Category: Time Series Analysis and Forecasting

The Method of Least Squares: NonLinear Trend

Secular Trend — Nonlinear

When a straight line does not describe accurately the long-term movement of a time series, then one might detect some curvature and decide to fit a curve instead of a straight line. The most commonly used curve, to describe the nonlinear secular trend in a time series, are:

  1. Exponential curve, and
  2. Second-degree parabola

1) Exponential (Nonlinear) Curve:

The exponential curve describes the trend (nonlinear) in a time series that changes by a constant percentage rate. The equation of the curve is $\hat{y} = ab^x$

Taking logarithm, we get the linear form $log\, \hat{y}=log\, a + (log\,b)x$

The method of least squares give the normal equations as:

\sum log\, y & = n\, log\, a + log\, b \sum x\\
\sum log\, y & = n\, log\, a \sum x + log\, b \sum x^2

However, if $\sum x=0$ the normal equations becomes

\sum log\,y & = n\, log a\\
\sum x log\, y &= log\, b \sum x^2

The values of $log\, a$ and $log\, b$ are

log\, a &=\frac{\sum log\, y}{n}\\
log\, b&= \frac{\sum x log\, y}{\sum x^2}

Taking $antilog$ of of $log\, a$ and $log\, b$, we get the values of $a$ and $b$.

Question: The population of a country for the years 1911 to 1971 in ten yearly intervals in millions is 5.38, 7.22, 9.64, 12.70, 17.80, 24.02, and 31.34. (i) Fit a curve of the type $\hat{y}=ab^x$ to this time series and find the trend values, (ii) Forecast the population for the year 1991.


(i) We have $\overline{t}=\frac{(1991+1971)}{2}=1941$. Let $x=\frac{t-\overline{t}}{10}=\frac{5-1941}{10}$ so that coded year number $x$ is measured in a unit of 10 years.

Coded year
$log y$$x log\, y$$x^2$$\hat{y}=13.029(1.345)^x$

The least squares exponential curve is $\hat{y} = ab^x$

Taking logarithm, $log\, \hat{y} = log a + (log\, b)x$

since $\sum x=0$, therefore

log\, a &= \frac{\sum log\, y}{n} = \frac{7.80429}{7}=1.1149\\
log\, b &= \frac{\sum x log\, y}{\sum x^2} = \frac{3.60636}{28}=0.12880\\
a &= antilog(1.1149)=13.029\\
b &= antilog(0.1288)=1.345\\
\hat{y} &=13.029 (1.345)^x,\quad \text{with origin at 1941 and unit of $x$ as 10 years}

(ii) For $t=1941$ we have $x=\frac{t-1941}{10}= \frac{1991-1994}{10}=5$. Putting $x=5$, in the least squares exponentail curve, we have
$\hat{y} = 13.029 (1.345)^5 = 57.348$ millions

Nonlinear method of least squares

2) Second Degree Parabola (Nonlinear)

It describes the trend (nonlinear) in a time series where a change in the amount of change is constant per unit time. The quadratic (parabolic) trend can be described by equation

\hat{y} = a + bx + cx^2

The method of least squares gives the normal equations as

\sum y &= na + b\sum x + c \sum x^2\\
\sum xy &= a\sum x + b\sum x^2 + c \sum x^3\\
\sum x^2y &= a \sum x^2 + b\sum x^3 + c\sum x^4

However if $\sum x = 0 \sum x^3$ then the normal equation reduces to

\sum y &= na + c\sum x^2\\
\sum xy &= b\sum x^2\\
\sum x^2 y &= a \sum x^2 + c \sum x^4\\
& \text{the values of $a$, $b$, and $c$ can be found as}\\
c &= \frac{n \sum x^2 y – (\sum x^2)(\sum y)}{n \sum x^2 -(\sum x^2)^2}\\
a&=\frac{\sum y – c\sum x^2}{n}\\
b&= \frac{\sum xy}{\sum x^2}

Question: Given the following time series

Price Index968791102108139307289
  1. Fit a second degree parabola taking the origin at 1938.
  2. Find the trend values
  3. What would have been the equation of parabola if origin were at 1933



Price index
Coded Year
$x^2$$x^4$$xy$$x^2y$Trend values
$y=110.2+15.48x+2.01 x^2$

(ii) Different trend values are already computed in the above table.

\hat{y} &= a + b x + c x^2\\
c &= \frac{n\sum x^2 y-(\sum x^2)(\sum y)}{n \sum x^4 -(\sum x^2)^2} =\frac{8(30995)-(168)(1219)}{8(6126)-(168)^2}=2.01\\
a &= \frac{\sum y – a \sum x^2}{n}=\frac{1219-(2.01)(168)}{8}=119.2\\
b &= \frac{\sum xy}{\sum x^2}=\frac{2601}{168} = 15.48\\
\hat{y} &= 110.2 + 15.48x + 2.01^2,\quad \text{with origin at the year 1938}

For different values of $x$, the trend values are obtained in table.

For shifting the origin at 1933, replace $x$ by $(x-5)$

\hat{y} &= 110.2 + 15.48(x-5)+2.01(x-5)^2\\
&= 110.2 + 15.48(x-5)+2.01(x^2 -10x + 25)\\
&= 110.2 + 15.48x -77.4 + 2.01x^2 – 20.1x + 50.25\\
&= 83.05 -4.62x + 2.01x^2, \quad \text{with origin at the year 1933}

Merits of Least Squares:

  • The method of least squares gives the most satisfactory measurement of the secular trend in a time series when the distribution of the deviations is approximately normal.
  • The least-squares estimates are unbiased estimates of the parameters.
  • The method can be used when the trend is linear, exponential, or quadratic.

Demerits of Least Squares:

  • The least-squares method gives too much weight to extremely large deviations from the trend
  • The least-squares line is the best only for the period to which it has reference.
  • The elimination or addition for a few or more time periods may change its position.

Learn about Method of Least Squares (Linear Trend)

The Method of Least Squares: Linear Trend

The least-squares principle says that “the sum of squares of the deviations of the observed values from the corresponding expected values should be least”. Among all the trend lines, the trend line is called a least-squares fit for which the sum of the squares of the deviations of the observed values form their corresponding expected values is the least.

Note that the usual probabilistic assumptions made in regression and correlation analysis are not met in the case of a time series data.

Secular Trend — Linear

It is useful to describe the trend in a time series where the amount of change is constant per unit time.

Let $(x_1, y_1), (x_2, y_2), \cdots, (x_n,y_n)$ be the $n$ pairs of observed sample values of a time series variable $y$, with $x$ representing the coded time value. We can plot these $n$ points on a graph.

Let us suppose that we want to fit a straight line expressed in slope-intercept form as:

\hat{y} = a + bx, \quad \quad (eq1)

The line (eq-1) will be called the least squares line if it makes $\sum(y-a-bx)^2$ minimum. The method of least squares yields the following normal equations:

\sum y &= na + b \sum x\\
\sum xy &= a \sum x + b \sum x^2

The normal equations give the value of $a$ and $b$ as:

b &= \frac{n \sum xy – (\sum x \sum y )}{n \sum x^2 -(\sum x)^2}\\
a & = \overline{y}-b\overline{x}

However, if $\sum x=0$ the usual normal equations reduces to

\sum y &= na\\
\sum xy & = b\sum x^2

The value of $a$ and $b$ also reduces to

a&=\frac{\sum y}{n}=\overline{y}\\
b&=\frac{\sum xy}{\sum x^2}

The trend values $\hat{y}$ are computed from the least-squares line $\hat{y}=a+bx$ by substituting the values of $x$ corresponding to the different time periods.

Properties of the Method of Least Squares

  • The least-squares line always passes through the point ($\overline{x}, \overline{y}$) call the center of gravity of the data.
  • The sum of deviations $\sum(y-\hat{y})$ of the observed values $y$ from their corresponding expected values $\hat{y}$ is zero, that is, $\sum(y-\hat{y})=0$, hence $\sum y= \sum \hat{y}$
  • The sum of squares of the deviations $\sum (y-\hat{y})^2$ measures how well the trend line fits the data. A smaller $\sum (y-\hat{y})^2$ means the better fit.

Moving Averages and Least Squares Linear Trend: The least-squares linear trend values corresponding to the central time period in each group of $k$ observations are equal to the $k$-period moving averages.

Question: Determine the trend line by the least-squares method from the following data. Plot the actual values and the linear trend on the same graph.



The equation of the trend line is

\hat{Y} = a + b\, X

Normal Equations are:

\Sigma Y & = n\, a + b \, \Sigma X \tag{i}\\
\Sigma XY& = a\, \Sigma X+ b\, \Sigma X^2 \tag{ii}


Putting the values in Normal equations:

81 &= 9a \tag*{1}\\
101&= 60b \tag*{2}

From (1) $a=\frac{81}{9}=9$, and from (2) $b=\frac{101}{60}=1.7$.

Fitted trend line is $\hat{Y}=9 + 1.7\,X$.

Least Squares Method

The Method of Least Squares: Non-Linear Trend

The Method of Moving Averages

The method of moving averages are of two types:

  1. Simple Moving Averages
  2. Weighted Moving Averages

Simple Moving Averages

If the observed values of a variable $Y$ are $y_1, y_2,\cdots, y_n$ corresponding to the time periods $t_1, t_2,\cdots, t_n$, respectively, the $k$-period simple moving averages are defined as

a_1 &= \frac{1}{k} \sum_{i=1}^{k} y_i\\
a_2 &= \frac{1}{k} \sum_{i=2}^{k+1} y_i,\\
a_3 &= \frac{1}{k} \sum_{i=3}^{k+2} y_i \\
\vdots &= \quad \vdots\\
a_m &= \frac{1}{k} \sum_{i=m}^{n} y_i

where $a_1, a_2, \cdots, a_m$ is the sequence of $k$-period simple moving averages. That is, the $k$-period simple moving averages are calculated by averaging first $k$ observations and then repeating this process of averaging the $k$ observations by dropping each time the first observation and including the next one. This process is continued till the last $k$ observations have been averaged. For example, the 3-period simple moving averages are given as:

a_1 &= \frac{1}{3} (y_1+y_2+y_3) = \frac{1}{3} \sum_{i=1}^{3} y_i\\
a_2 &= \frac{1}{3} (y_2+y_3+y_4) = \frac{1}{3} \sum_{i=2}^{4} y_i\\
a_3 &= \frac{1}{3} (y_3+y_4+y_5) = \frac{1}{3} \sum_{i=3}^{5} y_i\\
\vdots &= \quad \vdots\\
\text{and so on}

Each of these simple moving average of the sequence $a_1, a_2, a_3,\cdots$ is placed against the middle of each successive group. The $k$-period moving successive totals $S_1, S_2, S_3, \cdots$ are obtained by the following relations

S_1 = \sum_{i=1}^{k} y_i\\
S_2 &= S1+ y_{k+1}-y_1\\
S_3 &= S_2 + y_{k+2} – y_2\\
\vdots &= \quad \vdots\\
\text{so on}

The $k$-period simple moving averages are obtained by dividing these $k$-period moving successive totals ($S_1, S_2, S_3, \cdots$) by $k$, as given in the following relations

a_1 &= \frac{S_1}{k}\\
a_2 &= a_1 + \frac{y_{k_1}0y_1} {k}\\
a_3 &= a_2 + \frac{y_{k+2} -y_2}{k}\\
\vdots &= \quad \vdots\\
\text{so on}

method of moving averages
  • When $k$ is odd, the sequence $a_1, a_2, a_3, \cdots$ will be placed against the middle of its time period.
  • When $k$ is even, the sequence $a_1, a_2, a_3, \cdots$ of simple moving averages will be placed in the middle of two time periods. It is necessary to centralize these averages. For centralization, further 2-period moving averages of the former $k$-period moving averages are computed which are called $k$-period centered moving averages.

Weighted Moving Averages

For observed values ($y_1, y_2, \cdots, y_n$) of a variable $Y$ corresponding to the time periods $t_1, t_2, \cdots, t_n$, respectively, the $k$-period weighted moving averages with weights $w_1, w_2, \cdots, w_k$ are defined as

a_1 &= \frac{1}{\sum w} \sum_{i=1}^{k} y_i w\\
a_2 &= \frac{1}{\sum w} \sum_{i=2}^{k+1} y_i w\\
a_3 &= \frac{1}{\sum w} \sum_{i=3}^{k+2} y_i w\\
\vdots &= \vdots\\
a_m &= \frac{1}{\sum w} \sum_{i=m}^{n} y_i w\\

where $a_1, a_2, \cdots, a_m$ is a sequence of $k$-period weighted moving averages with weights $w_1, w_2, \cdots, w_k$, respectively. The $k$-period weighted moving averages are calculated by taking the weighted average of first $k$ observed values with weights $w_1, w_2, \cdots, w_k$ and then repeating this process of averaging the $k$ observations by dropping each time the first observation and including the next one. This process is continued until the last $k$ observations have been averaged.


  • The method of moving averages is simple and easy.
  • This method is appropriate to remove, seasonal variations, cyclical fluctuations, and irregular variations.


  • Some values at the beginning and the end of the series are lost.
  • Moving averages are greatly affected by extreme values.
  • The method does not provide a mathematical formula for the trend.

Example: Calculate 3-years simple moving averages for the following time series. Also, plot actual data and moving averages on a graph. Also, find the 3-years weighted moving averages with weights 2, 2, 1, respectively.



YearProduction3-year Simple MT3-year simple MA3-year WMT3-year WMA

*MT=moving total, MA=moving averages, WMT=weighted MT, WMA=Weighted MA

three year moving average

Method of Semi-Averages

The secular trend can also be measured by the method of semi-averages. The steps are:

  • Divide the time series data into two equal portions. If observations are odd then either omit the middle value or include the middle value in each half.
  • Take the average of each part and place these average values against the midpoints of the two parts.
  • Plot the semi-averages in the graph of the original values.
  • Draw the required trend line through these two potted points and extend it to cover the whole period.
  • It is simple to compute the slope and $y$-intercept of the line drawn from two points. The trend values can be found from the semi-average trend line or by estimated straight line as explained:

Let $y’_1$ and $y’_2$ be the semi-averages placed against the times $x_1$ and $x_2$. Let the estimated straight line $y’=a+bx$ is to pass through the points ($x_1$, $y’_1$) and ($x_2$, $y’_2$). The constant “$a$” and “$b$” can easily be determined. the equation of the line passing through the points ($x_1$, $y’_1$) and ($x_2$, $y’_2$) can be written as:

y’ – y’_1 &= \frac{y’_2-y’_1}{x_2-x_1}(x-x_1)\\
&= b(x-x_1)\\
\Rightarrow y’ &= (y’_1 – bx_1) + bx\\
&= a+bx, \quad \text{ where $a=y’_1-bx_1$}

For even number of observations the slope of trend line can be found as:

b&=\frac{1}{n/2}\left(\frac{S_2}{n/2} – \frac{S_1}{n/2} \right)\\
&= \frac{1}{n/2} \left(\frac{S_2-S_1}{n/2}\right)\\
&= \frac{4(S_2-S_1)}{n^2},

where $S_1$ is sum of $y$-values for the first half of the period, $S_2$ is sum of $y$-values of the second half of the period, and $n$ is the number of time units covered by the time-series.

Merits of Semi-Averages

  • The method of semi-averages is simple, easy, and quick.
  • It smooths out seasonal variations
  • It gives a better approximation to the trend because it is based on a mathematical model.

Demeris of Semi-Averages

  • It is a rough and objective method.
  • The arithmetic mean used in Semi Average is greatly affected by very large or by very small values.
  • The method of semi-averages is applicable when the trend is linear. This method is not appropriate if the trend is not linear.

Numerical Example 1:

The following table shows the property damaged by road accidents in Punjab for the year 1973 to 1979.

Property Damage201238392507484648742
  1. Obtain the semi-averages trend line
  2. Find out the trend values.


Let $x=t-1973$

YearProperty DamagedSemi TotalSemi AverageCoded YearTrend Values
1973201  0$y’=190+87(0)=190$
1975392  2$y’=190+87(2)=364$
1976507  3$y’=190+87(3)=451$
1977484  4$y’=190+87(4)=538$
1979742  6$y’=190+87(6)=712$
method of semi-averages (trend values)

y’_1 &= 277, x_1 = 1, y’_2 = 625, x_2=5\\
a&=y’_1 – bx_1 = 277-87(1)=190

The semi-average trend line $y’=190+87x$ (with the origin at 1973)

Numerical Example 2:

The following table gives the number of books in thousands sold at a book store for the year 1973 to 1981

No. of Books Sold423835253224201917
  1. Find the equation of the semi-averages trend line
  2. Compute the trend values
  3. Estimate the number of books sold for the year 1982.


Let $x=t-1973$

YearNo. of books (y)Semi TotalSemi AverageCoded yearTrend Values
197342  0$y’=39.5 – 3(0)=39.5$
197438140351$y’=39.5 – 3(1)=36.5$
1975352$y’=39.5 – 3(2)=33.5$
197625  3$y’=39.5 – 3(3)=30.5$
197732  4$y’=39.5 – 3(4)=27.5$
197824  5$y’=39.5 – 3(5)=24.5$
19792080206$y’=39.5 – 3(6)= 21.5$
198019  7$y’=39.5 – 3(7)=18.5$
198117  8$y’=39.5 – 3(8)=15.5$

y’_1 &= 35, x_1=1.5, y’_2=20, x_2=6.5\\
b &= \frac{y’_2 – y’_1}{x_2-x_1} = \frac{20-35}{6.5-1.5} =-3\\
a &= y’_1 – bx_1 = 35 – (-3)(1.5) = 39.5\\
y’&= 39.5 – 3x (\text{with origin at 1973})

For the year 1982, the estimated number of books sold is: $y’=39.5-3(9)=12.5$.

x Logo: Shield Security
This Site Is Protected By
Shield Security