The Method of Least Squares: Linear Trend
The least-squares principle says that “the sum of squares of the deviations of the observed values from the corresponding expected values should be least”. Among all the trend lines, the trend line is called a least-squares fit for which the sum of the squares of the deviations of the observed values form their corresponding expected values is the least.
Note that the usual probabilistic assumptions made in regression and correlation analysis are not met in the case of a time series data.
Secular Trend — Linear
It is useful to describe the trend in a time series where the amount of change is constant per unit time.
Let $(x_1, y_1), (x_2, y_2), \cdots, (x_n,y_n)$ be the $n$ pairs of observed sample values of a time series variable $y$, with $x$ representing the coded time value. We can plot these $n$ points on a graph.
Let us suppose that we want to fit a straight line expressed in slope-intercept form as:
\begin{align}
\hat{y} = a + bx, \quad \quad (eq1)
\end{align}
The line (eq-1) will be called the least squares line if it makes $\sum(y-a-bx)^2$ minimum. The method of least squares yields the following normal equations:
\begin{align*}
\sum y &= na + b \sum x\\
\sum xy &= a \sum x + b \sum x^2
\end{align*}
The normal equations give the value of $a$ and $b$ as:
\begin{align*}
b &= \frac{n \sum xy – (\sum x \sum y )}{n \sum x^2 -(\sum x)^2}\\
a & = \overline{y}-b\overline{x}
\end{align*}
However, if $\sum x=0$ the usual normal equations reduces to
\begin{align*}
\sum y &= na\\
\sum xy & = b\sum x^2
\end{align*}
The value of $a$ and $b$ also reduces to
\begin{align*}
a&=\frac{\sum y}{n}=\overline{y}\\
b&=\frac{\sum xy}{\sum x^2}
\end{align*}
The trend values $\hat{y}$ are computed from the least-squares line $\hat{y}=a+bx$ by substituting the values of $x$ corresponding to the different time periods.
Properties of the Method of Least Squares
- The least-squares line always passes through the point ($\overline{x}, \overline{y}$) call the center of gravity of the data.
- The sum of deviations $\sum(y-\hat{y})$ of the observed values $y$ from their corresponding expected values $\hat{y}$ is zero, that is, $\sum(y-\hat{y})=0$, hence $\sum y= \sum \hat{y}$
- The sum of squares of the deviations $\sum (y-\hat{y})^2$ measures how well the trend line fits the data. A smaller $\sum (y-\hat{y})^2$ means the better fit.
Moving Averages and Least Squares Linear Trend: The least-squares linear trend values corresponding to the central time period in each group of $k$ observations are equal to the $k$-period moving averages.
Question: Determine the trend line by the least-squares method from the following data. Plot the actual values and the linear trend on the same graph.
| Year | 1945 | 1946 | 1947 | 1948 | 1949 | 1950 | 1951 | 1952 | 1953 |
| Price | 3 | 6 | 2 | 10 | 7 | 9 | 14 | 12 | 18 |
Solution
The equation of the trend line is
\begin{align*}
\hat{Y} = a + b\, X
\end{align*}
Normal Equations are:
\begin{align*}
\Sigma Y & = n\, a + b \, \Sigma X \tag{i}\\
\Sigma XY& = a\, \Sigma X+ b\, \Sigma X^2 \tag{ii}
\end{align*}
| Year | Values | $X$ | $XY$ | $X^2$ | $\hat{Y}$ |
| 1945 | 3 | -4 | -12 | 16 | 22 |
| 1946 | 6 | -3 | -18 | 9 | 3.9 |
| 1947 | 2 | -2 | -4 | 4 | 5.6 |
| 1948 | 10 | -1 | -10 | 1 | 7.3 |
| 1949 | 7 | 0 | 0 | 0 | 9.0 |
| 1950 | 9 | 1 | 9 | 1 | 10.7 |
| 1951 | 14 | 2 | 28 | 4 | 12.4 |
| 1952 | 12 | 3 | 36 | 9 | 14.1 |
| 1953 | 18 | 4 | 72 | 16 | 15.8 |
| Total | 81 | 0 | 101 | 60 | 81.0 |
Putting the values in Normal equations:
\begin{align*}
81 &= 9a \tag*{1}\\
101&= 60b \tag*{2}
\end{align*}
From (1) $a=\frac{81}{9}=9$, and from (2) $b=\frac{101}{60}=1.7$.
Fitted trend line is $\hat{Y}=9 + 1.7\,X$.
