Autocorrelation

Post about the presence of autocorrelation, reasons for autocorrelation, detection of autocorrelation, remedy, and test statistics of autocorrelation.

Durbin-Watson Test Statistic (2021)

by

Durbin and Watson have suggested a test to detect the presence of autocorrelation which applies to small samples. However, the test is appropriate only for the first-order autoregressive scheme ($u_t =  \rho u_{t-1} + \varepsilon_t$).

Step by Step procedure for the Durbin-Watson Test

Step 1: Null and Alternative Hypothesis

The null hypothesis is $H_0:\rho=0$ (that is, $u$’s are not autocorrelated with a first-order scheme)

The alternative hypothesis is $H_1: \rho \ne 0$ (that is, $u$’s are autocorrelated with a first-order scheme)

Step 2: Level of Significance

Choose the appropriate level of significance, such as 5%, 1%, 10%, etc.

Step 3: Test Statistics

To test the null hypothesis, the Durbin-Watson Test statistic is

$$d = \frac{\sum\limits_{t=2}^n (u_t – u_{t-1})^2}{\sum\limits_{t=1}^n e_t^2}$$

The value of $d$ lies between 0 and 4, when $d=2$, then $\rho=0$. It means that $H_0:\rho=0$ is equivalent to testing $H_0:d=2$.

\begin{align*}
d&= \frac{\sum\limits_{t=2}^n (u_t – u_{t-1})^2}{\sum\limits_{t=1}^n u_t^2}\\
&=\frac{ \sum\limits_{t=2}^n (u_t^2 + u_{t-1}^2 – 2u_t u_{t-1} ) }{\sum\limits_{t=1}^n u_t^2} \\
&=\frac{ \sum\limits_{t=2}^n u_t^2 + \sum\limits_{t=2}^n u_{t-1}^2 – 2 \sum\limits_{t=2}^n u_t u_{t-1} }{\sum\limits_{t=1}^n u_t^2}
\end{align*}

Durbin-Watson Test Statistic is simply the ratio of the sum of squared differences in the successive residuals to the residual sum of squares. In the numerator, there will be $n-2$ observations because of lag values.

For large samples $\sum\limits_{t=2}^n u_t^2$, $\sum\limits_{t=2}^n u_{t-1}^2$ and $\sum\limits_{t=1}^n u_t^2$ are all approximately equal. Therefore,

\begin{align*}
d &\approx  \frac{2 \sum u_t^2 – 1}{\sum u_{t-1}^2} – \frac{2 \sum_{t=2}^n u_tu_{t-1} }{ \sum u_{t-1}^2 }\\
& \approx 2 \left[ 1- \frac{\sum u_t u_{t-1} }{ \sum u_{t-1}^2 }\right]\\
\text{but }\,\,\, \hat{\rho} &= \frac{\sum u_t u_{t-1}}{\sum u_{t-1}^2}
\end{align*}

Therefore $d\approx 2(1-\hat{\rho})$

It is obvious that the values of $d$ lie between 0 and 4.

Firstly: If there is no autocorrelation $\hat{\rho}=$ then $d=2$, it means that from the sample data $d^*\approx 2$. We accept that there is no autocorrelation.

Secondly: If $\hat{\rho}=+1$, we have perfect positive autocorrelation. Therefore, if $2<d^* <4$ there is some degree of positive autocorrelation (which is stronger the higher for the higher value of $d^*$).

Thirdly: If $\hat{\rho}=-1, d=4$. We have perfect negative autocorrelation. Therefore, if $2<d^*<4$, there is some degree of negative autocorrelation (which is stronger for the higher value of $d^*$).

The next step is to use the sample residual ($u_t$’s) and compute the empirical value of the Durbin-Watson statistic $d^*$.

Finally, the empirical $d^*$ must be compared with the theoretical values of $d$, that is, the values of $d$ which define the critical region of the test.

The problem with this test is that the exact distribution of $d$ is not known. However, Durbin and Watson have established upper ($d_u$) and lower ($d_l$) limits for the significance level of $d$ which are appropriate to the hypothesis of zero first-order autocorrelation against the alternative hypothesis of positive first-order autocorrelation. Durbin and Watson have tabulated these upper and lower values at 5% and 1% level of significance.

Critical Region of $d$ Durbin-Watson test

Durbin-Watson Test
  • If $d^*<d_l$ we reject the null hypothesis of no autocorrelation and accept that there is positive autocorrelation of the first order.
  • If $d^* >( 4-d_l)$ we reject the null hypothesis of no autocorrelation and accept that there is negative autocorrelation of the first order.
  • If $d_u < d^* < (4-d_u)$ we accept the null hypothesis of no autocorrelation
  • if $d_l < d^* < d_u$ or if $(4-d_u)<d^*<(4-d_l)$ the test is inconclusive.

Assumptions underlying the $d$ Statistics

  • The regression model includes the intercept term. It is not present as in the case of the regression through the origin, it is essential to return the regression including the intercept term to obtain the RSS.
  • The explanatory variables, $X$ are non-stochastic or fixed in repeated sampling.
  • The disturbances $u_t$ are generated by the first-order autoregressive scheme: $u_t=\rho + u_{t-1} +\varepsilon_t$ (it cannot be used to detect higher-order autoregression schemes.
  • The error term $u_t$ is assumed to be normally distributed.
  • The regression model does not include the lagged values(s) of the dependent variable as one of the explanatory variables. The Durbin-Watson test is inappropriate to the model of this type $$Y_t=\beta_1 + \beta_2X_{2t} + \beta_3 X_{3t} + \cdots+ \beta_k X_{kt} + \gamma Y_{t-1}+u_t$$, where $Y_{t-1}$ is the one period lagged values of $Y$.
  • There are no missing observations in the data.

Limitations or Shortcomings of Durbin-Watson Test Statistics

Durbin-Watson test has several shortcomings:

  • The $d$ statistics is not an appropriate measure of autocorrelation if, among the explanatory variables, there are lagged values of the endogenous variables.
  • Durbin-Watson test is inconclusive if the computed value lies between $d_l$ and $d_u$.
  • It is inappropriate for testing higher-order serial correlation or for other forms of autocorrelation.

An Asymptotic or Large Sample Test

Under the null hypothesis that $\rho=0$ and assuming that the sample size $n$ is large, it can be shown that $\sqrt{n}\hat{\rho}$ follows the normal distribution with 0 mean and variance 1, i.e. asymptotically,

$$\sqrt{n}\,\, \hat{\rho} \sim N(0, 1)$$

Introduction to Autocorrelation

Computer MCQs Test Online

Residuals plot for Detection of Autocorrelation (2020)

by

The existence and pattern of autocorrelation may be detected using a graphical representation of residuals obtained from ordinary least square regression. One can draw the following residual plot for the detection of autocorrelation:

Detection of Autocorrelation from Residual Plots

  • A plot of the residual plot against time.
  • A plot of the $hat{u}_t$ against $hat{u}_{t-1}$
  • A plot of standardized residuals against time.

Note that the population disturbances $u_t$ are not directly observable, we use their proxies, the residuals $hat{u}_t$.

Detection of Positive negative autocorrelation
  • A random pattern of residuals indicates the non-presence of autocorrelation.
  • A plot of residuals for detection of residuals used for visual examination of $hat{u}_t$ or  $hat{u}_t^2$ can provide useful information not only about the presence of autocorrelation but also about the presence of heteroscedasticity. Similarly, the examination of $hat{u}_t$ and $hat{u}_t^2$ provides useful information about model inadequacy or specification bias too.
  • The standardized residuals are computed as $frac{u_t}{hat{sigma}}$ where $hat{sigma}$ is standard error of regression.
Residuals plot for Autocorrelation

Note: The plot of residuals against time is called the sequence plot. For time-series data, the researcher can plot (graphically draw) the residuals versus time (called a time sequence plot), he may expect to observe some random pattern in the time series data, indicating that the data is not autocorrelated. However, if the researcher observes some pattern (other than random) in the graphical representation of the data, then it means that the data is autocorrelated. The existence of some patterns shown in the above Figure can be used for the detection of autocorrelation.

See more on Autocorrelation

R and Data Analysis

First Order Autocorrelation (2020)

by

To understand the First Order Autocorrelation, consider the multiple regression model as described below

$$Y_t=\beta_1+\beta_2 X_{2t}+\beta_3 X_{3t}+\cdots+\beta_k X_{kt}+u_t,$$

In the model above the current observation of the error term ($u_t$) is a function of the previous (lagged) observation of the error term ($u_{t-1}$). That is,

\begin{align*}
u_t = \rho u_{t-1} + \varepsilon_t, \tag*{eq 1}
\end{align*}

where $\rho$ is the parameter depicting the functional relationship among observations of the error term $u_t$ and $\varepsilon_t$ is a stochastic error term which is iid (identically independently distributed). It satisfies the standard OLS assumption:

\begin{align*}
E(\varepsilon) &=0\\
Var(\varepsilon) &=\sigma_t^2\\
Cov(\varepsilon_t, \varepsilon_{t+s} ) &=0
\end{align*}

Note if $\rho=1$, then all these assumptions are undefined.

The scheme (eq1) is known as a Markov first-order autoregressive scheme, usually denoted by AR(1). The eq1 is interpreted as the regression of $u_t$ on itself tagged on period. It is first-order because $u_t$ and its immediate past value are involved. Note the $Var(u_t)$ is still homoscedasticity under the AR(1) scheme.

The coefficient $\rho$ is called the first order autocorrelation coefficient (also called the coefficient of autocovariance) and takes values from -1 to 1 or ($|\rho|<1$). The size of $\rho$ determines the strength of autocorrelation (serial correlation).  There are three different cases:

  1. If $\rho$ is zero, then there is no autocorrelation because $u_t=\varepsilon_t$.
  2. If $\rho$ approaches 1, the value of the previous observation of the error ($u_t-1$) becomes more important in determining the value of the current error term ($u_t$), and therefore, greater positive autocorrelation exists. The negative error term will lead to negative and positive will lead to a positive error term.
  3. If $\rho$ approaches -1, there is a very high degree of negative autocorrelation. The signs of the error term tend to switch signs from negative to positive and vice versa in consecutive observations.
First order Autocorrelation

First Order Autocorrelation AR(1)

\begin{align*}
u_t &= \rho u_{t-1}+\varepsilon_t\\
E(u_t) &= \rho E(u_{t-1})+ E(\varepsilon_t)=0\\
Var(u_t)&=\rho^2 Var(u_{t-1}+var(\varepsilon_t)\\
\text{Because $u$’s and $\varepsilon$’s are uncorrelated}\\
Var(u_t)&=\sigma^2\\
Var(u_{t-1}) &=\sigma^2\\
Var(\varepsilon_t)&=\sigma_t^2\\
\Rightarrow Var(u_t) &=\rho^2 \sigma^2+\sigma_t^2\\
\Rightarrow \sigma^2-\rho^2\sigma^2 &=\sigma_t^2\\
\Rightarrow \sigma^2(1-\rho^2)&=\sigma_t^2\\
\Rightarrow Var(u_t)&=\sigma^2=\frac{\sigma_t^2}{1-\rho^2}
\end{align*}

For covariance, multiply equation (eq1) by $u_{t-1}$ and taking the expectations on both sides

\begin{align*}
u_t\cdot u_{t-1} &= \rho u_{t-1} \cdot u_{t-1} + \varepsilon_t \cdot u_{t-1}\\
E(u_t u_{t-1}) &= E[\rho u_{t-1}^2 + u_{t-1}\varepsilon_t ]\\
cov(u_t, u_{t-1}) &= E(u_t u_{t-1}) = E[\rho u_{t-1}^2 + u_{t-1}\varepsilon_t ]\\
&=\rho \frac{\sigma_t^2}{1-\rho^2}\tag*{$\because Var(u_t) = \frac{\sigma_t^2}{1-\rho^2}$}
\end{align*}

Similarly,
\begin{align*}
cov(u_t,u_{t-2}) &=\rho^2 \frac{\sigma_t^2}{(1-\rho^2)}\\
cov(u_t,u_{t-2}) &= \rho^2 \frac{\sigma_t^2}{(1-\rho^2)}\\
cov(u_t, u_{t+s}) &= \rho^p
\end{align*}

The strength and direction of the correlation (positive or negative) and its distance from zero determine the significance of the first-order autocorrelation. Values close to $+1$ or $-1$ indicate strong positive or negative autocorrelation, respectively. A value close to zero suggests little to no autocorrelation.

Software like R, Python, and MS Excel have built-in functions to calculate autocorrelation. The visualization of ACF is often a preferred method to assess autocorrelation across different lags, not just the first order autocorrelation.

In summary, first order autocorrelation refers to the correlation between a time series and lagged values of the same time series, specifically at a lag of one time period. It measures how much a variable in a time series is related to its immediate past value.

https://itfeature.com

Learn R Programming

Computer MCQs Online Test

What are the Consequences of Autocorrelation (2020)

by

Autocorrelation, when ignored, can lead to several issues in analyzing data, particularly in statistical models. In this post, we will discuss some important consequences of the existence of autocorrelation in the data. The consequences of the OLS estimators in the presence of Autocorrelation can be summarized as follows:

Consequences Patterns of Autocorrelation and Non-Autocorrelation

Consequences of Autocorrelation on OLS Estimators If Exists

  • When the disturbance terms are serially correlated then the OLS estimators of the $\hat{\beta}$s are still unbiased and consistent but the optimist property (minimum variance property) is not satisfied. This makes it harder to determine if the estimated effect of a variable is truly significant.
  • The OLS estimators will be inefficient and therefore, no longer BLUE. Inefficient means there could be better ways to estimate the model parameters that would produce more precise results with lower variance.
  • The estimated variance of the regression coefficients will be biased and inconsistent and will be greater than the variances of estimate calculated by other methods, therefore, hypothesis testing is no longer valid. In most of the cases, $R^2$ will be overestimated (indicating a better fit than the one that truly exists). The t- and F-statistics will tend to be higher. One might reject a true null hypothesis (meaning a relationship does not exist) or fail to reject a false one (meaning a relationship appears to exist when it does not).
  • The variance of random term $u$ may be under-estimated if the $u$’s are autocorrelated. That is, the random variance $\hat{\sigma}^2=\frac{\sum \hat{u}_i^2}{n-2}$ is likely to be under-estimate the true $\sigma^2$.
  • Among the consequences of autocorrelation, another is, that if the disturbance terms are autocorrelated then the OLS estimates are not asymptotic. That is $\hat{\beta}$s are not asymptotically efficient.

Therefore, autocorrelation may lead to misleading results and unreliable statistical tests. If autocorrelation is suspected in the data being analyzed, then use different statistical techniques to address it and improve the validity of your analysis.

https://itfeature.com

Learn about Autocorrelation and Reasons for Autocorrelations

Learn more about Autocorrelation on Wikipedia

MCQs General Knowledge