Durbin-Watson Test Statistic (2021)

by

Durbin and Watson have suggested a test to detect the presence of autocorrelation which applies to small samples. However, the test is appropriate only for the first-order autoregressive scheme ($u_t =  \rho u_{t-1} + \varepsilon_t$).

Step by Step procedure for the Durbin-Watson Test

Step 1: Null and Alternative Hypothesis

The null hypothesis is $H_0:\rho=0$ (that is, $u$’s are not autocorrelated with a first-order scheme)

The alternative hypothesis is $H_1: \rho \ne 0$ (that is, $u$’s are autocorrelated with a first-order scheme)

Step 2: Level of Significance

Choose the appropriate level of significance, such as 5%, 1%, 10%, etc.

Step 3: Test Statistics

To test the null hypothesis, the Durbin-Watson Test statistic is

$$d = \frac{\sum\limits_{t=2}^n (u_t – u_{t-1})^2}{\sum\limits_{t=1}^n e_t^2}$$

The value of $d$ lies between 0 and 4, when $d=2$, then $\rho=0$. It means that $H_0:\rho=0$ is equivalent to testing $H_0:d=2$.

\begin{align*}
d&= \frac{\sum\limits_{t=2}^n (u_t – u_{t-1})^2}{\sum\limits_{t=1}^n u_t^2}\\
&=\frac{ \sum\limits_{t=2}^n (u_t^2 + u_{t-1}^2 – 2u_t u_{t-1} ) }{\sum\limits_{t=1}^n u_t^2} \\
&=\frac{ \sum\limits_{t=2}^n u_t^2 + \sum\limits_{t=2}^n u_{t-1}^2 – 2 \sum\limits_{t=2}^n u_t u_{t-1} }{\sum\limits_{t=1}^n u_t^2}
\end{align*}

Durbin-Watson Test Statistic is simply the ratio of the sum of squared differences in the successive residuals to the residual sum of squares. In the numerator, there will be $n-2$ observations because of lag values.

For large samples $\sum\limits_{t=2}^n u_t^2$, $\sum\limits_{t=2}^n u_{t-1}^2$ and $\sum\limits_{t=1}^n u_t^2$ are all approximately equal. Therefore,

\begin{align*}
d &\approx  \frac{2 \sum u_t^2 – 1}{\sum u_{t-1}^2} – \frac{2 \sum_{t=2}^n u_tu_{t-1} }{ \sum u_{t-1}^2 }\\
& \approx 2 \left[ 1- \frac{\sum u_t u_{t-1} }{ \sum u_{t-1}^2 }\right]\\
\text{but }\,\,\, \hat{\rho} &= \frac{\sum u_t u_{t-1}}{\sum u_{t-1}^2}
\end{align*}

Therefore $d\approx 2(1-\hat{\rho})$

It is obvious that the values of $d$ lie between 0 and 4.

Firstly: If there is no autocorrelation $\hat{\rho}=$ then $d=2$, it means that from the sample data $d^*\approx 2$. We accept that there is no autocorrelation.

Secondly: If $\hat{\rho}=+1$, we have perfect positive autocorrelation. Therefore, if $2<d^* <4$ there is some degree of positive autocorrelation (which is stronger the higher for the higher value of $d^*$).

Thirdly: If $\hat{\rho}=-1, d=4$. We have perfect negative autocorrelation. Therefore, if $2<d^*<4$, there is some degree of negative autocorrelation (which is stronger for the higher value of $d^*$).

The next step is to use the sample residual ($u_t$’s) and compute the empirical value of the Durbin-Watson statistic $d^*$.

Finally, the empirical $d^*$ must be compared with the theoretical values of $d$, that is, the values of $d$ which define the critical region of the test.

The problem with this test is that the exact distribution of $d$ is not known. However, Durbin and Watson have established upper ($d_u$) and lower ($d_l$) limits for the significance level of $d$ which are appropriate to the hypothesis of zero first-order autocorrelation against the alternative hypothesis of positive first-order autocorrelation. Durbin and Watson have tabulated these upper and lower values at 5% and 1% level of significance.

Critical Region of $d$ Durbin-Watson test

Durbin-Watson Test
  • If $d^*<d_l$ we reject the null hypothesis of no autocorrelation and accept that there is positive autocorrelation of the first order.
  • If $d^* >( 4-d_l)$ we reject the null hypothesis of no autocorrelation and accept that there is negative autocorrelation of the first order.
  • If $d_u < d^* < (4-d_u)$ we accept the null hypothesis of no autocorrelation
  • if $d_l < d^* < d_u$ or if $(4-d_u)<d^*<(4-d_l)$ the test is inconclusive.

Assumptions underlying the $d$ Statistics

  • The regression model includes the intercept term. It is not present as in the case of the regression through the origin, it is essential to return the regression including the intercept term to obtain the RSS.
  • The explanatory variables, $X$ are non-stochastic or fixed in repeated sampling.
  • The disturbances $u_t$ are generated by the first-order autoregressive scheme: $u_t=\rho + u_{t-1} +\varepsilon_t$ (it cannot be used to detect higher-order autoregression schemes.
  • The error term $u_t$ is assumed to be normally distributed.
  • The regression model does not include the lagged values(s) of the dependent variable as one of the explanatory variables. The Durbin-Watson test is inappropriate to the model of this type $$Y_t=\beta_1 + \beta_2X_{2t} + \beta_3 X_{3t} + \cdots+ \beta_k X_{kt} + \gamma Y_{t-1}+u_t$$, where $Y_{t-1}$ is the one period lagged values of $Y$.
  • There are no missing observations in the data.

Limitations or Shortcomings of Durbin-Watson Test Statistics

Durbin-Watson test has several shortcomings:

  • The $d$ statistics is not an appropriate measure of autocorrelation if, among the explanatory variables, there are lagged values of the endogenous variables.
  • Durbin-Watson test is inconclusive if the computed value lies between $d_l$ and $d_u$.
  • It is inappropriate for testing higher-order serial correlation or for other forms of autocorrelation.

An Asymptotic or Large Sample Test

Under the null hypothesis that $\rho=0$ and assuming that the sample size $n$ is large, it can be shown that $\sqrt{n}\hat{\rho}$ follows the normal distribution with 0 mean and variance 1, i.e. asymptotically,

$$\sqrt{n}\,\, \hat{\rho} \sim N(0, 1)$$

Introduction to Autocorrelation

Computer MCQs Test Online

Heteroscedasticity Consistent Standard Errors

by

The post is about “Heteroscedasticity Consistent Standard Errors and Variances.

$\sigma_i^2$ are rarely known. However, there is a way of obtaining consistent estimates of variances and covariances of OLS estimators even if there is heteroscedasticity.

White’s Heteroscedasticity Consistent Standard Errors and Variances

White’s heteroscedasticity-corrected standard errors are known as robust standard errors. White’s heteroscedasticity-corrected standard errors are larger (maybe smaller too) than the OLS standard errors and therefore, the estimated $t$-values are much smaller (or maybe larger) than those obtained by the OLS.

Comparing the OLS output with White’s heteroscedasticity consistent standard errors (variances) may be useful to see whether heteroscedasticity is a serious problem in a particular set of data.

Plausible Assumptions about Heteroscedasticity Patterns

Assumption 1: The error variance is proportional to $X_i^2$

Heteroscedasticity Consistent standard errors and Variances

$$E(u_i^2)=\sigma^2 X_i^2$$
It is believed that the variance of $u_i$ is proportional to the square of the $X$ (in graphical methods or Park and Glejser approaches).

One may transform the original model as follows:

\begin{align}\label{assump1}
\frac{Y_i}{X_i} &=\frac{\beta_1}{X_i} + \beta_2 + \frac{u_i}{X_i} \nonumber \\
&=\beta_1 \frac{1}{X_i} + \beta_2 + v_i,\qquad \qquad (1)
\end{align}

where $v_i$ is the transformed disturbance term, equal to $\frac{u_i}{X_i}$. It can be verified that

\begin{align*}
E(v_i^2) &=E\left(\frac{u_i}{X_i}\right)^2\\
&=\frac{1}{X_i^2}E(u_i^2)=\sigma^2
\end{align*}

Hence, the variance of $v_i$ is now homoscedastic, and one may apply OLS to the transformed equation by regressing $\frac{Y_i}{X_i}$ on $\frac{1}{X_i}$.

Notice that in the transformed regression the intercept term $\beta_2$ is the slope coefficient in the original equation and the slope coefficient $\beta_1$ is the intercept term in the original model. Therefore, to get back to the original model multiply the estimated equation (1) by $X_i$.

Assumption 2: The Error Variance is Proportional to $X_i$

The square root transformation: $E(u_i^2) = \sigma^2 X_i$

Heteroscedasticity Consistent Variances

If it is believed that the variance of $u_i$ is proportional to $X_i$, then the original model can be transformed as

\begin{align*}
\frac{Y_i}{\sqrt{X_i}} &= \frac{\beta_1}{\sqrt{X_i}} + \beta_2 \sqrt{X_i} + \frac{u_i}{\sqrt{X_i}}\\
&=\beta_1 \frac{1}{\sqrt{X_i}} + \beta_2\sqrt{X_i}+v_i,\quad\quad (a)
\end{align*}

where $v_i=\frac{u_i}{\sqrt{X_i}}$ and $X_i>0$

$E(v_i^2)=\sigma^2$ (a homoscedastic situation)

One may proceed to apply OLS on equation (a), regressing $\frac{Y_i}{\sqrt{X_i}}$ on $\frac{1}{\sqrt{X_i}}$ and $\sqrt{X_i}$.

Note that the transformed model (a) has no intercept term. Therefore, use the regression through the origin model to estimate $\beta_1$ and $\beta_2$. To get back the original model simply multiply the equation (a) by $\sqrt{X_i}$.

Consider a case of $intercept = 0$, that is, $Y_i=\beta_2X_i+u_i$. The transformed model will be

\begin{align*}
\frac{Y_i}{\sqrt{X_i}} &= \beta_2 \sqrt{X_i} + \frac{u_i}{\sqrt{X_i}}\\
\beta_2 &=\frac{\overline{Y}}{\overline{X}}
\end{align*}

Here, the WLS estimator is simply the ratio of the means of the dependent and explanatory variables.

Assumption 3: The Error Variance is proportional to the Square of the Mean value of $Y$

$$E(u_i^2)=\sigma^2[E(Y_i)]^2$$

The original model is $Y_i=\beta_1 + \beta_2 X_i + u_I$ and $E(Y_i)=\beta_1 + \beta_2X_i$

The transformed model

\begin{align*}
\frac{Y_i}{E(Y_i)}&=\frac{\beta_1}{E(Y_i)} + \beta_2 \frac{X_i}{E(Y_i)} + \frac{u_i}{E(Y_i)}\\
&=\beta_1\left(\frac{1}{E(Y_i)}\right) + \beta_2 \frac{X_i}{E(Y_i)} + v_i, \quad \quad (b)
\end{align*}

where $v_i=\frac{u_i}{E(Y_i)}$, and $E(v_i^2)=\sigma^2$ (a situation of homoscedasticity).

Note that the transformed model (b) is inoperational as $E(Y_i)$ depends on $\beta_1$ and $\beta_2$ which are unknown. We know $\hat{Y}_i = \hat{\beta}_1 + \hat{\beta}_2X_i$ which is an estimator of $E(Y_i)$. Therefore, we proceed in two steps.

Step 1: Run the usual OLS regression ignoring the presence of heteroscedasticity problem and obtain $\hat{Y}_i$.

Step 2: Use the estimate of $\hat{Y}_i$ to transform the model as

\begin{align*}
\frac{Y_i}{\hat{Y}_i}&=\frac{\beta_1}{\hat{Y}_i} + \beta_2 \frac{X_i}{\hat{Y}_i} + \frac{u_i}{\hat{Y}_i}\\
&=\beta_1\left(\frac{1}{\hat{Y}_i}\right) + \beta_2 \frac{X_i}{\hat{Y}_i} + v_i, \quad \quad (c)
\end{align*}

where $v_i=\frac{u_i}{\hat{Y}_i}$.

Although $\hat{Y}_i$ is not exactly $E(Y_i)$, they are consistent estimates (as the sample size increases indefinitely; $\hat{Y}_i$ converges to true $E(Y_i)$). Therefore, the transformed model (c) will perform well if the sample size is reasonably large.

Assumption 4: Log Transformation

A log transformation

$$ ln Y_i = \beta_1 + \beta_2 ln X_i + u_i \tag*{log model-1}$$ usually reduces heteroscedasticity when compared to the regression $$Y_i=\beta_1+\beta_2X_i + u_i $$

It is because log transformation compresses the scales in which the variables are measured, by reducing a tenfold (دس گنا) difference between two values to a twofold (دگنا) difference. For example, 80 is 10 times the number 8, but ln(80) = 4.3280 is about twice as large as ln(8) = 2.0794.

By taking the log transformation, the slope coefficient $\beta_2$ measures the elasticity of $Y$ concerning $X$ (that is, the percentage change in $Y$ for the percentage change in $X$).

If $Y$ is consumption and $X$ is income in the model (log model-1) then $\beta_2$ measures income elasticity, while in the original model (model without any transformation: OLS model), $\beta_2$ measures only the rate of change of mean consumption for a unit change in income.

Note that the log transformation is not applicable if some of the $Y$ and $X$ values are zero or negative.

Note regarding all assumptions about the nature of heteroscedasticity, we are essentially speculating (سوچنا، منصوبہ بنانا) about the nature of $\sigma_i^2$.

  • There may be a problem of spurious correlation. For example, in the model $$Y_i = \beta_1+\beta_2X_i + u_i,$$ the $Y$ and $X$ variables may not be correlation but in transformed model $$\frac{Y_i}{X_i}=\beta_1\left(\frac{1}{X_i}\right) + \beta_2,$$ the $\frac{Y_i}{X_i}$ and $\frac{1}{X_i}$ are often found to be correlated.
  • $\sigma_i^2$ are not directly known, we estimate them from one or more of the transformations. All testing procedures are valid only in large samples. Therefore, be careful in interpreting the results based on the various transformations in small or finite samples.
  • For a model with more than one explanatory variable, one may not know in advance, which of the $X$ variables should be chosen for transforming data.

Read more about Heteroscedasticity and Homoscedasticity on Wikipedia

Heteroscedasticity Consistent Standard Errors

Heteroscedasticity in Regression

Learn R Programming Language

Heteroscedasticity in Regression (2020)

by

Heteroscedasticity in Regression

Heteroscedasticity in Regression: The term heteroscedasticity refers to the violation of the assumption of homoscedasticity in linear regression models (LRM). In the case of heteroscedasticity, the errors have unequal variances for different levels of the regressors, which leads to biased and inefficient estimators of the regression coefficients. The disturbances in the Classical Linear Regression Model (CLRM) appearing in the population regression function should be homoscedastic; that is they all have the same variance.

Mathematical Proof of $E(\hat{\sigma}^2)\ne \sigma^2$ when there is some presence of hetero in the data.

For the proof of $E(\hat{\sigma}^2)\ne \sigma^2$, consider the two-variable linear regression model in the presence of heteroscedasticity,

\begin{align}
Y_i=\beta_1 + \beta_2 X+ u_i, \quad\quad (eq1)
\end{align}

where $Var(u_i)=\sigma_i^2$ (Case of heteroscedasticity)

as

\begin{align}
\hat{\sigma^2} &= \frac{\sum \hat{u}_i^2 }{n-2}\\
&= \frac{\sum (Y_i – \hat{Y}_i)^2 }{n-2}\\
&=\frac{(\beta_1 + \beta_2 X_i + u_i – \hat{\beta}_1 -\hat{\beta}_2 X_i )^2}{n-2}\\
&=\frac{\sum \left( -(\hat{\beta}_1-\beta_1) – (\hat{\beta}_2 – \beta_2)X_i + u_i \right)^2 }{n-2}\quad\quad (eq2)
\end{align}

Noting that

\begin{align*}
(Y_i-\hat{Y}_i)&=0\\
\beta_1 + \beta_2 X + u_i\, – \,\hat{\beta}_1 – \hat{\beta}_2X &=0\\
-(\hat{\beta}_1 -\beta_1) – X(\hat{\beta}_2-\beta_2) – u_i & =0\\
(\hat{\beta}_1 -\beta_1) &= – X (\hat{\beta}_2-\beta_2) + u_i\\
\text{Applying summation on both side}&\\
\sum (\hat{\beta}_1-\beta_1) &= -(\hat{\beta}_2-\beta_2)\sum X + \sum u_i\\
(\hat{\beta}_1 – \beta_1) &= -(\hat{\beta}_2-\beta_2)\overline{X}+\overline{u}
\end{align*}

Substituting it in (eq2) and taking expectation on both sides:

\begin{align}
\hat{\sigma}^2 &= \frac{1}{n-2} \left[ -(-(\hat{\beta}_2 – \beta_2) \overline{X} + \overline{u} ) – (\hat{\beta}_2-\beta_2)X_i + u_i  \right]^2\\
&=\frac{1}{n-2}E\left[(\hat{\beta}_2-\beta_2)\overline{X} -\overline{u} – (\hat{\beta}_2-\beta_2)X_i-u_i \right]^2\\
&=\frac{1}{n-2} E\left[ -(\hat{\beta}_2 – \beta_2)(X_i-\overline{X}) + (u_i-\overline{u})\right]^2\\
&= \frac{1}{n-2}\left[-\sum x_i^2 Var(\hat{\beta}_2) + E[\sum(u_i-\overline{u}]^2 \right]\\
&=\frac{1}{n-2} \left[ -\frac{\sum x_i^2 \sigma_i^2}{(\sum x_i^2)} + \frac{(n-1)\sum \sigma_i^2}{n} \right]
\end{align}

If there is homoscedasticity, then $\sigma_i^2=\sigma^2$ for each $i$, $E(\hat{\sigma}_i^2)=\sigma^2$.

The expected value of the $\hat{\sigma}^2=\frac{\hat{u}_i^2}{n-2}$ will not be equal to the true $\sigma^2$ in the presence of heteroscedasticity.

Heteroscedasticity in regression

To address heteroscedasticity in regression analysis, several techniques can be used to stabilize the variance of the errors:

  1. Transformations: Transforming the variables (such as using logarithmic or square root transformations) can sometimes help stabilize the variance of the errors.
  2. Weighted Least Squares (WLS): WLS is a method that assigns different weights to observations based on their variances, thereby giving more weight to observations with smaller variances. This may also help to mitigate the impact of heteroscedasticity on the estimation of parameters.
  3. Robust Standard Errors: heteroscedasticity-consistent standard errors also known as Robust standard errors, provide a way to correct standard errors and hypothesis tests in the presence of heteroscedasticity without requiring assumptions about the specific form of heteroscedasticity.
  4. Generalized Least Squares (GLS): The GLS method allows to estimation of regression coefficients under a broader range of assumptions about the variance-covariance structure of the errors, including heteroscedasticity.

Overall, detecting and addressing heteroscedasticity is important for ensuring the validity and reliability of regression analysis results.

Read more on the Remedy of Heteroscedasticity

More on heteroscedasticity on Wikipedia

MCQs General Knowledge

R Programming Language