Heteroscedasticity

The post is about “Heteroscedasticity Consistent Standard Errors and Variances.

$\sigma_i^2$ are rarely known. However, there is a way of obtaining consistent estimates of variances and covariances of OLS estimators even if there is heteroscedasticity.

White’s Heteroscedasticity Consistent Standard Errors and Variances

White’s heteroscedasticity-corrected standard errors are known as robust standard errors. White’s heteroscedasticity-corrected standard errors are larger (maybe smaller too) than the OLS standard errors and therefore, the estimated $t$-values are much smaller (or maybe larger) than those obtained by the OLS.

Comparing the OLS output with White’s heteroscedasticity consistent standard errors (variances) may be useful to see whether heteroscedasticity is a serious problem in a particular set of data.

Plausible Assumptions about Heteroscedasticity Patterns

Assumption 1: The error variance is proportional to $X_i^2$

$$E(u_i^2)=\sigma^2 X_i^2$$
It is believed that the variance of $u_i$ is proportional to the square of the $X$ (in graphical methods or Park and Glejser approaches).

One may transform the original model as follows:

\begin{align}\label{assump1}
\frac{Y_i}{X_i} &=\frac{\beta_1}{X_i} + \beta_2 + \frac{u_i}{X_i} \nonumber \\
&=\beta_1 \frac{1}{X_i} + \beta_2 + v_i,\qquad \qquad (1)
\end{align}

where $v_i$ is the transformed disturbance term, equal to $\frac{u_i}{X_i}$. It can be verified that

\begin{align*}
E(v_i^2) &=E\left(\frac{u_i}{X_i}\right)^2\\
&=\frac{1}{X_i^2}E(u_i^2)=\sigma^2
\end{align*}

Hence, the variance of $v_i$ is now homoscedastic, and one may apply OLS to the transformed equation by regressing $\frac{Y_i}{X_i}$ on $\frac{1}{X_i}$.

Notice that in the transformed regression the intercept term $\beta_2$ is the slope coefficient in the original equation and the slope coefficient $\beta_1$ is the intercept term in the original model. Therefore, to get back to the original model multiply the estimated equation (1) by $X_i$.

Assumption 2: The Error Variance is Proportional to $X_i$

The square root transformation: $E(u_i^2) = \sigma^2 X_i$

If it is believed that the variance of $u_i$ is proportional to $X_i$, then the original model can be transformed as

\begin{align*}
\frac{Y_i}{\sqrt{X_i}} &= \frac{\beta_1}{\sqrt{X_i}} + \beta_2 \sqrt{X_i} + \frac{u_i}{\sqrt{X_i}}\\
&=\beta_1 \frac{1}{\sqrt{X_i}} + \beta_2\sqrt{X_i}+v_i,\quad\quad (a)
\end{align*}

where $v_i=\frac{u_i}{\sqrt{X_i}}$ and $X_i>0$

$E(v_i^2)=\sigma^2$ (a homoscedastic situation)

One may proceed to apply OLS on equation (a), regressing $\frac{Y_i}{\sqrt{X_i}}$ on $\frac{1}{\sqrt{X_i}}$ and $\sqrt{X_i}$.

Note that the transformed model (a) has no intercept term. Therefore, use the regression through the origin model to estimate $\beta_1$ and $\beta_2$. To get back the original model simply multiply the equation (a) by $\sqrt{X_i}$.

Consider a case of $intercept = 0$, that is, $Y_i=\beta_2X_i+u_i$. The transformed model will be

\begin{align*}
\frac{Y_i}{\sqrt{X_i}} &= \beta_2 \sqrt{X_i} + \frac{u_i}{\sqrt{X_i}}\\
\beta_2 &=\frac{\overline{Y}}{\overline{X}}
\end{align*}

Here, the WLS estimator is simply the ratio of the means of the dependent and explanatory variables.

Assumption 3: The Error Variance is proportional to the Square of the Mean value of $Y$

$$E(u_i^2)=\sigma^2[E(Y_i)]^2$$

The original model is $Y_i=\beta_1 + \beta_2 X_i + u_I$ and $E(Y_i)=\beta_1 + \beta_2X_i$

The transformed model

\begin{align*}
\frac{Y_i}{E(Y_i)}&=\frac{\beta_1}{E(Y_i)} + \beta_2 \frac{X_i}{E(Y_i)} + \frac{u_i}{E(Y_i)}\\
&=\beta_1\left(\frac{1}{E(Y_i)}\right) + \beta_2 \frac{X_i}{E(Y_i)} + v_i, \quad \quad (b)
\end{align*}

where $v_i=\frac{u_i}{E(Y_i)}$, and $E(v_i^2)=\sigma^2$ (a situation of homoscedasticity).

Note that the transformed model (b) is inoperational as $E(Y_i)$ depends on $\beta_1$ and $\beta_2$ which are unknown. We know $\hat{Y}_i = \hat{\beta}_1 + \hat{\beta}_2X_i$ which is an estimator of $E(Y_i)$. Therefore, we proceed in two steps.

Step 1: Run the usual OLS regression ignoring the presence of heteroscedasticity problem and obtain $\hat{Y}_i$.

Step 2: Use the estimate of $\hat{Y}_i$ to transform the model as

\begin{align*}
\frac{Y_i}{\hat{Y}_i}&=\frac{\beta_1}{\hat{Y}_i} + \beta_2 \frac{X_i}{\hat{Y}_i} + \frac{u_i}{\hat{Y}_i}\\
&=\beta_1\left(\frac{1}{\hat{Y}_i}\right) + \beta_2 \frac{X_i}{\hat{Y}_i} + v_i, \quad \quad (c)
\end{align*}

where $v_i=\frac{u_i}{\hat{Y}_i}$.

Although $\hat{Y}_i$ is not exactly $E(Y_i)$, they are consistent estimates (as the sample size increases indefinitely; $\hat{Y}_i$ converges to true $E(Y_i)$). Therefore, the transformed model (c) will perform well if the sample size is reasonably large.

Assumption 4: Log Transformation

A log transformation

$$ ln Y_i = \beta_1 + \beta_2 ln X_i + u_i \tag*{log model-1}$$ usually reduces heteroscedasticity when compared to the regression $$Y_i=\beta_1+\beta_2X_i + u_i $$

It is because log transformation compresses the scales in which the variables are measured, by reducing a tenfold (دس گنا) difference between two values to a twofold (دگنا) difference. For example, 80 is 10 times the number 8, but ln(80) = 4.3280 is about twice as large as ln(8) = 2.0794.

By taking the log transformation, the slope coefficient $\beta_2$ measures the elasticity of $Y$ concerning $X$ (that is, the percentage change in $Y$ for the percentage change in $X$).

If $Y$ is consumption and $X$ is income in the model (log model-1) then $\beta_2$ measures income elasticity, while in the original model (model without any transformation: OLS model), $\beta_2$ measures only the rate of change of mean consumption for a unit change in income.

Note that the log transformation is not applicable if some of the $Y$ and $X$ values are zero or negative.

Note regarding all assumptions about the nature of heteroscedasticity, we are essentially speculating (سوچنا، منصوبہ بنانا) about the nature of $\sigma_i^2$.

• There may be a problem of spurious correlation. For example, in the model $$Y_i = \beta_1+\beta_2X_i + u_i,$$ the $Y$ and $X$ variables may not be correlation but in transformed model $$\frac{Y_i}{X_i}=\beta_1\left(\frac{1}{X_i}\right) + \beta_2,$$ the $\frac{Y_i}{X_i}$ and $\frac{1}{X_i}$ are often found to be correlated.
• $\sigma_i^2$ are not directly known, we estimate them from one or more of the transformations. All testing procedures are valid only in large samples. Therefore, be careful in interpreting the results based on the various transformations in small or finite samples.
• For a model with more than one explanatory variable, one may not know in advance, which of the $X$ variables should be chosen for transforming data.

Read more about Heteroscedasticity and Homoscedasticity on Wikipedia

Heteroscedasticity in Regression

Learn R Programming Language