Normality of Error Term
In multiple linear regression models, the sum of squared residuals (SSR) is divided by $n-p$ (degrees of freedom, where $n$ is the total number of observations, and $p$ is the number of the parameter in the model) is a good estimate of the error variance. In the multiple linear regression model, the residual vector is
\begin{align*}
e &=(I-H)y\\
&=(I-H)(X\beta+e)\\
&=(I-H)\varepsilon
\end{align*}
where $H$ is the hat matrix for the regression model.
Each component $e_i=\varepsilon – \sum\limits_{i=1}^n h_{ij} \varepsilon_i$. Therefore, In multiple linear regression models, the normality of the residual is not simply the normality of the error term.
Note that:
\[Cov(\mathbf{e})=(I-H)\sigma^2 (I-H)’ = (I-H)\sigma^2\]
We can write $Var(e_i)=(1-h_{ii})\sigma^2$.
If the sample size ($n$) is much larger than the number of the parameters ($p$) in the model (i.e. $n > > p$), in other words, if sample size ($n$) is large enough, $h_{ii}$ will be small as compared to 1, and $Var(e_i) \approx \sigma^2$.
In multiple regression models, a residual behaves like an error if the sample size is large. However, this is not true for a small sample size.
It is unreliable to check the normality of error term assumption using residuals from multiple linear regression models when the sample size is small.
Learn more about Hat matrix: Role of Hat matrix in Diagnostics of Regression Analysis.
Learn R Programming Language
