Coefficient of Determination: A model Selection Criteria

$R^2$ pronounced R-Squared (Coefficient of determination) is a useful statistics to check the value of regression fit. $R^2$ measures the proportion of total variation about the mean $\bar{Y}$ explained by the regression. R is the correlation between $Y$ and $\hat{Y}$ and is usually the multiple correlation coefficient. Coefficient of determination ($R^2$) can take values as high as 1 or  (100%) when all the values are different i.e. $0\le R^2\le 1$. When repeats runs exist in the data the value of $R^2$ cannot attain 1, no matter how well model fits, because no model can explain the variation in the data due to the pure error. A perfect fit to data for which $\hat{Y}_i=Y_i$, $R^2=1$. If $\hat{Y}_i=\bar{Y}$, that is if $\beta_1=\beta_2=\cdots=\beta_{p-1}=0$ or if a model $Y=\beta_0 +\varepsilon$ alone has been fitted, then $R^2=0$. Therefore we can say that $R^2$ is a measure of the usefulness of the terms, other than $\beta_0$ in the model.

Note that we must sure that an improvement/ increase in $R^2$ value due to adding a new term (variable) to the model under study should have some real significance and is not due to the fact that the number of parameters in the model is getting else to saturation point. If there is no pure error $R^2$ can be made unity.

\begin{align*}
R^2 &= \frac{\text {SS due to regression given}\, b_0}{\text{Total SS corrected for mean} \, \bar{Y}} \\
&= \frac{SS \, (b_1 | b_0)}{S_{YY}} \\
&= \frac{\sum(\hat{Y_i}-\bar{Y})^2} {\sum(Y_i-\bar{Y})^2}r \\
&= \frac{S^2_{XY}}{(S_{XY})(S_{YY})}
\end{align*}

where summation are over i=1,2,…,n.

Note that when interpreting R-Square $R^2$ does not indicate whether:

• the independent variables (explanatory variables) are a cause of the changes in the dependent variable;
• omitted-variable bias exists;
• the correct regression was used;
• the most appropriate set of explanatory variables has been selected;
• there is collinearity (or multicollinearity) present in the data;
• the model might be improved by using transformed versions of the existing set of explanatory variables.

Learn more about

• Pearson’s Correlation Coefficient use, Interpretation, Properties, and Coefficient of Determination
• Coefficient of Determination as Model Selection Criteria
• Range of Correlation Coefficient
• Coefficient of Determination on Wikipedia