The coefficient of correlation is a statistic used to measure the strength and direction of the linear relationship between two Quantitative variables.
Properties of Correlation Coefficient
Understanding these properties helps us to interpret the correlation coefficient accurately and avoid misinterpretations. The following are some important Properties of Correlation Coefficient.
- The correlation coefficient ($r$) between $X$ and $Y$ is the same as the correlation between $Y$ and $X$. that is the correlation is symmetric with respect to $X$ and $Y$, i.e., $r_{XY} = r_{YX}$.
- The $r$ ranges from $-1$ to $+1$, i.e., $-1\le r \le +1$.
- There is no unit of $r$. The correlation coefficient $r$ is independent of the unit of measurement.
- It is not affected by the change of origin and scale, i.e., $r_{XY}=r_{YX}$. If a constant is added to each value of a variable, it is called a change of origin and if each value of a variable is multiplied by a constant, it is called a change of scale.
- The $r$ is the geometric mean of two regression coefficients, i.e., $\sqrt{b_{YX}\times b_{XY}}$.
In other words, if the two regression lines of $Y$ on $X$ and $X$ on $Y$ are written as $Y=a+bX$ and $X=c+dy$ respectively then $bd=r^2$. - The sign of $r_{XY}, b_{YX}$, and $b_{XY}$ is dependent on covariance which is common in the three as given below:
- $r=\frac{Cov(X, Y)}{\sqrt{Var(X) Var(Y)}},\,\, b_{YX} = \frac{Cov(Y, X)}{Var(X)}, \,\, b_{XY}=\frac{Cov(Y, X)}{Var(Y)}$
Hence, $r_{YX}, b_{YX}$, and $b_{XY}$ have the same sign.
- If $r=-1$ the correlation is perfectly negative, meaning as one variable increases the other increases proportionally.
- If $r=+1$ the correlation is perfectly positive, meaning as one variable increases the other decreases proportionally.
- If $r=0$ there is no correlation, i.e., there is no linear relationship between the variables. However, a non-linear relationship may exist but it does not necessarily mean that the variables are independent.
Theorem: Correlation: Independent of Origin and Scale. Show that the correlation coefficient is independent of origin and scale, i.e., $r_{XY}=r_{uv}$.
Proof: The formula for correlation coefficient is,
$$r_{XY}=\frac{\varSigma(X-\overline{X})((Y-\overline{Y})) }{\sqrt{[\varSigma(X-\overline{X})^2][\varSigma(Y-\overline{Y})^2]}}$$
\begin{align*}
\text{Let}\quad u&=\frac{X-a}{h}\\
\Rightarrow X&=a+hu \Rightarrow \overline{X}=a+h\overline{u} \\
\text{and}\quad v&=\frac{Y-b}{K}\\
\Rightarrow Y&=b+Kv \Rightarrow \overline{Y}=b+K\overline{v}\\
\text{Therefore}\\
r_{uv}&=\frac{\varSigma(u-\overline{u})((v-\overline{v})) }{\sqrt{[\varSigma(u-\overline{u})^2][\varSigma(v-\overline{v})^2]}}\\
&=\frac{\varSigma (a+hu-a-h\overline{u}) (b+Kv-b-K\overline{v})} {\sqrt{\varSigma(a+hu-a-h\overline{u})^2\varSigma(b+Kv-b-K\overline{v})^2}}\\
&=\frac{\varSigma(hu-h\overline{u})(Kv-K\overline{v})}{\sqrt{[\varSigma(hu-h\overline{u})^2][\varSigma(Kv-K\overline{v})^2]}}\\
&=\frac{hK\varSigma(u-\overline{u})(v-\overline{v})}{\sqrt{[h^2 K^2 \varSigma(u-\overline{u})^2] [\varSigma(v-\overline{v})^2]}}\\
&=\frac{hK\varSigma(u-\overline{u})(v-\overline{v})}{hK\,\sqrt{[\varSigma(u-\overline{u})^2] [\varSigma(v-\overline{v})^2]}}\\
&=\frac{\varSigma(u-\overline{u})(v-\overline{v}) }{\sqrt{[\varSigma(u-\overline{u})^2][\varSigma(v-\overline{v})^2]}}=
r_{uv}
\end{align*}
Note that
- Non-causality: Correlation does not imply causation. If two variables are strongly correlated, it does not necessarily mean that changes in one variable cause changes in the other. This is because the correlation only measures the strength and direction of the linear relationship between two quantitative variables, not the underlying cause-and-effect relationship.
- Sensitive to Outliers: The correlation coefficient can be sensitive to outliers, as outliers can disproportionately influence the correlation calculation.
- Assumption of Linearity: The correlation coefficient measures the linear relationship between variables. It may not accurately capture non-linear relationships between variables.
- Scale Invariance: The correlation coefficient is independent of the scale of the data. That is, multiplying or dividing all the values of one or both variables by a constant will not affect the strength and direction of correlation coefficient. This makes it useful for comparing relationships between variables measured in different units.
- Strength vs. Causation: A high correlation does not necessarily imply causation. It is because two variables are strongly correlated does not mean one causes the other. There might be a third unknown factor influencing both variables. Correlation analysis is a good starting point for exploring relationships, but further investigation is needed to establish causality.
