The coefficient of correlation is a statistic used to measure the strength and direction of the linear relationship between two Quantitative variables.
Properties of Correlation Coefficient
The following are some important Properties of Correlation Coefficient.
- The $r$ is symmetric with respect to $X$ and $Y$, i.e., $r_{XY} = r_{YX}$.
- The $r$ ranges from $-1$ to $+1$, i.e., $-1\le r \le +1$.
- There is no unit of $r$. The correlation coefficient $r$ is independent of the unit of measurement.
- It is not affected by the change of origin and scale, i.e., $r_{XY}=r_{YX}$. If a constant is added to each value of a variable, it is called a change of origin and if each value of a variable is multiplied by a constant, it is called a change of scale.
- The $r$ is the geometric mean of two regression coefficients, i.e., $\sqrt{b_{YX}\times b_{XY}}$.
In other words, if the two regression lines of $Y$ on $X$ and $X$ on $Y$ are written as $Y=a+bX$ and $X=c+dy$ respectively then $bd=r^2$. - The sign of $r_{XY}, b_{YX}$, and $b_{XY}$ is dependent on covariance which is common in the three as given below:
- $r=\frac{Cov(X, Y)}{\sqrt{Var(X) Var(Y)}},\,\, b_{YX} = \frac{Cov(Y, X)}{Var(X)}, \,\, b_{XY}=\frac{Cov(Y, X)}{Var(Y)}$
Hence, $r_{YX}, b_{YX}$, and $b_{XY}$ have the same sign.
- If $r=-1$ the correlation is perfectly negative.
- If $r=+1$ the correlation is perfectly positive.
- If $r=0$ there is no correlation, i.e., there is no linear relationship between the variables. However, a non-linear relationship may exist but it does not necessarily mean that the variables are independent.
Theorem: Correlation: Independent of Origin and Scale. Show that the correlation coefficient is independent of origin and scale, i.e., $r_{XY}=r_{uv}$.
Proof: The formula for correlation coefficient is,
$$r_{XY}=\frac{\varSigma(X-\overline{X})((Y-\overline{Y})) }{\sqrt{[\varSigma(X-\overline{X})^2][\varSigma(Y-\overline{Y})^2]}}$$
\begin{align*}
\text{Let}\quad u&=\frac{X-a}{h}\\
\Rightarrow X&=a+hu \Rightarrow \overline{X}=a+h\overline{u} \\
\text{and}\quad v&=\frac{Y-b}{K}\\
\Rightarrow Y&=b+Kv \Rightarrow \overline{Y}=b+K\overline{v}\\
\text{Therefore}\\
r_{uv}&=\frac{\varSigma(u-\overline{u})((v-\overline{v})) }{\sqrt{[\varSigma(u-\overline{u})^2][\varSigma(v-\overline{v})^2]}}\\
&=\frac{\varSigma (a+hu-a-h\overline{u}) (b+Kv-b-K\overline{v})} {\sqrt{\varSigma(a+hu-a-h\overline{u})^2\varSigma(b+Kv-b-K\overline{v})^2}}\\
&=\frac{\varSigma(hu-h\overline{u})(Kv-K\overline{v})}{\sqrt{[\varSigma(hu-h\overline{u})^2][\varSigma(Kv-K\overline{v})^2]}}\\
&=\frac{hK\varSigma(u-\overline{u})(v-\overline{v})}{\sqrt{[h^2 K^2 \varSigma(u-\overline{u})^2] [\varSigma(v-\overline{v})^2]}}\\
&=\frac{hK\varSigma(u-\overline{u})(v-\overline{v})}{hK\,\sqrt{[\varSigma(u-\overline{u})^2] [\varSigma(v-\overline{v})^2]}}\\
&=\frac{\varSigma(u-\overline{u})(v-\overline{v}) }{\sqrt{[\varSigma(u-\overline{u})^2][\varSigma(v-\overline{v})^2]}}=
r_{uv}
\end{align*}
Note that
- Non-causality: Correlation does not imply causation. If two variables are strongly correlated, it does not necessarily mean that changes in one variable cause changes in the other. This is because the correlation only measures the strength and direction of the linear relationship between two quantitative variables, not the underlying cause-and-effect relationship.
- Sensitive to Outliers: The correlation coefficient can be sensitive to outliers, as outliers can disproportionately influence the correlation calculation.
- Assumption of Linearity: The correlation coefficient measures the linear relationship between variables. It may not accurately capture non-linear relationships between variables.