Statistics for Data Science & Analytics

The coefficient of correlation is a statistic used to measure the strength and direction of the linear relationship between two Quantitative variables.

Properties of Correlation Coefficient

Understanding these properties helps us to interpret the correlation coefficient accurately and avoid misinterpretations. The following are some important Properties of Correlation Coefficient.

• The correlation coefficient ($r$) between $X$ and $Y$ is the same as the correlation between $Y$ and $X$. that is the correlation is symmetric with respect to $X$ and $Y$, i.e., $r_{XY} = r_{YX}$.
• The $r$ ranges from $-1$ to $+1$, i.e., $-1\le r \le +1$.
• There is no unit of $r$. The correlation coefficient $r$ is independent of the unit of measurement.
• It is not affected by the change of origin and scale, i.e., $r_{XY}=r_{YX}$. If a constant is added to each value of a variable, it is called a change of origin and if each value of a variable is multiplied by a constant, it is called a change of scale.
• The $r$ is the geometric mean of two regression coefficients, i.e., $\sqrt{b_{YX}\times b_{XY}}$.
  In other words, if the two regression lines of $Y$ on $X$ and $X$ on $Y$ are written as $Y=a+bX$ and $X=c+dy$ respectively then $bd=r^2$.
• The sign of $r_{XY}, b_{YX}$, and $b_{XY}$ is dependent on covariance which is common in the three as given below:
• $r=\frac{Cov(X, Y)}{\sqrt{Var(X) Var(Y)}},\,\, b_{YX} = \frac{Cov(Y, X)}{Var(X)}, \,\, b_{XY}=\frac{Cov(Y, X)}{Var(Y)}$

Hence, $r_{YX}, b_{YX}$, and $b_{XY}$ have the same sign.

• If $r=-1$ the correlation is perfectly negative, meaning as one variable increases the other increases proportionally.
• If $r=+1$ the correlation is perfectly positive, meaning as one variable increases the other decreases proportionally.
• If $r=0$ there is no correlation, i.e., there is no linear relationship between the variables. However, a non-linear relationship may exist but it does not necessarily mean that the variables are independent.

Examples of Correlation Coefficient

The following are some real-life examples of correlation coefficients (ranging from -1 to +1) to illustrate relationships between variables:

Positive Correlation ($r$ close to +1)

• The relationship between study time and exam scores: As students spend more time studying, their exam scores tend to increase. A correlation coefficient of $r = 0.85$ indicates a strong positive relationship.
• The relationship between advertising spending and sales revenue: Companies that invest more in advertising often see higher sales. A correlation coefficient of $r = 0.70$ suggests a strong positive link.

Negative Correlation ($r$ close to -1)

• The relationship between hours spent on social media and academic performance: As students spend more time on social media, their grades may decline. A correlation coefficient of $r = -0.65$ indicates a moderate negative relationship.
• The relationship between temperature and heating costs: As outdoor temperatures rise, heating costs tend to decrease. A correlation coefficient of $r = -0.90$ shows a strong negative correlation.

Weak or No Correlation ($r$ close to 0)

• The relationship between shoe size and IQ: There is no logical connection between shoe size and intelligence. A correlation coefficient of $r = 0.05$ indicates almost no correlation.
• The relationship between rainfall and stock market performance: Rainfall has no direct impact on stock market trends. A correlation coefficient of $r = -0.10$ suggests a very weak or negligible relationship.

Real-World Applications

• Healthcare: The correlation between exercise frequency and heart health.
• Economics: The correlation between unemployment rates and crime rates.
• Education: The correlation between parental income and children’s academic achievement.
• Environment: The correlation between carbon emissions and global temperatures.

Independence of Origin and Scale

Theorem: Correlation: Independent of Origin and Scale. Show that the correlation coefficient is independent of origin and scale, i.e., $r_{XY}=r_{uv}$.

Proof: The formula for correlation coefficient is,

$$r_{XY}=\frac{\varSigma(X-\overline{X})((Y-\overline{Y})) }{\sqrt{[\varSigma(X-\overline{X})^2][\varSigma(Y-\overline{Y})^2]}}$$

\begin{align*}
\text{Let}\quad u&=\frac{X-a}{h}\\
\Rightarrow X&=a+hu \Rightarrow \overline{X}=a+h\overline{u} \\
\text{and}\quad v&=\frac{Y-b}{K}\\
\Rightarrow Y&=b+Kv \Rightarrow \overline{Y}=b+K\overline{v}\\
\text{Therefore}\\
r_{uv}&=\frac{\varSigma(u-\overline{u})((v-\overline{v})) }{\sqrt{[\varSigma(u-\overline{u})^2][\varSigma(v-\overline{v})^2]}}\\
&=\frac{\varSigma (a+hu-a-h\overline{u}) (b+Kv-b-K\overline{v})} {\sqrt{\varSigma(a+hu-a-h\overline{u})^2\varSigma(b+Kv-b-K\overline{v})^2}}\\
&=\frac{\varSigma(hu-h\overline{u})(Kv-K\overline{v})}{\sqrt{[\varSigma(hu-h\overline{u})^2][\varSigma(Kv-K\overline{v})^2]}}\\
&=\frac{hK\varSigma(u-\overline{u})(v-\overline{v})}{\sqrt{[h^2 K^2 \varSigma(u-\overline{u})^2] [\varSigma(v-\overline{v})^2]}}\\
&=\frac{hK\varSigma(u-\overline{u})(v-\overline{v})}{hK\,\sqrt{[\varSigma(u-\overline{u})^2] [\varSigma(v-\overline{v})^2]}}\\
&=\frac{\varSigma(u-\overline{u})(v-\overline{v}) }{\sqrt{[\varSigma(u-\overline{u})^2][\varSigma(v-\overline{v})^2]}}=
r_{uv}
\end{align*}

Important Points about Correlation Analysis

1. Non-causality: Correlation does not imply causation. If two variables are strongly correlated, it does not necessarily mean that changes in one variable cause changes in the other. This is because the correlation only measures the strength and direction of the linear relationship between two quantitative variables, not the underlying cause-and-effect relationship.
2. Sensitive to Outliers: The correlation coefficient can be sensitive to outliers, as outliers can disproportionately influence the correlation calculation.
3. Assumption of Linearity: The correlation coefficient measures the linear relationship between variables. It may not accurately capture non-linear relationships between variables.
4. Scale Invariance: The correlation coefficient is independent of the scale of the data. That is, multiplying or dividing all the values of one or both variables by a constant will not affect the strength and direction of the correlation coefficient. This makes it useful for comparing relationships between variables measured in different units.
5. Strength vs. Causation: A high correlation does not necessarily imply causation. It is because two variables are strongly correlated does not mean one causes the other. There might be a third unknown factor influencing both variables. Correlation analysis is a good starting point for exploring relationships, but further investigation is needed to establish causality.

https://gmstat.com

https://rfaqs.com