Statistics for Data Science & Analytics - Learn Statistics: MCQs, Software & Data Analysi

Correlation Coefficient Range

Apr 4, 2025Oct 12, 2012 by Muhammad Imdad Ullah

Post Views: 939

The coefficient of correlation (r) measures the strength and direction of a linear relationship between two variables. In this post, we will discuss about coefficient of correlation and the coefficient of determination.

Correlation Coefficient Ranges

The correlation coefficient ranges from -1 to +1, where a value of +1 indicates the perfect positive correlation (as one variable increases, the other increases proportionally), the -1 value indicates the perfect negative correlation (as one variable increases, the other decreases proportionally), and the value of 0 indicates no linear correlation (no relationship between the variables).

The coefficient of correlation values between -1 and +1 indicates the degree of strength and direction of relationship:

The strength of correlation depends on the absolute value of r:

Range of Correlation Value	Interpretation
0.90 to 1.00	Very strong correlation
0.70 to 0.89	Strong correlation
0.40 to 0.69	Moderate correlation
0.10 to 0.39	Weak correlation
0.00 to 0.09	No or negligible correlation

The closer the value of the correlation coefficient is to ±1, the stronger the linear relationship.

Coefficient of Determination

We know that the ratio of the explained variation to the total variation is called the coefficient of determination, which is the square of the Correlation Coefficient Range and lies between $- 1$ and $+ 1$ . This ratio (coefficient of determination) is non-negative; therefore, denoted by $r^{2}$ , thus

$\begin{aligned} r^{2} & = \frac{Explained Variation}{Total Variation} \\ = \frac{\sum (\hat{Y} - \overset{―}{Y})^{2}}{\sum (Y - \overset{―}{Y})^{2}} \end{aligned}$

It can be seen that if the total variation is all explained, the ratio $r^{2}$ (Coefficient of Determination) is one, and if the total variation is all unexplained, then the explained variation and the ratio $r^{2}$ are zero.

The square root of the coefficient of determination is called the correlation coefficient, given by

$\begin{aligned} r & = \sqrt{\frac{Explained Variation}{Total Variation}} \\ = \pm \sqrt{\frac{\sum (\hat{Y} - \overset{―}{Y})^{2}}{\sum (Y - \overset{―}{Y})^{2}}} \end{aligned}$

and

$\sum (\hat{Y} - \overset{―}{Y})^{2} = \sum (Y - \overset{―}{Y})^{2} - \sum (Y - \hat{Y})^{2}$

Therefore

$\begin{aligned} r & = \sqrt{\frac{\sum (Y - \overset{―}{Y})^{2} - \sum (Y - \hat{Y})^{2}}{\sum (Y - \overset{―}{Y})^{2}}} \\ = \sqrt{1 - \frac{\sum (Y - \hat{Y})^{2}}{\sum (Y - \overset{―}{Y})^{2}}} \\ = \sqrt{1 - \frac{Unexplained Variation}{Total Variation}} = \sqrt{1 - \frac{S_{y . x}^{2}}{s_{y}^{2}}} \end{aligned}$

where $s_{y . x}^{2} = \frac{1}{n} \sum (Y - \hat{Y})^{2}$ and $s_{y}^{2} = \frac{1}{n} \sum (Y - \overset{―}{Y})^{2}$

$\begin{aligned} \Rightarrow r^{2} & = 1 - \frac{s_{y . x}^{2}}{s_{y}^{2}} \\ \Rightarrow s_{y . x}^{2} & = s_{y}^{2} (1 - r^{2}) \end{aligned}$

Since variances are non-negative

$\frac{s_{y . x}^{2}}{s_{y}^{2}} = 1 - r^{2} \geq 0$

Solving for inequality, we have

$\begin{aligned} 1 - r^{2} & \geq 0 \\ \Rightarrow r^{2} \leq 1 or | r | & \leq 1 \\ \Rightarrow & - 1 \leq r \leq 1 \end{aligned}$

Therefore, the Correlation Coefficient Range lies between $- 1$ and $+ 1$ inclusive.

Alternative Proof: Correlation Coefficient Range

Since $ρ (X, Y) = ρ (X^{*}, Y^{*})$ where $X^{*} = \frac{X - μ_{X}}{σ_{X}}$ and $Y^{*} = \frac{Y - Y^{*}}{σ_{Y}}$

and as covariance is bi-linear and $X^{*}, Y^{*}$ have zero mean and variance 1, therefore

$\begin{aligned} ρ (X^{*}, Y^{*}) & = C o v (X^{*}, Y^{*}) = C o v {\frac{X - μ_{X}}{σ_{X}}, \frac{Y - μ_{Y}}{σ_{Y}}} \\ = \frac{C o v (X - μ_{X}, Y - μ_{Y})}{σ_{X} σ_{Y}} \\ = \frac{C o v (X, Y)}{σ_{X} σ_{Y}} = ρ (X, Y) \end{aligned}$

We also know that the variance of any random variable is $\geq 0$ , it could be zero i.e. $(V a r (X) = 0)$ if and only if $X$ is a constant (almost surely), therefore

$V (X^{*} \pm Y^{*}) = V (X^{*}) + V (Y^{*}) \pm 2 C o v (X^{*}, Y^{*})$

As $V a r (X^{*}) = 1$ and $V a r (Y^{*}) = 1$ , the above equation would be negative if $C o v (X^{*}, Y^{*})$ is either greater than 1 or less than -1. Hence $1 \geq ρ (X, Y) = ρ (X^{*}, Y^{*}) \geq - 1$ .

If $Math input error$ then $Math input error$ making $X^{*} = Y^{*}$ almost surely. Similarly, if $Math input error$ then $X^{*} = - Y^{*}$ almost surely. In either case, $Y$ would be a linear function of $X$ almost surely.

For proof of Cauchy-Schwarz Inequality, please follow the link

We can see that the Correlation Coefficient range lies between $- 1$ and $+ 1$ .

Real-Life Example

Variable 1	Variable 2	Coefficient Value	Interpretation
Study hours	Exam scores	+0.85	Strong positive
Screen time	Sleep duration	-0.70	Strong negative
Age	Shoe size	~0.00	No linear correlation

FAQs about Correlation Coefficient

What is a coefficient of correlation?
What does a positive or negative correlation mean?
What is a strong or weak correlation?
Can correlation imply causation?
What are the types of correlation coefficients?
When should I use Pearson vs. Spearman correlation?
What are the assumptions of the Pearson correlation?
Can correlation be used for more than two variables?
How is correlation different from regression?
How is the correlation coefficient calculated?
What does a zero correlation mean?
Can correlation be misleading?

Learn more about

Multivariate Data Sets: Descriptive Statistics (2010)

May 23, 2024Oct 10, 2012 by Muhammad Imdad Ullah

Post Views: 433

Much of the information contained in the multivariate data sets can be assessed by calculating certain summary numbers, known as multivariate descriptive statistics such as Arithmetic mean (a measure of location), an average of the squares of the distances of all of the numbers from the mean (variation/spread i.e. measure of spread or variation), etc. Here we will discuss descriptive statistics and multivariate data sets.

Multivariate data sets are used in various fields, such as:

Social Sciences: Analyzing factors influencing social phenomena like voting behavior, educational attainment, or health outcomes.
Business: Understanding customer demographics and purchase patterns, market research, risk assessment, and financial modeling.
Natural Sciences: Studying relationships between environmental variables, analyzing climate data, or exploring genetic factors influencing diseases.

Multivariate Data Sets: Descriptive Analysis

We shall rely most heavily on descriptive statistics which is a measure of location, variation, and linear association. For descriptive statistics multivariate data set, let us start with a measure of location, a measure of spread, sample covariance, and sample correlation coefficient.

Measure of Location

The arithmetic average of $n$ measurements $(x_{11}, x_{21}, x_{31}, x_{41})$ on the first variable (defined in Multivariate Analysis: An Introduction) is

Sample Mean = $\bar{x} = \frac{1}{n} \sum_{j = 1}^{n} x_{j 1} where j = 1, 2, 3, \dots, n$

The sample mean for $n$ measurements on each of the p variables (there will be p sample means)

${\bar{x}}_{k} = \frac{1}{n} \sum_{j = 1}^{n} x_{j k} where k = 1, 2, \dots, p$

Measure of Spread

Measure of spread (variance) for $n$ measurements on the first variable for multivariate data sets can be found as
$s_{1}^{2} = \frac{1}{n} \sum_{j = 1}^{n} (x_{j 1} - {\bar{x}}_{1})^{2}$ where ${\bar{x}}_{1}$ is sample mean of the $x_{j}$ ’s for p variables.

Measure of spread (variance) for $n$ measurements on all variables can be found as

$s_{k}^{2} = \frac{1}{n} \sum_{j = 1}^{n} (x_{j k} - {\bar{x}}_{k})^{2} where k = 1, 2, \dots, p and j = 1, 2, \dots, p$

The Square Root of the sample variance is the sample standard deviation i.e

$S_{l}^{2} = S_{k k} = \frac{1}{n} \sum_{j = 1}^{n} (x_{j k} - {\bar{x}}_{k})^{2} where k = 1, 2, \dots, p$

Sample Covariance

Consider $n$ pairs of measurement on each of Variable 1 and Variable 2
$[\begin{matrix} x_{11} \\ x_{12} \end{matrix}], [\begin{matrix} x_{21} \\ x_{22} \end{matrix}], \dots, [\begin{matrix} x_{n 1} \\ x_{n 2} \end{matrix}]$
That is $x_{j 1}$ and $x_{j 2}$ are observed on the jth experimental item $(j = 1, 2, \dots, n)$ . So a measure of linear association between the measurements of $V_{1}$ and $V_{2}$ for multivariate data sets is provided by the sample covariance
$s_{12} = \frac{1}{n} \sum_{j = 1}^{n} (x_{j 1} - {\bar{x}}_{1}) (x_{j 2} - {\bar{x}}_{2})$
(the average product of the deviation from their respective means) therefore

$s_{i k} = \frac{1}{n} \sum_{j = 1}^{n} (x_{j i} - {\bar{x}}_{i}) (x_{j k} - {\bar{x}}_{k})$ ; $i = 1, 2, \dots, p$ and $k = 1, 2, \dots, p$ .

It measures the association between the kth variable.

Variance is the most commonly used measure of dispersion (variation) in the data and it is directly proportional to the amount of variation or information available in the data.

Sample Correlation Coefficient

For Multivariate Data Sets, the sample correlation coefficient for the ith and kth variables is

$r_{i k} = \frac{s_{i k}}{\sqrt{s_{i i}} \sqrt{s_{k k}}} = \frac{\sum_{j = 1}^{n} (x_{j i} - {\bar{x}}_{j}) (x_{j k} - {\bar{x}}_{k})}{\sqrt{\sum_{j = 1}^{n} (x_{j i} - {\bar{x}}_{i})^{2}} \sqrt{\sum_{j = 1}^{n} (x_{j k} - {\bar{x}}_{k})^{2}}}$
$where i = 1, 2, . ., p and k = 1, 2, \dots, p$

Note that $r_{i k} = r_{k i}$ for all $i$ and $k$ , and $r$ lies between $- 1$ and $+ 1$ . $r$ measures the strength of the linear association. If $r = 0$ the lack of linear association between the components exists. The sign of $r$ indicates the direction of the association.

Other Multivariate Analysis

Multiple Regression: It is used to model the relationship between a dependent variable (DV) and multiple independent variables (IV).

Principal Component Analysis (PCA): It reduces the dimensionality of data by identifying a smaller set of uncorrelated variables that capture most of the data’s variance.

Cluster Analysis: It groups the data points into clusters based on their similarities, helping identify subgroups within the data.

Discriminant Analysis: It classifies data points into predefined groups based on their characteristics.

Learn the use of matrices in R Language

Online MCQs Economics

Pearson Correlation Coefficient

Mar 15, 2025Sep 20, 2012 by Muhammad Imdad Ullah

Post Views: 1,138

Introduction to Pearson Correlation Coefficient

The correlation coefficient or Pearson Correlation Coefficient was originated by Karl Pearson in the 1900s. The Pearson Correlation Coefficient is a measure of the (degree of) strength of the linear relationship between two continuous random variables denoted by $ρ_{X Y}$ for population and for sample it is denoted by $r_{X Y}$ .

The Pearson Correlation coefficient can take values that occur in the interval $[1, - 1]$ . If the coefficient value is $1$ or $- 1$ , there will be a perfect linear relationship between the variables. A positive sign with a coefficient value shows a positive (direct, or supportive), while a negative sign with a coefficient value shows a negative (indirect, opposite) relationship between the variables.

The zero-value implies the absence of a linear relation and it also shows that variables are independent. Zero value also shows that there may be some other sort of relationship between the variables of interest such as a systematic or circular relationship between the variables.

Pearson Correlation Coefficient Scatter Diagram

Pearson Correlation Coefficient Formula

Mathematically, if two random variables such as $X$ and $Y$ follow an unknown joint distribution then the simple linear correlation coefficient is equal to covariance between $X$ and $Y$ divided by the product of their standard deviations i.e

$ρ = \frac{C o v (X, Y)}{σ_{X} σ_{Y}}$

where $C o v (X, Y)$ is a measure of covariance between $X$ and $Y$ , $σ_{X}$ and $σ_{Y}$ are the respective standard deviation of the random variables.

Sample Correlation Formula

For a sample of size $n$ , $(X_{1}, Y_{1}), (X_{2}, Y_{2}), \dots, (X_{n}, Y_{n})$ from the joint distribution, the quantity given below is an estimate of $ρ$ , called sampling correlation and denoted by $r$ .

$\begin{array}{rcl} r & = & \frac{\sum_{i = 1}^{n} (X_{i} - \bar{X}) (Y_{i} - \bar{Y})}{\sqrt{\sum_{i = 1}^{n} (X_{i} - \bar{X})^{2} \times \sum_{i = 1}^{n} (Y_{i} - \bar{Y})^{2}}} \\ = & \frac{C o v (X, Y)}{S_{X} X_{Y}} \end{array}$

Important Points to Remember

The existence of a statistical correlation does not mean that there exists a cause-and-effect relation between the variables. Cause and effect mean that a change in one variable does cause a change in the other variable.
The changes in the variables may be due to a common cause or random variations.
There are many kinds of correlation coefficients. The choice of which to use for a particular set of data depends on different factors such as
- Type of Scale (Level of Measurement or Measurement Scale) used to express the variables
- Nature of the underlying distribution (continuous or discrete)
- Characteristics of the distribution of the scores (linear or non-linear)
Correlation is perfectly linear if a constant change in $X$ is accompanied by a constant change in $Y$ . In this case, all the points in the scatter diagram will lie in a straight line.
A high correlation coefficient does not necessarily imply a direct dependence on the variables. For example, there may be a high correlation between the number of crimes and shoe prices. Such a kind of correlation is referred to as a non-sense or spurious correlation.

Properties of Pearson’s Correlation Coefficient

The following are important properties that a Pearson correlation coefficient can have:

The Pearson correlation coefficient is symmetrical for $X$ and $Y$ i.e. $r_{X Y} = r_{Y X}$ .
The Pearson correlation coefficient is a pure number and it does not depend upon the units in which the variables are measured.
The correlation coefficient is the geometric mean of the two regression coefficients. Thus if the two regression lines of $Y$ on $X$ and $X$ on $Y$ are written as $Y = a + b X$ and $X = c + d y$ respectively then $b d = r^{2}$ .
The Pearson correlation coefficient is independent of the choice of origin and scale of measurement of the variables, i.e. $r$ remains unchanged if constants are added to or subtracted from the variables and if the variables having the same size are multiplied or divided by the class interval size.
The correlation coefficient lies between -1 and +1, symbolically $- 1 \leq r \leq 1$ .

Take various Online MCQ quiz

Statistical Linear Models in R Language

Correlation Coefficient Range

Table of Contents

Correlation Coefficient Ranges

Coefficient of Determination

Alternative Proof: Correlation Coefficient Range

Real-Life Example

FAQs about Correlation Coefficient

Multivariate Data Sets: Descriptive Statistics (2010)

Multivariate Data Sets: Descriptive Analysis

Measure of Location

Measure of Spread

Sample Covariance

Sample Correlation Coefficient

Other Multivariate Analysis

Pearson Correlation Coefficient

Introduction to Pearson Correlation Coefficient

Table of Contents

Pearson Correlation Coefficient Formula

Sample Correlation Formula

Important Points to Remember

Properties of Pearson’s Correlation Coefficient

Table of Contents

Correlation Coefficient Ranges

Coefficient of Determination

Alternative Proof: Correlation Coefficient Range

Real-Life Example

FAQs about Correlation Coefficient

Share this:

Multivariate Data Sets: Descriptive Analysis

Measure of Location

Measure of Spread

Sample Covariance

Sample Correlation Coefficient

Other Multivariate Analysis

Share this:

Introduction to Pearson Correlation Coefficient

Table of Contents

Pearson Correlation Coefficient Formula

Sample Correlation Formula

Important Points to Remember

Properties of Pearson’s Correlation Coefficient

Share this: