Muhammad Imdad Ullah

Currently working as Assistant Professor of Statistics in Ghazi University, Dera Ghazi Khan. Completed my Ph.D. in Statistics from the Department of Statistics, Bahauddin Zakariya University, Multan, Pakistan. l like Applied Statistics, Mathematics, and Statistical Computing. Statistical and Mathematical software used is SAS, STATA, Python, GRETL, EVIEWS, R, SPSS, VBA in MS-Excel. Like to use type-setting LaTeX for composing Articles, thesis, etc.

Normal Probability Distribution

by

The Gaussian or normal probability distribution role is very important in statistics. It was investigated by researchers/ persons interested in gambling or in the distribution of errors made by people observing astronomical events. The normal probability distribution is important in other fields such as social sciences, behavioural statistics, business and management sciences, and engineering and technologies.

Importance of Normal Distribution

Some of the important reasons for the normal probability distribution are:

  • Many variables such as (weight, height, marks, measurement errors, IQ, etc.) are distributed as the symmetrical bell-shaped normal curve approximately.
  • Many inferential procedures (parametric tests: confidence intervals, hypothesis testing, regression analysis, etc.) assume that the variables follow the normal distribution.
  • All probability distributions approach a normal distribution under some conditions.
  • Even if a variable is not normally distributed, a distribution of sample sums or averages on that variable will be approximately normally distributed if the sample size is large enough.
  • The mathematics of a normal curve is well-known and relatively simple. One can find the probability that a score randomly sampled from a normal distribution falls within the interval $a$ and $b$ by integrating the normal probability density function (PDF) from $a$ to $b$. This is equivalent to finding the area under the curve between $a$ and $b$ assuming a total area of one.
  • Due to the Central Limit Theorem, the average of many independent random variables tends to follow a normal probability distribution, regardless of the original distribution of the variables.

Probability Density Functions of Normal Distribution

The probability density function known as the normal curve. $F(X)$ is the probability density, aka the height of the curve at value $X$.

$$F(X) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{(X-\mu)^2}{2\sigma^2} }$$

There are two parameters in the PDF of normal distribution, (i) the mean and (ii) the standard deviation. Everything else in the PDF of normal distribution on the right-hand side is a constant. There is a family of normal probability distribution with respect to their means and their standard deviations.

Standard Normal Probability Distribution

One can work with normal curve, even if one don’t know about integral calculus. One can use computer to compute the area under the normal curve or make use of the normal curve table. The normal curve table (standard normal table) is based on the standard normal curve ($Z$), which has a mean of 0 and a variance of 1. To use a standard normal curve table, one need to convert the raw score to $Z$-scores. A $Z$-score is the number of standard deviations ($\sigma$ or $s$) a score is above or below the mean of a reference distribution.

$$Z_X = \frac{X-\mu}{\sigma}$$

For example, suppose one wish to know the percentile rank of a score of 90 on an IQ test with $\mu = 100$ and $\sigma=10$. The $Z$-score will be

$$Z=\frac{X-\mu}{\sigma} = \frac{90-100}{10} = -1$$

One can either integrate the normal cure from $-\infty$ to $-1$ or use the standard normal table. The probability or area under the curve on the left of $-1$ is 0.1587 or 15.87%.

Standard Normal Probability distribution Curve

Key Characteristics of Normal Probability Distribution

  • Symmetry: In normal probability distribution, the mean, median, and mode are all equal and located at the center of the curve.
  • Spread: In normal distribution, the spread of the data is determined by the standard deviation. A larger standard deviation means that the curve is wider, and a smaller standard deviation means a narrower curve.
  • Area under the Normal Curve: The total area under the normal curve is always equal to 1 or 100%.
normal curve dnorm() normal probability distribuiton

Real-Life Applications of Normal Distribution

The following are some real-life applications of normal probability distribution.

  • Natural Phenomena:
    • Biological Traits: Many biological traits, such as weight, height, and IQ scores, tend to follow a normal distribution. This helps us to understand the typical range of values for different biological traits and identify outliers.
    • Physical Measurements: Errors in measurements often follow a normal distribution. This knowledge is crucial in fields like engineering and physics for quality control and precision.
  • Statistical Inference:
    • Hypothesis Testing: The normal distribution is used extensively in hypothesis testing to determine the statistical significance of the results. By understanding the distribution of sample means, one can make inferences about population parameters.
    • Confidence Intervals: Normal distribution helps calculate confidence intervals, which provide a range of values within which a population parameter is likely to fall with a certain level of confidence.
  • Machine Learning and Artificial Intelligence:
    • Feature Distribution: Many machine learning (ML) algorithms assume that features in data follow a normal distribution. The normality assumption about machine learning algorithms can influence the choice of algorithms and the effectiveness of models.
    • Error Analysis: The normal distribution is used to analyze the distribution of errors in machine learning models, helping to identify potential biases and improve accuracy.
  • Finance and Economics:
    • Asset Returns: While not perfectly normal, many financial assets, such as stock prices, follow an approximately normal distribution over short time periods. The assumption of normality is used in various financial models and risk assessments.
    • Economic Indicators: Economic indicators such as GDP growth rates and inflation rates often exhibit a normal distribution, allowing economists to analyze trends and make predictions.
  • Quality Control:
    • Process Control Charts: In manufacturing and other industries, normal distribution is used to create control charts that monitor the quality of products or processes. By tracking the distribution of measurements, one can identify when a process is going out of control.
  • Product Quality: Manufacturers use statistical quality control methods based on normal distribution to ensure that products meet quality standards.
  • Everyday Life:
    • Standardized Tests: The standardized Test scores, such as SAT and GRE, are often normalized to a standard normal distribution, allowing for comparisons between different test-takers.
    https://itfeature.com, Normal probability distribution

    R Programming Language, Online Quiz Website

    Presentation of Data in Statistics

    by

    Since the primary data is in raw form or haphazard, it is not easy to examine the unorganized data. The scientist or researcher has organized the data in an understandable and meaningful way. In this post, we will learn about the organization/ presentation of data in Statistics. The presentation of data in statistics is a vital aspect, as it transforms raw data into meaningful and understandable information.

    Classification/ Presentation of Data in Statistics

    The classification is a widely used data organization technique which is further classified into three categories

    • Tabulation (Frequency Distribution and Contingency Tables)
    • Graphical Presentation of Data (Bar charts, Pie charts, scatter diagrams. line charts, etc.)
    • Textual Presentation of Data (Descriptive Statistics)

    Classification of Data

    Classification is defined as the process of dividing a set of data into different groups or categories so that they are homogeneous with respect to their characteristics and mutually exclusive. In other words, classification is a method that divides a set of data into different heterogeneous groups or sorts the data into different heterogeneous groups, by sort we mean a systematic arrangement of objects, individuals, and units in such a way that different categories are created.

    The data can be classified/presented/organized in different ways, such as color classification, age classification, gender classification, and grade classification.

    Tabulation

    The classification of data in tabular form with suitable headings of tables, rows, and columns is called tabulation. There are different parts or components of a table: (i) Title, (ii) Column Caption, (iii) Row Caption, (iv) Footnotes, (v) Source note.

    Presentation of Data in Statistics
    • Table Number: A number is allocated to the table for identification, particularly when there are a lot of tables in the study.
    • Title: The title of the table should explain what is contained in the table. The title must be concise, clear, brief, and set in bold type font on the top of the table. It may also indicate the time and place to which the data refer.
    • Stub or Row Designations: Each row of the table should be given a brief heading called stubs or stub items. For columns, it is called the stub column.
    • Column Headings or Captions: column designation is given on top of each column to explain to what the figures in the column refer. It should be concise, clear, and precise. This is called caption, or heading. Columns can also be numbered if there are four or more columns in a table.
    • Body of the Table: The data should be organized/ arranged in such a way that any data point/ figure can be located easily. Various types of numerical variables should be arranged in ascending order from left to right in rows and from top to bottom in columns. The columns and rows totals can also be given.
    • Source: At the bottom of the table, a note should be added indicating the primary and secondary sources from which data have been collected
    • Footnotes and references: If any item has not been explained properly, a separate explanatory note should be added at the bottom of the table.

    Importance of Tabulation

    In Tabulation, data are arranged and it makes data brief.

    • In tabulation, data is divided into various parts and for each part, there are totals and sub totals. Therefore, relationships between different parts can easily be established.
    • Since data is organized in a table with a title and a number, data can be easily identified and used for the required purpose.
    • Tables can be easily presented in the form of graphs.
    • Tabulations make complex data simple making it easy to understand the data.
    • Tabulation also helps in identifying mistakes and errors.
    • Tabulation condenses the collected data and it becomes easy to analyze the data from tables.
    • Tabulation saves time and costs as it is the easiest and most comprehensive method used to organize the data.
    • Since tabulation summaries, the large scattered data, the maximum information may be gained/collected from these tables.

    Limitations of Tabulation

    • Tables contain only numerical data. The tables do not contain further details.
    • Qualitative expressions are not possible through tables.
    • Usually, tables are used by experts to conclude, but common men cannot understand them properly.

    Examples of Tabulation

    Consider, that a district is divided into two areas urban area and rural area, The Total population of the district is 271076 out of which only 46740 live in the urban area. The total male population of the district is 139699 and that of the urban area is 23083. The total unmarried population of the district is 112352 out of which 36864 are rural females. In the urban area unmarried people number 21072 out of which 12149 are males. Construct a table showing the population of the district by marital status, residence, and Gender.

    Tabulation Presentation of Data in Statistics
    Tabulation example presentation of data in statistics

    Graphical Presentation of Data In Statistics

    Visualization or Graphical presentation of data in statistics helps researchers visualize hidden information in a graphical/visual way. There are many types of graphical representations of the data:

    • Bar Charts: Bar charts are used to represent the frequency, percentage, or magnitude of different categories or groups in rectangular form. Simple bar charts are used to compare different categories while multiple bar charts are used to compare multiple categories over time or across groups. The stacked bar charts are used to show the composition of each category.
    • Pie Charts: Pie charts are used to represent the proportions of a whole as slices/sectors of a pie.
    • Line Graphs: Line graphs are used to show trends over time or relationships between variables.
    • Scatter plots: Scatter plots are used to visualize the relationship between two quantitative variables.
    • Histogram: Histograms are similar to bar charts where the bars are adjacent, representing the frequency distribution of a continuous variable.

    Textual Presentation of Data in Statistics

    Textual presentation of data includes descriptive statistics. Descriptive statistics summarizes the data using numerical measures like mean, median, mode, range, and standard deviation.

    Selection of the Right Method for the Presentation of Data

    For the presentation of data in statistics, one should be careful in selecting the right method of data representation. The selection or choice of the right method depends on:

    • Type of data: The visualization or textual presentation of data depends on the type of the data. For example, categorical data (such as gender, color, etc.) is often presented using bar charts or pie charts, while numerical data (such as age, marks, income, etc.) is better suited for histograms, line graphs, or scatter plots.
    • Purpose: To show the trends of data over time, one can use a line graph. A pie chart is suitable for comparing proportions. Therefore, the selection of presentation of data depends on the purpose, use, or application of data in real life.
    • Audience: The selection of different presentations of data depends on the familiarity of the audience with different types of graphs and charts. Simpler visualizations might be more effective for a general audience.

    FAQS about Presentation of Data in Statistics

    1. What is meant by the presentation of data?
    2. What is the difference between tabulation, graphical presentation, and textual presentation of the data?
    3. What are the different parts of a table? explain in detail.
    4. Discuss different graphical representations.
    5. Discuss the selection of the right method depending on the type of data.
    6. What is the importance of tabulation in statistics?

    https://rfaqs.com, https://gmstat.com

    Eigenvalue Multicollinearity Detection

    by

    In this post, we learn about the role of eigenvalue multicollinearity detection. In the context of the detection of multicollinearity, eigenvalues are used to assess the degree of linear dependence among explanatory (regressors, independent) variables in a regression model. Therefore, by understanding the role of eigenvalue multicollinearity detection, one can take appropriate steps to improve the reliability and interpretability of the regression models.

    Decomposition of Eigenvalues and Eigenvectors

    The pair-wise correlation matrix of explanatory variables is decomposed into eigenvalues and eigenvectors. Whereas Eigenvalues represent the variance explained by each Principal Component and Eigenvectors represent the directions of maximum variance.

    The Decomposition Process

    Firstly, compute the correlation coefficients between each pair of variables in the dataset.

    Secondly, find the Eigenvalues and Eigenvectors: solve the following equation for each eigenvalue ($\lambda$) and eigenvector ($vV)

    $$A v = \lambda v$$

    where $A$ is the correlation matrix, $v$ is the eigenvector, and $\lambda$ is the eigenvalue.

    The above equation essentially means that multiplying the correlation matrix ($A$) by the eigenvector ($v$) results in a scaled version of the eigenvector, where the scaling factor is the eigenvalue. This can be solved using various numerical methods, such as the power method or QR algorithm.

    Interpreting Eigenvalue Multicollinearity Detection

    A set of eigenvalues of relatively equal magnitudes indicates little multicollinearity (Freund and Littell 2000: 99). A small number of large eigenvalues suggests that a small number of component variables describe most of the variability of the original observed variables ($X$). Because of the score constraint, a number of large eigenvalues implies that there will be some small eigenvalues or some small variances of component variables.

    A zero eigenvalue means perfect multicollinearity among independent/explanatory variables and very small eigenvalues imply severe multicollinearity. Conventionally, an eigenvalue close to zero (less than 0.01) or condition number greater than 50 (30 for conservative persons) indicates significant multicollinearity. The condition index, calculated as the ratio of the largest eigenvalue to the smallest eigenvalue $\left(\frac{\lambda_{max}}{\lambda_{min}}\right)$, is a more sensitive measure of multicollinearity. A high condition index (often above 30) signals severe multicollinearity.

    Eigenvalue Multicollinearity Detection

    The proportion of variances tells how much percentage of the variance of parameter estimate (coefficient) is associated with each eigenvalue. A high proportion of variance of an independent variable coefficient reveals a strong association with the eigenvalue. If an eigenvalue is small enough and some independent variables show a high proportion of variation with respect to the eigenvalues then one may conclude that these independent variables have significant linear dependency (correlation).

    Presence of Multicollinearity in Regression Model

    Since Multicollinearity is a statistical phenomenon where two or more independent/explanatory variables in a regression model are highly correlated, the existence/presence of multicollinearity may result in

    • Unstable Coefficient Estimates: Estimates of regression coefficients become unstable in the presence of multicollinearity. A small change in the data can lead to large changes in the estimates of the regression coefficients.
    • Inflated Standard Errors: The standard errors of the regression coefficients inflated due to the presence of multicollinearity, making it difficult to assess the statistical significance of the coefficients.
    • Difficulty in Interpreting Coefficients: It becomes challenging to interpret the individual effects of the independent variables on the dependent variable when they are highly correlated.

    How to Mitigate the Effects of Multicollinearity

    If multicollinearity is detected, several strategies can be employed to mitigate the effects of multicollinearity. By examining the distribution of eigenvalues, researchers (statisticians and data analysts) can identify potential issues and take appropriate steps to address them, such as feature selection or regularization techniques.

    • Feature Selection: Remove redundant or highly correlated variables from the model.
    • Principal Component Regression (PCR): Transform the original variables into a smaller set of uncorrelated principal components.
    • Partial Least Squares Regression (PLSR): It is similar to PCR but also considers the relationship between the independent variables and the dependent variable.
    • Ridge Regression: Introduces a bias-variance trade-off to stabilize the coefficient estimates.
    • Lasso Regression: Shrinks some coefficients to zero, effectively performing feature selection.
    https://itfeature.com eigenvalue for multicollinearity detection

    https://rfaqs.com, https://gmstat.com

    Correlation Regression MCQs 6

    by

    The post is about a Quiz on Correlation Regression MCQs with Answers. There are 20 multiple-choice questions covering topics related to correlation and regression analysis, coefficient of determination, testing of correlation and regression coefficient, Interpretation of regression coefficients, and the method of least squares, etc. Let us start with Correlation Regression MCQs with answers.

    Online Multiple-Choice Questions about Correlation and Regression Analysis with Answers

    1. If the coefficient of determination is 0.49, the correlation coefficient may be

     
     
     
     

    2. The strength of the linear relationship between two numerical variables may be measured by the

     
     
     
     

    3. The sample correlation coefficient between $X$ and $Y$ is 0.375. It has been found that the p-value is 0.256 when testing $H_0:\rho = 0$ against the two-sided alternative $H_1:\rho\ne 0$. To test $H_0:\rho =0$ against the one-sided alternative $H_1:\rho >0$ at a significance level of 0.193, the p-value is

     
     
     
     

    4. If the correlation coefficient ($r=1.00$) then

     
     
     
     

    5. What do we mean when a simple linear regression model is “statistically” useful?

     
     
     
     

    6. The $Y$ intercept ($b_0$) represents the

     
     
     
     

    7. The slope ($b_1$) represents

     
     
     
     

    8. The correlation coefficient

     
     
     
     

    9. The estimated regression line relating the market value of a person’s stock portfolio to his annual income is $Y=5000+0.10X$. This means that each additional rupee of income will increase the stock portfolio by

     
     
     
     

    10. If the correlation coefficient $r=1.00$ then

     
     
     
     

    11. The sample correlation coefficient between $X$ and $Y$ is 0.375. It has been found that the p-value is 0.256 when testing $H_0:\rho = 0$ against the two-sided alternative $H_1:\rho\ne 0$. To test $H_0:\rho=0$ against the one-sided alternative $H_1:\rho<0$ at a significance level of 0.193, the p-value is

     
     
     
     

    12. Testing for the existence of correlation is equivalent to

     
     
     
     

    13. Which one of the following situations is inconsistent?

     
     
     
     

    14. Which of the following does the least squares method minimize?

     
     
     
     

    15. The sample correlation coefficient between $X$ and $Y$ is 0.375. It has been found that the p-value is 0.256 when testing $H_0:\rho=0$ against the one-sided alternative $H_1:\rho>0$. To test $H_0:\rho =04 against the two-sided alternative $H_1:\rho\ne 0$ at a significance level of 0.193, the p-value is

     
     
     
     

    16. Which one of the following statements is true?

     
     
     
     

    17. Assuming a linear relationship between $X$ and $Y$ if the coefficient of correlation equals $-0.30$

     
     
     
     

    18. The true correlation coefficient $\rho$ will be zero only if

     
     
     
     

    19. If you wanted to find out if alcohol consumption (measured in fluid oz.) and grade point average on a 4-point scale are linearly related, you would perform a

     
     
     
     

    20. In a simple linear regression problem, $r$ and $\beta_1$

     
     
     
     

    Online Correlation & Regression MCQs with Answers

    Online Correlation Regression MCQs

    • The $Y$ intercept ($b_0$) represents the
    • The slope ($b_1$) represents
    • Which of the following does the least squares method minimize?
    • What do we mean when a simple linear regression model is “statistically” useful?
    • If the correlation coefficient $r=1.00$ then
    • If the correlation coefficient ($r=1.00$) then
    • Assuming a linear relationship between $X$ and $Y$ if the coefficient of correlation equals $-0.30$
    • Testing for the existence of correlation is equivalent to
    • The strength of the linear relationship between two numerical variables may be measured by the
    • In a simple linear regression problem, $r$ and $\beta_1$
    • The sample correlation coefficient between $X$ and $Y$ is 0.375. It has been found that the p-value is 0.256 when testing $H_0:\rho = 0$ against the two-sided alternative $H_1:\rho\ne 0$. To test $H_0:\rho=0$ against the one-sided alternative $H_1:\rho<0$ at a significance level of 0.193, the p-value is The sample correlation coefficient between $X$ and $Y$ is 0.375. It has been found that the p-value is 0.256 when testing $H_0:\rho = 0$ against the two-sided alternative $H_1:\rho\ne 0$. To test $H_0:\rho =0$ against the one-sided alternative $H_1:\rho >0$ at a significance level of 0.193, the p-value is
    • The sample correlation coefficient between $X$ and $Y$ is 0.375. It has been found that the p-value is 0.256 when testing $H_0:\rho=0$ against the one-sided alternative $H_1:\rho>0$. To test $H_0:\rho =04 against the two-sided alternative $H_1:\rho\ne 0$ at a significance level of 0.193, the p-value is
    • If you wanted to find out if alcohol consumption (measured in fluid oz.) and grade point average on a 4-point scale are linearly related, you would perform a
    • The correlation coefficient
    • If the coefficient of determination is 0.49, the correlation coefficient may be
    • The estimated regression line relating the market value of a person’s stock portfolio to his annual income is $Y=5000+0.10X$. This means that each additional rupee of income will increase the stock portfolio by
    • Which one of the following situations is inconsistent?
    • Which one of the following statements is true?
    • The true correlation coefficient $\rho$ will be zero only if
    Statistics Help https://itfeature.com MCQs Correlation and Regression

    https://rfaqs.com, https://gmstat.com