Basic Statistics and Data Analysis

Statistics Lecture Notes, Online MCQs

Tagged: Miscellaneous Articles

Basic Statistics / Estimate and Estimation / Miscellaneous Articles

 by 

Bias: The Difference Between the Expected Value and True Value

Bias in Statistics is defined as the difference between the expected value of a statistic and the true value of the corresponding parameter. Therefore, the bias is a measure of the systematic error of an estimator. The bias indicates the distance of the estimator from the true value of the parameter. For example, if we calculate the mean of a large number of unbiased estimators, we will find the correct value.

In other words, the bias (sampling error) is a systematic error in measurement or sampling and it tells how far off on the average the model is from the truth.

Gauss, C.F. (1821) during his work on the least-squares method gave the concept of an unbiased estimator.

The bias of an estimator of a parameter should not be confused with its degree of precision as the degree of precision is a measure of the sampling error. The bias is favoring of one group or outcome intentionally or unintentionally over other groups or outcomes available in the population under study. Unlike random errors, bias is a serious problem and bias can be reduced by increasing the sample size and averaging the outcomes.

Bias and Variance

There are several types of bias that should not be considered mutually exclusive

  • Selection Bias (arise due to systematic differences between the groups compared)
  • Exclusion Bias (arise due to the systematic exclusion of certain individuals from the study)
  • Analytical Bias (arise due to the way that the results are evaluated)

Mathematically Bias can be defined as

Let statistics $T$ used to estimate a parameter $\theta$ if $E(T)=\theta+bias(\theta)$ then $bias(\theta)$ is called the bias of the statistic $T$, where $E(T)$ represents the expected value of the statistics $T$.
Note: that if $bias(\theta)=0$, then $E(T)=\theta$. So, $T$ is an unbiased estimator of the true parameter, say $θ$.

Reference:
Gauss, C.F. (1821, 1823, 1826). Theoria Combinations Observationum Erroribus Minimis Obnoxiae, Parts 1, 2 and suppl. Werke 4, 1-108.

For further reading about Statistical Bias visit: Statistical Bias.

Miscellaneous Articles

 by 

Common Log and Natural Log

Difference between Common Log and Natural Log

In this post, we will learn about the difference between Common Log and Natural Log.

The Logarithm of a number is the exponent by which another fixed value the base has to be raised to produce that number. For example, the logarithm of 1000 to base 10 is 3 as 1000=103. Logarithms were introduced by John Napier in the early 17th century for simplification of calculation and were widely adopted by scientists, engineers, and others to perform computations more easily using logarithm tables. The logarithm to base b=10 is called the common logarithm and has a lot of applications in science and engineering, while the natural logarithm has the constant e (2.718281828) as its base and is written as ln(x)or loge(x).

This common log is used in most of the exponential scales (such as 23) in chemistry such as pH scale (for measurement of acidity and alkalinity), Richter scale (for measurement of the intensity of earthquakes), and so on. It is so common that if you find no base written, you can assume it to be log x or common log.

The natural logarithm is widely used in pure mathematics, especially calculus. The natural logarithm of a number x is the power to which $e$ has to be raised to equal x. For example, ln(7.389…) is 2, because e2=7.389. The natural log of e itself (ln(e)) is 1 because $e^1=e$, while the natural logarithm of $1$ (ln(1))$ is 0, since $e^0=1$.

The question is “the reason for choosing 10 is obvious, but why e=2.718…”?

The answer is that it back to 300 years or more ago to Euler (which $e$ comes from his name). The function $e^x$ is the only function that its derivative (and consequently it’s integral) is itself. (ex’ =  ex ), no other function has this characteristic. The number e could be achieved by several numerical and analytical methods, more often infinite summations. This number has a more important rule in complex analysis.

Suppose you have a hundred rupees, and the interest rate is 10%, you will have Rs. 110, and the next time another 10% of Rs. 110, will raise your amount to Rs. 121, and so on…  What happens when the interest is being computed continuously (all the time)?  You might think you would soon have an infinite amount of money, but actually, you have your initial deposit times e to the power of the interest rate times the amount of time:

P=P0 ekt

where k is growth rate or interest rate and t is time period, P is Value at time t and P0 is Value it time t=0.

The intuitive explanation is: ex is the amount of continuous growth after a certain amount of time. The natural log gives you the time needed to reach a certain level of growth. That is, ex is the amount of continuous growth after a certain amount of time and a natural log is the amount of time needed to reach a certain level of continuous growth.

Learn more about Natural Logarithm