Statistics for Data Science & Analytics - Learn Statistics: MCQs, Software & Data Analysi

Unbiasedness of the Estimator (2013)

Apr 2, 2025Jul 27, 2013 by Muhammad Imdad Ullah

Post Views: 309

The unbiasedness of the estimator is probably the most important property that a good estimator should possess. In statistics, the bias (or bias function) of an estimator is the difference between this estimator’s expected value and the true value of the parameter being estimated. An estimator is said to be unbiased if its expected value equals the corresponding population parameter; otherwise, it is said to be biased. Let us discuss in detail the unbiasedness of the estimator.

Introduction to Unbasedness of the Estimator

In the world of statistics and data analysis, estimators play a crucial role in drawing conclusions from data. One of the most important properties of an estimator is unbiasedness. Understanding this concept helps statisticians and data scientists ensure that their estimates are as accurate and representative as possible. Let us explore the definition of unbiasedness, why it matters, and how it applies to real-world data analysis.

Unbiasedness of the Estimator

Suppose in the realization of a random variable $X$ taking values in probability space i.e. ( $χ, F, P_{θ}$ ), such that $θ ε Θ$ , a function $f : Θ \to Ω$ has to be estimated, mapping the parameter set $Θ$ into a certain set $Ω$ , and that as an estimator of $f (θ)$ a statistic $T = T (X)$ is chosen. if $T$ is such that
$E_{θ} [T] = \int_{χ} T (x) d P_{θ} (x) = f (θ)$
holds for $θ ε Θ$ then $T$ is called an unbiased estimator of $f (θ)$ . The unbiased estimator is frequently called free of systematic errors.

Unbiased Estimator

Suppose $\hat{θ}$ be an estimator of a parameter $θ$ , then $\hat{θ}$ is said to be unbiased estimator if $E (\hat{θ}) = 0$ .

If $E (\hat{θ}) = θ$ then $\hat{θ}$ is an unbiased estimator of a parameter $θ$ .
If $E (\hat{θ}) < θ$ then $\hat{θ}$ is a negatively biased estimator of a parameter $θ$ .
If $E (\hat{θ}) > θ$ then $\hat{θ}$ is a positively biased estimator of a parameter $θ$ .

Bias of an estimator $θ$ can be found by $[E (\hat{θ}) - θ]$

$\overset{―}{X}$ is an unbiased estimator of the mean of a population (whose mean exists).
$\overset{―}{X}$ is an unbiased estimator of $μ$ in a Normal distribution i.e. $N (μ, σ^{2})$ .
$\overset{―}{X}$ is an unbiased estimator of the parameter $p$ of the Bernoulli distribution.
$\overset{―}{X}$ is an unbiased estimator of the parameter $λ$ of the Poisson distribution.

In each of these cases, the parameter $μ, p$ or $λ$ is the mean of the respective population being sampled.

However, sample variance $S^{2}$ is not an unbiased estimator of population variance $σ^{2}$ , but consistent.

It is possible to have more than one unbiased estimator for an unknown parameter. The sample mean and the sample median are unbiased estimators of the population mean $μ$ if the population distribution is symmetrical.

Why is Unbiasedness Important?

Accuracy of Estimates: Unbiased estimators provide estimates that are correct on average, which reduces the chances of consistently overestimating or underestimating the true parameter.
Reliability in Statistical Inference: When conducting hypothesis tests or constructing confidence intervals, unbiased estimators ensure that statistical conclusions are valid and trustworthy.
Foundation for Further Statistical Properties: Many other desirable properties, such as consistency and efficiency, build upon the unbiasedness of an estimator.

Limitations of Unbiased Estimators

While unbiasedness is a desirable property, it is not the only criterion for a good estimator. Some unbiased estimators may have a high variance, making them unreliable for small samples. In such cases, biased but low-variance estimators (e.g., regularized estimators) might be preferred.

Summary

Unbiased estimators are a fundamental concept in statistics, ensuring that estimates are accurate on average. However, they must be used carefully, considering other factors like variance and sample size. Understanding unbiasedness helps in making informed statistical decisions and improving data-driven analysis.

Computer MCQs

R Programming Language

Absolute Measure of Dispersion

Apr 6, 2024May 25, 2013 by Muhammad Imdad Ullah

Post Views: 668

An absolute Measure of Dispersion gives an idea about the amount of dispersion/ spread in a set of observations. These quantities measure the dispersion in the same units as the units of original data. The absolute measure of dispersion cannot be used to compare the variation of two or more series/ data sets. The absolute measure of dispersion does not in itself, tell whether the variation is large or small.

Absolute Measure of Dispersion

The absolute Measure of Dispersion:

Range
Quartile Deviation
Mean Deviation
Variance or Standard Deviation

Range

The Range is the difference between the largest value and the smallest value in the data set. For ungrouped data, let $X_{0}$ be the smallest value and $X_{n}$ be the largest value in a data set then the range ( $R$ ) is defined as
$R = X_{n} - X_{0}$ .

For grouped data Range can be calculated in three different ways
R=Mid point of the highest class – Midpoint of the lowest class
R=Upper class limit of the highest class – Lower class limit of the lower class
R=Upper class boundary of the highest class – The lower class boundary of the lowest class

Quartile Deviation (Semi-Interquantile Range)

The Quartile deviation (an absolute measure of dispersion) is defined as the difference between the third and first quartiles, and half of this range is called the semi-interquartile range (SIQD) or simply quartile deviation (QD). $Q D = \frac{Q_{3} - Q_{1}}{2}$

The Quartile Deviation is superior to the range as it is not affected by extremely large or small observations, anyhow it does not give any information about the position of observation lying outside the two quantities. It is not amenable to mathematical treatment and is greatly affected by sampling variability. Although Quartile Deviation is not widely used as a measure of dispersion, it is used in situations in which extreme observations are thought to be unrepresentative/ misleading. Quartile Deviation is not based on all observations therefore it is affected by extreme observations.

Note: The range “Median ± QD” contains approximately 50% of the data.

Mean Deviation (Average Deviation)

The Mean Deviation is another absolute measure of dispersion and is defined as the arithmetic mean of the deviations measured either from the mean or from the median. All these deviations are counted as positive to avoid the difficulty arising from the property that the sum of deviations of observations from their mean is zero.

$M D = \frac{\sum | X - \overset{―}{X} |}{n}$ for ungrouped data for mean
$M D = \frac{\sum f | X - \overset{―}{X} |}{\sum f}$ for grouped data for mean
$M D = \frac{\sum | X - \tilde{X} |}{n}$ for ungrouped data for median
$M D = \frac{\sum f | X - \tilde{X} |}{\sum f}$ for grouped data for median
Mean Deviation can be calculated about other central tendencies but it is least when deviations are taken as the median.

The Mean Deviation gives more information than the range or the Quartile Deviation as it is based on all the observed values. The Mean Deviation does not give undue weight to occasional large deviations, so it should likely be used in situations where such deviations are likely to occur.

Variance and Standard Deviation

This absolute measure of dispersion is defined as the mean of the squares of deviations of all the observations from their mean. Traditionally population variance is denoted by $σ^{2}$ (sigma square) and for sample data denoted by $S^{2}$ or $s^{2}$ .

Symbolically
$σ^{2} = \frac{\sum (X_{i} - μ)^{2}}{N}$ Population Variance for ungrouped data
$S^{2} = \frac{\sum (X_{i} - \overset{―}{X})^{2}}{n}$ sample Variance for ungrouped data
$σ^{2} = \frac{\sum f (X_{i} - μ)^{2}}{\sum f}$ Population Variance for grouped data
$σ^{2} = \frac{\sum f (X_{i} - \overset{―}{X})^{2}}{\sum f}$ Sample Variance for grouped data

The variance is denoted by $V a r (X)$ for random variable $X$ . The term variance was introduced by R. A. Fisher (1890-1982) in 1918. The variance is in squares of units and the variance is a large number compared to observations themselves.
Note that there are alternative formulas to compute Variance or Standard Deviations.

The positive square root of the variance is called Standard Deviation (SD) to express the deviation in the same units as the original observation. It is a measure of the average spread about the mean and is symbolically defined as

$σ^{2} = \sqrt{\frac{\sum (X_{i} - μ)^{2}}{N}}$ Population Standard for ungrouped data
$S^{2} = \sqrt{\frac{\sum (X_{i} - \overset{―}{X})^{2}}{n}}$ Sample Standard Deviation for ungrouped data
$σ^{2} = \sqrt{\frac{\sum f (X_{i} - μ)^{2}}{\sum f}}$ Population Standard Deviation for grouped data
$σ^{2} = \sqrt{\frac{\sum f (X_{i} - \overset{―}{X})^{2}}{\sum f}}$ Sample Standard Deviation for grouped data
Standard Deviation is the most useful measure of dispersion and is credited with the name Standard Deviation by Karl Pearson (1857-1936).

In some text Sample, Standard Deviation is defined as $S^{2} = \frac{\sum (X_{i} - \overset{―}{X})^{2}}{n - 1}$ based on the argument that knowledge of any $n - 1$ deviations determines the remaining deviations as the sum of n deviations must be zero. This is an unbiased estimator of the population variance $σ^{2}$ . The Standard Deviation has a definite mathematical measure, it utilizes all the observed values and is amenable to mathematical treatment but affected by extreme values.

References

R Language Tutorial

MCQs about Business Mathematics

Random Walk Probability of Returning to Origin after n steps

Jun 23, 2024May 4, 2013 by Muhammad Imdad Ullah

Post Views: 406

Random Walk Probability of Returning to Origin

Assume that the walk starts at $x = 0$ with steps to the right or left occurring with probabilities $p$ and $q = 1 - p$ . We can write the position $X_{n}$ after $n$ steps as
$\begin{matrix} (1) & X_{n} = R_{n} - L_{n} \end{matrix}$
where $R_{n}$ is the number of right or positive steps (+1) and $L_{n}$ is the number of left or negative steps ( $- 1$ ).

Therefore the Total steps can be calculated as: $\begin{matrix} (2) & n = R_{n} + L_{n} \end{matrix}$
Hence
$\begin{aligned} L_{n} & = n - R_{n} \\ \Rightarrow X_{n} & = R_{n} - n + R_{n} \\ (3) & R_{n} & = \frac{1}{2} (n + X_{n}) \end{aligned}$
The equation (3) will be an integer only when $n$ and $X_{n}$ are both even or both odd (eg. To go from $x = 0$ to $x = 7$ we must take an odd number of steps).

Now, let $v_{n, x}$ be the probability that the walk is at state $x$ after $n$ steps assuming that $x$ is a positive integer. Then
$\begin{aligned} v_{n, x} & = P (X_{n} = x) \\ = P (R_{n} = \frac{1}{2} (n + x)) \end{aligned}$
$R_{n}$ is a binomial random variable with index $n$ having probability $p$ , since the walker either moves to the right or not at every step, and the steps are independent, then
$\begin{aligned} v_{n, x} & = (\binom{n}{\frac{1}{2} (n + x)}) p^{\frac{1}{2} (n + x)} q^{n - \frac{1}{2} (n + x)} \\ (4) & = (\binom{n}{\frac{1}{2} (n + x)}) p^{\frac{1}{2} (n + x)} q^{\frac{1}{2} (n - x)} \end{aligned}$
where $(n, x)$ are both even or both odd and $- n \leq x \leq n$ . Note that a similar argument can be constructed if $x$ is a negative integer.

Example

For a total number of steps is 2, the net displacement must be one of the three possibilities: (1) two steps to the left, (2) back to the start, (3) or two steps to the right. These correspond to values of $x = - 2, 0, + 2$ . It is impossible to get more than two units away from the origin if you take only two steps and it is equally impossible to end up exactly one unit from the origin if you take two steps.

For a symmetric case ( $p = \frac{1}{2}$ ), starting from the origin, there are $2^{n}$ different paths of length $n$ since there is a choice of right or left move at each step. Since the number of steps in the right direction must be $\frac{1}{2} (n + x)$ and the total number of paths must be the number of ways in which $\frac{1}{2} (n + x)$ can be chosen from $n$ : that is
$N_{n, x} = (\binom{n}{\frac{1}{2} (n + x)})$
provided that $\frac{1}{2} (n + x)$ is an integer.

By counting rule, the probability that the walk ends at $x$ after $n$ steps is given by the ratio of this number and the total number of paths (since all paths are equally likely). Hence
$v_{n, x} = \frac{N_{n, x}}{2^{n}} = (\binom{n}{\frac{1}{2} (n + x)}) \frac{1}{2^{n}}$
The probability $v_{n, x}$ is the probability that the walk ends at state $x$ after $n$ steps: the walk could have overshot x before returning there.

A related probability is the probability that the first visit to position x occurs at the $n$ th step. The following is a descriptive derivation of the associated probability-generating function of the symmetric random walk in which the walk starts at the origin, and we consider the probability that it returns to the origin.

From equation (4), the probability that a walk is at the origin at step $n$ is
$\begin{aligned} v_{n, x} & = (\binom{n}{\frac{1}{2} (n + x)}) p^{\frac{1}{2} (n + x)} q^{n - \frac{1}{2} (n + x)} \\ = (\binom{n}{\frac{1}{2} (n + 0)}) {(\frac{1}{2})}^{\frac{1}{2} n} {(\frac{1}{2})}^{\frac{1}{2} n} \\ (5) & = (\binom{n}{\frac{1}{2} n}) \frac{1}{2^{n}} = p_{n} (say), (n = 2, 4, 6, \dots) \end{aligned}$
Here $p_{n}$ is the probability that after $n$ steps the position of the walker is at the origin. We also assume that $p_{n} = 0$ if $n$ is odd. From equation (5) we can construct a generating function.
$\begin{aligned} H (s) & = \sum_{n = 0}^{\infty} p_{n} s^{n} \\ (6) & = \sum_{n = 0}^{\infty} p_{2 n} s^{2 n} = \sum_{n = 0}^{\infty} \frac{1}{2^{2 n}} (\binom{2 n}{n}) s^{2 n} \end{aligned}$
Note that $p_{0} = 1$ , and H(s) is not a probability generating function since $H (1) \neq 1$ .

The binomial coefficient can be re-arranged as follows:
$\begin{aligned} (\binom{2 n}{n}) & = \frac{(2 n)!}{n! n!} = \frac{2 n (2 n - 1) (2 n - 2) \dots 3.2 .1}{n! n!} \\ = \frac{2^{n} n! (2 n - 1) (2 n - 3) \dots 3.2 .1}{n! n!} \\ = \frac{2^{2 n}}{n!} \frac{1}{2} \frac{3}{2} \dots (n - \frac{1}{2}) \\ (7) & = (- 1)^{n} (\binom{- \frac{1}{2}}{n}) 2^{2 n} \end{aligned}$
Using equation (6) in (7)
$\begin{matrix} (8) & H (s) = \sum_{n = 0}^{\infty} \frac{1}{2^{2 n}} (- 1)^{n} (\binom{- \frac{1}{2}}{n}) s^{2 n} 2^{2 n} = (1 - s^{2})^{- \frac{1}{2}} \end{matrix}$
by binomial theorem, provided $| s | < 1$ . Note that this expansion guarantees that $p_{n} = 0$ if $n$ is odd.

Note that the equation (8) does not sum to one. This is called defective distribution which still gives the probability that the walk is at the origin at step n.

We can estimate the behavior of $p_{n}$ for large n by using Stirling’s Formula (asymptotic estimate for $n!$ for large $n$ ), $n! \approx \sqrt{2 π} n^{n + \frac{1}{2}} e^{- n}$

From equation (5)
$\begin{aligned} p_{2 n} & = \frac{1}{2^{2 n}} (\binom{2 n}{n}) = \frac{1}{2^{2 n}} \frac{(2 n)!}{n! n!} \\ \approx \frac{1}{2^{2 n}} \frac{\sqrt{2 π} (2 n)^{2 n + \frac{1}{2}} e^{- 2 n}}{[\sqrt{2 π} (n^{n + \frac{1}{2}} e^{- n})]^{2}} \\ = \frac{1}{\sqrt{π n}}; for large n \end{aligned}$
Hence $n p_{n} \to \infty$ confirming that the series $\sum_{n = 0}^{\infty} p_{n}$ must diverge.

Random Walk Probability of Returning to Origin after $n$ Steps Some EXAMPLES

Example: Consider a random walk starting from $x_{0} = 0$ and find the probability that after 5 steps the position is 3. i.e. $X_{5} = 3$ , $p = 0.6$ .

Solution: Here the number of steps is $n = 5$ and the position is $x = 3$ . Therefore positive and negative steps are

$R_{n} = \frac{1}{2} (n + x) = \frac{1}{2} (5 + 3) = 4$ and $X_{n} = R_{n} - L_{n} \Rightarrow 3 = 4 + L_{n} = 1$

The probability that the event $X_{5} = 3$ will occur in a random walk with $p = 0.6$ is
$P (X_{5} = 3) = (\binom{5}{\frac{1}{2} (5 + 3)}) (0.6)^{\frac{1}{2} (3 + 5)} (0.4)^{\frac{1}{2} (5 - 3)} = 0.2592$

Random Walk Probability of Returning to Origin after n steps

Click the links to learn Stochastic Processes Introduction, Random Walk Models, Simple Random Walk

Matrices in R Language

Unbiasedness of the Estimator (2013)

Table of Contents

Introduction to Unbasedness of the Estimator

Unbiasedness of the Estimator

Unbiased Estimator

Why is Unbiasedness Important?

Limitations of Unbiased Estimators

Summary

Absolute Measure of Dispersion

Absolute Measure of Dispersion

Range

Quartile Deviation (Semi-Interquantile Range)

Mean Deviation (Average Deviation)

Variance and Standard Deviation

Random Walk Probability of Returning to Origin after n steps

Table of Contents

Random Walk Probability of Returning to Origin

Example

Random Walk Probability of Returning to Origin after $n$ Steps Some EXAMPLES

Table of Contents

Introduction to Unbasedness of the Estimator

Unbiasedness of the Estimator

Unbiased Estimator

Why is Unbiasedness Important?

Limitations of Unbiased Estimators

Summary

Share this:

Absolute Measure of Dispersion

Range

Quartile Deviation (Semi-Interquantile Range)

Mean Deviation (Average Deviation)

Variance and Standard Deviation

Share this:

Table of Contents

Random Walk Probability of Returning to Origin

Example

Related Probability/ First Passage through x

Random Walk Probability of Returning to Origin after n Steps Some EXAMPLES

Share this:

Random Walk Probability of Returning to Origin after $n$ Steps Some EXAMPLES