Point Estimation of Parameters

by

Introduction to Point Estimation of Parameters

The objective of point estimation of parameters is to obtain a single number from the sample which will represent the unknown value of the parameter.

Practically we did not know about the population mean and standard deviation i.e. population parameters such as mean, standard deviation, etc. However, our goal is to measure (estimate) the mean and standard deviation of the population we are interested in from sample information to save time, cost, etc.  This can be done by estimating the sample mean and standard deviation as the best guess for the true population mean and standard deviation.  We can call this estimate a “best guess” and termed a “point estimate” as it is a single number summarized one.

Point Estimate

A Point Estimate is a statistic (a statistical measure from the sample) that gives a plausible estimate (or possibly a best guess) for the value in question.

$\overline{x}$ is a point estimate for $\mu$ and s is a point estimate for $\sigma$.

Or we can say that

A statistic used to estimate a parameter is called a point estimator or simply an estimator. The actual numerical value which we obtain for an estimator in a given problem is called an estimate.

Generally symbol $\theta$ (unknown constant) is used to denote a population parameter which may be a proportion, mean, or some measure of variability. The available information is in the form of a random sample $X_1, X_2, \cdots, X_n$ of size n drawn from the population. We wish to formulate a function of the sample observations $X_1, X_2, \cdots, X_n$; that is, we look for a statistic such that its value computed from the sample data would reflect the value of the population parameter as closely as possible. The estimator of $\theta$ is commonly denoted by $\hat{\theta}$. Different random samples usually provide different values of the statistic $\hat{\theta}$ having its sampling distribution.

Note that Unbiasedness, Efficiency, Consistency, and Sufficiency are the criteria (statistical properties of the estimator) to identify whether a statistic is a “good” estimator.

Application of Point Estimator Confidence Intervals

We can build intervals with confidence as we are not only interested in finding the point estimate for the mean but also in determining how accurate the point estimate is. Here the Central Limit Theorem plays a very important role in building confidence interval.  We assume that the sample standard deviation is close to the population standard deviation (which will almost always be true for large samples). The standard deviation of the sampling distribution of the estimator (here for mean) is

\[\sigma_x \approx \frac{\sigma}{\sqrt{n}}\]

Our interest is to find an interval around $\overline{x}$ such that there is a large probability that the actual (true) mean falls inside the computed interval.  This interval is called a confidence interval and the large probability is called the confidence level.

Example of Point Estimation of Parameters

Question: Suppose that we check for clarity in 50 locations in Lake and discover that the average depth of clarity of the lake is 14 feet with a standard deviation of 2 feet.  What can we conclude about the average clarity of the lake with a 95% confidence level?

Solution: Variable $x$ (depth of lack at 50 locations) can be used to provide a point estimate for $\mu$ and s to provide a point estimate for $s$. To answer how accurate is $x$ as a point estimate, we can construct a 95% confidence interval for $\mu$ as follows.

normal curve: Point Estimation of Parameters

Draw the picture given below and use the standard normal table to find the z-score associated with the probability of .025 (there is .025 to the left and .025 to the right i.e. two-tailed case).

The Z-score for a 95% confidence level is about $\pm 1.96$.

\begin{align*}
Z&=\frac{\overline{x}-\mu}{\frac{\sigma}{\sqrt{n}}}\\
\pm 1.96&=\frac{\overline{x}-\mu}{\frac{2}{\sqrt{n}}}\\
\overline{x}-14&=\pm 0.5488
\end{align*}

Note that $Z\frac{\sigma}{\sqrt{n}}$ is called the margin of error.

The 95% confidence interval for the mean clarity will be (13.45, 14.55)

In other words, there is a 95% chance that the mean clarity is between 13.45 and 14.55.

In general, if $z$ is the standard normal table value associated with a given level of confidence then a $\alpha$% confidence interval for the mean is

\[\overline{x} \pm Z_{\alpha}\frac{\sigma}{\sqrt{n}}\]

See more at Wikipedia about Point Estimation of Parameters

R Frequently Asked Questions

Unbiasedness of the Estimator (2013)

by

The unbiasedness of the estimator is probably the most important property that a good estimator should possess. In statistics, the bias (or bias function) of an estimator is the difference between this estimator’s expected value and the true value of the parameter being estimated. An estimator is said to be unbiased if its expected value equals the corresponding population parameter; otherwise, it is said to be biased. Let us discuss in detail the unbiasedness of the estimator.

Unbiasedness of the Estimator

Suppose in the realization of a random variable $X$ taking values in probability space i.e. ($\chi, \mathfrak{F}, P_\theta$), such that $\theta \varepsilon \Theta$, a function $f:\Theta \rightarrow \Omega $ has to be estimated, mapping the parameter set $\Theta$ into a certain set $\Omega$, and that as an estimator of $f(\theta)$ a statistic $T=T(X)$ is chosen. if $T$ is such that
\[E_\theta[T]=\int_\chi T(x) dP_\theta(x)=f(\theta)\]
holds for $\theta\varepsilon \Theta$ then $T$ is called an unbiased estimator of $f(\theta)$. The unbiased estimator is frequently called free of systematic errors.

Unbiased Estimator

Suppose $\hat{\theta}$ be an estimator of a parameter $\theta$, then $\hat{\theta}$ is said to be unbiased estimator if $E(\hat{\theta})=0$.

  • If $E(\hat{\theta})=\theta$ then $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$.
  • If $E(\hat{\theta})<\theta$ then $\hat{\theta}$ is a negatively biased estimator of a parameter $\theta$.
  • If $E(\hat{\theta})>\theta$ then $\hat{\theta}$ is a positively biased estimator of a parameter $\theta$.

Bias of an estimator $\theta$ can be found by $$[E(\hat{\theta})-\theta]$$

  • $\overline{X}$ is an unbiased estimator of the mean of a population (whose mean exists).
  • $\overline{X}$ is an unbiased estimator of $\mu$ in a Normal distribution i.e. $N(\mu, \sigma^2)$.
  • $\overline{X}$ is an unbiased estimator of the parameter $p$ of the Bernoulli distribution.
  • $\overline{X}$ is an unbiased estimator of the parameter $\lambda$ of the Poisson distribution.

In each of these cases, the parameter $\mu, p$ or $\lambda$ is the mean of the respective population being sampled.

However, sample variance $S^2$ is not an unbiased estimator of population variance $\sigma^2$, but consistent.

It is possible to have more than one unbiased estimator for an unknown parameter. The sample mean and the sample median are unbiased estimators of the population mean $\mu$ if the population distribution is symmetrical.

Unbiasedness of the Estimator

Computer MCQs

R Programming Language

Absolute Measure of Dispersion

by

An absolute Measure of Dispersion gives an idea about the amount of dispersion/ spread in a set of observations. These quantities measure the dispersion in the same units as the units of original data. The absolute measure of dispersion cannot be used to compare the variation of two or more series/ data sets. The absolute measure of dispersion does not in itself, tell whether the variation is large or small.

Absolute Measure of Dispersion

The absolute Measure of Dispersion:

  1. Range
  2. Quartile Deviation
  3. Mean Deviation
  4. Variance or Standard Deviation
Absolute Measures of Dispersion

Range

The Range is the difference between the largest value and the smallest value in the data set. For ungrouped data, let $X_0$ be the smallest value and $X_n$ be the largest  value in a data set then the range ($R$) is defined as
$R=X_n-X_0$.

For grouped data Range can be calculated in three different ways
R=Mid point of the highest class – Midpoint of the lowest class
R=Upper class limit of the highest class – Lower class limit of the lower class
R=Upper class boundary of the highest class – The lower class boundary of the lowest class

Quartile Deviation (Semi-Interquantile Range)

The Quartile deviation (an absolute measure of dispersion) is defined as the difference between the third and first quartiles, and half of this range is called the semi-interquartile range (SIQD) or simply quartile deviation (QD). $$QD=\frac{Q_3-Q_1}{2}$$

The Quartile Deviation is superior to the range as it is not affected by extremely large or small observations, anyhow it does not give any information about the position of observation lying outside the two quantities. It is not amenable to mathematical treatment and is greatly affected by sampling variability. Although Quartile Deviation is not widely used as a measure of dispersion, it is used in situations in which extreme observations are thought to be unrepresentative/ misleading. Quartile Deviation is not based on all observations therefore it is affected by extreme observations.

Note: The range “Median ± QD” contains approximately 50% of the data.

Mean Deviation (Average Deviation)

The Mean Deviation is another absolute measure of dispersion and is defined as the arithmetic mean of the deviations measured either from the mean or from the median. All these deviations are counted as positive to avoid the difficulty arising from the property that the sum of deviations of observations from their mean is zero.

$MD=\frac{\sum|X-\overline{X}|}{n}\quad$ for ungrouped data for mean
$MD=\frac{\sum f|X-\overline{X}|}{\sum f}\quad$ for grouped data for mean
$MD=\frac{\sum|X-\tilde{X}|}{n}\quad$ for ungrouped data for median
$MD=\frac{\sum f|X-\tilde{X}|}{\sum f}\quad$ for grouped data for median
Mean Deviation can be calculated about other central tendencies but it is least when deviations are taken as the median.

The Mean Deviation gives more information than the range or the Quartile Deviation as it is based on all the observed values. The Mean Deviation does not give undue weight to occasional large deviations, so it should likely be used in situations where such deviations are likely to occur.

Variance and Standard Deviation

This absolute measure of dispersion is defined as the mean of the squares of deviations of all the observations from their mean. Traditionally population variance is denoted by $\sigma^2$ (sigma square) and for sample data denoted by $S^2$ or $s^2$.

Symbolically
$\sigma^2=\frac{\sum(X_i-\mu)^2}{N}\quad$ Population Variance for ungrouped data
$S^2=\frac{\sum(X_i-\overline{X})^2}{n}\quad$ sample Variance for ungrouped data
$\sigma^2=\frac{\sum f(X_i-\mu)^2}{\sum f}\quad$ Population Variance for grouped data
$\sigma^2=\frac{\sum f (X_i-\overline{X})^2}{\sum f}\quad$ Sample Variance for grouped data

The variance is denoted by $Var(X)$ for random variable $X$. The term variance was introduced by R. A. Fisher (1890-1982) in 1918. The variance is in squares of units and the variance is a large number compared to observations themselves.
Note that there are alternative formulas to compute Variance or Standard Deviations.

The positive square root of the variance is called Standard Deviation (SD) to express the deviation in the same units as the original observation. It is a measure of the average spread about the mean and is symbolically defined as

$\sigma^2=\sqrt{\frac{\sum(X_i-\mu)^2}{N}}\quad$ Population Standard for ungrouped data
$S^2=\sqrt{\frac{\sum(X_i-\overline{X})^2}{n}}\quad$ Sample Standard Deviation for ungrouped data
$\sigma^2=\sqrt{\frac{\sum f(X_i-\mu)^2}{\sum f}}\quad$ Population Standard Deviation for grouped data
$\sigma^2=\sqrt{\frac{\sum f (X_i-\overline{X})^2}{\sum f}}\quad$ Sample Standard Deviation for grouped data
Standard Deviation is the most useful measure of dispersion and is credited with the name Standard Deviation by Karl Pearson (1857-1936).

In some text Sample, Standard Deviation is defined as $S^2=\frac{\sum (X_i-\overline{X})^2}{n-1}$ based on the argument that knowledge of any $n-1$ deviations determines the remaining deviations as the sum of n deviations must be zero. This is an unbiased estimator of the population variance $\sigma^2$. The Standard Deviation has a definite mathematical measure, it utilizes all the observed values and is amenable to mathematical treatment but affected by extreme values.

References

R Language Tutorial

MCQs about Business Mathematics