Sufficient statistics and Sufficient Estimators

Posted on by Muhammad Imdadullah
No Comments

An estimator $\hat{\theta}$ is sufficient if it make so much use of the information in the sample that no other estimator could extract from the sample, additional information about the population parameter being estimated.

The sample mean $\overline{X}$ utilizes all the values included in the sample so it is sufficient estimator of population mean $\mu$.

Sufficient estimators are often used to develop the estimator that have minimum variance among all unbiased estimators (MVUE).

If sufficient estimator exists, no other estimator from the sample can provide additional information about the population being estimated.

If there is a sufficient estimator, then there is no need to consider any of the non-sufficient estimator. Good estimator are function of sufficient statistics.

Let $X_1,X_2,\cdots,X_n$ be a random sample from a probability distribution with unknown parameter $\theta$, then this statistic (estimator) $U=g(X_1,X_,\cdots,X_n)$ observation gives $U=g(X_1,X_2,\cdots,X_n)$ does not depend upon population parameter $\Theta$.

Sufficient Statistic Example

The sample mean $\overline{X}$ is a sufficient for the population mean $\mu$ of a normal distribution with known variance. Once the sample mean is known, no further information about the population mean $\mu$ can be obtained from the sample itself, while median is not sufficient for the mean; even if the median of the sample is known, knowing the sample itself would provide further information about the population mean $\mu$.

Mathematical Definition of Sufficiency

Suppose that $X_1,X_2,\cdots,X_n \sim p(x;\theta)$. $T$ is sufficient for $\theta$ if the conditional distribution of $X_1,X_2,\cdots, X_n|T$ does not depend upon $\theta$. Thus
\[p(x_1,x_2,\cdots,x_n|t;\theta)=p(x_1,x_2,\cdots,x_n|t)\]
This means that we can replace $X_1,X_2,\cdots,X_n$ with $T(X_1,X_2,\cdots,X_n)$ without losing information.

For further reading visit: https://en.wikipedia.org/wiki/Sufficient_statistic

Download pdf file Sufficient Statistics:

 

Categories: Estimate and Estimation, Estimator Properties | Tags: , ,

Component of Time Series Data

Posted on by Muhammad Imdadullah
No Comments

Traditional methods of time series analysis are concerned with decomposing of a series into a trend, a seasonal variation and other irregular fluctuations. Although this approach is not always the best but still useful (Kendall and Stuart, 1996).

The components, by which time series is composed of, are called component of time series data. There are four basic Component of time series data described below.

Different Sources of Variation are:

  1. Seasonal effect (Seasonal Variation or Seasonal Fluctuations)
    Many of the time series data exhibits a seasonal variation which is annual period, such as sales and temperature readings.  This type of variation is easy to understand and can be easily measured or removed from the data to give de-seasonalized data.Seasonal Fluctuations describes any regular variation (fluctuation) with a period of less than one year for example cost of variation types of fruits and vegetables, cloths, unemployment figures, average daily rainfall, increase in sale of tea in winter, increase in sale of ice cream in summer etc., all show seasonal variations.The changes which repeat themselves within a fixed period, are also called seasonal variations, for example, traffic on roads in morning and evening hours, Sales at festivals like EID etc., increase in the number of passengers at weekend etc. Seasonal variations are caused by climate, social customs, religious activities etc.
  2. Other Cyclic Changes (Cyclical Variation or Cyclic Fluctuations)
    Time series exhibits Cyclical Variations at a fixed period due to some other physical cause, such as daily variation in temperature. Cyclical variation is a non-seasonal component which varies in recognizable cycle. sometime series exhibits oscillation which do not have a fixed period but are predictable to some extent. For example, economic data affected by business cycles with a period varying between about 5 and 7 years.In weekly or monthly data, the cyclical component may describes any regular variation (fluctuations) in time series data. The cyclical variation are periodic in nature and repeat themselves like business cycle, which has four phases (i) Peak (ii) Recession (iii) Trough/Depression (iv) Expansion.
  3. Trend (Secular Trend or Long Term Variation)
    It is a longer term change. Here we take into account the number of observations available and make a subjective assessment of what is long term. To understand the meaning of long term, let for example climate variables sometimes exhibit cyclic variation over a very long time period such as 50 years. If one just had 20 years data, this long term oscillation would appear to be a trend, but if several hundreds years of data is available, then long term oscillations would be visible.These movements are systematic in nature where the movements are broad, steady, showing slow rise or fall in the same direction. The trend may be linear or non-linear (curvilinear). Some examples of secular trend are: Increase in prices, Increase in pollution, increase in the need of wheat, increase in literacy rate, decrease in deaths due to advances in science.Taking averages over a certain period is a simple way of detecting trend in seasonal data. Change in averages with time is evidence of a trend in the given series, though there are more formal tests for detecting trend in time series.
  4. Other Irregular Variation (Irregular Fluctuations)
    When trend and cyclical variations are removed from a set of time series data, the residual left, which may or may not be random. Various techniques for analyzing series of this type examine to see “if irregular variation may be explained in terms of probability models such as moving average or autoregressive  models, i.e. we can see if any cyclical variation is still left in the residuals.These variation occur due to sudden causes are called residual variation (irregular variation or accidental or erratic fluctuations) and are unpredictable, for example rise in prices of steel due to strike in the factory, accident due to failure of break, flood, earth quick, war etc.
Component of Time Series Data

Component of Time Series Data

 

 

Categories: Time Series Analysis and Forecasting | Tags: , , ,

Frequency Distribution Table

Posted on by Muhammad Imdadullah
No Comments

Using Descriptive statistics we can organize the data to get the general pattern of the data and check where data values tend to concentrate and try to expose extreme or unusual data values.

A frequency distribution is a compact form of data in a table which displays the categories of observations according to there magnitudes and frequencies such that the similar or identical numerical values are grouped together. The categories are also known as groups, class intervals or simply classes. The classes must be mutually exclusive classes showing the number of observations in each class. The number of values falling in a particular category is called the frequency of that category denoted by f.

A Frequency Distribution shows us a summarized grouping of data divided into mutually exclusive classes and the number of occurrences in a class. Frequency distribution is a way of showing a raw (ungrouped or unorganized) data into grouped or organized data to show results of sales, production, income, loan, death rates, height, weight, temperature etc.

The relative frequency of a category is the proportion of observed frequency to the total frequency obtained by dividing observed frequency by the total frequency and denoted by r.f.  The sum of r.f. column should be one except for rounding error. Multiplying each relative frequency of class by 100 we can get percentage occurrence of a class. A relative frequency captures the relationship between a class total and the total number of observations.

The frequency distribution may be made for continuous data, discrete data and categorical data (for both qualitative and quantitative data). It can also be used to draw some graphs such as histogram, line chart, bar chart, pie chart, frequency polygon etc.

Steps to make a Frequency Distribution of data are:

  1. Decide about the number of classes. The number of classes usually between 5 and 20. Too many classes or too few classes might not reveal the basic shape of the data set, also it will be difficult to interpret such frequency distribution. The maximum number of classes may be determined by formula:
    \[\text{Number of Classes} = C = 1 + 3.3 log (n)\]
    \[\text{or} \quad C = \sqrt{n} \quad {approximately}\]where $n$ is the total number of observations in the data.
  2. Calculate the range of the data (Range = Max – Min) by finding minimum and maximum data value. Range will be used to determine the class interval or class width.
  3. Decide about width of the class denote by h and obtained by
    \[h = \frac{\text{Range}}{\text{Number of Classes}}= \frac{R}{C} \]
    Generally the class interval or class width is the same for all classes. The classes all taken together must cover at least the distance from the lowest value (minimum) in the data set up to the highest (maximum) value. Also note that equal class intervals are preferred in frequency distribution, while unequal class interval may be necessary in certain situations to avoid a large number of empty, or almost empty classes.
  4. Decide the individual class limits and select a suitable starting point of the first class which is arbitrary, it may be less than or equal to the minimum value. Usually it is started before the minimum value in such a way that the mid point (the average of lower and upper class limits of the first class) is properly placed.
  5. Take an observation and mark a vertical bar (|) for a class it belongs. A running tally is kept till the last observation. The tally counts  indicates five.
  6. Find the frequencies, relative frequency,  cumulative frequency etc. as required.
Frequency Distribution Table

Frequency Distribution Table

A frequency distribution is said to be skewed when its mean and median are different. The kurtosis of a frequency distribution is the concentration of scores at the mean, or how peaked the distribution appears if depicted graphically, for example, in a histogram. If the distribution is more peaked than the normal distribution it is said to be leptokurtic; if less peaked it is said to be platykurtic.

Further Reading: Frequency Distribution Table

 

Download pdf file of Frequency Distribution :

 

Categories: Basic Statistics | Tags: , ,

Objectives of Time Series Analysis

Posted on by Muhammad Imdadullah
No Comments

There are many objectives related to time series analysis, objectives of time series analysis may be classified as

  1. Description
  2. Explanation
  3. Prediction
  4. Control

The description of the objectives of time series analysis are as follows:

Description

The first step in the analysis is to plot the data and obtain simple descriptive measures (such as plotting data, looking for trends,  seasonal fluctuations and so on) of the main properties of the series.

Seasonal EffectIn above figure , there is a regular seasonal pattern of price change although this price pattern is not consistent.

Graph enables to look for “wild” observations or outlier (not appear to be consistent with the rest of the data). Graphing the time series make possible the presence of turning points where upward trend suddenly changed to a downward trend. If there is turning point, different models may have to be fitted to the two parts of the series.

Explanation

Observations taken on two or more variables, making possible to use the variation in one time series to explain the variation in another series. This may lead to deeper understanding. Multiple regression model may be helpful in this case.

Prediction

Given an observed time series, one may want to predict the future values of the series. It is an important task in sales of forecasting and is the analysis of economic and industrial time series. Prediction and forecasting used interchangeably.

Control

When time series generated to measure the quality of a manufacturing process (the aim may be) to control the process. Control procedures are of several different kinds.

In quality control, the observations are plotted on control chart and the controller takes action as a result of studying the charts.

A stochastic model is fitted to the series. Future values of the series are predicted and then the input process variables are adjusted so as to keep the process on target.

 

Categories: Time Series Analysis and Forecasting | Tags: ,

Binomial Random number Generation in R

Posted on by Muhammad Imdadullah
1 Comment

We will learn here how to generate Bernoulli or Binomial distribution in R with example of flip of a coin. This tutorial is based on how to generate random numbers according to different statistical distributions in R. Our focus is in binomial random number generation in R.

We know that in Bernoulli distribution, either something will happen or not such as coin flip has to outcomes head or tail (either head will occur or head will not occur i.e. tail will occur). For unbiased coin there will be 50%  chances that head or tail will occur in the long run. To generate a random number that are binomial in R, use rbinom(n, size,prob) command.

rbinom(n, size, prob) command has three parameters, namely

where
n is number of observations
size is number of trials (it may be zero or more)
prob is probability of success on each trial for example 1/2

Some Examples

  • One coin is tossed 10 times with probability of success=0.5
    coin will be fair (unbiased coin as p=1/2)
    >rbinom(n=10, size=1, prob=1/2)
    OUTPUT: 1 1 0 0 1 1 1 1 0 1
  • Two coins are tossed 10 times with probability of success=0.5
  • > rbinom(n=10, size=2, prob=1/2)
    OUTPUT: 2 1 2 1 2 0 1 0 0 1
  • One coin is tossed one hundred thousand times with probability of success=0.5
    > rbinom(n=100,000, size=1, prob=1/2)
  • store simulation results in $x$ vector
    > x<- rbinom(n=100,000, size=5, prob=1/2)
    count 1′s in x vector
    > sum(x)
    find the frequency distribution
    > table(x)
    creates a frequency distribution table with frequency
    > t=(table(x)/n *100)}
    plot frequency distribution table
    >plot(table(x),ylab=”Probability”,main=”size=5,prob=0.5″)

Download rbinom pdf file:
rbinom()112 downloads

View Video tutorial on rbinom command

Categories: Binomial Probability Distribution, R Language | Tags: , , ,