Stationary Stochastic Process
A stationary stochastic process is said to be stationary if its mean and variance are constant over time and the value of the covariance between the two time periods depends only on a distance or gap or lag between the two time periods and not the actual time at which the covariance is computed. Such a stochastic process is also known as weak stationary, covariance stationary, second-order stationary, or wide-sense stochastic process.
Table of Contents
In other words, a sequence of random variables {$y_t$} is covariance stationary if there is no trend, and if the covariance does not change over time.
Strictly Stationary (Covariance Stationary)
A time series is strictly stationary if all the moments of its probability distribution are invariance over time but not for the first two (mean and variance).
Let $y_t$ be a stochastic time series with
$E(y_t) = \mu $ $\Rightarrow$ Mean
$V(y_t) = E(y_t -\mu)^2=\sigma^2 $ $\Rightarrow$ Variance
$\gamma_k = E[(y_t-\mu)(y_{t+k}-\mu)]$ $\Rightarrow$ Covariance = $Cov(y_t, y_{t-k})$
$\gamma_k$ is covariance or autocovariance at lag $k$.
If $k=0$ then $Var(y_t)=\sigma^2$ i.e. $Cov(y_t)=Var(y_t)=\sigma^2$
If $k=1$ then we have covariance between two adjacent values of $y$.
If $y_t$ is to be stationary, the mean, variance, and autocovariance of $y_{t+m}$ (shift or origin of $y=m$) must be the same as those of $y_t$. OR
If a time series is stationary, its mean, variance, and autocovariance remain the same no matter at what point we measure them, i.e., they are time-invariant.
Non-Stationary Time Series
A time series having a time-varying mean or a time-varying variance or both is called a non-stationary time series.
Purely Random/ White Noise Process
A stochastic process having zero mean and constant variance ($\sigma^2$) and serially uncorrelated is called a purely random/ white noise process.
If it is independent also then such a process is called strictly white noise.
White noise denoted by $\mu_t$ as $\mu_t \sim N(0, \sigma^2)$ i.e. $\mu_t$ is independently and identically distributed as a normal distribution with zero mean and constant variance.
A stationary time series is important because if a time series is non-stationary, we can study its behavior only for the time period under consideration. Each set of time series data will, therefore, be for a particular episode. As a consequence, it is not possible to generalize it to other time periods. Therefore, for forecasting, such (non-stochastic) time series may be of little practical value. Our interest is in stationary time series.
