Category: Stochastic Processes

Dec 8, 2012

Random Walks Model: A Mathematical Formalization of Path

A random walk (first introduced by Karl Pearson in 1905) is a mathematical formalization of a path consisting series of random steps. Some examples may include,

The path traced by a molecule as it travels in a liquid or gas,
The search path of a foraging animal,
The price of fluctuating stock, and (iv) the financial status of a gambler.
All these random steps in the example can be modeled as random walks, although they may not be truly random in reality.

Suppose there are $a+1$ positions are marked out on a straight line and numbered 0,1,2,…, a. A person starts at $k$ where $0<k<a$. The walk proceeds in such a way that, at each step, there is probability p that the walker goes forward one step to $k+1$ and a probability $q=1-p$ that the walker goes back one step to $k-1$. The walk continues until either $0$ or $a$ is reached and then ends. In a random walk, the position of a walker after having moved $n$ times is known as the state of the walk after $n$ steps or after covering $n$ stages. Thus the walk described above starts at stage $k$ at step $0$ and moves to either stage $k-1$ or stage $k+1$ after 1 step and so on.

If the walk is bounded, then the ends of the walk are known as barriers and they may have various properties. In this case, the barriers are said to be absorbing implying that the walk must end once a barrier is reached since there is no escape.

A useful diagrammatic way of representing random walk is by a transition or process diagram. In a transition diagram, the possible states of the walker can be represented by points on a line. If a transition between two points can occur in one step then those points are joined by a curve or edge as shown with an arrow indicating the direction of the walk and a weighting denoting the probability of the step occurring. Transition diagram is also known as a direct graph.

For small Markov processes the simplest way to represent the process is often in terms of its state transition diagram. In-state transition diagram each state (outcome) represents the process as a node in a graph. The arcs in the graph represent possible transitions between states of the process. The arcs are labeled by the transition rates between the states.

Example: Suppose a meteorologist notices that the weather on a given day seems to depend on the weather conditions of the previous day. He/ She observes that if it is raining one day, then the next day is sunny 60% of the time and rainy 40% of the time; on the other hand, if it is sunny, the next day is sunny with probability 30% and rainy with probability 70%. Note that there are two outcomes (i) sunny (ii) rainy in this Markov process. The transition probability between sunny and rainy is 70%, between sunny and sunny is 30%, between rainy and sunny is 60% and between rainy and rainy is 40%. The simple weather forecasting Markov Process in transition diagram is

Random Walks

Random walks are widely being used in many fields such as Ecology, Economics, Psychology, Computer Science, Physics, Chemistry, and Biology, etc. Random walks explain the observed behavior of processes in all these fields, serving as a fundamental model for the recorded stochastic activity.

Stochastic Processes

Dec 2, 2012

by Muhammad Imdad Ullah

Stochastic Processes Introduction

Experiment: Any activity or situation having uncertain outcome.

Sample Space: The set of all possible outcomes is called sample space and every element ω of Ω is called sample point. In Stochastic process we will call it as state space.

Event and Event Space: An event is a subset of the sample space. The class of all events associated with a given experiment is defined to be the event space.

An event will always be a subset of the sample space, but for sufficiently large sample spaces, not all subsets will be events. Thus the class of all subsets of the sample space will not necessarily correspond to the event space.

Random Variable:
A random variable is a mapping function which assigns outcomes of a random experiment to real numbers. Occurrence of the outcome follows certain probability distribution. Therefore, a random variable is completely characterized by its probability density function (PDF). Or

A random variable is a map X: Ω→R such that {X ≤ x}={ω ε Ω : x(ω) ≤ x}ε F for all x ε R.

Probability Space: A probability space consists $(\Omega, \mathfrak{F}, P)$ of three parts, sample space, a collection of events and a probability measure.

Cumulative Distribution Function (CDF): Probability distribution function for the random variable X such that F(a)=P{X ≤ a }

Time: A point of time either discrete or continuous

State: It describe the attribute of a system at some point in time S=(s₁,s₂,…,s_t)

It is convenient to assign some unique non-negative integer as index to each possible value of the state vector S.

Activity: Something that takes some amount of time (duration) to occur. Activity culminates in an event.

Transition: Transition is caused by an event and it results in some movement from one state to another state.

Probability Measure: A probability measure intends to be a function defined for all subsets of Ω.

What is Stochastic Process?

The word stochastic is derived from the Greek word “stoΩ’kæstIk” meaning “to aim at a target”. Stochastic processes involves state which changes in a random way.
Given a probability space $(\Omega, \mathfrak{F}, P)$ stochastic process {X(t), t ε T} is a family of random variables, where the index set T may be discrete (T={0,1,2,…}) or continuous (T=[0, ∞)). The set of possible values which random variables {X(t), t ε T} may assume is called the state space of the process, and denoted by S. A continuous time stochastic process {X(t), t ε T}; (T=[0,∞)) is said to have independent increment of for all choices of {t₁,t₂,…,t_n}, the n random variables X(t₁)-X(t₀), X(t₂)-X(t₁),…,X(t_n)-X(t_n-1) are independent. Using discrete time the state of the process at time n+1 depends only on its state at time n.

It is often used to represent the evolution of some random value or system over time. Examples of processes modeled as stochastic time series include stock market and exchange rate fluctuations, signals such as speech, audio and video, medical data such as a patient’s EKG, EEG, blood pressure or temperature, random movement such as Brownian motion or random walks, counting process, Renewal process, Poisson process and Markov process.

Category: Stochastic Processes

Random Walks Model: A Mathematical Formalization of Path

Stochastic Processes Introduction

Categories