Stochastic Processes

Stochastic Processes, Markov Process, Random Walk Models, Poisson Process, Brownian Process, Random Walk Probability

Random Walks Model: A Mathematical Formalization of Path

by

A random walk (first introduced by Karl Pearson in 1905) is a mathematical formalization of a path consisting of a series of random steps.

Random Walks Example

The following are some example related to random walks

  1. The path traced by a molecule as it travels in a liquid or gas,
  2. The search path of a foraging animal,
  3. The price of a fluctuating stock, and (iv) the financial status of a gambler.
    All these random steps in the example can be modeled as random walks, although they may not be truly random in reality.

Suppose there are $a+1$ positions marked out on a straight line and numbered 0,1,2,…, a. A person starts at $k$ where $0<k<a$. The walk proceeds in such a way that, at each step, there is probability p that the walker goes forward one step to $k+1$ and a probability $q=1-p$ that the walker goes back one step to $k-1$. The walk continues until either $0$ or $a$ is reached and then ends.

In a random walk, the position of a walker after having moved $n$ times is known as the state of the walk after $n$ steps or after covering $n$ stages. Thus the walk described above starts at stage $k$ at step $0$ and moves to either stage $k-1$ or stage $k+1$ after 1 step and so on.

If the walk is bounded, then the ends of the walk are known as barriers and they may have various properties. In this case, the barriers are said to be absorbing implying that the walk must end once a barrier is reached since there is no escape.

A useful diagrammatic way of representing random walk is by a transition or process diagram. In a transition diagram, the possible states of the walker can be represented by points on a line. If a transition between two points can occur in one step then those points are joined by a curve or edge as shown with an arrow indicating the direction of the walk and a weighting denoting the probability of the step occurring. A transition diagram is also known as a direct graph.

For small Markov processes the simplest way to represent the process is often in terms of its state transition diagram. In-state transition diagram each state (outcome) represents the process as a node in a graph. The arcs in the graph represent possible transitions between states of the process. The arcs are labeled by the transition rates between the states.

Example:  Suppose a meteorologist notices that the weather on a given day seems to depend on the weather conditions of the previous day. He/ She observes that if it is raining one day, then the next day is sunny 60% of the time and rainy 40% of the time; on the other hand, if it is sunny, the next day is sunny with probability 30% and rainy with probability 70%. Note that there are two outcomes (i) sunny and (ii) rainy in this Markov process.

The transition probability between sunny and rainy is 70%, between sunny and sunny is 30%, between rainy and sunny is 60%, and between rainy and rainy is 40%. The simple weather forecasting Markov Process in the transition diagram is

Random Walks https://itfeature.com
Random Walks

Random walk models are widely used in many fields such as Ecology, Economics, Psychology, Computer Science, Physics, Chemistry, Biology, etc. Random walks explain the observed behavior of processes in all these fields, serving as a fundamental model for the recorded stochastic activity.

Overall, the random walk model is a versatile tool within stochastic processes. It provides a framework for studying systems influenced by randomness and helps understand the evolution of such systems over time.

https://itfeature.com

Learning Statistics by Using R Programming Language

Visit: Quiz website

Stochastic Processes Introduction (2012)

by

Before starting the introduction of Stochastic Processes, let us start with some important definitions related to statistics and stochastic processes.

Important Terms and Definitions

Experiment: Any activity or situation having an uncertain outcome.

Sample Space:  The set of all possible outcomes is called sample space and every element $\omega$ of $\Omega$ is called sample point. In the Stochastic process, we will call it state space.

Event and Event Space:  An event is a subset of the sample space. The class of all events associated with a given experiment is defined to be the event space.

An event will always be a subset of the sample space, but for sufficiently large sample spaces, not all subsets will be events. Thus the class of all subsets of the sample space will not necessarily correspond to the event space.

Random Variable: A random variable is a mapping function that assigns outcomes of a random experiment to real numbers. The occurrence of the outcome follows a certain probability distribution. Therefore, a random variable is completely characterized by its probability density function (PDF). Or

A random variable is a map $X:\Omega \rightarrow R$ such that $F\{X \le x\} = \{\omega \in \Omega:x(\omega)\le x\} \in F$ for all $x \in R$.

Probability Space:  A probability space consists of $(\Omega, \mathfrak{F}, P)$ of three parts, sample space, a collection of events, and a probability measure.

Cumulative Distribution Function (CDF): Probability distribution function for the random variable $X$ such that $F(a) = P\{X \le a\}$.

time line

Time: A point of time either discrete or continuous

State: It describes the attribute of a system at some point in time $S=(s_1, s_2, \cdots, s_t)$.

It is convenient to assign some unique non-negative integer as an index to each possible value of the state vector $S$.

Activity: Something that takes some amount of time (duration) to occur. The activity culminates in an event.

Transition (movement from one state to another) Stochastic Processes

Transition:  Transition is caused by an event and it results in some movement from one state to another state.

Probability Measure:  A probability measure intends to be a function defined for all subsets of $\Omega$.

What is a Stochastic Process?

The word stochastic is derived from the Greek word “stoΩ’kæstIk” meaning “to aim at a target”. Stochastic processes involve a state which changes randomly.
Given a probability space $(\Omega, \mathfrak{F}, P)$  stochastic process $\{X(t), t\in T\}$ is a family of random variables, where the index set $T may be discrete $(T=\{0,1,2,\cdots,\})$ or continuous $(T=[0, \infty))$. The set of possible values which random variables $\{X(t), t\in T\}$ may assume is called the state space of the process, and denoted by $S$.

A continuous time stochastic process $\{X(t), t \in T\}; (T=[0, \infty))$ is said to have an independent increment of for all choices of $\{t_1,t_2, \cdots, t_n\}$, the $n$ random variables $X(t_1) – X(t_0), X(t_2) – X(t_1), \cdots, X(t_n)-X(t_{n-1})$ are independent. Using discrete time the state of the process at time $n+1$ depends only on its state at time $n$.

It is often used to represent the evolution of some random value or system over time.

Examples of Stochastic Processes

Examples of processes modeled as stochastic time series include stock market and exchange rate fluctuations, signals such as speech, audio, and video, medical data such as a patient’s EKG, EEG, blood pressure or temperature, random movement such as Brownian motion or random walks, counting process, Renewal process, Poisson process and Markov process.

A stochastic process is a collection of random variables that evolve over time (or some other index). Stochastic processes are powerful tools for modeling real-world systems that exhibit randomness. They are used in a wide range of fields, including finance, engineering, physics, and even biology.

Introduction to the Random Walks Model

Generate Random Numbers in R Language

Online MCQs about Computer Science and Information Technology