# dtmc

Create discrete-time Markov chain

## Description

dtmc creates a discrete-time, finite-state, time-homogeneous Markov chain from a specified state transition matrix.

After creating a dtmc object, you can analyze the structure and evolution of the Markov chain, and visualize the Markov chain in various ways, by using the object functions. Also, you can use a dtmc object to specify the switching mechanism of a Markov-switching dynamic regression model (msVAR).

To create a switching mechanism, governed by threshold transitions and threshold variable data, for a threshold-switching dynamic regression model, see threshold and tsVAR.

## Creation

### Description

example

mc = dtmc(P) creates the discrete-time Markov chain object mc specified by the state transition matrix P.

example

mc = dtmc(P,'StateNames',stateNames) optionally associates the names stateNames to the states.

### Input Arguments

expand all

State transition matrix, specified as a numStates-by-numStates nonnegative numeric matrix.

P(i,j) is either the theoretical probability of a transition from state i to state j or an empirical count of observed transitions from state i to state j. P can be fully specified (all elements are nonnegative numbers), partially specified (elements are a mix of nonnegative numbers and NaN values), or unknown (completely composed of NaN values).

dtmc normalizes each row of P without any NaN values to sum to 1, then stores the normalized matrix in the property P.

Data Types: double

## Properties

expand all

You can set writable property values when you create the model object by using name-value pair argument syntax, or after you create the model object by using dot notation. For example, for the two-state model mc, to label the first and second states Depression and Recession, respectively, enter:

mc.StateNames = ["Depression" "Recession"];

Normalized transition matrix, specified as a numStates-by-numStates nonnegative numeric matrix.

If x is a row vector of length numStates specifying a distribution of states at time t (x sums to 1), then x*P is the distribution of states at time t + 1.

NaN entries indicate estimable transition probabilities. The estimate function of msVAR treats the known elements of P as equality constraints during optimization.

Data Types: double

Number of states, specified as a positive scalar.

Data Types: double

Unique state labels, specified as a string vector, cell vector of character vectors, or numeric vector of length numStates. Elements correspond to rows and columns of P.

Example: ["Depression" "Recession" "Stagnant" "Boom"]

Data Types: string

## Object Functions

dtmc objects require a fully specified transition matrix P.

expand all

 asymptotics Determine Markov chain asymptotics isergodic Check Markov chain for ergodicity isreducible Check Markov chain for reducibility classify Classify Markov chain states lazy Adjust Markov chain state inertia subchain Extract Markov subchain
 redistribute Compute Markov chain redistributions simulate Simulate Markov chain state walks
 distplot Plot Markov chain redistributions eigplot Plot Markov chain eigenvalues graphplot Plot Markov chain directed graph simplot Plot Markov chain simulations

## Examples

collapse all

Consider this theoretical, right-stochastic transition matrix of a stochastic process.

$P=\left[\begin{array}{cccc}0.5& 0.5& 0& 0\\ 0.5& 0& 0.5& 0\\ 0& 0& 0& 1\\ 0& 0& 1& 0\end{array}\right].$

Element ${P}_{ij}$ is the probability that the process transitions to state j at time t + 1 given that it is in state i at time t, for all t.

Create the Markov chain that is characterized by the transition matrix P.

P = [0.5 0.5 0 0; 0.5 0 0.5 0; 0 0 0 1; 0 0 1 0];
mc = dtmc(P);

mc is a dtmc object that represents the Markov chain.

Display the number of states in the Markov chain.

numstates = mc.NumStates
numstates = 4

Plot a directed graph of the Markov chain.

figure;
graphplot(mc);

Observe that states 3 and 4 form an absorbing class, while states 1 and 2 are transient.

Consider this transition matrix in which element $\left(i,j\right)$ is the observed number of times state i transitions to state j.

$P=\left[\begin{array}{cccc}16& 2& 3& 13\\ 5& 11& 10& 8\\ 9& 7& 6& 12\\ 4& 14& 15& 1\end{array}\right].$

For example, ${P}_{32}=7$ implies that state 3 transitions to state 2 seven times.

P = [16 2  3  13;
5  11 10 8;
9  7  6  12;
4  14 15 1];

Create the Markov chain that is characterized by the transition matrix P.

mc = dtmc(P);

Display the normalized transition matrix stored in mc. Verify that the elements within rows sum to 1 for all rows.

mc.P
ans = 4×4

0.4706    0.0588    0.0882    0.3824
0.1471    0.3235    0.2941    0.2353
0.2647    0.2059    0.1765    0.3529
0.1176    0.4118    0.4412    0.0294

sum(mc.P,2)
ans = 4×1

1
1
1
1

Plot a directed graph of the Markov chain.

figure;
graphplot(mc);

Consider the two-state business cycle of the US real gross national product (GNP) in [3] p. 697. At time t, real GNP can be in a state of expansion or contraction. Suppose that the following statements are true during the sample period.

• If real GNP is expanding at time t, then the probability that it will continue in an expansion state at time t + 1 is ${p}_{11}=0.90$.

• If real GNP is contracting at time t, then the probability that it will continue in a contraction state at time t + 1 is ${p}_{22}=0.75$.

Create the transition matrix for the model.

p11 = 0.90;
p22 = 0.75;

P = [p11 (1 - p11); (1 - p22) p22];

Create the Markov chain that is characterized by the transition matrix P. Label the two states.

mc = dtmc(P,'StateNames',["Expansion" "Contraction"])
mc =
dtmc with properties:

P: [2x2 double]
StateNames: ["Expansion"    "Contraction"]
NumStates: 2

Plot a directed graph of the Markov chain. Indicate the probability of transition by using edge colors.

figure;
graphplot(mc,'ColorEdges',true);

To help you explore the dtmc object functions, mcmix creates a Markov chain from a random transition matrix using only a specified number of states.

Create a five-state Markov chain from a random transition matrix.

rng(1); % For reproducibility
mc = mcmix(5)
mc =
dtmc with properties:

P: [5x5 double]
StateNames: ["1"    "2"    "3"    "4"    "5"]
NumStates: 5

mc is a dtmc object.

Plot the eigenvalues of the transition matrix on the complex plane.

figure;
eigplot(mc)

This spectrum determines structural properties of the Markov chain, such as periodicity and mixing rate.

Consider a Markov-switching autoregression (msVAR) model for the US GDP containing four economic regimes: depression, recession, stagnation, and expansion. To estimate the transition probabilities of the switching mechanism, you must supply a dtmc model with an unknown transition matrix entries to the msVAR framework.

Create a 4-regime Markov chain with an unknown transition matrix (all NaN entries). Specify the regime names.

P = nan(4);
statenames = ["Depression" "Recession" ...
"Stagnation" "Expansion"];

mcUnknown = dtmc(P,'StateNames',statenames)
mcUnknown =
dtmc with properties:

P: [4x4 double]
StateNames: ["Depression"    "Recession"    "Stagnation"    "Expansion"]
NumStates: 4

mcUnknown.P
ans = 4×4

NaN   NaN   NaN   NaN
NaN   NaN   NaN   NaN
NaN   NaN   NaN   NaN
NaN   NaN   NaN   NaN

Suppose economic theory states that the US economy never transitions to an expansion from a recession or depression. Create a 4-regime Markov chain with a partially known transition matrix representing the situation.

P(1,4) = 0;
P(2,4) = 0;

mcPartial = dtmc(P,'StateNames',statenames)
mcPartial =
dtmc with properties:

P: [4x4 double]
StateNames: ["Depression"    "Recession"    "Stagnation"    "Expansion"]
NumStates: 4

mcPartial.P
ans = 4×4

NaN   NaN   NaN     0
NaN   NaN   NaN     0
NaN   NaN   NaN   NaN
NaN   NaN   NaN   NaN

The estimate function of msVAR treats the known elements of mcPartial.P as equality constraints during optimization.

For more details on Markov-switching dynamic regression models, see msVAR.

## Alternatives

You also can create a Markov chain object using mcmix.

## References

[1] Gallager, R.G. Stochastic Processes: Theory for Applications. Cambridge, UK: Cambridge University Press, 2013.

[2] Haggstrom, O. Finite Markov Chains and Algorithmic Applications. Cambridge, UK: Cambridge University Press, 2002.

[3] Hamilton, James D. Time Series Analysis. Princeton, NJ: Princeton University Press, 1994.

[4] Norris, J. R. Markov Chains. Cambridge, UK: Cambridge University Press, 1997.

## Version History

Introduced in R2017b