Markov Chains Flashcards Preview

Decision Making > Markov Chains > Flashcards

Flashcards in Markov Chains Deck (16)
Loading flashcards...
1

Discrete stochastic process

Set of random variables
Characterise the state of a system
Discrete points in time

2

Markovian property

Memoryless property
Past doesn't matter

3

Completely ergodic

limit exists AND rows are identical
Unichain and Aperiodic
Steady State exists

4

States Accessible?

probability going from one to another is greater than 0

5

States communicate?

State i accessible from state j and state j accessible from state i
In the same class

6

State reflexive?

State communicates with itself

7

State symmetric?

If state i communicates with state j, then state j communicates with state i

8

State transitive?

if state i communicates with state j, and state j communicates with state k, then state i communicates with state k

9

Transient state

probability of returning to a state once it has left is <1 (or may never return)

10

Recurrent state

if and only if the state is not transient
Definitely return

11

Absorbing state

Special type of recurrent
Probability of being at a state is 1
Wont leave

12

Unichain

Markov chain consisting of a single recurrent class
Transient classes transition into this class

13

Multichain

More than one recurrent class

14

Periodic State

t steps to return > 1

15

Aperiodic State

t = o or cannot be found
i.e recurrent state with t = 1 or 𝑝𝑖𝑖 > 0

16

Key assumption

The next state depends only on the current state