Flashcards in Markov Chains Deck (16)

Loading flashcards...

1

## Discrete stochastic process

###
Set of random variables

Characterise the state of a system

Discrete points in time

2

## Markovian property

###
Memoryless property

Past doesn't matter

3

## Completely ergodic

###
limit exists AND rows are identical

Unichain and Aperiodic

Steady State exists

4

## States Accessible?

### probability going from one to another is greater than 0

5

## States communicate?

###
State i accessible from state j and state j accessible from state i

In the same class

6

## State reflexive?

### State communicates with itself

7

## State symmetric?

### If state i communicates with state j, then state j communicates with state i

8

## State transitive?

### if state i communicates with state j, and state j communicates with state k, then state i communicates with state k

9

## Transient state

### probability of returning to a state once it has left is <1 (or may never return)

10

## Recurrent state

###
if and only if the state is not transient

Definitely return

11

## Absorbing state

###
Special type of recurrent

Probability of being at a state is 1

Wont leave

12

## Unichain

###
Markov chain consisting of a single recurrent class

Transient classes transition into this class

13

## Multichain

### More than one recurrent class

14

## Periodic State

### t steps to return > 1

15

## Aperiodic State

###
t = o or cannot be found

i.e recurrent state with t = 1 or 𝑝𝑖𝑖 > 0

16