Recurrence and transience in Markov chain

We will study a second classification of states depending on the type of behavior of the chain.

Let x be a state of the chain, we note the time of attaining x, denoted by Tx, the first moment when x is visited after the departure. By convention, the reaching time is infinite if we never reach x. The formula is the following:


If the chain starts from state x, we use the term return time.

A state x is said to be recurrent if: proba17

The state x is said transient otherwise, ie when:


A state is recurrent if we are sure to return to it, it is transient if there is a non-zero probability of never returning to it, and thus to leave it definitively.

An equivalence class is said to be recurrent, respectively transient, if one of its vertices is recurrent, resp. transient.

A recurrent class is closed, that is, the probability of leaving a recurrent class is zero.

Let x be any state belonging to the recurrent class C. Assume that there exists y ∉ C such that x → y and show that we have a contradiction. Let us first notice that y does not lead to any vertex of C, because otherwise we would have y → x and therefore x ↔ y and y ∈ C. In addition, we have:


Now, the probability of not returning to x is inferiorly bounded by the probability of going into y in finite time (since y does not lead to any state of C). So, we have the following relationship:


Which is a contradiction with a reccurent state x. We see that a recurrent class is closed, but the reciprocal is false in general, it is still verified if this class is of finite cardinality.

It is important to remember the following corollary: a Markov chain defined on a finite state space has at least one recurrent state.