Contenus

Toggle## Steady state

In markov chain in continuous time (and irreducible in discrete time), the vector of stationary probabilities always exists and is independent of the initial distribution (steady state). This vector π is the solution of the following system:

The system is called the balance equations.

## Example

Two identical machines run continuously unless they are broken. A repairer available as needed to repair the machines.

The repair time follows an exponential distribution with an average of 0.5 day. Once repaired, the uptime of a machine before its next breakage follows an exponential distribution of an average of 1 day. We assume that these distributions are independent. Consider the random process defined in terms of the number of failed machines.

Consider the random variable X (t ') describing the number of machines down at time t'. The states of the random variable are {0, 1, 2}. The repair time and the breakage time follow an exponential distribution so we are in the presence of a continuous time Markov chain. The repair time follows an exponential distribution with an average of 0.5 day. The repair rate is the reverse, ie 2 machines per day. Likewise, we deduce that the debris rate is 1 day. When the two machines are working, we have a breakage rate = machine1 + machine2 = 2.

The states describe the number of machines that have failed. The two machines cannot break at the same time so q_{02} = 0. The repairer only repairs one machine at a time so q_{20} = 0. The repair rate is 2 machines per day. The breakage rate for a machine is 1 machine per day, and 2 per day if both machines are running. Which gives us the following continuous state Markov chain:

If we take the balance equations, we have the following system:

Which gives for solution the vector (0.4, 0.4, 0.2). If one seeks to calculate the average number of broken machines, it suffices to calculate the mathematical expectation since the states represent the number of broken machines: 0 * 0.4 + 1 * 0.4 + 2 * 0.2 = 0.8.