Master Equation for Markov Processes

Introduction

• Discussed the master equation for Markov processes.
• Defined conditional probability density for the system being in state k at time t given it was in state j at time 0.
• Equations are first-order coupled differential equations.
• Transition rates (W) describe going from state l to state k (l

Master Equation Form

• Written as:
  • [ \frac{d}{dt} p(k, t | j, 0) = \sum_{l=1}^{n} [W_{kl} p(l, t | j, 0) - W_{lk} p(k, t | j, 0)] ]
  • Constraint: l ≠ k
• Initial condition: [ p(k, 0 | j) = \delta_{kj} ]

Vector and Matrix Representation

• Represent as a column vector.
  • [ p(t) = \begin{pmatrix} p_1(t | j) \ p_2(t | j) \ \vdots\ \ p_n(t | j) \end{pmatrix} ]
• Transition matrix W elements:
  • Off-diagonal: [ W_{kj} = W_{kl} , \text{(transition rates)} , \text{for} , k , \ne , j ]
  • Diagonal: [ W_{kk} = -\sum_{l \ne k} W_{lk} ]
• Determinant of W is zero, W has real elements:
  • Off-diagonal elements: 0 or positive
  • Diagonal elements: negative

Special Cases and Detailed Balance

• Important cases include detailed balance where each transition rate is pairwise equal.
  • [ W_{kl} p(l) = W_{lk} p(k) ]
• Detailed balance often applies in thermodynamic equilibrium.
• General stationary distribution and normalization.
  • [ \sum_{k=1}^{n} p(k) = 1 ]

Explicit Solution for Special Cases

• Simple cases such as dichotomous Markov processes (two states):
  • Transition rates: λ1 (from state 1 to state 2) and λ2 (from state 2 to state 1).
  • Stationary distribution:
    • [ p_1 = \frac{\lambda_2}{\lambda_1 + \lambda_2} ]
    • [ p_2 = \frac{\lambda_1}{\lambda_1 + \lambda_2} ]
• Average residence time: τ₁ and τ₂ related to transition rates.
  • [ \lambda_1 = \frac{1}{\tau_1} ]
  • [ \lambda_2 = \frac{1}{\tau_2} ]

Exponential Solution

• General solution for p(t):
  • [ p(t) = e^{Wt} p(0) ]
  • Eigenvalues of W, rate of decay towards equilibrium determined via Gershgorin Disc Theorem.

Correlation Time

• Defined as the harmonic mean of individual residence times:
  • [ \tau_c = \frac{2}{\lambda_1 + \lambda_2} = \frac{\tau_1 \tau_2}{\tau_1 + \tau_2} ]
• Relates to how the process loses memory of its initial state.
• Autocorrelation function expected to decay exponentially.
  • [ \langle\delta x(0) \delta x(t) \rangle \propto e^{-2\lambda t} ]

Summary

• The master equation is foundational to understanding Markov processes and their long-term behavior.
• The stationary distribution and transition rates are key to the system's equilibrium state.
• Detailed balance simplifies solution finding in thermodynamic systems.
• For dichotomous Markov processes, explicit solutions provide insight into dynamic behavior and correlation times.
• Future discussion to detail exponential correlations and autocorrelation functions in Markov processes.