In Chapter 6 we present a general approach relying on the diffusion approximation to prove renewal theorems for Markov chains, so we consider Markov chains which may be approximated by a diffusion process. For a transient Markov chain with asymptotically zero drift, the average time spent by the chain in a unit interval is, roughly speaking, the reciprocal of the drift.

We apply a martingale-type technique and show that the asymptotic behaviour of the renewal measure depends heavily on the rate at which the drift vanishes. As in the last two chapters, two main cases are distinguished, either the drift of the chain decreases as 1/x or much more slowly than that. In contrast with the case of an asymptotically positive drift considered in Chapter 10, the case of vanishing drift is quite tricky to analyse since the Markov chain tends to infinity rather slowly.