Hostname: page-component-7b9c58cd5d-dlb68 Total loading time: 0 Render date: 2025-03-13T15:41:08.414Z Has data issue: false hasContentIssue false
  • English
  • Français

Denumerable state continuous time Markov decision processes with unbounded cost and transition rates under average criterion

Published online by Cambridge University Press:  17 February 2009

Xianping Guo  and
Weiping Zhu
Xianping Guo
Affiliation:
Department of Mathematics, Zhongshan University, P. R. China
Weiping Zhu
Affiliation:
School of Computer Science, ADFA, The University of New South Wales, Canberra, ACT 2600, Australia; e-mail: [email protected]
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

In this paper, we consider denumerable state continuous time Markov decision processes with (possibly unbounded) transition and cost rates under average criterion. We present a set of conditions and prove the existence of both average cost optimal stationary policies and a solution of the average optimality equation under the conditions. The results in this paper are applied to an admission control queue model and controlled birth and death processes.

Type
Research Article
Information
The ANZIAM Journal , Volume 43 , Issue 4 , April 2002 , pp. 541 - 557
Copyright
Copyright © Australian Mathematical Society 2002

References

[1]Andson, W. J., Continuous time Markov chains (Springer, New York, 1991).CrossRefGoogle Scholar
[2]Bather, J., “Optimal stationary policies for denumerable Markov chains in continuous time”, Adv. Appl. Prob. 8 (1976) 144–158.CrossRefGoogle Scholar
[3]Bertsekas, D. P., Dynamic programming: deterministic and stochastic models (Prentice-Hall, Englewood Cliffs, NJ, 1987).Google Scholar
[4]Chung, K. L., Markov chains with stationary transition probabilities (Springer, Berlin, 1960).CrossRefGoogle Scholar
[5]Dong, Z. Q., “Continuous time Markov decision programming with average reward criterion—countable state and action space”, Scientia Sinica SP ISS(2) (1979) 11–148.Google Scholar
[6]Filar, J. A. and Vrieze, K., Competitive Markov decision processes (Springer, New York, 1996).CrossRefGoogle Scholar
[7]Haviv, M. and Puterman, M. L., “Bias optimality in controlled queuing systems”, J. Appl. Prob. 35 (1998) 16–150.CrossRefGoogle Scholar
[8]Hou, B., “Continuous-time Markov decision processes programming with polynomial reward”, Ph. D. Thesis, Institute of Appl. Math. Academic Sinica, Beijing, 1986.Google Scholar
[9]Hou, Z. T. and Guo, X. P., Markov decision processes (Science and Technology Press of Hunan, Changsha, China, 1998).Google Scholar
[10]Howard, R. A., Dynamic programming and Markov processes (Wiley, New York, 1960).Google Scholar
[11]Kakuman, P., “Nondiscounted continuous time Markov decision processes with countable state space”, SIAM J. Control 10 (1972) 210–220.CrossRefGoogle Scholar
[12]Lembersky, M., “On maximal rewards and ε-optimal policies in continuous time Markov decision processes”, Ann. Stat. 2 (1974) 159–169.CrossRefGoogle Scholar
[13]Miller, R. L., “Finite state continuous time Markov decision processes with an infinite planning horizon”, J. Math. Anal. Appl. 22 (1968) 552–569.CrossRefGoogle Scholar
[14]Puterman, M. L., Markov decision processes (John Wiley & Sons, 1994).CrossRefGoogle Scholar
[15]Sennott, L. I., “Average cost optimal stationary policies in infinite state Markov decision processes with unbounded cost”, Oper. Res. 37 (1989) 623–633.CrossRefGoogle Scholar
[16]Sennott, L. I., “Another Set of conditions for average optimality in Markov decision processes”, Systems Control Lett. 24 (1995) 147–151.CrossRefGoogle Scholar
[17]Serfozo, R., “Optimal control of random walks, birth and death processes, and queues”, Adv. Appl. Prob. 13 (1981) 61–83.CrossRefGoogle Scholar
[18]Song, J. S., “Continuous time Markov decision programming with non-uniformly bounded transition rate”, Scientia Sinica 12 (1987) 1258–1267.Google Scholar
[19]Wu, C. B., “Continuous time Markov decision processes with unbounded reward and non-uniformly bounded transition rate under discounted criterion”, Acta Math. Appl. Sinica 20 (1997) 196–208.Google Scholar
[20]Yushkevich, A. A. and Feinberg, E. A., “On homogeneous Markov model with continuous time and finite or countable state space”, Theory, Prob. Appl. 24 (1979) 156–161.Google Scholar
[21]Zheng, S. H., “Continuous time Markov decision programming with average reward criterion and unbounded reward rate”, Acta Math. Appl. Sinica 7 (1991) 6–16.CrossRefGoogle Scholar