ANZIAM  J.  45 (2003), 49-74
Control of singularly perturbed Markov chains: A numerical study

H. Yang
  Department of Electrical and Computer Engineering
  University of Minnesota
  Minneapolis, MN 55455
G. Yin
  Department of Mathematics
  Wayne State University
  Detroit, MI 48202
K. Yin
  Department of Wood and Paper Science
  University of Minnesota
  St. Paul, MN 55108
Q. Zhang
  Department of Mathematics
  University of Georgia
  Athens, GA 30602

This work is devoted to numerical studies of nearly optimal controls of systems driven by singularly perturbed Markov chains. Our approach is based on the ideas of hierarchical controls applicable to many large-scale systems. A discrete-time linear quadratic control problem is examined. Its corresponding limit system is derived. The associated asymptotic properties and near optimality are demonstrated by numerical examples. Numerical experiments for a continuous-time hybrid linear quadratic regulator with Gaussian disturbances and a discrete-time Markov decision process are also presented. The numerical results have not only supported our theoretical findings but also provided insights for further applications.
Download the article in PDF format (size 329 Kb)

TeXAdel Scientific Publishing ©  Australian MS