Markov Chains With Maximum Entropy for Robotic Surveillance

• Published in: IEEE transactions on automatic control Vol. 64; no. 4; pp. 1566 - 1580
• Main Authors: George, Mishel, Jafarpour, Saber, Bullo, Francesco
• Format: Journal Article
• Language: English
• Published: New York IEEE 01.04.2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Computer simulation; Convex optimization; Entropy; entropy rate; Indexes; Iterative methods; Lagrange multiplier; Markov analysis; Markov chain; Markov chains; Markov processes; Maximum entropy; Measurement; Optimization; Robotics; Robots; Semidefinite programming; stochastic surveillance; Surveillance
• ISSN: 0018-9286; 1558-2523
• DOI: 10.1109/TAC.2018.2844120

This paper provides a comprehensive analysis of the following optimization problem: Maximize the entropy rate generated by a Markov chain over a connected graph of order <inline-formula><tex-math notation="LaTeX">n</tex-math></inline-formula> and subject to a prescribed stationary distribution. First, we show that this problem is strictly convex with global optimum lying in the interior of the feasible space. Second, using Lagrange multipliers, we provide a closed-form expression for the maxentropic Markov chain as a function of an <inline-formula><tex-math notation="LaTeX">n</tex-math></inline-formula>-dimensional vector, referred to as the maxentropic vector; we provide a provably converging iteration to compute this vector. Third, we show that the maxentropic Markov chain is reversible, compute its entropy rate and describe special cases, among other results. Fourth, through analysis and simulations, we show that our proposed procedure is more computationally efficient than semidefinite programming methods. Finally, we apply these results to robotic surveillance problems. We show realizations of the maxentropic Markov chains over prototypical robotic roadmaps and find that maxentropic Markov chains outperform minimum mean hitting time Markov chains for the so-called "intelligent intruders" with short attack durations. A comprehensive analysis of the following optimization problem: maximize the entropy rate generated by a Markov chain over a connected graph of order <inline-formula><tex-math notation="LaTeX">n</tex-math></inline-formula> and subject to a prescribed stationary distribution.