Markov decision processes in practice

This book presents classical Markov Decision Processes (MDP) for real-life applications and optimization. MDP allows users to develop and formally support approximate and simple decision rules, and this book showcases state-of-the-art applications in which MDP was key to the solution approach. The b...

Full description

Saved in:
Bibliographic Details
Other Authors: Boucherie, R. J. 1964- (Editor), Dijk, N. M. van, (Editor)
Format: eBook
Language: English
Published: Cham, Switzerland : Springer, [2017]
Series: International series in operations research & management science ; v. 248.
Subjects:
Markov processes.
Decision making.
elektronické knihy
electronic books
ISBN: 9783319477664
9783319477640
Physical Description: 1 online resource

Cover

Table of contents

LEADER 07339cam a2200517Ii 4500
001 98032
003 CZ-ZlUTB
005 20240914110950.0
006 m o d
007 cr cnu|||unuuu
008 170314s2017 sz ob 000 0 eng d
040 |a N$T  |b eng  |e rda  |e pn  |c N$T  |d IDEBK  |d N$T  |d OCLCO  |d EBLCP  |d YDX  |d NJR  |d UAB  |d OCLCF  |d IOG  |d AZU  |d UPM  |d MERER  |d ESU  |d VT2  |d Z5A  |d OCLCQ  |d JBG  |d IAD  |d ICW  |d ICN  |d OHI  |d OTZ  |d OCLCQ  |d IAS  |d OCLCQ  |d U3W  |d JG0  |d CAUOI  |d KSU  |d UKMGB  |d WYU  |d BRX  |d UKAHL  |d LQU  |d OCLCQ  |d ERF  |d UKBTH  |d LEATE  |d OCLCQ  |d SRU 
020 |a 9783319477664  |q (electronic bk.) 
020 |z 9783319477640 
024 7 |a 10.1007/978-3-319-47766-4  |2 doi 
024 8 |a 10.1007/978-3-319-47 
035 |a (OCoLC)975486905  |z (OCoLC)975928413  |z (OCoLC)976133801  |z (OCoLC)976240432  |z (OCoLC)978414155  |z (OCoLC)978750711  |z (OCoLC)979006523  |z (OCoLC)979315861  |z (OCoLC)979439346  |z (OCoLC)984868719  |z (OCoLC)1012018949  |z (OCoLC)1066585885  |z (OCoLC)1105183446  |z (OCoLC)1112588243  |z (OCoLC)1112914002  |z (OCoLC)1113159755  |z (OCoLC)1122813052  |z (OCoLC)1127134975  |z (OCoLC)1134830156  |z (OCoLC)1156121381 
245 0 0 |a Markov decision processes in practice /  |c Richard J. Boucherie, Nico M. van Dijk. 
264 1 |a Cham, Switzerland :  |b Springer,  |c [2017] 
264 4 |c ©2017 
300 |a 1 online resource 
336 |a text  |b txt  |2 rdacontent 
337 |a počítač  |b c  |2 rdamedia 
338 |a online zdroj  |b cr  |2 rdacarrier 
490 1 |a International series in operations research & management science ;  |v volume 248 
504 |a Includes bibliographical references. 
505 0 |a Foreword; Preface; Part I: General Theory; Part II: Healthcare; Part III: Transportation; Part IV: Production; Part V: Communications; Part VI: Financial Modeling; Summarizing; Acknowledgments; Contents; List of Contributors ; Part I General Theory; 1 One-Step Improvement Ideas and Computational Aspects; 1.1 Introduction; 1.2 The Average-Cost Markov Decision Model; 1.2.1 The Concept of Relative Values; 1.2.2 The Policy-Improvement Step; 1.2.3 The Odoni Bounds for Value Iteration; 1.3 Tailor-Made Policy-Iteration Algorithm; 1.3.1 A Queueing Control Problem with a Variable Service Rate. 
505 8 |a 1.4 One-Step Policy Improvement for Suboptimal Policies; 1.4.1 Dynamic Routing of Customers to Parallel Queues; 1.5 One-Stage-Look-Ahead Rule in Optimal Stopping; 1.5.1 Devil's Penny Problem; 1.5.2 A Game of Dropping Balls into Bins; 1.5.3 The Chow-Robbins Game; References; 2 Value Function Approximation in Complex Queueing Systems; 2.1 Introduction; 2.2 Difference Calculus for Markovian Birth-Death Systems; 2.3 Value Functions for Queueing Systems; 2.3.1 The M/Cox(r)/1 Queue; 2.3.2 Special Cases of the M/Cox(r)/1 Queue; 2.3.3 The M/M/s Queue; 2.3.4 The Blocking Costs in an M/M/s/s Queue. 
505 8 |a 2.3.5 Priority Queues; 2.4 Application: Routing to Parallel Queues; 2.5 Application: Dynamic Routing in Multiskill Call Centers; 2.6 Application: A Controlled Polling System; References; 3 Approximate Dynamic Programming by Practical Examples; 3.1 Introduction; 3.2 The Nomadic Trucker Example; 3.2.1 Problem Introduction; 3.2.2 MDP Model; 3.2.2.1 State; 3.2.2.2 Decision; 3.2.2.3 Costs; 3.2.2.4 New Information and Transition Function; 3.2.2.5 Solution; 3.2.3 Approximate Dynamic Programming; 3.2.3.1 Post-decision State; 3.2.3.2 Forward Dynamic Programming; 3.2.3.3 Value Function Approximation. 
505 8 |a 3.3 A Freight Consolidation Example; 3.3.1 Problem Introduction; 3.3.2 MDP Model; 3.3.2.1 State; 3.3.2.2 Decision; 3.3.2.3 Costs; 3.3.2.4 New Information and Transition Function; 3.3.2.5 Solution; 3.3.3 Approximate Dynamic Programming; 3.3.3.1 Post-decision State; 3.3.3.2 Forward Dynamic Programming; 3.3.3.3 Value Function Approximation; 3.4 A Healthcare Example; 3.4.1 Problem Introduction; 3.4.2 MDP Model; 3.4.2.1 State; 3.4.2.2 Decision; 3.4.2.3 Costs; 3.4.2.4 New Information and Transition Function; 3.4.2.5 Solution; 3.4.3 Approximate Dynamic Programming; 3.4.3.1 Post-decision State. 
505 8 |a 3.4.3.2 Forward Dynamic Programming; 3.4.3.3 Value Function Approximation; 3.5 What's More; 3.5.1 Policies; 3.5.2 Value Function Approximations; 3.5.3 Exploration vs Exploitation; Appendix; References; 4 Server Optimization of Infinite Queueing Systems; 4.1 Introduction; 4.2 Basic Definition and Notations; 4.3 Motivating Examples; 4.3.1 Optimization of a Queueing System with Two Different Servers; 4.3.2 Optimization of a Computational System with Power Saving Mode; 4.3.3 Structural Properties of These Motivating Examples; 4.4 Theoretical Background; 4.4.1 Subset Measures in Markov Chains. 
506 |a Plný text je dostupný pouze z IP adres počítačů Univerzity Tomáše Bati ve Zlíně nebo vzdáleným přístupem pro zaměstnance a studenty 
520 |a This book presents classical Markov Decision Processes (MDP) for real-life applications and optimization. MDP allows users to develop and formally support approximate and simple decision rules, and this book showcases state-of-the-art applications in which MDP was key to the solution approach. The book is divided into six parts. Part 1 is devoted to the state-of-the-art theoretical foundation of MDP, including approximate methods such as policy improvement, successive approximation and infinite state spaces as well as an instructive chapter on Approximate Dynamic Programming. It then continues with five parts of specific and non-exhaustive application areas. Part 2 covers MDP healthcare applications, which includes different screening procedures, appointment scheduling, ambulance scheduling and blood management. Part 3 explores MDP modeling within transportation. This ranges from public to private transportation, from airports and traffic lights to car parking or charging your electric car. Part 4 contains three chapters that illustrates the structure of approximate policies for production or manufacturing structures. In Part 5, communications is highlighted as an important application area for MDP. It includes Gittins indices, down-to-earth call centers and wireless sensor networks. Finally Part 6 is dedicated to financial modeling, offering an instructive review to account for financial portfolios and derivatives under proportional transactional costs. The MDP applications in this book illustrate a variety of both standard and non-standard aspects of MDP modeling and its practical use. This book should appeal to readers for practitioning, academic research and educational purposes, with a background in, among others, operations research, mathematics, computer science, and industrial engineering. 
590 |a SpringerLink  |b Springer Complete eBooks 
650 0 |a Markov processes. 
650 0 |a Decision making. 
655 7 |a elektronické knihy  |7 fd186907  |2 czenas 
655 9 |a electronic books  |2 eczenas 
700 1 |a Boucherie, R. J.  |q (Richard J.),  |d 1964-  |e editor. 
700 1 |a Dijk, N. M. van,  |e editor. 
776 0 8 |i Print version:  |t Markov decision processes in practice.  |d Cham, Switzerland : Springer, [2017]  |z 3319477641  |z 9783319477640  |w (OCoLC)959033672 
830 0 |a International series in operations research & management science ;  |v v. 248. 
856 4 0 |u https://proxy.k.utb.cz/login?url=https://link.springer.com/10.1007/978-3-319-47766-4  |y Plný text 
992 |c NTK-SpringerBM 
999 |c 98032  |d 98032 
993 |x NEPOSILAT  |y EIZ 

Similar Items