Stochastic dynamic programming solution to transmission scheduling: Multi sensor-multi process with wireless noisy channel

• Published in: Computers & electrical engineering Vol. 106; p. 108573
• Main Authors: Forootani, Ali, Forootani, Sara, Iervolino, Raffaele, Zarch, Majid Ghaniee
• Format: Journal Article
• Language: English
• Published: Elsevier Ltd 01.03.2023
• Subjects: Approximate dynamic programming; Kalman filter; Markov Decision Process; Neural Network; Random Forest; Sensor scheduling; Approximate dynamic programming; Neural Network; Sensor scheduling; Markov Decision Process; Kalman filter; Random Forest
• ISSN: 0045-7906; 1879-0755
• DOI: 10.1016/j.compeleceng.2022.108573

We investigate sensor scheduling for remote estimation when multiple smart sensors monitor multiple stochastic dynamical systems. The sensors transmit their measurements to a remote estimator through a noisy wireless communication channel. Such a remote estimator can receive multiple packets simultaneously sent by local sensors. Sensors transmit their measurements if their Signal Interference and Noise Ratio (SINR) is above a threshold. We compute the optimal policy for sensor scheduling by minimizing expected error covariance subject to total signal transmissions from all sensors. We model this problem as Markov Decision Process (MDP) with discounted cost per stage in the finite time horizon framework, then we employ stochastic Dynamic Programming as the optimization method. A novel algorithm based on sampling and machine learning techniques is proposed as the approximation. At each phase of the DP algorithm, samples are collected using a uniform probability distribution. The data is used to feed Neural Network (NN) and Random Forest (RF) models for cost function and policy approximation. The results of the proposed framework are supported by simulation examples comparing RF and NN as Approximate DP (ADP). Note that this idea builds a bridge among the recent advances in the area of data science, Machine Learning, and the ADP. •Modeling Multi Sensor-Multi Process (MSMP) sensor scheduling MDP.•New Approximate Dynamic Programming algorithm based on Machine Learning techniques.•Developing a Python based package to investigate the proposed NLS-ADP method.•Considering simulation scenarios to verify outcomes of the proposed NLS-ADP method.