Learning provably stabilizing neural controllers for discrete-time stochastic systems
Ansaripour M, Chatterjee K, Henzinger TA, Lechner M, Zikelic D. 2023. Learning provably stabilizing neural controllers for discrete-time stochastic systems. 21st International Symposium on Automated Technology for Verification and Analysis. ATVA: Automated Technology for Verification and Analysis, LNCS, vol. 14215, 357–379.
Download
No fulltext has been uploaded. References only!
Conference Paper
| Published
| English
Scopus indexed
Author
Ansaripour, Matin;
Chatterjee, KrishnenduISTA ;
Henzinger, Thomas AISTA ;
Lechner, MathiasISTA;
Zikelic, DjordjeISTA
Department
Grant
Series Title
LNCS
Abstract
We consider the problem of learning control policies in discrete-time stochastic systems which guarantee that the system stabilizes within some specified stabilization region with probability 1. Our approach is based on the novel notion of stabilizing ranking supermartingales (sRSMs) that we introduce in this work. Our sRSMs overcome the limitation of methods proposed in previous works whose applicability is restricted to systems in which the stabilizing region cannot be left once entered under any control policy. We present a learning procedure that learns a control policy together with an sRSM that formally certifies probability 1 stability, both learned as neural networks. We show that this procedure can also be adapted to formally verifying that, under a given Lipschitz continuous control policy, the stochastic system stabilizes within some stabilizing region with probability 1. Our experimental evaluation shows that our learning procedure can successfully learn provably stabilizing policies in practice.
Publishing Year
Date Published
2023-10-22
Proceedings Title
21st International Symposium on Automated Technology for Verification and Analysis
Publisher
Springer Nature
Acknowledgement
This work was supported in part by the ERC-2020-AdG 101020093, ERC CoG 863818 (FoRM-SMArt) and the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie Grant Agreement No. 665385.
Volume
14215
Page
357-379
Conference
ATVA: Automated Technology for Verification and Analysis
Conference Location
Singapore, Singapore
Conference Date
2023-10-24 – 2023-10-27
ISBN
ISSN
eISSN
IST-REx-ID
Cite this
Ansaripour M, Chatterjee K, Henzinger TA, Lechner M, Zikelic D. Learning provably stabilizing neural controllers for discrete-time stochastic systems. In: 21st International Symposium on Automated Technology for Verification and Analysis. Vol 14215. Springer Nature; 2023:357-379. doi:10.1007/978-3-031-45329-8_17
Ansaripour, M., Chatterjee, K., Henzinger, T. A., Lechner, M., & Zikelic, D. (2023). Learning provably stabilizing neural controllers for discrete-time stochastic systems. In 21st International Symposium on Automated Technology for Verification and Analysis (Vol. 14215, pp. 357–379). Singapore, Singapore: Springer Nature. https://doi.org/10.1007/978-3-031-45329-8_17
Ansaripour, Matin, Krishnendu Chatterjee, Thomas A Henzinger, Mathias Lechner, and Dorde Zikelic. “Learning Provably Stabilizing Neural Controllers for Discrete-Time Stochastic Systems.” In 21st International Symposium on Automated Technology for Verification and Analysis, 14215:357–79. Springer Nature, 2023. https://doi.org/10.1007/978-3-031-45329-8_17.
M. Ansaripour, K. Chatterjee, T. A. Henzinger, M. Lechner, and D. Zikelic, “Learning provably stabilizing neural controllers for discrete-time stochastic systems,” in 21st International Symposium on Automated Technology for Verification and Analysis, Singapore, Singapore, 2023, vol. 14215, pp. 357–379.
Ansaripour M, Chatterjee K, Henzinger TA, Lechner M, Zikelic D. 2023. Learning provably stabilizing neural controllers for discrete-time stochastic systems. 21st International Symposium on Automated Technology for Verification and Analysis. ATVA: Automated Technology for Verification and Analysis, LNCS, vol. 14215, 357–379.
Ansaripour, Matin, et al. “Learning Provably Stabilizing Neural Controllers for Discrete-Time Stochastic Systems.” 21st International Symposium on Automated Technology for Verification and Analysis, vol. 14215, Springer Nature, 2023, pp. 357–79, doi:10.1007/978-3-031-45329-8_17.