---
_id: '12738'
abstract:
- lang: eng
text: We study turn-based stochastic zero-sum games with lexicographic preferences
over objectives. Stochastic games are standard models in control, verification,
and synthesis of stochastic reactive systems that exhibit both randomness as well
as controllable and adversarial non-determinism. Lexicographic order allows one
to consider multiple objectives with a strict preference order. To the best of
our knowledge, stochastic games with lexicographic objectives have not been studied
before. For a mixture of reachability and safety objectives, we show that deterministic
lexicographically optimal strategies exist and memory is only required to remember
the already satisfied and violated objectives. For a constant number of objectives,
we show that the relevant decision problem is in NP∩coNP, matching the current
known bound for single objectives; and in general the decision problem is PSPACE-hard
and can be solved in NEXPTIME∩coNEXPTIME. We present an algorithm that computes
the lexicographically optimal strategies via a reduction to the computation of
optimal strategies in a sequence of single-objectives games. For omega-regular
objectives, we restrict our analysis to one-player games, also known as Markov
decision processes. We show that lexicographically optimal strategies exist and
need either randomization or finite memory. We present an algorithm that solves
the relevant decision problem in polynomial time. We have implemented our algorithms
and report experimental results on various case studies.
acknowledgement: Tobias Winkler and Joost-Pieter Katoen are supported by the DFG RTG
2236 UnRAVeL and the innovation programme under the Marie Skłodowska-Curie grant
agreement No. 101008233 (Mission). Krishnendu Chatterjee is supported by the ERC
CoG 863818 (ForM-SMArt) and the Vienna Science and Technology Fund (WWTF) Project
ICT15-003. Maximilian Weininger is supported by the DFG projects 383882557 Statistical
Unbounded Verification (SUV) and 427755713 Group-By Objectives in Probabilistic
Verification (GOPro). Stefanie Mohr is supported by the DFG RTG 2428 CONVEY. Open
Access funding enabled and organized by Projekt DEAL.
article_processing_charge: No
article_type: original
author:
- first_name: Krishnendu
full_name: Chatterjee, Krishnendu
id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87
last_name: Chatterjee
orcid: 0000-0002-4561-241X
- first_name: Joost P
full_name: Katoen, Joost P
id: 4524F760-F248-11E8-B48F-1D18A9856A87
last_name: Katoen
- first_name: Stefanie
full_name: Mohr, Stefanie
last_name: Mohr
- first_name: Maximilian
full_name: Weininger, Maximilian
last_name: Weininger
- first_name: Tobias
full_name: Winkler, Tobias
last_name: Winkler
citation:
ama: Chatterjee K, Katoen JP, Mohr S, Weininger M, Winkler T. Stochastic games with
lexicographic objectives. Formal Methods in System Design. 2023. doi:10.1007/s10703-023-00411-4
apa: Chatterjee, K., Katoen, J. P., Mohr, S., Weininger, M., & Winkler, T. (2023).
Stochastic games with lexicographic objectives. Formal Methods in System Design.
Springer Nature. https://doi.org/10.1007/s10703-023-00411-4
chicago: Chatterjee, Krishnendu, Joost P Katoen, Stefanie Mohr, Maximilian Weininger,
and Tobias Winkler. “Stochastic Games with Lexicographic Objectives.” Formal
Methods in System Design. Springer Nature, 2023. https://doi.org/10.1007/s10703-023-00411-4.
ieee: K. Chatterjee, J. P. Katoen, S. Mohr, M. Weininger, and T. Winkler, “Stochastic
games with lexicographic objectives,” Formal Methods in System Design.
Springer Nature, 2023.
ista: Chatterjee K, Katoen JP, Mohr S, Weininger M, Winkler T. 2023. Stochastic
games with lexicographic objectives. Formal Methods in System Design.
mla: Chatterjee, Krishnendu, et al. “Stochastic Games with Lexicographic Objectives.”
Formal Methods in System Design, Springer Nature, 2023, doi:10.1007/s10703-023-00411-4.
short: K. Chatterjee, J.P. Katoen, S. Mohr, M. Weininger, T. Winkler, Formal Methods
in System Design (2023).
date_created: 2023-03-19T23:00:59Z
date_published: 2023-03-08T00:00:00Z
date_updated: 2025-07-14T09:10:14Z
day: '08'
ddc:
- '000'
department:
- _id: KrCh
doi: 10.1007/s10703-023-00411-4
ec_funded: 1
external_id:
isi:
- '000946174300001'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.1007/s10703-023-00411-4
month: '03'
oa: 1
oa_version: Published Version
project:
- _id: 0599E47C-7A3F-11EA-A408-12923DDC885E
call_identifier: H2020
grant_number: '863818'
name: 'Formal Methods for Stochastic Models: Algorithms and Applications'
- _id: 25892FC0-B435-11E9-9278-68D0E5697425
grant_number: ICT15-003
name: Efficient Algorithms for Computer Aided Verification
publication: Formal Methods in System Design
publication_identifier:
eissn:
- 1572-8102
publication_status: epub_ahead
publisher: Springer Nature
quality_controlled: '1'
related_material:
record:
- id: '8272'
relation: earlier_version
status: public
scopus_import: '1'
status: public
title: Stochastic games with lexicographic objectives
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: journal_article
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2023'
...
---
_id: '8272'
abstract:
- lang: eng
text: We study turn-based stochastic zero-sum games with lexicographic preferences
over reachability and safety objectives. Stochastic games are standard models
in control, verification, and synthesis of stochastic reactive systems that exhibit
both randomness as well as angelic and demonic non-determinism. Lexicographic
order allows to consider multiple objectives with a strict preference order over
the satisfaction of the objectives. To the best of our knowledge, stochastic games
with lexicographic objectives have not been studied before. We establish determinacy
of such games and present strategy and computational complexity results. For strategy
complexity, we show that lexicographically optimal strategies exist that are deterministic
and memory is only required to remember the already satisfied and violated objectives.
For a constant number of objectives, we show that the relevant decision problem
is in NP∩coNP , matching the current known bound for single objectives; and
in general the decision problem is PSPACE -hard and can be solved in NEXPTIME∩coNEXPTIME
. We present an algorithm that computes the lexicographically optimal strategies
via a reduction to computation of optimal strategies in a sequence of single-objectives
games. We have implemented our algorithm and report experimental results on various
case studies.
alternative_title:
- LNCS
article_processing_charge: No
arxiv: 1
author:
- first_name: Krishnendu
full_name: Chatterjee, Krishnendu
id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87
last_name: Chatterjee
orcid: 0000-0002-4561-241X
- first_name: Joost P
full_name: Katoen, Joost P
id: 4524F760-F248-11E8-B48F-1D18A9856A87
last_name: Katoen
- first_name: Maximilian
full_name: Weininger, Maximilian
last_name: Weininger
- first_name: Tobias
full_name: Winkler, Tobias
last_name: Winkler
citation:
ama: 'Chatterjee K, Katoen JP, Weininger M, Winkler T. Stochastic games with lexicographic
reachability-safety objectives. In: International Conference on Computer Aided
Verification. Vol 12225. Springer Nature; 2020:398-420. doi:10.1007/978-3-030-53291-8_21'
apa: Chatterjee, K., Katoen, J. P., Weininger, M., & Winkler, T. (2020). Stochastic
games with lexicographic reachability-safety objectives. In International Conference
on Computer Aided Verification (Vol. 12225, pp. 398–420). Springer Nature.
https://doi.org/10.1007/978-3-030-53291-8_21
chicago: Chatterjee, Krishnendu, Joost P Katoen, Maximilian Weininger, and Tobias
Winkler. “Stochastic Games with Lexicographic Reachability-Safety Objectives.”
In International Conference on Computer Aided Verification, 12225:398–420.
Springer Nature, 2020. https://doi.org/10.1007/978-3-030-53291-8_21.
ieee: K. Chatterjee, J. P. Katoen, M. Weininger, and T. Winkler, “Stochastic games
with lexicographic reachability-safety objectives,” in International Conference
on Computer Aided Verification, 2020, vol. 12225, pp. 398–420.
ista: 'Chatterjee K, Katoen JP, Weininger M, Winkler T. 2020. Stochastic games with
lexicographic reachability-safety objectives. International Conference on Computer
Aided Verification. CAV: Computer Aided Verification, LNCS, vol. 12225, 398–420.'
mla: Chatterjee, Krishnendu, et al. “Stochastic Games with Lexicographic Reachability-Safety
Objectives.” International Conference on Computer Aided Verification, vol.
12225, Springer Nature, 2020, pp. 398–420, doi:10.1007/978-3-030-53291-8_21.
short: K. Chatterjee, J.P. Katoen, M. Weininger, T. Winkler, in:, International
Conference on Computer Aided Verification, Springer Nature, 2020, pp. 398–420.
conference:
name: 'CAV: Computer Aided Verification'
date_created: 2020-08-16T22:00:58Z
date_published: 2020-07-14T00:00:00Z
date_updated: 2025-07-14T09:10:14Z
day: '14'
ddc:
- '000'
department:
- _id: KrCh
doi: 10.1007/978-3-030-53291-8_21
ec_funded: 1
external_id:
arxiv:
- '2005.04018'
isi:
- '000695272500021'
file:
- access_level: open_access
checksum: 093d4788d7d5b2ce0ffe64fbe7820043
content_type: application/pdf
creator: dernst
date_created: 2020-08-17T11:32:44Z
date_updated: 2020-08-17T11:32:44Z
file_id: '8276'
file_name: 2020_LNCS_CAV_Chatterjee.pdf
file_size: 625056
relation: main_file
success: 1
file_date_updated: 2020-08-17T11:32:44Z
has_accepted_license: '1'
intvolume: ' 12225'
isi: 1
language:
- iso: eng
month: '07'
oa: 1
oa_version: Published Version
page: 398-420
project:
- _id: 0599E47C-7A3F-11EA-A408-12923DDC885E
call_identifier: H2020
grant_number: '863818'
name: 'Formal Methods for Stochastic Models: Algorithms and Applications'
- _id: 25892FC0-B435-11E9-9278-68D0E5697425
grant_number: ICT15-003
name: Efficient Algorithms for Computer Aided Verification
publication: International Conference on Computer Aided Verification
publication_identifier:
eissn:
- '16113349'
isbn:
- '9783030532901'
issn:
- '03029743'
publication_status: published
publisher: Springer Nature
quality_controlled: '1'
related_material:
record:
- id: '12738'
relation: later_version
status: public
scopus_import: '1'
status: public
title: Stochastic games with lexicographic reachability-safety objectives
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: conference
user_id: 4359f0d1-fa6c-11eb-b949-802e58b17ae8
volume: 12225
year: '2020'
...
---
_id: '79'
abstract:
- lang: eng
text: 'Markov Decision Processes (MDPs) are a popular class of models suitable for
solving control decision problems in probabilistic reactive systems. We consider
parametric MDPs (pMDPs) that include parameters in some of the transition probabilities
to account for stochastic uncertainties of the environment such as noise or input
disturbances. We study pMDPs with reachability objectives where the parameter
values are unknown and impossible to measure directly during execution, but there
is a probability distribution known over the parameter values. We study for the
first time computing parameter-independent strategies that are expectation optimal,
i.e., optimize the expected reachability probability under the probability distribution
over the parameters. We present an encoding of our problem to partially observable
MDPs (POMDPs), i.e., a reduction of our problem to computing optimal strategies
in POMDPs. We evaluate our method experimentally on several benchmarks: a motivating
(repeated) learner model; a series of benchmarks of varying configurations of
a robot moving on a grid; and a consensus protocol.'
alternative_title:
- LNCS
article_processing_charge: No
arxiv: 1
author:
- first_name: Sebastian
full_name: Arming, Sebastian
last_name: Arming
- first_name: Ezio
full_name: Bartocci, Ezio
last_name: Bartocci
- first_name: Krishnendu
full_name: Chatterjee, Krishnendu
id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87
last_name: Chatterjee
orcid: 0000-0002-4561-241X
- first_name: Joost P
full_name: Katoen, Joost P
id: 4524F760-F248-11E8-B48F-1D18A9856A87
last_name: Katoen
- first_name: Ana
full_name: Sokolova, Ana
last_name: Sokolova
citation:
ama: 'Arming S, Bartocci E, Chatterjee K, Katoen JP, Sokolova A. Parameter-independent
strategies for pMDPs via POMDPs. In: Vol 11024. Springer; 2018:53-70. doi:10.1007/978-3-319-99154-2_4'
apa: 'Arming, S., Bartocci, E., Chatterjee, K., Katoen, J. P., & Sokolova, A.
(2018). Parameter-independent strategies for pMDPs via POMDPs (Vol. 11024, pp.
53–70). Presented at the QEST: Quantitative Evaluation of Systems, Beijing, China:
Springer. https://doi.org/10.1007/978-3-319-99154-2_4'
chicago: Arming, Sebastian, Ezio Bartocci, Krishnendu Chatterjee, Joost P Katoen,
and Ana Sokolova. “Parameter-Independent Strategies for PMDPs via POMDPs,” 11024:53–70.
Springer, 2018. https://doi.org/10.1007/978-3-319-99154-2_4.
ieee: 'S. Arming, E. Bartocci, K. Chatterjee, J. P. Katoen, and A. Sokolova, “Parameter-independent
strategies for pMDPs via POMDPs,” presented at the QEST: Quantitative Evaluation
of Systems, Beijing, China, 2018, vol. 11024, pp. 53–70.'
ista: 'Arming S, Bartocci E, Chatterjee K, Katoen JP, Sokolova A. 2018. Parameter-independent
strategies for pMDPs via POMDPs. QEST: Quantitative Evaluation of Systems, LNCS,
vol. 11024, 53–70.'
mla: Arming, Sebastian, et al. Parameter-Independent Strategies for PMDPs via
POMDPs. Vol. 11024, Springer, 2018, pp. 53–70, doi:10.1007/978-3-319-99154-2_4.
short: S. Arming, E. Bartocci, K. Chatterjee, J.P. Katoen, A. Sokolova, in:, Springer,
2018, pp. 53–70.
conference:
end_date: 2018-09-07
location: Beijing, China
name: 'QEST: Quantitative Evaluation of Systems'
start_date: 2018-09-04
date_created: 2018-12-11T11:44:31Z
date_published: 2018-08-15T00:00:00Z
date_updated: 2023-09-13T09:38:28Z
day: '15'
department:
- _id: KrCh
- _id: ToHe
doi: 10.1007/978-3-319-99154-2_4
external_id:
arxiv:
- '1806.05126'
isi:
- '000548912200004'
intvolume: ' 11024'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/1806.05126
month: '08'
oa: 1
oa_version: Preprint
page: 53-70
publication_status: published
publisher: Springer
publist_id: '7975'
quality_controlled: '1'
scopus_import: '1'
status: public
title: Parameter-independent strategies for pMDPs via POMDPs
type: conference
user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1
volume: 11024
year: '2018'
...
---
_id: '10418'
abstract:
- lang: eng
text: We present a new proof rule for proving almost-sure termination of probabilistic
programs, including those that contain demonic non-determinism. An important question
for a probabilistic program is whether the probability mass of all its diverging
runs is zero, that is that it terminates "almost surely". Proving that can be
hard, and this paper presents a new method for doing so. It applies directly to
the program's source code, even if the program contains demonic choice. Like others,
we use variant functions (a.k.a. "super-martingales") that are real-valued and
decrease randomly on each loop iteration; but our key innovation is that the amount
as well as the probability of the decrease are parametric. We prove the soundness
of the new rule, indicate where its applicability goes beyond existing rules,
and explain its connection to classical results on denumerable (non-demonic) Markov
chains.
acknowledgement: "McIver and Morgan are grateful to David Basin and the Information
Security Group at ETH Zürich for hosting a six-month stay in Switzerland, during
part of which this work began. And thanks particularly to Andreas Lochbihler, who
shared with us the probabilistic termination problem that led to it. They acknowledge
the support of ARC grant DP140101119. Part of this work was carried out during the
Workshop on Probabilistic Programming Semantics\r\nat McGill University’s Bellairs
Research Institute on Barbados organised by Alexandra Silva and\r\nPrakash Panangaden.
Kaminski and Katoen are grateful to Sebastian Junges for spotting a flaw in §5.4."
article_number: '33'
article_processing_charge: No
article_type: original
arxiv: 1
author:
- first_name: Annabelle
full_name: Mciver, Annabelle
last_name: Mciver
- first_name: Carroll
full_name: Morgan, Carroll
last_name: Morgan
- first_name: Benjamin Lucien
full_name: Kaminski, Benjamin Lucien
last_name: Kaminski
- first_name: Joost P
full_name: Katoen, Joost P
id: 4524F760-F248-11E8-B48F-1D18A9856A87
last_name: Katoen
citation:
ama: Mciver A, Morgan C, Kaminski BL, Katoen JP. A new proof rule for almost-sure
termination. Proceedings of the ACM on Programming Languages. 2017;2(POPL).
doi:10.1145/3158121
apa: 'Mciver, A., Morgan, C., Kaminski, B. L., & Katoen, J. P. (2017). A new
proof rule for almost-sure termination. Proceedings of the ACM on Programming
Languages. Los Angeles, CA, United States: Association for Computing Machinery.
https://doi.org/10.1145/3158121'
chicago: Mciver, Annabelle, Carroll Morgan, Benjamin Lucien Kaminski, and Joost
P Katoen. “A New Proof Rule for Almost-Sure Termination.” Proceedings of the
ACM on Programming Languages. Association for Computing Machinery, 2017. https://doi.org/10.1145/3158121.
ieee: A. Mciver, C. Morgan, B. L. Kaminski, and J. P. Katoen, “A new proof rule
for almost-sure termination,” Proceedings of the ACM on Programming Languages,
vol. 2, no. POPL. Association for Computing Machinery, 2017.
ista: Mciver A, Morgan C, Kaminski BL, Katoen JP. 2017. A new proof rule for almost-sure
termination. Proceedings of the ACM on Programming Languages. 2(POPL), 33.
mla: Mciver, Annabelle, et al. “A New Proof Rule for Almost-Sure Termination.” Proceedings
of the ACM on Programming Languages, vol. 2, no. POPL, 33, Association for
Computing Machinery, 2017, doi:10.1145/3158121.
short: A. Mciver, C. Morgan, B.L. Kaminski, J.P. Katoen, Proceedings of the ACM
on Programming Languages 2 (2017).
conference:
end_date: 2018-01-13
location: Los Angeles, CA, United States
name: 'POPL: Programming Languages'
start_date: 2018-01-07
date_created: 2021-12-05T23:01:49Z
date_published: 2017-12-07T00:00:00Z
date_updated: 2021-12-07T08:04:14Z
day: '07'
department:
- _id: KrCh
- _id: ToHe
doi: 10.1145/3158121
external_id:
arxiv:
- '1711.03588'
intvolume: ' 2'
issue: POPL
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://dl.acm.org/doi/10.1145/3158121
month: '12'
oa: 1
oa_version: Published Version
publication: Proceedings of the ACM on Programming Languages
publication_identifier:
eissn:
- 2475-1421
publication_status: published
publisher: Association for Computing Machinery
quality_controlled: '1'
scopus_import: '1'
status: public
title: A new proof rule for almost-sure termination
type: journal_article
user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9
volume: 2
year: '2017'
...