---
_id: '8193'
abstract:
- lang: eng
  text: 'Multiple-environment Markov decision processes (MEMDPs) are MDPs equipped
    with not one, but multiple probabilistic transition functions, which represent
    the various possible unknown environments. While the previous research on MEMDPs
    focused on theoretical properties for long-run average payoff, we study them with
    discounted-sum payoff and focus on their practical advantages and applications.
    MEMDPs can be viewed as a special case of Partially observable and Mixed observability
    MDPs: the state of the system is perfectly observable, but not the environment.
    We show that the specific structure of MEMDPs allows for more efficient algorithmic
    analysis, in particular for faster belief updates. We demonstrate the applicability
    of MEMDPs in several domains. In particular, we formalize the sequential decision-making
    approach to contextual recommendation systems as MEMDPs and substantially improve
    over the previous MDP approach.'
acknowledgement: Krishnendu Chatterjee is supported by the Austrian ScienceFund (FWF)
  NFN Grant No. S11407-N23 (RiSE/SHiNE),and COST Action GAMENET. Petr Novotn ́y is
  supported bythe Czech Science Foundation grant No. GJ19-15134Y.
article_processing_charge: No
author:
- first_name: Krishnendu
  full_name: Chatterjee, Krishnendu
  id: 2E5DCA20-F248-11E8-B48F-1D18A9856A87
  last_name: Chatterjee
  orcid: 0000-0002-4561-241X
- first_name: Martin
  full_name: Chmelik, Martin
  id: 3624234E-F248-11E8-B48F-1D18A9856A87
  last_name: Chmelik
- first_name: Deep
  full_name: Karkhanis, Deep
  last_name: Karkhanis
- first_name: Petr
  full_name: Novotný, Petr
  id: 3CC3B868-F248-11E8-B48F-1D18A9856A87
  last_name: Novotný
- first_name: Amélie
  full_name: Royer, Amélie
  id: 3811D890-F248-11E8-B48F-1D18A9856A87
  last_name: Royer
  orcid: 0000-0002-8407-0705
citation:
  ama: 'Chatterjee K, Chmelik M, Karkhanis D, Novotný P, Royer A. Multiple-environment
    Markov decision processes: Efficient analysis and applications. In: <i>Proceedings
    of the 30th International Conference on Automated Planning and Scheduling</i>.
    Vol 30. Association for the Advancement of Artificial Intelligence; 2020:48-56.'
  apa: 'Chatterjee, K., Chmelik, M., Karkhanis, D., Novotný, P., &#38; Royer, A. (2020).
    Multiple-environment Markov decision processes: Efficient analysis and applications.
    In <i>Proceedings of the 30th International Conference on Automated Planning and
    Scheduling</i> (Vol. 30, pp. 48–56). Nancy, France: Association for the Advancement
    of Artificial Intelligence.'
  chicago: 'Chatterjee, Krishnendu, Martin Chmelik, Deep Karkhanis, Petr Novotný,
    and Amélie Royer. “Multiple-Environment Markov Decision Processes: Efficient Analysis
    and Applications.” In <i>Proceedings of the 30th International Conference on Automated
    Planning and Scheduling</i>, 30:48–56. Association for the Advancement of Artificial
    Intelligence, 2020.'
  ieee: 'K. Chatterjee, M. Chmelik, D. Karkhanis, P. Novotný, and A. Royer, “Multiple-environment
    Markov decision processes: Efficient analysis and applications,” in <i>Proceedings
    of the 30th International Conference on Automated Planning and Scheduling</i>,
    Nancy, France, 2020, vol. 30, pp. 48–56.'
  ista: 'Chatterjee K, Chmelik M, Karkhanis D, Novotný P, Royer A. 2020. Multiple-environment
    Markov decision processes: Efficient analysis and applications. Proceedings of
    the 30th International Conference on Automated Planning and Scheduling. ICAPS:
    International Conference on Automated Planning and Scheduling vol. 30, 48–56.'
  mla: 'Chatterjee, Krishnendu, et al. “Multiple-Environment Markov Decision Processes:
    Efficient Analysis and Applications.” <i>Proceedings of the 30th International
    Conference on Automated Planning and Scheduling</i>, vol. 30, Association for
    the Advancement of Artificial Intelligence, 2020, pp. 48–56.'
  short: K. Chatterjee, M. Chmelik, D. Karkhanis, P. Novotný, A. Royer, in:, Proceedings
    of the 30th International Conference on Automated Planning and Scheduling, Association
    for the Advancement of Artificial Intelligence, 2020, pp. 48–56.
conference:
  end_date: 2020-10-30
  location: Nancy, France
  name: 'ICAPS: International Conference on Automated Planning and Scheduling'
  start_date: 2020-10-26
date_created: 2020-08-02T22:00:58Z
date_published: 2020-06-01T00:00:00Z
date_updated: 2023-09-07T13:16:18Z
day: '01'
department:
- _id: KrCh
intvolume: '        30'
language:
- iso: eng
month: '06'
oa_version: None
page: 48-56
project:
- _id: 25863FF4-B435-11E9-9278-68D0E5697425
  call_identifier: FWF
  grant_number: S11407
  name: Game Theory
publication: Proceedings of the 30th International Conference on Automated Planning
  and Scheduling
publication_identifier:
  eissn:
  - '23340843'
  issn:
  - '23340835'
publication_status: published
publisher: Association for the Advancement of Artificial Intelligence
quality_controlled: '1'
related_material:
  record:
  - id: '8390'
    relation: dissertation_contains
    status: public
scopus_import: '1'
status: public
title: 'Multiple-environment Markov decision processes: Efficient analysis and applications'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 30
year: '2020'
...