---
_id: '7126'
abstract:
- lang: eng
  text: In the Minimum Description Length (MDL) principle, learning from the data
    is equivalent to an optimal coding problem. We show that the codes that achieve
    optimal compression in MDL are critical in a very precise sense. First, when they
    are taken as generative models of samples, they generate samples with broad empirical
    distributions and with a high value of the relevance, defined as the entropy of
    the empirical frequencies. These results are derived for different statistical
    models (Dirichlet model, independent and pairwise dependent spin models, and restricted
    Boltzmann machines). Second, MDL codes sit precisely at a second order phase transition
    point where the symmetry between the sampled outcomes is spontaneously broken.
    The order parameter controlling the phase transition is the coding cost of the
    samples. The phase transition is a manifestation of the optimality of MDL codes,
    and it arises because codes that achieve a higher compression do not exist. These
    results suggest a clear interpretation of the widespread occurrence of statistical
    criticality as a characterization of samples which are maximally informative on
    the underlying generative process.
article_number: '755'
article_processing_charge: No
article_type: original
author:
- first_name: Ryan J
  full_name: Cubero, Ryan J
  id: 850B2E12-9CD4-11E9-837F-E719E6697425
  last_name: Cubero
  orcid: 0000-0003-0002-1867
- first_name: Matteo
  full_name: Marsili, Matteo
  last_name: Marsili
- first_name: Yasser
  full_name: Roudi, Yasser
  last_name: Roudi
citation:
  ama: Cubero RJ, Marsili M, Roudi Y. Minimum description length codes are critical.
    <i>Entropy</i>. 2018;20(10). doi:<a href="https://doi.org/10.3390/e20100755">10.3390/e20100755</a>
  apa: Cubero, R. J., Marsili, M., &#38; Roudi, Y. (2018). Minimum description length
    codes are critical. <i>Entropy</i>. MDPI. <a href="https://doi.org/10.3390/e20100755">https://doi.org/10.3390/e20100755</a>
  chicago: Cubero, Ryan J, Matteo Marsili, and Yasser Roudi. “Minimum Description
    Length Codes Are Critical.” <i>Entropy</i>. MDPI, 2018. <a href="https://doi.org/10.3390/e20100755">https://doi.org/10.3390/e20100755</a>.
  ieee: R. J. Cubero, M. Marsili, and Y. Roudi, “Minimum description length codes
    are critical,” <i>Entropy</i>, vol. 20, no. 10. MDPI, 2018.
  ista: Cubero RJ, Marsili M, Roudi Y. 2018. Minimum description length codes are
    critical. Entropy. 20(10), 755.
  mla: Cubero, Ryan J., et al. “Minimum Description Length Codes Are Critical.” <i>Entropy</i>,
    vol. 20, no. 10, 755, MDPI, 2018, doi:<a href="https://doi.org/10.3390/e20100755">10.3390/e20100755</a>.
  short: R.J. Cubero, M. Marsili, Y. Roudi, Entropy 20 (2018).
date_created: 2019-11-26T22:18:05Z
date_published: 2018-10-01T00:00:00Z
date_updated: 2021-01-12T08:11:56Z
day: '01'
ddc:
- '519'
doi: 10.3390/e20100755
extern: '1'
file:
- access_level: open_access
  checksum: d642b7b661e1d5066b62e6ea9986b917
  content_type: application/pdf
  creator: rcubero
  date_created: 2019-11-26T22:23:08Z
  date_updated: 2020-07-14T12:47:50Z
  file_id: '7127'
  file_name: entropy-20-00755-v2.pdf
  file_size: 1366813
  relation: main_file
file_date_updated: 2020-07-14T12:47:50Z
has_accepted_license: '1'
intvolume: '        20'
issue: '10'
keyword:
- Minimum Description Length
- normalized maximum likelihood
- statistical criticality
- phase transitions
- large deviations
language:
- iso: eng
month: '10'
oa: 1
oa_version: Published Version
publication: Entropy
publication_identifier:
  issn:
  - 1099-4300
publication_status: published
publisher: MDPI
quality_controlled: '1'
status: public
title: Minimum description length codes are critical
tmp:
  image: /images/cc_by.png
  legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
  name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
  short: CC BY (4.0)
type: journal_article
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 20
year: '2018'
...