---
_id: '7606'
abstract:
- lang: eng
  text: We derive a tight lower bound on equivocation (conditional entropy), or equivalently
    a tight upper bound on mutual information between a signal variable and channel
    outputs. The bound is in terms of the joint distribution of the signals and maximum
    a posteriori decodes (most probable signals given channel output). As part of
    our derivation, we describe the key properties of the distribution of signals,
    channel outputs and decodes, that minimizes equivocation and maximizes mutual
    information. This work addresses a problem in data analysis, where mutual information
    between signals and decodes is sometimes used to lower bound the mutual information
    between signals and channel outputs. Our result provides a corresponding upper
    bound.
article_number: '8989292'
article_processing_charge: No
arxiv: 1
author:
- first_name: Michal
  full_name: Hledik, Michal
  id: 4171253A-F248-11E8-B48F-1D18A9856A87
  last_name: Hledik
- first_name: Thomas R
  full_name: Sokolowski, Thomas R
  id: 3E999752-F248-11E8-B48F-1D18A9856A87
  last_name: Sokolowski
  orcid: 0000-0002-1287-3779
- first_name: Gašper
  full_name: Tkačik, Gašper
  id: 3D494DCA-F248-11E8-B48F-1D18A9856A87
  last_name: Tkačik
  orcid: 0000-0002-6699-1455
citation:
  ama: 'Hledik M, Sokolowski TR, Tkačik G. A tight upper bound on mutual information.
    In: <i>IEEE Information Theory Workshop, ITW 2019</i>. IEEE; 2019. doi:<a href="https://doi.org/10.1109/ITW44776.2019.8989292">10.1109/ITW44776.2019.8989292</a>'
  apa: 'Hledik, M., Sokolowski, T. R., &#38; Tkačik, G. (2019). A tight upper bound
    on mutual information. In <i>IEEE Information Theory Workshop, ITW 2019</i>. Visby,
    Sweden: IEEE. <a href="https://doi.org/10.1109/ITW44776.2019.8989292">https://doi.org/10.1109/ITW44776.2019.8989292</a>'
  chicago: Hledik, Michal, Thomas R Sokolowski, and Gašper Tkačik. “A Tight Upper
    Bound on Mutual Information.” In <i>IEEE Information Theory Workshop, ITW 2019</i>.
    IEEE, 2019. <a href="https://doi.org/10.1109/ITW44776.2019.8989292">https://doi.org/10.1109/ITW44776.2019.8989292</a>.
  ieee: M. Hledik, T. R. Sokolowski, and G. Tkačik, “A tight upper bound on mutual
    information,” in <i>IEEE Information Theory Workshop, ITW 2019</i>, Visby, Sweden,
    2019.
  ista: Hledik M, Sokolowski TR, Tkačik G. 2019. A tight upper bound on mutual information.
    IEEE Information Theory Workshop, ITW 2019. Information Theory Workshop, 8989292.
  mla: Hledik, Michal, et al. “A Tight Upper Bound on Mutual Information.” <i>IEEE
    Information Theory Workshop, ITW 2019</i>, 8989292, IEEE, 2019, doi:<a href="https://doi.org/10.1109/ITW44776.2019.8989292">10.1109/ITW44776.2019.8989292</a>.
  short: M. Hledik, T.R. Sokolowski, G. Tkačik, in:, IEEE Information Theory Workshop,
    ITW 2019, IEEE, 2019.
conference:
  end_date: 2019-08-28
  location: Visby, Sweden
  name: Information Theory Workshop
  start_date: 2019-08-25
date_created: 2020-03-22T23:00:47Z
date_published: 2019-08-01T00:00:00Z
date_updated: 2025-06-30T13:21:05Z
day: '01'
department:
- _id: GaTk
doi: 10.1109/ITW44776.2019.8989292
ec_funded: 1
external_id:
  arxiv:
  - '1812.01475'
  isi:
  - '000540384500015'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://arxiv.org/abs/1812.01475
month: '08'
oa: 1
oa_version: Preprint
project:
- _id: 2564DBCA-B435-11E9-9278-68D0E5697425
  call_identifier: H2020
  grant_number: '665385'
  name: International IST Doctoral Program
publication: IEEE Information Theory Workshop, ITW 2019
publication_identifier:
  isbn:
  - '9781538669006'
publication_status: published
publisher: IEEE
quality_controlled: '1'
related_material:
  record:
  - id: '15020'
    relation: dissertation_contains
    status: public
scopus_import: '1'
status: public
title: A tight upper bound on mutual information
type: conference
user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1
year: '2019'
...