---
_id: '14202'
abstract:
- lang: eng
  text: "Approximating a probability density in a tractable manner is a central task\r\nin
    Bayesian statistics. Variational Inference (VI) is a popular technique that\r\nachieves
    tractability by choosing a relatively simple variational family.\r\nBorrowing
    ideas from the classic boosting framework, recent approaches attempt\r\nto \\emph{boost}
    VI by replacing the selection of a single density with a\r\ngreedily constructed
    mixture of densities. In order to guarantee convergence,\r\nprevious works impose
    stringent assumptions that require significant effort for\r\npractitioners. Specifically,
    they require a custom implementation of the greedy\r\nstep (called the LMO) for
    every probabilistic model with respect to an\r\nunnatural variational family of
    truncated distributions. Our work fixes these\r\nissues with novel theoretical
    and algorithmic insights. On the theoretical\r\nside, we show that boosting VI
    satisfies a relaxed smoothness assumption which\r\nis sufficient for the convergence
    of the functional Frank-Wolfe (FW) algorithm.\r\nFurthermore, we rephrase the
    LMO problem and propose to maximize the Residual\r\nELBO (RELBO) which replaces
    the standard ELBO optimization in VI. These\r\ntheoretical enhancements allow
    for black box implementation of the boosting\r\nsubroutine. Finally, we present
    a stopping criterion drawn from the duality gap\r\nin the classic FW analyses
    and exhaustive experiments to illustrate the\r\nusefulness of our theoretical
    and algorithmic contributions."
article_processing_charge: No
arxiv: 1
author:
- first_name: Francesco
  full_name: Locatello, Francesco
  id: 26cfd52f-2483-11ee-8040-88983bcc06d4
  last_name: Locatello
  orcid: 0000-0002-4850-0683
- first_name: Gideon
  full_name: Dresdner, Gideon
  last_name: Dresdner
- first_name: Rajiv
  full_name: Khanna, Rajiv
  last_name: Khanna
- first_name: Isabel
  full_name: Valera, Isabel
  last_name: Valera
- first_name: Gunnar
  full_name: Rätsch, Gunnar
  last_name: Rätsch
citation:
  ama: 'Locatello F, Dresdner G, Khanna R, Valera I, Rätsch G. Boosting black box
    variational inference. In: <i>Advances in Neural Information Processing Systems</i>.
    Vol 31. Neural Information Processing Systems Foundation; 2018.'
  apa: 'Locatello, F., Dresdner, G., Khanna, R., Valera, I., &#38; Rätsch, G. (2018).
    Boosting black box variational inference. In <i>Advances in Neural Information
    Processing Systems</i> (Vol. 31). Montreal, Canada: Neural Information Processing
    Systems Foundation.'
  chicago: Locatello, Francesco, Gideon Dresdner, Rajiv Khanna, Isabel Valera, and
    Gunnar Rätsch. “Boosting Black Box Variational Inference.” In <i>Advances in Neural
    Information Processing Systems</i>, Vol. 31. Neural Information Processing Systems
    Foundation, 2018.
  ieee: F. Locatello, G. Dresdner, R. Khanna, I. Valera, and G. Rätsch, “Boosting
    black box variational inference,” in <i>Advances in Neural Information Processing
    Systems</i>, Montreal, Canada, 2018, vol. 31.
  ista: 'Locatello F, Dresdner G, Khanna R, Valera I, Rätsch G. 2018. Boosting black
    box variational inference. Advances in Neural Information Processing Systems.
    NeurIPS: Neural Information Processing Systems vol. 31.'
  mla: Locatello, Francesco, et al. “Boosting Black Box Variational Inference.” <i>Advances
    in Neural Information Processing Systems</i>, vol. 31, Neural Information Processing
    Systems Foundation, 2018.
  short: F. Locatello, G. Dresdner, R. Khanna, I. Valera, G. Rätsch, in:, Advances
    in Neural Information Processing Systems, Neural Information Processing Systems
    Foundation, 2018.
conference:
  end_date: 2018-12-08
  location: Montreal, Canada
  name: 'NeurIPS: Neural Information Processing Systems'
  start_date: 2018-12-03
date_created: 2023-08-22T14:15:40Z
date_published: 2018-06-06T00:00:00Z
date_updated: 2023-09-13T07:38:24Z
day: '06'
department:
- _id: FrLo
extern: '1'
external_id:
  arxiv:
  - '1806.02185'
intvolume: '        31'
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://arxiv.org/abs/1806.02185
month: '06'
oa: 1
oa_version: Preprint
publication: Advances in Neural Information Processing Systems
publication_identifier:
  eissn:
  - 1049-5258
  isbn:
  - '9781510884472'
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
scopus_import: '1'
status: public
title: Boosting black box variational inference
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 31
year: '2018'
...