{"status":"public","title":"Boosting black box variational inference","quality_controlled":"1","date_created":"2023-08-22T14:15:40Z","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","oa_version":"Preprint","oa":1,"scopus_import":"1","publisher":"Neural Information Processing Systems Foundation","date_published":"2018-06-06T00:00:00Z","extern":"1","publication_status":"published","abstract":[{"text":"Approximating a probability density in a tractable manner is a central task\r\nin Bayesian statistics. Variational Inference (VI) is a popular technique that\r\nachieves tractability by choosing a relatively simple variational family.\r\nBorrowing ideas from the classic boosting framework, recent approaches attempt\r\nto \\emph{boost} VI by replacing the selection of a single density with a\r\ngreedily constructed mixture of densities. In order to guarantee convergence,\r\nprevious works impose stringent assumptions that require significant effort for\r\npractitioners. Specifically, they require a custom implementation of the greedy\r\nstep (called the LMO) for every probabilistic model with respect to an\r\nunnatural variational family of truncated distributions. Our work fixes these\r\nissues with novel theoretical and algorithmic insights. On the theoretical\r\nside, we show that boosting VI satisfies a relaxed smoothness assumption which\r\nis sufficient for the convergence of the functional Frank-Wolfe (FW) algorithm.\r\nFurthermore, we rephrase the LMO problem and propose to maximize the Residual\r\nELBO (RELBO) which replaces the standard ELBO optimization in VI. These\r\ntheoretical enhancements allow for black box implementation of the boosting\r\nsubroutine. Finally, we present a stopping criterion drawn from the duality gap\r\nin the classic FW analyses and exhaustive experiments to illustrate the\r\nusefulness of our theoretical and algorithmic contributions.","lang":"eng"}],"main_file_link":[{"url":"https://arxiv.org/abs/1806.02185","open_access":"1"}],"department":[{"_id":"FrLo"}],"day":"06","type":"conference","date_updated":"2023-09-13T07:38:24Z","article_processing_charge":"No","month":"06","volume":31,"intvolume":"        31","year":"2018","citation":{"chicago":"Locatello, Francesco, Gideon Dresdner, Rajiv Khanna, Isabel Valera, and Gunnar Rätsch. “Boosting Black Box Variational Inference.” In <i>Advances in Neural Information Processing Systems</i>, Vol. 31. Neural Information Processing Systems Foundation, 2018.","ista":"Locatello F, Dresdner G, Khanna R, Valera I, Rätsch G. 2018. Boosting black box variational inference. Advances in Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 31.","apa":"Locatello, F., Dresdner, G., Khanna, R., Valera, I., &#38; Rätsch, G. (2018). Boosting black box variational inference. In <i>Advances in Neural Information Processing Systems</i> (Vol. 31). Montreal, Canada: Neural Information Processing Systems Foundation.","ama":"Locatello F, Dresdner G, Khanna R, Valera I, Rätsch G. Boosting black box variational inference. In: <i>Advances in Neural Information Processing Systems</i>. Vol 31. Neural Information Processing Systems Foundation; 2018.","short":"F. Locatello, G. Dresdner, R. Khanna, I. Valera, G. Rätsch, in:, Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2018.","mla":"Locatello, Francesco, et al. “Boosting Black Box Variational Inference.” <i>Advances in Neural Information Processing Systems</i>, vol. 31, Neural Information Processing Systems Foundation, 2018.","ieee":"F. Locatello, G. Dresdner, R. Khanna, I. Valera, and G. Rätsch, “Boosting black box variational inference,” in <i>Advances in Neural Information Processing Systems</i>, Montreal, Canada, 2018, vol. 31."},"conference":{"start_date":"2018-12-03","name":"NeurIPS: Neural Information Processing Systems","end_date":"2018-12-08","location":"Montreal, Canada"},"_id":"14202","language":[{"iso":"eng"}],"publication":"Advances in Neural Information Processing Systems","author":[{"last_name":"Locatello","first_name":"Francesco","id":"26cfd52f-2483-11ee-8040-88983bcc06d4","full_name":"Locatello, Francesco","orcid":"0000-0002-4850-0683"},{"full_name":"Dresdner, Gideon","last_name":"Dresdner","first_name":"Gideon"},{"full_name":"Khanna, Rajiv","last_name":"Khanna","first_name":"Rajiv"},{"full_name":"Valera, Isabel","last_name":"Valera","first_name":"Isabel"},{"last_name":"Rätsch","first_name":"Gunnar","full_name":"Rätsch, Gunnar"}],"publication_identifier":{"eissn":["1049-5258"],"isbn":["9781510884472"]},"external_id":{"arxiv":["1806.02185"]}}