{"oa":1,"oa_version":"Published Version","user_id":"c635000d-4b10-11ee-a964-aac5a93f6ac1","date_created":"2021-11-03T10:59:08Z","publisher":"Springer","scopus_import":"1","title":"Optimal combination of linear and spectral estimators for generalized linear models","quality_controlled":"1","status":"public","file":[{"content_type":"application/pdf","file_name":"2021_Springer_Mondelli.pdf","access_level":"open_access","creator":"alisjak","relation":"main_file","checksum":"9ea12dd8045a0678000a3a59295221cb","file_id":"10542","success":1,"date_created":"2021-12-13T15:47:54Z","file_size":2305731,"date_updated":"2021-12-13T15:47:54Z"}],"type":"journal_article","day":"17","month":"08","article_processing_charge":"Yes (via OA deal)","tmp":{"image":"/images/cc_by.png","name":"Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)","short":"CC BY (4.0)","legal_code_url":"https://creativecommons.org/licenses/by/4.0/legalcode"},"date_updated":"2023-09-05T14:13:57Z","date_published":"2021-08-17T00:00:00Z","publication_status":"published","department":[{"_id":"MaMo"}],"abstract":[{"text":"We study the problem of recovering an unknown signal π‘₯π‘₯ given measurements obtained from a generalized linear model with a Gaussian sensing matrix. Two popular solutions are based on a linear estimator π‘₯π‘₯^L and a spectral estimator π‘₯π‘₯^s. The former is a data-dependent linear combination of the columns of the measurement matrix, and its analysis is quite simple. The latter is the principal eigenvector of a data-dependent matrix, and a recent line of work has studied its performance. In this paper, we show how to optimally combine π‘₯π‘₯^L and π‘₯π‘₯^s. At the heart of our analysis is the exact characterization of the empirical joint distribution of (π‘₯π‘₯,π‘₯π‘₯^L,π‘₯π‘₯^s) in the high-dimensional limit. This allows us to compute the Bayes-optimal combination of π‘₯π‘₯^L and π‘₯π‘₯^s, given the limiting distribution of the signal π‘₯π‘₯. When the distribution of the signal is Gaussian, then the Bayes-optimal combination has the form πœƒπ‘₯π‘₯^L+π‘₯π‘₯^s and we derive the optimal combination coefficient. In order to establish the limiting distribution of (π‘₯π‘₯,π‘₯π‘₯^L,π‘₯π‘₯^s), we design and analyze an approximate message passing algorithm whose iterates give π‘₯π‘₯^L and approach π‘₯π‘₯^s. Numerical simulations demonstrate the improvement of the proposed combination with respect to the two methods considered separately.","lang":"eng"}],"year":"2021","keyword":["Applied Mathematics","Computational Theory and Mathematics","Computational Mathematics","Analysis"],"isi":1,"acknowledgement":"M. Mondelli would like to thank Andrea Montanari for helpful discussions. All the authors would like to thank the anonymous reviewers for their helpful comments.","file_date_updated":"2021-12-13T15:47:54Z","ddc":["510"],"project":[{"name":"IST Austria Open Access Fund","_id":"B67AFEDC-15C9-11EA-A837-991A96BB2854"}],"doi":"10.1007/s10208-021-09531-x","author":[{"full_name":"Mondelli, Marco","id":"27EB676C-8706-11E9-9510-7717E6697425","first_name":"Marco","last_name":"Mondelli","orcid":"0000-0002-3242-7020"},{"first_name":"Christos","last_name":"Thrampoulidis","full_name":"Thrampoulidis, Christos"},{"first_name":"Ramji","last_name":"Venkataramanan","full_name":"Venkataramanan, Ramji"}],"publication":"Foundations of Computational Mathematics","has_accepted_license":"1","external_id":{"arxiv":["2008.03326"],"isi":["000685721000001"]},"publication_identifier":{"eissn":["1615-3383"],"issn":["1615-3375"]},"citation":{"mla":"Mondelli, Marco, et al. β€œOptimal Combination of Linear and Spectral Estimators for Generalized Linear Models.” Foundations of Computational Mathematics, Springer, 2021, doi:10.1007/s10208-021-09531-x.","ieee":"M. Mondelli, C. Thrampoulidis, and R. Venkataramanan, β€œOptimal combination of linear and spectral estimators for generalized linear models,” Foundations of Computational Mathematics. Springer, 2021.","ista":"Mondelli M, Thrampoulidis C, Venkataramanan R. 2021. Optimal combination of linear and spectral estimators for generalized linear models. Foundations of Computational Mathematics.","chicago":"Mondelli, Marco, Christos Thrampoulidis, and Ramji Venkataramanan. β€œOptimal Combination of Linear and Spectral Estimators for Generalized Linear Models.” Foundations of Computational Mathematics. Springer, 2021. https://doi.org/10.1007/s10208-021-09531-x.","apa":"Mondelli, M., Thrampoulidis, C., & Venkataramanan, R. (2021). Optimal combination of linear and spectral estimators for generalized linear models. Foundations of Computational Mathematics. Springer. https://doi.org/10.1007/s10208-021-09531-x","ama":"Mondelli M, Thrampoulidis C, Venkataramanan R. Optimal combination of linear and spectral estimators for generalized linear models. Foundations of Computational Mathematics. 2021. doi:10.1007/s10208-021-09531-x","short":"M. Mondelli, C. Thrampoulidis, R. Venkataramanan, Foundations of Computational Mathematics (2021)."},"article_type":"original","language":[{"iso":"eng"}],"_id":"10211"}