Optimal combination of linear and spectral estimators for generalized linear models

Mondelli M, Thrampoulidis C, Venkataramanan R. 2021. Optimal combination of linear and spectral estimators for generalized linear models. Foundations of Computational Mathematics.

Download
OA 2021_Springer_Mondelli.pdf 2.31 MB [Published Version]

Journal Article | Published | English

Scopus indexed
Author
Mondelli, MarcoISTA ; Thrampoulidis, Christos; Venkataramanan, Ramji
Department
Abstract
We study the problem of recovering an unknown signal π‘₯π‘₯ given measurements obtained from a generalized linear model with a Gaussian sensing matrix. Two popular solutions are based on a linear estimator π‘₯π‘₯^L and a spectral estimator π‘₯π‘₯^s. The former is a data-dependent linear combination of the columns of the measurement matrix, and its analysis is quite simple. The latter is the principal eigenvector of a data-dependent matrix, and a recent line of work has studied its performance. In this paper, we show how to optimally combine π‘₯π‘₯^L and π‘₯π‘₯^s. At the heart of our analysis is the exact characterization of the empirical joint distribution of (π‘₯π‘₯,π‘₯π‘₯^L,π‘₯π‘₯^s) in the high-dimensional limit. This allows us to compute the Bayes-optimal combination of π‘₯π‘₯^L and π‘₯π‘₯^s, given the limiting distribution of the signal π‘₯π‘₯. When the distribution of the signal is Gaussian, then the Bayes-optimal combination has the form πœƒπ‘₯π‘₯^L+π‘₯π‘₯^s and we derive the optimal combination coefficient. In order to establish the limiting distribution of (π‘₯π‘₯,π‘₯π‘₯^L,π‘₯π‘₯^s), we design and analyze an approximate message passing algorithm whose iterates give π‘₯π‘₯^L and approach π‘₯π‘₯^s. Numerical simulations demonstrate the improvement of the proposed combination with respect to the two methods considered separately.
Publishing Year
Date Published
2021-08-17
Journal Title
Foundations of Computational Mathematics
Publisher
Springer
Acknowledgement
M. Mondelli would like to thank Andrea Montanari for helpful discussions. All the authors would like to thank the anonymous reviewers for their helpful comments.
ISSN
eISSN
IST-REx-ID

Cite this

Mondelli M, Thrampoulidis C, Venkataramanan R. Optimal combination of linear and spectral estimators for generalized linear models. Foundations of Computational Mathematics. 2021. doi:10.1007/s10208-021-09531-x
Mondelli, M., Thrampoulidis, C., & Venkataramanan, R. (2021). Optimal combination of linear and spectral estimators for generalized linear models. Foundations of Computational Mathematics. Springer. https://doi.org/10.1007/s10208-021-09531-x
Mondelli, Marco, Christos Thrampoulidis, and Ramji Venkataramanan. β€œOptimal Combination of Linear and Spectral Estimators for Generalized Linear Models.” Foundations of Computational Mathematics. Springer, 2021. https://doi.org/10.1007/s10208-021-09531-x.
M. Mondelli, C. Thrampoulidis, and R. Venkataramanan, β€œOptimal combination of linear and spectral estimators for generalized linear models,” Foundations of Computational Mathematics. Springer, 2021.
Mondelli M, Thrampoulidis C, Venkataramanan R. 2021. Optimal combination of linear and spectral estimators for generalized linear models. Foundations of Computational Mathematics.
Mondelli, Marco, et al. β€œOptimal Combination of Linear and Spectral Estimators for Generalized Linear Models.” Foundations of Computational Mathematics, Springer, 2021, doi:10.1007/s10208-021-09531-x.
All files available under the following license(s):
Creative Commons Attribution 4.0 International Public License (CC-BY 4.0):
Main File(s)
Access Level
OA Open Access
Date Uploaded
2021-12-13
MD5 Checksum
9ea12dd8045a0678000a3a59295221cb


Export

Marked Publications

Open Data ISTA Research Explorer

Web of Science

View record in Web of Science®

Sources

arXiv 2008.03326

Search this title in

Google Scholar