---
_id: '12860'
abstract:
- lang: eng
text: 'Memorization of the relation between entities in a dataset can lead to privacy
issues when using a trained model for question answering. We introduce Relational
Memorization (RM) to understand, quantify and control this phenomenon. While bounding
general memorization can have detrimental effects on the performance of a trained
model, bounding RM does not prevent effective learning. The difference is most
pronounced when the data distribution is long-tailed, with many queries having
only few training examples: Impeding general memorization prevents effective learning,
while impeding only relational memorization still allows learning general properties
of the underlying concepts. We formalize the notion of Relational Privacy (RP)
and, inspired by Differential Privacy (DP), we provide a possible definition of
Differential Relational Privacy (DrP). These notions can be used to describe and
compute bounds on the amount of RM in a trained model. We illustrate Relational
Privacy concepts in experiments with large-scale models for Question Answering.'
article_number: '2203.16701'
article_processing_charge: No
arxiv: 1
author:
- first_name: Simone
full_name: Bombari, Simone
id: ca726dda-de17-11ea-bc14-f9da834f63aa
last_name: Bombari
- first_name: Alessandro
full_name: Achille, Alessandro
last_name: Achille
- first_name: Zijian
full_name: Wang, Zijian
last_name: Wang
- first_name: Yu-Xiang
full_name: Wang, Yu-Xiang
last_name: Wang
- first_name: Yusheng
full_name: Xie, Yusheng
last_name: Xie
- first_name: Kunwar Yashraj
full_name: Singh, Kunwar Yashraj
last_name: Singh
- first_name: Srikar
full_name: Appalaraju, Srikar
last_name: Appalaraju
- first_name: Vijay
full_name: Mahadevan, Vijay
last_name: Mahadevan
- first_name: Stefano
full_name: Soatto, Stefano
last_name: Soatto
citation:
ama: Bombari S, Achille A, Wang Z, et al. Towards differential relational privacy
and its use in question answering. arXiv. doi:10.48550/arXiv.2203.16701
apa: Bombari, S., Achille, A., Wang, Z., Wang, Y.-X., Xie, Y., Singh, K. Y., … Soatto,
S. (n.d.). Towards differential relational privacy and its use in question answering.
arXiv. https://doi.org/10.48550/arXiv.2203.16701
chicago: Bombari, Simone, Alessandro Achille, Zijian Wang, Yu-Xiang Wang, Yusheng
Xie, Kunwar Yashraj Singh, Srikar Appalaraju, Vijay Mahadevan, and Stefano Soatto.
“Towards Differential Relational Privacy and Its Use in Question Answering.” ArXiv,
n.d. https://doi.org/10.48550/arXiv.2203.16701.
ieee: S. Bombari et al., “Towards differential relational privacy and its
use in question answering,” arXiv. .
ista: Bombari S, Achille A, Wang Z, Wang Y-X, Xie Y, Singh KY, Appalaraju S, Mahadevan
V, Soatto S. Towards differential relational privacy and its use in question answering.
arXiv, 2203.16701.
mla: Bombari, Simone, et al. “Towards Differential Relational Privacy and Its Use
in Question Answering.” ArXiv, 2203.16701, doi:10.48550/arXiv.2203.16701.
short: S. Bombari, A. Achille, Z. Wang, Y.-X. Wang, Y. Xie, K.Y. Singh, S. Appalaraju,
V. Mahadevan, S. Soatto, ArXiv (n.d.).
date_created: 2023-04-23T16:11:48Z
date_published: 2022-03-30T00:00:00Z
date_updated: 2023-04-25T07:34:49Z
day: '30'
department:
- _id: GradSch
- _id: MaMo
doi: 10.48550/arXiv.2203.16701
external_id:
arxiv:
- '2203.16701'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2203.16701
month: '03'
oa: 1
oa_version: Preprint
publication: arXiv
publication_status: submitted
status: public
title: Towards differential relational privacy and its use in question answering
type: preprint
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2022'
...
---
_id: '10364'
abstract:
- lang: eng
text: 'This paper characterizes the latency of the simplified successive-cancellation
(SSC) decoding scheme for polar codes under hardware resource constraints. In
particular, when the number of processing elements P that can perform SSC decoding
operations in parallel is limited, as is the case in practice, the latency of
SSC decoding is O(N1-1/μ + N/P log2 log2 N/P), where N is the block length of
the code and μ is the scaling exponent of the channel. Three direct consequences
of this bound are presented. First, in a fully-parallel implementation where P
= N/2, the latency of SSC decoding is O(N1-1/μ), which is sublinear in the block
length. This recovers a result from our earlier work. Second, in a fully-serial
implementation where P = 1, the latency of SSC decoding scales as O(N log2 log2
N). The multiplicative constant is also calculated: we show that the latency of
SSC decoding when P = 1 is given by (2 + o(1))N log2 log2 N. Third, in a semi-parallel
implementation, the smallest P that gives the same latency as that of the fully-parallel
implementation is P = N1/μ. The tightness of our bound on SSC decoding latency
and the applicability of the foregoing results is validated through extensive
simulations.'
acknowledgement: "S. A. Hashemi is supported by a Postdoctoral Fellowship from the
Natural Sciences and\r\nEngineering Research Council of Canada (NSERC) and by Huawei.
M. Mondelli is partially\r\nsupported by the 2019 Lopez-Loreta Prize. A. Fazeli
and A. Vardy were supported in part by\r\nthe National Science Foundation under
Grant CCF-1764104."
article_processing_charge: No
article_type: original
arxiv: 1
author:
- first_name: Seyyed Ali
full_name: Hashemi, Seyyed Ali
last_name: Hashemi
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
- first_name: Arman
full_name: Fazeli, Arman
last_name: Fazeli
- first_name: Alexander
full_name: Vardy, Alexander
last_name: Vardy
- first_name: John
full_name: Cioffi, John
last_name: Cioffi
- first_name: Andrea
full_name: Goldsmith, Andrea
last_name: Goldsmith
citation:
ama: Hashemi SA, Mondelli M, Fazeli A, Vardy A, Cioffi J, Goldsmith A. Parallelism
versus latency in simplified successive-cancellation decoding of polar codes.
IEEE Transactions on Wireless Communications. 2022;21(6):3909-3920. doi:10.1109/TWC.2021.3125626
apa: Hashemi, S. A., Mondelli, M., Fazeli, A., Vardy, A., Cioffi, J., & Goldsmith,
A. (2022). Parallelism versus latency in simplified successive-cancellation decoding
of polar codes. IEEE Transactions on Wireless Communications. Institute
of Electrical and Electronics Engineers. https://doi.org/10.1109/TWC.2021.3125626
chicago: Hashemi, Seyyed Ali, Marco Mondelli, Arman Fazeli, Alexander Vardy, John
Cioffi, and Andrea Goldsmith. “Parallelism versus Latency in Simplified Successive-Cancellation
Decoding of Polar Codes.” IEEE Transactions on Wireless Communications.
Institute of Electrical and Electronics Engineers, 2022. https://doi.org/10.1109/TWC.2021.3125626.
ieee: S. A. Hashemi, M. Mondelli, A. Fazeli, A. Vardy, J. Cioffi, and A. Goldsmith,
“Parallelism versus latency in simplified successive-cancellation decoding of
polar codes,” IEEE Transactions on Wireless Communications, vol. 21, no.
6. Institute of Electrical and Electronics Engineers, pp. 3909–3920, 2022.
ista: Hashemi SA, Mondelli M, Fazeli A, Vardy A, Cioffi J, Goldsmith A. 2022. Parallelism
versus latency in simplified successive-cancellation decoding of polar codes.
IEEE Transactions on Wireless Communications. 21(6), 3909–3920.
mla: Hashemi, Seyyed Ali, et al. “Parallelism versus Latency in Simplified Successive-Cancellation
Decoding of Polar Codes.” IEEE Transactions on Wireless Communications,
vol. 21, no. 6, Institute of Electrical and Electronics Engineers, 2022, pp. 3909–20,
doi:10.1109/TWC.2021.3125626.
short: S.A. Hashemi, M. Mondelli, A. Fazeli, A. Vardy, J. Cioffi, A. Goldsmith,
IEEE Transactions on Wireless Communications 21 (2022) 3909–3920.
date_created: 2021-11-28T23:01:29Z
date_published: 2022-06-01T00:00:00Z
date_updated: 2024-09-10T13:03:18Z
day: '01'
department:
- _id: MaMo
doi: 10.1109/TWC.2021.3125626
external_id:
arxiv:
- '2012.13378'
isi:
- '000809406400028'
intvolume: ' 21'
isi: 1
issue: '6'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/2012.13378
month: '06'
oa: 1
oa_version: Preprint
page: 3909-3920
project:
- _id: 059876FA-7A3F-11EA-A408-12923DDC885E
name: Prix Lopez-Loretta 2019 - Marco Mondelli
publication: IEEE Transactions on Wireless Communications
publication_identifier:
eissn:
- 1558-2248
issn:
- 1536-1276
publication_status: published
publisher: Institute of Electrical and Electronics Engineers
quality_controlled: '1'
related_material:
record:
- id: '10053'
relation: earlier_version
status: public
scopus_import: '1'
status: public
title: Parallelism versus latency in simplified successive-cancellation decoding of
polar codes
type: journal_article
user_id: 4359f0d1-fa6c-11eb-b949-802e58b17ae8
volume: 21
year: '2022'
...
---
_id: '11420'
abstract:
- lang: eng
text: 'Understanding the properties of neural networks trained via stochastic gradient
descent (SGD) is at the heart of the theory of deep learning. In this work, we
take a mean-field view, and consider a two-layer ReLU network trained via noisy-SGD
for a univariate regularized regression problem. Our main result is that SGD with
vanishingly small noise injected in the gradients is biased towards a simple solution:
at convergence, the ReLU network implements a piecewise linear map of the inputs,
and the number of “knot” points -- i.e., points where the tangent of the ReLU
network estimator changes -- between two consecutive training inputs is at most
three. In particular, as the number of neurons of the network grows, the SGD dynamics
is captured by the solution of a gradient flow and, at convergence, the distribution
of the weights approaches the unique minimizer of a related free energy, which
has a Gibbs form. Our key technical contribution consists in the analysis of the
estimator resulting from this minimizer: we show that its second derivative vanishes
everywhere, except at some specific locations which represent the “knot” points.
We also provide empirical evidence that knots at locations distinct from the data
points might occur, as predicted by our theory.'
acknowledgement: "We would like to thank Mert Pilanci for several exploratory discussions
in the early stage\r\nof the project, Jan Maas for clarifications about Jordan et
al. (1998), and Max Zimmer for\r\nsuggestive numerical experiments. A. Shevchenko
and M. Mondelli are partially supported\r\nby the 2019 Lopez-Loreta Prize. V. Kungurtsev
acknowledges support to the OP VVV\r\nproject CZ.02.1.01/0.0/0.0/16 019/0000765
Research Center for Informatics.\r\n"
article_processing_charge: No
article_type: original
arxiv: 1
author:
- first_name: Aleksandr
full_name: Shevchenko, Aleksandr
id: F2B06EC2-C99E-11E9-89F0-752EE6697425
last_name: Shevchenko
- first_name: Vyacheslav
full_name: Kungurtsev, Vyacheslav
last_name: Kungurtsev
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
citation:
ama: Shevchenko A, Kungurtsev V, Mondelli M. Mean-field analysis of piecewise linear
solutions for wide ReLU networks. Journal of Machine Learning Research.
2022;23(130):1-55.
apa: Shevchenko, A., Kungurtsev, V., & Mondelli, M. (2022). Mean-field analysis
of piecewise linear solutions for wide ReLU networks. Journal of Machine Learning
Research. Journal of Machine Learning Research.
chicago: Shevchenko, Aleksandr, Vyacheslav Kungurtsev, and Marco Mondelli. “Mean-Field
Analysis of Piecewise Linear Solutions for Wide ReLU Networks.” Journal of
Machine Learning Research. Journal of Machine Learning Research, 2022.
ieee: A. Shevchenko, V. Kungurtsev, and M. Mondelli, “Mean-field analysis of piecewise
linear solutions for wide ReLU networks,” Journal of Machine Learning Research,
vol. 23, no. 130. Journal of Machine Learning Research, pp. 1–55, 2022.
ista: Shevchenko A, Kungurtsev V, Mondelli M. 2022. Mean-field analysis of piecewise
linear solutions for wide ReLU networks. Journal of Machine Learning Research.
23(130), 1–55.
mla: Shevchenko, Aleksandr, et al. “Mean-Field Analysis of Piecewise Linear Solutions
for Wide ReLU Networks.” Journal of Machine Learning Research, vol. 23,
no. 130, Journal of Machine Learning Research, 2022, pp. 1–55.
short: A. Shevchenko, V. Kungurtsev, M. Mondelli, Journal of Machine Learning Research
23 (2022) 1–55.
date_created: 2022-05-29T22:01:54Z
date_published: 2022-04-01T00:00:00Z
date_updated: 2024-09-10T13:03:17Z
day: '01'
ddc:
- '000'
department:
- _id: MaMo
- _id: DaAl
external_id:
arxiv:
- '2111.02278'
file:
- access_level: open_access
checksum: d4ff5d1affb34848b5c5e4002483fc62
content_type: application/pdf
creator: cchlebak
date_created: 2022-05-30T08:22:55Z
date_updated: 2022-05-30T08:22:55Z
file_id: '11422'
file_name: 21-1365.pdf
file_size: 1521701
relation: main_file
success: 1
file_date_updated: 2022-05-30T08:22:55Z
has_accepted_license: '1'
intvolume: ' 23'
issue: '130'
language:
- iso: eng
month: '04'
oa: 1
oa_version: Published Version
page: 1-55
project:
- _id: 059876FA-7A3F-11EA-A408-12923DDC885E
name: Prix Lopez-Loretta 2019 - Marco Mondelli
publication: Journal of Machine Learning Research
publication_identifier:
eissn:
- 1533-7928
issn:
- 1532-4435
publication_status: published
publisher: Journal of Machine Learning Research
quality_controlled: '1'
related_material:
link:
- relation: other
url: https://www.jmlr.org/papers/v23/21-1365.html
scopus_import: '1'
status: public
title: Mean-field analysis of piecewise linear solutions for wide ReLU networks
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: journal_article
user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9
volume: 23
year: '2022'
...
---
_id: '11639'
abstract:
- lang: eng
text: We study the list decodability of different ensembles of codes over the real
alphabet under the assumption of an omniscient adversary. It is a well-known result
that when the source and the adversary have power constraints P and N respectively,
the list decoding capacity is equal to 1/2logP/N. Random spherical codes achieve
constant list sizes, and the goal of the present paper is to obtain a better understanding
of the smallest achievable list size as a function of the gap to capacity. We
show a reduction from arbitrary codes to spherical codes, and derive a lower bound
on the list size of typical random spherical codes. We also give an upper bound
on the list size achievable using nested Construction-A lattices and infinite
Construction-A lattices. We then define and study a class of infinite constellations
that generalize Construction-A lattices and prove upper and lower bounds for the
same. Other goodness properties such as packing goodness and AWGN goodness of
infinite constellations are proved along the way. Finally, we consider random
lattices sampled from the Haar distribution and show that if a certain conjecture
that originates in analytic number theory is true, then the list size grows as
a polynomial function of the gap-to-capacity.
acknowledgement: "This work was done when Shashank Vatedka was at the Chinese University
of Hong Kong, where he was supported in part by CUHK Direct Grants 4055039 and 4055077.
He would like to acknowledge funding from a seed grant offered by IIT Hyderabad
and the Start-up Research Grant (SRG/2020/000910) from the Science and Engineering
Board, India. Yihan Zhang has received funding from the European Union’s Horizon
2020 research and innovation programme\r\nunder grant agreement No 682203-ERC-[Inf-Speed-Tradeoff]."
article_processing_charge: No
article_type: original
arxiv: 1
author:
- first_name: Yihan
full_name: Zhang, Yihan
id: 2ce5da42-b2ea-11eb-bba5-9f264e9d002c
last_name: Zhang
- first_name: Shashank
full_name: Vatedka, Shashank
last_name: Vatedka
citation:
ama: Zhang Y, Vatedka S. List decoding random Euclidean codes and Infinite constellations.
IEEE Transactions on Information Theory. 2022;68(12):7753-7786. doi:10.1109/TIT.2022.3189542
apa: Zhang, Y., & Vatedka, S. (2022). List decoding random Euclidean codes and
Infinite constellations. IEEE Transactions on Information Theory. IEEE.
https://doi.org/10.1109/TIT.2022.3189542
chicago: Zhang, Yihan, and Shashank Vatedka. “List Decoding Random Euclidean Codes
and Infinite Constellations.” IEEE Transactions on Information Theory.
IEEE, 2022. https://doi.org/10.1109/TIT.2022.3189542.
ieee: Y. Zhang and S. Vatedka, “List decoding random Euclidean codes and Infinite
constellations,” IEEE Transactions on Information Theory, vol. 68, no.
12. IEEE, pp. 7753–7786, 2022.
ista: Zhang Y, Vatedka S. 2022. List decoding random Euclidean codes and Infinite
constellations. IEEE Transactions on Information Theory. 68(12), 7753–7786.
mla: Zhang, Yihan, and Shashank Vatedka. “List Decoding Random Euclidean Codes and
Infinite Constellations.” IEEE Transactions on Information Theory, vol.
68, no. 12, IEEE, 2022, pp. 7753–86, doi:10.1109/TIT.2022.3189542.
short: Y. Zhang, S. Vatedka, IEEE Transactions on Information Theory 68 (2022) 7753–7786.
date_created: 2022-07-24T22:01:42Z
date_published: 2022-12-01T00:00:00Z
date_updated: 2023-08-03T12:12:19Z
day: '01'
department:
- _id: MaMo
doi: 10.1109/TIT.2022.3189542
external_id:
arxiv:
- '1901.03790'
isi:
- '000891796100007'
intvolume: ' 68'
isi: 1
issue: '12'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.1901.03790
month: '12'
oa: 1
oa_version: Preprint
page: 7753-7786
publication: IEEE Transactions on Information Theory
publication_identifier:
eissn:
- 1557-9654
issn:
- 0018-9448
publication_status: published
publisher: IEEE
quality_controlled: '1'
scopus_import: '1'
status: public
title: List decoding random Euclidean codes and Infinite constellations
type: journal_article
user_id: 4359f0d1-fa6c-11eb-b949-802e58b17ae8
volume: 68
year: '2022'
...
---
_id: '12011'
abstract:
- lang: eng
text: We characterize the capacity for the discrete-time arbitrarily varying channel
with discrete inputs, outputs, and states when (a) the encoder and decoder do
not share common randomness, (b) the input and state are subject to cost constraints,
(c) the transition matrix of the channel is deterministic given the state, and
(d) at each time step the adversary can only observe the current and past channel
inputs when choosing the state at that time. The achievable strategy involves
stochastic encoding together with list decoding and a disambiguation step. The
converse uses a two-phase "babble-and-push" strategy where the adversary chooses
the state randomly in the first phase, list decodes the output, and then chooses
state inputs to symmetrize the channel in the second phase. These results generalize
prior work on specific channels models (additive, erasure) to general discrete
alphabets and models.
acknowledgement: The work of ADS and ML was supported in part by the US National Science
Foundation under awards CCF-1909468 and CCF-1909451.
article_processing_charge: No
arxiv: 1
author:
- first_name: Yihan
full_name: Zhang, Yihan
id: 2ce5da42-b2ea-11eb-bba5-9f264e9d002c
last_name: Zhang
- first_name: Sidharth
full_name: Jaggi, Sidharth
last_name: Jaggi
- first_name: Michael
full_name: Langberg, Michael
last_name: Langberg
- first_name: Anand D.
full_name: Sarwate, Anand D.
last_name: Sarwate
citation:
ama: 'Zhang Y, Jaggi S, Langberg M, Sarwate AD. The capacity of causal adversarial
channels. In: 2022 IEEE International Symposium on Information Theory.
Vol 2022. IEEE; 2022:2523-2528. doi:10.1109/ISIT50566.2022.9834709'
apa: 'Zhang, Y., Jaggi, S., Langberg, M., & Sarwate, A. D. (2022). The capacity
of causal adversarial channels. In 2022 IEEE International Symposium on Information
Theory (Vol. 2022, pp. 2523–2528). Espoo, Finland: IEEE. https://doi.org/10.1109/ISIT50566.2022.9834709'
chicago: Zhang, Yihan, Sidharth Jaggi, Michael Langberg, and Anand D. Sarwate. “The
Capacity of Causal Adversarial Channels.” In 2022 IEEE International Symposium
on Information Theory, 2022:2523–28. IEEE, 2022. https://doi.org/10.1109/ISIT50566.2022.9834709.
ieee: Y. Zhang, S. Jaggi, M. Langberg, and A. D. Sarwate, “The capacity of causal
adversarial channels,” in 2022 IEEE International Symposium on Information
Theory, Espoo, Finland, 2022, vol. 2022, pp. 2523–2528.
ista: 'Zhang Y, Jaggi S, Langberg M, Sarwate AD. 2022. The capacity of causal adversarial
channels. 2022 IEEE International Symposium on Information Theory. ISIT: Internation
Symposium on Information Theory vol. 2022, 2523–2528.'
mla: Zhang, Yihan, et al. “The Capacity of Causal Adversarial Channels.” 2022
IEEE International Symposium on Information Theory, vol. 2022, IEEE, 2022,
pp. 2523–28, doi:10.1109/ISIT50566.2022.9834709.
short: Y. Zhang, S. Jaggi, M. Langberg, A.D. Sarwate, in:, 2022 IEEE International
Symposium on Information Theory, IEEE, 2022, pp. 2523–2528.
conference:
end_date: 2022-07-01
location: Espoo, Finland
name: 'ISIT: Internation Symposium on Information Theory'
start_date: 2022-06-26
date_created: 2022-09-04T22:02:03Z
date_published: 2022-08-03T00:00:00Z
date_updated: 2022-09-05T09:09:15Z
day: '03'
department:
- _id: MaMo
doi: 10.1109/ISIT50566.2022.9834709
external_id:
arxiv:
- '2205.06708'
intvolume: ' 2022'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: ' https://doi.org/10.48550/arXiv.2205.06708'
month: '08'
oa: 1
oa_version: Preprint
page: 2523-2528
publication: 2022 IEEE International Symposium on Information Theory
publication_identifier:
isbn:
- '9781665421591'
issn:
- 2157-8095
publication_status: published
publisher: IEEE
quality_controlled: '1'
scopus_import: '1'
status: public
title: The capacity of causal adversarial channels
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 2022
year: '2022'
...
---
_id: '12012'
abstract:
- lang: eng
text: This paper is eligible for the Jack Keil Wolf ISIT Student Paper Award. We
generalize a previous framework for designing utility-optimal differentially private
(DP) mechanisms via graphs, where datasets are vertices in the graph and edges
represent dataset neighborhood. The boundary set contains datasets where an individual’s
response changes the binary-valued query compared to its neighbors. Previous work
was limited to the homogeneous case where the privacy parameter ε across all datasets
was the same and the mechanism at boundary datasets was identical. In our work,
the mechanism can take different distributions at the boundary and the privacy
parameter ε is a function of neighboring datasets, which recovers an earlier definition
of personalized DP as special case. The problem is how to extend the mechanism,
which is only defined at the boundary set, to other datasets in the graph in a
computationally efficient and utility optimal manner. Using the concept of strongest
induced DP condition we solve this problem efficiently in polynomial time (in
the size of the graph).
article_processing_charge: No
arxiv: 1
author:
- first_name: Sahel
full_name: Torkamani, Sahel
id: 0503e7f8-2d05-11ed-aa17-db0640c720fc
last_name: Torkamani
- first_name: Javad B.
full_name: Ebrahimi, Javad B.
last_name: Ebrahimi
- first_name: Parastoo
full_name: Sadeghi, Parastoo
last_name: Sadeghi
- first_name: Rafael G.L.
full_name: D'Oliveira, Rafael G.L.
last_name: D'Oliveira
- first_name: Muriel
full_name: Médard, Muriel
last_name: Médard
citation:
ama: 'Torkamani S, Ebrahimi JB, Sadeghi P, D’Oliveira RGL, Médard M. Heterogeneous
differential privacy via graphs. In: 2022 IEEE International Symposium on Information
Theory. Vol 2022. IEEE; 2022:1623-1628. doi:10.1109/ISIT50566.2022.9834711'
apa: 'Torkamani, S., Ebrahimi, J. B., Sadeghi, P., D’Oliveira, R. G. L., & Médard,
M. (2022). Heterogeneous differential privacy via graphs. In 2022 IEEE International
Symposium on Information Theory (Vol. 2022, pp. 1623–1628). Espoo, Finland:
IEEE. https://doi.org/10.1109/ISIT50566.2022.9834711'
chicago: Torkamani, Sahel, Javad B. Ebrahimi, Parastoo Sadeghi, Rafael G.L. D’Oliveira,
and Muriel Médard. “Heterogeneous Differential Privacy via Graphs.” In 2022
IEEE International Symposium on Information Theory, 2022:1623–28. IEEE, 2022.
https://doi.org/10.1109/ISIT50566.2022.9834711.
ieee: S. Torkamani, J. B. Ebrahimi, P. Sadeghi, R. G. L. D’Oliveira, and M. Médard,
“Heterogeneous differential privacy via graphs,” in 2022 IEEE International
Symposium on Information Theory, Espoo, Finland, 2022, vol. 2022, pp. 1623–1628.
ista: 'Torkamani S, Ebrahimi JB, Sadeghi P, D’Oliveira RGL, Médard M. 2022. Heterogeneous
differential privacy via graphs. 2022 IEEE International Symposium on Information
Theory. ISIT: Internation Symposium on Information Theory vol. 2022, 1623–1628.'
mla: Torkamani, Sahel, et al. “Heterogeneous Differential Privacy via Graphs.” 2022
IEEE International Symposium on Information Theory, vol. 2022, IEEE, 2022,
pp. 1623–28, doi:10.1109/ISIT50566.2022.9834711.
short: S. Torkamani, J.B. Ebrahimi, P. Sadeghi, R.G.L. D’Oliveira, M. Médard, in:,
2022 IEEE International Symposium on Information Theory, IEEE, 2022, pp. 1623–1628.
conference:
end_date: 2022-07-01
location: Espoo, Finland
name: 'ISIT: Internation Symposium on Information Theory'
start_date: 2022-06-26
date_created: 2022-09-04T22:02:04Z
date_published: 2022-08-03T00:00:00Z
date_updated: 2022-09-05T10:28:35Z
day: '03'
department:
- _id: MaMo
doi: 10.1109/ISIT50566.2022.9834711
external_id:
arxiv:
- '2203.15429'
intvolume: ' 2022'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2203.15429
month: '08'
oa: 1
oa_version: Preprint
page: 1623-1628
publication: 2022 IEEE International Symposium on Information Theory
publication_identifier:
isbn:
- '9781665421591'
issn:
- 2157-8095
publication_status: published
publisher: IEEE
quality_controlled: '1'
scopus_import: '1'
status: public
title: Heterogeneous differential privacy via graphs
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 2022
year: '2022'
...
---
_id: '12013'
abstract:
- lang: eng
text: We consider the problem of communication over adversarial channels with feedback.
Two parties comprising sender Alice and receiver Bob seek to communicate reliably.
An adversary James observes Alice's channel transmission entirely and chooses,
maliciously, its additive channel input or jamming state thereby corrupting Bob's
observation. Bob can communicate over a one-way reverse link with Alice; we assume
that transmissions over this feedback link cannot be corrupted by James. Our goal
in this work is to study the optimum throughput or capacity over such channels
with feedback. We first present results for the quadratically-constrained additive
channel where communication is known to be impossible when the noise-to-signal
(power) ratio (NSR) is at least 1. We present a novel achievability scheme to
establish that positive rate communication is possible even when the NSR is as
high as 8/9. We also present new converse upper bounds on the capacity of this
channel under potentially stochastic encoders and decoders. We also study feedback
communication over the more widely studied q-ary alphabet channel under additive
noise. For the q -ary channel, where q > 2, it is well known that capacity is
positive under full feedback if and only if the adversary can corrupt strictly
less than half the transmitted symbols. We generalize this result and show that
the same threshold holds for positive rate communication when the noiseless feedback
may only be partial; our scheme employs a stochastic decoder. We extend this characterization,
albeit partially, to fully deterministic schemes under partial noiseless feedback.
We also present new converse upper bounds for q-ary channels under full feedback,
where the encoder and/or decoder may privately randomize. Our converse results
bring to the fore an interesting alternate expression for the well known converse
bound for the q—ary channel under full feedback which, when specialized to the
binary channel, also equals its known capacity.
article_processing_charge: No
author:
- first_name: Pranav
full_name: Joshi, Pranav
last_name: Joshi
- first_name: Amritakshya
full_name: Purkayastha, Amritakshya
last_name: Purkayastha
- first_name: Yihan
full_name: Zhang, Yihan
id: 2ce5da42-b2ea-11eb-bba5-9f264e9d002c
last_name: Zhang
- first_name: Amitalok J.
full_name: Budkuley, Amitalok J.
last_name: Budkuley
- first_name: Sidharth
full_name: Jaggi, Sidharth
last_name: Jaggi
citation:
ama: 'Joshi P, Purkayastha A, Zhang Y, Budkuley AJ, Jaggi S. On the capacity of
additive AVCs with feedback. In: 2022 IEEE International Symposium on Information
Theory. Vol 2022. IEEE; 2022:504-509. doi:10.1109/ISIT50566.2022.9834850'
apa: 'Joshi, P., Purkayastha, A., Zhang, Y., Budkuley, A. J., & Jaggi, S. (2022).
On the capacity of additive AVCs with feedback. In 2022 IEEE International
Symposium on Information Theory (Vol. 2022, pp. 504–509). Espoo, Finland:
IEEE. https://doi.org/10.1109/ISIT50566.2022.9834850'
chicago: Joshi, Pranav, Amritakshya Purkayastha, Yihan Zhang, Amitalok J. Budkuley,
and Sidharth Jaggi. “On the Capacity of Additive AVCs with Feedback.” In 2022
IEEE International Symposium on Information Theory, 2022:504–9. IEEE, 2022.
https://doi.org/10.1109/ISIT50566.2022.9834850.
ieee: P. Joshi, A. Purkayastha, Y. Zhang, A. J. Budkuley, and S. Jaggi, “On the
capacity of additive AVCs with feedback,” in 2022 IEEE International Symposium
on Information Theory, Espoo, Finland, 2022, vol. 2022, pp. 504–509.
ista: 'Joshi P, Purkayastha A, Zhang Y, Budkuley AJ, Jaggi S. 2022. On the capacity
of additive AVCs with feedback. 2022 IEEE International Symposium on Information
Theory. ISIT: Internation Symposium on Information Theory vol. 2022, 504–509.'
mla: Joshi, Pranav, et al. “On the Capacity of Additive AVCs with Feedback.” 2022
IEEE International Symposium on Information Theory, vol. 2022, IEEE, 2022,
pp. 504–09, doi:10.1109/ISIT50566.2022.9834850.
short: P. Joshi, A. Purkayastha, Y. Zhang, A.J. Budkuley, S. Jaggi, in:, 2022 IEEE
International Symposium on Information Theory, IEEE, 2022, pp. 504–509.
conference:
end_date: 2022-07-01
location: Espoo, Finland
name: 'ISIT: Internation Symposium on Information Theory'
start_date: 2022-06-26
date_created: 2022-09-04T22:02:04Z
date_published: 2022-08-03T00:00:00Z
date_updated: 2022-09-05T10:23:35Z
day: '03'
department:
- _id: MaMo
doi: 10.1109/ISIT50566.2022.9834850
intvolume: ' 2022'
language:
- iso: eng
month: '08'
oa_version: None
page: 504-509
publication: 2022 IEEE International Symposium on Information Theory
publication_identifier:
isbn:
- '9781665421591'
issn:
- 2157-8095
publication_status: published
publisher: IEEE
quality_controlled: '1'
scopus_import: '1'
status: public
title: On the capacity of additive AVCs with feedback
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 2022
year: '2022'
...
---
_id: '12014'
abstract:
- lang: eng
text: We study the problem of high-dimensional multiple packing in Euclidean space.
Multiple packing is a natural generalization of sphere packing and is defined
as follows. Let N > 0 and L∈Z≥2. A multiple packing is a set C of points in Rn
such that any point in Rn lies in the intersection of at most L – 1 balls of radius
nN−−−√ around points in C. Given a well-known connection with coding theory, multiple
packings can be viewed as the Euclidean analog of list-decodable codes, which
are well-studied for finite fields. In this paper, we exactly pin down the asymptotic
density of (expurgated) Poisson Point Processes under a stronger notion called
average-radius multiple packing. To this end, we apply tools from high-dimensional
geometry and large deviation theory. This gives rise to the best known lower bound
on the largest multiple packing density. Our result corrects a mistake in a previous
paper by Blinovsky [Bli05].
article_processing_charge: No
author:
- first_name: Yihan
full_name: Zhang, Yihan
id: 2ce5da42-b2ea-11eb-bba5-9f264e9d002c
last_name: Zhang
- first_name: Shashank
full_name: Vatedka, Shashank
last_name: Vatedka
citation:
ama: 'Zhang Y, Vatedka S. List-decodability of Poisson Point Processes. In: 2022
IEEE International Symposium on Information Theory. Vol 2022. IEEE; 2022:2559-2564.
doi:10.1109/ISIT50566.2022.9834512'
apa: 'Zhang, Y., & Vatedka, S. (2022). List-decodability of Poisson Point Processes.
In 2022 IEEE International Symposium on Information Theory (Vol. 2022,
pp. 2559–2564). Espoo, Finland: IEEE. https://doi.org/10.1109/ISIT50566.2022.9834512'
chicago: Zhang, Yihan, and Shashank Vatedka. “List-Decodability of Poisson Point
Processes.” In 2022 IEEE International Symposium on Information Theory,
2022:2559–64. IEEE, 2022. https://doi.org/10.1109/ISIT50566.2022.9834512.
ieee: Y. Zhang and S. Vatedka, “List-decodability of Poisson Point Processes,” in
2022 IEEE International Symposium on Information Theory, Espoo, Finland,
2022, vol. 2022, pp. 2559–2564.
ista: 'Zhang Y, Vatedka S. 2022. List-decodability of Poisson Point Processes. 2022
IEEE International Symposium on Information Theory. ISIT: Internation Symposium
on Information Theory vol. 2022, 2559–2564.'
mla: Zhang, Yihan, and Shashank Vatedka. “List-Decodability of Poisson Point Processes.”
2022 IEEE International Symposium on Information Theory, vol. 2022, IEEE,
2022, pp. 2559–64, doi:10.1109/ISIT50566.2022.9834512.
short: Y. Zhang, S. Vatedka, in:, 2022 IEEE International Symposium on Information
Theory, IEEE, 2022, pp. 2559–2564.
conference:
end_date: 2022-07-01
location: Espoo, Finland
name: 'ISIT: Internation Symposium on Information Theory'
start_date: 2022-06-26
date_created: 2022-09-04T22:02:04Z
date_published: 2022-08-03T00:00:00Z
date_updated: 2022-09-05T09:23:04Z
day: '03'
department:
- _id: MaMo
doi: 10.1109/ISIT50566.2022.9834512
intvolume: ' 2022'
language:
- iso: eng
month: '08'
oa_version: None
page: 2559-2564
publication: 2022 IEEE International Symposium on Information Theory
publication_identifier:
isbn:
- '9781665421591'
issn:
- 2157-8095
publication_status: published
publisher: IEEE
quality_controlled: '1'
scopus_import: '1'
status: public
title: List-decodability of Poisson Point Processes
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 2022
year: '2022'
...
---
_id: '12015'
abstract:
- lang: eng
text: We study the problem of high-dimensional multiple packing in Euclidean space.
Multiple packing is a natural generalization of sphere packing and is defined
as follows. Let P, N > 0 and L∈Z≥2. A multiple packing is a set C of points in
Bn(0–,nP−−−√) such that any point in ℝ n lies in the intersection of at most L
– 1 balls of radius nN−−−√ around points in C. 1 In this paper, we derive two
lower bounds on the largest possible density of a multiple packing. These bounds
are obtained through a stronger notion called average-radius multiple packing.
Specifically, we exactly pin down the asymptotics of (expurgated) Gaussian codes
and (expurgated) spherical codes under average-radius multiple packing. To this
end, we apply tools from high-dimensional geometry and large deviation theory.
The bound for spherical codes matches the previous best known bound which was
obtained for the standard (weaker) notion of multiple packing through a curious
connection with error exponents [Bli99], [ZV21]. The bound for Gaussian codes
suggests that they are strictly inferior to spherical codes.
article_processing_charge: No
author:
- first_name: Yihan
full_name: Zhang, Yihan
id: 2ce5da42-b2ea-11eb-bba5-9f264e9d002c
last_name: Zhang
- first_name: Shashank
full_name: Vatedka, Shashank
last_name: Vatedka
citation:
ama: 'Zhang Y, Vatedka S. Lower bounds for multiple packing. In: 2022 IEEE International
Symposium on Information Theory. Vol 2022. IEEE; 2022:3085-3090. doi:10.1109/ISIT50566.2022.9834443'
apa: 'Zhang, Y., & Vatedka, S. (2022). Lower bounds for multiple packing. In
2022 IEEE International Symposium on Information Theory (Vol. 2022, pp.
3085–3090). Espoo, Finland: IEEE. https://doi.org/10.1109/ISIT50566.2022.9834443'
chicago: Zhang, Yihan, and Shashank Vatedka. “Lower Bounds for Multiple Packing.”
In 2022 IEEE International Symposium on Information Theory, 2022:3085–90.
IEEE, 2022. https://doi.org/10.1109/ISIT50566.2022.9834443.
ieee: Y. Zhang and S. Vatedka, “Lower bounds for multiple packing,” in 2022 IEEE
International Symposium on Information Theory, Espoo, Finland, 2022, vol.
2022, pp. 3085–3090.
ista: 'Zhang Y, Vatedka S. 2022. Lower bounds for multiple packing. 2022 IEEE International
Symposium on Information Theory. ISIT: Internation Symposium on Information Theory
vol. 2022, 3085–3090.'
mla: Zhang, Yihan, and Shashank Vatedka. “Lower Bounds for Multiple Packing.” 2022
IEEE International Symposium on Information Theory, vol. 2022, IEEE, 2022,
pp. 3085–90, doi:10.1109/ISIT50566.2022.9834443.
short: Y. Zhang, S. Vatedka, in:, 2022 IEEE International Symposium on Information
Theory, IEEE, 2022, pp. 3085–3090.
conference:
end_date: 2022-07-01
location: Espoo, Finland
name: 'ISIT: Internation Symposium on Information Theory'
start_date: 2022-06-26
date_created: 2022-09-04T22:02:05Z
date_published: 2022-08-03T00:00:00Z
date_updated: 2022-09-05T10:39:04Z
day: '03'
department:
- _id: MaMo
doi: 10.1109/ISIT50566.2022.9834443
intvolume: ' 2022'
language:
- iso: eng
month: '08'
oa_version: None
page: 3085-3090
publication: 2022 IEEE International Symposium on Information Theory
publication_identifier:
isbn:
- '9781665421591'
issn:
- 2157-8095
publication_status: published
publisher: IEEE
quality_controlled: '1'
scopus_import: '1'
status: public
title: Lower bounds for multiple packing
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 2022
year: '2022'
...
---
_id: '12016'
abstract:
- lang: eng
text: We consider the problem of coded distributed computing using polar codes.
The average execution time of a coded computing system is related to the error
probability for transmission over the binary erasure channel in recent work by
Soleymani, Jamali and Mahdavifar, where the performance of binary linear codes
is investigated. In this paper, we focus on polar codes and unveil a connection
between the average execution time and the scaling exponent μ of the family of
codes. In the finite-length characterization of polar codes, the scaling exponent
is a key object capturing the speed of convergence to capacity. In particular,
we show that (i) the gap between the normalized average execution time of polar
codes and that of optimal MDS codes is O(n –1/μ ), and (ii) this upper bound can
be improved to roughly O(n –1/2 ) by considering polar codes with large kernels.
We conjecture that these bounds could be improved to O(n –2/μ ) and O(n –1 ),
respectively, and provide a heuristic argument as well as numerical evidence supporting
this view.
acknowledgement: D. Fathollahi and M. Mondelli were partially supported by the 2019
Lopez-Loreta Prize. The authors thank Hamed Hassani and Hessam Mahdavifar for helpful
discussions.
article_processing_charge: No
arxiv: 1
author:
- first_name: Dorsa
full_name: Fathollahi, Dorsa
last_name: Fathollahi
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
citation:
ama: 'Fathollahi D, Mondelli M. Polar coded computing: The role of the scaling exponent.
In: 2022 IEEE International Symposium on Information Theory. Vol 2022.
IEEE; 2022:2154-2159. doi:10.1109/ISIT50566.2022.9834712'
apa: 'Fathollahi, D., & Mondelli, M. (2022). Polar coded computing: The role
of the scaling exponent. In 2022 IEEE International Symposium on Information
Theory (Vol. 2022, pp. 2154–2159). Espoo, Finland: IEEE. https://doi.org/10.1109/ISIT50566.2022.9834712'
chicago: 'Fathollahi, Dorsa, and Marco Mondelli. “Polar Coded Computing: The Role
of the Scaling Exponent.” In 2022 IEEE International Symposium on Information
Theory, 2022:2154–59. IEEE, 2022. https://doi.org/10.1109/ISIT50566.2022.9834712.'
ieee: 'D. Fathollahi and M. Mondelli, “Polar coded computing: The role of the scaling
exponent,” in 2022 IEEE International Symposium on Information Theory,
Espoo, Finland, 2022, vol. 2022, pp. 2154–2159.'
ista: 'Fathollahi D, Mondelli M. 2022. Polar coded computing: The role of the scaling
exponent. 2022 IEEE International Symposium on Information Theory. ISIT: Internation
Symposium on Information Theory vol. 2022, 2154–2159.'
mla: 'Fathollahi, Dorsa, and Marco Mondelli. “Polar Coded Computing: The Role of
the Scaling Exponent.” 2022 IEEE International Symposium on Information Theory,
vol. 2022, IEEE, 2022, pp. 2154–59, doi:10.1109/ISIT50566.2022.9834712.'
short: D. Fathollahi, M. Mondelli, in:, 2022 IEEE International Symposium on Information
Theory, IEEE, 2022, pp. 2154–2159.
conference:
end_date: 2022-07-01
location: Espoo, Finland
name: 'ISIT: Internation Symposium on Information Theory'
start_date: 2022-06-26
date_created: 2022-09-04T22:02:05Z
date_published: 2022-08-03T00:00:00Z
date_updated: 2024-09-10T13:03:17Z
day: '03'
department:
- _id: MaMo
doi: 10.1109/ISIT50566.2022.9834712
external_id:
arxiv:
- '2201.10082'
intvolume: ' 2022'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://doi.org/10.48550/arXiv.2201.10082
month: '08'
oa: 1
oa_version: Preprint
page: 2154-2159
project:
- _id: 059876FA-7A3F-11EA-A408-12923DDC885E
name: Prix Lopez-Loretta 2019 - Marco Mondelli
publication: 2022 IEEE International Symposium on Information Theory
publication_identifier:
isbn:
- '9781665421591'
issn:
- 2157-8095
publication_status: published
publisher: IEEE
quality_controlled: '1'
scopus_import: '1'
status: public
title: 'Polar coded computing: The role of the scaling exponent'
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 2022
year: '2022'
...
---
_id: '12017'
abstract:
- lang: eng
text: 'In the classic adversarial communication problem, two parties communicate
over a noisy channel in the presence of a malicious jamming adversary. The arbitrarily
varying channels (AVCs) offer an elegant framework to study a wide range of interesting
adversary models. The optimal throughput or capacity over such AVCs is intimately
tied to the underlying adversary model; in some cases, capacity is unknown and
the problem is known to be notoriously hard. The omniscient adversary, one which
knows the sender’s entire channel transmission a priori, is one of such classic
models of interest; the capacity under such an adversary remains an exciting open
problem. The myopic adversary is a generalization of that model where the adversary’s
observation may be corrupted over a noisy discrete memoryless channel. Through
the adversary’s myopicity, one can unify the slew of different adversary models,
ranging from the omniscient adversary to one that is completely blind to the transmission
(the latter is the well known oblivious model where the capacity is fully characterized).In
this work, we present new results on the capacity under both the omniscient and
myopic adversary models. We completely characterize the positive capacity threshold
over general AVCs with omniscient adversaries. The characterization is in terms
of two key combinatorial objects: the set of completely positive distributions
and the CP-confusability set. For omniscient AVCs with positive capacity, we present
non-trivial lower and upper bounds on the capacity; unlike some of the previous
bounds, our bounds hold under fairly general input and jamming constraints. Our
lower bound improves upon the generalized Gilbert-Varshamov bound for general
AVCs while the upper bound generalizes the well known Elias-Bassalygo bound (known
for binary and q-ary alphabets). For the myopic AVCs, we build on prior results
known for the so-called sufficiently myopic model, and present new results on
the positive rate communication threshold over the so-called insufficiently myopic
regime (a completely insufficient myopic adversary specializes to an omniscient
adversary). We present interesting examples for the widely studied models of adversarial
bit-flip and bit-erasure channels. In fact, for the bit-flip AVC with additive
adversarial noise as well as random noise, we completely characterize the omniscient
model capacity when the random noise is sufficiently large vis-a-vis the adversary’s
budget.'
article_processing_charge: No
author:
- first_name: Anuj Kumar
full_name: Yadav, Anuj Kumar
last_name: Yadav
- first_name: Mohammadreza
full_name: Alimohammadi, Mohammadreza
last_name: Alimohammadi
- first_name: Yihan
full_name: Zhang, Yihan
id: 2ce5da42-b2ea-11eb-bba5-9f264e9d002c
last_name: Zhang
- first_name: Amitalok J.
full_name: Budkuley, Amitalok J.
last_name: Budkuley
- first_name: Sidharth
full_name: Jaggi, Sidharth
last_name: Jaggi
citation:
ama: 'Yadav AK, Alimohammadi M, Zhang Y, Budkuley AJ, Jaggi S. New results on AVCs
with omniscient and myopic adversaries. In: 2022 IEEE International Symposium
on Information Theory. Vol 2022. Institute of Electrical and Electronics Engineers;
2022:2535-2540. doi:10.1109/ISIT50566.2022.9834632'
apa: 'Yadav, A. K., Alimohammadi, M., Zhang, Y., Budkuley, A. J., & Jaggi, S.
(2022). New results on AVCs with omniscient and myopic adversaries. In 2022
IEEE International Symposium on Information Theory (Vol. 2022, pp. 2535–2540).
Espoo, Finland: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ISIT50566.2022.9834632'
chicago: Yadav, Anuj Kumar, Mohammadreza Alimohammadi, Yihan Zhang, Amitalok J.
Budkuley, and Sidharth Jaggi. “New Results on AVCs with Omniscient and Myopic
Adversaries.” In 2022 IEEE International Symposium on Information Theory,
2022:2535–40. Institute of Electrical and Electronics Engineers, 2022. https://doi.org/10.1109/ISIT50566.2022.9834632.
ieee: A. K. Yadav, M. Alimohammadi, Y. Zhang, A. J. Budkuley, and S. Jaggi, “New
results on AVCs with omniscient and myopic adversaries,” in 2022 IEEE International
Symposium on Information Theory, Espoo, Finland, 2022, vol. 2022, pp. 2535–2540.
ista: 'Yadav AK, Alimohammadi M, Zhang Y, Budkuley AJ, Jaggi S. 2022. New results
on AVCs with omniscient and myopic adversaries. 2022 IEEE International Symposium
on Information Theory. ISIT: Internation Symposium on Information Theory vol.
2022, 2535–2540.'
mla: Yadav, Anuj Kumar, et al. “New Results on AVCs with Omniscient and Myopic Adversaries.”
2022 IEEE International Symposium on Information Theory, vol. 2022, Institute
of Electrical and Electronics Engineers, 2022, pp. 2535–40, doi:10.1109/ISIT50566.2022.9834632.
short: A.K. Yadav, M. Alimohammadi, Y. Zhang, A.J. Budkuley, S. Jaggi, in:, 2022
IEEE International Symposium on Information Theory, Institute of Electrical and
Electronics Engineers, 2022, pp. 2535–2540.
conference:
end_date: 2022-07-01
location: Espoo, Finland
name: 'ISIT: Internation Symposium on Information Theory'
start_date: 2022-06-26
date_created: 2022-09-04T22:02:06Z
date_published: 2022-08-03T00:00:00Z
date_updated: 2023-02-13T09:00:14Z
day: '03'
department:
- _id: MaMo
doi: 10.1109/ISIT50566.2022.9834632
intvolume: ' 2022'
language:
- iso: eng
month: '08'
oa_version: None
page: 2535-2540
publication: 2022 IEEE International Symposium on Information Theory
publication_identifier:
isbn:
- '9781665421591'
issn:
- 2157-8095
publication_status: published
publisher: Institute of Electrical and Electronics Engineers
quality_controlled: '1'
scopus_import: '1'
status: public
title: New results on AVCs with omniscient and myopic adversaries
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 2022
year: '2022'
...
---
_id: '12018'
abstract:
- lang: eng
text: We study the problem of characterizing the maximal rates of list decoding
in Euclidean spaces for finite list sizes. For any positive integer L ≥ 2 and
real N > 0, we say that a subset C⊂Rn is an (N,L – 1)-multiple packing or an (N,L–
1)-list decodable code if every Euclidean ball of radius nN−−−√ in ℝ n contains
no more than L − 1 points of C. We study this problem with and without ℓ 2 norm
constraints on C, and derive the best-known lower bounds on the maximal rate for
(N,L−1) multiple packing. Our bounds are obtained via error exponents for list
decoding over Additive White Gaussian Noise (AWGN) channels. We establish a curious
inequality which relates the error exponent, a quantity of average-case nature,
to the list-decoding radius, a quantity of worst-case nature. We derive various
bounds on the error exponent for list decoding in both bounded and unbounded settings
which could be of independent interest beyond multiple packing.
article_processing_charge: No
author:
- first_name: Yihan
full_name: Zhang, Yihan
id: 2ce5da42-b2ea-11eb-bba5-9f264e9d002c
last_name: Zhang
- first_name: Shashank
full_name: Vatedka, Shashank
last_name: Vatedka
citation:
ama: 'Zhang Y, Vatedka S. Lower bounds on list decoding capacity using error exponents.
In: 2022 IEEE International Symposium on Information Theory. Vol 2022.
Institute of Electrical and Electronics Engineers; 2022:1324-1329. doi:10.1109/ISIT50566.2022.9834815'
apa: 'Zhang, Y., & Vatedka, S. (2022). Lower bounds on list decoding capacity
using error exponents. In 2022 IEEE International Symposium on Information
Theory (Vol. 2022, pp. 1324–1329). Espoo, Finland: Institute of Electrical
and Electronics Engineers. https://doi.org/10.1109/ISIT50566.2022.9834815'
chicago: Zhang, Yihan, and Shashank Vatedka. “Lower Bounds on List Decoding Capacity
Using Error Exponents.” In 2022 IEEE International Symposium on Information
Theory, 2022:1324–29. Institute of Electrical and Electronics Engineers, 2022.
https://doi.org/10.1109/ISIT50566.2022.9834815.
ieee: Y. Zhang and S. Vatedka, “Lower bounds on list decoding capacity using error
exponents,” in 2022 IEEE International Symposium on Information Theory,
Espoo, Finland, 2022, vol. 2022, pp. 1324–1329.
ista: 'Zhang Y, Vatedka S. 2022. Lower bounds on list decoding capacity using error
exponents. 2022 IEEE International Symposium on Information Theory. ISIT: Internation
Symposium on Information Theory vol. 2022, 1324–1329.'
mla: Zhang, Yihan, and Shashank Vatedka. “Lower Bounds on List Decoding Capacity
Using Error Exponents.” 2022 IEEE International Symposium on Information Theory,
vol. 2022, Institute of Electrical and Electronics Engineers, 2022, pp. 1324–29,
doi:10.1109/ISIT50566.2022.9834815.
short: Y. Zhang, S. Vatedka, in:, 2022 IEEE International Symposium on Information
Theory, Institute of Electrical and Electronics Engineers, 2022, pp. 1324–1329.
conference:
end_date: 2022-07-01
location: Espoo, Finland
name: 'ISIT: Internation Symposium on Information Theory'
start_date: 2022-06-26
date_created: 2022-09-04T22:02:06Z
date_published: 2022-08-03T00:00:00Z
date_updated: 2023-02-13T09:02:06Z
day: '03'
department:
- _id: MaMo
doi: 10.1109/ISIT50566.2022.9834815
intvolume: ' 2022'
language:
- iso: eng
month: '08'
oa_version: None
page: 1324-1329
publication: 2022 IEEE International Symposium on Information Theory
publication_identifier:
isbn:
- '9781665421591'
issn:
- 2157-8095
publication_status: published
publisher: Institute of Electrical and Electronics Engineers
quality_controlled: '1'
scopus_import: '1'
status: public
title: Lower bounds on list decoding capacity using error exponents
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 2022
year: '2022'
...
---
_id: '12019'
abstract:
- lang: eng
text: This paper studies combinatorial properties of codes for the Z-channel. A
Z-channel with error fraction τ takes as input a length-n binary codeword and
injects in an adversarial manner up to nτ asymmetric errors, i.e., errors that
only zero out bits but do not flip 0’s to 1’s. It is known that the largest (L
− 1)-list-decodable code for the Z-channel with error fraction τ has exponential
(in n) size if τ is less than a critical value that we call the Plotkin point
and has constant size if τ is larger than the threshold. The (L−1)-list-decoding
Plotkin point is known to be L−1L−1−L−LL−1. In this paper, we show that the largest
(L−1)-list-decodable code ε-above the Plotkin point has size Θ L (ε −3/2 ) for
any L − 1 ≥ 1.
article_processing_charge: No
author:
- first_name: Nikita
full_name: Polyanskii, Nikita
last_name: Polyanskii
- first_name: Yihan
full_name: Zhang, Yihan
id: 2ce5da42-b2ea-11eb-bba5-9f264e9d002c
last_name: Zhang
citation:
ama: 'Polyanskii N, Zhang Y. List-decodable zero-rate codes for the Z-channel. In:
2022 IEEE International Symposium on Information Theory. Vol 2022. Institute
of Electrical and Electronics Engineers; 2022:2553-2558. doi:10.1109/ISIT50566.2022.9834829'
apa: 'Polyanskii, N., & Zhang, Y. (2022). List-decodable zero-rate codes for
the Z-channel. In 2022 IEEE International Symposium on Information Theory
(Vol. 2022, pp. 2553–2558). Espoo, Finland: Institute of Electrical and Electronics
Engineers. https://doi.org/10.1109/ISIT50566.2022.9834829'
chicago: Polyanskii, Nikita, and Yihan Zhang. “List-Decodable Zero-Rate Codes for
the Z-Channel.” In 2022 IEEE International Symposium on Information Theory,
2022:2553–58. Institute of Electrical and Electronics Engineers, 2022. https://doi.org/10.1109/ISIT50566.2022.9834829.
ieee: N. Polyanskii and Y. Zhang, “List-decodable zero-rate codes for the Z-channel,”
in 2022 IEEE International Symposium on Information Theory, Espoo, Finland,
2022, vol. 2022, pp. 2553–2558.
ista: 'Polyanskii N, Zhang Y. 2022. List-decodable zero-rate codes for the Z-channel.
2022 IEEE International Symposium on Information Theory. ISIT: Internation Symposium
on Information Theory vol. 2022, 2553–2558.'
mla: Polyanskii, Nikita, and Yihan Zhang. “List-Decodable Zero-Rate Codes for the
Z-Channel.” 2022 IEEE International Symposium on Information Theory, vol.
2022, Institute of Electrical and Electronics Engineers, 2022, pp. 2553–58, doi:10.1109/ISIT50566.2022.9834829.
short: N. Polyanskii, Y. Zhang, in:, 2022 IEEE International Symposium on Information
Theory, Institute of Electrical and Electronics Engineers, 2022, pp. 2553–2558.
conference:
end_date: 2022-07-01
location: Espoo, Finland
name: 'ISIT: Internation Symposium on Information Theory'
start_date: 2022-06-26
date_created: 2022-09-04T22:02:07Z
date_published: 2022-08-03T00:00:00Z
date_updated: 2023-02-13T09:02:18Z
day: '03'
department:
- _id: MaMo
doi: 10.1109/ISIT50566.2022.9834829
intvolume: ' 2022'
language:
- iso: eng
month: '08'
oa_version: None
page: 2553-2558
publication: 2022 IEEE International Symposium on Information Theory
publication_identifier:
isbn:
- '9781665421591'
issn:
- 2157-8095
publication_status: published
publisher: Institute of Electrical and Electronics Engineers
quality_controlled: '1'
scopus_import: '1'
status: public
title: List-decodable zero-rate codes for the Z-channel
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 2022
year: '2022'
...
---
_id: '13146'
abstract:
- lang: eng
text: 'A recent line of work has analyzed the theoretical properties of deep neural
networks via the Neural Tangent Kernel (NTK). In particular, the smallest eigenvalue
of the NTK has been related to the memorization capacity, the global convergence
of gradient descent algorithms and the generalization of deep nets. However, existing
results either provide bounds in the two-layer setting or assume that the spectrum
of the NTK matrices is bounded away from 0 for multi-layer networks. In this paper,
we provide tight bounds on the smallest eigenvalue of NTK matrices for deep ReLU
nets, both in the limiting case of infinite widths and for finite widths. In the
finite-width setting, the network architectures we consider are fairly general:
we require the existence of a wide layer with roughly order of N neurons, N being
the number of data samples; and the scaling of the remaining layer widths is arbitrary
(up to logarithmic factors). To obtain our results, we analyze various quantities
of independent interest: we give lower bounds on the smallest singular value of
hidden feature matrices, and upper bounds on the Lipschitz constant of input-output
feature maps.'
acknowledgement: The authors would like to thank the anonymous reviewers for their
helpful comments. MM was partially supported by the 2019 Lopez-Loreta Prize. QN
and GM acknowledge support from the European Research Council (ERC) under the European
Union’s Horizon 2020 research and innovation programme (grant agreement no 757983).
article_processing_charge: No
arxiv: 1
author:
- first_name: Quynh
full_name: Nguyen, Quynh
last_name: Nguyen
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
- first_name: Guido
full_name: Montufar, Guido
last_name: Montufar
citation:
ama: 'Nguyen Q, Mondelli M, Montufar G. Tight bounds on the smallest Eigenvalue
of the neural tangent kernel for deep ReLU networks. In: Proceedings of the
38th International Conference on Machine Learning. Vol 139. ML Research Press;
2021:8119-8129.'
apa: 'Nguyen, Q., Mondelli, M., & Montufar, G. (2021). Tight bounds on the smallest
Eigenvalue of the neural tangent kernel for deep ReLU networks. In Proceedings
of the 38th International Conference on Machine Learning (Vol. 139, pp. 8119–8129).
Virtual: ML Research Press.'
chicago: Nguyen, Quynh, Marco Mondelli, and Guido Montufar. “Tight Bounds on the
Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks.” In Proceedings
of the 38th International Conference on Machine Learning, 139:8119–29. ML
Research Press, 2021.
ieee: Q. Nguyen, M. Mondelli, and G. Montufar, “Tight bounds on the smallest Eigenvalue
of the neural tangent kernel for deep ReLU networks,” in Proceedings of the
38th International Conference on Machine Learning, Virtual, 2021, vol. 139,
pp. 8119–8129.
ista: Nguyen Q, Mondelli M, Montufar G. 2021. Tight bounds on the smallest Eigenvalue
of the neural tangent kernel for deep ReLU networks. Proceedings of the 38th International
Conference on Machine Learning. International Conference on Machine Learning vol.
139, 8119–8129.
mla: Nguyen, Quynh, et al. “Tight Bounds on the Smallest Eigenvalue of the Neural
Tangent Kernel for Deep ReLU Networks.” Proceedings of the 38th International
Conference on Machine Learning, vol. 139, ML Research Press, 2021, pp. 8119–29.
short: Q. Nguyen, M. Mondelli, G. Montufar, in:, Proceedings of the 38th International
Conference on Machine Learning, ML Research Press, 2021, pp. 8119–8129.
conference:
end_date: 2021-07-24
location: Virtual
name: International Conference on Machine Learning
start_date: 2021-07-18
date_created: 2023-06-18T22:00:48Z
date_published: 2021-07-01T00:00:00Z
date_updated: 2024-09-10T13:03:17Z
day: '01'
ddc:
- '000'
department:
- _id: MaMo
external_id:
arxiv:
- '2012.11654'
file:
- access_level: open_access
checksum: 19489cf5e16a0596b1f92e317d97c9b0
content_type: application/pdf
creator: dernst
date_created: 2023-06-19T10:49:12Z
date_updated: 2023-06-19T10:49:12Z
file_id: '13155'
file_name: 2021_PMLR_Nguyen.pdf
file_size: 591332
relation: main_file
success: 1
file_date_updated: 2023-06-19T10:49:12Z
has_accepted_license: '1'
intvolume: ' 139'
language:
- iso: eng
month: '07'
oa: 1
oa_version: Published Version
page: 8119-8129
project:
- _id: 059876FA-7A3F-11EA-A408-12923DDC885E
name: Prix Lopez-Loretta 2019 - Marco Mondelli
publication: Proceedings of the 38th International Conference on Machine Learning
publication_identifier:
eissn:
- 2640-3498
isbn:
- '9781713845065'
publication_status: published
publisher: ML Research Press
quality_controlled: '1'
scopus_import: '1'
status: public
title: Tight bounds on the smallest Eigenvalue of the neural tangent kernel for deep
ReLU networks
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 139
year: '2021'
...
---
_id: '9002'
abstract:
- lang: eng
text: ' We prove that, for the binary erasure channel (BEC), the polar-coding paradigm
gives rise to codes that not only approach the Shannon limit but do so under the
best possible scaling of their block length as a function of the gap to capacity.
This result exhibits the first known family of binary codes that attain both optimal
scaling and quasi-linear complexity of encoding and decoding. Our proof is based
on the construction and analysis of binary polar codes with large kernels. When
communicating reliably at rates within ε>0 of capacity, the code length n often
scales as O(1/εμ), where the constant μ is called the scaling exponent. It is
known that the optimal scaling exponent is μ=2, and it is achieved by random linear
codes. The scaling exponent of conventional polar codes (based on the 2×2 kernel)
on the BEC is μ=3.63. This falls far short of the optimal scaling guaranteed by
random codes. Our main contribution is a rigorous proof of the following result:
for the BEC, there exist ℓ×ℓ binary kernels, such that polar codes constructed
from these kernels achieve scaling exponent μ(ℓ) that tends to the optimal value
of 2 as ℓ grows. We furthermore characterize precisely how large ℓ needs to be
as a function of the gap between μ(ℓ) and 2. The resulting binary codes maintain
the recursive structure of conventional polar codes, and thereby achieve construction
complexity O(n) and encoding/decoding complexity O(nlogn).'
article_processing_charge: No
article_type: original
arxiv: 1
author:
- first_name: Arman
full_name: Fazeli, Arman
last_name: Fazeli
- first_name: Hamed
full_name: Hassani, Hamed
last_name: Hassani
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
- first_name: Alexander
full_name: Vardy, Alexander
last_name: Vardy
citation:
ama: 'Fazeli A, Hassani H, Mondelli M, Vardy A. Binary linear codes with optimal
scaling: Polar codes with large kernels. IEEE Transactions on Information Theory.
2021;67(9):5693-5710. doi:10.1109/TIT.2020.3038806'
apa: 'Fazeli, A., Hassani, H., Mondelli, M., & Vardy, A. (2021). Binary linear
codes with optimal scaling: Polar codes with large kernels. IEEE Transactions
on Information Theory. IEEE. https://doi.org/10.1109/TIT.2020.3038806'
chicago: 'Fazeli, Arman, Hamed Hassani, Marco Mondelli, and Alexander Vardy. “Binary
Linear Codes with Optimal Scaling: Polar Codes with Large Kernels.” IEEE Transactions
on Information Theory. IEEE, 2021. https://doi.org/10.1109/TIT.2020.3038806.'
ieee: 'A. Fazeli, H. Hassani, M. Mondelli, and A. Vardy, “Binary linear codes with
optimal scaling: Polar codes with large kernels,” IEEE Transactions on Information
Theory, vol. 67, no. 9. IEEE, pp. 5693–5710, 2021.'
ista: 'Fazeli A, Hassani H, Mondelli M, Vardy A. 2021. Binary linear codes with
optimal scaling: Polar codes with large kernels. IEEE Transactions on Information
Theory. 67(9), 5693–5710.'
mla: 'Fazeli, Arman, et al. “Binary Linear Codes with Optimal Scaling: Polar Codes
with Large Kernels.” IEEE Transactions on Information Theory, vol. 67,
no. 9, IEEE, 2021, pp. 5693–710, doi:10.1109/TIT.2020.3038806.'
short: A. Fazeli, H. Hassani, M. Mondelli, A. Vardy, IEEE Transactions on Information
Theory 67 (2021) 5693–5710.
date_created: 2021-01-10T23:01:18Z
date_published: 2021-09-01T00:00:00Z
date_updated: 2024-03-07T12:18:50Z
day: '01'
department:
- _id: MaMo
doi: 10.1109/TIT.2020.3038806
external_id:
arxiv:
- '1711.01339'
intvolume: ' 67'
issue: '9'
language:
- iso: eng
month: '09'
oa_version: Preprint
page: 5693-5710
publication: IEEE Transactions on Information Theory
publication_identifier:
eissn:
- 1557-9654
issn:
- 0018-9448
publication_status: published
publisher: IEEE
quality_controlled: '1'
related_material:
record:
- id: '6665'
relation: earlier_version
status: public
scopus_import: '1'
status: public
title: 'Binary linear codes with optimal scaling: Polar codes with large kernels'
type: journal_article
user_id: 3E5EF7F0-F248-11E8-B48F-1D18A9856A87
volume: 67
year: '2021'
...
---
_id: '9047'
abstract:
- lang: eng
text: This work analyzes the latency of the simplified successive cancellation (SSC)
decoding scheme for polar codes proposed by Alamdar-Yazdi and Kschischang. It
is shown that, unlike conventional successive cancellation decoding, where latency
is linear in the block length, the latency of SSC decoding is sublinear. More
specifically, the latency of SSC decoding is O(N1−1/μ) , where N is the block
length and μ is the scaling exponent of the channel, which captures the speed
of convergence of the rate to capacity. Numerical results demonstrate the tightness
of the bound and show that most of the latency reduction arises from the parallel
decoding of subcodes of rate 0 or 1.
acknowledgement: M. Mondelli was partially supported by grants NSF DMS-1613091, CCF-1714305,
IIS-1741162, and ONR N00014-18-1-2729. S. A. Hashemi is supported by a Postdoctoral
Fellowship from the Natural Sciences and Engineering Research Council of Canada
(NSERC) and by Huawei. The authors would like to thank the anonymous reviewers for
their comments that helped improving the quality of the manuscript.
article_processing_charge: No
article_type: original
arxiv: 1
author:
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
- first_name: Seyyed Ali
full_name: Hashemi, Seyyed Ali
last_name: Hashemi
- first_name: John M.
full_name: Cioffi, John M.
last_name: Cioffi
- first_name: Andrea
full_name: Goldsmith, Andrea
last_name: Goldsmith
citation:
ama: Mondelli M, Hashemi SA, Cioffi JM, Goldsmith A. Sublinear latency for simplified
successive cancellation decoding of polar codes. IEEE Transactions on Wireless
Communications. 2021;20(1):18-27. doi:10.1109/TWC.2020.3022922
apa: Mondelli, M., Hashemi, S. A., Cioffi, J. M., & Goldsmith, A. (2021). Sublinear
latency for simplified successive cancellation decoding of polar codes. IEEE
Transactions on Wireless Communications. IEEE. https://doi.org/10.1109/TWC.2020.3022922
chicago: Mondelli, Marco, Seyyed Ali Hashemi, John M. Cioffi, and Andrea Goldsmith.
“Sublinear Latency for Simplified Successive Cancellation Decoding of Polar Codes.”
IEEE Transactions on Wireless Communications. IEEE, 2021. https://doi.org/10.1109/TWC.2020.3022922.
ieee: M. Mondelli, S. A. Hashemi, J. M. Cioffi, and A. Goldsmith, “Sublinear latency
for simplified successive cancellation decoding of polar codes,” IEEE Transactions
on Wireless Communications, vol. 20, no. 1. IEEE, pp. 18–27, 2021.
ista: Mondelli M, Hashemi SA, Cioffi JM, Goldsmith A. 2021. Sublinear latency for
simplified successive cancellation decoding of polar codes. IEEE Transactions
on Wireless Communications. 20(1), 18–27.
mla: Mondelli, Marco, et al. “Sublinear Latency for Simplified Successive Cancellation
Decoding of Polar Codes.” IEEE Transactions on Wireless Communications,
vol. 20, no. 1, IEEE, 2021, pp. 18–27, doi:10.1109/TWC.2020.3022922.
short: M. Mondelli, S.A. Hashemi, J.M. Cioffi, A. Goldsmith, IEEE Transactions on
Wireless Communications 20 (2021) 18–27.
date_created: 2021-01-31T23:01:21Z
date_published: 2021-01-01T00:00:00Z
date_updated: 2023-08-07T13:36:25Z
day: '01'
department:
- _id: MaMo
doi: 10.1109/TWC.2020.3022922
external_id:
arxiv:
- '1909.04892'
isi:
- '000607808800002'
intvolume: ' 20'
isi: 1
issue: '1'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/1909.04892
month: '01'
oa: 1
oa_version: Preprint
page: 18-27
publication: IEEE Transactions on Wireless Communications
publication_identifier:
eissn:
- '15582248'
issn:
- '15361276'
publication_status: published
publisher: IEEE
quality_controlled: '1'
related_material:
record:
- id: '8536'
relation: earlier_version
status: public
scopus_import: '1'
status: public
title: Sublinear latency for simplified successive cancellation decoding of polar
codes
type: journal_article
user_id: 4359f0d1-fa6c-11eb-b949-802e58b17ae8
volume: 20
year: '2021'
...
---
_id: '10053'
abstract:
- lang: eng
text: 'This paper characterizes the latency of the simplified successive-cancellation
(SSC) decoding scheme for polar codes under hardware resource constraints. In
particular, when the number of processing elements P that can perform SSC decoding
operations in parallel is limited, as is the case in practice, the latency of
SSC decoding is O(N1−1 μ+NPlog2log2NP), where N is the block length of the code
and μ is the scaling exponent of polar codes for the channel. Three direct consequences
of this bound are presented. First, in a fully-parallel implementation where P=N2
, the latency of SSC decoding is O(N1−1/μ) , which is sublinear in the block length.
This recovers a result from an earlier work. Second, in a fully-serial implementation
where P=1 , the latency of SSC decoding scales as O(Nlog2log2N) . The multiplicative
constant is also calculated: we show that the latency of SSC decoding when P=1
is given by (2+o(1))Nlog2log2N . Third, in a semi-parallel implementation, the
smallest P that gives the same latency as that of the fully-parallel implementation
is P=N1/μ . The tightness of our bound on SSC decoding latency and the applicability
of the foregoing results is validated through extensive simulations.'
acknowledgement: "S. A. Hashemi is supported by a Postdoctoral Fellowship from the
Natural Sciences and Engineering Research Council\r\nof Canada (NSERC) and by Huawei.
M. Mondelli is partially supported by the 2019 Lopez-Loreta Prize. A. Fazeli and
A. Vardy were supported in part by the National Science Foundation under Grant CCF-1764104."
article_processing_charge: No
arxiv: 1
author:
- first_name: Seyyed Ali
full_name: Hashemi, Seyyed Ali
last_name: Hashemi
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
- first_name: Arman
full_name: Fazeli, Arman
last_name: Fazeli
- first_name: Alexander
full_name: Vardy, Alexander
last_name: Vardy
- first_name: John
full_name: Cioffi, John
last_name: Cioffi
- first_name: Andrea
full_name: Goldsmith, Andrea
last_name: Goldsmith
citation:
ama: 'Hashemi SA, Mondelli M, Fazeli A, Vardy A, Cioffi J, Goldsmith A. Parallelism
versus latency in simplified successive-cancellation decoding of polar codes.
In: 2021 IEEE International Symposium on Information Theory. Institute
of Electrical and Electronics Engineers; 2021:2369-2374. doi:10.1109/ISIT45174.2021.9518153'
apa: 'Hashemi, S. A., Mondelli, M., Fazeli, A., Vardy, A., Cioffi, J., & Goldsmith,
A. (2021). Parallelism versus latency in simplified successive-cancellation decoding
of polar codes. In 2021 IEEE International Symposium on Information Theory
(pp. 2369–2374). Melbourne, Australia: Institute of Electrical and Electronics
Engineers. https://doi.org/10.1109/ISIT45174.2021.9518153'
chicago: Hashemi, Seyyed Ali, Marco Mondelli, Arman Fazeli, Alexander Vardy, John
Cioffi, and Andrea Goldsmith. “Parallelism versus Latency in Simplified Successive-Cancellation
Decoding of Polar Codes.” In 2021 IEEE International Symposium on Information
Theory, 2369–74. Institute of Electrical and Electronics Engineers, 2021.
https://doi.org/10.1109/ISIT45174.2021.9518153.
ieee: S. A. Hashemi, M. Mondelli, A. Fazeli, A. Vardy, J. Cioffi, and A. Goldsmith,
“Parallelism versus latency in simplified successive-cancellation decoding of
polar codes,” in 2021 IEEE International Symposium on Information Theory,
Melbourne, Australia, 2021, pp. 2369–2374.
ista: 'Hashemi SA, Mondelli M, Fazeli A, Vardy A, Cioffi J, Goldsmith A. 2021. Parallelism
versus latency in simplified successive-cancellation decoding of polar codes.
2021 IEEE International Symposium on Information Theory. ISIT: International Symposium
on Information Theory, 2369–2374.'
mla: Hashemi, Seyyed Ali, et al. “Parallelism versus Latency in Simplified Successive-Cancellation
Decoding of Polar Codes.” 2021 IEEE International Symposium on Information
Theory, Institute of Electrical and Electronics Engineers, 2021, pp. 2369–74,
doi:10.1109/ISIT45174.2021.9518153.
short: S.A. Hashemi, M. Mondelli, A. Fazeli, A. Vardy, J. Cioffi, A. Goldsmith,
in:, 2021 IEEE International Symposium on Information Theory, Institute of Electrical
and Electronics Engineers, 2021, pp. 2369–2374.
conference:
end_date: 2021-07-20
location: Melbourne, Australia
name: 'ISIT: International Symposium on Information Theory'
start_date: 2021-07-12
date_created: 2021-09-27T14:33:14Z
date_published: 2021-09-01T00:00:00Z
date_updated: 2024-09-10T13:03:18Z
day: '01'
department:
- _id: MaMo
doi: 10.1109/ISIT45174.2021.9518153
external_id:
arxiv:
- '2012.13378'
isi:
- '000701502202078'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/2012.13378
month: '09'
oa: 1
oa_version: Preprint
page: 2369-2374
project:
- _id: 059876FA-7A3F-11EA-A408-12923DDC885E
name: Prix Lopez-Loretta 2019 - Marco Mondelli
publication: 2021 IEEE International Symposium on Information Theory
publication_identifier:
eisbn:
- 978-1-5386-8209-8
isbn:
- 978-1-5386-8210-4
issn:
- 2157-8095
publication_status: published
publisher: Institute of Electrical and Electronics Engineers
quality_controlled: '1'
related_material:
record:
- id: '10364'
relation: later_version
status: public
scopus_import: '1'
status: public
title: Parallelism versus latency in simplified successive-cancellation decoding of
polar codes
type: conference
user_id: 4359f0d1-fa6c-11eb-b949-802e58b17ae8
year: '2021'
...
---
_id: '10211'
abstract:
- lang: eng
text: "We study the problem of recovering an unknown signal \U0001D465\U0001D465
given measurements obtained from a generalized linear model with a Gaussian sensing
matrix. Two popular solutions are based on a linear estimator \U0001D465\U0001D465^L
and a spectral estimator \U0001D465\U0001D465^s. The former is a data-dependent
linear combination of the columns of the measurement matrix, and its analysis
is quite simple. The latter is the principal eigenvector of a data-dependent matrix,
and a recent line of work has studied its performance. In this paper, we show
how to optimally combine \U0001D465\U0001D465^L and \U0001D465\U0001D465^s. At
the heart of our analysis is the exact characterization of the empirical joint
distribution of (\U0001D465\U0001D465,\U0001D465\U0001D465^L,\U0001D465\U0001D465^s)
in the high-dimensional limit. This allows us to compute the Bayes-optimal combination
of \U0001D465\U0001D465^L and \U0001D465\U0001D465^s, given the limiting distribution
of the signal \U0001D465\U0001D465. When the distribution of the signal is Gaussian,
then the Bayes-optimal combination has the form \U0001D703\U0001D465\U0001D465^L+\U0001D465\U0001D465^s
and we derive the optimal combination coefficient. In order to establish the limiting
distribution of (\U0001D465\U0001D465,\U0001D465\U0001D465^L,\U0001D465\U0001D465^s),
we design and analyze an approximate message passing algorithm whose iterates
give \U0001D465\U0001D465^L and approach \U0001D465\U0001D465^s. Numerical simulations
demonstrate the improvement of the proposed combination with respect to the two
methods considered separately."
acknowledgement: M. Mondelli would like to thank Andrea Montanari for helpful discussions.
All the authors would like to thank the anonymous reviewers for their helpful comments.
article_processing_charge: Yes (via OA deal)
article_type: original
arxiv: 1
author:
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
- first_name: Christos
full_name: Thrampoulidis, Christos
last_name: Thrampoulidis
- first_name: Ramji
full_name: Venkataramanan, Ramji
last_name: Venkataramanan
citation:
ama: Mondelli M, Thrampoulidis C, Venkataramanan R. Optimal combination of linear
and spectral estimators for generalized linear models. Foundations of Computational
Mathematics. 2021. doi:10.1007/s10208-021-09531-x
apa: Mondelli, M., Thrampoulidis, C., & Venkataramanan, R. (2021). Optimal combination
of linear and spectral estimators for generalized linear models. Foundations
of Computational Mathematics. Springer. https://doi.org/10.1007/s10208-021-09531-x
chicago: Mondelli, Marco, Christos Thrampoulidis, and Ramji Venkataramanan. “Optimal
Combination of Linear and Spectral Estimators for Generalized Linear Models.”
Foundations of Computational Mathematics. Springer, 2021. https://doi.org/10.1007/s10208-021-09531-x.
ieee: M. Mondelli, C. Thrampoulidis, and R. Venkataramanan, “Optimal combination
of linear and spectral estimators for generalized linear models,” Foundations
of Computational Mathematics. Springer, 2021.
ista: Mondelli M, Thrampoulidis C, Venkataramanan R. 2021. Optimal combination of
linear and spectral estimators for generalized linear models. Foundations of Computational
Mathematics.
mla: Mondelli, Marco, et al. “Optimal Combination of Linear and Spectral Estimators
for Generalized Linear Models.” Foundations of Computational Mathematics,
Springer, 2021, doi:10.1007/s10208-021-09531-x.
short: M. Mondelli, C. Thrampoulidis, R. Venkataramanan, Foundations of Computational
Mathematics (2021).
date_created: 2021-11-03T10:59:08Z
date_published: 2021-08-17T00:00:00Z
date_updated: 2023-09-05T14:13:57Z
day: '17'
ddc:
- '510'
department:
- _id: MaMo
doi: 10.1007/s10208-021-09531-x
external_id:
arxiv:
- '2008.03326'
isi:
- '000685721000001'
file:
- access_level: open_access
checksum: 9ea12dd8045a0678000a3a59295221cb
content_type: application/pdf
creator: alisjak
date_created: 2021-12-13T15:47:54Z
date_updated: 2021-12-13T15:47:54Z
file_id: '10542'
file_name: 2021_Springer_Mondelli.pdf
file_size: 2305731
relation: main_file
success: 1
file_date_updated: 2021-12-13T15:47:54Z
has_accepted_license: '1'
isi: 1
keyword:
- Applied Mathematics
- Computational Theory and Mathematics
- Computational Mathematics
- Analysis
language:
- iso: eng
month: '08'
oa: 1
oa_version: Published Version
project:
- _id: B67AFEDC-15C9-11EA-A837-991A96BB2854
name: IST Austria Open Access Fund
publication: Foundations of Computational Mathematics
publication_identifier:
eissn:
- 1615-3383
issn:
- 1615-3375
publication_status: published
publisher: Springer
quality_controlled: '1'
scopus_import: '1'
status: public
title: Optimal combination of linear and spectral estimators for generalized linear
models
tmp:
image: /images/cc_by.png
legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
short: CC BY (4.0)
type: journal_article
user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1
year: '2021'
...
---
_id: '10593'
abstract:
- lang: eng
text: 'We study the problem of estimating a rank-$1$ signal in the presence of rotationally
invariant noise-a class of perturbations more general than Gaussian noise. Principal
Component Analysis (PCA) provides a natural estimator, and sharp results on its
performance have been obtained in the high-dimensional regime. Recently, an Approximate
Message Passing (AMP) algorithm has been proposed as an alternative estimator
with the potential to improve the accuracy of PCA. However, the existing analysis
of AMP requires an initialization that is both correlated with the signal and
independent of the noise, which is often unrealistic in practice. In this work,
we combine the two methods, and propose to initialize AMP with PCA. Our main result
is a rigorous asymptotic characterization of the performance of this estimator.
Both the AMP algorithm and its analysis differ from those previously derived in
the Gaussian setting: at every iteration, our AMP algorithm requires a specific
term to account for PCA initialization, while in the Gaussian case, PCA initialization
affects only the first iteration of AMP. The proof is based on a two-phase artificial
AMP that first approximates the PCA estimator and then mimics the true AMP. Our
numerical simulations show an excellent agreement between AMP results and theoretical
predictions, and suggest an interesting open direction on achieving Bayes-optimal
performance.'
acknowledgement: "M. Mondelli would like to thank László Erdős for helpful discussions.
M. Mondelli was partially supported by the 2019 Lopez-Loreta Prize. R. Venkataramanan
was partially supported by the Alan Turing Institute under the EPSRC grant EP/N510129/1.\r\n"
article_processing_charge: No
arxiv: 1
author:
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
- first_name: Ramji
full_name: Venkataramanan, Ramji
last_name: Venkataramanan
citation:
ama: 'Mondelli M, Venkataramanan R. PCA initialization for approximate message passing
in rotationally invariant models. In: 35th Conference on Neural Information
Processing Systems. Vol 35. Neural Information Processing Systems Foundation;
2021:29616-29629.'
apa: 'Mondelli, M., & Venkataramanan, R. (2021). PCA initialization for approximate
message passing in rotationally invariant models. In 35th Conference on Neural
Information Processing Systems (Vol. 35, pp. 29616–29629). Virtual: Neural
Information Processing Systems Foundation.'
chicago: Mondelli, Marco, and Ramji Venkataramanan. “PCA Initialization for Approximate
Message Passing in Rotationally Invariant Models.” In 35th Conference on Neural
Information Processing Systems, 35:29616–29. Neural Information Processing
Systems Foundation, 2021.
ieee: M. Mondelli and R. Venkataramanan, “PCA initialization for approximate message
passing in rotationally invariant models,” in 35th Conference on Neural Information
Processing Systems, Virtual, 2021, vol. 35, pp. 29616–29629.
ista: 'Mondelli M, Venkataramanan R. 2021. PCA initialization for approximate message
passing in rotationally invariant models. 35th Conference on Neural Information
Processing Systems. NeurIPS: Neural Information Processing Systems vol. 35, 29616–29629.'
mla: Mondelli, Marco, and Ramji Venkataramanan. “PCA Initialization for Approximate
Message Passing in Rotationally Invariant Models.” 35th Conference on Neural
Information Processing Systems, vol. 35, Neural Information Processing Systems
Foundation, 2021, pp. 29616–29.
short: M. Mondelli, R. Venkataramanan, in:, 35th Conference on Neural Information
Processing Systems, Neural Information Processing Systems Foundation, 2021, pp.
29616–29629.
conference:
end_date: 2021-12-14
location: Virtual
name: 'NeurIPS: Neural Information Processing Systems'
start_date: 2021-12-06
date_created: 2022-01-03T10:50:02Z
date_published: 2021-12-01T00:00:00Z
date_updated: 2024-09-10T13:03:19Z
day: '01'
department:
- _id: MaMo
external_id:
arxiv:
- '2106.02356'
intvolume: ' 35'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/2106.02356
month: '12'
oa: 1
oa_version: Preprint
page: 29616-29629
project:
- _id: 059876FA-7A3F-11EA-A408-12923DDC885E
name: Prix Lopez-Loretta 2019 - Marco Mondelli
publication: 35th Conference on Neural Information Processing Systems
publication_identifier:
isbn:
- '9781713845393'
issn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
scopus_import: '1'
status: public
title: PCA initialization for approximate message passing in rotationally invariant
models
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 35
year: '2021'
...
---
_id: '10594'
abstract:
- lang: eng
text: 'The question of how and why the phenomenon of mode connectivity occurs in
training deep neural networks has gained remarkable attention in the research
community. From a theoretical perspective, two possible explanations have been
proposed: (i) the loss function has connected sublevel sets, and (ii) the solutions
found by stochastic gradient descent are dropout stable. While these explanations
provide insights into the phenomenon, their assumptions are not always satisfied
in practice. In particular, the first approach requires the network to have one
layer with order of N neurons (N being the number of training samples), while
the second one requires the loss to be almost invariant after removing half of
the neurons at each layer (up to some rescaling of the remaining ones). In this
work, we improve both conditions by exploiting the quality of the features at
every intermediate layer together with a milder over-parameterization condition.
More specifically, we show that: (i) under generic assumptions on the features
of intermediate layers, it suffices that the last two hidden layers have order
of N−−√ neurons, and (ii) if subsets of features at each layer are linearly separable,
then no over-parameterization is needed to show the connectivity. Our experiments
confirm that the proposed condition ensures the connectivity of solutions found
by stochastic gradient descent, even in settings where the previous requirements
do not hold.'
acknowledgement: MM was partially supported by the 2019 Lopez-Loreta Prize. QN and
PB acknowledge support from the European Research Council (ERC) under the European
Union’s Horizon 2020 research and innovation programme (grant agreement no 757983).
article_processing_charge: No
arxiv: 1
author:
- first_name: Quynh
full_name: Nguyen, Quynh
last_name: Nguyen
- first_name: Pierre
full_name: Bréchet, Pierre
last_name: Bréchet
- first_name: Marco
full_name: Mondelli, Marco
id: 27EB676C-8706-11E9-9510-7717E6697425
last_name: Mondelli
orcid: 0000-0002-3242-7020
citation:
ama: 'Nguyen Q, Bréchet P, Mondelli M. When are solutions connected in deep networks?
In: 35th Conference on Neural Information Processing Systems. Vol 35. Neural
Information Processing Systems Foundation; 2021.'
apa: 'Nguyen, Q., Bréchet, P., & Mondelli, M. (2021). When are solutions connected
in deep networks? In 35th Conference on Neural Information Processing Systems
(Vol. 35). Virtual: Neural Information Processing Systems Foundation.'
chicago: Nguyen, Quynh, Pierre Bréchet, and Marco Mondelli. “When Are Solutions
Connected in Deep Networks?” In 35th Conference on Neural Information Processing
Systems, Vol. 35. Neural Information Processing Systems Foundation, 2021.
ieee: Q. Nguyen, P. Bréchet, and M. Mondelli, “When are solutions connected in deep
networks?,” in 35th Conference on Neural Information Processing Systems,
Virtual, 2021, vol. 35.
ista: Nguyen Q, Bréchet P, Mondelli M. 2021. When are solutions connected in deep
networks? 35th Conference on Neural Information Processing Systems. 35th Conference
on Neural Information Processing Systems vol. 35.
mla: Nguyen, Quynh, et al. “When Are Solutions Connected in Deep Networks?” 35th
Conference on Neural Information Processing Systems, vol. 35, Neural Information
Processing Systems Foundation, 2021.
short: Q. Nguyen, P. Bréchet, M. Mondelli, in:, 35th Conference on Neural Information
Processing Systems, Neural Information Processing Systems Foundation, 2021.
conference:
end_date: 2021-12-14
location: Virtual
name: 35th Conference on Neural Information Processing Systems
start_date: 2021-12-06
date_created: 2022-01-03T10:56:20Z
date_published: 2021-12-01T00:00:00Z
date_updated: 2024-09-10T13:03:19Z
day: '01'
department:
- _id: MaMo
external_id:
arxiv:
- '2102.09671'
intvolume: ' 35'
language:
- iso: eng
main_file_link:
- open_access: '1'
url: https://arxiv.org/abs/2102.09671
month: '12'
oa: 1
oa_version: Preprint
project:
- _id: 059876FA-7A3F-11EA-A408-12923DDC885E
name: Prix Lopez-Loretta 2019 - Marco Mondelli
publication: 35th Conference on Neural Information Processing Systems
publication_identifier:
isbn:
- '9781713845393'
issn:
- 1049-5258
publication_status: published
publisher: Neural Information Processing Systems Foundation
quality_controlled: '1'
status: public
title: When are solutions connected in deep networks?
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 35
year: '2021'
...