Elastic Coordination for Scalable Machine Learning
Project Period: 2019-03-01 – 2024-02-29
Funding Organisation:
EC/H2020
Acronym
ScaleML
Principal Investigator
Department(s)
Grant Number
805223
Funding Organisation
EC/H2020
38 Publications
2023 | Published | Journal Article | IST-REx-ID: 12566 | 
Wait-free approximate agreement on graphs
D.-A. Alistarh, F. Ellen, J. Rybicki, Theoretical Computer Science 948 (2023).
[Published Version]
View
| Files available
| DOI
| WoS
D.-A. Alistarh, F. Ellen, J. Rybicki, Theoretical Computer Science 948 (2023).
2023 | Accepted | Conference Paper | IST-REx-ID: 13053 |
CrAM: A Compression-Aware Minimizer
E.-A. Peste, A. Vladu, E. Kurtic, C. Lampert, D.-A. Alistarh, in:, 11th International Conference on Learning Representations , n.d.
[Preprint]
View
| Files available
| Download Preprint (ext.)
| arXiv
E.-A. Peste, A. Vladu, E. Kurtic, C. Lampert, D.-A. Alistarh, in:, 11th International Conference on Learning Representations , n.d.
2023 | Published | Thesis | IST-REx-ID: 13074 |
Efficiency and generalization of sparse neural networks
E.-A. Peste, Efficiency and Generalization of Sparse Neural Networks, Institute of Science and Technology Austria, 2023.
[Published Version]
View
| Files available
| DOI
E.-A. Peste, Efficiency and Generalization of Sparse Neural Networks, Institute of Science and Technology Austria, 2023.
2023 | Published | Journal Article | IST-REx-ID: 14364 |
Why extension-based proofs fail
D.-A. Alistarh, J. Aspnes, F. Ellen, R. Gelashvili, L. Zhu, SIAM Journal on Computing 52 (2023) 913–944.
[Preprint]
View
| Files available
| DOI
| Download Preprint (ext.)
| WoS
| arXiv
D.-A. Alistarh, J. Aspnes, F. Ellen, R. Gelashvili, L. Zhu, SIAM Journal on Computing 52 (2023) 913–944.
2023 | Published | Conference Paper | IST-REx-ID: 14458 |
SparseGPT: Massive language models can be accurately pruned in one-shot
E. Frantar, D.-A. Alistarh, in:, Proceedings of the 40th International Conference on Machine Learning, ML Research Press, 2023, pp. 10323–10337.
[Preprint]
View
| Download Preprint (ext.)
| arXiv
E. Frantar, D.-A. Alistarh, in:, Proceedings of the 40th International Conference on Machine Learning, ML Research Press, 2023, pp. 10323–10337.
2023 | Published | Conference Paper | IST-REx-ID: 14460 |
SparseProp: Efficient sparse backpropagation for faster training of neural networks at the edge
M. Nikdan, T. Pegolotti, E.B. Iofinova, E. Kurtic, D.-A. Alistarh, in:, Proceedings of the 40th International Conference on Machine Learning, ML Research Press, 2023, pp. 26215–26227.
[Preprint]
View
| Download Preprint (ext.)
| arXiv
M. Nikdan, T. Pegolotti, E.B. Iofinova, E. Kurtic, D.-A. Alistarh, in:, Proceedings of the 40th International Conference on Machine Learning, ML Research Press, 2023, pp. 26215–26227.
2023 | Published | Conference Paper | IST-REx-ID: 14461 |
Quantized distributed training of large models with convergence guarantees
I. Markov, A. Vladu, Q. Guo, D.-A. Alistarh, in:, Proceedings of the 40th International Conference on Machine Learning, ML Research Press, 2023, pp. 24020–24044.
[Preprint]
View
| Download Preprint (ext.)
| arXiv
I. Markov, A. Vladu, Q. Guo, D.-A. Alistarh, in:, Proceedings of the 40th International Conference on Machine Learning, ML Research Press, 2023, pp. 24020–24044.
2023 | Published | Conference Paper | IST-REx-ID: 14771 |
Bias in pruned vision models: In-depth analysis and countermeasures
E.B. Iofinova, E.-A. Peste, D.-A. Alistarh, in:, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2023, pp. 24364–24373.
[Preprint]
View
| Files available
| DOI
| Download Preprint (ext.)
| WoS
| arXiv
E.B. Iofinova, E.-A. Peste, D.-A. Alistarh, in:, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2023, pp. 24364–24373.
2022 | Published | Conference Paper | IST-REx-ID: 12299 |
How well do sparse ImageNet models transfer?
E.B. Iofinova, E.-A. Peste, M. Kurtz, D.-A. Alistarh, in:, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Institute of Electrical and Electronics Engineers, 2022, pp. 12256–12266.
[Preprint]
View
| Files available
| DOI
| Download Preprint (ext.)
| WoS
| arXiv
E.B. Iofinova, E.-A. Peste, M. Kurtz, D.-A. Alistarh, in:, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Institute of Electrical and Electronics Engineers, 2022, pp. 12256–12266.
2022 | Published | Conference Paper | IST-REx-ID: 11180 |
Multi-queues can be state-of-the-art priority schedulers
A. Postnikova, N. Koval, G. Nadiradze, D.-A. Alistarh, in:, Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Association for Computing Machinery, 2022, pp. 353–367.
[Preprint]
View
| Files available
| DOI
| Download Preprint (ext.)
| WoS
| arXiv
A. Postnikova, N. Koval, G. Nadiradze, D.-A. Alistarh, in:, Proceedings of the 27th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, Association for Computing Machinery, 2022, pp. 353–367.
2022 | Published | Conference Paper | IST-REx-ID: 11183 |
Beyond distributed subgraph detection: Induced subgraphs, multicolored problems and graph parameters
A. Nikabadi, J. Korhonen, in:, Q. Bramas, V. Gramoli, A. Milani (Eds.), 25th International Conference on Principles of Distributed Systems, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.
[Published Version]
View
| Files available
| DOI
A. Nikabadi, J. Korhonen, in:, Q. Bramas, V. Gramoli, A. Milani (Eds.), 25th International Conference on Principles of Distributed Systems, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.
2022 | Published | Conference Paper | IST-REx-ID: 11184 |
Fast graphical population protocols
D.-A. Alistarh, R. Gelashvili, J. Rybicki, in:, Q. Bramas, V. Gramoli, A. Milani (Eds.), 25th International Conference on Principles of Distributed Systems, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.
[Published Version]
View
| Files available
| DOI
| arXiv
D.-A. Alistarh, R. Gelashvili, J. Rybicki, in:, Q. Bramas, V. Gramoli, A. Milani (Eds.), 25th International Conference on Principles of Distributed Systems, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2022.
2022 | Published | Conference Paper | IST-REx-ID: 11844 |
Near-optimal leader election in population protocols on graphs
D.-A. Alistarh, J. Rybicki, S. Voitovych, in:, Proceedings of the Annual ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2022, pp. 246–256.
[Published Version]
View
| Files available
| DOI
| arXiv
D.-A. Alistarh, J. Rybicki, S. Voitovych, in:, Proceedings of the Annual ACM Symposium on Principles of Distributed Computing, Association for Computing Machinery, 2022, pp. 246–256.
2021 | Published | Journal Article | IST-REx-ID: 8286 |
Dynamic averaging load balancing on cycles
D.-A. Alistarh, G. Nadiradze, A. Sabour, Algorithmica (2021).
[Published Version]
View
| Files available
| DOI
| WoS
| arXiv
D.-A. Alistarh, G. Nadiradze, A. Sabour, Algorithmica (2021).
2021 | Published | Journal Article | IST-REx-ID: 8723 |
Breaking (global) barriers in parallel stochastic optimization with wait-avoiding group averaging
S. Li, T.B.-N. Tal Ben-Nun, G. Nadiradze, S.D. Girolamo, N. Dryden, D.-A. Alistarh, T. Hoefler, IEEE Transactions on Parallel and Distributed Systems 32 (2021).
[Preprint]
View
| DOI
| Download Preprint (ext.)
| WoS
| arXiv
S. Li, T.B.-N. Tal Ben-Nun, G. Nadiradze, S.D. Girolamo, N. Dryden, D.-A. Alistarh, T. Hoefler, IEEE Transactions on Parallel and Distributed Systems 32 (2021).
2021 | Published | Conference Paper | IST-REx-ID: 13147 |
Communication-efficient distributed optimization with quantized preconditioners
F. Alimisis, P. Davies, D.-A. Alistarh, in:, Proceedings of the 38th International Conference on Machine Learning, ML Research Press, 2021, pp. 196–206.
[Published Version]
View
| Files available
| arXiv
F. Alimisis, P. Davies, D.-A. Alistarh, in:, Proceedings of the 38th International Conference on Machine Learning, ML Research Press, 2021, pp. 196–206.
2021 | Published | Conference Paper | IST-REx-ID: 11436 |
Asynchronous optimization methods for efficient training of deep neural networks with guarantees
V. Kungurtsev, M. Egan, B. Chatterjee, D.-A. Alistarh, in:, 35th AAAI Conference on Artificial Intelligence, AAAI 2021, AAAI Press, 2021, pp. 8209–8216.
[Preprint]
View
| Download Preprint (ext.)
| arXiv
V. Kungurtsev, M. Egan, B. Chatterjee, D.-A. Alistarh, in:, 35th AAAI Conference on Artificial Intelligence, AAAI 2021, AAAI Press, 2021, pp. 8209–8216.
2021 | Published | Conference Paper | IST-REx-ID: 11452 |
Distributed principal component analysis with limited communication
F. Alimisis, P. Davies, B. Vandereycken, D.-A. Alistarh, in:, Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2021, pp. 2823–2834.
[Published Version]
View
| Download Published Version (ext.)
| arXiv
F. Alimisis, P. Davies, B. Vandereycken, D.-A. Alistarh, in:, Advances in Neural Information Processing Systems - 35th Conference on Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2021, pp. 2823–2834.
2021 | Published | Conference Paper | IST-REx-ID: 11458 |
AC/DC: Alternating Compressed/DeCompressed training of deep neural networks
E.-A. Peste, E.B. Iofinova, A. Vladu, D.-A. Alistarh, in:, 35th Conference on Neural Information Processing Systems, Curran Associates, 2021, pp. 8557–8570.
[Published Version]
View
| Files available
| Download Published Version (ext.)
| arXiv
E.-A. Peste, E.B. Iofinova, A. Vladu, D.-A. Alistarh, in:, 35th Conference on Neural Information Processing Systems, Curran Associates, 2021, pp. 8557–8570.
2021 | Published | Conference Paper | IST-REx-ID: 11463 |
M-FAC: Efficient matrix-free approximations of second-order information
E. Frantar, E. Kurtic, D.-A. Alistarh, in:, 35th Conference on Neural Information Processing Systems, Curran Associates, 2021, pp. 14873–14886.
[Published Version]
View
| Download Published Version (ext.)
| arXiv
E. Frantar, E. Kurtic, D.-A. Alistarh, in:, 35th Conference on Neural Information Processing Systems, Curran Associates, 2021, pp. 14873–14886.