{"date_updated":"2023-09-19T15:12:45Z","article_processing_charge":"No","month":"12","day":"01","type":"conference","abstract":[{"text":"This paper studies the problem of distributed stochastic optimization in an adversarial setting where, out of m machines which allegedly compute stochastic gradients every iteration, an α-fraction are Byzantine, and may behave adversarially. Our main result is a variant of stochastic gradient descent (SGD) which finds ε-approximate minimizers of convex functions in T=O~(1/ε²m+α²/ε²) iterations. In contrast, traditional mini-batch SGD needs T=O(1/ε²m) iterations, but cannot tolerate Byzantine failures. Further, we provide a lower bound showing that, up to logarithmic factors, our algorithm is information-theoretically optimal both in terms of sample complexity and time complexity.","lang":"eng"}],"main_file_link":[{"url":"https://arxiv.org/abs/1803.08917","open_access":"1"}],"department":[{"_id":"DaAl"}],"date_published":"2018-12-01T00:00:00Z","publication_status":"published","scopus_import":"1","publisher":"Neural Information Processing Systems Foundation","user_id":"c635000d-4b10-11ee-a964-aac5a93f6ac1","date_created":"2019-06-13T08:22:37Z","oa_version":"Published Version","oa":1,"status":"public","quality_controlled":"1","title":"Byzantine stochastic gradient descent","external_id":{"isi":["000461823304061"],"arxiv":["1803.08917"]},"publication":"Advances in Neural Information Processing Systems","author":[{"orcid":"0000-0003-3650-940X","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","full_name":"Alistarh, Dan-Adrian","last_name":"Alistarh","first_name":"Dan-Adrian"},{"full_name":"Allen-Zhu, Zeyuan","first_name":"Zeyuan","last_name":"Allen-Zhu"},{"last_name":"Li","first_name":"Jerry","full_name":"Li, Jerry"}],"page":"4613-4623","_id":"6558","language":[{"iso":"eng"}],"citation":{"short":"D.-A. Alistarh, Z. Allen-Zhu, J. Li, in:, Advances in Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2018, pp. 4613–4623.","ama":"Alistarh D-A, Allen-Zhu Z, Li J. Byzantine stochastic gradient descent. In: <i>Advances in Neural Information Processing Systems</i>. Vol 2018. Neural Information Processing Systems Foundation; 2018:4613-4623.","apa":"Alistarh, D.-A., Allen-Zhu, Z., &#38; Li, J. (2018). Byzantine stochastic gradient descent. In <i>Advances in Neural Information Processing Systems</i> (Vol. 2018, pp. 4613–4623). Montreal, Canada: Neural Information Processing Systems Foundation.","chicago":"Alistarh, Dan-Adrian, Zeyuan Allen-Zhu, and Jerry Li. “Byzantine Stochastic Gradient Descent.” In <i>Advances in Neural Information Processing Systems</i>, 2018:4613–23. Neural Information Processing Systems Foundation, 2018.","ista":"Alistarh D-A, Allen-Zhu Z, Li J. 2018. Byzantine stochastic gradient descent. Advances in Neural Information Processing Systems. NeurIPS: Conference on Neural Information Processing Systems vol. 2018, 4613–4623.","ieee":"D.-A. Alistarh, Z. Allen-Zhu, and J. Li, “Byzantine stochastic gradient descent,” in <i>Advances in Neural Information Processing Systems</i>, Montreal, Canada, 2018, vol. 2018, pp. 4613–4623.","mla":"Alistarh, Dan-Adrian, et al. “Byzantine Stochastic Gradient Descent.” <i>Advances in Neural Information Processing Systems</i>, vol. 2018, Neural Information Processing Systems Foundation, 2018, pp. 4613–23."},"conference":{"name":"NeurIPS: Conference on Neural Information Processing Systems","start_date":"2018-12-02","location":"Montreal, Canada","end_date":"2018-12-08"},"year":"2018","volume":2018,"intvolume":"      2018","isi":1}