{"title":"Fast quantized arithmetic on x86: Trading compute for data movement","quality_controlled":"1","isi":1,"status":"public","year":"2018","publisher":"IEEE","scopus_import":"1","oa_version":"None","volume":"2018-October","user_id":"c635000d-4b10-11ee-a964-aac5a93f6ac1","date_created":"2019-02-17T22:59:25Z","article_number":"8598402","_id":"6031","language":[{"iso":"eng"}],"department":[{"_id":"DaAl"}],"abstract":[{"lang":"eng","text":"We introduce Clover, a new library for efficient computation using low-precision data, providing mathematical routines required by fundamental methods in optimization and sparse recovery. Our library faithfully implements variants of stochastic quantization that guarantee convergence at low precision, and supports data formats from 4-bit quantized to 32-bit IEEE-754 on current Intel processors. In particular, we show that 4-bit can be implemented efficiently using Intel AVX despite the lack of native support for this data format. Experimental results with dot product, matrix-vector multiplication (MVM), gradient descent (GD), and iterative hard thresholding (IHT) demonstrate that the attainable speedups are in many cases close to linear with respect to the reduction of precision due to reduced data movement. Finally, for GD and IHT, we show examples of absolute speedup achieved by 4-bit versus 32-bit, by iterating until a given target error is achieved."}],"conference":{"start_date":"2018-10-21","name":"SiPS: Workshop on Signal Processing Systems","end_date":"2018-10-24","location":"Cape Town, South Africa"},"publication_status":"published","citation":{"ieee":"A. Stojanov, T. M. Smith, D.-A. Alistarh, and M. Puschel, “Fast quantized arithmetic on x86: Trading compute for data movement,” in <i>2018 IEEE International Workshop on Signal Processing Systems</i>, Cape Town, South Africa, 2018, vol. 2018–October.","mla":"Stojanov, Alen, et al. “Fast Quantized Arithmetic on X86: Trading Compute for Data Movement.” <i>2018 IEEE International Workshop on Signal Processing Systems</i>, vol. 2018–October, 8598402, IEEE, 2018, doi:<a href=\"https://doi.org/10.1109/SiPS.2018.8598402\">10.1109/SiPS.2018.8598402</a>.","short":"A. Stojanov, T.M. Smith, D.-A. Alistarh, M. Puschel, in:, 2018 IEEE International Workshop on Signal Processing Systems, IEEE, 2018.","ista":"Stojanov A, Smith TM, Alistarh D-A, Puschel M. 2018. Fast quantized arithmetic on x86: Trading compute for data movement. 2018 IEEE International Workshop on Signal Processing Systems. SiPS: Workshop on Signal Processing Systems vol. 2018–October, 8598402.","chicago":"Stojanov, Alen, Tyler Michael Smith, Dan-Adrian Alistarh, and Markus Puschel. “Fast Quantized Arithmetic on X86: Trading Compute for Data Movement.” In <i>2018 IEEE International Workshop on Signal Processing Systems</i>, Vol. 2018–October. IEEE, 2018. <a href=\"https://doi.org/10.1109/SiPS.2018.8598402\">https://doi.org/10.1109/SiPS.2018.8598402</a>.","apa":"Stojanov, A., Smith, T. M., Alistarh, D.-A., &#38; Puschel, M. (2018). Fast quantized arithmetic on x86: Trading compute for data movement. In <i>2018 IEEE International Workshop on Signal Processing Systems</i> (Vol. 2018–October). Cape Town, South Africa: IEEE. <a href=\"https://doi.org/10.1109/SiPS.2018.8598402\">https://doi.org/10.1109/SiPS.2018.8598402</a>","ama":"Stojanov A, Smith TM, Alistarh D-A, Puschel M. Fast quantized arithmetic on x86: Trading compute for data movement. In: <i>2018 IEEE International Workshop on Signal Processing Systems</i>. Vol 2018-October. IEEE; 2018. doi:<a href=\"https://doi.org/10.1109/SiPS.2018.8598402\">10.1109/SiPS.2018.8598402</a>"},"date_published":"2018-12-31T00:00:00Z","month":"12","article_processing_charge":"No","external_id":{"isi":["000465106800060"]},"date_updated":"2023-09-19T14:41:51Z","type":"conference","day":"31","author":[{"last_name":"Stojanov","first_name":"Alen","full_name":"Stojanov, Alen"},{"first_name":"Tyler Michael","last_name":"Smith","full_name":"Smith, Tyler Michael"},{"orcid":"0000-0003-3650-940X","id":"4A899BFC-F248-11E8-B48F-1D18A9856A87","full_name":"Alistarh, Dan-Adrian","first_name":"Dan-Adrian","last_name":"Alistarh"},{"full_name":"Puschel, Markus","last_name":"Puschel","first_name":"Markus"}],"publication":"2018 IEEE International Workshop on Signal Processing Systems","doi":"10.1109/SiPS.2018.8598402"}