{"date_updated":"2023-09-11T11:31:59Z","publication_identifier":{"isbn":["9781713845393"]},"month":"07","article_processing_charge":"No","external_id":{"arxiv":["2107.01057"]},"publication":"35th Conference on Neural Information Processing Systems","day":"02","author":[{"full_name":"Träuble, Frederik","first_name":"Frederik","last_name":"Träuble"},{"full_name":"Kügelgen, Julius von","last_name":"Kügelgen","first_name":"Julius von"},{"full_name":"Kleindessner, Matthäus","last_name":"Kleindessner","first_name":"Matthäus"},{"full_name":"Locatello, Francesco","id":"26cfd52f-2483-11ee-8040-88983bcc06d4","first_name":"Francesco","last_name":"Locatello","orcid":"0000-0002-4850-0683"},{"first_name":"Bernhard","last_name":"Schölkopf","full_name":"Schölkopf, Bernhard"},{"full_name":"Gehler, Peter","last_name":"Gehler","first_name":"Peter"}],"page":"116-128","type":"conference","main_file_link":[{"open_access":"1","url":"https://arxiv.org/abs/2107.01057"}],"abstract":[{"text":"When machine learning systems meet real world applications, accuracy is only\r\none of several requirements. In this paper, we assay a complementary\r\nperspective originating from the increasing availability of pre-trained and\r\nregularly improving state-of-the-art models. While new improved models develop\r\nat a fast pace, downstream tasks vary more slowly or stay constant. Assume that\r\nwe have a large unlabelled data set for which we want to maintain accurate\r\npredictions. Whenever a new and presumably better ML models becomes available,\r\nwe encounter two problems: (i) given a limited budget, which data points should\r\nbe re-evaluated using the new model?; and (ii) if the new predictions differ\r\nfrom the current ones, should we update? Problem (i) is about compute cost,\r\nwhich matters for very large data sets and models. Problem (ii) is about\r\nmaintaining consistency of the predictions, which can be highly relevant for\r\ndownstream applications; our demand is to avoid negative flips, i.e., changing\r\ncorrect to incorrect predictions. In this paper, we formalize the Prediction\r\nUpdate Problem and present an efficient probabilistic approach as answer to the\r\nabove questions. In extensive experiments on standard classification benchmark\r\ndata sets, we show that our method outperforms alternative strategies along key\r\nmetrics for backward-compatible prediction updates.","lang":"eng"}],"_id":"14182","language":[{"iso":"eng"}],"department":[{"_id":"FrLo"}],"citation":{"ieee":"F. Träuble, J. von Kügelgen, M. Kleindessner, F. Locatello, B. Schölkopf, and P. Gehler, “Backward-compatible prediction updates: A probabilistic approach,” in <i>35th Conference on Neural Information Processing Systems</i>, Virtual, 2021, vol. 34, pp. 116–128.","mla":"Träuble, Frederik, et al. “Backward-Compatible Prediction Updates: A Probabilistic Approach.” <i>35th Conference on Neural Information Processing Systems</i>, vol. 34, 2021, pp. 116–28.","short":"F. Träuble, J. von Kügelgen, M. Kleindessner, F. Locatello, B. Schölkopf, P. Gehler, in:, 35th Conference on Neural Information Processing Systems, 2021, pp. 116–128.","chicago":"Träuble, Frederik, Julius von Kügelgen, Matthäus Kleindessner, Francesco Locatello, Bernhard Schölkopf, and Peter Gehler. “Backward-Compatible Prediction Updates: A Probabilistic Approach.” In <i>35th Conference on Neural Information Processing Systems</i>, 34:116–28, 2021.","ista":"Träuble F, Kügelgen J von, Kleindessner M, Locatello F, Schölkopf B, Gehler P. 2021. Backward-compatible prediction updates: A probabilistic approach. 35th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 34, 116–128.","apa":"Träuble, F., Kügelgen, J. von, Kleindessner, M., Locatello, F., Schölkopf, B., &#38; Gehler, P. (2021). Backward-compatible prediction updates: A probabilistic approach. In <i>35th Conference on Neural Information Processing Systems</i> (Vol. 34, pp. 116–128). Virtual.","ama":"Träuble F, Kügelgen J von, Kleindessner M, Locatello F, Schölkopf B, Gehler P. Backward-compatible prediction updates: A probabilistic approach. In: <i>35th Conference on Neural Information Processing Systems</i>. Vol 34. ; 2021:116-128."},"publication_status":"published","date_published":"2021-07-02T00:00:00Z","conference":{"end_date":"2021-12-10","location":"Virtual","name":"NeurIPS: Neural Information Processing Systems","start_date":"2021-12-07"},"extern":"1","year":"2021","volume":34,"date_created":"2023-08-22T14:05:41Z","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","intvolume":"        34","oa_version":"Preprint","oa":1,"status":"public","title":"Backward-compatible prediction updates: A probabilistic approach","quality_controlled":"1"}