[{"date_published":"2023-05-01T00:00:00Z","month":"05","year":"2023","language":[{"iso":"eng"}],"title":"Relative representations enable zero-shot latent space communication","external_id":{"arxiv":["2209.15430"]},"department":[{"_id":"FrLo"}],"main_file_link":[{"open_access":"1","url":"https://arxiv.org/abs/2209.15430"}],"date_created":"2023-08-22T14:22:20Z","conference":{"start_date":"2023-05-01","end_date":"2023-05-05","name":"International Conference on Machine Learning Representations","location":"Kigali, Rwanda"},"type":"conference","publication_status":"published","day":"01","citation":{"chicago":"Moschella, Luca, Valentino Maiorca, Marco Fumero, Antonio Norelli, Francesco Locatello, and Emanuele Rodolà. “Relative Representations Enable Zero-Shot Latent Space Communication.” In <i>The 11th International Conference on Learning Representations</i>, 2023.","apa":"Moschella, L., Maiorca, V., Fumero, M., Norelli, A., Locatello, F., &#38; Rodolà, E. (2023). Relative representations enable zero-shot latent space communication. In <i>The 11th International Conference on Learning Representations</i>. Kigali, Rwanda.","ieee":"L. Moschella, V. Maiorca, M. Fumero, A. Norelli, F. Locatello, and E. Rodolà, “Relative representations enable zero-shot latent space communication,” in <i>The 11th International Conference on Learning Representations</i>, Kigali, Rwanda, 2023.","ista":"Moschella L, Maiorca V, Fumero M, Norelli A, Locatello F, Rodolà E. 2023. Relative representations enable zero-shot latent space communication. The 11th International Conference on Learning Representations. International Conference on Machine Learning Representations.","short":"L. Moschella, V. Maiorca, M. Fumero, A. Norelli, F. Locatello, E. Rodolà, in:, The 11th International Conference on Learning Representations, 2023.","mla":"Moschella, Luca, et al. “Relative Representations Enable Zero-Shot Latent Space Communication.” <i>The 11th International Conference on Learning Representations</i>, 2023.","ama":"Moschella L, Maiorca V, Fumero M, Norelli A, Locatello F, Rodolà E. Relative representations enable zero-shot latent space communication. In: <i>The 11th International Conference on Learning Representations</i>. ; 2023."},"author":[{"last_name":"Moschella","full_name":"Moschella, Luca","first_name":"Luca"},{"first_name":"Valentino","last_name":"Maiorca","full_name":"Maiorca, Valentino"},{"last_name":"Fumero","full_name":"Fumero, Marco","first_name":"Marco"},{"first_name":"Antonio","full_name":"Norelli, Antonio","last_name":"Norelli"},{"first_name":"Francesco","full_name":"Locatello, Francesco","last_name":"Locatello","orcid":"0000-0002-4850-0683","id":"26cfd52f-2483-11ee-8040-88983bcc06d4"},{"full_name":"Rodolà, Emanuele","last_name":"Rodolà","first_name":"Emanuele"}],"status":"public","abstract":[{"lang":"eng","text":"Neural networks embed the geometric structure of a data manifold lying in a high-dimensional space into latent representations. Ideally, the distribution of the data points in the latent space should depend only on the task, the data, the loss, and other architecture-specific constraints. However, factors such as the random weights initialization, training hyperparameters, or other sources of randomness in the training phase may induce incoherent latent spaces that hinder any form of reuse. Nevertheless, we empirically observe that, under the same data and modeling choices, the angles between the encodings within distinct latent spaces do not change. In this work, we propose the latent similarity between each sample and a fixed set of anchors as an alternative data representation, demonstrating that it can enforce the desired invariances without any additional training. We show how neural architectures can leverage these relative representations to guarantee, in practice, invariance to latent isometries and rescalings, effectively enabling latent space communication: from zero-shot model stitching to latent space comparison between diverse settings. We extensively validate the generalization capability of our approach on different datasets, spanning various modalities (images, text, graphs), tasks (e.g., classification, reconstruction) and architectures (e.g., CNNs, GCNs, transformers)."}],"date_updated":"2023-09-13T09:44:26Z","oa":1,"article_processing_charge":"No","arxiv":1,"publication":"The 11th International Conference on Learning Representations","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","oa_version":"Preprint","quality_controlled":"1","_id":"14217","extern":"1"},{"arxiv":1,"publication":"The 11th International Conference on Learning Representations","oa":1,"date_updated":"2023-09-13T11:37:03Z","article_processing_charge":"No","_id":"14218","extern":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","oa_version":"Preprint","quality_controlled":"1","publication_status":"published","day":"10","citation":{"mla":"Seitzer, Maximilian, et al. “Bridging the Gap to Real-World Object-Centric Learning.” <i>The 11th International Conference on Learning Representations</i>, 2023.","ama":"Seitzer M, Horn M, Zadaianchuk A, et al. Bridging the gap to real-world object-centric learning. In: <i>The 11th International Conference on Learning Representations</i>. ; 2023.","short":"M. Seitzer, M. Horn, A. Zadaianchuk, D. Zietlow, T. Xiao, C.-J.S.-G. Carl-Johann Simon-Gabriel, T. He, Z. Zhang, B. Schölkopf, T. Brox, F. Locatello, in:, The 11th International Conference on Learning Representations, 2023.","ista":"Seitzer M, Horn M, Zadaianchuk A, Zietlow D, Xiao T, Carl-Johann Simon-Gabriel C-JS-G, He T, Zhang Z, Schölkopf B, Brox T, Locatello F. 2023. Bridging the gap to real-world object-centric learning. The 11th International Conference on Learning Representations. ICLR: International Conference on Learning Representations.","ieee":"M. Seitzer <i>et al.</i>, “Bridging the gap to real-world object-centric learning,” in <i>The 11th International Conference on Learning Representations</i>, Kigali, Rwanda, 2023.","apa":"Seitzer, M., Horn, M., Zadaianchuk, A., Zietlow, D., Xiao, T., Carl-Johann Simon-Gabriel, C.-J. S.-G., … Locatello, F. (2023). Bridging the gap to real-world object-centric learning. In <i>The 11th International Conference on Learning Representations</i>. Kigali, Rwanda.","chicago":"Seitzer, Maximilian, Max Horn, Andrii Zadaianchuk, Dominik Zietlow, Tianjun Xiao, Carl-Johann Simon-Gabriel Carl-Johann Simon-Gabriel, Tong He, et al. “Bridging the Gap to Real-World Object-Centric Learning.” In <i>The 11th International Conference on Learning Representations</i>, 2023."},"type":"conference","abstract":[{"text":"Humans naturally decompose their environment into entities at the appropriate level of abstraction to act in the world. Allowing machine learning algorithms to derive this decomposition in an unsupervised way has become an important line of research. However, current methods are restricted to simulated data or require additional information in the form of motion or depth in order to successfully discover objects. In this work, we overcome this limitation by showing that reconstructing features from models trained in a self-supervised manner is a sufficient training signal for object-centric representations to arise in a fully unsupervised way. Our approach, DINOSAUR, significantly out-performs existing image-based object-centric learning models on simulated data and is the first unsupervised object-centric model that scales to real-world datasets such as COCO and PASCAL VOC. DINOSAUR is conceptually simple and shows competitive performance compared to more involved pipelines from the computer vision literature.","lang":"eng"}],"status":"public","author":[{"full_name":"Seitzer, Maximilian","last_name":"Seitzer","first_name":"Maximilian"},{"last_name":"Horn","full_name":"Horn, Max","first_name":"Max"},{"last_name":"Zadaianchuk","full_name":"Zadaianchuk, Andrii","first_name":"Andrii"},{"first_name":"Dominik","last_name":"Zietlow","full_name":"Zietlow, Dominik"},{"first_name":"Tianjun","last_name":"Xiao","full_name":"Xiao, Tianjun"},{"first_name":"Carl-Johann Simon-Gabriel","last_name":"Carl-Johann Simon-Gabriel","full_name":"Carl-Johann Simon-Gabriel, Carl-Johann Simon-Gabriel"},{"first_name":"Tong","full_name":"He, Tong","last_name":"He"},{"last_name":"Zhang","full_name":"Zhang, Zheng","first_name":"Zheng"},{"full_name":"Schölkopf, Bernhard","last_name":"Schölkopf","first_name":"Bernhard"},{"first_name":"Thomas","last_name":"Brox","full_name":"Brox, Thomas"},{"orcid":"0000-0002-4850-0683","full_name":"Locatello, Francesco","last_name":"Locatello","first_name":"Francesco","id":"26cfd52f-2483-11ee-8040-88983bcc06d4"}],"department":[{"_id":"FrLo"}],"main_file_link":[{"open_access":"1","url":"https://arxiv.org/abs/2209.14860"}],"conference":{"end_date":"2023-05-05","location":"Kigali, Rwanda","name":"ICLR: International Conference on Learning Representations","start_date":"2023-05-01"},"date_created":"2023-08-22T14:22:41Z","year":"2023","month":"05","date_published":"2023-05-10T00:00:00Z","title":"Bridging the gap to real-world object-centric learning","external_id":{"arxiv":["2209.14860"]},"language":[{"iso":"eng"}]},{"date_published":"2023-05-01T00:00:00Z","year":"2023","month":"05","language":[{"iso":"eng"}],"title":"Unsupervised semantic segmentation with self-supervised object-centric representations","external_id":{"arxiv":["2207.05027"]},"department":[{"_id":"FrLo"}],"main_file_link":[{"open_access":"1","url":"https://arxiv.org/abs/2207.05027"}],"date_created":"2023-08-22T14:22:58Z","conference":{"start_date":"2023-05-01","name":"ICLR: International Conference on Learning Representations","location":"Kigali, Rwanda","end_date":"2023-05-05"},"type":"conference","publication_status":"published","citation":{"short":"A. Zadaianchuk, M. Kleindessner, Y. Zhu, F. Locatello, T. Brox, in:, The 11th International Conference on Learning Representations, 2023.","ista":"Zadaianchuk A, Kleindessner M, Zhu Y, Locatello F, Brox T. 2023. Unsupervised semantic segmentation with self-supervised object-centric representations. The 11th International Conference on Learning Representations. ICLR: International Conference on Learning Representations.","mla":"Zadaianchuk, Andrii, et al. “Unsupervised Semantic Segmentation with Self-Supervised Object-Centric Representations.” <i>The 11th International Conference on Learning Representations</i>, 2023.","ama":"Zadaianchuk A, Kleindessner M, Zhu Y, Locatello F, Brox T. Unsupervised semantic segmentation with self-supervised object-centric representations. In: <i>The 11th International Conference on Learning Representations</i>. ; 2023.","chicago":"Zadaianchuk, Andrii, Matthaeus Kleindessner, Yi Zhu, Francesco Locatello, and Thomas Brox. “Unsupervised Semantic Segmentation with Self-Supervised Object-Centric Representations.” In <i>The 11th International Conference on Learning Representations</i>, 2023.","apa":"Zadaianchuk, A., Kleindessner, M., Zhu, Y., Locatello, F., &#38; Brox, T. (2023). Unsupervised semantic segmentation with self-supervised object-centric representations. In <i>The 11th International Conference on Learning Representations</i>. Kigali, Rwanda.","ieee":"A. Zadaianchuk, M. Kleindessner, Y. Zhu, F. Locatello, and T. Brox, “Unsupervised semantic segmentation with self-supervised object-centric representations,” in <i>The 11th International Conference on Learning Representations</i>, Kigali, Rwanda, 2023."},"day":"01","author":[{"first_name":"Andrii","full_name":"Zadaianchuk, Andrii","last_name":"Zadaianchuk"},{"first_name":"Matthaeus","full_name":"Kleindessner, Matthaeus","last_name":"Kleindessner"},{"first_name":"Yi","full_name":"Zhu, Yi","last_name":"Zhu"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","full_name":"Locatello, Francesco","last_name":"Locatello","orcid":"0000-0002-4850-0683","first_name":"Francesco"},{"first_name":"Thomas","last_name":"Brox","full_name":"Brox, Thomas"}],"status":"public","abstract":[{"text":"In this paper, we show that recent advances in self-supervised feature\r\nlearning enable unsupervised object discovery and semantic segmentation with a\r\nperformance that matches the state of the field on supervised semantic\r\nsegmentation 10 years ago. We propose a methodology based on unsupervised\r\nsaliency masks and self-supervised feature clustering to kickstart object\r\ndiscovery followed by training a semantic segmentation network on pseudo-labels\r\nto bootstrap the system on images with multiple objects. We present results on\r\nPASCAL VOC that go far beyond the current state of the art (50.0 mIoU), and we\r\nreport for the first time results on MS COCO for the whole set of 81 classes:\r\nour method discovers 34 categories with more than $20\\%$ IoU, while obtaining\r\nan average IoU of 19.6 for all 81 categories.","lang":"eng"}],"oa":1,"date_updated":"2023-09-13T11:25:43Z","article_processing_charge":"No","arxiv":1,"publication":"The 11th International Conference on Learning Representations","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","quality_controlled":"1","oa_version":"Preprint","_id":"14219","extern":"1"},{"main_file_link":[{"open_access":"1","url":"https://arxiv.org/abs/2110.06562"}],"article_number":"2110.06562","department":[{"_id":"FrLo"}],"conference":{"start_date":"2023-04-11","end_date":"2023-04-14","location":"Tübingen, Germany","name":"CLeaR: Conference on Causal Learning and Reasoning"},"date_created":"2023-08-22T14:23:54Z","month":"04","year":"2023","date_published":"2023-04-15T00:00:00Z","title":"Unsupervised object learning via common fate","external_id":{"arxiv":["2110.06562"]},"language":[{"iso":"eng"}],"publication":"2nd Conference on Causal Learning and Reasoning","arxiv":1,"article_processing_charge":"No","oa":1,"date_updated":"2023-09-13T11:31:14Z","extern":"1","_id":"14222","quality_controlled":"1","oa_version":"Preprint","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","day":"15","citation":{"short":"M. Tangemann, S. Schneider, J. von Kügelgen, F. Locatello, P. Gehler, T. Brox, M. Kümmerer, M. Bethge, B. Schölkopf, in:, 2nd Conference on Causal Learning and Reasoning, 2023.","ista":"Tangemann M, Schneider S, Kügelgen J von, Locatello F, Gehler P, Brox T, Kümmerer M, Bethge M, Schölkopf B. 2023. Unsupervised object learning via common fate. 2nd Conference on Causal Learning and Reasoning. CLeaR: Conference on Causal Learning and Reasoning, 2110.06562.","mla":"Tangemann, Matthias, et al. “Unsupervised Object Learning via Common Fate.” <i>2nd Conference on Causal Learning and Reasoning</i>, 2110.06562, 2023.","ama":"Tangemann M, Schneider S, Kügelgen J von, et al. Unsupervised object learning via common fate. In: <i>2nd Conference on Causal Learning and Reasoning</i>. ; 2023.","chicago":"Tangemann, Matthias, Steffen Schneider, Julius von Kügelgen, Francesco Locatello, Peter Gehler, Thomas Brox, Matthias Kümmerer, Matthias Bethge, and Bernhard Schölkopf. “Unsupervised Object Learning via Common Fate.” In <i>2nd Conference on Causal Learning and Reasoning</i>, 2023.","apa":"Tangemann, M., Schneider, S., Kügelgen, J. von, Locatello, F., Gehler, P., Brox, T., … Schölkopf, B. (2023). Unsupervised object learning via common fate. In <i>2nd Conference on Causal Learning and Reasoning</i>. Tübingen, Germany.","ieee":"M. Tangemann <i>et al.</i>, “Unsupervised object learning via common fate,” in <i>2nd Conference on Causal Learning and Reasoning</i>, Tübingen, Germany, 2023."},"publication_status":"published","type":"conference","abstract":[{"text":"Learning generative object models from unlabelled videos is a long standing problem and required for causal scene modeling. We decompose this problem into three easier subtasks, and provide candidate solutions for each of them. Inspired by the Common Fate Principle of Gestalt Psychology, we first extract (noisy) masks of moving objects via unsupervised motion segmentation. Second, generative models are trained on the masks of the background and the moving objects, respectively. Third, background and foreground models are combined in a conditional \"dead leaves\" scene model to sample novel scene configurations where occlusions and depth layering arise naturally. To evaluate the individual stages, we introduce the Fishbowl dataset positioned between complex real-world scenes and common object-centric benchmarks of simplistic objects. We show that our approach allows learning generative models that generalize beyond the occlusions present in the input videos, and represent scenes in a modular fashion that allows sampling plausible scenes outside the training distribution by permitting, for instance, object numbers or densities not observed in the training set.","lang":"eng"}],"status":"public","author":[{"last_name":"Tangemann","full_name":"Tangemann, Matthias","first_name":"Matthias"},{"first_name":"Steffen","full_name":"Schneider, Steffen","last_name":"Schneider"},{"last_name":"Kügelgen","full_name":"Kügelgen, Julius von","first_name":"Julius von"},{"orcid":"0000-0002-4850-0683","last_name":"Locatello","full_name":"Locatello, Francesco","first_name":"Francesco","id":"26cfd52f-2483-11ee-8040-88983bcc06d4"},{"last_name":"Gehler","full_name":"Gehler, Peter","first_name":"Peter"},{"last_name":"Brox","full_name":"Brox, Thomas","first_name":"Thomas"},{"first_name":"Matthias","last_name":"Kümmerer","full_name":"Kümmerer, Matthias"},{"first_name":"Matthias","last_name":"Bethge","full_name":"Bethge, Matthias"},{"last_name":"Schölkopf","full_name":"Schölkopf, Bernhard","first_name":"Bernhard"}]},{"language":[{"iso":"eng"}],"scopus_import":"1","publisher":"ML Research Press","date_published":"2022-04-01T00:00:00Z","month":"04","date_created":"2023-08-21T09:27:43Z","conference":{"start_date":"2022-03-28","location":"Virtual","end_date":"2022-03-30","name":"AISTATS: Conference on Artificial Intelligence and Statistics"},"department":[{"_id":"FrLo"}],"status":"public","intvolume":"       151","type":"conference","day":"01","page":"8439-8457","publication":"Proceedings of the 25th International Conference on Artificial Intelligence and Statistics","title":" Faster one-sample stochastic conditional gradient method for composite convex minimization","external_id":{"arxiv":["2202.13212"]},"year":"2022","main_file_link":[{"open_access":"1","url":"https://arxiv.org/abs/2202.13212"}],"alternative_title":["PMLR"],"author":[{"first_name":"Gideon","last_name":"Dresdner","full_name":"Dresdner, Gideon"},{"first_name":"Maria-Luiza","full_name":"Vladarean, Maria-Luiza","last_name":"Vladarean"},{"last_name":"Rätsch","full_name":"Rätsch, Gunnar","first_name":"Gunnar"},{"orcid":"0000-0002-4850-0683","last_name":"Locatello","full_name":"Locatello, Francesco","first_name":"Francesco","id":"26cfd52f-2483-11ee-8040-88983bcc06d4"},{"last_name":"Cevher","full_name":"Cevher, Volkan","first_name":"Volkan"},{"first_name":"Alp","full_name":"Yurtsever, Alp","last_name":"Yurtsever"}],"abstract":[{"text":" We propose a stochastic conditional gradient method (CGM) for minimizing convex finite-sum objectives formed as a sum of smooth and non-smooth terms. Existing CGM variants for this template either suffer from slow convergence rates, or require carefully increasing the batch size over the course of the algorithm’s execution, which leads to computing full gradients. In contrast, the proposed method, equipped with a stochastic average gradient (SAG) estimator, requires only one sample per iteration. Nevertheless, it guarantees fast convergence rates on par with more sophisticated variance reduction techniques. In applications we put special emphasis on problems with a large number of separable constraints. Such problems are prevalent among semidefinite programming (SDP) formulations arising in machine learning and theoretical computer science. We provide numerical experiments on matrix completion, unsupervised clustering, and sparsest-cut SDPs. ","lang":"eng"}],"citation":{"short":"G. Dresdner, M.-L. Vladarean, G. Rätsch, F. Locatello, V. Cevher, A. Yurtsever, in:, Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, ML Research Press, 2022, pp. 8439–8457.","ista":"Dresdner G, Vladarean M-L, Rätsch G, Locatello F, Cevher V, Yurtsever A. 2022.  Faster one-sample stochastic conditional gradient method for composite convex minimization. Proceedings of the 25th International Conference on Artificial Intelligence and Statistics. AISTATS: Conference on Artificial Intelligence and Statistics, PMLR, vol. 151, 8439–8457.","mla":"Dresdner, Gideon, et al. “ Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization.” <i>Proceedings of the 25th International Conference on Artificial Intelligence and Statistics</i>, vol. 151, ML Research Press, 2022, pp. 8439–57.","ama":"Dresdner G, Vladarean M-L, Rätsch G, Locatello F, Cevher V, Yurtsever A.  Faster one-sample stochastic conditional gradient method for composite convex minimization. In: <i>Proceedings of the 25th International Conference on Artificial Intelligence and Statistics</i>. Vol 151. ML Research Press; 2022:8439-8457.","chicago":"Dresdner, Gideon, Maria-Luiza Vladarean, Gunnar Rätsch, Francesco Locatello, Volkan Cevher, and Alp Yurtsever. “ Faster One-Sample Stochastic Conditional Gradient Method for Composite Convex Minimization.” In <i>Proceedings of the 25th International Conference on Artificial Intelligence and Statistics</i>, 151:8439–57. ML Research Press, 2022.","apa":"Dresdner, G., Vladarean, M.-L., Rätsch, G., Locatello, F., Cevher, V., &#38; Yurtsever, A. (2022).  Faster one-sample stochastic conditional gradient method for composite convex minimization. In <i>Proceedings of the 25th International Conference on Artificial Intelligence and Statistics</i> (Vol. 151, pp. 8439–8457). Virtual: ML Research Press.","ieee":"G. Dresdner, M.-L. Vladarean, G. Rätsch, F. Locatello, V. Cevher, and A. Yurtsever, “ Faster one-sample stochastic conditional gradient method for composite convex minimization,” in <i>Proceedings of the 25th International Conference on Artificial Intelligence and Statistics</i>, Virtual, 2022, vol. 151, pp. 8439–8457."},"publication_status":"published","quality_controlled":"1","oa_version":"Preprint","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","publication_identifier":{"issn":["2640-3498"]},"extern":"1","_id":"14093","article_processing_charge":"No","oa":1,"volume":151,"date_updated":"2023-09-06T10:28:17Z","arxiv":1},{"status":"public","intvolume":"        35","type":"conference","day":"15","page":"16548-16562","publication":"36th Conference on Neural Information Processing Systems","language":[{"iso":"eng"}],"scopus_import":"1","publisher":"Neural Information Processing Systems Foundation","date_published":"2022-12-15T00:00:00Z","month":"12","date_created":"2023-08-21T12:12:42Z","conference":{"start_date":"2022-11-28","name":"NeurIPS: Neural Information Processing Systems","end_date":"2022-12-09","location":"New Orleans, LA, United States"},"department":[{"_id":"FrLo"}],"author":[{"first_name":"Michael","last_name":"Lohaus","full_name":"Lohaus, Michael"},{"full_name":"Kleindessner, Matthäus","last_name":"Kleindessner","first_name":"Matthäus"},{"last_name":"Kenthapadi","full_name":"Kenthapadi, Krishnaram","first_name":"Krishnaram"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","first_name":"Francesco","last_name":"Locatello","full_name":"Locatello, Francesco","orcid":"0000-0002-4850-0683"},{"last_name":"Russell","full_name":"Russell, Chris","first_name":"Chris"}],"abstract":[{"lang":"eng","text":"We show that deep networks trained to satisfy demographic parity often do so\r\nthrough a form of race or gender awareness, and that the more we force a network\r\nto be fair, the more accurately we can recover race or gender from the internal state\r\nof the network. Based on this observation, we investigate an alternative fairness\r\napproach: we add a second classification head to the network to explicitly predict\r\nthe protected attribute (such as race or gender) alongside the original task. After\r\ntraining the two-headed network, we enforce demographic parity by merging the\r\ntwo heads, creating a network with the same architecture as the original network.\r\nWe establish a close relationship between existing approaches and our approach\r\nby showing (1) that the decisions of a fair classifier are well-approximated by our\r\napproach, and (2) that an unfair and optimally accurate classifier can be recovered\r\nfrom a fair classifier and our second head predicting the protected attribute. We use\r\nour explicit formulation to argue that the existing fairness approaches, just as ours,\r\ndemonstrate disparate treatment and that they are likely to be unlawful in a wide\r\nrange of scenarios under US law."}],"citation":{"mla":"Lohaus, Michael, et al. “Are Two Heads the Same as One? Identifying Disparate Treatment in Fair Neural Networks.” <i>36th Conference on Neural Information Processing Systems</i>, vol. 35, Neural Information Processing Systems Foundation, 2022, pp. 16548–62.","ama":"Lohaus M, Kleindessner M, Kenthapadi K, Locatello F, Russell C. Are two heads the same as one? Identifying disparate treatment in fair neural networks. In: <i>36th Conference on Neural Information Processing Systems</i>. Vol 35. Neural Information Processing Systems Foundation; 2022:16548-16562.","short":"M. Lohaus, M. Kleindessner, K. Kenthapadi, F. Locatello, C. Russell, in:, 36th Conference on Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2022, pp. 16548–16562.","ista":"Lohaus M, Kleindessner M, Kenthapadi K, Locatello F, Russell C. 2022. Are two heads the same as one? Identifying disparate treatment in fair neural networks. 36th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems, Advances in Neural Information Processing Systems, vol. 35, 16548–16562.","ieee":"M. Lohaus, M. Kleindessner, K. Kenthapadi, F. Locatello, and C. Russell, “Are two heads the same as one? Identifying disparate treatment in fair neural networks,” in <i>36th Conference on Neural Information Processing Systems</i>, New Orleans, LA, United States, 2022, vol. 35, pp. 16548–16562.","apa":"Lohaus, M., Kleindessner, M., Kenthapadi, K., Locatello, F., &#38; Russell, C. (2022). Are two heads the same as one? Identifying disparate treatment in fair neural networks. In <i>36th Conference on Neural Information Processing Systems</i> (Vol. 35, pp. 16548–16562). New Orleans, LA, United States: Neural Information Processing Systems Foundation.","chicago":"Lohaus, Michael, Matthäus Kleindessner, Krishnaram Kenthapadi, Francesco Locatello, and Chris Russell. “Are Two Heads the Same as One? Identifying Disparate Treatment in Fair Neural Networks.” In <i>36th Conference on Neural Information Processing Systems</i>, 35:16548–62. Neural Information Processing Systems Foundation, 2022."},"publication_status":"published","oa_version":"Preprint","quality_controlled":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","extern":"1","publication_identifier":{"isbn":["9781713871088"]},"_id":"14106","article_processing_charge":"No","date_updated":"2023-09-06T10:29:42Z","volume":35,"oa":1,"arxiv":1,"external_id":{"arxiv":["2204.04440"]},"title":"Are two heads the same as one? Identifying disparate treatment in fair neural networks","year":"2022","main_file_link":[{"url":"https://arxiv.org/abs/2204.04440","open_access":"1"}],"alternative_title":["Advances in Neural Information Processing Systems"]},{"abstract":[{"text":"Amodal perception requires inferring the full shape of an object that is partially occluded. This task is particularly challenging on two levels: (1) it requires more information than what is contained in the instant retina or imaging sensor, (2) it is difficult to obtain enough well-annotated amodal labels for supervision. To this end, this paper develops a new framework of\r\nSelf-supervised amodal Video object segmentation (SaVos). Our method efficiently leverages the visual information of video temporal sequences to infer the amodal mask of objects. The key intuition is that the occluded part of an object can be explained away if that part is visible in other frames, possibly deformed as long as the deformation can be reasonably learned.\r\nAccordingly, we derive a novel self-supervised learning paradigm that efficiently utilizes the visible object parts as the supervision to guide the training on videos. In addition to learning type prior to complete masks for known types, SaVos also learns the spatiotemporal prior, which is also useful for the amodal task and could generalize to unseen types. The proposed\r\nframework achieves the state-of-the-art performance on the synthetic amodal segmentation benchmark FISHBOWL and the real world benchmark KINS-Video-Car. Further, it lends itself well to being transferred to novel distributions using test-time adaptation, outperforming existing models even after the transfer to a new distribution.","lang":"eng"}],"status":"public","author":[{"full_name":"Yao, Jian","last_name":"Yao","first_name":"Jian"},{"first_name":"Yuxin","full_name":"Hong, Yuxin","last_name":"Hong"},{"first_name":"Chiyu","full_name":"Wang, Chiyu","last_name":"Wang"},{"first_name":"Tianjun","full_name":"Xiao, Tianjun","last_name":"Xiao"},{"last_name":"He","full_name":"He, Tong","first_name":"Tong"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","orcid":"0000-0002-4850-0683","last_name":"Locatello","full_name":"Locatello, Francesco","first_name":"Francesco"},{"first_name":"David","full_name":"Wipf, David","last_name":"Wipf"},{"first_name":"Yanwei","full_name":"Fu, Yanwei","last_name":"Fu"},{"last_name":"Zhang","full_name":"Zhang, Zheng","first_name":"Zheng"}],"publication_status":"published","citation":{"apa":"Yao, J., Hong, Y., Wang, C., Xiao, T., He, T., Locatello, F., … Zhang, Z. (2022). Self-supervised amodal video object segmentation. In <i>36th Conference on Neural Information Processing Systems</i>. New Orleans, LA, United States. <a href=\"https://doi.org/10.48550/arXiv.2210.12733\">https://doi.org/10.48550/arXiv.2210.12733</a>","ieee":"J. Yao <i>et al.</i>, “Self-supervised amodal video object segmentation,” in <i>36th Conference on Neural Information Processing Systems</i>, New Orleans, LA, United States, 2022.","chicago":"Yao, Jian, Yuxin Hong, Chiyu Wang, Tianjun Xiao, Tong He, Francesco Locatello, David Wipf, Yanwei Fu, and Zheng Zhang. “Self-Supervised Amodal Video Object Segmentation.” In <i>36th Conference on Neural Information Processing Systems</i>, 2022. <a href=\"https://doi.org/10.48550/arXiv.2210.12733\">https://doi.org/10.48550/arXiv.2210.12733</a>.","mla":"Yao, Jian, et al. “Self-Supervised Amodal Video Object Segmentation.” <i>36th Conference on Neural Information Processing Systems</i>, 2022, doi:<a href=\"https://doi.org/10.48550/arXiv.2210.12733\">10.48550/arXiv.2210.12733</a>.","ama":"Yao J, Hong Y, Wang C, et al. Self-supervised amodal video object segmentation. In: <i>36th Conference on Neural Information Processing Systems</i>. ; 2022. doi:<a href=\"https://doi.org/10.48550/arXiv.2210.12733\">10.48550/arXiv.2210.12733</a>","ista":"Yao J, Hong Y, Wang C, Xiao T, He T, Locatello F, Wipf D, Fu Y, Zhang Z. 2022. Self-supervised amodal video object segmentation. 36th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems.","short":"J. Yao, Y. Hong, C. Wang, T. Xiao, T. He, F. Locatello, D. Wipf, Y. Fu, Z. Zhang, in:, 36th Conference on Neural Information Processing Systems, 2022."},"day":"23","type":"conference","_id":"14107","extern":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","oa_version":"Preprint","arxiv":1,"publication":"36th Conference on Neural Information Processing Systems","date_updated":"2023-09-11T09:34:17Z","oa":1,"article_processing_charge":"No","title":"Self-supervised amodal video object segmentation","external_id":{"arxiv":["2210.12733"]},"language":[{"iso":"eng"}],"doi":"10.48550/arXiv.2210.12733","year":"2022","month":"10","date_published":"2022-10-23T00:00:00Z","conference":{"location":"New Orleans, LA, United States","end_date":"2022-12-01","name":"NeurIPS: Neural Information Processing Systems","start_date":"2022-11-28"},"date_created":"2023-08-21T12:13:25Z","department":[{"_id":"FrLo"}],"main_file_link":[{"open_access":"1","url":"https://doi.org/10.48550/arXiv.2210.12733"}]},{"day":"01","type":"conference","status":"public","publication":"2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition","page":"10400-10411","month":"07","date_published":"2022-07-01T00:00:00Z","publisher":"Institute of Electrical and Electronics Engineers","scopus_import":"1","language":[{"iso":"eng"}],"department":[{"_id":"FrLo"}],"conference":{"name":"CVPR: Conference on Computer Vision and Pattern Recognition","location":"New Orleans, LA, United States","end_date":"2022-06-24","start_date":"2022-06-18"},"date_created":"2023-08-21T12:18:00Z","publication_status":"published","citation":{"short":"D. Zietlow, M. Lohaus, G. Balakrishnan, M. Kleindessner, F. Locatello, B. Scholkopf, C. Russell, in:, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Institute of Electrical and Electronics Engineers, 2022, pp. 10400–10411.","ista":"Zietlow D, Lohaus M, Balakrishnan G, Kleindessner M, Locatello F, Scholkopf B, Russell C. 2022. Leveling down in computer vision: Pareto inefficiencies in fair deep classifiers. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR: Conference on Computer Vision and Pattern Recognition, 10400–10411.","ama":"Zietlow D, Lohaus M, Balakrishnan G, et al. Leveling down in computer vision: Pareto inefficiencies in fair deep classifiers. In: <i>2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition</i>. Institute of Electrical and Electronics Engineers; 2022:10400-10411. doi:<a href=\"https://doi.org/10.1109/cvpr52688.2022.01016\">10.1109/cvpr52688.2022.01016</a>","mla":"Zietlow, Dominik, et al. “Leveling down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers.” <i>2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition</i>, Institute of Electrical and Electronics Engineers, 2022, pp. 10400–11, doi:<a href=\"https://doi.org/10.1109/cvpr52688.2022.01016\">10.1109/cvpr52688.2022.01016</a>.","chicago":"Zietlow, Dominik, Michael Lohaus, Guha Balakrishnan, Matthaus Kleindessner, Francesco Locatello, Bernhard Scholkopf, and Chris Russell. “Leveling down in Computer Vision: Pareto Inefficiencies in Fair Deep Classifiers.” In <i>2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition</i>, 10400–411. Institute of Electrical and Electronics Engineers, 2022. <a href=\"https://doi.org/10.1109/cvpr52688.2022.01016\">https://doi.org/10.1109/cvpr52688.2022.01016</a>.","ieee":"D. Zietlow <i>et al.</i>, “Leveling down in computer vision: Pareto inefficiencies in fair deep classifiers,” in <i>2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition</i>, New Orleans, LA, United States, 2022, pp. 10400–10411.","apa":"Zietlow, D., Lohaus, M., Balakrishnan, G., Kleindessner, M., Locatello, F., Scholkopf, B., &#38; Russell, C. (2022). Leveling down in computer vision: Pareto inefficiencies in fair deep classifiers. In <i>2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition</i> (pp. 10400–10411). New Orleans, LA, United States: Institute of Electrical and Electronics Engineers. <a href=\"https://doi.org/10.1109/cvpr52688.2022.01016\">https://doi.org/10.1109/cvpr52688.2022.01016</a>"},"abstract":[{"text":"Algorithmic fairness is frequently motivated in terms of a trade-off in which overall performance is decreased so as to improve performance on disadvantaged groups where the algorithm would otherwise be less accurate. Contrary to this, we find that applying existing fairness approaches to computer vision improve fairness by degrading the performance of classifiers across all groups (with increased degradation on the best performing groups). Extending the bias-variance decomposition for classification to fairness, we theoretically explain why the majority of fairness methods designed for low capacity models should not be used in settings involving high-capacity models, a scenario common to computer vision. We corroborate this analysis with extensive experimental support that shows that many of the fairness heuristics used in computer vision also degrade performance on the most disadvantaged groups. Building on these insights, we propose an adaptive augmentation strategy that, uniquely, of all methods tested, improves performance for the disadvantaged groups.","lang":"eng"}],"author":[{"last_name":"Zietlow","full_name":"Zietlow, Dominik","first_name":"Dominik"},{"first_name":"Michael","full_name":"Lohaus, Michael","last_name":"Lohaus"},{"full_name":"Balakrishnan, Guha","last_name":"Balakrishnan","first_name":"Guha"},{"full_name":"Kleindessner, Matthaus","last_name":"Kleindessner","first_name":"Matthaus"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","full_name":"Locatello, Francesco","last_name":"Locatello","orcid":"0000-0002-4850-0683","first_name":"Francesco"},{"first_name":"Bernhard","full_name":"Scholkopf, Bernhard","last_name":"Scholkopf"},{"first_name":"Chris","full_name":"Russell, Chris","last_name":"Russell"}],"arxiv":1,"oa":1,"date_updated":"2023-09-11T09:19:14Z","article_processing_charge":"No","_id":"14114","extern":"1","publication_identifier":{"issn":["1063-6919"],"isbn":["9781665469470"],"eissn":["2575-7075"]},"user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","oa_version":"Preprint","quality_controlled":"1","year":"2022","doi":"10.1109/cvpr52688.2022.01016","title":"Leveling down in computer vision: Pareto inefficiencies in fair deep classifiers","external_id":{"arxiv":["2203.04913"]},"main_file_link":[{"url":"https://arxiv.org/abs/2203.04913","open_access":"1"}]},{"month":"10","year":"2022","date_published":"2022-10-14T00:00:00Z","title":"Neural attentive circuits","external_id":{"arxiv":["2210.08031"]},"language":[{"iso":"eng"}],"alternative_title":[" Advances in Neural Information Processing Systems"],"department":[{"_id":"FrLo"}],"main_file_link":[{"open_access":"1","url":"https://doi.org/10.48550/arXiv.2210.08031"}],"conference":{"name":"NeurIPS: Neural Information Processing Systems","end_date":"2022-12-01","location":"New Orleans, United States","start_date":"2022-11-29"},"date_created":"2023-08-22T13:57:27Z","publication_status":"published","day":"14","citation":{"ieee":"N. Rahaman <i>et al.</i>, “Neural attentive circuits,” in <i>36th Conference on Neural Information Processing Systems</i>, New Orleans, United States, 2022, vol. 35.","apa":"Rahaman, N., Weiss, M., Locatello, F., Pal, C., Bengio, Y., Schölkopf, B., … Ballas, N. (2022). Neural attentive circuits. In <i>36th Conference on Neural Information Processing Systems</i> (Vol. 35). New Orleans, United States.","chicago":"Rahaman, Nasim, Martin Weiss, Francesco Locatello, Chris Pal, Yoshua Bengio, Bernhard Schölkopf, Li Erran Li, and Nicolas Ballas. “Neural Attentive Circuits.” In <i>36th Conference on Neural Information Processing Systems</i>, Vol. 35, 2022.","mla":"Rahaman, Nasim, et al. “Neural Attentive Circuits.” <i>36th Conference on Neural Information Processing Systems</i>, vol. 35, 2022.","ama":"Rahaman N, Weiss M, Locatello F, et al. Neural attentive circuits. In: <i>36th Conference on Neural Information Processing Systems</i>. Vol 35. ; 2022.","short":"N. Rahaman, M. Weiss, F. Locatello, C. Pal, Y. Bengio, B. Schölkopf, L.E. Li, N. Ballas, in:, 36th Conference on Neural Information Processing Systems, 2022.","ista":"Rahaman N, Weiss M, Locatello F, Pal C, Bengio Y, Schölkopf B, Li LE, Ballas N. 2022. Neural attentive circuits. 36th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems,  Advances in Neural Information Processing Systems, vol. 35."},"type":"conference","intvolume":"        35","abstract":[{"text":"Recent work has seen the development of general purpose neural architectures\r\nthat can be trained to perform tasks across diverse data modalities. General\r\npurpose models typically make few assumptions about the underlying\r\ndata-structure and are known to perform well in the large-data regime. At the\r\nsame time, there has been growing interest in modular neural architectures that\r\nrepresent the data using sparsely interacting modules. These models can be more\r\nrobust out-of-distribution, computationally efficient, and capable of\r\nsample-efficient adaptation to new data. However, they tend to make\r\ndomain-specific assumptions about the data, and present challenges in how\r\nmodule behavior (i.e., parameterization) and connectivity (i.e., their layout)\r\ncan be jointly learned. In this work, we introduce a general purpose, yet\r\nmodular neural architecture called Neural Attentive Circuits (NACs) that\r\njointly learns the parameterization and a sparse connectivity of neural modules\r\nwithout using domain knowledge. NACs are best understood as the combination of\r\ntwo systems that are jointly trained end-to-end: one that determines the module\r\nconfiguration and the other that executes it on an input. We demonstrate\r\nqualitatively that NACs learn diverse and meaningful module configurations on\r\nthe NLVR2 dataset without additional supervision. Quantitatively, we show that\r\nby incorporating modularity in this way, NACs improve upon a strong non-modular\r\nbaseline in terms of low-shot adaptation on CIFAR and CUBs dataset by about\r\n10%, and OOD robustness on Tiny ImageNet-R by about 2.5%. Further, we find that\r\nNACs can achieve an 8x speedup at inference time while losing less than 3%\r\nperformance. Finally, we find NACs to yield competitive results on diverse data\r\nmodalities spanning point-cloud classification, symbolic processing and\r\ntext-classification from ASCII bytes, thereby confirming its general purpose\r\nnature.","lang":"eng"}],"status":"public","author":[{"first_name":"Nasim","full_name":"Rahaman, Nasim","last_name":"Rahaman"},{"first_name":"Martin","last_name":"Weiss","full_name":"Weiss, Martin"},{"first_name":"Francesco","orcid":"0000-0002-4850-0683","full_name":"Locatello, Francesco","last_name":"Locatello","id":"26cfd52f-2483-11ee-8040-88983bcc06d4"},{"first_name":"Chris","full_name":"Pal, Chris","last_name":"Pal"},{"first_name":"Yoshua","last_name":"Bengio","full_name":"Bengio, Yoshua"},{"last_name":"Schölkopf","full_name":"Schölkopf, Bernhard","first_name":"Bernhard"},{"last_name":"Li","full_name":"Li, Li Erran","first_name":"Li Erran"},{"last_name":"Ballas","full_name":"Ballas, Nicolas","first_name":"Nicolas"}],"arxiv":1,"publication":"36th Conference on Neural Information Processing Systems","volume":35,"oa":1,"date_updated":"2023-09-11T09:29:09Z","article_processing_charge":"No","_id":"14168","extern":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","oa_version":"Preprint"},{"alternative_title":["PMLR"],"main_file_link":[{"url":"https://arxiv.org/abs/2107.00637","open_access":"1"}],"title":"Generalization and robustness implications in object-centric learning","external_id":{"arxiv":["2107.00637"]},"year":"2022","extern":"1","_id":"14170","oa_version":"Preprint","quality_controlled":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","arxiv":1,"article_processing_charge":"No","date_updated":"2023-09-11T10:08:14Z","oa":1,"volume":2022,"abstract":[{"text":"The idea behind object-centric representation learning is that natural scenes can better be modeled as compositions of objects and their relations as opposed to distributed representations. This inductive bias can be injected into neural networks to potentially improve systematic generalization and performance of downstream tasks in scenes with multiple objects. In this paper, we train state-of-the-art unsupervised models on five common multi-object datasets and evaluate segmentation metrics and downstream object property prediction. In addition, we study generalization and robustness by investigating the settings where either a single object is out of distribution -- e.g., having an unseen color, texture, or shape -- or global properties of the scene are altered -- e.g., by occlusions, cropping, or increasing the number of objects. From our experimental study, we find object-centric representations to be useful for\r\ndownstream tasks and generally robust to most distribution shifts affecting objects. However, when the distribution shift affects the input in a less structured manner, robustness in terms of segmentation and downstream task performance may vary significantly across models and distribution shifts. ","lang":"eng"}],"author":[{"first_name":"Andrea","full_name":"Dittadi, Andrea","last_name":"Dittadi"},{"first_name":"Samuele","full_name":"Papa, Samuele","last_name":"Papa"},{"full_name":"Vita, Michele De","last_name":"Vita","first_name":"Michele De"},{"first_name":"Bernhard","last_name":"Schölkopf","full_name":"Schölkopf, Bernhard"},{"full_name":"Winther, Ole","last_name":"Winther","first_name":"Ole"},{"first_name":"Francesco","full_name":"Locatello, Francesco","last_name":"Locatello","orcid":"0000-0002-4850-0683","id":"26cfd52f-2483-11ee-8040-88983bcc06d4"}],"citation":{"mla":"Dittadi, Andrea, et al. “Generalization and Robustness Implications in Object-Centric Learning.” <i>Proceedings of the 39th International Conference on Machine Learning</i>, vol. 2022, ML Research Press, pp. 5221–85.","ama":"Dittadi A, Papa S, Vita MD, Schölkopf B, Winther O, Locatello F. Generalization and robustness implications in object-centric learning. In: <i>Proceedings of the 39th International Conference on Machine Learning</i>. Vol 2022. ML Research Press; :5221-5285.","ista":"Dittadi A, Papa S, Vita MD, Schölkopf B, Winther O, Locatello F. Generalization and robustness implications in object-centric learning. Proceedings of the 39th International Conference on Machine Learning. International Conference on Machine Learning, PMLR, vol. 2022, 5221–5285.","short":"A. Dittadi, S. Papa, M.D. Vita, B. Schölkopf, O. Winther, F. Locatello, in:, Proceedings of the 39th International Conference on Machine Learning, ML Research Press, n.d., pp. 5221–5285.","ieee":"A. Dittadi, S. Papa, M. D. Vita, B. Schölkopf, O. Winther, and F. Locatello, “Generalization and robustness implications in object-centric learning,” in <i>Proceedings of the 39th International Conference on Machine Learning</i>, Baltimore, MD, United States, vol. 2022, pp. 5221–5285.","apa":"Dittadi, A., Papa, S., Vita, M. D., Schölkopf, B., Winther, O., &#38; Locatello, F. (n.d.). Generalization and robustness implications in object-centric learning. In <i>Proceedings of the 39th International Conference on Machine Learning</i> (Vol. 2022, pp. 5221–5285). Baltimore, MD, United States: ML Research Press.","chicago":"Dittadi, Andrea, Samuele Papa, Michele De Vita, Bernhard Schölkopf, Ole Winther, and Francesco Locatello. “Generalization and Robustness Implications in Object-Centric Learning.” In <i>Proceedings of the 39th International Conference on Machine Learning</i>, 2022:5221–85. ML Research Press, n.d."},"publication_status":"submitted","conference":{"location":"Baltimore, MD, United States","end_date":"2022-07-23","name":"International Conference on Machine Learning","start_date":"2022-07-17"},"date_created":"2023-08-22T13:59:55Z","department":[{"_id":"FrLo"}],"publisher":"ML Research Press","language":[{"iso":"eng"}],"month":"07","date_published":"2022-07-22T00:00:00Z","publication":"Proceedings of the 39th International Conference on Machine Learning","page":"5221-5285","intvolume":"      2022","status":"public","day":"22","type":"conference"},{"abstract":[{"lang":"eng","text":"This paper demonstrates how to recover causal graphs from the score of the\r\ndata distribution in non-linear additive (Gaussian) noise models. Using score\r\nmatching algorithms as a building block, we show how to design a new generation\r\nof scalable causal discovery methods. To showcase our approach, we also propose\r\na new efficient method for approximating the score's Jacobian, enabling to\r\nrecover the causal graph. Empirically, we find that the new algorithm, called\r\nSCORE, is competitive with state-of-the-art causal discovery methods while\r\nbeing significantly faster."}],"author":[{"last_name":"Rolland","full_name":"Rolland, Paul","first_name":"Paul"},{"last_name":"Cevher","full_name":"Cevher, Volkan","first_name":"Volkan"},{"last_name":"Kleindessner","full_name":"Kleindessner, Matthäus","first_name":"Matthäus"},{"first_name":"Chris","last_name":"Russel","full_name":"Russel, Chris"},{"first_name":"Bernhard","last_name":"Schölkopf","full_name":"Schölkopf, Bernhard"},{"full_name":"Janzing, Dominik","last_name":"Janzing","first_name":"Dominik"},{"last_name":"Locatello","full_name":"Locatello, Francesco","orcid":"0000-0002-4850-0683","first_name":"Francesco","id":"26cfd52f-2483-11ee-8040-88983bcc06d4"}],"publication_status":"published","citation":{"short":"P. Rolland, V. Cevher, M. Kleindessner, C. Russel, B. Schölkopf, D. Janzing, F. Locatello, in:, Proceedings of the 39th International Conference on Machine Learning, ML Research Press, 2022, pp. 18741–18753.","ista":"Rolland P, Cevher V, Kleindessner M, Russel C, Schölkopf B, Janzing D, Locatello F. 2022. Score matching enables causal discovery of nonlinear additive noise  models. Proceedings of the 39th International Conference on Machine Learning. International Conference on Machine Learning, PMLR, vol. 162, 18741–18753.","mla":"Rolland, Paul, et al. “Score Matching Enables Causal Discovery of Nonlinear Additive Noise  Models.” <i>Proceedings of the 39th International Conference on Machine Learning</i>, vol. 162, ML Research Press, 2022, pp. 18741–53.","ama":"Rolland P, Cevher V, Kleindessner M, et al. Score matching enables causal discovery of nonlinear additive noise  models. In: <i>Proceedings of the 39th International Conference on Machine Learning</i>. Vol 162. ML Research Press; 2022:18741-18753.","chicago":"Rolland, Paul, Volkan Cevher, Matthäus Kleindessner, Chris Russel, Bernhard Schölkopf, Dominik Janzing, and Francesco Locatello. “Score Matching Enables Causal Discovery of Nonlinear Additive Noise  Models.” In <i>Proceedings of the 39th International Conference on Machine Learning</i>, 162:18741–53. ML Research Press, 2022.","ieee":"P. Rolland <i>et al.</i>, “Score matching enables causal discovery of nonlinear additive noise  models,” in <i>Proceedings of the 39th International Conference on Machine Learning</i>, Baltimore, MD, United States, 2022, vol. 162, pp. 18741–18753.","apa":"Rolland, P., Cevher, V., Kleindessner, M., Russel, C., Schölkopf, B., Janzing, D., &#38; Locatello, F. (2022). Score matching enables causal discovery of nonlinear additive noise  models. In <i>Proceedings of the 39th International Conference on Machine Learning</i> (Vol. 162, pp. 18741–18753). Baltimore, MD, United States: ML Research Press."},"_id":"14171","extern":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","quality_controlled":"1","oa_version":"Preprint","arxiv":1,"volume":162,"date_updated":"2023-09-11T10:14:20Z","oa":1,"article_processing_charge":"No","external_id":{"arxiv":["2203.04413"]},"title":"Score matching enables causal discovery of nonlinear additive noise  models","year":"2022","alternative_title":["PMLR"],"main_file_link":[{"open_access":"1","url":"https://arxiv.org/abs/2203.04413"}],"intvolume":"       162","status":"public","day":"22","type":"conference","publication":"Proceedings of the 39th International Conference on Machine Learning","page":"18741-18753","publisher":"ML Research Press","language":[{"iso":"eng"}],"month":"07","date_published":"2022-07-22T00:00:00Z","conference":{"name":"International Conference on Machine Learning","end_date":"2022-07-23","location":"Baltimore, MD, United States","start_date":"2022-07-17"},"date_created":"2023-08-22T14:00:18Z","department":[{"_id":"FrLo"}]},{"arxiv":1,"publication":"10th International Conference on Learning Representations","date_updated":"2023-09-11T09:40:52Z","oa":1,"article_processing_charge":"No","_id":"14172","extern":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","quality_controlled":"1","oa_version":"Preprint","publication_status":"published","day":"25","citation":{"chicago":"Schott, Lukas, Julius von Kügelgen, Frederik Träuble, Peter Gehler, Chris Russell, Matthias Bethge, Bernhard Schölkopf, Francesco Locatello, and Wieland Brendel. “Visual Representation Learning Does Not Generalize Strongly within the  Same Domain.” In <i>10th International Conference on Learning Representations</i>, 2022.","ieee":"L. Schott <i>et al.</i>, “Visual representation learning does not generalize strongly within the  same domain,” in <i>10th International Conference on Learning Representations</i>, Virtual, 2022.","apa":"Schott, L., Kügelgen, J. von, Träuble, F., Gehler, P., Russell, C., Bethge, M., … Brendel, W. (2022). Visual representation learning does not generalize strongly within the  same domain. In <i>10th International Conference on Learning Representations</i>. Virtual.","short":"L. Schott, J. von Kügelgen, F. Träuble, P. Gehler, C. Russell, M. Bethge, B. Schölkopf, F. Locatello, W. Brendel, in:, 10th International Conference on Learning Representations, 2022.","ista":"Schott L, Kügelgen J von, Träuble F, Gehler P, Russell C, Bethge M, Schölkopf B, Locatello F, Brendel W. 2022. Visual representation learning does not generalize strongly within the  same domain. 10th International Conference on Learning Representations. ICLR: International Conference on Learning Representations.","mla":"Schott, Lukas, et al. “Visual Representation Learning Does Not Generalize Strongly within the  Same Domain.” <i>10th International Conference on Learning Representations</i>, 2022.","ama":"Schott L, Kügelgen J von, Träuble F, et al. Visual representation learning does not generalize strongly within the  same domain. In: <i>10th International Conference on Learning Representations</i>. ; 2022."},"type":"conference","abstract":[{"lang":"eng","text":"An important component for generalization in machine learning is to uncover underlying latent factors of variation as well as the mechanism through which each factor acts in the world. In this paper, we test whether 17 unsupervised, weakly supervised, and fully supervised representation learning approaches correctly infer the generative factors of variation in simple datasets (dSprites, Shapes3D, MPI3D) from controlled environments, and on our contributed CelebGlow dataset. In contrast to prior robustness work that introduces novel factors of variation during test time, such as blur or other (un)structured noise, we here recompose, interpolate, or extrapolate only existing factors of variation from the training data set (e.g., small and medium-sized objects during training and large objects during testing). Models\r\nthat learn the correct mechanism should be able to generalize to this benchmark. In total, we train and test 2000+ models and observe that all of them struggle to learn the underlying mechanism regardless of supervision signal and architectural bias. Moreover, the generalization capabilities of all tested models drop significantly as we move from artificial datasets towards\r\nmore realistic real-world datasets. Despite their inability to identify the correct mechanism, the models are quite modular as their ability to infer other in-distribution factors remains fairly stable, providing only a single factoris out-of-distribution. These results point to an important yet understudied problem of learning mechanistic models of observations that can facilitate\r\ngeneralization."}],"status":"public","author":[{"first_name":"Lukas","last_name":"Schott","full_name":"Schott, Lukas"},{"last_name":"Kügelgen","full_name":"Kügelgen, Julius von","first_name":"Julius von"},{"first_name":"Frederik","full_name":"Träuble, Frederik","last_name":"Träuble"},{"full_name":"Gehler, Peter","last_name":"Gehler","first_name":"Peter"},{"first_name":"Chris","full_name":"Russell, Chris","last_name":"Russell"},{"full_name":"Bethge, Matthias","last_name":"Bethge","first_name":"Matthias"},{"full_name":"Schölkopf, Bernhard","last_name":"Schölkopf","first_name":"Bernhard"},{"last_name":"Locatello","full_name":"Locatello, Francesco","orcid":"0000-0002-4850-0683","first_name":"Francesco","id":"26cfd52f-2483-11ee-8040-88983bcc06d4"},{"full_name":"Brendel, Wieland","last_name":"Brendel","first_name":"Wieland"}],"department":[{"_id":"FrLo"}],"main_file_link":[{"open_access":"1","url":"https://doi.org/10.48550/arXiv.2107.08221"}],"conference":{"start_date":"2022-04-25","end_date":"2022-04-29","location":"Virtual","name":"ICLR: International Conference on Learning Representations"},"date_created":"2023-08-22T14:00:50Z","year":"2022","month":"04","date_published":"2022-04-25T00:00:00Z","title":"Visual representation learning does not generalize strongly within the  same domain","external_id":{"arxiv":["2107.08221"]},"language":[{"iso":"eng"}]},{"type":"conference","day":"15","status":"public","intvolume":"        35","page":"7181-7198","publication":"36th Conference on Neural Information Processing Systems","date_published":"2022-12-15T00:00:00Z","month":"12","language":[{"iso":"eng"}],"scopus_import":"1","publisher":"Neural Information Processing Systems Foundation","department":[{"_id":"FrLo"}],"date_created":"2023-08-22T14:01:13Z","conference":{"start_date":"2022-11-28","name":"NeurIPS: Neural Information Processing Systems","location":"New Orleans, LA, United States","end_date":"2022-12-09"},"citation":{"apa":"Wenzel, F., Dittadi, A., Gehler, P. V., Carl-Johann Simon-Gabriel, C.-J. S.-G., Horn, M., Zietlow, D., … Locatello, F. (2022). Assaying out-of-distribution generalization in transfer learning. In <i>36th Conference on Neural Information Processing Systems</i> (Vol. 35, pp. 7181–7198). New Orleans, LA, United States: Neural Information Processing Systems Foundation.","ieee":"F. Wenzel <i>et al.</i>, “Assaying out-of-distribution generalization in transfer learning,” in <i>36th Conference on Neural Information Processing Systems</i>, New Orleans, LA, United States, 2022, vol. 35, pp. 7181–7198.","chicago":"Wenzel, Florian, Andrea Dittadi, Peter Vincent Gehler, Carl-Johann Simon-Gabriel Carl-Johann Simon-Gabriel, Max Horn, Dominik Zietlow, David Kernert, et al. “Assaying Out-of-Distribution Generalization in Transfer Learning.” In <i>36th Conference on Neural Information Processing Systems</i>, 35:7181–98. Neural Information Processing Systems Foundation, 2022.","mla":"Wenzel, Florian, et al. “Assaying Out-of-Distribution Generalization in Transfer Learning.” <i>36th Conference on Neural Information Processing Systems</i>, vol. 35, Neural Information Processing Systems Foundation, 2022, pp. 7181–98.","ama":"Wenzel F, Dittadi A, Gehler PV, et al. Assaying out-of-distribution generalization in transfer learning. In: <i>36th Conference on Neural Information Processing Systems</i>. Vol 35. Neural Information Processing Systems Foundation; 2022:7181-7198.","ista":"Wenzel F, Dittadi A, Gehler PV, Carl-Johann Simon-Gabriel C-JS-G, Horn M, Zietlow D, Kernert D, Russell C, Brox T, Schiele B, Schölkopf B, Locatello F. 2022. Assaying out-of-distribution generalization in transfer learning. 36th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems, Advances in Neural Information Processing Systems, vol. 35, 7181–7198.","short":"F. Wenzel, A. Dittadi, P.V. Gehler, C.-J.S.-G. Carl-Johann Simon-Gabriel, M. Horn, D. Zietlow, D. Kernert, C. Russell, T. Brox, B. Schiele, B. Schölkopf, F. Locatello, in:, 36th Conference on Neural Information Processing Systems, Neural Information Processing Systems Foundation, 2022, pp. 7181–7198."},"publication_status":"published","author":[{"last_name":"Wenzel","full_name":"Wenzel, Florian","first_name":"Florian"},{"first_name":"Andrea","last_name":"Dittadi","full_name":"Dittadi, Andrea"},{"full_name":"Gehler, Peter Vincent","last_name":"Gehler","first_name":"Peter Vincent"},{"first_name":"Carl-Johann Simon-Gabriel","last_name":"Carl-Johann Simon-Gabriel","full_name":"Carl-Johann Simon-Gabriel, Carl-Johann Simon-Gabriel"},{"last_name":"Horn","full_name":"Horn, Max","first_name":"Max"},{"first_name":"Dominik","full_name":"Zietlow, Dominik","last_name":"Zietlow"},{"full_name":"Kernert, David","last_name":"Kernert","first_name":"David"},{"first_name":"Chris","full_name":"Russell, Chris","last_name":"Russell"},{"last_name":"Brox","full_name":"Brox, Thomas","first_name":"Thomas"},{"full_name":"Schiele, Bernt","last_name":"Schiele","first_name":"Bernt"},{"full_name":"Schölkopf, Bernhard","last_name":"Schölkopf","first_name":"Bernhard"},{"last_name":"Locatello","full_name":"Locatello, Francesco","orcid":"0000-0002-4850-0683","first_name":"Francesco","id":"26cfd52f-2483-11ee-8040-88983bcc06d4"}],"abstract":[{"lang":"eng","text":"Since out-of-distribution generalization is a generally ill-posed problem, various proxy targets (e.g., calibration, adversarial robustness, algorithmic corruptions, invariance across shifts) were studied across different research programs resulting in different recommendations. While sharing the same aspirational goal, these approaches have never been tested under the same\r\nexperimental conditions on real data. In this paper, we take a unified view of previous work, highlighting message discrepancies that we address empirically, and providing recommendations on how to measure the robustness of a model and how to improve it. To this end, we collect 172 publicly available dataset pairs for training and out-of-distribution evaluation of accuracy, calibration error, adversarial attacks, environment invariance, and synthetic corruptions. We fine-tune over 31k networks, from nine different architectures in the many- and\r\nfew-shot setting. Our findings confirm that in- and out-of-distribution accuracies tend to increase jointly, but show that their relation is largely dataset-dependent, and in general more nuanced and more complex than posited by previous, smaller scale studies."}],"article_processing_charge":"No","volume":35,"oa":1,"date_updated":"2023-09-06T10:34:43Z","arxiv":1,"oa_version":"Preprint","quality_controlled":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","publication_identifier":{"isbn":["9781713871088"]},"extern":"1","_id":"14173","year":"2022","external_id":{"arxiv":["2207.09239"]},"title":"Assaying out-of-distribution generalization in transfer learning","main_file_link":[{"open_access":"1","url":"https://arxiv.org/abs/2207.09239"}],"alternative_title":["Advances in Neural Information Processing Systems"]},{"conference":{"start_date":"2022-04-25","end_date":"2022-04-29","name":"ICLR: International Conference on Learning Representations","location":"Virtual"},"date_created":"2023-08-22T14:02:13Z","department":[{"_id":"FrLo"}],"main_file_link":[{"open_access":"1","url":" https://doi.org/10.48550/arXiv.2107.05686"}],"external_id":{"arxiv":["2107.05686"]},"title":"The role of pretrained representations for the OOD generalization of  reinforcement learning agents","language":[{"iso":"eng"}],"year":"2022","month":"04","date_published":"2022-04-25T00:00:00Z","_id":"14174","extern":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","quality_controlled":"1","oa_version":"Preprint","arxiv":1,"publication":"10th International Conference on Learning Representations","date_updated":"2023-09-11T09:48:36Z","oa":1,"article_processing_charge":"No","abstract":[{"text":"Building sample-efficient agents that generalize out-of-distribution (OOD) in real-world settings remains a fundamental unsolved problem on the path towards achieving higher-level cognition. One particularly promising approach is to begin with low-dimensional, pretrained representations of our world, which should facilitate efficient downstream learning and generalization. By training 240 representations and over 10,000 reinforcement learning (RL) policies on a simulated robotic setup, we evaluate to what extent different properties of\r\npretrained VAE-based representations affect the OOD generalization of downstream agents. We observe that many agents are surprisingly robust to realistic distribution shifts, including the challenging sim-to-real case. In addition, we find that the generalization performance of a simple downstream proxy task reliably predicts the generalization performance of our RL agents\r\nunder a wide range of OOD settings. Such proxy tasks can thus be used to select pretrained representations that will lead to agents that generalize.","lang":"eng"}],"author":[{"last_name":"Dittadi","full_name":"Dittadi, Andrea","first_name":"Andrea"},{"first_name":"Frederik","last_name":"Träuble","full_name":"Träuble, Frederik"},{"first_name":"Manuel","last_name":"Wüthrich","full_name":"Wüthrich, Manuel"},{"last_name":"Widmaier","full_name":"Widmaier, Felix","first_name":"Felix"},{"full_name":"Gehler, Peter","last_name":"Gehler","first_name":"Peter"},{"first_name":"Ole","full_name":"Winther, Ole","last_name":"Winther"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","first_name":"Francesco","orcid":"0000-0002-4850-0683","full_name":"Locatello, Francesco","last_name":"Locatello"},{"first_name":"Olivier","full_name":"Bachem, Olivier","last_name":"Bachem"},{"first_name":"Bernhard","full_name":"Schölkopf, Bernhard","last_name":"Schölkopf"},{"full_name":"Bauer, Stefan","last_name":"Bauer","first_name":"Stefan"}],"status":"public","publication_status":"published","day":"25","citation":{"ieee":"A. Dittadi <i>et al.</i>, “The role of pretrained representations for the OOD generalization of  reinforcement learning agents,” in <i>10th International Conference on Learning Representations</i>, Virtual, 2022.","apa":"Dittadi, A., Träuble, F., Wüthrich, M., Widmaier, F., Gehler, P., Winther, O., … Bauer, S. (2022). The role of pretrained representations for the OOD generalization of  reinforcement learning agents. In <i>10th International Conference on Learning Representations</i>. Virtual.","chicago":"Dittadi, Andrea, Frederik Träuble, Manuel Wüthrich, Felix Widmaier, Peter Gehler, Ole Winther, Francesco Locatello, Olivier Bachem, Bernhard Schölkopf, and Stefan Bauer. “The Role of Pretrained Representations for the OOD Generalization of  Reinforcement Learning Agents.” In <i>10th International Conference on Learning Representations</i>, 2022.","mla":"Dittadi, Andrea, et al. “The Role of Pretrained Representations for the OOD Generalization of  Reinforcement Learning Agents.” <i>10th International Conference on Learning Representations</i>, 2022.","ama":"Dittadi A, Träuble F, Wüthrich M, et al. The role of pretrained representations for the OOD generalization of  reinforcement learning agents. In: <i>10th International Conference on Learning Representations</i>. ; 2022.","short":"A. Dittadi, F. Träuble, M. Wüthrich, F. Widmaier, P. Gehler, O. Winther, F. Locatello, O. Bachem, B. Schölkopf, S. Bauer, in:, 10th International Conference on Learning Representations, 2022.","ista":"Dittadi A, Träuble F, Wüthrich M, Widmaier F, Gehler P, Winther O, Locatello F, Bachem O, Schölkopf B, Bauer S. 2022. The role of pretrained representations for the OOD generalization of  reinforcement learning agents. 10th International Conference on Learning Representations. ICLR: International Conference on Learning Representations."},"type":"conference"},{"language":[{"iso":"eng"}],"title":"You mostly walk alone: Analyzing feature attribution in trajectory prediction","external_id":{"arxiv":["2110.05304"]},"date_published":"2022-04-25T00:00:00Z","month":"04","year":"2022","date_created":"2023-08-22T14:02:34Z","conference":{"start_date":"2022-04-25","end_date":"2022-04-29","name":"ICLR: International Conference on Learning Representations","location":"Virtual"},"main_file_link":[{"open_access":"1","url":"https://doi.org/10.48550/arXiv.2110.05304"}],"department":[{"_id":"FrLo"}],"status":"public","author":[{"full_name":"Makansi, Osama","last_name":"Makansi","first_name":"Osama"},{"first_name":"Julius von","full_name":"Kügelgen, Julius von","last_name":"Kügelgen"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","last_name":"Locatello","full_name":"Locatello, Francesco","orcid":"0000-0002-4850-0683","first_name":"Francesco"},{"last_name":"Gehler","full_name":"Gehler, Peter","first_name":"Peter"},{"full_name":"Janzing, Dominik","last_name":"Janzing","first_name":"Dominik"},{"last_name":"Brox","full_name":"Brox, Thomas","first_name":"Thomas"},{"full_name":"Schölkopf, Bernhard","last_name":"Schölkopf","first_name":"Bernhard"}],"abstract":[{"text":"Predicting the future trajectory of a moving agent can be easy when the past trajectory continues smoothly but is challenging when complex interactions with other agents are involved. Recent deep learning approaches for trajectory prediction show promising performance and partially attribute this to successful reasoning about agent-agent interactions. However, it remains unclear which features such black-box models actually learn to use for making predictions. This paper proposes a procedure that quantifies the contributions\r\nof different cues to model performance based on a variant of Shapley values. Applying this procedure to state-of-the-art trajectory prediction methods on standard benchmark datasets shows that they are, in fact, unable to reason about interactions. Instead, the past trajectory of the target is the only feature used for predicting its future. For a task with richer social\r\ninteraction patterns, on the other hand, the tested models do pick up such interactions to a certain extent, as quantified by our feature attribution method. We discuss the limits of the proposed method and its links to causality.","lang":"eng"}],"type":"conference","day":"25","citation":{"ieee":"O. Makansi <i>et al.</i>, “You mostly walk alone: Analyzing feature attribution in trajectory prediction,” in <i>10th International Conference on Learning Representations</i>, Virtual, 2022.","apa":"Makansi, O., Kügelgen, J. von, Locatello, F., Gehler, P., Janzing, D., Brox, T., &#38; Schölkopf, B. (2022). You mostly walk alone: Analyzing feature attribution in trajectory prediction. In <i>10th International Conference on Learning Representations</i>. Virtual.","chicago":"Makansi, Osama, Julius von Kügelgen, Francesco Locatello, Peter Gehler, Dominik Janzing, Thomas Brox, and Bernhard Schölkopf. “You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction.” In <i>10th International Conference on Learning Representations</i>, 2022.","ama":"Makansi O, Kügelgen J von, Locatello F, et al. You mostly walk alone: Analyzing feature attribution in trajectory prediction. In: <i>10th International Conference on Learning Representations</i>. ; 2022.","mla":"Makansi, Osama, et al. “You Mostly Walk Alone: Analyzing Feature Attribution in Trajectory Prediction.” <i>10th International Conference on Learning Representations</i>, 2022.","ista":"Makansi O, Kügelgen J von, Locatello F, Gehler P, Janzing D, Brox T, Schölkopf B. 2022. You mostly walk alone: Analyzing feature attribution in trajectory prediction. 10th International Conference on Learning Representations. ICLR: International Conference on Learning Representations.","short":"O. Makansi, J. von Kügelgen, F. Locatello, P. Gehler, D. Janzing, T. Brox, B. Schölkopf, in:, 10th International Conference on Learning Representations, 2022."},"publication_status":"published","oa_version":"Preprint","quality_controlled":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","extern":"1","_id":"14175","article_processing_charge":"No","oa":1,"date_updated":"2023-09-11T09:52:20Z","publication":"10th International Conference on Learning Representations","arxiv":1},{"type":"conference","publication_status":"submitted","day":"04","citation":{"mla":"Rahaman, Nasim, et al. “A General Purpose Neural Architecture for Geospatial Systems.” <i>36th Conference on Neural Information Processing Systems</i>.","ama":"Rahaman N, Weiss M, Träuble F, et al. A general purpose neural architecture for geospatial systems. In: <i>36th Conference on Neural Information Processing Systems</i>.","ista":"Rahaman N, Weiss M, Träuble F, Locatello F, Lacoste A, Bengio Y, Pal C, Li LE, Schölkopf B. A general purpose neural architecture for geospatial systems. 36th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems.","short":"N. Rahaman, M. Weiss, F. Träuble, F. Locatello, A. Lacoste, Y. Bengio, C. Pal, L.E. Li, B. Schölkopf, in:, 36th Conference on Neural Information Processing Systems, n.d.","apa":"Rahaman, N., Weiss, M., Träuble, F., Locatello, F., Lacoste, A., Bengio, Y., … Schölkopf, B. (n.d.). A general purpose neural architecture for geospatial systems. In <i>36th Conference on Neural Information Processing Systems</i>. New Orleans, LA, United States.","ieee":"N. Rahaman <i>et al.</i>, “A general purpose neural architecture for geospatial systems,” in <i>36th Conference on Neural Information Processing Systems</i>, New Orleans, LA, United States.","chicago":"Rahaman, Nasim, Martin Weiss, Frederik Träuble, Francesco Locatello, Alexandre Lacoste, Yoshua Bengio, Chris Pal, Li Erran Li, and Bernhard Schölkopf. “A General Purpose Neural Architecture for Geospatial Systems.” In <i>36th Conference on Neural Information Processing Systems</i>, n.d."},"author":[{"full_name":"Rahaman, Nasim","last_name":"Rahaman","first_name":"Nasim"},{"first_name":"Martin","full_name":"Weiss, Martin","last_name":"Weiss"},{"first_name":"Frederik","full_name":"Träuble, Frederik","last_name":"Träuble"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","last_name":"Locatello","full_name":"Locatello, Francesco","orcid":"0000-0002-4850-0683","first_name":"Francesco"},{"last_name":"Lacoste","full_name":"Lacoste, Alexandre","first_name":"Alexandre"},{"full_name":"Bengio, Yoshua","last_name":"Bengio","first_name":"Yoshua"},{"first_name":"Chris","last_name":"Pal","full_name":"Pal, Chris"},{"first_name":"Li Erran","last_name":"Li","full_name":"Li, Li Erran"},{"first_name":"Bernhard","last_name":"Schölkopf","full_name":"Schölkopf, Bernhard"}],"status":"public","abstract":[{"lang":"eng","text":"Geospatial Information Systems are used by researchers and Humanitarian Assistance and Disaster Response (HADR) practitioners to support a wide variety of important applications. However, collaboration between these actors is difficult due to the heterogeneous nature of geospatial data modalities (e.g., multi-spectral images of various resolutions, timeseries, weather data) and diversity of tasks (e.g., regression of human activity indicators or detecting forest fires). In this work, we present a roadmap towards the construction of a general-purpose neural architecture (GPNA) with a geospatial inductive bias, pre-trained on large amounts of unlabelled earth observation data in a self-supervised manner. We envision how such a model may facilitate cooperation between members of the community. We show preliminary results on the first step of the roadmap, where we instantiate an architecture that can process a wide variety of geospatial data modalities and demonstrate that it can achieve competitive performance with domain-specific architectures on tasks relating to the U.N.'s Sustainable Development Goals."}],"oa":1,"date_updated":"2023-09-13T09:35:59Z","article_processing_charge":"No","arxiv":1,"publication":"36th Conference on Neural Information Processing Systems","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","quality_controlled":"1","oa_version":"Preprint","_id":"14215","extern":"1","date_published":"2022-11-04T00:00:00Z","year":"2022","month":"11","language":[{"iso":"eng"}],"external_id":{"arxiv":["2211.02348"]},"title":"A general purpose neural architecture for geospatial systems","department":[{"_id":"FrLo"}],"main_file_link":[{"open_access":"1","url":"https://doi.org/10.48550/arXiv.2211.02348"}],"date_created":"2023-08-22T14:21:47Z","conference":{"start_date":"2022-11-28","location":"New Orleans, LA, United States","end_date":"2022-12-09","name":"NeurIPS: Neural Information Processing Systems"}},{"type":"preprint","publication_status":"submitted","day":"04","citation":{"mla":"Norelli, Antonio, et al. “ASIF: Coupled Data Turns Unimodal Models to Multimodal without Training.” <i>ArXiv</i>, 2210.01738, doi:<a href=\"https://doi.org/10.48550/arXiv.2210.01738\">10.48550/arXiv.2210.01738</a>.","ama":"Norelli A, Fumero M, Maiorca V, Moschella L, Rodolà E, Locatello F. ASIF: Coupled data turns unimodal models to multimodal without training. <i>arXiv</i>. doi:<a href=\"https://doi.org/10.48550/arXiv.2210.01738\">10.48550/arXiv.2210.01738</a>","short":"A. Norelli, M. Fumero, V. Maiorca, L. Moschella, E. Rodolà, F. Locatello, ArXiv (n.d.).","ista":"Norelli A, Fumero M, Maiorca V, Moschella L, Rodolà E, Locatello F. ASIF: Coupled data turns unimodal models to multimodal without training. arXiv, 2210.01738.","ieee":"A. Norelli, M. Fumero, V. Maiorca, L. Moschella, E. Rodolà, and F. Locatello, “ASIF: Coupled data turns unimodal models to multimodal without training,” <i>arXiv</i>. .","apa":"Norelli, A., Fumero, M., Maiorca, V., Moschella, L., Rodolà, E., &#38; Locatello, F. (n.d.). ASIF: Coupled data turns unimodal models to multimodal without training. <i>arXiv</i>. <a href=\"https://doi.org/10.48550/arXiv.2210.01738\">https://doi.org/10.48550/arXiv.2210.01738</a>","chicago":"Norelli, Antonio, Marco Fumero, Valentino Maiorca, Luca Moschella, Emanuele Rodolà, and Francesco Locatello. “ASIF: Coupled Data Turns Unimodal Models to Multimodal without Training.” <i>ArXiv</i>, n.d. <a href=\"https://doi.org/10.48550/arXiv.2210.01738\">https://doi.org/10.48550/arXiv.2210.01738</a>."},"author":[{"first_name":"Antonio","full_name":"Norelli, Antonio","last_name":"Norelli"},{"last_name":"Fumero","full_name":"Fumero, Marco","first_name":"Marco"},{"last_name":"Maiorca","full_name":"Maiorca, Valentino","first_name":"Valentino"},{"first_name":"Luca","full_name":"Moschella, Luca","last_name":"Moschella"},{"first_name":"Emanuele","last_name":"Rodolà","full_name":"Rodolà, Emanuele"},{"full_name":"Locatello, Francesco","last_name":"Locatello","orcid":"0000-0002-4850-0683","first_name":"Francesco","id":"26cfd52f-2483-11ee-8040-88983bcc06d4"}],"status":"public","abstract":[{"lang":"eng","text":"CLIP proved that aligning visual and language spaces is key to solving many vision tasks without explicit training, but required to train image and text encoders from scratch on a huge dataset. LiT improved this by only training the text encoder and using a pre-trained vision network. In this paper, we show that a common space can be created without any training at all, using single-domain encoders (trained with or without supervision) and a much smaller amount of image-text pairs. Furthermore, our model has unique properties. Most notably, deploying a new version with updated training samples can be done in a matter of seconds. Additionally, the representations in the common space are easily interpretable as every dimension corresponds to the similarity of the input to a unique entry in the multimodal dataset. Experiments on standard zero-shot visual benchmarks demonstrate the typical transfer ability of image-text models. Overall, our method represents a simple yet surprisingly strong baseline for foundation multi-modal models, raising important questions on their data efficiency and on the role of retrieval in machine learning."}],"oa":1,"date_updated":"2024-02-12T09:57:14Z","article_processing_charge":"No","arxiv":1,"publication":"arXiv","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","oa_version":"Preprint","_id":"14216","date_published":"2022-10-04T00:00:00Z","year":"2022","month":"10","doi":"10.48550/arXiv.2210.01738","language":[{"iso":"eng"}],"title":"ASIF: Coupled data turns unimodal models to multimodal without training","external_id":{"arxiv":["2210.01738"]},"department":[{"_id":"FrLo"}],"article_number":"2210.01738","main_file_link":[{"url":"https://doi.org/10.48550/arXiv.2210.01738","open_access":"1"}],"date_created":"2023-08-22T14:22:04Z"},{"date_created":"2023-08-22T14:23:16Z","department":[{"_id":"FrLo"}],"article_number":"2201.13388","main_file_link":[{"open_access":"1","url":"https://doi.org/10.48550/arXiv.2201.13388"}],"external_id":{"arxiv":["2201.13388"]},"title":"Compositional multi-object reinforcement learning with linear relation networks","language":[{"iso":"eng"}],"month":"01","doi":"10.48550/arXiv.2201.13388","year":"2022","date_published":"2022-01-31T00:00:00Z","_id":"14220","extern":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","oa_version":"Preprint","arxiv":1,"publication":"arXiv","oa":1,"date_updated":"2023-09-11T11:49:40Z","article_processing_charge":"No","abstract":[{"lang":"eng","text":"Although reinforcement learning has seen remarkable progress over the last years, solving robust dexterous object-manipulation tasks in multi-object settings remains a challenge. In this paper, we focus on models that can learn manipulation tasks in fixed multi-object settings and extrapolate this skill zero-shot without any drop in performance when the number of objects changes. We consider the generic task of bringing a specific cube out of a set to a goal position. We find that previous approaches, which primarily leverage attention and graph neural network-based architectures, do not generalize their skills when the number of input objects changes while scaling as K2. We propose an alternative plug-and-play module based on relational inductive biases to overcome these limitations. Besides exceeding performances in their training environment, we show that our approach, which scales linearly in K, allows agents to extrapolate and generalize zero-shot to any new object number."}],"status":"public","author":[{"full_name":"Mambelli, Davide","last_name":"Mambelli","first_name":"Davide"},{"full_name":"Träuble, Frederik","last_name":"Träuble","first_name":"Frederik"},{"last_name":"Bauer","full_name":"Bauer, Stefan","first_name":"Stefan"},{"last_name":"Schölkopf","full_name":"Schölkopf, Bernhard","first_name":"Bernhard"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","full_name":"Locatello, Francesco","last_name":"Locatello","orcid":"0000-0002-4850-0683","first_name":"Francesco"}],"publication_status":"submitted","day":"31","citation":{"apa":"Mambelli, D., Träuble, F., Bauer, S., Schölkopf, B., &#38; Locatello, F. (n.d.). Compositional multi-object reinforcement learning with linear relation networks. <i>arXiv</i>. <a href=\"https://doi.org/10.48550/arXiv.2201.13388\">https://doi.org/10.48550/arXiv.2201.13388</a>","ieee":"D. Mambelli, F. Träuble, S. Bauer, B. Schölkopf, and F. Locatello, “Compositional multi-object reinforcement learning with linear relation networks,” <i>arXiv</i>. .","chicago":"Mambelli, Davide, Frederik Träuble, Stefan Bauer, Bernhard Schölkopf, and Francesco Locatello. “Compositional Multi-Object Reinforcement Learning with Linear Relation Networks.” <i>ArXiv</i>, n.d. <a href=\"https://doi.org/10.48550/arXiv.2201.13388\">https://doi.org/10.48550/arXiv.2201.13388</a>.","ama":"Mambelli D, Träuble F, Bauer S, Schölkopf B, Locatello F. Compositional multi-object reinforcement learning with linear relation networks. <i>arXiv</i>. doi:<a href=\"https://doi.org/10.48550/arXiv.2201.13388\">10.48550/arXiv.2201.13388</a>","mla":"Mambelli, Davide, et al. “Compositional Multi-Object Reinforcement Learning with Linear Relation Networks.” <i>ArXiv</i>, 2201.13388, doi:<a href=\"https://doi.org/10.48550/arXiv.2201.13388\">10.48550/arXiv.2201.13388</a>.","short":"D. Mambelli, F. Träuble, S. Bauer, B. Schölkopf, F. Locatello, ArXiv (n.d.).","ista":"Mambelli D, Träuble F, Bauer S, Schölkopf B, Locatello F. Compositional multi-object reinforcement learning with linear relation networks. arXiv, 2201.13388."},"type":"preprint"},{"month":"07","year":"2021","date_published":"2021-07-23T00:00:00Z","title":"Representation learning for out-of-distribution generalization in reinforcement learning","language":[{"iso":"eng"}],"department":[{"_id":"FrLo"}],"conference":{"end_date":"2021-07-23","location":"Virtual","name":"ICML: International Conference on Machine Learning","start_date":"2021-07-23"},"date_created":"2023-09-13T12:43:14Z","day":"23","citation":{"ama":"Träuble F, Dittadi A, Wuthrich M, et al. Representation learning for out-of-distribution generalization in reinforcement learning. In: <i>ICML 2021 Workshop on Unsupervised Reinforcement Learning</i>. ; 2021.","mla":"Träuble, Frederik, et al. “Representation Learning for Out-of-Distribution Generalization in Reinforcement Learning.” <i>ICML 2021 Workshop on Unsupervised Reinforcement Learning</i>, 2021.","ista":"Träuble F, Dittadi A, Wuthrich M, Widmaier F, Gehler PV, Winther O, Locatello F, Bachem O, Schölkopf B, Bauer S. 2021. Representation learning for out-of-distribution generalization in reinforcement learning. ICML 2021 Workshop on Unsupervised Reinforcement Learning. ICML: International Conference on Machine Learning.","short":"F. Träuble, A. Dittadi, M. Wuthrich, F. Widmaier, P.V. Gehler, O. Winther, F. Locatello, O. Bachem, B. Schölkopf, S. Bauer, in:, ICML 2021 Workshop on Unsupervised Reinforcement Learning, 2021.","ieee":"F. Träuble <i>et al.</i>, “Representation learning for out-of-distribution generalization in reinforcement learning,” in <i>ICML 2021 Workshop on Unsupervised Reinforcement Learning</i>, Virtual, 2021.","apa":"Träuble, F., Dittadi, A., Wuthrich, M., Widmaier, F., Gehler, P. V., Winther, O., … Bauer, S. (2021). Representation learning for out-of-distribution generalization in reinforcement learning. In <i>ICML 2021 Workshop on Unsupervised Reinforcement Learning</i>. Virtual.","chicago":"Träuble, Frederik, Andrea Dittadi, Manuel Wuthrich, Felix Widmaier, Peter Vincent Gehler, Ole Winther, Francesco Locatello, Olivier Bachem, Bernhard Schölkopf, and Stefan Bauer. “Representation Learning for Out-of-Distribution Generalization in Reinforcement Learning.” In <i>ICML 2021 Workshop on Unsupervised Reinforcement Learning</i>, 2021."},"publication_status":"published","type":"conference","abstract":[{"lang":"eng","text":"Learning data representations that are useful for various downstream tasks is a cornerstone of artificial intelligence. While existing methods are typically evaluated on downstream tasks such as classification or generative image quality, we propose to assess representations through their usefulness in downstream control tasks, such as reaching or pushing objects. By training over 10,000 reinforcement learning policies, we extensively evaluate to what extent different representation properties affect out-of-distribution (OOD) generalization. Finally, we demonstrate zero-shot transfer of these policies from simulation to the real world, without any domain randomization or fine-tuning. This paper aims to establish the first systematic characterization of the usefulness of learned representations for real-world OOD downstream tasks."}],"author":[{"full_name":"Träuble, Frederik","last_name":"Träuble","first_name":"Frederik"},{"full_name":"Dittadi, Andrea","last_name":"Dittadi","first_name":"Andrea"},{"full_name":"Wuthrich, Manuel","last_name":"Wuthrich","first_name":"Manuel"},{"first_name":"Felix","full_name":"Widmaier, Felix","last_name":"Widmaier"},{"first_name":"Peter Vincent","full_name":"Gehler, Peter Vincent","last_name":"Gehler"},{"last_name":"Winther","full_name":"Winther, Ole","first_name":"Ole"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","first_name":"Francesco","orcid":"0000-0002-4850-0683","full_name":"Locatello, Francesco","last_name":"Locatello"},{"first_name":"Olivier","full_name":"Bachem, Olivier","last_name":"Bachem"},{"last_name":"Schölkopf","full_name":"Schölkopf, Bernhard","first_name":"Bernhard"},{"first_name":"Stefan","last_name":"Bauer","full_name":"Bauer, Stefan"}],"status":"public","publication":"ICML 2021 Workshop on Unsupervised Reinforcement Learning","article_processing_charge":"No","date_updated":"2023-09-13T12:44:00Z","extern":"1","_id":"14332","oa_version":"None","quality_controlled":"1","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87"},{"date_created":"2023-08-21T12:19:30Z","department":[{"_id":"FrLo"}],"language":[{"iso":"eng"}],"publisher":"Institute of Electrical and Electronics Engineers","scopus_import":"1","date_published":"2021-05-01T00:00:00Z","article_type":"original","month":"05","page":"612-634","issue":"5","publication":"Proceedings of the IEEE","status":"public","intvolume":"       109","type":"journal_article","day":"01","main_file_link":[{"open_access":"1","url":"https://doi.org/10.1109/JPROC.2021.3058954"}],"external_id":{"arxiv":["2102.11107"]},"title":"Toward causal representation learning","year":"2021","doi":"10.1109/jproc.2021.3058954","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","quality_controlled":"1","oa_version":"Published Version","_id":"14117","publication_identifier":{"eissn":["1558-2256"],"issn":["0018-9219"]},"extern":"1","date_updated":"2023-09-11T11:43:35Z","volume":109,"oa":1,"article_processing_charge":"No","arxiv":1,"keyword":["Electrical and Electronic Engineering"],"author":[{"first_name":"Bernhard","last_name":"Scholkopf","full_name":"Scholkopf, Bernhard"},{"id":"26cfd52f-2483-11ee-8040-88983bcc06d4","full_name":"Locatello, Francesco","last_name":"Locatello","orcid":"0000-0002-4850-0683","first_name":"Francesco"},{"full_name":"Bauer, Stefan","last_name":"Bauer","first_name":"Stefan"},{"last_name":"Ke","full_name":"Ke, Nan Rosemary","first_name":"Nan Rosemary"},{"first_name":"Nal","last_name":"Kalchbrenner","full_name":"Kalchbrenner, Nal"},{"full_name":"Goyal, Anirudh","last_name":"Goyal","first_name":"Anirudh"},{"full_name":"Bengio, Yoshua","last_name":"Bengio","first_name":"Yoshua"}],"abstract":[{"text":"The two fields of machine learning and graphical causality arose and are developed separately. However, there is, now, cross-pollination and increasing interest in both fields to benefit from the advances of the other. In this article, we review fundamental concepts of causal inference and relate them to crucial open problems of machine learning, including transfer and generalization, thereby assaying how causality can contribute to modern machine learning research. This also applies in the opposite direction: we note that most work in causality starts from the premise that the causal variables are given. A central problem for AI and causality is, thus, causal representation learning, that is, the discovery of high-level causal variables from low-level observations. Finally, we delineate some implications of causality for machine learning and propose key research areas at the intersection of both communities.","lang":"eng"}],"publication_status":"published","citation":{"ista":"Scholkopf B, Locatello F, Bauer S, Ke NR, Kalchbrenner N, Goyal A, Bengio Y. 2021. Toward causal representation learning. Proceedings of the IEEE. 109(5), 612–634.","short":"B. Scholkopf, F. Locatello, S. Bauer, N.R. Ke, N. Kalchbrenner, A. Goyal, Y. Bengio, Proceedings of the IEEE 109 (2021) 612–634.","mla":"Scholkopf, Bernhard, et al. “Toward Causal Representation Learning.” <i>Proceedings of the IEEE</i>, vol. 109, no. 5, Institute of Electrical and Electronics Engineers, 2021, pp. 612–34, doi:<a href=\"https://doi.org/10.1109/jproc.2021.3058954\">10.1109/jproc.2021.3058954</a>.","ama":"Scholkopf B, Locatello F, Bauer S, et al. Toward causal representation learning. <i>Proceedings of the IEEE</i>. 2021;109(5):612-634. doi:<a href=\"https://doi.org/10.1109/jproc.2021.3058954\">10.1109/jproc.2021.3058954</a>","chicago":"Scholkopf, Bernhard, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. “Toward Causal Representation Learning.” <i>Proceedings of the IEEE</i>. Institute of Electrical and Electronics Engineers, 2021. <a href=\"https://doi.org/10.1109/jproc.2021.3058954\">https://doi.org/10.1109/jproc.2021.3058954</a>.","ieee":"B. Scholkopf <i>et al.</i>, “Toward causal representation learning,” <i>Proceedings of the IEEE</i>, vol. 109, no. 5. Institute of Electrical and Electronics Engineers, pp. 612–634, 2021.","apa":"Scholkopf, B., Locatello, F., Bauer, S., Ke, N. R., Kalchbrenner, N., Goyal, A., &#38; Bengio, Y. (2021). Toward causal representation learning. <i>Proceedings of the IEEE</i>. Institute of Electrical and Electronics Engineers. <a href=\"https://doi.org/10.1109/jproc.2021.3058954\">https://doi.org/10.1109/jproc.2021.3058954</a>"}}]