--- _id: '14176' abstract: - lang: eng text: "Intensive care units (ICU) are increasingly looking towards machine learning for methods to provide online monitoring of critically ill patients. In machine learning, online monitoring is often formulated as a supervised learning problem. Recently, contrastive learning approaches have demonstrated promising improvements over competitive supervised benchmarks. These methods rely on well-understood data augmentation techniques developed for image data which do not apply to online monitoring. In this work, we overcome this limitation by\r\nsupplementing time-series data augmentation techniques with a novel contrastive\r\nlearning objective which we call neighborhood contrastive learning (NCL). Our objective explicitly groups together contiguous time segments from each patient while maintaining state-specific information. Our experiments demonstrate a marked improvement over existing work applying contrastive methods to medical time-series." alternative_title: - PMLR article_processing_charge: No arxiv: 1 author: - first_name: Hugo full_name: Yèche, Hugo last_name: Yèche - first_name: Gideon full_name: Dresdner, Gideon last_name: Dresdner - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Matthias full_name: Hüser, Matthias last_name: Hüser - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch citation: ama: 'Yèche H, Dresdner G, Locatello F, Hüser M, Rätsch G. Neighborhood contrastive learning applied to online patient monitoring. In: Proceedings of 38th International Conference on Machine Learning. Vol 139. ML Research Press; 2021:11964-11974.' apa: 'Yèche, H., Dresdner, G., Locatello, F., Hüser, M., & Rätsch, G. (2021). Neighborhood contrastive learning applied to online patient monitoring. In Proceedings of 38th International Conference on Machine Learning (Vol. 139, pp. 11964–11974). Virtual: ML Research Press.' chicago: Yèche, Hugo, Gideon Dresdner, Francesco Locatello, Matthias Hüser, and Gunnar Rätsch. “Neighborhood Contrastive Learning Applied to Online Patient Monitoring.” In Proceedings of 38th International Conference on Machine Learning, 139:11964–74. ML Research Press, 2021. ieee: H. Yèche, G. Dresdner, F. Locatello, M. Hüser, and G. Rätsch, “Neighborhood contrastive learning applied to online patient monitoring,” in Proceedings of 38th International Conference on Machine Learning, Virtual, 2021, vol. 139, pp. 11964–11974. ista: Yèche H, Dresdner G, Locatello F, Hüser M, Rätsch G. 2021. Neighborhood contrastive learning applied to online patient monitoring. Proceedings of 38th International Conference on Machine Learning. International Conference on Machine Learning, PMLR, vol. 139, 11964–11974. mla: Yèche, Hugo, et al. “Neighborhood Contrastive Learning Applied to Online Patient Monitoring.” Proceedings of 38th International Conference on Machine Learning, vol. 139, ML Research Press, 2021, pp. 11964–74. short: H. Yèche, G. Dresdner, F. Locatello, M. Hüser, G. Rätsch, in:, Proceedings of 38th International Conference on Machine Learning, ML Research Press, 2021, pp. 11964–11974. conference: end_date: 2021-07-24 location: Virtual name: International Conference on Machine Learning start_date: 2021-07-18 date_created: 2023-08-22T14:03:04Z date_published: 2021-08-01T00:00:00Z date_updated: 2023-09-11T10:16:55Z day: '01' department: - _id: FrLo extern: '1' external_id: arxiv: - '2106.05142' intvolume: ' 139' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2106.05142 month: '08' oa: 1 oa_version: Preprint page: 11964-11974 publication: Proceedings of 38th International Conference on Machine Learning publication_status: published publisher: ML Research Press quality_controlled: '1' scopus_import: '1' status: public title: Neighborhood contrastive learning applied to online patient monitoring type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 139 year: '2021' ... --- _id: '14177' abstract: - lang: eng text: "The focus of disentanglement approaches has been on identifying independent factors of variation in data. However, the causal variables underlying real-world observations are often not statistically independent. In this work, we bridge the gap to real-world scenarios by analyzing the behavior of the most prominent disentanglement approaches on correlated data in a large-scale empirical study (including 4260 models). We show and quantify that systematically induced correlations in the dataset are being learned and reflected in the latent representations, which has implications for downstream applications of disentanglement such as fairness. We also demonstrate how to resolve these latent correlations, either using weak supervision during\r\ntraining or by post-hoc correcting a pre-trained model with a small number of labels." alternative_title: - PMLR article_processing_charge: No arxiv: 1 author: - first_name: Frederik full_name: Träuble, Frederik last_name: Träuble - first_name: Elliot full_name: Creager, Elliot last_name: Creager - first_name: Niki full_name: Kilbertus, Niki last_name: Kilbertus - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Andrea full_name: Dittadi, Andrea last_name: Dittadi - first_name: Anirudh full_name: Goyal, Anirudh last_name: Goyal - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer citation: ama: 'Träuble F, Creager E, Kilbertus N, et al. On disentangled representations learned from correlated data. In: Proceedings of the 38th International Conference on Machine Learning. Vol 139. ML Research Press; 2021:10401-10412.' apa: 'Träuble, F., Creager, E., Kilbertus, N., Locatello, F., Dittadi, A., Goyal, A., … Bauer, S. (2021). On disentangled representations learned from correlated data. In Proceedings of the 38th International Conference on Machine Learning (Vol. 139, pp. 10401–10412). Virtual: ML Research Press.' chicago: Träuble, Frederik, Elliot Creager, Niki Kilbertus, Francesco Locatello, Andrea Dittadi, Anirudh Goyal, Bernhard Schölkopf, and Stefan Bauer. “On Disentangled Representations Learned from Correlated Data.” In Proceedings of the 38th International Conference on Machine Learning, 139:10401–12. ML Research Press, 2021. ieee: F. Träuble et al., “On disentangled representations learned from correlated data,” in Proceedings of the 38th International Conference on Machine Learning, Virtual, 2021, vol. 139, pp. 10401–10412. ista: 'Träuble F, Creager E, Kilbertus N, Locatello F, Dittadi A, Goyal A, Schölkopf B, Bauer S. 2021. On disentangled representations learned from correlated data. Proceedings of the 38th International Conference on Machine Learning. ICML: International Conference on Machine Learning, PMLR, vol. 139, 10401–10412.' mla: Träuble, Frederik, et al. “On Disentangled Representations Learned from Correlated Data.” Proceedings of the 38th International Conference on Machine Learning, vol. 139, ML Research Press, 2021, pp. 10401–12. short: F. Träuble, E. Creager, N. Kilbertus, F. Locatello, A. Dittadi, A. Goyal, B. Schölkopf, S. Bauer, in:, Proceedings of the 38th International Conference on Machine Learning, ML Research Press, 2021, pp. 10401–10412. conference: end_date: 2021-07-24 location: Virtual name: 'ICML: International Conference on Machine Learning' start_date: 2021-07-18 date_created: 2023-08-22T14:03:47Z date_published: 2021-08-01T00:00:00Z date_updated: 2023-09-11T10:18:48Z day: '01' department: - _id: FrLo extern: '1' external_id: arxiv: - '2006.07886' intvolume: ' 139' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2006.07886 month: '08' oa: 1 oa_version: Published Version page: 10401-10412 publication: Proceedings of the 38th International Conference on Machine Learning publication_status: published publisher: ML Research Press quality_controlled: '1' scopus_import: '1' status: public title: On disentangled representations learned from correlated data type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 139 year: '2021' ... --- _id: '14178' abstract: - lang: eng text: Learning meaningful representations that disentangle the underlying structure of the data generating process is considered to be of key importance in machine learning. While disentangled representations were found to be useful for diverse tasks such as abstract reasoning and fair classification, their scalability and real-world impact remain questionable. We introduce a new high-resolution dataset with 1M simulated images and over 1,800 annotated real-world images of the same setup. In contrast to previous work, this new dataset exhibits correlations, a complex underlying structure, and allows to evaluate transfer to unseen simulated and real-world settings where the encoder i) remains in distribution or ii) is out of distribution. We propose new architectures in order to scale disentangled representation learning to realistic high-resolution settings and conduct a large-scale empirical study of disentangled representations on this dataset. We observe that disentanglement is a good predictor for out-of-distribution (OOD) task performance. article_processing_charge: No arxiv: 1 author: - first_name: Andrea full_name: Dittadi, Andrea last_name: Dittadi - first_name: Frederik full_name: Träuble, Frederik last_name: Träuble - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Manuel full_name: Wüthrich, Manuel last_name: Wüthrich - first_name: Vaibhav full_name: Agrawal, Vaibhav last_name: Agrawal - first_name: Ole full_name: Winther, Ole last_name: Winther - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf citation: ama: 'Dittadi A, Träuble F, Locatello F, et al. On the transfer of disentangled representations in realistic settings. In: The Ninth International Conference on Learning Representations. ; 2021.' apa: Dittadi, A., Träuble, F., Locatello, F., Wüthrich, M., Agrawal, V., Winther, O., … Schölkopf, B. (2021). On the transfer of disentangled representations in realistic settings. In The Ninth International Conference on Learning Representations. Virtual. chicago: Dittadi, Andrea, Frederik Träuble, Francesco Locatello, Manuel Wüthrich, Vaibhav Agrawal, Ole Winther, Stefan Bauer, and Bernhard Schölkopf. “On the Transfer of Disentangled Representations in Realistic Settings.” In The Ninth International Conference on Learning Representations, 2021. ieee: A. Dittadi et al., “On the transfer of disentangled representations in realistic settings,” in The Ninth International Conference on Learning Representations, Virtual, 2021. ista: 'Dittadi A, Träuble F, Locatello F, Wüthrich M, Agrawal V, Winther O, Bauer S, Schölkopf B. 2021. On the transfer of disentangled representations in realistic settings. The Ninth International Conference on Learning Representations. ICLR: International Conference on Learning Representations.' mla: Dittadi, Andrea, et al. “On the Transfer of Disentangled Representations in Realistic Settings.” The Ninth International Conference on Learning Representations, 2021. short: A. Dittadi, F. Träuble, F. Locatello, M. Wüthrich, V. Agrawal, O. Winther, S. Bauer, B. Schölkopf, in:, The Ninth International Conference on Learning Representations, 2021. conference: end_date: 2021-05-07 location: Virtual name: 'ICLR: International Conference on Learning Representations' start_date: 2021-05-03 date_created: 2023-08-22T14:04:16Z date_published: 2021-05-04T00:00:00Z date_updated: 2023-09-11T10:55:30Z day: '04' department: - _id: FrLo extern: '1' external_id: arxiv: - '2010.14407' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2010.14407 month: '05' oa: 1 oa_version: Preprint publication: The Ninth International Conference on Learning Representations publication_status: published quality_controlled: '1' status: public title: On the transfer of disentangled representations in realistic settings type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2021' ... --- _id: '14179' abstract: - lang: eng text: Self-supervised representation learning has shown remarkable success in a number of domains. A common practice is to perform data augmentation via hand-crafted transformations intended to leave the semantics of the data invariant. We seek to understand the empirical success of this approach from a theoretical perspective. We formulate the augmentation process as a latent variable model by postulating a partition of the latent representation into a content component, which is assumed invariant to augmentation, and a style component, which is allowed to change. Unlike prior work on disentanglement and independent component analysis, we allow for both nontrivial statistical and causal dependencies in the latent space. We study the identifiability of the latent representation based on pairs of views of the observations and prove sufficient conditions that allow us to identify the invariant content partition up to an invertible mapping in both generative and discriminative settings. We find numerical simulations with dependent latent variables are consistent with our theory. Lastly, we introduce Causal3DIdent, a dataset of high-dimensional, visually complex images with rich causal dependencies, which we use to study the effect of data augmentations performed in practice. article_processing_charge: No arxiv: 1 author: - first_name: Julius von full_name: Kügelgen, Julius von last_name: Kügelgen - first_name: Yash full_name: Sharma, Yash last_name: Sharma - first_name: Luigi full_name: Gresele, Luigi last_name: Gresele - first_name: Wieland full_name: Brendel, Wieland last_name: Brendel - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Michel full_name: Besserve, Michel last_name: Besserve - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 citation: ama: 'Kügelgen J von, Sharma Y, Gresele L, et al. Self-supervised learning with data augmentations provably isolates content from style. In: Advances in Neural Information Processing Systems. Vol 34. ; 2021:16451-16467.' apa: Kügelgen, J. von, Sharma, Y., Gresele, L., Brendel, W., Schölkopf, B., Besserve, M., & Locatello, F. (2021). Self-supervised learning with data augmentations provably isolates content from style. In Advances in Neural Information Processing Systems (Vol. 34, pp. 16451–16467). Virtual. chicago: Kügelgen, Julius von, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. “Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style.” In Advances in Neural Information Processing Systems, 34:16451–67, 2021. ieee: J. von Kügelgen et al., “Self-supervised learning with data augmentations provably isolates content from style,” in Advances in Neural Information Processing Systems, Virtual, 2021, vol. 34, pp. 16451–16467. ista: 'Kügelgen J von, Sharma Y, Gresele L, Brendel W, Schölkopf B, Besserve M, Locatello F. 2021. Self-supervised learning with data augmentations provably isolates content from style. Advances in Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 34, 16451–16467.' mla: Kügelgen, Julius von, et al. “Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style.” Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 16451–67. short: J. von Kügelgen, Y. Sharma, L. Gresele, W. Brendel, B. Schölkopf, M. Besserve, F. Locatello, in:, Advances in Neural Information Processing Systems, 2021, pp. 16451–16467. conference: end_date: 2021-12-10 location: Virtual name: 'NeurIPS: Neural Information Processing Systems' start_date: 2021-12-07 date_created: 2023-08-22T14:04:36Z date_published: 2021-06-08T00:00:00Z date_updated: 2023-09-11T10:33:19Z day: '08' department: - _id: FrLo extern: '1' external_id: arxiv: - '2106.04619' intvolume: ' 34' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2106.04619 month: '06' oa: 1 oa_version: Preprint page: 16451-16467 publication: Advances in Neural Information Processing Systems publication_identifier: isbn: - '9781713845393' publication_status: published quality_controlled: '1' status: public title: Self-supervised learning with data augmentations provably isolates content from style type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 34 year: '2021' ... --- _id: '14180' abstract: - lang: eng text: 'Modern neural network architectures can leverage large amounts of data to generalize well within the training distribution. However, they are less capable of systematic generalization to data drawn from unseen but related distributions, a feat that is hypothesized to require compositional reasoning and reuse of knowledge. In this work, we present Neural Interpreters, an architecture that factorizes inference in a self-attention network as a system of modules, which we call \emph{functions}. Inputs to the model are routed through a sequence of functions in a way that is end-to-end learned. The proposed architecture can flexibly compose computation along width and depth, and lends itself well to capacity extension after training. To demonstrate the versatility of Neural Interpreters, we evaluate it in two distinct settings: image classification and visual abstract reasoning on Raven Progressive Matrices. In the former, we show that Neural Interpreters perform on par with the vision transformer using fewer parameters, while being transferrable to a new task in a sample efficient manner. In the latter, we find that Neural Interpreters are competitive with respect to the state-of-the-art in terms of systematic generalization. ' article_processing_charge: No arxiv: 1 author: - first_name: Nasim full_name: Rahaman, Nasim last_name: Rahaman - first_name: Muhammad Waleed full_name: Gondal, Muhammad Waleed last_name: Gondal - first_name: Shruti full_name: Joshi, Shruti last_name: Joshi - first_name: Peter full_name: Gehler, Peter last_name: Gehler - first_name: Yoshua full_name: Bengio, Yoshua last_name: Bengio - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf citation: ama: 'Rahaman N, Gondal MW, Joshi S, et al. Dynamic inference with neural interpreters. In: Advances in Neural Information Processing Systems. Vol 34. ; 2021:10985-10998.' apa: Rahaman, N., Gondal, M. W., Joshi, S., Gehler, P., Bengio, Y., Locatello, F., & Schölkopf, B. (2021). Dynamic inference with neural interpreters. In Advances in Neural Information Processing Systems (Vol. 34, pp. 10985–10998). Virtual. chicago: Rahaman, Nasim, Muhammad Waleed Gondal, Shruti Joshi, Peter Gehler, Yoshua Bengio, Francesco Locatello, and Bernhard Schölkopf. “Dynamic Inference with Neural Interpreters.” In Advances in Neural Information Processing Systems, 34:10985–98, 2021. ieee: N. Rahaman et al., “Dynamic inference with neural interpreters,” in Advances in Neural Information Processing Systems, Virtual, 2021, vol. 34, pp. 10985–10998. ista: 'Rahaman N, Gondal MW, Joshi S, Gehler P, Bengio Y, Locatello F, Schölkopf B. 2021. Dynamic inference with neural interpreters. Advances in Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 34, 10985–10998.' mla: Rahaman, Nasim, et al. “Dynamic Inference with Neural Interpreters.” Advances in Neural Information Processing Systems, vol. 34, 2021, pp. 10985–98. short: N. Rahaman, M.W. Gondal, S. Joshi, P. Gehler, Y. Bengio, F. Locatello, B. Schölkopf, in:, Advances in Neural Information Processing Systems, 2021, pp. 10985–10998. conference: end_date: 2021-12-10 location: Virtual name: 'NeurIPS: Neural Information Processing Systems' start_date: 2021-12-07 date_created: 2023-08-22T14:04:55Z date_published: 2021-10-12T00:00:00Z date_updated: 2023-09-11T11:33:46Z day: '12' department: - _id: FrLo extern: '1' external_id: arxiv: - '2110.06399' intvolume: ' 34' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2110.06399 month: '10' oa: 1 oa_version: Preprint page: 10985-10998 publication: Advances in Neural Information Processing Systems publication_identifier: isbn: - '9781713845393' publication_status: published quality_controlled: '1' status: public title: Dynamic inference with neural interpreters type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 34 year: '2021' ... --- _id: '14181' abstract: - lang: eng text: Variational Inference makes a trade-off between the capacity of the variational family and the tractability of finding an approximate posterior distribution. Instead, Boosting Variational Inference allows practitioners to obtain increasingly good posterior approximations by spending more compute. The main obstacle to widespread adoption of Boosting Variational Inference is the amount of resources necessary to improve over a strong Variational Inference baseline. In our work, we trace this limitation back to the global curvature of the KL-divergence. We characterize how the global curvature impacts time and memory consumption, address the problem with the notion of local curvature, and provide a novel approximate backtracking algorithm for estimating local curvature. We give new theoretical convergence rates for our algorithms and provide experimental validation on synthetic and real-world datasets. article_processing_charge: No arxiv: 1 author: - first_name: Gideon full_name: Dresdner, Gideon last_name: Dresdner - first_name: Saurav full_name: Shekhar, Saurav last_name: Shekhar - first_name: Fabian full_name: Pedregosa, Fabian last_name: Pedregosa - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch citation: ama: 'Dresdner G, Shekhar S, Pedregosa F, Locatello F, Rätsch G. Boosting variational inference with locally adaptive step-sizes. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. International Joint Conferences on Artificial Intelligence; 2021:2337-2343. doi:10.24963/ijcai.2021/322' apa: 'Dresdner, G., Shekhar, S., Pedregosa, F., Locatello, F., & Rätsch, G. (2021). Boosting variational inference with locally adaptive step-sizes. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (pp. 2337–2343). Montreal, Canada: International Joint Conferences on Artificial Intelligence. https://doi.org/10.24963/ijcai.2021/322' chicago: Dresdner, Gideon, Saurav Shekhar, Fabian Pedregosa, Francesco Locatello, and Gunnar Rätsch. “Boosting Variational Inference with Locally Adaptive Step-Sizes.” In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, 2337–43. International Joint Conferences on Artificial Intelligence, 2021. https://doi.org/10.24963/ijcai.2021/322. ieee: G. Dresdner, S. Shekhar, F. Pedregosa, F. Locatello, and G. Rätsch, “Boosting variational inference with locally adaptive step-sizes,” in Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, Montreal, Canada, 2021, pp. 2337–2343. ista: 'Dresdner G, Shekhar S, Pedregosa F, Locatello F, Rätsch G. 2021. Boosting variational inference with locally adaptive step-sizes. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence. IJCAI: International Joint Conference on Artificial Intelligence, 2337–2343.' mla: Dresdner, Gideon, et al. “Boosting Variational Inference with Locally Adaptive Step-Sizes.” Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, International Joint Conferences on Artificial Intelligence, 2021, pp. 2337–43, doi:10.24963/ijcai.2021/322. short: G. Dresdner, S. Shekhar, F. Pedregosa, F. Locatello, G. Rätsch, in:, Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, International Joint Conferences on Artificial Intelligence, 2021, pp. 2337–2343. conference: end_date: 2021-08-27 location: Montreal, Canada name: 'IJCAI: International Joint Conference on Artificial Intelligence' start_date: 2021-08-19 date_created: 2023-08-22T14:05:14Z date_published: 2021-05-19T00:00:00Z date_updated: 2023-09-11T11:14:30Z day: '19' department: - _id: FrLo doi: 10.24963/ijcai.2021/322 extern: '1' external_id: arxiv: - '2105.09240' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2105.09240 month: '05' oa: 1 oa_version: Published Version page: 2337-2343 publication: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence publication_identifier: eisbn: - '9780999241196' publication_status: published publisher: International Joint Conferences on Artificial Intelligence quality_controlled: '1' status: public title: Boosting variational inference with locally adaptive step-sizes type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2021' ... --- _id: '14182' abstract: - lang: eng text: "When machine learning systems meet real world applications, accuracy is only\r\none of several requirements. In this paper, we assay a complementary\r\nperspective originating from the increasing availability of pre-trained and\r\nregularly improving state-of-the-art models. While new improved models develop\r\nat a fast pace, downstream tasks vary more slowly or stay constant. Assume that\r\nwe have a large unlabelled data set for which we want to maintain accurate\r\npredictions. Whenever a new and presumably better ML models becomes available,\r\nwe encounter two problems: (i) given a limited budget, which data points should\r\nbe re-evaluated using the new model?; and (ii) if the new predictions differ\r\nfrom the current ones, should we update? Problem (i) is about compute cost,\r\nwhich matters for very large data sets and models. Problem (ii) is about\r\nmaintaining consistency of the predictions, which can be highly relevant for\r\ndownstream applications; our demand is to avoid negative flips, i.e., changing\r\ncorrect to incorrect predictions. In this paper, we formalize the Prediction\r\nUpdate Problem and present an efficient probabilistic approach as answer to the\r\nabove questions. In extensive experiments on standard classification benchmark\r\ndata sets, we show that our method outperforms alternative strategies along key\r\nmetrics for backward-compatible prediction updates." article_processing_charge: No arxiv: 1 author: - first_name: Frederik full_name: Träuble, Frederik last_name: Träuble - first_name: Julius von full_name: Kügelgen, Julius von last_name: Kügelgen - first_name: Matthäus full_name: Kleindessner, Matthäus last_name: Kleindessner - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Peter full_name: Gehler, Peter last_name: Gehler citation: ama: 'Träuble F, Kügelgen J von, Kleindessner M, Locatello F, Schölkopf B, Gehler P. Backward-compatible prediction updates: A probabilistic approach. In: 35th Conference on Neural Information Processing Systems. Vol 34. ; 2021:116-128.' apa: 'Träuble, F., Kügelgen, J. von, Kleindessner, M., Locatello, F., Schölkopf, B., & Gehler, P. (2021). Backward-compatible prediction updates: A probabilistic approach. In 35th Conference on Neural Information Processing Systems (Vol. 34, pp. 116–128). Virtual.' chicago: 'Träuble, Frederik, Julius von Kügelgen, Matthäus Kleindessner, Francesco Locatello, Bernhard Schölkopf, and Peter Gehler. “Backward-Compatible Prediction Updates: A Probabilistic Approach.” In 35th Conference on Neural Information Processing Systems, 34:116–28, 2021.' ieee: 'F. Träuble, J. von Kügelgen, M. Kleindessner, F. Locatello, B. Schölkopf, and P. Gehler, “Backward-compatible prediction updates: A probabilistic approach,” in 35th Conference on Neural Information Processing Systems, Virtual, 2021, vol. 34, pp. 116–128.' ista: 'Träuble F, Kügelgen J von, Kleindessner M, Locatello F, Schölkopf B, Gehler P. 2021. Backward-compatible prediction updates: A probabilistic approach. 35th Conference on Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 34, 116–128.' mla: 'Träuble, Frederik, et al. “Backward-Compatible Prediction Updates: A Probabilistic Approach.” 35th Conference on Neural Information Processing Systems, vol. 34, 2021, pp. 116–28.' short: F. Träuble, J. von Kügelgen, M. Kleindessner, F. Locatello, B. Schölkopf, P. Gehler, in:, 35th Conference on Neural Information Processing Systems, 2021, pp. 116–128. conference: end_date: 2021-12-10 location: Virtual name: 'NeurIPS: Neural Information Processing Systems' start_date: 2021-12-07 date_created: 2023-08-22T14:05:41Z date_published: 2021-07-02T00:00:00Z date_updated: 2023-09-11T11:31:59Z day: '02' department: - _id: FrLo extern: '1' external_id: arxiv: - '2107.01057' intvolume: ' 34' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2107.01057 month: '07' oa: 1 oa_version: Preprint page: 116-128 publication: 35th Conference on Neural Information Processing Systems publication_identifier: isbn: - '9781713845393' publication_status: published quality_controlled: '1' status: public title: 'Backward-compatible prediction updates: A probabilistic approach' type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 34 year: '2021' ... --- _id: '14221' abstract: - lang: eng text: 'The world is structured in countless ways. It may be prudent to enforce corresponding structural properties to a learning algorithm''s solution, such as incorporating prior beliefs, natural constraints, or causal structures. Doing so may translate to faster, more accurate, and more flexible models, which may directly relate to real-world impact. In this dissertation, we consider two different research areas that concern structuring a learning algorithm''s solution: when the structure is known and when it has to be discovered.' article_number: '2111.13693' article_processing_charge: No arxiv: 1 author: - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 citation: ama: Locatello F. Enforcing and discovering structure in machine learning. arXiv. doi:10.48550/arXiv.2111.13693 apa: Locatello, F. (n.d.). Enforcing and discovering structure in machine learning. arXiv. https://doi.org/10.48550/arXiv.2111.13693 chicago: Locatello, Francesco. “Enforcing and Discovering Structure in Machine Learning.” ArXiv, n.d. https://doi.org/10.48550/arXiv.2111.13693. ieee: F. Locatello, “Enforcing and discovering structure in machine learning,” arXiv. . ista: Locatello F. Enforcing and discovering structure in machine learning. arXiv, 2111.13693. mla: Locatello, Francesco. “Enforcing and Discovering Structure in Machine Learning.” ArXiv, 2111.13693, doi:10.48550/arXiv.2111.13693. short: F. Locatello, ArXiv (n.d.). date_created: 2023-08-22T14:23:35Z date_published: 2021-11-26T00:00:00Z date_updated: 2023-09-12T07:04:44Z day: '26' department: - _id: FrLo doi: 10.48550/arXiv.2111.13693 extern: '1' external_id: arxiv: - '2111.13693' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2111.13693 month: '11' oa: 1 oa_version: Preprint publication: arXiv publication_status: submitted status: public title: Enforcing and discovering structure in machine learning type: preprint user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2021' ... --- _id: '14326' abstract: - lang: eng text: "Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features. Yet, most deep learning approaches learn distributed representations that do not capture the compositional properties of natural scenes. In this paper, we present the Slot Attention module, an architectural component that interfaces with perceptual representations such as the output of a convolutional neural network and produces a set of task-dependent abstract representations which we call slots. These slots are exchangeable and can bind to any object in the input by specializing through a competitive procedure over multiple rounds of attention. We empirically demonstrate that Slot Attention can extract object-centric representations that enable generalization to unseen compositions when trained on unsupervised object discovery and supervised property prediction tasks.\r\n\r\n" article_processing_charge: No arxiv: 1 author: - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Dirk full_name: Weissenborn, Dirk last_name: Weissenborn - first_name: Thomas full_name: Unterthiner, Thomas last_name: Unterthiner - first_name: Aravindh full_name: Mahendran, Aravindh last_name: Mahendran - first_name: Georg full_name: Heigold, Georg last_name: Heigold - first_name: Jakob full_name: Uszkoreit, Jakob last_name: Uszkoreit - first_name: Alexey full_name: Dosovitskiy, Alexey last_name: Dosovitskiy - first_name: Thomas full_name: Kipf, Thomas last_name: Kipf citation: ama: 'Locatello F, Weissenborn D, Unterthiner T, et al. Object-centric learning with slot attention. In: Advances in Neural Information Processing Systems. Vol 33. Curran Associates; 2020:11525-11538.' apa: 'Locatello, F., Weissenborn, D., Unterthiner, T., Mahendran, A., Heigold, G., Uszkoreit, J., … Kipf, T. (2020). Object-centric learning with slot attention. In Advances in Neural Information Processing Systems (Vol. 33, pp. 11525–11538). Virtual: Curran Associates.' chicago: Locatello, Francesco, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. “Object-Centric Learning with Slot Attention.” In Advances in Neural Information Processing Systems, 33:11525–38. Curran Associates, 2020. ieee: F. Locatello et al., “Object-centric learning with slot attention,” in Advances in Neural Information Processing Systems, Virtual, 2020, vol. 33, pp. 11525–11538. ista: 'Locatello F, Weissenborn D, Unterthiner T, Mahendran A, Heigold G, Uszkoreit J, Dosovitskiy A, Kipf T. 2020. Object-centric learning with slot attention. Advances in Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 33, 11525–11538.' mla: Locatello, Francesco, et al. “Object-Centric Learning with Slot Attention.” Advances in Neural Information Processing Systems, vol. 33, Curran Associates, 2020, pp. 11525–38. short: F. Locatello, D. Weissenborn, T. Unterthiner, A. Mahendran, G. Heigold, J. Uszkoreit, A. Dosovitskiy, T. Kipf, in:, Advances in Neural Information Processing Systems, Curran Associates, 2020, pp. 11525–11538. conference: end_date: 2020-12-12 location: Virtual name: 'NeurIPS: Neural Information Processing Systems' start_date: 2020-12-06 date_created: 2023-09-13T12:03:46Z date_published: 2020-01-01T00:00:00Z date_updated: 2023-09-13T12:19:19Z department: - _id: FrLo extern: '1' external_id: arxiv: - '2006.15055' intvolume: ' 33' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2006.15055 oa: 1 oa_version: Preprint page: 11525-11538 publication: Advances in Neural Information Processing Systems publication_identifier: isbn: - '9781713829546' publication_status: published publisher: Curran Associates quality_controlled: '1' status: public title: Object-centric learning with slot attention type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 33 year: '2020' ... --- _id: '14125' abstract: - lang: eng text: "Motivation: Recent technological advances have led to an increase in the production and availability of single-cell data. The ability to integrate a set of multi-technology measurements would allow the identification of biologically or clinically meaningful observations through the unification of the perspectives afforded by each technology. In most cases, however, profiling technologies consume the used cells and thus pairwise correspondences between datasets are lost. Due to the sheer size single-cell datasets can acquire, scalable algorithms that are able to universally match single-cell measurements carried out in one cell to its corresponding sibling in another technology are needed.\r\nResults: We propose Single-Cell data Integration via Matching (SCIM), a scalable approach to recover such correspondences in two or more technologies. SCIM assumes that cells share a common (low-dimensional) underlying structure and that the underlying cell distribution is approximately constant across technologies. It constructs a technology-invariant latent space using an autoencoder framework with an adversarial objective. Multi-modal datasets are integrated by pairing cells across technologies using a bipartite matching scheme that operates on the low-dimensional latent representations. We evaluate SCIM on a simulated cellular branching process and show that the cell-to-cell matches derived by SCIM reflect the same pseudotime on the simulated dataset. Moreover, we apply our method to two real-world scenarios, a melanoma tumor sample and a human bone marrow sample, where we pair cells from a scRNA dataset to their sibling cells in a CyTOF dataset achieving 90% and 78% cell-matching accuracy for each one of the samples, respectively." article_processing_charge: No article_type: original author: - first_name: Stefan G full_name: Stark, Stefan G last_name: Stark - first_name: Joanna full_name: Ficek, Joanna last_name: Ficek - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Ximena full_name: Bonilla, Ximena last_name: Bonilla - first_name: Stéphane full_name: Chevrier, Stéphane last_name: Chevrier - first_name: Franziska full_name: Singer, Franziska last_name: Singer - first_name: Rudolf full_name: Aebersold, Rudolf last_name: Aebersold - first_name: Faisal S full_name: Al-Quaddoomi, Faisal S last_name: Al-Quaddoomi - first_name: Jonas full_name: Albinus, Jonas last_name: Albinus - first_name: Ilaria full_name: Alborelli, Ilaria last_name: Alborelli - first_name: Sonali full_name: Andani, Sonali last_name: Andani - first_name: Per-Olof full_name: Attinger, Per-Olof last_name: Attinger - first_name: Marina full_name: Bacac, Marina last_name: Bacac - first_name: Daniel full_name: Baumhoer, Daniel last_name: Baumhoer - first_name: Beatrice full_name: Beck-Schimmer, Beatrice last_name: Beck-Schimmer - first_name: Niko full_name: Beerenwinkel, Niko last_name: Beerenwinkel - first_name: Christian full_name: Beisel, Christian last_name: Beisel - first_name: Lara full_name: Bernasconi, Lara last_name: Bernasconi - first_name: Anne full_name: Bertolini, Anne last_name: Bertolini - first_name: Bernd full_name: Bodenmiller, Bernd last_name: Bodenmiller - first_name: Ximena full_name: Bonilla, Ximena last_name: Bonilla - first_name: Ruben full_name: Casanova, Ruben last_name: Casanova - first_name: Stéphane full_name: Chevrier, Stéphane last_name: Chevrier - first_name: Natalia full_name: Chicherova, Natalia last_name: Chicherova - first_name: Maya full_name: D'Costa, Maya last_name: D'Costa - first_name: Esther full_name: Danenberg, Esther last_name: Danenberg - first_name: Natalie full_name: Davidson, Natalie last_name: Davidson - first_name: Monica-Andreea Dră full_name: gan, Monica-Andreea Dră last_name: gan - first_name: Reinhard full_name: Dummer, Reinhard last_name: Dummer - first_name: Stefanie full_name: Engler, Stefanie last_name: Engler - first_name: Martin full_name: Erkens, Martin last_name: Erkens - first_name: Katja full_name: Eschbach, Katja last_name: Eschbach - first_name: Cinzia full_name: Esposito, Cinzia last_name: Esposito - first_name: André full_name: Fedier, André last_name: Fedier - first_name: Pedro full_name: Ferreira, Pedro last_name: Ferreira - first_name: Joanna full_name: Ficek, Joanna last_name: Ficek - first_name: Anja L full_name: Frei, Anja L last_name: Frei - first_name: Bruno full_name: Frey, Bruno last_name: Frey - first_name: Sandra full_name: Goetze, Sandra last_name: Goetze - first_name: Linda full_name: Grob, Linda last_name: Grob - first_name: Gabriele full_name: Gut, Gabriele last_name: Gut - first_name: Detlef full_name: Günther, Detlef last_name: Günther - first_name: Martina full_name: Haberecker, Martina last_name: Haberecker - first_name: Pirmin full_name: Haeuptle, Pirmin last_name: Haeuptle - first_name: Viola full_name: Heinzelmann-Schwarz, Viola last_name: Heinzelmann-Schwarz - first_name: Sylvia full_name: Herter, Sylvia last_name: Herter - first_name: Rene full_name: Holtackers, Rene last_name: Holtackers - first_name: Tamara full_name: Huesser, Tamara last_name: Huesser - first_name: Anja full_name: Irmisch, Anja last_name: Irmisch - first_name: Francis full_name: Jacob, Francis last_name: Jacob - first_name: Andrea full_name: Jacobs, Andrea last_name: Jacobs - first_name: Tim M full_name: Jaeger, Tim M last_name: Jaeger - first_name: Katharina full_name: Jahn, Katharina last_name: Jahn - first_name: Alva R full_name: James, Alva R last_name: James - first_name: Philip M full_name: Jermann, Philip M last_name: Jermann - first_name: André full_name: Kahles, André last_name: Kahles - first_name: Abdullah full_name: Kahraman, Abdullah last_name: Kahraman - first_name: Viktor H full_name: Koelzer, Viktor H last_name: Koelzer - first_name: Werner full_name: Kuebler, Werner last_name: Kuebler - first_name: Jack full_name: Kuipers, Jack last_name: Kuipers - first_name: Christian P full_name: Kunze, Christian P last_name: Kunze - first_name: Christian full_name: Kurzeder, Christian last_name: Kurzeder - first_name: Kjong-Van full_name: Lehmann, Kjong-Van last_name: Lehmann - first_name: Mitchell full_name: Levesque, Mitchell last_name: Levesque - first_name: Sebastian full_name: Lugert, Sebastian last_name: Lugert - first_name: Gerd full_name: Maass, Gerd last_name: Maass - first_name: Markus full_name: Manz, Markus last_name: Manz - first_name: Philipp full_name: Markolin, Philipp last_name: Markolin - first_name: Julien full_name: Mena, Julien last_name: Mena - first_name: Ulrike full_name: Menzel, Ulrike last_name: Menzel - first_name: Julian M full_name: Metzler, Julian M last_name: Metzler - first_name: Nicola full_name: Miglino, Nicola last_name: Miglino - first_name: Emanuela S full_name: Milani, Emanuela S last_name: Milani - first_name: Holger full_name: Moch, Holger last_name: Moch - first_name: Simone full_name: Muenst, Simone last_name: Muenst - first_name: Riccardo full_name: Murri, Riccardo last_name: Murri - first_name: Charlotte KY full_name: Ng, Charlotte KY last_name: Ng - first_name: Stefan full_name: Nicolet, Stefan last_name: Nicolet - first_name: Marta full_name: Nowak, Marta last_name: Nowak - first_name: Patrick GA full_name: Pedrioli, Patrick GA last_name: Pedrioli - first_name: Lucas full_name: Pelkmans, Lucas last_name: Pelkmans - first_name: Salvatore full_name: Piscuoglio, Salvatore last_name: Piscuoglio - first_name: Michael full_name: Prummer, Michael last_name: Prummer - first_name: Mathilde full_name: Ritter, Mathilde last_name: Ritter - first_name: Christian full_name: Rommel, Christian last_name: Rommel - first_name: María L full_name: Rosano-González, María L last_name: Rosano-González - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch - first_name: Natascha full_name: Santacroce, Natascha last_name: Santacroce - first_name: Jacobo Sarabia del full_name: Castillo, Jacobo Sarabia del last_name: Castillo - first_name: Ramona full_name: Schlenker, Ramona last_name: Schlenker - first_name: Petra C full_name: Schwalie, Petra C last_name: Schwalie - first_name: Severin full_name: Schwan, Severin last_name: Schwan - first_name: Tobias full_name: Schär, Tobias last_name: Schär - first_name: Gabriela full_name: Senti, Gabriela last_name: Senti - first_name: Franziska full_name: Singer, Franziska last_name: Singer - first_name: Sujana full_name: Sivapatham, Sujana last_name: Sivapatham - first_name: Berend full_name: Snijder, Berend last_name: Snijder - first_name: Bettina full_name: Sobottka, Bettina last_name: Sobottka - first_name: Vipin T full_name: Sreedharan, Vipin T last_name: Sreedharan - first_name: Stefan full_name: Stark, Stefan last_name: Stark - first_name: Daniel J full_name: Stekhoven, Daniel J last_name: Stekhoven - first_name: Alexandre PA full_name: Theocharides, Alexandre PA last_name: Theocharides - first_name: Tinu M full_name: Thomas, Tinu M last_name: Thomas - first_name: Markus full_name: Tolnay, Markus last_name: Tolnay - first_name: Vinko full_name: Tosevski, Vinko last_name: Tosevski - first_name: Nora C full_name: Toussaint, Nora C last_name: Toussaint - first_name: Mustafa A full_name: Tuncel, Mustafa A last_name: Tuncel - first_name: Marina full_name: Tusup, Marina last_name: Tusup - first_name: Audrey Van full_name: Drogen, Audrey Van last_name: Drogen - first_name: Marcus full_name: Vetter, Marcus last_name: Vetter - first_name: Tatjana full_name: Vlajnic, Tatjana last_name: Vlajnic - first_name: Sandra full_name: Weber, Sandra last_name: Weber - first_name: Walter P full_name: Weber, Walter P last_name: Weber - first_name: Rebekka full_name: Wegmann, Rebekka last_name: Wegmann - first_name: Michael full_name: Weller, Michael last_name: Weller - first_name: Fabian full_name: Wendt, Fabian last_name: Wendt - first_name: Norbert full_name: Wey, Norbert last_name: Wey - first_name: Andreas full_name: Wicki, Andreas last_name: Wicki - first_name: Bernd full_name: Wollscheid, Bernd last_name: Wollscheid - first_name: Shuqing full_name: Yu, Shuqing last_name: Yu - first_name: Johanna full_name: Ziegler, Johanna last_name: Ziegler - first_name: Marc full_name: Zimmermann, Marc last_name: Zimmermann - first_name: Martin full_name: Zoche, Martin last_name: Zoche - first_name: Gregor full_name: Zuend, Gregor last_name: Zuend - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch - first_name: Kjong-Van full_name: Lehmann, Kjong-Van last_name: Lehmann citation: ama: 'Stark SG, Ficek J, Locatello F, et al. SCIM: Universal single-cell matching with unpaired feature sets. Bioinformatics. 2020;36(Supplement_2):i919-i927. doi:10.1093/bioinformatics/btaa843' apa: 'Stark, S. G., Ficek, J., Locatello, F., Bonilla, X., Chevrier, S., Singer, F., … Lehmann, K.-V. (2020). SCIM: Universal single-cell matching with unpaired feature sets. Bioinformatics. Oxford University Press. https://doi.org/10.1093/bioinformatics/btaa843' chicago: 'Stark, Stefan G, Joanna Ficek, Francesco Locatello, Ximena Bonilla, Stéphane Chevrier, Franziska Singer, Rudolf Aebersold, et al. “SCIM: Universal Single-Cell Matching with Unpaired Feature Sets.” Bioinformatics. Oxford University Press, 2020. https://doi.org/10.1093/bioinformatics/btaa843.' ieee: 'S. G. Stark et al., “SCIM: Universal single-cell matching with unpaired feature sets,” Bioinformatics, vol. 36, no. Supplement_2. Oxford University Press, pp. i919–i927, 2020.' ista: 'Stark SG et al. 2020. SCIM: Universal single-cell matching with unpaired feature sets. Bioinformatics. 36(Supplement_2), i919–i927.' mla: 'Stark, Stefan G., et al. “SCIM: Universal Single-Cell Matching with Unpaired Feature Sets.” Bioinformatics, vol. 36, no. Supplement_2, Oxford University Press, 2020, pp. i919–27, doi:10.1093/bioinformatics/btaa843.' short: S.G. Stark, J. Ficek, F. Locatello, X. Bonilla, S. Chevrier, F. Singer, R. Aebersold, F.S. Al-Quaddoomi, J. Albinus, I. Alborelli, S. Andani, P.-O. Attinger, M. Bacac, D. Baumhoer, B. Beck-Schimmer, N. Beerenwinkel, C. Beisel, L. Bernasconi, A. Bertolini, B. Bodenmiller, X. Bonilla, R. Casanova, S. Chevrier, N. Chicherova, M. D’Costa, E. Danenberg, N. Davidson, M.-A.D. gan, R. Dummer, S. Engler, M. Erkens, K. Eschbach, C. Esposito, A. Fedier, P. Ferreira, J. Ficek, A.L. Frei, B. Frey, S. Goetze, L. Grob, G. Gut, D. Günther, M. Haberecker, P. Haeuptle, V. Heinzelmann-Schwarz, S. Herter, R. Holtackers, T. Huesser, A. Irmisch, F. Jacob, A. Jacobs, T.M. Jaeger, K. Jahn, A.R. James, P.M. Jermann, A. Kahles, A. Kahraman, V.H. Koelzer, W. Kuebler, J. Kuipers, C.P. Kunze, C. Kurzeder, K.-V. Lehmann, M. Levesque, S. Lugert, G. Maass, M. Manz, P. Markolin, J. Mena, U. Menzel, J.M. Metzler, N. Miglino, E.S. Milani, H. Moch, S. Muenst, R. Murri, C.K. Ng, S. Nicolet, M. Nowak, P.G. Pedrioli, L. Pelkmans, S. Piscuoglio, M. Prummer, M. Ritter, C. Rommel, M.L. Rosano-González, G. Rätsch, N. Santacroce, J.S. del Castillo, R. Schlenker, P.C. Schwalie, S. Schwan, T. Schär, G. Senti, F. Singer, S. Sivapatham, B. Snijder, B. Sobottka, V.T. Sreedharan, S. Stark, D.J. Stekhoven, A.P. Theocharides, T.M. Thomas, M. Tolnay, V. Tosevski, N.C. Toussaint, M.A. Tuncel, M. Tusup, A.V. Drogen, M. Vetter, T. Vlajnic, S. Weber, W.P. Weber, R. Wegmann, M. Weller, F. Wendt, N. Wey, A. Wicki, B. Wollscheid, S. Yu, J. Ziegler, M. Zimmermann, M. Zoche, G. Zuend, G. Rätsch, K.-V. Lehmann, Bioinformatics 36 (2020) i919–i927. date_created: 2023-08-21T12:28:20Z date_published: 2020-12-01T00:00:00Z date_updated: 2023-09-11T10:21:00Z day: '01' department: - _id: FrLo doi: 10.1093/bioinformatics/btaa843 extern: '1' external_id: pmid: - '33381818' intvolume: ' 36' issue: Supplement_2 keyword: - Computational Mathematics - Computational Theory and Mathematics - Computer Science Applications - Molecular Biology - Biochemistry - Statistics and Probability language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.1093/bioinformatics/btaa843 month: '12' oa: 1 oa_version: Published Version page: i919-i927 pmid: 1 publication: Bioinformatics publication_identifier: eissn: - 1367-4811 publication_status: published publisher: Oxford University Press quality_controlled: '1' related_material: link: - relation: software url: https://github.com/ratschlab/scim scopus_import: '1' status: public title: 'SCIM: Universal single-cell matching with unpaired feature sets' type: journal_article user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 36 year: '2020' ... --- _id: '14186' abstract: - lang: eng text: "The goal of the unsupervised learning of disentangled representations is to\r\nseparate the independent explanatory factors of variation in the data without\r\naccess to supervision. In this paper, we summarize the results of Locatello et\r\nal., 2019, and focus on their implications for practitioners. We discuss the\r\ntheoretical result showing that the unsupervised learning of disentangled\r\nrepresentations is fundamentally impossible without inductive biases and the\r\npractical challenges it entails. Finally, we comment on our experimental\r\nfindings, highlighting the limitations of state-of-the-art approaches and\r\ndirections for future research." article_processing_charge: No arxiv: 1 author: - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer - first_name: Mario full_name: Lucic, Mario last_name: Lucic - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch - first_name: Sylvain full_name: Gelly, Sylvain last_name: Gelly - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Olivier full_name: Bachem, Olivier last_name: Bachem citation: ama: 'Locatello F, Bauer S, Lucic M, et al. A commentary on the unsupervised learning of disentangled representations. In: The 34th AAAI Conference on Artificial Intelligence. Vol 34. Association for the Advancement of Artificial Intelligence; 2020:13681-13684. doi:10.1609/aaai.v34i09.7120' apa: 'Locatello, F., Bauer, S., Lucic, M., Rätsch, G., Gelly, S., Schölkopf, B., & Bachem, O. (2020). A commentary on the unsupervised learning of disentangled representations. In The 34th AAAI Conference on Artificial Intelligence (Vol. 34, pp. 13681–13684). New York, NY, United States: Association for the Advancement of Artificial Intelligence. https://doi.org/10.1609/aaai.v34i09.7120' chicago: Locatello, Francesco, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. “A Commentary on the Unsupervised Learning of Disentangled Representations.” In The 34th AAAI Conference on Artificial Intelligence, 34:13681–84. Association for the Advancement of Artificial Intelligence, 2020. https://doi.org/10.1609/aaai.v34i09.7120. ieee: F. Locatello et al., “A commentary on the unsupervised learning of disentangled representations,” in The 34th AAAI Conference on Artificial Intelligence, New York, NY, United States, 2020, vol. 34, no. 9, pp. 13681–13684. ista: 'Locatello F, Bauer S, Lucic M, Rätsch G, Gelly S, Schölkopf B, Bachem O. 2020. A commentary on the unsupervised learning of disentangled representations. The 34th AAAI Conference on Artificial Intelligence. AAAI: Conference on Artificial Intelligence vol. 34, 13681–13684.' mla: Locatello, Francesco, et al. “A Commentary on the Unsupervised Learning of Disentangled Representations.” The 34th AAAI Conference on Artificial Intelligence, vol. 34, no. 9, Association for the Advancement of Artificial Intelligence, 2020, pp. 13681–84, doi:10.1609/aaai.v34i09.7120. short: F. Locatello, S. Bauer, M. Lucic, G. Rätsch, S. Gelly, B. Schölkopf, O. Bachem, in:, The 34th AAAI Conference on Artificial Intelligence, Association for the Advancement of Artificial Intelligence, 2020, pp. 13681–13684. conference: end_date: 2020-02-12 location: New York, NY, United States name: 'AAAI: Conference on Artificial Intelligence' start_date: 2020-02-07 date_created: 2023-08-22T14:07:26Z date_published: 2020-07-28T00:00:00Z date_updated: 2023-09-12T07:44:48Z day: '28' department: - _id: FrLo doi: 10.1609/aaai.v34i09.7120 extern: '1' external_id: arxiv: - '2007.14184' intvolume: ' 34' issue: '9' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2007.14184 month: '07' oa: 1 oa_version: Preprint page: 13681-13684 publication: The 34th AAAI Conference on Artificial Intelligence publication_identifier: eissn: - 2374-3468 isbn: - '9781577358350' publication_status: published publisher: Association for the Advancement of Artificial Intelligence quality_controlled: '1' scopus_import: '1' status: public title: A commentary on the unsupervised learning of disentangled representations type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 34 year: '2020' ... --- _id: '14187' abstract: - lang: eng text: "We propose a novel Stochastic Frank-Wolfe (a.k.a. conditional gradient)\r\nalgorithm for constrained smooth finite-sum minimization with a generalized\r\nlinear prediction/structure. This class of problems includes empirical risk\r\nminimization with sparse, low-rank, or other structured constraints. The\r\nproposed method is simple to implement, does not require step-size tuning, and\r\nhas a constant per-iteration cost that is independent of the dataset size.\r\nFurthermore, as a byproduct of the method we obtain a stochastic estimator of\r\nthe Frank-Wolfe gap that can be used as a stopping criterion. Depending on the\r\nsetting, the proposed method matches or improves on the best computational\r\nguarantees for Stochastic Frank-Wolfe algorithms. Benchmarks on several\r\ndatasets highlight different regimes in which the proposed method exhibits a\r\nfaster empirical convergence than related methods. Finally, we provide an\r\nimplementation of all considered methods in an open-source package." alternative_title: - PMLR article_processing_charge: No arxiv: 1 author: - first_name: Geoffrey full_name: Négiar, Geoffrey last_name: Négiar - first_name: Gideon full_name: Dresdner, Gideon last_name: Dresdner - first_name: Alicia full_name: Tsai, Alicia last_name: Tsai - first_name: Laurent El full_name: Ghaoui, Laurent El last_name: Ghaoui - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Robert M. full_name: Freund, Robert M. last_name: Freund - first_name: Fabian full_name: Pedregosa, Fabian last_name: Pedregosa citation: ama: 'Négiar G, Dresdner G, Tsai A, et al. Stochastic Frank-Wolfe for constrained finite-sum minimization. In: Proceedings of the 37th International Conference on Machine Learning. Vol 119. ; 2020:7253-7262.' apa: Négiar, G., Dresdner, G., Tsai, A., Ghaoui, L. E., Locatello, F., Freund, R. M., & Pedregosa, F. (2020). Stochastic Frank-Wolfe for constrained finite-sum minimization. In Proceedings of the 37th International Conference on Machine Learning (Vol. 119, pp. 7253–7262). Virtual. chicago: Négiar, Geoffrey, Gideon Dresdner, Alicia Tsai, Laurent El Ghaoui, Francesco Locatello, Robert M. Freund, and Fabian Pedregosa. “Stochastic Frank-Wolfe for Constrained Finite-Sum Minimization.” In Proceedings of the 37th International Conference on Machine Learning, 119:7253–62, 2020. ieee: G. Négiar et al., “Stochastic Frank-Wolfe for constrained finite-sum minimization,” in Proceedings of the 37th International Conference on Machine Learning, Virtual, 2020, vol. 119, pp. 7253–7262. ista: Négiar G, Dresdner G, Tsai A, Ghaoui LE, Locatello F, Freund RM, Pedregosa F. 2020. Stochastic Frank-Wolfe for constrained finite-sum minimization. Proceedings of the 37th International Conference on Machine Learning. International Conference on Machine Learning, PMLR, vol. 119, 7253–7262. mla: Négiar, Geoffrey, et al. “Stochastic Frank-Wolfe for Constrained Finite-Sum Minimization.” Proceedings of the 37th International Conference on Machine Learning, vol. 119, 2020, pp. 7253–62. short: G. Négiar, G. Dresdner, A. Tsai, L.E. Ghaoui, F. Locatello, R.M. Freund, F. Pedregosa, in:, Proceedings of the 37th International Conference on Machine Learning, 2020, pp. 7253–7262. conference: end_date: 2020-07-18 location: Virtual name: International Conference on Machine Learning start_date: 2020-07-13 date_created: 2023-08-22T14:07:52Z date_published: 2020-07-27T00:00:00Z date_updated: 2023-09-12T08:03:40Z day: '27' department: - _id: FrLo extern: '1' external_id: arxiv: - '2002.11860' intvolume: ' 119' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2002.11860 month: '07' oa: 1 oa_version: Preprint page: 7253-7262 publication: Proceedings of the 37th International Conference on Machine Learning publication_status: published quality_controlled: '1' status: public title: Stochastic Frank-Wolfe for constrained finite-sum minimization type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 119 year: '2020' ... --- _id: '14188' abstract: - lang: eng text: "Intelligent agents should be able to learn useful representations by\r\nobserving changes in their environment. We model such observations as pairs of\r\nnon-i.i.d. images sharing at least one of the underlying factors of variation.\r\nFirst, we theoretically show that only knowing how many factors have changed,\r\nbut not which ones, is sufficient to learn disentangled representations.\r\nSecond, we provide practical algorithms that learn disentangled representations\r\nfrom pairs of images without requiring annotation of groups, individual\r\nfactors, or the number of factors that have changed. Third, we perform a\r\nlarge-scale empirical study and show that such pairs of observations are\r\nsufficient to reliably learn disentangled representations on several benchmark\r\ndata sets. Finally, we evaluate our learned representations and find that they\r\nare simultaneously useful on a diverse suite of tasks, including generalization\r\nunder covariate shifts, fairness, and abstract reasoning. Overall, our results\r\ndemonstrate that weak supervision enables learning of useful disentangled\r\nrepresentations in realistic scenarios." alternative_title: - PMLR article_processing_charge: No arxiv: 1 author: - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Ben full_name: Poole, Ben last_name: Poole - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Olivier full_name: Bachem, Olivier last_name: Bachem - first_name: Michael full_name: Tschannen, Michael last_name: Tschannen citation: ama: 'Locatello F, Poole B, Rätsch G, Schölkopf B, Bachem O, Tschannen M. Weakly-supervised disentanglement without compromises. In: Proceedings of the 37th International Conference on Machine Learning. Vol 119. ; 2020:6348–6359.' apa: Locatello, F., Poole, B., Rätsch, G., Schölkopf, B., Bachem, O., & Tschannen, M. (2020). Weakly-supervised disentanglement without compromises. In Proceedings of the 37th International Conference on Machine Learning (Vol. 119, pp. 6348–6359). Virtual. chicago: Locatello, Francesco, Ben Poole, Gunnar Rätsch, Bernhard Schölkopf, Olivier Bachem, and Michael Tschannen. “Weakly-Supervised Disentanglement without Compromises.” In Proceedings of the 37th International Conference on Machine Learning, 119:6348–6359, 2020. ieee: F. Locatello, B. Poole, G. Rätsch, B. Schölkopf, O. Bachem, and M. Tschannen, “Weakly-supervised disentanglement without compromises,” in Proceedings of the 37th International Conference on Machine Learning, Virtual, 2020, vol. 119, pp. 6348–6359. ista: Locatello F, Poole B, Rätsch G, Schölkopf B, Bachem O, Tschannen M. 2020. Weakly-supervised disentanglement without compromises. Proceedings of the 37th International Conference on Machine Learning. International Conference on Machine Learning, PMLR, vol. 119, 6348–6359. mla: Locatello, Francesco, et al. “Weakly-Supervised Disentanglement without Compromises.” Proceedings of the 37th International Conference on Machine Learning, vol. 119, 2020, pp. 6348–6359. short: F. Locatello, B. Poole, G. Rätsch, B. Schölkopf, O. Bachem, M. Tschannen, in:, Proceedings of the 37th International Conference on Machine Learning, 2020, pp. 6348–6359. conference: end_date: 2020-07-18 location: Virtual name: International Conference on Machine Learning start_date: 2020-07-13 date_created: 2023-08-22T14:08:14Z date_published: 2020-07-07T00:00:00Z date_updated: 2023-09-12T07:59:29Z day: '07' department: - _id: FrLo extern: '1' external_id: arxiv: - '2002.02886' intvolume: ' 119' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/2002.02886 month: '07' oa: 1 oa_version: Preprint page: 6348–6359 publication: Proceedings of the 37th International Conference on Machine Learning publication_status: published quality_controlled: '1' scopus_import: '1' status: public title: Weakly-supervised disentanglement without compromises type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 119 year: '2020' ... --- _id: '14195' abstract: - lang: eng text: "The idea behind the unsupervised learning of disentangled representations is that real-world data is generated by a few explanatory factors of variation which can be recovered by unsupervised learning algorithms. In this paper, we provide a sober look at recent progress in the field and challenge some common assumptions. We first theoretically show that the unsupervised learning of disentangled representations is fundamentally impossible without inductive biases on both the models and the data. Then, we train over 14000\r\n models covering most prominent methods and evaluation metrics in a reproducible large-scale experimental study on eight data sets. We observe that while the different methods successfully enforce properties “encouraged” by the corresponding losses, well-disentangled models seemingly cannot be identified without supervision. Furthermore, different evaluation metrics do not always agree on what should be considered “disentangled” and exhibit systematic differences in the estimation. Finally, increased disentanglement does not seem to necessarily lead to a decreased sample complexity of learning for downstream tasks. Our results suggest that future work on disentanglement learning should be explicit about the role of inductive biases and (implicit) supervision, investigate concrete benefits of enforcing disentanglement of the learned representations, and consider a reproducible experimental setup covering several data sets." article_number: '209' article_processing_charge: No article_type: original arxiv: 1 author: - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer - first_name: Mario full_name: Lucic, Mario last_name: Lucic - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch - first_name: Sylvain full_name: Gelly, Sylvain last_name: Gelly - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Olivier full_name: Bachem, Olivier last_name: Bachem citation: ama: Locatello F, Bauer S, Lucic M, et al. A sober look at the unsupervised learning of disentangled representations and their evaluation. Journal of Machine Learning Research. 2020;21. apa: Locatello, F., Bauer, S., Lucic, M., Rätsch, G., Gelly, S., Schölkopf, B., & Bachem, O. (2020). A sober look at the unsupervised learning of disentangled representations and their evaluation. Journal of Machine Learning Research. MIT Press. chicago: Locatello, Francesco, Stefan Bauer, Mario Lucic, Gunnar Rätsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. “A Sober Look at the Unsupervised Learning of Disentangled Representations and Their Evaluation.” Journal of Machine Learning Research. MIT Press, 2020. ieee: F. Locatello et al., “A sober look at the unsupervised learning of disentangled representations and their evaluation,” Journal of Machine Learning Research, vol. 21. MIT Press, 2020. ista: Locatello F, Bauer S, Lucic M, Rätsch G, Gelly S, Schölkopf B, Bachem O. 2020. A sober look at the unsupervised learning of disentangled representations and their evaluation. Journal of Machine Learning Research. 21, 209. mla: Locatello, Francesco, et al. “A Sober Look at the Unsupervised Learning of Disentangled Representations and Their Evaluation.” Journal of Machine Learning Research, vol. 21, 209, MIT Press, 2020. short: F. Locatello, S. Bauer, M. Lucic, G. Rätsch, S. Gelly, B. Schölkopf, O. Bachem, Journal of Machine Learning Research 21 (2020). date_created: 2023-08-22T14:10:34Z date_published: 2020-09-01T00:00:00Z date_updated: 2023-09-12T09:23:56Z day: '01' ddc: - '000' department: - _id: FrLo extern: '1' external_id: arxiv: - '2010.14766' has_accepted_license: '1' intvolume: ' 21' language: - iso: eng main_file_link: - open_access: '1' url: https://jmlr.csail.mit.edu/papers/v21/19-976.html month: '09' oa: 1 oa_version: Published Version publication: Journal of Machine Learning Research publication_status: published publisher: MIT Press quality_controlled: '1' scopus_import: '1' status: public title: A sober look at the unsupervised learning of disentangled representations and their evaluation tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: journal_article user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 21 year: '2020' ... --- _id: '14184' abstract: - lang: eng text: "Learning disentangled representations is considered a cornerstone problem in\r\nrepresentation learning. Recently, Locatello et al. (2019) demonstrated that\r\nunsupervised disentanglement learning without inductive biases is theoretically\r\nimpossible and that existing inductive biases and unsupervised methods do not\r\nallow to consistently learn disentangled representations. However, in many\r\npractical settings, one might have access to a limited amount of supervision,\r\nfor example through manual labeling of (some) factors of variation in a few\r\ntraining examples. In this paper, we investigate the impact of such supervision\r\non state-of-the-art disentanglement methods and perform a large scale study,\r\ntraining over 52000 models under well-defined and reproducible experimental\r\nconditions. We observe that a small number of labeled examples (0.01--0.5\\% of\r\nthe data set), with potentially imprecise and incomplete labels, is sufficient\r\nto perform model selection on state-of-the-art unsupervised models. Further, we\r\ninvestigate the benefit of incorporating supervision into the training process.\r\nOverall, we empirically validate that with little and imprecise supervision it\r\nis possible to reliably learn disentangled representations." article_processing_charge: No arxiv: 1 author: - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Michael full_name: Tschannen, Michael last_name: Tschannen - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer - first_name: Gunnar full_name: Rätsch, Gunnar last_name: Rätsch - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Olivier full_name: Bachem, Olivier last_name: Bachem citation: ama: 'Locatello F, Tschannen M, Bauer S, Rätsch G, Schölkopf B, Bachem O. Disentangling factors of variation using few labels. In: 8th International Conference on Learning Representations. ; 2019.' apa: Locatello, F., Tschannen, M., Bauer, S., Rätsch, G., Schölkopf, B., & Bachem, O. (2019). Disentangling factors of variation using few labels. In 8th International Conference on Learning Representations. Virtual. chicago: Locatello, Francesco, Michael Tschannen, Stefan Bauer, Gunnar Rätsch, Bernhard Schölkopf, and Olivier Bachem. “Disentangling Factors of Variation Using Few Labels.” In 8th International Conference on Learning Representations, 2019. ieee: F. Locatello, M. Tschannen, S. Bauer, G. Rätsch, B. Schölkopf, and O. Bachem, “Disentangling factors of variation using few labels,” in 8th International Conference on Learning Representations, Virtual, 2019. ista: 'Locatello F, Tschannen M, Bauer S, Rätsch G, Schölkopf B, Bachem O. 2019. Disentangling factors of variation using few labels. 8th International Conference on Learning Representations. ICLR: International Conference on Learning Representations.' mla: Locatello, Francesco, et al. “Disentangling Factors of Variation Using Few Labels.” 8th International Conference on Learning Representations, 2019. short: F. Locatello, M. Tschannen, S. Bauer, G. Rätsch, B. Schölkopf, O. Bachem, in:, 8th International Conference on Learning Representations, 2019. conference: end_date: 2020-05-01 location: Virtual name: 'ICLR: International Conference on Learning Representations' start_date: 2020-04-26 date_created: 2023-08-22T14:06:37Z date_published: 2019-12-20T00:00:00Z date_updated: 2023-09-12T07:01:34Z day: '20' department: - _id: FrLo extern: '1' external_id: arxiv: - '1905.01258' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1905.01258 month: '12' oa: 1 oa_version: Preprint publication: 8th International Conference on Learning Representations publication_status: published quality_controlled: '1' scopus_import: '1' status: public title: Disentangling factors of variation using few labels type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2019' ... --- _id: '14189' abstract: - lang: eng text: "We consider the problem of recovering a common latent source with independent\r\ncomponents from multiple views. This applies to settings in which a variable is\r\nmeasured with multiple experimental modalities, and where the goal is to\r\nsynthesize the disparate measurements into a single unified representation. We\r\nconsider the case that the observed views are a nonlinear mixing of\r\ncomponent-wise corruptions of the sources. When the views are considered\r\nseparately, this reduces to nonlinear Independent Component Analysis (ICA) for\r\nwhich it is provably impossible to undo the mixing. We present novel\r\nidentifiability proofs that this is possible when the multiple views are\r\nconsidered jointly, showing that the mixing can theoretically be undone using\r\nfunction approximators such as deep neural networks. In contrast to known\r\nidentifiability results for nonlinear ICA, we prove that independent latent\r\nsources with arbitrary mixing can be recovered as long as multiple,\r\nsufficiently different noisy views are available." alternative_title: - PMLR article_processing_charge: No arxiv: 1 author: - first_name: Luigi full_name: Gresele, Luigi last_name: Gresele - first_name: Paul K. full_name: Rubenstein, Paul K. last_name: Rubenstein - first_name: Arash full_name: Mehrjou, Arash last_name: Mehrjou - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf citation: ama: 'Gresele L, Rubenstein PK, Mehrjou A, Locatello F, Schölkopf B. The incomplete Rosetta Stone problem: Identifiability results for multi-view nonlinear ICA. In: Proceedings of the 35th Conference on Uncertainty in Artificial  Intelligence. Vol 115. ML Research Press; 2019:217-227.' apa: 'Gresele, L., Rubenstein, P. K., Mehrjou, A., Locatello, F., & Schölkopf, B. (2019). The incomplete Rosetta Stone problem: Identifiability results for multi-view nonlinear ICA. In Proceedings of the 35th Conference on Uncertainty in Artificial  Intelligence (Vol. 115, pp. 217–227). Tel Aviv, Israel: ML Research Press.' chicago: 'Gresele, Luigi, Paul K. Rubenstein, Arash Mehrjou, Francesco Locatello, and Bernhard Schölkopf. “The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICA.” In Proceedings of the 35th Conference on Uncertainty in Artificial  Intelligence, 115:217–27. ML Research Press, 2019.' ieee: 'L. Gresele, P. K. Rubenstein, A. Mehrjou, F. Locatello, and B. Schölkopf, “The incomplete Rosetta Stone problem: Identifiability results for multi-view nonlinear ICA,” in Proceedings of the 35th Conference on Uncertainty in Artificial  Intelligence, Tel Aviv, Israel, 2019, vol. 115, pp. 217–227.' ista: 'Gresele L, Rubenstein PK, Mehrjou A, Locatello F, Schölkopf B. 2019. The incomplete Rosetta Stone problem: Identifiability results for multi-view nonlinear ICA. Proceedings of the 35th Conference on Uncertainty in Artificial  Intelligence. UAI: Uncertainty in Artificial Intelligence, PMLR, vol. 115, 217–227.' mla: 'Gresele, Luigi, et al. “The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICA.” Proceedings of the 35th Conference on Uncertainty in Artificial  Intelligence, vol. 115, ML Research Press, 2019, pp. 217–27.' short: L. Gresele, P.K. Rubenstein, A. Mehrjou, F. Locatello, B. Schölkopf, in:, Proceedings of the 35th Conference on Uncertainty in Artificial  Intelligence, ML Research Press, 2019, pp. 217–227. conference: end_date: 2019-07-25 location: Tel Aviv, Israel name: 'UAI: Uncertainty in Artificial Intelligence' start_date: 2019-07-22 date_created: 2023-08-22T14:08:35Z date_published: 2019-05-16T00:00:00Z date_updated: 2023-09-12T08:07:38Z day: '16' department: - _id: FrLo extern: '1' external_id: arxiv: - '1905.06642' intvolume: ' 115' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1905.06642 month: '05' oa: 1 oa_version: Preprint page: 217-227 publication: Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence publication_status: published publisher: ML Research Press quality_controlled: '1' scopus_import: '1' status: public title: 'The incomplete Rosetta Stone problem: Identifiability results for multi-view nonlinear ICA' type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 115 year: '2019' ... --- _id: '14190' abstract: - lang: eng text: "Learning meaningful and compact representations with disentangled semantic\r\naspects is considered to be of key importance in representation learning. Since\r\nreal-world data is notoriously costly to collect, many recent state-of-the-art\r\ndisentanglement models have heavily relied on synthetic toy data-sets. In this\r\npaper, we propose a novel data-set which consists of over one million images of\r\nphysical 3D objects with seven factors of variation, such as object color,\r\nshape, size and position. In order to be able to control all the factors of\r\nvariation precisely, we built an experimental platform where the objects are\r\nbeing moved by a robotic arm. In addition, we provide two more datasets which\r\nconsist of simulations of the experimental setup. These datasets provide for\r\nthe first time the possibility to systematically investigate how well different\r\ndisentanglement methods perform on real data in comparison to simulation, and\r\nhow simulated data can be leveraged to build better representations of the real\r\nworld. We provide a first experimental study of these questions and our results\r\nindicate that learned models transfer poorly, but that model and hyperparameter\r\nselection is an effective means of transferring information to the real world." article_processing_charge: No arxiv: 1 author: - first_name: Muhammad Waleed full_name: Gondal, Muhammad Waleed last_name: Gondal - first_name: Manuel full_name: Wüthrich, Manuel last_name: Wüthrich - first_name: Đorđe full_name: Miladinović, Đorđe last_name: Miladinović - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Martin full_name: Breidt, Martin last_name: Breidt - first_name: Valentin full_name: Volchkov, Valentin last_name: Volchkov - first_name: Joel full_name: Akpo, Joel last_name: Akpo - first_name: Olivier full_name: Bachem, Olivier last_name: Bachem - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer citation: ama: 'Gondal MW, Wüthrich M, Miladinović Đ, et al. On the transfer of inductive bias from simulation to the real world: a new disentanglement dataset. In: Advances in Neural Information Processing Systems. Vol 32. ; 2019.' apa: 'Gondal, M. W., Wüthrich, M., Miladinović, Đ., Locatello, F., Breidt, M., Volchkov, V., … Bauer, S. (2019). On the transfer of inductive bias from simulation to the real world: a new disentanglement dataset. In Advances in Neural Information Processing Systems (Vol. 32). Vancouver, Canada.' chicago: 'Gondal, Muhammad Waleed, Manuel Wüthrich, Đorđe Miladinović, Francesco Locatello, Martin Breidt, Valentin Volchkov, Joel Akpo, Olivier Bachem, Bernhard Schölkopf, and Stefan Bauer. “On the Transfer of Inductive Bias from Simulation to the Real World: A New Disentanglement Dataset.” In Advances in Neural Information Processing Systems, Vol. 32, 2019.' ieee: 'M. W. Gondal et al., “On the transfer of inductive bias from simulation to the real world: a new disentanglement dataset,” in Advances in Neural Information Processing Systems, Vancouver, Canada, 2019, vol. 32.' ista: 'Gondal MW, Wüthrich M, Miladinović Đ, Locatello F, Breidt M, Volchkov V, Akpo J, Bachem O, Schölkopf B, Bauer S. 2019. On the transfer of inductive bias from simulation to the real world: a new disentanglement dataset. Advances in Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 32.' mla: 'Gondal, Muhammad Waleed, et al. “On the Transfer of Inductive Bias from Simulation to the Real World: A New Disentanglement Dataset.” Advances in Neural Information Processing Systems, vol. 32, 2019.' short: M.W. Gondal, M. Wüthrich, Đ. Miladinović, F. Locatello, M. Breidt, V. Volchkov, J. Akpo, O. Bachem, B. Schölkopf, S. Bauer, in:, Advances in Neural Information Processing Systems, 2019. conference: end_date: 2019-12-14 location: Vancouver, Canada name: 'NeurIPS: Neural Information Processing Systems' start_date: 2019-12-08 date_created: 2023-08-22T14:09:13Z date_published: 2019-06-07T00:00:00Z date_updated: 2023-09-13T09:46:38Z day: '07' department: - _id: FrLo extern: '1' external_id: arxiv: - '1906.03292' intvolume: ' 32' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1906.03292 month: '06' oa: 1 oa_version: Preprint publication: Advances in Neural Information Processing Systems publication_identifier: isbn: - '9781713807933' publication_status: published quality_controlled: '1' status: public title: 'On the transfer of inductive bias from simulation to the real world: a new disentanglement dataset' type: conference user_id: c635000d-4b10-11ee-a964-aac5a93f6ac1 volume: 32 year: '2019' ... --- _id: '14191' abstract: - lang: eng text: A broad class of convex optimization problems can be formulated as a semidefinite program (SDP), minimization of a convex function over the positive-semidefinite cone subject to some affine constraints. The majority of classical SDP solvers are designed for the deterministic setting where problem data is readily available. In this setting, generalized conditional gradient methods (aka Frank-Wolfe-type methods) provide scalable solutions by leveraging the so-called linear minimization oracle instead of the projection onto the semidefinite cone. Most problems in machine learning and modern engineering applications, however, contain some degree of stochasticity. In this work, we propose the first conditional-gradient-type method for solving stochastic optimization problems under affine constraints. Our method guarantees O(k−1/3) convergence rate in expectation on the objective residual and O(k−5/12) on the feasibility gap. article_processing_charge: No arxiv: 1 author: - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Alp full_name: Yurtsever, Alp last_name: Yurtsever - first_name: Olivier full_name: Fercoq, Olivier last_name: Fercoq - first_name: Volkan full_name: Cevher, Volkan last_name: Cevher citation: ama: 'Locatello F, Yurtsever A, Fercoq O, Cevher V. Stochastic Frank-Wolfe for composite convex minimization. In: Advances in Neural Information Processing Systems. Vol 32. ; 2019:14291–14301.' apa: Locatello, F., Yurtsever, A., Fercoq, O., & Cevher, V. (2019). Stochastic Frank-Wolfe for composite convex minimization. In Advances in Neural Information Processing Systems (Vol. 32, pp. 14291–14301). Vancouver, Canada. chicago: Locatello, Francesco, Alp Yurtsever, Olivier Fercoq, and Volkan Cevher. “Stochastic Frank-Wolfe for Composite Convex Minimization.” In Advances in Neural Information Processing Systems, 32:14291–14301, 2019. ieee: F. Locatello, A. Yurtsever, O. Fercoq, and V. Cevher, “Stochastic Frank-Wolfe for composite convex minimization,” in Advances in Neural Information Processing Systems, Vancouver, Canada, 2019, vol. 32, pp. 14291–14301. ista: 'Locatello F, Yurtsever A, Fercoq O, Cevher V. 2019. Stochastic Frank-Wolfe for composite convex minimization. Advances in Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 32, 14291–14301.' mla: Locatello, Francesco, et al. “Stochastic Frank-Wolfe for Composite Convex Minimization.” Advances in Neural Information Processing Systems, vol. 32, 2019, pp. 14291–14301. short: F. Locatello, A. Yurtsever, O. Fercoq, V. Cevher, in:, Advances in Neural Information Processing Systems, 2019, pp. 14291–14301. conference: end_date: 2019-12-14 location: Vancouver, Canada name: 'NeurIPS: Neural Information Processing Systems' start_date: 2019-12-08 date_created: 2023-08-22T14:09:35Z date_published: 2019-12-29T00:00:00Z date_updated: 2023-09-12T08:48:45Z day: '29' department: - _id: FrLo extern: '1' external_id: arxiv: - '1901.10348' intvolume: ' 32' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1901.10348 month: '12' oa: 1 oa_version: Preprint page: 14291–14301 publication: Advances in Neural Information Processing Systems publication_identifier: isbn: - '9781713807933' publication_status: published quality_controlled: '1' scopus_import: '1' status: public title: Stochastic Frank-Wolfe for composite convex minimization type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 32 year: '2019' ... --- _id: '14193' abstract: - lang: eng text: "A disentangled representation encodes information about the salient factors\r\nof variation in the data independently. Although it is often argued that this\r\nrepresentational format is useful in learning to solve many real-world\r\ndown-stream tasks, there is little empirical evidence that supports this claim.\r\nIn this paper, we conduct a large-scale study that investigates whether\r\ndisentangled representations are more suitable for abstract reasoning tasks.\r\nUsing two new tasks similar to Raven's Progressive Matrices, we evaluate the\r\nusefulness of the representations learned by 360 state-of-the-art unsupervised\r\ndisentanglement models. Based on these representations, we train 3600 abstract\r\nreasoning models and observe that disentangled representations do in fact lead\r\nto better down-stream performance. In particular, they enable quicker learning\r\nusing fewer samples." article_processing_charge: No arxiv: 1 author: - first_name: Sjoerd van full_name: Steenkiste, Sjoerd van last_name: Steenkiste - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Jürgen full_name: Schmidhuber, Jürgen last_name: Schmidhuber - first_name: Olivier full_name: Bachem, Olivier last_name: Bachem citation: ama: 'Steenkiste S van, Locatello F, Schmidhuber J, Bachem O. Are disentangled representations helpful for abstract visual reasoning? In: Advances in Neural Information Processing Systems. Vol 32. ; 2019.' apa: Steenkiste, S. van, Locatello, F., Schmidhuber, J., & Bachem, O. (2019). Are disentangled representations helpful for abstract visual reasoning? In Advances in Neural Information Processing Systems (Vol. 32). Vancouver, Canada. chicago: Steenkiste, Sjoerd van, Francesco Locatello, Jürgen Schmidhuber, and Olivier Bachem. “Are Disentangled Representations Helpful for Abstract Visual Reasoning?” In Advances in Neural Information Processing Systems, Vol. 32, 2019. ieee: S. van Steenkiste, F. Locatello, J. Schmidhuber, and O. Bachem, “Are disentangled representations helpful for abstract visual reasoning?,” in Advances in Neural Information Processing Systems, Vancouver, Canada, 2019, vol. 32. ista: 'Steenkiste S van, Locatello F, Schmidhuber J, Bachem O. 2019. Are disentangled representations helpful for abstract visual reasoning? Advances in Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 32.' mla: Steenkiste, Sjoerd van, et al. “Are Disentangled Representations Helpful for Abstract Visual Reasoning?” Advances in Neural Information Processing Systems, vol. 32, 2019. short: S. van Steenkiste, F. Locatello, J. Schmidhuber, O. Bachem, in:, Advances in Neural Information Processing Systems, 2019. conference: end_date: 2019-12-14 location: Vancouver, Canada name: 'NeurIPS: Neural Information Processing Systems' start_date: 2019-12-08 date_created: 2023-08-22T14:09:53Z date_published: 2019-05-29T00:00:00Z date_updated: 2023-09-12T09:02:43Z day: '29' department: - _id: FrLo extern: '1' external_id: arxiv: - '1905.12506' intvolume: ' 32' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.1905.12506 month: '05' oa: 1 oa_version: Preprint publication: Advances in Neural Information Processing Systems publication_identifier: isbn: - '9781713807933' publication_status: published quality_controlled: '1' status: public title: Are disentangled representations helpful for abstract visual reasoning? type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 32 year: '2019' ... --- _id: '14197' abstract: - lang: eng text: "Recently there has been a significant interest in learning disentangled\r\nrepresentations, as they promise increased interpretability, generalization to\r\nunseen scenarios and faster learning on downstream tasks. In this paper, we\r\ninvestigate the usefulness of different notions of disentanglement for\r\nimproving the fairness of downstream prediction tasks based on representations.\r\nWe consider the setting where the goal is to predict a target variable based on\r\nthe learned representation of high-dimensional observations (such as images)\r\nthat depend on both the target variable and an \\emph{unobserved} sensitive\r\nvariable. We show that in this setting both the optimal and empirical\r\npredictions can be unfair, even if the target variable and the sensitive\r\nvariable are independent. Analyzing the representations of more than\r\n\\num{12600} trained state-of-the-art disentangled models, we observe that\r\nseveral disentanglement scores are consistently correlated with increased\r\nfairness, suggesting that disentanglement may be a useful property to encourage\r\nfairness when sensitive variables are not observed." article_processing_charge: No arxiv: 1 author: - first_name: Francesco full_name: Locatello, Francesco id: 26cfd52f-2483-11ee-8040-88983bcc06d4 last_name: Locatello orcid: 0000-0002-4850-0683 - first_name: Gabriele full_name: Abbati, Gabriele last_name: Abbati - first_name: Tom full_name: Rainforth, Tom last_name: Rainforth - first_name: Stefan full_name: Bauer, Stefan last_name: Bauer - first_name: Bernhard full_name: Schölkopf, Bernhard last_name: Schölkopf - first_name: Olivier full_name: Bachem, Olivier last_name: Bachem citation: ama: 'Locatello F, Abbati G, Rainforth T, Bauer S, Schölkopf B, Bachem O. On the fairness of disentangled representations. In: Advances in Neural Information Processing Systems. Vol 32. ; 2019:14611–14624.' apa: Locatello, F., Abbati, G., Rainforth, T., Bauer, S., Schölkopf, B., & Bachem, O. (2019). On the fairness of disentangled representations. In Advances in Neural Information Processing Systems (Vol. 32, pp. 14611–14624). Vancouver, Canada. chicago: Locatello, Francesco, Gabriele Abbati, Tom Rainforth, Stefan Bauer, Bernhard Schölkopf, and Olivier Bachem. “On the Fairness of Disentangled Representations.” In Advances in Neural Information Processing Systems, 32:14611–14624, 2019. ieee: F. Locatello, G. Abbati, T. Rainforth, S. Bauer, B. Schölkopf, and O. Bachem, “On the fairness of disentangled representations,” in Advances in Neural Information Processing Systems, Vancouver, Canada, 2019, vol. 32, pp. 14611–14624. ista: 'Locatello F, Abbati G, Rainforth T, Bauer S, Schölkopf B, Bachem O. 2019. On the fairness of disentangled representations. Advances in Neural Information Processing Systems. NeurIPS: Neural Information Processing Systems vol. 32, 14611–14624.' mla: Locatello, Francesco, et al. “On the Fairness of Disentangled Representations.” Advances in Neural Information Processing Systems, vol. 32, 2019, pp. 14611–14624. short: F. Locatello, G. Abbati, T. Rainforth, S. Bauer, B. Schölkopf, O. Bachem, in:, Advances in Neural Information Processing Systems, 2019, pp. 14611–14624. conference: end_date: 2019-12-14 location: Vancouver, Canada name: 'NeurIPS: Neural Information Processing Systems' start_date: 2019-12-08 date_created: 2023-08-22T14:12:28Z date_published: 2019-12-08T00:00:00Z date_updated: 2023-09-12T09:37:22Z day: '08' department: - _id: FrLo extern: '1' external_id: arxiv: - '1905.13662' intvolume: ' 32' language: - iso: eng main_file_link: - open_access: '1' url: https://arxiv.org/abs/1905.13662 month: '12' oa: 1 oa_version: Preprint page: 14611–14624 publication: Advances in Neural Information Processing Systems publication_identifier: isbn: - '9781713807933' publication_status: published quality_controlled: '1' scopus_import: '1' status: public title: On the fairness of disentangled representations type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 32 year: '2019' ...