{"status":"public","title":"Self-compatibility: Evaluating causal discovery without ground truth","year":"2023","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","date_created":"2023-09-13T12:44:59Z","oa_version":"Preprint","oa":1,"abstract":[{"lang":"eng","text":"As causal ground truth is incredibly rare, causal discovery algorithms are\r\ncommonly only evaluated on simulated data. This is concerning, given that\r\nsimulations reflect common preconceptions about generating processes regarding\r\nnoise distributions, model classes, and more. In this work, we propose a novel\r\nmethod for falsifying the output of a causal discovery algorithm in the absence\r\nof ground truth. Our key insight is that while statistical learning seeks\r\nstability across subsets of data points, causal learning should seek stability\r\nacross subsets of variables. Motivated by this insight, our method relies on a\r\nnotion of compatibility between causal graphs learned on different subsets of\r\nvariables. We prove that detecting incompatibilities can falsify wrongly\r\ninferred causal relations due to violation of assumptions or errors from finite\r\nsample effects. Although passing such compatibility tests is only a necessary\r\ncriterion for good performance, we argue that it provides strong evidence for\r\nthe causal models whenever compatibility entails strong implications for the\r\njoint distribution. We also demonstrate experimentally that detection of\r\nincompatibilities can aid in causal model selection."}],"main_file_link":[{"url":"https://doi.org/10.48550/arXiv.2307.09552","open_access":"1"}],"_id":"14333","article_number":"2307.09552","language":[{"iso":"eng"}],"department":[{"_id":"FrLo"}],"publication_status":"submitted","citation":{"short":"P.M. Faller, L.C. Vankadara, A.A. Mastakouri, F. Locatello, D. Janzing, ArXiv (n.d.).","apa":"Faller, P. M., Vankadara, L. C., Mastakouri, A. A., Locatello, F., &#38; Janzing, D. (n.d.). Self-compatibility: Evaluating causal discovery without ground truth. <i>arXiv</i>. <a href=\"https://doi.org/10.48550/arXiv.2307.09552\">https://doi.org/10.48550/arXiv.2307.09552</a>","ista":"Faller PM, Vankadara LC, Mastakouri AA, Locatello F, Janzing D. Self-compatibility: Evaluating causal discovery without ground truth. arXiv, 2307.09552.","chicago":"Faller, Philipp M., Leena Chennuru Vankadara, Atalanti A. Mastakouri, Francesco Locatello, and Dominik Janzing. “Self-Compatibility: Evaluating Causal Discovery without Ground Truth.” <i>ArXiv</i>, n.d. <a href=\"https://doi.org/10.48550/arXiv.2307.09552\">https://doi.org/10.48550/arXiv.2307.09552</a>.","ama":"Faller PM, Vankadara LC, Mastakouri AA, Locatello F, Janzing D. Self-compatibility: Evaluating causal discovery without ground truth. <i>arXiv</i>. doi:<a href=\"https://doi.org/10.48550/arXiv.2307.09552\">10.48550/arXiv.2307.09552</a>","ieee":"P. M. Faller, L. C. Vankadara, A. A. Mastakouri, F. Locatello, and D. Janzing, “Self-compatibility: Evaluating causal discovery without ground truth,” <i>arXiv</i>. .","mla":"Faller, Philipp M., et al. “Self-Compatibility: Evaluating Causal Discovery without Ground Truth.” <i>ArXiv</i>, 2307.09552, doi:<a href=\"https://doi.org/10.48550/arXiv.2307.09552\">10.48550/arXiv.2307.09552</a>."},"date_published":"2023-07-18T00:00:00Z","extern":"1","date_updated":"2023-09-13T12:47:53Z","month":"07","article_processing_charge":"No","external_id":{"arxiv":["2307.09552"]},"publication":"arXiv","author":[{"full_name":"Faller, Philipp M.","first_name":"Philipp M.","last_name":"Faller"},{"first_name":"Leena Chennuru","last_name":"Vankadara","full_name":"Vankadara, Leena Chennuru"},{"full_name":"Mastakouri, Atalanti A.","last_name":"Mastakouri","first_name":"Atalanti A."},{"last_name":"Locatello","first_name":"Francesco","full_name":"Locatello, Francesco","id":"26cfd52f-2483-11ee-8040-88983bcc06d4","orcid":"0000-0002-4850-0683"},{"first_name":"Dominik","last_name":"Janzing","full_name":"Janzing, Dominik"}],"day":"18","doi":"10.48550/arXiv.2307.09552","type":"preprint"}