{"title":"The effect of shape and illumination on material perception: Model and applications","quality_controlled":"1","status":"public","publisher":"Association for Computing Machinery","scopus_import":"1","oa_version":"Submitted Version","oa":1,"date_created":"2021-08-08T22:01:28Z","user_id":"4359f0d1-fa6c-11eb-b949-802e58b17ae8","department":[{"_id":"BeBi"}],"main_file_link":[{"url":"https://zaguan.unizar.es/record/110704/files/texto_completo.pdf","open_access":"1"}],"abstract":[{"text":"Material appearance hinges on material reflectance properties but also surface geometry and illumination. The unlimited number of potential combinations between these factors makes understanding and predicting material appearance a very challenging task. In this work, we collect a large-scale dataset of perceptual ratings of appearance attributes with more than 215,680 responses for 42,120 distinct combinations of material, shape, and illumination. The goal of this dataset is twofold. First, we analyze for the first time the effects of illumination and geometry in material perception across such a large collection of varied appearances. We connect our findings to those of the literature, discussing how previous knowledge generalizes across very diverse materials, shapes, and illuminations. Second, we use the collected dataset to train a deep learning architecture for predicting perceptual attributes that correlate with human judgments. We demonstrate the consistent and robust behavior of our predictor in various challenging scenarios, which, for the first time, enables estimating perceived material attributes from general 2D images. Since our predictor relies on the final appearance in an image, it can compare appearance properties across different geometries and illumination conditions. Finally, we demonstrate several applications that use our predictor, including appearance reproduction using 3D printing, BRDF editing by integrating our predictor in a differentiable renderer, illumination design, or material recommendations for scene design.","lang":"eng"}],"publication_status":"published","date_published":"2021-08-01T00:00:00Z","issue":"4","article_processing_charge":"No","month":"08","date_updated":"2023-08-10T14:20:10Z","type":"journal_article","day":"01","isi":1,"acknowledgement":"This project has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie, grant agreement Nº 765911 (RealVision) and from the European Research Council (ERC), grant agreement Nº 804226 (PERDY).","year":"2021","intvolume":"        40","volume":40,"_id":"9820","article_number":"125","language":[{"iso":"eng"}],"article_type":"original","citation":{"ieee":"A. Serrano <i>et al.</i>, “The effect of shape and illumination on material perception: Model and applications,” <i>ACM Transactions on Graphics</i>, vol. 40, no. 4. Association for Computing Machinery, 2021.","mla":"Serrano, Ana, et al. “The Effect of Shape and Illumination on Material Perception: Model and Applications.” <i>ACM Transactions on Graphics</i>, vol. 40, no. 4, 125, Association for Computing Machinery, 2021, doi:<a href=\"https://doi.org/10.1145/3450626.3459813\">10.1145/3450626.3459813</a>.","short":"A. Serrano, B. Chen, C. Wang, M. Piovarci, H.P. Seidel, P. Didyk, K. Myszkowski, ACM Transactions on Graphics 40 (2021).","ama":"Serrano A, Chen B, Wang C, et al. The effect of shape and illumination on material perception: Model and applications. <i>ACM Transactions on Graphics</i>. 2021;40(4). doi:<a href=\"https://doi.org/10.1145/3450626.3459813\">10.1145/3450626.3459813</a>","chicago":"Serrano, Ana, Bin Chen, Chao Wang, Michael Piovarci, Hans Peter Seidel, Piotr Didyk, and Karol Myszkowski. “The Effect of Shape and Illumination on Material Perception: Model and Applications.” <i>ACM Transactions on Graphics</i>. Association for Computing Machinery, 2021. <a href=\"https://doi.org/10.1145/3450626.3459813\">https://doi.org/10.1145/3450626.3459813</a>.","ista":"Serrano A, Chen B, Wang C, Piovarci M, Seidel HP, Didyk P, Myszkowski K. 2021. The effect of shape and illumination on material perception: Model and applications. ACM Transactions on Graphics. 40(4), 125.","apa":"Serrano, A., Chen, B., Wang, C., Piovarci, M., Seidel, H. P., Didyk, P., &#38; Myszkowski, K. (2021). The effect of shape and illumination on material perception: Model and applications. <i>ACM Transactions on Graphics</i>. Association for Computing Machinery. <a href=\"https://doi.org/10.1145/3450626.3459813\">https://doi.org/10.1145/3450626.3459813</a>"},"external_id":{"isi":["000674930900090"]},"publication_identifier":{"issn":["07300301"],"eissn":["15577368"]},"author":[{"full_name":"Serrano, Ana","first_name":"Ana","last_name":"Serrano"},{"full_name":"Chen, Bin","last_name":"Chen","first_name":"Bin"},{"full_name":"Wang, Chao","first_name":"Chao","last_name":"Wang"},{"last_name":"Piovarci","first_name":"Michael","full_name":"Piovarci, Michael","id":"62E473F4-5C99-11EA-A40E-AF823DDC885E","orcid":"0000-0002-5062-4474"},{"full_name":"Seidel, Hans Peter","last_name":"Seidel","first_name":"Hans Peter"},{"full_name":"Didyk, Piotr","first_name":"Piotr","last_name":"Didyk"},{"full_name":"Myszkowski, Karol","first_name":"Karol","last_name":"Myszkowski"}],"publication":"ACM Transactions on Graphics","doi":"10.1145/3450626.3459813"}