---
_id: '9957'
abstract:
- lang: eng
text: The reflectance field of a face describes the reflectance properties responsible
for complex lighting effects including diffuse, specular, inter-reflection and
self shadowing. Most existing methods for estimating the face reflectance from
a monocular image assume faces to be diffuse with very few approaches adding a
specular component. This still leaves out important perceptual aspects of reflectance
as higher-order global illumination effects and self-shadowing are not modeled.
We present a new neural representation for face reflectance where we can estimate
all components of the reflectance responsible for the final appearance from a
single monocular image. Instead of modeling each component of the reflectance
separately using parametric models, our neural representation allows us to generate
a basis set of faces in a geometric deformation-invariant space, parameterized
by the input light direction, viewpoint and face geometry. We learn to reconstruct
this reflectance field of a face just from a monocular image, which can be used
to render the face from any viewpoint in any light condition. Our method is trained
on a light-stage training dataset, which captures 300 people illuminated with
150 light conditions from 8 viewpoints. We show that our method outperforms existing
monocular reflectance reconstruction methods, in terms of photorealism due to
better capturing of physical premitives, such as sub-surface scattering, specularities,
self-shadows and other higher-order effects.
acknowledgement: "We thank Tarun Yenamandra and Duarte David for helping us with the
comparisons. This work was supported by the\r\nERC Consolidator Grant 4DReply (770784).
We also acknowledge support from InterDigital."
article_processing_charge: No
arxiv: 1
author:
- first_name: Mallikarjun
full_name: B R, Mallikarjun
last_name: B R
- first_name: Ayush
full_name: Tewari, Ayush
last_name: Tewari
- first_name: Tae-Hyun
full_name: Oh, Tae-Hyun
last_name: Oh
- first_name: Tim
full_name: Weyrich, Tim
last_name: Weyrich
- first_name: Bernd
full_name: Bickel, Bernd
id: 49876194-F248-11E8-B48F-1D18A9856A87
last_name: Bickel
orcid: 0000-0001-6511-9385
- first_name: Hans-Peter
full_name: Seidel, Hans-Peter
last_name: Seidel
- first_name: Hanspeter
full_name: Pfister, Hanspeter
last_name: Pfister
- first_name: Wojciech
full_name: Matusik, Wojciech
last_name: Matusik
- first_name: Mohamed
full_name: Elgharib, Mohamed
last_name: Elgharib
- first_name: Christian
full_name: Theobalt, Christian
last_name: Theobalt
citation:
ama: 'B R M, Tewari A, Oh T-H, et al. Monocular reconstruction of neural face reflectance
fields. In: Proceedings of the IEEE Computer Society Conference on Computer
Vision and Pattern Recognition. IEEE; 2021:4791-4800. doi:10.1109/CVPR46437.2021.00476'
apa: 'B R, M., Tewari, A., Oh, T.-H., Weyrich, T., Bickel, B., Seidel, H.-P., …
Theobalt, C. (2021). Monocular reconstruction of neural face reflectance fields.
In Proceedings of the IEEE Computer Society Conference on Computer Vision and
Pattern Recognition (pp. 4791–4800). Nashville, TN, United States; Virtual:
IEEE. https://doi.org/10.1109/CVPR46437.2021.00476'
chicago: B R, Mallikarjun, Ayush Tewari, Tae-Hyun Oh, Tim Weyrich, Bernd Bickel,
Hans-Peter Seidel, Hanspeter Pfister, Wojciech Matusik, Mohamed Elgharib, and
Christian Theobalt. “Monocular Reconstruction of Neural Face Reflectance Fields.”
In Proceedings of the IEEE Computer Society Conference on Computer Vision and
Pattern Recognition, 4791–4800. IEEE, 2021. https://doi.org/10.1109/CVPR46437.2021.00476.
ieee: M. B R et al., “Monocular reconstruction of neural face reflectance
fields,” in Proceedings of the IEEE Computer Society Conference on Computer
Vision and Pattern Recognition, Nashville, TN, United States; Virtual, 2021,
pp. 4791–4800.
ista: 'B R M, Tewari A, Oh T-H, Weyrich T, Bickel B, Seidel H-P, Pfister H, Matusik
W, Elgharib M, Theobalt C. 2021. Monocular reconstruction of neural face reflectance
fields. Proceedings of the IEEE Computer Society Conference on Computer Vision
and Pattern Recognition. CVPR: Conference on Computer Vision and Pattern Recognition,
4791–4800.'
mla: B R, Mallikarjun, et al. “Monocular Reconstruction of Neural Face Reflectance
Fields.” Proceedings of the IEEE Computer Society Conference on Computer Vision
and Pattern Recognition, IEEE, 2021, pp. 4791–800, doi:10.1109/CVPR46437.2021.00476.
short: M. B R, A. Tewari, T.-H. Oh, T. Weyrich, B. Bickel, H.-P. Seidel, H. Pfister,
W. Matusik, M. Elgharib, C. Theobalt, in:, Proceedings of the IEEE Computer Society
Conference on Computer Vision and Pattern Recognition, IEEE, 2021, pp. 4791–4800.
conference:
end_date: 2021-06-25
location: Nashville, TN, United States; Virtual
name: 'CVPR: Conference on Computer Vision and Pattern Recognition'
start_date: 2021-06-20
date_created: 2021-08-24T06:03:00Z
date_published: 2021-09-01T00:00:00Z
date_updated: 2023-08-11T11:08:35Z
day: '01'
ddc:
- '000'
department:
- _id: BeBi
doi: 10.1109/CVPR46437.2021.00476
external_id:
arxiv:
- '2008.10247'
isi:
- '000739917304096'
file:
- access_level: open_access
checksum: 961db0bde76dd87cf833930080bb9f38
content_type: application/pdf
creator: bbickel
date_created: 2021-08-24T06:02:15Z
date_updated: 2021-08-24T06:02:15Z
file_id: '9958'
file_name: R_Monocular_Reconstruction_of_Neural_Face_Reflectance_Fields_CVPR_2021_paper[1].pdf
file_size: 4746649
relation: main_file
file_date_updated: 2021-08-24T06:02:15Z
has_accepted_license: '1'
isi: 1
language:
- iso: eng
month: '09'
oa: 1
oa_version: Preprint
page: 4791-4800
publication: Proceedings of the IEEE Computer Society Conference on Computer Vision
and Pattern Recognition
publication_identifier:
isbn:
- 978-166544509-2
issn:
- 1063-6919
publication_status: published
publisher: IEEE
quality_controlled: '1'
scopus_import: '1'
status: public
title: Monocular reconstruction of neural face reflectance fields
type: conference
user_id: 4359f0d1-fa6c-11eb-b949-802e58b17ae8
year: '2021'
...