Face + Hair reconstruction

Description

I worked on the problem of 3D reconstruction of the face and hair from images and camera poses. Given a set of 2D images, the goal was to recover the underlying 3D geometry. Training data consisted of 3D head scans captured with phones. The pipeline, illustrated below, first extracted features from the images and then used a small MLP to predict an occupancy field in canonical space, combining neural networks with traditional computer vision techniques to achieve accurate reconstruction. The solution was integrated into production, delivering robust results for in-the-wild scenarios

BibTeX

@patent{ulianov2022methods,
      title={Methods and systems for generating a resolved threedimensional (R3D) avatar},
      author={Ulianov, Dmitrii and Pasechnik, Igor and Shabanov, Ahmedkhan and Lebedev, Vadim and Krotov, Ilya and Chinaev, Nikolai and Yakupov, Bulat and Poletaev, Vsevolod},
      year={2022},
      publisher={Google Patents},
      note={US Patent 11,494,963}
    }