3D Morphable Models of faces for medical applications

Outline

  • Preliminary 3D
  • Introduction to 3DMM
  • 3DMM and Orthognathic Surgery
  • Q&A
  • Conclusion

Preliminary 3D

  • Points (Vertices): x, y, z
  • A set of points: PointCloud (>80.000 points)
  • Edges: Connection between two points
  • Faces: Three connected Points, Triangle
  • 3D Mesh: Points + Edges+ Faces
  • Texture, Normals

Demo of 3D Mesh

3DMM

  • Introduced by Blanz and Vetter in 1999
  • Statistical shape and texture models
  • A 3DMM is a generative model for 3D object
  • Key elements: The mean object $\mu$ and the principal variations $\mathbf{U}$

How to create a 3DMM

  • Collect a dataset of 3D Meshes
  • Bring every mesh of your dataset to correspondence
  • Find the mean object and its principal variations

How to create a 3DMM

Large Scale Facial Model (LSFM)

  • Mein3D database of more 10000 faces has been used
  • 35 models for specific demographic groups
  • Wide variety of age, gender (48% male, 52% female), and ethnicity (82% White, 9% Asian, 5% mixed heritage, 3% Black and 1% other)

Age histogram

Specificity

Generalisation

Compactness

Comparison

Visualization

Mimic Me

  • A new dataset collected from April 2017-July 2017 at Science Museum London
  • ~5000 subjects, in various expresions
  • In total, more than 300,000 meshes
  • Variety in ethnicity(White 76.57%, Asian 12.41%, Black 2.27%, Other 6.61%, Mixed 2.13%)s, age, gender

Data Collection

3DMM and Orthognathic Surgery

Automated computer assisted plastic and reconstructive surgery diagnosis and planning

Purpose

  • Describe what a mean face looks like
    • Normal face
    • Patient face (preoperative and postoperative)
  • Automated diagnosis of a face (e.g. GP)
  • Automated surgery planning (e.g. specialist)

Datasets

  • Data from Boston and Harvard Hospital
  • ~151 subjects, with pre and post data
  • Mean age 18.4 (2.4), range 14-28
  • Variety in ethnicity(White 72%, Asian 10%, Black 8%, Other/Mixed 10%)

Intrinsic characteristics

Down arrow

Mean faces

Down arrow

Differences between mean faces

Down arrow

T-SNE

Down arrow

Classification

Down arrow

Surgery simulation

Down arrow
Down arrow

Q&A

Down arrow

Conclusion

  • 3DMM powerfull and flexible representations for normal cases
  • 3DMM can be used for craniosynostosis syndromes for diagnosis and prediction
  • Orthognathic surgery can also be benefited

Thank you

Down arrow
  • Lara Van de Lande, MD, PhD Candidate
  • Paul Knoops, Bioengineer, PhD Candidate