To achieve movie-level special effects, Apple's new patent can simulate facial expressions in real time to monsters similar to "Lord of the Rings" Gollum.

CTOnews.com, February 7 (Xinhua)-- Apple acquired Zurich-based Faceshift in September October 2015 and launched Animoji and Memoji after digestion and absorption. According to a list published by the United States Trademark and Patent Office (USPTO) on January 26, 2023, Apple received a patent from Faceshift dating back to 2013.

The inventors of the patent include Mark Pauley, a senior software engineer at Apple, and Sofien Bouaziz, a former Faceshift engineer. The latter's profile shows that he "designed, developed and manufactured a real-time facial tracking algorithm for iPhone X Animoji and provided it to third-party developers through ARKit." Bouaziz worked at Google after leaving Apple in 2018 and is now the XR Presence director of Meta.

The patent relates to a real-time facial animation production method and a real-time facial animation processing device. The method includes providing a dynamic expression model, receiving tracking data corresponding to a user's facial expression, estimating tracking parameters based on the dynamic expression model and tracking data, and improving the dynamic expression model based on the tracking data and estimated tracking parameters.

CTOnews.com learned from Apple's patent map that the system can dynamically simulate faces to Gollum monsters similar to the Lord of the Rings, and can produce different facial expressions according to facial changes.