Baidu's new patent enables virtual digital people to learn lip movements deeply.

CTOnews.com February 13 news, February 7, Beijing Baidu Netcom Technology Co., Ltd., "virtual object-driven method, deep learning network training method and device" patent was granted.

The patent summary shows that the patent relates to the field of artificial intelligence, especially in the fields of deep learning, computer vision, virtual / augmented reality and image processing technology, and can be applied to virtual digital human, meta-universe and other scenarios. The specific implementation scheme includes: determining the initial speech feature in response to the acquired speech data; performing time sequence enhancement processing on the initial speech feature to obtain the target speech feature; based on the target speech feature and the reference facial image of the target virtual object, generate a lip image sequence; according to the lip image sequence, drive the target virtual object to make the target virtual object perform lip actions matching with the speech data.

Tu Yuan PixabayCTOnews.com found that, in short, the patent makes the lip movement of a virtual digital human closer to a real person when speaking. It is worth mentioning that Baidu has officially confirmed that it will release the similar ChatGPT project "Literary Heart word" (ERNIE Bot). If combined with this technology, it will be able to create a more realistic virtual digital human.