We are thrilled to announce the beta version of the Avatar SDK Leap. Leap is facial motion capture software that transforms a video of a person speaking on camera into a 3D animation of the person’s avatar. By leveraging neural networks to predict the shape and texture of the avatar, Leap delivers an unprecedented level of visual quality that transcends the uncanny valley. Currently, users will need an iPhone with the Leap Mobile application to record a video, but support for videos captured with standard cameras will be added shortly.

Leap consists of the Leap Mobile application that allows you to capture a video with face animation and plugins for Unity and Unreal Engine to transfer this animation to a 3d avatar on a scene. After capturing data with the Leap Mobile application, you can export the collected data to a computer running UE or Unity. The Avatar SDK Leap plugins can import this data and create realistic facial animation sequences for your 3D avatar in just a few clicks. The plugins will make all the required assets, including the skeletal mesh, textures, materials, and animations, in just a few minutes.

Leap Mobile application

Leap UE plugin

The Avatar SDK Leap UE plugin streamlines the integration of the groundbreaking Leap technology into your Unreal Engine projects. The Avatar SDK Leap UE plugin can import datasets captured by Leap Mobile and create realistic facial animation sequences for your 3D avatar in UE Editor. The ready-to-use Level Sequence will be added to your level, so you only need to play it to see your animation in action. 

Leap Unity plugin

The Avatar SDK Leap Unity plugin simplifies the integration of Leap technology into your Unity projects. The plugin imports this data seamlessly, enabling the creation of realistic facial animation sequences for your 3D avatars directly within the Unity Editor. A ready-to-use animation clip is automatically added to your scene, allowing you to preview your animations with a single play-action.

Based on this plugin, we also added the Windows Demo Application, which can help you visualize your captured dataset with an animation without using Unity. It requires only a Windows PC with the same prerequisites.

Sign up for a waitlist to be the first to get access!