Lip Sync - Intimidating Animation Tab ScreenShot - Blender 2.8

Automated Lip Sync Animation for Any 3D Model – Blender Rhubarb Plugin

It is possible to set up an automated lip sync animation in Blender 2.8 for any character or a random object you find on a file repository website. Want to see a talking pipe? Yes, there is an automatically lip synced talking pipe animation on YouTube. Nearly every file on STL sharing websites can be animated so that it speaks automatically in sync with your speech.

Animation Tool in Blender 2.8

Blender3D is a sandbox. The more I use it, the more functionalities I discover. My most recent finds are animation rigs and armatures, tools that allow to create movement for any object you want, in a smooth and natural fashion. Additionally, you can change the colors of the objects in various transitions. There may be tools out there better suited for particular tasks, but I find it very convenient to have it all, designing, sculpting, etc., in a software that I am familiar with. And Blender 3D completely nailed it.

Automatic Lip Syncing - 3DPG Mouth Positions - Shapekeys
Automatic Lip Syncing – 3DPG Mouth Positions – Shapekeys
(Image: 3DWithUs)

Lip Syncing

Lip syncing seems to be one of the most difficult tasks in character/object animation. There are dozens of different mouth positions, each covering a particular sound: E, A, O, W, etc. For a simple 2D animation, four mouth positions may be sufficient. However, for a 3D model face, for example, we would recommend using all positions described in the YouTube video by YOGYOG.

Pipe and Poses by YOGYOG
Pipe and Poses
(Image: YOGYOG)

Automatic Lip Syncing – Rhubarb Plugin

It is also possible to lip sync manually by simply adding your model’s mouth positions to “Object Mode – Object Data – Shape Keys”. Then start saving keyframes for every sound effect to create your desired speech (viewing a YouTube tutorial is highly recommended). This is a precise but very time consuming process. I would only use this method for a short speech animation, like greetings, etc. Alternatively, you can download the automatic lip-syncing plugin “Rhubarb”. It is hard to set it up properly, but when it is done, your mesh will speak any speech script you upload. And you can come back to this Blender animation setup time and time again.

Preparing the Mesh

When I uploaded an existing 3DPG face to Blender, I struggled to get the mouth positions to function. My mesh broke frequently. Only after combing through YouTube tutorials I understood that it is very important to correctly set up the mesh.

I went through a simple yet lengthy process and set up the lower face mesh how it should be. You can refer to the image below. In this case, the less vertexes and edges, the merrier.

Lip Sync - Before After - Mesh Editing in Blender 2.8
Lip Sync – Before After – Mesh Editing in Blender 2.8
(Image: 3DWithUs)

Sound for The Automated Lip Sync Animation

Audacity is an old, trusty (and free) software that will help you to prepare your own voice. And if necessary, modify it further.
One place in particular where I got stuck when working with Rhubarb lip-syncing was the format. To work properly in Blender 2.8, the sound format should be “16 Bit – PCM” (and exported as WAV).

Audacity - 16 bit PCM WAV - The Best Rhubarb Lipsyncing Format
16 bit PCM WAV – The Best Rhubarb Lipsyncing Format
(ScreenSHot: Audacity)

Amazon Polly – Text to Speech Service

The 3D Printing Guardian’s voice was achieved thanks to Amazon Polly (AWS) text-to-voice application. “25-45 years old Joey from the USA” helped us in his English and American accents. From now on 3DPG will present videos with voice and fun animation. Stay tuned. Please don’t forget to like and subscribe.

Off-Topic: Learning New 3D Designing Skills – in an Upside-down Way

Students used to take a few design courses in order to be able to create designs, and in most cases, their sketches and works were never realized. Today, 3D printer owners first print someone else’s object, then 3D design something simple, and later more advanced. In my case, I found myself animating my own 3D printing model that I created some time ago. I’m curious to see what comes next! See you soon and happy 3D designing and 3D printing.

Files & Links

Blender 2.7 plus Rhubarb Lipsync – Shapekeys
Rhubarb Plugin/Addon
by DanielSWolf
Download Blender Software
Blender for 3D Printing Courses
on Udemy

Lip Sync - Intimidating Animation Tab Screenshot - Blender 2.8
Lip Sync – Animation Tab in Blender 2.8
(ScreenShot: Blender2.9)

Read more: 3D Character Design Workflow in Blender

In this article, we’ll be going over the full 3D character design workflow as seen in games, illustration, & animation. These steps can be used to start your journey into character creation for 3D printing with a few adjustments.

QUDITECH 3D Printer Promotion Banner

Leave a comment

We encourage and welcome topic related comments. Comments with links will be held briefly for moderation. Links to social media profiles can be inserted into the Website tab. Your email address will not be published. To have an image/avatar, users must be registered at Gravatar. Happy Commenting!