Neural networks are reshaping many professions. Recently, Heygen’s neural system faced infection that converts video text into spoken words in the video hero’s voice while mirroring lip movements. The result is meme videos spreading across the internet once again.
Heygen Labs operates on a straightforward pipeline. After a video is uploaded, a single neural network analyzes it to convert the spoken parts into text. That text is then translated into another language by a dedicated module, currently supporting eight languages. A subsequent component recreates the voice with the same timbre and accent as the original speaker. Finally, a final neuron handles lip synchronization so the lips move to the new language.
Have you ever used neural networks?
Instructions for working with Heygen Labs
Here is a concise guide to using Heygen Labs:
- Register for the service.
- Prepare a video with a resolution between 360 and 4096 pixels and a duration of 30 to 59 seconds. Longer projects cost more time, and hovering over the Requirements label reveals additional rules. Editing such a video is free.
- Upload the prepared video by dropping it into the Upload area or by selecting the file from the panel.
- Select the target language for translation and click Send.
- Wait for processing to finish and then download the file.
Note that at the time of writing, the service’s popularity has turned it into a public joke: the equipment owners cannot cope with the demand, and the queue length can vary from thousands to many tens of thousands depending on the moment.
After a while, a Queue label appears, followed by an Upgrade option to skip the queue. The service may inform users that they are stuck in a queue and offer a paid way to move ahead.
One issue reported during testing was an error stating that the angle of view was too wide in some frames. The workaround is to use videos where faces face the camera head on. Refunds were issued in those cases. When selecting footage, consider the facial angle and the mask alignment requirements.
Memes showing how the neural network works
Now the results from Heygen Lab include a well-known clip featuring Natalia often referred to as a Marine Corps figure. The person in that video spoke German.
The system handles Russian language nuances as well. An English translation of the meme We do not know what it is, if we knew what it is proved to be even funnier than the original.
Another popular case places a video at a riverside scene. The question remains whether a director like Christopher Nolan might someday reference this in a new film.
Evgeniy Ponasenkov is another familiar name the tool can emulate, producing convincing German voice performance that suggests a strong stance.
Moving on to classic meme clips, the question arises whether international audiences will appreciate food humor like cutlets with mashed potatoes expressed through lifelike speech.
There is a nod to a famous monologue about the power of a brother, now rendered in a modern vocal style for memes and viral clips.
The neural network preserves the character’s voice in many languages. Even without seeing the image, viewers can often identify the character and the context, such as a familiar dish like borscht with cabbage rather than its red counterpart.
The system shows adaptability across a broad spectrum of voices, including esteemed performers like Nikita Mikhalkov, to illustrate the range of possible tone and accentations.
There are limits. Two voices cannot be shown simultaneously in a single video, and rapid head movements can throw off the mask alignment. Lighting and angle pose additional challenges for accurate lip synchronization.
Despite some noise and imperfect microphone quality in source material, the technology still preserves the core voice characteristics effectively. Voice acting remains a field where human performers still hold a strong edge for dynamic expressions, but the progress is remarkable for a technology still in early stages. It invites speculation about what future improvements AI voice systems will bring.
Are voice actors in danger?
Attribution: VG Times