How Sora’s Neural Network creates videos – specific examples

We already told you that neural networks have learned to create images from text queries and edit existing videos. However, until recently, artificial intelligence could not generate videos on its own. The situation started to change with the arrival of Sora’s neural network, which is now in testing phase – and also heavily hyped. You can find examples of her work in our article.

You can try signing up for a beta test via the link (however, the chances of being selected are extremely low).

Entered text: A stylish woman walks down a Tokyo street full of warm glowing neon and animated city signs. She wears a black leather jacket, a long red dress and black boots, and carries a black bag. She is wearing sunglasses and red lipstick. She walks confidently and nonchalantly. The street is damp and reflective, creating a mirror effect from the colorful lights. There are a lot of pedestrians walking around.

Translation: A stylish woman walks down the street of Tokyo. She is surrounded by animated signs and warm neon lights. The girl is dressed in a black leather jacket, a long red dress and black boots, and in her hands she carries a black handbag. She is wearing sunglasses and red lipstick. The beauty walks confidently and at ease. There are puddles of rain on the road that reflect the light of multi-colored lamps like a mirror. There are a lot of pedestrians around the woman.

Entered text: The camera follows a white vintage SUV with a black roof rack as it drives up a steep dirt road surrounded by pine trees on a steep mountainside.

Translation: The camera follows a white vintage SUV with a black roof rack as it drives down a dirt road surrounded by pine trees on a steep mountainside.

Entered text: A cat that wakes its sleeping owner and demands breakfast. The owner tries to ignore the cat, but the cat tries new tactics and eventually the owner removes a secret stash of treats from under the pillow to keep the cat at bay a little longer.

Translation: The cat wakes its sleeping owner and demands to be fed. The man tries to ignore the cat, but she tries a new tactic. After this, the owner takes treats from under the pillow to calm the cat for a while.

Entered text: Photorealistic close-up video of two pirate ships fighting each other while sailing in a cup of coffee.

Translation: Photorealistic close-up video of two pirate ships fighting each other in a cup of coffee.

Entered text: A young man in his twenties is sitting on a cloud in the sky reading a book.

Translation: A young man of about twenty is sitting on a cloud in the sky and reading a book.

Here you can see a comparison of real Minecraft gameplay and a video created by a neural network. There is of course a difference. But someone who hasn’t played this game for a long time might not notice the catch.

Other examples

A butterfly diving into a pond

It is true that the neural network does not always produce realistic video. In some frames you may notice some unnaturalness.

And sometimes the result looks completely strange.

Are you impressed with Sora’s abilities? Or were you lucky enough to test it yourself? Share your stories in the comments!