Adobe offered a preview of experimental generative AI tools for editing video and audio at its Adobe Max conference.
Adobe uses its “Sneaks” presentations to showcase the latest tech its engineers are working on and it looks like generative AI is about to change video editing in a big way.
Firefly Image 2 and Project Stardust were Adobe Max headline acts for using generative AI to edit images, but until now there haven’t been similar tools for video.
Project Fast Fill allows an editor to add or remove elements to video scenes as easily as editing a still image. The feature is powered by Adobe Firefly. A user can edit a single frame using simple text prompts, and then Project Fast Fill will carry the edit through the rest of the frames.
One example Adobe demonstrated showed how easy it was to remove people walking in the background of a scene.
And after the edit, here’s the result:
In the second example an area on a man’s shirt was selected and then entering the text prompt “tie” resulted in 4 images of the man wearing a tie were generated. Selecting one of the options resulted in the video now showing the man wearing a tie.
Once generated, here’s the result:
Imagine how much time and money this will save in production and editing.
If a scene gets ruined by having someone walk into the shot, there’s no need to do another take. If the director decides he didn’t like the clothing an actor wore in the shot, there’s no need for a wardrobe change and a reshoot.
The Sneaks presentation also featured a few other powerful video editing features.
Project Scene Change enables a video editor to take two videos shot from different angles, and create a composite scene with synchronized camera motion.
Project Dub Dub Dub uses AI to automate the dubbing process, which is a cost and labor-intensive process. The feature automatically creates translated audio in all supported languages while preserving the voice of the speaker.
Do you wish you could watch your favorite old movie in better resolution? Project Res Up uses diffusion-based upsampling technology to convert a low-resolution video into a high-resolution one.
Check out how well this works.
If you’ve ever tried to edit even a simple video for YouTube you’ll know how much time it can take. AI video editing tools that require little more than text prompts will be a game changer.
If Hollywood eventually works out a deal with actors and writers let’s hope these tools help us get to watch our favorite shows a little sooner.
