Adobe’s Firefly to Get Video Generation Capabilities

In a few months, Adobe users can try out its AI model to make videos for the first time

The business says that features that use Adobe’s Firefly Video model will be added to the Premiere Pro test app and a free website before the end of 2024.

Adobe says that Generative Extend, Text to Video, and Image to Video are in a secret beta stage but will soon be available to everyone.

Generative Extend will be built into the Premiere Pro beta app later this year. It will let you add two seconds to any movie you input. Later this year, Firefly’s specialized website will also have the Text-to-video and Image-to-video models. These models turn prompts or images into five-second videos. (Adobe said the time cap could go up.)

For decades, creative people have loved Adobe’s software. However, generative AI tools like these could change the business that Adobe works in, for better or worse. Adobe’s Firefly is their response to the new wave of creative AI models, such as OpenAI’s Sora and Runway’s Gen-3 Alpha. People are fascinated by the tools because they can make clips in minutes that would have taken a person-hours to make. Most people think that these early efforts at tools are too unpredictable to use in professional settings, though.

However, Adobe thinks it can stand out by being easier to control. Ely Greenfield, Adobe’s CTO of digital media, told TechCrunch that there is a “huge appetite” for Firefly’s AI tools in situations where they can speed up or add to existing processes.

For example, Greenfield says that the generative fill feature in Firefly that was added to Adobe Photoshop last year is “one of the most-used features we’ve added in the last ten years.”

Adobe wouldn’t say how much these AI video tools cost. Adobe gives Creative Cloud users some “generative credits” for other Firefly tools. Usually, one credit leads to one generation result. Plans that cost more come with more points.

Greenfield showed off the Firefly-powered features that will be coming later this year in a test for TechCrunch.

With Generative Extend, you can add two seconds of extra footage that doesn’t look like it’s part of the original movie. This function looks at the last few frames of a scene and uses Firefly’s Video model to guess what will happen in the next few seconds. For the scene’s sound, Generative Extend will make background noise like traffic or nature sounds, but not music or people talking. Greenfield says that’s what the music business requires to get a license.

Once, Greenfield showed a video clip that had been changed to include the feature. It showed an astronaut looking out into space. I knew it had been extended because the screen had a strange lens flare. The camera pan and objects in the scene stayed the same, though. It might be helpful when the scene stops too soon, and you must add more time to transition or fade out.

You may already know about Firefly’s Text-to-video and Image-to-video features. You can put in a text or picture prompt and get up to five seconds of video back. People can use these AI video producers on firefly.adobe.com, though Adobe did not say if there would be rate limits.

Adobe also says that Firefly’s Text to Video features are pretty good at spelling words properly, which is something that AI video models often have trouble with.

Beginning with, Adobe is being extra careful when it comes to safety. Greenfield says that Firefly’s video models have blocks that stop them from making movies about drugs, alcohol, and being naked. Not only that, but he also said that Adobe’s video generation models are not trained on famous people or leaders. Some of the competition isn’t even close to being that good.