Adobe’s AI video model is now available in Premiere Pro

Adobe is entering the field of AI-powered generative video. Business Firefly video modelwhich has been talked about since the beginning of this year, is launching today with several fresh tools, including some available directly in Premiere Pro, that will allow creators to augment footage and generate video from still images and text prompts.

The first tool, Generative Extend, will launch in the beta version of Premiere Pro. You can employ it to extend the end or beginning of footage that’s a bit too low, or to make mid-shot adjustments, such as correcting shifting eyelines or unexpected movement.

Clips can only be extended by two seconds, so Generative Extend is really only good for minor corrections, but it can replace the need to re-record footage to correct minor errors. Extended clips can be generated in 720p or 1080p at 24 frames per second. It can also be used for audio to make editing easier, albeit with limitations. This will, for example, extend sound effects and ambient “room tone” by up to ten seconds, but will not extend voice dialogue or music.

Two other video generating tools are appearing on the Internet. Adobe’s text-to-video and image-to-video tools, first announced in September, are now available as part of a circumscribed public beta of the Firefly web app.

The text-to-video feature works similarly to other video generators like Runway and Sora from OpenAI – users just need to add a text description of what they want to generate. It can emulate a variety of styles such as regular “real” film, 3D animation, and stop-motion animation, and the generated clips can be further refined with select “camera controls” that simulate things such as camera angle, movement, and shooting distance.

The Image-to-Video feature takes it a step further by allowing users to add a reference image to a text prompt to give them more control over the results. Adobe suggests this can be used to create b-rolls from images and photographs, or to visualize reshoots by uploading a still from an existing video. However, the before and after example below shows that it is not really able to directly replace reshoots, as several bugs such as cable wobble and background shifting are perceptible in the results.

You won’t be shooting entire videos using this technology any time soon. The maximum length for text-to-video and image-to-video clips is currently five seconds, and the highest quality is 720p and 24 frames per second. By comparison, OpenAI claims that Sora can generate videos up to a minute long “while maintaining visual quality and compliance with user commands” – but this is not yet publicly available, even though it was announced months before Adobe’s tools were introduced.

Text-to-video, image-to-video, and generative extensions take about 90 seconds to generate, but Adobe says it’s working on a “turbo mode” to reduce that time. While Adobe’s tools based on its AI video model are circumscribed, they claim to be “commercially safe” because they have been trained on content that the original software giant has been permitted to employ. Models from other providers like Runway are checked for alleged training on thousands of YouTube videos – and in Meta’s case, it could even be yours personal Videos – For some users, commercial viability may be the key to closing a deal.

Another benefit is that videos created or edited using Adobe’s Firefly video model can be embedded with content credentials to assist reveal AI employ and ownership once published online. It’s unclear when these tools will come out of beta, but at least they’re publicly available – and that’s more than we can say Sora from OpenAIMeta’s Movie Gen generators and Google’s Veo generators.

The AI ​​video launch was announced today at Adobe MAX, where the company is also rolling out a number of other AI-powered features across its original apps.