A new AI model from Runway, the startup that helped develop the well-known Stable Diffusion AI image generator, can take any word description, such as “turtles flying in the sky,” and produce three seconds of relevant video.
Runway is not initially making the model widely available, nor will it be open-sourced like Stable Diffusion, citing safety and business concerns. Gen-2, the text-to-video version, will initially be accessible on Discord through a waitlist on the Runway website.
It’s not new to use AI to create videos from text inputs. Late last year, research papers on text-to-video AI models were published by Meta Platforms Inc. and Google, respectively. The key distinction, according to Runway CEO Cristobal Valenzuela, is that the text-to-video AI technology is now accessible to the entire public.
Read More: Microsoft unveils AI office copilot for Microsoft 365
In its first iteration, Gen-1, users could input a poor 3D animation or a shaky smartphone clip and apply an AI-generated overlay.
Gen-2, in contrast, appears to be more focused on creating videos from scratch, albeit there are numerous limitations to be aware of. First, access is restricted, and second, the demo clips supplied by Runway are brief, unsteady, and most definitely not photorealistic.
