Meta
Metarevealed Movie Gen , its third - moving ridge multimodal video AI , on Friday . It promise to“produce custom videos and sound , edit exist videos , and transubstantiate your personal trope into a unique video , ” while outperforming similar models like Runway’sGen-3,Kuaishou Technology’sKling 1.5 , or OpenAI’sSora .
Meta Movie Gen builds off of the company ’s earlier work , first with its multimodalMake - A - Scenemodels , and then Llama ’s look-alike groundwork poser . Movie Gen is a collection of all of these models — specifically , video genesis , personalized picture multiplication , precise telecasting editing , and audio contemporaries — that improve the creator ’s fine - grain control . “ We anticipate these models enabling various newfangled products that could speed up creativeness , ” the company spell inits promulgation post .
Meta
For video propagation , Movie Gen relies on a 30B - parameter model that outputs up to 16 - moment clips , albeit at a pokey 16 frames per second ( fps ) . “ These models can argue about aim motion , subject - object interactions , and camera movement , and they can teach plausible motions for a blanket salmagundi of concept , ” Meta said , “ making them land - of - the - art models in their class . ” Using that same model , Movie Gen can create personalized video for Jehovah from still image .
Meta apply a variance of that video - generation model that utilize both video- and textbook - based stimulation to precisely edit the content that it generates . It can impress both localized edits such as add up , removing , or supercede elements , and world edits like applying a new cinematic trend . To generate audio , Movie Gen trust on a freestanding 13B - parameter model that can create up to 45 endorsement of audio — be it ambient setting stochasticity , sound core , or implemental lashings — while automatically syncing that subject matter to the television .
grant toMeta ’s whitened report , Movie Gen consistently won out in A / B-complex vitamin tests against other state - of - the - art picture AIs include Gen3,Sora , and Kling 1.5 in the class of video generation . It also transcend ID - animator in personalised video generation and Pika Labs Sound Gen for audio coevals . It also bested Gen3 a 2d time , in television editing capabilities . Based on the demo videos we ’ve regard so far , Movie Gen far outclassesthe current batch of free - to - purpose video recording generatorsas well .
The company says it design to “ work nearly with filmmakers and creators to integrate their feedback ” as it continues to develop these models , but was spry to head out that it has no design of dismiss human creators with AI . ” We ’re sharing this research because we conceive in the power of this engineering science to facilitate citizenry express themselves in new ways and to provide opportunities to people who might not otherwise have them , ” the fellowship wrote . “ Our Bob Hope is that perhaps one day in the future , everyone will have the chance to impart their esthetic visions to life and make high - definition video and audio using Movie Gen. ”
