Telegram-канал f_s_c_p - FSCP: Неотсортированное

FSCP

08 апреля 2025 14:10

One-Minute Video Generation with Test-Time Training

Ого, вот это уже интересно.

По крайней мере тема с 60-секундными сериалами закрыта.

Когда вы смотрите все эти "нейросетьсделалахарипотера", вам наливают нарезку по 5 секунд из несовместимых движений камеры. Пока только видео-2-видео и говорящие головы могут генерить контент больше минуты.

А тут вот авторе взялись порешать задачу длинных видосов. По сравнению Mamba 2, Gated DeltaNet и слоями со скользящим окном внимания, слои Test-Time Training генерируют гораздо более целостные видео, рассказывающие сложные истории.

Срочно шуруйте вот сюда и смотрите, что они сотворили с Том и Джерри:
https://test-time-training.github.io/video-dit/

Там внутри CogVideoX 5B model. Авторы говорят, что можно генерить и более длинные видео, а артефакты в будущем можно пофиксить взяв более тяжелые модели.

Ну и обратите внимание на промпты, они величиной с Юпитер. Но вы же понимаете, кто будет их писать...
В оригинале 21 параграф:

An underwater scene stretches across the entire screen. Amidst the colorful reef, a small, rolled-up parchment map lays on the sea floor. Jerry, the brown mouse, swims calmly into the scene from the left, his large eyes wide with curiosity, carefully looking around while his tiny paws propel him forward gently. A small, rolled-up parchment map is laying on the sea floor. The map has a single small red x on it.

An underwater scene stretches across the entire screen. Jerry, the brown mouse, pauses, his attention drawn to a small, rolled-up parchment map laying on the sea floor. With gentle care, Jerry grasps the delicate parchment with both paws and slowly unrolls it, examining its smooth surface intently. The map has a single small red x on it.<end_scene>

<start_scene>A coral-covered underwater area stretches across the screen, rich with delicate coral formations in pastel hues gently moving with the rhythm of the ocean. Jerry, the brown mouse, swims thoughtfully toward a gently glowing sea anemone, its soft tentacles subtly pulsing with luminescence. Jerry's expression is curious and determined, his tiny paws methodically propelling him through the water. As he swims, he turns his head left and right, eyes carefully scanning for something important. The camera smoothly pans along with Jerry's steady forward swimming, highlighting his thoughtful and attentive demeanor as he continues his careful exploration of the vibrant underwater environment.<end_scene>

<start_scene>Underwater, a close-up reveals Tom, the blue-gray cat, with a hungry and predatory expression, his sharp eyes narrowed as he keenly scans the surroundings. Tom turns his head methodically from left to right, squinting his eyes carefully as he searches for something, a cunning gleam evident in his gaze. Small bubbles of air gently rise behind him, highlighting his location beneath the ocean surface. In the softly swaying background, a green coral plant moves rhythmically with the water's flow. Tom is depicted naturally, free of any clothing, his sleek fur softly illuminated by the refracted sunlight filtering down from above.<end_scene>

<start_scene>Underwater, a large purple coral sways gently and rhythmically from side to side in the mild ocean current. From behind the broad coral branches, Tom, the blue-gray cat, peers mischievously with a sly and calculating smile, his eyes following Jerry intently. Jerry, the brown mouse, swims slowly and obliviously past Tom, calmly navigating the serene underwater landscape without sensing any threat. The camera smoothly pans horizontally, tracking Jerry's leisurely movement across the scene, emphasizing Tom's stealthy observation, his sly grin deepening as he patiently waits for the ideal moment to act upon his mischievous intentions toward Jerry.

Кстати, код есть
https://github.com/test-time-training/ttt-video-dit

Ну и прежде чем писать о том, какая же отстойная анимация, представьте, что все это смотрят двухлетки, которым все равно на принципы диснея, а траффик, как известно не пахнет.

@cgevent