Multi-Frame Keyframe Video with Sound in ComfyUI (Workflow Included) – MiniMax H3 Multiframe Reference
Standard first-and-last-frame video models only let you pin the start and end of a clip, leaving the middle frames to drift or morph unpredictably. With the Add Guide for MiniMax H3 (MiniMaxH3AddGuide) node in ComfyUI, you can lock any number of keyframe images, video clips, or audio tracks onto…
Annons
Annons
Standard first-and-last-frame video models only let you pin the start and end of a clip, leaving the middle frames to drift or morph unpredictably. With the Add Guide for MiniMax H3 (MiniMaxH3AddGuide) node in ComfyUI, you can lock any number of keyframe images, video clips, or audio tracks onto exact timestamps along a continuous 24fps timeline while generating synchronized stereo dialogue, sound effects, and music in one pass. Generated in a single 5-second pass from four keyframe images anchored at 0.0s, 1.3s, 2.6s, and 3.8s, complete with Arctic wind, crackling lightning, a frost-wolf growl, lip-synced English dialogue, and orchestral score. From left to right: the four keyframe shots anchored at 0.0s (), 1.3s (Frame 31), 2.6s (Frame 62), and 3.8s (Frame 91). In this tutorial, you will learn how to: Set up the MiniMax H3 multiframe reference (MiniMaxH3AddGuide) workflow in ComfyUI. Chain multiple keyframe images at exact timestamps (frame_idx) across a single generation. Fix the stock template’s 5.0-second cutoff bug so every shot gets balanced screen time. Extend existing videos seamlessly with both motion and stereo audio using a 22-frame guide slice. Software needed We will use ComfyUI, a free AI image and video generator. You can use it on Windows, Mac, or Google Colab. Think Diffusion provides an online ComfyUI service if you don’t have a capable local GPU. (20% extra credit for SDA readers.) What is MiniMax H3 Multiframe Reference? MiniMax H3 is an open-weights, omni-modal diffusion transformer that generates 24fps video and stereo audio simultaneously. While Wan 2.2 first and last frame video and MiniMax H3 image-to-video only anchor the opening and closing frames, the Multiframe Reference workflow pairs MiniMax H3 reference-to-video (ref2va) with daisy-chained Add Guide for MiniMax H3 (MiniMaxH3AddGuide) nodes. In our testing, this is the most controllable way to direct multi-shot sequences or complex single-shot choreography in ComfyUI. Instead of hoping the sampler bridges a 5-second gap cleanly, you place hard visual and audio guardrails anywhere along the timeline: Arbitrary timeline keyframes (frame_idx): Pin still images to any frame number (or negative frame offsets counted from the end of the clip) to force exact camera cuts or mid-shot waypoints. Identity lock plus frame lock: Your first image connects to MiniMaxH3ReferenceToVideo as to lock character identity globally, while chained MiniMaxH3AddGuide nodes pin subsequent keyframes directly onto the target video latent’s 3D RoPE time axis. Seamless video and audio continuation: Unlike single-frame image-to-video chaining that stalls motion and resets the soundtrack, MiniMaxH3AddGuide accepts multi-frame video slices (5, 22, 39 frames) and audio waveforms at frame_idx: 0 to continue camera velocity and sound without a seam. Joint stereo sound design: Every keyframe transition automatically aligns with generated Foley, ambient soundscapes, spoken dialogue, and music. MiniMax H3 Multiframe Reference Step-by-Step Step 0: Update ComfyUI The MiniMaxH3AddGuide node requires ComfyUI 0.33.0 or newer (0.36.0+ is recommended for faster VAE decoding). Update ComfyUI before loading the workflow using ComfyUI Manager. Click the Manager button on the top toolbar. Select Update ComfyUI. Restart ComfyUI. Step 1: Download the models Download these five files and place them in their matching directories: Download minimax_h3_ref2va_pruned_int8_convrot.safetensors (19.5 GB) and put it in ComfyUI > models > diffusion_models. Download qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors (14.6 GB) and put it in ComfyUI > models > text_encoders. Download minimax_h3_video_vae_fp16.safetensors (4.85 GB) and put it in ComfyUI > models > vae. Download minimax_h3_audio_vae_fp32.safetensors (577 MB) and put it in ComfyUI > models > vae. Download minimax_h3_ref2v_turbo_4step_v0.1_comfyui_bf16.safetensors (1.82 GB) and put it in ComfyUI > models > loras. If you already completed our MiniMax H3 reference-to-video tutorial, you already have every single file above installed! You can also reuse minimax_h3_video_vae_int8_convrot.safetensors (2.62 GB) from the MiniMax H3 text-to-video guide by selecting it in the Load VAE node (#119). Optional style shortcuts: Download any of the 10 style embeddings (minimaxh3_*.safetensors) and put them in ComfyUI > models > embeddings. Step 2: Load the workflow Download the custom workflow JSON file below and drag it into the ComfyUI window. Download This workflow places all nodes directly on the main canvas across seven groups: Models, User Inputs, Video Settings, Conditioning, Extra Image Guidance Conditioning, Switch Model and Settings, Sampling, and Decoding and create video. Step 3: Install missing nodes This workflow uses native built-in ComfyUI nodes only. However, if you see nodes with red borders after loading the workflow, you may need to install missing nodes in ComfyUI Manager or update your ComfyUI build. Click Manager > Install Missing Custom Nodes. Install any nodes that are listed as missing (and update ComfyUI if prompted). Restart ComfyUI and refresh your browser. Step 4: Upload your keyframe images and set their timestamps In the User Inputs group along the top of the graph, upload your four keyframe images to the four Load Image nodes (#164, #157, #159, and #167): Load Image (#164, Shot 1 at 0.0s): Connects to ref_image_0 on MiniMax H3 Reference to Video (#136). This image serves two roles at once: it is (locking your character’s face and outfit across the whole clip) and the opening frame of Shot 1. Load Image (#157, Shot 2 at 1.3s): Connects to the first Add Guide for MiniMax H3 node (#147), with Frame Index (Second) (#154) set to 1.3 (frame 31 at 24fps). Load Image (#159, Shot 3 at 2.6s): Connects to the second Add Guide for MiniMax H3 node (#162), with Frame Index (Second) (#160) set to 2.6 (frame 62). Load Image (#167, Shot 4 at 3.8s): Connects to the third Add Guide for MiniMax H3 node (#169), with Frame Index (Second) (#168) set to 3.8 (frame 91). You can download and use these four 16:9 keyframe images to reproduce the Nordic valkyrie demo: Keyframe 1 (#164 at 0.0s / ): Volcanic cliff under the aurora borealis (valkyrie_kf1.jpg). Keyframe 2 (#157 at 1.3s / Frame 31): Glacial ice cavern with lightning spear (valkyrie_kf2.jpg). Keyframe 3 (#159 at 2.6s / Frame 62): Snowy mountain ridge beside the frost-wolf (valkyrie_kf3.jpg). Keyframe 4 (#167 at 3.8s / Frame 91): Golden celestial portal close-up for spoken dialogue (valkyrie_kf4.jpg). Why we changed the third timestamp from 5.0s to 3.8s: In the stock ComfyUI template, the third Frame Index (Second) node (#168) is set to 5.0 seconds (frame 120) inside a 5.0-second (124-frame) video. That leaves only 4 frames (0.16 seconds) for Shot 4 before the video ends! Spacing the three guides at 1.3s, 2.6s, and 3.8s gives all four shots a balanced 1.25 to 1.35 seconds of animation. Step 5: Set the resolution and duration In the Video Settings group on the left: On the Resolution Selector (Size) node (#115), set aspect_ratio to 16:9 (Widescreen). Set megapixels to 0.4 (864x480) for fast iteration, or 0.98 (1344x768) for full 768p quality. Leave multiple at 32. Set Float (Duration) (#132) to 5 seconds. The Math Expression node (#131) snaps 5 seconds at 24fps up to 124 frames (~5.17 seconds) to satisfy H3’s $17k + 5$ latent frame grid. Step 6: Revise the prompt The full multi-frame prompt is already loaded inside Input Text (Prompt) (#138). When writing your own multiframe prompt, keep these rules in mind: Match your prompt timestamps to your Frame Index (Second) nodes: Start each cut on a new line with the exact timestamp set on your guide nodes ([Shot 1] At 00:00.000..., [Shot 2] At 00:01.300..., [Shot 3] At 00:02.600..., [Shot 4] At 00:03.800...). If your prompt timestamps don’t match your Add Guide frame indices, the model may cut early or flash the guide image for a single frame. Know how works with Add Guide: Only images wired to ref_image_0..8 on MiniMax H3 Reference to Video (#136) are sent to the 32B text encoder as ... Images connected only to Add Guide for MiniMax H3 lock the video latent at that frame, but the text encoder cannot see them as unless you also describe the shot visually in text (or wire those images to extra ref_image slots on #136). Include [keyframe completion + reference generation] in summary:: Keeping this task tag at the start of the summary: block tells the ref2va model to lock onto both the identity tokens and the timeline keyframe latents. Wrap spoken lines in tags: To make your character speak on a specific keyframe shot, assign a speaker tag ((S1)) and wrap the language and line in [English] ... : Looking straight into the lens with calm, fearless intensity, the young woman (S1) says: [English] The northern gate is open. Follow the starlight. Close with overall_soundscape: and non_diegetic_music:: List your chronological sound effects (howling wind, crackling spear lightning, frost-wolf growl, portal drone) in overall_soundscape: and your score in non_diegetic_music:. Step 7: Generate the video Click the Run button to start generation. When complete, the rendered MP4 with stereo audio plays inside the Save Video node (#92) and saves to ComfyUI > output > video. Frame from Shot 1 of minimax_h3_multiframe_valkyrie.mp4, with golden starlight sparks drifting outward under the aurora borealis. Settings SettingValue in the workflowEffectLoad Image (#164)valkyrie_kf1.jpgWired to ref_image_0 () for global identity lock and Shot 1 start.Add Guide 1 (#147, #154)valkyrie_kf2.jpg at 1.3sPins Shot 2 keyframe at frame_idx: 31 (round(1.3 * 24)).Add Guide 2 (#162, #160)valkyrie_kf3.jpg at 2.6sPins Shot 3 keyframe at frame_idx: 62 (round(2.6 * 24)).Add Guide 3 (#169, #168)valkyrie_kf4.jpg at 3.8sPins Shot 4 keyframe at frame_idx: 91 (round(3.8 * 24)).Aspect ratio / Megapixels16:9 / 0.4 (864x480)Safe up to 0.98 MP (1344x768, native 768p).Float (Duration)5 seconds (124 frames)Snaps up to H3’s $17k + 5$ frame grid at 24fps.Boolean (Enable Lightning LoRA)false (20 steps)Set to true to use the 4-step ref2v Turbo LoRA.Sampler / schedulerres_multistep / betabeta holds facial identity tighter across multi-frame cuts than simple.Denoise1.0Keep at 1.0 (guide frames are injected as clean conditioning segments). Tips Multi-shot cuts vs. single-shot continuous waypoints: You can use Add Guide for MiniMax H3 in two ways. If your guide images show different camera angles and your prompt says At 00:01.300, the camera cuts to..., the model makes a clean editorial cut. If your guide images show the same camera angle at different poses (first, middle, and last frame) and you describe one continuous motion without the word cut, the model smoothly interpolates through each pose as a waypoint. Use negative frame_idx to lock the final frame automatically: MiniMaxH3AddGuide supports negative frame numbers that count backward from the end of the video (resolved_frame_index = frame_count + frame_idx). Setting frame_idx directly to -1 always locks an image to the very last frame of the clip, even if you later change Float (Duration) from 5 seconds to 8 or 10 seconds. Extend videos seamlessly with a 22-frame clip + audio guide: Extending a video by feeding only its final still frame into first_frame causes motion stalls, audio resets, and progressive blur. Instead, slice the last 22 frames (0.92 seconds) and matching audio tail of Clip 1 and wire both image and audio into Add Guide for MiniMax H3 at frame_idx: 0. Because 22 frames is a full $17k + 5$ latent block (valid multi-frame guide lengths are 5, 22, 39, 56... frames; batches under 5 frames use only the first image), H3 preserves camera momentum and audio phase across the join. How to add a 5th or 6th keyframe: Duplicate the 4-node guide block (Load Image + Frame Index (Second) + Math Expression + Add Guide for MiniMax H3), connect the previous guide node’s positive output to the new guide node’s positive input, and connect the last guide node’s positive output to Basic Guider (#126). Always reuse the same vae (#119), audio_vae (#120), and latent (#136). Avoid out-of-bounds frame_idx errors: Make sure round(seconds * 24) plus the length of your guide image or clip stays strictly below your total frame_count (124 frames for a 5-second video, indexed 0 to 123). If a guide timestamp resolves to frame 124 or higher, ComfyUI stops with ValueError: frame_idx is outside the video's 124 frames. Keep guide image aspect ratios matched to your canvas: MiniMaxH3AddGuide resizes every guide image to the target canvas using a center crop (lanczos). Use 16:9 guide images when generating 16:9 video so your character’s head or props are not cropped off at the top and bottom. Upscale after generating instead of exceeding 0.98 MP: Native 768p widescreen is 1344x768 (0.98 MP). Do not set megapixels to 1.0 (1376x768), which crosses H3’s pixel ceiling (768 * 1344) and introduces artifacts. Render at 0.4 MP or 0.98 MP and pass the finished clip through an AI upscaler or video-to-video refiner if you need 1080p or 4K. What’s next MiniMax H3 reference-to-video MiniMax H3 image-to-video MiniMax H3 text-to-video Wan 2.2 first and last frame video Wan 2.2 image-to-video Video-to-video AI upscalers The post Multi-Frame Keyframe Video with Sound in ComfyUI (Workflow Included) – MiniMax H3 Multiframe Reference first appeared on Stable Diffusion Art.