Skip to content

Video (Wan2.2 Image-to-Video) Pipeline

Production pipeline for generating short video clips from source images using the Wan2.2 image-to-video model via Tensor.art.

End-to-End Flow

flowchart LR
    subgraph Generate["1. Generate"]
        IMG[Source Image] --> UP[grit_upload_image]
        UP --> GEN[grit_generate_video]
        GEN --> POLL[grit_check_status]
        POLL -->|Pending| POLL
        POLL -->|Complete| VID[Generated MP4]
    end

    subgraph Deliver["2. Deliver"]
        VID --> INGEST[grit_ingest_video]
        INGEST --> S3[S3 Upload + .meta.json]
        S3 --> SLACK[Slack Review]
        SLACK -->|Approved| UE[Game-Ready Video]
    end

Unlike the Audio SFX Pipeline, video generation is asynchronous — the grit_generate_video call submits a job to Tensor.art and returns immediately. You must poll grit_check_status until the job completes before downloading or ingesting the result.

MCP Tool Reference

Four MCP tools form the video I2V pipeline:

Tool Depends On Description
grit_upload_image Tensor.art API Upload a source image to Tensor.art for use as the I2V input frame.
grit_generate_video Tensor.art API Submit an image-to-video generation job using Wan2.2 I2V. Returns a job ID for status polling.
grit_check_status Tensor.art API Poll the status of a generation job. Returns pending, processing, completed, or failed.
grit_ingest_video boto3 (S3) Upload the generated video to S3 with a provenance .meta.json sidecar.

See Grit Tooling — Video Pipeline for full parameter details.

Prerequisites

Dependency Install Notes
Tensor.art API key Environment variable Required by grit_upload_image, grit_generate_video, and grit_check_status
boto3 / S3 credentials Environment variable Required by grit_ingest_video

No local binary dependencies are required — all generation and processing happens server-side on Tensor.art's GPU infrastructure.

Generation Workflow

Step 1: Upload Source Image

The source image serves as the first frame (or key reference frame) for the generated video. Quality of the input image directly affects output quality.

grit_upload_image(file_path="/path/to/source_frame.png")
# → Returns an image_id for use in generation

Source image tips

  • Use a high-resolution image (at least 1024×1024) for best results.
  • Ensure the subject is clearly composed — Wan2.2 will animate from this frame.
  • Avoid heavy text overlays or UI elements in the source image, as these can produce artifacts during animation.

Step 2: Generate Video

Submit the image-to-video job with a motion prompt describing the desired animation.

grit_generate_video(image_id="<image_id>",
                    prompt="slow camera pan across a misty forest clearing, gentle wind")
# → Returns a job_id

The prompt guides the motion and style of the generated video. Be descriptive about:

  • Camera movement — pan, zoom, dolly, static
  • Subject motion — walking, flowing, flickering
  • Atmosphere — lighting changes, weather effects, particle effects

Step 3: Poll for Completion

Video generation is asynchronous and typically takes 30–120 seconds depending on Tensor.art queue depth.

grit_check_status(job_id="<job_id>")
# → { "status": "processing", "progress": 45 }
# ... wait and retry ...
grit_check_status(job_id="<job_id>")
# → { "status": "completed", "output_url": "https://..." }
Status Meaning
pending Job is queued, waiting for GPU allocation
processing Generation in progress
completed Video is ready for download / ingest
failed Generation failed — check error message and retry with adjusted prompt

Polling etiquette

Wait 10–15 seconds between status checks to avoid rate-limiting. Most jobs complete within 2 minutes.

Step 4: Ingest to S3

Once the job completes, ingest the generated video to S3 with provenance metadata.

grit_ingest_video(job_id="<job_id>",
                  group="lore_cinematic", element="forest_clearing")
# → s3://<asset-bucket>/video/lore_cinematic/forest_clearing/VID_LORE_CINEMATIC_FOREST_CLEARING.mp4

S3 Naming Convention

Video assets are stored under a consistent S3 path:

grit/video/{group}/{element}/VID_{GROUP}_{ELEMENT}.mp4
Segment Example Description
{group} lore_cinematic Content category or gameplay system
{element} forest_clearing Specific scene or subject

Each uploaded video is accompanied by a provenance sidecar (.meta.json) containing:

  • license — Rights / usage terms
  • generator — tensor-art-wan2.2-i2v
  • prompt — The motion prompt used for generation
  • source_image — Reference to the original input image
  • issue — GitLab issue reference for traceability

Runtime Notes

Video Output Characteristics

Property Typical Value
Format MP4 (H.264)
Resolution Matches source image aspect ratio (commonly 1280×720 or 1024×1024)
Duration 3–5 seconds per generation
Frame rate 24 fps

UE5 Integration

Generated video clips can be imported into Unreal Engine 5 as Media Source assets:

  1. Import — Place the MP4 in the project's Content/Video/ directory.
  2. Media Player — Create a MediaPlayer asset and a MediaTexture for playback.
  3. Material — Apply the MediaTexture to a material for in-world display (e.g., lore screens, cutscene backdrops).
  4. Sequencer — For cinematic use, trigger playback via Level Sequencer events.

Storyboard Assembly

Multiple generated clips can be assembled into longer sequences using the Storyboard Assembly Pipeline. Use grit_list_clips to find ingested video assets and grit_assemble_premiere to create a Premiere Pro timeline.

Known Limitations

Limitation Impact Workaround
Short clip duration Wan2.2 generates 3–5 second clips; longer sequences require multiple generations. Use the Storyboard Assembly pipeline to concatenate clips.
Async generation Jobs can take 30–120 seconds; no streaming preview is available. Poll with grit_check_status at 10–15 second intervals.
Tensor.art rate limits Concurrent job limits may apply depending on account tier. Queue jobs sequentially or stagger submissions.
Motion consistency Complex multi-subject scenes may produce inconsistent motion between elements. Use a clear, focused source image with a single primary subject. Simplify the motion prompt.
No audio track Generated videos are silent (video-only MP4). Add audio in post using the Audio SFX Pipeline or a DAW.
Resolution tied to input Output resolution depends on the source image dimensions and Tensor.art's processing. Provide source images at your target resolution.

Typical Operator Workflow

A step-by-step example for producing a "misty forest clearing" lore cinematic clip:

# 1. Upload source image
grit_upload_image(file_path="/assets/forest_clearing_keyframe.png")
# → image_id: "img_abc123"

# 2. Submit generation job
grit_generate_video(image_id="img_abc123",
                    prompt="slow camera pan across a misty forest clearing, gentle wind rustling leaves, morning light filtering through trees")
# → job_id: "job_xyz789"

# 3. Poll for completion
grit_check_status(job_id="job_xyz789")
# → { "status": "processing", "progress": 60 }
# ... wait 15 seconds ...
grit_check_status(job_id="job_xyz789")
# → { "status": "completed", "output_url": "https://..." }

# 4. Ingest to S3
grit_ingest_video(job_id="job_xyz789",
                  group="lore_cinematic", element="forest_clearing")
# → s3://<asset-bucket>/video/lore_cinematic/forest_clearing/VID_LORE_CINEMATIC_FOREST_CLEARING.mp4

See Also