ali-video

Aliyun Bailian Video Generation (HappyHorse)

Overview

This skill enables AI-powered video generation through Aliyun Bailian's DashScope API using HappyHorse models. Three generation modes are supported: text-to-video, image-to-video (first-frame based), and reference-to-video (multiple reference images). Video generation typically takes 1-5 minutes.

Critical: Always Use the Bundled Script

The ali_video.py script in the scripts/ directory is the only correct way to generate videos with this skill. Never write temporary scripts or call the DashScope API directly — the bundled script handles authentication, base64 conversion, error handling, and watermark-free output correctly. Writing ad-hoc scripts will produce videos with watermarks or other issues.

Script Path

The Python scripts for this skill are located in the scripts/ subdirectory relative to this SKILL.md file. Always use the absolute path when executing scripts. Determine the absolute path based on where this skill is installed.

For example, if this SKILL.md is at /path/to/skills/ali-video/SKILL.md, then:

• Auth script: python "/path/to/skills/ali-video/scripts/ali_auth.py"
• Video script: python "/path/to/skills/ali-video/scripts/ali_video.py"
• Query script: python "/path/to/skills/ali-video/scripts/ali_query_task.py"

When executing, construct the command using the resolved absolute path:

python "<absolute_path_to_skill_dir>/scripts/ali_video.py" <args>

Authorization

The script reads the API key from ~/.ali/.env (in the user's home directory). Do NOT ask the user for an API key upfront. Just run the script directly.

If the script fails with an authentication error (invalid/expired key), THEN tell the user:

1. Their API key is invalid or expired
2. Get a new key from: https://bailian.console.aliyun.com/
3. Pass directly: --api-key YOUR_KEY

The key passed via --api-key is automatically saved to ~/.ali/.env. Future sessions will pick it up without needing to pass the key again.

Models

Mode	Model	Description	Best For
T2V	happyhorse-1.0-t2v	Text-to-Video	Default, generating video from text descriptions
I2V	happyhorse-1.0-i2v	Image-to-Video (first frame)	Animating a single image
R2V	happyhorse-1.0-r2v	Reference-to-Video	Combining multiple reference images into video

Default mode: t2v (text-to-video)

Workflow

1. Text-to-Video (T2V)

User request: "做一段 10 秒钟横屏的宇宙飞船穿梭星际的视频"

Constraints:

• Do NOT pass --image-path or --reference-image for T2V mode.

Agent processing:

1. Extract parameters: duration=10, ratio="16:9"
2. Enhance prompt with cinematic details, camera movements, scene descriptions
3. Call API with enriched prompt
4. Monitor progress with heartbeat updates (every 60 seconds)
5. Download to ./ali_output/

Example enhanced prompt:

Cinematic wide shot of a spaceship rapidly flying through space, passing glowing
nebulae and distant stars, lens flares, dramatic camera movement, epic scale,
movie-quality visual effects, smooth 24fps motion

2. Image-to-Video (I2V)

User request: "让这张人物合影 family.jpg 动起来" or "让风景照动起来"

Agent processing:

1. Verify image path (must be a valid local file or accessible URL)
2. Enhance prompt with motion instructions
3. Monitor progress with heartbeat updates
4. Download results

Parameters for I2V:

• --image-path: First-frame image (required for I2V). Supports:
  • Local file path (preferred): C:/photos/cat.jpg or ./my_image.jpg
  • HTTP/HTTPS URL: https://example.com/image.png
• Prompt: Optional text describing desired motion/animation

Important: Always prefer local absolute paths. Do NOT upload images to free image hosting sites to obtain a URL — this is unreliable and may expose private images.

Note: I2V does not support --ratio. Output aspect ratio matches the input image.

Path validation: When specifying image paths, use valid absolute paths for your OS:

• Windows: C:/Users/name/photos/cat.jpg or D:\photos\cat.jpg
• macOS/Linux: /home/name/photos/cat.jpg
• Paths like /c/Users/... or /d/photos/... are incorrect on Windows — use C:/Users/... or D:/photos/... instead.
• Always prefer local absolute paths. Do NOT upload images to free image hosting sites.

Image constraints:

• Formats: JPEG, JPG, PNG, WEBP
• Min resolution: 300x300 (width and height each >= 300px)
• Aspect ratio: 1:2.5 to 2.5:1
• Max size: 20MB

3. Reference-to-Video (R2V)

User request: "用这几张参考图生成一段视频" or "根据参考人物和场景生成视频"

Agent processing:

1. Collect 1-9 reference images (local paths preferred)
2. Craft prompt using [Image 1], [Image 2] tags to reference images by position
3. Monitor progress with heartbeat updates
4. Download results

Parameters for R2V:

• --reference-image: Reference images (1-9). Each supports:
  • Local file path (preferred): C:/photos/ref.jpg or ./ref.jpg
  • HTTP/HTTPS URL: https://example.com/image.jpg
• Prompt: Must use [Image N] tags to reference images

Example prompt:

[Image 1]中身着红色旗袍的女性缓缓展开[Image 2]中的折扇，镜头推进特写

Image constraints:

• 1-9 reference images
• Formats: JPEG, PNG, WEBP
• Min short side: 400px
• Max size: 10MB per image

4. Resolution and Aspect Ratio

Aspect ratios (T2V and R2V only):

• 16:9 - Horizontal (YouTube, standard video) - default
• 9:16 - Vertical (TikTok, Reels, Shorts)
• 1:1 - Square
• 4:3 - Standard
• 3:4 - Portrait

Resolutions:

• 720P - HD quality - default
• 1080P - Full HD

5. Duration Options

• Range: 3-15 seconds
• Default: 5 seconds
• Longer videos take proportionally more time to generate

Progress Tracking

Video generation times vary by duration:

• 5s video: ~1-2 minutes
• 10s video: ~2-4 minutes
• 15s video: ~5+ minutes

Keep users informed:

Waiting for video generation to complete...
   (This may take 1-5 minutes, longer for 15s videos - please be patient)
Status: RUNNING (elapsed: 45s)
Video rendering in progress, elapsed 60s, please wait...
Video rendering in progress, elapsed 120s, please wait...
Task completed!

Heartbeat interval: Print encouraging message every 60 seconds.

Batch Video Generation

When generating multiple videos at once (e.g., a series of product videos, multiple scenes), follow this batch workflow to run generation tasks concurrently.

File Naming Convention

For batch generation, prepare reference images and prompt files with sequential number prefixes so they can be correctly paired:

batch_input/
├── 01_image.jpg        # Reference image for task #1
├── 01_prompt.txt       # Prompt for task #1
├── 02_image.jpg        # Reference image for task #2
├── 02_prompt.txt       # Prompt for task #2
├── 03_image.jpg        # Reference image for task #3
├── 03_prompt.txt       # Prompt for task #3

The number prefix (e.g., 01_, 02_) establishes the pairing. Image and prompt files with the same prefix must correspond to the same video. This correspondence must never be wrong.

Batch Execution

Launch each task as a separate ali_video.py invocation concurrently (in parallel). Use --prompt-file to read the prompt from the corresponding text file:

# Launch all tasks in parallel (each in background)
python "<skill_dir>/scripts/ali_video.py" --prompt-file ./batch_input/01_prompt.txt --model i2v --image-path ./batch_input/01_image.jpg --output-dir ./batch_output &

python "<skill_dir>/scripts/ali_video.py" --prompt-file ./batch_input/02_prompt.txt --model i2v --image-path ./batch_input/02_image.jpg --output-dir ./batch_output &

python "<skill_dir>/scripts/ali_video.py" --prompt-file ./batch_input/03_prompt.txt --model i2v --image-path ./batch_input/03_image.jpg --output-dir ./batch_output &

wait  # Wait for all background tasks to complete

Batch Rules

1. Sequential number prefixes — Files must be named starting with 01_, 02_, 03_, etc. Zero-padded numbers ensure correct sorting.
2. Strict pairing — The Nth image file and Nth prompt file (by number prefix) are always paired together. Never mix up the order.
3. Concurrent execution — All tasks should be launched in parallel to maximize throughput. The DashScope API handles queuing.
4. Shared output directory — Use a common --output-dir for the batch so all results land in one place. Each task's output file is named by task ID, so there's no collision.
5. Always use ali_video.py — Never write temporary scripts for batch generation. Call ali_video.py once per task, using --prompt-file to load the prompt from the paired text file.

Using the Script

Dependencies: The script requires requests and pyyaml libraries. Install before first use:

pip install requests pyyaml

Execute the Python script directly:

# Text-to-video (T2V, default 5s, 16:9)
python "<absolute_path_to_skill_dir>/scripts/ali_video.py" "a spaceship flying through space"

# 10 second vertical video
python "<absolute_path_to_skill_dir>/scripts/ali_video.py" "sunset over ocean waves" --duration 10 --ratio 9:16

# Image-to-video (I2V) with local image path (preferred)
python "<absolute_path_to_skill_dir>/scripts/ali_video.py" "a cat running on grass" --model i2v --image-path C:/photos/cat.jpg

# I2V with relative path
python "<absolute_path_to_skill_dir>/scripts/ali_video.py" "gentle waves" --model i2v --image-path ./my_photo.png

# Reference-to-video (R2V) with multiple images
python "<absolute_path_to_skill_dir>/scripts/ali_video.py" "[Image 1] girl in dress holding [Image 2] fan" --model r2v --reference-image C:/photos/girl.jpg --reference-image C:/photos/fan.jpg

# High quality 1080P
python "<absolute_path_to_skill_dir>/scripts/ali_video.py" "epic mountain timelapse" --resolution 1080P --duration 10

# Read prompt from file (useful for long prompts or batch generation)
python "<absolute_path_to_skill_dir>/scripts/ali_video.py" --prompt-file ./prompt.txt --model i2v --image-path ./photo.jpg

# Custom output directory
python "<absolute_path_to_skill_dir>/scripts/ali_video.py" "a sunset timelapse" --output-dir ./my_videos

Querying Task Status After Timeout

If video generation times out, the script prints the task ID and suggests using the query script. To check status or download a completed task later:

# Poll and wait for completion (default 120s), auto-download on success
python "<absolute_path_to_skill_dir>/scripts/ali_query_task.py" TASK_ID

# Wait longer (e.g. 10 minutes)
python "<absolute_path_to_skill_dir>/scripts/ali_query_task.py" TASK_ID --wait 600

# Poll with custom interval and output directory
python "<absolute_path_to_skill_dir>/scripts/ali_query_task.py" TASK_ID --wait 600 --poll-interval 15 --output-dir ./my_videos

# Single query, just check status (no waiting)
python "<absolute_path_to_skill_dir>/scripts/ali_query_task.py" TASK_ID --wait 0

# Single query + download if already succeeded
python "<absolute_path_to_skill_dir>/scripts/ali_query_task.py" TASK_ID --wait 0 --download

The script polls until the task reaches a terminal state (SUCCEEDED/FAILED/UNKNOWN) or the timeout expires. --wait defaults to 120 seconds; pass --wait 0 for a single query. On success it auto-downloads the video. On timeout it prints the task ID again so the agent can retry.

Task Status Flow

Status	Meaning
PENDING	Task waiting in queue
RUNNING	Currently generating
SUCCEEDED	Done, video ready for download
FAILED	Error occurred, check error code and message
UNKNOWN	Task expired (>24 hours) or not found

Error Handling

Scenario	User Message
No API key / key invalid	"API key not found or invalid. Get a new key from https://bailian.console.aliyun.com/ and set via --api-key"
Task failed	Show the specific error code and message from API
Timeout	"Generation timed out after N seconds. Task ID: xxx. Use query script to check later: python ali_query_task.py xxx"
Invalid parameters	Show validation error

Output

All videos are saved to output directory with naming pattern:

• Default: ./ali_output/ (current working directory)
• Custom: Use --output-dir or -o to specify a different path
• Naming: {task_id}.mp4 - e.g., 0385dc79-5ff8-4d82-bcb6-xxxxxx.mp4

URL mapping: The script also saves file-to-URL mappings in ./ali_output/urls.yaml. This file tracks the original URLs for each downloaded file and accumulates entries across multiple runs. When you need the original URL of a generated video, read this file.

Important: Video URLs expire after 24 hours. Download promptly.

After completion, the script outputs both the local file path and the remote URL. Inform user with both:

Video generation complete!
   - File: ./ali_output/{filename}
   - URL: {remote_url}

Tips for Better Results

Text-to-Video (T2V)

• Include cinematic terms: "wide shot", "close-up", "pan", "dolly"
• Describe motion: "flying rapidly", "slowly drifting", "zooming in"
• Specify style: "cinematic", "documentary style", "dreamy"
• Prompt supports up to 5000 English chars or 2500 Chinese chars

Image-to-Video (I2V)

• Describe the desired motion: "gentle sway", "subtle movement"
• For landscapes: "clouds moving", "water flowing", "leaves rustling"
• The output ratio matches the input image automatically

Reference-to-Video (R2V)

• Use [Image 1], [Image 2] tags in prompt to reference specific images
• Be specific about how elements from different images should interact
• 1-9 reference images supported

Resources

• scripts/ali_video.py: Main API client with full CLI support
• scripts/ali_query_task.py: Standalone task query/download script (use after timeout)
• scripts/ali_auth.py: API key management module
• references/api_reference.md: Detailed DashScope HappyHorse API documentation
• DashScope Console: https://bailian.console.aliyun.com/