AirSpark/skills/automation/json-schemas.md

Air Spark — JSON Schemas（后端数据契约）

后端与 AI 技能层之间的数据契约。 Stage 2 输出 → Stage 3/4/5 输入。 Stage 5 输出 → Stage 6/7 输入。

---

文件列表

文件名	由谁生成	被谁使用
characters.json	Stage 2 (storyboard-automation)	Stage 3, Stage 5, Stage 6
scenes.json	Stage 2 (storyboard-automation)	Stage 3, Stage 5, Stage 6
keyshots.json	Stage 2 (storyboard-automation)	Stage 4, Stage 5, Stage 6
segments.json	Stage 5 (segmentation-automation)	Stage 6, Stage 7

---

characters.json

{
  "characters": [
    {
      "id": "char_001",
      "name": "T仔",
      "name_en": "T-Zai",
      "species": "T-Rex",
      "prompt_en": "string (Banana Pro English prompt, narrative paragraph)",
      "distinctive_features": ["string", "string", "string"],
      "asset_filename": "character_char_001.jpg",
      "asset_status": "pending"
    }
  ]
}

字段说明：

• id：全局唯一，格式 char_XXX，三位数
• asset_filename：预定义命名，Stage 3 生成后按此名存入资产库
• asset_status：pending → completed → failed（Stage 3 更新）

---

scenes.json

{
  "scenes": [
    {
      "id": "scene_001",
      "name": "T仔的单身公寓",
      "environment": "indoor",
      "time_of_day": "morning",
      "estimated_duration_sec": 30,
      "characters_present": ["char_001", "char_002"],
      "prompt_en": "string (Banana Pro English prompt, narrative paragraph, No characters in the scene.)",
      "asset_filename": "scene_scene_001.jpg",
      "asset_status": "pending"
    }
  ]
}

字段说明：

• environment：indoor 或 outdoor
• time_of_day：morning / day / evening / night
• characters_present：该场景中出现的角色 id 列表
• asset_status：pending → completed → failed（Stage 3 更新）

---

keyshots.json

{
  "video_ratio": "16:9",
  "keyshots": [
    {
      "scene_id": "scene_001",
      "scene_name": "T仔的单身公寓",
      "keyshot_index": 1,
      "grid_type": "4",
      "grid_rows": 2,
      "grid_cols": 2,
      "total_cells": 4,
      "gen_width": 2560,
      "gen_height": 1440,
      "cell_width": 1280,
      "cell_height": 720,
      "scene_duration_sec": 30,
      "coverage_start_sec": 0,
      "coverage_end_sec": 30,
      "cell_duration_sec": 7.5,
      "prompt_en": "string (full Banana Pro prompt for the grid image)",
      "grid_asset_filename": "keyshot_scene_001_1_grid.jpg",
      "asset_status": "pending",
      "cells": [
        {
          "num": 1,
          "row": 1,
          "col": 1,
          "timecode_start": "0:00",
          "timecode_end": "0:08",
          "spatial_description_en": "string",
          "asset_filename": "keyshot_scene_001_1_01.jpg",
          "asset_status": "pending"
        }
      ]
    }
  ]
}

字段说明：

• video_ratio：项目级配置，16:9 / 9:16 / 21:9
• keyshot_index：同场景有两个9宫格时区分（1或2），其余场景固定为1
• grid_type："4" 或 "9"
• gen_width / gen_height：调用 Banana Pro 时的目标尺寸
• cell_width / cell_height：PIL 裁切后每格尺寸（固定1280×720）
• coverage_start_sec / coverage_end_sec：该宫格在场景内覆盖的秒数范围
• cell_duration_sec：(coverage_end_sec - coverage_start_sec) / total_cells
• grid_asset_filename：宫格整图文件名（裁切前）
• grid_asset_filename 命名：keyshot_{scene_id}_{keyshot_index}_grid.jpg
• 每格 asset_filename：keyshot_{scene_id}_{keyshot_index}_{cell_num 两位数}.jpg
• asset_status 更新：Stage 4 生成整图后 → completed；PIL 裁切每格后 → 各格 completed

---

segments.json

{
  "episode_id": "ep01",
  "total_segments": 12,
  "total_duration_sec": 160,
  "segments": [
    {
      "id": "seg_001",
      "index": 1,
      "total": 12,
      "timecode_start": "0:00",
      "timecode_end": "0:15",
      "duration_sec": 15,
      "scene_id": "scene_001",
      "scene_number": "1-1",
      "scene_name": "T仔的单身公寓",
      "environment": "indoor",
      "time_of_day": "day",
      "character_ids": ["char_001"],
      "script_text": "string (原始剧本内容，\\n 换行，一字不改)",
      "reference_images": [
        {"type": "character", "id": "char_001"},
        {"type": "scene", "id": "scene_001"},
        {"type": "prop", "id": "prop_001", "note": "闹钟"},
        {"type": "keyshot", "scene_id": "scene_001", "keyshot_index": 1, "cell_num": 1}
      ],
      "is_action_scene": false,
      "seedance_status": "pending",
      "seedance_job_id": null,
      "seedance_video_url": null,
      "seedance_local_path": null,
      "retry_count": 0
    }
  ],
  "visual_warnings": [
    {
      "segment_id": "seg_003",
      "type": "missing_initial_state",
      "message": "开头缺少角色初始状态"
    }
  ]
}

reference_images 类型说明：

type	必填字段	后端处理方式
character	id	查 asset_library → character_{id}.jpg
scene	id	查 asset_library → scene_{id}.jpg
prop	id, note	查 asset_library → prop_{id}.jpg
keyshot	scene_id, keyshot_index, cell_num	查 asset_library → keyshot_{scene_id}_{keyshot_index}_{cell_num:02d}.jpg

不使用 prev_frame：keyshot cell 图承担空间位置锚点职责，所有片段无顺序依赖，Stage 6 全并发提交。

Seedance 状态字段：

• seedance_status：pending / running / completed / failed
• seedance_job_id：提交 Seedance API 后返回的任务 ID
• seedance_video_url：Seedance 返回的下载 URL
• seedance_local_path：下载到本地后的路径
• retry_count：当前重试次数（最多 3 次）

---

Asset Library 命名规范

所有资产文件统一存放在 projects/{project_id}/episodes/{episode_id}/assets/：

资产类型	文件名格式	示例
角色人设图	character_{id}.jpg	character_char_001.jpg
场景图	scene_{id}.jpg	scene_scene_001.jpg
道具图	prop_{id}.jpg	prop_prop_001.jpg
Keyshot 宫格整图	keyshot_{scene_id}_{keyshot_index}_grid.jpg	keyshot_scene_001_1_grid.jpg
Keyshot 裁切格	keyshot_{scene_id}_{keyshot_index}_{cell_num:02d}.jpg	keyshot_scene_001_1_01.jpg
片段视频	segment_{seg_id}.mp4	segment_seg_001.mp4
成片	final_{episode_id}.mp4	final_ep01.mp4

---

PIL 裁切逻辑（参考）

from PIL import Image

def crop_keyshot_cells(grid_image_path, keyshot: dict, output_dir: str):
    """
    精确裁切宫格图为独立格子图。
    keyshot: keyshots.json 中的一个 keyshot 对象
    """
    img = Image.open(grid_image_path)
    rows = keyshot["grid_rows"]
    cols = keyshot["grid_cols"]
    cell_w = keyshot["cell_width"]   # 1280
    cell_h = keyshot["cell_height"]  # 720

    for cell in keyshot["cells"]:
        row = cell["row"] - 1  # 转为0-indexed
        col = cell["col"] - 1
        left = col * cell_w
        top = row * cell_h
        right = left + cell_w
        bottom = top + cell_h
        cropped = img.crop((left, top, right, bottom))
        cropped.save(f"{output_dir}/{cell['asset_filename']}")

---

Seedance API 调用参考（Stage 6）

# 模型 ID：doubao-seedance-1-5-pro-251215（过渡期）/ doubao-seedance-2-0（正式）
# 端点：https://ark.cn-beijing.volces.com/api/v3/contents/generations/tasks

# 参考图拼装顺序（固定，决定 [图N] 编号）：
#   1. 角色人设图（按 character_ids 顺序，每角色1张）
#   2. 场景图（1张）
#   3. keyshot cell 图（1张）
#   4. 道具图（如有，0-2张）

# 提示词文本结构（后端自动拼装）：
#   {script_text}（剧本原文，一字不改）
#
#   {render_style}，[图1]是{char1_name}，[图2]是{scene_name}，[图3]是{keyshot描述}，
#   你是一位专业的动画导演，自行安排分镜设计，切镜充满电影感，画面氛围也有电影感，
#   不要有背景音乐，但要有音效。
#   （动作戏追加：动作戏可以有一点荷兰式倾斜镜头，动作戏的镜头具有视觉张力和空间感。）

# 请求体结构（content 数组，text 在前，图片按顺序追加）：
# {
#   "model": "doubao-seedance-1-5-pro-251215",
#   "content": [
#     {"type": "text", "text": "<上述拼装的完整提示词>"},
#     {"type": "image_url", "image_url": {"url": "<图1 URL>"}, "role": "reference_image"},
#     {"type": "image_url", "image_url": {"url": "<图2 URL>"}, "role": "reference_image"},
#     ...
#   ],
#   "generate_audio": true,
#   "duration": <segment.duration_sec>,
#   "ratio": "<project.video_ratio>",
#   "watermark": false
# }

# 异步轮询：POST 创建 → GET 轮询（每10秒）→ status=="succeeded" → 下载 video_url
# 错误重试：最多 3 次，指数退避 1s/4s/16s
# 429 限速：按响应头 Retry-After 等待

---

FFmpeg 拼接逻辑（Stage 7）

# 根据 segments.json 按 index 顺序生成 concat.txt
# 格式：
# file 'projects/xxx/episodes/ep01/assets/segment_seg_001.mp4'
# file 'projects/xxx/episodes/ep01/assets/segment_seg_002.mp4'
# ...

ffmpeg -f concat -safe 0 -i concat.txt -c copy final_ep01.mp4
# 无损拼接，保留 Seedance 原生音画同步
# 3-5 秒完成 30-40 片段