acbd2e30ad
- frontend/: Next.js 16 app (App Router, React 19, Tailwind v4) - skills/: project skills (seedance, automation, trae-agents, etc.) - Docs: PRD, UI-Design-System, DEV-LOG, seedance integration notes - skills-lock.json: skills version lock Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
297 lines
9.4 KiB
Markdown
297 lines
9.4 KiB
Markdown
# Air Spark — JSON Schemas（后端数据契约）
|
||
|
||
> 后端与 AI 技能层之间的数据契约。
|
||
> Stage 2 输出 → Stage 3/4/5 输入。
|
||
> Stage 5 输出 → Stage 6/7 输入。
|
||
|
||
---
|
||
|
||
## 文件列表
|
||
|
||
| 文件名 | 由谁生成 | 被谁使用 |
|
||
|--------|----------|----------|
|
||
| `characters.json` | Stage 2 (storyboard-automation) | Stage 3, Stage 5, Stage 6 |
|
||
| `scenes.json` | Stage 2 (storyboard-automation) | Stage 3, Stage 5, Stage 6 |
|
||
| `keyshots.json` | Stage 2 (storyboard-automation) | Stage 4, Stage 5, Stage 6 |
|
||
| `segments.json` | Stage 5 (segmentation-automation) | Stage 6, Stage 7 |
|
||
|
||
---
|
||
|
||
## characters.json
|
||
|
||
```json
|
||
{
|
||
"characters": [
|
||
{
|
||
"id": "char_001",
|
||
"name": "T仔",
|
||
"name_en": "T-Zai",
|
||
"species": "T-Rex",
|
||
"prompt_en": "string (Banana Pro English prompt, narrative paragraph)",
|
||
"distinctive_features": ["string", "string", "string"],
|
||
"asset_filename": "character_char_001.jpg",
|
||
"asset_status": "pending"
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
**字段说明**：
|
||
- `id`：全局唯一，格式 `char_XXX`，三位数
|
||
- `asset_filename`：预定义命名，Stage 3 生成后按此名存入资产库
|
||
- `asset_status`：`pending` → `completed` → `failed`（Stage 3 更新）
|
||
|
||
---
|
||
|
||
## scenes.json
|
||
|
||
```json
|
||
{
|
||
"scenes": [
|
||
{
|
||
"id": "scene_001",
|
||
"name": "T仔的单身公寓",
|
||
"environment": "indoor",
|
||
"time_of_day": "morning",
|
||
"estimated_duration_sec": 30,
|
||
"characters_present": ["char_001", "char_002"],
|
||
"prompt_en": "string (Banana Pro English prompt, narrative paragraph, No characters in the scene.)",
|
||
"asset_filename": "scene_scene_001.jpg",
|
||
"asset_status": "pending"
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
**字段说明**：
|
||
- `environment`：`indoor` 或 `outdoor`
|
||
- `time_of_day`：`morning` / `day` / `evening` / `night`
|
||
- `characters_present`：该场景中出现的角色 id 列表
|
||
- `asset_status`：`pending` → `completed` → `failed`（Stage 3 更新）
|
||
|
||
---
|
||
|
||
## keyshots.json
|
||
|
||
```json
|
||
{
|
||
"video_ratio": "16:9",
|
||
"keyshots": [
|
||
{
|
||
"scene_id": "scene_001",
|
||
"scene_name": "T仔的单身公寓",
|
||
"keyshot_index": 1,
|
||
"grid_type": "4",
|
||
"grid_rows": 2,
|
||
"grid_cols": 2,
|
||
"total_cells": 4,
|
||
"gen_width": 2560,
|
||
"gen_height": 1440,
|
||
"cell_width": 1280,
|
||
"cell_height": 720,
|
||
"scene_duration_sec": 30,
|
||
"coverage_start_sec": 0,
|
||
"coverage_end_sec": 30,
|
||
"cell_duration_sec": 7.5,
|
||
"prompt_en": "string (full Banana Pro prompt for the grid image)",
|
||
"grid_asset_filename": "keyshot_scene_001_1_grid.jpg",
|
||
"asset_status": "pending",
|
||
"cells": [
|
||
{
|
||
"num": 1,
|
||
"row": 1,
|
||
"col": 1,
|
||
"timecode_start": "0:00",
|
||
"timecode_end": "0:08",
|
||
"spatial_description_en": "string",
|
||
"asset_filename": "keyshot_scene_001_1_01.jpg",
|
||
"asset_status": "pending"
|
||
}
|
||
]
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
**字段说明**：
|
||
- `video_ratio`：项目级配置，`16:9` / `9:16` / `21:9`
|
||
- `keyshot_index`：同场景有两个9宫格时区分（1或2），其余场景固定为1
|
||
- `grid_type`：`"4"` 或 `"9"`
|
||
- `gen_width` / `gen_height`：调用 Banana Pro 时的目标尺寸
|
||
- `cell_width` / `cell_height`：PIL 裁切后每格尺寸（固定1280×720）
|
||
- `coverage_start_sec` / `coverage_end_sec`：该宫格在场景内覆盖的秒数范围
|
||
- `cell_duration_sec`：`(coverage_end_sec - coverage_start_sec) / total_cells`
|
||
- `grid_asset_filename`：宫格整图文件名（裁切前）
|
||
- `grid_asset_filename` 命名：`keyshot_{scene_id}_{keyshot_index}_grid.jpg`
|
||
- 每格 `asset_filename`：`keyshot_{scene_id}_{keyshot_index}_{cell_num 两位数}.jpg`
|
||
- `asset_status` 更新：Stage 4 生成整图后 → `completed`；PIL 裁切每格后 → 各格 `completed`
|
||
|
||
---
|
||
|
||
## segments.json
|
||
|
||
```json
|
||
{
|
||
"episode_id": "ep01",
|
||
"total_segments": 12,
|
||
"total_duration_sec": 160,
|
||
"segments": [
|
||
{
|
||
"id": "seg_001",
|
||
"index": 1,
|
||
"total": 12,
|
||
"timecode_start": "0:00",
|
||
"timecode_end": "0:15",
|
||
"duration_sec": 15,
|
||
"scene_id": "scene_001",
|
||
"scene_number": "1-1",
|
||
"scene_name": "T仔的单身公寓",
|
||
"environment": "indoor",
|
||
"time_of_day": "day",
|
||
"character_ids": ["char_001"],
|
||
"script_text": "string (原始剧本内容，\\n 换行，一字不改)",
|
||
"reference_images": [
|
||
{"type": "character", "id": "char_001"},
|
||
{"type": "scene", "id": "scene_001"},
|
||
{"type": "prop", "id": "prop_001", "note": "闹钟"},
|
||
{"type": "keyshot", "scene_id": "scene_001", "keyshot_index": 1, "cell_num": 1}
|
||
],
|
||
"is_action_scene": false,
|
||
"seedance_status": "pending",
|
||
"seedance_job_id": null,
|
||
"seedance_video_url": null,
|
||
"seedance_local_path": null,
|
||
"retry_count": 0
|
||
}
|
||
],
|
||
"visual_warnings": [
|
||
{
|
||
"segment_id": "seg_003",
|
||
"type": "missing_initial_state",
|
||
"message": "开头缺少角色初始状态"
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
**reference_images 类型说明**：
|
||
|
||
| type | 必填字段 | 后端处理方式 |
|
||
|------|----------|-------------|
|
||
| `character` | `id` | 查 asset_library → `character_{id}.jpg` |
|
||
| `scene` | `id` | 查 asset_library → `scene_{id}.jpg` |
|
||
| `prop` | `id`, `note` | 查 asset_library → `prop_{id}.jpg` |
|
||
| `keyshot` | `scene_id`, `keyshot_index`, `cell_num` | 查 asset_library → `keyshot_{scene_id}_{keyshot_index}_{cell_num:02d}.jpg` |
|
||
|
||
**不使用 `prev_frame`**：keyshot cell 图承担空间位置锚点职责，所有片段无顺序依赖，Stage 6 全并发提交。
|
||
|
||
**Seedance 状态字段**：
|
||
- `seedance_status`：`pending` / `running` / `completed` / `failed`
|
||
- `seedance_job_id`：提交 Seedance API 后返回的任务 ID
|
||
- `seedance_video_url`：Seedance 返回的下载 URL
|
||
- `seedance_local_path`：下载到本地后的路径
|
||
- `retry_count`：当前重试次数（最多 3 次）
|
||
|
||
---
|
||
|
||
## Asset Library 命名规范
|
||
|
||
所有资产文件统一存放在 `projects/{project_id}/episodes/{episode_id}/assets/`：
|
||
|
||
| 资产类型 | 文件名格式 | 示例 |
|
||
|----------|------------|------|
|
||
| 角色人设图 | `character_{id}.jpg` | `character_char_001.jpg` |
|
||
| 场景图 | `scene_{id}.jpg` | `scene_scene_001.jpg` |
|
||
| 道具图 | `prop_{id}.jpg` | `prop_prop_001.jpg` |
|
||
| Keyshot 宫格整图 | `keyshot_{scene_id}_{keyshot_index}_grid.jpg` | `keyshot_scene_001_1_grid.jpg` |
|
||
| Keyshot 裁切格 | `keyshot_{scene_id}_{keyshot_index}_{cell_num:02d}.jpg` | `keyshot_scene_001_1_01.jpg` |
|
||
| 片段视频 | `segment_{seg_id}.mp4` | `segment_seg_001.mp4` |
|
||
| 成片 | `final_{episode_id}.mp4` | `final_ep01.mp4` |
|
||
|
||
---
|
||
|
||
## PIL 裁切逻辑（参考）
|
||
|
||
```python
|
||
from PIL import Image
|
||
|
||
def crop_keyshot_cells(grid_image_path, keyshot: dict, output_dir: str):
|
||
"""
|
||
精确裁切宫格图为独立格子图。
|
||
keyshot: keyshots.json 中的一个 keyshot 对象
|
||
"""
|
||
img = Image.open(grid_image_path)
|
||
rows = keyshot["grid_rows"]
|
||
cols = keyshot["grid_cols"]
|
||
cell_w = keyshot["cell_width"] # 1280
|
||
cell_h = keyshot["cell_height"] # 720
|
||
|
||
for cell in keyshot["cells"]:
|
||
row = cell["row"] - 1 # 转为0-indexed
|
||
col = cell["col"] - 1
|
||
left = col * cell_w
|
||
top = row * cell_h
|
||
right = left + cell_w
|
||
bottom = top + cell_h
|
||
cropped = img.crop((left, top, right, bottom))
|
||
cropped.save(f"{output_dir}/{cell['asset_filename']}")
|
||
```
|
||
|
||
---
|
||
|
||
## Seedance API 调用参考（Stage 6）
|
||
|
||
```python
|
||
# 模型 ID：doubao-seedance-1-5-pro-251215（过渡期）/ doubao-seedance-2-0（正式）
|
||
# 端点：https://ark.cn-beijing.volces.com/api/v3/contents/generations/tasks
|
||
|
||
# 参考图拼装顺序（固定，决定 [图N] 编号）：
|
||
# 1. 角色人设图（按 character_ids 顺序，每角色1张）
|
||
# 2. 场景图（1张）
|
||
# 3. keyshot cell 图（1张）
|
||
# 4. 道具图（如有，0-2张）
|
||
|
||
# 提示词文本结构（后端自动拼装）：
|
||
# {script_text}（剧本原文，一字不改）
|
||
#
|
||
# {render_style}，[图1]是{char1_name}，[图2]是{scene_name}，[图3]是{keyshot描述}，
|
||
# 你是一位专业的动画导演，自行安排分镜设计，切镜充满电影感，画面氛围也有电影感，
|
||
# 不要有背景音乐，但要有音效。
|
||
# （动作戏追加：动作戏可以有一点荷兰式倾斜镜头，动作戏的镜头具有视觉张力和空间感。）
|
||
|
||
# 请求体结构（content 数组，text 在前，图片按顺序追加）：
|
||
# {
|
||
# "model": "doubao-seedance-1-5-pro-251215",
|
||
# "content": [
|
||
# {"type": "text", "text": "<上述拼装的完整提示词>"},
|
||
# {"type": "image_url", "image_url": {"url": "<图1 URL>"}, "role": "reference_image"},
|
||
# {"type": "image_url", "image_url": {"url": "<图2 URL>"}, "role": "reference_image"},
|
||
# ...
|
||
# ],
|
||
# "generate_audio": true,
|
||
# "duration": <segment.duration_sec>,
|
||
# "ratio": "<project.video_ratio>",
|
||
# "watermark": false
|
||
# }
|
||
|
||
# 异步轮询：POST 创建 → GET 轮询（每10秒）→ status=="succeeded" → 下载 video_url
|
||
# 错误重试：最多 3 次，指数退避 1s/4s/16s
|
||
# 429 限速：按响应头 Retry-After 等待
|
||
```
|
||
|
||
---
|
||
|
||
## FFmpeg 拼接逻辑（Stage 7）
|
||
|
||
```bash
|
||
# 根据 segments.json 按 index 顺序生成 concat.txt
|
||
# 格式：
|
||
# file 'projects/xxx/episodes/ep01/assets/segment_seg_001.mp4'
|
||
# file 'projects/xxx/episodes/ep01/assets/segment_seg_002.mp4'
|
||
# ...
|
||
|
||
ffmpeg -f concat -safe 0 -i concat.txt -c copy final_ep01.mp4
|
||
# 无损拼接，保留 Seedance 原生音画同步
|
||
# 3-5 秒完成 30-40 片段
|
||
```
|