新增 提取资产 skill 修复获取小说tool 修复多项目剧本agent故事策略丢失,增加事件提取并发

This commit is contained in:
zhishi 2026-03-27 00:11:22 +08:00
parent bc4f3107a5
commit 9b3806e989
10 changed files with 310 additions and 112 deletions

View File

@ -0,0 +1,95 @@
---
name: universal_agent
description: 专注于从剧本内容中提取所使用的资产(角色、场景、道具)并生成结构化资产列表的助手。
---
# Script Assets Extract
你是一个专业的剧本内容分析助手,专注于从剧本文本中识别和提取所有涉及的资产(角色、场景、道具),并为每项资产生成可供下游制作流程使用的结构化描述和提示词。
## 何时使用
用户提供剧本内容,你需要逐段阅读并提取其中涉及的所有资产(人物角色、场景地点、道具物件),输出为结构化的资产列表。产出的资产描述将用于后续 AI 图片生成和制作流程。
## 与系统的对应关系
- 资产类型:
- `role` — 角色(对应 `o_assets.type = "role"`
- `scene` — 场景(对应 `o_assets.type = "scene"`
- `tool` — 道具(对应 `o_assets.type = "tool"`
- `clip` — 素材片段(对应 `o_assets.type = "clip"`
- 下游用途:资产提示词生成 → AI 资产图生成 → 分镜制作
## 输出要求
**必须通过调用 `resultTool` 工具返回结果**禁止以纯文本、Markdown 表格或 JSON 代码块等形式直接输出资产列表。
`resultTool` 的 schema 会对字段类型和枚举值做强校验,调用时请严格按照下方字段定义填写,确保数据结构正确、字段完整、类型匹配。
每个资产对象包含以下字段:
| 字段 | 类型 | 必填 | 说明 |
| ---- | ---- | ---- | ---- |
| `name` | string | 是 | 资产名称,使用剧本中的原始称呼,不做其他多余描述 |
| `desc` | string | 是 | 资产描述30-80 字的视觉化描述 |
| `prompt` | string | 是 | 生成提示词,英文,用于 AI 图片生成 |
| `type` | enum | 是 | 资产类型:`role` / `scene` / `tool` / `clip` |
## 提取规则
### 角色role
- 提取剧本中出现的所有有名字的角色
- `desc`:包含外貌特征、服饰风格、体态气质等视觉要素
- `prompt`:英文提示词,描述角色的外观特征,适用于 AI 角色图生成
- 同一角色有多个称呼时,取最常用的作为 `name`
- 无名龙套(如"路人甲"、"士兵")可跳过,除非其造型对剧情有重要视觉意义
### 场景scene
- 提取剧本中出现的所有场景/地点
- `desc`:包含空间结构、光照氛围、关键陈设、色调基调等视觉要素
- `prompt`:英文提示词,描述场景的整体视觉风格,适用于 AI 场景图生成
- 同一场景的不同状态(如白天/夜晚)不重复提取,在 `desc` 中注明即可
### 道具tool
- 提取剧本中出现的重要道具/物品
- `desc`:包含外观形状、颜色材质、尺寸参考、特殊效果等视觉要素
- `prompt`:英文提示词,描述道具的外观细节,适用于 AI 道具图生成
- 仅提取有独立视觉意义或剧情功能的道具,通用物品可跳过
### 素材片段clip
- 提取剧本中需要的特殊素材片段(如特效、转场、特殊画面等)
- `desc`:描述素材的视觉效果和用途
- `prompt`:英文提示词,描述素材的视觉特征
## 提示词prompt生成规范
- 采用逗号分隔的关键词/短语格式
- 优先描述**视觉特征**,避免抽象概念
- 包含风格关键词(如 anime style, manga style 等,根据项目风格决定)
- 角色 prompt 示例:`a young man, sharp eyebrows, black hair, pale skin, wearing a gray Taoist robe, slender build, cold expression`
- 场景 prompt 示例:`dark cave interior, glowing crystals on walls, misty atmosphere, dim blue lighting, stone altar in center`
- 道具 prompt 示例:`ancient jade pendant, oval shape, translucent green, carved dragon pattern, glowing faintly`
## 提取流程
1. 通读剧本全文,识别所有出现的角色、场景、道具
2. 对每个资产生成结构化的 `name``desc``prompt``type`
3. 去重:同一资产不重复提取
4. **必须通过调用 `resultTool` 工具输出完整资产列表**,不要分多次调用,一次性将所有资产放入 `assetsList` 数组中提交
## 提取原则
1. **忠于剧本**:所有提取基于剧本中的实际内容,不臆造未出现的资产
2. **视觉优先**:描述和提示词聚焦视觉特征,便于 AI 图片生成
3. **精简实用**:只提取对制作有实际意义的资产,避免过度提取
4. **分类准确**:严格按照 role/scene/tool/clip 分类,不混淆
5. **提示词质量**:英文提示词应具体、可执行,能直接用于 AI 图片生成
## 注意事项
- 资产列表中**不要包含剧本内容本身**,仅提取所使用到的资产
- 角色的随身物品如果有独立剧情功能,应单独作为道具提取
- 场景中的固定陈设不需要单独提取为道具,除非该物件有独立剧情作用

View File

@ -44,6 +44,13 @@ description: 通用文本分析与内容提取 Agent支持小说事件提取
- **资产类型**`tool`(对应 `o_assets.type = "tool"`
- **输出**:结构化道具资产表(道具名称、类别、外观描述、尺寸参考、材质质感、功能/用途、首次出场、关联角色、状态变体)+ 高频道具排名
### 6. 剧本资产提取script_assets_extract
- **触发条件**:用户提供剧本内容,要求从剧本中提取所使用的资产(角色、场景、道具、素材片段)
- **参考文件**`references/script_assets_extract.md`
- **资产类型**`role``scene``tool``clip`(对应 `o_assets.type`
- **输出**:结构化资产列表(资产名称、资产描述、生成提示词、资产类型),通过 `resultTool` 工具调用返回
## 资产提取分工说明
当用户要求从小说中提取"所有资产"或"角色场景道具"时,三个资产提取技能应按以下分工协作:

View File

@ -42,6 +42,7 @@ export default (resTool: ResTool, toolsNames?: string[]) => {
console.log("[tools] get_novel_events", ids);
const data = await u
.db("o_novel")
.where("projectId",resTool.data.projectId)
.select("id", "chapterIndex as index", "reel", "chapter", "chapterData", "event", "eventState")
.whereIn("id", ids);
const eventString = data.map((i: any) => [`${i.index}章,标题:${i.chapter},事件:${i.event}`].join("\n")).join("\n");

View File

@ -1,4 +1,4 @@
// @routes-hash 758e343e27eb25780faf00eeab306216
// @routes-hash 8097a5206252be753261d3f059243260
import { Express } from "express";
import route1 from "./routes/agents/clearMemory";
@ -66,37 +66,38 @@ import route62 from "./routes/project/getProject";
import route63 from "./routes/script/addScript";
import route64 from "./routes/script/delScript";
import route65 from "./routes/script/exportScript";
import route66 from "./routes/script/getScrptApi";
import route67 from "./routes/script/updateScript";
import route68 from "./routes/scriptAgent/getPlanData";
import route69 from "./routes/scriptAgent/setPlanData";
import route70 from "./routes/setting/agentDeploy/agentSetKey";
import route71 from "./routes/setting/agentDeploy/deployAgentModel";
import route72 from "./routes/setting/agentDeploy/getAgentDeploy";
import route73 from "./routes/setting/dbConfig/clearData";
import route74 from "./routes/setting/fileManagement/openFolder";
import route75 from "./routes/setting/getTextModel";
import route76 from "./routes/setting/loginConfig/getUser";
import route77 from "./routes/setting/loginConfig/updateUserPwd";
import route78 from "./routes/setting/memoryConfig/delAllMemory";
import route79 from "./routes/setting/memoryConfig/getMemory";
import route80 from "./routes/setting/memoryConfig/sureMemory";
import route81 from "./routes/setting/skillManagement/addSkill";
import route82 from "./routes/setting/skillManagement/deleteSkill";
import route83 from "./routes/setting/skillManagement/embeddingSkill";
import route84 from "./routes/setting/skillManagement/generateDescription";
import route85 from "./routes/setting/skillManagement/getSkillList";
import route86 from "./routes/setting/skillManagement/scanSkills";
import route87 from "./routes/setting/skillManagement/updateSkill";
import route88 from "./routes/setting/vendorConfig/addVendor";
import route89 from "./routes/setting/vendorConfig/deleteVendor";
import route90 from "./routes/setting/vendorConfig/getVendorList";
import route91 from "./routes/setting/vendorConfig/modelTest";
import route92 from "./routes/setting/vendorConfig/updateVendor";
import route93 from "./routes/task/getTaskApi";
import route94 from "./routes/task/getTaskCategories";
import route95 from "./routes/task/taskDetails";
import route96 from "./routes/test/test";
import route66 from "./routes/script/extractAssets";
import route67 from "./routes/script/getScrptApi";
import route68 from "./routes/script/updateScript";
import route69 from "./routes/scriptAgent/getPlanData";
import route70 from "./routes/scriptAgent/setPlanData";
import route71 from "./routes/setting/agentDeploy/agentSetKey";
import route72 from "./routes/setting/agentDeploy/deployAgentModel";
import route73 from "./routes/setting/agentDeploy/getAgentDeploy";
import route74 from "./routes/setting/dbConfig/clearData";
import route75 from "./routes/setting/fileManagement/openFolder";
import route76 from "./routes/setting/getTextModel";
import route77 from "./routes/setting/loginConfig/getUser";
import route78 from "./routes/setting/loginConfig/updateUserPwd";
import route79 from "./routes/setting/memoryConfig/delAllMemory";
import route80 from "./routes/setting/memoryConfig/getMemory";
import route81 from "./routes/setting/memoryConfig/sureMemory";
import route82 from "./routes/setting/skillManagement/addSkill";
import route83 from "./routes/setting/skillManagement/deleteSkill";
import route84 from "./routes/setting/skillManagement/embeddingSkill";
import route85 from "./routes/setting/skillManagement/generateDescription";
import route86 from "./routes/setting/skillManagement/getSkillList";
import route87 from "./routes/setting/skillManagement/scanSkills";
import route88 from "./routes/setting/skillManagement/updateSkill";
import route89 from "./routes/setting/vendorConfig/addVendor";
import route90 from "./routes/setting/vendorConfig/deleteVendor";
import route91 from "./routes/setting/vendorConfig/getVendorList";
import route92 from "./routes/setting/vendorConfig/modelTest";
import route93 from "./routes/setting/vendorConfig/updateVendor";
import route94 from "./routes/task/getTaskApi";
import route95 from "./routes/task/getTaskCategories";
import route96 from "./routes/task/taskDetails";
import route97 from "./routes/test/test";
export default async (app: Express) => {
app.use("/api/agents/clearMemory", route1);
@ -164,35 +165,36 @@ export default async (app: Express) => {
app.use("/api/script/addScript", route63);
app.use("/api/script/delScript", route64);
app.use("/api/script/exportScript", route65);
app.use("/api/script/getScrptApi", route66);
app.use("/api/script/updateScript", route67);
app.use("/api/scriptAgent/getPlanData", route68);
app.use("/api/scriptAgent/setPlanData", route69);
app.use("/api/setting/agentDeploy/agentSetKey", route70);
app.use("/api/setting/agentDeploy/deployAgentModel", route71);
app.use("/api/setting/agentDeploy/getAgentDeploy", route72);
app.use("/api/setting/dbConfig/clearData", route73);
app.use("/api/setting/fileManagement/openFolder", route74);
app.use("/api/setting/getTextModel", route75);
app.use("/api/setting/loginConfig/getUser", route76);
app.use("/api/setting/loginConfig/updateUserPwd", route77);
app.use("/api/setting/memoryConfig/delAllMemory", route78);
app.use("/api/setting/memoryConfig/getMemory", route79);
app.use("/api/setting/memoryConfig/sureMemory", route80);
app.use("/api/setting/skillManagement/addSkill", route81);
app.use("/api/setting/skillManagement/deleteSkill", route82);
app.use("/api/setting/skillManagement/embeddingSkill", route83);
app.use("/api/setting/skillManagement/generateDescription", route84);
app.use("/api/setting/skillManagement/getSkillList", route85);
app.use("/api/setting/skillManagement/scanSkills", route86);
app.use("/api/setting/skillManagement/updateSkill", route87);
app.use("/api/setting/vendorConfig/addVendor", route88);
app.use("/api/setting/vendorConfig/deleteVendor", route89);
app.use("/api/setting/vendorConfig/getVendorList", route90);
app.use("/api/setting/vendorConfig/modelTest", route91);
app.use("/api/setting/vendorConfig/updateVendor", route92);
app.use("/api/task/getTaskApi", route93);
app.use("/api/task/getTaskCategories", route94);
app.use("/api/task/taskDetails", route95);
app.use("/api/test/test", route96);
app.use("/api/script/extractAssets", route66);
app.use("/api/script/getScrptApi", route67);
app.use("/api/script/updateScript", route68);
app.use("/api/scriptAgent/getPlanData", route69);
app.use("/api/scriptAgent/setPlanData", route70);
app.use("/api/setting/agentDeploy/agentSetKey", route71);
app.use("/api/setting/agentDeploy/deployAgentModel", route72);
app.use("/api/setting/agentDeploy/getAgentDeploy", route73);
app.use("/api/setting/dbConfig/clearData", route74);
app.use("/api/setting/fileManagement/openFolder", route75);
app.use("/api/setting/getTextModel", route76);
app.use("/api/setting/loginConfig/getUser", route77);
app.use("/api/setting/loginConfig/updateUserPwd", route78);
app.use("/api/setting/memoryConfig/delAllMemory", route79);
app.use("/api/setting/memoryConfig/getMemory", route80);
app.use("/api/setting/memoryConfig/sureMemory", route81);
app.use("/api/setting/skillManagement/addSkill", route82);
app.use("/api/setting/skillManagement/deleteSkill", route83);
app.use("/api/setting/skillManagement/embeddingSkill", route84);
app.use("/api/setting/skillManagement/generateDescription", route85);
app.use("/api/setting/skillManagement/getSkillList", route86);
app.use("/api/setting/skillManagement/scanSkills", route87);
app.use("/api/setting/skillManagement/updateSkill", route88);
app.use("/api/setting/vendorConfig/addVendor", route89);
app.use("/api/setting/vendorConfig/deleteVendor", route90);
app.use("/api/setting/vendorConfig/getVendorList", route91);
app.use("/api/setting/vendorConfig/modelTest", route92);
app.use("/api/setting/vendorConfig/updateVendor", route93);
app.use("/api/task/getTaskApi", route94);
app.use("/api/task/getTaskCategories", route95);
app.use("/api/task/taskDetails", route96);
app.use("/api/test/test", route97);
}

View File

@ -5,30 +5,88 @@ import { error, success } from "@/lib/responseFormat";
import compressing from "compressing";
import { validateFields } from "@/middleware/middleware";
import { useSkill } from "@/utils/agent/skillsTools";
import { Output, tool } from "ai";
const router = express.Router();
export const AssetSchema = z.object({
prompt: z.string().describe("生成提示词"),
name: z.string().describe("资产名称,仅为名称不做其他任何表述"),
desc: z.string().describe("资产描述"),
type: z.enum(["role", "tool", "scene"]).describe("资产类型"),
});
export default router.post(
"/",
validateFields({
scriptIds: z.array(z.number()),
projectId: z.number(),
}),
async (req, res) => {
const { scriptIds } = req.body;
const { scriptIds, projectId } = req.body;
if (!scriptIds.length) return res.status(400).send(error("请先选择剧本"));
const scripts = await u.db("o_script").whereIn("id", scriptIds);
const intansce = u.Ai.Text("universalAgent");
const skill = await useSkill("universal_agent.md");
const novelData = await u.db("o_novel").where("projectId", projectId).select("chapterData");
if (!novelData || novelData.length === 0) return res.status(400).send(error("请先上传小说"));
const resData = await intansce.invoke({
system: skill.prompt,
messages: [
{
role: "user",
content: "请根据以下小说章节生成事件摘要:\n",
async function getAssets() {
return await u.db("o_assets").where("projectId", projectId).select("id", "name");
}
for (const scriptId of scriptIds) {
const resultTool = tool({
description: "返回结果时必须调用这个工具,",
inputSchema: z.object({
assetsList: z.array(AssetSchema).describe("剧本所使用资产列表,注意不要包含剧本内容,仅为所使用到的 道具、人物、场景、素材"),
}),
execute: async ({ assetsList }) => {
console.log("[tools] set_flowData script", assetsList);
if (assetsList && assetsList.length) {
const assetId = [];
const existingAssets = await getAssets();
for (const i of assetsList) {
if (existingAssets.length) {
const exist = existingAssets.find((j) => j.name === i.name);
if (exist) {
assetId.push(exist.id);
continue;
}
}
const [id] = await u.db("o_assets").insert({
name: i.name,
prompt: i.prompt,
type: i.type,
describe: i.desc,
projectId: projectId,
startTime: Date.now(),
});
assetId.push(id);
}
await u.db("o_scriptAssets").insert(assetId.map((i) => ({ scriptId: scriptId, assetId: i })));
}
return true;
},
],
tools: skill.tools,
});
});
try {
const skill = await useSkill("universal_agent.md");
const resData = await intansce.invoke({
messages: [
{
role: "system",
content:
skill.prompt +
"\n\n提取剧本中涉及的资产角色、场景、道具参考技能 script_assets_extract 规范,结果必须通过 resultTool 工具返回。",
},
{
role: "user",
content: `请根据以下剧本提取对应的剧本资产(角色、场景、道具、素材片段):\n\n${scripts.map((i) => i.content).join("\n\n---\n\n")}`,
},
],
tools: { ...skill.tools, resultTool },
});
console.log("%c Line:47 🥝 resData", "background:#2eafb0", resData);
} catch (e) {
console.log("%c Line:52 🍢 e", "background:#42b983", e);
}
}
},
);

View File

@ -13,11 +13,11 @@ export default router.post(
}),
async (req, res) => {
const { projectId, agentType } = req.body;
const data = await u.db("o_agentWorkData").where({ id: projectId, key: agentType }).first();
const data = await u.db("o_agentWorkData").where({ projectId: projectId, key: agentType }).first();
if (!data) {
await u.db("o_agentWorkData").insert({
id: projectId,
projectId: projectId,
key: agentType,
data: JSON.stringify({
storySkeleton: "",

View File

@ -19,7 +19,7 @@ export default router.post(
const { projectId, agentType, data } = req.body;
await u
.db("o_agentWorkData")
.where({ id: projectId, key: agentType })
.where({ projectId: projectId, key: agentType })
.update({
data: JSON.stringify(data),
});

View File

@ -1,6 +1,17 @@
// @db-hash ce28b6d566911952421c2661e14bfde5
// @db-hash 579a004cc745580469a24ee71f5f51c3
//该文件由脚本自动生成,请勿手动修改
export interface _o_project_old_20260326 {
'artStyle'?: string | null;
'createTime'?: number | null;
'id'?: number | null;
'intro'?: string | null;
'name'?: string | null;
'projectType'?: string | null;
'type'?: string | null;
'userId'?: number | null;
'videoRatio'?: string | null;
}
export interface _o_storyboard_old_20260325 {
'camera'?: string | null;
'createTime'?: number | null;
@ -127,11 +138,13 @@ export interface o_project {
'artStyle'?: string | null;
'createTime'?: number | null;
'id'?: number | null;
'imageModel'?: string | null;
'intro'?: string | null;
'name'?: string | null;
'projectType'?: string | null;
'type'?: string | null;
'userId'?: number | null;
'videoModel'?: string | null;
'videoRatio'?: string | null;
}
export interface o_script {
@ -237,6 +250,7 @@ export interface o_videoConfig {
}
export interface DB {
"_o_project_old_20260326": _o_project_old_20260326;
"_o_storyboard_old_20260325": _o_storyboard_old_20260325;
"memories": memories;
"o_agentDeploy": o_agentDeploy;

View File

@ -16,46 +16,68 @@ export interface EventType {
class CleanNovel {
emitter: EventEmitter;
constructor() {
/** 最大并发数 */
concurrency: number;
constructor(concurrency: number = 5) {
this.emitter = new EventEmitter();
this.concurrency = concurrency;
}
private async processChapter(novel: o_novel, intansce: ReturnType<typeof u.Ai.Text>): Promise<EventType | null> {
try {
const skill = await useSkill("universal_agent.md");
const resData = await intansce.invoke({
system: skill.prompt,
messages: [
{
role: "user",
content: "请根据以下小说章节生成事件摘要:\n" + novel.chapterData!,
},
],
tools: skill.tools,
});
const preData = resData.text;
this.emitter.emit("item", { id: novel.id, event: preData });
return { id: novel.id!, event: preData };
} catch (e) {
this.emitter.emit("item", { id: novel.id, event: null, errorReason: u.error(e).message });
return null;
}
}
async start(allChapters: o_novel[], projectId: number): Promise<EventType[]> {
//所有事件
let totalEvent: EventType[] = [];
const totalEvent: EventType[] = [];
const intansce = u.Ai.Text("universalAgent");
try {
for (let gi = 0; gi < allChapters.length; gi++) {
const novel = allChapters[gi];
let resData;
try {
const skill = await useSkill("universal_agent.md");
// 并发控制:通过信号量限制同时执行的任务数
let running = 0;
let index = 0;
const results: Promise<void>[] = [];
resData = await intansce.invoke({
system: skill.prompt,
messages: [
{
role: "user",
content: "请根据以下小说章节生成事件摘要:\n" + novel.chapterData!,
},
],
tools: skill.tools,
});
console.log("%c Line:35 🍆 resData", "background:#fca650", resData);
const runNext = (): Promise<void> => {
if (index >= allChapters.length) return Promise.resolve();
const novel = allChapters[index++];
running++;
const preData = resData.text;
return this.processChapter(novel, intansce).then((result) => {
if (result) totalEvent.push(result);
running--;
return runNext();
});
};
// 启动最多 concurrency 个并发任务
const workers = Array.from(
{ length: Math.min(this.concurrency, allChapters.length) },
() => runNext()
);
await Promise.all(workers);
this.emitter.emit("item", { id: novel.id, event: preData });
totalEvent.push({ id: novel.id!, event: preData });
} catch (e) {
console.log("%c Line:51 🍩 e", "background:#93c0a4", e);
this.emitter.emit("item", { id: novel.id, event: null, errorReason: u.error(e).message });
}
}
} catch (e) {
console.error(e);
throw e;
}
return totalEvent;
}
}

View File

@ -43,7 +43,6 @@ export default function runCode(code: string) {
return exports as Record<string, any>;
}
/**
* size
*/