完成数字人模式 UI 的"背景图叠加 + 实时字幕"功能。所有改动基于 EAF 框架(Phase 10 commit 31982ba),保持 0 个 lv_* UI 函数链接进固件。 Step 1: JPG 背景图叠加 - ai_chat_ui_eaf.c 加 esp_jpeg 解码 Background_360x360.jpg → RGB565 buffer (252KB PSRAM) → gfx_img_create 作为底层 - z-index 通过创建顺序控制: 背景 → 数字人 anim → 字幕 label - 选项 A 保留 JPG (~20KB SPIFFS) 比选项 B (252KB .bin) 省 232KB 数字人透明: esp_emote_gfx local patch (gfx_anim.c::gfx_anim_render_24bit_pixels) - 根因: 在线 EAF Packer 默认导出 24-bit 模式,工具不暴露 bit_depth 选项,alpha 滑块拉到 0 无法保存,导致 GIF 透明像素被烘焙成屏幕背景 色 (黑色 RGB888 #000000) - 解决: 在 24-bit 渲染函数加 chroma key,跳过近黑像素让背景图露出 - 阈值演化 v1 (0x0000) → v3 (16) → v4 (24),最终 RGB888 ≤ (24,24,24) - 保留 R/G/B AND 关系(三分量都小才透明),保护数字人本体暗色不破洞 - 双字节序判定,兼容 disp_config_t.flags.swap = true Step 2: 中文字幕 (gfx_label + LVGL bitmap font 方案 A) - 字体方案对比 3 方案后选方案 A(C 数组 XIP from Flash): • A: 1.4MB Flash + 0 RAM (推荐) • B: xiaozhi-fonts .bin 1.18MB SPIFFS + 1.18MB PSRAM • C: 自转 .bin ~2.8MB 总占用 - extern const lv_font_t font_puhui_20_4 → gfx_label_set_font 直接喂 - linker 副作用: 仅引入 7 个 LVGL 函数 ~2.2KB(lv_font_get_bitmap_fmt_txt / lv_mem_* 幽灵符号),无 lv_obj/lv_disp/lv_indev 等 UI 框架函数 - 字幕参数: 300×56 (2 行限制) + 行间距 4 + 贴底 y_ofs=-4 - GFX_LABEL_LONG_WRAP 字符级断行(中文友好),CENTER 居中 - 流式 TTS 节流 50ms(比 LVGL 100ms 短,EAF 渲染更快) 工具脚本 (tools/patch_eaf_transparency.py) - 探索性脚本:解析 hiyori-assets.bin 尝试修补 EAF palette alpha - 实际未生效(工具导出 24-bit 无 palette),保留作为 EAF bin layout 解析参考 固件大小: 2.75MB → 4.30MB(+1.55MB = 字体 1.4MB + 字幕代码 + 背景图代码) 分区余量: 50% → 25% (1.42MB 空闲,安全) 完整踩坑经验已沉淀到 ~/.claude/CLAUDE.md §13 + 项目 memory。 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
198 lines
7.6 KiB
Python
198 lines
7.6 KiB
Python
#!/usr/bin/env python3
|
||
"""
|
||
Phase 10 step 1.5: 修补 hiyori-assets.bin 让 hiyori GIF 黑色背景变透明
|
||
|
||
ESP Emote GFX Packer 在线工具导出 EAF 时把 GIF 透明像素填充为不透明黑色
|
||
(palette idx ? = BGRA 0x00 0x00 0x00 0xFF),导致叠加到背景图上显示黑色矩形。
|
||
|
||
本脚本直接 patch .bin 文件内嵌的 EAF 数据:
|
||
1. 解析 MMAP bin 找每个 EAF 资源
|
||
2. 对每个 EAF 解析每帧 header → 找 palette 中 RGB=黑色 entry
|
||
3. 把这些 entry 的 alpha 字节从 0xFF 改 0x00(让 idx 视为透明)
|
||
4. 重算 EAF stored_chk 写回
|
||
|
||
EAF frame header layout (from gfx_eaf_dec.c):
|
||
[9] bit_depth (1B): 4/8/24
|
||
[10-11] width
|
||
[12-13] height
|
||
[14-15] blocks
|
||
[16-17] block_height
|
||
[18+] block_len table: blocks × 4B
|
||
[+] palette: num_colors × 4B (BGRA)
|
||
[+] data
|
||
|
||
EAF main header (from gfx_eaf_dec.h):
|
||
[0] format magic 0x89
|
||
[1-3] "EAF"
|
||
[4-7] total_frames (uint32 LE)
|
||
[8-11] stored_chk (uint32 LE)
|
||
[12-15] stored_len (uint32 LE)
|
||
[16+] frame table: total_frames × 8B (frame_size + frame_offset)
|
||
[16 + total_frames*8] frame data (each frame: 2B 0x5A5A magic + frame_header + data)
|
||
"""
|
||
import sys
|
||
import struct
|
||
from pathlib import Path
|
||
|
||
BIN_PATH = Path(__file__).parent.parent / "spiffs_image" / "hiyori-assets.bin"
|
||
|
||
def patch_eaf(eaf_bytes: bytearray) -> int:
|
||
"""对单个 EAF 二进制内容做透明 patch,返回修改的 palette entry 数"""
|
||
# 校验 EAF magic
|
||
if eaf_bytes[0] != 0x89 or eaf_bytes[1:4] not in (b"EAF", b"AAF"):
|
||
raise ValueError(f"非 EAF/AAF 数据 (开头: {eaf_bytes[:4].hex()})")
|
||
|
||
total_frames = struct.unpack("<I", eaf_bytes[4:8])[0]
|
||
stored_chk_orig = struct.unpack("<I", eaf_bytes[8:12])[0]
|
||
stored_len = struct.unpack("<I", eaf_bytes[12:16])[0]
|
||
|
||
# 校验原 checksum
|
||
calc_orig = sum(eaf_bytes[16:16 + stored_len]) & 0xFFFFFFFF
|
||
if calc_orig != stored_chk_orig:
|
||
print(f" ⚠️ 原 checksum 不匹配 (stored={stored_chk_orig:#x} calc={calc_orig:#x})")
|
||
print(f" stored_len={stored_len} 数据长度={len(eaf_bytes)-16}")
|
||
# 容忍:esp_mmap_assets 短读 2 字节也可能影响这里,不强中止
|
||
|
||
print(f" EAF: frames={total_frames}, stored_chk={stored_chk_orig:#x}, stored_len={stored_len}")
|
||
|
||
# frame table 起点 = offset 16
|
||
# 每 entry 8B: frame_size(4) + frame_offset(4)
|
||
# 帧数据起点 = 16 + total_frames * 8
|
||
frame_table_off = 16
|
||
frame_data_base = frame_table_off + total_frames * 8
|
||
|
||
patched_entries = 0
|
||
for fi in range(total_frames):
|
||
entry_off = frame_table_off + fi * 8
|
||
frame_size = struct.unpack("<I", eaf_bytes[entry_off:entry_off+4])[0]
|
||
frame_offset = struct.unpack("<I", eaf_bytes[entry_off+4:entry_off+8])[0]
|
||
|
||
# 帧绝对位置 = frame_data_base + frame_offset
|
||
# 前 2B 是 0x5A 0x5A magic prefix(EAF_MAGIC_HEAD)
|
||
abs_frame_pos = frame_data_base + frame_offset
|
||
if abs_frame_pos + 2 > len(eaf_bytes):
|
||
print(f" ❌ frame {fi} 偏移越界 (abs={abs_frame_pos}, total={len(eaf_bytes)})")
|
||
continue
|
||
# 验证 frame magic
|
||
if eaf_bytes[abs_frame_pos] != 0x5A or eaf_bytes[abs_frame_pos+1] != 0x5A:
|
||
print(f" ❌ frame {fi} magic 错误 (got {eaf_bytes[abs_frame_pos]:#x} {eaf_bytes[abs_frame_pos+1]:#x})")
|
||
continue
|
||
|
||
# frame_header 从 magic 后开始,但 BIT_DEPTH_OFFSET = 9 是相对什么?
|
||
# 看 gfx_eaf_dec.c line 198: file_data[EAF_FRAME_BIT_DEPTH_OFFSET]
|
||
# 其中 file_data = entries[i].frame_mem (注:含 0x5A5A magic 前缀)
|
||
# 所以 file_data + 9 = 帧 magic 前缀之后第 7 字节
|
||
# layout (相对 abs_frame_pos):
|
||
# [0-1] 0x5A 0x5A magic
|
||
# [2-8] format (3) + version (4)
|
||
# [9] bit_depth
|
||
# [10-11] width
|
||
# [12-13] height
|
||
# [14-15] blocks
|
||
# [16-17] block_height
|
||
# [18+] block_len: blocks × 4B
|
||
# [+] palette: (1<<bit_depth) × 4B (BGRA)
|
||
|
||
bit_depth = eaf_bytes[abs_frame_pos + 9]
|
||
if bit_depth not in (4, 8, 24):
|
||
print(f" ❌ frame {fi} bit_depth 异常: {bit_depth}")
|
||
continue
|
||
if bit_depth == 24:
|
||
# 24-bit 无 palette
|
||
continue
|
||
|
||
blocks = struct.unpack("<H", eaf_bytes[abs_frame_pos+14:abs_frame_pos+16])[0]
|
||
num_colors = 1 << bit_depth
|
||
palette_off = abs_frame_pos + 18 + blocks * 4
|
||
|
||
# 扫 palette 找 RGB=0,0,0 entries
|
||
for ci in range(num_colors):
|
||
entry = abs_frame_pos + 18 + blocks * 4 + ci * 4 - abs_frame_pos + palette_off - palette_off
|
||
entry_abs = palette_off + ci * 4
|
||
b = eaf_bytes[entry_abs + 0]
|
||
g = eaf_bytes[entry_abs + 1]
|
||
r = eaf_bytes[entry_abs + 2]
|
||
a = eaf_bytes[entry_abs + 3]
|
||
# 找 R=G=B=0 且 A 非 0 的 entry,改 A=0
|
||
if b == 0 and g == 0 and r == 0 and a != 0:
|
||
eaf_bytes[entry_abs + 3] = 0
|
||
patched_entries += 1
|
||
if patched_entries <= 5 or fi == 0:
|
||
print(f" frame[{fi}] palette[{ci}]: BGRA={b:02x}{g:02x}{r:02x}{a:02x} → 00000000 透明")
|
||
|
||
# 重算 stored_chk
|
||
new_chk = sum(eaf_bytes[16:16 + stored_len]) & 0xFFFFFFFF
|
||
struct.pack_into("<I", eaf_bytes, 8, new_chk)
|
||
print(f" ✓ checksum 重算: {stored_chk_orig:#x} → {new_chk:#x}, patched {patched_entries} 个 palette entries")
|
||
|
||
return patched_entries
|
||
|
||
|
||
def main():
|
||
if not BIN_PATH.exists():
|
||
print(f"❌ 文件不存在: {BIN_PATH}")
|
||
sys.exit(1)
|
||
|
||
data = bytearray(BIN_PATH.read_bytes())
|
||
print(f"载入 {BIN_PATH} ({len(data)} bytes)")
|
||
|
||
# 解析 MMAP header
|
||
if bytes(data[:4]) != b"MMAP":
|
||
print(f"❌ 不是 MMAP 文件 (head={data[:4]})")
|
||
sys.exit(1)
|
||
|
||
file_count = struct.unpack("<I", data[12:16])[0]
|
||
print(f"MMAP file_count = {file_count}")
|
||
|
||
ENTRY_SIZE = 28
|
||
DATA_START = 0x20 + file_count * ENTRY_SIZE
|
||
|
||
total_patched = 0
|
||
for i in range(file_count):
|
||
entry_off = 0x20 + i * ENTRY_SIZE
|
||
name = data[entry_off:entry_off+16].split(b'\0')[0].decode('utf-8', errors='replace')
|
||
size = struct.unpack("<I", data[entry_off+16:entry_off+20])[0]
|
||
offset = struct.unpack("<I", data[entry_off+20:entry_off+24])[0]
|
||
|
||
if not name.endswith('.eaf'):
|
||
print(f"\n[{i}] 跳过非 EAF: {name}")
|
||
continue
|
||
|
||
# 真实位置 = DATA_START + offset + 2 (跳 0x5A5A magic)
|
||
real_off = DATA_START + offset + 2
|
||
print(f"\n[{i}] {name} @ file_offset={real_off} size={size}")
|
||
|
||
# 抽出 EAF 数据(含 magic prefix 跳过)
|
||
eaf_view = data[real_off:real_off + size]
|
||
if len(eaf_view) != size:
|
||
print(f" ❌ 数据截断: 期望 {size}, 实际 {len(eaf_view)}")
|
||
continue
|
||
|
||
# 转 bytearray 才能 in-place 修改
|
||
eaf_buf = bytearray(eaf_view)
|
||
try:
|
||
patched = patch_eaf(eaf_buf)
|
||
except Exception as e:
|
||
print(f" ❌ patch 失败: {e}")
|
||
continue
|
||
total_patched += patched
|
||
|
||
# 写回原 data
|
||
data[real_off:real_off + size] = eaf_buf
|
||
|
||
# 备份原文件
|
||
backup = BIN_PATH.with_suffix('.bin.bak')
|
||
if not backup.exists():
|
||
backup.write_bytes(BIN_PATH.read_bytes())
|
||
print(f"\n✓ 原文件备份到: {backup}")
|
||
else:
|
||
print(f"\n(备份已存在: {backup})")
|
||
|
||
BIN_PATH.write_bytes(bytes(data))
|
||
print(f"✓ 已写回 {BIN_PATH}")
|
||
print(f"✓ 共 patched {total_patched} 个 palette entries(黑色 RGB+A=FF → A=00)")
|
||
|
||
|
||
if __name__ == "__main__":
|
||
main()
|