Baji_Rtc_Toy/tools/patch_eaf_transparency.py
Rdzleo eceadda807 feat(ui): Phase 10 step 1+2 - 背景图 + 中文字幕 + 数字人透明
完成数字人模式 UI 的"背景图叠加 + 实时字幕"功能。所有改动基于 EAF
框架(Phase 10 commit 31982ba),保持 0 个 lv_* UI 函数链接进固件。

Step 1: JPG 背景图叠加
- ai_chat_ui_eaf.c 加 esp_jpeg 解码 Background_360x360.jpg →
  RGB565 buffer (252KB PSRAM) → gfx_img_create 作为底层
- z-index 通过创建顺序控制: 背景 → 数字人 anim → 字幕 label
- 选项 A 保留 JPG (~20KB SPIFFS) 比选项 B (252KB .bin) 省 232KB

数字人透明: esp_emote_gfx local patch (gfx_anim.c::gfx_anim_render_24bit_pixels)
- 根因: 在线 EAF Packer 默认导出 24-bit 模式,工具不暴露 bit_depth
  选项,alpha 滑块拉到 0 无法保存,导致 GIF 透明像素被烘焙成屏幕背景
  色 (黑色 RGB888 #000000)
- 解决: 在 24-bit 渲染函数加 chroma key,跳过近黑像素让背景图露出
- 阈值演化 v1 (0x0000) → v3 (16) → v4 (24),最终 RGB888 ≤ (24,24,24)
- 保留 R/G/B AND 关系(三分量都小才透明),保护数字人本体暗色不破洞
- 双字节序判定,兼容 disp_config_t.flags.swap = true

Step 2: 中文字幕 (gfx_label + LVGL bitmap font 方案 A)
- 字体方案对比 3 方案后选方案 A(C 数组 XIP from Flash):
  • A: 1.4MB Flash + 0 RAM (推荐)
  • B: xiaozhi-fonts .bin 1.18MB SPIFFS + 1.18MB PSRAM
  • C: 自转 .bin ~2.8MB 总占用
- extern const lv_font_t font_puhui_20_4 → gfx_label_set_font 直接喂
- linker 副作用: 仅引入 7 个 LVGL 函数 ~2.2KB(lv_font_get_bitmap_fmt_txt
  / lv_mem_* 幽灵符号),无 lv_obj/lv_disp/lv_indev 等 UI 框架函数
- 字幕参数: 300×56 (2 行限制) + 行间距 4 + 贴底 y_ofs=-4
- GFX_LABEL_LONG_WRAP 字符级断行(中文友好),CENTER 居中
- 流式 TTS 节流 50ms(比 LVGL 100ms 短,EAF 渲染更快)

工具脚本 (tools/patch_eaf_transparency.py)
- 探索性脚本:解析 hiyori-assets.bin 尝试修补 EAF palette alpha
- 实际未生效(工具导出 24-bit 无 palette),保留作为 EAF bin layout
  解析参考

固件大小: 2.75MB → 4.30MB(+1.55MB = 字体 1.4MB + 字幕代码 + 背景图代码)
分区余量: 50% → 25% (1.42MB 空闲,安全)

完整踩坑经验已沉淀到 ~/.claude/CLAUDE.md §13 + 项目 memory。

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 17:38:31 +08:00

198 lines
7.6 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

#!/usr/bin/env python3
"""
Phase 10 step 1.5: 修补 hiyori-assets.bin 让 hiyori GIF 黑色背景变透明
ESP Emote GFX Packer 在线工具导出 EAF 时把 GIF 透明像素填充为不透明黑色
(palette idx ? = BGRA 0x00 0x00 0x00 0xFF),导致叠加到背景图上显示黑色矩形。
本脚本直接 patch .bin 文件内嵌的 EAF 数据:
1. 解析 MMAP bin 找每个 EAF 资源
2. 对每个 EAF 解析每帧 header → 找 palette 中 RGB=黑色 entry
3. 把这些 entry 的 alpha 字节从 0xFF 改 0x00让 idx 视为透明)
4. 重算 EAF stored_chk 写回
EAF frame header layout (from gfx_eaf_dec.c):
[9] bit_depth (1B): 4/8/24
[10-11] width
[12-13] height
[14-15] blocks
[16-17] block_height
[18+] block_len table: blocks × 4B
[+] palette: num_colors × 4B (BGRA)
[+] data
EAF main header (from gfx_eaf_dec.h):
[0] format magic 0x89
[1-3] "EAF"
[4-7] total_frames (uint32 LE)
[8-11] stored_chk (uint32 LE)
[12-15] stored_len (uint32 LE)
[16+] frame table: total_frames × 8B (frame_size + frame_offset)
[16 + total_frames*8] frame data (each frame: 2B 0x5A5A magic + frame_header + data)
"""
import sys
import struct
from pathlib import Path
BIN_PATH = Path(__file__).parent.parent / "spiffs_image" / "hiyori-assets.bin"
def patch_eaf(eaf_bytes: bytearray) -> int:
"""对单个 EAF 二进制内容做透明 patch返回修改的 palette entry 数"""
# 校验 EAF magic
if eaf_bytes[0] != 0x89 or eaf_bytes[1:4] not in (b"EAF", b"AAF"):
raise ValueError(f"非 EAF/AAF 数据 (开头: {eaf_bytes[:4].hex()})")
total_frames = struct.unpack("<I", eaf_bytes[4:8])[0]
stored_chk_orig = struct.unpack("<I", eaf_bytes[8:12])[0]
stored_len = struct.unpack("<I", eaf_bytes[12:16])[0]
# 校验原 checksum
calc_orig = sum(eaf_bytes[16:16 + stored_len]) & 0xFFFFFFFF
if calc_orig != stored_chk_orig:
print(f" ⚠️ 原 checksum 不匹配 (stored={stored_chk_orig:#x} calc={calc_orig:#x})")
print(f" stored_len={stored_len} 数据长度={len(eaf_bytes)-16}")
# 容忍esp_mmap_assets 短读 2 字节也可能影响这里,不强中止
print(f" EAF: frames={total_frames}, stored_chk={stored_chk_orig:#x}, stored_len={stored_len}")
# frame table 起点 = offset 16
# 每 entry 8B: frame_size(4) + frame_offset(4)
# 帧数据起点 = 16 + total_frames * 8
frame_table_off = 16
frame_data_base = frame_table_off + total_frames * 8
patched_entries = 0
for fi in range(total_frames):
entry_off = frame_table_off + fi * 8
frame_size = struct.unpack("<I", eaf_bytes[entry_off:entry_off+4])[0]
frame_offset = struct.unpack("<I", eaf_bytes[entry_off+4:entry_off+8])[0]
# 帧绝对位置 = frame_data_base + frame_offset
# 前 2B 是 0x5A 0x5A magic prefixEAF_MAGIC_HEAD
abs_frame_pos = frame_data_base + frame_offset
if abs_frame_pos + 2 > len(eaf_bytes):
print(f" ❌ frame {fi} 偏移越界 (abs={abs_frame_pos}, total={len(eaf_bytes)})")
continue
# 验证 frame magic
if eaf_bytes[abs_frame_pos] != 0x5A or eaf_bytes[abs_frame_pos+1] != 0x5A:
print(f" ❌ frame {fi} magic 错误 (got {eaf_bytes[abs_frame_pos]:#x} {eaf_bytes[abs_frame_pos+1]:#x})")
continue
# frame_header 从 magic 后开始,但 BIT_DEPTH_OFFSET = 9 是相对什么?
# 看 gfx_eaf_dec.c line 198: file_data[EAF_FRAME_BIT_DEPTH_OFFSET]
# 其中 file_data = entries[i].frame_mem (注:含 0x5A5A magic 前缀)
# 所以 file_data + 9 = 帧 magic 前缀之后第 7 字节
# layout (相对 abs_frame_pos):
# [0-1] 0x5A 0x5A magic
# [2-8] format (3) + version (4)
# [9] bit_depth
# [10-11] width
# [12-13] height
# [14-15] blocks
# [16-17] block_height
# [18+] block_len: blocks × 4B
# [+] palette: (1<<bit_depth) × 4B (BGRA)
bit_depth = eaf_bytes[abs_frame_pos + 9]
if bit_depth not in (4, 8, 24):
print(f" ❌ frame {fi} bit_depth 异常: {bit_depth}")
continue
if bit_depth == 24:
# 24-bit 无 palette
continue
blocks = struct.unpack("<H", eaf_bytes[abs_frame_pos+14:abs_frame_pos+16])[0]
num_colors = 1 << bit_depth
palette_off = abs_frame_pos + 18 + blocks * 4
# 扫 palette 找 RGB=0,0,0 entries
for ci in range(num_colors):
entry = abs_frame_pos + 18 + blocks * 4 + ci * 4 - abs_frame_pos + palette_off - palette_off
entry_abs = palette_off + ci * 4
b = eaf_bytes[entry_abs + 0]
g = eaf_bytes[entry_abs + 1]
r = eaf_bytes[entry_abs + 2]
a = eaf_bytes[entry_abs + 3]
# 找 R=G=B=0 且 A 非 0 的 entry改 A=0
if b == 0 and g == 0 and r == 0 and a != 0:
eaf_bytes[entry_abs + 3] = 0
patched_entries += 1
if patched_entries <= 5 or fi == 0:
print(f" frame[{fi}] palette[{ci}]: BGRA={b:02x}{g:02x}{r:02x}{a:02x} → 00000000 透明")
# 重算 stored_chk
new_chk = sum(eaf_bytes[16:16 + stored_len]) & 0xFFFFFFFF
struct.pack_into("<I", eaf_bytes, 8, new_chk)
print(f" ✓ checksum 重算: {stored_chk_orig:#x}{new_chk:#x}, patched {patched_entries} 个 palette entries")
return patched_entries
def main():
if not BIN_PATH.exists():
print(f"❌ 文件不存在: {BIN_PATH}")
sys.exit(1)
data = bytearray(BIN_PATH.read_bytes())
print(f"载入 {BIN_PATH} ({len(data)} bytes)")
# 解析 MMAP header
if bytes(data[:4]) != b"MMAP":
print(f"❌ 不是 MMAP 文件 (head={data[:4]})")
sys.exit(1)
file_count = struct.unpack("<I", data[12:16])[0]
print(f"MMAP file_count = {file_count}")
ENTRY_SIZE = 28
DATA_START = 0x20 + file_count * ENTRY_SIZE
total_patched = 0
for i in range(file_count):
entry_off = 0x20 + i * ENTRY_SIZE
name = data[entry_off:entry_off+16].split(b'\0')[0].decode('utf-8', errors='replace')
size = struct.unpack("<I", data[entry_off+16:entry_off+20])[0]
offset = struct.unpack("<I", data[entry_off+20:entry_off+24])[0]
if not name.endswith('.eaf'):
print(f"\n[{i}] 跳过非 EAF: {name}")
continue
# 真实位置 = DATA_START + offset + 2 (跳 0x5A5A magic)
real_off = DATA_START + offset + 2
print(f"\n[{i}] {name} @ file_offset={real_off} size={size}")
# 抽出 EAF 数据(含 magic prefix 跳过)
eaf_view = data[real_off:real_off + size]
if len(eaf_view) != size:
print(f" ❌ 数据截断: 期望 {size}, 实际 {len(eaf_view)}")
continue
# 转 bytearray 才能 in-place 修改
eaf_buf = bytearray(eaf_view)
try:
patched = patch_eaf(eaf_buf)
except Exception as e:
print(f" ❌ patch 失败: {e}")
continue
total_patched += patched
# 写回原 data
data[real_off:real_off + size] = eaf_buf
# 备份原文件
backup = BIN_PATH.with_suffix('.bin.bak')
if not backup.exists():
backup.write_bytes(BIN_PATH.read_bytes())
print(f"\n✓ 原文件备份到: {backup}")
else:
print(f"\n(备份已存在: {backup})")
BIN_PATH.write_bytes(bytes(data))
print(f"✓ 已写回 {BIN_PATH}")
print(f"✓ 共 patched {total_patched} 个 palette entries黑色 RGB+A=FF → A=00")
if __name__ == "__main__":
main()