修复 4 个配网模式核心问题, 让 Rtc_AIavatar 分支 (含火山 RTC SDK + 软件 AEC + 完整业务)
能像 adaptation_dzbjImg_shar 一样正常配网, 同时显示居中提示文字.
============ 问题与修复 ============
### 问题 1: 配网模式 BLE 广播 ADV_DATA malloc 失败 (手机搜不到设备)
日志:
E (3731) BLE_INIT: Malloc failed
E (3731) BT_HCI: CC evt: op=0x2008 (HCI_BLE_WRITE_ADV_DATA), status=0x7
I (3731) BluetoothProvisioning: ✅ 广播启动成功 (假成功, 广播数据空)
根因:
Rtc_AIavatar 比 adaptation_dzbjImg_shar 多 ~50-80 KB DRAM 业务 .bss
(软件 AEC + HTTPS 完整状态机 + dialog watchdog + 完整 RTC 状态),
+ 火山 RTC SDK 静态库 .bss ~30-50 KB (g_cnxMgr 14.6KB, ack$14 12.6KB 等),
配网模式时 BLE Bluedroid stack 抢不到广播数据 malloc 所需的 ~10KB DRAM.
修复 (前次 commit 已做): sdkconfig 关闭 BLE 5.0 6 个特性 (项目实际只用 4.2 legacy),
省 ~15 KB controller DRAM, 广播数据 malloc 成功.
### 问题 2: 配网模式下 LCD 绘制跟 WiFi/BLE 初始化抢 DRAM 导致 reboot
日志:
E (1200) wifi:Expected to init 10 rx buffer, actual is 1
E (1220) BluetoothProvisioning: WiFi初始化失败: ESP_ERR_NO_MEM
assert failed: vQueueDelete queue.c:2355 (pxQueue) ← BLE GATT fixed_queue_new 失败 → 反向清理 NULL 队列
排查路径 (失败方案记录):
- esp_lcd_panel_draw_bitmap 一次画 360x360 (253KB): SPI queue 满, 下半屏未画 + DRAM 抢 WiFi
- 分块画 (60 行/块) + vTaskDelay 块间: SPI driver 内部 queue 持续保留 DRAM, 仍然抢
- 强制 codec output_only=false 完整 duplex: 多 15KB DRAM, BLE BTU_StartUp malloc 失败 reboot
- CONFIG_BT_BLE_DYNAMIC_ENV_MEMORY=y: 引入 BTU bt_workqueue 分配失败 → vQueueDelete NULL → assert
修复 (本次 commit): EAF 最小化初始化
movecall_moji_esp32s3.cc 配网模式调用新增的 ai_chat_screen_init_provisioning(),
跳过 8 张 EAF 资源加载 (省 4.32 MB PSRAM + ~10KB DRAM), 跳过数字人 anim,
只启用 gfx_emote renderer + 单个 gfx_label, flush buffer 启动时预分配 (~30KB DRAM 一次到位),
跟 BLE 初始化不再有动态分配冲突. 跟 adaptation_dzbjImg_shar 用 LVGL 显示 GIF 同思路,
用 EAF 替代 LVGL 避免引入 50-80KB LVGL .bss.
### 问题 3: 配网模式音效不播放
根因:
ResetWifiConfiguration 由 BOOT 按键 OnClick 调用, 跑在 esp_timer task 上下文,
vTaskDelay(4000ms) 实测只等了 1.1 秒就被唤醒, 音效没播完就 esp_restart.
修复 (前次 commit 已做): 派发到独立 task 跑 PlaySound + vTaskDelay + esp_restart,
独立 task 中 vTaskDelay 正常工作, 等 4 秒确保 解码 + DMA + 功放尾音完整.
### 问题 4: 配网时屏幕黑屏 (UX 不友好)
实施: ai_chat_screen_init_provisioning("请使用APP\n蓝牙配网~")
LCD 黑底白字居中显示提示文字, 用户感知"配网中".
label height=64 (恰好包 2 行 + 余白), GFX_ALIGN_CENTER 上下左右居中.
============ 文件改动 ============
main/application.cc:
Application 构造时显式注释: 不能在配网模式置 background_task_=nullptr
(OnAudioOutput 无判空, 会 std::mutex::lock 异常 abort).
main/boards/common/wifi_board.cc:
ResetWifiConfiguration 派发独立 task 跑 PlaySound + 4s delay + esp_restart,
EnterWifiConfigMode BLE 启动后早 return (StartBleProvisioning 内部已 Alert 音效).
main/boards/movecall-moji-esp32s3/movecall_moji_esp32s3.cc:
AI 对话模式分支: 配网时调 ai_chat_screen_init_provisioning() 显示文字,
正常模式调 ai_chat_screen_init() 显示数字人.
main/dzbj/ai_chat_ui.h:
新增 ai_chat_screen_init_provisioning(const char* hint_text) 声明.
main/dzbj/ai_chat_ui_eaf.c:
新增 ai_chat_screen_init_provisioning() 实现, EAF 最小化路径:
gfx_emote_init + gfx_disp_add + 单 label 显示文字, 跳过 EAF 资源/anim/背景图.
============ 测试结果 (设备实测) ============
- 按 BOOT 触发配网: 听到完整配网音效 (P3_LALA_WIFICONFIG 约 1 秒)
- 设备重启 → 配网模式启动 → LCD 显示"请使用APP\n蓝牙配网~" 居中
- 手机能搜到 Airhub_d0:cf:13:03:bb:f2 → 能连接 → 配网完成
- 配网完成重启 → 正常模式数字人 + RTC 对话功能正常
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
633 lines
26 KiB
C
633 lines
26 KiB
C
/*
|
||
* Phase 10: 数字人 RTC 模式 EAF UI 实现
|
||
*
|
||
* 完全替代 ai_chat_ui.c (LVGL 版),提供相同的 C API 签名让 AiChatDisplay 桥接层无需改动。
|
||
*
|
||
* 架构:
|
||
* esp_emote_gfx (gfx_emote_init + gfx_disp_add + gfx_anim)
|
||
* ↓
|
||
* mmap_assets (use_fs 模式,从 /spiflash/hiyori-assets.bin 加载)
|
||
* ↓
|
||
* panel_handle (lcd.c 暴露,已由 lcd_init 完成硬件初始化)
|
||
*
|
||
* PoC 阶段说明:
|
||
* - 只显示数字人动画(核心目的:验证显示 + 听感效果)
|
||
* - 字幕/状态文字: 仅日志输出(字体接驳留待后续,需要打包 .bin 字体到 mmap_assets)
|
||
* - 资源限制: 当前 hiyori-assets.bin 只含 m06 + m07(用户在线工具未导入 m03)
|
||
* sad/angry 等负面情绪暂时降级到 m07
|
||
*/
|
||
|
||
#include "ai_chat_ui.h"
|
||
#include "lcd.h" // 引用 panel_handle / lcd_io_handle
|
||
#include "fatfs.h" // Phase 10 step 1: DecodeImg 用于背景 JPG 解码
|
||
#include "jpeg_decoder.h" // esp_jpeg_image_output_t
|
||
#include "lvgl.h" // Phase 10 step 2: 引用 lv_font_t 类型(不调任何 LVGL 函数)
|
||
|
||
// Phase 10 step 2: 字幕字体(LVGL bitmap font C 数组,编译进 Flash .rodata,0 RAM)
|
||
// esp_emote_gfx 的 gfx_font_lv_init_adapter 接收 lv_font_t* 直接用,
|
||
// 独立解析 LVGL bitmap 数据,不调用任何 LVGL 字体函数。
|
||
extern const lv_font_t font_puhui_20_4;
|
||
|
||
#include "gfx.h"
|
||
#include "esp_mmap_assets.h"
|
||
#include "esp_log.h"
|
||
#include "esp_lcd_panel_io.h"
|
||
#include "esp_lcd_panel_ops.h"
|
||
#include "esp_spiffs.h" // Phase 10: SPIFFS 自动挂载
|
||
#include "esp_heap_caps.h"
|
||
|
||
#include <stdio.h>
|
||
#include <stdlib.h>
|
||
#include <string.h>
|
||
|
||
// Phase 10 v2 修复:
|
||
// esp_mmap_assets v2.0.0 在 use_fs=true 模式下,mmap_assets_get_mem() 返回的是文件内偏移量
|
||
// 而不是 RAM 指针(看 esp_mmap_assets.c line 523 + line 353 的 fseek 用法)。
|
||
// 把 offset 当指针 dereference 会导致 LoadProhibited panic。
|
||
// 修复:开机时用 mmap_assets_copy_by_index 把所有 EAF 数据 fread 到 PSRAM buffer,
|
||
// 运行时直接用 buffer 指针给 gfx_anim 使用。
|
||
typedef struct {
|
||
uint8_t *data; // EAF 数据 PSRAM 指针(malloc 出来)
|
||
size_t size; // EAF 大小
|
||
char name[40]; // 文件名
|
||
} eaf_cache_entry_t;
|
||
|
||
static eaf_cache_entry_t s_eaf_cache[8]; // 预留 8 个表情槽位
|
||
static int s_eaf_cache_count = 0;
|
||
|
||
// Phase 10 step 1: 背景图
|
||
#define EAF_BG_JPG_PATH "/spiflash/Background_360x360.jpg"
|
||
static uint8_t *s_bg_rgb565_data = NULL; // 解码后的 RGB565 buffer(永不释放)
|
||
static gfx_image_dsc_t s_bg_image_dsc;
|
||
static gfx_obj_t *s_bg_img_obj = NULL;
|
||
|
||
// Phase 10 step 2: 字幕
|
||
static gfx_obj_t *s_chat_label = NULL;
|
||
static char s_last_chat_content[256] = {0}; // 去重缓存
|
||
static int64_t s_last_chat_update_us = 0; // 100ms 节流
|
||
|
||
static const char *TAG = "AI_CHAT_EAF";
|
||
|
||
// ==========================================================
|
||
// 配置常量
|
||
// ==========================================================
|
||
#define EAF_ASSETS_PATH "/spiflash/hiyori-assets.bin"
|
||
#define EAF_MAX_FILES 3 // index.json + 2 个 EAF (m06 + m07)
|
||
#define EAF_DEFAULT_FPS 14 // 与工具配置一致
|
||
#define LCD_W 360
|
||
#define LCD_H 360
|
||
|
||
// ==========================================================
|
||
// 全局 EAF 上下文
|
||
// ==========================================================
|
||
static gfx_handle_t s_emote_handle = NULL;
|
||
static gfx_disp_t *s_disp = NULL;
|
||
static gfx_obj_t *s_anim_obj = NULL;
|
||
static mmap_assets_handle_t s_assets = NULL;
|
||
static int s_current_emotion_idx = -1;
|
||
static bool s_initialized = false;
|
||
|
||
// ==========================================================
|
||
// 情绪 → asset 名字 映射表
|
||
// ==========================================================
|
||
typedef struct {
|
||
const char *emotion;
|
||
const char *asset_name;
|
||
} eaf_emotion_map_t;
|
||
|
||
static const eaf_emotion_map_t s_emotion_map[] = {
|
||
// 22 种情绪 → 8 张 EAF (m01~m08, 2026-05-20 全 8 张接入)
|
||
// 注: 具体动作内容需要用户在 EAF Packer 中确认后调整, 当前是按情绪类别 +
|
||
// 编号顺序的初版分配, 烧录后看效果再精准调整
|
||
// 默认/积极组 (12 → m01..m05 均分, 每张约 2-3 种情绪)
|
||
{"neutral", "hiyori_m01.eaf"},
|
||
{"happy", "hiyori_m01.eaf"},
|
||
{"blink", "hiyori_m01.eaf"},
|
||
{"laughing", "hiyori_m02.eaf"},
|
||
{"funny", "hiyori_m02.eaf"},
|
||
{"curious", "hiyori_m02.eaf"},
|
||
{"loving", "hiyori_m03.eaf"},
|
||
{"relaxed", "hiyori_m03.eaf"},
|
||
{"delicious", "hiyori_m04.eaf"},
|
||
{"kissy", "hiyori_m04.eaf"},
|
||
{"confident", "hiyori_m05.eaf"},
|
||
{"silly", "hiyori_m05.eaf"},
|
||
// 思考/疲倦组 (5) → m06
|
||
{"sleepy", "hiyori_m06.eaf"},
|
||
{"thinking", "hiyori_m06.eaf"},
|
||
{"confused", "hiyori_m06.eaf"},
|
||
{"embarrassed", "hiyori_m06.eaf"},
|
||
{"dizzy", "hiyori_m06.eaf"},
|
||
// 负面/严肃组 (3) → m07
|
||
{"sad", "hiyori_m07.eaf"},
|
||
{"crying", "hiyori_m07.eaf"},
|
||
{"angry", "hiyori_m07.eaf"},
|
||
// 惊讶组 (2) → m08
|
||
{"surprised", "hiyori_m08.eaf"},
|
||
{"shocked", "hiyori_m08.eaf"},
|
||
};
|
||
#define EMOTION_MAP_SIZE (sizeof(s_emotion_map) / sizeof(s_emotion_map[0]))
|
||
|
||
// ==========================================================
|
||
// LCD flush 回调 (gfx → esp_lcd_panel_draw_bitmap)
|
||
// ==========================================================
|
||
static void eaf_disp_flush_cb(gfx_disp_t *disp, int x1, int y1, int x2, int y2, const void *data) {
|
||
esp_lcd_panel_handle_t panel = (esp_lcd_panel_handle_t)gfx_disp_get_user_data(disp);
|
||
esp_lcd_panel_draw_bitmap(panel, x1, y1, x2, y2, data);
|
||
}
|
||
|
||
// panel IO 完成回调,通知 gfx flush 完毕
|
||
static bool eaf_flush_io_ready(esp_lcd_panel_io_handle_t panel_io,
|
||
esp_lcd_panel_io_event_data_t *edata,
|
||
void *user_ctx) {
|
||
gfx_disp_t *disp = (gfx_disp_t *)user_ctx;
|
||
if (disp) {
|
||
gfx_disp_flush_ready(disp, true);
|
||
}
|
||
return true;
|
||
}
|
||
|
||
// ==========================================================
|
||
// 按名字查找 cache 中的 EAF entry
|
||
// ==========================================================
|
||
static int find_cache_index_by_name(const char *name) {
|
||
if (!name) return -1;
|
||
for (int i = 0; i < s_eaf_cache_count; i++) {
|
||
if (strcmp(s_eaf_cache[i].name, name) == 0 && s_eaf_cache[i].data) {
|
||
return i;
|
||
}
|
||
}
|
||
// Fallback: 找不到精确匹配时 (例如 PoC 阶段 bin 里只有一张表情但代码映射到 m06/m07),
|
||
// 用 cache 里第一个可用 EAF 替代, 保证设备能显示动画 (验证阶段方便用)
|
||
// 注: 实际产品阶段所有情绪都该有对应 EAF, 这条 fallback 仅 PoC 兜底
|
||
if (s_eaf_cache_count > 0 && s_eaf_cache[0].data) {
|
||
ESP_LOGW(TAG, "Asset %s 不在 cache, fallback 到 %s", name, s_eaf_cache[0].name);
|
||
return 0;
|
||
}
|
||
return -1;
|
||
}
|
||
|
||
// ==========================================================
|
||
// 切换表情到指定 asset (用 PSRAM 中 cache 的 EAF 数据)
|
||
// ==========================================================
|
||
static esp_err_t switch_emotion_by_asset(const char *asset_name) {
|
||
if (!s_initialized || !s_anim_obj) return ESP_ERR_INVALID_STATE;
|
||
|
||
int idx = find_cache_index_by_name(asset_name);
|
||
if (idx < 0) {
|
||
ESP_LOGW(TAG, "Asset 未在 cache: %s", asset_name);
|
||
return ESP_ERR_NOT_FOUND;
|
||
}
|
||
if (idx == s_current_emotion_idx) {
|
||
return ESP_OK; // 已是当前表情
|
||
}
|
||
|
||
uint8_t *eaf_data = s_eaf_cache[idx].data;
|
||
size_t eaf_size = s_eaf_cache[idx].size;
|
||
|
||
esp_err_t ret = gfx_emote_lock(s_emote_handle);
|
||
if (ret != ESP_OK) return ret;
|
||
|
||
gfx_anim_stop(s_anim_obj);
|
||
|
||
gfx_anim_src_t src = {
|
||
.type = GFX_ANIM_SRC_TYPE_MEMORY,
|
||
.data = eaf_data,
|
||
.data_len = eaf_size,
|
||
};
|
||
gfx_anim_set_src_desc(s_anim_obj, &src);
|
||
|
||
// 贴底显示, 让 240×320 数字人脚部紧贴屏底
|
||
// 底部 40px (= 360-320) 给数字人脚部 (跟之前 360×360 EAF 视觉一致)
|
||
// 顶部 40px 留透明空白 (chroma key 透明, 露出背景图)
|
||
// 字幕 56 高度在数字人上层 (z-index), 显示时覆盖最下方约 56px (含脚部)
|
||
gfx_obj_align(s_anim_obj, GFX_ALIGN_BOTTOM_MID, 0, 0);
|
||
|
||
// 全部帧 + EAF_DEFAULT_FPS + 永远循环
|
||
gfx_anim_set_segment(s_anim_obj, 0, 0xFFFFFFFF, EAF_DEFAULT_FPS, true);
|
||
gfx_anim_start(s_anim_obj);
|
||
|
||
gfx_emote_unlock(s_emote_handle);
|
||
|
||
s_current_emotion_idx = idx;
|
||
ESP_LOGI(TAG, "切换表情: %s (idx=%d, size=%d)", asset_name, idx, eaf_size);
|
||
return ESP_OK;
|
||
}
|
||
|
||
// ==========================================================
|
||
// 公开 C API(与 ai_chat_ui.c 完全相同的签名)
|
||
// ==========================================================
|
||
|
||
void ai_chat_screen_init(void) {
|
||
if (s_initialized) {
|
||
ESP_LOGW(TAG, "已初始化,跳过");
|
||
return;
|
||
}
|
||
|
||
ESP_LOGI(TAG, "============================");
|
||
ESP_LOGI(TAG, "=== EAF 数字人 UI 初始化 ===");
|
||
ESP_LOGI(TAG, "============================");
|
||
|
||
// 0. 确保 SPIFFS 挂载(mmap_assets_new with use_fs=true 需要 vfs 路径可访问)
|
||
size_t spiffs_total = 0, spiffs_used = 0;
|
||
esp_err_t mount_ret = esp_spiffs_info("storage", &spiffs_total, &spiffs_used);
|
||
if (mount_ret != ESP_OK) {
|
||
ESP_LOGI(TAG, "SPIFFS 未挂载,自动挂载到 /spiflash...");
|
||
esp_vfs_spiffs_conf_t spiffs_cfg = {
|
||
.base_path = "/spiflash",
|
||
.partition_label = "storage",
|
||
.max_files = 5,
|
||
.format_if_mount_failed = false,
|
||
};
|
||
mount_ret = esp_vfs_spiffs_register(&spiffs_cfg);
|
||
if (mount_ret != ESP_OK) {
|
||
ESP_LOGE(TAG, "SPIFFS 挂载失败: %s", esp_err_to_name(mount_ret));
|
||
return;
|
||
}
|
||
esp_spiffs_info("storage", &spiffs_total, &spiffs_used);
|
||
}
|
||
ESP_LOGI(TAG, "SPIFFS 已就绪: total=%u KB, used=%u KB",
|
||
(unsigned)(spiffs_total / 1024), (unsigned)(spiffs_used / 1024));
|
||
|
||
// 1. 自己解析 hiyori-assets.bin(绕过 esp_mmap_assets v2.0.0 use_fs 模式的严重 offset bug)
|
||
//
|
||
// MMAP bin 实际 layout(hex 反推得出):
|
||
// [0x00-0x03] "MMAP" magic
|
||
// [0x04-0x07] version + checksum (2B + 2B)
|
||
// [0x08-0x0B] header_size = 16
|
||
// [0x0C-0x0F] file_count
|
||
// [0x10-0x1F] reserved (16B)
|
||
// [0x20+] file entry table,每 entry = 28B (16B name + 4B size + 4B offset + 4B pad)
|
||
// [data] table 后是数据段。每个文件: 2B 0x5A 0x5A magic prefix + size 字节数据。
|
||
// entry.offset 是相对数据段起点的偏移(指向文件的 magic prefix 起点)
|
||
ESP_LOGI(TAG, "解析 hiyori-assets.bin:");
|
||
FILE *f = fopen(EAF_ASSETS_PATH, "rb");
|
||
if (!f) {
|
||
ESP_LOGE(TAG, "打开 %s 失败", EAF_ASSETS_PATH);
|
||
return;
|
||
}
|
||
uint8_t header[16];
|
||
if (fread(header, 1, 16, f) != 16 || memcmp(header, "MMAP", 4) != 0) {
|
||
ESP_LOGE(TAG, "MMAP 头解析失败");
|
||
fclose(f);
|
||
return;
|
||
}
|
||
uint32_t file_count = header[12] | (header[13] << 8) | (header[14] << 16) | (header[15] << 24);
|
||
ESP_LOGI(TAG, " MMAP file_count=%u", (unsigned)file_count);
|
||
// 自动适配新旧两版 EAF Packer 的 MMAP 格式 (2026-05-20 新版 Packer entry 变 32B)
|
||
// 旧版: header[8] = 0x10, entry 28B = name(16) + size(4) + offset(4) + pad(4)
|
||
// 新版: header[8] = 0x14, entry 32B = name(16) + reserved(4) + size(4) + offset(4) + pad(4)
|
||
const bool is_new_mmap = (header[8] == 0x14);
|
||
const size_t ENTRY_SIZE = is_new_mmap ? 32 : 28;
|
||
const size_t FSIZE_OFFSET = is_new_mmap ? 20 : 16;
|
||
const size_t FOFFSET_OFFSET = is_new_mmap ? 24 : 20;
|
||
ESP_LOGI(TAG, " MMAP 版本=%s, ENTRY_SIZE=%zu",
|
||
is_new_mmap ? "新版(0x14)" : "旧版(0x10)", ENTRY_SIZE);
|
||
// 跳过 reserved 16B 到 entry table 起点 (0x20)
|
||
fseek(f, 0x20, SEEK_SET);
|
||
const size_t DATA_START = 0x20 + file_count * ENTRY_SIZE;
|
||
|
||
s_eaf_cache_count = 0;
|
||
for (uint32_t i = 0; i < file_count; i++) {
|
||
uint8_t entry[32]; // 按最大 entry 大小分配, 兼容旧/新版
|
||
fseek(f, 0x20 + i * ENTRY_SIZE, SEEK_SET);
|
||
if (fread(entry, 1, ENTRY_SIZE, f) != ENTRY_SIZE) {
|
||
ESP_LOGE(TAG, " entry[%u] 读取失败", (unsigned)i);
|
||
continue;
|
||
}
|
||
char name[17] = {0};
|
||
memcpy(name, entry, 16);
|
||
name[16] = '\0';
|
||
uint32_t fsize = entry[FSIZE_OFFSET]
|
||
| (entry[FSIZE_OFFSET + 1] << 8)
|
||
| (entry[FSIZE_OFFSET + 2] << 16)
|
||
| (entry[FSIZE_OFFSET + 3] << 24);
|
||
uint32_t foffset = entry[FOFFSET_OFFSET]
|
||
| (entry[FOFFSET_OFFSET + 1] << 8)
|
||
| (entry[FOFFSET_OFFSET + 2] << 16)
|
||
| (entry[FOFFSET_OFFSET + 3] << 24);
|
||
|
||
// 只缓存 .eaf 文件
|
||
size_t nlen = strlen(name);
|
||
if (nlen < 4 || strcmp(name + nlen - 4, ".eaf") != 0) {
|
||
ESP_LOGI(TAG, " 跳过非 EAF: %s (size=%u)", name, (unsigned)fsize);
|
||
continue;
|
||
}
|
||
|
||
// 真实文件位置 = data_section_start + entry.offset + 2 (跳过 0x5A 0x5A magic prefix)
|
||
size_t real_offset = DATA_START + foffset + 2;
|
||
|
||
uint8_t *buf = heap_caps_malloc(fsize, MALLOC_CAP_SPIRAM);
|
||
if (!buf) {
|
||
ESP_LOGE(TAG, " PSRAM malloc 失败: %s (size=%u)", name, (unsigned)fsize);
|
||
continue;
|
||
}
|
||
if (fseek(f, real_offset, SEEK_SET) != 0 || fread(buf, 1, fsize, f) != fsize) {
|
||
ESP_LOGE(TAG, " fread 失败: %s @ offset %zu", name, real_offset);
|
||
heap_caps_free(buf);
|
||
continue;
|
||
}
|
||
|
||
// 验证 EAF format magic
|
||
if (buf[0] != 0x89 || buf[1] != 'E' || buf[2] != 'A' || buf[3] != 'F') {
|
||
ESP_LOGE(TAG, " EAF magic 失败: %s (got %02x %02x %02x %02x)",
|
||
name, buf[0], buf[1], buf[2], buf[3]);
|
||
heap_caps_free(buf);
|
||
continue;
|
||
}
|
||
|
||
if (s_eaf_cache_count >= (int)(sizeof(s_eaf_cache)/sizeof(s_eaf_cache[0]))) {
|
||
ESP_LOGW(TAG, " cache 已满,丢弃: %s", name);
|
||
heap_caps_free(buf);
|
||
break;
|
||
}
|
||
s_eaf_cache[s_eaf_cache_count].data = buf;
|
||
s_eaf_cache[s_eaf_cache_count].size = fsize;
|
||
strncpy(s_eaf_cache[s_eaf_cache_count].name, name, sizeof(s_eaf_cache[0].name) - 1);
|
||
s_eaf_cache[s_eaf_cache_count].name[sizeof(s_eaf_cache[0].name) - 1] = '\0';
|
||
ESP_LOGI(TAG, " ✓ Cached [%d] %s (%u bytes) @ %p (file_offset=%zu)",
|
||
s_eaf_cache_count, name, (unsigned)fsize, buf, real_offset);
|
||
s_eaf_cache_count++;
|
||
}
|
||
fclose(f);
|
||
|
||
if (s_eaf_cache_count == 0) {
|
||
ESP_LOGE(TAG, "没有 EAF 资源被加载,初始化中止");
|
||
return;
|
||
}
|
||
ESP_LOGI(TAG, "EAF 预加载完成,共 %d 个表情可用", s_eaf_cache_count);
|
||
|
||
// 2. 初始化 gfx 核心(绑 Core 0,与原 LVGL 一致避免抢音频 Core 1)
|
||
gfx_core_config_t gfx_cfg = {
|
||
.fps = 25,
|
||
.task = GFX_EMOTE_INIT_CONFIG(),
|
||
};
|
||
gfx_cfg.task.task_priority = 4;
|
||
gfx_cfg.task.task_affinity = 0; // Core 0
|
||
gfx_cfg.task.task_stack = 8 * 1024;
|
||
s_emote_handle = gfx_emote_init(&gfx_cfg);
|
||
if (!s_emote_handle) {
|
||
ESP_LOGE(TAG, "gfx_emote_init 失败");
|
||
return;
|
||
}
|
||
|
||
// 3. 添加 display(接管 panel_handle)
|
||
gfx_disp_config_t disp_cfg = {
|
||
.h_res = LCD_W,
|
||
.v_res = LCD_H,
|
||
.flush_cb = eaf_disp_flush_cb,
|
||
.update_cb = NULL,
|
||
.user_data = (void *)panel_handle,
|
||
.flags = {
|
||
.swap = true, // RGB565 字节序(与 LVGL 配置一致)
|
||
.buff_dma = true,
|
||
.buff_spiram = false,
|
||
.double_buffer = true,
|
||
},
|
||
.buffers = { .buf1 = NULL, .buf2 = NULL, .buf_pixels = LCD_W * 20 },
|
||
};
|
||
s_disp = gfx_disp_add(s_emote_handle, &disp_cfg);
|
||
if (!s_disp) {
|
||
ESP_LOGE(TAG, "gfx_disp_add 失败");
|
||
gfx_emote_deinit(s_emote_handle);
|
||
s_emote_handle = NULL;
|
||
return;
|
||
}
|
||
|
||
// 注册 panel IO 完成回调
|
||
const esp_lcd_panel_io_callbacks_t cbs = { .on_color_trans_done = eaf_flush_io_ready };
|
||
esp_lcd_panel_io_register_event_callbacks(lcd_io_handle, &cbs, s_disp);
|
||
|
||
// 4. 设置背景色 = BG_COLOR (0x000000 黑色,与 LVGL 版一致)
|
||
gfx_disp_set_bg_color(s_disp, GFX_COLOR_HEX(0x000000));
|
||
|
||
// 4.5 加载 360x360 静态背景图(JPG → esp_jpeg 解码 → RGB565 → gfx_img)
|
||
// 必须先于 gfx_anim_create,让背景在最底层(数字人在上层)
|
||
{
|
||
esp_jpeg_image_output_t bg_outdata;
|
||
esp_err_t bg_ret = DecodeImg((char *)EAF_BG_JPG_PATH, &s_bg_rgb565_data, &bg_outdata);
|
||
if (bg_ret != ESP_OK || !s_bg_rgb565_data) {
|
||
ESP_LOGW(TAG, "背景图加载失败: %s(继续无背景)", EAF_BG_JPG_PATH);
|
||
} else {
|
||
s_bg_image_dsc.header.magic = C_ARRAY_HEADER_MAGIC;
|
||
s_bg_image_dsc.header.cf = GFX_COLOR_FORMAT_RGB565;
|
||
s_bg_image_dsc.header.w = bg_outdata.width;
|
||
s_bg_image_dsc.header.h = bg_outdata.height;
|
||
s_bg_image_dsc.header.stride = bg_outdata.width * 2;
|
||
s_bg_image_dsc.data_size = bg_outdata.width * bg_outdata.height * 2;
|
||
s_bg_image_dsc.data = s_bg_rgb565_data;
|
||
|
||
gfx_emote_lock(s_emote_handle);
|
||
s_bg_img_obj = gfx_img_create(s_disp);
|
||
if (s_bg_img_obj) {
|
||
gfx_img_src_t bg_src = {
|
||
.type = GFX_IMG_SRC_TYPE_IMAGE_DSC,
|
||
.data = &s_bg_image_dsc,
|
||
};
|
||
gfx_img_set_src_desc(s_bg_img_obj, &bg_src);
|
||
gfx_obj_align(s_bg_img_obj, GFX_ALIGN_CENTER, 0, 0);
|
||
ESP_LOGI(TAG, "背景图已加载: %dx%d (%u KB RGB565 @ %p)",
|
||
bg_outdata.width, bg_outdata.height,
|
||
(unsigned)(bg_outdata.width * bg_outdata.height * 2 / 1024),
|
||
s_bg_rgb565_data);
|
||
} else {
|
||
ESP_LOGE(TAG, "gfx_img_create 失败");
|
||
}
|
||
gfx_emote_unlock(s_emote_handle);
|
||
}
|
||
}
|
||
|
||
// 5. 创建动画对象 + 加载默认表情 m06(在背景之后创建 → 显示在背景上层)
|
||
s_anim_obj = gfx_anim_create(s_disp);
|
||
if (!s_anim_obj) {
|
||
ESP_LOGE(TAG, "gfx_anim_create 失败");
|
||
return;
|
||
}
|
||
|
||
s_initialized = true;
|
||
|
||
// 默认表情 = neutral → m06
|
||
switch_emotion_by_asset("hiyori_m06.eaf");
|
||
|
||
// 6. 字幕 label(在 anim 之后创建,确保 z-index 在最上层)
|
||
gfx_emote_lock(s_emote_handle);
|
||
s_chat_label = gfx_label_create(s_disp);
|
||
if (s_chat_label) {
|
||
// 字体:LVGL bitmap font C 数组直接喂入(esp_emote_gfx 独立解析,不调 LVGL 函数)
|
||
gfx_label_set_font(s_chat_label, (gfx_font_t)&font_puhui_20_4);
|
||
// 尺寸:宽 300 / 高 56(最多 2 行 × 24px 行高 + 8px 余量;超出部分自动 clip)
|
||
// - 字体 20px + line_spacing 4 = 24px/行
|
||
// - height = 56 → 2 行 + 余白,避免裁剪到字底
|
||
// - 超过 2 行的字幕被 height clip 不显示(长字幕只看到前两行)
|
||
// - 300 比 320 略小,给左右各 ~30px 余白
|
||
gfx_obj_set_size(s_chat_label, 300, 56);
|
||
// 自动换行(按 obj geometry width 断行,中文支持字符级断行)
|
||
gfx_label_set_long_mode(s_chat_label, GFX_LABEL_LONG_WRAP);
|
||
// 居中对齐
|
||
gfx_label_set_text_align(s_chat_label, GFX_TEXT_ALIGN_CENTER);
|
||
// 黑色字(与原 LVGL 版一致,背景图浅色)
|
||
gfx_label_set_color(s_chat_label, GFX_COLOR_HEX(0x000000));
|
||
gfx_label_set_bg_enable(s_chat_label, false); // 无背景框
|
||
gfx_label_set_line_spacing(s_chat_label, 4); // 行间距 4px
|
||
// 底部贴边:y_ofs = -4(贴近屏幕底,留 4px 余白防溢出)
|
||
gfx_obj_align(s_chat_label, GFX_ALIGN_BOTTOM_MID, 0, -4);
|
||
gfx_label_set_text(s_chat_label, "");
|
||
// 隐藏,等收到字幕再显示
|
||
gfx_obj_set_visible(s_chat_label, false);
|
||
ESP_LOGI(TAG, "字幕 label 创建成功(300×90, 黑字, WRAP, 居中)");
|
||
} else {
|
||
ESP_LOGE(TAG, "字幕 label 创建失败");
|
||
}
|
||
gfx_emote_unlock(s_emote_handle);
|
||
|
||
ESP_LOGI(TAG, "=== EAF 数字人 UI 初始化完成 ===");
|
||
}
|
||
|
||
void ai_chat_set_status(const char* status) {
|
||
// PoC 阶段不显示状态文字(gfx_label 需要字体资源接驳,留待后续)
|
||
if (status) {
|
||
ESP_LOGI(TAG, "状态: %s(PoC 阶段暂不显示)", status);
|
||
}
|
||
}
|
||
|
||
void ai_chat_set_emotion(const char* emotion) {
|
||
if (!emotion || !s_initialized) return;
|
||
|
||
// 查映射表
|
||
const char *asset_name = "hiyori_m06.eaf"; // 默认 fallback
|
||
for (size_t i = 0; i < EMOTION_MAP_SIZE; i++) {
|
||
if (strcmp(emotion, s_emotion_map[i].emotion) == 0) {
|
||
asset_name = s_emotion_map[i].asset_name;
|
||
break;
|
||
}
|
||
}
|
||
switch_emotion_by_asset(asset_name);
|
||
}
|
||
|
||
void ai_chat_set_chat_message(const char* role, const char* content) {
|
||
(void)role;
|
||
if (!s_initialized || !s_chat_label) return;
|
||
if (!content) content = "";
|
||
|
||
// 去重:流式 ASR 大量重复中间结果,相同内容直接返回
|
||
if (strncmp(s_last_chat_content, content, sizeof(s_last_chat_content)) == 0) {
|
||
return;
|
||
}
|
||
|
||
// 50ms 节流(EAF 渲染比 LVGL 快很多,缩短节流让字幕更跟手)
|
||
// 例外:空内容(清空字幕)不节流
|
||
int64_t now_us = esp_timer_get_time();
|
||
if (content[0] != '\0' && (now_us - s_last_chat_update_us) < 50000) {
|
||
return;
|
||
}
|
||
s_last_chat_update_us = now_us;
|
||
|
||
esp_err_t lk = gfx_emote_lock(s_emote_handle);
|
||
if (lk != ESP_OK) {
|
||
ESP_LOGW(TAG, "字幕 lock 失败,跳过");
|
||
return;
|
||
}
|
||
gfx_label_set_text(s_chat_label, content);
|
||
// 空内容时隐藏(避免空 label 占位)
|
||
gfx_obj_set_visible(s_chat_label, content[0] != '\0');
|
||
gfx_emote_unlock(s_emote_handle);
|
||
|
||
// 缓存当前内容
|
||
strncpy(s_last_chat_content, content, sizeof(s_last_chat_content) - 1);
|
||
s_last_chat_content[sizeof(s_last_chat_content) - 1] = '\0';
|
||
}
|
||
|
||
void ai_chat_resume_animation(void) {
|
||
// EAF 动画由 gfx_anim_start 持续播放,无需手动 resume
|
||
ESP_LOGD(TAG, "resume_animation(EAF 模式下自动循环,无需操作)");
|
||
}
|
||
|
||
// ==========================================================
|
||
// 配网模式专用 - 最小化 EAF 初始化, 只显示文字提示
|
||
// 设计目的: 配网模式下 DRAM 紧张 (RTC SDK + 应用 .bss 占 30-50KB),
|
||
// 不能完整初始化 EAF (加载 4.32 MB EAF 资源 + 数字人 anim).
|
||
// 只启用 gfx renderer + 单个 label, 在启动早期预分配 flush buffer,
|
||
// 避免跟后续 BLE Bluedroid 初始化抢动态分配的 DRAM.
|
||
// 跟 adaptation_dzbjImg_shar (用 LVGL 显示 GIF) 同思路, 用 EAF 替代 LVGL 省 .bss
|
||
// ==========================================================
|
||
void ai_chat_screen_init_provisioning(const char* hint_text) {
|
||
if (s_initialized) {
|
||
ESP_LOGW(TAG, "[配网] EAF 已初始化, 跳过");
|
||
return;
|
||
}
|
||
ESP_LOGI(TAG, "============================");
|
||
ESP_LOGI(TAG, "=== EAF 最小化 (配网模式) ===");
|
||
ESP_LOGI(TAG, "============================");
|
||
|
||
// 1. 初始化 gfx 核心 (Core 0, 跟原版一致, 不抢音频 Core 1)
|
||
gfx_core_config_t gfx_cfg = {
|
||
.fps = 25,
|
||
.task = GFX_EMOTE_INIT_CONFIG(),
|
||
};
|
||
gfx_cfg.task.task_priority = 4;
|
||
gfx_cfg.task.task_affinity = 0;
|
||
gfx_cfg.task.task_stack = 8 * 1024;
|
||
s_emote_handle = gfx_emote_init(&gfx_cfg);
|
||
if (!s_emote_handle) {
|
||
ESP_LOGE(TAG, "[配网] gfx_emote_init 失败");
|
||
return;
|
||
}
|
||
|
||
// 2. 添加 display (接管 panel_handle, 启动时预分配 flush buffer ~30KB DRAM)
|
||
gfx_disp_config_t disp_cfg = {
|
||
.h_res = LCD_W,
|
||
.v_res = LCD_H,
|
||
.flush_cb = eaf_disp_flush_cb,
|
||
.update_cb = NULL,
|
||
.user_data = (void *)panel_handle,
|
||
.flags = {
|
||
.swap = true,
|
||
.buff_dma = true,
|
||
.buff_spiram = false,
|
||
.double_buffer = true,
|
||
},
|
||
.buffers = { .buf1 = NULL, .buf2 = NULL, .buf_pixels = LCD_W * 20 },
|
||
};
|
||
s_disp = gfx_disp_add(s_emote_handle, &disp_cfg);
|
||
if (!s_disp) {
|
||
ESP_LOGE(TAG, "[配网] gfx_disp_add 失败");
|
||
gfx_emote_deinit(s_emote_handle);
|
||
s_emote_handle = NULL;
|
||
return;
|
||
}
|
||
|
||
const esp_lcd_panel_io_callbacks_t cbs = { .on_color_trans_done = eaf_flush_io_ready };
|
||
esp_lcd_panel_io_register_event_callbacks(lcd_io_handle, &cbs, s_disp);
|
||
|
||
// 3. 设置背景色 = 黑色
|
||
gfx_disp_set_bg_color(s_disp, GFX_COLOR_HEX(0x000000));
|
||
|
||
s_initialized = true;
|
||
|
||
// 4. 创建文字 label - 不加载 EAF 资源 + 不创建 anim
|
||
// 上下/左右居中: label height 恰好包文字 (避免 gfx_label 内部顶对齐导致视觉偏上),
|
||
// 配合 GFX_ALIGN_CENTER 整体居中到屏幕中央
|
||
gfx_emote_lock(s_emote_handle);
|
||
s_chat_label = gfx_label_create(s_disp);
|
||
if (s_chat_label) {
|
||
gfx_label_set_font(s_chat_label, (gfx_font_t)&font_puhui_20_4);
|
||
// 字体 20px + line_spacing 8 = 28px/行, 2 行文字 = 56px, 留 8px 余白 = 64px
|
||
gfx_obj_set_size(s_chat_label, 300, 64);
|
||
gfx_label_set_long_mode(s_chat_label, GFX_LABEL_LONG_WRAP);
|
||
gfx_label_set_text_align(s_chat_label, GFX_TEXT_ALIGN_CENTER); // 文字水平居中
|
||
gfx_label_set_color(s_chat_label, GFX_COLOR_HEX(0xFFFFFF)); // 白字 (黑底背景)
|
||
gfx_label_set_bg_enable(s_chat_label, false);
|
||
gfx_label_set_line_spacing(s_chat_label, 8);
|
||
// label 整体居中到屏幕正中 → 文字视觉上上下左右居中
|
||
gfx_obj_align(s_chat_label, GFX_ALIGN_CENTER, 0, 0);
|
||
gfx_label_set_text(s_chat_label, hint_text ? hint_text : "请使用APP\n蓝牙配网~");
|
||
gfx_obj_set_visible(s_chat_label, true);
|
||
ESP_LOGI(TAG, "[配网] 文字 label 创建成功 (居中显示)");
|
||
} else {
|
||
ESP_LOGE(TAG, "[配网] gfx_label_create 失败");
|
||
}
|
||
gfx_emote_unlock(s_emote_handle);
|
||
|
||
ESP_LOGI(TAG, "=== EAF 最小化初始化完成 ===");
|
||
}
|