Rdzleo
e95d0c414e
实现 ESP32-S3 上单摄像头人脸追踪的核心代码骨架，替代 Grove Vision AI V2
模块，通过 UART 发送人脸坐标驱动 RP2040 控制的眼球/YAW 舵机。
## 规划文档（docs/phase-01-face-tracking/）
- GOAL.md Phase 目标与 5 大成功标准
- RESEARCH.md esp-dl v3.2/3.3 + human_face_detect 0.4.1 技术调研
- PLAN.md 15 个原子任务的执行计划（T01-T15）
- PLAN_CHECK.md 计划审查报告（PASS_WITH_NOTES）
- PROGRESS.md 执行进度追踪（批次 1-3 已完成）
## 批次 1：依赖与开关（T01-T03）
- main/idf_component.yml
新增 esp-dl ~3.3.0 + human_face_detect 0.4.1（仅 S3/P4）
esp-sr 从 ~2.2.0 升级到 ~2.3.1，解决 esp-dsp 1.6/1.7 版本冲突
- main/Kconfig.projbuild
新增 CONFIG_XIAOZHI_ENABLE_FACE_TRACKING 开关（默认 y，depends on S3）
新增 CONFIG_XIAOZHI_FACE_TRACKING_FPS_CHOICE（5/10/15）
- main/boards/common/esp32_camera.{h,cc}
新增 ProbeFrameCapture() 最小 V4L2 DQBUF/QBUF 探针（T01）
- main/application.cc
Start() 末尾调用 probe 验证摄像头硬件链路
## 批次 2：人脸检测核心（T04-T06）
- main/boards/common/esp32_camera.{h,cc}
新增 FrameRef 结构体 + CaptureForDetection/ReleaseDetectionFrame
双超时 mutex 策略：face_tracker 10ms timeout 跳帧，Capture() RAII guard
- main/face_tracker.{h,cc}（新建）
Core 0 / 优先级 2 / 栈 8KB 独立任务
集成 esp-dl HumanFaceDetect 推理
坐标归一化 cx*224/W-112，匹配 RP2040 pixel_centre=112
多人脸遍历挑 score 最高，避免多脸时眼球摇摆
三重保护：Kconfig depends on S3 + 源文件 #if 守卫 + CMake 条件排除
- main/CMakeLists.txt
非 S3 目标从 SOURCES 移除 face_tracker.cc
## 批次 3：UART 协议扩展（T07）
- main/uart_component.{h,cc}
新增 uart_send_face(x,y) 发送 face:x,y\r\n 协议
extern "C" 链接名配合 face_tracker 的弱符号声明
全局 TX mutex 保护所有 UART 写入，防并发帧交织
uart_send_string 同步加锁保持一致性
## 编译验证
idf.py build 通过，固件 2.51MB / 剩余 1.46MB (36% free)
当前 face_tracker 未被 application 激活（留到 T11），
UART/摄像头现有功能零影响。
## 未完成（下次继续）
- T01 硬件 probe 实机验证
- T08-T10 RP2040 端 parse_face + facetrack 双数据源改造
- T11-T15 application 接入 + 端到端联调 + 性能调优 + 最终验收
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
126 lines
3.5 KiB
YAML
126 lines
3.5 KiB
YAML
## IDF Component Manager Manifest File
|
||
dependencies:
|
||
waveshare/esp_lcd_sh8601: 1.0.2
|
||
espressif/esp_lcd_ili9341: ==1.2.0
|
||
espressif/esp_lcd_gc9a01: ==2.0.1
|
||
espressif/esp_lcd_st77916: ^1.0.1
|
||
espressif/esp_lcd_axs15231b: ^1.0.0
|
||
|
||
espressif/esp_lcd_st7701:
|
||
version: ^1.1.4
|
||
rules:
|
||
- if: target in [esp32s3, esp32p4]
|
||
espressif/esp_lcd_st7796:
|
||
version: 1.3.5
|
||
rules:
|
||
- if: target in [esp32, esp32s2, esp32s3, esp32p4]
|
||
espressif/esp_lcd_spd2010: ==1.0.2
|
||
espressif/esp_io_expander_tca9554: ==2.0.0
|
||
espressif/esp_lcd_panel_io_additions: ^1.0.1
|
||
78/esp_lcd_nv3023: ~1.0.0
|
||
78/esp-wifi-connect: ~2.6.2
|
||
78/esp-opus-encoder: ~2.4.1
|
||
78/esp-ml307: ~3.5.1
|
||
78/xiaozhi-fonts: ~1.5.5
|
||
espressif/led_strip: ~3.0.1
|
||
espressif/esp_codec_dev: ~1.5
|
||
# [Phase 01] 2026-04-17 升级：从 ~2.2.0 升到 ~2.3.1 以解决 esp-dsp 版本冲突
|
||
# esp-sr 2.2.x 依赖 esp-dsp==1.6.0，而 esp-dl 3.3.0 依赖 esp-dsp==1.7.0
|
||
# esp-sr 2.3.0+ 已切换到 esp-dsp 1.7.0，与 esp-dl 兼容
|
||
espressif/esp-sr: ~2.3.1
|
||
espressif/button: ~4.1.3
|
||
espressif/knob: ^1.0.0
|
||
espressif/esp_video:
|
||
version: ==1.3.1 # for compatibility. update version may need to modify this project code.
|
||
rules:
|
||
- if: target not in [esp32]
|
||
espressif/esp_image_effects:
|
||
version: ^1.0.1
|
||
rules:
|
||
- if: target not in [esp32]
|
||
espressif/esp_lcd_touch_ft5x06: ~1.0.7
|
||
espressif/esp_lcd_touch_gt911: ^1
|
||
espressif/esp_lcd_touch_gt1151: ^1
|
||
waveshare/esp_lcd_touch_cst9217: ^1.0.3
|
||
espressif/esp_lcd_touch_cst816s: ^1.0.6
|
||
lvgl/lvgl: ~9.3.0
|
||
esp_lvgl_port: ~2.6.0
|
||
espressif/esp_io_expander_tca95xx_16bit: ^2.0.0
|
||
espressif2022/image_player: ^1.1.1
|
||
espressif2022/esp_emote_gfx: ^1.1.2
|
||
espressif/adc_mic: ^0.2.1
|
||
espressif/esp_mmap_assets: '>=1.2'
|
||
txp666/otto-emoji-gif-component:
|
||
version: ^1.0.3
|
||
rules:
|
||
- if: target in [esp32s3]
|
||
espressif/adc_battery_estimation: ^0.2.0
|
||
espressif/esp_new_jpeg: ^0.6.1
|
||
|
||
# [Phase 01] esp-dl 人脸检测依赖（仅 S3 目标）
|
||
# 偏差记录 2026-04-17: PLAN 原定 esp-dl==3.2.0，但 human_face_detect 0.4.1
|
||
# 实际依赖 esp-dl ~3.3.0（registry 版本约束），升级到 ~3.3.0 以解决版本冲突
|
||
espressif/esp-dl:
|
||
version: "~3.3.0"
|
||
rules:
|
||
- if: target in [esp32s3, esp32p4]
|
||
espressif/human_face_detect:
|
||
version: "==0.4.1"
|
||
rules:
|
||
- if: target in [esp32s3, esp32p4]
|
||
|
||
# SenseCAP Watcher Board
|
||
wvirgil123/sscma_client:
|
||
version: 1.0.2
|
||
rules:
|
||
- if: target in [esp32s3]
|
||
|
||
tny-robotics/sh1106-esp-idf: ^1.0.0
|
||
waveshare/esp_lcd_jd9365_10_1:
|
||
version: '*'
|
||
rules:
|
||
- if: target in [esp32p4]
|
||
waveshare/esp_lcd_st7703:
|
||
version: '*'
|
||
rules:
|
||
- if: target in [esp32p4]
|
||
espressif/esp32_p4_function_ev_board:
|
||
version: ^5.0.3
|
||
rules:
|
||
- if: target in [esp32p4]
|
||
espressif/esp_lcd_ili9881c:
|
||
version: ^1.0.1
|
||
rules:
|
||
- if: target in [esp32p4]
|
||
espressif/esp_lcd_ek79007:
|
||
version: ^1.0.3
|
||
rules:
|
||
- if: target in [esp32p4]
|
||
espressif/esp_hosted:
|
||
version: 2.0.17
|
||
rules:
|
||
- if: target in [esp32h2, esp32p4]
|
||
espressif/esp_wifi_remote:
|
||
version: '*'
|
||
rules:
|
||
- if: target in [esp32p4]
|
||
espfriends/servo_dog_ctrl:
|
||
version: ^0.1.8
|
||
rules:
|
||
- if: target in [esp32c3]
|
||
|
||
llgok/cpp_bus_driver:
|
||
version: 1.1.0
|
||
rules:
|
||
- if: target in [esp32p4]
|
||
|
||
espressif/bmi270_sensor:
|
||
version: ^0.1.0
|
||
rules:
|
||
- if: target in [esp32s3, esp32c5]
|
||
|
||
## Required IDF version
|
||
idf:
|
||
version: '>=5.4.0'
|
||
espressif/esp_lcd_touch_st7123: ^1.0.0
|