Rdzleo
31982ba7b9
✅ 验证完成： - 音频卡顿明显改善（用户实测） - 数字人 hiyori 动画正常显示 - nm 验证：固件中 0 个 lv_*/lvgl_* 函数符号 - kapi.bin: 4.7MB → 2.75MB（-42%） 关键改动: - main/dzbj/ai_chat_ui_eaf.c (404 行新增)： 完全替代 LVGL 版 ai_chat_ui.c，提供同名 C API（ai_chat_screen_init / set_status / set_emotion / set_chat_message / resume_animation）。 AiChatDisplay C++ 桥接层无需改动。 内部用 gfx_emote_init + gfx_disp_add + gfx_anim + mmap_assets。 - main/CMakeLists.txt：双轨编译 CONFIG_BAJI_BADGE_MODE=y → ai_chat_ui.c (LVGL) + bg_gif_demo.c CONFIG_BAJI_BADGE_MODE=n → ai_chat_ui_eaf.c (esp_emote_gfx) - main/dzbj/dzbj_init.c：EAF 模式跳过 lvgl_lcd_init() 调用 - main/dzbj/lcd.c/h：暴露 lcd_io_handle 给 EAF 注册 IO 完成回调 踩坑修复（commit message 留档供后续参考）: 1. esp_mmap_assets v2.0.0 在 use_fs=true 模式下 mmap_assets_get_mem() 返回的是文件内偏移量而非 RAM 指针（fseek bug + offset 没加 data_section_start），导致 LoadProhibited panic。 解决：完全绕过 mmap_assets，自己 fopen + 解析 MMAP bin 头 （layout: 头 16B + 每 entry 28B + data 段每文件 2B magic + 数据）。 2. esp_emote_gfx 期望 esp_lcd_touch v2.x 新 API，项目用 v1.1.2 旧 API。 在 managed_components 内 gfx_touch.c 加 shim 桥接（local patch， reconfigure 后需 reapply）。 3. EAF format magic 是 0x89 'EAF'（gfx_eaf_dec.h），不是 0x5A5A （那是 esp_mmap_assets 内部文件分隔符）。 4. SPIFFS 需要在 ai_chat_screen_init 入口自动挂载（不能依赖 bg_gif_demo 的惰性挂载，那个已被 CONFIG 排除）。 依赖增量: - espressif2022/esp_emote_gfx: ~3.0.5 - espressif/esp_mmap_assets: * (仅用于声明依赖，运行时被绕过) 数字人模式核心 UI 范围: - 显示数字人动画 ✅ (hiyori_m06/m07, 居中循环) - 情绪 → GIF 映射 ✅ (23 情绪 → 2 EAF，sad/angry 暂用 m07，m03 待补) - 字幕/状态文字: stub ⏳（字体接驳留待后续，需打包 .bin 字体到资源） - 触摸: 不支持（PoC 阶段不需要） Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
XiaoZhi AI Chatbot
Introduction
👉 Build your AI chat companion with ESP32+SenseVoice+Qwen72B!【bilibili】
👉 Equipping XiaoZhi with DeepSeek's smart brain【bilibili】
👉 Build your own AI companion, a beginner's guide【bilibili】
Project Purpose
This is an open-source project released under the MIT license, allowing anyone to use it freely, including for commercial purposes.
Through this project, we aim to help more people get started with AI hardware development and understand how to implement rapidly evolving large language models in actual hardware devices. Whether you're a student interested in AI or a developer exploring new technologies, this project offers valuable learning experiences.
Everyone is welcome to participate in the project's development and improvement. If you have any ideas or suggestions, please feel free to raise an Issue or join the chat group.
Learning & Discussion QQ Group: 376893254
Implemented Features
- Wi-Fi / ML307 Cat.1 4G
- BOOT button wake-up and interruption, supporting both click and long-press triggers
- Offline voice wake-up ESP-SR
- Streaming voice dialogue (WebSocket or UDP protocol)
- Support for 5 languages: Mandarin, Cantonese, English, Japanese, Korean SenseVoice
- Voice print recognition to identify who's calling AI's name 3D Speaker
- Large model TTS (Volcano Engine or CosyVoice)
- Large Language Models (Qwen, DeepSeek, Doubao)
- Configurable prompts and voice tones (custom characters)
- Short-term memory, self-summarizing after each conversation round
- OLED / LCD display showing signal strength or conversation content
- Support for LCD image expressions
- Multi-language support (Chinese, English)
Hardware Section
Breadboard DIY Practice
See the Feishu document tutorial:
👉 XiaoZhi AI Chatbot Encyclopedia
Breadboard demonstration:
Supported Open Source Hardware
- LiChuang ESP32-S3 Development Board
- Espressif ESP32-S3-BOX3
- M5Stack CoreS3
- AtomS3R + Echo Base
- AtomMatrix + Echo Base
- Magic Button 2.4
- Waveshare ESP32-S3-Touch-AMOLED-1.8
- LILYGO T-Circle-S3
- XiaGe Mini C3
- Moji XiaoZhi AI Derivative Version
- CuiCan AI pendant
- WMnologo-Xingzhi-1.54TFT
- SenseCAP Watcher
Firmware Section
Flashing Without Development Environment
For beginners, it's recommended to first use the firmware that can be flashed without setting up a development environment.
The firmware connects to the official xiaozhi.me server by default. Currently, personal users can register an account to use the Qwen real-time model for free.
👉 Flash Firmware Guide (No IDF Environment)
Development Environment
- Cursor or VSCode
- Install ESP-IDF plugin, select SDK version 5.3 or above
- Linux is preferred over Windows for faster compilation and fewer driver issues
- Use Google C++ code style, ensure compliance when submitting code
Developer Documentation
- Board Customization Guide - Learn how to create custom board adaptations for XiaoZhi
- IoT Control Module - Understand how to control IoT devices through AI voice commands
AI Agent Configuration
If you already have a XiaoZhi AI chatbot device, you can configure it through the xiaozhi.me console.
👉 Backend Operation Tutorial (Old Interface)
Technical Principles and Private Deployment
👉 Detailed WebSocket Communication Protocol Documentation
For server deployment on personal computers, refer to another MIT-licensed project xiaozhi-esp32-server