Rdzleo
b1577d8418
问题与修复（按发现顺序）: 【问题1】AudioLoop guard 冻结 OnAudioOutput → 待命音队列永远不被消费 - 现象: WaitForAudioPlayback 3秒超时, 无声 - 根因: EnterIdleHibernate Step 0 设 hibernating_=true, AudioLoop 顶层 if(hibernating_) continue 会同时跳过 OnAudioInput + OnAudioOutput, PlaySound 入队的 Opus 永远不解码。 - 修复: guard 下沉到 OnAudioInput 入口（仅 input 侧关 codec 有 bad_alloc 风险）, OnAudioOutput 自带 codec->output_enabled() 保护。 【问题2】volc_rtc_stop 副作用关 I2S 通道 → codec 状态错位 - 现象: 听到杂音而非待命音; i2s_channel_disable "not enabled yet" 错误 - 根因: 火山 RTC SDK 的 stop 内部关闭 ES8311 I2S, 但 codec class 内部 output_enabled_ 标志仍是 true → 状态错位, PlaySound 写入到 disabled 的 I2S。 - 修复: EnterIdleHibernate 在 PlaySound 前显式 EnableOutput(false→true) 强制重新激活 I2S, 并灌 200ms silence 覆盖 DMA 残留。 【问题3 - 真因】protocol downlink_is_pcm_ 标志位污染 → Opus 被当 PCM 字节流写出 - 现象: 杂音仍在 - 根因: 火山 RTC 下行音频是 PCM, DataCallback 设 downlink_is_pcm_=true。 LeaveRoom 没重置这个 flag, 后续 hibernate 中 PlaySound 入队的 Opus 包, OnAudioOutput 读到 protocol_->downlink_is_pcm() 返回 true → treat_as_pcm=true → 跳过 opus_decoder, 直接把 Opus 编码字节当 int16 PCM 样本写到 codec → 杂音。 - 修复: VolcRtcProtocol::LeaveRoom 末尾重置 downlink_is_pcm_=false + first_downlink_logged_=false。唤醒重连后 DataCallback 收到首包会立即 重新设置该 flag, 不影响欢迎语 PCM 播放。 【问题4】WaitForAudioPlayback 完成 ≠ DMA 输出完成 → 尾音被截 - 现象: 待命音能听见但提前结束约 1 秒 - 根因: WaitForAudioPlayback 只判断 audio_decode_queue_ 出队完毕。 OnAudioOutput 是 background_task Schedule 异步执行 codec write, 队列空 ≠ codec 写完; codec.Write 返回 ≠ I2S DMA + ES8311 FIFO 输出完毕。 - 修复: WaitForAudioPlayback 之后追加 background_task->WaitForCompletion + vTaskDelay(1000) 让 DMA 尾音自然衰减, 才关 player_pipeline。 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
XiaoZhi AI Chatbot
Introduction
👉 Build your AI chat companion with ESP32+SenseVoice+Qwen72B!【bilibili】
👉 Equipping XiaoZhi with DeepSeek's smart brain【bilibili】
👉 Build your own AI companion, a beginner's guide【bilibili】
Project Purpose
This is an open-source project released under the MIT license, allowing anyone to use it freely, including for commercial purposes.
Through this project, we aim to help more people get started with AI hardware development and understand how to implement rapidly evolving large language models in actual hardware devices. Whether you're a student interested in AI or a developer exploring new technologies, this project offers valuable learning experiences.
Everyone is welcome to participate in the project's development and improvement. If you have any ideas or suggestions, please feel free to raise an Issue or join the chat group.
Learning & Discussion QQ Group: 376893254
Implemented Features
- Wi-Fi / ML307 Cat.1 4G
- BOOT button wake-up and interruption, supporting both click and long-press triggers
- Offline voice wake-up ESP-SR
- Streaming voice dialogue (WebSocket or UDP protocol)
- Support for 5 languages: Mandarin, Cantonese, English, Japanese, Korean SenseVoice
- Voice print recognition to identify who's calling AI's name 3D Speaker
- Large model TTS (Volcano Engine or CosyVoice)
- Large Language Models (Qwen, DeepSeek, Doubao)
- Configurable prompts and voice tones (custom characters)
- Short-term memory, self-summarizing after each conversation round
- OLED / LCD display showing signal strength or conversation content
- Support for LCD image expressions
- Multi-language support (Chinese, English)
Hardware Section
Breadboard DIY Practice
See the Feishu document tutorial:
👉 XiaoZhi AI Chatbot Encyclopedia
Breadboard demonstration:
Supported Open Source Hardware
- LiChuang ESP32-S3 Development Board
- Espressif ESP32-S3-BOX3
- M5Stack CoreS3
- AtomS3R + Echo Base
- AtomMatrix + Echo Base
- Magic Button 2.4
- Waveshare ESP32-S3-Touch-AMOLED-1.8
- LILYGO T-Circle-S3
- XiaGe Mini C3
- Moji XiaoZhi AI Derivative Version
- CuiCan AI pendant
- WMnologo-Xingzhi-1.54TFT
- SenseCAP Watcher
Firmware Section
Flashing Without Development Environment
For beginners, it's recommended to first use the firmware that can be flashed without setting up a development environment.
The firmware connects to the official xiaozhi.me server by default. Currently, personal users can register an account to use the Qwen real-time model for free.
👉 Flash Firmware Guide (No IDF Environment)
Development Environment
- Cursor or VSCode
- Install ESP-IDF plugin, select SDK version 5.3 or above
- Linux is preferred over Windows for faster compilation and fewer driver issues
- Use Google C++ code style, ensure compliance when submitting code
Developer Documentation
- Board Customization Guide - Learn how to create custom board adaptations for XiaoZhi
- IoT Control Module - Understand how to control IoT devices through AI voice commands
AI Agent Configuration
If you already have a XiaoZhi AI chatbot device, you can configure it through the xiaozhi.me console.
👉 Backend Operation Tutorial (Old Interface)
Technical Principles and Private Deployment
👉 Detailed WebSocket Communication Protocol Documentation
For server deployment on personal computers, refer to another MIT-licensed project xiaozhi-esp32-server