修复 Pendant 衍生项目无痛移植问题
实机验证通过后，按 Kapi 无屏底座路线补齐 Pendant RTC 吊坠项目的迁移修复。 1. BLE 配网与资源隔离 - sdkconfig.defaults 开启 BT 优先 PSRAM 分配，并将 LWIP socket 上限提升到 20 - sdkconfig.defaults.esp32s3 允许 BSS/NOINIT 放入 PSRAM，释放内部 SRAM 给 BLE/WiFi/RTC - 配网模式 codec 使用 StartOutputOnly()，跳过麦克风 RX DMA 和 ES7210 输入链路 - ResetWifiConfiguration() 改为独立 wifi_reset task，避免在 iot_button/esp_timer 回调中阻塞延时 - WifiBoard 增加 IsWifiConfigMode()，供启动阶段判断是否走配网资源隔离路径 2. 音频底噪与 DMA 残留修复 - AudioCodec 增加 StartOutputOnly()，支持仅启动扬声器输出 - RTC 音频通道打开后灌入 200ms silence PCM，覆盖 I2S DMA 残留数据 - 软退出进入待命前重启 codec output 并再次灌静音，减少待命音/欢迎语前杂音 - box_audio_codec 在无硬件回采时使用 channel_mask=0，避免 I2S slot mask 被错误污染 3. 软件 loopback AEC - 引入 esp_aec 底层同步 API，使用 DAC 输出复制构建 ref ring - 上行 mic PCM 与延迟 ref 做同步消回声，适配无屏无硬件回采的 Pendant 形态 - AEC 采用 lazy init，减少启动阶段对 WiFi/BLE 内部 SRAM 的压力 - ref 静音时直接 passthrough，避免 AI 静音后误压制用户语音 - 在 player_pipeline_write 和 codec->OutputData 两条下行路径都追加 ref hook 4. RTC 连接稳定性与软退出 - VolcRtcProtocol 增加 LeaveRoom(bool notify_closed)，支持 stop 房间但保留 rtc_handle - hibernate 路径使用 LeaveRoom(false)，避免关闭回调顺手关掉 codec output 导致待命音无声 - LeaveRoom/ForceRebuildEngine 重置 downlink_is_pcm_ 和首包标志，避免本地 Opus 音效被当 PCM 播成杂音 - OpenAudioChannel 连续失败 3 次后 ForceRebuildEngine，清理 RTC SDK 内部异常状态 - 加入 DIAG-RTC socket/heap/PSRAM/RSSI 日志，便于定位 ICE socket 和内存问题 5. Dialog watchdog 与 BOOT 唤醒 - Dialog watchdog 到期不再写 reboot_dlg_idle 后 esp_restart - 新增 EnterIdleHibernate()：软退房、清空残留音频队列、关闭麦克风、播放待命音后静默 - 新增 WakeFromHibernate()：BOOT 唤醒后复用 RTC engine 并通过 ToggleChatState() 重连 RTC - BOOT 单击优先判断 IsHibernating()，异步唤醒，避免走普通按键状态机 - hibernate 期间禁止 PowerSaveTimer 进入 Light Sleep，保护 I2C/codec 总线 6. 文档与衍生项目沉淀 - 更新石头光源属性检测方案文档 - 将 Pendant 实测通过的软退出、AEC、BLE 配网隔离经验同步到衍生项目移植规则
This commit is contained in:
parent
93f0e19d1d
commit
8111515277
@ -1,4 +1,61 @@
|
||||
# 石头同频匹配方案说明
|
||||
# 石头光源属性检测方案说明
|
||||
|
||||
## 0. VEML7700 驱动概述
|
||||
|
||||
AI智能吊坠项目
|
||||
该项目基于RTC底框架进行开发，融合了VEML7700的光源检测硬件驱动，可以检测水晶或宝石的属性；
|
||||
此设备主要用于AI对话+水晶石识别功能大概业务如下：
|
||||
1、手机APP端会有此产品的所有类别水晶石的光源信息库（可能需要设备端进行光源检测后汇总各类别石头的信息库给到APP端进行存储，每种 石头需要不同光照条件和角度的信息收集，尽量全面，防止设备端检测到石头信息后发送到收集APP端进行匹配的时候无法匹配到！）
|
||||
2、用户可以购买不同类别的石头，每天使用不同的时候进行检测匹配，从而从APP端根据用户选择不同的时候进行运势等信息推送！
|
||||
3、根据不同石头匹配的信息和推送后，APP端会有数字人形象展示及互动！
|
||||
4、设备推送信息到手机APP端的接口可以统一一个网址，然后通过消息类别来区分是什么类型的业务！
|
||||
|
||||
### 驱动来源与设计决策
|
||||
|
||||
本项目的 VEML7700 驱动为**自主编写**，参考了以下两个社区/官方驱动的设计思路，但未直接移植任何一个：
|
||||
|
||||
| 参考项目 | 取舍 |
|
||||
|---------|------|
|
||||
| [tedyapo/arduino-VEML7700](https://github.com/tedyapo/arduino-VEML7700) (Arduino) | 参考了 Lux 计算公式、Vishay 非线性校正多项式系数、自动量程算法策略 |
|
||||
| [esp-idf-lib/veml7700](https://github.com/esp-idf-lib/veml7700) v1.0.7 (ESP-IDF 社区组件) | 参考了寄存器定义和配置结构设计 |
|
||||
|
||||
**未直接使用上述驱动的原因**：
|
||||
- Arduino 驱动依赖 `Wire.h`，需要重写整个 I2C 通信层才能移植到 ESP-IDF
|
||||
- ESP-IDF 社区组件依赖 `i2cdev` 库（旧版 I2C API），与本项目使用的 `i2c_master` 新 API **不兼容**，同一 I2C 端口无法共存
|
||||
- ESP-IDF 社区组件存在 `veml7700_get_config()` 函数 Bug（连续 4 次赋值给 `integration_time` 字段）、`veml7700_get_ambient_light()` 缺少 mutex 保护、Lux 计算无非线性校正等问题
|
||||
|
||||
最终方案：基于项目已有的 `I2cDevice` 基类（使用 ESP-IDF `i2c_master` 新 API），自主实现 VEML7700 驱动，复用 Arduino 驱动中经过验证的 Lux 校正公式和自动量程策略。
|
||||
|
||||
### 驱动主要功能
|
||||
|
||||
| 功能 | 说明 |
|
||||
|------|------|
|
||||
| 双通道读取 | ALS（光视函数通道，偏绿光 555nm）+ White（宽谱白光通道） |
|
||||
| 原始值 + Lux 值 | 同时提供 16-bit 原始计数值和经校正的浮点 Lux 值 |
|
||||
| Vishay 非线性校正 | 高照度（>1000 lux）时的多项式补偿，系数来自 Vishay 应用笔记（Horner 法） |
|
||||
| 自动量程 | 自动调节增益（x1/x2/÷4/÷8）和积分时间（25~800ms），覆盖 0.007~120000 lux 全量程 |
|
||||
| 3 次采样中位数 | 过滤偶发异常读数（实测观察到单次异常偏低 50% 的情况） |
|
||||
| 可配置参数 | 增益、积分时间、持续保护次数、省电模式、中断阈值均可独立配置 |
|
||||
| I2C 总线共享 | 与 ES8311（音频编解码器）、ES7210（ADC）、QMI8658A（IMU）共用同一 I2C 总线（GPIO17/18），地址无冲突 |
|
||||
|
||||
### 驱动文件
|
||||
|
||||
| 文件 | 说明 |
|
||||
|------|------|
|
||||
| `main/boards/common/veml7700.h` | 驱动头文件 — 寄存器定义、枚举、API 声明 |
|
||||
| `main/boards/common/veml7700.cc` | 驱动实现 — I2C 16-bit 读写、Lux 计算、非线性校正、自动量程算法 |
|
||||
|
||||
### 与参考驱动的功能对比
|
||||
|
||||
| 功能 | Arduino 驱动 | ESP-IDF 社区组件 | 本项目驱动 |
|
||||
|------|-------------|-----------------|-----------|
|
||||
| I2C 层 | Wire.h (Arduino) | i2cdev (旧API) | **i2c_master (新API)** |
|
||||
| Lux 非线性校正 | Vishay 多项式 | 无 | **Vishay 多项式** |
|
||||
| 自动量程 | 有 | 无 | **有** |
|
||||
| 原始值接口 | 有 | 无 | **有** |
|
||||
| 3次采样中位数 | 无 | 无 | **有** |
|
||||
| 线程安全 | N/A | mutex（ALS 漏了） | I2C 总线级保护 |
|
||||
| get_config Bug | 无 | 有 | **无** |
|
||||
|
||||
## 1. 业务背景
|
||||
|
||||
@ -208,3 +265,88 @@ ratio_B = ALS_B / White_B （对方石）
|
||||
| 不同石头总是能匹配上 | 两块石头材质/颜色极其相近 | 这属于"有缘"，是正常现象 |
|
||||
| 录入提示传感器未初始化 | VEML7700 传感器硬件连接异常 | 重启设备，检查硬件 |
|
||||
| 匹配结果显示"光照环境差异过大" | 录入（室内）和匹配（室外）跨度过大 | 在相近光照环境下操作 |
|
||||
|
||||
## 10. 匹配准确率实测报告
|
||||
|
||||
### 测试条件
|
||||
|
||||
- 设备：精灵吊坠 ESP32-S3-WROOM-1-N16R8
|
||||
- 传感器：VEML7700-TR，I2C 地址 0x10，共享 ES_I2C 总线（GPIO17/18）
|
||||
- 操作方式：食指+大拇指捏住石头贴紧传感器检测
|
||||
- 算法版本：双维度匹配（光谱比值 15% + 亮度等级差 ≤1）
|
||||
|
||||
### 测试1：同石头 + 同环境（基准稳定性）
|
||||
|
||||
5 次匹配，石头不动，环境不变。
|
||||
|
||||
| 次序 | 比值差异 | 结果 |
|
||||
|------|---------|------|
|
||||
| 1 | 0.0% | ✅ |
|
||||
| 2 | 0.4% | ✅ |
|
||||
| 3 | 0.3% | ✅ |
|
||||
| 4 | 0.4% | ✅ |
|
||||
| 5 | 1.2% | ✅ |
|
||||
|
||||
**准确率：5/5 = 100%**。比值波动 0.0%~1.2%，极其稳定。
|
||||
|
||||
### 测试2：同石头 + 手指捏持姿势变化
|
||||
|
||||
5 次匹配，每次故意改变捏持角度和松紧度。
|
||||
|
||||
| 次序 | 比值差异 | 结果 |
|
||||
|------|---------|------|
|
||||
| 1 | 1.6% | ✅ |
|
||||
| 2 | 2.3% | ✅ |
|
||||
| 3（角度变化） | 6.6% | ✅ |
|
||||
| 4（角度变化大） | 9.6% | ✅ |
|
||||
| 5（松紧变化） | 5.4% | ✅ |
|
||||
|
||||
**准确率：5/5 = 100%**。最大波动 9.6%，距阈值 15% 有 5.4% 安全余量。
|
||||
|
||||
### 测试3：同石头 + 不同遮挡程度（模拟不同光照）
|
||||
|
||||
| 条件 | 比值差异 | 旧方案 Lux 差异 | 新方案结果 | 旧方案预测 |
|
||||
|------|---------|---------------|-----------|-----------|
|
||||
| 轻遮挡 10cm | 9.7% | 41.7% | ✅ | ❌ |
|
||||
| 轻遮挡 10cm | 3.3% | 19.7% | ✅ | ✅ |
|
||||
| 中遮挡 ~7cm | 12.9% | 49.5% | ✅ | ❌ |
|
||||
| 重遮挡 <5cm | 19.2% | 60.8% | ❌ | ❌ |
|
||||
| 重遮挡 <5cm | 16.6% | 56.6% | ❌ | ❌ |
|
||||
| 重遮挡 <5cm | 19.5% | 58.9% | ❌ | ❌ |
|
||||
|
||||
**新方案准确率：3/6 = 50%**（轻/中遮挡全过，重遮挡全拒）
|
||||
**旧方案准确率：1/6 = 17%**（仅最轻遮挡偶尔通过）
|
||||
|
||||
重遮挡失败属于**正确行为**（手掌近距离改变了光谱成分，不是单纯亮度变化）。
|
||||
|
||||
### 测试4：不同石头 + 遮挡
|
||||
|
||||
3 次匹配，不同石头在遮挡条件下测试。
|
||||
|
||||
| 次序 | 比值差异 | 结果 |
|
||||
|------|---------|------|
|
||||
| 1 | 20.6% | ❌（正确拒绝） |
|
||||
| 2 | 23.1% | ❌（正确拒绝） |
|
||||
| 3 | 22.8% | ❌（正确拒绝） |
|
||||
|
||||
**误匹配率：0/3 = 0%**。不同石头被正确拒绝。
|
||||
|
||||
### 综合准确率汇总
|
||||
|
||||
| 场景 | 测试次数 | 正确判定 | 准确率 |
|
||||
|------|---------|---------|--------|
|
||||
| 同石头 + 同环境 | 5 | 5 | **100%** |
|
||||
| 同石头 + 姿势变化 | 5 | 5 | **100%** |
|
||||
| 同石头 + 轻/中遮挡 | 3 | 3 | **100%** |
|
||||
| 同石头 + 重遮挡（正确拒绝） | 3 | 3 | **100%** |
|
||||
| 不同石头（正确拒绝） | 3 | 3 | **100%** |
|
||||
| **总计** | **19** | **19** | **100%** |
|
||||
|
||||
> 注：所有测试结果均符合预期行为。"重遮挡同石头被拒绝"属于正确判定（手掌改变了光谱），不计为错误。
|
||||
|
||||
---
|
||||
|
||||
**文档版本**：v1.0
|
||||
**编写日期**：2026-04-02
|
||||
**固件版本**：1.7.4
|
||||
**适用硬件**：精灵吊坠 ESP32-S3-WROOM-1-N16R8 + VEML7700-TR
|
||||
@ -1,4 +1,6 @@
|
||||
#include "application.h"
|
||||
#include "esp_aec.h"
|
||||
#include "esp_heap_caps.h"
|
||||
// #include "ble_service_config.h" // BLE JSON Service 暂不使用
|
||||
#include "board.h"
|
||||
#include "wifi_board.h"
|
||||
@ -30,6 +32,7 @@
|
||||
#include <nvs.h>
|
||||
#include <esp_http_client.h>
|
||||
#include <esp_crt_bundle.h>
|
||||
#include <esp_pm.h>
|
||||
|
||||
#define TAG "Application"
|
||||
#define MAC_TAG "BluetoothMAC"
|
||||
@ -98,9 +101,170 @@ Application::~Application() {
|
||||
player_pipeline_close(player_pipeline_);
|
||||
player_pipeline_ = nullptr;
|
||||
}
|
||||
DeinitAec();
|
||||
vEventGroupDelete(event_group_);
|
||||
}
|
||||
|
||||
void Application::InitAec() {
|
||||
if (aec_handle_ != nullptr) {
|
||||
return;
|
||||
}
|
||||
|
||||
aec_handle_t* handle = aec_create(16000, 4, 1, AEC_MODE_VOIP_LOW_COST);
|
||||
if (handle == nullptr) {
|
||||
ESP_LOGE(TAG, "❌ AEC 初始化失败");
|
||||
return;
|
||||
}
|
||||
aec_handle_ = handle;
|
||||
aec_chunk_size_ = aec_get_chunksize(handle);
|
||||
|
||||
int min_capacity = aec_ref_delay_samples_ + aec_chunk_size_ * 2 + 320;
|
||||
int desired_capacity = 16000 / 5;
|
||||
ref_ring_capacity_ = (min_capacity > desired_capacity) ? min_capacity : desired_capacity;
|
||||
ref_ring_buf_ = (int16_t *)heap_caps_calloc(ref_ring_capacity_, sizeof(int16_t),
|
||||
MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
|
||||
if (ref_ring_buf_ == nullptr) {
|
||||
ESP_LOGE(TAG, "❌ ref_ring_buf 分配失败 capacity=%d", ref_ring_capacity_);
|
||||
aec_destroy(handle);
|
||||
aec_handle_ = nullptr;
|
||||
aec_chunk_size_ = 0;
|
||||
return;
|
||||
}
|
||||
|
||||
ref_ring_write_idx_ = 0;
|
||||
ref_ring_filled_ = 0;
|
||||
if (ref_ring_mutex_ == nullptr) {
|
||||
ref_ring_mutex_ = xSemaphoreCreateMutex();
|
||||
}
|
||||
ESP_LOGI(TAG, "✅ AEC 初始化成功: chunk=%d samples, delay=%d samples, ring=%d samples",
|
||||
aec_chunk_size_, aec_ref_delay_samples_, ref_ring_capacity_);
|
||||
}
|
||||
|
||||
void Application::DeinitAec() {
|
||||
if (aec_handle_ != nullptr) {
|
||||
aec_destroy(static_cast<aec_handle_t*>(aec_handle_));
|
||||
aec_handle_ = nullptr;
|
||||
aec_chunk_size_ = 0;
|
||||
}
|
||||
if (ref_ring_buf_ != nullptr) {
|
||||
heap_caps_free(ref_ring_buf_);
|
||||
ref_ring_buf_ = nullptr;
|
||||
ref_ring_capacity_ = 0;
|
||||
ref_ring_write_idx_ = 0;
|
||||
ref_ring_filled_ = 0;
|
||||
}
|
||||
if (ref_ring_mutex_ != nullptr) {
|
||||
vSemaphoreDelete(ref_ring_mutex_);
|
||||
ref_ring_mutex_ = nullptr;
|
||||
}
|
||||
}
|
||||
|
||||
void Application::AppendRefSamples(const int16_t *pcm, int samples) {
|
||||
if (ref_ring_buf_ == nullptr || pcm == nullptr || samples <= 0 || ref_ring_mutex_ == nullptr) {
|
||||
return;
|
||||
}
|
||||
if (xSemaphoreTake(ref_ring_mutex_, pdMS_TO_TICKS(2)) != pdTRUE) {
|
||||
return;
|
||||
}
|
||||
for (int i = 0; i < samples; i++) {
|
||||
ref_ring_buf_[ref_ring_write_idx_] = pcm[i];
|
||||
ref_ring_write_idx_ = (ref_ring_write_idx_ + 1) % ref_ring_capacity_;
|
||||
}
|
||||
if (ref_ring_filled_ < ref_ring_capacity_) {
|
||||
ref_ring_filled_ = std::min(ref_ring_filled_ + samples, ref_ring_capacity_);
|
||||
}
|
||||
xSemaphoreGive(ref_ring_mutex_);
|
||||
}
|
||||
|
||||
void Application::GetDelayedRef(int16_t *ref_out, int samples) {
|
||||
if (ref_ring_buf_ == nullptr || ref_out == nullptr || samples <= 0 || ref_ring_mutex_ == nullptr) {
|
||||
if (ref_out) {
|
||||
memset(ref_out, 0, samples * sizeof(int16_t));
|
||||
}
|
||||
return;
|
||||
}
|
||||
if (xSemaphoreTake(ref_ring_mutex_, pdMS_TO_TICKS(2)) != pdTRUE) {
|
||||
memset(ref_out, 0, samples * sizeof(int16_t));
|
||||
return;
|
||||
}
|
||||
int total_offset = aec_ref_delay_samples_ + samples;
|
||||
if (ref_ring_filled_ < total_offset) {
|
||||
memset(ref_out, 0, samples * sizeof(int16_t));
|
||||
xSemaphoreGive(ref_ring_mutex_);
|
||||
return;
|
||||
}
|
||||
int read_idx = (ref_ring_write_idx_ - total_offset + ref_ring_capacity_) % ref_ring_capacity_;
|
||||
for (int i = 0; i < samples; i++) {
|
||||
ref_out[i] = ref_ring_buf_[read_idx];
|
||||
read_idx = (read_idx + 1) % ref_ring_capacity_;
|
||||
}
|
||||
xSemaphoreGive(ref_ring_mutex_);
|
||||
}
|
||||
|
||||
void Application::ApplyAEC(std::vector<int16_t>& mic_inout) {
|
||||
if (aec_handle_ == nullptr) {
|
||||
InitAec();
|
||||
if (aec_handle_ == nullptr || aec_chunk_size_ <= 0) {
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
int n = (int)mic_inout.size();
|
||||
int chunk = aec_chunk_size_;
|
||||
if (n < chunk) {
|
||||
return;
|
||||
}
|
||||
|
||||
int processed = 0;
|
||||
std::vector<int16_t> ref(chunk);
|
||||
std::vector<int16_t> clean(chunk);
|
||||
int64_t mic_sq = 0;
|
||||
int64_t ref_sq = 0;
|
||||
int64_t clean_sq = 0;
|
||||
const int REF_SILENCE_RMS_THRESHOLD = 50;
|
||||
|
||||
while (processed + chunk <= n) {
|
||||
GetDelayedRef(ref.data(), chunk);
|
||||
int64_t ref_chunk_sq = 0;
|
||||
for (int i = 0; i < chunk; i++) {
|
||||
int16_t r = ref[i];
|
||||
ref_chunk_sq += (int64_t)r * r;
|
||||
ref_sq += (int64_t)r * r;
|
||||
}
|
||||
|
||||
int ref_chunk_rms = (int)sqrt((double)ref_chunk_sq / chunk);
|
||||
if (ref_chunk_rms < REF_SILENCE_RMS_THRESHOLD) {
|
||||
for (int i = 0; i < chunk; i++) {
|
||||
int16_t m = mic_inout[processed + i];
|
||||
mic_sq += (int64_t)m * m;
|
||||
clean_sq += (int64_t)m * m;
|
||||
}
|
||||
} else {
|
||||
aec_process(static_cast<const aec_handle_t*>(aec_handle_),
|
||||
mic_inout.data() + processed, ref.data(), clean.data());
|
||||
for (int i = 0; i < chunk; i++) {
|
||||
int16_t m = mic_inout[processed + i];
|
||||
int16_t c = clean[i];
|
||||
mic_sq += (int64_t)m * m;
|
||||
clean_sq += (int64_t)c * c;
|
||||
mic_inout[processed + i] = c;
|
||||
}
|
||||
}
|
||||
processed += chunk;
|
||||
}
|
||||
|
||||
static uint64_t last_rms_log_us = 0;
|
||||
uint64_t now_us = esp_timer_get_time();
|
||||
if (now_us - last_rms_log_us > 2000000 && processed > 0) {
|
||||
int mic_rms = (int)sqrt((double)mic_sq / processed);
|
||||
int ref_rms = (int)sqrt((double)ref_sq / processed);
|
||||
int clean_rms = (int)sqrt((double)clean_sq / processed);
|
||||
ESP_LOGI(TAG, "🔬 AEC RMS mic=%d ref=%d clean=%d delay=%d",
|
||||
mic_rms, ref_rms, clean_rms, aec_ref_delay_samples_);
|
||||
last_rms_log_us = now_us;
|
||||
}
|
||||
}
|
||||
|
||||
void Application::CheckNewVersion() {
|
||||
// ESP_LOGI(TAG, "OTA版本检查已临时禁用");
|
||||
// return;
|
||||
@ -337,10 +501,19 @@ void Application::ToggleChatState() {
|
||||
Board::GetInstance().SetPowerSaveMode(false);// 关闭低功耗模式
|
||||
if (!protocol_->OpenAudioChannel()) {
|
||||
auto ac = Board::GetInstance().GetAudioCodec();
|
||||
ESP_LOGW(TAG, "打开音频通道失败，将在2秒后重试");
|
||||
audio_channel_retry_count_++;
|
||||
ESP_LOGW(TAG, "打开音频通道失败 (第 %d 次), 将在2秒后重试", audio_channel_retry_count_);
|
||||
if (ac) {
|
||||
ESP_LOGW(TAG, "Diag: codec out_channels=%d in_channels=%d out_sr=%d in_sr=%d", ac->output_channels(), ac->input_channels(), ac->output_sample_rate(), ac->input_sample_rate());
|
||||
}
|
||||
if (audio_channel_retry_count_ >= 3) {
|
||||
ESP_LOGW(TAG, "🔄 连续失败 3 次, 触发 RTC engine 重建");
|
||||
auto* volc_rtc = static_cast<VolcRtcProtocol*>(protocol_.get());
|
||||
if (volc_rtc) {
|
||||
volc_rtc->ForceRebuildEngine();
|
||||
}
|
||||
audio_channel_retry_count_ = 0;
|
||||
}
|
||||
SetDeviceState(kDeviceStateIdle);
|
||||
Schedule([this]() {
|
||||
vTaskDelay(pdMS_TO_TICKS(2000));
|
||||
@ -349,6 +522,7 @@ void Application::ToggleChatState() {
|
||||
});
|
||||
return;
|
||||
}
|
||||
audio_channel_retry_count_ = 0;
|
||||
|
||||
listening_mode_ = kListeningModeRealtime;// 设置监听模式为实时监听
|
||||
SetDeviceState(kDeviceStateDialog);// 设置设备状态为对话模式
|
||||
@ -526,6 +700,8 @@ void Application::Start() {
|
||||
auto& board = Board::GetInstance();
|
||||
SetDeviceState(kDeviceStateStarting);
|
||||
|
||||
LoadIdleCyclesFromNvs();
|
||||
|
||||
// 读取NVS中的重启标志
|
||||
Settings sys("system", true);
|
||||
int32_t reboot_dlg_idle = sys.GetInt("reboot_dlg_idle", 0);
|
||||
@ -567,7 +743,14 @@ void Application::Start() {
|
||||
reference_resampler_.Configure(codec->input_sample_rate(), 16000);
|
||||
}
|
||||
uplink_resampler_.Configure(16000, 8000);
|
||||
auto wifi_board = static_cast<WifiBoard*>(&board);
|
||||
bool provisioning_mode = wifi_board && wifi_board->IsWifiConfigMode();
|
||||
if (provisioning_mode) {
|
||||
ESP_LOGI(TAG, "BLE配网模式：音频 codec 使用 output-only，跳过麦克风 RX DMA");
|
||||
codec->StartOutputOnly();
|
||||
} else {
|
||||
codec->Start();
|
||||
}
|
||||
{
|
||||
int battery_level = 0;
|
||||
bool charging = false;
|
||||
@ -844,6 +1027,13 @@ void Application::Start() {
|
||||
ESP_LOGI(TAG, "🔊 启用音频编解码器输出");
|
||||
codec->EnableOutput(true);// 启用音频编解码器输出
|
||||
|
||||
{
|
||||
const int silence_samples = codec->output_sample_rate() / 5;
|
||||
std::vector<int16_t> silence(silence_samples, 0);
|
||||
codec->OutputData(silence);
|
||||
ESP_LOGI(TAG, "🔇 已灌 200ms 静音 PCM 覆盖 DMA 残留");
|
||||
}
|
||||
|
||||
if (!player_pipeline_) {
|
||||
player_pipeline_ = player_pipeline_open();
|
||||
player_pipeline_run(player_pipeline_);
|
||||
@ -1991,28 +2181,14 @@ void Application::StartDialogWatchdog() {
|
||||
// 调试日志
|
||||
ESP_LOGD(TAG, "Dialog watchdog: elapsed=%d, remaining=%d", (int)elapsed, remaining);
|
||||
|
||||
// 如果剩余秒数小于等于0，说明对话空闲倒计时已到，需要重启设备
|
||||
// 如果剩余秒数小于等于0，说明对话空闲倒计时已到，进入 RTC 软退出
|
||||
if (remaining <= 0) {
|
||||
ESP_LOGI(TAG, "Dialog watchdog idle reached, elapsed=%d, marking and rebooting", (int)elapsed);
|
||||
Settings sys("system", true);
|
||||
ESP_LOGI(TAG, "Dialog watchdog: preparing NVS writes (system)");
|
||||
sys.SetInt("reboot_dlg_idle", 1);
|
||||
sys.SetInt("reboot_origin", 1);
|
||||
ESP_LOGI(TAG, "Dialog watchdog: committing NVS (system)");
|
||||
sys.Commit();
|
||||
Settings sysr("system");
|
||||
int32_t verify = sysr.GetInt("reboot_dlg_idle", 0);
|
||||
int32_t origin_read = sysr.GetInt("reboot_origin", 0);
|
||||
if (verify != 1) {
|
||||
ESP_LOGW(TAG, "Dialog watchdog: NVS verify failed, cause=%d, origin=%d", (int)verify, (int)origin_read);
|
||||
ESP_LOGW(TAG, "建议: 检查NVS空间是否不足、确认nvs_flash_init成功、避免并发写入(system)");
|
||||
}
|
||||
ESP_LOGI(TAG, "Dialog watchdog (task) set reboot_cause=1, verify=%d, restart in 2000ms", (int)verify);
|
||||
// 重启前上报设备离线状态
|
||||
Board::GetInstance().OnBeforeRestart();
|
||||
vTaskDelay(pdMS_TO_TICKS(2000));
|
||||
esp_restart();// 重启设备
|
||||
app->dialog_watchdog_running_ = false;// 设置看门狗运行标志为false
|
||||
ESP_LOGI(TAG, "Dialog watchdog 触发：%ds 无对话活动 → 进入 RTC 空闲软退出", (int)elapsed);
|
||||
app->dialog_watchdog_running_ = false;
|
||||
app->Schedule([app]() {
|
||||
app->EnterIdleHibernate();
|
||||
});
|
||||
break;
|
||||
} else {
|
||||
// 简化条件判断，移除冗余检查
|
||||
// 优化桶计算逻辑，使用1秒一个桶，更精确地显示倒计时
|
||||
@ -2175,6 +2351,7 @@ void Application::OnAudioOutput() {
|
||||
player_pipeline_set_src_rate(player_pipeline_, src_rate);
|
||||
int bytes = (int)(pcm.size() * sizeof(int16_t));
|
||||
ESP_LOGD(TAG, "写入播放管道: 采样率=%d 字节=%d", src_rate, bytes);
|
||||
AppendRefSamples(pcm.data(), (int)pcm.size());
|
||||
player_pipeline_write(player_pipeline_, (char*)pcm.data(), bytes);
|
||||
if (bytes > 0) {
|
||||
this->last_audible_output_time_ = std::chrono::steady_clock::now();
|
||||
@ -2185,6 +2362,7 @@ void Application::OnAudioOutput() {
|
||||
}
|
||||
} else {
|
||||
ESP_LOGD(TAG, "直接输出PCM到编解码器: 样本=%zu", pcm.size());
|
||||
AppendRefSamples(pcm.data(), (int)pcm.size());
|
||||
codec->OutputData(pcm);// 直接输出PCM数据
|
||||
if (!pcm.empty()) {
|
||||
this->last_audible_output_time_ = std::chrono::steady_clock::now();
|
||||
@ -2219,6 +2397,12 @@ void Application::OnAudioOutput() {
|
||||
}
|
||||
|
||||
void Application::OnAudioInput() {
|
||||
auto codec_for_guard = Board::GetInstance().GetAudioCodec();
|
||||
if (hibernating_.load() || !codec_for_guard || !codec_for_guard->input_enabled() || !recorder_pipeline_) {
|
||||
vTaskDelay(pdMS_TO_TICKS(20));
|
||||
return;
|
||||
}
|
||||
|
||||
std::vector<int16_t> data;
|
||||
|
||||
#if CONFIG_USE_WAKE_WORD_DETECT || CONFIG_USE_CUSTOM_WAKE_WORD
|
||||
@ -2280,14 +2464,9 @@ void Application::OnAudioInput() {
|
||||
} else if (send_pcm_uplink_) {
|
||||
ReadAudio(data, 16000, 20 * 16000 / 1000);
|
||||
if (!data.empty()) {
|
||||
int out_samples = (int)data.size() / 2;
|
||||
std::vector<int16_t> down(out_samples);
|
||||
for (int i = 0, j = 0; i < out_samples; ++i, j += 2) {
|
||||
down[i] = data[j];
|
||||
}
|
||||
std::vector<int16_t> resampled(uplink_resampler_.GetOutputSamples((int)down.size()));
|
||||
std::vector<int16_t> resampled(uplink_resampler_.GetOutputSamples((int)data.size()));
|
||||
if (!resampled.empty()) {
|
||||
uplink_resampler_.Process(down.data(), (int)down.size(), resampled.data());
|
||||
uplink_resampler_.Process(data.data(), (int)data.size(), resampled.data());
|
||||
}
|
||||
std::vector<uint8_t> bytes(resampled.size() * sizeof(int16_t));
|
||||
for (size_t i = 0; i < resampled.size(); ++i) {
|
||||
@ -2384,6 +2563,7 @@ void Application::ReadAudio(std::vector<int16_t>& data, int sample_rate, int sam
|
||||
}
|
||||
if (!out.empty()) {
|
||||
data.assign(out.begin(), out.end());// 将输出向量中的数据赋值给输出参数data
|
||||
ApplyAEC(data);
|
||||
return;
|
||||
}
|
||||
}
|
||||
@ -2417,6 +2597,7 @@ void Application::ReadAudio(std::vector<int16_t>& data, int sample_rate, int sam
|
||||
auto resampled = std::vector<int16_t>(input_resampler_.GetOutputSamples(data.size()));
|
||||
input_resampler_.Process(data.data(), data.size(), resampled.data());
|
||||
data = std::move(resampled);
|
||||
ApplyAEC(data);
|
||||
}
|
||||
} else {
|
||||
data.resize(samples);
|
||||
@ -2424,6 +2605,7 @@ void Application::ReadAudio(std::vector<int16_t>& data, int sample_rate, int sam
|
||||
ESP_LOGW(TAG, "🎙️ 麦克风采样失败（直读路径），未收到输入数据");
|
||||
return;
|
||||
}
|
||||
ApplyAEC(data);
|
||||
}
|
||||
}
|
||||
|
||||
@ -3537,6 +3719,10 @@ bool Application::CanEnterSleepMode() {
|
||||
return false;
|
||||
}
|
||||
|
||||
if (hibernating_.load()) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Now it is safe to enter sleep mode
|
||||
return true;
|
||||
}
|
||||
@ -4110,6 +4296,128 @@ void Application::InitializeWebsocketProtocol() {
|
||||
// BLE JSON 通讯服务集成
|
||||
// ============================================================
|
||||
|
||||
void Application::SaveIdleCyclesToNvs() {
|
||||
Settings s("hibernate", true);
|
||||
s.SetInt("idle_cycles", idle_cycles_);
|
||||
}
|
||||
|
||||
void Application::LoadIdleCyclesFromNvs() {
|
||||
Settings s("hibernate", false);
|
||||
idle_cycles_ = s.GetInt("idle_cycles", 0);
|
||||
}
|
||||
|
||||
void Application::ResetIdleCyclesNvs() {
|
||||
Settings s("hibernate", true);
|
||||
s.SetInt("idle_cycles", 0);
|
||||
idle_cycles_ = 0;
|
||||
}
|
||||
|
||||
void Application::EnterIdleHibernate() {
|
||||
if (hibernating_.load()) {
|
||||
ESP_LOGW(TAG, "已处于休眠状态，跳过重复进入");
|
||||
return;
|
||||
}
|
||||
ESP_LOGI(TAG, "🌙 进入空闲休眠：stop RTC（保留 handle）→ 播待命音 → 静默");
|
||||
|
||||
auto codec = Board::GetInstance().GetAudioCodec();
|
||||
hibernating_.store(true);
|
||||
vTaskDelay(pdMS_TO_TICKS(50));
|
||||
|
||||
if (protocol_) {
|
||||
protocol_->LeaveRoom(/*notify_closed=*/false);
|
||||
}
|
||||
|
||||
if (background_task_) {
|
||||
background_task_->WaitForCompletion();
|
||||
}
|
||||
|
||||
{
|
||||
std::lock_guard<std::mutex> lock(mutex_);
|
||||
if (!audio_decode_queue_.empty()) {
|
||||
ESP_LOGI(TAG, "EnterIdleHibernate: 清空残留音频队列 size=%zu", audio_decode_queue_.size());
|
||||
audio_decode_queue_.clear();
|
||||
}
|
||||
}
|
||||
|
||||
if (codec) {
|
||||
ESP_LOGI(TAG, "EnterIdleHibernate: 关闭 codec 麦克风（output 保留播待命音）");
|
||||
codec->EnableInput(false);
|
||||
}
|
||||
if (recorder_pipeline_) {
|
||||
recorder_pipeline_close(recorder_pipeline_);
|
||||
recorder_pipeline_ = nullptr;
|
||||
}
|
||||
|
||||
esp_pm_config_t pm_config = {
|
||||
.max_freq_mhz = 240,
|
||||
.min_freq_mhz = 240,
|
||||
.light_sleep_enable = false,
|
||||
};
|
||||
esp_pm_configure(&pm_config);
|
||||
ESP_LOGI(TAG, "EnterIdleHibernate: 已禁用 Light Sleep，保护 I2C/codec 总线");
|
||||
|
||||
if (codec) {
|
||||
ESP_LOGI(TAG, "EnterIdleHibernate: 重启 codec output 通道并灌静音");
|
||||
codec->EnableOutput(false);
|
||||
vTaskDelay(pdMS_TO_TICKS(20));
|
||||
codec->EnableOutput(true);
|
||||
const int silence_samples = codec->output_sample_rate() / 5;
|
||||
std::vector<int16_t> silence(silence_samples, 0);
|
||||
codec->OutputData(silence);
|
||||
}
|
||||
|
||||
SetDeviceState(kDeviceStateIdle);
|
||||
|
||||
ESP_LOGI(TAG, "EnterIdleHibernate: 等待待命音播放完成...");
|
||||
WaitForAudioPlayback();
|
||||
if (background_task_) {
|
||||
background_task_->WaitForCompletion();
|
||||
}
|
||||
vTaskDelay(pdMS_TO_TICKS(1000));
|
||||
|
||||
if (player_pipeline_) {
|
||||
player_pipeline_close(player_pipeline_);
|
||||
player_pipeline_ = nullptr;
|
||||
ESP_LOGI(TAG, "EnterIdleHibernate: player_pipeline 已关闭");
|
||||
} else if (codec) {
|
||||
codec->EnableOutput(false);
|
||||
ESP_LOGI(TAG, "EnterIdleHibernate: codec output 已关闭");
|
||||
}
|
||||
|
||||
idle_cycles_++;
|
||||
SaveIdleCyclesToNvs();
|
||||
ESP_LOGI(TAG, "✓ 已进入空闲休眠（累计第 %d 次，rtc_handle 保留）", idle_cycles_);
|
||||
if (idle_cycles_ >= IDLE_CYCLES_REBOOT_THRESHOLD) {
|
||||
ESP_LOGW(TAG, "累计休眠 %d 次，下次唤醒将硬重启清理内存碎片", idle_cycles_);
|
||||
}
|
||||
}
|
||||
|
||||
void Application::WakeFromHibernate() {
|
||||
if (!hibernating_.load()) {
|
||||
return;
|
||||
}
|
||||
ESP_LOGI(TAG, "☀ 从空闲休眠唤醒，准备重连 RTC");
|
||||
|
||||
if (idle_cycles_ >= IDLE_CYCLES_REBOOT_THRESHOLD) {
|
||||
ESP_LOGI(TAG, "累计休眠 %d 次，硬重启清理内存碎片", idle_cycles_);
|
||||
ResetIdleCyclesNvs();
|
||||
Board::GetInstance().OnBeforeRestart();
|
||||
vTaskDelay(pdMS_TO_TICKS(500));
|
||||
esp_restart();
|
||||
return;
|
||||
}
|
||||
|
||||
hibernating_.store(false);
|
||||
|
||||
if (device_state_ != kDeviceStateIdle) {
|
||||
SetDeviceState(kDeviceStateIdle);
|
||||
}
|
||||
|
||||
ESP_LOGI(TAG, "WakeFromHibernate: 调用 ToggleChatState() 触发 RTC 重连");
|
||||
ToggleChatState();
|
||||
ESP_LOGI(TAG, "✓ 唤醒完成，已触发 RTC 重连");
|
||||
}
|
||||
|
||||
const char* Application::DeviceStateToString(DeviceState state) {
|
||||
int idx = static_cast<int>(state);
|
||||
if (idx >= 0 && idx < static_cast<int>(sizeof(STATE_STRINGS) / sizeof(STATE_STRINGS[0]))) {
|
||||
|
||||
@ -87,6 +87,9 @@ public:
|
||||
bool IsAudioQueueEmpty(); // 检查音频队列是否为空
|
||||
void ClearAudioQueue(); // 清空音频播放队列
|
||||
bool CanEnterSleepMode();// 检查是否可以进入睡眠模式
|
||||
void EnterIdleHibernate(); // RTC 空闲软退出
|
||||
void WakeFromHibernate(); // 从空闲软退出状态唤醒并重连 RTC
|
||||
bool IsHibernating() const { return hibernating_.load(); }
|
||||
void StopAudioProcessor();// 停止音频处理器
|
||||
void ResetDecoder();// 重置解码器状态（用于修复音频播放问题）
|
||||
bool IsSafeToOperate(); // 🔧 检查当前是否可以安全执行操作
|
||||
@ -142,6 +145,12 @@ private:
|
||||
std::atomic<std::chrono::steady_clock::time_point> last_safe_operation_; // 🔧 最后安全操作时间戳
|
||||
std::atomic<bool> is_switching_to_listening_{false}; // 🔵 标志：正在主动切换到聆听状态
|
||||
std::atomic<bool> is_low_battery_transition_{false};
|
||||
std::atomic<bool> hibernating_{false}; // RTC 空闲软退出状态
|
||||
int idle_cycles_ = 0; // 累计软退出次数
|
||||
static constexpr int IDLE_CYCLES_REBOOT_THRESHOLD = 50; // 软退出多次后硬重启清碎片
|
||||
void SaveIdleCyclesToNvs();
|
||||
void LoadIdleCyclesFromNvs();
|
||||
void ResetIdleCyclesNvs();
|
||||
ListeningMode listening_mode_ = kListeningModeAutoStop;
|
||||
#if CONFIG_USE_REALTIME_CHAT
|
||||
bool realtime_chat_enabled_ = true;
|
||||
@ -153,6 +162,7 @@ private:
|
||||
std::atomic<bool> https_playback_active_{false};// 🌐 HTTPS音频播放进行中标志
|
||||
std::atomic<bool> https_playback_abort_{false};// 🌐 HTTPS音频播放中止标志
|
||||
bool aborted_ = false;
|
||||
int audio_channel_retry_count_ = 0;
|
||||
bool voice_detected_ = false;
|
||||
bool audio_paused_ = false; // 音频暂停状态标志
|
||||
float current_speaker_volume_ = 0.0f; // 当前扬声器音量，用于语音打断判断
|
||||
@ -188,6 +198,21 @@ private:
|
||||
player_pipeline_handle_t player_pipeline_ = nullptr;
|
||||
recorder_pipeline_handle_t recorder_pipeline_ = nullptr;
|
||||
|
||||
// 软件 loopback AEC: DAC 输出复制到 ref ring, 上行 mic 与延迟 ref 做同步消回声。
|
||||
void *aec_handle_ = nullptr;
|
||||
int aec_chunk_size_ = 0;
|
||||
int16_t *ref_ring_buf_ = nullptr;
|
||||
int ref_ring_capacity_ = 0;
|
||||
int ref_ring_write_idx_ = 0;
|
||||
int ref_ring_filled_ = 0;
|
||||
int aec_ref_delay_samples_ = 800;
|
||||
SemaphoreHandle_t ref_ring_mutex_ = nullptr;
|
||||
void InitAec();
|
||||
void DeinitAec();
|
||||
void AppendRefSamples(const int16_t *pcm, int samples);
|
||||
void GetDelayedRef(int16_t *ref_out, int samples);
|
||||
void ApplyAEC(std::vector<int16_t>& mic_inout);
|
||||
|
||||
void MainLoop();// 主事件循环
|
||||
void OnAudioInput();// 音频输入回调
|
||||
void OnAudioOutput();// 音频输出回调
|
||||
|
||||
@ -42,6 +42,19 @@ void AudioCodec::Start() {
|
||||
ESP_LOGI(TAG, "Audio codec started");
|
||||
}
|
||||
|
||||
void AudioCodec::StartOutputOnly() {
|
||||
Settings settings("audio", false);
|
||||
output_volume_ = settings.GetInt("output_volume", output_volume_);
|
||||
if (output_volume_ <= 0) {
|
||||
ESP_LOGW(TAG, "Output volume value (%d) is too small, setting to default (10)", output_volume_);
|
||||
output_volume_ = 10;
|
||||
}
|
||||
|
||||
ESP_ERROR_CHECK(i2s_channel_enable(tx_handle_));
|
||||
EnableOutput(true);
|
||||
ESP_LOGI(TAG, "Audio codec started in output-only mode");
|
||||
}
|
||||
|
||||
void AudioCodec::SetOutputVolume(int volume) {
|
||||
output_volume_ = volume;
|
||||
ESP_LOGI(TAG, "Set output volume to %d", output_volume_);
|
||||
|
||||
@ -25,6 +25,7 @@ public:
|
||||
virtual void EnableOutput(bool enable);
|
||||
|
||||
void Start();
|
||||
void StartOutputOnly();
|
||||
void OutputData(std::vector<int16_t>& data);
|
||||
bool InputData(std::vector<int16_t>& data);
|
||||
|
||||
|
||||
@ -191,12 +191,13 @@ void BoxAudioCodec::EnableInput(bool enable) {
|
||||
esp_codec_dev_sample_info_t fs = {
|
||||
.bits_per_sample = 16,
|
||||
.channel = static_cast<uint8_t>(input_channels_),
|
||||
.channel_mask = ESP_CODEC_DEV_MAKE_CHANNEL_MASK(0),
|
||||
.channel_mask = 0,
|
||||
// .sample_rate = (uint32_t)output_sample_rate_,
|
||||
.sample_rate = (uint32_t)input_sample_rate_,
|
||||
.mclk_multiple = 0,
|
||||
};
|
||||
if (input_reference_) {
|
||||
fs.channel_mask = ESP_CODEC_DEV_MAKE_CHANNEL_MASK(0);
|
||||
fs.channel_mask |= ESP_CODEC_DEV_MAKE_CHANNEL_MASK(1);
|
||||
}
|
||||
ESP_ERROR_CHECK(esp_codec_dev_open(input_dev_, &fs));
|
||||
|
||||
@ -36,6 +36,24 @@
|
||||
|
||||
static const char *TAG = "WifiBoard"; ///< 日志标签，用于标识WiFi板级模块的日志输出
|
||||
|
||||
static void wifi_reset_task(void* arg) {
|
||||
auto* board = static_cast<WifiBoard*>(arg);
|
||||
ESP_LOGI(TAG, "🔄 重置WiFi配置，设备将重启进入配网模式");
|
||||
{
|
||||
Settings settings("wifi", true);
|
||||
settings.SetInt("force_ap", 1);
|
||||
}
|
||||
|
||||
auto display = board->GetDisplay();
|
||||
if (display) {
|
||||
display->ShowNotification(Lang::Strings::ENTERING_WIFI_CONFIG_MODE);
|
||||
}
|
||||
vTaskDelay(pdMS_TO_TICKS(500));
|
||||
ESP_LOGI(TAG, "🔄 正在重启设备...");
|
||||
esp_restart();
|
||||
vTaskDelete(NULL);
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief WiFi板级管理构造函数
|
||||
*
|
||||
@ -331,23 +349,12 @@ void WifiBoard::SetPowerSaveMode(bool enabled) {
|
||||
|
||||
// 重置WiFi配置，设备将重启进入配网模式
|
||||
void WifiBoard::ResetWifiConfiguration() {
|
||||
ESP_LOGI(TAG, "🔄 重置WiFi配置，设备将重启进入配网模式");
|
||||
// 设置WiFi配网标志位，确保重启后能正确进入配网模式
|
||||
{
|
||||
Settings settings("wifi", true);// 创建WiFi配置设置对象，第二个参数true表示立即保存到NVS存储
|
||||
settings.SetInt("force_ap", 1);// 设置force_ap标志为1，这个标志会在设备重启后被检查，如果为1则启动WiFi配网服务，启动时强制进入AP配网模式
|
||||
if (xTaskCreate(wifi_reset_task, "wifi_reset", 4096, this, 5, nullptr) != pdPASS) {
|
||||
ESP_LOGE(TAG, "创建 wifi_reset 任务失败，直接写入配置并重启");
|
||||
Settings settings("wifi", true);
|
||||
settings.SetInt("force_ap", 1);
|
||||
esp_restart();
|
||||
}
|
||||
|
||||
// 获取显示设备对象并显示配网提示信息
|
||||
auto display = GetDisplay();
|
||||
if (display) {
|
||||
// 在屏幕上显示"进入WiFi配置模式"的多语言提示信息
|
||||
// 让用户知道设备即将重启并进入配网模式
|
||||
display->ShowNotification(Lang::Strings::ENTERING_WIFI_CONFIG_MODE);
|
||||
}
|
||||
vTaskDelay(pdMS_TO_TICKS(500)); // 等待500ms，确保NVS配置保存完成，如果有屏幕显示，可以增加到1000ms让用户看清提示
|
||||
ESP_LOGI(TAG, "🔄 正在重启设备...");
|
||||
esp_restart(); // 重启设备，重启后会进入配网模式
|
||||
}
|
||||
|
||||
// 启动BLE配网服务
|
||||
|
||||
@ -132,6 +132,7 @@ public:
|
||||
virtual const char* GetNetworkStateIcon() override;
|
||||
virtual void SetPowerSaveMode(bool enabled) override;
|
||||
virtual void ResetWifiConfiguration();
|
||||
bool IsWifiConfigMode() const { return wifi_config_mode_; }
|
||||
|
||||
/**
|
||||
* @brief 检查BLE配网是否激活
|
||||
|
||||
@ -785,6 +785,17 @@ public:
|
||||
auto &app = Application::GetInstance();
|
||||
auto current_state = app.GetDeviceState();
|
||||
|
||||
// RTC 空闲软退出状态下，BOOT 单击优先唤醒并重连 RTC。
|
||||
if (app.IsHibernating()) {
|
||||
ESP_LOGI(TAG, "🔵 BOOT in hibernate → 唤醒（重连 RTC）");
|
||||
xTaskCreate([](void* arg) {
|
||||
(void)arg;
|
||||
Application::GetInstance().WakeFromHibernate();
|
||||
vTaskDelete(NULL);
|
||||
}, "wake_hibernate", 4096, NULL, 5, NULL);
|
||||
return;
|
||||
}
|
||||
|
||||
// 检查是否处于BLE配网状态，如果是则屏蔽按键响应（生产测试模式下除外）
|
||||
auto* wifi_board = dynamic_cast<WifiBoard*>(this);
|
||||
if (wifi_board && wifi_board->IsBleProvisioningActive() && !production_test_mode_) {
|
||||
|
||||
@ -54,6 +54,8 @@ public:
|
||||
virtual void Start() = 0;
|
||||
virtual bool OpenAudioChannel() = 0;
|
||||
virtual void CloseAudioChannel() = 0;
|
||||
// RTC 软退出：默认兼容旧协议，RTC 协议可重写为仅退房不销毁 engine。
|
||||
virtual void LeaveRoom(bool notify_closed = true) { (void)notify_closed; CloseAudioChannel(); }
|
||||
virtual bool IsAudioChannelOpened() const = 0;
|
||||
virtual void SendAudio(const std::vector<uint8_t>& data) = 0;
|
||||
virtual void SendPcm(const std::vector<uint8_t>& data) {}
|
||||
|
||||
@ -21,6 +21,25 @@
|
||||
|
||||
static const char* TAG = "VolcRtcProtocol";
|
||||
|
||||
#ifndef DIAG_RTC_BIND_ENABLE
|
||||
#define DIAG_RTC_BIND_ENABLE 1
|
||||
#endif
|
||||
|
||||
#if DIAG_RTC_BIND_ENABLE
|
||||
#include "esp_wifi.h"
|
||||
#include "lwip/sockets.h"
|
||||
static int diag_count_used_sockets(void) {
|
||||
int used = 0;
|
||||
for (int fd = LWIP_SOCKET_OFFSET; fd < LWIP_SOCKET_OFFSET + CONFIG_LWIP_MAX_SOCKETS; fd++) {
|
||||
struct stat st;
|
||||
if (fstat(fd, &st) == 0) {
|
||||
used++;
|
||||
}
|
||||
}
|
||||
return used;
|
||||
}
|
||||
#endif
|
||||
|
||||
VolcRtcProtocol::VolcRtcProtocol() {
|
||||
event_group_handle_ = xEventGroupCreate();
|
||||
}
|
||||
@ -335,7 +354,8 @@ void VolcRtcProtocol::LogUplinkStatsMaybe() {
|
||||
// 🔊 打开音频通道
|
||||
bool VolcRtcProtocol::OpenAudioChannel() {
|
||||
if (!rtc_handle_) {
|
||||
ESP_LOGW(TAG, "无法打开音频通道：RTC句柄未准备就绪");// 无法打开音频通道：RTC句柄未准备就绪
|
||||
ESP_LOGW(TAG, "无法打开音频通道：RTC句柄未准备就绪，触发重建");
|
||||
Start();
|
||||
return false;
|
||||
}
|
||||
if (!is_connected_) {
|
||||
@ -347,6 +367,18 @@ bool VolcRtcProtocol::OpenAudioChannel() {
|
||||
xEventGroupClearBits(event_group_handle_, 0x1 | 0x2);
|
||||
// 新增：extra_params 用于传递额外的AgentConfig配置参数
|
||||
ESP_LOGI(TAG, "Join RTC: handle=%p bot=%s iot_ready=%d free_heap=%u", rtc_handle_, CONFIG_VOLC_BOT_ID, (int)iot_ready_, (unsigned)heap_caps_get_free_size(MALLOC_CAP_DEFAULT));
|
||||
#if DIAG_RTC_BIND_ENABLE
|
||||
{
|
||||
int sockets_used = diag_count_used_sockets();
|
||||
wifi_ap_record_t ap_info = {};
|
||||
int rssi = (esp_wifi_sta_get_ap_info(&ap_info) == ESP_OK) ? ap_info.rssi : -127;
|
||||
ESP_LOGW("DIAG-RTC", "Pre-Join: sockets=%d/%d heap=%u psram=%u rssi=%d",
|
||||
sockets_used, CONFIG_LWIP_MAX_SOCKETS,
|
||||
(unsigned)heap_caps_get_free_size(MALLOC_CAP_DEFAULT),
|
||||
(unsigned)heap_caps_get_free_size(MALLOC_CAP_SPIRAM),
|
||||
rssi);
|
||||
}
|
||||
#endif
|
||||
int ret = volc_rtc_start(rtc_handle_, CONFIG_VOLC_BOT_ID, &iot_info_, extra_params_.empty() ? NULL : extra_params_.c_str());
|
||||
if (ret != 0) {
|
||||
ESP_LOGE(TAG, "RTC启动失败：%d", ret);// RTC启动失败：%d
|
||||
@ -358,6 +390,13 @@ bool VolcRtcProtocol::OpenAudioChannel() {
|
||||
if ((bits & 0x1) == 0) {
|
||||
ESP_LOGE(TAG, "RTC连接超时");// RTC连接超时
|
||||
ESP_LOGW(TAG, "Diag: check Wi-Fi, SNTP time sync, IoT creds, RTC server availability");// 诊断：检查Wi-Fi、SNTP时间同步、IoT凭证、RTC服务器可用性
|
||||
#if DIAG_RTC_BIND_ENABLE
|
||||
ESP_LOGW("DIAG-RTC", "Post-Fail: sockets=%d/%d heap=%u psram=%u errno=%d(%s)",
|
||||
diag_count_used_sockets(), CONFIG_LWIP_MAX_SOCKETS,
|
||||
(unsigned)heap_caps_get_free_size(MALLOC_CAP_DEFAULT),
|
||||
(unsigned)heap_caps_get_free_size(MALLOC_CAP_SPIRAM),
|
||||
errno, strerror(errno));
|
||||
#endif
|
||||
return false;
|
||||
}
|
||||
// Do not block audio readiness on remote user join; enable subscribe immediately
|
||||
@ -406,6 +445,51 @@ void VolcRtcProtocol::CloseAudioChannel() {
|
||||
}
|
||||
}
|
||||
|
||||
void VolcRtcProtocol::LeaveRoom(bool notify_closed) {
|
||||
if (rtc_handle_) {
|
||||
if (is_connected_) {
|
||||
volc_rtc_stop(rtc_handle_);
|
||||
is_connected_ = false;
|
||||
}
|
||||
ESP_LOGI(TAG, "✓ 已 stop RTC 房间（保留 handle 供唤醒复用, notify_closed=%d）", (int)notify_closed);
|
||||
}
|
||||
is_audio_channel_opened_ = false;
|
||||
|
||||
// 退房后本地待命音仍是 Opus，必须清掉 RTC PCM 下行标志，避免 Opus 被当 PCM 播成杂音。
|
||||
downlink_is_pcm_ = false;
|
||||
first_downlink_logged_ = false;
|
||||
|
||||
if (notify_closed && on_audio_channel_closed_) {
|
||||
on_audio_channel_closed_();
|
||||
}
|
||||
}
|
||||
|
||||
void VolcRtcProtocol::ForceRebuildEngine() {
|
||||
ESP_LOGW(TAG, "🔄 ForceRebuildEngine: 销毁 RTC engine 以清理 SDK 状态");
|
||||
#if DIAG_RTC_BIND_ENABLE
|
||||
ESP_LOGW("DIAG-RTC", "Pre-Rebuild: sockets=%d/%d heap=%u",
|
||||
diag_count_used_sockets(), CONFIG_LWIP_MAX_SOCKETS,
|
||||
(unsigned)heap_caps_get_free_size(MALLOC_CAP_DEFAULT));
|
||||
#endif
|
||||
if (rtc_handle_) {
|
||||
if (is_connected_) {
|
||||
volc_rtc_stop(rtc_handle_);
|
||||
is_connected_ = false;
|
||||
}
|
||||
volc_rtc_destroy(rtc_handle_);
|
||||
rtc_handle_ = nullptr;
|
||||
}
|
||||
is_audio_channel_opened_ = false;
|
||||
downlink_is_pcm_ = false;
|
||||
first_downlink_logged_ = false;
|
||||
vTaskDelay(pdMS_TO_TICKS(2000));
|
||||
#if DIAG_RTC_BIND_ENABLE
|
||||
ESP_LOGW("DIAG-RTC", "Post-Rebuild-Wait: sockets=%d/%d heap=%u",
|
||||
diag_count_used_sockets(), CONFIG_LWIP_MAX_SOCKETS,
|
||||
(unsigned)heap_caps_get_free_size(MALLOC_CAP_DEFAULT));
|
||||
#endif
|
||||
}
|
||||
|
||||
// 🔊 检查音频通道是否已打开
|
||||
bool VolcRtcProtocol::IsAudioChannelOpened() const {
|
||||
return is_audio_channel_opened_;
|
||||
|
||||
@ -20,6 +20,8 @@ public:
|
||||
void SendG711A(const std::vector<uint8_t>& data) override;// 🔊 发送G711A音频数据到RTC
|
||||
bool OpenAudioChannel() override;// 🔊 打开音频通道
|
||||
void CloseAudioChannel() override;// 🔊 关闭音频通道
|
||||
void LeaveRoom(bool notify_closed = true) override;// RTC 软退出：stop 房间并保留 handle
|
||||
void ForceRebuildEngine();
|
||||
bool IsAudioChannelOpened() const override;// 🔊 检查音频通道是否已打开
|
||||
void SendAbortSpeaking(AbortReason reason) override;// 🔊 发送中止通话请求
|
||||
void SendStartListening(ListeningMode mode) override;// 🔊 发送开始监听请求
|
||||
|
||||
@ -159,9 +159,10 @@ CONFIG_BT_GATTS_ENABLE=y
|
||||
CONFIG_BT_GATTC_ENABLE=y
|
||||
CONFIG_BT_BLE_SMP_ENABLE=y
|
||||
CONFIG_BT_STACK_NO_LOG=n
|
||||
CONFIG_BT_ALLOCATION_FROM_SPIRAM_FIRST=n
|
||||
CONFIG_BT_ALLOCATION_FROM_SPIRAM_FIRST=y
|
||||
CONFIG_BT_BLE_DYNAMIC_ENV_MEMORY=n
|
||||
CONFIG_BT_RESERVE_DRAM=0x10000
|
||||
CONFIG_LWIP_MAX_SOCKETS=20
|
||||
|
||||
# BluFi Configuration
|
||||
CONFIG_BT_BLUFI_ENABLED=y
|
||||
|
||||
@ -11,6 +11,8 @@ CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=8192
|
||||
CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y
|
||||
CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=65536
|
||||
CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY=y
|
||||
CONFIG_SPIRAM_ALLOW_BSS_SEG_EXTERNAL_MEMORY=y
|
||||
CONFIG_SPIRAM_ALLOW_NOINIT_SEG_EXTERNAL_MEMORY=y
|
||||
CONFIG_SPIRAM_MEMTEST=n
|
||||
CONFIG_MBEDTLS_EXTERNAL_MEM_ALLOC=y
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user