实机验证通过后,按 Kapi 无屏底座路线补齐 Pendant RTC 吊坠项目的迁移修复。 1. BLE 配网与资源隔离 - sdkconfig.defaults 开启 BT 优先 PSRAM 分配,并将 LWIP socket 上限提升到 20 - sdkconfig.defaults.esp32s3 允许 BSS/NOINIT 放入 PSRAM,释放内部 SRAM 给 BLE/WiFi/RTC - 配网模式 codec 使用 StartOutputOnly(),跳过麦克风 RX DMA 和 ES7210 输入链路 - ResetWifiConfiguration() 改为独立 wifi_reset task,避免在 iot_button/esp_timer 回调中阻塞延时 - WifiBoard 增加 IsWifiConfigMode(),供启动阶段判断是否走配网资源隔离路径 2. 音频底噪与 DMA 残留修复 - AudioCodec 增加 StartOutputOnly(),支持仅启动扬声器输出 - RTC 音频通道打开后灌入 200ms silence PCM,覆盖 I2S DMA 残留数据 - 软退出进入待命前重启 codec output 并再次灌静音,减少待命音/欢迎语前杂音 - box_audio_codec 在无硬件回采时使用 channel_mask=0,避免 I2S slot mask 被错误污染 3. 软件 loopback AEC - 引入 esp_aec 底层同步 API,使用 DAC 输出复制构建 ref ring - 上行 mic PCM 与延迟 ref 做同步消回声,适配无屏无硬件回采的 Pendant 形态 - AEC 采用 lazy init,减少启动阶段对 WiFi/BLE 内部 SRAM 的压力 - ref 静音时直接 passthrough,避免 AI 静音后误压制用户语音 - 在 player_pipeline_write 和 codec->OutputData 两条下行路径都追加 ref hook 4. RTC 连接稳定性与软退出 - VolcRtcProtocol 增加 LeaveRoom(bool notify_closed),支持 stop 房间但保留 rtc_handle - hibernate 路径使用 LeaveRoom(false),避免关闭回调顺手关掉 codec output 导致待命音无声 - LeaveRoom/ForceRebuildEngine 重置 downlink_is_pcm_ 和首包标志,避免本地 Opus 音效被当 PCM 播成杂音 - OpenAudioChannel 连续失败 3 次后 ForceRebuildEngine,清理 RTC SDK 内部异常状态 - 加入 DIAG-RTC socket/heap/PSRAM/RSSI 日志,便于定位 ICE socket 和内存问题 5. Dialog watchdog 与 BOOT 唤醒 - Dialog watchdog 到期不再写 reboot_dlg_idle 后 esp_restart - 新增 EnterIdleHibernate():软退房、清空残留音频队列、关闭麦克风、播放待命音后静默 - 新增 WakeFromHibernate():BOOT 唤醒后复用 RTC engine 并通过 ToggleChatState() 重连 RTC - BOOT 单击优先判断 IsHibernating(),异步唤醒,避免走普通按键状态机 - hibernate 期间禁止 PowerSaveTimer 进入 Light Sleep,保护 I2C/codec 总线 6. 文档与衍生项目沉淀 - 更新石头光源属性检测方案文档 - 将 Pendant 实测通过的软退出、AEC、BLE 配网隔离经验同步到衍生项目移植规则
249 lines
8.4 KiB
C++
249 lines
8.4 KiB
C++
#include "box_audio_codec.h"
|
|
|
|
#include <esp_log.h>
|
|
#include <driver/i2c.h>
|
|
#include <driver/i2s_tdm.h>
|
|
|
|
static const char TAG[] = "BoxAudioCodec";
|
|
|
|
BoxAudioCodec::BoxAudioCodec(void* i2c_master_handle, int input_sample_rate, int output_sample_rate,
|
|
gpio_num_t mclk, gpio_num_t bclk, gpio_num_t ws, gpio_num_t dout, gpio_num_t din,
|
|
gpio_num_t pa_pin, uint8_t es8311_addr, uint8_t es7210_addr, bool input_reference) {
|
|
duplex_ = true; // 是否双工
|
|
input_reference_ = input_reference; // 是否使用参考输入,实现回声消除
|
|
input_channels_ = input_reference_ ? 2 : 1; // 输入通道数
|
|
input_sample_rate_ = input_sample_rate;
|
|
output_sample_rate_ = output_sample_rate;
|
|
|
|
CreateDuplexChannels(mclk, bclk, ws, dout, din);
|
|
|
|
// Do initialize of related interface: data_if, ctrl_if and gpio_if
|
|
audio_codec_i2s_cfg_t i2s_cfg = {
|
|
.port = I2S_NUM_0,
|
|
.rx_handle = rx_handle_,
|
|
.tx_handle = tx_handle_,
|
|
};
|
|
data_if_ = audio_codec_new_i2s_data(&i2s_cfg);
|
|
assert(data_if_ != NULL);
|
|
|
|
// Output
|
|
audio_codec_i2c_cfg_t i2c_cfg = {
|
|
.port = (i2c_port_t)1,
|
|
.addr = es8311_addr,
|
|
.bus_handle = i2c_master_handle,
|
|
};
|
|
out_ctrl_if_ = audio_codec_new_i2c_ctrl(&i2c_cfg);
|
|
assert(out_ctrl_if_ != NULL);
|
|
|
|
gpio_if_ = audio_codec_new_gpio();
|
|
assert(gpio_if_ != NULL);
|
|
|
|
es8311_codec_cfg_t es8311_cfg = {};
|
|
es8311_cfg.ctrl_if = out_ctrl_if_;
|
|
es8311_cfg.gpio_if = gpio_if_;
|
|
es8311_cfg.codec_mode = ESP_CODEC_DEV_WORK_MODE_DAC;
|
|
es8311_cfg.pa_pin = pa_pin;
|
|
es8311_cfg.use_mclk = true;
|
|
es8311_cfg.hw_gain.pa_voltage = 5.0;
|
|
es8311_cfg.hw_gain.codec_dac_voltage = 3.3;
|
|
out_codec_if_ = es8311_codec_new(&es8311_cfg);
|
|
assert(out_codec_if_ != NULL);
|
|
|
|
esp_codec_dev_cfg_t dev_cfg = {
|
|
.dev_type = ESP_CODEC_DEV_TYPE_OUT,
|
|
.codec_if = out_codec_if_,
|
|
.data_if = data_if_,
|
|
};
|
|
output_dev_ = esp_codec_dev_new(&dev_cfg);
|
|
assert(output_dev_ != NULL);
|
|
|
|
// Input
|
|
i2c_cfg.addr = es7210_addr;
|
|
in_ctrl_if_ = audio_codec_new_i2c_ctrl(&i2c_cfg);
|
|
assert(in_ctrl_if_ != NULL);
|
|
|
|
es7210_codec_cfg_t es7210_cfg = {};
|
|
es7210_cfg.ctrl_if = in_ctrl_if_;
|
|
es7210_cfg.mic_selected = ES7120_SEL_MIC1 | ES7120_SEL_MIC2;
|
|
in_codec_if_ = es7210_codec_new(&es7210_cfg);
|
|
assert(in_codec_if_ != NULL);
|
|
|
|
dev_cfg.dev_type = ESP_CODEC_DEV_TYPE_IN;
|
|
dev_cfg.codec_if = in_codec_if_;
|
|
input_dev_ = esp_codec_dev_new(&dev_cfg);
|
|
assert(input_dev_ != NULL);
|
|
|
|
ESP_LOGI(TAG, "BoxAudioDevice initialized");
|
|
}
|
|
|
|
BoxAudioCodec::~BoxAudioCodec() {
|
|
ESP_ERROR_CHECK(esp_codec_dev_close(output_dev_));
|
|
esp_codec_dev_delete(output_dev_);
|
|
ESP_ERROR_CHECK(esp_codec_dev_close(input_dev_));
|
|
esp_codec_dev_delete(input_dev_);
|
|
|
|
audio_codec_delete_codec_if(in_codec_if_);
|
|
audio_codec_delete_ctrl_if(in_ctrl_if_);
|
|
audio_codec_delete_codec_if(out_codec_if_);
|
|
audio_codec_delete_ctrl_if(out_ctrl_if_);
|
|
audio_codec_delete_gpio_if(gpio_if_);
|
|
audio_codec_delete_data_if(data_if_);
|
|
}
|
|
|
|
void BoxAudioCodec::CreateDuplexChannels(gpio_num_t mclk, gpio_num_t bclk, gpio_num_t ws, gpio_num_t dout, gpio_num_t din) {
|
|
assert(input_sample_rate_ == output_sample_rate_);
|
|
|
|
i2s_chan_config_t chan_cfg = {
|
|
.id = I2S_NUM_0,
|
|
.role = I2S_ROLE_MASTER,
|
|
.dma_desc_num = 6,
|
|
.dma_frame_num = 240,
|
|
.auto_clear_after_cb = true,
|
|
.auto_clear_before_cb = false,
|
|
.intr_priority = 0,
|
|
};
|
|
ESP_ERROR_CHECK(i2s_new_channel(&chan_cfg, &tx_handle_, &rx_handle_));
|
|
|
|
i2s_std_config_t std_cfg = {
|
|
.clk_cfg = {
|
|
.sample_rate_hz = (uint32_t)output_sample_rate_,
|
|
.clk_src = I2S_CLK_SRC_DEFAULT,
|
|
.ext_clk_freq_hz = 0,
|
|
.mclk_multiple = I2S_MCLK_MULTIPLE_256
|
|
},
|
|
.slot_cfg = {
|
|
.data_bit_width = I2S_DATA_BIT_WIDTH_16BIT,
|
|
.slot_bit_width = I2S_SLOT_BIT_WIDTH_AUTO,
|
|
.slot_mode = I2S_SLOT_MODE_STEREO,
|
|
.slot_mask = I2S_STD_SLOT_BOTH,
|
|
.ws_width = I2S_DATA_BIT_WIDTH_16BIT,
|
|
.ws_pol = false,
|
|
.bit_shift = true,
|
|
.left_align = true,
|
|
.big_endian = false,
|
|
.bit_order_lsb = false
|
|
},
|
|
.gpio_cfg = {
|
|
.mclk = mclk,
|
|
.bclk = bclk,
|
|
.ws = ws,
|
|
.dout = dout,
|
|
.din = I2S_GPIO_UNUSED,
|
|
.invert_flags = {
|
|
.mclk_inv = false,
|
|
.bclk_inv = false,
|
|
.ws_inv = false
|
|
}
|
|
}
|
|
};
|
|
|
|
i2s_tdm_config_t tdm_cfg = {
|
|
.clk_cfg = {
|
|
.sample_rate_hz = (uint32_t)input_sample_rate_,
|
|
.clk_src = I2S_CLK_SRC_DEFAULT,
|
|
.ext_clk_freq_hz = 0,
|
|
.mclk_multiple = I2S_MCLK_MULTIPLE_256,
|
|
.bclk_div = 8,
|
|
},
|
|
.slot_cfg = {
|
|
.data_bit_width = I2S_DATA_BIT_WIDTH_16BIT,
|
|
.slot_bit_width = I2S_SLOT_BIT_WIDTH_AUTO,
|
|
.slot_mode = I2S_SLOT_MODE_STEREO,
|
|
.slot_mask = i2s_tdm_slot_mask_t(I2S_TDM_SLOT0 | I2S_TDM_SLOT1 | I2S_TDM_SLOT2 | I2S_TDM_SLOT3),
|
|
.ws_width = I2S_TDM_AUTO_WS_WIDTH,
|
|
.ws_pol = false,
|
|
.bit_shift = true,
|
|
.left_align = false,
|
|
.big_endian = false,
|
|
.bit_order_lsb = false,
|
|
.skip_mask = false,
|
|
.total_slot = I2S_TDM_AUTO_SLOT_NUM
|
|
},
|
|
.gpio_cfg = {
|
|
.mclk = mclk,
|
|
.bclk = bclk,
|
|
.ws = ws,
|
|
.dout = I2S_GPIO_UNUSED,
|
|
.din = din,
|
|
.invert_flags = {
|
|
.mclk_inv = false,
|
|
.bclk_inv = false,
|
|
.ws_inv = false
|
|
}
|
|
}
|
|
};
|
|
|
|
ESP_ERROR_CHECK(i2s_channel_init_std_mode(tx_handle_, &std_cfg));
|
|
ESP_ERROR_CHECK(i2s_channel_init_tdm_mode(rx_handle_, &tdm_cfg));
|
|
ESP_LOGI(TAG, "Duplex channels created");
|
|
}
|
|
|
|
void BoxAudioCodec::SetOutputVolume(int volume) {
|
|
ESP_ERROR_CHECK(esp_codec_dev_set_out_vol(output_dev_, volume));
|
|
AudioCodec::SetOutputVolume(volume);
|
|
}
|
|
|
|
void BoxAudioCodec::EnableInput(bool enable) {
|
|
if (enable == input_enabled_) {
|
|
return;
|
|
}
|
|
if (enable) {
|
|
esp_codec_dev_sample_info_t fs = {
|
|
.bits_per_sample = 16,
|
|
.channel = static_cast<uint8_t>(input_channels_),
|
|
.channel_mask = 0,
|
|
// .sample_rate = (uint32_t)output_sample_rate_,
|
|
.sample_rate = (uint32_t)input_sample_rate_,
|
|
.mclk_multiple = 0,
|
|
};
|
|
if (input_reference_) {
|
|
fs.channel_mask = ESP_CODEC_DEV_MAKE_CHANNEL_MASK(0);
|
|
fs.channel_mask |= ESP_CODEC_DEV_MAKE_CHANNEL_MASK(1);
|
|
}
|
|
ESP_ERROR_CHECK(esp_codec_dev_open(input_dev_, &fs));
|
|
ESP_ERROR_CHECK(esp_codec_dev_set_in_channel_gain(input_dev_, ESP_CODEC_DEV_MAKE_CHANNEL_MASK(0), 27.0));
|
|
if (input_reference_) {
|
|
ESP_ERROR_CHECK(esp_codec_dev_set_in_channel_gain(input_dev_, ESP_CODEC_DEV_MAKE_CHANNEL_MASK(1), 21.0));
|
|
}
|
|
ESP_LOGI(TAG, "Input opened: sr=%u ch=%u mask=0x%x ref=%d", (unsigned)fs.sample_rate, (unsigned)fs.channel, (unsigned)fs.channel_mask, (int)input_reference_);
|
|
} else {
|
|
ESP_ERROR_CHECK(esp_codec_dev_close(input_dev_));
|
|
}
|
|
AudioCodec::EnableInput(enable);
|
|
}
|
|
|
|
void BoxAudioCodec::EnableOutput(bool enable) {
|
|
if (enable == output_enabled_) {
|
|
return;
|
|
}
|
|
if (enable) {
|
|
// Play 16bit audio with configured channels
|
|
esp_codec_dev_sample_info_t fs = {
|
|
.bits_per_sample = 16,
|
|
.channel = static_cast<uint8_t>(output_channels_),
|
|
.channel_mask = 0,
|
|
.sample_rate = (uint32_t)output_sample_rate_,
|
|
.mclk_multiple = 0,
|
|
};
|
|
ESP_ERROR_CHECK(esp_codec_dev_open(output_dev_, &fs));
|
|
ESP_ERROR_CHECK(esp_codec_dev_set_out_vol(output_dev_, output_volume_));
|
|
} else {
|
|
ESP_ERROR_CHECK(esp_codec_dev_close(output_dev_));
|
|
}
|
|
AudioCodec::EnableOutput(enable);
|
|
}
|
|
|
|
int BoxAudioCodec::Read(int16_t* dest, int samples) {
|
|
if (input_enabled_) {
|
|
ESP_ERROR_CHECK_WITHOUT_ABORT(esp_codec_dev_read(input_dev_, (void*)dest, samples * sizeof(int16_t)));
|
|
}
|
|
return samples;
|
|
}
|
|
|
|
int BoxAudioCodec::Write(const int16_t* data, int samples) {
|
|
if (output_enabled_) {
|
|
ESP_ERROR_CHECK_WITHOUT_ABORT(esp_codec_dev_write(output_dev_, (void*)data, samples * sizeof(int16_t)));
|
|
}
|
|
return samples;
|
|
}
|