Directive 3: ESP32 MQTT auto-reconnect, heartbeat, MQTT-state LED, drop logging
- Reconnect and re-subscribe on every (re)connect; log the subscribe result. - Heartbeat every 5 min on voice/audio_stream (valid silent PCM frame) so the Directive-1 watchdog can tell "device dead" from "nobody spoke". voice_bridge treats it as silence and ignores it. - Retained LWT "offline" / "online" presence on voice/status for layer attribution in the watchdog. - LED: green now means MQTT connected AND publishing. Wi-Fi down = red blink, Wi-Fi up but MQTT down = amber blink. Never green without MQTT. - Log the reason on every drop: PubSubClient state code, or the Wi-Fi disconnect reason from the Wi-Fi event. - Bound the I2S read (50 ms) so Wi-Fi/MQTT housekeeping cannot stall; reset a stale streaming state on disconnect (the stuck-green bug). - Document the MQTT/LED contract in README; flag config.h and led_control.h as stale drafts so they cannot creep back as a second config. - Credentials stay placeholders; nothing secret is committed. Not flashed/tested: no ESP32 and no arduino-cli is reachable from the build environment. Hardware-in-the-loop verification needs a human at the device. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
47
README.md
47
README.md
@@ -45,13 +45,35 @@ Location: `~/voice_bridge/` (On Server .13)
|
||||
## 🚦 LED Status Indicators (ESP32)
|
||||
The LED Ring provides real-time feedback on the device state.
|
||||
|
||||
**Green now means "MQTT connected and publishing", not merely "Wi-Fi joined".**
|
||||
This is the Directive 3 fix: the old ring could sit on green (streaming) while the
|
||||
socket was dead and nothing reached the broker.
|
||||
|
||||
| Color | State | Trigger |
|
||||
| :--- | :--- | :--- |
|
||||
| **Off** | Idle | Listening for sound (VAD). |
|
||||
| **Green** | Recording | Sound detected above threshold. Streaming audio. |
|
||||
| **Blue** | Processing | Silence detected. End of stream sent to server. |
|
||||
| **Rainbow** | Acknowledged | "Wake Word" detected by server OR Command executed. |
|
||||
| **Red** | Error | Wi-Fi or MQTT connection loss. |
|
||||
| **Red blink** | Wi-Fi down | Not joined to the access point. Never green. |
|
||||
| **Amber blink** | MQTT down | Wi-Fi up but the broker link is down. Never green. |
|
||||
| **Green (steady)** | Idle + healthy | MQTT connected, subscribed, heartbeats publishing. |
|
||||
| **Green (bright)** | Recording | Sound detected above threshold, streaming audio. |
|
||||
| **Blue** | Processing | Silence detected, end of stream sent to server, waiting for reply. |
|
||||
| **Yellow** | Acknowledged | Server replied `OK` (command executed). |
|
||||
|
||||
## 🔌 MQTT reliability (Hub Fixes — Directive 3)
|
||||
|
||||
- **Auto-reconnect + re-subscribe on every (re)connect.** PubSubClient reopens
|
||||
the socket but does not restore subscriptions. `connectMqtt()` re-subscribes to
|
||||
`voice/status` on every connect and logs the result. Without this the client can
|
||||
hold a live socket with no subscription and go silently deaf.
|
||||
- **Heartbeat every 5 minutes on `voice/audio_stream`.** The watchdog (Directive 1)
|
||||
watches that topic, so topic silence means "the publisher is dead", not "nobody
|
||||
spoke". The heartbeat is a valid silent PCM frame (256 bytes), so `voice_bridge`
|
||||
treats it as silence and ignores it.
|
||||
- **Presence.** The device sets a retained Last Will `offline` on `voice/status`
|
||||
and publishes retained `online` on connect. This gives the watchdog a way to
|
||||
attribute a failure to the ESP32 layer.
|
||||
- **Drop logging.** On every connected→disconnected transition the serial console
|
||||
prints the PubSubClient state code (for example `CONNECTION_LOST`) or the Wi-Fi
|
||||
disconnect reason code.
|
||||
|
||||
## ⚙️ Configuration Details
|
||||
|
||||
@@ -59,10 +81,14 @@ The LED Ring provides real-time feedback on the device state.
|
||||
* **Mic SCK:** GPIO 4
|
||||
* **Mic WS:** GPIO 5
|
||||
* **Mic SD:** GPIO 6
|
||||
* **LED Pin:** GPIO 5
|
||||
* **LED Pin:** GPIO 1
|
||||
* **VAD Threshold:** `1000` (Adjust based on noise floor)
|
||||
* **Soft Gain:** `x6` (Multiplies raw input to match Whisper requirements)
|
||||
|
||||
> `config.h` and `led_control.h` are stale early drafts and are **not** included
|
||||
> by the sketch. `voice_assistant.ino` is the single source of truth. Do not add
|
||||
> a second copy of the broker/topic/pin config.
|
||||
|
||||
### Docker Environment
|
||||
* **Network:** Internal bridge network between Bridge and Whisper.
|
||||
* **Models:**
|
||||
@@ -72,10 +98,19 @@ The LED Ring provides real-time feedback on the device state.
|
||||
## 🚀 Setup & Commands
|
||||
|
||||
**1. Flash ESP32**
|
||||
|
||||
Credentials in `voice_assistant.ino` are placeholders in this repo. Set the real
|
||||
values locally before flashing, and never commit them. Flashing needs the ESP32
|
||||
physically connected over USB to a machine with `arduino-cli` (this is the step
|
||||
that cannot be done from the server):
|
||||
|
||||
```bash
|
||||
arduino-cli upload -v -p /dev/ttyACM0 --fqbn esp32:esp32:esp32s3:CDCOnBoot=cdc,USBMode=hwcdc,PSRAM=disabled,FlashMode=dio,DebugLevel=info --input-dir ./build
|
||||
```
|
||||
|
||||
Watch the serial console (`arduino-cli monitor -p /dev/ttyACM0 -c baudrate=115200`)
|
||||
to confirm the `[mqtt] re-subscribe` and `[heartbeat]` lines.
|
||||
|
||||
**2. Start Backend**
|
||||
```bash
|
||||
cd ~/voice_bridge
|
||||
|
||||
6
config.h
6
config.h
@@ -1,5 +1,11 @@
|
||||
#pragma once
|
||||
|
||||
// STALE / UNUSED. This file is an early draft and is NOT compiled: the sketch
|
||||
// voice_assistant.ino does not include it and holds the live config itself.
|
||||
// The values below are wrong for the running device (broker .30 and the
|
||||
// homeassistant/voice/* topics belong to an older design). Kept only so old
|
||||
// checkouts do not break; do not add a second config here.
|
||||
|
||||
// --- WiFi ---
|
||||
const char* WIFI_SSID = "Aussie Broadband 8729";
|
||||
const char* WIFI_PASS = "WIFI_PASS_REDACTED";
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
#pragma once
|
||||
// STALE / UNUSED. This header is an early LED draft and is NOT compiled:
|
||||
// voice_assistant.ino does not include it. It also defines a conflicting
|
||||
// `strip` / `DeviceState` / `currentState`, so including it would break the
|
||||
// sketch. LED behaviour now lives in voice_assistant.ino (see renderLed()).
|
||||
#include <Freenove_WS2812_Lib_for_ESP32.h>
|
||||
#include "config.h"
|
||||
|
||||
|
||||
@@ -1,21 +1,62 @@
|
||||
/*
|
||||
* voice_assistant.ino — ESP32-S3 microphone-to-MQTT bridge
|
||||
*
|
||||
* Hub Fixes — Directive 3 hardening (2026-10-14).
|
||||
*
|
||||
* What changed and why:
|
||||
*
|
||||
* 1. MQTT auto-reconnect + re-subscribe on EVERY (re)connect.
|
||||
* PubSubClient reconnects to a restarted broker but does NOT restore
|
||||
* subscriptions. Without an explicit re-subscribe the client can hold a
|
||||
* live socket with no subscription and go silently deaf. That is the same
|
||||
* bug class that silenced voice_bridge on 2026-10-13.
|
||||
*
|
||||
* 2. Heartbeat every 5 minutes on voice/audio_stream.
|
||||
* The Directive-1 watchdog watches that topic. The heartbeat is a valid
|
||||
* silent PCM frame, so voice_bridge treats it as silence and ignores it,
|
||||
* while the watchdog sees that the publisher is alive. This makes silence
|
||||
* a reliable "the device is dead" signal instead of "nobody spoke".
|
||||
*
|
||||
* 3. LED shows the MQTT state, not just Wi-Fi.
|
||||
* GREEN = MQTT connected and publishing (idle: steady green, streaming:
|
||||
* bright green). Wi-Fi down = RED blink. Wi-Fi up but MQTT down = AMBER
|
||||
* blink. Green is never shown when the device cannot publish.
|
||||
*
|
||||
* 4. The serial console logs the reason whenever the broker connection drops
|
||||
* (the PubSubClient state code, or the Wi-Fi disconnect reason).
|
||||
*
|
||||
* Broker: the shared .13 voice broker on port 1883. That plane is plain and
|
||||
* shared with n8n and the voice path. Do not change it, and do not put TLS on
|
||||
* this client. config.h and led_control.h in this folder are stale; this
|
||||
* sketch is the single source of truth for the running firmware.
|
||||
*/
|
||||
|
||||
#include <WiFi.h>
|
||||
#include <PubSubClient.h>
|
||||
#include <driver/i2s.h>
|
||||
#include <Freenove_WS2812_Lib_for_ESP32.h>
|
||||
#include <Freenove_WS2812_Lib_for_ESP32.h>
|
||||
|
||||
// --- CONNECTIVITY ---
|
||||
// ---------------------------------------------------------------------------
|
||||
// CONFIG — edit the credential placeholders before flashing.
|
||||
// Never commit real credentials.
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// --- WiFi ---
|
||||
const char* WIFI_SSID = "Aussie Broadband 8729";
|
||||
const char* WIFI_PASS = "WIFI_PASS_REDACTED";
|
||||
|
||||
const char* MQTT_BROKER = "192.168.20.13";
|
||||
const int MQTT_PORT = 1883;
|
||||
const char* MQTT_USER = "MQTT_USER_REDACTED";
|
||||
const char* MQTT_PASS = "MQTT_PASS_REDACTED";
|
||||
// --- MQTT (.13 voice broker, plain 1883) ---
|
||||
const char* MQTT_BROKER = "192.168.20.13";
|
||||
const int MQTT_PORT = 1883;
|
||||
const char* MQTT_USER = "MQTT_USER_REDACTED";
|
||||
const char* MQTT_PASS = "MQTT_PASS_REDACTED";
|
||||
const char* MQTT_CLIENT_ID = "ESP32Mic";
|
||||
|
||||
const char* TOPIC_AUDIO = "voice/audio_stream";
|
||||
const char* TOPIC_STATUS = "voice/status";
|
||||
// --- Topics ---
|
||||
const char* TOPIC_AUDIO = "voice/audio_stream"; // audio + heartbeat liveness
|
||||
const char* TOPIC_STATUS = "voice/status"; // WAKE / OK / processing / presence
|
||||
|
||||
// --- HARDWARE ---
|
||||
// --- Hardware (ESP32-S3) ---
|
||||
#define I2S_SCK 4
|
||||
#define I2S_WS 5
|
||||
#define I2S_SD 6
|
||||
@@ -23,9 +64,17 @@ const char* TOPIC_STATUS = "voice/status";
|
||||
#define LED_PIN 1
|
||||
#define NUM_LEDS 12
|
||||
|
||||
// --- TUNING ---
|
||||
#define VAD_THRESHOLD 1000
|
||||
const int BLOCK_SIZE = 512;
|
||||
// --- Tuning ---
|
||||
#define VAD_THRESHOLD 1000
|
||||
#define SAMPLE_RATE 16000
|
||||
#define BLOCK_SIZE 512
|
||||
|
||||
// --- Timing ---
|
||||
#define HEARTBEAT_INTERVAL_MS (5UL * 60UL * 1000UL) // 5 minutes
|
||||
#define MQTT_RECONNECT_INTERVAL_MS 5000UL
|
||||
#define WIFI_RECONNECT_INTERVAL_MS 5000UL
|
||||
#define WIFI_CONNECT_TIMEOUT_MS 20000UL
|
||||
|
||||
int16_t sBuffer[BLOCK_SIZE];
|
||||
|
||||
WiFiClient espClient;
|
||||
@@ -33,20 +82,27 @@ PubSubClient mqtt(espClient);
|
||||
Freenove_ESP32_WS2812 strip(NUM_LEDS, LED_PIN, 0, TYPE_GRB);
|
||||
|
||||
// --- STATES ---
|
||||
enum DeviceState {
|
||||
STATE_IDLE,
|
||||
STATE_STREAMING, // Noise detected (Green)
|
||||
STATE_LISTENING, // Wake Word Confirmed (Blue)
|
||||
STATE_PROCESSING, // Sending to Server (Red)
|
||||
STATE_SUCCESS, // Done (Yellow)
|
||||
STATE_ERROR // Timeout (Blink Red)
|
||||
enum DeviceState {
|
||||
STATE_IDLE, // connected, no voice
|
||||
STATE_STREAMING, // voice detected, publishing audio (bright green)
|
||||
STATE_LISTENING, // wake word confirmed, publishing audio (bright green)
|
||||
STATE_PROCESSING, // waiting for the server (blue)
|
||||
STATE_SUCCESS, // command executed (yellow)
|
||||
STATE_ERROR // timeout (red blink)
|
||||
};
|
||||
DeviceState currentState = STATE_IDLE;
|
||||
unsigned long stateTimer = 0;
|
||||
|
||||
// --- CONNECTION TRACKING ---
|
||||
bool wasMqttConnected = false;
|
||||
unsigned long lastMqttAttempt = 0;
|
||||
unsigned long lastWifiAttempt = 0;
|
||||
unsigned long lastHeartbeat = 0;
|
||||
uint32_t ledValue = 0xFFFFFFFF; // sentinel so the first draw always happens
|
||||
|
||||
const i2s_config_t i2s_config = {
|
||||
.mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX),
|
||||
.sample_rate = 16000,
|
||||
.sample_rate = SAMPLE_RATE,
|
||||
.bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,
|
||||
.channel_format = I2S_CHANNEL_FMT_ONLY_LEFT,
|
||||
.communication_format = I2S_COMM_FORMAT_STAND_I2S,
|
||||
@@ -63,132 +119,315 @@ const i2s_pin_config_t pin_config = {
|
||||
.data_in_num = I2S_SD
|
||||
};
|
||||
|
||||
// --- LED HELPER ---
|
||||
void setRingColor(uint8_t r, uint8_t g, uint8_t b) {
|
||||
// ---------------------------------------------------------------------------
|
||||
// LED
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// Only push a frame when the colour actually changed. strip.show() disables
|
||||
// interrupts for the duration of the frame, so we must not call it every loop.
|
||||
void applyLed(uint8_t r, uint8_t g, uint8_t b) {
|
||||
uint32_t value = ((uint32_t)r << 16) | ((uint32_t)g << 8) | (uint32_t)b;
|
||||
if (value == ledValue) return;
|
||||
ledValue = value;
|
||||
strip.setAllLedsColor(r, g, b);
|
||||
strip.show();
|
||||
}
|
||||
|
||||
// --- MQTT HANDLING ---
|
||||
// The LED contract:
|
||||
// Wi-Fi down -> RED blink
|
||||
// Wi-Fi up, MQTT down -> AMBER blink
|
||||
// MQTT connected + publishing -> GREEN (steady when idle)
|
||||
// ... activity overlays -> blue / yellow while still connected.
|
||||
// Green therefore means "connected and publishing", never "Wi-Fi only".
|
||||
void renderLed(bool mqttUp) {
|
||||
const bool blink = (millis() / 400) % 2 == 0;
|
||||
|
||||
if (WiFi.status() != WL_CONNECTED) {
|
||||
applyLed(blink ? 90 : 0, 0, 0);
|
||||
return;
|
||||
}
|
||||
if (!mqttUp) {
|
||||
applyLed(blink ? 90 : 0, blink ? 45 : 0, 0);
|
||||
return;
|
||||
}
|
||||
|
||||
switch (currentState) {
|
||||
case STATE_IDLE: applyLed(0, 120, 0); break; // healthy green
|
||||
case STATE_STREAMING: applyLed(0, 255, 0); break; // recording
|
||||
case STATE_LISTENING: applyLed(0, 255, 0); break; // talking to Jarvis
|
||||
case STATE_PROCESSING: applyLed(0, 0, 255); break; // thinking
|
||||
case STATE_SUCCESS: applyLed(255, 200, 0); break; // done
|
||||
case STATE_ERROR: applyLed(blink ? 255 : 0, 0, 0); break; // fault
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// CONNECTION LOGGING
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const char* mqttStateStr(int state) {
|
||||
switch (state) {
|
||||
case -4: return "CONNECTION_TIMEOUT";
|
||||
case -3: return "CONNECTION_LOST";
|
||||
case -2: return "CONNECT_FAILED";
|
||||
case -1: return "DISCONNECTED";
|
||||
case 0: return "CONNECTED";
|
||||
case 1: return "BAD_PROTOCOL";
|
||||
case 2: return "BAD_CLIENT_ID";
|
||||
case 3: return "UNAVAILABLE";
|
||||
case 4: return "BAD_CREDENTIALS";
|
||||
case 5: return "UNAUTHORIZED";
|
||||
default: return "UNKNOWN";
|
||||
}
|
||||
}
|
||||
|
||||
// Log the Wi-Fi reason from the driver, not a guess.
|
||||
void onWifiEvent(WiFiEvent_t event, WiFiEventInfo_t info) {
|
||||
switch (event) {
|
||||
case ARDUINO_EVENT_WIFI_STA_CONNECTED:
|
||||
Serial.println("[wifi] joined access point");
|
||||
break;
|
||||
case ARDUINO_EVENT_WIFI_STA_GOT_IP:
|
||||
Serial.printf("[wifi] got IP %s\n", WiFi.localIP().toString().c_str());
|
||||
break;
|
||||
case ARDUINO_EVENT_WIFI_STA_DISCONNECTED:
|
||||
Serial.printf("[wifi] DISCONNECTED reason=%d\n", info.wifi_sta_disconnected.reason);
|
||||
break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// MQTT
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
void mqttCallback(char* topic, byte* payload, unsigned int length) {
|
||||
char msg[length + 1];
|
||||
memcpy(msg, payload, length);
|
||||
msg[length] = '\0';
|
||||
|
||||
|
||||
String message = String(msg);
|
||||
// Serial.print("Recv: "); Serial.println(message); // Debug
|
||||
|
||||
if (message == "WAKE") {
|
||||
// Server heard "Hey Jarvis" -> Turn BLUE
|
||||
currentState = STATE_LISTENING;
|
||||
stateTimer = millis(); // Reset timer to prevent premature timeout
|
||||
setRingColor(0, 0, 255);
|
||||
}
|
||||
else if (message == "OK") {
|
||||
// Command executed -> Turn YELLOW
|
||||
stateTimer = millis();
|
||||
} else if (message == "OK") {
|
||||
currentState = STATE_SUCCESS;
|
||||
stateTimer = millis();
|
||||
setRingColor(255, 200, 0); // Yellow/Orange
|
||||
}
|
||||
}
|
||||
|
||||
// Connect (or re-connect) and re-subscribe. Called on every (re)connect so the
|
||||
// client is never left holding a live socket with no subscription.
|
||||
bool connectMqtt() {
|
||||
lastMqttAttempt = millis();
|
||||
Serial.printf("[mqtt] connecting to %s:%d (wifi=%d)\n",
|
||||
MQTT_BROKER, MQTT_PORT, (int)WiFi.status());
|
||||
|
||||
// Retained Last Will on voice/status: a hard drop marks the device offline.
|
||||
// voice_bridge only acts on "processing"/"WAKE"/"OK", so this is inert to it.
|
||||
bool ok = mqtt.connect(MQTT_CLIENT_ID, MQTT_USER, MQTT_PASS,
|
||||
TOPIC_STATUS, 0, true, "offline");
|
||||
if (!ok) {
|
||||
Serial.printf("[mqtt] connect FAILED state=%d (%s)\n",
|
||||
mqtt.state(), mqttStateStr(mqtt.state()));
|
||||
return false;
|
||||
}
|
||||
|
||||
Serial.println("[mqtt] connected");
|
||||
bool sub = mqtt.subscribe(TOPIC_STATUS);
|
||||
Serial.printf("[mqtt] re-subscribe %s -> %s\n", TOPIC_STATUS, sub ? "ok" : "FAILED");
|
||||
|
||||
bool online = mqtt.publish(TOPIC_STATUS, "online", true);
|
||||
Serial.printf("[mqtt] presence online -> %s\n", online ? "ok" : "FAILED");
|
||||
|
||||
lastHeartbeat = millis(); // do not fire a heartbeat immediately after connect
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// WIFI
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
void wifiBegin() {
|
||||
Serial.printf("[wifi] connecting to SSID \"%s\"\n", WIFI_SSID);
|
||||
WiFi.mode(WIFI_STA);
|
||||
WiFi.begin(WIFI_SSID, WIFI_PASS);
|
||||
lastWifiAttempt = millis();
|
||||
}
|
||||
|
||||
bool wifiEnsure() {
|
||||
if (WiFi.status() == WL_CONNECTED) return true;
|
||||
if (millis() - lastWifiAttempt >= WIFI_RECONNECT_INTERVAL_MS) {
|
||||
Serial.printf("[wifi] down (status=%d); reconnecting\n", (int)WiFi.status());
|
||||
WiFi.disconnect();
|
||||
WiFi.begin(WIFI_SSID, WIFI_PASS);
|
||||
lastWifiAttempt = millis();
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// SETUP
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
void setup() {
|
||||
delay(2000);
|
||||
delay(2000); // let USB CDC serial come up
|
||||
Serial.begin(115200);
|
||||
|
||||
mqtt.setBufferSize(2048);
|
||||
|
||||
Serial.println();
|
||||
Serial.println("[boot] voice_assistant starting (Directive 3 firmware)");
|
||||
|
||||
strip.begin();
|
||||
strip.setBrightness(30);
|
||||
setRingColor(50, 50, 50); // White Boot
|
||||
applyLed(60, 60, 60); // white boot
|
||||
|
||||
i2s_driver_install(I2S_PORT, &i2s_config, 0, NULL);
|
||||
i2s_set_pin(I2S_PORT, &pin_config);
|
||||
|
||||
WiFi.begin(WIFI_SSID, WIFI_PASS);
|
||||
while (WiFi.status() != WL_CONNECTED) {
|
||||
delay(500);
|
||||
setRingColor(0, 0, 100); delay(100); setRingColor(0,0,0);
|
||||
}
|
||||
|
||||
|
||||
mqtt.setBufferSize(2048);
|
||||
mqtt.setServer(MQTT_BROKER, MQTT_PORT);
|
||||
mqtt.setCallback(mqttCallback);
|
||||
|
||||
setRingColor(0,0,0); // Idle
|
||||
mqtt.setKeepAlive(15);
|
||||
|
||||
WiFi.onEvent(onWifiEvent);
|
||||
wifiBegin();
|
||||
|
||||
// Bounded first-join wait. If it fails, loop() keeps retrying non-stop.
|
||||
const unsigned long start = millis();
|
||||
while (WiFi.status() != WL_CONNECTED &&
|
||||
millis() - start < WIFI_CONNECT_TIMEOUT_MS) {
|
||||
delay(250);
|
||||
applyLed(0, 0, ((millis() / 400) % 2) ? 80 : 0); // blue blink = joining
|
||||
}
|
||||
if (WiFi.status() == WL_CONNECTED) {
|
||||
Serial.printf("[wifi] connected, ip=%s\n", WiFi.localIP().toString().c_str());
|
||||
} else {
|
||||
Serial.println("[wifi] no connection yet; will keep retrying");
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// LOOP
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
void loop() {
|
||||
if (!mqtt.connected()) {
|
||||
if (mqtt.connect("ESP32Mic", MQTT_USER, MQTT_PASS)) {
|
||||
mqtt.subscribe(TOPIC_STATUS);
|
||||
// 1) Wi-Fi first: MQTT cannot work without it.
|
||||
const bool wifiUp = wifiEnsure();
|
||||
bool nowConnected = false;
|
||||
|
||||
// 2) MQTT connect / reconnect with a non-blocking backoff.
|
||||
if (wifiUp) {
|
||||
if (!mqtt.connected() &&
|
||||
(millis() - lastMqttAttempt >= MQTT_RECONNECT_INTERVAL_MS)) {
|
||||
connectMqtt();
|
||||
}
|
||||
if (mqtt.connected()) {
|
||||
mqtt.loop();
|
||||
nowConnected = true;
|
||||
}
|
||||
}
|
||||
mqtt.loop();
|
||||
|
||||
// --- TIMEOUT LOGIC (Fixes the "Lock Up") ---
|
||||
// 3) Log every drop exactly once, and name the reason.
|
||||
if (wasMqttConnected && !nowConnected) {
|
||||
if (WiFi.status() != WL_CONNECTED) {
|
||||
Serial.printf("[mqtt] connection DROPPED: Wi-Fi lost (wifi=%d). "
|
||||
"Reconnect + re-subscribe is automatic.\n",
|
||||
(int)WiFi.status());
|
||||
} else {
|
||||
Serial.printf("[mqtt] connection DROPPED: state=%d (%s). "
|
||||
"Reconnect + re-subscribe is automatic.\n",
|
||||
mqtt.state(), mqttStateStr(mqtt.state()));
|
||||
}
|
||||
currentState = STATE_IDLE; // never leave a stale green/streaming state
|
||||
} else if (!wasMqttConnected && nowConnected) {
|
||||
Serial.println("[mqtt] link up: connected and subscribed");
|
||||
}
|
||||
wasMqttConnected = nowConnected;
|
||||
|
||||
// 4) Heartbeat: prove the publisher is alive so topic silence means "dead".
|
||||
if (nowConnected &&
|
||||
currentState != STATE_STREAMING && currentState != STATE_LISTENING &&
|
||||
(millis() - lastHeartbeat >= HEARTBEAT_INTERVAL_MS)) {
|
||||
lastHeartbeat = millis();
|
||||
static const uint8_t silence[256] = {0}; // 128 samples of silent PCM
|
||||
bool ok = mqtt.publish(TOPIC_AUDIO, silence, sizeof(silence));
|
||||
Serial.printf("[heartbeat] %u bytes on %s -> %s\n",
|
||||
(unsigned)sizeof(silence), TOPIC_AUDIO, ok ? "ok" : "FAILED");
|
||||
if (!ok) {
|
||||
Serial.println("[heartbeat] publish failed; forcing MQTT reconnect");
|
||||
mqtt.disconnect();
|
||||
lastMqttAttempt = millis();
|
||||
}
|
||||
}
|
||||
|
||||
// 5) Device-state timers.
|
||||
if (currentState == STATE_PROCESSING && millis() - stateTimer > 6000) {
|
||||
// We waited 6 seconds for an answer, but got nothing.
|
||||
// Blink Red 3 times then Reset.
|
||||
for(int i=0; i<3; i++) {
|
||||
setRingColor(255, 0, 0); delay(200);
|
||||
setRingColor(0, 0, 0); delay(200);
|
||||
for (int i = 0; i < 3; i++) {
|
||||
applyLed(255, 0, 0); delay(200);
|
||||
applyLed(0, 0, 0); delay(200);
|
||||
}
|
||||
currentState = STATE_IDLE;
|
||||
}
|
||||
|
||||
// Reset "Success" yellow light after 3 seconds
|
||||
if (currentState == STATE_SUCCESS && millis() - stateTimer > 3000) {
|
||||
currentState = STATE_IDLE;
|
||||
setRingColor(0,0,0);
|
||||
}
|
||||
|
||||
size_t bytesRead;
|
||||
i2s_read(I2S_PORT, &sBuffer, sizeof(sBuffer), &bytesRead, portMAX_DELAY);
|
||||
renderLed(nowConnected);
|
||||
|
||||
// Gain Boost (x6)
|
||||
for (int i = 0; i < bytesRead / 2; i++) {
|
||||
int32_t boosted = sBuffer[i] * 6;
|
||||
// 6) Read audio with a bounded timeout. portMAX_DELAY would stall Wi-Fi and
|
||||
// MQTT housekeeping forever if the I2S peripheral stopped producing data.
|
||||
size_t bytesRead = 0;
|
||||
i2s_read(I2S_PORT, &sBuffer, sizeof(sBuffer), &bytesRead, pdMS_TO_TICKS(50));
|
||||
if (bytesRead < 2) {
|
||||
return;
|
||||
}
|
||||
|
||||
const int samples = bytesRead / 2;
|
||||
|
||||
// Gain boost (x6).
|
||||
for (int i = 0; i < samples; i++) {
|
||||
int32_t boosted = sBuffer[i] * 6;
|
||||
if (boosted > 32767) boosted = 32767;
|
||||
if (boosted < -32768) boosted = -32768;
|
||||
sBuffer[i] = (int16_t)boosted;
|
||||
}
|
||||
|
||||
|
||||
long sum = 0;
|
||||
for (int i = 0; i < bytesRead / 2; i++) {
|
||||
for (int i = 0; i < samples; i++) {
|
||||
sum += abs(sBuffer[i]);
|
||||
}
|
||||
int avg = sum / (bytesRead / 2);
|
||||
const int avg = samples > 0 ? (int)(sum / samples) : 0;
|
||||
|
||||
// --- AUDIO LOGIC ---
|
||||
|
||||
// 1. Detect Noise -> Green
|
||||
// Never publish audio into a dead socket.
|
||||
if (!nowConnected) {
|
||||
currentState = STATE_IDLE;
|
||||
return;
|
||||
}
|
||||
|
||||
// Speech detected while idle -> start streaming (bright green).
|
||||
if (currentState == STATE_IDLE && avg > VAD_THRESHOLD) {
|
||||
currentState = STATE_STREAMING;
|
||||
stateTimer = millis();
|
||||
setRingColor(0, 255, 0); // Green (I hear you)
|
||||
|
||||
// Send trigger buffer
|
||||
mqtt.publish(TOPIC_AUDIO, (const uint8_t*)sBuffer, bytesRead);
|
||||
}
|
||||
|
||||
// 2. Stream Audio
|
||||
// Stream while active.
|
||||
if (currentState == STATE_STREAMING || currentState == STATE_LISTENING) {
|
||||
mqtt.publish(TOPIC_AUDIO, (const uint8_t*)sBuffer, bytesRead);
|
||||
|
||||
|
||||
if (avg > VAD_THRESHOLD) stateTimer = millis();
|
||||
|
||||
// Silence Timeout (1.5s)
|
||||
|
||||
// Silence timeout (1.5 s).
|
||||
if (millis() - stateTimer > 1500) {
|
||||
if (currentState == STATE_LISTENING) {
|
||||
// We were talking to Jarvis, now we wait for reply
|
||||
// We were talking to Jarvis; now wait for the reply.
|
||||
currentState = STATE_PROCESSING;
|
||||
stateTimer = millis(); // Start timeout timer
|
||||
mqtt.publish(TOPIC_STATUS, "processing");
|
||||
setRingColor(255, 0, 0); // Red (Thinking/Wait)
|
||||
stateTimer = millis();
|
||||
mqtt.publish(TOPIC_STATUS, "processing");
|
||||
} else {
|
||||
// Just noise, never woke up
|
||||
// Just noise; never woke up.
|
||||
currentState = STATE_IDLE;
|
||||
setRingColor(0, 0, 0);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user