A commercial smart speaker sends your living room audio to a provider and takes two to three seconds to execute a command that never needed to leave the house. This build keeps wake word, transcription and intent parsing on local hardware, and gets the round trip under a second.
The Problem with Commercial Smart Speakers
Most 'smart' speakers route raw audio to the cloud, taking 2-3 seconds round-trip to execute a command that only touches the local network. They also lack spatial context: 'turn it off' from the living room is ambiguous to a device that does not know which room it is in't know what 'it' is. OpenClaw does.
1. The Air-Gapped Architecture
The whole stack runs on a single Mac Mini M4 Pro in a network closet, hosting OpenClaw alongside Home Assistant on a VLAN with no route to the internet.
2. The Hardware Nodes
Rather than $50 commercial speakers, the satellites are custom nodes:
- M5Stack Atom Echo ($13): Tiny ESP32-based smart speakers placed in every room. They stream wake-word activated audio via Wi-Fi.
- Mac Mini M4 Pro: The 'Brain'. Runs the local Whisper model for transcription, Llama 3 for intent parsing, and TTS for the voice response.
- Home Assistant: The 'Muscles'. OpenClaw sends JSON RPC commands directly to the HA inference API to toggle relays.
3. Reaching Sub-Second Response Times
Latency is the hard part. OpenClaw's Streaming Audio API transcribes with Whisper while the user is still speaking, so by the time the sentence ends the LLM has already begun parsing intent. In practice the lights toggle before the TTS reply finishes speaking.
4. The "Room Aware" Context
Because each M5Stack satellite is bound to a zone in the OpenClaw config, the agent knows which room the request came from. 'Lights out' spoken in the bedroom does not touch the kitchen β the zone binding is what gives it spatial awareness.
The Result
The result is a controller that responds faster than a cloud assistant on local device commands, keeps all audio on the LAN, and handles chained instructions. What it does not do is answer general knowledge questions β it is a home controller, not a search engine.