Reachy Stage Chat

A Reachy Mini chat app built for live stage demos. Audio input comes from an external microphone paired to the host computer (Bluetooth mic/headset, or wireless lav/handheld with a USB receiver dongle) instead of the robot's onboard mic array, which picks up crowd noise and PA bleed on stage. Audio output still plays through Reachy Mini's speaker, so the robot appears to talk on stage.

Two supported setups:

1. App on your computer (any mic the computer can see, including Bluetooth headsets). Launch with the launcher for your OS (below); the SDK finds the robot on the network and streams Reachy's voice to his speaker over WebRTC.
2. Wireless robot + USB mic plugged into the robot (wired/USB mics only). The Wireless's USB-C is a host port: plug in a USB mic or a wireless mic's USB receiver dongle (USB-C→A adapter if needed), run the app from the dashboard as usual, and the mic appears in the device dropdown within a few seconds. Fully self-contained — no computer-side install.

What does not work: the app running on-board with no USB mic attached (it can then only see the robot's own mic, reachymini_audio_src — the settings page warns about this), and Bluetooth headphones paired to the robot (the robot has no Bluetooth audio stack; its Bluetooth is a setup/provisioning channel only).

Pipeline: external mic → sounddevice capture → OpenAI Realtime API (gpt-realtime-2, speech-to-speech) → resample → Reachy Mini speaker via the SDK (push_audio_sample()). Turn detection is done locally (VAD + gating + push-to-talk), not by the API — the app decides when a turn starts and ends.

The SDK handles output only. Input is captured directly from the chosen macOS device — the robot's mics are never opened.

Features

• Input device selection — dropdown in the web UI, or MIC_DEVICE env var / config file for scripted launches. Devices are matched by name substring (device indices shift when devices connect/disconnect). The last-used device is persisted.
• Graceful fallback — if the chosen device disappears mid-session (Bluetooth dropout), the app logs clearly and falls back to the system default input. It never crashes on stage.
• Echo/feedback gating — the mic is muted while the robot is speaking (playback duration computed from sample length ÷ samplerate), so a presenter mic held near the robot's speaker doesn't feed back into the pipeline.
• Push-to-talk — toggle in the UI (hold the big button or the spacebar), so crowd Q&A can't accidentally trigger the robot.
• Input level meter — in the web UI, so the presenter can confirm the mic is live before going on.
• Speech-reactive motion — head wobble + antenna bounce while the robot speaks, driven by playback state.
• Vision — ask "what can you see?" and the model calls its camera tool: a photo from Reachy's camera is attached to the conversation and he describes it, like the official conversation app.
• Barge-in — in push-to-talk mode, pressing the talk button or spacebar while Reachy is speaking cuts him off and opens the mic immediately.
• Idle moves — after 1 minute with no input, Reachy does a random little dance (sway, nod groove, look-around, head circle, antenna party), and again every minute he stays idle. Any speech input or reply cancels it instantly.
• Breathing — a gentle continuous breathing motion (small vertical bob, slight pitch, antennas lifting on the inhale) whenever he's idle or listening, so he always looks alive between exchanges.
• OpenAI key in the UI — paste the API key on the settings page (stored owner-only in the user config dir); the OPENAI_API_KEY env var still works as a fallback.
• Speaker volume — 0–200% gain slider for Reachy's speaker, default 200%.
• Opening line + system prompt — editable on the settings page. "Apply now" changes them for the session; "Save for next launch" persists them. The opening line is spoken automatically when the app starts, and can be re-triggered from the UI.

Quick start (presenters)

See docs/RUN_SHEET.md for the one-page run sheet. Short version: pair the mic → open the Reachy Mini desktop app → launch Reachy Stage Chat (or double-click launch_stage_chat.command) → open the app's settings page → pick your mic and check the level meter is green → go.

Configuration

Settings chosen in the web UI are persisted to the platform user config directory and restored on the next launch. Env vars override persisted config for that session.

Install as a Reachy Mini app

The app follows the Reachy Mini app format: a Python package exposing a ReachyMiniApp subclass through the reachy_mini_apps entry-point group. Once the Hugging Face Space mirror is tagged reachy_mini, it is discoverable and installable from the Reachy Mini desktop app dashboard.

Manual install for development:

pip install -e .
python -m reachy_stage_chat.main

Repo layout

• reachy_stage_chat/ — the app package
  • main.py — ReachyMiniApp subclass, FastAPI settings endpoints, main loop
  • audio_input.py — sounddevice capture, device matching, VAD, gating, fallback
  • realtime_pipeline.py — OpenAI Realtime API session (gpt-realtime-2)
  • speech_output.py — resampling, volume, robot speaker playback
  • settings.py — env/config-file settings with persistence
  • static/ — settings web UI (device dropdown, level meter, push-to-talk)
• launch_stage_chat.command — double-clickable macOS launcher for mentors
• docs/RUN_SHEET.md — one-page presenter run sheet
• .github/workflows/sync-to-hf.yml — mirrors main to the Hugging Face Space

GitHub → Hugging Face sync

The GitHub repo is the source of truth; the HF Space mirrors it. Every push to main force-pushes to the Space via .github/workflows/sync-to-hf.yml. It requires an HF_TOKEN repository secret (write-scoped Hugging Face token).

Known gotchas

• macOS mic permission: the first run prompts for microphone access for the desktop app / Python. Approve it (System Settings → Privacy & Security → Microphone) before going on stage.
• Bluetooth latency (100–300 ms) and dropouts: fine for chat turn-taking; prefer a USB-dongle wireless mic for the actual event.
• Device indices shift when devices connect/disconnect — always configure the mic by name substring, not raw index.
• Assumes the app runs on the Mac — launch it with launch_stage_chat.command (or python -m reachy_stage_chat.main) on the computer, with the robot reachable on the network. On a Wireless robot, installing from the dashboard runs the app on-board the Pi: the device dropdown then only shows reachymini_audio_src (the robot's own mic), and mics paired to your computer can never appear. The settings page warns when it detects this.