Abstract
A Python desktop program that listens to a live microphone stream via SoundDevice, classifies each audio chunk as active or quiet using a simple amplitude threshold, and switches an OpenCV-rendered skull between talking and idle frames in real time. The interesting part is the tight coupling between a raw audio signal and a per-frame visual state with no intermediate recording step.
1. What This Is
The project is an audio-reactive visual toy. It does not perform speech recognition or transcription; it answers one question per audio chunk — is the signal loud enough to count as speech? — and maps the binary answer to a skull animation state. SoundDevice captures the microphone stream, a threshold check produces the state, and OpenCV draws the matching frame in a window that updates every loop iteration.
2. How It Works
The main loop runs continuously while the window is open. Each iteration pulls a short chunk from the live audio stream, computes its amplitude, compares it against a fixed threshold, and redraws the skull accordingly. There is no queue and no separate thread for audio versus rendering — the single Python loop handles both, which keeps the code simple but ties frame rate to audio chunk size.
| # | Stage | Input | Tool | Output |
|---|---|---|---|---|
| 01 | Open audio stream | Microphone device | SoundDevice | Live sample buffer |
| 02 | Read chunk & measure level | Sample buffer | Python | Amplitude value |
| 03 | Classify state | Amplitude value | Threshold check | Active / Quiet flag |
| 04 | Select skull frame | Active / Quiet flag | Frame lookup | Target image |
| 05 | Render & loop | Target image | OpenCV imshow | Updated window |
3. Constraints
-
Binary state only
The threshold produces a single active/quiet flag. There is no volume scaling, so a whisper and a shout look identical to the animation.
-
Fixed threshold, no calibration
The cutoff is hard-coded. A quiet room or a loud environment shifts the operating point, and the user has no way to adjust sensitivity at runtime.
-
Single-threaded loop
Audio capture and OpenCV rendering share one Python loop. If the display call blocks, the next audio chunk is delayed, which can cause visible stutter.
-
Desktop-only
OpenCV's highgui window is a native desktop widget. There is no web or headless rendering path, so the project cannot run in a browser or on a server.
4. Next
- a. Add a startup calibration step that samples ambient noise and sets the threshold relative to the measured floor.
- b. Replace the binary flag with a small set of amplitude bands (idle, talking, loud) to drive multiple skull frames.
- c. Move audio capture into a separate thread or callback so the render loop is not blocked by I/O.
— end of report —