Abstract
Two Dockerized FastAPI services wrapping local speech models: NeuTTS for offline voice-cloning TTS and Nemotron (ONNX Runtime GenAI) for ASR. Both expose REST and WebSocket endpoints with persistent storage and health checks, letting other pipelines call speech generation and recognition without embedding model initialization logic.
1. What This Is
This consolidated project groups two related speech-AI deployment services. NeuTTS provides offline text-to-speech with reference-voice cloning: you save a reference voice, then synthesize new speech by referencing a stored keyword. Nemotron wraps an ONNX Runtime GenAI ASR model behind FastAPI, offering file transcription and a WebSocket streaming path for incremental PCM audio.
Both services share the same deployment pattern: Docker containers with persistent mounted volumes for model data, generated audio, and reference assets, plus startup health and readiness checks. The goal is to make local speech models behave like predictable infrastructure that media pipelines and agent systems can call over HTTP.
2. How It Works
Each service follows the same request lifecycle: container starts, model loads, readiness check passes, then the service accepts work.
| # | Stage | Input | Tool | Output |
|---|---|---|---|---|
| 01 | Container start | Docker image + mounted volumes | Docker / Compose | Running process |
| 02 | Model load + self-test | Model weights on volume | ONNX Runtime GenAI / NeuTTS | Ready state |
| 03 | Health / readiness check | HTTP GET /health | FastAPI | 200 OK or 503 |
| 04 | Inference | Text (TTS) or audio file / PCM stream (ASR) | NeuTTS / Nemotron | WAV file or transcript |
| 05 | Persistence + response | Generated artifact | Mounted volume + HTTP | Download ref / JSON transcript |
3. Implementation Notes
3.1 NeuTTS voice persistence
Reference voices and their transcripts are stored in a mounted data folder. The API exposes a voice-save endpoint (upload reference audio plus transcript) and a TTS endpoint that accepts a voice keyword and target text. Generated audio is written to a second mounted folder, so container recreation does not lose voices or outputs.
3.2 Nemotron ASR streaming
Beyond the file-upload transcription endpoint (which accepts language, VAD, and chunk-size parameters), the service opens a WebSocket path for incremental PCM audio. The model runs on ONNX Runtime GenAI, and a startup self-test verifies the runtime can decode a short probe before the readiness flag flips to true.
3.3 Shared deployment pattern
Both services use the same FastAPI plus Docker skeleton: a Compose file defines the container, mounts persistent volumes for model weights and outputs, and wires a health-check endpoint. This keeps the two services deployable independently while sharing operational conventions.
4. Constraints
-
Local-only inference
Both services run entirely on the host GPU or CPU. There is no cloud fallback or autoscaling; if the container is down, the pipeline is down.
-
No authentication or rate limiting
Endpoints are open on the local network. Fine for a single-operator setup, but not safe to expose without an auth proxy in front.
-
Single-request inference
Each request blocks the model thread. Concurrent TTS or ASR calls will queue or time out; there is no batching or worker pool.
-
Voice-cloning fidelity is reference-dependent
NeuTTS quality tracks the reference clip. Short or noisy references degrade output noticeably, and there is no automatic quality gate before a voice is saved.
5. Next
- a. Add a lightweight auth token and per-IP rate limit so the services can sit behind a reverse proxy on a shared dev box.
- b. Introduce a simple request queue (asyncio semaphore or small task worker) to handle concurrent inference calls without blocking.
- c. Wire a CI step that runs the health-check and a short end-to-end transcription or TTS round-trip against the Docker image on every push.
— end of report —