Abstract
NetherTeller is a staged Python pipeline that converts raw web-novel chapters into multi-character English audiobook videos. It separates narration from dialogue, assigns persistent voices to characters across 200+ chapters, synthesizes speech in segments, aligns ASS subtitles, and renders final video with FFmpeg. The central engineering problem is long-form voice and speaker consistency at batch scale.
1. What This Is
NetherTeller treats a web-novel chapter as structured data rather than a single TTS block. Raw chapter text is cleaned, narration and dialogue are separated, each quoted line is attributed to a character, and that character is resolved against a persistent voice map. Speech is generated in segments, subtitle timing is calculated, and FFmpeg composes the final upload-ready video.
Public output on the NetherTeller YouTube channel spans chapters 1 through the 200+ range. The implementation is closed-source; the architecture described here is reconstructed from the public output batches and the project brief.
2. How It Works
The pipeline is intentionally staged so each phase can be debugged or rerun independently. A single failed chapter, voice line, or render can be regenerated without rebuilding the entire batch.
| # | Stage | Input | Tool | Output |
|---|---|---|---|---|
| 01 | Load & Normalize | Raw chapter text | Python | Cleaned, formatted text |
| 02 | LLM Cleanup | Normalized text | Local LLM | Structured narration/dialogue transcript |
| 03 | Speaker ID & Voice Map | Structured transcript | Speaker identification + persistent map | Character-tagged lines with voice assignments |
| 04 | Speech Generation | Tagged lines | TTS / Voice cloning | Per-segment audio files |
| 05 | Alignment & Subtitles | Audio segments | Audio alignment + ASS | Timed ASS subtitle track |
| 06 | Final Render | Audio + subtitles + visual assets | FFmpeg | Upload-ready chapter/batch video |
3. Implementation Notes
3.1 Persistent voice mapping
Each character is bound to a voice profile that survives across chapter batches. When a character reappears in chapter 150, the pipeline resolves them against the same mapping used in chapter 5, reducing the voice drift that plagues naive per-chapter TTS runs. The map is maintained as state between batches rather than re-derived each time.
3.2 Segmented generation and regeneration
Audio is produced in per-line segments, not as one continuous file. This means a single misidentified speaker or a failed TTS call can be regenerated in isolation. Batch orchestration in Python tracks which segments are complete and which need a rerun, keeping the workflow resumable.
3.3 LLM-assisted cleanup and speaker identification
A local LLM handles two jobs: converting noisy raw chapter text into a consistent narration-versus-dialogue structure, and assisting with speaker attribution for quoted lines. Running the model locally keeps the pipeline self-contained and avoids per-token API costs at the 200+ chapter scale.
4. Constraints
-
Closed-source implementation
No public repository. Architecture and stage boundaries are inferred from the project brief and public output batches, so internal error handling and state management details remain opaque.
-
Voice drift at scale
Even with a persistent mapping, 200+ chapters of accumulated TTS segments make subtle timbre shifts audible. The mapping reduces but does not eliminate drift, especially for characters with sparse dialogue.
-
Local LLM quality ceiling
Cleanup and speaker identification are bounded by the local model's capability. Ambiguous dialogue attribution or malformed source text can propagate errors into downstream audio and subtitle stages.
-
No automated regression suite
The brief does not mention test coverage for voice consistency, subtitle timing, or render output. Regression detection relies on manual review of batch outputs.
5. Next
- a. Open-source the Python orchestration layer and voice-map state management so the pipeline is auditable and reproducible.
- b. Add automated regression checks: compare voice embeddings across chapters to flag drift before a batch ships.
- c. Extend the LLM cleanup stage to handle multi-language source novels, generalizing the narration/dialogue structuring beyond English.
— end of report —