Abstract
A three-utility Python/FFmpeg tool that sorts a video library by orientation and resolution bucket, batch re-encodes to HEVC/AAC with optional NVENC acceleration, and flags same-size duplicate candidates. The design goal is batch resilience: one corrupt or unsupported file never halts a multi-thousand-file run.
1. What This Is
Three related CLI utilities for recurring video-library cleanup, built in June 2024. The sorter reads each file's metadata and moves it into an orientation/resolution folder (144p through 2160p). The re-encoder runs FFmpeg in batch mode targeting HEVC video and AAC audio, with an optional NVIDIA NVENC path for hardware acceleration. The duplicate finder scans the library and flags files that share an identical byte size.
The project was designed as a standalone media-automation script set; it later became background for larger automated video and audiobook pipelines.
2. How It Works
The pipeline is sequential but each stage is independently runnable. The sorter and re-encoder share the same file-discovery and metadata-reading code; the duplicate finder is a separate pass.
| # | Stage | Input | Tool | Output |
|---|---|---|---|---|
| 01 | Scan & probe | Input library directory | Python + FFprobe | Per-file dimensions, codec, size |
| 02 | Classify & sort | Probed metadata | Python | Files moved to orientation/resolution folders |
| 03 | Queue for re-encode | Eligible sorted files | Python (size-descending sort) | Ordered encode queue |
| 04 | Batch encode | Queued files | FFmpeg (libx265 / NVENC) + AAC | HEVC/AAC output files |
| 05 | Error isolation | Failed encodes | Python | Files routed to error path; batch continues |
| 06 | Duplicate scan | Full library | Python (file-size match) | Duplicate-candidate list |
3. Implementation Notes
3.1 Resolution & orientation classification
Files are bucketed by height into classes from 144p up to 2160p, and by aspect ratio into landscape or portrait. Files whose metadata cannot be read or that fall outside supported classes are moved to a dedicated error folder rather than silently dropped, so a reviewer can inspect what was skipped.
3.2 NVENC fallback path
When an NVIDIA GPU and the NVENC driver are detected, the encoder switches from libx265 to h264_nvenc (or h265_nvenc) for faster throughput. If the GPU is absent or the driver is missing, the script falls back to software encoding without changing the output codec target. The selection is a one-time check at startup, not per-file.
3.3 Duplicate detection by file size
The duplicate finder compares byte size only. Two files with identical size are flagged as candidates; no content hash or perceptual comparison is performed. This is a deliberate simplicity trade-off: it is fast on large libraries but will produce false positives for files that happen to share a size.
4. Constraints
-
Size-only duplicate matching
Two unrelated files of the same byte length are flagged as duplicates. No MD5/SHA or perceptual hash is computed, so the output list requires manual review before any deletion.
-
NVENC is all-or-nothing per run
The hardware-encoder check happens once at startup. A mid-run GPU failure or driver crash is not caught; the batch either uses NVENC throughout or falls back to software for the entire run.
-
Single-machine batch
All encoding runs on one host. There is no queue, worker pool, or distributed mechanism, so a 500-file library at 1080p HEVC can take many hours on software encoding.
-
No dry-run or undo
The sorter moves files and the re-encoder overwrites or writes alongside originals. There is no preview mode and no transactional rollback if a run is interrupted partway.
5. Next
- a. Add a content-hash (SHA-256) pass behind the size-match filter so the duplicate list is actionable without manual review.
- b. Introduce a --dry-run flag for both the sorter and re-encoder that prints the planned moves/encodes without touching the filesystem.
- c. Move the NVENC availability check to a per-file guard so a mid-batch GPU failure degrades gracefully to software for the remaining files.
— end of report —