Abstract
Paint Buddy takes a black-and-white source image, extracts horizontal line segments with OpenCV, and replays them as mouse-drag strokes inside Microsoft Paint via PyAutoGUI. The core engineering question is the coordinate-mapping layer between image space and a live GUI canvas.
1. What This Is
A desktop automation tool that makes a computer draw an image in Paint. Rather than replicating every pixel, it reduces the source to a compact sequence of horizontal strokes and drives a real GUI application to render them. The project separates image-analysis logic from UI-control logic, so the stroke list is a portable intermediate representation.
2. How It Works
The pipeline is strictly sequential: load, analyze, map, draw, finish. Each stage hands off to the next with a well-defined artifact.
| # | Stage | Input | Tool | Output |
|---|---|---|---|---|
| 01 | Load & normalize | Source image file | OpenCV | Grayscale / binary frame |
| 02 | Line extraction | Normalized frame | OpenCV | List of horizontal segments |
| 03 | Coordinate mapping | Segment endpoints | Python (affine transform) | Screen-space (x, y) pairs |
| 04 | Stroke execution | Screen-space pairs | d>PyAutoGUI | Mouse click-drag in Paint |
| 05 | Completion | All segments drawn | PyAutoGUI | Finished drawing in Paint |
3. Implementation Notes
3.1 Line extraction over pixel copying
OpenCV scans the normalized image row by row and emits horizontal segments for contiguous visible runs. This is a deliberate simplification: the output is a vector-like stroke list, not a raster. It keeps the action count low enough for a GUI mouse to complete in reasonable time while preserving the overall shape.
3.2 Canvas coordinate mapping
Image-space endpoints are translated to screen coordinates that match the configured Paint canvas position and zoom level. This is the most fragile link in the chain—any mismatch in window placement or canvas scale shifts the entire drawing.
4. Constraints
-
GUI fragility
Window position, canvas zoom, and OS-level timing all affect output. If Paint is moved or loses focus mid-run, strokes land in the wrong place with no recovery.
-
Binary-image scope
The pipeline assumes high-contrast black-and-white input. Color or low-contrast images are not handled; there is no per-channel stroke pass.
-
Sequential, single-threaded
Strokes are drawn one after another with no parallelism. Complex images with many segments take proportionally longer to complete.
-
No error recovery
There is no mid-run abort, retry, or progress checkpoint. A failed stroke leaves a partial drawing with no way to resume cleanly.
5. Next
- a. Add a calibration step that auto-detects the Paint canvas bounds via screenshot analysis, removing the manual coordinate configuration.
- b. Extend line extraction to color images by splitting into per-channel stroke passes (R, G, B) and layering them.
- c. Introduce a progress counter and a keyboard-interrupt handler so long drawings can be paused or aborted without corrupting the canvas.
— end of report —