Abstract
Luma is a Telegram-first personal AI assistant that routes incoming messages through a manager-style LLM orchestrator, delegates to modular skills for research, browser, and sandbox tasks, and maintains per-user semantic memory via Qdrant. The architecture separates interactive chat handling from background scheduler and watcher workers, so recurring jobs and condition checks run independently of any single Telegram request.
1. What This Is
Luma is a persistent, chat-based automation layer rather than a command-line demo. Allowed users or groups message a Telegram bot; the assistant loads per-user state, retrieves relevant context from a Qdrant vector store, and hands the request to a manager LLM that decides whether to answer directly or invoke a registered skill. Skills cover research, browser/sandbox execution, reminders, food logging, fitness workflows, and summary generation.
The whole stack—bot, agent workers, Qdrant, SQLite, and a local FastAPI debug UI—runs as a Docker Compose multi-service deployment. Per-user prompts, SQLite data, and supporting files are isolated so each user's state can be inspected independently.
2. How It Works
A single user message flows through six stages. Scheduler and watcher workers operate in parallel and do not block the interactive path.
| # | Stage | Input | Tool | Output |
|---|---|---|---|---|
| 01 | Message intake | Telegram update | Bot polling / webhook | Authenticated user/group payload |
| 02 | State & memory load | User ID + message text | Qdrant, SQLite | Per-user prompt, retrieved context |
| 03 | Manager routing | Enriched prompt | LLM endpoint (local/cloud) | Direct answer or skill invocation plan |
| 04 | Skill execution | Skill name + parameters | Research / browser / sandbox / reminder module | Structured result |
| 05 | Persist & reply | Result + updated state | SQLite, Qdrant, Telegram API | User-facing reply |
| 06 | Background workers | Cron config, watcher rules | Scheduler loop, watcher loop | Recurring jobs, condition-triggered actions |
3. Implementation Notes
3.1 Manager-style orchestration
Rather than a fixed intent classifier, the manager LLM receives the full context (user prompt, retrieved memory, available skill list) and emits either a direct textual answer or a structured skill-invocation plan. This keeps the routing logic in the model layer, which simplifies adding new skills but makes behaviour dependent on the underlying model's instruction-following quality.
3.2 Semantic memory as retrieval
Memory is stored in Qdrant and queried by relevance at each turn instead of appending an ever-growing transcript. This bounds prompt length and lets the assistant recall information from weeks ago without re-reading the full history. The trade-off is that retrieval quality depends on embedding similarity; very recent or highly specific facts can be missed if the query phrasing diverges from the stored vector.
3.3 Pluggable skill modules
Each skill (research, browser, sandbox, reminder, food log, fitness, summary) is a self-contained Python module registered with the agent builder. Adding a new tool does not require touching the Telegram handler or the manager prompt template, which keeps the core path stable as the skill set grows.
3.4 Parallel background workers
Scheduler and watcher loops run as separate worker processes inside the Docker Compose stack. They execute recurring jobs (e.g., daily reminders) and condition-based checks (e.g., file or webhook triggers) without occupying the Telegram reply path, so a long-running watcher task does not delay user-facing responses.
4. Constraints
-
Single-operator scope
Access is gated by an allowed-user/group list. There is no multi-tenant auth, rate limiting, or per-user resource quotas, so the system is designed for one operator or a small trusted group.
-
LLM dependency for routing
The manager LLM decides skill invocation. A weaker or misconfigured model can mis-route requests, skip memory retrieval, or produce malformed skill parameters, and there is no deterministic fallback path.
-
Local deployment ceiling
Docker Compose provides a repeatable local runtime but no horizontal scaling, health-check orchestration, or CI/CD pipeline. Running in production would require a container orchestrator and external service management.
-
No automated test coverage
The repository does not include a test suite for the agent loop, skill modules, or worker processes. Regression risk is managed manually through the FastAPI debug UI.
5. Next
- a. Add structured error handling and retry logic around LLM calls and skill execution so a transient model timeout does not drop a user message.
- b. Introduce a skill registry with hot-reload so new tools can be added without restarting the Compose stack.
- c. Extend the FastAPI debug UI with request tracing and per-skill latency metrics to make production debugging less ad-hoc.
— end of report —