Helios

Helios — Completion Plan to Production-Ideal

Snapshot date: 2026-05-18, after phase 43.

Snapshot date: 2026-05-18, after phase 43. Sources: git log (40 commits), code survey, ADR index, parameters registry (89 keys), feature matrix (23 features × 4 tiers), 71 test files, 7 Alembic migrations.

Living document — update when a section ships or scope shifts. The code is the source of truth; this file is a map.

This document answers one question: what is left between the current state and a Helios we would deploy, sell, and operate without embarrassment?

It is organized so the team can slice the work without losing dependency order:

  • §1 — honest audit of what exists, what's partial, what's stub.
  • §2 — Tier A: shipping correctness (close the critical gaps).
  • §3 — Tier B: feature breadth (the long tail of plugins / channels / importers / surfaces).
  • §4 — Tier C: scale, polish, durability.
  • §5 — cross-cutting quality (coverage, fuzz, soak, error budgets).
  • §6 — risk register.
  • §7 — open product decisions for the owner.
  • §8 — recommended order of operations (next 14 phases).
  • §9 — anti-goals. Things that look tempting but actively harm the product.
  • §10 — done-when checklist per workstream.

1. Current state — an honest audit

1.1 Foundation (strong, load-bearing, well-tested)

  • Parameters registry (89 keys, 9 scope levels). Append-only overrides, validators (type/min/max/regex/enum), Pub/Sub bus, audit-log integration, hot-reload cache. Owner panel renders 89 forms generated from the registry.
  • Stealth guard — runtime assert_stealth_safe + CI AST whitelist scan
    • 100 % coverage on apps/userbot/stealth/. Cannot reach a forbidden Pyrogram method without raising StealthViolation.
  • Multi-tenant RLShelios_app non-superuser role, FORCE ROW LEVEL SECURITY, NULL-handling fix in 0003, integration-tested isolation.
  • Hash-chained audit log — daily chain, tamper-evident.
  • Signed bot callbacks — HMAC-SHA256 truncated to 8 bytes (fits the 64-byte Telegram ceiling), nonce-replay protection, ttl from bot.callback.signature.ttl_seconds.
  • Telegram initData HMAC — two-step WebAppData keying, max-age cap.
  • AES-256-GCM at rest — AAD-bound encryption for session blobs and route secrets; master key abstracted via MasterKeyProvider (env + AWS-KMS stub).
  • TOTP (RFC 6238) for owner — require_owner checks X-TOTP-Code when OWNER_TOTP_ENABLED.
  • Resilience dispatcher — Pyrogram-agnostic mapping of exceptions to FailureKind, health-score updates, quarantine, owner alerts on critical.
  • Userbot pool runtime — boot loop, factory wrapping every Client with the stealth guard, presence poller emitting helios:events Redis Stream.
  • Real ingester — consumer-group reads + ack-after-handle, derives online_sessions from status_events.
  • Aggregator + daily rollupstarget_daily_stats table, idempotent upsert, compute_daily() pure.
  • Health watchdog — pool, queue, stale-row checks; SETNX dedup of owner-DM alerts.
  • Signed-callback validation — every callback through SignedCallback factory + filter; CI rejects raw InlineKeyboardMarkup.
  • WebSocket /v1/stream (phase 41) — tier-gated, initData-auth'd, Redis pub/sub pump with keepalive, consumed by Mini App's useEventStream hook (phase 42).

1.2 Functional surfaces (partial, working)

SurfaceState
API20+ endpoints. me, targets, analytics, heatmap, daily, routes CRUD, target_settings, personas, userbots/import, business/*, owner/*, audit, gdpr/*, settings/parameters/visible, WS /stream. Auth: initData + scoped Bearer + TOTP for owner.
Mini App10 pages (Me, Targets, TargetDetail, TargetSettings, Business, OwnerPanel, Userbots, Personas, AuditLog, Settings). HashRouter, TanStack Query, Zustand. Live-events card on home. Bundle 76 kB gzipped (budget 200).
Bot/start (only slash), main menu, language picker, target list/add/remove, billing menu, Stealth Mode purchase, About / Help pages. Business Bot scaffolding (business_connection, business_message, edited/deleted) consumes updates and writes to business_* tables.
BillingTelegram Stars only: SKU catalog, invoice creation, payment confirmation, tier grant, Stealth Mode plans (1m / 3m / 1y / lifetime). No CryptoBot / CryptAPI / xRocket / TON Connect.
NotificationsBotMessageNotifier for offline→online transitions. Health-watchdog DMs for owner. No webhook spam yet — logging_router worker fans helios:events to enabled webhook routes.
Delivery channelsbot_message (direct path via notifier), webhook (HMAC-signed, retries). Missing: tg_channel, tg_topic, external_bot, email, discord, slack, file_archive.
Session importerspyrogram_session (SQLite file), pyrogram_string, telethon_string (phase 43). Missing: telethon_session (SQLite file), tdata (Desktop archive), tdlib.
Paymentstelegram_stars. Empty dirs: cryptobot/, cryptapi/, xrocket/, ton_connect/.
EnrichmentProtocol stubs only (ocr/base, transcription/base, sentiment/base). No concrete providers.
ObservabilityPrometheus metrics (API + domain + worker servers on 9091-9096), Grafana overview dashboard (8 panels), alert rules (5 groups).
Documentationmkdocs-material site, strict mode in CI, ADRs 0001-0011 indexed.

1.3 Stubs and zeros (the long tail — Tier B / C)

  • Web portal (ADR-0007) — no portal.html OIDC flow wired (build emits the second entry, but the Telegram-Login handshake + JWT exchange aren't built; portal.tsx is the bare entry).
  • Continuous aggregates — none. Analytics resolves directly from hypertables on every request.
  • Hybrid analytics engine (ADR-0010) — heatmap / daily exist but the tier-aware resolve(query, scope_ctx) router does not.
  • Meilisearch indexer — backend wrapper exists, no worker, no target_messages table (Business Bot has its own business_messages).
  • Media pipeline (ADR-0009) — no media_downloader, no media_compactor, no per-file HKDF key derivation. Storage backends are stubs.
  • Persona warmup — picker exists; automatic warmup (read-only chat lurking, slow contact imports) does not.
  • Chat tracking — no tracked_chats migration, no joiners running, no message capture workflows beyond business messages.
  • GDPR purger workercore/gdpr repository + ops endpoints exist; the scheduled background worker that physically purges expired-retention rows does not.
  • Promo / coupon engine — no UI, no redemption flow.
  • Broadcast composer — no UI, no audience filter, no scheduler.
  • Predictions (next online / sleep schedule) — none.
  • Backup + restore drill — runbook exists; no WAL-G wiring; no quarterly drill log.

1.4 Test inventory (snapshot)

BucketFilesNotes
tests/unit/30Pure logic; no I/O. Includes pure pyrogram / telethon importer parsers, billing grant, health checks, heatmap, Grafana validators.
tests/integration/30Real Postgres + Redis (+ MinIO + Meili in CI). Every API route hit. WS stream tested end-to-end.
tests/e2e/1Skeleton — aiogram-tests not yet exhaustive.
tests/resilience/2Dispatcher mapping table; quarantine state machine.
tests/security/1Init-data forgery, signed-callback tampering.
tests/settings/3Parameters: type / scope / hot-reload propagation / audit row.
tests/stealth/4100 % line coverage on apps/userbot/stealth/; whitelist diff.

Coverage gates are wired in CI; the measurement of every floor in CLAUDE.md (core/ ≥ 85 %, apps/ ≥ 70 %, etc.) is enforced per-package by the coverage job in .github/workflows/ci.yml.

1.5 Migrations history

#What
0001clients, audit_actors, audit_log, settings_overrides, parameters_catalog, subscriptions, etc. RLS bootstrap.
0002tracked_users, target_subscriptions, status_events (hyper), online_sessions, userbots, personas. RLS extension.
0003RLS NULL-handling fix (COALESCE(current_setting,'')).
0004billing_invoices.
0005target_daily_stats.
0006business_connections, business_messages.
0007log_routes (with AES-encrypted secret).

Pending: tracked_chats, target_messages (hyper), target_message_edits, identity_changes, username_events, target_media, predictions, scheduled_overrides table, presets, client_presets_applied.


2. Tier A — Shipping correctness

Definition of done for this tier: if we onboarded a paying client today, we would not need to hand-hold them through any documented feature. The product matches its claims. This is the bar.

A1. Coverage measurement (currently gated, not yet audited)

  • CI already enforces per-package floors via --cov-fail-under. Add a bi-weekly make coverage-report artifact uploaded to Loki / GitHub pages so the team sees trends, not just pass/fail.
  • Wire pytest-cov HTML report into the artifacts bundle of every PR.

A2. Chat tracking — the missing half of the read path

The product README and chat_tracking.md docs promise message capture from shared chats. We have business-bot capture but not general chat-tracking. This is the single largest gap.

Migrationtracked_chats, target_chat_subscriptions, target_messages (TimescaleDB hyper), target_message_edits (append-only), target_media. RLS on all client-scoped tables.

Userbot workapps/userbot/handlers/chats/{messages,edits,deletions} already exist as stubs; wire them through pyrotgfork's update filters, emit chat.message_captured, chat.message_edited, chat.message_deleted events through the existing Redis Stream pipeline.

Joiner flowapps/userbot/joiners/{base,public,invite,persona_picker, verifier} exist as stubs; bring them online behind the chat-join policy parameters (chat_join.*) already in the registry.

Bot UXapps/bot/dialogs/add_chat/ aiogram-dialog flow + consent acknowledgement step (one-time prompt per client).

APIGET /v1/chats, POST /v1/chats, DELETE /v1/chats/{id}, GET /v1/chats/{id}/messages, GET /v1/chats/{id}/search (the search endpoint stays Postgres-trigram-only until A4 lands).

Mini AppChatsPage, ChatDetailPage, ChatSearchPage.

This is one full week of careful work; can be split into 3–4 phases.

A3. Userbot warmup + chat-join policy enforcement

The picker exists; the warmup path does not. Without warmup, fresh userbots get banned within hours when they start joining popular chats.

  • New worker apps/workers/persona_warmup.py — for each newly imported userbot, schedule a humanized warmup track: profile photo upload, bio text, contact imports of low-risk accounts, lurk in 1–2 public read-only channels for ≥ N days (parameter).
  • Block joining a "real" tracked chat until the userbot's health_score
    • warmup_done_at predicate is satisfied.
  • Audit each warmup step. Owner can pause/resume per userbot.

A4. Meilisearch indexer (gated on A2)

  • apps/workers/indexer.py consumes chat.message_captured, chat.message_edited, target.identity_changed, target.username_added.
  • Writes to per-tenant indices messages:{client_id} and identities:{client_id}. The wrapper already injects client_ids = <id> on the read path.
  • Search latency budget: p99 ≤ 200 ms on Premium, ≤ 800 ms on Pro.
  • Test floor: ≥ 90 % coverage on infrastructure/search/ + indexer worker.

A5. Identity / username / photo capture

Migration: identity_changes, username_events, target_photos. Pollers apps/userbot/pollers/{identity,photo,usernames,stories} exist as stubs; bring them online with cadences from the parameters registry (polling.identity.cadence_seconds, etc.).

A6. Predictions + sleep schedule (the "intelligence" claim)

The README sells "pattern detection" and "predictions". Today: not built.

  • core/analytics/predictions.py — pure, Prophet-style or ARIMA-on-hourly baseline. Output expected_next_online_at per target.
  • core/analytics/sleep_schedule.py — pure detection over 30 days of hourly buckets. Output {sleep_start_hour, sleep_end_hour, confidence}.
  • API: GET /v1/targets/{id}/predict, GET /v1/targets/{id}/sleep.
  • Tier-gate: Premium+. Predictions card in Mini App TargetDetailPage.

A7. GDPR purger worker (ops continuity)

  • apps/workers/gdpr_purger.py — once an opt-out hits protected_users(reason='gdpr_opt_out'), the worker physically removes rows from event tables once the longest retention window on any subscriber has expired.
  • Audit each purge as gdpr.opt_out_confirmed.
  • This is ops-only (no user-facing button — Stealth Mode is the paid product). Already enforced by Phase 24-fix.

A8. Hybrid analytics engine (ADR-0010 long-tail)

The heatmap / daily endpoints exist but do not route by tier.

  • core/analytics/engine.py:resolve(query, scope_ctx) — single resolver that picks: on-read + 60 s memoize (Free) → 5-min rollup (Pro) → 1-min rollup + WS push (Premium) → continuous aggregate (Enterprise).
  • TimescaleDB continuous aggregates: target_hourly_stats, chat_daily_stats. Refresh policies tuned per tier.

A9. Backup + restore drill

  • WAL-G hourly to S3 (parameter: backup.s3.endpoint).
  • Quarterly restore-to-staging drill — manual; the runbook lives in docs/operations/runbook.md already; the drill log should land in docs/operations/drill-history.md.

3. Tier B — Feature breadth

Definition of done: every claim in the README, ADRs, and admin-panel mandate has a concrete implementation that an owner can enable from the panel without us shipping code.

B1. All delivery channels (each ≤ ½ day)

webhook + bot_message ship. The remaining six live under one folder each in apps/workers/delivery/:

ChannelNotes
tg_channelPost to a Telegram channel via the platform bot; templates from i18n/locales/<lang>/notify.yaml.
tg_topicSame as channel, with message_thread_id.
external_botOwner-supplied bot token in route config (HMAC-secret-encrypted).
emailSMTP per-route. DKIM out of scope; we just hand off to a transactional provider.
discordWebhook URL + content limits.
slackBlock kit; same posture as Discord.
file_archiveAppend-JSONL per day per route to S3; signed-URL download from Mini App.

Each is one class XChannel(DeliveryChannel) + the existing exponential backoff base + a single integration test that pins request shape.

B2. All session importers

Telethon .session (SQLite, different schema than Pyrogram), tdata (Telegram Desktop archive — zip / tar.gz / 7z / rar), TDLib td.binlog.

Each is one file under apps/userbot/auth/importers/, registered in REGISTRY. The canonical encoder/decoder pipeline already handles storage; the importers only need to parse.

B3. All payment plugins

telegram_stars ships. CryptoBot, CryptAPI, xRocket, TON Connect each:

  • One class XProvider(PaymentProvider).
  • settings_schema() for the auto-rendered admin form.
  • Webhook router mounted at /payments/{name}/webhook with per-route HMAC-SHA256 verification.
  • One integration test that pins the happy-path round-trip.

B4. Enrichment plugins (Premium+ only)

OCR (tesseract local + google_vision / openai_vision cloud), transcription (whisper_local / whisper_api / google_stt), sentiment (transformers_local / openai). Same Protocol shape; dispatcher picks the active provider per kind from parameters.

B5. Owner pages — broadcast, promos, GDPR queue

Three more Mini App pages, all generators-on-top-of-the-registry:

  • Broadcast composer — i18n variants, audience filter (tier, locale, churn risk = parameter), schedule, preview, send. Post-stats land in the bot DM.
  • Promo / coupon engine — time-bounded settings_overrides on billing.* keys for a cohort filter. Coupon redemption flow lives in the bot's billing menu.
  • GDPR ops queue — pending opt-outs / exports / deletions with one-click confirm.

B6. Web portal (ADR-0007) — full OIDC handshake

The dual-entry build emits portal.html today; the OIDC flow is not wired. Path:

  • Server: /portal/oidc/init (PKCE state generation), /portal/oidc/callback (JWKS-verified token exchange, Helios-JWT issuance into HttpOnly Secure SameSite=Lax cookie).
  • Client: portal.tsx Telegram-Login button → server handshake → hydrated React tree shared with Mini App via feature detection.
  • Security headers (HSTS, CSP, COOP) already enforced.

B7. Persona editor + persona pool view (owner Mini App)

  • Per-persona: name, bio, avatar, languages (multi), interests (tag set used by picker scoring), age plausibility window.
  • Pool view: health-score histogram, current capacity, last-success timestamp, manual cooldown button. Most already half-built in PersonasPage + UserbotsPage; finish the editing forms.

B8. Story / NFT-username surfaces

The capture pollers exist; UI is not built. One Mini App tab per kind (stories, nft_usernames) inside TargetDetailPage.

B9. Media pipeline (ADR-0009)

  • apps/workers/media_downloader.py — pulls Telegram file IDs, writes to S3 /orig slot. Per-file AES-256-GCM with HKDF-derived key from file_unique_id.
  • apps/workers/media_compactor.py — runs the per-class codec ladder (AVIF/Opus/AV1/zstd) to /opt. CPU budget capped by compression.cpu_quota_percent.
  • Mini App: media gallery in chat detail view; thumbnails come from /opt, exports from /orig.

B10. Auto-warmup-and-grow for the userbot pool

Building on A3 — once warmup works, the owner panel gets a "request N more userbots" button that triggers the operator workflow (still human-mediated for cost reasons).


4. Tier C — Scale & polish

Definition of done: the platform absorbs ten times current load with no architectural changes and one operator on call.

C1. Multi-region

  • Stateless workers behind KEDA; replica count tracks queue depth.
  • Postgres logical replication to a read replica per region; analytics endpoints route to the nearest replica.
  • Redis Cluster (or Sentinel) for HA. Streams remain single-region; cross-region delivery is opt-in via tg_channel.
  • CDN for Mini App + portal static (Cloudflare or Fastly).

C2. Cold storage

Decide between ClickHouse, S3+Parquet, and long-retention Postgres (parameter: cold_storage.backend). The decision lives in §7.

C3. SLOs and error budgets

Prometheus alert rules already exist; add explicit SLOs:

  • API p99 latency ≤ 250 ms (≥ 30-day rolling).
  • Capture-to-notify p99 ≤ 200 ms (Premium).
  • Stealth violations rate = 0 (any non-zero pages immediately).
  • Payment-webhook success rate ≥ 99.5 %.

Burn-rate alerts at 2 % / 5 % / 10 % windows.

C4. Property-based + fuzz tests

  • Hypothesis for state machines: status ingestion, scope resolver, RLS filter, signed-callback parser, init-data parser, session-string importers.
  • Atheris / boofuzz for parsers: tdata archive, OIDC tokens, payment webhooks. Run in CI on a weekly schedule, not on every PR.

C5. Soak + load tests

tests/soak/ with locust + a synthetic-event generator: 1 000 clients, 50 targets each, mixed tier distribution. Acceptance gate before any billing-affecting release.

C6. UI polish

  • Per-tier theme accents (Free / Pro / Premium / Enterprise badge colours via design tokens).
  • Empty-state illustrations (SVG only — ADR-0008).
  • Premium emoji rendering for client-facing strings (cached is_premium flag, 24-hour TTL).
  • Mobile haptics on key actions; Mini App fullscreen + safe-area insets already wired.

C7. Onboarding

Owner runbook for first deploy → first paying client. The bot's About page → docs/operations/onboarding.md link tree.


5. Cross-cutting quality

#ConcernStatusAction
Q1Coverage measurementgated, not yet trendedA1 — bi-weekly artifact.
Q2Property + fuzz testsnoneC4 — Hypothesis for parsers; weekly fuzz job.
Q3Soak / loadnoneC5 — locust; gate before billing changes.
Q4Migration safetymanual reviewForward-test on staging snapshot in CI; quarterly downgrade drill.
Q5Feature flags vs parameterscleanKeep boot-flags in config/feature_flags.yaml; runtime in registry.
Q6SLOs / error budgetsalerts exist, SLOs missingC3.
Q7Docs disciplinemkdocs strict in CIAdd ADR per requires_restart=True parameter and per new layer.
Q8Dependency hygienepip-audit wiredRenovate or Dependabot weekly.
Q9Type / lint completenessmypy strict, ruff cleanNo outstanding # type: ignore without comment.
Q10Time disciplineUTC everywhereAlready enforced by core.time; ban naive datetimes via ruff rule.

6. Risk register (updated)

#RiskLikelihoodImpactMitigation
1Stealth violation in new userbot codemediumcatastrophicAST whitelist scan in CI; 100 % stealth coverage; runtime guard; ADR-required to add an RPC.
2RLS bypass via raw SQL or wrong sessionlowcatastrophicnon-superuser helios_app; FORCE RLS; integration tests; lint rule.
3Userbot mass-banmedium-highhighhumanizer ceilings; persona discipline; warmup (A3); proxy rotation; capacity caps.
4Payment-webhook spooflowhighHMAC verification per provider; secret rotation in panel.
5Session-blob leakvery lowcatastrophicAES-256-GCM at rest, KMS in prod, never on disk plaintext, never in logs.
6Owner-account hijacklowcatastrophicTOTP; OWNER_TG_USER_ID whitelist; audit-log alerts on owner actions.
7Postgres data losslowcatastrophicWAL-G continuous backup (A9); hourly logical; quarterly drill.
8Regulatory actionmediumhighclear ToS, paid Stealth Mode + ops-only GDPR purger, jurisdictional legal review.
9Cost overrun (Redis, S3, Stars fees)mediummediumper-tier quotas as parameters; budget alerts; OpenTelemetry.
10Single-developer bus factorhighhighThis doc, ADRs, runbooks, onboarding guide.
11Chat capture exposes ourselves to detection (visible joins)mediumhighA3 warmup + ADR-0005 join ceiling (max_userbots_per_chat=2) + blacklist on first kick.
12Mini App / portal bundle bloat as features landmediumlow200 kB Brotli initial-route budget enforced; code-splitting per route.

7. Open decisions for the owner

These are real forks where the cheap answer and the right answer diverge.

  1. Hosting target — single-VPS docker-compose (cheap, fragile, slow to scale) vs managed Kubernetes (expensive, robust, harder to debug). Decide before Tier C work; affects KEDA wiring.
  2. Cold storage — ClickHouse (rich queries, ops cost) vs S3+Parquet (cheap, slower queries) vs long-retention Postgres (operationally simplest, expensive at scale). Decide before A8 finishes.
  3. Plugin marketplace — third-party plugins eventually, or always first-party? Affects security review process.
  4. Userbot supply model — bring-your-own session vs operator-managed pool. Likely both, with a tier gate. Affects A3 priority.
  5. Mini App vs Web portal feature split — feature-equal, or portal only for power users? Decide before B6.
  6. Pricing experimentation — coupons + promos (B5) only, or a real experimentation framework? Affects audit-log schema.
  7. Localization scope — beyond en / ru / uk, which markets first? Affects A1 coverage gates and onboarding doc.
  8. AI assist — anomaly detection is statistical; do we also want LLM-driven content tags / sentiment / topic clustering as Premium? Affects B4 sentiment plugin shape.

If we keep the cadence of one phase ≈ one commit, the most defensible sequence puts the largest unbuilt promise first:

#PhaseTierNotes
44Chat tracking migration + repositoryA2Schema + RLS + repositories. No userbot work yet.
45Chat-add bot dialog + consent acknowledgementA2Pure UX; lets us seed test data through real flow.
46Chat-message capture in userbot (read-only)A2Stealth-whitelist diff for messages.getHistory.
47Chats Mini App pages (list / detail / search)A2Postgres trigram search until A4.
48Persona warmup workerA3Unblocks chat joining at scale.
49Meilisearch indexer worker + per-tenant indicesA4After A2 there is data to index.
50Identity / username / photo capture pollersA5All three together — same shape.
51Predictions + sleep-schedule analyticsA6Pure; reuses existing daily stats.
52Hybrid analytics resolver (tier-aware)A8Behind a feature flag; no UX change.
53GDPR purger workerA7Ops-only; consumes existing protected_users rows.
54Delivery channel: tg_channel + file_archiveB1Two channels in one phase — both small.
55Session importer: Telethon .session (SQLite)B2Sister to phase 43.
56Payment plugin: CryptoBotB3Most-requested crypto provider.
57Owner Mini App: broadcast composerB5Read-only audience filter first.

After phase 53 we have Tier A complete: a real, paying client can connect a userbot, add chats, get notifications, see predictions, opt out, and the platform purges them when retention expires. After phase 57 the breadth claims start ringing true: an operator can stand up crypto billing and a tg_channel sink without us shipping code.


9. Anti-goals

These look tempting and would actively harm the project.

  • A second slash command. Even one. (CLAUDE.md §5.)
  • Hardcoded constants for any tunable value. A "temporary" hardcode always survives.
  • A second writer to streamed event tables. Only the ingester writes.
  • A custom diff library when diff_match_patch exists.
  • Reinventing OAuth instead of using the Telegram Login OIDC SDK.
  • A microservice split before the monolith proves it needs one.
  • An in-house ML stack before classical methods are shipped.
  • A custom CI runner before GitHub Actions is exhausted.
  • A "rewrite in Rust" branch. Python 3.12 + async + the libraries already chosen are sufficient for many years of traffic.
  • A user-facing GDPR opt-out button. The paid product is Stealth Mode; regulatory erasure stays ops-only. (Phase 24-fix.)
  • Active-visibility RPCs, ever. messages.sendReaction, stories.incrementStoryViews, messages.setTyping — never.

10. Done-when checklist per workstream

Each workstream is "done" only when every box is ticked.

Userbot

  • Stealth guard wraps every Pyrogram Client.
  • Resilience dispatcher catches and classifies every exception.
  • Presence poller emits events through Redis Streams.
  • Pyrogram session importers (file + string).
  • Telethon string importer.
  • Telethon .session file importer (B2).
  • tdata + tdlib importers (B2).
  • Warmup worker (A3).
  • Identity / username / photo / stories pollers (A5).
  • Chat join + verifier + persona-collision check (A2/A3).
  • Chat message capture handlers wired through stealth diff (A2).

Workers

  • Ingester (real, consumer-group, ack-after-handle).
  • Aggregator + daily rollups.
  • Health watchdog with owner-DM alerts.
  • Notifier (bot DMs on offline→online).
  • Status-ingestion service.
  • Logging router (events → routes → channels).
  • Synthetic-status generator (dev/test).
  • GDPR purger (A7).
  • Persona warmup (A3).
  • Media downloader + compactor (B9).
  • Meilisearch indexer (A4).

API

  • me / targets / analytics / heatmap / daily / routes / target_settings.
  • business / personas / userbots / audit / gdpr / settings.
  • WebSocket /v1/stream (Premium+).
  • chats CRUD + search (A2).
  • predict + sleep (A6).
  • export endpoints (CSV / JSON) actually streaming, not 501.

Bot

  • Single /start, buttons only.
  • Tier-gated keyboards via role filter.
  • Billing menu + Stealth Mode purchase.
  • About / Help pages.
  • Business Bot scaffolding.
  • Add-chat dialog with consent step (A2).
  • Per-target settings dialog (A8 polish).
  • Coupon redemption flow (B5).
  • Business Bot diff-aware editing (B / S7 from prior plan).

Mini App

  • 10 pages, HashRouter, TanStack Query, Zustand.
  • Live-events card (phase 42).
  • Owner panel: parameters catalog with 89 forms.
  • Chats list / detail / search pages (A2).
  • Predictions + sleep-schedule cards in TargetDetail (A6).
  • Media gallery in chat detail (B9).
  • Broadcast composer (B5).
  • Promos page (B5).
  • GDPR ops queue (B5).
  • Story / NFT-username tabs (B8).

Plugins

  • Payments: telegram_stars.
  • Delivery: bot_message, webhook.
  • Session importers: pyrogram_session, pyrogram_string, telethon_string.
  • Payments: cryptobot, cryptapi, xrocket, ton_connect (B3).
  • Delivery: tg_channel, tg_topic, external_bot, email, discord, slack, file_archive (B1).
  • Session importers: telethon_session, tdata, tdlib (B2).
  • Enrichment: ocr, transcription, sentiment — one concrete each (B4).

Observability

  • Prometheus metrics on API + workers (ports 9090–9096).
  • Grafana overview dashboard (8 panels).
  • Alert rules (5 groups).
  • Loki-shaped structlog.
  • SLO definitions + burn-rate alerts (C3).
  • OpenTelemetry traces wired end-to-end (currently spans only on API).

Security

  • AES-256-GCM at rest, KMS-abstracted master key.
  • TOTP second factor for owner.
  • Signed callbacks + nonce-replay protection.
  • HMAC initData verification.
  • HSTS/CSP/COOP headers.
  • pip-audit in CI.
  • Stealth AST whitelist scan + runtime guard.
  • OIDC portal flow with JWKS + PKCE + Ed25519 verify (B6).
  • Per-route HMAC rotation UX in admin panel.
  • Weekly fuzz job (C4).

Documentation

  • mkdocs-material site, strict mode in CI.
  • 11 ADRs indexed.
  • Runbook + observability + deployment + security pages.
  • First-deploy onboarding guide (C7).
  • User-facing help articles linked from bot About page.
  • Drill history log (A9).

Quality gates

  • Per-package coverage floors enforced in CI.
  • mypy strict + ruff format + ruff check all-clean.
  • Translation completeness check.
  • Hypothesis state-machine tests (C4).
  • Atheris fuzz parsers (C4).
  • Locust soak suite (C5).
  • Quarterly migration-downgrade drill (Q4).

11. The one-paragraph version

Helios has a strong skeleton: parameters registry (89 keys), stealth guard, RLS, hash-chained audit log, signed callbacks, OIDC-ready security headers, TOTP for owner, AES-256-GCM at rest, Telegram Stars billing, status ingestion, daily rollups, 30 integration tests passing, mkdocs docs in CI, Grafana dashboards, Prometheus alerts, and a live /v1/stream WebSocket that the Mini App already consumes. The largest remaining promise is general chat tracking (capture, search, edits, deletions, media) — it is the next 4 phases. After that, the long tail is plugin work: more delivery channels, more payment providers, more session importers, more enrichment backends. None of those introduce architectural surprises; they extend Protocols that already exist. The work between today and "production-ideal" is execution, not invention.

On this page

1. Current state — an honest audit1.1 Foundation (strong, load-bearing, well-tested)1.2 Functional surfaces (partial, working)1.3 Stubs and zeros (the long tail — Tier B / C)1.4 Test inventory (snapshot)1.5 Migrations history2. Tier A — Shipping correctnessA1. Coverage measurement (currently gated, not yet audited)A2. Chat tracking — the missing half of the read pathA3. Userbot warmup + chat-join policy enforcementA4. Meilisearch indexer (gated on A2)A5. Identity / username / photo captureA6. Predictions + sleep schedule (the "intelligence" claim)A7. GDPR purger worker (ops continuity)A8. Hybrid analytics engine (ADR-0010 long-tail)A9. Backup + restore drill3. Tier B — Feature breadthB1. All delivery channels (each ≤ ½ day)B2. All session importersB3. All payment pluginsB4. Enrichment plugins (Premium+ only)B5. Owner pages — broadcast, promos, GDPR queueB6. Web portal (ADR-0007) — full OIDC handshakeB7. Persona editor + persona pool view (owner Mini App)B8. Story / NFT-username surfacesB9. Media pipeline (ADR-0009)B10. Auto-warmup-and-grow for the userbot pool4. Tier C — Scale & polishC1. Multi-regionC2. Cold storageC3. SLOs and error budgetsC4. Property-based + fuzz testsC5. Soak + load testsC6. UI polishC7. Onboarding5. Cross-cutting quality6. Risk register (updated)7. Open decisions for the owner8. Recommended order of operations (next 14 phases)9. Anti-goals10. Done-when checklist per workstreamUserbotWorkersAPIBotMini AppPluginsObservabilitySecurityDocumentationQuality gates11. The one-paragraph version