Case study / 01
Relay
Human-in-the-loop orchestration for student work, planning, and collaboration.
Relay turns lecture material, deadlines, and meeting transcripts into reviewable actions across Learn, Plan, and Collaborate. Models interpret messy content; typed application logic resolves identities, validates proposals, and schedules study sessions with CP-SAT. Users approve exact payloads before external writes. A runtime contract separates those workflow decisions from execution, while persisted artifacts record what Notion, Google Calendar, and GitHub actually created—including partial success.
- Next.js
- FastAPI
- PostgreSQL
- Pydantic
- OR-Tools
AI Interprets
Models structure content and intent; they do not directly write to providers.
Typed Proposals
Validated model output becomes reviewable application state.
Human Approval
Users review and approve exact payloads before external writes.
Reliable Execution
Approved snapshots cross a typed runtime boundary with idempotency keys.
System architecture
Model proposes. Code resolves. Human approves. Runtime executes.
Relay implements local execution and an HTTP runtime adapter. Live compatibility with the separate Agent Runtime service needs integration validation; normal CI uses local and mocked execution.
Engineering decisions
| Decision | Why | Tradeoff |
|---|---|---|
| LLM interpretation, deterministic resolution. | Messy inputs benefit from models; execution-critical fields need explicit rules and validation. | More application logic than directly executing generated output. |
| Typed proposals and approval before writes. | Users can correct actions and references before external systems change. | More interaction and state; execution is not fully autonomous. |
| Separate Relay from Agent Runtime. | Product-specific workflow decisions stay separate from reusable execution reliability. | A typed contract and cross-service compatibility checks. |
Under the hood
Orchestration / State
- Pydantic proposals and PostgreSQL workflow state
- Ambiguous owners retain candidate matches for review
- Immutable approval snapshots and audit events
Plan / Scheduling
- Normalize Notion tasks, deadlines, and preferences
- Load Google Calendar busy intervals
- OR-Tools CP-SAT places sessions within constraints
Execution / Integrations
- RuntimeClient with local and HTTP implementations
- Notion, Google Calendar, and GitHub connectors
- Per-action outcomes and ExternalArtifact records
Technical deep dives
Learn · Plan · Collaborate
View full diagramThree workflows share typed proposals and approval gates. PLAN uses CP-SAT placement, not model-selected time slots.
Human-in-the-Loop Action Lifecycle
Interpretation is a proposal. Approval authorizes the write.
Input
Notes, tasks, or a meeting transcript.
Interpret
Schema-validated model output where needed.
Resolve
Domain rules resolve identities and constraints; ambiguity stays visible.
Propose
Typed actions enter reviewable workflow state.
Approve
Owner-scoped approval freezes the exact payload.
Execute
RuntimeClient receives the snapshot and idempotency key.
Record
Provider outcomes and external artifacts return to Relay state.
Partial success stays visible. Successful artifacts survive sibling failures. Retries reuse approved payloads and keys; provider APIs do not guarantee exactly-once side effects.
System checks
- Ambiguous owners remain unresolved until corrected.
- Unapproved actions cannot bypass the execution gate.
- Scheduling excludes busy time and reports infeasibility.
- Recovery reuses approved payloads and idempotency keys.
- Partial failures preserve successful external artifacts.
Next
Add a live end-to-end contract check between Relay’s HTTP adapter and Agent Runtime, covering approved execution, uncertain responses, and per-action artifact recovery. Current CI uses local runtime behavior and mocked providers.