playground/docs/architecture.md

System Architecture

Below is a summary of the Production VPS and Development Laptop architectures. Both environments use Docker containers for consistency, with near-identical stacks where practical.

flowchart LR
%% Client
    A(Browser / PWA)
    Y(iOS App / Android App)
    
    subgraph User
        A
        Y
    end

%% LLM / Realtime
    B(OpenAI Realtime API)
    Z(Gemini Live API)

    subgraph Large Language Model
        B
        Z
    end

%% Server-side
    C(Caddy)
    I(Gitea + Actions + Repositories)
    J(Gitea Runner)

    D(Next.js Frontend)
    E(FastAPI Backend + Agent Runtime)
    G(LiveKit Server)
    H[(PostgreSQL + pgvector)]

%% Client ↔ VPS
    A <-- https://www.avaaz.ai --> C
    A <-- https://app.avaaz.ai --> C
    A & Y <-- https://api.avaaz.ai --> C
    A & Y <-- wss://rtc.avaaz.ai --> C
    A & Y <-- "udp://rtc.avaaz.ai:50000-60000 (WebRTC Media)" --> G

%% Caddy ↔ App
    C <-- "http://frontend:3000 (app)" --> D
    C <-- "http://backend:8000 (api)" --> E
    C <-- "ws://livekit:7880 (WebRTC signaling)" --> G
    C <-- "http://gitea:3000 (git)" --> I

%% App internal
    D <-- "http://backend:8000" --> E
    E <-- "postgresql://postgres:5432" --> H
    E <-- "http://livekit:7880 (control)" --> G
    E <-- "Agent joins via WebRTC" --> G

%% Agent ↔ LLM
    E <-- "WSS/WebRTC (realtime)" --> B
    E <-- "WSS (streaming)" --> Z

%% CI/CD
    I <-- "CI/CD triggers" --> J

    subgraph VPS
        subgraph Infra
            C
            I
            J
        end
        subgraph App
            D
            E
            G
            H
        end        
    end

%% Development Environment
    L(VS Code + Git + Docker)
    M(Local Docker Compose)
    N(Local Browser)
    O(Local Frontend)
    P(Local Backend)
    Q[(Local Postgres)]
    R(Local LiveKit)

    L <-- "https://git.avaaz.ai/...git" --> C
    L <-- "ssh://git@git.avaaz.ai:2222/..." --> I
    L -- "docker compose up" --> M

    M -- "Build & Run" --> O & P & Q & R

    N <-- HTTP --> O & P
    N <-- WebRTC --> R

    O <-- HTTP --> P
    P <-- SQL --> Q
    P <-- HTTP/WebRTC --> R
    P <-- WSS/WebRTC --> B
    P <-- WSS --> Z

    subgraph Development Laptop
        L
        M
        N
        subgraph Local App
            O
            P
            Q
            R
        end
    end

1. Production VPS

1.1 Components

Infra Stack

Docker Compose: ./infra/docker-compose.yml.

Container	Description
caddy	Caddy – Reverse proxy with automatic HTTPS (TLS termination via Let’s Encrypt).
gitea	Gitea + Actions – Git server using SQLite. Automated CI/CD workflows.
gitea-runner	Gitea Runner – Executes CI/CD jobs defined in Gitea Actions workflows.

App Stack

Docker Compose: ./app/docker-compose.yml.

Container	Description
frontend	Next.js Frontend – SPA/PWA interface served from a Node.js-based Next.js server.
backend	FastAPI + Uvicorn Backend – API, auth, business logic, LiveKit orchestration, agent.
postgres	PostgreSQL + pgvector – Persistent relational database with vector search.
livekit	LiveKit Server – WebRTC signaling plus UDP media for real-time audio and data.

The backend uses several Python packages such as UV, Ruff, FastAPI, FastAPI Users, FastAPI-pagination, FastStream, FastMCP, Pydantic, PydanticAI, Pydantic-settings, LiveKit Agent, Google Gemini Live API, OpenAI Realtime API, SQLAlchemy, Alembic, docling, Gunicorn, Uvicorn[standard], Pyright, Pytest, Hypothesis, and Httpx to deliver the services.

1.2 Network

• All containers join a shared proxy Docker network.
• Caddy can route to any service by container name.
• App services communicate internally:
  • Frontend ↔ Backend
  • Backend ↔ Postgres
  • Backend ↔ LiveKit
  • Backend (agent) ↔ LiveKit & external LLM realtime APIs

1.3 Public DNS Records

Hostname	Record Type	Target	Purpose
www.avaaz.ai	CNAME	avaaz.ai	Marketing / landing site
avaaz.ai	A	217.154.51.242	Root domain
app.avaaz.ai	A	217.154.51.242	Next.js frontend (SPA/PWA)
api.avaaz.ai	A	217.154.51.242	FastAPI backend
rtc.avaaz.ai	A	217.154.51.242	LiveKit signaling + media
git.avaaz.ai	A	217.154.51.242	Gitea (HTTPS + SSH)

1.4 Public Inbound Firewall Ports & Protocols

Port	Protocol	Purpose
80	TCP	HTTP, ACME HTTP-01 challenge
443	TCP	HTTPS, WSS (frontend, backend, LiveKit)
2222	TCP	Git SSH via Gitea
2885	TCP	VPS SSH access
3478	UDP	STUN/TURN
5349	TCP	TURN over TLS
7881	TCP	LiveKit TCP fallback
50000–60000	UDP	LiveKit WebRTC media

1.5 Routing

Caddy

Caddy routes traffic from public ports 80 and 443 to internal services.

• https://www.avaaz.ai → http://frontend:3000
• https://app.avaaz.ai → http://frontend:3000
• https://api.avaaz.ai → http://backend:8000
• wss://rtc.avaaz.ai → ws://livekit:7880
• https://git.avaaz.ai → http://gitea:3000

Internal Container Network

• frontend → http://backend:8000
• backend → postgres://postgres:5432
• backend → http://livekit:7880 (control)
• backend → ws://livekit:7880 (signaling)
• backend → udp://livekit:50000-60000 (media)
• gitea-runner → /var/run/docker.sock (Docker API on host)

Outgoing

• backend → https://api.openai.com/v1/realtime/sessions
• backend → wss://api.openai.com/v1/realtime?model=gpt-realtime
• backend → wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent

1.6 Functional Layers

Data Layer

Infra:

• SQLite (Gitea)

  • Gitea stores Git metadata (users, repos, issues, Actions metadata) in /data/gitea/gitea.db.
  • This is a file-backed SQLite database inside a persistent Docker volume.
  • Repository contents are stored under /data/git/, also volume-backed.

• Gitea Runner State

  • Gitea Actions runner stores its registration information and job metadata under /data/.runner.

App:

• PostgreSQL with pgvector

  • Primary relational database for users, lessons, transcripts, embeddings, and conversational context.
  • Hosted in the postgres container with a persistent Docker volume.
  • Managed via SQLAlchemy and Alembic migrations in the backend.

• LiveKit Ephemeral State

  • Room metadata, participant states, and signaling information persist in memory within the livekit container.
  • LiveKit’s SFU media buffers and room state are not persisted across restarts.

Control Layer

Infra:

• Caddy

  • TLS termination (Let’s Encrypt).
  • Reverse proxy and routing for all public domains.
  • ACME certificate renewal.

• Gitea

  • Git hosting, pull/clone over SSH and HTTPS.
  • CI/CD orchestration via Actions and internal APIs.

• Gitea Runner

  • Executes workflows and controls the Docker engine via /var/run/docker.sock.

App:

• FastAPI Backend

  • Authentication and authorization (/auth/login, /auth/refresh, /auth/me).
  • REST APIs for lessons, progress, documents, and file handling.
  • LiveKit session management (room mapping /sessions/default, token minting /sessions/default/token, agent configuration).
  • Calls out to OpenAI Realtime and Gemini Live APIs for AI-driven conversational behavior.

• LiveKit Server

  • Manages room signaling, participant permissions, and session state.
  • Exposes HTTP control endpoint for room and participant management.

Media Layer

App:

• User Audio Path

  • Browser/mobile → LiveKit:
    • WSS signaling via rtc.avaaz.ai → Caddy → livekit:7880.
    • UDP audio and data channels via rtc.avaaz.ai:50000–60000 directly to LiveKit on the VPS.
  • WebRTC handles ICE, STUN/TURN, jitter buffers, and Opus audio encoding.

• AI Agent Audio Path

  • The agent logic inside the backend uses LiveKit Agent SDK to join rooms as a participant.
  • Agent → LiveKit:
    • WS signaling over the internal Docker network (ws://livekit:7880).
    • UDP audio transport as part of its WebRTC session.
  • Agent → LLM realtime API:
    • Secure WSS/WebRTC connection to OpenAI Realtime or Gemini Live.
  • The agent transcribes, processes, and generates audio responses, publishing them into the LiveKit room so the user hears natural speech.

1.7 CI/CD Pipeline

Production CI/CD is handled by Gitea Actions running on the VPS. The gitea-runner container has access to the host Docker daemon and is responsible for both validation and deployment:

• .gitea/workflows/ci.yml – Continuous Integration (branch/PR validation, no deployment).
• .gitea/workflows/cd.yml – Continuous Deployment (tag-based releases to production).

Build Phase (CI Workflow: ci.yml)

Triggers

• push to:
  • feature/**
  • bugfix/**
• pull_request targeting main.

Runner & Environment

• Runs on the self-hosted runner labeled linux_amd64.
• Checks out the relevant branch or PR commit from the avaaz-app repository into the runner’s workspace.

Steps

1. Checkout code
   Uses actions/checkout@v4 to fetch the branch or PR head commit.

2. Report triggering context
   Logs the event type (push or pull_request) and branches:

   • For push: the source branch (e.g., feature/foo).
   • For pull_request: source and target (main).

3. Static analysis & tests

   • Run linters, type checkers, and unit tests for backend and frontend.
   • Ensure the application code compiles/builds.

4. Build Docker images for CI

   • Build images (e.g., frontend:ci and backend:ci) to validate Dockerfiles and build chain.
   • These images are tagged for CI only and not used for production.

5. Cleanup CI images

   • Remove CI-tagged images at the end of the job (even on failure) to prevent disk usage from accumulating.

Outcome

• A green CI result on a branch/PR signals that:
  • The code compiles/builds.
  • Static checks and tests pass.
  • Docker images can be built successfully.
• CI does not modify the production stack and does not depend on tags.

Deploy Phase (CD Workflow: cd.yml)

Triggers

• Creation of a Git tag matching v* that points to a commit on the main branch in the avaaz-app repository.

Runner & Environment

• Runs on the same linux_amd64 self-hosted runner.
• Checks out the exact commit referenced by the tag.

Steps

1. Checkout tagged commit

   • Uses actions/checkout@v4 with ref: ${{ gitea.ref }} to check out the tagged commit.

2. Tag validation

   • Fetches origin/main.
   • Verifies that the tag commit is an ancestor of origin/main (i.e., the tag points to code that has been merged into main).
   • Fails the deployment if the commit is not in main’s history.

3. Build & publish release

   • Builds production Docker images for frontend, backend, LiveKit, etc., tagged with the version (e.g., v0.1.0).
   • Applies database migrations (e.g., via Alembic) if required.

4. Restart production stack

   • Restarts or recreates the app stack containers using the newly built/tagged images (e.g., via docker compose -f docker-compose.yml up -d).

5. Health & readiness checks

   • Probes key endpoints with curl -f, such as:
     • https://app.avaaz.ai
     • https://api.avaaz.ai/health
     • wss://rtc.avaaz.ai (signaling-level check)
   • If checks fail, marks the deployment as failed and automatically rolls back to previous images.

Outcome

• Only tagged releases whose commits are on the main branch are deployed.
• Deployment is explicit (tag-based), separated from CI validation.

1.8 Typical Workflows

User Login

1. Browser loads the frontend from https://app.avaaz.ai.
2. Frontend submits credentials to POST https://api.avaaz.ai/auth/login.
3. Backend validates credentials and returns:
   • A short-lived JWT access token
   • A long-lived opaque refresh token
   • A minimal user profile for immediate UI hydration
4. Frontend stores tokens appropriately (access token in memory; refresh token in secure storage or an httpOnly cookie).

Load Persistent Session

1. Frontend calls GET https://api.avaaz.ai/sessions/default.
2. Backend retrieves or creates the user’s persistent conversational session, which encapsulates:
   • Long-running conversation state
   • Lesson and progress context
   • Historical summary for LLM context initialization
3. Backend prepares the session’s LLM context so that the agent can join with continuity.

Join the Live Conversation Session

1. Frontend requests a LiveKit access token via POST https://api.avaaz.ai/sessions/default/token.
2. Backend generates a new LiveKit token (short-lived, room-scoped), containing:
   • Identity
   • Publish/subscribe permissions
   • Expiration (affecting initial join)
   • Room ID corresponding to the session
3. Frontend connects to the LiveKit server:
   • WSS for signaling
   • UDP/SCTP for low-latency audio and file transfer
4. If the user disconnects, the frontend requests a new LiveKit token before rejoining, ensuring seamless continuity.

Conversation with AI Agent

1. Backend configures the session’s AI agent using:
   • Historical summary
   • Current lesson state
   • Language settings and mode (lesson, mock exam, free talk)
2. The agent joins the same LiveKit room as a participant.
3. All media flows through LiveKit:
   • User → audio → LiveKit → Agent
   • Agent → LLM realtime API → synthesized audio → LiveKit → User
4. The agent guides the user verbally: continuing lessons, revisiting material, running mock exams, or free conversation.

The user experiences this as a continuous, ongoing session with seamless reconnection and state persistence.

1.9 Hardware

Class	Description
system	Standard PC (i440FX + PIIX, 1996)
bus	Motherboard
memory	96KiB BIOS
processor	AMD EPYC-Milan Processor
memory	8GiB System Memory
bridge	440FX - 82441FX PMC [Natoma]
bridge	82371SB PIIX3 ISA [Natoma/Triton II]
communication	PnP device PNP0501
input	PnP device PNP0303
input	PnP device PNP0f13
storage	PnP device PNP0700
system	PnP device PNP0b00
storage	82371SB PIIX3 IDE [Natoma/Triton II]
bus	82371SB PIIX3 USB [Natoma/Triton II]
bus	UHCI Host Controller
input	QEMU USB Tablet
bridge	82371AB/EB/MB PIIX4 ACPI
display	QXL paravirtual graphic card
generic	Virtio RNG
storage	Virtio block device
disk	257GB Virtual I/O device
volume	238GiB EXT4 volume
volume	4095KiB BIOS Boot partition
volume	105MiB Windows FAT volume
volume	913MiB EXT4 volume
network	Virtio network device
network	Ethernet interface
input	Power Button
input	AT Translated Set 2 keyboard
input	VirtualPS/2 VMware VMMouse

2. Development Laptop

2.1 Components

App Stack (local Docker)

• frontend (Next.js SPA)
• backend (FastAPI)
• postgres (PostgreSQL + pgvector)
• livekit (local LiveKit Server)

No Caddy is deployed locally; the browser talks directly to the mapped container ports on localhost.

2.2 Network

• All services run as Docker containers on a shared Docker network.
• Selected ports are published to localhost for direct access from the browser and local tools.
• No public domains are used in development; everything is addressed via http://localhost/....

2.3 Domains & IP Addresses

Local development uses:

• http://localhost:3000 → frontend (Next.js dev/server container)
• http://localhost:8000 → backend API (FastAPI)
  • Example auth/session endpoints:
    • POST http://localhost:8000/auth/login
    • GET http://localhost:8000/sessions/default
    • POST http://localhost:8000/sessions/default/token
• ws://localhost:7880 → LiveKit signaling (local LiveKit server)
• udp://localhost:50000–60000 → LiveKit/WebRTC media

No /etc/hosts changes or TLS certificates are required; localhost acts as a secure origin for WebRTC.

2.4 Ports & Protocols

Port	Protocol	Purpose
3000	TCP	Frontend (Next.js)
8000	TCP	Backend API (FastAPI)
5432	TCP	Postgres + pgvector
7880	TCP	LiveKit HTTP + WS signaling
50000–60000	UDP	LiveKit WebRTC media (audio, data)

2.5 Routing

No local Caddy or reverse proxy layer is used; routing is direct via published ports.

Internal Container Routing (Docker network)

• Backend → Postgres: postgres://postgres:5432
• Backend → LiveKit: http://livekit:7880
• Frontend (server-side) → Backend: http://backend:8000

Browser → Containers (via localhost)

• Browser → Frontend: http://localhost:3000
• Browser → Backend API: http://localhost:8000

Outgoing (from Backend)

• backend → https://api.openai.com/v1/realtime/sessions
• backend → wss://api.openai.com/v1/realtime?model=gpt-realtime
• backend → wss://generativelanguage.googleapis.com/ws/google.ai.generativelanguage.v1beta.GenerativeService.BidiGenerateContent

These calls mirror production agent behavior while pointing to the same cloud LLM realtime endpoints.

2.6 Functional Layers

Data Layer

• Local Postgres instance mirrors the production schema (including pgvector).
• Database migrations are applied via backend tooling (e.g., Alembic) to keep schema in sync.

Control Layer

• Backend runs full application logic locally:
  • Authentication and authorization
  • Lesson and progress APIs
  • LiveKit session management (/sessions/default, /sessions/default/token) and agent control
• Frontend integrates against the same API surface as production, only with localhost URLs.

Media Layer

• Local LiveKit instance handles:
  • WS/HTTP signaling on port 7880
  • WebRTC media (audio + data channels) on UDP 50000–60000
• Agent traffic mirrors production logic:
  • LiveKit ↔ Backend ↔ LLM realtime APIs (OpenAI / Gemini).

2.7 Typical Workflows

Developer Pushes Code

1. Developer pushes to git.avaaz.ai over HTTPS/SSL or SSH.
2. CI runs automatically (linting, tests, build validation). No deployment occurs.
3. When a release is ready, the developer creates a version tag (v*) on a commit in main.
4. CD triggers: validates the tag, rebuilds from the tagged commit, deploys updated containers, then performs post-deploy health checks.

App Development

• Start the stack: docker compose -f docker-compose.dev.yml up -d
• Open the app in the browser: http://localhost:3000
• Frontend calls the local backend for:
  • POST http://localhost:8000/auth/login
  • GET http://localhost:8000//sessions/default
  • POST http://localhost:8000//sessions/default/token

API Testing

• Health check: curl http://localhost:8000/health

• Auth and session testing:

  curl -X POST http://localhost:8000/auth/login \
       -H "Content-Type: application/json" \
       -d '{"email": "user@example.com", "password": "password"}'
  
  curl http://localhost:8000/sessions/default \
       -H "Authorization: Bearer <access_token>"
  

LiveKit Testing

• Frontend connects to LiveKit via:
  • Signaling: ws://localhost:7880
  • WebRTC media: udp://localhost:50000–60000
• Backend issues local LiveKit tokens via POST http://localhost:8000//sessions/default/token, then connects the AI agent to the local room.

2.8 Hardware

Class	Description
system	HP Laptop 14-em0xxx
bus	8B27 motherboard bus
memory	128KiB BIOS
processor	AMD Ryzen 3 7320U
memory	256KiB L1 cache
memory	2MiB L2 cache
memory	4MiB L3 cache
memory	8GiB System Memory
bridge	Family 17h-19h PCIe Root Complex
generic	Family 17h-19h IOMMU
storage	SK hynix BC901 HFS256GE SSD
disk	256GB NVMe disk
volume	299MiB Windows FAT volume
volume	238GiB EXT4 volume
network	RTL8852BE PCIe 802.11ax Wi-Fi
display	Mendocino integrated graphics
multimedia	Rembrandt Radeon High Definition Audio
generic	Family 19h PSP/CCP
bus	AMD xHCI Host Controller
input	Logitech M705 Mouse
input	Logitech K370s/K375s Keyboard
multimedia	Jabra SPEAK 510 USB
multimedia	Logitech Webcam C925e
communication	Bluetooth Radio
multimedia	HP True Vision HD Camera
bus	FCH SMBus Controller
bridge	FCH LPC Bridge
power	AE03041 Battery
input	Power Button
input	Lid Switch
input	HP WMI Hotkeys
input	AT Translated Set 2 Keyboard
input	Video Bus
input	SYNA32D9:00 06CB:CE17 Mouse
input	SYNA32D9:00 06CB:CE17 Touchpad
network	Ethernet Interface