madava/playground
1
0
Fork 0
Code Pull Requests Actions Releases Activity
Files
01ebc23e3fe77f003e1bf99f15faa39673afd655
playground/infra/README.md

22 KiB
Raw Blame History

Configuration

1. Configure the firewall at the VPS host

Public IP
217.154.51.242
Action Allowed IP Protocol Port(s) Description
Allow Any TCP 80 HTTP
Allow Any TCP 443 HTTPS
Allow Any TCP 2222 Git SSH
Allow Any TCP 2885 VPS SSH
Allow Any UDP 3478 STUN/TURN
Allow Any TCP 5349 TURN/TLS
Allow Any TCP 7881 LiveKit TCP
Allow Any UDP 50000-60000 LiveKit Media

2. Configure the DNS settings at domain registrar

Host (avaaz.ai) Type Value
@ A 217.154.51.242
www CNAME avaaz.ai
app A 217.154.51.242
api A 217.154.51.242
rtc A 217.154.51.242
git A 217.154.51.242

3. Change the SSH port from 22 to 2885

  1. Connect to the server.

    ssh username@avaaz.ai

  2. Edit the SSH configuration file.

    sudo nano /etc/ssh/sshd_config

  3. Add port 2885 to the file and comment out port 22.

    #Port 22
Port 2885

  4. Save the file and exit the editor.

    • Press Ctrl+O, then Enter to save, and Ctrl+X to exit.

  5. Restart the SSH service.

    sudo systemctl daemon-reload && sudo systemctl restart ssh.socket && sudo systemctl restart ssh.service

  6. Before closing the current session, open a new terminal window and connect to the server to verify the changes work correctly.

    ssh username@avaaz.ai         # ssh: connect to host avaaz.ai port 22: Connection timed out
ssh username@avaaz.ai -p 2885

  7. Once the connection is successful, close the original session safely.

4. Build and deploy the infrastructure

  1. Check with dig git.avaaz.ai +short wether the DNS settings have been propagated.

  2. SSH into the VPS to install Docker & docker compose.

    ssh username@avaaz.ai -p 2885

  3. Update system packages.

    sudo apt update && sudo apt upgrade -y

  4. Install dependencies for Docker’s official repo

    sudo apt install -y \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

  5. Add Docker’s official APT repo.

    sudo install -m 0755 -d /etc/apt/keyrings

curl -fsSL https://download.docker.com/linux/ubuntu/gpg \
    sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

echo \
    "deb [arch=$(dpkg --print-architecture) \
    signed-by=/etc/apt/keyrings/docker.gpg] \
    https://download.docker.com/linux/ubuntu \
    $(lsb_release -cs) stable" \
    sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update

  6. Install Docker Engine + compose plugin.

    sudo apt install -y \
    docker-ce \
    docker-ce-cli \
    containerd.io \
    docker-buildx-plugin \
    docker-compose-plugin

  7. Verify the installation.

    sudo docker --version
sudo docker compose version

  8. Create the /etc/docker/daemon.json file to avoid issues with overusing disk for log data.

    sudo nano /etc/docker/daemon.json

  9. Paste the following.

    {
    "log-driver": "local",
    "log-opts": {
        "max-size": "10m",
        "max-file": "3"
    }
}

  10. Save the file and exit the editor.

    • Press Ctrl+O, then Enter to save, and Ctrl+X to exit.

  11. Restart the docker service to apply changes.

    sudo systemctl daemon-reload
sudo systemctl restart docker

  12. Create directory for infra stack in /srv/infra.

    sudo mkdir -p /srv/infra
sudo chown -R $USER:$USER /srv/infra
cd /srv/infra

  13. Create directories for Gitea (repos, config, etc.) and Runner persistent data. Gitea runs as UID/GID 1000 by default.

    mkdir -p gitea-data gitea-runner-data

  14. Create the /srv/infra/docker-compose.yml (Caddy + Gitea + Runner) file.

    nano docker-compose.yml

  15. Paste the following.

    services:
    caddy:
        # Use the latest official Caddy image
        image: caddy:latest
        # Docker Compose automatically generates container names: <folder>_<service>_<index>
        container_name: caddy # Fixed name used by Docker engine
        # Automatically restart unless manually stopped
        restart: unless-stopped
        ports:
            # Expose HTTP (ACME + redirect)
            - "80:80"
            # Expose HTTPS/WSS (frontend, backend, LiveKit)
            - "443:443"
        volumes:
            # Mount the Caddy config file read-only
            - ./Caddyfile:/etc/caddy/Caddyfile:ro 
            # Caddy TLS certs (persistent Docker volume)
            - caddy_data:/data
            # Internal Caddy state/config
            - caddy_config:/config
        networks:
            # Attach to the shared "proxy" network
            - proxy

    gitea:
        # Official Gitea image with built-in Actions
        image: gitea/gitea:latest
        container_name: gitea # Fixed name used by Docker engine
        # Auto-restart service
        restart: unless-stopped
        environment:
            # Run Gitea as host user 1000 (prevents permission issues)
            - USER_UID=1000
            # Same for group
            - USER_GID=1000
            # Use SQLite (stored inside /data)
            - GITEA__database__DB_TYPE=sqlite3
            # Location of the SQLite DB
            - GITEA__database__PATH=/data/gitea/gitea.db
            # Custom config directory
            - GITEA_CUSTOM=/data/gitea
        volumes:
            # Bind mount instead of Docker volume because:
            # - We want repos, configs, SSH keys, and SQLite DB **visible and editable** on host
            # - Easy backups (just copy `./gitea-data`)
            # - Easy migration
            # - Avoids losing data if Docker volumes are pruned
            - ./gitea-data:/data
        networks:
            - proxy
        ports:
            # SSH for Git operations mapped to host 2222
            - "2222:22"

    gitea-runner:
        # Official Gitea Actions Runner
        image: gitea/act_runner:latest
        container_name: gitea-runner # Fixed name used by Docker engine
        restart: unless-stopped
        depends_on:
            # Runner requires Gitea to be available
            - gitea
        volumes:
            # Runner uses host Docker daemon to spin up job containers (Docker-out-of-Docker)
            - /var/run/docker.sock:/var/run/docker.sock
            # Bind mount instead of volume because:
            # - Runner identity is stored in /data/.runner
            # - Must persist across container recreations
            # - Prevents duplicated runner registrations in Gitea
            # - Easy to inspect/reset via `./gitea-runner-data/.runner`
            - ./gitea-runner-data:/data
        environment:
            # Base URL of your Gitea instance
            - GITEA_INSTANCE_URL=${GITEA_INSTANCE_URL}
            # One-time registration token
            - GITEA_RUNNER_REGISTRATION_TOKEN=${GITEA_RUNNER_REGISTRATION_TOKEN}
            # Human-readable name for the runner
            - GITEA_RUNNER_NAME=${GITEA_RUNNER_NAME}
            # Runner labels (e.g., ubuntu-latest)
            - GITEA_RUNNER_LABELS=${GITEA_RUNNER_LABELS}
            # Set container timezone to UTC for consistent logs
            - TZ=Etc/UTC
        networks:
            - proxy
        # Start runner using persisted config
        command: ["act_runner", "daemon", "--config", "/data/.runner"]

networks:
    proxy:
        # Shared network for Caddy + Gitea (+ later app stack)
        name: proxy
        # Default Docker bridge network
        driver: bridge

volumes:
    # Docker volume for Caddy TLS data (safe to keep inside Docker)
    caddy_data:
            name: caddy_data
    # Docker volume for internal Caddy configs/state
    caddy_config:
            name: caddy_config

  16. Save the file and exit the editor.

    • Press Ctrl+O, then Enter to save, and Ctrl+X to exit.

  17. Create the /srv/infra/.env file with environment variables.

    nano .env

  18. Paste the following:

    # Base URL of your Gitea instance (used by the runner to register itself
# and to send/receive workflow job information).
GITEA_INSTANCE_URL=https://git.avaaz.ai

# One-time registration token generated in:
# Gitea → Site Administration → Actions → Runners → "Generate Token"
# This MUST be filled in once, so the runner can register.
# After registration, the runner stores its identity inside ./gitea-runner-data/.runner
# and this value is no longer needed (can be left blank).
GITEA_RUNNER_REGISTRATION_TOKEN=

# Human-readable name for this runner.
# This is shown in the Gitea UI so you can distinguish multiple runners:
# Example: "vps-runner", "staging-runner", "gpu-runner"
GITEA_RUNNER_NAME=gitea-runner

# Runner labels allow workflows to choose specific runners.
# The label format is: label[:schema[:args]]
# - "ubuntu-latest" is the <label> name that workflows request using runs-on: [ "ubuntu-latest" ].
# - "docker://" is the <schema> indicating the job runs inside a separate Docker container.
# - "catthehacker/ubuntu:act-latest" is the <args>, specifying the Docker image to use for the container.
# Workflows can target this using:
#   runs-on: [ "ubuntu-latest" ]
GITEA_RUNNER_LABELS=ubuntu-latest:docker://catthehacker/ubuntu:act-latest

  19. Save the file and exit the editor.

    • Press Ctrl+O, then Enter to save, and Ctrl+X to exit.

  20. Create /srv/infra/Caddyfile to configure Caddy.

    nano Caddyfile

  21. Paste the following:

    {
    # Global Caddy options.
    #
    # auto_https on
    #   - Caddy listens on port 80 for every host (ACME + redirect).
    #   - Automatically issues HTTPS certificates.
    #   - Automatically redirects HTTP → HTTPS unless disabled.
    #
}

# ------------------------------------------------------------
# Redirect www → root domain
# ------------------------------------------------------------
www.avaaz.ai {
    # Permanent redirect to naked domain
    redir https://avaaz.ai{uri} permanent
}

# ------------------------------------------------------------
# Marketing site (optional — if frontend handles it, remove this)
# Redirect root → app
# ------------------------------------------------------------
avaaz.ai {
    # If you have a static marketing page, serve it here.
    # If not, redirect visitors to the app.
    redir https://app.avaaz.ai{uri}
}

# ------------------------------------------------------------
# Frontend (Next.js)
# Public URL: https://app.avaaz.ai
# Internal target: frontend:3000
# ------------------------------------------------------------
app.avaaz.ai {
    # Reverse-proxy HTTPS traffic to the frontend container
    reverse_proxy frontend:3000

    # Access log for debugging frontend activity
    log {
        output file /data/app-access.log
    }

    # Compression for faster delivery of JS, HTML, etc.
    encode gzip zstd
}

# ------------------------------------------------------------
# Backend (FastAPI)
# Public URL: https://api.avaaz.ai
# Internal target: backend:8000
# ------------------------------------------------------------
api.avaaz.ai {
    # Reverse-proxy all API traffic to FastAPI
    reverse_proxy backend:8000

    # Access log — useful for monitoring API traffic and debugging issues
    log {
        output file /data/api-access.log
    }

    # Enable response compression (JSON, text, etc.)
    encode gzip zstd
}

# ------------------------------------------------------------
# LiveKit (signaling only — media uses direct UDP)
# Public URL: wss://rtc.avaaz.ai
# Internal target: livekit:7880
# ------------------------------------------------------------
rtc.avaaz.ai {
    # LiveKit uses WebSocket signaling, so we reverse-proxy WS → WS
    reverse_proxy livekit:7880

    # Access log — helps diagnose WebRTC connection failures
    log {
        output file /data/rtc-access.log
    }

    # Compression not needed for WS traffic, but harmless
    encode gzip zstd
}

# ------------------------------------------------------------
# Gitea (Git server UI + HTTPS + SSH clone)
# Public URL: https://git.avaaz.ai
# Internal target: gitea:3000
# ------------------------------------------------------------
git.avaaz.ai {
    # Route all HTTPS traffic to Gitea’s web UI
    reverse_proxy gitea:3000

    # Log all Git UI requests and API access
    log {
        output file /data/git-access.log
    }

    # Compress UI responses
    encode gzip zstd
}

  22. Save the file and exit the editor.

    • Press Ctrl+O, then Enter to save, and Ctrl+X to exit.

  23. Start the stack from /srv/infra.

    sudo docker compose pull         # fetch images: caddy, gitea, act_runner
sudo docker compose up -d        # start all containers in the background

  24. Verify that the status of all the containers are Up.

    sudo docker compose ps -a

  25. Open https://git.avaaz.ai in your browser. Caddy should have already obtained a cert and you should see the Gitea installer.

  26. Configure database settings.

    • Database Type: SQLite3
    • Path: /data/gitea/gitea.db (matches GITEA__database__PATH)

  27. Configure general settings.

    • Site Title: default (Gitea: Git with a cup of tea)
    • Repository Root Path: default (/data/git/repositories)
    • LFS Root Path: default (/data/git/lfs)

  28. Configure server settings.

    • Domain: git.avaaz.ai (external HTTPS via Caddy)
    • SSH Port: 2222 (external SSH port)
    • HTTP Port: 3000 (internal HTTP port)
    • Gitea Base URL / ROOT_URL: https://git.avaaz.ai/

  29. Create the admin account (username + password + email) and finish installation.

  30. Edit Gitea /data/gitea/conf/app.ini at the host bind mount /srv/infra/gitea-data/gitea/conf/app.ini.

    nano gitea-data/gitea/conf/app.ini

  31. Add/verify the following sections.

    [server]
; Gitea serves HTTP internally (Caddy handles HTTPS externally)
PROTOCOL            = http                 
; External hostname used for links and redirects
DOMAIN              = git.avaaz.ai         
; Hostname embedded in SSH clone URLs
SSH_DOMAIN          = git.avaaz.ai         
; Internal container port Gitea listens on (Caddy reverse-proxies to this)
HTTP_PORT           = 3000                 
; Public-facing base URL (MUST be HTTPS when behind Caddy)
ROOT_URL            = https://git.avaaz.ai/
; Enable Gitea's built-in SSH server inside the container 
DISABLE_SSH         = false                
; Host-side SSH port exposed by Docker (mapped to container:22)
SSH_PORT            = 2222                 
; Container-side SSH port (always 22 inside the container)
SSH_LISTEN_PORT     = 22                   

[database]
; SQLite database file stored in bind-mounted volume
PATH                = /data/gitea/gitea.db 
; Using SQLite (sufficient for single-node small/medium setups)
DB_TYPE             = sqlite3              

[security]
; Prevent web-based reinstallation (crucial for a secured instance)
INSTALL_LOCK        = true                 
; Auto-generated on first startup; DO NOT change or delete
SECRET_KEY          =                      

[actions]
; Enable Gitea Actions (CI/CD)
ENABLED             = true                 
; Default platform to get action plugins, github for https://github.com, self for the current Gitea instance.
DEFAULT_ACTIONS_URL = github

  32. Restart Gitea to apply changes.

    sudo docker compose restart gitea

  33. Check if Actions is enabled.

    1. Log in as admin at https://git.avaaz.ai.
    2. Go to Site Administration.
    3. Look for a menu item Actions. If [actions] ENABLED = true in app.ini, there will be options related to Runners, allowing management of instance-level action runners. Otherwise, the Actions menu item in the Site Administration panel will not appear, indicating the feature is globally disabled.

  34. Get registration token to register the Gitea Actions runner and create a user account.

    1. Log in as admin at https://git.avaaz.ai.
    2. Go to Site Administration → Actions → Runners.
    3. Choose Create new Runner.
    4. Copy the Registration Token.
    5. Create a user account.

  35. Edit .env to add the token.

    nano .env

  36. Paste the Registration Token after = without spaces.

    # One-time registration token generated in:
# Gitea → Site Administration → Actions → Runners → "Generate Token"
# This MUST be filled in once, so the runner can register.
# After registration, the runner stores its identity inside ./gitea-runner-data/.runner
# and this value is no longer needed (can be left blank).
GITEA_RUNNER_REGISTRATION_TOKEN=

  37. Check for configuration changes and restart the container gitea-runner.

    sudo docker compose up -d gitea-runner

  38. Confirm that the Gitea instance URL, Runner name, and Runner labels in gitea-runner-data/.runner file are the same as the values in the .env file. Fix it using nano gitea-runner-data/.runner if different.

  39. Verify that the Runner is connected to https://git.avaaz.ai and is polling for jobs.

    sudo docker logs -f gitea-runner

  40. Generate an SSH key on laptop. Accept the defaults and optionally set a passphrase. The public key is placed in ~/.ssh/id_ed25519.pub.

    ssh-keygen -t ed25519 -C "user@avaaz.ai"

  41. Add the public key to Gitea.

    1. Log into https://git.avaaz.ai as user.
    2. Go to Profile → Settings → SSH / GPG Keys → Add Key.
    3. Paste the contents starting with ssh-ed25519 in ~/.ssh/id_ed25519.pub.
    4. Save.

  42. Test SSH remote on laptop.

    ssh -T -p 2222 git@git.avaaz.ai

  43. Type yes to tell SSH client to trust the fingerprint and press Enter. Enter the passphrase and verify the response You've successfully authenticated..., but Gitea does not provide shell access.

  44. Confirm that Gitea’s clone URLs of a repo show ssh://git@git.avaaz.ai:2222/<user>/<repo>.git.

  45. Upgrade Docker images safely.

    sudo docker compose pull         # pull newer images
sudo docker compose up -d        # recreate containers with new images

  46. Restart the whole infra stack.

    sudo docker compose restart      # restart all containers

  47. Check logs for troubleshooting.

    sudo docker logs -f caddy        # shows “obtaining certificate” or ACME errors if HTTPS fails.
sudo docker logs -f gitea        # shows DB/permissions problems, config issues, etc.
sudo docker logs -f gitea-runner # shows registration/connection/job-execution issues.

5. Validate the infrastructure

  1. Confirm that all containers caddy, gitea, and gitea-runner are Up.

    sudo docker compose ps -a

  2. Confirm that https://git.avaaz.ai shows Gitea login page with a valid TLS cert (padlock icon) when opened in a browser.

  3. Confirm the response You've successfully authenticated..., but Gitea does not provide shell access. when connecting to Gitea over SSH.

    ssh -T -p 2222 git@git.avaaz.ai

  4. Create a test repo in Gitea and confirm cloning it.

    git clone ssh://git@git.avaaz.ai:2222/<your-user>/test.git

  5. Confirm that the Actions runner gitea-runner is registered and online with status Idle.

    1. Log in as admin at https://git.avaaz.ai.
    2. Go to Site Administration → Actions → Runners.

  6. Add .gitea/workflows/test.yml to the test repo root, commit and push.

    # Workflow Name
name: Test Workflow

# Trigger on a push event to any branch
on:
    push:
        branches:
            # This means 'any branch'
            - '**'

# Define the jobs to run
jobs:
    hello:
        # Specify the runner image to use
        runs-on: [ "ubuntu-latest" ]

        # Define the steps for this job
        steps:
            - name: Run a Test Script
              run: echo "Hello from Gitea Actions!"

  7. Confirm a workflow run appears in Gitea → test repo → Actions tab and progresses from queued → in progress → success.

  8. Confirm the logs show the job picked up, container created, and the “Hello from Gitea Actions!” output.

    sudo docker logs -f gitea-runner
View Git Blame Copy Permalink