Skip to content

Taryn Deployment Coordinator

Cross-system deployment via Ansible playbooks with health checks and Gritters notifications.

Overview

Taryn manages deployments across The Smithy's infrastructure:

  • TheSmithy (Evennia game server)
  • Spyder (integration services)
  • Endless Cascade (web frontend)
  • Grit (utility services)
  • Frankenstein engine (interactive narrative engine on dedicated EC2)

Key Components

Component Location Role
Playbooks taryn/ansible/*.yml Deployment workflows per target system
Roles taryn/ansible/roles/ Reusable Ansible roles for each service
Group vars taryn/ansible/group_vars/all.yml Shared variables (repo paths, Gritters config, SSH keys)
Inventory taryn/ansible/inventory/hosts Target host definitions per environment
Health monitor taryn/src/services/coworker_health_monitor.py Cross-agent health polling via integration endpoints
Health service taryn/src/services/coworker_health_service.py Fleet health snapshots via MCP client
Fleet tripwires taryn/scripts/fleet-tripwires.sh Daily read-only sweep for fleet rot (stale MRs, red pipelines, deploy lag, pod health)

Deploy Targets

Repo GitLab Path Deploy Type
TheSmithy the-smithy1/TheSmithy git pull + evennia reload
Spyder the-smithy1/spyder git pull + service restart
Endless Cascade the-smithy1/core/endless-cascade Binary archive promote
Grit (lore) the-smithy1/grit/lore-repo git pull (content only)
Frankenstein engine the-smithy1/interactive-narrative-engine Ansible role frankenstein-engine (install + rolling deploy over SSM)

Playbook Inventory

Playbook Purpose
deploy-to-dev.yml Deploy current branch to the dev environment
deploy-to-staging.yml Deploy to the staging environment
deploy-to-prod.yml Deploy TheSmithy main to production via SSM Automation runbook (roland-prod-deploy-mud) with Slack approval gate — see Production Deployment
promote-to-staging.yml Merge develop → main across all product repos, bump version, deploy to staging, run health checks, sync develop version forward, and notify via Gritters
promote-to-prod.yml ⚠️ DEPRECATED — SSH-over-port-22 path no longer works for prod after the 2026-05-08 prod hardening. Kept as reference for version-bump and release-tag logic only
frankenstein-install.yml First-time provisioning of the Frankenstein engine EC2 (after Terraform stands it up)
frankenstein-deploy.yml Rolling deploy for Frankenstein: git pull + migrate-on-change + evennia reload + health check
bootstrap-postgres.yml One-time idempotent bootstrap of Taryn's local Postgres user + database
ship-mr.yml Ship a single MR end-to-end: merge → staging deploy → verify gate → prod deploy → Gritters notification
seed-local-claude-oauth.yml Seed Claude OAuth credentials for the local CI runner (Docker-based)

Ansible Roles

Role Path Responsibility
smithy-deploy taryn/ansible/roles/smithy-deploy/ TheSmithy deploy: git pull, dependency install, evennia reload
spyder-deploy taryn/ansible/roles/spyder-deploy/ Spyder deploy: git pull, dependency install, service restart
cascade-deploy taryn/ansible/roles/cascade-deploy/ Endless Cascade deploy: binary archive promote
frankenstein-engine taryn/ansible/roles/frankenstein-engine/ Full lifecycle for the Frankenstein engine EC2 — install, rolling deploy, secrets management
version-bump taryn/ansible/roles/version-bump/ Reads/increments the VERSION file during promotions

Frankenstein Engine Role Structure

The frankenstein-engine role is the most complex, managing a dedicated EC2 instance via SSM (port 22 is closed):

ansible/roles/frankenstein-engine/
├── defaults/main.yml       # Paths, ports, hostnames, secret IDs
├── handlers/main.yml       # Service reload handlers
├── tasks/
│   ├── install.yml         # First-time provisioning
│   ├── main.yml            # Rolling deploy (git pull + migrate + reload + health check)
│   └── secrets.yml         # Renders systemd EnvironmentFile from AWS Secrets Manager
└── templates/
    ├── frankenstein.service.j2    # systemd unit (Type=forking + PIDFile + ExecStop)
    ├── frankenstein.env.j2        # EnvironmentFile (secrets read once per deploy)
    └── nginx-proxy.conf.j2       # Nginx reverse proxy config

Connection uses the community.aws.aws_ssm plugin — no SSH key or bastion required. AWS credentials must have SSM + Secrets Manager read access on frankenstein/*.

Workflow

  1. Version bump triggered (manual or CI)
  2. Taryn runs the appropriate playbook
  3. Health checks confirm successful deployment
  4. Gritters notification posted to team channel

Staging Promotion Flow

promote-to-staging.yml performs the full staging cycle:

  1. Merge develop → main across all four product repos (TheSmithy, Spyder, Endless Cascade, Grit)
  2. Bump the VERSION file
  3. Deploy each service to staging via its respective role
  4. Run health checks
  5. Sync develop version forward
  6. Post a Gritters notification to the team
source ~/Smithy/.env
ansible-playbook -i ansible/inventory/hosts ansible/promote-to-staging.yml

Production Deployment (SSM)

After the 2026-05-08 prod hardening, production EC2 has port 22 firewalled. SSH-over-Ansible no longer works for prod. Production deployments now go through the SSM Automation runbook roland-prod-deploy-mud via deploy-to-prod.yml, which includes a Slack approval gate.

source ~/Smithy/.env
ansible-playbook -i ansible/inventory/hosts ansible/deploy-to-prod.yml

Key details:

  • Requesting identity: any of the five agents (pearl, arturo, chisel, minh, taryn) — all hold the agent-prod-runbook-request IAM permission. Default is arturo; override via -e requesting_agent=pearl.
  • Approval identity: human admin (WILL_AWS_*) or autonomous-approval agent (TARYN_APPROVER_AWS_*). Approval flows via Slack (#prod-approvals, channel C0A4LHCUM9N) or AWS Console/CLI. The playbook does not auto-approve — the human gate is the whole point.
  • Poll budget: 30 min default (60 polls × 30s). Override with -e poll_max_attempts=120 (60 min) or -e poll_interval_seconds=60.

Note: promote-to-prod.yml is deprecated. It is kept only as reference for version-bump and release-tag logic. Do not use it for production deployments.

Health Checks

Taryn runs two complementary health-monitoring systems:

Cross-Agent Health Monitor (coworker_health_monitor.py)

Polls each conducted-set agent's GET /api/v1/health/integrations endpoint in-cluster. Designed for state-transition alerts, not poll spam — a healthy→unhealthy transition emits one alert; continued unhealthy is silent; unhealthy→healthy emits a recovery message.

  • Conducted set: Minh, Pearl, Arturo, Roland, Chisel (configurable via TARYN_COWORKER_HEALTH_AGENTS)
  • Auth: per-agent <AGENT>_INTERNAL_API_TOKEN for Bearer auth
  • Scheduling: managed by bot_schedulers.py, gated by TARYN_COWORKER_HEALTH_ENABLED

Fleet Health Service (coworker_health_service.py)

Consumes the CoworkersMCPClient to produce point-in-time health snapshots of each coworker and shared infrastructure (Postgres, Redis, Neo4j). Powers /taryn standup (fleet table) and /taryn watch <coworker> (deep-dive view).

Fleet Tripwires (fleet-tripwires.sh)

A read-only daily sweep (launchd com.glassumbrella.fleet-tripwires, 07:23) that catches fleet rot within a day:

  • Open MRs older than 7 days (drafts excluded)
  • Latest main pipeline red on a blocking, non-deploy job
  • Deploy lag — running pod older than the last src/-touching main commit by >48 hours
  • Pod health — not Ready, restarts >5, or CrashLoopBackOff
  • Missing deployment for a repo expected to run
scripts/fleet-tripwires.sh              # sweep + alert on trips
scripts/fleet-tripwires.sh --dry-run    # print trips, POST nothing
scripts/fleet-tripwires.sh --test-alert # one test Gritters post

Trips post one consolidated Gritters alert as @taryn; a clean sweep posts nothing (daily silence is the signal).

Gritters Notifications

Deployment playbooks notify the team via the Gritters API using the Ansible uri module. Configuration is in taryn/ansible/group_vars/all.yml:

gritters:
  api_url: https://app.glassumbrella.io/api/gritters/post
  api_key: "{{ lookup('env', 'SPYDER_API_KEY') }}"
  originator: "@minh"
  recipient: collab

Notifications are sent from the following playbooks:

Playbook Notification
promote-to-staging.yml Staging promotion complete with version number
ship-mr.yml MR shipped end-to-end announcement
fleet-tripwires.sh Consolidated alert on fleet rot (only when trips are detected)

Configuration

Deployment targets and credentials are managed through Ansible inventory and vault files.

  • Shared variables: taryn/ansible/group_vars/all.yml — repo paths, Gritters config, SSH keys, branch model, service names, Evennia settings
  • Environment-specific overrides: staging.yml and prod.yml in group_vars/
  • Inventory: taryn/ansible/inventory/hosts
  • Frankenstein-specific: taryn/ansible/group_vars/frankenstein.yml (SSM connection params, instance ID)

See taryn/CLAUDE.md for local development setup and the full environment variable reference.