Istari Deploy Agent

Run ID	Type	Outcome	Started	Duration	Chart	Phases	Logs
Loading...

End-to-End Release Validation

Every Istari platform release goes through this pipeline before shipping to customers. Two full deployment cycles (fresh install + upgrade) run autonomously overnight on the CS sandbox.

Plan Creation

~5 min (you)

Fresh Install

14 phases, ~10h

Review

~5 min (you)

Upgrade Test

8 phases, ~8h

Review

~5 min (you)

Ship to Customers

~10 min report

Timeline

Both runs fit in one overnight + one business day. ~25 minutes of engineer time total.

Evening

/release-test-coverage <name> in Claude Code. Queries JFrog, creates plans, uploads to S3, starts EC2.

You

Overnight

Agent runs fresh install: teardown, EKS, terraform, Zitadel, helm, SCS, MCP, SpiceDB, smoke tests. Slack updates arrive throughout.

Agent

Morning

Review fresh install results in Slack. If passed, trigger upgrade plan.

You

Daytime

Agent runs upgrade: pre-checks, breaking changes analysis, helm upgrade, post-verify, smoke tests.

Agent

Afternoon

Review upgrade results. Report to engineering. Both must pass before release ships.

You

Swimlane View

Who does what, and what moves where.

You

/release-test-coverage → Confirm versions → Plans created → Review fresh → Trigger upgrade → Review upgrade → Report

Deploy Agent

Pick up plan → 14 phases (fresh) → Finalize → 8 phases (upgrade) → Finalize → Stop EC2

Outputs

Slack updates S3 logs Dashboard Git results Terraform package Release assessment

Fresh Install Phases (14)

Full deployment from scratch. Creates EKS cluster, RDS, all infrastructure, identity, platform, and runs end-to-end smoke tests.

Infrastructure

Teardown

Destroy previous deployment (terraform destroy, namespace wipe)

Opus

Env Setup

VPC discovery, subnet validation, AWS provider config

Opus

EKS Apply

EKS cluster, node groups, OIDC provider in BYOVPC

Opus

Pull Secret

JFrog registry credentials, image pull test

Sonnet

Full Terraform

RDS PostgreSQL, S3 buckets, IAM roles, KMS, security groups

Opus

ALB + DNS

Application Load Balancer, ACM cert, Route53 records

Sonnet

Identity & Secrets

Configurator

Zitadel org, admin user, OIDC clients (frontend, registry, MCP)

Opus

Secrets

K8s secrets for frontend, registry, fileservice, OIDC creds

Sonnet

08u

SCS Enablement

S3 inbox/outbox, DB schema, Zitadel client, SCS secrets

Opus

Platform

Helm Install

Platform chart with all subcharts enabled

Opus

Verify Pods

All pods running, readiness probes, no crash loops

Sonnet

MCP Enable

MCP service, health endpoint, AI chat connectivity

Sonnet

SpiceDB

Connection pool hardening, dispatch, permission latency

Opus

Validation

Final Validate

HTTPS endpoints, TLS certs, auth flow end-to-end

Sonnet

Smoke Tests

API tests (httpx + PAT auth) + browser tests (Playwright)

Python

Upgrade Phases (8)

Deploys the previous release, then upgrades to the new one. Skips infrastructure phases (03, 05, 06) since the cluster already exists.

Assessment

U01

Pre-Upgrade Check

Discover existing cluster state, endpoints, domains

Opus

U02

Pre-Upgrade Infra

Verify infrastructure is ready for upgrade

Opus

U03

Breaking Changes

Analyze release notes, plan migration steps

Opus

Migration

U04

Configurator Upgrade

Update Zitadel clients, add new OIDC configs

Opus

U05

SCS Enablement

Activate new SCS features, preserve existing state

Opus

U06

Platform Upgrade

Helm upgrade with schema migrations, new subcharts

Opus

Validation

U07

Post-Upgrade Verify

All services restart cleanly, data survives

Sonnet

U08

Smoke Tests

API + browser tests against upgraded platform

Python

Safety Architecture

Guardrails that prevent the agent from causing damage or running away with spend.

👁

Opus 4.7 Advisor

Independent model reviews every terraform apply and helm upgrade before execution. Can BLOCK the run.

💰

Per-Phase Budget Caps

Each phase has a max LLM spend ($1-$8). Prevents runaway token consumption on any single phase.

⏱

16-Hour Deadline

Warns at 15h, shuts down at 16h. Operator can reply "continue" in Slack to extend.

🔒

Permissions Boundary

IAM boundary denies IAM mutation, Organizations access, and tag-scoped resource creation.

📈

AWS Spend Guardrail

Monthly budget alert ($200). cost-check.sh runs before every deployment.

💬

Slack Check-in

9am ET EventBridge asks if the run should continue. Operator oversight without waking up overnight.

Finalization Flow

After all phases complete (or fail), the agent produces deliverables and notifies.

Determine

Classify outcome → success.json / failure.json / advisor_block.json

Generate

Recommendation Worklog Release Assessment Version Check

Deliver

Sync logs to S3 Move plan (running → completed) Update dashboard manifest Terraform package Git commit results

Notify

Slack: result + AI summary → Listen 15 min for replies → Stop EC2

What does the deploy agent test?

Before every platform release ships to customers, the deploy agent runs two full deployment cycles on the CS sandbox (AWS account 669640508343). Each cycle exercises the complete Istari infrastructure and application stack.

Fresh Install

Tears down everything and rebuilds from scratch: EKS cluster, RDS, ALB, DNS, Zitadel, platform helm chart, SCS, MCP, SpiceDB. Validates the full "day one" customer experience.

Upgrade Path

Deploys the previous release, then upgrades to the new one. Tests helm upgrade, database migrations, subchart activation, and verifies existing data survives.

Smoke Tests

API tests (httpx + PAT auth) and browser tests (Playwright) run against the live deployment. Login, create system, upload file, verify workflows.

Report

Results posted to Slack, logs archived in S3, artifacts committed to git. Both a fresh install and upgrade must pass before the release ships.

What each phase validates

Phase	Name	What it tests
01	Teardown	Clean destruction of previous deployment (terraform destroy, namespace wipe)
02	Env Setup	VPC discovery, subnet validation, AWS provider configuration, existing resource detection
03	EKS Apply	EKS cluster creation in BYOVPC, node groups, OIDC provider (fresh install only)
04	Pull Secret	JFrog registry pull secret creation, image pull verification
05	Full Apply	Complete terraform: RDS PostgreSQL, S3 buckets, IAM roles, security groups, KMS (fresh only)
06	ALB+DNS	Application Load Balancer, ACM certificate, Route53 DNS records (fresh only)
07	Configurator	Zitadel identity: organization, admin user, OIDC clients for frontend/registry/MCP
08	Secrets	Kubernetes secrets for frontend, registry service, fileservice, OIDC credentials
08u	SCS	Secure Connection Service: S3 inbox/outbox, database schema, Zitadel client, secrets
09	Platform	Helm install/upgrade of the platform chart with all subcharts enabled
10	Verify	All pods running, readiness probes passing, no crash loops, resource utilization normal
11	MCP	MCP service enablement, health endpoint, AI chat connectivity
12	SpiceDB	Connection pool hardening, dispatch authority, permission resolution latency
13	Validate	HTTPS endpoints responding, TLS certificates valid, authentication flow working
14	Smoke	API smoke tests (PAT auth, CRUD ops) + browser tests (Playwright login, upload, workflow)

How a run works

An engineer runs /release-test-coverage <name> in Claude Code. The skill queries JFrog for the latest gated chart versions, creates a plan JSON, pushes to git, uploads to S3, and starts the EC2 instance. The agent picks up the plan on boot and runs through all phases autonomously (8-10 hours for fresh, 6-8 for upgrade). An independent Opus 4.7 advisor reviews every terraform apply and helm upgrade before execution. Results go to Slack, S3, and this dashboard.