Managing Your Deployment¶
Day-to-day Commands¶
pulumi up # deploy changes
pulumi preview # preview without deploying
pulumi stack output --json # view outputs (API URL, DB endpoint, etc.)
pulumi refresh # sync Pulumi state with actual AWS resources
Updating Hawk¶
Pull the latest code and redeploy:
Database migrations run automatically during deployment.
Updating Inspect AI or Scout¶
Use the release preparation script:
cd hawk
./scripts/ops/prepare-release.py --inspect-ai 0.3.50 # specific PyPI version
./scripts/ops/prepare-release.py --inspect-ai abc123 # specific git commit
./scripts/ops/prepare-release.py --inspect-scout 0.2.10 # update Scout
Then deploy and run smoke tests to validate.
Smoke Tests¶
Validate that a deployed environment is working end-to-end:
cd hawk
hawk login
./scripts/dev/smoke # test current stack
./scripts/dev/smoke --stack my-org # test a specific stack
./scripts/dev/smoke --warehouse # include database checks
./scripts/dev/smoke -k test_real_llm # run a specific test
Smoke tests submit real evals against real models and verify results end up in the warehouse and viewer.
After updating Inspect AI or Scout dependencies:
Tearing Down¶
Warning
Always wait for pulumi destroy to finish before running stack rm. Running stack rm first will orphan AWS resources in your account.
Troubleshooting¶
Pulumi State Issues¶
# See what Pulumi thinks exists vs what's actually in AWS
pulumi refresh
# If a resource is stuck, remove it from state (doesn't delete from AWS)
pulumi state delete '<resource-urn>'
# Import an existing AWS resource into Pulumi state
pulumi import aws:ec2/securityGroup:SecurityGroup my-sg sg-0123456789