Upgrade the CosmicAC stack
Upgrade a running CosmicAC deployment to a new release tag.
Upgrade a running CosmicAC deployment to a new release tag. One tag covers every image in the Compose stack.
In-cluster job images upgrade separately
task update upgrades only the Compose platform stack, not the GPU and inference images that run as jobs in your Kubernetes cluster. To change those, update their tags in the Kubernetes worker's config/common.json.
Prerequisites
You need the following before you start:
- A running CosmicAC deployment. See Deploy CosmicAC.
- The deployment repository on your host.
- The new release
TAGyou want to deploy.
Steps
Set the new tag
Set TAG=<new-tag> in .env, or pass it on the command line in the next step.
Run the update
Pull the new images, add any new configuration keys, and recreate the stack at the new tag:
task updateTo skip editing .env, pass the tag on the command line instead:
task update TAG=<new-tag>To upgrade one service rather than the whole stack, name it in SERVICES:
task update SERVICES="cosmicac-app-node"Optional: Change a configuration value
Do this only if you want a different value for one of the new keys. task update sets each new key to the release default and leaves your existing values unchanged.
Find the key in the service's config/common.json, then set it in .env. This example sets pagination.limit to 100 for cosmicac-app-node:
APPNODE_COMMON_CONFIG__pagination__limit=100Apply the change, then restart the service:
task apply-app-node-common-config
task restart SERVICES="cosmicac-app-node"See Task deployment commands for each service's prefix and apply command, and for the other variable forms, including the ones that set values in config/facs/.
task update restarts the services it upgrades, so this restart applies only to your .env change. Any later change you make in .env needs both commands, because the apply command writes the config file without starting containers.
Verify the deployment
Confirm every service is up:
task psFor the full health checks, see Verify the deployment.
If a worker won't connect, refresh the shared runtime keys, then recreate the worker so it picks them up:
task wire
task restart SERVICES="cosmicac-wrk-server-k8s-nvidia"Roll back
To roll back, deploy the previous tag. A rollback preserves your state. Back up first so you have a snapshot:
task backup
task update TAG=<previous-tag>