Upgrades

Meeting BaaS v2 deployments are designed to be CI/CD-friendly. This guide explains how to upgrade your deployment when new images are released.

Upgrade Process Overview

The upgrade process is straightforward:

Receive Image Tag: Meeting BaaS provides new image tags periodically
Set Environment Variables: Export ENVIRON and IMAGE_TAG
Run Upgrade Script: Use baas_controller.sh to upgrade services
Verify: Check that services are running correctly

Prerequisites

Access to the latest image tag from Meeting BaaS
kubectl configured and connected to your cluster
helm installed
Deployment script (baas_controller.sh) accessible

Upgrade Steps

Step 1: Set Environment Variables

# Set environment (prod or preprod)
export ENVIRON=prod

# Set the new image tag (provided by Meeting BaaS)
export IMAGE_TAG=2026-01-27abc123def456...

Note: The same IMAGE_TAG is used for all services (API server, jobs, and bots) since they're built from the same codebase commit.

Step 2: Pull and Push Images (If Using Your Own Registry)

If you're mirroring images to your own registry:

# Login to Meeting BaaS registry
docker login rg.fr-par.scw.cloud

# Pull new images
docker pull rg.fr-par.scw.cloud/meeting-baas-prod-api-server/api-server-v2:$IMAGE_TAG
docker pull rg.fr-par.scw.cloud/meeting-baas-prod-bots/zoom-bots-v2:$IMAGE_TAG
docker pull rg.fr-par.scw.cloud/meeting-baas-prod-bots/meet-teams-bots-v2:$IMAGE_TAG

# Tag for your registry
docker tag rg.fr-par.scw.cloud/meeting-baas-prod-api-server/api-server-v2:$IMAGE_TAG \
  YOUR_REGISTRY/api-server-v2:$IMAGE_TAG
docker tag rg.fr-par.scw.cloud/meeting-baas-prod-bots/zoom-bots-v2:$IMAGE_TAG \
  YOUR_REGISTRY/zoom-bots-v2:$IMAGE_TAG
docker tag rg.fr-par.scw.cloud/meeting-baas-prod-bots/meet-teams-bots-v2:$IMAGE_TAG \
  YOUR_REGISTRY/meet-teams-bots-v2:$IMAGE_TAG

# Push to your registry
docker push YOUR_REGISTRY/api-server-v2:$IMAGE_TAG
docker push YOUR_REGISTRY/zoom-bots-v2:$IMAGE_TAG
docker push YOUR_REGISTRY/meet-teams-bots-v2:$IMAGE_TAG

Step 3: Upgrade Services

Navigate to the helm-charts directory and upgrade each service:

cd helm-charts

export ENVIRON=prod
export IMAGE_TAG=2026-01-27abc123def456...

# Upgrade API server (includes migration job)
./baas_controller.sh api-v2 upgrade

# Upgrade background jobs
./baas_controller.sh job-v2 upgrade

# Upgrade bot services
./baas_controller.sh zoom-bots-v2 upgrade
./baas_controller.sh meet-teams-bots-v2 upgrade

What Happens During Upgrade:

Migration Job: Runs automatically as a pre-upgrade hook (if migrations exist)
Rolling Update: Pods are updated one at a time (zero-downtime)
Health Checks: New pods must pass health checks before old pods are terminated

Step 4: Verify Upgrade

# Check API server pods
kubectl get pods -n services -l app.kubernetes.io/instance=api-server-v2

# Check pod images (should show new tag)
kubectl get pods -n services -l app.kubernetes.io/instance=api-server-v2 -o jsonpath='{.items[*].spec.containers[*].image}'

# Check API server logs for errors
kubectl logs -n services -l app.kubernetes.io/instance=api-server-v2 --tail=50

# Test health endpoint
curl https://api.yourcompany.com/health

# Test feature flags endpoint
curl https://api.yourcompany.com/status/features

Database Migrations

Database migrations run automatically during API server upgrades:

How It Works

Helm creates a migration Job before upgrading the deployment
The Job runs node dist/cli/migrate.js using the new image
The Job completes (or fails after retries) before pods roll out
Old Job is cleaned up automatically

Migration Job Details

Configuration (in api_server_v2_chart/values.yaml):

migration:
  enabled: true                    # Enable automatic migrations
  backoffLimit: 3                  # Retries on failure
  activeDeadlineSeconds: 300       # Timeout (5 minutes)
  ttlSecondsAfterFinished: 600     # Cleanup after (10 minutes)

Checking Migration Status

# Check migration job
kubectl get jobs -n services | grep migration

# Check migration logs
kubectl logs -n services -l app.kubernetes.io/component=migration --tail=100

# Check migration job details
kubectl describe job -n services api-server-v2-migration

Migration Failures

If migration fails:

Check Logs: Review migration job logs for errors
Fix Issues: Address database connectivity or permission issues
Retry: Delete the failed job and upgrade again:

# Delete failed migration job
kubectl delete job -n services api-server-v2-migration

# Retry upgrade
./baas_controller.sh api-v2 upgrade

CI/CD Integration

GitHub Actions Example

name: Upgrade Meeting BaaS

on:
  workflow_dispatch:
    inputs:
      image_tag:
        description: 'Image tag to deploy'
        required: true

jobs:
  upgrade:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
        with:
          submodules: recursive
      
      - name: Set up kubectl
        uses: azure/setup-kubectl@v3
      
      - name: Set up Helm
        uses: azure/setup-helm@v3
      
      - name: Configure kubectl
        run: |
          echo "${{ secrets.KUBECONFIG }}" > kubeconfig.yaml
          export KUBECONFIG=$PWD/kubeconfig.yaml
      
      - name: Upgrade services
        env:
          ENVIRON: prod
          IMAGE_TAG: ${{ github.event.inputs.image_tag }}
        run: |
          cd helm-charts
          chmod +x baas_controller.sh
          ./baas_controller.sh api-v2 upgrade
          ./baas_controller.sh job-v2 upgrade
          ./baas_controller.sh zoom-bots-v2 upgrade
          ./baas_controller.sh meet-teams-bots-v2 upgrade
      
      - name: Verify deployment
        run: |
          export KUBECONFIG=$PWD/kubeconfig.yaml
          kubectl get pods -n services
          curl -f https://api.yourcompany.com/health || exit 1

GitLab CI Example

upgrade:
  stage: deploy
  script:
    - export ENVIRON=prod
    - export IMAGE_TAG=$CI_COMMIT_TAG
    - cd helm-charts
    - chmod +x baas_controller.sh
    - ./baas_controller.sh api-v2 upgrade
    - ./baas_controller.sh job-v2 upgrade
    - ./baas_controller.sh zoom-bots-v2 upgrade
    - ./baas_controller.sh meet-teams-bots-v2 upgrade
  only:
    - tags

Rollback Procedure

If an upgrade causes issues, you can rollback:

Rollback API Server

# List revisions
helm history api-server-v2 -n services

# Rollback to previous revision
helm rollback api-server-v2 -n services

# Or rollback to specific revision
helm rollback api-server-v2 3 -n services

Rollback Other Services

# Rollback jobs
helm rollback job-v2 -n services

# Rollback bots
helm rollback zoom-bots-v2 -n services
helm rollback meet-teams-bots-v2 -n services

Note: Rolling back may require database migrations to be compatible. Check migration compatibility before rolling back.

Upgrade Best Practices

Pre-Upgrade Checklist

Review release notes for breaking changes
Backup database (if possible)
Test upgrade in staging environment first
Verify image tag is correct
Check that all required secrets are still valid
Ensure sufficient cluster resources

During Upgrade

Monitor pod status: kubectl get pods -n services -w
Watch API server logs: kubectl logs -n services -l app.kubernetes.io/instance=api-server-v2 -f
Check migration job: kubectl get jobs -n services | grep migration
Test health endpoint: curl https://api.yourcompany.com/health

Post-Upgrade Verification

All pods are running: kubectl get pods -n services
Health endpoint responds: curl https://api.yourcompany.com/health
Feature flags are correct: curl https://api.yourcompany.com/status/features
Test bot creation: Create a test bot via API
Check logs for errors: Review recent logs for any issues
Verify background jobs: Check that CronJobs are running

Updating Helm Charts

When Helm charts are updated:

# Update submodule to latest
cd helm-charts
git pull origin main
cd ..

# Review changes
git diff helm-charts

# Test upgrade (dry-run)
cd helm-charts
helm upgrade --dry-run api-server-v2 ./api_server_v2_chart \
  -f ../environment-overrides/api_server_v2_chart/prod.yaml \
  --set image.tag=$IMAGE_TAG

# If everything looks good, upgrade
./baas_controller.sh api-v2 upgrade

Version Compatibility

API Server and Jobs: Must use the same image tag (same codebase)
Bot Images: Can be upgraded independently, but should match API server version for compatibility
Video Device Plugin: Uses fixed version (1.0.0), rarely updated

Troubleshooting Upgrades

Migration Job Stuck

# Check job status
kubectl describe job -n services api-server-v2-migration

# Check logs
kubectl logs -n services -l app.kubernetes.io/component=migration

# Delete and retry
kubectl delete job -n services api-server-v2-migration
./baas_controller.sh api-v2 upgrade

Pods Not Starting

# Check pod status
kubectl describe pod -n services -l app.kubernetes.io/instance=api-server-v2

# Check events
kubectl get events -n services --sort-by='.lastTimestamp'

# Common issues:
# - Image pull errors (check registry access)
# - Resource constraints (check node resources)
# - Configuration errors (check logs)

Health Checks Failing

# Check liveness probe
kubectl describe pod -n services -l app.kubernetes.io/instance=api-server-v2 | grep Liveness

# Check readiness probe
kubectl describe pod -n services -l app.kubernetes.io/instance=api-server-v2 | grep Readiness

# Test endpoints manually
kubectl port-forward -n services svc/api-server-v2 3001:3001
curl http://localhost:3001/health
curl http://localhost:3001/liveness

Next Steps

Troubleshooting - Common issues and solutions
Review Deployment Guide for initial setup