detections/_docs/04_deploy/ci_cd_pipeline.md

CI/CD Pipeline

Platform

GitHub Actions (default recommendation; adaptable to Azure Pipelines).

Pipeline Stages

Stage	Trigger	Steps	Quality Gate
Lint	Every push	black --check, Cython syntax check	Zero errors
Unit Test	Every push	pytest tests/ -v --csv=report.csv	All pass
Security Scan	Every push	pip-audit, Trivy image scan	Zero critical/high CVEs
Build	PR merge to dev	Build detections-cpu and detections-gpu images, tag with git SHA	Build succeeds
E2E Test	After build	docker compose -f e2e/docker-compose.test.yml up --abort-on-container-exit	All e2e tests pass
Push	After e2e	Push images to container registry	Push succeeds
Deploy Staging	After push	Deploy to staging via scripts/deploy.sh	Health check passes
Deploy Production	Manual approval	Deploy to production via scripts/deploy.sh	Health check passes

Workflow Definition (GitHub Actions)

name: CI/CD

on:
  push:
    branches: [dev, main]
  pull_request:
    branches: [dev]

env:
  REGISTRY: ${{ secrets.REGISTRY }}

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install black
      - run: black --check src/ tests/

  test:
    runs-on: ubuntu-latest
    needs: lint
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install -r requirements.txt
      - run: python setup.py build_ext --inplace
      - run: cd src && pytest ../tests/ -v

  security:
    runs-on: ubuntu-latest
    needs: lint
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install pip-audit
      - run: pip-audit -r requirements.txt

  build:
    runs-on: ubuntu-latest
    needs: [test, security]
    if: github.event_name == 'push'
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ secrets.REGISTRY_USER }}
          password: ${{ secrets.REGISTRY_PASSWORD }}
      - uses: docker/build-push-action@v5
        with:
          context: .
          file: Dockerfile
          push: true
          tags: ${{ env.REGISTRY }}/azaion/detections-cpu:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

  e2e:
    runs-on: ubuntu-latest
    needs: build
    steps:
      - uses: actions/checkout@v4
      - run: |
          cd e2e
          COMPOSE_PROFILES=cpu docker compose -f docker-compose.test.yml up \
            --build --abort-on-container-exit --exit-code-from e2e-runner

  deploy-staging:
    runs-on: ubuntu-latest
    needs: e2e
    if: github.ref == 'refs/heads/dev'
    environment: staging
    steps:
      - uses: actions/checkout@v4
      - run: |
          IMAGE_TAG=${{ github.sha }} \
          DEPLOY_HOST=${{ secrets.STAGING_HOST }} \
          DEPLOY_USER=${{ secrets.STAGING_USER }} \
          bash scripts/deploy.sh

  deploy-production:
    runs-on: ubuntu-latest
    needs: e2e
    if: github.ref == 'refs/heads/main'
    environment:
      name: production
      url: ${{ secrets.PRODUCTION_URL }}
    steps:
      - uses: actions/checkout@v4
      - run: |
          IMAGE_TAG=${{ github.sha }} \
          DEPLOY_HOST=${{ secrets.PRODUCTION_HOST }} \
          DEPLOY_USER=${{ secrets.PRODUCTION_USER }} \
          bash scripts/deploy.sh

Caching Strategy

Cache Type	Scope	Tool
Python dependencies	Per requirements.txt hash	actions/cache + pip cache dir
Docker layers	Per Dockerfile hash	BuildKit GHA cache
Cython compiled modules	Per src/ hash	actions/cache

Parallelization

• test and security jobs run in parallel after lint
• build waits for both test and security
• GPU image build can be added as a parallel job to CPU build

Notifications

Event	Channel
Build failure	GitHub PR status check (blocks merge)
Security scan failure	GitHub PR status check + team notification
Deployment success/failure	Deployment environment status