CI Integration

Run your Checksum tests automatically in your CI pipeline. This guide covers setup for GitHub Actions and GitLab CI/CD.

---

Overview

The typical CI setup:

1. Get your API key from the Checksum web app and store it as a CI secret
2. Choose a running strategy (on PR merge, on schedule, manual trigger)
3. If tests are in a separate repo, create a Personal Access Token (PAT) with repo access
4. Add the CI configuration below

---

GitHub Actions

Basic Setup

Create .github/workflows/checksum-tests.yml:

name: Run Checksum Tests

on:
  # Choose your triggers:
  # schedule:
  #   - cron: '0 0 * * *'  # Runs every day at 00:00 UTC
  workflow_dispatch:        # Allows manual triggering

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install npm dependencies
        run: npm install

      - name: Install Playwright with dependencies
        run: npx playwright install --with-deps

      - name: Download .env from Checksum
        run: npx checksumai dotenv --download --api-key="${{ secrets.CHECKSUM_API_KEY }}"

      - name: Run Checksum tests
        run: npx checksumai test
        env:
          CHECKSUM_API_KEY: ${{ secrets.CHECKSUM_API_KEY }}
          USERNAME: ${{ secrets.USERNAME }}
          PASSWORD: ${{ secrets.PASSWORD }}
          LOGIN_URL: ${{ secrets.LOGIN_URL }}
          BASE_URL: ${{ secrets.BASE_URL }}
          CI: true

Cross-Repo Setup

If your tests live in a different repository than the one running the action, checkout the test repo using a Personal Access Token:

steps:
  - uses: actions/checkout@v4

  - name: Checkout test repo
    uses: actions/checkout@v4
    with:
      repository: org/your-test-repo
      ref: main
      token: ${{ secrets.TEST_REPO_PAT }}
      path: .

  # ... rest of the steps

Setting Up Secrets

1. Go to your GitHub repository → Settings → Secrets and variables → Actions
2. Add the following secrets:
   • CHECKSUM_API_KEY — your Checksum API key
   • USERNAME — test user credentials
   • PASSWORD — test user password
   • BASE_URL — your application URL
   • LOGIN_URL — your login page URL
   • TEST_REPO_PAT (if cross-repo) — Personal Access Token with repo permissions

Testing the Action

1. Commit the workflow file to your main branch
2. Go to Actions → select your workflow → Run Workflow
3. Monitor the run to confirm tests execute successfully

---

GitLab CI/CD

Basic Setup

Add to your .gitlab-ci.yml:

image: node:20-bookworm

stages:
  - test

run-checksum-tests:
  stage: test
  rules:
    # - if: '$CI_PIPELINE_SOURCE == "schedule"'  # Runs for scheduled pipelines
    - if: '$CI_PIPELINE_SOURCE == "web"'          # Allows manual triggering
      when: manual

  before_script:
    - npm ci
    - npx playwright install --with-deps
    - npx checksumai dotenv --download --api-key="${CHECKSUM_API_KEY}"

  script:
    - npx checksumai test

  variables:
    CHECKSUM_API_KEY: $CHECKSUM_API_KEY
    USERNAME: $USERNAME
    PASSWORD: $PASSWORD
    LOGIN_URL: $LOGIN_URL
    BASE_URL: $BASE_URL
    CI: "true"

  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/

Cross-Repo Setup

If tests are in a separate repository:

  before_script:
    - git clone "https://gitlab-ci-token:${TEST_REPO_PAT}@gitlab.com/org/your-test-repo.git" tests
    - cd tests
    - npm ci
    - npx playwright install --with-deps
    - npx checksumai dotenv --download --api-key="${CHECKSUM_API_KEY}"

Setting Up CI/CD Variables

1. Go to your GitLab project → Settings → CI/CD → Variables
2. Add the same variables as listed in the GitHub section above

Programmatic CI Integration (API)

Instead of running the CLI directly, you can trigger and manage test runs programmatically using the Checksum API. This is useful for custom CI pipelines, orchestration tools, or when you need to chain test runs with other operations.

Run Tests

# Run all tests
curl -X POST https://api.checksum.ai/public-api/v1/execution/suite \
  -H "Authorization: Bearer $CHECKSUM_API_KEY"
# Response: { "name": "job-name-12345" }

Poll for Results

# Check status
curl https://api.checksum.ai/public-api/v1/execution/status/job-name-12345 \
  -H "Authorization: Bearer $CHECKSUM_API_KEY"
# Response: { "status": "passed", "passed": 45, "failed": 2, "recovered": 1, "bug": 0 }

Get Detailed Results

curl https://api.checksum.ai/public-api/v1/test-runs/<RUN_ID>/results \
  -H "Authorization: Bearer $CHECKSUM_API_KEY"

Trigger Auto-Healing

curl -X POST https://api.checksum.ai/public-api/v1/auto-heal \
  -H "Authorization: Bearer $CHECKSUM_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "testRunId": "<RUN_ID>",
    "autoCreatePR": true,
    "branch": "main",
    "prNumber": 42
  }'
# Response: { "batchId": "...", "sessionIds": [...], "failureCount": 3, "testIds": [...] }

Poll Healing Progress

curl https://api.checksum.ai/public-api/v1/auto-heal/batch/<BATCH_ID> \
  -H "Authorization: Bearer $CHECKSUM_API_KEY"

For the complete API documentation, see API Reference.

---

Running Strategies

Strategy	When to Use	Configuration
Manual trigger	Testing and debugging	workflow_dispatch (GitHub) / when: manual (GitLab)
On schedule	Regular health checks	schedule: cron (GitHub) / $CI_PIPELINE_SOURCE == "schedule" (GitLab)
On PR merge	Verify after deployments	push: branches: [main] (GitHub) / $CI_PIPELINE_SOURCE == "push" (GitLab)

Start with manual triggers to verify your setup, then add scheduled or merge-triggered runs.