Checksum CLI

The Checksum CLI (checksumai) is the command-line tool for running your generated Playwright tests. It handles environment setup, test execution, result uploading, and integrates with the Checksum dashboard for reporting.

---

For the full repository initialization guide (creating a test repo, running npx checksumai init, folder structure, and configuration), see Repo & CLI Setup.

Installation

The CLI is included as an npm dependency in your Checksum test repository. After cloning your test repo:

npm install
npx playwright install --with-deps

---

Authentication

Download your project’s environment variables using your API key:

npx checksumai dotenv --download --api-key=<YOUR_API_KEY>

This creates a .env file with your configured environment settings (environment URL, login URL, credentials, etc.). You can find your API key in Settings → Project Settings in the web app.

---

Running Tests

Run all tests

npx checksumai test

Run specific tests by name

npx checksumai test -g "User Can Create Opportunity"

Verify setup with the example test

npx checksumai test -g "example"

The “example” test is a special quick test that validates your login process works correctly. Run it first to confirm everything is set up.

---

Test Run Modes

Mode	Description
Normal	Standard test execution — run all tests and report results
Auto-Heal	When a test fails, the CLI attempts to heal it in real time (see Auto-Recovery)

---

Environment Variables

The CLI uses environment variables from the downloaded .env file. You can override any variable:

Configuration

Your test setup is configured in checksum.config.ts inside the checksum/ directory. Key options include:

Option	Description	Default
apiKey	Your Checksum API key	Required
environments	Test environments with URLs and credentials	Required
options.useChecksumSelectors	Smart selector recovery	true
options.useChecksumAI	AI-powered action/assertion recovery	{ actions: true, assertions: false }
options.hostReports	Upload reports to Checksum dashboard	true when CI=true
options.autoHealPRs	Auto-create PRs with healed tests	true when CI=true

For the full configuration reference, see Repo & CLI Setup.

# Override via environment
BASE_URL=http://localhost:3000 npx checksumai test

# Or set in your CI environment
env:
  BASE_URL: ${{ secrets.BASE_URL }}

Explicit environment variables always take precedence over the .env file.

---

What the CLI Uploads

After each test run, the CLI uploads results to the Checksum dashboard when hostReports is enabled in checksum.config.ts. By default, this is true when CI=true (which is set automatically by most CI providers). For local runs, reports are saved locally only unless you explicitly enable hostReports.

When uploading is enabled, the CLI sends:

• Test results — pass/fail/healed status for each test
• Videos — screen recordings of test execution
• Screenshots — captured at key points and on failures
• HAR files — network traffic recordings
• Playwright traces — detailed execution traces for debugging

These artifacts are available in the web app under Test Results. See Reports and Traces for details.

---

Next Steps

• Set up CI integration with GitHub Actions or GitLab
• View test reports and traces