Repository and CLI Setup

Before using Checksum, you need to set up your test repository and connect it to Checksum. This guide walks you through creating or configuring a repository for your Playwright tests, installing the Checksum CLI, and verifying that everything works.

Code Repository vs Tests Repository

Checksum works with up to two repositories:

• Tests repository — This is where your Playwright tests live. Checksum writes to this repository by opening pull requests with generated or healed tests. Every project must have a tests repository connected.

• Code repository (optional) — This is your application’s source code. Checksum reads this repository to understand your app’s routes, components, and interactions, which significantly improves test detection accuracy. Checksum never writes to the code repository.

If your tests live in the same repository as your application code, simply connect the same repository for both.

Connecting a code repository is optional but recommended. Projects with a connected code repo benefit from faster, more accurate test detection.

---

Setting Up Your Test Repository

There are two ways to set up a test repository depending on your project structure.

Option A: New Dedicated Test Repository (Recommended for Getting Started)

Create a new repository dedicated to your Checksum tests. This is the simplest way to get started and keeps your test suite cleanly separated from your application code.

1. Create a new repository on GitHub or GitLab
2. Clone it locally
3. Initialize the project and install dependencies:

npm init -y
npm install @checksum-ai/runtime playwright
npx checksumai init

Option B: Add to an Existing Repository

If you want your tests to live alongside your application code, we recommend creating a dedicated checksum/ directory with its own package.json. This keeps Checksum’s test dependencies separate from your production dependencies.

mkdir checksum
cd checksum
npm init -y
npm install @checksum-ai/runtime playwright
npx checksumai init

In both cases, the key steps are the same: install @checksum-ai/runtime and playwright, then run npx checksumai init to scaffold the project.

---

What Init Creates

Running npx checksumai init generates the following project structure:

checksum/
├── checksum.config.ts        # Checksum configuration
├── playwright.config.ts      # Playwright config
├── tsconfig.json              # TypeScript config
├── login.ts                   # Login helper
├── .gitignore
└── tests/
    ├── example.checksum.spec.ts  # Example test to verify setup
    └── ...                       # Generated tests go here

• checksum.config.ts — Central configuration file for your Checksum project (see below for a full reference)
• playwright.config.ts — Standard Playwright configuration, pre-configured to work with Checksum
• login.ts — A helper module for authenticating test users during test runs
• tests/example.checksum.spec.ts — A sample test that validates your setup is working correctly

---

Configuration (checksum.config.ts)

The checksum.config.ts file is the central configuration for your Checksum project. Below is a complete reference of all available fields.

apiKey

Your Checksum API key. You can find this in Settings → Project Settings in the Checksum web app.

apiKey: process.env.CHECKSUM_API_KEY

runMode

Controls how tests behave when they encounter failures.

Value	Description
RunMode.Normal	Tests run normally and fail on assertion errors (default)
RunMode.Heal	Tests attempt to self-heal when encountering failures

environments

Environment settings configured here must match what you’ve set in the Checksum web app. See Environment Configuration for how to manage environments, credentials, and custom variables through the UI.

An array of environment configurations. Each environment describes a target application instance that tests can run against.

Field	Type	Description
name	string	Environment name — must match the name configured in the Checksum web app
baseURL	string	Base URL of the application under test (e.g., https://staging.myapp.com)
loginURL	string	URL of the login page
default	boolean	Whether this is the default environment
users	array	Test user credentials for this environment

Each entry in the users array has the following fields:

Field	Type	Description
role	string	A descriptive role name (e.g., "admin", "viewer")
username	string	Login username or email
password	string	Login password
default	boolean	Whether this is the default user for the environment

options

Fine-grained controls for test execution behavior.

Option	Type	Default	Description

| useChecksumSelectors | boolean | true | Enable Smart Selector recovery. | | useChecksumAI | object | { actions: true, assertions: false } | AI-powered recovery. actions: true lets the AI retry failed interactions. assertions: false means assertion failures are not auto-recovered. |

| useMockData | boolean | false | Use mock API data during test runs. | | hostReports | boolean | true when CI=true | Upload test reports and traces to the Checksum dashboard. |

| autoHealPRs | boolean | true when CI=true | Automatically create PRs with healed tests in CI. |

Complete Example

import { ChecksumConfig, RunMode } from "@checksum-ai/runtime";

const config: ChecksumConfig = {
  apiKey: process.env.CHECKSUM_API_KEY,
  runMode: RunMode.Normal,
  environments: [
    {
      name: "staging",
      baseURL: "https://staging.myapp.com",
      loginURL: "https://staging.myapp.com/login",
      default: true,
      users: [
        {
          role: "admin",
          username: process.env.ADMIN_USERNAME,
          password: process.env.ADMIN_PASSWORD,
          default: true,
        },
        {
          role: "viewer",
          username: process.env.VIEWER_USERNAME,
          password: process.env.VIEWER_PASSWORD,
          default: false,
        },
      ],
    },
  ],
  options: {
    useChecksumSelectors: true,
    useChecksumAI: {
      actions: true,
      assertions: false,
    },
    useMockData: false,
    hostReports: true,
    autoHealPRs: true,
  },
};

export default config;

---

Verifying Your Setup

After initializing your project, verify that everything is configured correctly by running the example test.

1. Install Playwright browsers:

npx playwright install --with-deps

2. Download your environment variables from Checksum:

npx checksumai dotenv --download --api-key=<YOUR_API_KEY>

3. Run the example test:

npx checksumai test -g "example"

The example test validates that your login process and basic configuration are working. If it passes, your setup is complete and you’re ready to start detecting and generating tests.

---

Next Steps

• Connect your Git provider to enable test delivery via pull requests
• Detect test flows from your application