Story and Test Format

Checksum generates two files for each test: a story file (.checksum.md) that describes the test in human-readable format, and a test file (.checksum.spec.ts) that contains the actual Playwright code.

---

Story Files (.checksum.md)

Story files describe the test scenario in a structured markdown format with YAML frontmatter. They serve as the source of truth for what the test does.

Example

---
title: "User Can Create Opportunity"
checksumTestId: OFJlm
startUrl: /opportunities
envUser:
  username: [e-mail]
---

## Data Setup
**Method:** API

- POST `/api/contacts` - Create contact with name "John Doe"

## Data Cleanup
**Method:** API

- DELETE `/api/contacts/{contactId}` - Remove test contact

## Steps

1. **Navigate to Opportunities**
   - Action: Go to the opportunities page
   - Verify: Opportunities list is visible

2. **Create New Opportunity**
   - Action: Click "New Opportunity", fill in the form with contact "John Doe"
   - Verify: Form fields are populated correctly

3. **Save and Confirm**
   - Action: Click "Save"
   - Verify: New opportunity appears in the list with contact name

Frontmatter Fields

Field	Description
title	Name of the test scenario
checksumTestId	Unique identifier for the test
startUrl	The URL path where the test begins
envUser	Credentials to use (references an environment user)

Sections

• Data Setup — How to prepare test data before the test runs (API calls, database seeds)
• Data Cleanup — How to clean up after the test (delete created records)
• Steps — The test steps with actions the user takes and verifications to check

---

Test Files (.checksum.spec.ts)

Test files are standard Playwright TypeScript tests. They’re generated from the story file and can be run with the Checksum CLI or directly with Playwright.

Checksum Fixtures

Tests use Checksum fixtures (init() from @checksum-ai/runtime) which provide automatic healing, smart selector recovery, report uploading, and a variable store for sharing data between steps. However, tests are pure Playwright under the hood — you can run them with vanilla Playwright by replacing the Checksum imports with standard Playwright imports.

Page Object Model

Generated tests can also use the Page Object Model pattern for better organization:

// pages/opportunities.page.ts
import { Page, Locator } from "@playwright/test";

export class OpportunitiesPage {
  readonly newButton: Locator;
  readonly contactField: Locator;
  readonly saveButton: Locator;

  constructor(private page: Page) {
    this.newButton = page.getByRole("button", { name: "New Opportunity" });
    this.contactField = page.getByLabel("Contact");
    this.saveButton = page.getByRole("button", { name: "Save" });
  }

  async createOpportunity(contactName: string) {
    await this.newButton.click();
    await this.contactField.fill(contactName);
    await this.saveButton.click();
  }
}

// tests/opportunities.checksum.spec.ts
import { init } from "@checksum-ai/runtime";
import { OpportunitiesPage } from "../pages/opportunities.page";

const { test, defineChecksumTest, expect } = init();

test(
  defineChecksumTest("User Can Create Opportunity", "OFJlm"),
  async ({ page }) => {
    const opportunities = new OpportunitiesPage(page);
    await page.goto("/opportunities");
    await opportunities.createOpportunity("John Doe");
    await expect(page.getByText("John Doe")).toBeVisible();
  }
);

Example

import { init } from "@checksum-ai/runtime";

const { test, defineChecksumTest, login, expect, checksumAI } = init();

test(
  defineChecksumTest("User Can Create Opportunity", "OFJlm"),
  async ({ page, vs }) => {
    // Step 1: Navigate to Opportunities
    await page.goto("/opportunities");
    await expect(page.getByRole("heading", { name: "Opportunities" })).toBeVisible();

    // Step 2: Create New Opportunity
    await page.getByRole("button", { name: "New Opportunity" }).click();
    await page.getByLabel("Contact").fill("John Doe");

    // Step 3: Save and Confirm
    await page.getByRole("button", { name: "Save" }).click();
    await expect(page.getByText("John Doe")).toBeVisible();
  }
);

Key Characteristics

• Built on Playwright — Tests use Playwright under the hood via the @checksum-ai/runtime wrapper, which adds Checksum-specific features (auto-recovery, reporting, variable store).
• Grounded selectors — Checksum grounds selectors in your actual app code (using data-testid, role attributes, text content) rather than guessing.
• defineChecksumTest — Wraps each test with a name and Checksum test ID for tracking and reporting.
• Variable store (vs) — A shared store for passing data between test steps (e.g., created record IDs for cleanup).
• Web-first assertions — Uses Playwright’s built-in assertions (toBeVisible(), toHaveText()) that automatically retry.

---

Running Generated Tests

You can run generated tests locally or in CI:

# Run all tests
npx checksumai test

# Run a specific test by name
npx checksumai test -g "User Can Create Opportunity"

See Checksum CLI for more details on running tests.