Generated Tests

Understanding the Playwright tests we generate.

Generated Tests

Plaintest generates real Playwright tests that you can run, export, and customize. This guide explains how they work.

Test Structure

Every generated test follows this structure:

import { test, expect } from '@playwright/test';

test.describe('Login Flow', () => {
  test.beforeEach(async ({ page }) => {
    // Setup code (navigation, login, etc.)
    await page.goto('https://myapp.com');
  });

  test('should log in with valid credentials', async ({ page }) => {
    // Test steps
    await page.getByRole('textbox', { name: 'Email' }).fill('user@example.com');
    await page.getByRole('textbox', { name: 'Password' }).fill('password123');
    await page.getByRole('button', { name: 'Sign In' }).click();

    // Assertions
    await expect(page).toHaveURL(/dashboard/);
    await expect(page.getByText('Welcome back')).toBeVisible();
  });
});

Selectors

We use Playwright's recommended selectors in this priority:

  1. Role-based: getByRole('button', { name: 'Submit' })
  2. Label-based: getByLabel('Email address')
  3. Text-based: getByText('Welcome')
  4. Test ID: getByTestId('submit-button')
  5. CSS: locator('.btn-primary') (last resort)

Role-based selectors are most resilient to UI changes.

Assertions

Tests include assertions to verify behavior:

// URL assertions
await expect(page).toHaveURL(/dashboard/);
await expect(page).toHaveURL('https://myapp.com/dashboard');

// Content assertions
await expect(page.getByText('Success')).toBeVisible();
await expect(page.getByRole('heading')).toHaveText('Dashboard');

// Element state
await expect(page.getByRole('button')).toBeEnabled();
await expect(page.getByRole('checkbox')).toBeChecked();

Handling Authentication

If your app requires login, tests include authentication:

test.beforeEach(async ({ page }) => {
  // Navigate to login
  await page.goto('https://myapp.com/login');

  // Fill credentials (from environment)
  await page.getByLabel('Email').fill(process.env.USERNAME!);
  await page.getByLabel('Password').fill(process.env.PASSWORD!);
  await page.getByRole('button', { name: 'Sign In' }).click();

  // Wait for login to complete
  await expect(page).toHaveURL(/dashboard/);
});

Credentials are stored securely and injected at runtime.

Exporting Tests

You can export any test to run locally:

  1. Click Export on a test
  2. Download the .spec.ts file
  3. Run with Playwright: npx playwright test exported-test.spec.ts

Running Exported Tests

# Install Playwright if needed
npm init playwright@latest

# Run the exported test
npx playwright test my-test.spec.ts

# Run with UI mode
npx playwright test my-test.spec.ts --ui

# Run headed (see the browser)
npx playwright test my-test.spec.ts --headed

Customizing Tests

Exported tests are standard Playwright - customize as needed:

// Add more assertions
await expect(page.locator('.error-message')).not.toBeVisible();

// Add wait conditions
await page.waitForLoadState('networkidle');

// Add screenshots
await page.screenshot({ path: 'checkout.png' });

// Add custom timeouts
await page.getByRole('button').click({ timeout: 10000 });

Test Results

When tests run, you see:

  • Status: Pass, Fail, or Skipped
  • Duration: How long the test took
  • Screenshots: Captured at key steps
  • Error Details: What went wrong (if failed)
  • Video: Recording of the test run (if enabled)

AI Analysis on Failure

When a test fails, AI analyzes the failure:

  • Verdict: Is it a test issue or an app bug?
  • Suggestion: How to fix the test
  • Auto-fix: Some tests are automatically fixed and re-run

Verdicts

  • fix_test: The test needs updating (UI changed)
  • app_bug: The app has a bug
  • inconclusive: Needs human review

Best Practices

  1. Test one thing: Each test should verify a single flow
  2. Use descriptive names: should complete checkout not test1
  3. Avoid flaky selectors: Prefer roles and labels over classes
  4. Add waits when needed: Dynamic content may need explicit waits
  5. Keep tests independent: Each test should work in isolation

Next Steps