Table of contents

Using The Playwright Test Runner With Checkly

Checkly natively supports running browser checks using the Playwright Test Runner, allowing you to write tests and use assertions using the popular testing framework. Read more on how to utilise Playwright Test best in the official documentation.

Playwright Test Runner elevates your monitoring and debugging experience by providing a number of neat functionalities:

• Detailed trace files with step-by-step information on your test cases.
• Video recordings of browser sessions
• The expect() function comes with built-in awaiting.
• Lots of web-first assertions like toContainText(), toHaveURL()
• High-level locators like getByTitle, getByRole
• Visual regression testing with the toMatchSnapshot() assertion.
• Independent nested test cases that make your Checkly check even more powerful.

Playwright Test is available from runtime 2022.02 onwards.

This is the list of Playwright Test Runner features that are currently supported. We will update it as more features become supported.

One of the key benefits of using Playwright Test is that you can split your check into multiple independent test cases, and group them using the test.describe function. Your Checkly check will fail if at least one of the test cases fails.

import { test } from '@playwright/test';

test.describe('two tests', () => {
 test('one', async ({ page }) => {
   // ...
 })

 test('two', async ({ page }) => {
   // ...
 })
})

const { test } = require('@playwright/test')

test.describe('two tests', () => {
 test('one', async ({ page }) => {
   // ...
 })

 test('two', async ({ page }) => {
   // ...
 })
})

Playwright Test Runner offers hook functions such as test.afterEach() and test.beforeEach() that run before or after individual test cases or test.afterAll() and test.beforeAll() that run before or after all tests have started/finished.

You can find more information on available methods in the official documentation.

When a @playwright/test test case fails, Checkly will record and make its trace files available via the UI. You can download the trace files for manual inspection or view them directly with trace.playwright.dev.

Using the Playwright Trace Viewer you can effortlessly view your test, skip back and forth between actions, view snapshots and metadata, and more. This makes it very easy to inspect individual traces and debug failing tests.

When running tests from the editor page, trace files are always available for download and preview, regardless of whether the check is passing or failing. For scheduled check runs traces are only preserved when the check failed.

When a @playwright/test test case fails, Checkly will record a video for each page navigation and make it available in the UI. It is a great tool to get a first look of the actions and their outcome to quickly identify what failed, and to visualize regressions.

Here’s an example of a Playwright Test script that fails, and provides a video of the test sequence.

When running tests from the editor page, video files are always available for download and preview, regardless of whether the check is passing or failing. For scheduled check runs videos are only preserved when the check failed.

If you are structuring your test codebase following the PageObject pattern, you can use the Checkly CLI out of the box. Just make sure that:

• the folder you initialize your CLI in when building your project sits above your test spec files and their dependencies
• your testMatch is pointing to the path(s) where your test specs live

To see one way this can look like, see our example repository.

We are gradually rolling out support for global configuration options for the Playwright Test Runner. This allows you to configure your Playwright tests in a single place, instead of having to repeat the same configuration for each test file.

There are three things you should be aware of:

1. You can only use a subset of the Playwright config options. See the supported configuration options section for more information.
2. You need to add the playwrightConfig section to your checkly.config.ts file, nested under the browserChecks section.
3. We explicitly do not read from the existing playwright.config.ts or playwright.config.js file in your project. This is to avoid any confusion about which config file is used to run your tests and to prevent any unexpected behaviour.

If you have an existing playwright.config.ts or playwright.config.js file in your project that you want to import, you can simply run the sync-playwright

npx checkly sync-playwright

This command will add currently supported Playwright config option to your checkly.config.ts file.

import { defineConfig } from 'checkly'

export default defineConfig({
  projectName: 'Website Monitoring',
  logicalId: 'website-monitoring-1',
  repoUrl: 'https://github.com/acme/website',
  checks: {
    checkMatch: '**/*.check.js',
    playwrightConfig: {         // note the extra playwrightConfig section
      timeout: 1234,
      use: {
        baseURL: 'https://www.checklyhq.com',
        isMobile: true,
      },
      expect: {
        toHaveScreenshot: {
          maxDiffPixels: 10,
        }
      }
    },
    browserChecks: {
      testMatch: '**/*.spec.js',
    },
  },
  cli: {
    runLocation: 'eu-west-1',
    privateRunLocation: 'private-dc1'
  }
})

const { defineConfig } = require('checkly')

const config = defineConfig({
  projectName: 'Website Monitoring',
  logicalId: 'website-monitoring-1',
  repoUrl: 'https://github.com/acme/website',
  checks: {
    checkMatch: '**/*.check.js',
    playwrightConfig: {           // note the extra playwrightConfig section
      timeout: 1234,
      use: {
        baseURL: 'https://www.checklyhq.com',
        isMobile: true,
      },
      expect: {
        toHaveScreenshot: {
          maxDiffPixels: 10,
        }
      }
    },
    browserChecks: {
      testMatch: '**/*.spec.js',
    },
  },
  cli: {
    runLocation: 'eu-west-1',
    privateRunLocation: 'private-dc1'
  }
})

module.exports = config;

We currently support the following configuration options. We will update this list as more options become supported.

Getting started with Browser Monitoring Scraping & asserting on page elements

Last updated on November 6, 2024. You can contribute to this documentation by editing this page on Github