Getting started with Playwright

Automated testing gives programmers the ability to run more tests in less time while exploring their program’s functionality. With so many test automation frameworks on the scene, it can be difficult to figure out which is best to use for your specific application.

This article takes a look at Playwright, Microsoft’s new test automation framework, and compares it to some of the other major frameworks. Read on to discover how Playwright holds up against veteran frameworks and when it makes the most sense to use.

What is Playwright?

Playwright is a new test automation framework designed to test web apps from end to end. The framework is Microsoft’s offering to the test automation space. Although new, Playwright packs a punch out of the gate with speedy testing that goes where most other frameworks do not.

Playwright works with some of the most popular programming languages, including JavaScript, Python, Java, and C#. It also supports Chromium, Firefox, and WebKit, providing a wide range of cross-browser testing capabilities.

The framework is designed to be fast and reliable, and runs on Windows, Linux, and macOS operating systems. Further, the tool is fully open-source and only requires NodeJS as a prerequisite.

Playwright has the ability to run in CI environments as well as locally on your machine. The framework has everything you need to write and perform UI tests completely offline.

A Playwright test example

This article assumes you’re already familiar with the Playwright installation process. But in case you need a refresher, check out Mailosaur’s Playwright setup guide.

To get a glimpse of Playwright in action with Mailosaur, check out this example:

const playwright = require('playwright');
const assert = require('chai').assert;
const Mailosaur = require('mailosaur');

(async () => {
  const apiKey = 'YOUR_API_KEY';
  const mailosaur = new Mailosaur(apiKey);

  // Save our email domain information so that we know
  // what test email address to use.
  const serverId = 'abcd1234';
  const serverDomain = 'abcd1234.mailosaur.net';

  // Launch a new browser window and create a new browser context.
  const browser = await playwright.chromium.launch();
  const context = await browser.newContext();

  // Create a new page and navigate to the password reset URL.
  const page = await context.newPage();    
  await page.goto('https://example.mailosaur.com/password-reset');

  // Save a screenshot of how the page looks, to analyse later if needed.
  await page.screenshot({ path: `password-request.png` });

  // Make up a random email address for this test.
  const randomString = new Date().getTime();
  const testEmail = `${randomString}@${serverDomain}`;

  // Request a password reset for our test email address.
  await page.fill('#email', testEmail);
  await page.click('button[type="submit"]');

  // Save another screenshot — this time of the password request sent screen.
  await page.screenshot({ path: `password-request-sent.png` });

  // Search for the email with password reset instructions.
  const email = await mailosaur.messages.get(serverId, {
    sentTo: testEmail
  });

  // Validate that the email arrived and contains the right subject.
  assert.equal(email.subject, 'Set your new password for ACME Product');

  await browser.close();
})();

In a nutshell, our program is using our test email address from Mailosaur to request a password reset. The code then navigates to the password reset site and inputs a randomly generated email address from our Mailosaur domain.

After requesting the reset, the program watches the Mailosaur mail server for the password reset instructions, verifying that the subject is correct.

As a bonus, we have Playwright snap a few screenshots along the way that we can analyse and troubleshoot as needed.

How Playwright compares to other end-to-end test frameworks

Playwright is an up-and-comer in a world of established players like Cypress and Selenium. Let’s take a look at how it compares.

Selenium

There are a few similarities between Playwright and Selenium. Both frameworks are easy to install, open source, and completely free to use.

They both support multiple programming languages, although Selenium supports a few more (namely PHP, Ruby, and Perl).

Playwright works cross-platform with Chromium, Firefox, and Webkit, whereas Selenium is compatible with every browser under the sun. In addition, Playwright’s headless approach allows for faster testing in even the most complex of situations.

When comparing the two frameworks, Playwright typically works best for complex tests with a limited scope of coverage. For any tests requiring a wider area of coverage, Selenium is likely the way to go. With Playwright being so new, it remains to be seen how much it will branch out into other languages and platforms.

Cypress

Cypress has some similarities with Playwright, but the two frameworks differ in many ways. Cypress is also an open source framework that’s free for testers to use, and both are best installed through npm.

Playwright allows a single test to be run in multiple browsers, supporting Chrome, Webkit, and Firefox. Cypress, at the time of writing, does not work with Webkit browsers such as Safari and does not support multi-page or third-party comparisons.

The Cypress framework does have easy-to-understand documentation with a broader community of users. The framework is an all-in-one test kit, which gives it more of a plug-and-play feel. Playwright lets you select the test runner of your choice and has fewer dependencies to worry about.

When looking at the two frameworks side-by-side, Cypress offers a better approach for those new to testing or programmers who are looking for one package that contains everything they need to get started. Playwright is probably the better choice for anyone who needs to test in Webkit or cover scenarios requiring multiple pages and domains.

Automated testing frameworks at a glance

If you’re wondering whether a framework supports your preferred programming language or browser at a glance, look no further than the table below:

Key features of Playwright

Playwright is not without a few tricks of its own.

Browser contexts

A browser context allows you to create an isolated session within a browser instance. These fast sessions are easy to create and appear in an incognito-like state. This feature lets programmers run tests in a unique browser setting to isolate the state of the browser each time.

Browser contexts are also able to emulate multi-page scenarios involving colour schemes, permissions, locale, or even mobile devices. Other frameworks such as those listed above struggle to produce something similar.

WebKit support

While not quite a unique feature, Playwright is one of a few frameworks to feature compatibility with WebKit. Since WebKit is the engine that powers the Safari browser, this unlocks access to a range of possibilities you won’t have with other frameworks.

Playwright: the future of automated testing

Playwright is still a new framework but comes pre-loaded with lots of support from Microsoft and their entire ecosystem. It already offers features such as cross-platform functionality that help it stand toe-to-toe with some of the big names out there.

Although still relatively unknown, a majority those who have used Playwright would use it again. Out less than a year, the Playwright framework already has a high satisfaction rating compared to other popular frameworks.

It stands to reason that we’ll see an abundance of new features as Playwright continues to grow. We may even see interesting cross-functionality between other Microsoft apps like Azure in the future.

Mailosaur and Playwright can save time and frustration

When you’re ready to test your own application functionality requiring email or SMS messages, look no further than Playwright and Mailosaur. At Mailosaur, we make it easier to discover issues with your application before it hits the market.