Playwright 1.64 Added --shuffle. Run It Before You Trust Your Suite Again.

Playwright 1.64 Added --shuffle. Run It Before You Trust Your Suite Again.

ยท 5 min readPublished inEngineering Quality
QAAssist graphic: Playwright 1.64 added --shuffle. Run it first.

Playwright 1.64 shipped on October 7 with a --shuffle flag and a reproducible seed. It's the cheapest way to find tests that only pass because of the order they run in.

Quick answer

Playwright 1.64, released October 7, 2026, adds a --shuffle option that runs tests in random order and prints a seed you can pass back to reproduce that order. It helps expose tests that depend on each other. Roll it out in a nightly job first, log the seed, replay failures with it, and fix or lock any shared state before enabling it on pull requests.

Key takeaways

  • Playwright 1.64 (October 7, 2026) adds --shuffle, which randomizes test order and prints a reproducible seed.
  • Replaying a failure with the same seed separates order dependence from timing problems.
  • The new describe-level lock option makes genuinely shared resources explicit.
  • Check the four breaking changes, including hidden iframes now counting as hidden, before upgrading.

Playwright 1.64 shipped on October 7. The headline feature is WebMCP support, and that'll get most of the attention. The feature I'd actually run this week is a single flag: --shuffle.

The release notes describe it plainly: it "schedules tests in a random order, which helps to find tests that accidentally depend on each other." It prints a seed, and you can pass that seed back to get the same order again:

npx playwright test --shuffle
# Running 42 tests using 4 workers, shuffle seed 271828182

npx playwright test --shuffle 271828182

That second line is what makes it useful rather than just chaotic. A random failure you can't replay is noise. A random failure with a seed is a bug report.

Why order dependence hides so well

Most suites pass because they've always run in the same order. Test A creates a user. Test B happens to rely on that user existing. Test C clears a setting that test D quietly expects to be cleared. None of this is written down. It just works, until someone adds a test in the middle, splits a file, changes the worker count, or runs one spec on its own while debugging.

Then you get the classic flaky-test conversation. "It fails on CI but passes locally." "It passes when I run it alone." "Just rerun it." Some of those failures are timing and network. Some of them are tests leaning on state left behind by other tests. You can't see that dependency by reading the code, because it lives in the order.

Shuffling makes the order a variable instead of a constant. If a test only passes when it runs after another one, a few shuffled runs will usually find out.

What else changed that QA folks should know

The release has a few other things worth a look.

Test locks for groups. There's a new lock option in test.describe.configure() that adds a lock to every test in a file or group, for example test.describe.configure({ lock: 'user-settings' }). I read this as the honest fix for some of what shuffle will expose. If two groups of tests really do share one resource, say a single settings record, a lock says so out loud instead of relying on luck and file order.

Default projects. testProject.default lets you keep Firefox and WebKit projects in the config but only run Chromium when someone types a plain npx playwright test. You can still run everything with --project="*", which is what the notes suggest for CI. That's a nice change for local speed, as long as CI really does run the full set.

locator.within(). You can now say "this button, within that dialog" by combining two locators you already have. Relative locators like nth() resolve inside each parent, so "the third cell of every row" becomes one expression.

Breaking changes to check before you upgrade. There are four. Device descriptors now forward the screen property, so media queries see the emulated screen size. JSX in test files now follows your tsconfig.json. --update-snapshots=missing now passes the run when it only writes missing snapshots. And elements inside hidden iframes are now treated as hidden by actions and toBeVisible(). That last one could flip assertions in suites that test embedded widgets, so run the upgrade on a branch first.

WebMCP. The new page.webmcp and frame.webmcp APIs give you access to tools a page registers through the experimental WebMCP browser API, so you can test them like any other part of the app. You need to launch Chromium with --enable-features=WebMCP. If your product doesn't expose WebMCP tools yet, park this. If it does, those tools are now a surface that needs tests like any form or endpoint.

How I'd roll shuffle out

Don't flip --shuffle on your main pull-request pipeline on day one. If your suite has hidden dependencies, you'll turn every PR red for reasons unrelated to the change, and people will learn to ignore failures. That's the opposite of the point.

A calmer rollout looks like this:

  1. Add one nightly job that runs with --shuffle. Keep the normal PR job as it is.
  2. Log the seed with every failure. Playwright prints it at the start of the run. Make sure it ends up in the CI summary or the ticket, not buried in a log nobody scrolls.
  3. Replay before you triage. When the nightly fails, rerun locally with the same seed. If it fails again, you've got an order dependency. If it doesn't, you're probably looking at a timing problem instead, which is a different fix.
  4. Fix the dependency or declare it. Either make the test set up its own data, or, if two groups genuinely share a resource, put a lock on them so the dependency is explicit.
  5. Move shuffle into the PR job once nightly has been quiet for a while. You decide what "a while" means for your team. I'd want a couple of weeks of clean runs.

The point of step 3 is worth stressing. A lot of teams have a single bucket called "flaky" and treat everything in it the same way, usually with retries. Shuffle gives you a cheap way to split that bucket in two. Order-dependent tests are deterministic once you have the seed. They're bugs in your tests, and they're usually quick to fix once you can see them.

My take

Random ordering isn't a new idea. Plenty of test runners have had it for years, and some teams have rigged it into Playwright themselves. What's new is that it's now a first-class flag with a reproducible seed, which removes the excuse.

I'd treat the first shuffled run as an audit. Whatever it turns up was already there. It was just hiding behind a fixed order and a retry policy. Better to find it on a quiet Tuesday night than in the week someone reorganizes the test folders before a release.

And if the shuffled run comes back clean the first time? Good. Keep the nightly job anyway. Test suites pick up new dependencies every time someone writes a test in a hurry, and that's every week.

Playwrighttest orderflaky testsCItest automation

Frequently asked questions

It schedules tests in a random order to help find tests that accidentally depend on each other. Playwright prints a shuffle seed, and passing that seed back reproduces the same order.

About this article

Gulshan KumarProduct & Engineering
Ship release-ready quality

Try QA Assist

Plans, runs, and defect handoff in one workspace.

Get it free