Data as of Aug 25, 2026 · Based on 38 AI responses from ChatGPT Search and Google AI Mode · See how Parse measures this
Your brand can be here too.
Shipping daily with a small team means your end-to-end (E2E) tests can easily become a bottleneck rather than a safety net. When UI changes daily, brittle tests turn into a constant stream of false alarms.
Here is a practical playbook to keep your E2E suite stable, fast, and actually useful:
- Shift left by writing and updating E2E scenarios *as part of the PR* , before code hits main. If the feature author is responsible for the test, it matches reality.
- Pair acceptance criteria with automated tests so the definition of "done" includes a passing E2E check.[[1]](https://google.com/goto?url=CAESjQEB6zswFQvrL_ZBxinc-LAuDCXhkbtwi2i6l4yqM2lkHElQnC67DBajHPDs5QY4Vqgd5ldz7DImLfgkipIjcbZHBlxWZAwM3nO-kFa91RhCuydRbTt_ndcaXom9p0fYhMq76RvQQ9IUjvWazzTWuVUIx7WfIouHsksHTAMQvifjKXF48nninWRqOiCOiQY)
- Avoid testing every edge case or UI permutation in E2E. Save those for unit or integration tests.
- Focus exclusively on core revenue or operational paths (e.g., signup, checkout, core dashboard action). Aim for high-value coverage over high-volume coverage.[[1]](https://google.com/goto?url=CAESfgHrOzAVGiSZSOFt1d0tBs7jDC2cgYuW-PpzsN5Jml7hgmJ_XKcq5V1YKYgTTQUvRVgJY9DGpIi9o7zCSxT5feOaYhj8zOseIkHIBxpfViyLvWTpJA91sXctXOGaKgc2annsLu-kjL_-Et-pJoyGfTIQO4LcwyopekbfqE4ODA)[[2]](https://google.com/goto?url=CAESggEB6zswFd3989zkcekoMGNZbmtcofKQ4eb5pKunKWn8nrz33QlN_IuYhpN0ub8jqXEgTJI2trUnvnYGW56OL_TOvCoWvA6QCi8WJKktwcb-qtSjGB4JOKLgCTFrA7pY10z4pArDnwK-DAcdEsNHUxNxLgdcp9VWWKdzhd3w017iJb-D)[[3]](https://google.com/goto?url=CAESZwHrOzAVYi76nd-MPSvkby_VDDVt1Ex0AG3GIAAqO18ygBaC5wthHrFW3nC6r51IiWcJgFfAobEJ1ZLSH-rE9WPdZ_zpkQeytpLpXjaZJinBoigS807Gi-N_sNGPVYxGFgnAJqs6-a8)[[4]](https://google.com/goto?url=CAESigEB6zswFZJMfCZYeB_3T6hrhx_p8vpNvTpC9HUOgd887MRQfAju5V8frwakDpeCM4scl2qWhmQvGC8fJWSs76XJ6Zla5hIspOOUrtN6OZcVWRugoL3IgLUpFHO94FLcFofhdwyK92ZHKdhwgNDAkWJaG13aKweh5x3_2rOtdPXg4RB9d-A8Xx1yj1A)
- Stop using fragile CSS classes, dynamic generated IDs, or deep DOM paths that change with minor styling updates.
- Implement dedicated attributes like `data-testid="submit-btn"` across your frontend codebase so selectors never break during refactors.[[1]](https://google.com/goto?url=CAESZwHrOzAVUX-AZ32dgN1bLLGcuqV4bQ79SBX0IxK6yebhAW0LeAuzzpsclTtMLkGS-I09NVybPe1BggPFUqEUvLwJRXSMhgUKe99dKkTODEocHu_6xCepNWjT4rywu_dc3Zt5aPiniYU)[[2]](https://google.com/goto?url=CAESWwHrOzAV-KGj7Q9pmtIbMYYjaavK3GBx1ZskwOG2rxIe4p3VS1dHAbhX66PON2BoIl0g_mcm--36880HeGGmscLMoVPXbN4cIpW436IOCkeHqWO9fbP32MVTXYg)[[3]](https://google.com/goto?url=CAESlgEB6zswFQRFVjGJuqAVKCJCY9Eg_iOcxcjJlCyvIkwIhGqiqX2xGZf1qxpHu1Z5Pcv_cLwFfcFrGzBns3W7ARJ8ZrUy7XVg8acZsIn-dOG_xZXgb34bRa--8qgoIGUGFIFJnSANpBG-l2BxUqWq-XRW71Kw5CV9Y4tTFKvPCEjPPGgqarRUhQMu981jrRwJgI-M5Fe4eLA)[[4]](https://google.com/goto?url=CAESfgHrOzAVDQ53jsim7v4crE9LC8IEZ2bqGc18CH6FeruQG3tbPv-xHBD1L2LHgySHJlXw-e3oMVnHLpHxaSnekbBftsPbY21AL8PbgJD9zt9kAte9z5P0ywm2KsLiHGu6kMbQg7zuX-Yu8nVNS70eZ8ZQ0nXn2cLHYKS64iHfOw)[[5]](https://google.com/goto?url=CAESWwHrOzAV5yOTH73NQ7Ml2sCiLey2E5l4WacWrj7O3jrqbrTPxOPBGtMDisyzNLwfckxC8wq1EL41LnOqMLIomc26vw-azARkNJBSwjr-9KisobXrdirTWcMjGOk)
- Never let tests depend on state left behind by previous runs. Flakiness spikes when test #2 fails because test #1 didn't clean up a database row.
- Seed state via API requests during test setup rather than clicking through the UI to get there (e.g., use an API endpoint to create a user and log them in instantly).[[1]](https://google.com/goto?url=CAESVAHrOzAVRHafkYWoru9POg79n92eJ2ejto3S95DqXwKsSEnw8EsRRsb9YSUNokgkzgvkk888rHyqjxjGUsRHP1IB56Xku0-DsLnBPq_Z8Re0N4JjTQ)[[2]](https://google.com/goto?url=CAEShAEB6zswFV1_gvZReEB11z1lrvGScwHh8ExpR-Blf4fohdXk8uEDX-DGTFJPndDXNzTABmUhQTpeoOTdB_HV4LC9pXByHb_hvFVrM6_YhapQvfzIgzkrS4BRtitc9W55fp2W2ZVaB96slJBw_IYS8xAZ3Wo8HWkqqv9PJnYFXmV75bwfzf4)[[3]](https://google.com/goto?url=CAESjwEB6zswFetrq1uvKka5n22NQxJq-mq4goqbc96EjR4f__PocfU_p830DgNdkeigPltBSs2I2IN6ckde135hgLGtoX35TDvvrw278sSqBoWxGupaEP44jEhObBR-9-3imsRzcx5ye2er6AWuise6NyzSHwhgzZflTHh996sCpxnyEproZT2f6l8RDRtYcC2J8Q)[[4]](https://google.com/goto?url=CAESfgHrOzAVDQ53jsim7v4crE9LC8IEZ2bqGc18CH6FeruQG3tbPv-xHBD1L2LHgySHJlXw-e3oMVnHLpHxaSnekbBftsPbY21AL8PbgJD9zt9kAte9z5P0ywm2KsLiHGu6kMbQg7zuX-Yu8nVNS70eZ8ZQ0nXn2cLHYKS64iHfOw)
- Set a rotation or rule: if a test is flaky, it gets quarantined or fixed immediately. Do not let a culture of "ignoring red builds" take root.
- Delete tests that no longer map to active features. A smaller, ruthless suite is better than a massive graveyard of broken legacy checks.[[1]](https://google.com/goto?url=CAESSwHrOzAV4G4yPps6iClO1REqKsGbT5yUq6Nooa9wk1qOgzHzatd3zfN58ZX9LWu0fq5-SNaMIohQentTFb7f0rwaKZMUe7ADXSOjnw)[[2]](https://google.com/goto?url=CAESrgEB6zswFcfHIQ7BVqhR3OtlYwZQzKmvcgo0R9W8JwvOnHQ5uAiPlP7uV3QRGGuLwySZfqZ5zbY93mEMMxYforoDO0ebwr0lHADCI1kzlSlq31gEOYjwrMUCMagxdJCs6UQwwOqmDwFu9-21l2VDiyGoNXibEgCue6R-tHNXo9fVMcWo_EeKtu8HVX-LQ59CgxTQM1eNeM1Z6VG29GfGH7DETDJnSs1ZoOg1ku1-XjI)[[3]](https://google.com/goto?url=CAESZgHrOzAV9CGKpLaaISicmugg4_kiGyOTbPZsiJiAW70jm0BZDuAobNQeKuAHgaFKAPkBVUrOsOTQaKpdgoFNj7TC7xg0KV1UM2pYYv8isc3yuxdMBjBI6ZkrZrhknskePj1RxEvnbQ)[[4]](https://google.com/goto?url=CAESXQHrOzAVknLxdVAxvSSoQ-TRypZhIfnl-MvckMQ0ujkRwVL7JMrGs8l0Mr52eZ4aID2ArQYWsvqJI5zyK5Lbb5tTWLOl4Ep7hssAJLGgHwdi6-021egX-5gHMiN-Rw)[[5]](https://google.com/goto?url=CAESUAHrOzAVmc8sMq89mRJVDAN2xQVSHhdroaYcHfFl6edPMqRW3rBRhrzrmzqd-eNDFS8YrUoomGJuUU9GgSN6xjfDxssUZUIjl2oJWJVO5s-s)
To help tailor this to your stack, could you tell me:
For a small team shipping daily, the goal isn't “zero flaky tests.” It's making failures cheap to diagnose and preventing the common causes of flakiness.
This is the biggest lever.
Both Playwright and Cypress explicitly recommend independent tests because leaked state is a major source of nondeterministic failures.
A useful rule:
If I run this one test 20 times on its own, it should behave exactly the same as when the whole suite runs.
Avoid:
click()
sleep(2s)
assert()
Prefer waiting for something meaningful:
click()
wait until "Order confirmed" is visible
assert()
Modern E2E frameworks already provide actionability/auto-waiting and retryable assertions; use those rather than guessing how long the application needs.
If you're seeing lots of wait(1000) or sleep(5000), treat those as flakiness debt.
Don't select elements based on implementation details:
div:nth-child(3) .blue-button
Prefer things deliberately exposed for testing:
[data-testid="checkout-submit"]
or, where appropriate, user-facing roles/names such as “Submit order.” Playwright recommends user-facing locators, while Cypress recommends stable data-* selectors.
This also creates a nice team convention: if a UI element is important enough to E2E-test, give it a stable interface.
Don't make your checkout E2E test depend on Stripe's availability, a marketing API, an email provider, or some random third-party service.
Test your integration with the third party separately, then make the E2E test deterministic by controlling the external response. Playwright explicitly recommends avoiding tests against third-party dependencies.
Keep a small number of true “everything is real” tests for critical paths.
For a small team, I'd roughly aim for:
Your E2E suite should answer questions like:
It shouldn't contain 47 variations of input validation that could be tested much faster below the browser.
This is particularly important for a daily-shipping team.
If a test fails intermittently:
Don't just increase retries until green.
Retries are useful for distinguishing “failed once, passed on retry” from deterministic failures, but a test that routinely needs three attempts is still broken. Cypress, for example, supports tracking tests that pass only after retry.
When CI fails, developers shouldn't have to reproduce it blindly.
Save:
The ideal experience is:
“Checkout failed on step 4 because
/api/orders/123returned 500.” —not:
“Expected button to be visible after 30 seconds.”
Even if you only have 100 tests today, avoid shared mutable state.
For example, instead of every test editing [email protected], generate something like:
test-user-{unique-test-id}@example.com
Two parallel tests modifying the same backend record are a classic source of races. Playwright specifically recommends unique backend data for parallel tests.
Every E2E test must:
And I'd put one metric on the team's dashboard:
“What percentage of CI failures this week were actual product failures?”
If that number is low, don't respond by adding more E2E tests. Fix the test system first. A smaller, boring, trustworthy E2E suite is far more valuable to a team shipping every day than a huge suite everyone has learned to ignore.
For example, instead of every test editing [email protected], generate something like:
test-user-{unique-test-id}@example.com
Two parallel tests modifying the same backend record are a classic source of races. Playwright specifically recommends unique backend data for parallel tests.
Every E2E test must:
Test your integration with the third party separately, then make the E2E test deterministic by controlling the external response. Playwright explicitly recommends avoiding tests against third-party dependencies.
Keep a small number of true “everything is real” tests for critical paths.
For a small team, I'd roughly aim for:
For a small team shipping daily, the goal isn't “make E2E tests perfect.” It's “make failures trustworthy.” A good rule is: if the team routinely reruns CI without investigating the first failure, the suite has already lost its value.
Here’s the approach I'd use:
Use E2E tests for critical user journeys, not every edge case.
Push detailed business logic into unit/integration tests. E2E tests are slower and more sensitive to environmental changes.
This is probably the biggest flake reducer.
Each test should create or seed the state it needs and should not depend on a previous test having run. Playwright explicitly recommends isolated tests, and Cypress similarly treats test isolation as a best practice.
For example, prefer:
test: create invoice
→ seed customer
→ log in
→ create invoice
→ assert invoice exists
over:
test 1: create customer
test 2: log in as customer
test 3: create invoice
The latter creates cascading failures and makes parallelization painful.
Use deterministic test data:
Programmatic login and explicit state control are also recommended by Cypress.
This is the classic:
await page.waitForTimeout(2000)
It makes tests both slower and flaky.
Instead, wait for something meaningful:
await expect(page.getByRole('button', { name: 'Save' }))
.toBeEnabled()
or wait for a specific network/state transition.
Modern E2E frameworks have built-in retry/wait behavior; use that rather than inventing timing assumptions.
Avoid selectors like:
div:nth-child(3) > button.sc-abc123
Prefer something deliberately stable:
<button data-testid="submit-order">
or, even better when appropriate, user-facing semantics:
getByRole('button', { name: 'Submit order' })
Cypress specifically recommends resilient data-* selectors rather than selectors coupled to CSS/implementation details.
A retry can be useful in CI because transient infrastructure/network issues happen. But don't configure 5 retries and declare victory.
A useful policy is:
Cypress's current guidance similarly recommends keeping retries low and treating frequently retried tests as technical debt.
When CI fails, the developer should get enough information to answer “what happened?” without reproducing it three times.
Capture:
The difference between “E2E failed” and “checkout failed after /api/payment returned 503; here's the trace” is enormous for a small team.
I'd maintain a tiny dashboard or even a GitHub issue label:
Flaky tests this week
---------------------
checkout-payment.spec 3 flakes
invite-user.spec 1 flake
search.spec 0
If a test flakes, someone owns fixing it. Don't leave it permanently quarantined.
The important metric isn't merely pass rate; it's “how often does a red build represent a real regression?” Flaky tests erode trust in the entire suite.
For a daily-shipping team, I'd aim for something like:
PR: unit/integration → critical E2E → merge
Post-merge/nightly: full E2E matrix → browsers → less-common workflows
Don't make every PR wait 30 minutes for every browser/device combination unless there's a strong reason.
Don't just patch the individual test.
If five tests break because a button's CSS class changed, introduce a selector convention.
If tests constantly fight shared database state, improve test-data isolation.
If tests constantly time out waiting for asynchronous jobs, create a reliable test-only mechanism for waiting on job completion.
If login is 20% of every test, create a supported authenticated fixture.
The small-team sweet spot is a boring, deterministic E2E suite with perhaps 20–50 excellent tests rather than 500 fragile ones.
The litmus test I'd use is:
Can a developer believe a red E2E build without rerunning it? If the answer is “yes” most of the time, you're doing E2E testing right.
Shipping daily with a small team means your end-to-end (E2E) tests can easily become a bottleneck rather than a safety net. When tests are flaky or constantly breaking due to minor UI tweaks, teams start ignoring them—defeating their entire purpose.
Here is a practical playbook to bulletproof your E2E suite and keep it fast, reliable, and low-maintenance.
sleep(5000) . Instead, rely on modern test runner features (like Playwright's or Cypress's built-in auto-waiting and web-first assertions) that wait for specific network states, elements to be visible, or animations to finish..btn-primary-v2-xyz ) that change during styling updates. Use dedicated test attributes like data-testid="submit-button" or stable accessibility roles (getByRole).Would you like to dive deeper into any of these areas? Tell me:
I can give you specific code patterns or a refactoring strategy tailored to your setup.
Keeping end-to-end (E2E) tests stable while shipping daily requires shifting how you write, maintain, and trigger them. When a small team deploys constantly, flaky or brittle E2E suites quickly become a bottleneck that people learn to ignore—defeating their purpose.
Here is a practical playbook to keep your E2E tests green and reliable:
sleep() or waitForTimeout() calls, which either waste time or fail under random network lag. Use modern assertion-based auto-retrying and explicit element-state waiters built into frameworks like Playwright or Cypress.If you'd like, tell me:
I can give you more specific advice on optimizing your CI/CD integration.
For a small team shipping daily, the goal isn't “never have flaky E2E tests.” It's “make flakiness cheap to detect, diagnose, and eliminate.”
The practices that give you the biggest payoff:
Make every test independently runnable.
Each test should create its own data, establish its own auth/session, and clean up after itself. Don't have test B depend on what test A did. Both Playwright and Cypress explicitly recommend isolation because it prevents cascading failures and makes parallelization/retries much safer.
Create test state through APIs, not the UI.
If a checkout test needs a customer with an existing order, create that customer/order through a database/API fixture rather than clicking through 15 UI screens first. Keep the UI portion focused on what you're actually testing. Cypress specifically recommends programmatically controlling application state and authentication.
Never use arbitrary sleeps as synchronization.
sleep(2000) means “I hope the system is ready by then.” Instead, wait for something meaningful: a specific network response, element state, URL change, or application condition. Modern E2E frameworks have built-in retry/wait behavior for this.
Use stable selectors.
Prefer things explicitly intended for testing—e.g. data-testid="submit-order"—over CSS classes, DOM position, or implementation-specific selectors. Your designer changing .btn-primary shouldn't break 40 tests.
Keep the E2E suite small and high-value.
Don't test every validation rule through a browser. Put most logic in unit/component/API tests and reserve E2E for critical user journeys:
Retry once in CI—but treat a retry as a failure signal.
A single retry can prevent a transient browser/network hiccup from blocking a deploy, while still exposing tests that are flaky. Playwright categorizes “failed first, passed on retry” as flaky; Cypress similarly recommends keeping retries low and using flake data to find root causes.
In other words: green-after-retry ≠ healthy test.
Make CI failures diagnosable without reproducing them locally.
Capture screenshots/video where useful, but especially traces/network information. For example, Playwright recommends its trace viewer for CI failures because it lets you inspect actions, DOM snapshots, and network activity after the fact.
Track flakiness as an engineering metric.
Keep a simple list of:
If a test flakes repeatedly, quarantine it or fix/delete it rather than teaching everyone to ignore it.
I'd use something like:
E2E tests must be isolated, deterministic, and independently runnable. CI gets one retry. A test that passes only after retry is reported as flaky and gets fixed rather than permanently tolerated.
And structure your suite roughly as:
PR: critical E2E smoke tests + unit/component tests
Main branch: full E2E suite, parallelized
Nightly: broader browser/device/integration coverage
The biggest mindset shift is that test maintenance is part of shipping, not a separate QA activity. If a test breaks because a legitimate UI change was made, updating it should be part of that same PR—not a backlog item someone discovers three weeks later.
For a small team shipping daily, the goal isn't “lots of E2E coverage.” It's “a small set of E2E tests we trust.” E2E tests are inherently slower and more maintenance-heavy than unit/integration tests, so push most assertions down the pyramid.
Keep the E2E suite small and business-critical
If a behavior can be confidently tested at the API/integration level, don't test it again through the browser.
Make every test independent
Each test should create the data it needs and clean up—or use disposable test data. Don't have “test B” depend on “test A” having created something. Isolation is one of the biggest defenses against cascading and parallel-execution failures.
A useful rule:
You should be able to run any single E2E test 100 times, by itself, and get the same answer.
Stop depending on timing
Avoid:
await sleep(2000)
await page.click(...)
Prefer waiting for an observable condition:
await expect(page.getByRole('button', { name: 'Submit' }))
.toBeEnabled();
await page.getByRole('button', { name: 'Submit' }).click();
await expect(page.getByText('Order confirmed')).toBeVisible();
Modern browser frameworks such as Playwright provide auto-waiting and retryable assertions specifically to reduce timing-related flakiness.
Test the UI like a user
Prefer stable user-facing contracts—roles, labels, accessible names—over CSS classes or DOM structure. That way a harmless UI refactor doesn't break ten tests.
Control your dependencies
Don't let your critical CI test depend on Stripe's availability, somebody else's API, an external email provider, etc. Stub/fake dependencies where you're testing your own behavior. Keep a much smaller number of genuinely end-to-end tests for the integrations themselves.
Give every test deterministic data
A particularly nasty source of flakiness is shared databases:
Test A ──┐
Test B ──┼──> same "test-user"
Test C ──┘
Instead:
Test A ──> user-A
Test B ──> user-B
Test C ──> user-C
This becomes essential once you parallelize CI.
Use retries as a smoke alarm, not a solution
One retry is useful operationally: if a test fails and then passes, you've learned that it's flaky. But don't make “passes on the third try” equivalent to “healthy.” Playwright, for example, explicitly categorizes retry-passing tests as flaky.
I'd make the policy:
flaky test → ticket → owner → fix quickly
If necessary, temporarily quarantine it so it doesn't block every deploy—but don't leave quarantined tests indefinitely. Google describes a similar quarantine approach while emphasizing that it can otherwise hide real bugs.
Make CI failures ridiculously easy to diagnose
On failure, automatically retain:
For Playwright specifically, traces are particularly useful because they let you inspect the timeline, DOM snapshots, actions, and network activity.
I'd aim for something like:
┌─ 500+ unit tests → every commit
│
├─ 50 integration tests → every PR
│
└─ 10–30 E2E tests → every PR/deploy
│
├─ deterministic
├─ isolated
├─ fast
└─ extremely high-value
And track flakiness rate, not just pass rate.
The key cultural rule for a small team is:
A flaky E2E test is a broken test, not an acceptable test that occasionally fails.
If the team starts ignoring red CI because “that one's flaky,” you've lost the primary value of the suite: trust. Reliable, fast feedback is more valuable than maximizing the number of scenarios exercised end-to-end.
Keeping end-to-end (E2E) tests stable on a fast-moving, daily-shipping team requires shifting your philosophy from "test everything in the UI" to "test high-value user journeys with surgical precision." When tests constantly break, they become noise that the team ignores.
Here is a practical, structured approach to keeping your E2E suite green and reliable:
div > div > button.submit ) or volatile text that changes with localization. Use dedicated data attributes (e.g., data-testid="submit-button" ) or stable ARIA roles that rarely change during refactoring.sleep(5000) ), which are slow and still fail under network lag. Rely on modern framework auto-waiting assertions and explicit state-based waits (e.g., waiting for a specific network request or DOM visibility).If you'd like, tell me:
I can give you more specific, tool-tailored strategies to fix your pipeline.
Keeping end-to-end (E2E) tests stable while shipping daily requires shifting from treating tests as a final gatekeeper to treating them as a living product infrastructure. When a small team ships daily, flaky, slow, or brittle tests will quickly get bypassed or abandoned.
data-testid="submit-button" —across your front-end components so developers know not to touch them during routine UI changes.waitForTimeout($5000$) or sleep(10) , which slow down suites and still fail under network load. Use modern framework assertions that poll and wait implicitly for specific network responses, element states, or URL transitions to resolve naturally.If you'd like, let me know:
I can give you targeted strategies to fix your specific bottleneck.
The trick is to treat flakiness as an engineering defect, not as an unavoidable property of E2E tests. For a small team shipping daily, I’d optimize for a suite that is small, deterministic, and easy to diagnose rather than trying to cover everything through the browser.
Use E2E tests for the workflows where you need confidence that the whole system works together:
Push most edge cases down to unit/integration tests. A 30-test E2E suite that you trust is much more valuable than 300 tests that everyone habitually reruns.
The biggest rule: test B must not depend on test A having run first.
Each test should create its own data/state and be runnable by itself. Both Playwright and Cypress explicitly recommend this because shared state and ordering create nondeterministic failures.
For example, prefer:
test:
create user "test-<unique-id>"
log in as that user
perform workflow
clean up
over:
test 1: create shared user
test 2: assumes shared user exists
test 3: modifies shared user
This becomes especially important once CI runs tests in parallel.
wait(2000) is basically a bet that the application will always be ready within two seconds.
Instead, wait for observable state:
click "Save"
expect "Saved successfully" to be visible
or:
submit form
expect URL to change
expect result row to appear
Modern E2E frameworks provide retrying assertions and auto-waiting specifically to avoid these races.
Avoid selectors coupled to implementation details:
div:nth-child(3) > button.button-blue
Prefer something representing what the user actually interacts with:
button "Create account"
or an intentionally stable test ID where a semantic locator isn't appropriate. Playwright specifically recommends user-facing locators and explicit contracts over brittle CSS/XPath selectors.
A useful team convention is: if changing CSS can break an E2E test, the selector probably isn't good enough.
Don't have five CI workers fighting over:
[email protected]
Generate unique records, or give each test/worker its own namespace/tenant/database state.
Also control things that otherwise introduce randomness:
Mock external systems when the external system itself isn't what you're testing.
When CI fails, you want to answer "why?" without reproducing it locally.
Capture, at minimum:
For example, Playwright recommends traces for CI failures because they let you inspect the timeline, DOM snapshots, actions, and network activity.
A retry can keep a transient infrastructure hiccup from blocking a deploy. But:
"Passed on retry" should be visible and count as a flaky test.
Don't let:
failed → retry → passed → green
become indistinguishable from a clean pass.
Retries are useful precisely because isolated tests can be retried independently; Playwright's documentation makes the same point.
Track something like:
E2E reliability
----------------
Clean pass rate: 99.2%
Passed after retry: 0.6%
Failed after retry: 0.2%
Then put a threshold on the team: if a test flakes twice in a week, someone fixes it.
This is probably the biggest cultural change.
Don't say:
"The checkout test is flaky."
Say:
"The checkout test has a defect; let's find the cause."
Otherwise developers learn to ignore red CI, and eventually the entire suite loses its credibility. Research on test flakiness similarly identifies the loss of trust and debugging/release cost as major consequences.
I'd aim for:
PR
Nightly
Team rule
If you're shipping daily with a 3–8 person team, I'd actually prioritize test isolation + deterministic data + good diagnostics before adding more tests. Those three usually give you far more leverage than sophisticated CI machinery.