How To Avoid Flaky Tests in Cypress?
1. What is a Flaky Test?
A test is considered to be flaky when it can pass and fail across multiple retry attempts without any code changes.
For example, if a test is executed and fails, then the test is executed again without any code change, but this time it passes.
2. Common Mistakes That Causes Flaky Tests in Cypress
Identifying flaky tests in Cypress can be tricky. Because of the nature of flaky tests, it becomes difficult to discover the cause or reproduce the error. This becomes frustrating. That said, there are a few common mistakes that one makes which might result in the test becoming flaky.
- Hardcoded wait
- Poorly designed selectors
- Less understanding in retry-ability
- Not using network wait
- Less understanding of how the state of an element changes in the application
These are the common mistakes seen from user complaints across platforms.
3. Best Practices to Avoid Writing Flaky Tests in Cypress
✅ Use Assertions
Use assertions to check that the expected behavior of your application is met. This can help to catch errors early and avoid flaky tests.
cy.get(‘#my-element’).should(‘be.visible’)
cy.get(‘#my-element’).should(‘exist’)
cy.get(‘#my-input’).should(‘have.value’, ‘example value’)
✅ Use Retries
Cypress automatically retries commands and assertions that fail. By default, it will retry up to 4 times. You can increase the number of retries using the retries configuration option in your cypress.json file.
cy.get(‘#my-element’).click({ retries: 4})
cy.get(‘#my-element’).should(‘exist’, { timeout: 5000, retryInterval: 1000 })
✅ Use Waits
Use explicit wait statements to ensure that actions are timed appropriately, such as waiting for an element to load.
cy.wait(1000).get(‘#my-element’).click()
✅ Define Timeouts
Cypress allows timeouts which ensures the balance between giving operations enough time to complete within a given time and anything exceeding throw timeout error without slowing time for inefficient tests.
Cypress.config(‘defaultCommandTimeout’, 5000)
cy.get(‘#my-element’, { timeout: 5000 }).should(‘exist’)
cy.get(‘#my-element’).should(‘exist’, { timeout: 5000 })
✅ Define Unique Identifiers for Element Selectors
It’s important to choose selectors that are unique and stable to ensure that your tests are reliable and maintainable.
cy.get(‘#my-element’)
cy.get(‘.my-class’)
✅ Use Fixtures or Mocks for Test Data
Poorly or inconsistent data can introduce flakiness. To minimize variability, we should prepare consistent and reliable test data. Use dedicated databases or mocks to control the test environment and ensure stable execution conditions.
Cypress allows for easily controlling and manipulating the test data, making it easier to write more robust and reusable tests.
{
“user”: {
“username”: “testuser”,
“password”: “testpass”
}
}
✅ Run Tests in Consistent and Stable Environments
Use tools and services to ensure that tests are run in a consistent and stable environment every time you push changes to your codebase.
🚫 Avoid Relying on External Dependencies
Avoid tests relying on external dependencies, such as APIs or databases, that may not always be available or return the expected data.
🚫 Avoid Data Dependency
Avoid tests that may rely on specific data, and if that data is changed or deleted, the tests may fail or produce unexpected results.