Nowadays, a remarkable technology is gaining traction in the field of test automation. In this blog, we are going to explore this skyrocketing technology known as Visual AI. We are going to delve deep into its impact and why it is going mainstream in the current scenario.
Why Visual AI and What Is Its Purpose?
Visual Testing
Visual testing is the testing technique that concentrates more on the visual aspects of the application. This technique is about validating that the elements in the web application appear correct and consistent. Visual bugs can lead to customer dissatisfaction, loss of credibility, damage to the brand image, negative SEO effects, and increased development and maintenance costs. Below is an example of a visual bug encountered in one of the leading applications.
Why Visual AI testing over other visual testing methods
As demonstrated in the above images, these types of bugs are difficult to catch with traditional web UI testing where we put functional assertions in our test cases. These assertions are directly proportional to the elements, properties, and environments. With the increase in a number of these factors, the functional assertions also increase. This is where Visual AI testing plays an important role.
Before we get into the details of Visual AI testing, let’s touch base with the conventional visual testing practices. DOM comparison and Pixel-to-Pixel comparison. In the DOM comparison, there is a comparison between two DOMs, and bugs are reported if there is any difference found. But two identical DOMs can be rendered differently which can result in visual bugs leakage. On the other hand, the pixel-by-pixel approach is about comparing the images by taking an account of the image pixel and it will only pass if images are 100% identical. So, a small shift in property or image pixel can lead to failures.
Visual AI is a powerful testing technique that mimics the human eyes. Using artificial intelligence, it is smart enough to report the actual deviations and ignore the irrelevant or trivial deviations in the web application.
Above are a few leading AI-powered Visual AI testing tools. In this blog, we are going to focus on Applitools eyes for writing our first test case.
Writing your first test via Eyes
Brief Introduction to Applitools Eyes
Applitools Eyes is an AI-powered tool that carries out visual regression testing. It enables teams to efficiently deliver top-notch applications across various browsers and devices. Applitools Eyes achieves this by mimicking the human eye’s capabilities and autonomously identifying issues and flaws during each release cycle. It can recognize dynamic content like ads and it ignores it while executing.
- It also provides leverage to integrate your tests with leading CI/CD tools such as Jenkins, Github Actions, CircleCI, etc.
- Applitools SDK integrates with popular automation tools such as Appium, Selenium, Cypress, etc
Step by Step guide to create a basic test
- Sign up on the Applitools eyes to access its benefits.
- Once registered and signed in, you are able to access the dashboard.
- In the below diagram, you can observe the basic workflow of Applitools eyes.
- We are utilizing the Selenium JUnit framework using Maven to execute our test cases. But, Eyes can be employed with other frameworks as well.
- Add the following dependency in the pom.xml file
<dependency>
<groupId>com.applitools</groupId>
<artifactId>eyes-selenium-java5</artifactId>
<version>5.64.4</version>
</dependency>- Now, go to your dashboard. Hover over the profile and then click on the API key. A dialog box appears where we can copy the API key.
- Add the following code in the setup method annotated with @BeforeMethod
eyes=new Eyes();
eyes.setApiKey(API_KEY);
eyes.open(driver,"Getting Started with Eyes",testInfo.getTestMethod().get().getName(),new RectangleSize(1000,600));- Write your test under @Test annotation and add
eyes.check(Target.window())that provides coverage for visual validation of the web application. - In your teardown method, add the following snippet to close the server :
eyes.closeAsync(); - Below is an example of a test case for visual testing using Applitools Eyes.
import com.applitools.eyes.RectangleSize;
import com.applitools.eyes.selenium.Eyes;
import com.applitools.eyes.selenium.fluent.Target;
import org.junit.jupiter.api.*;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
public class VisualAITest {
WebDriver driver;
Eyes eyes;
@BeforeEach
public void beforeEach(TestInfo testInfo) {
//set up the connection with Applitools Eyes and add access to Eyes dashboard
eyes=new Eyes();
eyes.setApiKey("API_KEY");
ChromeOptions options = new ChromeOptions();
options.addArguments("--remote-allow-origins=*");
driver = new ChromeDriver(options);
eyes.open(driver,"Getting Started with Eyes",testInfo.getTestMethod().get().getName(),new RectangleSize(1000,600));
}
@Test
public void firstVisualTestCase() {
driver.get("https://www.google.com/");
//visually validates the test case
eyes.check(Target.window());
}
@AfterEach
public void tearDown() {
//close the connection with Applitools server
eyes.closeAsync();
driver.close();
}
}Reviewing the test results
Baselines
- Before diving into this topic, it is important to know about the baseline. The baseline consists of 5 variables: App Name, Test Name, Browser, OS, and Viewport size which represents a test run.
- If you run your test for the first time, the Applitools server will check for the match of the above 5 parameters. If no match is found, it will create a new baseline and mark the status as New.
- On consecutive runs, if the Applitools server finds the match, it will compare it against the existing baseline and represent the result.
Test Status
- New: There is no match for the variable values defined in the existing baseline or there is no existing baseline in your account
- Passed: If all the variables match with the existing baseline, then Applitools eyes will check for visual differences. The test case is passed if there is no difference.
- Unresolved: The test is marked as unresolved if there is any visual difference and the variables are the same as the existing baseline. At this stage, the tester has to determine if it is a valid change or a visual bug.
- If the change is valid, then the user will approve the change with a thumbs up. This will replace the existing baseline with the new checkpoint.
- If the difference is valid, then the user can reject it with a thumbs down. This will mark the test case as Failed.
Conclusion
Visual AI testing is valuable for ensuring that applications look and behave consistently across different browsers, platforms, and devices, as well as for validating responsive design, accessibility, and localization aspects. It enhances the accuracy and efficiency of testing processes, helping teams deliver high-quality applications with a superior user experience.
