Dynamic Test Selection Using Reinforce Learning

In software testing, reinforcement learning is a technique in which a computer program or agent determines which tests should be run based on performance, so that it can catch errors effectively.

Reinforcement Learning

Reinforcement learning is a way to test software by trying out different testing actions, seeing what works (like finding bugs), and then getting better at making test decisions over time. In software testing, reinforcement learning means automatically finding the best test cases by rewarding the system when it finds errors and fixing them when it doesn’t.

• Saves money on cloud, CI, and CD while cutting down on testing time
• Provides developers with feedback more quickly.
• Makes pipelines more intelligent with each execution.

The Problem with Static Test Suites in RL

Traditionally, testing pipelines rely on static test suites:

• Each code change triggers a large number of tests.
• Testers may run tests even if they don’t relate to the most recent modification.
• Redundant tests slow down feedback cycles and waste computing time.

Dynamic Test Selection (DTS) to the Rescue

DTS now only runs those tests that have the highest chances of failing. People generally use several strategies to achieve this goal, such as prioritising tests, evaluating the impact of changes, and using machine learning for prediction.

• Code coverage analysis
• Change impact analysis
• Historical failure data

Project Structure

RL/
├── src/main/java/RL/
│   ├── RLAgent.java     # Reinforcement Learning agent logic
│   └── RLRunner.java    # Main class to run selective tests
├── qvalues.json         # Stores learned test behaviors (Q-values)├── pom.xml              # Maven project setup
└── testng.xml           # TestNG configuration

Purpose

Instead of running the entire test suite every time, RL is primarily used in dynamic test cases by running only the most valuable or high-risk test cases to improve the testing process. Best approaches:-

• Reduce feedback loops and speed up releases to increase the efficiency of the CI/CD pipeline.
• Avoid low-impact tests to save time and money.
• To increase the defect detection rate, focus on those areas where changes have increased the failure rate.
• Keep learning from previous results and continuously improve and adapt your testing process to make improve the decisions in the future.

How It Operates

RLAgent.java

• Maintains a Q-table (test name → score) in qvalues.json
• Uses epsilon-greedy strategy to balance:
• Exploration (random tests)
• Exploitation (run tests with high failure likelihood)
• Update scores:
• Failing test → Higher score
• Passing test → Lower score

RLRunner.java

Runs the pipeline:

1. Outlines every test that could be conducted (hardcoded list)
2. Determines which tests to run using RLAgent.shouldRun().
3. Tests are run (using mvn test -Dtest=…)
4. Uses the test result to update the agent.
5. Keeps updated Q-values.

Example Output (qvalues.json)

{
  "tests.SignInTests.testValidLogin": 0.1,
  "tests.SignInTests.testInvalidLogin": 0.85
}

After multiple runs, flaky tests build up a higher score and get selected more often.

How to Run

Run the main class
mvn compile exec:java -Dexec.mainClass="RL.RLRunner"

Benefits of RL

Future Improvements

• Auto-discover tests using file scanning or reflection should come first.
• Use the TestNG or JUnit API in place of ProcessBuilder for more integration and flexibility.
• To further visualise the learning performance, present Q-values as graphs.
• Additionally, ensure you preserve test metadata, such as flaky, slow, or stable, for future analysis and smarter test execution.

Reference:-

https://www.sciencedirect.com/science/article/abs/pii/S0950584924002593