When we do automation test in our project, one important mindset is we need to treat the automation project as another software development project beside the main product. It means we need to invest to build automation framework, apply OOP/Design Pattern/ Coding convention or spend effort to maintain the automation project. One important process we need to care is code review to make sure about the quality of automation script. In this article, we will share about:
- The popular problems with the new code committed.
- Review code with Azure DevOps.
1. The popular problems with the new code committed
- Unknown purpose of the new code: The description of the pull request is not clear enough to understand the purpose of the new code. We should link the work item like PBI/Bug to the pull request so that the reviewer can understand the purpose of code changes.
- Poor code quality: the code doesn’t follow the coding convention or have the bad logic. That’s the reason why we need to review code. We can do the crossed review or apply the coding convention tool to make sure about the code quality.
- Undefined reviewer: When we create a pull request, we need to assign the reviewer for that pull request.
- Effort for reviewing code: After the reviewer reviews the new committed code, they may have some comments on the commit. If we only discuss verbally, we may forget the item we need to fix in the discussion. It will take a lot of time for re-discussing when something is missed. Therefore, we need a way to store the information about the comment.
- Effort for re-running the test script: When we make changes in the test script, we need to re-run the test script again to make sure that the changes won’t impact the existing test cases, and the new test cases added work well. One more issue usually happens is the new committed code can’t run on the CI/CD system. We’ll need more effort to investigate this issue. Whenever we make changes, we need to run test again and attach the result to the pull request. It will take a lot of effort. Therefore, we need a solution to re-run test automatically when we push the new code to the repository.
2. Review code with Azure DevOps
Azure DevOps repository allow us setting the branch policy for merging code so that it will help us save effort for reviewing code process. Here are some policies when consider setting to protect our code branch.
- Minimum number of reviewers: set the minimum number of reviewers. Eg: When we set it to 2, we need at least 2 approvals from your team to merge to pull request. We also can set the name of required reviewers after creating pull request.
- Check for linked work items: we can add the PBI/UserStory/Bug ID here, and it can show us what the purpose of the pull request is. If we don’t link any work item, we are not allowed to merge the Pull Request
- Check for comment resolution: When reviewing the pull request, the reviewer can leaves the comments for the code. After fixing the code based on the comments, the comments can be resolved. We only can merge code after all the comments are resolved.
- Build Validation: This is great feature in setting the branch policy. We can create a pipeline for running the important test cases and link it here. Thanks to that, whenever there is a new pull request to this branch or the test script is changed, the pipeline will be triggered automatically. We don’t need to re-run and attach the result manually when the code is changed. In case there is any failed test case, we can’t merge code into this branch. It helps us make sure that we can run the script on the pipeline successfully after merging the pull request. When we add Build Validation as a check point in the pull request policy, we need to care about the duration of this pipeline. It should short enough to merge code quickly and avoid consuming too much resource for running test.
Let’s assume that we’ve already had a repository on Azure DevOps repository and created a pipeline for running test cases. Here are the steps to setting the policy to protect our branch.
- Select the “Branches” menu item in Repos and choose the branch you want to set the policy
- Configure information related to Reviewer, link working item and check for comment resolution
- Add a pipeline for validate the pull request
After setting the policy, you can try creating a pull request and play with it. You can see the checkpoints here. You’re not allowed to merge code if any checkpoints here are failed.
We can see the test result easily after clicking on the checkpoint related to the pipeline.
Based on our experience, the test case can be failed because of the testing environment or existing bug, and there is no problem with the automation script. This prevents us from merging the new code. To solve this problem, the admin user can bypass the policy with the following steps.
- Select the “Branches” menu item in Repos and choose the branch you want to set the branch security
- Allow bypass policies when completing PR
3. Conclusion
The quality of automation script is really important. It will help us get quality information of our product early and exactly. Applying the supported feature of Azure Dev Ops repository will help us save a lot of effort for reviewing code. Moreover, the history and quality of our code are traceability with this approach.
