Two Years as a Business Analyst in a Data Processing Project
Over the past two years, I have worked as a Business Analyst (BA) on a data processing project. During this time, I learned that the Business Analyst role in data projects is significantly different from the role in traditional application development.
In data projects, a BA focuses less on screens and user flows. Instead, the emphasis is on data understanding, data quality, reporting logic, and business processes. Therefore, in this blog, I share my real experience and explain what a Business Analyst really does in data projects, based on the workflow I followed.
Project Overview
The main goal of this project was not only to migrate data from multiple data sources into Azure Data Factory, but also to improve the client’s overall business process.
More specifically, the project aimed to:
1. Centralise data from multiple teams
2. Reduce manual data collection
3. Improve data accuracy and consistency
4. Enable faster and more reliable reporting
As a result, users no longer needed to collect and merge data manually from different departments.
Consequently, reporting became faster and more consistent across the organisation.
Overall Workflow of a Business Analyst in Data Projects
Before diving into each step in detail, it is important to understand the overall workflow of a Business Analyst in data projects.
Unlike traditional application development, data projects follow a more iterative and data-driven approach.
The Business Analyst is involved from early requirement discovery through data validation, reporting, and final delivery, working closely with stakeholders, data engineers, and reporting teams.
The steps described below represent the typical lifecycle I followed during the project
Step 1: Understand the Client’s Expectations and Pain Points
This is the initial but most important stage.
First of all, as a BA, you must clearly understand:
1. What the client expects to achieve?
2. What problems they face?
3. Which reports need to be built
4. What key figures or metrics will appear in each report.
Most importantly, understanding the client’s pain points allows the team to propose the right solution.
The IIBA’s guidance on preparing for requirements elicitation clearly emphasizes this step as critical for successful outcomes.
As a result, during testing, the team can verify whether the delivered outputs truly solve the client’s issues.
Step 2: Exploring Data Source
The purpose of exploring a data source is, first and foremost, to become familiar with the data before you start working with it.
In some cases, data providers supply detailed documentation. In other cases, they may only share sample data.
Regardless, at this stage, the BA’s responsibility is to thoroughly explore and understand the following:
1. How many data sources need to be integrated
2. Connection methods, such as file-based connections, web connections, or API connections
3. The structure of the data response, including:
+ Fields or columns returned
+ Whether the structure is nested or flat
+ The order and hierarchy of the data
+ Data formats, such as CSV, JSON, or XML
+ Data types, including string, integer, and date values
+ In addition, the correctness and consistency of these data types
+ Entity relationships, which help identify what data is actually needed and how entities relate to each other, including:
. What entities (tables or objects) exist
. The purpose of each entity
. The relationship between them
As a result, any unclear or inconsistent issues should be synchronised with the data provider.
Step 3: Break Down and Estimate Effort
Once we understand both the client’s expectations and the data sources, the next step is to break down the work into detailed tasks. The purpose of this task is to translate client requirements and data understanding into a detailed execution plan with accurate timelines, resources, and risk control.
The BA collaborates with the development team to estimate effort and synchronise with the client to ensure:
1. The project timeline and delivery schedule meet expectations
2. The right priorities are set (e.g., which data source to migrate first, which reports to focus on first)
Step 4: Support Implementation
The purpose of Support Implementation is to ensure that the solution being built by the technical team correctly implements business requirements, follows agreed data logic, and produces accurate, high-quality data for reporting and analytics.
During development, the BA works in parallel with the technical team to:
1. Define transformation rules for each data layer
2. Document the data mapping between layers: a typical data flow looks like this:
Data Source → Staging → Data Storage→ Data Presentation
3. The BA also helps define data quality check rules, such as:
+ Missing mapping IDs between transaction and dimension data
+ Invalid data formats
+ Violations of general business rules
…
4. Clarify requirements and answer questions
+ Resolve functional questions from developers
+ Confirm assumptions and edge cases
5. Validate implementation outcomes
+ Ensure data aligns with business expectations
+ Review sample outputs and test results
Step 5: Reporting Support
The purpose of reporting support is to ensure that reports accurately reflect business requirements, present data clearly to users, and apply correct calculation logic
1. Designing the report UI layout
2. Defining logic for each field
3. Documenting the mapping between UI elements and report data fields
+ Documenting for logic calculation details
Step 6: Validation and Demonstration
After development is completed, the BA starts validating the data flow and ensuring that:
1. Data quality checks are applied correctly
2. Figures in reports match client expectations
3. If required, the BA prepares demonstrations for clients to explain report logic and gather feedback.
Conclusion
To summarize, the tasks above reflect my experience as a Business Analyst working in a data processing and improvement project.
Joining such a project allows you not only to understand data from a technical perspective but also to contribute to making business operations more efficient by transforming how data is managed, processed, and presented.