Analytics QA
How a small analytics QA toolkit protects dashboard trust.
A dashboard can look polished and still be wrong. The risk usually appears before the dashboard layer: missing dates, null KPIs, duplicate rows, inconsistent naming and silent data spikes inside exports that nobody has time to inspect manually.
The tool is not meant to replace analysts. It gives analytics teams a repeatable way to catch obvious reporting risks before the meeting where numbers are questioned.
The situation
Imagine a marketing or eCommerce team receiving weekly dashboard exports from multiple sources: retailer files, campaign reports, web analytics data and manually maintained mapping sheets. The dashboard team is expected to report sessions, clicks, revenue, orders, conversion rate and media performance with confidence.
The difficult part is that each source can fail differently. A retailer export may skip a date. A campaign field may use a new naming format. A country code may not match the dashboard mapping. A revenue column may be blank for one market. A duplicate row may inflate a KPI. None of those issues need to be dramatic to create a credibility problem.
The business problem
When data quality checks live only in someone’s memory, reporting becomes fragile. Stakeholders ask why numbers changed, analysts lose time re-checking old files, and project managers cannot easily tell whether the problem is tracking, ingestion, mappings, dashboard logic or source data.
That is why I built the Analytics QA Toolkit as a portfolio project: a lightweight Python/pandas workflow that reads CSV or Excel exports and produces a clear QA report. It is intentionally practical, not over-engineered.
What the toolkit checks
| Check | Why it matters | Example risk |
|---|---|---|
| Missing dates | Find gaps in expected reporting periods. | A weekly report skips Sunday and undercounts revenue. |
| Null KPIs | Detect important metrics that are blank. | Orders are missing for one market while sessions are present. |
| Duplicate rows | Catch repeated records at the expected grain. | One campaign/date row is counted twice. |
| Spike detection | Flag abnormal jumps that need investigation. | Revenue jumps 400% because a file used the wrong currency or grain. |
| Mapping validation | Compare country, brand or source fields to approved values. | A new country label breaks dashboard filters. |
| Naming consistency | Catch inconsistent campaign or stream naming. | Paid social campaigns split into multiple dashboard buckets. |
How it helps a real team
The value is not only technical. The value is operational. A QA report gives the team a shared object to discuss: here are the failed checks, here are warnings, here is the affected file, here is the likely business risk, and here are the next actions.
That changes the stakeholder conversation from “the dashboard looks wrong” to “we found three source-file issues and one mapping issue; the dashboard logic is probably not the first problem.” That distinction saves time and reduces blame.
The workflow
- Export or receive the reporting file.
- Run the QA toolkit against the file and mapping config.
- Review the Markdown and CSV QA report.
- Route each issue to the right owner: source data, tracking, naming, vendor, mapping or dashboard logic.
- Attach the report to the delivery ticket or monthly reporting documentation.
Why this matters for analytics project management
Analytics delivery often fails in the middle layer. The tools exist. The data exists. The dashboard exists. But nobody owns the validation logic, documentation and follow-up rhythm. A small QA workflow creates a bridge between technical checks and business confidence.
That is the exact space I like to own: enough technical depth to inspect the problem, enough project management discipline to coordinate the fix, and enough stakeholder communication to make the outcome usable.
What I would improve next
The next version should support configurable severity levels, richer Excel summaries, GitHub Actions examples for scheduled QA, and Streamlit upload screens for non-technical users. The first version stays intentionally simple so it can be understood, adapted and trusted.