The Six Dimensions of Data Quality
The Six Dimensions of Data Quality
Accuracy
Does data correctly represent the real-world entity? Example: Customer address matches their actual address. Verify against source of truth.
Completeness
Are all expected records and fields present? Example: Missing customer emails in a 95% complete dataset. Measure: NULL percentage per column.
Consistency
Is data the same across systems? Example: Revenue differs between finance and analytics dashboards. Reconcile across systems.
Timeliness
Is data available when needed? Example: Dashboard shows yesterday's data instead of today's. Measure: Data freshness SLA.
Validity
Does data conform to defined formats/constraints? Example: Email field contains valid email format. Enforce with schema validation.
Uniqueness
Are there duplicate records? Example: Same order appearing twice in the warehouse. Detect with COUNT(DISTINCT) vs COUNT(*).
Code Example
Data quality checks with SQL
-- Completeness check
SELECT
COUNT() AS total_rows,
COUNT(email) AS non_null_emails,
ROUND(COUNT(email) * 100.0 / COUNT(), 2) AS completeness_pct
FROM users;
-- Uniqueness check
SELECT
order_id,
COUNT() AS occurrences
FROM orders
GROUP BY order_id
HAVING COUNT() > 1; -- Duplicates!
-- Validity check
SELECT *
FROM users
WHERE email NOT LIKE '%@%.%'; -- Invalid emails
-- Timeness check
SELECT
MAX(updated_at) AS last_update,
EXTRACT(EPOCH FROM (NOW() - MAX(updated_at))) / 3600 AS hours_since_update
FROM fact_orders;
Best Practices
- Define quality dimensions for each dataset
- Implement automated checks in pipelines
- Create monitoring dashboards
- Establish escalation procedures
Interview Tips
- Explain the six dimensions
- How do you implement quality checks?
- Describe a data quality incident
Practice Problems
Design and implement a solution that demonstrates understanding of data quality dimensions in a data engineering context. Consider edge cases and performance.
Your implementation needs to handle 10x the current data volume. Identify bottlenecks and propose solutions.
Quiz
1. What is the primary benefit of data quality dimensions?
2. When would you choose data quality dimensions over alternatives?
Flashcards
Question
What is Data Quality Dimensions?
Click to reveal answer
Answer
Understand the six dimensions: accuracy, completeness, consistency, timeliness, validity, and uniqueness. Key for Data Quality.
Question
When to use Data Quality Dimensions?
Click to reveal answer
Answer
Use when requirements match its strengths. Consider trade-offs vs alternatives.
Revision Notes
Key Takeaways
- 1. Understand the six dimensions: accuracy, completeness, consistency, timeliness, validity, and uniqueness.
- 2. Master data quality dimensions for Data Quality
- 3. Practice with hands-on projects
- 4. Understand trade-offs and alternatives
Interview Tips
- • Explain data quality dimensions with real examples
- • Discuss trade-offs and alternatives
- • Show how this connects to the broader data stack
Cheat Sheet
Data Quality Dimensions — Quick Reference
Description
Understand the six dimensions: accuracy, completeness, consistency, timeliness, validity, and uniqueness.
Key Points
- Important concept in Data Quality
- Understanding this is essential for data engineering interviews
- Practice with real-world scenarios