TDWI Data Quality Management: Techniques for Data Profiling, Assessment, and Improvement Training

Data quality is one of the most difficult challenges for nearly every business, IT organization, and BI program. The most common approach to data quality problems is reactive—a process of fixing problems when they are discovered and reported. But reactive data quality methods are not quality management; they are simply quality maintenance—a never-ending cycle of continuously fixing defects but rarely removing the causes. The only proven path to sustainable data quality is through a comprehensive quality management program that includes data profiling, data quality assessment, root cause analysis, data cleansing, and process improvement.

There is no prerequisites.

BI, MDM, and data governance program and project managers and practitioners; data stewards; data warehouse designers and developers; data quality professionals

• Techniques for column, table, and cross-table data profiling
• How to analyze data profiles and find the stories within them
• Subjective and objective methods to assess and measure data quality
• How to apply OLAP and performance scorecards for data quality management
• How to get beyond symptoms and understand the real causes of data quality defects
• Data cleansing techniques to effectively remediate existing data quality deficiencies
• Process improvement methods to eliminate root causes and prevent future defects

Module One: Data Quality Basics

• Data Quality Concepts 
  • Defining Data Quality
  • Integrity and Correctness
  • Data Quality and Metadata Quality
  • Rules as Metadata
  • Metadata and DQ Management
• Data Quality Processes
  • Quality Control, Assurance, & Management
  • Data Profiling
  • Data Quality Assessment
  • Data Cleansing
  • Root Cause Analysis
  • Process Improvement

Module Two: Profiling Data

• Data Profiling Concepts
  • Purpose and Processes
• Column Profiling
  • Extracting Values Metadata
  • Examining Values Metadata
• Table Profiling
  • Examining Dependencies
  • Keys and Uniqueness
  • Column Dependencies
• Cross-Table Profiling
  • Examining Redundancy and Relationships
  • Redundancy
  • Relationships
• Analyzing Data Profiles
  • Column Profiles
  • Values Frequency
  • The Story in the Profiles
  • Data Quality Rules
• Data Profiling in Practice
  • Profiling and Projects
  • People and Technology

Module Three: Assessing Data Quality

• DQ Assessment Concepts
  • DQ Assessment Defined
  • Subjective Assessment
  • Objective Assessment
  • Assessment in DQ Management
• Subjective Assessment
  • Subjective Assessment Process
  • A Survey of Data Quality Perceptions
• Objective Assessment
  • Objective Assessment Process
  • Planning and Preparation
  • Cataloging Data Quality Rules
  • Cataloging Data Quality Errors
  • Testing and Tuning Rules
  • Reporting and Analyzing Data Quality
• Assessment in Practice
  • Assessment and Projects
  • People, Roles, and Skills

Module Four: Fixing Data Quality Defects

• Data Cleansing Concepts
  • Data Cleansing Defined
  • Techniques
  • Cleansing Process Overview
  • Planning & Preparation
• Procedural Data Cleansing
  • Names and Addresses
  • De-Duplication and Consolidation
• Rule-Based Data Cleansing
  • Data Cleansing Rules
  • Rules Processing
• Data Cleansing in Practice
  • Data Cleansing Workflow
  • Cleansing and the Data Warehouse
  • Data Cleansing and Projects
  • People, Roles, and Responsibilities

Module Five: Preventing Data Quality Defects

• Root Cause Analysis
  • RCA Overview
  • The RCA Process
  • Causal Modeling
  • Five Whys
  • Fishbone Diagrams
  • Causal Loop Diagrams
• Process Improvement
  • Process Improvement Principles
  • Process Improvement Overview
  • Prerequisites
  • Target Processes
  • Change Planning
  • Action Planning
  • Executing Change

Module Six: Summary and Conclusion

• Sustaining Data Quality
  • Awareness, Accountability, Action
  • Mistakes to Avoid
• References and Resources
  • To Learn More