Data Migration Best Practice: Orphan Analysis
What’s an Orphan? An orphan transaction is a transaction in a “child” table without an associated transaction in a “parent” table. For instance, an address record in the Address table without a link to a customer record in the Customer table. Orphaned records lead to various issues in business operations like marketing and business analytics. Challenges [...]
The role of data quality in ETL design: DQETL
Introduction Data integration is nothing new. Since the concept of data warehousing, data integration has been a major initiative for most large organizations. On the most common obstacles of integrating data into a warehouse has been the fact that assumptions about the state of the source data have been either false or flawed at best. [...]
Big Data … Little Data Quality
Is Big Data better Data Quality? Big Data is everywhere. Chances are you’ve used a big data solution today. However, are big data solutions delivering big data quality? High Availability versus High Data Quality Typically, Big Data solutions are designed to ensure high availability. High availability is based on the concept that it is more important to collect [...]
The Seven Habits of Highly Effective Data Quality
7 Habits of Highly Effective Data Quality I’ve been reading Stephen Covey’s The 7 Habits of Highly Effective People and I couldn’t help but notice the parallels between effective people and effective data management. In the book Covey discloses that there are principles, centered on self-discipline, that lead to success and fulfillment. Sounds great, right? The [...]
Data Quality Poll: Data Profiling and Data Migration
I’m interested to hear the thoughts of my fellow data quality practitioners about the role of data quality, more specifically data profiling, in the data migration process. Vote, leave a comment, whatever … I’m looking for some consensus around the approach.