Zephyrnet Logo

Start Small and Scale Up with Data Profiling, Data Quality, and Data Governance

Date:

Organizations today are rallying business users around using data to make better business decisions. Business users want to know where that data lives, understand if people are accessing the right data at the right time, and be assured that the data is of high quality. But they are not always out shopping for Data Quality or Data Governance tools.

Rather, the trend we are seeing with our customers and in the marketplace is that business leaders are looking for quick ways to answer these questions. They are having a lot of success with a bite-sized way of doing things, where companies start small, get a quick win, get the business comfortable with the approach, and scale up from there. 

TRAIN TO BECOME A CERTIFIED DATA MANAGEMENT PROFESSIONAL

Our online training program in CDMP preparation provides a solid foundation of different data disciplines.

The business doesn’t care how the capabilities get deployed, as long as it doesn’t take a long time, it doesn’t cost a lot, and they see results. But the IT organization that lives and breathes data and infrastructure has a lot of requirements to make this possible. So, the challenge for vendors is to bridge these polar-opposite dimensions, making data access easy for the business and making it easy and cost-effective for IT to deploy new data capabilities.  

Of course, doing that takes new tools or requires better leveraging of solutions already implemented. But it also means moving away from platforms that take years to implement, cost in the millions of dollars, and are designed to crawl the entire universe of an organization’s data and clean and catalog it all. By the time you get to the end of this kind of project, everything has changed, and you have not effectively achieved the business objectives.

Instead, we are seeing companies embracing smaller solutions for Data Quality and Data Governance that can be replicated across multiple use cases, and the marketplace is responding. Today, organizations can acquire Data Management solutions composed of modular components that can be used as standalone tools but also integrate with each other and across other market-leading toolsets. That lets organizations grow as they acquire data skills and progress Data Quality and Data Governance.  

Behind this trend is a business-centric approach to deciding what tactical steps to take when. For example, think about a key KPI for the business, such as sales pipeline growth, as a place to get a quick win. You start with understanding what data feeds the pipeline and where it comes from. It’s most likely multiple repositories, such as CRM for customer data and seller activity, a home-grown application for tracking RFP responses, and the ERP system for contract data. 

Today, we see data profiling tools appearing as standalone capabilities that serve as a necessary starting point for data initiatives such as our pipeline report example. Rather than being embedded in either Data Quality or Data Governance toolsets, standalone data profiling is serving as a bridge between them. Data profiling produces basic statistics both about the data itself – what the fields are and the state of the data, which feed into Data Quality tools – and the about metadata that feed Data Governance.  

Once profiling is complete, it’s a straightforward next step to apply Data Quality rules that validate a field or flag an error. Getting started with Data Governance, however, requires addressing significant cultural challenges. In fact, most of the data professionals in a recent study from Drexel University’s Lebow Center for Analytics said cultural issues are the leading roadblock to Data Governance.

The reason is that Data Governance is most successful when there is up-front participation from business users. They hold the knowledge that data stewards need to apply the business context and business logic around what those data assets are when you catalog metadata. It takes a collaboration between the data steward and the business partner in the sales organization, going back to our pipeline project. A Data Governance tool can assist in the process both by being the repository of the information gathered and by facilitating the workflow between the two organizations. 

Here again is where those quick wins can help. They demonstrate that there is significant value for business users on the back end of a project. Ultimately, you want to get to having a data catalog that business users can log into, identify what data repositories would go into that pipeline report, and visualize the relationships and ownership. They can also dig deeper to find out what fields are available, what the definition of each field is, and what the Data Quality metric is for each. 

With this ease of use, you get business users who are enabled to access and interact with data in ways that we haven’t seen before. What we’re really encouraged about is seeing organizations thinking more proactively about Data Quality and bringing it together with Data Governance. Through these efforts, organizations can achieve data integrity – data that is accurate, consistent, and presented with its business context – and trust the data-driven business decisions their people make. 

spot_img

Latest Intelligence

spot_img

Chat with us

Hi there! How can I help you?