Friday, 30th October 2020

Data virtualisation – the Chief Data Officer’s best friend

The rise of the Chief Data Officer (CDO) has been meteoric in recent years. Despite being one of the newest executive roles for many businesses, research suggests the number of companies hiring a CDO has more than quadrupled since 2012. With Dixons Carphone Warehouse and even the Welsh Government recently joining this figure, investment in the coveted position doesn’t look set to slow down anytime soon. By Alberto Pan, Chief Technical Officer at Denodo.

And it’s no wonder! After all, data is every business’s most valuable asset, regardless of size or sector. The CDO’s role is to both manage and protect this asset; ensuring its quality so the wider team can utilise the information gleaned from it in order to inform the wider business strategy.


Add to this the explosion of big data, alongside the introduction of new regulatory requirements – such as the GDPR. Both have meant that organisations have finally accepted the responsibility for data quality rests with them. Needless to say, in light of this, the role of CDO has become more important than ever before.


However, hiring a CDO is only the start of a long journey towards ensuring a business is making the most of its data and achieving a good level of governance. There are likely to be many bumps in the road and CDOs will need the correct tools and technologies – as well as support and buy-in from other board-level members – if they are to have any hope of success.


Data quality and governance is NOT guaranteed


All data, no matter where it is stored within an organisation, is within the CDO’s remit. That data might be gathered via the Internet of Things or social media, it might be structured or unstructured, it might be in a data lake or a data warehouse. With so much data to consider, getting a handle on data quality can be challenging.


For instance, whilst data lakes and data warehouses both have their benefits when it comes to storing data, if they are in different locations and utilise different technologies, it can be difficult for the CDO to combine data from each to cleanse and contextualise – both of which are essential to ensure quality and therefore effectively implement governance. In the same way, if some consumers and applications access the data warehouse while others target the data lake, the CDO will be forced to enforce data governance and security policies in both systems, which is costly and prone to inconsistencies.


Probably an even more pressing problem is how to accelerate the provision of data to consumers in formats they can use effectively, while at the same time conforming to standard glossaries and policies. The traditional method for this involved creating new carefully curated physical data marts for each specific information need. But this simply does not scale for modern organisations today, where all business decisions, even the ones taken by middle managers, need to be supported by data. Now, we need to deliver curated data to many different types of users, and for a much wider variety of needs.


Enter data virtualisation


To increase agility and facilitate governance, instead of physically transferring information, businesses need to play the data ‘where it lies’, accessing and joining it across multiple stores to create a logical data warehouse. As the Market Analyst Gartner said: “stop collecting data, start connecting to data”.


Data virtualisation has become the key to achieving this. Rather than replicating data, and moving it to a new, consolidated repository, this type of technology connects users and applications to a view of the data, no matter where it resides. It stitches together information abstracted from various underlying sources – without the need to actually move it – and delivers it to consuming applications in real time. This approach means every time a person, or application, runs a query that retrieves data from two or more disparate sources, the results are joined into a single response. And users can do this from any BI, data science or data vitulisation tool.


CDOs can utilise this method in several ways to drive data governance and ensure data quality. First, Data virtualisation allows the agile creation of virtual views that can expose curated data adapted to the needs of each consumer without needing to physically replicate data in a new system. This is much faster than the traditional methods.


Second, as the key component in the logical data warehouse architecture, data virtualisation, can provide a single access point to any data in the enterprise. That means the CDO has a single entry point to apply consistent data quality, security and governance policies across all data sources and consumers. For instance, data virtualisation allows a semantic layer to be created, where data is offered according to the formats and definitions specified in company-wide glossaries and policies, and data quality rules are checked and enforced.


This also grants the CDO heightened visability when it comes to the quality of an organisation’s data, meaning that any potential data governance issues are flagged, and therefore dealt with before data delivery.


Once hired, many CDOs face a long road towards providing agile data delivery, achieving data governance and ensuring data quality. They often inherit systems that aren’t fit for purpose and need to prove their worth. Along the way, they’ll likely come up against hurdles and challenges and data virtualisation will emerge as more than a tool, becoming a means of support and the trusted companion they’ve been looking for.


A recent HP panel discussion sought to provide some answers to this question – topics covered includ...
Pascal Geenens, director of threat intelligence, Radware, offers some fascinating insights into some...
You may be surprised to learn that one of the first computer viruses to bring millions of computers...
How IT managers protect corporate networks from targeted attacks By Chris Connell, Deputy Vice Pre...
Why business decision makers should expand their network security strategy, By Chris Connell, Deput...
By Miles Tappin, Vice President, EMEA at ThreatConnect.
By Mikkel Stegmann, Principal Scientist at Fingerprints.
Digital transformation needs security at heart, says Jonathan Whiteside, Principal Technical Consult...