Saturday, 19th September 2020

Data warehousing dissatisfaction

Siloed apps and data, outdated legacy tech, and slow and manual data movement are blocking success and costing organizations millions.

New research published by SnapLogic reveals that 83% of organizations are not fully satisfied with the performance and output of their data management and data warehousing initiatives. IT leaders cite a growing number of disconnected applications and data sources, outdated legacy systems, and slow and manual data movement as reasons for their frustration, all of which are stalling progress and costing them millions.


The new study, conducted by independent research firm Vanson Bourne, found that the average organization has 115 distinct applications and data sources across their enterprise, but almost half of them (49%) are siloed and disconnected from one another. Respondents expressed clear concern, with 89% of IT Decision Makers (ITDMs) from organizations where these apps and systems aren’t integrated worried that these data silos are holding them back. Indeed, ITDMs confirmed they are losing, on average, more than $1 million USD annually due to poor data management.

Still, organizations are committed to building a data-driven enterprise and are funding their data management initiatives accordingly. Over three quarters (76%) of survey respondents indicated that their companies have increased their data budgets over the past year.

Delivering on the Promise
ITDMs are investing in data storage solutions such as data warehouses and data lakes to help them make the most of their valuable data assets and deliver on the promise of agile analytics and actionable business insights. With this data, respondents hope to achieve faster and more informed decision making (70%), an improvement in business operations (64%), accelerated innovation and new business development (63%), an enhanced customer experience (61%), and more efficient and productive employees (57%). However, the process of getting data into a data warehouse or data lake is not always straightforward, all too often inhibiting progress and success.

Key research findings:

Nearly nine in ten (88%) ITDMs experience challenges trying to load data into data warehouses, with the biggest inhibitors being legacy technology (49%), complex data types and formats (44%), data silos (40%), and data access issues tied to regulatory requirements (40%).

Nearly half (48%) of the data migrated into organizations’ data warehouses or other storage solutions requires cleaning before it can be useful. This likely contributes to the more than 4 hours of time wasted on average, per employee per week, resolving issues around missing data, duplicate data, or data that needs to be reformatted.

ITDMs indicated that, on average, 42% of their data management and warehousing processes are currently being done manually, but could be automated, which would save valuable time, resources, and money.

As a result of these issues, almost all respondents (93%) believe improvements are needed in how they collect, manage, and analyze data.

“We are not only in a period of exponential data growth, but also one in which data lies at the heart of the successful adoption of technologies such as AI, automation, and advanced analytics,” said Craig Stewart, CTO at SnapLogic. “This means that having an effective data management and warehousing strategy is business-critical”.

Stewart continued: “To get data warehousing right, organizations must break down data silos, retire legacy tools, automate manual processes, and accelerate the integration and movement of data across the enterprise. Only then can teams across the business truly harness the full power of data to drive better decisions, actions, and outcomes.”

Study identifies pressures to capitalize on a new era and the opportunity found in data.
With Cluster Linking, Confluent Platform 6.0 provides a single platform to build a globally connecte...
More than 73,000 users collaborate on new online platform set up by the European Open Science Cloud...
Long-standing MariaDB user turns to the source for cloud database-as-a-service.
Announced at the 14th annual MIT Chief Data Officer and Information Quality Symposium, study finds C...
Talend Data Fabric now certified on Cloudera Data Platform.
New joint go-to-market initiative will accelerate cloud analytics with BigQuery and Robust, proven S...
Leading digital automotive marketplace embraces the power of data with Looker.