What is dark data?
First up, it is imperative to understand that dark data is not “illicit data” and does not bear negative connotations. Put simply, dark data is data that is collected by an organization through its processing chains but is not actively analyzed, enriched, or used. Anything that is not structured—such as data in log files and missing attributes from a transaction—and lacks labels can be considered dark data.
Here are some examples:
It is estimated that 40% to 90% of enterprise data is dark data, depending on the industry. According to a 2022 report from Enterprise Strategy Group, 47% of all data is dark data, on average, with a fifth of respondents saying more than 70% of their data is dark data.
What is the potential impact of dark data?
This paradigm shift will generate more data waiting to be unlocked, spurring an infinite progression of valuable intelligence and business value.
Dark data: The BFSI story
From the days of the earliest banks, BFSI companies have always used data to improve the customer experience. We have come a long way from personal customer information written on paper documents to credit scores, purchase histories, and telematics data used by an increasing number of insurance companies.
Yet, this long history of data collection is part of the problem.
As financial services companies evolve, old data loses its strategic and business value, eventually going dark. With today’s limitless cloud storage systems, it is far easier to make use of digital data than it is physical written records. Inevitably, the latter is filed away and eventually lost to the void of dark data.
BFSI companies are particularly vulnerable to this problem. The industry holds huge backlogs of stale data, 20% of which are made up of old document files. As smart contracts and blockchain transactions grow in popularity, such old data is rapidly losing its relevance and value.
The financial services industry’s heavily regulated environment is partly responsible for creating a culture that is hard-pressed to delete everything. The consequence of this “save everything often” mentality is that old data ends up taking up valuable storage space.
The way forward: Dark data best practices
The out of sight, out of mind nature of dark data has also led some organizations to stop properly managing, maintaining, and protecting such information. Over time, this can pose a major security risk to financial services companies and their customers.
With data privacy regulations like GDPR now in effect, consumers are more likely to take action against irresponsible financial services firms than any other sector, so dark data is a pressing issue in terms of data security compliance.
What exactly does an organization’s ideal data analytics practice look like? Here are four ways you can infuse data analytics into your team going forward.
It's time to look past the misconceptions surrounding dark data and approach it as a well of untapped potential. With the right strategies and technologies, organizations will be poised to reap the benefits.
How does Resulticks help you bring value from the dark data?
Meeting new consumer needs and expectations is critical to business success.Learn more
According to Deloitte digital report, Customer Experience is the top most priority for businessLearn more
According to Omnichannel Research Report India, more than two in five Indian marketers have setLearn more
Agile marketing is more than just a buzzword these days – it’s the new standard. The success ofLearn more
Excellent customer service and experience are key in the insurance industry.Learn more