Dark data is a subset of big data that organizations will collect and store from their business activities but are unable to use for any of the decision-making processes. The amount of data that the organizations collect for their business purposes are increasing over the years. Organizations store these data in different systems and applications. These systems do not have sync features hence do not support seamless sync. Organizations are spending a lot on technology and on talent to grab the value for the data. In spite of these efforts by the organizations, it is pointed out that on average 60-70 percent of data goes unused for data analytics. It is also found by the manufacturing industry that almost 90 percent of the information generated by the sensors and analog-to-digital conversions never get used. If the organizations work on using these unused data, it can make a huge difference in improving business operations and decisions

Depending on the company and industry, there is a wide range of examples for dark data. Some are listed as Customer call records, Geolocation data, Surveillance video footage, financial statements, previous employee details, website log files, email correspondences, and so on.Most of these unstructured data never get used. According to the ICD, 90% of the data which are created is never processed or analyzed you can always compare these dark data to the hidden iceberg. Only the tip of the data collected is analyzed and the decision is taken in the business operations and the tip of these data is also used to make the decision but the dark data will be hidden beneath. The dark data can also be analyzed and the organizations may get benefitted better. Analyzing these unstructured data requires tools. The tools that can be used to process the data and present a meaningful analysis in the form of different types of data visualization techniques and in terms we can you better insights for decision making.

How to tackle these dark data should be dealt with the proper manner. The first step to deal with this dark data is by discovering all the unstructured data hiding in the organization. What type of data the organizations should search for? The list goes on the E-mails, log files, images, call records, social posts and all these data will be stored in the cloud, in the laptops, in dropbox, and so on. The report says that 77% of the respondents agreed that finding the data and capturing the dark data should be a top priority. Then the unstructured data which are discovered needed to be mapped against their structured operational data to find the way to the untapped opportunities. Once these dark data is aligned and analyzed can be used for additional insights for value creation to the organization.

In today’s technology-driven world, data is considered money. Till now the application of analytics was limited to structured data but now the situation has changed. The recognition of dark data‘s importance and also advancement of technology may be the key to improving the customer relationships, greater efficiency that may also lead to higher profits.

Prof.Roopa U
Assistant Professor


Subscribe to get updates about our new posts!

We don’t spam! Read our privacy policy for more info.