May 17, 2020

Knowing the limitations of big data

Big Data
Data Analytics
Data Privacy
Gaurav Gupta
3 min
Big data
Big data is feeding a revolution in how businesses make decisions and evaluate performance. Regular use of data and analytics to make decisions about bu...

Big data is feeding a revolution in how businesses make decisions and evaluate performance. Regular use of data and analytics to make decisions about business strategy and operations is now commonplace. Any operational metrics a business leader wants can be at their fingertips within seconds of it being requested. This can be a good thing – a great thing, even. 

Yet leaders need to be judicious in their use of data especially around privacy and sensitivity concerns. The well-known case of the department store Target using analysis of buyers’ shopping trends to market selectively to expectant mothers is a cautionary tale for leaders who want to rely on big data and analytics. While a great idea in theory, it backfired when a teenager who was hiding her pregnancy from her parents was called out by the coupons she received in the mail based on Target’s data analytics. 

See also:

A search for big data failures will reveal many such cases and each one of them stems from a reliance on data analytics without human intervention. The problem is not with big data and analytics but with how we choose to use them.

Another example is from the use of traffic light-based dashboards to review business performance. Hundreds of thousands of years of evolution have helped our brains develop a mechanism for spotting threats and problems. Our survival instincts cause us to want to eliminate threats. These mechanisms lead us to focus on the few ‘problem’ metrics that are invariably red, at the expense of potential upside opportunities represented by already ‘green’ metrics. 

Big data often leads managers to rely too much on the data and abdicate decision making. Using what the data tells you to inform a considered decision is of course commendable, but blindly following it without question or leaving room for experience and gut instincts can lead to poor decisions.  

In our effort to make what appear to be fact based and unbiased decisions we too often either ignore the uncertainty in our inputs or overstate the conclusions of our data analysis. Senior leaders need to set an example by ensuring that their teams can challenge the conclusions data lead towards, and talk openly about their own biases and opinions. Lots of research has shown the value of ‘gut feelings’ and the ability of the human brain to subconsciously process information and draw on experiences to arrive at intuitive decisions. The best decision making combines intuitive gut feeling with robust data analysis. 

The ever-increasing availability of data and the introduction of automation and artificial intelligence may seem to reduce the need for leadership and decision making, however the opposite is true. Big data and artificial intelligence allow us to ask ever more complex questions.  Making decisions that combine the answers they provide with intuition and experience will be the challenge for leaders. Data supports and amplifies the ability of people in a business to make well informed decisions and take action but should not be the sole driver for these decisions. 

The individuals, teams, and organisations that are most effectively able to harness the potential of big data will be those that are best able to integrate it into their existing decision-making methodology. 

Gaurav Gupta, Affiliate at Kotter

Share article

Jun 4, 2021

Logi Analytics Webinar: Meet the speaker

2 min
Join Technology Magazine and Logi Analytics for this exclusive webinar as we explore the revolutionary power of next-generation embedded analytics

Data allows business owners to leverage digital insights and embrace the power of data-driven business intelligence to make more informed decisions that are better for business growth and evolution. By using data to drive its actions, an organisation can contextualise and/or personalise its messaging to its prospects and customers for a more customer-centric approach.

BizClik Media Group and Logi Analytics invite you to explore next-gen embedded analytics in our live webinar. There’s still time to sign up for the event entitled ‘Application Imperative: How Next-Gen Embedded Analytics Power Data-Driven Action’, which is taking place on 10 June at 4 pm BST.

The webinar will be led by Constellation Research’s Principal Analyst, Doug Henschen, who focuses on data-driven decision-making. Henschen’s Data-to-Decisions research examines how organisations employ data analysis to reimagine their business models and gain a deeper understanding of their customers.

Henschen's research acknowledges that innovative data analysis applications require a multi-disciplinary approach starting with information and orchestration technologies, continuing through business intelligence, data visualisation, and analytics, and moving into NoSQL and big-data analysis, third-party data enrichment, and decision-management technologies.

Constellation Research is a technology research and advisory firm based in Silicon Valley. Prior to joining Constellation, Doug Henschen led analytics, big data, business intelligence, optimisation, smart applications research, and news coverage at InformationWeek.

What will the webinar cover?

This exclusive webinar will explain next-gen embedding capabilities that will enable your company to:

  • Eliminate unproductive toggling between transactional interfaces and purely analytic dashboards
  • Drive two-way interactions between app features and embedded analytics to drive data-driven action
  • The compounding impact of embedded analytics on your overall ROI
  • Harness analytics as triggers for automated workflows and suggested next-best actions
  • Enable developers to build quickly without coding while customising self-service options for end users

Logi Analytics is the only developer-grade analytics solutions provider focused exclusively on embedding analytics in commercial and enterprise applications, empowering the world’s software teams with the most intuitive data analytics solutions and a team of dedicated professionals invested in your company’s success.

Why not sign up today to find out exactly how Logi Analytics can revolutionise your data analytics game?

We look forward to seeing you there!

Share article