Really? We Could Be Getting It Wrong.

These days we rely on data to help us make decisions either predictive analytics or diagnostic analytics. Either way, you need collective intelligence from both humans and AI.

As the saying goes, “without data, you’re just another person with an opinion, without an opinion, you’re just another person with data”

AI provides very objective insights. But you will still need human to interpret and devise actions from the data. As humans, we are all subjected to cognitive biases that could derail our insights. The following are some key cognitive biases that could cause you to make the wrong diagnosis and eventually misguided execution. Let’s take a look here.

(1) Survivorship Bias

Survivorship bias arises due to the human tendency to focus over data points that are selected or successful over an underlying criteria.

For example, if we are set out to build a model to predict the characteristics of a successful entrepreneur it is very likely the available data is focused mainly about those who made their fortune in the business and not about those who failed in their entrepreneurial journey. We all know how it’s gonna end if we build the model with only positive labels.

(2) Confirmation Bias

Confirmation bias is the most popular and prevalent cognitive bias among all. We only hear what we want to hear, we only see what we want to see. Our strong belief system forces us to ignore any information that doesn’t conform with our preconceived notions.Confirmation bias refers to the need to prove a hypothesis and therefore to lean heavily on data that might lead this way. For example, a data collection may want to prove that Twitter users were more engaged with a TV show while it was on air – and may neglect to take into account that the greater cumulative engagement occurred in the days after viewers had had a chance to digest the episode. So recommendations could result in companies producing show-related online materials at the wrong time. One of the most galling examples of confirmation bias occurred after the 2016 US Presidential election, where polls were gathered based on a Clinton win, ignoring evidence that might prove otherwise.

(3) Availability Bias

Availability bias is a cognitive shortcut that results in over-reliance of the events or data that we can immediately think of. Often times, this bias may result in neglecting new sources of data that can potentially improve the model performance. Essentially, availability bias refers to the way in which people make decisions based only on information readily available to them. For example, a data collection may discover that respondents spend time looking at a website’s blog – and will use this information to develop the blog in order to convert to a sale or returning customer. However, the availability bias may cause other factors to be neglected due to the information that the blog is successful being the only piece relied on. 

For instance, the blog could be successful but could create very little engagement, meaning solely developing the blog would create no conversions. 

(4) Selection Bias

Selection bias refers to the sample the data has been collected from being unrepresentative of people on the whole. Imagine a console game has collected data on how long players spend on the game and then begin to use this in their game development. The data only looks at existing users, and doesn’t take into account factors that might convert a non-user to a fan of the game. For example, a survey found that Xbox gamers were overestimating the prevalence of the “red ring of death” console fault due to the likelihood of those who had experienced it to complete the survey.

(5) Clustering Illusion

Clustering illusion refers to the human tendency to find patterns in random events, when they don’t exist. Although randomness is prevalent and familiar to us, research in human psychology says we are poor at recognising it.

Since the goal of data exploration is to find patterns int he data, we are more susceptible of making mistakes due to this illusion. Proper statistical tests and being skeptical to do additional checks helps overcoming this illusion.

(6) Bandwagon Effect

Band Wagon Effect is a cognitive bias which explains the impulse to choose certain option or follow particular behaviour, because other people are doing it. This leads to a dangerous cycle, as more people continue to follow a trend makes it more likely that other people hop on the band wagon.

We observe this quite often in analytics, where practitioners often go after the buzz words such as deep learning/reinforcement learning without understanding the constraints and associated costs.

I hope this short article provides you ideas on what to be conscious about. Cognitive biases are unconscious decisions but it can be overcome if you are aware if them. When interpreting data, it is critical that we get rid of such biases as the wrong insights can be costly.

If you are interested in exploring more about cognitive biases, check out this wonderful article written by Buster Benson.