Cognitive Bias in AI: Why Machine Learning Applications Are Like People

Main Message: When we train AI systems using human data, the result is human bias.

Abstract: We would like to think that AI-based machine learning systems always produce the right answer within their problem domain. However, in reality their performance is a direct result of the data used to train them. The answers in production are only as good as that training data.

But data collected by human means, such as surveys, observations, or estimates can have built-in human biases, such as the confirmation bias or the representative bias. Even seemingly objective measurements can be measuring the wrong things, or can be missing essential information about the problem domain.

The effects of biased data can be even more insidious. AI systems often function as black boxes, which means technologists are unaware of how an AI came to its conclusion. This can make it particularly hard to identify any inequality, bias, or discrimination feeding into a particular decision.

This presentation explains how AI systems can suffer from the same biases as human experts, and how that could lead to biased results. It examines how data scientist, testers and other stakeholders can develop test cases to recognize biases, both in data and the resulting system, and how to address those biases.

1 favorite thumb_down thumb_up 0 comments visibility_off  Remove from Watchlist visibility  Add to Watchlist

Outline/structure of the Session

Introduction, problem framing

Examples of training machine learning systems

Research and conclusions on biases and how they can creep into training data

How to recognize biased data and decisions

How to correct for bias

Learning Outcome

  • What kind of systems produce nondeterministic results
  • How machine learning results can be biased
  • How to recognize and understand bias in machine learning systems

Target Audience

Data scientists, machine language developers and testers.


Basic understanding of neural network concepts, especially training.

schedule Submitted 5 months ago

Comments Subscribe to Comments

comment Comment on this Proposal