Cognitive Bias in AI: Why Machine Learning Applications Are Like People
Main Message: When we train AI systems using human data, the result is human bias.
Abstract: We would like to think that AI-based machine learning systems always produce the right answer within their problem domain. However, in reality their performance is a direct result of the data used to train them. The answers in production are only as good as that training data.
But data collected by human means, such as surveys, observations, or estimates can have built-in human biases, such as the confirmation bias or the representative bias. Even seemingly objective measurements can be measuring the wrong things, or can be missing essential information about the problem domain.
The effects of biased data can be even more insidious. AI systems often function as black boxes, which means technologists are unaware of how an AI came to its conclusion. This can make it particularly hard to identify any inequality, bias, or discrimination feeding into a particular decision.
This presentation explains how AI systems can suffer from the same biases as human experts, and how that could lead to biased results. It examines how data scientist, testers and other stakeholders can develop test cases to recognize biases, both in data and the resulting system, and how to address those biases.
Outline/structure of the Session
Introduction, problem framing
Examples of training machine learning systems
Research and conclusions on biases and how they can creep into training data
How to recognize biased data and decisions
How to correct for bias
- What kind of systems produce nondeterministic results
- How machine learning results can be biased
- How to recognize and understand bias in machine learning systems
Data scientists, machine language developers and testers.
Basic understanding of neural network concepts, especially training.