The Problem of Bias
An AI model is only as good as the data it's trained on. Because LLMs are trained on vast amounts of text from the internet, they learn the biases that are present in that text. If the data reflects historical inequalities or stereotypes, the AI will learn and can even amplify those biases.
Case Study: Biased Hiring Tool
A major tech company created an AI tool to help them screen CVs. The goal was to find the best candidates. They trained the model on the CVs of people they had hired over the last 10 years. However, because the company had historically hired more men than women, the AI learned that male candidates were preferable. It started penalizing CVs that included the word "women's" (e.g., "women's chess club captain") and downgrading graduates from all-women's colleges. The company had to scrap the tool.
Critical Thinking
Who is responsible for the bias in the AI hiring tool? The programmers? The company? The historical data? Explain your reasoning.