So, the Washington Post published a fascinating article about a woman who described how she played with LinkedIn, pretending to be two different people —a man and a woman—with the same skills. Needless to say, she had better success as a man than a woman. The question, of course, is why and what does it tell us more generally about decision-making in a world in which AI transforms our information?
And here is the basic problem: Imagine a human activity with a bias. For example, preferring men for jobs compared to women. What is AI going to do? AI is going to repeat human behavior because success is defined as making a choice. AI will not going to follow real people in the workplace and evaluate them five years later. These models work on what’s called reinforcement learning. They figure out what people say yes to, what they say no to, and they reinforce those choices.
Now, later, the algorithm cannot evaluate whether the men are less successful than the women or not. It evaluates what people choose and because of this approach, AI is likely to amplify human bias.
I’ll give an example. Many years ago, we did research on the importance of height in online dating and we found that height was hugely important. Let’s say that I’m 5’9”. If I wanted to be as attractive as somebody who is 5’10” I would have had to make about $30,000 more a year. That’s a lot for a one-inch difference. Why did this happen? Because the algorithms take height very seriously. And a woman that sets her height minimum to a man who is 5’10” will never see somebody like me who is very sweet, but only 5’9”. So, algorithms have a way of exaggerating things. Do women really care about one inch to the tune of $30,000 a year (This was quite a few years ago and the dollar was worth more)? I don’t think so. The algorithm can amplify something like this. And the same thing can happen in other areas.
The point is we need to be very worried about this. We need to understand that algorithms are tools that are only as good as the quality of the data that they’re receiving. And if they’re getting biased data in, they’ll put biased data out. And not only biased data out, the algorithms will potentially be even more biased than people, because it will be very hard for it to figure out nuances.
When we used to do equations and the statistical models were equations, we could look at a coefficient. The way that AI models work, it’s very hard to figure out this out. These algorithms measure lots of things and they capture lots of things and it’s very hard to figure out where the attention is focused.
Anyway, all of this is to say that these algorithms are amazing. They’re going to do lots for us. But there’s a danger that when they learn from human activity that contains a bias, there’s a chance the algorithms will create more of that bias and, moreover, that we will not know exactly where the bias is coming from. Let’s be careful when we use these algorithms on data that has a built-in human bias.
Good luck to all of us.
