Have you been denied a business loan recently, for a reason you can’t figure out? Have you tried to get insurance and failed? It might have been the fault of artificial intelligence (AI) and something called data bias — which we’re about to discuss.
There’s a surplus of hype about artificial intelligence right now, especially fear of it taking away jobs. However, that’s just its most obvious fault, and it’s not destroying jobs that makes people like Elon Musk say they’re afraid of AI. After all, he makes a self-driving car.
The biggest problem of AI is that it does not do what we wanted it to do, which was eliminate human bias from business decision-making.
Instead, AI is making some insidious things happen that we thought we had under control. They’re called data bias.
Disclaimer: This is not intended to be legal advice; rather, by being aware of these issues, we can use AI more effectively and responsibly. This content is for informational purposes only; it does not represent any affiliation, endorsement or sponsorship with the third parties listed.
AI: The good and bad
Artificial intelligence is computers teaching computers to do things humans used to do, with the goal to eliminate the problems associated with human error. It is made possible because we now have extremely powerful, fast computers that can take large amounts of data and organize it into algorithms — which are rules. And the more data to which these rules are applied, the more accurate the rules are supposed to get.
AI is already operating pretty well in some areas of our lives.
For example, facial recognition software in the later models of the iPhone allows you to log in without a password. And my Roomba robotic vacuum cleaner has taught itself to get back to its docking station after wandering around my house. On the other hand, you can see where AI is not ready for primetime, because all the Uber self-driving cars were taken off the road after one of them hit a pedestrian recently.
But AI is rolling out in even more business situations, and it can hurt as well as help you in business — because artificial intelligence isn’t as “artificial” as you think. It’s a prediction model based on past performance. So if you did something in the past, but have completely changed your life, the algorithm might just miss that human factor.
Why do we have data bias?
The algorithms that run AI were made by humans, and that’s both the glorious and the damnable part of them (e.g., data bias). Just when we thought we had eliminated the problem of human bias by training computers, we find that human bias is inescapable. This is giving rise to the concern about data ethics among data scientists who now have to contend with data bias.
About 40 years ago, it was discovered by a couple of economists, Daniel Kahneman and Adam Tversky, that the human brain has certain cognitive biases that rule its behavior. No matter how “objective” a person tries to be, she is prey to those biases. We’re not rational in our decision making. The book they wrote, Thinking Fast and Slow, is a tough read, but the guys won a Nobel Prize for their discovery.
The list of cognitive biases is long and includes some benign familiar ones, like the tendency to endow animals with human traits, and some more frightening ones, like the tendency to believe things just because other people believe them. Taken together, they often make us act in non-rational ways — sometimes even against our best interests.
The cost of data bias
In business, bias can be costly. For example, confirmation bias is the tendency to search for, interpret, focus on and remember information in a way that confirms one's preconceptions. This can lead to the wrong product design decisions. And familiarity bias, the tendency to express undue preference for things merely because of familiarity with them, might prevent people from trying new things. That may make the introduction of new solutions into a market difficult.
Many medical practices, for example, still use fax machines to exchange data, often because the office staff knows how to use them. Familiarity bias is a hallmark of healthcare businesses, which are already overwhelmed with data and out of time to interpret it.
When computers came along, one of their promises for business was greater “objectivity,” because machines would overcome human cognitive bias. But data scientists have been coming to terms with the unintended consequences of relying too heavily on data.
In the financial industry, for example, data bias may cause results that violate the Equal Credit Opportunity Act (fair lending) for mortgages. That law, enacted in 1974, prohibits credit discrimination based on race, color, religion, national origin, sex, marital status, age or source of income. But ZIP code can be used as a proxy for race in some neighborhoods, and you can be denied a mortgage because you live in a neighborhood the algorithm thinks does not contain residents who can afford to pay back a loan. This bias has caused minorities to be disproportionately denied mortgages.
Garbage in, garbage out
Unfortunately, most data — except perhaps for that generated by sensors on machines — is generated by humans. A doctor still makes a diagnosis of a patient’s condition, and a police person takes down an accident report. These things are later fed into a computer, but they’ve already got the human bias built in. When that data is fed into a computer and used to train a machine, the machine’s learning reflects that bias.
“Garbage in, garbage out,” we used to say.
Microsoft thought it would show off its capabilities and made a self-learning Twitter bot a few years ago, and in less than a day the bot had adopted the biases of some of the worst people on Twitter, letting loose with racist, abusive posts. Microsoft had to take it down.
The more complex AI becomes, the more data scientists face a conundrum: because there is an element of bias in every data set, the more specifically we tailor a predictive model to the past the less likely it is to reflect the future.
Now, data scientists can control for things like data bias, but it still makes artificial intelligence less than perfect.
As a small business person, you need to watch artificial intelligences advances carefully, not because they might take away your business — but because they might affect you adversely as a human being.
