IBM has launched an open-source toolkit to discern bias in artificial intelligence (AI) algorithms.
The project, in Python, presents developers with fairness metrics to check for bias in machine learning workflows, and bias mitigators to overcome bias and produce a fairer outcome. IBM has also made available a “gentle intro” video, a series of tutorials, and a complete API with its so-called AI Fairness 360 toolkit.
IBM described the release as “another step in our mission to democratise AI and bring it closer to developers”. It joins three new IBM open-source releases for AI developers, including a tool to defend deep neural networks against adversarial attacks, a Kubernetes platform for deep-learning frameworks, and an updated version of its MAX AI model exchange.
“It’s an exciting time. Our mission to grow the next generation of AI systems continues,” it said in a note to developers. “We hope you’ll try the toolkit, and we look forward to hearing your experiences and feedback. Join us, and together let’s raise AI right.”
Data does not always tell the truth; machines lie. Algorithmic bias means fairness and equality, the ultimate promises made by technology to re-write the rulebook, remain relative.
“Data reflects the social, historical and political conditions in which it was created. Artificial intelligence systems ‘learn’ based on the data they are given. This, along with many other factors, can lead to biased, inaccurate, and unfair outcomes.”
So says the AI Now Institute, an interdisciplinary research centre based at New York University, established to probe the social implications of AI. There are many examples of algorithmic bias, revealing inadvertently the implicit values of humans at the controls of computer systems.
In many cases, algorithmic bias is illegal. AI-governed systems that approve mortgage loans based on race, religion, or gender are illegal, IBM noted as an example. But not all bias in machine learning is illegal; sometimes it exists in more subtle ways, it said.
“For example, a loan company might want a diverse portfolio of customers across all income levels, and could therefore deem it undesirable if they are making more loans to high income levels over low income levels. Although this scenario is not illegal or unethical, it’s undesirable for the company’s strategy,” said IBM.
“Bias can enter the system anywhere in the data-gathering, model-training, and model-serving phases. The training data set might be biased towards particular types of instances. The algorithm that creates the model could be biased so that it generates models that are weighted towards particular variables in the input.
“The test data set might be biased in that it has expectations on correct answers that themselves are biased. Testing and mitigating bias should take place at each of these three steps in the machine learning process.”
