Examples That Illustrate Why Transparency Is Crucial In AI

AI is rapidly transforming the world of business as it becomes increasingly woven into the fabric of organizations and the day-to-day lives of customers.

However the speed of this transformation creates risks, as organizations struggle with challenges around deploying AI in ways that are responsible and minimize the risk of harm.

One of the cornerstones of responsible AI is transparency. AI systems – including algorithms themselves as well as the data sources – should be understandable so we can comprehend how decisions are made and ensure it’s done in a fair, unbiased and ethical way.

Today, many businesses that use AI are taking steps towards ensuring this happens. However there have been cases where use of AI has been worryingly opaque.

Here we will look at real-world examples, good and bad, that illustrate the benefits of transparent AI and the dangers of obscure or unexplainable algorithms.

Transparent AI Done Well

When Adobe released its Firefly generative AI toolset, it reassured users that it is open and transparent about the data used to train its models. Unlike other generative AI tools (eg OpenAI’s Dall-E, it published information on all of the images that were used, along with reassurance that it owned all the rights to these images, or that they were in the public domain. This means users can make informed choices about whether to trust that their tool hasn’t been trained in a way that infringes copyrights.

Salesforce includes transparency as an important element of “accuracy” – one of its five guidelines for developing trustworthy AI. This means that they take steps to make it clear when AI provides answers that it isn’t sure are completely correct. This includes citing sources and highlighting areas that users of their tools might want to double-check to ensure there haven’t been mistakes!

Microsoft’s Python SDK for Azure Machine Learning includes a function called model_explainability, which in recent versions is set to “true” by default. This gives developers insights into interpretability, meaning they can understand the decisions and ensure they are made fairly and ethically.

Cognizant recommends creating centers of excellence to centralize AI oversight, allowing best practices around transparency to be adopted across an organization. Understanding what steps are being taken to ensure AI is accountable and explainable means these practices can be iterated across all AI projects in a responsible way.

Transparent AI Done Badly!

OpenAI – creators of ChatGPT and the image generation model Dall-E – have been accused of failing to be transparent over what data is used to train their models. This has led to lawsuits from artists and writers claiming that their material was used without permission. However, some believe that OpenAI’s users could themselves face legal action in the future if copyright holders are able to successfully argue that material created with the help of OpenAI’s tools also infringes their IP rights. This example demonstrates how opacity around training data can potentially lead to a breakdown in trust between an AI service provider and its customers.

Other image generators – including Google’s Imagen and Midjourney – have been criticized for overly depicting professionals as white men and historical inaccuracies, such as showing the US Founding Fathers and German Nazi soldiers as people of color. A Lack of transparency in AI decision-making hinders developers from easily identifying and fixing these issues.

In banking and insurance, AI is increasingly frequently being used to assess risk and detect fraud. If these systems aren’t transparent, it could lead to customers being refused credit, having transactions blocked, or even facing criminal investigations, while having no way of understanding why they have been singled out or put under suspicion.

Even more worrying is the dangers posed by non-transparency around systems and data used in healthcare. As AI is increasingly used for routine tasks like spotting signs of cancer in medical imagery, biased data can lead to dangerous mistakes and worse patient outcomes. With no measures in place to ensure transparency, biased data is less likely to be identified and removed from systems used to train AI tools.

The Benefits Of Transparent AI

Ensuring AI is deployed transparently is essential for building trust with customers. They want to know what, how and why decisions are being made with their data and have an inherent distrust of “black box” machines that refuse to explain what they are doing!

On top of that it allows us to identify and eliminate problems that can be caused by biased data, by ensuring that all the data used is thoroughly audited and cleansed.

Last but not least, the amount of regulation around AI is increasing. Legislation such as the upcoming EU AI Act specifically rules that AI systems in critical use cases must be transparent and explainable. This means that businesses using opaque, black-box AI could leave themselves open to big fines.

Building transparency and accountability into AI systems is increasingly being seen as a critical part of developing ethical and responsible AI. Although the highly complex nature of today’s advanced AI models means this isn’t always straightforward, it's a challenge that will have to be overcome if AI is to fulfill its potential for creating positive change and value.