How can we build explainable AI?

Post by
Executive Office
Learn how data scientists can build explainable AI

Why do we need explainable AI?

With the increasing complexity of AI models and the use of these models in real-world applications, the impact these AI models have on our lives is incalculable. It will not be a stretch to say that these AI models know us better than we know ourselves! While this is quite amazing, it is also, at the same time, quite frightening. With increasing algorithmic complexity, we are paying the cost of decreasing interpretability and trust. It means that the more complex a model is, the less likely we understand how it works. It is the main reason why these AI models are called black-boxes: we do not know the why behind these complex algorithms.

Accuracy vs. Interpretability. ExplainX Internal

The reality is that these algorithms are becoming a part of our daily lives: from the Facebook newsfeed to TikTok to our recent online credit card applications are interactions all driven by power AI algorithms — most of which are black-boxes. As we are actively using these algorithms in mission-critical use cases like predicting cancer or accidents or evaluating a candidate based on text representing them, we need frameworks and techniques to help us navigate and open up these black-boxes.

In short, model developers need to answer these five major questions:

  1. Why did my model give this prediction?
  2. Does my model prediction align with business/domain logic? If the answer is no, then identify and treat irrelevant features that have more weight.
  3. Is my model behaviour consistent among different subsets of data? If the answer is yes, then compare and contrast to identify where and how the behaviour varies.
  4. Is my model bias towards a specific feature within my dataset? If the answer is yes, then find where that bias is and get rid of it?
  5. What can I do to influence the prediction or what actions can I take to achieve a desirable outcome?
  6. Optional: does my model enable auditing to meet compliance with regulatory requirements? (relevant mainly for financial institutions)

The role of the data scientist or the model developer is to answer all these questions confidently. Responsibly building these algorithms, removing biases, understanding model behaviour and ensuring trust are the core of what a data scientist needs to accomplish.

Let’s talk action!

In light of the increasing importance of xAI, I provide a model explainability framework to ensure interpretability and fairness across the ML lifecycle.

Please note, I will not dive into various explainability techniques in this article, but I will be publishing a series of articles and videos that will introduce explainability concepts in a thorough but fun way. So stay tuned for them.

To circumvent the scope of the framework, I will only focus on the Model Building & Validation Stage of the machine learning lifecycle. I also assume that you have already defined the use case, prepared the data and are now ready to build the model.

In the model building and validation stage, there are two personal goals you have to achieve:

Confidence: ensure your model is robust, reliable and fair.
Communication & Trust: share model insights with stakeholders

So here is the five-step framework:

  1. Start with interpretable models and work your way up to black-box models
  2. Define test cases and edge cases
  3. Use interpretability techniques to understand model behaviour, remove biases and explain predictions — take advantage of explainX
  4. Build a narrative and communicate results with stakeholders through an interactive dashboard
  5. Improve your model through feedback

First things first, in high-stake environments where even a single mistake can have a dramatic impact or can cost a lot of money, you should always start with interpretable models. Starting with black-box models can usually be an uphill battle.

Regardless of what type of model you decided to go with, start with defining test cases — sets of cases for which the model should behave as expected. Do this exercise to verify and validate your model. This step also requires collaboration with other stakeholders who have the domain and business knowledge to offer.

After defining the test cases, go one step further and build edge cases — sets of cases for which the model behaviour is particularly uncertain or unexpected. It is an excellent strategy to stress-test your machine learning model.

Next, you should employ model agnostic interpretability techniques like SHAP, counterfactuals, surrogate decision trees, ProtoDash or IG. These methods provide explanations on a global level (explaining the whole model) and on a local level (individual predictions made by the model).

Unfortunately, there are so many techniques out there that it is overwhelming to choose the right one. Once you finalize the interpretability method, you have to code it, customize it to fit your model and data, fix any errors along the way, optimize it for speed and use other toolsets for visualizing the results.

Every different technique comes with its own set of challenges. Luckily, you can use explainX, which is an open-source explainable AI library that provides state of the art model interpretability techniques under one roof and makes them accessible with just a single line of code — saving a tremendous amount of hours of coding, debugging and effort from your side.

By accessing these interpretability techniques, you should describe your model through four lenses:

  1. Focus on overall model behaviour
  2. Focus on instances
  3. Focus on features
  4. Focus on model comparisons

Most of these lenses are pretty straightforward, but you should also focus on features to gain a better understanding of your model. You can quickly test your model plausibility and suitability by analyzing feature rankings. Additionally, explaining a single model is helpful but comparing & contrasting two or more models in parallel is more effective. It helps you progressively build, compare and refine your models.

After you understand your model through multiple lenses, it is time to communicate your narrative with stakeholders. The goal here is to improve your model through feedback and obtain their trust. Remember, the communication will be iterative, and feedback will be crucial in helping you further optimize your model performance.

Ideally, share an interactive dashboard with insights using language and representations that stakeholders can understand. Stakeholders should be able to add their comments or ask for clarification seamlessly.

Let there be light.

Kudos for making it till the end. Explaining and debugging a black-box machine learning model is a difficult task. But with the right framework and explainability techniques, you will be able to do it in no time. Good news for you: expainX is open-source:

In the next article, I will share an interpretability framework for deployment, maintenance and the usage stage.

Till then, let there be light.

Share this with your friends.

Ready to dive in?
Create your free account now.