Use Case & Tutorial

Explainable AI Use Case: Loan Application Approval

Post by
R&D Team
Learn how to explain and debug a blackbox CatBoost model on Loan Application dataset with explainable AI

Black-Box AI Creates Risk for Businesses!

Explainability is extremely crucial in AI systems that are responsible for carrying out mission-critical tasks. For example, in healthcare, if scientists are relying on AI models to help them figure out whether the patient will have cancer or not, they need to be 100% sure of their diagnoses otherwise this can result in death, a lot of lawsuits and a lot of damage done in trust. Nature is this problem is so intense that explainability sits at the core of this problem: the data scientists and the human operators, in this case, the doctors need to understand how the machine learning system is behaving and how did it come to a decision.

Explainable AI is also extremely important in financial services or FinTech in particular due to the growing adoption of machine learning solutions for credit scoring, loan approval, insurance, investment decisions and so on. Here again, there is a clear cost associated with implementing wrong decisions by the machine learning system: so there is a huge need to understand how the model actually works.

Using black-box AI increases business risk and exposes the businesses to a deep downside. The reason why black-box models are not desirable becomes more clear when we look at how the business functions as a whole:

For the business decision-maker, data scientists need to answer the question of why they can trust our models, for IT & Operation, data scientists need to tell them how can they monitor and debug if an error occurs, for the data scientist, they need to know how they can further improve the accuracy of their models and finally, for regulators and auditors, they need to be able to get an answer to whether our AI system is fair or not?

Enter explainable AI

Explainable AI aims at providing clear and transparent predictions - an end-to-end system that translates complicated reasoning into simple language and relies on automated feedback to constantly improve the AI system. Remember, xAI is highly driven by the feedback so is a two-way interaction between the human and the AI system.

In the end, we should understand why behind the model, the impact of the model, where the model fails and what recommendations to provide.

Explainx: An open-source, fast and scalable explainable AI platform.

explainx is an open-source explainable AI platform created by Explainx aims to help data scientists explain, monitor and debug black-box AI models — the purpose is to enable companies to build robust, unbiased and transparent AI solutions.

Data scientists can explain any blackbox model in a single line with explainX xai.
explainX high-level workflow

Within the explainx architecture, explainx provides access to state of the art interpretability techniques in just a single line of code within our Jupyter notebook or IDE.

For this example, we will use the HELOC dataset provided by FICO. The customers in this dataset have requested a credit line in the range of USD 5,000–150,000. Our job is to predict RiskPerformance: whether they will make timely payments over a two year period. The prediction can then be used to decide whether the homeowner qualifies for a line of credit.

Loan application approval explainable AI use case
Loan Application Customer Journey -

For this example, we will train CatBoost Classifier model. After the training is done, we will use the explainx xAI module to explain our model and build a narrative for business stakeholders to understand! You can also follow the tutorial by simply copying the code in your own notebook!

Let’s start with opening up our Jupyter notebooks and installing the explainx library.

You can also clone the repository from the link:

So let's begin!

Install the library.
Import relevant packages
Load and pre-process our dataset for model building. The dataset is already available in the explainX library.
Train our black-box model.
Simply call the explainX function and pass the parameters
Once explainx and running, all you need to do is point your browser to and you’ll see a very nice user interface called explainX.Dashboard.

Note: If you want to view it inline, simple pass mode="inline” argument into the explainx function.

For this tutorial, we will not go in the nitty-gritty of model building, model metrics and evaluation. Instead, we will dive right into the explaining AI powered decisions and understand how models behave. So let’s start opening up the black-box!

In explainX, we can explain the model on four levels:

1. Global Level Explanation

2. Local Prediction Explanation

3. Scenario Analysis

4. Feature Interaction & Distributions

  1. Global Level Explanation

Let's use the overall feature importance and overall feature impact graph to identify top/bottom performing features.

explainX explainable AI identifies top features important to the blackbox model

This tells that according to the CatBoost model, ExternalRiskEstimate, MSinceMostRecentInq and PercentTradesNeverDelqare the top three variables with the largest impact on RiskPerformance. These three represents Risk Estimate, Credit Inquiries and Debt Level information: extremely important categories when assessing risk.

This information gives us a general idea of feature contribution but to understand whether each of these features has a positive or a negative impact on RiskPerformance we need to look at the feature impact graph.

explainX identifies which features have a negative or a positive impact on the model output

This graph gives us even more insight into the model logic. We can clearly observe that ExternalRiskEstimate impacts the predicting variable positively — pushes it towards “Good Credit Risk Performance” and that matches our intuition as well. We have to use a little bit of our domain knowledge in finance for this: ExternalRiskEstimate is a consolidated version of some risk markers (higher is better) so automatically we learn that this variable will always positively affect the prediction. Then we have NumSatisfactoryTrades the number of “satisfactory” accounts (“trades”) has a significant positive effect on the predicted probability of good credit.

However, in the lower end, delinquency variables push the overall prediction towards 0 (in this case, application denied). This is interesting and we can even dig deeper and see when does this negative effect of delinquency wears off? (Something to try on your own!)

Now as we have an idea of how each feature/variable affects the overall decision, we can move on to explaining the decision for a specific customer. For that, we will be using an impact graph or a decision plot that will help us get attribution scores for that particular prediction. To further understand model behavior, we will also find similar customers that had a similar profile and decision to further explain the model.

  1. Local Prediction Explanation

So let’s explain the decision for customer # 9 in our data. For this specific customer, the application was Approved because the RiskPerformance was “Good” and our model classified it correctly!


So this graph clearly shows the top three positive and top three negatively impacting variables. According to the model, MSinceMostRecentInq had the most positive impact on the variable. So this tells us that the higher value of this variable means that there is no penalty for having more than one month since the most recent inquiry. Then we have ExternalRiskEstimate that again plays a positive role in pushing the prediction towards “good credit behavior”. However, PercentTradesNeverDelq affected the prediction negatively: this might be the case if the value of this variable is extremely small because the smaller value for this variable lowers the probability of good credit score - and we can confirm this by looking at the data.

To keep it short, these findings match our mental-models as the contribution or impact score of each variable is correctly assigned. To further explain the model and support our analysis, we will find similar customers!

ExplainX comes with an in-built prototypical analysis module which provides a much more well rounded and comprehensive view of why the decision for the applicant may be justifiable by identifying similar customers!

explainX finds similar data points from the data that support model output

The above table depicts the five closest customers to the chosen applicant. Based on importance weight assigned to each profile by the method, we see that the prototype under column zero is the most representative user profile by far. This is (intuitively) confirmed from the feature similarity where more than 50% of the features (12 out of 23) of this prototype are identical to that of the chosen user whose prediction we want to explain. Also, the bank employee looking at the prototypical users and their features surmise that the approved applicant belongs to a group with high values of ExternalRiskEstimate. This justification gives the bank employees more confidence in approving the user’s application.

  1. Scenario Analysis

Now, let’s explore different scenarios and see how the model performs - we can use scenario analysis to identify biases within our model. It might be the case where for specific group of customers, the model is behaving differently. We can apply the data filters within the data table (no need to write SQL queries to filter your data) and explain multiple instances and scenarios very easily.

This is extremely useful when you are trying to understand behavior on a specific cluster or group of data. For example, we want to see how the model behaved when the ExternalRiskEstimate was > 60 and MSinceOldestTradeOpen was greater than 200.

We can clearly see the dominance of ExternalRiskEstimate from the information above!


When the ExternalRiskEstimate is greater than 60, it is seen as a positive sign and most of the applications were approved which matches our internal mental model as well: ExternalRiskEstimate is actually monotonically decreasing which means that as the value of ExternalRiskEstimate increases, the probability of Bad Credit Score decreases! So in this cluster where the ExternalRiskEstimate is greater than 60, we will have more approved applications and customers who were extended the credit line. We can confirm this by visualizing the marginal effect of changing ExternalRiskEstimate has on the model decision.

  1. Feature Interaction & Distribution
explainx xai tells you what will happen if you increase or decrease the value of your feature

We can clearly see that pattern that as the value of ExternalRiskEstimate increases, the model is more likely to approve the application of the customer. We see more concentration of red dots as the ExternalRiskEstimate value goes up and that makes a lot of sense! Let's not forget that we have to look at other variables as well to learn more but now, with explainX, we finally have an in-depth insight into how the model is actually behaving.


Data Scientists can easily use explainX to further explore patterns by looking at interactions between different variables and how they impact the overall decision of the model. So let’s summarize our learning for a business user to understand:

  • According to the CatBoost model, ExternalRiskEstimate, MSinceMostRecentInq and PercentTradesNeverDelqare the top three variables with the largest impact on RiskPerformance - this is confirmed by looking at our PDP plot.
  • ExternalRiskEstimate impacts the predicting variable positively — increases the chances of getting the application approved but delinquency variables can pose as red flags and the model may deny the application.
  • For our customer # 9 whose loan application was approved, high-value of ExternalRiskEstimate played a very positive role in getting the application accepted. However, PercentTradesNeverDelq affected the prediction negatively so the loan officer can review the data and validate the learnings.
  • We successfully found customers that had similar profiles to our applicant according to the model, thus strengthening our hypothesis even more.
  • We were able to validate the model’s logic by further exploring the PDP that clearly showed us how increasing the value of ExternalRiskEstimate increases the probability of loan approval as well.
  • Data Scientist can further explore, debug their models and provide transparency about decisions to business stakeholders and customers.

I hope you all enjoyed this case study. Explainability is extremely crucial and more than relevant today — so the ability to present a narrative that shows your understanding of how AI works is a vital skill for a data scientist. This is the real essence of human-AI understanding and democratization of AI.

Share this with your friends.

Ready to dive in?
Create your free account now.