AI models need to be transparent and we, as consumers of these algorithms, need to understand how decisions are made by these AI models that directly impact us.
One of the biggest challenges in machine learning is the consistency of the decision making logic. We have seen time and time again how machine learning models have performed better in certain situations and poorly in others.
This makes us suspicious of the overall performance of the model: we cannot take global level metrics at face value. We need to dive deeper.
In light of this, we are introducing a new feature in the explainX model interpretability toolkit: evaluate model performance with cohort analysis. The purpose of this feature is to help data scientists compare model performance across different cohorts or subsets within their datasets - at scale and at speed.
We are releasing the first version of the cohort analysis that will help data scientists and model developers compare their model performance across multiple sub-cohorts within their datasets.
With this feature, data scientists can compare the accuracy, precision, recall, false negative rate and false positive rate of their classification models across multiple subsets within their data. Similarly, we have added crucial metrics for regression based algorithms as well.
Additionally, cohort analysis also gives them the option to compare the distribution of their predictions and identify imbalances in the ground truth values or even model predictions themselves. This capability will enable the model developer to ensure their models are performing consistently across the entire dataset instead of showing bias towards a specific sub-cohort. We will be launching a full tutorial in the coming weeks so stay tuned.
As we believe in constantly improving the user experience, we have also given a fresh look to our browser application. Here is a little sneak peak:
With improved visualizations and insights, data scientists will be able to get the maximum value and understanding of their models without customizing code of various other open-source tools.
This is completely open-source so head over to explainx.ai/register to try it out today.