There is an ever-increasing prevalence of modern AI systems which are becoming more capable and finding their way to new applications. These enhanced capabilities mean more complexity, and that makes these systems more difficult to understand.When IBM Watson was marketed to hospitals to help the oncology department detect cancer, it failed miserably. The doctors as well as the patients were unable to trust the machine at each stage of consulting and treatment as Watson wouldn’t provide the reasons for its results. Moreover when its results agreed with the doctor’s, it couldn’t provide a diagnosis.The lack of explainability and trust hampers our ability to fully trust AI systems.

XAI could alleviate this situation by proposing novel ways of explaining the underlying thinking process of AI systems.Explainable AI (XAI) is an emerging field in machine learning that aims to address how black box decisions of AI systems are made. XAI is expected to answer questions like: Why did the AI system make a specific prediction or decision? Why didn’t the AI system do something else? When did the AI system succeed and when did it fail? Explainability refers not only to whether the decisions a model outputs are interpretable, but also whether or not the whole process and intention surrounding the model can be properly accounted for.

There are two main set of techniques used to develop explainable systems; post-hoc and ante-hoc. Ante-hoc techniques (e.g. RETAIN, BDL) entail baking explainability into a model from the beginning. Post-hoc techniques (e.g. LIME, LRP, BETA) allow models to be trained normally, with explainability only being incorporated at testing time. The advantages of XAI are human understandable rationale in decision making, trust in system, regulatory compliance, generalization, debugging and enhancement of AI models, detection of bias and openness of discovery and scientific research. The primary applications of XAI systems can be in healthcare, driverless cars or even drones being deployed during war. Despite the advantages there will always be tradeoff decisions to be made between explainability and accuracy depending on the application field of the algorithm and the end-user to whom it’s accountable.Thusas AI becomes more profound in our lives, explainable AI becomes even more important.

The author of this article is Ramkrishna M, Asst. Prof., Dept of Computer Engineering, International Institute of Information Technology, Hinjawaid, Pune.