Back to AI Research

AI Research

Explainability Assistant: A Conversational XAI Inte... | AI Research

Key Takeaways

  • Explainability Assistant: A Conversational XAI Interface for Interpreting Energy Consumption Models As machine learning models become more complex, it is inc...
  • Explainable Artificial Intelligence (XAI) techniques address this opacity, but traditional XAI dashboards require substantial technical expertise and provide limited flexibility for dynamic, context-aware inquiry.
  • Conversational XAI systems offer a promising alternative; however, previous approaches, such as TalkToModel, were constrained by rigid custom grammars and achieved only 76.8% intent-parsing accuracy.
  • This paper introduces the Explainability Assistant, an open-source conversational XAI system that leverages the function-calling capabilities of modern Large Language Models (LLMs) to overcome these limitations.
  • The system achieves 94% intent-parsing accuracy, supports flexible natural language interaction, and adapts to different ML problem types without task-specific fine-tuning.
Paper AbstractExpand

Energy consumption forecasting relies on increasingly complex machine learning (ML) models, such as Genetic Programming-based symbolic regressors, whose predictions can be difficult for facility managers and building operators to interpret. Explainable Artificial Intelligence (XAI) techniques address this opacity, but traditional XAI dashboards require substantial technical expertise and provide limited flexibility for dynamic, context-aware inquiry. Conversational XAI systems offer a promising alternative; however, previous approaches, such as TalkToModel, were constrained by rigid custom grammars and achieved only 76.8% intent-parsing accuracy. This paper introduces the Explainability Assistant, an open-source conversational XAI system that leverages the function-calling capabilities of modern Large Language Models (LLMs) to overcome these limitations. The system achieves 94% intent-parsing accuracy, supports flexible natural language interaction, and adapts to different ML problem types without task-specific fine-tuning. We present the system's architecture and report results from a comparative evaluation conducted with energy domain specialists, contrasting the Explainability Assistant with a traditional XAI dashboard. The evaluation suggests improved usability and consistent task accuracy, with all experts unanimously preferring the conversational interface for practical use.

Explainability Assistant: A Conversational XAI Interface for Interpreting Energy Consumption Models
As machine learning models become more complex, it is increasingly difficult for non-technical professionals, such as building managers and engineers, to understand why a model makes a specific prediction. While traditional "dashboard" tools exist to explain these models, they often require high technical expertise and can be rigid to use. This paper introduces the Explainability Assistant, an open-source conversational interface that allows users to ask questions about machine learning model behavior using natural language, making complex AI insights more accessible and easier to interpret.

How the System Works

The Explainability Assistant moves away from older, rigid systems that relied on custom grammars. Instead, it uses the "function-calling" capabilities of modern Large Language Models (LLMs). When a user asks a question—such as "Why was this energy consumption prediction so high?"—the system translates that request into a structured command. It then executes specific explainability tools, such as SHAP for feature importance or DiCE for counterfactual reasoning, to provide an answer. Because the system is modular, it can be adapted to different types of machine learning tasks, such as energy forecasting or medical classification, simply by updating configuration files rather than retraining the model. The xai story also surfaces in AI tools are subtly rewriting user..., adding another angle.

Improving Accuracy and Transparency

A major challenge for conversational AI is accurately understanding user intent. Previous systems often struggled with this, achieving only 76.8% accuracy. By leveraging modern LLMs, the Explainability Assistant reaches up to 94% intent-parsing accuracy. To ensure the system remains trustworthy, it is designed to be transparent: before it executes a task, it provides a clear, written explanation of what it intends to do. Furthermore, the system is designed with data privacy in mind, as the actual data processing and model execution happen locally on the back-end, keeping sensitive information protected.

Expert Feedback and Performance

The researchers tested the system with energy domain specialists who compared it against a traditional point-and-click dashboard. The results showed that the conversational interface was highly effective: participants achieved 100% accuracy on assigned tasks using the Explainability Assistant, compared to 93% with the traditional dashboard. Most notably, the experts unanimously preferred the conversational interface for its ease of use and indicated a strong likelihood of using it in their future work. The same large language models question is explored in Efficient Test-Time Adaptation through Human-AI Interaction, which adds a research perspective.

Considerations for Future Use

While the conversational interface outperformed traditional dashboards in usability, the study noted that participants maintained a healthy level of caution regarding the system's outputs. This suggests that users are not blindly trusting the AI, which is a positive outcome for decision-support tools. The authors suggest that future developments could explore "hybrid" interfaces, which might combine the flexibility of natural language conversation with the visual overviews provided by traditional dashboards to offer the best of both worlds. as detailed in the full paper on Arxiv

More from Franklin

AI tools are subtly rewriting user...newsFLY-EVAL++researchXiaomi-TabLDMresearchLarge Language Models (LLMs) for Telecom...researchEfficient Test-Time Adaptation through Human-AI InteractionresearchKernel-Managed Shared Memory for System-Wide Personalizationresearch

Comments (0)

No comments yet

Be the first to share your thoughts!