FiMI Banking: A Sovereign Model for Indian Retail B... | AI Research

FiMI Banking: A Sovereign Model for Indian Retail Banking
Banks require conversational AI that can handle sensitive financial tasks—such as KYC verification, loan EMI calculations, and insurance claims—while strictly adhering to regulatory and operational rules. General-purpose AI models often struggle in this environment because they may hallucinate information, use tools incorrectly, or fail to handle sensitive banking situations with the necessary caution. This paper introduces FiMI Banking, a specialized, controlled setting designed to train and evaluate AI agents for the Indian retail banking sector. By building a system that operates on bank-controlled infrastructure, the researchers provide a way for financial institutions to deploy reliable, secure, and sovereign AI assistants. The same ai evaluation question is explored in DRACO, which adds a research perspective.

Building a Controlled Banking Environment

To ensure the model is both safe and functional, the researchers created a specialized environment that mimics real-world banking operations. This environment is built using vetted banking documents, structured ground truth, and synthetic customer data. Because real customer data is private and sensitive, the team used synthetic personas and account states to simulate interactions. The system includes a comprehensive catalog of banking tools that allow the AI to perform actual tasks, such as retrieving account balances or processing service requests, rather than just answering questions about them. This setup is fully deterministic and replayable, meaning that training and evaluation occur in a consistent, verifiable space.

Training for Safe and Accurate Performance

The researchers tested two distinct post-training methods to improve the model's performance: preference optimization and reinforcement learning. Preference optimization focuses on the model’s conversational behavior, specifically training it to recognize when a request is outside its scope or safety guidelines. By creating "preference pairs"—examples where the model compares a correct response to a failed one—the system learns to refuse inappropriate requests more effectively. Meanwhile, reinforcement learning is used to master multi-turn tool-use tasks. This method rewards the model for completing a sequence of actions correctly, such as checking eligibility before approving a transaction, ensuring that the agent follows the logical order required by banking workflows. The ai agents story also surfaces in Stanford Researchers Develop TRACE to Fix..., adding another angle.

Key Results and Improvements

The study demonstrates that these two approaches address different, complementary needs for a banking agent. Preference optimization led to a significant increase in safety, with the model’s ability to correctly refuse out-of-scope requests rising from 52% to 80%. Reinforcement learning proved highly effective for complex, multi-step tasks, improving edge-case performance from 0.509 to 0.718 and order-sensitive task performance from 0.590 to 0.679. Notably, the reinforcement learning approach achieved these gains while using 29% fewer generated tokens, indicating that the model became more efficient at reaching the correct outcome.

Considerations for Implementation

A central goal of FiMI Banking is to provide a solution that banks can host on their own infrastructure, including in air-gapped settings where the system is disconnected from the public internet. By using an open-model family, banks retain full control over the model weights and can customize the system to fit their specific product offerings and tool contracts. The researchers emphasize that because the training environment uses the same verifiable rewards as the evaluation process, the model is directly aligned with the specific requirements and regulatory standards of the bank. This ensures that the agent remains reliable throughout its deployment, as its performance can be continuously measured against the same rigorous standards used during its training. The ai agents story also surfaces in Moonshot AI and kvcache-ai Open Source..., adding another angle. as detailed in the full paper on Arxiv