Back to AI Research

AI Research

MIRROR: Learning from the Other View for Multi-Moda... | AI Research

Key Takeaways

  • MIRROR: Learning from the Other View for Multi-Modal Reasoning Vision-language models (VLMs) often struggle with reasoning, even when the information provide...
  • We show that these views often elicit different behaviors: a model may solve a problem from text but fail on the corresponding diagram, or succeed visually while failing textually.
  • This inconsistency suggests that different views expose complementary reasoning paths and failure modes that standard multimodal post-training does not fully exploit.
  • We then develop Modality-Informed Reciprocal Reasoning Optimization (MIRROR), a reinforcement learning approach for improving multimodal reasoning via self supervision.
  • For each problem, MIRROR evaluates the model under all views, selects the best-performing view as a teacher, and trains other views with a reverse-KL objective towards the teacher.
Paper AbstractExpand

Unlike large language models (LLMs) that exhibit strong reasoning capabilities, vision-language models (VLMs) struggle with visual reasoning, even on geometry problems that admit equivalent text, diagram, and combined diagram+text views. We show that these views often elicit different behaviors: a model may solve a problem from text but fail on the corresponding diagram, or succeed visually while failing textually. This inconsistency suggests that different views expose complementary reasoning paths and failure modes that standard multimodal post-training does not fully exploit. To study and exploit this phenomenon, we construct ODA-Data, a high-quality paired multimodal geometry dataset with text-dominant, image-dominant, and combined image+text views of the same problems, together with splits for training and evaluating modality-dependent reasoning behaviors. We then develop Modality-Informed Reciprocal Reasoning Optimization (MIRROR), a reinforcement learning approach for improving multimodal reasoning via self supervision. For each problem, MIRROR evaluates the model under all views, selects the best-performing view as a teacher, and trains other views with a reverse-KL objective towards the teacher. Across reasoning benchmarks that evaluate on geometry problems, MIRROR improves over standard RL and yields more accurate and consistent behavior across modalities

MIRROR: Learning from the Other View for Multi-Modal Reasoning
Vision-language models (VLMs) often struggle with reasoning, even when the information provided is identical. A model might solve a geometry problem easily when described in text, but fail when presented with a diagram, or vice versa. This inconsistency reveals that different ways of presenting the same problem—text, images, or both—expose different strengths and weaknesses in a model's reasoning process. The researchers behind MIRROR aim to turn this inconsistency into a learning opportunity, using the model's own successful reasoning in one "view" to teach it how to reason better in another.

The Problem of View-Dependent Reasoning

Current models are highly sensitive to how information is presented. In geometry, a problem can be expressed through a text description, a diagram, or a combination of both. The researchers found that these views are not equally effective; a diagram might make a geometric structure obvious, while a text description might clarify a specific constraint that is hard to "see" in an image. Because these views are essentially different paths to the same mathematical truth, the failure of a model to solve a problem in one view while succeeding in another is not just a random error—it is a signal that the model has not fully learned how to ground its reasoning across different types of input.

How MIRROR Works

MIRROR (Modality-Informed Reciprocal Reasoning Optimization) is a self-supervised training framework that uses these performance gaps to improve reasoning. Instead of relying on external teachers or human-written solutions, the model acts as its own teacher. During training, the system evaluates the model’s performance on a problem across all three views (text-dominant, image-dominant, and combined). It then identifies the "strongest" view for that specific problem and uses it as a guide.
The student model is trained using reinforcement learning, where it attempts to solve the problem using a restricted view (e.g., only the image). While the student generates its own reasoning steps, the "teacher" (the model's own performance on the strongest view) provides guidance. This ensures the student learns to improve its reasoning without being forced to rely on information that isn't actually present in its current, restricted view.

Results and Performance

To test this, the researchers created ODA-Data, a high-quality dataset of geometry problems that preserves the same underlying logic across text, image, and combined views. The results show that MIRROR significantly outperforms standard training methods. By using cross-modal supervision, the model becomes more accurate and consistent, meaning it is more likely to solve the same problem correctly regardless of whether it is presented with text or an image. Specifically, the model showed notable gains in accuracy on both internal benchmarks and external geometry tests like GeoInt and MathVerse, proving that paired views of the same problem provide a powerful signal for self-improvement.

Key Takeaways

The research demonstrates that simply exposing a model to multiple views of a problem during training is not enough; without a structured way to transfer knowledge, the model often fails to learn effectively. MIRROR succeeds because it treats the model's own cross-modal inconsistencies as a source of supervision. By dynamically selecting the best view to act as a teacher for each individual problem, the model learns to internalize reasoning structures that were previously hidden, leading to more robust and reliable multimodal intelligence.

Comments (0)

No comments yet

Be the first to share your thoughts!