PathView-Bench: Can Multimodal Large Language Model...

PathView-Bench: Can Multimodal Large Language Model... | AI Research

Key Takeaways

PathView-Bench is a new benchmark designed to evaluate whether Multimodal Large Language Models (MLLMs) truly understand the fine-grained, multiscale visual...
Multimodal large language models (MLLMs) are increasingly used to analyze pathology images.
However, dominant multimodal benchmarks in pathology mainly score final diagnostic answers, captions, or reports.
These evaluations provide limited insight into whether a model understands the multiscale visual content needed for pathology reasoning and decision-making.
We introduce PathVU, a vision-anchored benchmark for fine-grained and multiscale visual understanding in computational pathology.

Paper AbstractExpand

Multimodal large language models (MLLMs) are increasingly used to analyze pathology images. However, dominant multimodal benchmarks in pathology mainly score final diagnostic answers, captions, or reports. These evaluations provide limited insight into whether a model understands the multiscale visual content needed for pathology reasoning and decision-making. We introduce PathVU, a vision-anchored benchmark for fine-grained and multiscale visual understanding in computational pathology. Built from 23 public pathology imaging datasets with human-supervised labels and spatial annotations, PathVU evaluates MLLM understanding in two fields of view: Region FOV for high-resolution local regions and Slide FOV for macro whole-slide views. By converting raw annotations into deterministic task targets, PathVU enables programmatic scoring of region localization, visual recognition, quantity estimation, spatial reasoning, and insufficient-context judgment. The benchmark contains 14 VQA-style tasks, 61,673 images, and 308,070 samples across 28 organs and 7,253,526 annotations. Evaluating 18 representative general-purpose, medical-domain, and pathology-oriented MLLMs, we observe substantial limitations even in advanced models on fine-grained visual tasks across multiscale pathology images. PathVU provides a reproducible basis for developing and evaluating pathology MLLMs with explicit multiscale visual understanding.

PathView-Bench is a new benchmark designed to evaluate whether Multimodal Large Language Models (MLLMs) truly understand the fine-grained, multiscale visual details required for computational pathology. While existing benchmarks often focus on final diagnostic answers or report generation, this research argues that such metrics do not confirm if a model’s reasoning is actually grounded in the underlying image content.

Multiscale Visual Understanding

Pathology images, specifically whole-slide images (WSIs), contain critical information at different scales, ranging from cellular morphology to broad tissue architecture. PathView-Bench organizes its evaluation into two distinct fields of view:

Region-FOV: Focuses on high-resolution local regions to test understanding of cellular structures and micro-level details.
Slide-FOV: Focuses on macro whole-slide views to test understanding of tissue organization, lesion distribution, and large-scale spatial patterns.

Benchmark Construction

The researchers built PathView-Bench by aggregating 23 public pathology datasets, resulting in 61,673 images and over 308,000 samples. To ensure the benchmark is objective, the team converted human-supervised labels and spatial annotations into deterministic task targets. This allows for programmatic scoring of 14 different VQA-style tasks—such as object counting, region classification, and spatial reasoning—without relying on LLM-based evaluation, which the authors note can be inconsistent.

Performance of Current Models

The study evaluated 18 representative MLLMs, including general-purpose, medical-domain, and pathology-oriented models. The results indicate a consistent performance gap: while many models can name visual categories, they struggle with tasks requiring precise grounding, counting, spatial organization, or judging whether an image contains enough information to answer a question.
Franklin analysis: The data suggests that high performance on slide-level diagnostic tasks can mask significant failures in basic visual operations. The researchers observed that neither increasing model scale nor using pathology-specific training consistently resolved these grounding issues.

Limitations and Future Directions

The authors note that current models often fail when asked to identify when a view provides insufficient evidence, such as when a model is asked to count cells in a low-magnification macro view where individual cells are not visible. By providing a reproducible, auditable framework, the researchers aim to shift the focus of pathology AI development toward models that demonstrate explicit, evidence-based visual understanding rather than just linguistic fluency.