AI Research

Multimodal Knowledge Edit-Scoped Generalization for... | AI Research

Key Takeaways

Multimodal Large Language Models (MLLMs) are powerful tools that combine visual and textual information, but their knowledge can become outdated as facts cha...
Online multimodal knowledge editing requires injecting a continual stream of visual-textual corrections into multimodal large language models (MLLMs) with bounded overhead and minimal disruption to unrelated behaviors.
Existing editors mainly emphasize edit reliability and long-horizon stability, but rarely control the semantic boundary of each edit.
Therefore, we formulate Edit-Scoped Generalization, reframing online MLLM editing from merely correcting an instance to controlling the propagation boundary of each edit.
To this end, we propose ScopeEdit, a scope-aware online editor that decomposes each update into a modality-local absorption branch and an evidence-gated shared generalization branch.

Paper AbstractExpand

Online multimodal knowledge editing requires injecting a continual stream of visual-textual corrections into multimodal large language models (MLLMs) with bounded overhead and minimal disruption to unrelated behaviors. Existing editors mainly emphasize edit reliability and long-horizon stability, but rarely control the semantic boundary of each edit. Our pilot analyses of post-edit behaviors and internal neuronal activities reveal a scope gap behind reliable edits: instance-level success neither guarantees transfer to valid cross-modal variants nor prevents leakage to unrelated inputs, while edit-related cross-modal responses concentrate in deeper semantic layers. Therefore, we formulate Edit-Scoped Generalization, reframing online MLLM editing from merely correcting an instance to controlling the propagation boundary of each edit. To this end, we propose ScopeEdit, a scope-aware online editor that decomposes each update into a modality-local absorption branch and an evidence-gated shared generalization branch. The local branch supports stable edit absorption, whereas the shared branch enables cross-modal propagation only when visual and textual evidence are sufficiently aligned. Both branches perform scope-separated write geometries in orthogonal low-rank spaces and maintain branch-wise preconditioners via Sherman--Morrison recursions, yielding constant per-edit overhead. Extensive experiments across diverse benchmarks, long-horizon edit streams, MLLM backbones, real-world VLKEB scenarios, and complex vision-language architectures show that ScopeEdit consistently improves the trade-off between in-scope cross-modal transfer and out-of-scope locality, while preserving edit reliability, stability and online efficiency. Our code is available at this https URL .

Multimodal Large Language Models (MLLMs) are powerful tools that combine visual and textual information, but their knowledge can become outdated as facts change. While researchers have developed ways to "edit" these models with new information, these updates often struggle to balance reliability with the need to generalize correctly. This paper introduces a new framework called ScopeEdit, which moves beyond simple fact-correction to explicitly control the "semantic boundary" of an edit—ensuring that new information is applied appropriately without causing unintended side effects.

The Problem: Scope Errors in MLLM Editing

Current editing methods focus primarily on "reliability"—ensuring the model gives the correct answer to the specific question it was just taught. However, the authors found that reliability does not guarantee that the model will handle related questions correctly or avoid leaking new information into unrelated topics. Their analysis revealed two main issues: "under-generalization," where the model fails to apply the new knowledge to similar cross-modal variants, and "over-generalization," where the model incorrectly applies the new knowledge to unrelated inputs.

How ScopeEdit Works

To address these issues, ScopeEdit treats every update as a two-part process. It decomposes the model's learning into two distinct branches:

Modality-Local Absorption Branch: This branch focuses on stable, reliable updates that ensure the model learns the specific fact while preserving the integrity of unrelated information (out-of-scope locality).
Evidence-Gated Shared Generalization Branch: This branch is responsible for spreading the new knowledge to semantically related cross-modal contexts. It only activates when the system detects that the visual and textual evidence are sufficiently aligned, preventing the model from making unwarranted assumptions.
By using orthogonal low-rank spaces for these branches, the system maintains efficiency and prevents the two branches from interfering with each other.

Key Findings and Performance

The researchers tested ScopeEdit across various benchmarks, long-horizon edit streams, and different MLLM architectures. Their results show that this approach consistently improves the trade-off between in-scope generalization and out-of-scope locality. By focusing on the deeper semantic layers of the model—where visual and textual information are most synchronized—ScopeEdit achieves more stable and precise updates. The method maintains a constant per-edit overhead, making it a practical solution for the ongoing, sequential nature of online model editing.

Comments (0)

No comments yet

Be the first to share your thoughts!