Back to AI Research

AI Research

EdiTikZ: Scientific Figure Editing from Revision Tr... | AI Research

Key Takeaways

  • EdiTikZ: Scientific Figure Editing from Revision Trajectories Scientific figures are essential for academic communication, but creating and refining them is...
  • Vision-language models (VLMs) have shown strong performance in generating scientific figures from text or images.
  • However, producing publication-ready figures requires iterative refinement, making scientific figure editing an important yet largely unexplored task.
  • Existing approaches rely on costly proprietary agentic systems, focus primarily on evaluation, or construct training supervision from synthetically generated edits.
  • Instead, we leverage naturally occurring scientific revision and development trajectories as a scalable source of supervision.
Paper AbstractExpand

Vision-language models (VLMs) have shown strong performance in generating scientific figures from text or images. However, producing publication-ready figures requires iterative refinement, making scientific figure editing an important yet largely unexplored task. Existing approaches rely on costly proprietary agentic systems, focus primarily on evaluation, or construct training supervision from synthetically generated edits. Instead, we leverage naturally occurring scientific revision and development trajectories as a scalable source of supervision. To this end, we introduce DaEdiTikZ, the first large-scale dataset of revision-derived scientific figure edits, constructed by mining 391K plausible TikZ edit pairs from arXiv, GitHub, and TeX SE and inferring 781K directed edit instructions with a VLM conditioned on rendered figures and TikZ code. We further introduce DaEdiTikZ-Bench, a human-refined benchmark with 790 instances, and train two compact Qwen3.5-based EdiTikZ models (4B and 9B) by jointly learning reconstruction and editing, followed by reinforcement learning (RL) with complementary rewards for rendered fidelity and edit application. Automatic evaluation places our 9B model above all tested baselines, while human evaluation with 9 annotators and 4,320 ratings places it above GPT-5.6-Sol and on par with Gemini-3.1-Pro. Under severe out-of-distribution shifts, it remains competitive with GPT-5.6-Sol near its 2K training sequence-length regime. Models and datasets will be released.

EdiTikZ: Scientific Figure Editing from Revision Trajectories
Scientific figures are essential for academic communication, but creating and refining them is a time-consuming process. While modern vision-language models (VLMs) have become proficient at generating figures from scratch, they struggle with the nuanced task of editing existing figures based on natural language instructions. This paper introduces EdiTikZ, a framework designed to bridge this gap by enabling models to modify scientific figures while preserving their original structure and content.

Leveraging Natural Revision Histories

The researchers observed that scientific figures are not static; they evolve through iterative revisions during the research and writing process. By mining these "revision trajectories" from platforms like arXiv, GitHub, and TeX SE, the team created DaEdiTikZ, a large-scale dataset containing 391,000 pairs of scientific figures. To make this data useful for training, they used a VLM to analyze the differences between these figure pairs and automatically generate 781,000 specific, natural-language editing instructions. This approach provides a scalable way to teach models how to apply precise changes to complex diagrams.

Training for Precision and Fidelity

To build the EdiTikZ models, the authors trained compact Qwen3.5-based models (4B and 9B parameters) using a two-step process. First, they performed supervised fine-tuning that forced the models to learn both how to reconstruct figures from scratch and how to apply edits to existing ones. Second, they used reinforcement learning (RL) to further refine the models. This RL process utilized a dual-reward system: one reward measures how well the final image matches the visual intent, while another ensures the model faithfully follows the specific text-based edit instructions provided by the user.

Performance and Evaluation

The researchers evaluated their models using DaEdiTikZ-Bench, a curated set of 790 human-refined editing instances. Human evaluations, involving nine annotators and over 4,000 ratings, showed that the 9B model outperforms existing solutions like GPT-5.6-Sol and performs on par with Gemini-3.1-Pro. Furthermore, the model demonstrated strong generalization capabilities, remaining competitive even when tasked with editing complex figures that fall outside the distribution of its training data.

Key Takeaways

The success of EdiTikZ highlights the value of using real-world scientific data to train specialized models. By grounding the model's training in both visual renders and the underlying TikZ code, the researchers found that the model could better interpret spatial and numeric instructions. This combination of large-scale, revision-derived data and reinforcement learning provides a robust foundation for automating the iterative refinement of scientific figures, potentially saving researchers significant time in the publication process.

Comments (0)

No comments yet

Be the first to share your thoughts!