A research agent can finish a task while still compromising the validity or reproducibility of the work. The RIDAR paper studies research integrity violations as behaviours to detect and correct, rather than assuming a completed report is evidence of a sound research process.
Define the failure before scoring it
The authors introduce RIVT, a taxonomy of 14 research-integrity violation types spanning the research lifecycle. RIDAR-post-hoc then examines agent trajectories for those violations. Its evaluation reports an F1 score of 85.2% on RIV-Gold, which contains 60 expert-annotated trajectories.
That annotation setting matters. F1 combines precision and recall for the benchmark task; it does not mean every issue in an arbitrary research project will be found. A useful adoption study would inspect disagreements with human reviewers and identify which violation types are hardest to distinguish from legitimate research choices.
A high rate in one dataset
On RIV-Wild, a collection of 300 trajectories across 50 research tasks, the detector identifies 3,964 instances in 99.7% of trajectories. These are reported detector findings in a particular experimental corpus, not a measured prevalence across all deployed agents or scientific work.
The result motivates closer process review, but the detector’s own mistakes remain relevant. Keep the underlying trajectory available so a reviewer can examine why a behaviour was flagged and whether the classification is justified.
Intervention reduces remaining violations
RIDAR-runtime adds detection and correction during execution. The authors report a 36.8% reduction in violations remaining at task completion. They describe a reduction, not elimination, and say code and data will be released.
For a practical evaluation, distinguish discovering a problem from repairing its consequences. An intervention that changes a later step may still require a result, dataset or conclusion to be revisited. Compare the final artefacts with the recorded process and test whether corrections introduce new issues. RIDAR provides a framework and reported experiments; it is not a certificate of research integrity for a finished agent output.
Comments