Correlation Is Not Enough: Embedding Human Metadata... | AI Research

The Problem with Current Biomedical Encoders

The paper "Correlation Is Not Enough: Embedding Human Metadata for Individual Causal Discovery" addresses a critical failure in modern biomedical language models. When these models are asked to compare unrelated concepts—such as a specific cortisol level and stock-market volatility—they often assign them a high similarity score. While traditional retrieval systems can filter out this noise, "Large Behavioural Models" (LBMs) cannot. Because LBMs map out a person's life events to infer causal links, they treat this false similarity as evidence of a real connection, leading to significant errors in causal reasoning.

Improving Embedding Accuracy

To fix this, the authors introduce a two-step refinement process to improve how models distinguish between related and unrelated concepts. First, they perform a contrastive training pass over 72,034 pairs, which significantly improves the model's ability to separate different domains. Second, they introduce a method called BODHI, which mines "hard negatives"—pairs that are clearly unrelated—from a biomedical knowledge graph. This further sharpens the model's discrimination capabilities, ensuring that embedding geometry accurately reflects true causal relationships rather than just superficial similarity.

Performance and Hardware Optimization

The researchers also focused on the practical deployment of these models using Intel Xeon 6737P hardware with AMX acceleration. By utilizing OpenVINO, they achieved a 133x increase in speed, reducing query latency from 1367 ms to just 10 ms. Interestingly, the study found that FP16 precision outperformed INT8 on this specific hardware, a result that contradicts standard industry advice. The authors provide an explanation for this performance quirk and note that their models run significantly slower on hardware lacking AMX support.

Key Takeaways for Future Research

The authors emphasize that for LBMs, embedding geometry is not merely a technical detail—it is the foundation of correctness. By releasing their benchmark suite, training corpora, the BODHI generator, and OpenVINO scripts, they aim to provide the community with the tools necessary to build more reliable causal discovery systems. The findings highlight that as we move toward models that reason over individual human data, the accuracy of the underlying embeddings becomes a primary requirement for preventing the propagation of false causal edges.