Paper AbstractExpand
Pseudorandom number generators (PRNGs) constitute indispensable computational tools across multiple scientific domains, including Monte Carlo simulations, stochastic computing, and artificial intelligence (AI). The reproducibility of such applications critically depends on the ability of PRNG implementations to generate identical sequences across software environments when initialized from the same internal state. These algorithms enable the simulation of stochastic processes while providing deterministic and repeatable behaviour, thereby facilitating reproducible experiments. Modern PRNG implementations may be initialized through either a seed or, more accurately, an initial state that exceeds the capacity of a conventional integer seed. However, reliance on a simple seed alone frequently proves insufficient to ensure consistent program execution traces across different implementations. A natural assumption is that transferring the complete internal state of a generator should guarantee identical outputs regardless of the software library used. This study examines the validity of this assumption by investigating whether complete initial states can ensure cross-library fidelity and portability of PRNG streams. We focus on two widely deployed generators, Mersenne Twister and Philox, and evaluate their implementations across four major Python ecosystems-Random, NumPy, PyTorch, and TensorFlow. We compare the sequences produced by these implementations against those generated by the original reference algorithms under identical initialization conditions. Our results demonstrate that reproducibility cannot be assumed from PRNG state transfer alone, even when implementations claim to follow the same underlying algorithm. While fidelity was successfully achieved for several implementations, significant discrepancies were observed in others. Most notably, the Philox implementation in PyTorch exhibits fundamental incompatibilities with the reference algorithm, preventing exact reproduction of generator outputs across environments. These findings challenge the common expectation that access to a full internal state of a PRNG is sufficient to ensure reproducibility across software stacks. They further highlight that implementation-specific design choices can introduce hidden barriers to experimental replication, particularly in AI workflows that rely on multiple frameworks. This work shows that implementation fidelity of a PRNG is a necessary condition for scientific reproducibility and makes two primary contributions. First, it identifies practical guidelines for achieving reliable PRNG usage and reproducibility within the Python scientific and AI ecosystem. Second, it evaluates the extent to which cross-library portability and fidelity can be recovered through user-level techniques, without requiring modifications to library source code.
Reproducible AI Requires Reproducible Randomness
This paper investigates a critical challenge in scientific computing and artificial intelligence: the assumption that transferring the internal state of a pseudorandom number generator (PRNG) ensures identical results across different software environments. While researchers often rely on PRNGs to create deterministic, repeatable experiments, this study reveals that simply sharing an initial state is not always enough to guarantee consistent output. By testing common PRNG algorithms across major Python ecosystems, the authors demonstrate that implementation-specific design choices can create hidden barriers to reproducing AI research.
Testing Cross-Library Fidelity
To determine if PRNG state transfer is truly reliable, the researchers evaluated two widely used generators—Mersenne Twister and Philox—across four prominent Python environments: Random, NumPy, PyTorch, and TensorFlow. They compared the sequences produced by these libraries against the original reference algorithms under identical initialization conditions. The goal was to see if the same starting state would yield the same stream of numbers regardless of the library being used. The same ai systems question is explored in ScienceBuddy, which adds a research perspective.
Key Findings on Implementation Discrepancies
The results show that reproducibility cannot be taken for granted. While some implementations successfully maintained fidelity, others failed to produce the expected sequences. A significant finding is that the Philox implementation in PyTorch is fundamentally incompatible with the reference algorithm, making it impossible to achieve exact output reproduction across different software stacks. These discrepancies prove that even when libraries claim to use the same underlying algorithm, the way they are built can lead to different results, complicating efforts to replicate AI workflows. The same ai safety question is explored in Compositional Policy Violations, which adds a research perspective.
Implications for Scientific Reproducibility
The study highlights that implementation fidelity is a necessary, yet often overlooked, requirement for scientific reproducibility. Because modern AI workflows frequently rely on multiple frameworks, these hidden inconsistencies can undermine the reliability of experimental results. To address this, the authors provide practical guidelines for researchers to achieve more reliable PRNG usage. Furthermore, they explore user-level techniques that can help recover portability and fidelity without requiring developers to modify the underlying library source code. The same ai safety question is explored in Learning Cardiac Features, which adds a research perspective. as detailed in the full paper on Arxiv
Comments (0)
to join the discussion
No comments yet
Be the first to share your thoughts!