Ceci n'est pas une pipe: AI systems as semantic... | AI Research

Ceci n'est pas une pipe: AI systems as semantic abstractions
This paper argues that we should stop viewing AI systems as "magic" or "oracles" that inherently produce truth. Instead, the authors propose that an AI system’s output is merely an engineered representation—a "semantic abstraction"—that must be carefully checked against reality. By creating a formal framework to distinguish between what an AI system uses, what its sources say, and what is actually known in a domain, the authors provide a way to identify and define common AI failures like hallucinations, stale information, and unsupported assertions.

Moving Beyond the "Oracle" Myth

The authors contend that treating AI as an oracle—where generated text is conflated with factual reality—is a fundamental mistake. They compare an AI’s output to René Magritte’s famous painting of a pipe, which is a representation of a pipe, not the object itself. To move toward more reliable systems, they suggest that AI outputs, tool calls, and actions should be treated as messages that require interpretation and verification against an underlying, objective semantics rather than being accepted based on their "fluent" or convincing tone.

A Framework for Information States

To evaluate whether an AI system is acting correctly, the authors define an "Information State" consisting of three distinct layers:

• Universal Knowledge Base: The accepted domain knowledge and rules that represent the "ground truth" of a subject.

• Source-derived Knowledge Base: The information extracted from specific references (like files or databases), which may be incomplete, incorrect, or outdated.

• Effective Knowledge Base: The material the system is actually using at a specific moment, which is often a compressed or lossy version of the other two layers.
  By separating these, the framework allows developers to pinpoint exactly where a system fails. For example, a system might fail because it is using a "stale" source, or because it is making an "extrapolation" that the underlying domain rules do not support.

Tracing and Accountability

The paper emphasizes the importance of maintaining a "trace"—a record of events, observations, and actions taken by the system. In their example of an AI-aided passport renewal app, the authors show that a system cannot simply claim a form is "ready." Instead, the system must be able to point to a specific sequence of events: the user’s input, the retrieval of official guidance, the result of a picture-check, and the user’s confirmation. If the system cannot justify its conclusion through this chain of evidence, the output is considered unsupported.

Practical Implications

The authors suggest that this framework provides a necessary vocabulary for building and auditing AI systems. By formalizing how messages are interpreted and checked, developers can move away from relying on the apparent fluency of a model and toward a system where actions are justified by explicit authority and reliable claims. While the authors acknowledge that some domains (like prose) are harder to map than others, they believe this approach helps clarify a system's obligations and makes its failure modes easier to diagnose and correct.