Back to AI Research

AI Research

Why Sampling Is Not Choosing: Intentionality, Agenc... | AI Research

Key Takeaways

  • Why Sampling Is Not Choosing: Intentionality, Agency, and Moral Responsibility in Large Language Models This paper examines the growing trend of attributing...
  • Recent advances in large language models (LLMs) have prompted claims that such systems exhibit agency or qualify as moral agents.
  • This paper argues that these attributions are misguided.
  • We maintain that moral responsibility requires commitment-bearing agency grounded in intrinsic intentionality and self-attributed action, and that such agency constitutes the form of free will relevant to responsibility.
  • Although LLMs generate coherent and normatively evaluable outputs, their operation is fully characterized by probabilistic input-output mappings learned from data.
Paper AbstractExpand

Recent advances in large language models (LLMs) have prompted claims that such systems exhibit agency or qualify as moral agents. This paper argues that these attributions are misguided. We maintain that moral responsibility requires commitment-bearing agency grounded in intrinsic intentionality and self-attributed action, and that such agency constitutes the form of free will relevant to responsibility. Although LLMs generate coherent and normatively evaluable outputs, their operation is fully characterized by probabilistic input-output mappings learned from data. Their apparent intentionality is derived rather than intrinsic, and their outputs are neither owned as commitments nor guided by reasons. Variability introduced by stochastic sampling does not amount to choice or authorship. We address objections from the intentional stance, functionalism, compatibilism, and the presence of moral reasoning in model outputs, arguing that none suffice to establish genuine agency.

Why Sampling Is Not Choosing: Intentionality, Agency, and Moral Responsibility in Large Language Models
This paper examines the growing trend of attributing moral agency and responsibility to large language models (LLMs). The author argues that these attributions are fundamentally misguided. While LLMs can produce outputs that are normatively evaluated—meaning they can be judged as correct or incorrect—the paper contends that they lack the essential qualities of a moral agent. By defining the necessary conditions for moral responsibility, the research demonstrates that current AI architectures are incapable of meeting the requirements for genuine accountability.

The Requirements for Moral Agency

The paper establishes that moral responsibility is not merely about the output an entity produces, but about the nature of the entity itself. For a system to be a moral agent, it must possess "commitment-bearing agency." This requires two primary components: intrinsic intentionality and the capacity for self-attributed action. Intrinsic intentionality means that the system’s thoughts or states have meaning for the system itself, rather than just for the people observing it. Self-attributed action refers to the ability of an agent to recognize its own actions as its own, to endorse them, and to be answerable for them. The author argues that this form of agency is the foundation of the free will necessary for moral responsibility.

Why LLMs Fall Short

The research highlights that transformer-based models are essentially probabilistic input-output mechanisms. During training, these models learn to predict the next token in a sequence based on statistical patterns in data. They do not possess a "first-person" perspective or an internal understanding of the world. Because their "aboutness" is derived from human interpretation rather than originating within the system, they lack intrinsic intentionality. Furthermore, the paper notes that even advanced techniques like chain-of-thought prompting or tree-based search do not constitute deliberation. These processes are simply more complex forms of statistical selection, not the evaluation of reasons by an autonomous agent.

The Role of Stochastic Sampling

A central point of the paper is that the variability introduced by sampling—the process by which an LLM chooses from a probability distribution of possible next words—is not the same as making a choice. While sampling makes a model’s output unpredictable, this randomness does not equate to authorship or free will. True agency involves an agent taking ownership of a decision, whereas an LLM’s output is the result of a mathematical computation. Therefore, the stochastic nature of these models provides no evidence of the agency required for moral responsibility.

Implications for Responsibility

The paper concludes that because LLMs lack the necessary structural conditions for agency, they cannot be held morally responsible for their actions. This does not mean that the behavior of these systems is unimportant; rather, it shifts the focus of responsibility back to the humans involved. Designers, developers, and users remain the true bearers of moral responsibility. Efforts to regulate or align AI should be viewed as methods for managing tools within human normative frameworks, rather than as attempts to govern autonomous moral agents.

Comments (0)

No comments yet

Be the first to share your thoughts!