Back to AI Research

AI Research

Abductive Reasoning with Probabilistic Commonsense | AI Research

Key Takeaways

  • Abductive Reasoning with Probabilistic Commonsense Recent advancements in AI have sought to improve the reasoning capabilities of Large Language Models (LLMs...
  • Recent efforts to improve the reasoning abilities of Large Language Models (LLMs) have focused on integrating formal logic solvers within neurosymbolic frameworks.
  • A key challenge is that formal solvers lack commonsense world knowledge, preventing them from making reasoning steps that humans find obvious.
  • Prior methods address this by using LLMs to supply missing commonsense assumptions, but these approaches implicitly assume universal agreement on such commonsense facts.
  • In reality, commonsense beliefs vary across individuals.
Paper AbstractExpand

Recent efforts to improve the reasoning abilities of Large Language Models (LLMs) have focused on integrating formal logic solvers within neurosymbolic frameworks. A key challenge is that formal solvers lack commonsense world knowledge, preventing them from making reasoning steps that humans find obvious. Prior methods address this by using LLMs to supply missing commonsense assumptions, but these approaches implicitly assume universal agreement on such commonsense facts. In reality, commonsense beliefs vary across individuals. We propose a probabilistic framework for abductive commonsense reasoning that explicitly models this variation, aiming to determine whether most people would judge a statement as true or false. We introduce Probabilistic Abductive CommonSense (PACS), a novel algorithm that uses an LLM and a formal solver to sample proofs as observations of individuals' distinct commonsense beliefs, and aggregates conclusions across these samples. Empirically, PACS outperforms chain-of-thought reasoning, prior neurosymbolic methods, and search-based approaches across multiple benchmarks.

Abductive Reasoning with Probabilistic Commonsense
Recent advancements in AI have sought to improve the reasoning capabilities of Large Language Models (LLMs) by connecting them to formal logic solvers. A significant hurdle in this field is that while these solvers are excellent at processing strict rules, they lack the "commonsense" knowledge that humans use to navigate everyday life. Previous attempts to fix this have relied on LLMs to provide missing facts, but these methods often incorrectly assume that there is a single, universally agreed-upon set of commonsense beliefs. This paper introduces a new framework that acknowledges that commonsense is subjective and varies from person to person.

Modeling Diverse Perspectives

The core idea behind this research is that human beliefs are not monolithic. When faced with an ambiguous situation, different people may hold different, yet internally consistent, sets of beliefs that lead them to opposite conclusions. Rather than forcing an LLM to find one "correct" set of facts, the authors propose a probabilistic framework. They define the "abductive probability" of a statement as the average of how many people would judge that statement to be true versus false. By modeling this variation, the system can determine if a query is "abductively true"—meaning it is the conclusion most people would reach.

The PACS Algorithm

To implement this, the researchers developed the Probabilistic Abductive CommonSense (PACS) algorithm. This approach uses an LLM to sample various "commonsense vectors," which represent the distinct belief systems of different individuals. The algorithm works by:

  • Translating a reasoning problem into formal logic premises and a query.

  • Sampling potential commonsense assumptions one step at a time.

  • Using a formal logic solver to check if these assumptions, combined with the premises, lead to a clear conclusion.

  • Employing an early-stopping mechanism to minimize computational cost, halting the process as soon as a conclusion is reached.
    Once multiple samples are collected, the system aggregates the results to estimate the probability of the query being true or false, ultimately providing the answer that aligns with the majority of the sampled perspectives.

Performance and Results

The researchers tested PACS across several abductive reasoning benchmarks to evaluate its effectiveness. The results indicate that PACS outperforms traditional chain-of-thought reasoning, as well as existing neurosymbolic and search-based methods. By explicitly accounting for the diversity of human belief rather than treating commonsense as a fixed, universal truth, the framework demonstrates improved accuracy and provides a more credible approach to handling complex, ambiguous reasoning tasks.

Comments (0)

No comments yet

Be the first to share your thoughts!