Collaborative Agent Reasoning Engineering (CARE): A Three-Party Design Methodology for Systematically Engineering AI Agents with Subject Matter Experts, Developers, and Helper Agents This paper introduces CARE, a structured methodology designed to move AI agent development away from unreliable, trial-and-error "prompt tinkering." Instead of relying on ad-hoc adjustments, CARE provides a disciplined, stage-gated framework that treats agent design as a formal engineering process. By involving a specific trio of participants—Subject Matter Experts (SMEs), developers, and AI-based "helper agents"—the methodology ensures that an agent’s reasoning, tool usage, and verification criteria are clearly defined, testable, and maintainable. The Three-Party Collaboration CARE relies on a triadic workflow to bridge the gap between human expertise and technical implementation. SMEs provide the necessary domain knowledge and constraints, while developers handle the technical architecture. The "helper agents" act as essential facilitation infrastructure. These helper agents automate the heavy lifting of the design process by asking targeted questions, drafting documentation in structured Markdown, and proposing revisions. This allows human team members to focus on reviewing and approving the agent’s behavior at defined "stage gates," ensuring that the final system aligns with the intended goals. Breaking Down Agent Design The methodology deconstructs an AI agent into four core design targets that must be explicitly defined: Interaction Policy: How the agent interprets goals, manages uncertainty, and handles complex, multi-step reasoning. Domain Grounding: The authoritative knowledge and constraints that prevent the agent from producing plausible but incorrect information. Tool Orchestration: The logic governing how the agent selects and uses external tools, including error handling and data retrieval. Evaluation and Verification: The criteria used to measure success and detect performance regressions as the system evolves. By creating versioned artifacts for each of these targets, teams can audit their design decisions and ensure that the agent remains reliable even as underlying models or tools change. Performance and Validation To test the effectiveness of CARE, the authors applied the methodology to a NASA Earth Science data discovery task. They compared a CARE-designed agent against a baseline agent that used the same model and tools but lacked the structured design process. Using a two-gate evaluation—a large synthetic benchmark for rapid iteration and an SME-created "gold" benchmark for high-confidence validation—the CARE-designed agent demonstrated superior performance. It achieved higher retrieval accuracy (Recall@K) in both testing environments, proving that a systematic, artifact-driven approach leads to more capable and consistent agent behavior. Considerations and Limitations While CARE offers a repeatable path for building robust agents, the authors note several important limitations. The methodology’s success is heavily dependent on the quality of the helper agents and the expertise of the human team; if the helper agents fail to ask the right questions or if human reviews are superficial, the system may still harbor hidden flaws. Additionally, there is a risk of "overfitting" to benchmarks, where the design process becomes too focused on passing specific tests rather than handling the full range of real-world queries. Finally, because the methodology relies on LLMs, periodic re-verification is necessary to account for potential "model drift" over time.