Back to AI Research

AI Research

From Intent to Execution: Composing Agentic Workflo... | AI Research

Key Takeaways

  • From Intent to Execution: Composing Agentic Workflows with Agent Recommendation Creating Multi-Agent Systems (MAS)—where multiple AI agents work together to...
  • Multi-Agent Systems (MAS) built using AI agents fulfill a variety of user intents that may be used to design and build a family of related applications.
  • However, the creation of such MAS currently involves manual composition of the plan, manual selection of appropriate agents, and manual creation of execution graphs.
  • This paper introduces a framework for the automated creation of multi-agent systems which replaces multiple manual steps with an automated framework.
  • The proposed framework consists of software modules and a workflow to orchestrate the requisite task- specific application.
Paper AbstractExpand

Multi-Agent Systems (MAS) built using AI agents fulfill a variety of user intents that may be used to design and build a family of related applications. However, the creation of such MAS currently involves manual composition of the plan, manual selection of appropriate agents, and manual creation of execution graphs. This paper introduces a framework for the automated creation of multi-agent systems which replaces multiple manual steps with an automated framework. The proposed framework consists of software modules and a workflow to orchestrate the requisite task- specific application. The modules include: an LLM-derived planner, a set of tasks described in natural language, a dynamic call graph, an orchestrator for map agents to tasks, and an agent recommender that finds the most suitable agent(s) from local and global agent registries. The agent recommender uses a two-stage information retrieval (IR) system comprising a fast retriever and an LLM-based re-ranker. We implemented a series of experiments exploring the choice of embedders, re- rankers, agent description enrichment, and supervising critique agent. We benchmarked this system end-to-end, evaluating the combination of planning, agent selection, and task completion, with our proposed approach. Our experimental results show that our approach outperforms the state-of-the- art in terms of the recall rate and is more robust and scalable compared to previous approaches. The critique agent holistically reevaluates both agent and tool recommendations against the overall plan. We show that the inclusion of the critique agent further enhances the recall score, proving that the comprehensive review and revision of task-based agent selection is an essential step in building end-to-end multi-agent systems.

From Intent to Execution: Composing Agentic Workflows with Agent Recommendation
Creating Multi-Agent Systems (MAS)—where multiple AI agents work together to fulfill a user's goal—is currently a manual, labor-intensive process. Developers must write plans, select specific agents, and build execution graphs by hand. This paper introduces "AutoMAS," an automated framework designed to streamline this process. By using an LLM-based planner and a sophisticated agent recommendation system, the framework can take a user's natural language intent and automatically generate, configure, and execute a multi-agent workflow with minimal human intervention.

Automating Workflow Design

The AutoMAS framework functions by breaking down a user's request into a series of subtasks. It represents these tasks as a finite state machine, which allows for flexible, non-sequential execution. To ensure the system is robust, it uses a "Variable Call Graph" that can reroute tasks if an agent becomes unavailable or if environmental conditions change. This architecture allows the system to adapt to real-time constraints, such as prioritizing lower costs or faster execution speeds based on user requirements.

The Two-Stage Agent Recommender

A central challenge in building these systems is finding the right agent for a specific job from a vast registry. AutoMAS solves this with a two-stage retrieval process:

  • Retriever: A fast, hybrid search (combining keywords and vector embeddings) that quickly identifies a broad list of potentially relevant agents.

  • Re-ranker: An LLM-based module that analyzes the candidates from the first stage to provide a more nuanced, accurate selection based on the specific task context.
    To further improve accuracy, the framework uses "agent description enrichment," where it generates synthetic queries from agent documentation at the time of indexing. This creates a richer, more searchable profile for each agent, making it easier for the system to match them to user needs.

The Role of the Critique Agent

The framework includes an optional "critique" mechanism that acts as a quality control layer. Unlike the recommender, which focuses on finding the best individual agent, the critique agent evaluates the entire plan. It checks whether the selected agents are compatible with one another and whether the overall workflow adheres to user constraints like security, cost, and latency. Experiments showed that including this critique step significantly improved the system's recall scores, proving that a holistic review of the entire workflow is essential for building reliable, end-to-end multi-agent systems.

Performance and Scalability

The researchers benchmarked AutoMAS using established datasets to evaluate both planning and agent selection. The results indicate that the two-stage retrieval approach, combined with description enrichment and the critique mechanism, outperforms existing state-of-the-art methods. The system demonstrated high robustness and scalability, effectively navigating the "needle in an agentic stack" problem as the number of available agents grows. By automating the transition from intent to execution, AutoMAS provides a scalable path for developing complex, adaptive AI applications.

Comments (0)

No comments yet

Be the first to share your thoughts!