An Agentic Approach for Active Data Collection, Tra... | AI Research

An Agentic Approach for Active Data Collection, Travel Behavior Modeling, and Weather-Sensitive Demand Prediction proposes a three-agent workflow to streamline travel behavior research. By integrating conversational data collection, structured data processing, and predictive modeling, the researchers aim to bridge the gap between digital survey administration and behavioral analysis. The study evaluates how different configurations of large language models (LLMs) perform in predicting commuter mode choices under varying weather conditions, benchmarking these results against traditional statistical and machine-learning methods.

A Three-Agent Workflow

The researchers developed a framework consisting of three specialized agents: a Data Collection Agent, a Data Processing Agent, and a Data Modeling Agent. Each agent is designed to handle a specific stage of the research process:

• Data Collection Agent: Uses a chatbot to administer an image-augmented stated-preference survey, collecting mode choices from 92 student commuters across five weather scenarios.

• Data Processing Agent: Converts raw survey responses into a structured format suitable for analysis, handling tasks like data validation and feature construction.

• Data Modeling Agent: Executes predictive models, including discrete choice models, machine learning, and various LLM configurations, to estimate travel behavior.
  This modular design allows researchers to maintain an auditable workflow where each step is clearly defined and reproducible.

Evaluating LLM Predictive Performance

The study tested nine locally deployed LLMs, ranging from 2 to 35 billion parameters, to see how well they could predict individual travel choices without task-specific training. The researchers experimented with several prompting strategies:

• Zero-shot: The models made predictions without prior examples.

• Few-shot: The models were provided with a small number of examples to guide their predictions.

• Persona and History: The models were given information about the traveler’s habitual travel patterns or specific personas.

• Vision-based: The models were shown the same weather images presented to the human respondents.
  The best text-only zero-shot LLM achieved a 69.9% five-class accuracy, slightly outperforming the random forest benchmark of 69.6%. When vision-capable models were used, the accuracy reached 71.5%, suggesting that visual context provides meaningful information for predicting travel behavior.

Key Findings on Travel Behavior

The empirical results from the survey indicated that commuter mode choices are highly sensitive to environmental context. Specifically, the researchers observed that cycling was particularly sensitive to adverse weather, while public transit usage increased significantly during snowy conditions. The study found that providing LLMs with habitual travel information consistently improved prediction accuracy. Furthermore, "Expert" framing generally yielded better results than "Role-Play" prompts, and persona information proved most valuable when habitual travel data was missing.

Limitations and Considerations

While the framework provides a structured approach to travel behavior research, the authors note that the study is based on a limited sample of 92 student commuters. Because the findings rely on this specific group, the authors emphasize that broader application of this multi-agent workflow requires further validation using larger and more representative samples of travelers. Additionally, while LLMs show promise in predicting behavior, the researchers caution that simulated responses should not be treated as equivalent to human-subject evidence without independent verification.