AI GitHub
BrowserGym for evaluating browser agents
Run browser agents in a Gym-style task environment with explicit observations and actions.
Key Takeaways
- BrowserGym wraps web task environments behind a common interface.
- Start with one supported benchmark integration and its documented dependencies.
- Run one supported task with a simple agent and record its observation and action sequence.
- Benchmark integrations have their own environment requirements.
How it works
BrowserGym wraps web task environments behind a common interface. Observations and action representations can be varied while keeping a task and scoring setup fixed.
What you need
Start with one supported benchmark integration and its documented dependencies. Different tasks may have different login, browser or application setup requirements.
What to try first
Run one supported task with a simple agent and record its observation and action sequence. Compare it against another agent using the same interface.
Replay a scripted action sequence against the same task. That gives you a baseline to distinguish an environment regression from a model's choices.
Where to look for failures
Benchmark integrations have their own environment requirements. Pin task IDs, action settings and benchmark versions to make scores comparable.
Related Franklin pages
Choose the benchmark that matches the interface you care about. Compare environment setup and scoring evidence before treating an aggregate as a measure of agent ability.
- Test agents on simulated workplace tasks that span software and office tools. TheAgentCompany for workplace agent evaluation
- Evaluate agents that operate real desktop application interfaces in a controlled environment. OSWorld for computer-use benchmarks
For an agent interface that reaches mobile apps, see Franklin’s guide to Phone-Use. Phone-Use
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗