AI GitHub
TheAgentCompany for workplace agent evaluation
Test agents on simulated workplace tasks that span software and office tools.
Key Takeaways
- The benchmark supplies a simulated company environment, task definitions and evaluators.
- Read the environment setup before attempting multiple tasks.
- Choose one task and inspect its starting data and scoring code before running an agent.
- A score is tied to the provided environment and evaluator.
How it works
The benchmark supplies a simulated company environment, task definitions and evaluators. Agents interact with systems seeded with records instead of answering isolated question prompts.
What you need
Read the environment setup before attempting multiple tasks. Each benchmark task expects a particular starting snapshot and can depend on several simulated applications.
What to try first
Choose one task and inspect its starting data and scoring code before running an agent. Compare the saved artifacts to the expected result.
Read the evaluator's checks before adapting the task. Some goals require changes in more than one application, and a plausible screenshot may miss those requirements.
Where to look for failures
A score is tied to the provided environment and evaluator. Keep the environment version, agent settings and tool access fixed when comparing runs.
Related Franklin pages
Choose the benchmark that matches the interface you care about. Compare environment setup and scoring evidence before treating an aggregate as a measure of agent ability.
- Evaluate agents that operate real desktop application interfaces in a controlled environment. OSWorld for computer-use benchmarks
- Run browser agents in a Gym-style task environment with explicit observations and actions. BrowserGym for evaluating browser agents
For an agent interface that reaches mobile apps, see Franklin’s guide to Phone-Use. Phone-Use
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗