Andrew Ng Launches OpenWorker: Local-First Desktop AI Agent

Andrew Ng has introduced OpenWorker, an open-source, local-first desktop AI agent designed to shift the paradigm from conversational AI to task-based execution. Unlike traditional AI assistants that provide text-based responses, OpenWorker focuses on delivering finished work, such as polished documents, updated calendars, triaged inboxes, or Slack replies containing specific data. By breaking down high-level user outcomes into actionable steps, the agent operates across local files and connected applications, requiring human confirmation before executing consequential tasks.

Architecture and Local-First Design

The OpenWorker architecture is built as a four-layer system that runs entirely on the user's machine. The desktop shell utilizes Tauri 2 to wrap a React 18 user interface, which supervises a Python 3.10+ agent server built on FastAPI and uvicorn. The engine leverages aisuite, Andrew Ng’s provider-agnostic LLM library, to manage interactions. This setup ensures that model calls, connector tokens, and API keys remain local, with secrets designed to stay outside of the model’s context, prompts, or traces.
The project is highly modular, featuring a capability and connector layer that supports vetted local tools, hosted integrations, and the Model Context Protocol (MCP). While the application includes an optional cloud broker for OAuth handshakes, it remains fully functional for users who prefer to manually input credentials, ensuring that connector tokens are never stored in the cloud.

Model Flexibility and Tool Integration

OpenWorker follows a bring-your-own-model approach, offering a curated matrix of 30 entries. Users can connect to native providers including OpenAI, Anthropic, and Google, or utilize OpenAI-compatible vendors such as DeepSeek, Mistral, and Qwen. For those seeking fully local execution, the platform supports open-weight models through Together AI and Fireworks, as well as local runtimes via Ollama, which requires no API key.
The agent’s capability layer allows it to interact with a wide range of environments, including file systems, Git repositories, and shell commands. To mitigate security risks, the built-in operations persona is explicitly programmed to treat all incoming data from logs, web sources, and files as untrusted, establishing a robust posture against prompt injection.

Advanced Permission and Risk Management

A defining feature of OpenWorker is its typed permission engine, which categorizes every tool call into four risk classes: read, write_local, exec, and external. These classes are governed by five distinct permission modes, ranging from read-only planning to fully automated execution. This system ensures that the agent operates within defined boundaries, such as path-scoped workspace mutations.
The design prioritizes human oversight through specific operational rules. Even in unattended mode, the agent does not bypass the need for human input; instead, it routes prompts to an inbox and suspends operations until the user responds. Furthermore, the system is designed to prevent the automation of shell commands, which require manual approval by design, ensuring that the agent remains a controlled coworker rather than an autonomous black box.