Foundry Local
Foundry Local is a developer SDK for embedding local AI inference in applications. According to the official website, it downloads, caches, loads, and calls optimized models directly on a user's device, without requiring an Azure subscription.
What it does
Foundry Local provides native SDKs for building apps that run AI models in-process. A typical lifecycle initializes the manager, selects a model alias, downloads and caches the model, loads it, and then calls a chat or audio client. The site shows Python and JavaScript as starting points, with C# and Rust available for production applications.
The SDK can select and register execution providers for NPU, GPU, or CPU hardware, reducing the need for application developers to handle device-specific setup. The runtime and cached models remain on the device, and the publisher says prompts, audio, and responses stay local. After models have been downloaded, the site describes the runtime as supporting offline user workflows.
Notable capabilities
- Native SDKs for Python, JavaScript, C#, and Rust
- Model download, local caching, loading, and in-process inference
- Chat and audio client access through the SDK lifecycle
- Automatic targeting of NPU, GPU, or CPU execution providers
- Optional OpenAI-compatible server access for development workflows and frameworks such as LangChain
Who it helps
Foundry Local is aimed at developers shipping applications that need on-device AI rather than a required hosted inference service. It may fit products where keeping prompts, audio, and responses on the user's hardware is important, or where an app should continue working offline after its models are available. It is a developer-focused runtime and SDK rather than a general-purpose chat interface. For comparison, Atomic Chat is presented as a desktop and mobile application for downloading models and chatting with them locally, while Foundry Local focuses on embedding local inference into an app.
How it fits a workflow
Developers can install the SDK package, choose a catalog model alias, download it, load it, and call inference from their application process. The SDK is positioned as the primary path for production use. CLI or REST tools can supplement development, while the optional OpenAI-compatible server can support frameworks that expect an OpenAI-style endpoint.
Strengths and limits
Its main strengths are local execution, a multi-language native SDK approach, cached models for offline use, and automatic handling of common processor targets. The site does not specify the full model catalog, supported operating systems, hardware requirements, or pricing, so those details should be assessed for a particular deployment. Model downloads also form part of the setup workflow, even though later inference can use the local runtime and cache.