Google Launches LiteRT.js for High-Performance Browser AI Inference

Google has released LiteRT.js, a new JavaScript binding for its on-device inference library, LiteRT. By compiling the native runtime to WebAssembly, Google enables developers to execute .tflite models directly within web browsers. This approach allows the same model artifacts used on Android and iOS to run locally on the web, offering benefits such as enhanced user privacy, zero server costs, and ultra-low latency.

Architecture and Performance

Unlike earlier web AI solutions like TensorFlow.js, which rely on JavaScript-based kernels, LiteRT.js ships the native cross-platform runtime. This allows web applications to leverage existing optimizations for hardware, quantization, and performance developed for mobile and desktop environments. Google reports that LiteRT.js is up to 3x faster than other web runtimes across CPU and GPU inference. Furthermore, when utilizing GPU or NPU acceleration, the runtime can achieve a 5–60x speedup compared to its own CPU execution path.
The runtime supports three distinct backends: XNNPACK for CPU, ML Drift over WebGPU for GPU, and the experimental WebNN API for NPUs. The system operates on an all-or-nothing delegation model, meaning a graph cannot be split across different hardware. If a model cannot be fully delegated to a chosen accelerator, the runtime falls back to WebAssembly execution.

Implementation and Memory Management

To integrate models, developers can convert PyTorch models to .tflite using LiteRT Torch, provided they meet strict requirements such as being TorchDynamo-exportable without Python conditional branches or dynamic input dimensions. Once a model is loaded, the runtime provides a straightforward API for execution. However, developers must be diligent regarding memory management. Unlike some other frameworks, LiteRT.js does not provide automatic garbage collection for tensors. Every tensor must be explicitly deleted using the delete method to prevent device memory leaks.

Positioning Within the Ecosystem

Google positions LiteRT.js as a specialized replacement for TensorFlow.js Graph Models rather than a complete replacement for the entire TensorFlow.js library. The two tools are designed to be interoperable, with the @litertjs/tfjs-interop package allowing tensors to pass between them. While TensorFlow.js remains the recommended tool for pre- and post-processing tasks, LiteRT.js provides a high-performance path for inference. To assist with development, Google has released tools such as the model-tester, which allows developers to verify model performance across WebNN, WebGPU, and CPU backends using random inputs before finalizing their implementation.