ExecuTorch for edge deployment of PyTorch models

How it works

The export flow captures a graph and lowers it for a target backend before the runtime executes it. Deployment choices such as quantization and delegation enter at the export stage.

What you need

The normal path starts with PyTorch export and ends with an artifact the device runtime can load. Keep the export program and model revision together so a changed operator can be traced.

What to try first

Export a small model first and compare its outputs on the target device against PyTorch. Measure app startup, execution and peak memory with production-shaped inputs.

Measure the artifact size and startup time before and after adding the backend delegate. These costs matter when the model is part of a mobile release.

Where to look for failures

Operators and dynamic shapes must work through the selected export and backend path. Add preprocessing parity tests when image or audio features are prepared outside the model.

Related Franklin pages

Use the same input tensors and output tolerance for export tests. An inference runtime may support the model format while the preferred device delegate covers only part of its graph.

• Convert trained models for inference in mobile and edge applications. MNN for embedded neural network inference
• Deploy small machine-learning models into mobile and embedded applications. LiteRT for Android and edge inference

For the constraints that shape mobile and embedded model deployment, see Franklin’s guide. On-device AI

Project documentation

• Current project README
• README at the reviewed revision