AI GitHub
MNN for embedded neural network inference
Convert trained models for inference in mobile and edge applications.
Key Takeaways
- MNN combines a model converter with a runtime and device-specific backends.
- Plan conversion and deployment on the same model revision.
- Convert a small known model and compare its outputs with the source model using identical input tensors.
- A conversion can succeed while output changes because of preprocessing, precision or layout.
How it works
MNN combines a model converter with a runtime and device-specific backends. A deployment can configure how the graph runs after converting the original model to the format expected by the runtime.
What you need
Plan conversion and deployment on the same model revision. The converter and runtime versions are useful provenance when a new device backend produces different results.
What to try first
Convert a small known model and compare its outputs with the source model using identical input tensors. Run the same test on the slowest device you plan to support.
Compare outputs after switching from the device accelerator to a reference CPU path. This helps isolate a precision difference from an error in the converter.
Where to look for failures
A conversion can succeed while output changes because of preprocessing, precision or layout. Compare results before choosing a faster backend based on latency alone.
Related Franklin pages
Use the same input tensors and output tolerance for export tests. An inference runtime may support the model format while the preferred device delegate covers only part of its graph.
- Deploy small machine-learning models into mobile and embedded applications. LiteRT for Android and edge inference
- Execute exported ONNX models with execution providers suited to the target device. ONNX Runtime for portable inference
For the constraints that shape mobile and embedded model deployment, see Franklin’s guide. On-device AI
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗