AI GitHub
Modal Examples for remote GPU jobs
Study runnable patterns for GPU inference, training and jobs on Modal.
Key Takeaways
- The repository contains example applications with their dependency images, functions and resource declarations.
- The example code is available to inspect before you use the hosted runtime.
- Pick a small example with comparable inputs and inspect its image definition, secrets and resource request before running it.
- Review resource settings and the provider's usage terms before running an example at scale, particularly one that requests multiple GPUs.
How it works
The repository contains example applications with their dependency images, functions and resource declarations. It is a reference for adapting a specific workload to Modal's remote execution model.
What you need
The example code is available to inspect before you use the hosted runtime. Read its image, GPU and volume declarations to understand what the remote environment supplies.
What to try first
Pick a small example with comparable inputs and inspect its image definition, secrets and resource request before running it. Time a cold invocation and a subsequent warm one.
Use a time-limited test job and confirm shutdown behavior. Distinguish build time, remote queueing and execution in the results so you can compare example recipes fairly.
Where to look for failures
The examples target a hosted service. Review resource settings and the provider's usage terms before running an example at scale, particularly one that requests multiple GPUs.
Related Franklin pages
The execution environment matters as much as the Python code. Compare startup, dependency packaging and worker recovery before changing where a compute job runs.
- Declare inference services on Kubernetes with model serving runtimes and lifecycle controls. KServe for Kubernetes model deployments
- Scale Python ML work across workers while keeping resource needs explicit. Ray for distributed training and serving
For the concepts behind hosting model endpoints, see Franklin’s glossary. Model serving
Project documentation
Try it for your workflow
Explore the project’s documentation for setup and usage, or save this page to a collection for later.
Read the project documentation ↗