Amazon has announced new customization capabilities for its Nova models within Amazon SageMaker AI. These features enable customers to tailor Nova Micro, Nova Lite, and Nova Pro models thro…
Amazon has announced new customization capabilities for its Nova models within Amazon SageMaker AI. These features enable customers to tailor Nova Micro, Nova Lite, and Nova Pro models throughout the training lifecycle, including pre-training, supervised fine-tuning, and alignment.
The goal is to allow businesses to adapt these powerful foundation models to their specific needs, improving accuracy, controlling costs, and optimizing latency for business-critical applications. The customizations are designed to work seamlessly with Amazon Bedrock, offering both on-demand and provisioned throughput inference options.
The article details various customization techniques available for Nova models. These include supervised fine-tuning, which uses input-output pairs for task-specific adaptation; alignment, which shapes model output based on preferences like brand voice using methods like Direct Preference Optimization (DPO) and Proximal Policy Optimization (PPO); continued pre-training, which expands the model's knowledge with proprietary data; and knowledge distillation, which transfers knowledge from a larger model to a smaller, more efficient one.
The choice of technique depends on factors like the customer's goals, data availability, and computing resources. Supervised fine-tuning offers two approaches: Parameter-efficient fine-tuning (PEFT), which updates only a subset of parameters, and full fine-tuning (FFT), which updates all parameters.
Alignment techniques like DPO and PPO allow for control over the model's outputs, ensuring they align with desired preferences. Continued pre-training allows the models to learn from large amounts of unlabeled data. Knowledge distillation is useful when adequate input-output samples are unavailable, allowing a more powerful model to enhance training data.
The article highlights that early access customers are already utilizing these customization capabilities. It provides a step-by-step example of customizing the Nova Micro model using direct preference optimization in Amazon SageMaker Studio, showing how to select a model, choose a fine-tuning recipe, and run the recipe within either SageMaker training jobs or SageMaker HyperPod.
This streamlined process allows users to readily adapt the Nova models to their unique requirements.