Google DeepMind Releases Gemma 4 12B: Encoder-Free Multimodal AI

Google DeepMind has officially released Gemma 4 12B, a dense, multimodal model that marks a significant shift in architecture by eliminating traditional, separate encoders. By allowing vision and audio data to flow directly into the LLM backbone, the model achieves high-level performance while remaining efficient enough to run on consumer-grade hardware with 16 GB of RAM. The model is available under the Apache 2.0 license, with weights accessible on Hugging Face and Kaggle.

A Streamlined, Encoder-Free Architecture

The core innovation of Gemma 4 12B is its unified, decoder-only transformer design. Previous mid-sized models relied on heavy, separate components—such as 550M-parameter vision encoders and 300M-parameter audio encoders—which added significant latency and overhead. In this new iteration, those components have been removed entirely.
The vision pipeline now utilizes a 35M-parameter embedder that processes raw image patches independently using a single matrix multiplication, avoiding the need for attention layers. Similarly, the audio pipeline projects raw 16 kHz audio frames directly into the same embedding space as text tokens, bypassing the need for feature extraction or conformer layers. This unified weight space simplifies the fine-tuning process, as LoRA or full-model tuning can now update vision, audio, and text processing in a single pass.

Multimodal Capabilities and Local Performance

Gemma 4 12B is the first mid-sized Gemma model to feature native audio input, while also supporting text, image, and video modalities. Despite its smaller footprint, Google DeepMind reports that the model performs near the level of the 26B Mixture of Experts variant on standard benchmarks, all while utilizing less than half the memory.
The model is designed for practical, agentic workflows. Its demonstrated capabilities include automatic speech recognition, speaker diarization, and video understanding. In internal testing, the model successfully processed a five-minute video segment by analyzing frames at 1 FPS. Furthermore, the model has shown strong performance in coding tasks, including the ability to generate and serve its own image-processing applications.

Deployment and Ecosystem Integration

To support local deployment, Google DeepMind has released a dedicated Multi-Token Prediction (MTP) drafter model to reduce inference latency. The model is compatible with a wide range of tools and frameworks, including llama.cpp, MLX, vLLM, Ollama, SGLang, Unsloth, and LM Studio.
Developers can integrate the model using Hugging Face Transformers or the LiteRT-LM CLI, which provides an OpenAI-compatible local API server. For those looking to build agentic applications, the official Gemma Skills repository offers pre-built capabilities. The model is also ready for broader enterprise deployment via Google Cloud Run, GKE, or the Gemini Enterprise Agent Platform Model Garden.