DeepSeek AI Releases Fire-Flyer File System (3FS): A High-Performance Distributed File System Designed to Address the Challenges of AI Training and Inference Workload

Key Takeaways

• DeepSeek AI has launched the Fire-Flyer File System (3FS), a distributed file system designed for AI training and inference workloads.
• 3FS addresses the growing data and computational demands of AI by offering a high-throughput, scalable storage solution.
• It combines the capabilities of thousands of SSDs with numerous storage nodes, bypassing traditional data locality limitations.
• Key features include Chain Replication with Apportioned Queries (CRAQ) for strong consistency and stateless metadata services supported by a transactional key-value store.
• Performance benchmarks show 3FS achieving a read throughput of 6.6 TiB/s and efficient data sorting, along with a high KVCache read throughput of 40 GiB/s for inference tasks.