Hugging Face Rearchitecting Uploads and Downloads — Content-Addressed Store (CAS)

AI Impact Summary

Hugging Face is redesigning its upload and download architecture to improve performance and scalability, introducing a content-addressed store (CAS) to optimize file transfers. This shift moves beyond the current S3 and Cloudfront setup, enabling byte-level analysis and deduplication, particularly for large model files like the 131GB Llama 3-70B weights. The new protocol aims to reduce upload speeds by 10-25% and provides enhanced security and operational telemetry for enterprise users.