Hugging Face Transformers RAG gains Ray-based distributed document retrieval for faster fine-tuning

AI Impact Summary

Hugging Face Transformers' RAG integration with Ray decouples external-document retrieval from training, delivering about a 2x faster retrieval and improved multi-GPU fine-tuning scalability. This shifts the retrieval bottleneck from a PyTorch torch.distributed setup to Ray actors, enabling more scalable and parallel experimentation with RAG using PyTorch Lightning and Ray Tune. Adoption requires installing Ray and enabling the distributed_retriever option with multiple retrieval workers, replacing older retrieval pipelines and bringing Ray infrastructure into the training workflow.