Optimum 1.2 adds accelerated ONNX Runtime inference for Hugging Face Transformers pipelines

AI Impact Summary

Optimum 1.2 now embeds ONNX Runtime inference into the Hugging Face pipeline flow, enabling transformers pipelines to run with optimized graphs and quantization. The change introduces ORTModelForQuestionAnswering and related tools (ORTOptimizer, ORTQuantizer) as API-compatible replacements for PyTorch-backed models, letting production QA workloads like deepset/roberta-base-squad2 be converted to ONNX and served via the transformers pipeline. This will alter latency, model size, and throughput characteristics for inference-heavy workloads, particularly on CPU instances with AVX512 support, and is designed to push transformer workloads from experimentation to production at scale. Teams can pull optimized checkpoints from the Hugging Face Hub and apply dynamic quantization and graph optimization to squeeze performance gains.