LocalAI: LocalAI 3.11.0 Release: Realtime Audio, Music Generation, and Expanded ASR | SignalBreak

LocalAI: LocalAI 3.11.0 Release: Realtime Audio, Music Generation, and Expanded ASR | SignalBreak | SignalBreak

Source text

🎉 LocalAI 3.11.0 Release! 🚀 LocalAI 3.11.0 is a massive update for Audio and Multimodal capabilities . We are introducing Realtime Audio Conversations , a dedicated Music Generation UI , and a massive expansion of ASR (Speech-to-Text) and TTS backends. Whether you want to talk to your AI, clone voices, transcribe with speaker identification, or generate songs, this release has you covered. Check out the highlights below! 📌 TL;DR Feature Summary Realtime Audio Native support for audio conversations , enabling fluid voice interactions similar to OpenAI's Realtime API. Documentation Music Generation UI New UI interface for MusicGen (Ace-Step), allowing you to generate music from text prompts directly in the browser. New ASR Backends Added WhisperX (with Speaker Diarization), VibeVoice , Qwen-ASR , and Nvidia NeMo . TTS Streaming Text-to-Speech now supports streaming mode for lower latency responses. (VoxCPM only for now) vLLM Omni Added support for vLLM Omni , expanding our high-performance inference capabilities. Speaker Diarization Native support for identifying different speakers in transcriptions via WhisperX . Hardware Expansion Expanded build support for CUDA 12/13, L4T (Jetson), SBSA, and better Metal (Apple Silicon) integration with MLX backends Breaking Changes ExLlama (deprecated) and Bark (unmaintained) backends have been removed. 🚀 New Features & Major Enhancements 🎙️ Realtime Audio Conversations LocalAI 3.11.0 introduces native support for Realtime Audio Conversations . Enables fluid, low-latency voice interaction with agents. Logic handled directly within the LocalAI pipeline for seamless audio-in/audio-out workflows. Support for STT/TTS and voice-to-voice models (experimental) Support for tool calls 🗣️ Talk to your LocalAI : This brings us one step closer to a fully local, voice-native assistant experience compatible with standard client implementations. Check here for detailed documentation. 🎵 Music Generation UI & Ace-Step We have added a dedicated int

View original source

LocalAI 3.11.0 Release: Realtime Audio, Music Generation, and Expanded ASR

More from LocalAI

Get alerts for LocalAI