RL framework adds UCB exploration via Q-ensembles

AI Impact Summary

Enabling UCB exploration via Q-ensembles introduces an ensemble-based upper confidence bound for action selection, shifting exploration from epsilon-greedy to confidence-driven decisions. This can improve sample efficiency and policy quality in environments with sparse rewards, especially when combining multiple Q heads to estimate uncertainty. Expect higher compute and memory costs due to maintaining multiple Q-functions, and you’ll need new hyperparameters (ensemble size, confidence scaling) and migration in training loops to replace standard epsilon-greedy. Performance will hinge on careful tuning to avoid over-exploration and instability during updates.