Hugging Face Data Measurements Tool released — interactive dataset analysis with Streamlit

AI Impact Summary

HF's Data Measurements Tool (DMT) provides an open-source library plus a no-code UI to quantify dataset properties (descriptive stats, Zipf’s law adherence, nPMI, duplicates, label balance) and surface biases via interactive visualizations. It integrates with the Hugging Face Datasets Hub and Spaces, using Streamlit for the UI, enabling dataset creators and users to measure, compare, and iterate on datasets without heavy coding. The alpha v0 release currently demonstrates support on popular English datasets (e.g., SQuAD, IMDb, C4) and plans to extend to more languages and datasets in coming weeks, expanding coverage for multilingual and diverse data. This capability accelerates data quality audits and bias checks, supporting safer, more responsible model training pipelines and faster iteration in data-centric ML projects.