Embedded AI evaluators won't be an easy fix, says Hugging Face policy chief
Irene Solaiman, chief policy officer at Hugging Face, says AI model testing is not yet a mature science. She argues that third-party evaluators placed inside AI companies need far more standardization.
Benchmarking of artificial intelligence (AI) models is not yet a consistent science, and the industry would struggle to rely on it as it stands. That was the message from Irene Solaiman, chief policy officer at the AI platform Hugging Face, speaking at TechCrunch Disrupt last week. She addressed whether the tests AI companies use to show one product beats another can be trusted.
Reproducibility and comparability
Solaiman called trust a weighted term. In her view, what matters is whether an evaluation's results can be reproduced and whether evaluators have enough access to the model itself, which is where open weights play a big role. Reproducibility has long been one of the trickiest parts of benchmarking and remains debated in the industry.
There is also the question of how different tests compare with one another. According to Solaiman, tests become ineffective relatively quickly, and the industry lacks a sufficiently thriving ecosystem of evaluators.
Embedded evaluators
The embedded evaluator concept places supposedly neutral third-party observers inside AI organizations. It has gained popularity in recent months as labs have faced a series of safety incidents that raised concerns about the technology's potential harms. Hugging Face was itself the target of one such incident. Rather than active government regulation of model safety, labs have pushed for the third-party approach.
Solaiman said the qualifications that determine who is an effective evaluator for which tasks need much more standardization. Doubts about neutrality and effectiveness persist: Anthropic's decision to hire the consulting firm Accenture as its first evaluator drew considerable backlash.
From models to whole systems
Evaluation is shifting from testing the model alone to testing the whole system around it, including the harness, the tool a user uses to access the model, and the agent. A weaker model with a better harness could outperform a frontier model, Solaiman said. Agentic evaluations are hard and expensive, and there are more evaluations for models than for harnesses.
In short, the evaluation field is currently something of a free-for-all, and for benchmarking to remain a useful tool, a more uniform approach across the industry may be needed.
