Hybrid setup pairing cheap Jev model with Opus matches frontier accuracy at over 4x lower cost

Data company Hightouch has published a benchmark testing whether language models can automatically grade identity resolution (IDR) results. IDR is the process of mapping data records to real people, for example so that all of one customer's purchases can be tied together. The most common problem is over-merging, where a single profile ends up containing records from several people.

Why it is hard

Real data has no answer key, so quality is checked by hand at roughly 50 identities per hour. Getting the over-merge rate to within ±1% takes more than 1,000 samples. Hightouch wanted models to take over this work. It focused on Jev and GPT-6 Luna, models that return a 0–1 confidence score and are faster and cheaper than frontier models.

Test results

The benchmark used pseudopeople, a tool that simulates a fictional US town and adds realistic data errors. From it, 1,000 identities were built: half were one person with noisy records, half were over-merges.

Claude Opus 5.5 reached 99.0% accuracy at $3.86 per 1,000 identities. Luna with a 70% threshold scored 97.3%, and Jev with a 60% threshold scored 95.7%. The two cheaper models cost $0.05–$0.10 per 1,000 identities and respond in about 0.15 seconds.

On a harder test with twins, juniors and seniors, and same-name people, Luna fell clearly behind the frontier models. Jev's confidence scores proved better calibrated: its mistakes mostly came with low confidence, while Luna often erred with high confidence.

Hybrid model

When Jev forwarded only uncertain cases (30–90% confidence, 22% of identities) to Opus, accuracy reached 98.8% at $0.88 per 1,000. On the harder test this setup scored 95.3% against Opus's 95.1%, at $2.94 versus $5.39. The Luna plus Opus combination reached 91.6% on the harder test.