Research

Every environment we ship is checked by people who do the work it is about: clinicians for health tasks, engineers and operators for industrial ones, scientists and site reliability engineers for the lab and the data centre. Their decisions are already in the records we build from, their rules set the thresholds, they review how we grade, and every lesson from their reviews goes into the next environment we build.

Static benchmarks wait for all the data before grading an agent. For agents that act while events unfold, that measures the wrong skill, and training on it teaches models to wait.

Version 7 of our leaderboard puts 15 models on industrial and scientific systems and 14 on personal agents. No model earns half of the possible reward in any family, and different labs lead different families.

Real operating records have gaps, drift, late corrections and sources that disagree. Synthetic data rarely has them. An environment built from real records teaches the work as it really is.

Each of our environments has easier levels and a hard level. The best models clear the easier levels and still fail the hard one. That gradient makes an environment useful for training, not only for evaluation.

Frontier models handle simple questions about time series well. Long histories, decision cutoffs and questions with a trap in them still break them. For frontier evals of LLM reasoning over time series data, reach out to us.