Quick Take: EdgeBench is a benchmark from ByteDance Seed, released July 6, 2026, that measures how AI agents learn from a real environment while they work — across 134 expert-built tasks that each run for at least 12 hours. Claude Opus 4.8 leads the full suite with a score of 51.3, ahead of GPT-5.5 at 48.4, GPT-5.4 at 39.3, GLM-5.1 at 37.4, and DeepSeek-V4-Pro at 31.0. The headline finding: agent performance follows a log-sigmoid scaling law with R-squared of 0.998, and a fit across model generations puts learning speed on track to roughly double every three months.
EdgeBench in five points
- A new axis for agents. EdgeBench scores day-scale learning — how much an agent improves over 12 hours of trying — instead of a single one-shot answer.
- Opus 4.8 on top. Claude Opus 4.8 leads the 134-task suite at 51.3, with GPT-5.5 second at 48.4 and a wide gap down to DeepSeek-V4-Pro at 31.0.
- A clean scaling law. Best-so-far performance follows a log-sigmoid curve against interaction time, fitting with R-squared of 0.998.
- Built by hand, at scale. 134 tasks across six domains, averaging 57.2 expert hours to build each one, and roughly 38,000 hours of agent interaction analyzed.
- A credibility signal. A Chinese lab published a leaderboard that puts a US model first and its own country's models lower — the opposite of a home-team bias.
What EdgeBench Is, and What It Actually Measures
EdgeBench is a benchmark that asks a different question from almost every leaderboard before it. Instead of "how good is this model's answer," it asks "how fast does this agent get better while it works." The distinction matters because 2026's frontier systems are no longer one-shot chatbots; they are autonomous agents that plan, run code, read the results, and try again — sometimes for hours. A benchmark that grades only the first attempt misses the entire loop that makes an agentic model useful.
According to the EdgeBench paper on arXiv, each task keeps an agent operating for at least 12 hours inside a live environment with "rich, multilevel feedback." The score is not a pass or fail on a single output; it is the best result the agent reaches as its interaction time grows. That design is what lets the authors measure a learning curve rather than a static capability. The code and an official leaderboard are published on GitHub under ByteDance-Seed.
The name is a tell. "Edge" refers to the frontier of what an agent can reach at the edge of its current ability, given time and feedback — the zone where learning, not recall, decides the outcome. It is a benchmark aimed squarely at the thing everyone now cares about: agents that improve on the job.
The Leaderboard: Claude Opus 4.8 Leads, GPT-5.5 Close Behind
On the complete 134-task suite scored at the 12-hour mark, Claude Opus 4.8 comes first with 51.3. GPT-5.5 is a genuine second at 48.4 — close enough that on many individual tasks the two trade places. Then the field drops off: GPT-5.4 at 39.3, GLM-5.1 at 37.4, and DeepSeek's V4 line (the V4-Pro variant) at 31.0. The gap between first and last is roughly 20 points on a 0-to-100 scale, which is large for a frontier-only field.
The EdgeBench dataset card on Hugging Face also reports a smaller 51-task public subset, and the ordering there is identical — a good sign that the ranking is not an artifact of the hidden tasks. Note the subset numbers run a few points below the full-suite numbers, because the public subset is a different (and slightly harder-scoring) slice, not a contradiction.
| Model | Full suite (134 tasks) | Public subset (51 tasks) |
|---|---|---|
| Claude Opus 4.8 | 51.3 | 44.2 |
| GPT-5.5 | 48.4 | 43.1 |
| GPT-5.4 | 39.3 | 34.2 |
| GLM-5.1 | 37.4 | 30.4 |
| DeepSeek-V4-Pro | 31.0 | 25.7 |
One caveat worth stating up front: the benchmark tests specific model versions. It scored GLM-5.1 and the V4-Pro build of DeepSeek, not the newest checkpoints either lab may have shipped since. Leaderboards are snapshots, and this one is a July 2026 snapshot.
The Log-Sigmoid Scaling Law — and the Three-Month Number
The most cited result in the paper is not the leaderboard; it is the shape of the learning curve. The authors find that an agent's best-so-far score rises with interaction time along a log-sigmoid path — slow at the start, a steep climb in the middle, then a plateau — and that this curve fits the data with R-squared of 0.998. That is an unusually tight fit, and it is the paper's central claim: learning from an environment is not chaotic, it is predictable. The full text on arXiv spells out the functional form and the conditions under which it holds.
The number that will travel furthest is the second one. Across model generations, a log-linear fit to the frontier models "corresponds to an approximate doubling every three months" in learning speed, per the abstract. Read carefully, that is a trend line through a handful of frontier data points — closer in spirit to a Moore's-law-style observation than to a proven physical constant. It rhymes with other 2026 scaling claims, like Huawei's Tau scaling law for chips, and like all of them it should be treated as a signal to watch, not a guarantee to bank on.
Still, the direction is the point. If agents are not just getting smarter but getting faster at learning, then the useful metric shifts from "what can this model do today" to "how quickly can it figure out something new." EdgeBench is the first widely shared benchmark built to measure exactly that.
Six Domains, 134 Tasks, 38,000 Hours of Agent Time
What separates EdgeBench from a quick eval is the sheer cost of building it. The 134 tasks span six domains, and the task counts are uneven by design: Scientific Problems and ML leads with 39 tasks, Systems and Software Engineering has 36, Combinatorial Optimization and Professional Knowledge Work have 19 each, Formal Math and Theorem Proving has 13, and Interactive Games and Simulators has 8. Those add up cleanly to 134.
Each task took an average of 57.2 recorded human hours to construct, and every task is engineered to keep an agent busy for at least 12 hours. Multiply that out across models and runs and the study analyzes roughly 38,000 hours of agent interaction — the raw material that makes an R-squared of 0.998 credible rather than lucky. The task data and domain breakdown are documented on the Hugging Face dataset page, with 51 of the 134 tasks released publicly and the rest held back as a hidden set. The paper's methodology section details how the environments deliver continuous feedback rather than a single end-of-run grade.
Holding back most of the tasks is a deliberate anti-gaming move. A public benchmark that ships all its answers eventually leaks into training data and stops measuring anything; keeping 83 tasks hidden is how EdgeBench tries to stay honest past its first month.
Why a Chinese Lab Ranking a US Model First Is the Real Story
Benchmarks published by a model's own maker always carry a discount for self-interest. EdgeBench does not have that problem. It comes from ByteDance Seed, a Chinese research lab, and it puts Anthropic's Claude Opus 4.8 — a US model — at the top, while placing the Chinese-built GLM-5.1 and DeepSeek-V4-Pro in the bottom half. When the scorekeeper's home team finishes fourth and fifth, the leaderboard reads as a measurement rather than a marketing asset.
That neutrality is worth more than any single score. ByteDance has been shipping its own frontier models aggressively — its multimodal blitz earlier in 2026 made that plain — so a benchmark from the same house that still ranks a rival first is a stronger endorsement of Opus 4.8 than Anthropic could credibly make itself. The open leaderboard on GitHub lets anyone re-run the public subset and check, which is the other half of what makes a result trustworthy.
It also complicates the lazy narrative that Chinese and US labs are running separate, incomparable races. Here is a Chinese lab building the measuring stick, and a US model winning on it, with the code in the open. The competition is real, but so is the shared yardstick.
What EdgeBench Does and Doesn't Tell Us
EdgeBench is a genuine step forward, but it is not the last word, and the authors are refreshingly clear about that. It deliberately excludes tasks whose difficulty is mainly visual — GUI operation, for instance — because, as the paper notes, when success hinges on the vision backbone it becomes impossible to separate learning ability from raw perception. So a strong EdgeBench score says little about an agent driving a screen.
The scaling law comes with its own fine print: the authors state it "is not expected to hold for every environment-learning process" and can break when task structures contain strong bottlenecks or uneven difficulty. And the domain sizes are small — with just 8 tasks in Games and 13 in Formal Math, per-domain rankings are noisy and should not be over-read. Tasks built by experts are also, inevitably, shaped by those experts' choices; another team might have drawn the frontier differently. None of this is disqualifying, but all of it argues for reading EdgeBench the way you should read any leaderboard: as one lens, not the whole view. The 51 public tasks are open if you want to probe the scoring yourself, and that habit of checking pairs well with the caution we urged during the VibeThinker benchmark fight.
How to Read This Result
The safest way to use EdgeBench is alongside the benchmarks it does not replace. A one-shot test like SWE-bench Verified tells you how capable a model is at a single fix; EdgeBench tells you how fast it improves over hours of iteration. A model can look great on one and ordinary on the other. Together they describe an agent's raw skill and its learning slope — two properties that used to be conflated and now, finally, have separate numbers.
For anyone choosing a model to run agents in production, the takeaway is narrow but useful: on ByteDance's day-scale test, Claude Opus 4.8 currently learns fastest, GPT-5.5 is within striking distance, and the rest of the field trails. That is a July 2026 reading, and if the paper's own three-month-doubling trend holds, the next snapshot will look different.
Frequently Asked Questions
What is EdgeBench?
EdgeBench is a benchmark from ByteDance Seed that measures how well AI agents learn from a real environment while they work, rather than how well they answer a single prompt. It uses 134 expert-built, real-world tasks across six domains, each of which keeps an agent running for at least 12 hours under continuous feedback. The paper analyzes roughly 38,000 hours of agent interaction and was released on July 6, 2026.
Who created EdgeBench and when was it released?
EdgeBench was built by ByteDance Seed, the AI research group inside ByteDance, and posted to arXiv on July 6, 2026 as paper 2607.05155, led by Deyao Zhu and 46 co-authors. The code and an official leaderboard live on GitHub under ByteDance-Seed, and the task data is published on Hugging Face.
What does EdgeBench measure that other benchmarks do not?
Most benchmarks score a single attempt: the model reads a problem and answers once. EdgeBench instead measures day-scale learning — how much an agent improves over many hours of trying, testing, failing, and adjusting inside one environment. It tracks the best-so-far score as a function of interaction time, so it rewards agents that get better through iteration, not just agents that are strong on the first try.
Which model tops the EdgeBench leaderboard?
Claude Opus 4.8 leads the full 134-task suite with a score of 51.3 at the 12-hour mark, ahead of GPT-5.5 at 48.4, GPT-5.4 at 39.3, GLM-5.1 at 37.4, and DeepSeek-V4-Pro at 31.0. On the smaller 51-task public subset the ranking is identical: Opus 4.8 at 44.2, GPT-5.5 at 43.1, GPT-5.4 at 34.2, GLM-5.1 at 30.4, and DeepSeek-V4-Pro at 25.7.
What is the log-sigmoid scaling law in EdgeBench?
The paper reports that an agent best-so-far performance rises with interaction time along a log-sigmoid curve — flat at first, then a steep climb, then a plateau — and that this shape fits the aggregate data with R-squared of 0.998. In plain terms, learning from an environment is predictable: give an agent more time and its score follows a smooth, S-shaped path rather than jumping around randomly.
Does agent learning speed really double every three months?
The paper reports that, across model generations, a log-linear fit to the frontier models corresponds to agent learning speed roughly doubling every three months. That is a trend line drawn through a small number of frontier data points, not a guaranteed law of nature, and the authors themselves note the scaling curve can break down in certain environments. It is a striking headline number that deserves to be read as an early signal, not a promise.
How many tasks and domains does EdgeBench cover?
EdgeBench has 134 tasks spread across six domains: Scientific Problems and ML (39 tasks), Systems and Software Engineering (36), Combinatorial Optimization (19), Professional Knowledge Work (19), Formal Math and Theorem Proving (13), and Interactive Games and Simulators (8). The counts add up to the full 134, and 51 of those tasks are released publicly.
How were EdgeBench tasks built and how long do they run?
Each task was constructed through substantial expert effort — an average of 57.2 recorded human hours per task — and is designed to sustain at least 12 hours of continuous agent operation with rich, multilevel feedback. Summed across the whole suite, the study analyzes roughly 38,000 hours of agent interaction, which is what makes the scaling-law fit statistically meaningful.
Why does it matter that a Chinese lab ranked a US model first?
EdgeBench comes from ByteDance Seed, a Chinese lab, yet it places Anthropic Claude Opus 4.8 — a US model — at the top, ahead of GPT-5.5 and well ahead of the Chinese-built GLM-5.1 and DeepSeek-V4-Pro. When a benchmark ranks a competitor model above the home team, that removes an obvious source of bias and makes the leaderboard more credible than a vendor publishing scores for its own model.
Is EdgeBench open source and can I access it?
Partly. Of the 134 tasks, 51 are released as a public subset, published on Hugging Face under ByteDance-Seed, with evaluation code and the leaderboard on GitHub. The remaining 83 tasks are held back, which is a common design choice that keeps a hidden test set from leaking into future training data and inflating scores.
What are EdgeBench limitations?
EdgeBench deliberately excludes tasks whose main difficulty is visual understanding or GUI operation, because there success depends on the vision backbone rather than on iterative reasoning. The authors also warn that the log-sigmoid law is not expected to hold everywhere and can fail when task structures contain strong bottlenecks or uneven difficulty. And with as few as 8 tasks in the smallest domain, single-domain results should be read with caution.
How does EdgeBench compare to SWE-bench or one-shot benchmarks?
They measure different things. SWE-bench Verified and similar tests score a single fix or a single answer, so they capture raw capability at one moment. EdgeBench measures the slope of improvement over 12 hours of interaction, so a model can be strong on a one-shot benchmark yet mediocre at learning, or vice versa. The two are complementary, and reading either one in isolation gives an incomplete picture of an agent.
Sources
- ByteDance Seed, "EdgeBench: Unveiling Scaling Laws of Learning from Real-World Environments," arXiv 2607.05155 (submitted July 6, 2026) — full HTML.
- EdgeBench code and official leaderboard, GitHub: ByteDance-Seed/EdgeBench.
- EdgeBench task data (51-task public subset), Hugging Face: ByteDance-Seed/EdgeBench.
- ByteDance Seed research lab, seed.bytedance.com.
