Agent Performance
Pass rates on EvoAgentBench · 4 domains · 2 scaffolds · 3 backbones
Mean scores from Table 3 of the paper; standard errors are omitted.
79.8%
Best Method Score
+3.3
Mean Transfer Gain
96
Method Comparisons
Filter · Agent
Filter · Domain
Filter · Method
Sort by
| # | Agent | Base Model | Domain | Method | Vanilla | Method Score | Δ | |
|---|---|---|---|---|---|---|---|---|
| 1 | Nanobot | Qwen3.5-397B | Software Engineering | Anchor Skill†| 62.5% | 79.8% | +17.3 | |
| 2 | OpenClaw | Qwen3.5-397B | Software Engineering | Anchor Skill†| 66.7% | 76.8% | +10.1 | |
| 3 | Nanobot | Gemma-4-31B | Algorithmic Reasoning | Anchor Skill†| 69.0% | 73.3% | +4.3 | |
| 4 | OpenClaw | Gemma-4-31B | Algorithmic Reasoning | Anchor Skill†| 72.1% | 72.5% | +0.4 | |
| 5 | Nanobot | Gemma-4-31B | Algorithmic Reasoning | ReasoningBank | 69.0% | 72.5% | +3.5 | |
| 6 | Nanobot | Gemma-4-31B | Algorithmic Reasoning | Memento | 69.0% | 72.4% | +3.4 | |
| 7 | Nanobot | Qwen3.5-397B | Software Engineering | GEPA | 62.5% | 71.4% | +8.9 | |
| 8 | Nanobot | Gemma-4-31B | Algorithmic Reasoning | GEPA | 69.0% | 70.2% | +1.2 | |
| 9 | Nanobot | Qwen3.5-397B | Algorithmic Reasoning | Anchor Skill†| 57.8% | 69.8% | +12.0 | |
| 10 | OpenClaw | Gemma-4-31B | Algorithmic Reasoning | GEPA | 72.1% | 69.8% | -2.3 | |
| 11 | OpenClaw | Qwen3.5-397B | Software Engineering | GEPA | 66.7% | 67.9% | +1.2 | |
| 12 | OpenClaw | Gemma-4-31B | Algorithmic Reasoning | Memento | 72.1% | 67.8% | -4.3 | |
| 13 | Nanobot | Qwen3.5-397B | Algorithmic Reasoning | GEPA | 57.8% | 65.1% | +7.3 | |
| 14 | Nanobot | Qwen3.5-397B | Software Engineering | Memento | 62.5% | 64.9% | +2.4 | |
| 15 | Nanobot | Qwen3.5-397B | Algorithmic Reasoning | ReasoningBank | 57.8% | 64.3% | +6.5 | |
| 16 | OpenClaw | Qwen3.5-397B | Algorithmic Reasoning | Anchor Skill†| 52.7% | 64.0% | +11.3 | |
| 17 | OpenClaw | Qwen3.5-397B | Software Engineering | Memento | 66.7% | 63.7% | -3.0 | |
| 18 | Nanobot | Qwen3.5-397B | Algorithmic Reasoning | Memento | 57.8% | 63.5% | +5.7 | |
| 19 | Nanobot | Qwen3.5-27B | Algorithmic Reasoning | Anchor Skill†| 52.7% | 63.2% | +10.5 | |
| 20 | Nanobot | Qwen3.5-397B | Software Engineering | ReasoningBank | 62.5% | 63.1% | +0.6 | |
| 21 | Nanobot | Qwen3.5-27B | Algorithmic Reasoning | ReasoningBank | 52.7% | 62.4% | +9.7 | |
| 22 | Nanobot | Qwen3.5-397B | Knowledge Work | Anchor Skill†| 53.4% | 62.0% | +8.6 | |
| 23 | OpenClaw | Qwen3.5-397B | Algorithmic Reasoning | GEPA | 52.7% | 61.6% | +8.9 | |
| 24 | OpenClaw | Qwen3.5-397B | Software Engineering | ReasoningBank | 66.7% | 61.3% | -5.4 | |
| 25 | OpenClaw | Gemma-4-31B | Algorithmic Reasoning | ReasoningBank | 72.1% | 61.2% | -10.9 | |
| 26 | OpenClaw | Qwen3.5-397B | Algorithmic Reasoning | Memento | 52.7% | 60.9% | +8.2 | |
| 27 | OpenClaw | Qwen3.5-27B | Algorithmic Reasoning | Anchor Skill†| 51.9% | 60.5% | +8.6 | |
| 28 | Nanobot | Qwen3.5-27B | Algorithmic Reasoning | GEPA | 52.7% | 60.5% | +7.8 | |
| 29 | OpenClaw | Qwen3.5-397B | Knowledge Work | Anchor Skill†| 55.8% | 59.6% | +3.8 | |
| 30 | OpenClaw | Qwen3.5-397B | Algorithmic Reasoning | ReasoningBank | 52.7% | 59.3% | +6.6 | |
| 31 | Nanobot | Qwen3.5-27B | Knowledge Work | Anchor Skill†| 43.9% | 57.7% | +13.8 | |
| 32 | OpenClaw | Qwen3.5-27B | Algorithmic Reasoning | ReasoningBank | 51.9% | 57.4% | +5.5 | |
| 33 | OpenClaw | Qwen3.5-397B | Knowledge Work | ReasoningBank | 55.8% | 57.3% | +1.5 | |
| 34 | Nanobot | Qwen3.5-27B | Algorithmic Reasoning | Memento | 52.7% | 57.0% | +4.3 | |
| 35 | OpenClaw | Qwen3.5-397B | Knowledge Work | GEPA | 55.8% | 56.5% | +0.7 | |
| 36 | OpenClaw | Gemma-4-31B | Knowledge Work | Anchor Skill†| 44.1% | 54.8% | +10.7 | |
| 37 | OpenClaw | Qwen3.5-27B | Algorithmic Reasoning | Memento | 51.9% | 53.9% | +2.0 | |
| 38 | OpenClaw | Qwen3.5-27B | Algorithmic Reasoning | GEPA | 51.9% | 53.9% | +2.0 | |
| 39 | Nanobot | Qwen3.5-27B | Software Engineering | ReasoningBank | 45.8% | 53.0% | +7.2 | |
| 40 | Nanobot | Qwen3.5-397B | Knowledge Work | ReasoningBank | 53.4% | 52.5% | -0.9 | |
| 41 | OpenClaw | Qwen3.5-27B | Software Engineering | Anchor Skill†| 42.3% | 52.4% | +10.1 | |
| 42 | Nanobot | Qwen3.5-397B | Knowledge Work | Memento | 53.4% | 52.2% | -1.2 | |
| 43 | Nanobot | Qwen3.5-397B | Knowledge Work | GEPA | 53.4% | 52.2% | -1.2 | |
| 44 | OpenClaw | Qwen3.5-397B | Knowledge Work | Memento | 55.8% | 50.0% | -5.8 | |
| 45 | Nanobot | Qwen3.5-27B | Software Engineering | Anchor Skill†| 45.8% | 49.4% | +3.6 | |
| 46 | Nanobot | Gemma-4-31B | Knowledge Work | Anchor Skill†| 43.2% | 49.3% | +6.1 | |
| 47 | Nanobot | Qwen3.5-27B | Knowledge Work | ReasoningBank | 43.9% | 49.2% | +5.3 | |
| 48 | Nanobot | Qwen3.5-27B | Software Engineering | GEPA | 45.8% | 48.8% | +3.0 | |
| 49 | OpenClaw | Qwen3.5-27B | Knowledge Work | Anchor Skill†| 43.6% | 48.6% | +5.0 | |
| 50 | OpenClaw | Qwen3.5-27B | Knowledge Work | ReasoningBank | 43.6% | 48.2% | +4.6 | |
| 51 | Nanobot | Gemma-4-31B | Knowledge Work | GEPA | 43.2% | 47.9% | +4.7 | |
| 52 | Nanobot | Qwen3.5-27B | Knowledge Work | Memento | 43.9% | 47.6% | +3.7 | |
| 53 | Nanobot | Qwen3.5-27B | Knowledge Work | GEPA | 43.9% | 46.6% | +2.7 | |
| 54 | Nanobot | Gemma-4-31B | Knowledge Work | ReasoningBank | 43.2% | 46.5% | +3.3 | |
| 55 | Nanobot | Gemma-4-31B | Knowledge Work | Memento | 43.2% | 46.2% | +3.0 | |
| 56 | OpenClaw | Qwen3.5-397B | Web Research | Anchor Skill†| 31.8% | 45.3% | +13.5 | |
| 57 | OpenClaw | Qwen3.5-27B | Software Engineering | Memento | 42.3% | 45.2% | +2.9 | |
| 58 | OpenClaw | Gemma-4-31B | Knowledge Work | GEPA | 44.1% | 44.8% | +0.7 | |
| 59 | OpenClaw | Qwen3.5-27B | Knowledge Work | Memento | 43.6% | 44.3% | +0.7 | |
| 60 | OpenClaw | Gemma-4-31B | Knowledge Work | ReasoningBank | 44.1% | 42.7% | -1.4 | |
| 61 | OpenClaw | Gemma-4-31B | Software Engineering | GEPA | 25.0% | 42.3% | +17.3 | |
| 62 | OpenClaw | Qwen3.5-27B | Software Engineering | GEPA | 42.3% | 41.7% | -0.6 | |
| 63 | OpenClaw | Qwen3.5-397B | Web Research | ReasoningBank | 31.8% | 39.0% | +7.2 | |
| 64 | Nanobot | Gemma-4-31B | Software Engineering | GEPA | 17.3% | 37.5% | +20.2 | |
| 65 | OpenClaw | Qwen3.5-27B | Software Engineering | ReasoningBank | 42.3% | 36.9% | -5.4 | |
| 66 | OpenClaw | Qwen3.5-397B | Web Research | Memento | 31.8% | 33.8% | +2.0 | |
| 67 | OpenClaw | Gemma-4-31B | Knowledge Work | Memento | 44.1% | 33.2% | -10.9 | |
| 68 | OpenClaw | Gemma-4-31B | Software Engineering | Anchor Skill†| 25.0% | 32.7% | +7.7 | |
| 69 | OpenClaw | Qwen3.5-397B | Web Research | GEPA | 31.8% | 32.3% | +0.5 | |
| 70 | OpenClaw | Gemma-4-31B | Software Engineering | ReasoningBank | 25.0% | 31.5% | +6.5 | |
| 71 | OpenClaw | Qwen3.5-27B | Knowledge Work | GEPA | 43.6% | 31.3% | -12.3 | |
| 72 | Nanobot | Gemma-4-31B | Software Engineering | Anchor Skill†| 17.3% | 26.8% | +9.5 | |
| 73 | Nanobot | Gemma-4-31B | Software Engineering | ReasoningBank | 17.3% | 24.4% | +7.1 | |
| 74 | Nanobot | Gemma-4-31B | Software Engineering | Memento | 17.3% | 21.4% | +4.1 | |
| 75 | OpenClaw | Gemma-4-31B | Software Engineering | Memento | 25.0% | 20.2% | -4.8 | |
| 76 | Nanobot | Qwen3.5-397B | Web Research | Anchor Skill†| 12.8% | 20.0% | +7.2 | |
| 77 | OpenClaw | Gemma-4-31B | Web Research | Anchor Skill†| 13.3% | 17.9% | +4.6 | |
| 78 | OpenClaw | Gemma-4-31B | Web Research | GEPA | 13.3% | 17.4% | +4.1 | |
| 79 | Nanobot | Qwen3.5-397B | Web Research | Memento | 12.8% | 16.9% | +4.1 | |
| 80 | OpenClaw | Gemma-4-31B | Web Research | Memento | 13.3% | 16.4% | +3.1 | |
| 81 | Nanobot | Qwen3.5-397B | Web Research | ReasoningBank | 12.8% | 15.9% | +3.1 | |
| 82 | OpenClaw | Qwen3.5-27B | Web Research | GEPA | 10.8% | 14.9% | +4.1 | |
| 83 | OpenClaw | Qwen3.5-27B | Web Research | Anchor Skill†| 10.8% | 14.9% | +4.1 | |
| 84 | Nanobot | Qwen3.5-397B | Web Research | GEPA | 12.8% | 14.9% | +2.1 | |
| 85 | Nanobot | Qwen3.5-27B | Web Research | Anchor Skill†| 8.7% | 13.3% | +4.6 | |
| 86 | OpenClaw | Gemma-4-31B | Web Research | ReasoningBank | 13.3% | 13.3% | +0.0 | |
| 87 | OpenClaw | Qwen3.5-27B | Web Research | ReasoningBank | 10.8% | 12.8% | +2.0 | |
| 88 | Nanobot | Qwen3.5-27B | Web Research | Memento | 8.7% | 12.3% | +3.6 | |
| 89 | Nanobot | Gemma-4-31B | Web Research | Anchor Skill†| 9.2% | 12.3% | +3.1 | |
| 90 | Nanobot | Qwen3.5-27B | Web Research | GEPA | 8.7% | 11.8% | +3.1 | |
| 91 | OpenClaw | Qwen3.5-27B | Web Research | Memento | 10.8% | 11.3% | +0.5 | |
| 92 | Nanobot | Gemma-4-31B | Web Research | Memento | 9.2% | 10.3% | +1.1 | |
| 93 | Nanobot | Gemma-4-31B | Web Research | GEPA | 9.2% | 9.7% | +0.5 | |
| 94 | Nanobot | Qwen3.5-27B | Software Engineering | Memento | 45.8% | 9.5% | -36.3 | |
| 95 | Nanobot | Qwen3.5-27B | Web Research | ReasoningBank | 8.7% | 9.2% | +0.5 | |
| 96 | Nanobot | Gemma-4-31B | Web Research | ReasoningBank | 9.2% | 4.6% | -4.6 |
Means from Table 3 of the paper; standard errors are intentionally omitted. Overall is derived by equally weighting four domains and both scaffolds. †Anchor Skill is a diagnostic reference, not a deployable automatic method.
Turn Cost Analysis
Percentage change in agent turns relative to Vanilla, equal-weighted across the four domains. Negative values mean fewer turns; positive values mean additional overhead.
| Agent | Base Model | Method | All | Solved | Unsolved |
|---|---|---|---|---|---|
| OpenClaw | Qwen3.5-27B | Memento | +40.7% | +39.4% | +23.5% |
| OpenClaw | Qwen3.5-27B | ReasoningBank | -6.3% | -5.7% | -16.3% |
| OpenClaw | Qwen3.5-27B | GEPA | -4.8% | +4.4% | -18.2% |
| OpenClaw | Qwen3.5-27B | Anchor Skill†| -3.4% | -13.6% | -2.8% |
| Nanobot | Qwen3.5-27B | Memento | -2.1% | +4.8% | -0.9% |
| Nanobot | Qwen3.5-27B | ReasoningBank | +3.6% | +2.4% | +7.9% |
| Nanobot | Qwen3.5-27B | GEPA | -7.2% | +8.0% | -13.0% |
| Nanobot | Qwen3.5-27B | Anchor Skill†| -8.8% | -10.6% | -9.8% |
| OpenClaw | Qwen3.5-397B | Memento | -3.5% | +4.3% | -18.8% |
| OpenClaw | Qwen3.5-397B | ReasoningBank | -10.3% | -2.1% | -26.1% |
| OpenClaw | Qwen3.5-397B | GEPA | +25.4% | +27.6% | +13.1% |
| OpenClaw | Qwen3.5-397B | Anchor Skill†| -8.0% | -5.4% | -11.3% |
| Nanobot | Qwen3.5-397B | Memento | -10.8% | -9.3% | -20.8% |
| Nanobot | Qwen3.5-397B | ReasoningBank | -0.4% | +1.0% | +4.7% |
| Nanobot | Qwen3.5-397B | GEPA | +13.7% | +24.4% | -8.2% |
| Nanobot | Qwen3.5-397B | Anchor Skill†| -4.0% | +1.4% | +8.3% |
| OpenClaw | Gemma-4-31B | Memento | +13.6% | +38.8% | +10.4% |
| OpenClaw | Gemma-4-31B | ReasoningBank | +34.8% | +31.6% | +42.8% |
| OpenClaw | Gemma-4-31B | GEPA | +44.6% | +40.3% | +52.7% |
| OpenClaw | Gemma-4-31B | Anchor Skill†| +7.6% | +16.5% | +26.9% |
| Nanobot | Gemma-4-31B | Memento | +43.9% | +12.8% | -16.0% |
| Nanobot | Gemma-4-31B | ReasoningBank | +37.6% | +30.4% | +42.4% |
| Nanobot | Gemma-4-31B | GEPA | +41.5% | +9.3% | -8.3% |
| Nanobot | Gemma-4-31B | Anchor Skill†| +32.7% | +11.5% | -6.4% |
Values reproduce Table 4 of the paper. All, Solved, and Unsolved are separately averaged across domains, so All need not lie between the other two columns. †Diagnostic reference.