primer — Evaluation #1
No measurable effect
0.0 pp
pp = percentage pointsWe ran the measurement. The delta (0.0 pp) is real but can't yet be distinguished from natural variance at this task count. You're protected from acting on noise.
n=5 tasks · 3 runs/config · σ ±0.000 pp
open-bridgeegress opensha256 …@sha256:a0685355n=5×3σ ±0.0 ppGemini (experimental)/gemini-2.5-flash
Metrics
Without0.0%success rate
With0.0%success rate
Variance±0.0 ppmin 0.0% / max 0.0%
Tasks53 runs/config
Cost without≈ $0.0000
Cost with≈ $0.0000
Cost deltaN/Aestimated — provider cost approximate
PRIMER overhead$0.0001
Generation cost (Layer-1 provider). Never included in eval cost delta.
Per-task flips
Fixed by the file 0Broken by the file 0Passed with and without 0Failed with and without 5
- stub_57fad84178stub_functionFailed with and without0.0% / 0.0%0.0 pp
- stub_b9274fb078stub_functionFailed with and without0.0% / 0.0%0.0 pp
- stub_e9b7aa08f6stub_functionFailed with and without0.0% / 0.0%0.0 pp
- stub_cd25b2a9a2stub_functionFailed with and without0.0% / 0.0%0.0 pp
- stub_a9e3859795stub_functionFailed with and without0.0% / 0.0%0.0 pp
| Task | Flip | Without / With | Delta | Flaky | Details |
|---|---|---|---|---|---|
stub_57fad84178stub_function | Failed with and without | 0.0% / 0.0% | 0.0 pp | ||
stub_b9274fb078stub_function | Failed with and without | 0.0% / 0.0% | 0.0 pp | ||
stub_e9b7aa08f6stub_function | Failed with and without | 0.0% / 0.0% | 0.0 pp | ||
stub_cd25b2a9a2stub_function | Failed with and without | 0.0% / 0.0% | 0.0 pp | ||
stub_a9e3859795stub_function | Failed with and without | 0.0% / 0.0% | 0.0 pp |