The ARC-AGI-3 scoring methodology page appears to have an internal arithmetic inconsistency in the per-game aggregation example.
Docs page:
Current content:
- Levels 1-3 are described as
0.25 each
- Levels 4-7 are
0
- The formula line then shows
(0.25x1 + 0.25x2 + 0.5x3 + 0x4 + 0x5 + 0x6 + 0x7) / (1+2+3+4+5+6+7) = 0.01289 (1.29%)
Why this is inconsistent:
- The bullet list says the first three level scores are all
0.25, but the formula uses 0.5 for the third weighted term.
- The stated result
0.01289 matches neither interpretation.
Checked arithmetic:
- If levels 1-3 are all
0.25, then the weighted average is:
(0.25*1 + 0.25*2 + 0.25*3) / 28 = 1.5 / 28 = 0.0535714286 (about 5.36%)
- If the intended formula was
0.25, 0.25, 0.5, then:
(0.25*1 + 0.25*2 + 0.5*3) / 28 = 2.25 / 28 = 0.0803571429 (about 8.04%)
So the example result should be corrected, and either the bullet list or the formula line should be made consistent with the other.
This looks like a docs/example bug rather than a bug in the scoring rules themselves.
The ARC-AGI-3 scoring methodology page appears to have an internal arithmetic inconsistency in the per-game aggregation example.
Docs page:
Current content:
0.25each0(0.25x1 + 0.25x2 + 0.5x3 + 0x4 + 0x5 + 0x6 + 0x7) / (1+2+3+4+5+6+7) = 0.01289 (1.29%)Why this is inconsistent:
0.25, but the formula uses0.5for the third weighted term.0.01289matches neither interpretation.Checked arithmetic:
0.25, then the weighted average is:(0.25*1 + 0.25*2 + 0.25*3) / 28 = 1.5 / 28 = 0.0535714286(about5.36%)0.25, 0.25, 0.5, then:(0.25*1 + 0.25*2 + 0.5*3) / 28 = 2.25 / 28 = 0.0803571429(about8.04%)So the example result should be corrected, and either the bullet list or the formula line should be made consistent with the other.
This looks like a docs/example bug rather than a bug in the scoring rules themselves.