Quiz answer-position bias still present on main (follow-up to #239)

[Johnny-my-boi]

Quiz answer-position bias still present on main (follow-up to #239)

#239 reported that the correct answer is almost always option B, but it was closed as completed. The fix is not in main yet, so I'm opening a fresh issue as suggested.

Evidence

I scanned every quiz.json on the current main (commit 86bbb61):

308 quiz files, 1846 questions total
  A (0):  265  (14%)
  B (1): 1108  (60%)   <-- correct answer
  C (2):  256  (14%)
  D (3):  217  (12%)

A uniform distribution would put each option near 25%. Some phases were authored correctly (05-nlp, 17-infra, 18-ethics are well balanced), but many are almost entirely B:

• Phase 0: 54/60
• Phase 1: 100/110
• Phase 3: 54/65
• Phase 10: 52/55

This makes the answer trivially guessable and defeats the self-evaluation purpose of the quizzes.

Reproduction

import json, glob
from collections import Counter
c = Counter()
for f in glob.glob('phases/**/quiz.json', recursive=True):
    d = json.load(open(f))
    qs = d if isinstance(d, list) else d.get('questions', [])
    for q in qs:
        if isinstance(q, dict) and 'correct' in q:
            c[q['correct']] += 1
print(sorted(c.items()))

Proposed fix

I'm happy to send a PR that shuffles each question's options (updating the correct index accordingly) and rebuilds site/data.js. Let me know if that approach works.