Skip to content

[codex] clarify KBR ChatGPT import instructions #291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
DickHorner merged 3 commits into from
Jun 7, 2026
Merged

[codex] clarify KBR ChatGPT import instructions #291

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
efec383
clarify kbr chatgpt import instructions
DickHorner May 27, 2026
03092e0
address kbr prompt review feedback
DickHorner May 27, 2026
1ee362a
Merge branch 'main' into codex/kbr-chatgpt-json-instructions
DickHorner Jun 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions modules/exams/rule-packs/default/contract.template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@

## Matching Rule

- The user may upload multiple Leistung files in the same message.
- Treat every uploaded file as one separate Leistung unless the user explicitly says that multiple files belong together.
- Process each Leistung independently and keep its correction data isolated.
- The user does not need to provide a Leistung `chatRef`.
- Resolve the matching Leistung `chatRef` from the submitted Leistung and the candidate data listed under `Chat References`.
- Use visible candidate information from the submitted Leistung for this matching step.
Expand Down
58 changes: 56 additions & 2 deletions modules/exams/rule-packs/default/prompt.template.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
You are assisting with a correction session based on a structured contract.

Initial response:
- `Bereit. Bitte laden Sie die erste Leistung hoch.`
- `Bereit. Bitte laden Sie die erste Leistung oder mehrere Leistungen gleichzeitig hoch.`

Session workflow (generic and strict):
- process exactly one Leistung at a time
- the user may upload multiple Leistung files in the same message
- treat every uploaded file as one separate Leistung unless the user explicitly says that multiple files belong together
- process one Leistung at a time internally, even when multiple files were uploaded together
- keep each Leistung isolated; do not mix data between Leistungen
- use the submitted Leistung only to resolve the matching candidate and Leistung `chatRef`
- do not require the user to provide a Leistung `chatRef`
Expand Down Expand Up @@ -56,6 +58,58 @@ Required import bundle fields:
- include `importedTaskScores`
- include optional fields such as `rulePack`, `evidence`, or `metadata` only when supported by the loaded contract, rules, and import bundle schema

JSON structure to output for `Zwischenexport`:
{
"contract": <contract_json_object>,
"chatRef": "chat-0001",
"importedTaskScores": [
{
"taskId": "task-1",
"points": 0,
"maxPoints": 0,
"scoringUnitId": "task-1.score",
"comment": "Kurzbegründung mit Bezug zur Leistung",
"confidence": 0.8,
"evidenceIds": ["evidence-1"]
}
],
"evidence": [
{
"id": "evidence-1",
"kind": "quote",
"value": "Kurzer Beleg aus der Leistung"
}
],
"metadata": {
"generalComment": "Kurzer Gesamtkommentar"
}
}

JSON structure to output for `Ende Korrektur`:
[
{
"contract": <contract_json_object>,
"chatRef": "chat-0001",
"importedTaskScores": []
},
{
"contract": <contract_json_object>,
"chatRef": "chat-0002",
"importedTaskScores": []
}
]

Formatting notes for the JSON structures:
- replace `<contract_json_object>` with the full JSON object provided in the Contract JSON section below
- replace example `chatRef`, `taskId`, `scoringUnitId`, points, comments, evidence, and metadata with the resolved values
- use only task IDs and scoring-unit IDs that exist in the loaded contract
- omit optional fields when they are empty or unsupported
- for `Ende Korrektur`, output only the array and include one object per resolved Leistung

## Contract JSON

{{contractJson}}

## Contract

{{contractMarkdown}}
Comment on lines +61 to 115
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot May 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Inlining the entire contract JSON multiple times inside the prompt's example structures (once in Zwischenexport and twice in Ende Korrektur) will cause massive token bloat and potentially exceed the LLM's context window limits for larger exams.

Instead, we can define a single Contract JSON section at the bottom of the prompt to hold the full contract JSON, and use a placeholder like <contract_json_object> in the examples. This reduces the prompt size significantly while keeping the instructions clear.

JSON structure to output for Zwischenexport:
{
  "contract": <contract_json_object>,
  "chatRef": "chat-0001",
  "importedTaskScores": [
    {
      "taskId": "task-1",
      "points": 0,
      "maxPoints": 0,
      "scoringUnitId": "task-1.score",
      "comment": "Kurzbegruendung mit Bezug zur Leistung",
      "confidence": 0.8,
      "evidenceIds": ["evidence-1"]
    }
  ],
  "evidence": [
    {
      "id": "evidence-1",
      "kind": "quote",
      "value": "Kurzer Beleg aus der Leistung"
    }
  ],
  "metadata": {
    "generalComment": "Kurzer Gesamtkommentar"
  }
}

JSON structure to output for Ende Korrektur:
[
  {
    "contract": <contract_json_object>,
    "chatRef": "chat-0001",
    "importedTaskScores": []
  },
  {
    "contract": <contract_json_object>,
    "chatRef": "chat-0002",
    "importedTaskScores": []
  }
]

Formatting notes for the JSON structures:
- replace <contract_json_object> with the full JSON object provided in the Contract JSON section below
- replace example chatRef, taskId, scoringUnitId, points, comments, evidence, and metadata with the resolved values
- use only task IDs and scoring-unit IDs that exist in the loaded contract
- omit optional fields when they are empty or unsupported
- for Ende Korrektur, output only the array and include one object per resolved Leistung

## Contract JSON

    {{contractJson}}

## Contract

{{contractMarkdown}}

Expand Down
61 changes: 59 additions & 2 deletions modules/exams/src/rule-packs/default-pack.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,9 @@ const DEFAULT_CONTRACT_TEMPLATE = `# Correction Session Contract

## Matching Rule

- The user may upload multiple Leistung files in the same message.
- Treat every uploaded file as one separate Leistung unless the user explicitly says that multiple files belong together.
- Process each Leistung independently and keep its correction data isolated.
- The user does not need to provide a Leistung \`chatRef\`.
- Resolve the matching Leistung \`chatRef\` from the submitted Leistung and the candidate data listed under \`Chat References\`.
- Use visible candidate information from the submitted Leistung for this matching step.
Expand Down Expand Up @@ -125,10 +128,12 @@ const DEFAULT_CONTRACT_TEMPLATE = `# Correction Session Contract
const DEFAULT_PROMPT_TEMPLATE = `You are assisting with a correction session based on a structured contract.

Initial response:
- \`Bereit. Bitte laden Sie die erste Leistung hoch.\`
- \`Bereit. Bitte laden Sie die erste Leistung oder mehrere Leistungen gleichzeitig hoch.\`

Session workflow (generic and strict):
- process exactly one Leistung at a time
- the user may upload multiple Leistung files in the same message
- treat every uploaded file as one separate Leistung unless the user explicitly says that multiple files belong together
- process one Leistung at a time internally, even when multiple files were uploaded together
- keep each Leistung isolated; do not mix data between Leistungen
- use the submitted Leistung only to resolve the matching candidate and Leistung \`chatRef\`
- do not require the user to provide a Leistung \`chatRef\`
Expand Down Expand Up @@ -180,6 +185,58 @@ Required import bundle fields:
- include \`importedTaskScores\`
- include optional fields such as \`rulePack\`, \`evidence\`, or \`metadata\` only when supported by the loaded contract, rules, and import bundle schema

JSON structure to output for \`Zwischenexport\`:
{
"contract": <contract_json_object>,
"chatRef": "chat-0001",
"importedTaskScores": [
{
"taskId": "task-1",
"points": 0,
"maxPoints": 0,
"scoringUnitId": "task-1.score",
"comment": "Kurzbegründung mit Bezug zur Leistung",
"confidence": 0.8,
"evidenceIds": ["evidence-1"]
}
],
"evidence": [
{
"id": "evidence-1",
"kind": "quote",
"value": "Kurzer Beleg aus der Leistung"
}
],
"metadata": {
"generalComment": "Kurzer Gesamtkommentar"
}
}

JSON structure to output for \`Ende Korrektur\`:
[
{
"contract": <contract_json_object>,
"chatRef": "chat-0001",
"importedTaskScores": []
},
{
"contract": <contract_json_object>,
"chatRef": "chat-0002",
"importedTaskScores": []
}
]

Formatting notes for the JSON structures:
- replace \`<contract_json_object>\` with the full JSON object provided in the Contract JSON section below
- replace example \`chatRef\`, \`taskId\`, \`scoringUnitId\`, points, comments, evidence, and metadata with the resolved values
- use only task IDs and scoring-unit IDs that exist in the loaded contract
- omit optional fields when they are empty or unsupported
- for \`Ende Korrektur\`, output only the array and include one object per resolved Leistung

## Contract JSON

{{contractJson}}

## Contract

{{contractMarkdown}}
Comment on lines +188 to 242
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot May 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Inlining the entire contract JSON multiple times inside the prompt's example structures (once in Zwischenexport and twice in Ende Korrektur) will cause massive token bloat and potentially exceed the LLM's context window limits for larger exams.

Instead, we can define a single Contract JSON section at the bottom of the prompt to hold the full contract JSON, and use a placeholder like <contract_json_object> in the examples. This reduces the prompt size significantly while keeping the instructions clear.

JSON structure to output for Zwischenexport:
{
  "contract": <contract_json_object>,
  "chatRef": "chat-0001",
  "importedTaskScores": [
    {
      "taskId": "task-1",
      "points": 0,
      "maxPoints": 0,
      "scoringUnitId": "task-1.score",
      "comment": "Kurzbegruendung mit Bezug zur Leistung",
      "confidence": 0.8,
      "evidenceIds": ["evidence-1"]
    }
  ],
  "evidence": [
    {
      "id": "evidence-1",
      "kind": "quote",
      "value": "Kurzer Beleg aus der Leistung"
    }
  ],
  "metadata": {
    "generalComment": "Kurzer Gesamtkommentar"
  }
}

JSON structure to output for Ende Korrektur:
[
  {
    "contract": <contract_json_object>,
    "chatRef": "chat-0001",
    "importedTaskScores": []
  },
  {
    "contract": <contract_json_object>,
    "chatRef": "chat-0002",
    "importedTaskScores": []
  }
]

Formatting notes for the JSON structures:
- replace <contract_json_object> with the full JSON object provided in the Contract JSON section below
- replace example chatRef, taskId, scoringUnitId, points, comments, evidence, and metadata with the resolved values
- use only task IDs and scoring-unit IDs that exist in the loaded contract
- omit optional fields when they are empty or unsupported
- for Ende Korrektur, output only the array and include one object per resolved Leistung

## Contract JSON

    {{contractJson}}

## Contract

{{contractMarkdown}}

Expand Down
1 change: 1 addition & 0 deletions modules/exams/src/use-cases/export-correction-session.use-case.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ function buildPromptArtifacts(
): string {
return renderTemplate(rulePack.templates.prompt, {
contractMarkdown,
contractJson: JSON.stringify(contract, null, 2),
importBundleSchema: JSON.stringify(rulePack.importBundleSchema, null, 2),
rulePackManifest: JSON.stringify(rulePack.manifest, null, 2),
rulePackRules: JSON.stringify(rulePack.rules, null, 2),
Expand Down
25 changes: 25 additions & 0 deletions modules/exams/tests/export-correction-session.use-case.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,31 @@ describe('ExportCorrectionSessionArtifactsUseCase', () => {
expect(artifact.promptFile.content).toContain('"importedTaskScores"');
expect(artifact.promptFile.content).toContain('must return exactly one raw JSON array');
expect(artifact.promptFile.content).toContain('do not invent a wrapper object');
expect(artifact.promptFile.content).toContain(
'Bitte laden Sie die erste Leistung oder mehrere Leistungen gleichzeitig hoch.'
);
expect(artifact.promptFile.content).toContain(
'the user may upload multiple Leistung files in the same message'
);
Comment on lines +218 to +223
expect(artifact.promptFile.content).toContain('JSON structure to output for `Zwischenexport`');
expect(artifact.promptFile.content).toContain('JSON structure to output for `Ende Korrektur`');
expect(artifact.promptFile.content).toContain('"contract": <contract_json_object>');
expect(artifact.promptFile.content).toContain('## Contract JSON');
expect(artifact.promptFile.content).toContain('"id": "contract-session-session-2026-04-17"');
expect(
artifact.promptFile.content.match(/"id": "contract-session-session-2026-04-17"/g)
).toHaveLength(1);
expect(artifact.promptFile.content).toContain('"chatRef": "chat-0001"');
expect(artifact.promptFile.content).toContain('"importedTaskScores": [');
expect(artifact.promptFile.content).toContain('"evidence": [');
expect(artifact.promptFile.content).toContain('"metadata": {');
Comment on lines +224 to +235
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot May 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Update the test assertions to match the optimized prompt structure that uses a single Contract JSON section and placeholders in the example structures.

Suggested change
expect(artifact.promptFile.content).toContain('JSON structure to output for `Zwischenexport`');
expect(artifact.promptFile.content).toContain('JSON structure to output for `Ende Korrektur`');
expect(artifact.promptFile.content).toContain('"contract": {');
expect(artifact.promptFile.content).toContain('"id": "contract-session-session-2026-04-17"');
expect(artifact.promptFile.content).toContain('"chatRef": "chat-0001"');
expect(artifact.promptFile.content).toContain('"importedTaskScores": [');
expect(artifact.promptFile.content).toContain('"evidence": [');
expect(artifact.promptFile.content).toContain('"metadata": {');
expect(artifact.promptFile.content).toContain('JSON structure to output for Zwischenexport');
expect(artifact.promptFile.content).toContain('JSON structure to output for Ende Korrektur');
expect(artifact.promptFile.content).toContain('"contract": <contract_json_object>');
expect(artifact.promptFile.content).toContain('"id": "contract-session-session-2026-04-17"');
expect(artifact.promptFile.content).toContain('"chatRef": "chat-0001"');
expect(artifact.promptFile.content).toContain('"importedTaskScores": [');
expect(artifact.promptFile.content).toContain('"evidence": [');
expect(artifact.promptFile.content).toContain('"metadata": {');

expect(artifact.promptFile.content).toContain('Kurzbegründung mit Bezug zur Leistung');
expect(artifact.contractFile.content).toContain(
'The user may upload multiple Leistung files in the same message.'
);
expect(artifact.contractFile.content).toContain(
'Process each Leistung independently and keep its correction data isolated.'
);
});

it('does not require users to provide Leistung chatRefs before evaluation', () => {
Expand Down