Skip to content

Commit 4b7ae5b

Browse files
committed
Document setting timelimit strategy
1 parent bf78085 commit 4b7ae5b

File tree

3 files changed

+19
-0
lines changed

3 files changed

+19
-0
lines changed

doc/manual/config-basic.rst

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,7 @@ Before a contest, you will want to have tested your reference
129129
solutions on the system to see whether those are judged as expected
130130
and maybe use their runtimes to set timelimits for the problems.
131131

132+
132133
The simplest way to do this is to include the jury solutions in a
133134
problem zip file and upload this. You can also upload a zip file
134135
containing just solutions to an existing problem. The zip
@@ -140,4 +141,11 @@ it need not have started yet). You can verify whether the submissions
140141
gave the expected answer in the Judging Verifier, available from
141142
the jury index page.
142143

144+
After this check change the evaluation to full in the configuration
145+
and rejudge with a reasonable overshoot to distinguish between the
146+
slowest *Accepted* solution and the fastest *Timelimit Exceeded*.
147+
As a rule of thumb set the timelimit to twice the slowest *Accepted*
148+
solution. The Statistics/Analytics page has a graphic overview for those
149+
submissions.
150+
143151
.. _ICPC-compatible teams.tsv files: https://ccs-specs.icpc.io/2021-11/ccs_system_requirements#teamstsv

webapp/src/Controller/Jury/JuryMiscController.php

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -272,6 +272,8 @@ public function judgingVerifierAction(Request $request): Response
272272

273273
$verifyMultiple = (bool)$request->get('verify_multiple', false);
274274

275+
$problems = $this->dj->getCurrentContest()->getProblems() ?? [];
276+
275277
foreach ($submissions as $submission) {
276278
// As we only load the needed judging, this will automatically be the first one
277279
/** @var Judging $judging */
@@ -329,6 +331,7 @@ public function judgingVerifierAction(Request $request): Response
329331
'verified' => $verified,
330332
'nomatch' => $nomatch,
331333
'earlier' => $earlier,
334+
'problems' => $problems,
332335
'verifyMultiple' => $verifyMultiple,
333336
]);
334337
}

webapp/templates/jury/check_judgings.html.twig

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,4 +99,12 @@
9999
{{ checkJudgings.verifyResults('earlier', 'Verified earlier', earlier, true) }}
100100
{{ checkJudgings.verifyResults('nomatch', 'Without magic string', nomatch, true) }}
101101

102+
<h2>Problem runtime analytics</h2>
103+
You probably want to <a href="{{ path('jury_config') }}">fully</a> judge all submissions with enough overshoot to see the maximum runtime for Accepted solutions and tune those against
104+
expected Timelimit Exceeded solutions.<br>
105+
{% for p in problems %}
106+
{% set link = path('analysis_problem', {'probid': p.probid, 'view': 'hidden'}) %}
107+
<a href="{{ link }}">{{ p | problemBadge }}</a>
108+
{% endfor %}
109+
102110
{% endblock %}

0 commit comments

Comments
 (0)