Test Evaluation¶

We highly recommend testing all phases of your challenge before allowing participants to submit their solutions. This allows you to find issues with your evaluation or scoring, gets you familiar with the debugging options and allows you to verify that the chosen resources are sufficient for all phases and data.

Deadline: 6 weeks post-challenge acceptance

Submit to your phase.¶

To test your evaluation, you can submit for instance the example algorithm created earlier in the process. As a challenge organizer, you can submit as many times as you like, provided there is budget available. Refer to this page for more information on how to create a submission, and our tip on debugging failed submissions

New Evaluation¶

When uploading a new evaluation container it's possible to re-evaluate existing submissions to create new evaluation results. Re-evaluation is a manual step since re-evaluating all submissions can be quite expensive. Navigate to Admin ⟶ [name of phase] ⟶ Submission & Evaluations:

Clicking on one of the submission ID links will bring up the submission detail page. On this page, shown below, you will find a button to re-evaluate the submission with the new (active) evaluation method. Doing this re-uses the existing algorithm submissions results: saving compute time and having to upload a new algorithm image every time the evaluation method changes.