Tips for running a challenge

Test your submission process¶

Once you have everything set up, we advise you to test the submission process thoroughly before opening it up for participants. You can create a dummy user account and create test submissions. The grand challenge support team is happy to help you debug any issues you face during this process.

Guide your participants to submission tips¶

Please guide your participants to the documentation presented here about submission tips.

Provide tutorials¶

It is also advisable to create an example docker container containing an example algorithm for your participants. We provide a generic tutorial for algorithm creation with Docker here that you can use as a blueprint for your own, more tailored tutorial. Examples of demo repositories for previous challenges can be found here and here.

Walking participants through the submission process in a video tutorial can also help prevent questions. You can refer your participants to this section of the documentation or use the information there for your own video tutorial. Examples from previous challenges can be found here and here.

Enable the Forum for your challenge¶

Participants can then ask questions about their submissions or other problems they face. In contrast to private emails, this open forum has the benefit that other participants can read these questions and your answers to them, hopefully reducing the question load for you as an admin. Previous challenges have usually posted a list of tips for submission on the forum.

Help your participants debug their submissions¶

We do not publish the submission logs openly on GC. Doing so could potentially leak information about the test data set. Debugging their submission is thus difficult for participants to do on their own. They will require your assistance. To inspect the logs, navigate to the Admin menu in your challenge and select Evaluations from the tab menu. Click on the algorithm of the evaluation that failed and that you'd like to get more information about. This will lead you to a list of results that this algorithm has produced. Select the appropriate result and you will be able to see the logs that the algorithm produced. You can then share the logs with the participants provided that they do not leak sensitive information about the test data. So please do read the logs yourself before posting them.

Note that we do tell participants (as part of the submission tips that we advise you to publish in the forum) that they should thoroughly test their algorithm not only locally, but also on Grand Challenge. Once they have uploaded their container, they can try out their algorithm on test images of their choosing and even inspect the result in an online image viewer. Nonetheless, submission failures will happen and so it is important for you to know how to access the logs and help.