FAQ
←
Set up communication channels
Start your challenge
→

Tips for running a challenge

Test your submission process

Once you have everything set up, we advise you to test the submission process thoroughly before opening it up for participants. The Grand-Challenge support team is happy to help you debug any issues you face during this process.

Set up budget alerts

To help you stay on top of your spending, you will receive automatic budget alerts when your usage hits:

  • 70%
  • 90%
  • 100% of your total budget

If you'd like to set custom alert levels, please contact us at support@grand-challenge.org.

These alerts give you the opportunity to proactively:

  • Monitor your challenge usage
  • Take action to avoid interruptions, either by:

Guide your participants to submission tips

Please guide your participants to the documentation presented here about submission tips.

Provide tutorials

It is also advisable to share an example algorithm script (including the Dockerfile and instructions on how to build the image) for your participants. You can use the challenge Starter Kit example as a baseline, but you will likely want to customize it a littler further before sharing with your participants.

Walking participants through the submission process in a video tutorial can also help prevent questions. You can refer your participants to this section of the documentation or use the information there for your video tutorial. Examples from previous challenges can be found here and here.

🛠 Help Your Participants Debug Their Submissions

By default, submission logs are not visible to participants on Grand Challenge. This is to prevent any risk of inadvertently revealing details about the test dataset. As a result, participants may need your help to debug failed submissions.

🔍 How to Access Submission Logs

To inspect a participant’s submission logs:

  1. Go to the Admin menu of your challenge.
  2. Navigate to the phase-specific tab and click Submissions & Evaluations.
  3. Click the Evaluation ID of the failed submission.
  4. Scroll to the Evaluation Admin section.
  5. Under Prerequisite jobs, click on each job ID to inspect their logs.

You can share these logs with the participant, as long as they do not contain sensitive information. Be sure to review the logs carefully before sharing them.

📣 Encourage Testing During Development

Participants are advised (via submission tips and forum announcements) to thoroughly test their algorithm, both locally and on the Grand Challenge platform:

  • After uploading a container, they can run tests on public or synthetic data.
  • They can inspect the results using the online image viewer before final submission.

Still, submission failures are common — knowing how to access and interpret logs will help you support them effectively.

Optional: Open Submission Phase

During early development or debugging phases, you can make a phase more permissive by enabling open submission settings. This allows participants to:

  • Automatically view their logs, input data, and outputs
  • Debug their algorithms independently
  • Save you significant time in troubleshooting

⚠️ Warning: Enabling this setting makes the entire dataset for that phase accessible to participants. It should only be used with public or synthetic data, never with hidden test sets. If this setting is active, challenge admins will receive email alerts, and the platform will show clear warnings.

To enable this:

  • Go to the Algorithm section of your Phase settings
  • Enable "Give Algorithm Editors Job View Permissions"

Optional: keep results hidden until a big reveal 🎊

By default, leaderboard rankings or submission details are updated every time someone submits to your challenge. An alternative is to keep them hidden until a specific day or event, making the publication a big reveal. Follow this workflow to enable such a big reveal:

1. Set Up the Phase:

Ensure Admin ⟶ Phase Settings ⟶ Scoring Tab ⟶ Auto publish new results is set to False.

2. Participant Submissions:

Allow participants to submit their algorithms. Note that participants will not receive their results immediately.

3. Close the Submission Phase:

Close the phase by either:

  • Letting the submission date pass, or
  • Setting the submission limit to zero.
4. Set Visibility to Private:

Change the phase visibility setting to hidden via Admin ⟶ Phase Settings ⟶ Public to False.

5. Publish Selected Evaluations:

Carefully publish the selected evaluations by manually pressing the 'Publish Result' button for evaluations listed in Admin ⟶ Phase Settings ⟶ Submissions & Evaluations.

6. Reveal Day:

On the designated reveal day, set Admin ⟶ Phase Settings ⟶ Public to True to make results visible to participants.

←
Set up communication channels
Start your challenge
→