Making a Challenge Submission

Depending on the setup of the challenge you participate in, you can have two types of challenge submissions: a set of predictions or an algorithm container. You can make your submission by navigating to the Submit tab as shown below. In this example, the sub-tab "Challenge" is a phase that accepts prediction file(s) submissions, and "Algorithm" is a phase that accepts algorithm containers as submissions.

Submitting your predictions file(s)¶

Under the Submit tab, you will have a field to drag and drop your predictions or browse through your computer to upload the predictions.

Submitting your Algorithm container¶

Navigate to the phase that accepts Algorithm submissions as shown in the example below. In this example, the sub-tab "Algorithms" is the phase that accepts Algorithm containers as Challenge submissions.

Under this sub-tab, you will have the option to select an already existing Algorithm on grand-challenge.org for which you are an editor. Additionally, you also have options to provide a brief description of your Algorithm in a PDF or attach publications to appear along with your Algorithm.

⚠️It is highly advisable to test your Algorithm containers locally before you upload your containers to grand-challenge.org. For more instructions on creating your Algorithm containers, please check the documentation here. Please check this page for testing your containers locally.

⚠️Note that there are submission limits for each challenge. Submission limits are imposed to prevent abuse of the system and to prevent participants from iteratively optimizing the test set by making multiple submissions. Also, note that we have limited computational resources and we'd like to keep our Challenges fair as described in this piece by the MIDOG2021 organizers.

Submission tips¶

• Please make sure that you are a verified user to submit your solution.
• Please remember to set the GPU and memory requirements correctly when uploading your container. If you enable GPU support, inference jobs will be assigned 1x Nvidia T4 card.
• The duration of inference jobs will be limited by the challenge you created the container for, this can be from 1 minute up to 1 hour.
• Note that the container you upload for your algorithm cannot exceed 10GB. We recommend you compress your container with gzip to ensure a smooth upload: docker save your_algorithm | gzip -c > YourAlgorithm.tar.gz.
• It may take some time to validate your algorithm, if your container is not marked as active after 24 hours, please contact grand challenge support.
• You do not need to create a new algorithm for each submission, you can update your existing algorithm by uploading a new docker container image on the container management page or tagging your linked Github repository. If you update your algorithm, don't forget to make a new submission to the challenge with it, this will not happen automatically.
• Once your container is active, please test it out before submitting it to the challenge, you can upload data using the "Try Out Algorithm" button. Please use a representative image and check that the outputs are what you expect. You can inspect the error messages on the Results page of your algorithm.
• Remember that your container will not have access to the internet when executing, so all model weights must be present in your container image. You can test this locally using the --network=none option of docker run. If you do not do this, you will see errors like Temporary failure in name resolution in your logs.
• Evaluation of the submitted algorithms can take a while depending on how many other submissions are happening at the same time. So, please be patient :) If it has not been executed after 24 hours, please contact us.
• Debugging after submission: as a participant, you cannot always see the submission logs as this could leak information about the test set. Please contact the challenge organizers if your submission failed and you want to know why. Please be reminded that any errors about your algorithm container should have been caught and eliminated before submitting to the challenge – this is where trying your algorithm out on the GC platform comes in (see the previous point).