Common submission errors

A small QA on questions we get regarding the submission:

Q: I created an algorithm but haven't received any mail whether my submission was successful or not.
A: After creating an algorithm you have to go back to the submission panel on the challenge website and submit the created algorithm to the desired phase

Q: My algorithm failed on one or more cases. Whats wrong?
A: More specific error descriptions can be found when you look at your algorithm results. For that please go to your algorithm/ALGO_NAME/results (https://grand-challenge.org/algorithms/ALGO_NAME/jobs/) and click on the error.
Most common errors:

• Output directory 'images/automated-petct-lesion-segmentation' is empty. Your final segmentation was not written to the dedicated output path.
• Output file 'data-centric-model.json' was not produced. The category json was not written to the dedicated output path.
• Time limit exceeded. Your algorithm took longer than 5min for one case. If your algorithm fails on just one case the whole submission will fail. If your algorithm is at the edge of 5min you could also implement a timer or a function which just saves an empty mask at the beginning. This will reduce your performance but ensures, that you predict all cases.
• RuntimeError: Current CUDA Device does not support bfloat16. Please switch dtype to float16.
• The output file 'data-centric-model.json' is not valid. JSON does not fulfill schema: instance is not of type 'boolean'. The json should just contain a single boolean. Please use the code from the baselines to save the file.

Q: Can you share some test samples?
A: The data for the preliminary submission is drawn from the public train dataset (you could search for the largest file by filesize and the file with highest number of lesions) and the final test data wont be released.

Hi Team,

We submitted our algorithm to Preliminary Test Set but got failed evaluation. However, the algorithm was proceeded successfully IN 'Try-out Algorithm'. No error was reported in the log panel. How could I find out the errors?

See the answer to question 2 above. All logs that are produced by your algorithm container are reported on Grand Challenge. You can see the error "Time limit exceeded" on both the job list and job detail view.

We submitted our algorithm to the Preliminary Test Set, but it failed the evaluation (The algorithm failed on one or more cases). However, the algorithm successfully passed the 'Try-out Algorithm' phase without any errors reported in the log panel. How can I identify the issues?

As described in the second answer you have to go to your job results. You can find them by going to https://grand-challenge.org/algorithms/ALGO_NAME/jobs/. Your algorithm is named b1 so you have to look at https://grand-challenge.org/algorithms/b1/jobs/. Here you can see that the time limit was exeeded, probably due to the test-time augmentation.

I don't know why, but it's empty.

Sorry for the confusion about the logs. Algorithm details should now be visible for participants.