Hi everyone!

I would like to clarify a few things regarding the final test set submission.

  1. The submission deadline is on Jan 25, 11.59pm ** (AoE**, https://time.is/Anywhere_on_Earth). Your submission does not need to be evaluated until the deadline, it just means, you should submit your algorithm to the NODE21 track latest at that time. The evaluation might take up to 2 days as you have noticed with the current overload of jobs in grand-challenge.

  2. For the generation track: the algorithms will be running on the same 1000 CXR images (the faster r-CNN model will be trained with them, but the final model will be evaluated on the final test set this time). Because of how grand-challenge handles identical algorithm submissions, when you try to submit the same algorithm you have submitted before to the generation track, you might get a notification something like 'A submission for this algorithm container image for this phase already exists.' If that is the case, please rebuild and upload your new docker container (or retag your container after adding print lines or comment lines) to handle this issue. Sorry for the inconvenience, but that is the only solution right now.

  3. After participants submit their algorithm, if it fails for some reason, we will give them a few days of grace period to fix their container and submit it again. However, in this period, they will not be allowed to make any changes in their algorithm, or in the pdf file they have uploaded to describe their algorithm. However, hopefully, there will not be any failed algorithms since they have been already tested in the experimental test set.

  4. If you are participating in a challenge with a team (let's say a team of 2 people), and you have two unique solutions that are different from each other, then you can submit both of them. However, they have to be completely different from each other (e.g., completely different network architectures), similar models with different hyperparameters or different ensembles cannot be submitted. Model selection should have been done through the experimental test set; this is the final test set where each algorithm will be evaluated once.

Again, thanks a lot for participating in the challenge, and I wish you good luck with the final push. 💪

Best, Ecem