Submission Github Requirements

First off, congratulations of a successful challenge! It is exciting to see the large number of teams that were able to submit a working model. Moving into Friday, when the participation Google form is due, could I ask for clarification on what the mandatory public Github repository for each submission must include?

The model architecture is an obvious requirement, but does this also include:

• Trained instances of the model (model weights) matching the submitted Docker image?
• Trained instances of the model (model weights) trained with only ISLES24 data in the event that private training data was also used?
• Image pre/postprocessing code?
• Code for loading the preprocessed data, including any real-time augmentation?
• Code for training the model, including hyperparameter optimization?
• Code for loading the model weights and data, and passing the data through the model for inference?

I do not want to omit anything that is strictly required to meet the participation guidelines but, at the same time, it is a laborious process to document and refactor code for public release, and so it would be helpful to know the exact scope of what must be released in order to qualify as a participant.

Best,

Anthony

Hi Anthony,

Thank you for the kind feedback!

Regarding Git repos:

The goal is to make it easy for anyone interested in testing your algorithms—whether they want to reproduce the paper’s results, annotate local data, compare it with another method, etc. This way, the challenge outputs are reusable by the community, rather than just being numbers reported in a paper. I think it also adds transparency, showing that the laeding teams truly performed what they reported, and that there was no 'magic' involved.

That said, we encourage participants to include everything needed to run their algorithms 'out of the box,' which should at least include:

• Inference script

• Model weights

• Appropriate license (e.g. Apache)

• [Optional ]Training scripts

The Docker image can be the one you submitted for the challenge (since all participants are familiar with it and can easily reuse it), or you can create an even simpler, more user-friendly solution. An example of a complete Git repo from our last challenge is this one.

I hope this clears things up!

Best, Ezequiel