Should we use the optimal frame mask or the whole case masks for evaluation?

Based on my understanding, we only need to input the optimal frame index (int) and its corresponding mask (.mha) for metric evalution. However, in evaluate.py from evaluation repo (line 118-123), fetal_abdomen_frame_number is used to extract one frame from the output segmentation mask (840 frames .mha) and I met indexing bugs when I only provide the optimal frame mask (1 frame .mha) with its frame index. It seems unreasonable and increases the memory overheads during the evaluation.

Is my understanding correct? Thank you very much.

Hi Icchen,

Thank you for your question. Your algorithm should output a 2D mask and its corresponding frame number. The baseline code has a default saving function (write_array_as_image_file) you should ideally use. This function internally converts the 2D mask to a 3D mask in which all other images are blank. While this indeed seems unreasonable, it is meant for internal debugging, as it allows me to visualize output solutions on the grand-challenge viewer. For this, The mask must match the 3D dimensions of the input images.

If you use the function I mentioned above and ensure your frame number is an integer within [0, 840) or -1 (for cases where you find no good frame), your indexing issues should be solved.

Good luck!

Thank you for your response!

Also, other questions that I am still having are:

• Is the deadline preliminary testing phase at August 6 ?

• Are there any requirements to enter the final test phase in the preliminary testing phase?

You're welcome!

Regarding your questions:

The Final Test Phase will be open from July 15th until July 31st. Only one submission per team is allowed in this phase. The Preliminary Development Phase will remain open during this period to allow for testing your model before your final submission.

There are no requirements for participating in the Final Test Phase. However, note that, along with your submission, you must provide:

• A valid and publicly accessible link to the GitHub repository containing your model.

• A PDF file with the following details:

  • A full description of all team members, including:
    • Name and surname
    • Affiliation(s)
    • Email address
    • Grand-challenge username (if applicable)
  • A clear explanation of your method. This explanation should be detailed enough for others to reproduce your proposed solution.

This information can also be found here.

Dear organisation, can you provide us with more information regarding the PDF? In particular extension and level of detail, it is supposed to be a manuscript?

I guess there will be a lot of low level detail to be covered in order to make solutions reproducible, so a high level explaination + code repo would be enough?

Best, C

Hi Carlos,

Thank you for your question. The document does not need to follow the traditional structure of a research article: there is no need to include introduction, results, and discussion sections. Instead, it should be formatted like a methods section of a paper.

It should include information regarding the following (where applicable):

• Data Processing:

  • How the data was cleaned and pre-processed.

  • How the data was split for training.

• Network Architecture:

  • Description of the neural network architecture(s) used.
  • Hyperparameters chosen.
• Training Details:
  • Information on the training process, including the number of epochs and any techniques used to improve training (e.g., early stopping, data augmentation).
• Post-processing:
  • Any post-processing steps applied to the predicted masks.

Hope this helps!