Doubt about ground truth mask P141.mha in the evaluation github repo

Hi,

I have a doubt regarding the label file/mask named P141.mha that can be found in the evaluation algorithm github repo. It has 3 different labels (pixel values ranging from 0 to 3). This causes the evaluation code to calculate an incorrect dice coefficient. Will this happen also in the final evaluation of the code? Because I'm getting very bad results in my submission, and I'm afraid this is the reason. Is it something I'm missing?

Thank you very much.

Vicent

Hi Vicent, I might have converted to .MHA the wrong P141 file, but I can assure that all the ground truths in both the evaluation and test phase here on grand-challenge are binary images.

Best regards, Luca Lumetti

Hi Luca,

I have another doubt. My code seems to be working, but I don't know why the first mask that is generated seems to be correct (with a DICE coefficient of 0.7+), but the following ones seem to be very bad performing, with DICEs of 0.2 and 0.0 (probably meaning that the file is completely empty). I have been trying to pinpoint what is happening, but I am completely stuck. Is there any way to take a look at the masks that I'm generating, or the output (stdout/stderr) that is being generated? Just to make sure I'm not making a mistake when dockerizing the algorithm, or generating the output in the docker.

Thank you very much. Vicent Caselles

Hi Vicent, I cannot provide you the masks generated by your algorithm, as it would make unfair the challenge and they can be exploited to produce an algorithm tailored for the test cases, hence overfitting the set. The stdout and stderr of your algorithm do not contain any significative message. What I can guess is that you have correctly dockerized your algorithm, but probably it cannot generalize well on some of the images.

Regards, Luca Lumetti

Thanks Luca, I'll keep on working to get better results.

Vicent