By: kaifengsong0 on July 11, 2021, 2:39 p.m.
Hello, can you explain "The uncertainty measures won't be evaluated during the validation phase"? Do we need to provide an uncertainty map when task 3 is submitted?I would really appreciate it if you reply.
By: kvanwijnen on July 12, 2021, 9:18 a.m.
Thank you for your message and great to hear you are participating in our challenge!
For the final submission of task 3 you should indeed output a predicted segmentation and a predicted uncertainty map (see https://valdo.grand-challenge.org/Task3/ section 'Goal' and https://valdo.grand-challenge.org/Evaluation/ 'Task 3'). The evaluation of this task is subdivided in evaluating segmentation, detection, volume prediction and uncertainty. So the uncertainty map only affects part of the evaluation.
If you prefer not to adapt your method to also output an uncertainty map, you can output any image for this uncertainty map as long as the values range between 0 (certain) and 1 (uncertain). So you could also use the (inverted) predicted segmentation even though this is not really an uncertainty map (watch out with the definition of the uncertainty map though, since 1 is defined as uncertain). We will report the performance per metric at MICCAI and also on the leaderboard, so you are free to choose which metrics you want to focus most on. However it's important to realize the ranking is based on all metrics.
