How is final ranking determined?

How is final ranking determined?  

  By: 89douner on July 20, 2022, 8:39 a.m.

Q. How is the ranking determined? - On leaderboard, there are three sections, "Lesion segmentation", "Lesion segmentation - Hidden test set", and " Lesion segmentation - Hidden test set - Algorithm test". - Are all three sections' ranking used for final ranking? or Is the raking determined through "Lesion segmentation - Hidden test set - Algorithm test"? - I'd like to know more detail.

Thanks!

Re: How is final ranking determined?  

  By: sliew on July 25, 2022, 9:19 p.m.

Hello - the final MICCAI rankings will be calculated from a combination of the automated evaluation (e.g., Lesion Segmentation; this phase is currently open until Aug 15 and leaderboard is public) and the hidden algorithm evaluation (e.g., using a Docker submission to evaluate performance on a completely hidden generalizability dataset; this phase will be open from Aug 3-15). In order to discourage participants from overfitting or retraining over the public test data (e.g., "Lesion Segmentation"), the hidden test cases (Docker submission) will be weighted four times higher than the public test ones. The final results will be announced at MICCAI, in conjunction with our partners at the ISLES'22 challenge (see: https://www.isles-challenge.org/ and https://isles22.grand-challenge.org/).

 Last edited by: sliew on Aug. 15, 2023, 12:56 p.m., edited 1 time in total.

Re: How is final ranking determined?  

  By: 89douner on July 26, 2022, 1:12 a.m.

Thanks for the answer!