By: 89douner on July 20, 2022, 8:39 a.m.
Q. How is the ranking determined? - On leaderboard, there are three sections, "Lesion segmentation", "Lesion segmentation - Hidden test set", and " Lesion segmentation - Hidden test set - Algorithm test". - Are all three sections' ranking used for final ranking? or Is the raking determined through "Lesion segmentation - Hidden test set - Algorithm test"? - I'd like to know more detail.
Thanks!
By: sliew on July 25, 2022, 9:19 p.m.
Hello - the final MICCAI rankings will be calculated from a combination of the automated evaluation (e.g., Lesion Segmentation; this phase is currently open until Aug 15 and leaderboard is public) and the hidden algorithm evaluation (e.g., using a Docker submission to evaluate performance on a completely hidden generalizability dataset; this phase will be open from Aug 3-15). In order to discourage participants from overfitting or retraining over the public test data (e.g., "Lesion Segmentation"), the hidden test cases (Docker submission) will be weighted four times higher than the public test ones. The final results will be announced at MICCAI, in conjunction with our partners at the ISLES'22 challenge (see: https://www.isles-challenge.org/ and https://isles22.grand-challenge.org/).
