Update on Final Test Phase Info ❗️❗️❗️ ¶
By: simongraham73 on Feb. 28, 2022, 3:41 p.m.
We would like to give you 2 updates:
1) We have found for task 1, the PQ per class in the leaderboard and the output logs do not correspond to the correct nuclei type. This does not affect the final mPQ+ result, but we should inform you in case you were optimising for a particular class. The correct class mapping is as follows:
(type on leaderboard: actual type) Epithelial: Neutrophil Neutrophil: Plasma Lymphocyte: Epithelial Plasma: Lymphocyte Eosinophil: Eosinophil Connective: Connective
Note, for task 2, the leaderboard stats per class is correct!
2) The maximum run time allowed for the final test phase is 60 minutes. Note, this is for a single input array of size 1010x256x256x3 (1010 images). If your submission does not pass (but it did pass in the preliminary phase), then this will be because it exceeded the max time limit. You should ensure that you test how long your docker container takes to run before submitting. If you can't get your solution to run under 60 minutes - an alternative is to send across your Docker container and we can run your container locally - but first please try to see whether your submission passes under the time limit. However, if doing this, your algorithm MUST have passed during the preliminary test phase. We will provide more information on this very soon, but please note that this should only be done as a last resort!
Due to the above, we are going to reopen the preliminary test phase for an extra 2 days (closing on the 2nd March at 23:59 GMT). We will still keep the final test phase open and so you are still welcome to submit your algorithm when you are ready.
Team CoNIC 🚀
2022-03-01 Edited by vqdang: In case that you want to know your docker runtime for the preliminary test set, please fill your team name here , we will periodically check this form every 2h to add more information.