Submission for preliminary test phase is now open! ¶
By: simongraham73 on Feb. 14, 2022, 12:11 p.m.
We are really pleased to announce that the challenge is now accepting submissions for the preliminary test set of both tasks !
We have provided detailed steps and instructional videos in the Submission tab of the challenge. Here, you will need to complete the following three steps to make a valid submission:
- Dockerize code using our template
- Upload algorithm to grand challenge page
- Submit algorithm to our challenge
We have given detailed instructions on how to make the docker container in our associated GitHub page! For this, we have a dedicated
docker_template branch. Take a look at both this repo and our videos to fully understand how to make the container successfully.
The preliminary test set is from a single centre from TCGA. In fact, we have chosen some difficult images for this and so don’t be disheartened if your initial results are significantly lower than what you were getting on the development set! Remember, what you get on the final test set is what counts. Please start thinking about making submissions on the preliminary test set as soon as you can to test your submission and so that you are ready for your final submission in a couple of weeks !
You can also now see the HoVer-Net baseline scores on the preliminary test set on the leaderboard. The leaderboards are ranked by mPQ+ and R2 for segmentation & classification and cellular composition tasks respectively. However, you can view additional metrics by clicking on the appropriate button on the leaderboard! For example, we give the binary PQ, binary PQ+ and PQ+ per class for segmentation & classification. We also give the mean MAE, mean MAAPE and R2 for each class for cellular composition. Note, for both MAE and MAAPE- the lower the score, the better!
Remember, only one submission per team and per task is allowed per day - we will be manually tracking this and will strictly not consider teams in the final leaderboard that ignores this. This means that each time can actually make 2 submissions in a single day - one for seg & class and one for counting.
All models will be run on a single NVIDIA T4 GPU with 8 CPUs and a max memory of 32 GB.
Good luck !