Submission for preliminary test phase is now open!

We are really pleased to announce that the challenge is now accepting submissions for the preliminary test set of both tasks !

We have provided detailed steps and instructional videos in the Submission tab of the challenge. Here, you will need to complete the following three steps to make a valid submission:

• Dockerize code using our template
• Upload algorithm to grand challenge page
• Submit algorithm to our challenge

We have given detailed instructions on how to make the docker container in our associated GitHub page! For this, we have a dedicated docker_template branch. Take a look at both this repo and our videos to fully understand how to make the container successfully.

The preliminary test set is from a single centre from TCGA. In fact, we have chosen some difficult images for this and so don’t be disheartened if your initial results are significantly lower than what you were getting on the development set! Remember, what you get on the final test set is what counts. Please start thinking about making submissions on the preliminary test set as soon as you can to test your submission and so that you are ready for your final submission in a couple of weeks !

You can also now see the HoVer-Net baseline scores on the preliminary test set on the leaderboard. The leaderboards are ranked by mPQ+ and R2 for segmentation & classification and cellular composition tasks respectively. However, you can view additional metrics by clicking on the appropriate button on the leaderboard! For example, we give the binary PQ, binary PQ+ and PQ+ per class for segmentation & classification. We also give the mean MAE, mean MAAPE and R2 for each class for cellular composition. Note, for both MAE and MAAPE- the lower the score, the better!

Remember, only one submission per team and per task is allowed per day - we will be manually tracking this and will strictly not consider teams in the final leaderboard that ignores this. This means that each time can actually make 2 submissions in a single day - one for seg & class and one for counting.

All models will be run on a single NVIDIA T4 GPU with 8 CPUs and a max memory of 32 GB.

Good luck !

Team CoNIC

Dear Simon,

thank you for providing the docker template and the instructional videos. I have a question just for clarification: 1 submission per day means 1 succesful submission? And is there any way to get some information why a submission failed?

Best regards Tim

Hi Tim,

You can make 1 successful submission per task per day. So you can make 2 submissions per day (1 on each task). I see you have now been successful making the submission on each task. Nice work!

Thank you!

I have noticed that in the "conic_template" main.py the counts are saved as

f'{output_dir}/{type_name}.json'

whereas in the "conic_baseline" main.py the counts are saved as

f'{OUT_DIR}/{type_name}-count.json'.

Yesterday my submission failed but this morning it worked after adding "-count" to the filename. So, I guess this suffix is required (and may need a fix in the repo).

Edit: I see now that I could also have read the comments above the file saving which clearly state that the "-count" is needed.

@iait

Hi, yes that is indeed a bug. I have uploaded a minor fix to add "-count" to make it in line with the comment above it.

Thanks for your notification