FINAL SUBMISSIONS OPEN! ¶
By: f.ciompi on June 19, 2022, 4:32 p.m.
Dear participants,
In the past two weeks, we have been working together with the grand-challenge support team to solve all the issues that some of you had experienced during the last submissions of your methods, to both experimental leaderboard 1 and 2. We have managed to solve most of them and now we are finally ready to enable the final submissions!
Today, we are opening the final leaderboard 1 and leaderboard 2, the submission system should be active after 18:00h (Amsterdam time). The deadline for submitting your final method is June 24 at 23:59 (CEST); for details on how time zones work on grand-challenge, see this previous discussion (https://grand-challenge.org/forums/forum/tiger-601/topic/timezone-deadline-time-robust-algorithm-and-refactored-baseline-algorithm-696/)
About your final submission, here we explain a bit how this works, also based on the rules that we defined at the beginning of the challenge and the discussions that we have had with some of you during the challenge, including during the webinars that we organized:
- each participant must submit ONLY ONE METHOD, meaning a single algorithm in a docker container; it is not possible to submit two different algorithms to the two leaderboards (even if they are packed into the same container); the only accepted difference between processing in L1 and L2 is in the post-processing of detection and segmentation, for example to filter out some detections when computing the TIL score, or thigns like that;
- if participants submitted to both L1 and L2 during the experimental phase, they have to pick a single method, and submit it to both L1 and L2 final;
- if participants submitted to only L1 or only L2, they have to pick one of the submtited methods and submit it to the corresponding final leaderboard;
- only one submission (of the same method) per leaderboard is allowed; should your submission not work, we will help you fix the problem, together with grand-challenge support;
- as detailed in the rules (https://tiger.grand-challenge.org/Rules/), you should attach a description of your algorithm to your final submission; for this, we have enabled the submission of an additional file, which can be based on the method template that we shared with you, or can be in the form of a short conference paper (for example, using the two-column IEEE paper template);
- as detailed in the rules, the top-3 algorithms should make their solution publicly available as open-source code; since the top-3 methods are not known at the time of submission, we enabled an additional field to add a URL, but we did not make it mandatory, so it can be updated later. However, we would like to encourage participants to make their solution open-source anyway, to foster the development of AI for quantification of TILs within the scientific community, and therefore already add a link to their repository with your final submission.
Finally, looking already beyond the final results of this challenge, in the past few months we have been working on preparing additional experiments on several external datasets of breast cancer cases (both surgical resections and biopsies) from multiple sources, touching upon multiple clinically relevant questions, such as prognosis (disease-free survival, as in the challenge, and overall survival), but also comparison of AI-based TILs and visual TILs by (panels of) pathologists, and prediction of treatment response. The idea is to include results on these additional cohorts in the paper that we will write with the results of this challenge. As mentioned before, we have been working on writing this paper already for some time, aiming at a quick submission within a few months after the end of TIGER. For this reason, as stated here (https://tiger.grand-challenge.org/Rules/), we will invite the authors of top algorithms (at least the top-3) to co-author this paper, and in order to run algorithms on these external datasets, which will be done mostly on AWS, we will ask the authors to authorize us to use these algorithms (essentially, to make us editors of the algorithm, as some of you did during the challenge to allow us to check for technical issues in your submissions). We envision that this will happen in the form of a research collaboration between us and the authors, which we would like to agree on with you as soon as the final results are announced.
About the latter, although TIGER is not attached to any conference, we would like to organize a web event (similar to webinars that we organized before) where we will ask you to present your solutions and where we will announce the results. Because of the recent technical issues with grand-challenge, we had to change our planned deadlines, and assuming that the final evaluation runs smoothly, we will decide on a date for this event and communicate this with you in the next few days.
But now, it's time to get ready for your final submissions, please check the submission pages and let us know if you have any questions or problems.
We wish you good luck with your final submissions, and thanks again for participating in this challenge, probably not the easiest one, but with the potential to make a significant impact on breast cancer treatment via AI-based TIL quantification.
Looking forward to your results!
Regards, The TIGER orgaizing team
