STOIC2021 results webinar
Published 12 Oct. 2022
The final results for the STOIC2021: A COVID-19 AI challenge are in! In this challenge that was held with 10,000+ CT scans of the STOIC project, participants predicted from a CT scan who will develop severe COVID-19, defined as intubation or death.
Please join the webinar through Zoom on October 18th at 3PM CEST in which the results of the challenge will be disclosed and the winning teams will give a presentation, reveal their secret sauce, and receive their prizes!
An improved challenge format¶
STOIC2021 implements a new AI challenge format. Earlier medical image analysis challenges follow challenge formats (called Type 1 and Type 2) where participants develop and train their solutions with public training data. In STOIC2021, participants still developed their solutions with a public set of training data, but instead of training their final solution themselves, they submitted a training codebase to the challenge organizers who trained these systems on a very large secret training set. We call this challenge format Type 3.
With Type 3 challenges, the challenge organizers can train the challenge solutions on much bigger data, including data that cannot be publicly released. The submitted training code bases will also be made publicly available with an open-source license for downloading and re-use after the challenge.
STOIC2021 consisted of a Type 2 Qualification phase in which the finalists were selected, followed by a Type 3 Final phase where 8 finalists competed for top 3 to win AWS credit prizes totaling $20,000.
Qualification phase results¶
During the Qualification phase, participating teams developed solutions using 2,000 CT scans that were made publicly available at the Registry of Open Data on AWS. They submitted solutions in the form of grand-challenge.org Algorithms. 68 participants competed in 30 teams. Combined, they developed and successfully submitted 119 solutions. These submissions and their performance can be found on the Qualification Leaderboard.
20 teams competed for admission to the Final phase of the challenge by making a single submission to the Qualification (last submission) Leaderboard. The finalists were the best performing 8 teams that accepted a Final phase invitation.
Final phase results¶
In the Final phase, these 8 finalist teams submitted a single code base for training and inference. This code base was trained by the challenge organizers on a secret training set of 9,724 CT scans.
The results of the Final phase will be disclosed during the STOIC2021 Webinar. The top 3 finalist teams will present their winning solutions. Feel free to join the webinar through Zoom on October 18th at 3PM CEST!
Webinar Schedule¶
3.00 | Welcome and introduction to STOIC2021
3.05 | The STOIC Project: clinical background
3:10 | Award ceremony
3:15 | Team #3 Solution: how did they do it?
3:20 | Team #2 Solution: how did they do it?
3:25 | Team #1 Solution: how did they do it?
3:30 | Q&A
3:40 | Outline of the paper, next steps & closing
