Summer Newsletter 2022

Published 8 July 2022

🛠️ New features of Grand Challenge¶

Summer started early for Grand Challenge with fresh new developments and how could that be better than by improving our very own Challenges?

We further explored the strength of Type 2 challenges, where the algorithms submitted by the participants remain directly accessible on our platform after the end of each challenge. More information regarding Type 2 challenges, can be found in our challenge documentation. We highly encourage challenge hosts to share the outcomes of their challenge and their experiences with Type 2 challenges in the form of a blog post!

We've updated our rules for hosting your own challenge. Read more about our new request procedure which includes an automated form for challenge cost estimation.

Read up about other new features in our latest development blog post.

💡 Blogposts¶

“AIROGS Challenge Report: AI models can be used for glaucoma screening, but do they know when they cannot?”¶

Recently, the results of the challenge “AIROGS: Artificial Intelligence for RObust Glaucoma Screening” were presented at ISBI 2022. Participating teams presented their artificial intelligence (AI) solutions for robust glaucoma screening on which they had been working for the past four months. In this blog post, the rationale behind the challenge is presented, an overview of the results is given and the experiences with the Type 2 challenge features that were recently introduced on grand-challenge.org are discussed.

“One year of MDR: 17% of products comply”¶

It has been a year since the more stringent Medical Device Regulation (MDR) replaced the former Medical Device Directive (MDD). With two years left on the clock for the 'transition period', only 17% of the AI products for radiology have obtained clearance under the MDR so far. If the market continues at this pace, half of the current CE marked AI products for radiology will not comply in time. The likely consequence is that software will be taken off the market and can no longer be used clinically after May 2024. Does that mean users stop buying MDD compliant AI software?

🔦 Highlighted algorithms¶

CORADS-AI

This algorithm analyzes non-contrast CT scans for COVID-19. The algorithm includes the following features: lobe segmentation, CO-RADS scoring, GGO and consolidation segmentation. The algorithm has an AUC of 0.88 for differentiating COVID-19 positive and negative patients. Try out this algorithm with your own data!

Pulmonary Nodule Malignancy Prediction

This is a deep learning (DL) algorithm based on convolutional neural networks (CNNs) that accepts a low-dose chest CT and the coordinates of a pulmonary nodule as input to estimate the malignancy risk of the nodule. This algorithm was developed with pulmonary nodules from the National Lung Screening Trial (NLST) and validated externally with pulmonary nodules from the Danish Lung Cancer Screening Trial (DLCST). The AUC for the whole test cohort is 0.93. Try out this algorithm with your own data!

🚀 Upcoming and ongoing challenges¶

Currently, there are two types of challenges on Grand Challenge. A type 1 challenge is where the training data (including ground truth) and testing data (excluding ground truth) are made publicly available with only the ground truth files for testing data to be kept secret. Participants are expected to upload predictions of the publicly available testing data, and Grand Challenge automatically evaluates the predictions by comparing them with the secret ground truth labels. In a type 2 challenge, participants submit an algorithm as a solution. This algorithm is then run on a secret test set (uploaded to an archive by the challenge admins) on the Grand Challenge platform. The results that the algorithm produces are subsequently evaluated using a custom evaluation method provided by the challenge admins. We encourage all challenge to run as type 2 challenges.

🥅 Goal: To benchmark AI against radiologists at prostate cancer detection in MRI, using the largest public training dataset and private testing cohort reported in prostate-AI literature till date
✍️ Register: Accepting AI algorithms submissions now for Open Development Phase - Validation and Tuning
⏰ Closing: 20th of October, 2022, is the closing date for Open Development Phase - Validation and Tuning submissions
🏆 Prize: To-be-announced

🥅 Goal: The objective of AI is to evaluate automatic tumor lesion segmentation in whole body FDG-PET/CT for accurate and gast lesion segmentation and avoid false positives
✍️ Register: Accepting submissions now
⏰ Close: 31st of August, 2022
🏆 Prize: 

€6.000,-
€3.000,-
€2.000,-
till 7th place: €1.000,-

🥅 Goal:

The goal of the segmentation task (Task 1) is to segment two key brain structures (tumour and cochlea) involved in the follow-up and treatment planning of vestibular schwannoma
The goal of the classification task (Task 2) is to automatically classify high resolution T2 images with VS according to the Koos grade

✍️ Register: Accepting submissions now
⏰ Close: 5th of August, 2022
🏆 Prize: Not applicable

Additional to these challenges, more challenges can be found here.

🏆 Leadboard of finished challenges¶

The goal is to develop algorithms that perform segmentation, classification and counting of 6 different types of nuclei within the current largest known publicly available nuclei-level dataset in CPath, containing around half a million labelled nuclei. The winners for the segmentation and classification track are Martin Weigert (mPQ+: 0.50132), Elias Bauman (mPQ+: 0.47616) and Winnie Zhang (mPQ+: 0.46310). The winners for the cellular composition track are Winnie Zhang (r2: 0.76413), xx wang (r2: 0.76250) and Martin Weigert (r2: 0.75498). Congratulations!