Dear AIROGS participants,

Change in algorithm input/output requirements We have made a change in the requirements for the algorithm input/output. The algorithm containers previously needed to process a single image. However, because of the relatively large amount of cases in our test set, this caused resource-related issues on the Grand Challenge evaluation platform.

It is now required that a single run of a Docker container can process multiple fundus images. Since the Grand Challenge evaluation platform can technically only process one image, we employ a workaround in which we put multiple fundus images into a single file. We use a stacked TIFF file that includes multiple color fundus images.

Example template update The GitHub repository with example algorithm has been updated to process files in the expected way and contains an example TIFF file in /test/images/phase_1.tiff with 10 color fundus images (see here).

Teams that already submitted their algorithm to Preliminary Test Phase 1 need to update their algorithm according to the new requirements (preferably pulling the changes from the updated GitHub repo).

All images within the TIFF stack need to be processed within 10 seconds. Per TIFF stack, an additional 5 minutes will be available to load model weights etc.

Preliminary Test Phase 2 Next to the Preliminary Test Phase 1, to which an unlimited number of submissions is allowed, Preliminary Test Phase 2 has been introduced and will open on February 1st at 12:00 pm CET. This phase contains more test cases and will also produce the same metrics as the final phase. A total limit of three submissions to this phase will be allowed. A week later, on February 8th at 12:00 pm CET, the Final Test Phase will open. Only one submission to this phase will be allowed.

All test phases will end on March 1st at 23:59 CET.

Best regards, Coen de Vente