Task 3 Continuation Information + New Dataset + Baseline

Dear Participants,

we want to share some new information on the continuation of the L2R Hidden Task Challenge:

New Timeline: October 28th: Submission of algorithms for sanity checks (Upload Link) Participants are encouraged (but not enforced) to submit their self-configuring algorithms for sanity checking until October 28th. We will provide feedback in the following week. November 9th: Q&A / sanity check feedback meeting We will offer another Q&A meeting regarding questions which may have surfaced while implementing your solution and potential technical caveats which occurred during sanity checking. November 28th: Final submission of algorithms Participants who submitted to the sanity check will receive 3 chances to correct their final submission if it fails to produce reasonable results. Teams who did not participate in the sanity check will be given only one opportunity to correct their final submission after deadline.

Dataset information we would like to share: * We will evaluate on 3 Tasks (H1: Lung; H2 Abdomen, H3: Brain) * Please expect the possibility of large datasets * H2,H3 are MRI Images, normalised (by clipping intensity values between 0,3500), followed by deviation by standard deviation) * Datasets might originate from mixed sources, so random sampling is highly enrouraged * Label image might not include all possible labels. Please be aware that a label could be missing in fixed and moving label images (check your similarity function for unexpected outputs)

Our environment: * We will train and run your algorithms on an nVidia A100-80GB, CUDA Version 11.6. * OS: Ubuntu 20.04.4 LTS * CPU: AMD EPYC 7513 * 120 GB disk storage * If you need more information regarding our training/inference environment, please open a new topic on this forum. We will update this post accordingly.

Algorithm requirements: * We have provided a containerized example algorithm on our Github repository. However, as before we do not want to limit submissions to docker containers. If you'd like to submit source code, please add a valid requirements.txt to create an environment. * Be aware that test data will not be present at time of training. Therefore, participants will have to submit a script for training and inference respectively. * Algorithm training must be scaleable with respect to training duration, since we want to perform a sanity check after one hour of training. Your algorithm is expected to produce a somewhat reasonable output (i.e. deformation fields). If this check is successful, we will continue training for a longer duration. * We strongly encourage logging your algorithms activities, which will (hopefully) simplify debugging. * Please provide detailed information, including but not necessary limited to: - preferred and complete contact information (i.e. possible multiple recipients) - installation, training and inferring of your algorithm (i.e. Docker commands) - algorithm setting for sanity checking (e.g. a certain number of iterations to produce results in under 1h training time)

To ease implementing a suitable algorithm, we have prepared a new dataset AbdomenMRMR: https://cloud.imi.uni-luebeck.de/s/PjnKjxyR9gXxFr8 (570 MB)

We have also published a simplified Self-Configuring Baseline (L2R-SCB), which serves as a baseline in our self-configuring experiments: https://github.com/MDL-UzL/L2R/tree/main/examples/l2r_SCB

Update 27.10.: Since it was not explicitly mentioned in the original post, I would like to point out that we provide automatically generated labels in the hidden datasets. Analogous to the folder structure ../labelsTs/.. these are stored under ../predictedlabelsTs/... Please do see the additional dataset AbdomenMRMR(see above) for a complete example of a hidden dataset we will use in inference.

The labels are generated by utilizing the nnUNet(https://github.com/MIC-DKFZ/nnUNet), a powerful self-configuring segmentation method.