LUMIR - Test Phase Docker Submission

Dear challenge participants,

The test set consists of 590 images, making it impractical to distribute and collect the deformation fields. As a result, the test set will not be made available to challenge participants. Instead, participants are required to containerize their methods with Docker and submit their Docker containers for test phase evaluation.

Detailed instructions on how to build a Docker image for your model are provided here. You can find an example for building a Docker image for TransMorph on our GitHub.

If you have any questions, feel free to reach out to me at jchen245 [at] jhmi.edu.

Best, Junyu

Dear Junyu,

I have a few questions regarding the Docker submission. If I understand correctly, the fixed and moving image pairs are selected using the provided JSON file. However, the Docker submission should work for both validation and test data, as you mentioned that "we will compare your validation predictions with the ones obtained on our local machine."

My questions are: how is the key, such as "validation" or "testing," selected to filter the JSON file content? Should we include an argument in the Docker run command to specify the correct phase? Additionally, what is the correct key for the test data? Is it "testing"? This is unclear since all the examples provided use the key "validation."

Thank you for your clarification.

Best regards, TS (Spoderman)

Dear Spoderman,

Thank you for bringing this up. The template Docker image we provided was designed to minimize the effort for participants in building their own Docker images. This allows them to easily test their containers on the validation data, while also enabling us to verify everything is functioning correctly. During the actual testing on the test set, the "validation" field in the JSON file will be updated with image pairs for the test set, and the scores will be calculated accordingly. So, it's perfectly fine to keep the template script's data-reading process as is.

I apologize for any confusion, and I hope this clears up your questions.

Thanks, Junyu