Validation Docker Submission Failed

Dear challenge organizers,

We have met a problem when submitting the docker image: "You have an existing container image importing, please wait for it to complete".

One of our submitted dockers are still in the status of "Import Status Initialized", It seems it stucked and we cannot move on.

May you kindly do us a favor to check this problem and help us ni this validation phase?

Best regards,

You might try creating a new algorithm in the meantime where you can upload your new docker container. Maybe this will help and the suck import will time out eventually.

When you upload a Docker image, it takes some time to initialize and get ready for evaluation against the validation dataset as defined here. After successfully uploading a Docker image, please wait for some time (from a few minutes to an hour at maximum) and keep refreshing the page to see the progress.

In your case, your Docker image should be ready to test its performance against the validation data. You just need to click on the submit icon as described here, then from the dropdown menu, select your uploaded algorithm and run it.

If you wish, you can upload a new Docker image and test it against the validation data. However, please note that it will take some time for the initialization and for the Grand-Challenge server to process your Docker image and make it ready for evaluation.

I hope this helps! Let me know if you need any further adjustments.

Thank you for your reply, however, we have met the problem 'The algorithm failed on one or more cases.'

May you do us a favor to provide more details on this problem?

Best regards,

Hi,

For the log of your algorithm, please check your inbox on the Grand-Challenge website.

Additionally, I have a tip that might help: when using sliding window inference, try transferring the data to the CPU instead of the GPU. You can do this as shown below:

sliding_window_inference(image, (patch_size, patch_size, patch_size), num_samples, model, device='cpu')

I hope this resolves the issue.

Hi,

Thanks for your kind reply.

However, if using CPU instead of GPU, the inference time will be very long which may not satisfy the requirement (5 min / per case).

In addition, the local test is well (using the provided tesh.sh provided in the github), within 32 GB CPU memory and 16GB GPU memory, however, it failed when submitting to the platform.

Best regards,

Using a CPU instead of a GPU can indeed lead to longer inference times, but it should not cause the inference time to exceed the 5-minute requirement per case. However, this is just a suggestion and not the solution to your error.

To help diagnose the issue with your code, we have shared the error log from your algorithm in your Grand Challenge account inbox. Reviewing this log should provide more insight into what might be causing the submission to fail on the platform.

It’s worth noting that many participants have successfully submitted their Docker images multiple times, achieving a Dice score of 0.75. This indicates that with properly written code, it is possible to run your Docker image successfully and obtain good results.

Thank you for your kind help. We will try to figure it out.

One confusing thing is that we tested our prepared docker successfully, followed by here: https://github.com/ImranNust/AortaSeg24/tree/main/Docker_Preparation_For_Submission, and run ./test.sh. The local log showed that the program is within 16GB GPU memory, and 32 GB CPU memory. However, we fail to obtain results when submitting to the platform.

Another concern is that you conduct careful memory management in the provided link. We wanna know whether it is necessary for this challenge?
For example" "del model # to save some memory" ... "gc.collect() "

Thank you so much for your interest and for participating in the challenge. We hope to see some good results from you.

No, you don't need to use del model or gc.collect() if you don't deem them necessary. Many participants who followed our guidelines submitted their work and obtained good results. If you diligently follow the guidelines, we hope you will be able to achieve good results as well.