try-out succeeded, but submit failed

Hi

I upload my algorithm and try out in the stack1.mha data. The result is succeeded, with warnings. The output Json file is the same with my local result.

But when I submit the algorithm, the evaluation result is failed: the algorithm failed on one or more cases.

I don't know why, and I wonder how can I find out why my submission algorithm is wrong.

Thank you~

Hello, wan77pc,

Thank you for your question.

Regarding the problem you encountered, I am guessing that the model is taking too long to predict the landmarks? Because the validation set contains 50 images and the test set contains 150 images, which is much more than the 2 images in the stack1.mha file.

Please also check carefully to see if the GPU is used and how long it takes to predict an image.

Hi

Thank you for your reply.

Is there a running time limitation in evaluation ?

I modified the algorithm workflow and the validation passed. But lost some accuracy. If not due to time limitation, I guess the algorithm is over-bounds accessing on some data.

Thank you~

Hello, wan77pc,

I sincerely apologize for the delay in responding to your inquiry. We have received a significant volume of email feedback from other participants, which caused the delay. Please accept my sincere apologies for any inconvenience this may have caused.

Regarding the Grand-challenge feedback, our configuration utilizes Amazon's ml.m5.large instance with a maximum GPU memory of 32GB, 2 CPU cores, and 8GB of RAM. The maximum allowed running time for processing 50 test images is 10 minutes.