Do We have the inference time limit in the final phase (testing phase)？

Hello, we wonder, for example, is that ok if it takes 72 hours for us to get all result in ouput directory? What is the time limit of the testing phase?

best regards daisy

Dear daisy,

Thank you for your good question. We believe that at this stage it is more worthwhile to pursue performance than to spend time, however, as you say, if the submitted model takes too extreme a time, this is also unacceptable. However, it is too late to set limits on the teams, so we decided not to limit the model computation time, and we hope that the teams will not need too many models for ensembling (which can sometimes lead to performance degradation) to maintain a reasonable inference time.

Best regards

Dear @jyuanfeng8, some guidance on what is considered reasonable for inference time is needed. Even a rough estimate would help out a lot. I think there exist very different opinions on what is reasonable: a couple hours? a day? a week? a month? What are you planning to do with dockers that run too long? It would be really frustrating to get disqualified because of this, especially because there hasn't been clear communication regarding that from the start ;-) Now there is still time for us to adapt our solution to these requirements. Best, Fabian

Hi, Issense, thank you for your information, We are discussing the matter and trying to come up with a reasonable solution.

Dear all, we have clarified the limitations of the docker submssion, please see https://grand-challenge.org/forums/forum/miccai-abdominal-multi-organ-segmentation-challenge-2022-660/topic/offical-model-size-inference-time-limit-on-docker-submission-950/. Sorry for uconvenience.