Success 107 times then failed

On Sept. 12, 2023, at 2:10 p.m., the algorithm I submitted during the Final Testing Phase succeeded 107 times and then failed. I would like to obtain the log information. This algorithm had a successful submission during the Prelim Testing Phase on Sept. 12, 2023, at 12:49 a.m.

Your submission was running correctly but ran out of the time limit set by grand challenge for each algorithm run (which is 1 hr). The allotted time does seem more than sufficient for inference purposes on the final test set so please ensure your code is running efficiently. We have asked grand challenge support to increase time limit already but we are not sure when and how much of an increase would be possible. If the time limit does increase, we will make an announcement regarding it right away.

Best, SurgToolLoc Organizing Committee

Unfortunately the limit for inference cannot be increased and there is a hard limit of 1 hr from grand challenge. Though please note that this is 1 hr / video, which is more than sufficient for inference purposes. Please ensure that your algorithm is efficiently running since your submissions are all running out of time and erroring out.

Best, SurgToolLoc Organizing Committee

Moreover, the highest number of frames in a video within the test set is 3818 (~63 mins). The inference speed needs to be only slightly more than 1 fps for the longest video to stay within the 1 hr limit, which is very reasonable. We are not sure if GPU usage could be causing any issue, but it could be worth trying to run on CPU only to see if that makes any difference.

Thank you for your attention and response! I have modified my model deployment strategy, and the model prediction part (Executing Algorithm) has successfully completed. It predicted a total of 108 videos.