Time limit per video increased to resolve time limit issue and deadline extension to 8th Sept

Hi teams,

The time limit for each container run on a video has been increased to 1 hr (from 20 mins) which should be more than enough for algorithm runs for all teams for the longer video as well. Please re-submit your containers on the main phase if your container errored out due to time limit exceeded. We have extended the deadline by another day to 8th Sept. At the end of this deadline, we will take the last successful run of each team as the final submission to finalize the leaderboard.

Best, SurgToolLoc 2022 Organizing Committee

Thank you aneeqzia_isi

"we will take the last successful run of each team as the final submission to finalize the leaderboard"

Just to clarify, you'll use the last one of the two, not the highest of the two submissions? I.e. if the 2nd submission scores lower than 1st submission, it will overwrite the 1st submission?

Thanks, Bo

By taking the final submission, the teams that submitted previous to this extension get to pick the "best of two" from their submissions, whereas the other teams have to hope their final submission is the better one.

No one can be 100% sure that his last submission was the best unless he knows what the test set was like. Common sense, we want to take the best grade as our final result. In addition, our current proposal has been successfully submitted, proving that the current test set is available, and whether other teams should bear the time-consuming risk to some extent themselves, rather than blindly appealing for postponement.

The point of picking the last successful submission was primarily meant to be in the case IF we would have removed the long video data sample from test set (in which case we would have needed to run all submissions again for fair comparison with already successful submissions). Since we did not do that, we will take the best submission for each team in finalizing the leaderboard. This is conditional on timely submission of methodology report.

There were some bugs on our side, while some things were on GC side which were not in our control. We felt this warranted flexibility from our side in extending deadline by a few days so that we can also help teams as much as possible in making their final submissions. However, there will be no more extensions now and the leaderboard will be finalized right after the final phases end.

Best. SurgToolLoc 2022 Organizing Committee

Thank you for your patient answer. We now understand your original intention . Best Wishes!