Surgical tool classification

Dear Organizers, I have the following questions about the Surgical tool classification task： 1. Is the first task to make a prediction for each frame in a fragment, or for the whole fragment? 2. If a clip corresponds to a classification true label with duplicates, should the predicted label also have duplicates? For example: clip_001876,its true value is [needle driver, nan, needle driver, cadiere forceps], so should the predicted label be [needle driver, needle driver, cadiere forceps], or [needle driver, cadiere forceps]?

Thanks again and best regards Ma

Dear ok@@@,

Thanks for posting your question.

1. The predictions would need to be made for each frame within the fragment.
2. You will only be required to predict the tool once even if there are two of the same tools present in the image.

Exact details regarding the output format for both categories along with submisson instructions will be uploaded on the website within the next few weeks so please keep checking the website for updates.

Best, SurgToolLoc 2022 Organizing Committee

Dear Organizers,

I have a follow-up question. The training data is provided as video clips of 30 seconds, and we are required to develop models that detect tools in each frame. Will the test set have same data structure (clips of 30 seconds) or just images? This is important to know in case of developing temporal models.

Best regards, NourJL

Dear NourJL,

The testing data will consist of similar videos to training data at 1FPS and the teams are required to produce outputes per frame. The videos in the testing set will be of variable length, however they will be temporally consistent (example: one video from a whole surgical task).

Best, SurgToolLoc 2022 Organizing Committee

Dear Organizers,

Thank you for your reply.

As far as I understood, The input of the models in the evaluation phase will be video clips. So, do the test videos have 1 Hz frequency (1 fps), OR they have 60 Hz, and the outputs should be produced only at 1 fps? If the second one is correct (60 Hz), can the model employ temporal information from intermediate frames (frames that won't be tested)?

Should the tool classification be performed online? OR information from future frames can be utilised?

Best regards, NourJL

Dear NourJL,

The test videos will be at 1Hz (1fps) and you will be required to produce a prediction for each frame in that video.

The tool predictions can be made offline - so future frame information can be utilized when making predicitons. However, please note that you will have to access to only a video path within the predictions function in the algorithm container - so you will need to figure out logic of reading and using future frame information for submission.

Best, SurgToolLoc 2022 Organizing Committee

Dear organizers,

Thank you for your reply!

I have a follow-up question. Are the testing labels also noisy as training labels? In other words, were testing labels obtained using inforrmation from the robotic system OR by annotating each frame separately?

Best regards, NourJL

Hi NourJL,

Thank you for your question.

The labels for the test set will not be noisy, as the test set will be annotated for grount truth bounding boxes (for category 2) which will be used for tool presence labels as well (category 1).

Best, SurgToolLoc 2022 Organizing Committee