Format of corners in category 2

Format of corners in category 2  

  By: SKJP on Sept. 1, 2022, 10:33 a.m.

Dear Organizers,

I would like to confirm the format of corners in category 2.

In the submission instruction (https://surgtoolloc.grand-challenge.org/submission/), the first corner should be "bottom-right" as

"corners": [             [ 92.66666412353516, 136.06668090820312, 0.5],             [ 54.79999923706055, 136.06668090820312, 0.5],             [ 54.79999923706055, 95.53333282470703, 0.5],             [ 92.66666412353516, 95.53333282470703, 0.5]         ],

In the example codes (https://github.com/aneeqzia-isi/surgtoolloc2022-category-2), the first cornder should be "top-left" as

(from line 110 to 120 in process.py) # bbox coordinates are the four corners of a box: [x, y, 0.5] # Starting with top left as first corner, then following the clockwise sequence # origin is defined as the top left corner of the video frame num_predictions = 2 predictions = [] for n in range(num_predictions): name = f'slice_nr_{frame_id}_' + self.tool_list[n] bbox = [[54.7, 95.5, 0.5], [92.6, 95.5, 0.5], [92.6, 136.1, 0.5], [54.7, 136.1, 0.5]]

Which is correct ? I tried both cases by changing the corner order only. I got the mean mAP > 0 with "bottom-right" format and the mean mAP = 0 with "top-left" format.

Best regards, Satoshi

Re: Format of corners in category 2  

  By: aneeqzia_isi on Sept. 1, 2022, 9:40 p.m.

Hi Satoshi,

Thanks a lot for bringing this discrepency out!

The correct format is "top left" based on your terminology - which is in the sample github repo's. The one in the submission instructions was copied over incorrectly from the GC documentation but has been corrected now.

We are currently trying to check a few things in the evaluation container of C2 that may be causing inaccurate calculations of the mAP value. We will make an announcement regarding it accordingly if there is any change.

Best, SurgToolLoc 2022 Organizing Committee