About the ranking score

Hi everyone,

The final score is computed as S=0.25(DSC_FH+DSC_PS+DSC_ALL)/3.0+0.25[0.5(1-HD_FH/100+1-HD_PS/100+1-HD_ALL/100)/3.0+0.5(1-ASD_FH/100+1-ASD_PS/100+1-ASD_ALL/100)/3.0]+0.5(1-△AoP/180). Here, FH, PS and ALL, respectively, denote the region of fetal head, Pubic Symphysis and the whole image. Note: In our output: score=S; aop=△AoP; hd_fh=HD_FH; hd_ps=HD_PS; asd_fh=ASD_FH; asd_ps=ASD_PS; hd_all=HD_ALL; ads_all=ASD_ALL; dice_fh=DSC_FH; dice_ps=DSC_ALL; (Sorry for our mistake). There is no DSC_PS in the output, but it can be computed based on the above equation.

Jieyun Bai

Dear Dr. Bai,

Thank you for your previous response and the clarification you provided regarding the mistake. However, I would like to bring to your attention that I have observed illogical scores in my final submissions.

In both of my submissions, the aggregate values for the metrics have increased, indicating an improvement in performance. However, the final scores remained identical up to the 16th decimal place. This raises concerns about the precision of the code implementation of the score equation.

While it is theoretically possible to find a precise and specific value for dice_ps that satisfies both equations up to such a precise decimal place. It would be greatly appreciated if you could review the precision of calculations and verify if there are any potential errors or inconsistencies that could have led to the identical scores. For instance, are HD and ASD treated as floating points in the evaluated expression?

Thank you in advance,

Sincerely,

Marawan

Hi Marawan,

We have checked the code and there is no problem as you mentioned. In our ratings, AOP accounts for 50%, Dice accounts for 25%, HD accounts for 12.5%, and ASD accounts for 12.5%. Please evaluate your results overall.

Jieyun Bai

Thank you for your response. Would it be possible for you to kindly provide us with the correct precise value of the evaluation container output for the dice_ps of both submissions backed with the evaluation script? Having this information would enable us to calculate the final score accurately.

I am still curious to understand why the two submissions have the exact same scores up to 16 decimal places. Is it merely a coincidence that the dice_ps value for the first submission is 0.892866362535384, while the dice_ps value for the second submission is 0.88815708776772 (although all other segmentation metrics are better in the second submission)?

Thank you,

Marawan Elbatel