Dose metric

Dose metric  

  By: Yuliang on May 5, 2025, 4:04 p.m.

Hi, I just spot some issues in the implementation of the dose metric.

On the challenge website, it is said that "In cases where an algorithm produces no output on a given frame with ground truth label available, the following default metric values will be used: DSC=0, HD95/MASD/CD=image size along the largest dimension in mm, dose set to zero for that frame, inference time calculated as in normal cases."

However, the code in the trackrad2025 repo are not implemented accordingly. More specifically, the DoseMetric class does not filter out empty predictions, which will cause center of mass to have null value after this line https://github.com/LMUK-RADONC-PHYS-RES/trackrad2025/blob/c5380c16b3e0b264ee8713d2c68de0c90cb463fd/evaluation/evaluate.py#L325C8-L325C98.

Could you please have a look at this issue? Many thanks!

Re: Dose metric  

  By: tom-julius on May 6, 2025, 8:17 a.m.

Hello Yuliang,

thank you for your interest in our challenge and spotting this issue.

You are correct that the array of center of mass locations contains null (or more precisely NaN) values after this line in case of empty predictions.

Luckily, this is not really a problem, as that array is only ever used for computing the shifted dose distribution. And the scipy.ndimage.shift function used to shift the dose produces a zero result in case of nan shifts, which is the behaviour we want to achieve.

But I still agree, that it is not an elegant solution to rely on the edge-case handling of a third-party library. So I'll add explicit handling of this case and add some comments to the source code.

Thank you for bringing our attention to this.

Sincerely, Tom

 Last edited by: tom-julius on May 6, 2025, 7:27 p.m., edited 1 time in total.