Data evaluation

Data evaluation  

  By: noelie.debs on Aug. 29, 2022, 4:06 p.m.

Hi,

Knowing that some detection models can return bounding box outputs (e.g. nnDetection), in what form should the ground truth of these models be? Do we have to build bounding boxes around each lesion of the ground truth? or is it the dense contours of the lesion directly?

And is the IoU (overlap to determine whether there is a match or not) the same when it is a bounding box model or a dense contour model ?

Best regards,

Noëlie

 Last edited by: noelie.debs on Aug. 15, 2023, 12:57 p.m., edited 1 time in total.

Re: Data evaluation  

  By: joeran.bosma on Aug. 30, 2022, 2:36 p.m.

Hi Noëlle,

For a given case, we expect all submitted AI models to predict a csPCa detection map with the same spatial dimensions and resolution as the T2W image. We will evaluate these detection maps against the dense lesion contours, no matter the underlying model (e.g., nnU-Net or nnDetection). So indeed, the IoU computation is not changed for bounding box predictions.

To train a detection model that returns bounding boxes, the ground truth label format varies from framework to framework. For nnDetection, the training labels should be dense lesion maps, in a similar structure as nnU-Net. You can find details on this in the documentation of nnDetection. The label format of nnDetection is more general than that of nnUNet, but in the case of PI-CAI, all ground truth lesion annotations are distinct non-touching structures. Because of that, the nnDetection label format can be derived from the nnU-Net label format. We have done so in the nnDetection baseline, so you can follow that approach.

Please note that for the nnDetection baseline we have converted the bounding box predictions to simple cubes. This is by no means optimal, it is likely that converting the boxes to e.g. spheres would provide a better match with the ground truth lesion contours. We have left this to the participants to determine.

Hope this helps and let me know if you have any other queries, Joeran

Re: Data evaluation  

  By: noelie.debs on Aug. 30, 2022, 3:53 p.m.

Thank you for your very clear answer ! Best,

Noëlie