Is FROC computation image-wise or patch-wise?

Dear organizers,

I have a question regarding the FROC computation. Is the FROC score compute per image or per patch of image?

For each input image, the detection output is JSON file where "points" have all the "point"s for the detected TILs. However when inferencing the image, the image may be divided into patches and each patch is processed separately. Elaborating the above questions, is FROC computed per divided patches then all FROC scores are averaged as mean FROC or are points for each patch is concatenated into single list corresponding to detection of input image then FROC is computed?

Thank you for opening such interesting challenge!

Yours sincerely, Sungduk Cho

Hi,

the froc is computed for the complete dataset (the true and false positives and the false negatives are computed per roi and aggregated).

Dear Witali,

Could you please share the code that converts the json file per slide into predictions in each roi? Thank you very much!

Yours sincerely, Qinghe

Hi,

see the fct "read pred points probs" (with underscores) in the commit I just pushed for the froc_snippet.

Dear Witali,

Thank you for sharing the code! It looks like that the tps, tns, and fps are computed per slide and then aggregated. It should give the same results as per roi but perhaps slower? Please correct me if I've misunderstood.

Yours sincerely, Qinghe

On a similar thread, does this mean that the compute_scores function:

https://github.com/DIAGNijmegen/pathology-tiger-algorithm-example/blob/main/evaluations/eval_det_froc_snippet.py#L11

is meant to return the aggregated tps, fns and fps?

And if that is the case, does this mean that the area_mm here:

https://github.com/DIAGNijmegen/pathology-tiger-algorithm-example/blob/main/evaluations/eval_det_froc_snippet.py#L13

is the total tissue area across all slides?

Thank you for the clarifications!

Hi,

the tps, tns and fps are computed per roi (per slide) and summed up.

compute-scores returns indeed the aggregated tps, fns, fps. It gets as input a list of ground_truths and predictions (as arrays or lists, one per roi), computes for each the tp, fn and fp and sums them up. (this is done for each possible probability-threshold)

area-mm is indeed the total area of all rois.

let me know if there are more questions, Witali