impact on the results

When I output the result of task 1 in multiple-referable-glaucoma-binary-decisions.json and task 2 in stacked-referable-glaucomatous-features.json. Does the probability value in multiple-referable-glaucoma-likelihoods.json affect the outcome of the evaluation process?Or can we simply say that in the evaluation process, only when the probability value is greater than 0.5 can we enter the evaluation process of Task 2, or when the result of Task 1 is 1 can we enter the evaluation process of Task 2?

Hi Mamaa,

Thanks for your question. I'll try to answer it - I'm sure Yeganeh will correct me if this information is incorrect.

For task 1 (binary decision) you need to produce a likelihood for all images in the data set. It does not need to be a probability; when evaluating the results, the threshold will be adapted to produce the required 95% specificity and the corresponding sensitivity will be reported.

For task 2 (glaucomatous features), only the results on those images that are labeled as 'referable glaucoma' in the JustRAIGS data set will be evaluated. If you produce glaucomatous feature labels for images that have a 'no referable glaucoma' ground truth label, those will simply be ignored. So, I'd recommend to always produce these glaucomatous feature labels, even for images that have a low referable glaucoma likelihood according to your algorithm. In that way, even if your binary decision classification is wrong, you still have a chance of getting reasonable glaucomatous features labels.

Let me know if you have additional questions!

Best, Koen

Hi Mamaa,

Task 1: Referral performance

The algorithm will input the images and will output the binary labels (0 or 1) along with the likelihood.

No referable glaucoma: 0, Referable glaucoma:1

Task 2: Justification performance

The algorithm will output binary labels (0,1) for each ten additional features for only referable glaucoma. For example, if image is classified as referral glaucoma in task 1 then classification of ten additional features is required.

Feature is absent: 0, Feature is present: 1

For more information, please see the JustRAIGS GitHub at https://github.com/DM2LL/JustRAIGS-IEEE-ISBI-2024/tree/main/Example%20algorithm