Evaluation metrics

Dear Organizers,

Regarding the evaluation metrics, is there a script or recommended package to guarantee the consistency of submissions?

Thank you.

Hi,

You can use this: https://github.com/chaimeleon-eu/OpenChallenge/blob/master/ProstateCancerRiskPredictionEvaluation/evaluation.py

Hope we could help!

Hi,

Could we have a similar evaluation code for the lung type ? It is not so easy to understand Concordance Index (C-index) and Time-dependent Area Under the Curve (AUC).

Best,

Hi,

We'll upload to Github shortly a similar archive, but in the meantime you can use theses ones: https://scikit-survival.readthedocs.io/en/stable/api/generated/sksurv.metrics.concordance_index_censored.html https://scikit-survival.readthedocs.io/en/stable/api/generated/sksurv.metrics.cumulative_dynamic_auc.html

Hope we could help!

Hello Organizers,

The functions you have in evaluation.py file are a little different from what I have. I have two risk probabilities (probability of class 0 and probability of class 1). Is that okay or should I change my script according to your function in evaluation.py ?

Hi,

You must give only one probability. Hope we could help!