Variance-balanced accuracy?

Hello again,

It seems I am unable to find the definition of this metric in your eval repo:

https://github.com/CoWBenchmark/TopCoW_Eval_Metrics/tree/master/topcow24_eval/metrics/edg_metrics

I could not find a definition elsewhere either. Maybe this refers to per-class balanced accuracy?

Thanks,

Adrian

Hi Adrian,

Thank you for your question.

It is "variant" (as in CoW anatomical variants in terms of graph topology). The variant-balanced-accuracy is calculated during the aggregate stage. We use sklearn's balanced accuracy to compute. The aggregate calculation is specified here:

https://github.com/CoWBenchmark/TopCoW_Eval_Metrics/blob/master/topcow24_eval/aggregate/aggregate_all_graph_dicts.py

Here is the test file with the expected output and example usages:

https://github.com/CoWBenchmark/TopCoW_Eval_Metrics/blob/master/topcow24_eval/aggregate/test_aggregate_all_graph_dicts.py

Best, Kaiyuan

Thanks Kaiyuan, it makes sense. Nice work with the evaluation, it's obvious that lots of thought went into that repo.

Adrian