Possibly conflicting Evaluation Metrics

Possibly conflicting Evaluation Metrics  

  By: xleratorxlerator9 on July 6, 2024, 4:01 a.m.

The metrics used in the score function used for ranking may be conflicting. Namely,

  • NAE_ac : lower is better
  • DSC : higher is better
  • WFSS: higher is better

hence the composite score: Score = 0.5NAE + 0.25DSC + 0.25*WFSS may not rank better algorithms higher.

 Last edited by: xleratorxlerator9 on July 6, 2024, 4:02 a.m., edited 1 time in total.

Re: Possibly conflicting Evaluation Metrics  

  By: sofisappia on July 8, 2024, 1:36 p.m.

Thank you for pointing this out. You are completely right. I have updated the evaluation code to reflect this:

score = 0.5 (1 - NAE_AC) + 0.25 DSC + 0.25 WFSS

I am rerunning algorithm evaluations with this updated formula.