One of our participants in the data confirmation phase pointed out a curious result, which turned out to be a bug in the evaluation code (details below if you are interested). The bug has now been fixed, and all the successful submissions so far are being re-run. Hopefully, the full updated leaderboard will be available later today.

The good news is that this is one of the reasons we are running the data confirmation phase - to iron out any bugs like these prior to the proper challenge. The bad news is the scores are likely to go down (even for the person that pointed out the error).

Bug details: The D95 is the dose for which 95% of the volume has a higher dose. However, when one takes the DVH as a vector sorted from lowest to highest dose, the dose representing the D95 is actually the 5th percentile in this vector, not the 95th percentile! Turns out the scoring code was erroneously calculating the D5, and evaluating this against the target for the D95. Consequently, very high scores were being achieved for this measure.