Re-evaluation of validation submissions and leaderboard positions ¶
By: evihuijben on July 17, 2023, 9:48 p.m.
Dear participants,
As previously mentioned (see post), we have re-evaluated all validation submissions made before July 14th, 11:30 CET, using the correct SSIM implementation. The average SSIM rank shift for Task 1 and Task 2 is now 7.0 and 6.7 positions respectively. However, the rankings based on PSNR and MAE remain the same as the implementation for these metrics was unchanged. As a result, there has been an overall average shift in leaderboard positions of 2.7 and 2.4 places for Task 1 and Task 2 respectively.
We understand the impact this may have on your submissions. To accommodate this situation, we have extended the validation period until the end of the testing period. We recommend that you carry out your final validation checks before submitting to the testing phase. Please note that the final scoring will take into account both image metrics (3) and dose assessment metrics (6), so focusing solely on the three image metrics during the validation set may not yield the best overall result.
We thank you for your understanding and patience.
Best regards,
Evi Huijben