Issues with three cases in validation set fixed

Hi everybody!

Three cases in the validation set (E_P000004, E_P000005, and E_P000012) had some issues —shout out to wildsquirrel for noticing and letting us know. It took us a while to figure out what was causing them because on our end in ASAP, they did not show up, but only on GC.

I'm going to request a re-evaluation of all the submissions with GC. Hopefully, this will happen tomorrow or Tuesday. The good news is it means that your performance metrics are currently lower than they should be, so expect a nice jump in performance after the re-evaluation.

For anybody who's had issues with submissions that run well on the debugging but fail on the live leaderboard, this will hopefully fix that, too (I've manually rerun one case, and it indeed did fix it).

The corrected data has been live since yesterday, so any submissions you make from now on will show the correct performance.

Let me know if there are any further issues!

Happy holidays! Linda

Hi Linda,

Have all the submissions been reevaluated by GC yet? We were wondering if there have been any changes to the logs we receive, as our algorithm is still failing the leaderboard.

Thanks and happy holidays!

Thank you for fixing the issues! I'm also wondering when will re-evaluation happen?

No they have not been re-run yet. GC told me that it will take a bit longer due to the holidays as most people are not working at the moment.

I'll post an update here. I think you should also get a notification that there is a new result for your algorithm once it's done.

@Irem: I see that your latest submission succeeded so I hope you're all good now?