Submission Error

Dear organizers,

My latest submission to the prostate classification track throws an error message ('The algorithm failed on one or more cases.'), could you please give me some clue for debugging it, regarding the GC log of the error? The container passes my internal tests, and I already had two other similar and successful submissions, so I have no idea what is going on.

It's my last one, created on Dec. 18, 2023, 3:07 p.m.. Thanks!

Adrian

Hi, unfortunately you cannot see the logs of your submissions, it's something that the grand-challenge platform does to protect the validation data. Hope we could help.

Hello, and thanks for your answer.

Yes, I know I cannot see the logs, but in other competitions the organizers, who could see them internally, used to give me a description of the error message to help me debug, copy pasting the parts that were helpful to understand what went wrong. Like, did I forget to move the model to the gpu? Or did I run out of memory? Things like that. I would appreciate a lot if you could help me, I have been looking through the code for a long time and cant find any mistake.

Cheers,

Adrian

I'm sure that Alvaro can provide specific information for you, but in the meantime you can also use the "Try Out Algorithm" button on your algorithm page. If you upload a representative case there you will get the logs and that will reveal problems with containerisation at least.

Hey @jmsmkn, thanks for the advice. I found out that when loading the json file the keys are not the same as in the training data, where we had 'age' during training, at inference we will encounter 'patient_age'. A bit confusingly, the other key, which was 'psa', does not need to be turned into 'patient_psa'. The try out thing has finished with a key_error, and I am rebuilding the container now to fix that. Later today I will reupload and resubmit, I am quite sure it will work this time.

Not sure if you are part of the GC team, but if you are or have access to them, and GC is willing to take feedback on the platform for future improvement, I would say that, in case the algo fails, providing with some technical logs to the participants so that we can debug without having to bother the organizers all the time would be a very nice addition. Thanks!

It worked, thank you very much. I also saw that after finishing the "Try out algorithm", you get a log with the errors, so I guess that works well for debugging. Maybe GC users should be made aware of this feature, I personally didn't know it and it works very nicely.

Cheers,

Adrian

I am one of the developers of Grand Challenge so cannot comment on the difference between released and internal data for this challenge.

in case the algo fails, providing with some technical logs to the participants so that we can debug without having to bother the organizers all the time would be a very nice addition

That is actually a feature we have just added, but Challenge administrators would need to opt-in to it as it makes the logs (and hence the data used) accessible to the Challenge participants. There will be a blog post about it in January. We will be encouraging future Challenges to use this as it's also oft-requested from Challenge administrators.

Maybe GC users should be made aware of this feature, I personally didn't know it and it works very nicely.

It should be documented in the help section.

Hi all, I'm getting the following message when trying the Try Out Algorithm: "Unexpected Error Something went wrong..." I'm not getting any logs, and my algorithm "Failed on one or more cases" upon submission. The container passes my internal tests. What may be the issue here?

Hello,

I am also getting that error message from the Try Out Algorithm functionality, with the lung survival algorithm. It is very hard to understand what is happening, with this sort of uninformative message. Any clues, please?

Adrian