Submission still executing

Submission still executing  

  By: Andreas on May 4, 2022, 7:11 a.m.

Hello,

We submitted one algorithm for leaderboard 1, 4 days ago, but it's still running. Do you know if this is normal, or whther our running script is too slow, or if there's another problem? We can make it faster still if necessary.

Mart is an editor if you need to poke around.

Best, Andreas

Re: Submission still executing  

  By: mart.vanrijthoven on May 4, 2022, 8:57 a.m.

Dear Andreas,

Thank you for the notification. I have checked your submission, but I have not clue why it is still executing. I have contacted grand-challenge support and will get back to you.

Best wishes, Mart

Re: Submission still executing  

  By: mart.vanrijthoven on May 6, 2022, 8:40 a.m.

Dear Andreas,

I am sorry to say that grand-challenge support has not found an answer to why your job was hanging. It has been canceled now. I checked, and only one image in the test set did not succeed. I also had a look at your logs. However, the logs for the images that succeeded are similar to the log for the image that did not succeed.

You might try to add information about your outputs in the logs, e.g., the number of predicted lymphocytes. Maybe that can give us a clue.

Best wishes, Mart

Re: Submission still executing  

  By: Andreas on May 6, 2022, 7:29 p.m.

Hi Mart,

Thanks a lot for your time and help. I think it might have just been too slow. We submitted a new one which should be much faster. If there was no error then it might have just been taking too long. Do you know if it was cancelled due to a time limit?

I intentionally removed all logging because i couldn't see the output myself when testing it. But perhaps i can plug it back in if the files can be accessed and reviewed. I will wait for the new submission which should take about a third of the time. And if there's another error I will submit one with logging so that we can debug.

I thank you, again, for your assistance on the matter.

Best, Andreas

Re: Submission still executing  

  By: mart.vanrijthoven on May 12, 2022, 7:21 a.m.

Dear Andreas,

Your submission was not canceled due to a timeout. I am still not sure what happened with that submission. But it seems your next submission succeeded, which is great!

Best wishes, Mart

Re: Submission still executing  

  By: arian.arab on May 26, 2022, 5:53 p.m.

Hi Mart,

I have encountered the same issue since yesterday. The submission is still executing. Could you please see what's the problem?

Thanks a lot. Arian

Re: Submission still executing  

  By: mart.vanrijthoven on May 27, 2022, 8:45 a.m.

Hi Arian,

I emailed you with this information but i also put it here for other participants to see:

This problem can happen due to GPUs not being available or hanging jobs. In the case of GPU availability, it should resolve when GPUs come available again. I contacted the grand-challenge support team about the hanging jobs, and they mentioned they would fix these jobs. However, because yesterday was a holiday, this can take more time than usual.

Best wishes, Mart