The algorithm failed on one or more cases.

The algorithm failed on one or more cases.  

  By: qqqwwwee on Aug. 24, 2022, 5:46 a.m.

Dear organizers,

My algorithm has passed the offline test with the provided "test.sh". But it failed after I submit it. Unfortunately, I can not find any error information about it.

May I ask how to solve this problem?

P.s. I have submitted some algorithms with the same way and all of them can pass the online test.

Thanks!

Re: The algorithm failed on one or more cases.  

  By: Marcel.Frueh on Aug. 25, 2022, 6:22 a.m.

Hi, as far as I see, your algorithms are running properly now, also in the online system. Is this correct?

Re: The algorithm failed on one or more cases.  

  By: ludovic on Aug. 26, 2022, 11:29 a.m.

Dear organizers,

I have exactly the same issue: my container is passing the the preliminary test but failing the final submission.

How do we get more info about failures?

Thanks, Ludovic

Re: The algorithm failed on one or more cases.  

  By: qqqwwwee on Aug. 28, 2022, 8:15 a.m.

Hi, organizers,

"Aug. 24, 2022, 12:23 a.m. Challenge Preliminary Test Set User Mugshot qqqwwwee adp Failed The algorithm failed on one or more cases."

This container has passed the offline test. However, it failed after I submitted. Could you please help me check why only this container failed?

Sincerely, Eugene

Re: The algorithm failed on one or more cases.  

  By: Marcel.Frueh on Aug. 29, 2022, 4:32 a.m.

Hi,

@ludovic: "The container ran out of memory." is the error message we receive based on your submission. Can you create a new algorithm container to which you assign more memory?

@qqqwwwee, I can only find a submission for the 23rd of August. This algorithm exceeds the time limit.

Thanks for participating Marcel

 Last edited by: Marcel.Frueh on Aug. 15, 2023, 12:57 p.m., edited 2 times in total.

Re: The algorithm failed on one or more cases.  

  By: qqqwwwee on Aug. 31, 2022, 2:42 a.m.

Hi organizers,

May I ask how I know the max time limit?

Or could you please cancel this limit?

Regards. Eugene

Re: The algorithm failed on one or more cases.  

  By: Marcel.Frueh on Aug. 31, 2022, 8:32 a.m.

Hi, the time limit is 20 min per subject. Unfortunately we cannot increase this limit. I will investigate the shapes which lead to the timeout so you can try cropping or resampling for these few subjects .

Re: The algorithm failed on one or more cases.  

  By: Marcel.Frueh on Aug. 31, 2022, 8:39 a.m.

Try resampling or cropping to a max of 550-600 axial slices if necessary.

Re: The algorithm failed on one or more cases.  

  By: ArMo on Sept. 1, 2022, 4:07 a.m.

Dear organizers,

I have the same issue too, may I know what kind of error I met? Is the algorithm exceed the time limit?

Re: The algorithm failed on one or more cases.  

  By: Marcel.Frueh on Sept. 1, 2022, 4:34 a.m.

ArMo: You error is: sitk::ERROR: Source image size of [ 350, 350, 287 ] does not match this image's size of [ 400, 400, 287 ]! or sitk::ERROR: Source image size of [ 343, 343, 275 ] does not match this image's size of [ 400, 400, 275 ]!

Re: The algorithm failed on one or more cases.  

  By: ArMo on Sept. 1, 2022, 5:18 p.m.

Dear organizers,

I am wondering if there is a part of images in the final test set, which have different size with the train set images' size: [400,400,H]. As is told above, some source images have the size of [350,350,287] or [343,343,275].

May I have the information of the shape of final test set?

Thank you

Re: The algorithm failed on one or more cases.  

  By: Marcel.Frueh on Sept. 1, 2022, 8:37 p.m.

Hi, all volumes differ in size. Some have only 300 axial slices and some have up to 650.

Re: The algorithm failed on one or more cases.  

  By: junma on Sept. 1, 2022, 9:44 p.m.

Dear organizers,

I got the same error. Could you please get back to me the details of the error info?

Re: The algorithm failed on one or more cases.  

  By: ArMo on Sept. 2, 2022, 2 a.m.

hi, thanks for your reply what about the size of axial slices, are they all 400 * 400 ? or they are all different?

Re: The algorithm failed on one or more cases.  

  By: Marcel.Frueh on Sept. 2, 2022, 4:02 a.m.

Hi, they are all different.

@junma, you algorithm is exceeding the time limit of 20 min per subject.