lung task: The algorithm failed on one or more cases

lung task: The algorithm failed on one or more cases  

  By: junma on Jan. 14, 2024, 12:49 a.m.

Dear organizers,

Could you please let me know the docker error log of our lung validation task submission?

I tried the algorithm by uploading a case but got "unexpected error, something went wrong". there is still no detailed error log.

I'm also not allowed to create new algorithms. Is there any way to delete my previously failed algorithms?

 Last edited by: junma on Jan. 14, 2024, 12:50 a.m., edited 1 time in total.

Re: lung task: The algorithm failed on one or more cases  

  By: jhendriksen on Jan. 14, 2024, 2:23 a.m.

You don't need to delete your old algorithm and create a new one. From within your algorithm information page you should be able to click containers on the left and then choose to upload a new container which will allow you to try new versions of your code without creating new algorithm.

Regarding failures - with how difficult it is to get helpful logs from Grand Challenge when trying the algorithms in the cloud I found it was much easier to copy a set of test images and metadata to the docker image in the same format as they should be on Grand Challenge and then run the docker image locally. This way you can add as much logging as necessary to see why your image is failing and it won't be hidden from you in any way.

Re: lung task: The algorithm failed on one or more cases  

  By: junma on Jan. 14, 2024, 3:05 a.m.

Thank you so much for your great suggestions.

My docker works under the following input

        self.image_input_dir = "chest-ct" # "/input/images/chest-ct/"
        self.image_input_path = list(Path(self.image_input_dir).glob("*.mha"))[0]

        # load clinical information
        # dictionary with patient_age and psa information
        # with open("/input/clinical-information-lung-ct.json") as fp:
        with open("chest-ct/case_0094.json") as fp:
            self.clinical_info = json.load(fp)

I submitted a new docker but still got the error: The algorithm failed on one or more cases

Could you please let me know the docker error logs?

Re: lung task: The algorithm failed on one or more cases  

  By: anajimenez on Jan. 14, 2024, 9:59 a.m.

Dear @junma,

It seems that the problem is when checking the content of 'overall-survival-months.json' file. Grand Challenge platform rejects is as valid with the following message: "The output file 'overall-survival-months.json' is not valid"

Can you double-check that you are storing a float number in the JSON file? I recommend you cast it to a native python float number (with float(your_number)) since it seemed to cause problems to other participants.