Update on submissions - Submitting to the final test set

Dear participants,

We have added a new blog post on the challenge website regarding the final submission to the MIDOG challenge: https://imi.thi.de/midog/2021/08/31/update-on-submissions/

If you haven't seen the blog post yet, please have a look on how to submit your solution to the final test set. Please note that you will to comit your final algorithm version to this phase yourself.

Looking forward to learning more about your approaches!

Best, Katharina

Dear participants,

submitting to the final test phase is now possible.

Best, Katharina

Hi, I wanted to submit for the final test phase but got an error "The algorithm failed on one or more cases". I used this docker container for preliminary test phase without any error. I also tried the algorithm and it showed the outputs. What should be the problem?

update: I tried to re-submit this docker container and I got this error: "A submission for this algorithm container image for this phase already exists."

hi, same problem here! Have you solved it? @salar.razavi

Unfortunately, no.

Dear salar.razavi,

we will look into your case.

@zerostarcraft: Your algorithm did run without any issues - note that we won't publish the results on the private test set until after the challenge.

Best, Katharina

got it, thanks.

Hi Salar Razavi,

we have had a closer look and your algorithm showed the error message for one of the images. /opt/algorithm/detection.py:29: RuntimeWarning: Degrees of freedom <= 0 for slice eigvals, eigvecs = np.linalg.eigh(np.cov(ODhat.T)) ... numpy.linalg.LinAlgError: Eigenvalues did not converge

Is there a way of how you can fix this aspect easily? I will talk to grand-challenge on how we can proceed in your case.

Best, Katharina

Hi, Thanks for your response. I have been using the "np.linalg.eigh" function in Stain Normalization to get eigen vectors for a long time and it is always working. It is so weird and I looked for multiple sources and in this short time I am not sure how to fix this problem.
It could be related to the compilation of the numpy with specific libraries. Is there anyway that I could run the algorithm again as I noticed it gives error sometimes. Best

Hello Katharina,

I have had the same issue of the algorithm failing on the final test set despite having worked on the preliminary test set. Could you look in to this for me please?

Thanks, Jack

Hello Katharina, I think changing the numpy with scipy to find the eigen value would be a solution. Could I submit my Container Docker again to check if it works? Thanks

Hi Jack, Hi Salar,

Jack: Your submission should now have gone through now - it is listed as "succeeded".

Salar: That might be an option - since I am not familar with the details of your method, I am not sure whether there is still a chance that the determination does not converge. Potentially you want to add a catch phase here and proceed with some default setting in case that it doesn't work. I will be in contact with regards to fixing your submission.

Best, Katharina

Can I create another algorithm and submit a new another Docker file? I cannot submit again with the previous algorithm. I am building a new container and was wondering whether I would be able to submit with a new algorithm?

Hi Salar,

Just got some additional confirmation from the GC-platform. It should be sufficient to upload a new, updated container to your current algorithm. Only successfull submissions count toward the submission limit. If there is no change to the algorithm, then the platform does not allow for a resubmissions (since it is very unlikely that the same code version will run through in the next iteration).

Kindly try to upload an updated container, submit again, and let me know whether it now works.

Kind regards, Katharina

Hi, I uploaded another docker container and it also failed on the final test set. Could you please look in to this again? It is so weird as the model is working with all other images. Thanks