Getting "The images are not unique, please submit a unique image for each case."

Getting "The images are not unique, please submit a unique image for each case."  

  By: ckoirala on July 16, 2023, 4:24 p.m.

Hello team, I have made some submissions to the Development Stage. My predictions are in the same format as the 3Label. I submitted a zip file containing the predictions. However, I encountered an error stating, "evalutils.exceptions.ValidationError: The images are not unique, please submit a unique image for each case." I have reviewed my predictions, but I couldn't find any duplicates or non-unique predictions. Could you please assist me with this issue? Is it possible to obtain a detailed log?

Re: Getting "The images are not unique, please submit a unique image for each case."  

  By: baorinawork on July 17, 2023, 4:24 a.m.

Hi Ckoirala, Can you try using the evaluation docker that I provided to first test your submission locally?( That evaluation docker is the same as the one that runs on the grand challenge) Can your submission be evaluated successfully locally?

Re: Getting "The images are not unique, please submit a unique image for each case."  

  By: baorinawork on July 19, 2023, 2:27 a.m.

Hi Ckoirala,

This error happens here in evaltuils.Validator:

class UniqueImagesValidator(DataFrameValidator):
"""
Validates that each image in the set is unique
"""

def validate(self, *, df: DataFrame):
    try:
        hashes = df["hash"]
    except KeyError:
        raise ValidationError("Column `hash` not found in DataFrame.")

    if len(set(hashes)) != len(hashes):
        raise ValidationError(
            "The images are not unique, please submit a unique image for "
            "each case."
        )

You may have the same two predictions (a special case is that two predictions all have 0s with the same shape). I have updated our evaluation docker to remove this check function, allowing the two predictions to have the same results.   You can use the evaluation docker to test locally or submit to grand-challenge again.

Thanks for the information!

Best regards, Rina

Re: Getting "The images are not unique, please submit a unique image for each case."  

  By: ckoirala on July 19, 2023, 4:04 p.m.

Thank you baorinawork. It is working now.