Failed submission

Hi

I had a submission on July 18, 2023, 9:12 a.m., that was failed with this error "The algorithm failed on one or more cases.". Could you please send me the error description, let me figure out why it was failed?

Thank you~

Hi

I encounter the same problem. Could you please give me some suggestions how to solve this problem?

Thank you~

Hi, I am facing the same issue of failed submission. Could you please reply if you have found a solution for this?

Hello, wan77pc,

Thank you for your submission. According to our records, your submission has encountered an error displaying "Time limit exceeded." It appears that the Docker container used for your submission may have taken longer than the allowed time for processing.

To ensure successful submissions, please consider optimizing the execution time of your Docker container or the underlying processes. Additionally, you may want to check for any inefficiencies or bottlenecks that could be causing the time limit to be exceeded.

If you encounter any specific issues or require further assistance, please don't hesitate to reach out. We are here to help and ensure a smooth experience for all participants.

Hello, wangc,

It seems that I couldn't locate any submission records under the name "wangc". To further investigate this matter, could you please provide me with additional information regarding your submission?

Please share the following details:

• The date and time of your submission attempt.

• Any error messages or codes you encountered during the submission process.

With this information, I'll be better equipped to assist you in resolving the issue and ensuring that your submission is properly recorded.

Thank you for your cooperation, and I look forward to hearing back from you.

Hello, kpkreddy,

Thank you for your interest in our challenge, and we appreciate your submission. However, upon reviewing the submission in the backend, we encountered an "AssertionError: Image must be non-zero" error. The specific traceback is as follows:

2023-07-30T09:05:00.539000+00:00 Traceback (most recent call last):
2023-07-30T09:05:00.539000+00:00   File "/opt/algorithm/process.py", line 169, in <module>
2023-07-30T09:05:00.539000+00:00     process_case(filename)
2023-07-30T09:05:00.539000+00:00   File "/opt/algorithm/process.py", line 164, in process_case
2023-07-30T09:05:00.539000+00:00     all_images_predict_landmarks_list = predict(input_path)
2023-07-30T09:05:00.539000+00:00   File "/opt/algorithm/process.py", line 153, in predict
2023-07-30T09:05:00.539000+00:00     predict_landmarks = postprocess_model_prediction(predict_heatmap, processed_image, invTx)
2023-07-30T09:05:00.539000+00:00   File "/opt/algorithm/process.py", line 124, in postprocess_model_prediction
2023-07-30T09:05:00.539000+00:00     mywarpedheatmap.plot()
2023-07-30T09:05:00.539000+00:00   File "/home/algorithm/.local/lib/python3.8/site-packages/ants/viz/plot.py", line 2257, in plot
2023-07-30T09:05:00.539000+00:00     assert image.sum() > 0, "Image must be non-zero"
2023-07-30T09:05:00.539000+00:00 AssertionError: Image must be non-zero

To address this issue, we kindly request you to thoroughly review your code and prioritize testing the algorithm using the "Try-out Algorithm" option on the algorithm page.

Additionally, you may find helpful guidance in the following repository: CL_detection2023_reference_docker. The README file provides detailed instructions for testing.

Thank you for your understanding, and we look forward to your successful participation.

Hi, our submission also failed, we tried the try-out-algorithm and it's computing much longer than on my local machine (15 mins). test.sh runs successfully locally in a shorter time (3.16 mins). Any help appreciated!

Hello, lisak,

Yes, I noticed your failed submission in the backend, which encountered a "Time limit exceeded" error. We are currently discussing how to address this time limit issue and prevent it from happening during the testing phase. My suggestion is that you can try submitting a simplified version for the validation phase.

If you have any further questions or need assistance, please don't hesitate to reach out. We are here to help.

Hi jeremyzhang,

thanks for the answer! We are trying to simply the algorithm now. Can you tell us what the time limit is roughly, so that we know to what extend we need to optimize?

Thanks!

Hello, Sarina,

Our current configuration utilizes Amazon's ml.m5.large instance with a maximum GPU memory of 32GB, 2 CPU cores, and 8GB of RAM. The allowed running time for processing 50 test images is limited to 10 minutes.

We understand that this time limit may present some challenges for participants, and I want to assure you that we are actively working to address this issue. Our team is dedicated to providing a fair and optimal testing environment for all participants.

During the testing phase, we will ensure that there is sufficient time for participants to optimize their algorithms effectively. Your feedback is valuable to us, and we appreciate your understanding and patience as we work towards a solution. If you have any further questions or concerns, please don't hesitate to reach out. We are here to support you throughout the competition.

Hi, thanks for clarification! we worked on simplifying the algorithm due to the time limitation issue but when we wanted to re-submit it, the valdiation tab was already closed even though it was still 1 1/2 hour until the TWT 00.00 deadline.

Hello, Sarina,

Yes, you are right. I have modified the deadline. Please submit your algorithm docker in time.

Hi jeremyzhang, the validation submission is still closed. Did you mean that you will open it again for the 1 1/2 hours that it was closed too early? Because it would probably need some time for the docker verification. Thank you and sorry for the last minute submission, we weren't expecting the timing issue.

Hello, Sarina,

I have made some configurations in the backend, but it seems that they haven't taken effect. This issue appears to be related to the Grand-challenge platform. I sincerely apologize for any inconvenience this may have caused in your submissions.

To mitigate the impact of this time issue on algorithm validation pharse, we are considering increasing the number of allowed submissions slightly during the testing phase. This way, we hope to provide everyone with more opportunities to validate their algorithms.

Hello, Kpkreddy,

I've seen your Docker submission on the Grand Challenge platform backend. The error logged is as follows:

2023-08-15T09:08:13.117000+00:00   File "/opt/algorithm/process.py", line 105, in preprocess_one_image
2023-08-15T09:08:13.117000+00:00     fwdTx = defaultTx
2023-08-15T09:08:13.117000+00:00 NameError: name 'defaultTx' is not defined

An error has occurred pointing to line 105 of the file "/opt/algorithm/process.py". It seems that an undefined variable 'defaultTx' is being used there.

Please feel free to let me know if you require further assistance.