How long does it take to evaluate a submitted algorithm?

Dear Orginizer

In gneneral, how long does it take from submitting the algorithm and getting the results? I have submitted an algorithm, but it was in queue for quite sometime. I would like to know how long an algorithm will stay in queue.

Thanks

It really depends on how many other submissions are happening at the same time. If it has not been executed after 24 hours you can contact support @ grand-challenge.org who can investigate during EU working hours.

Thanks for your rapid reply.

I have another question. The system notified us that the submitted algorithm failed in 1 case. Is it possible to know the error message exactly? Since the submitted algorithm was executed successfully on the try-out experiment, we would like to know where the alglrotihm went wrong so that we can fix it.

Sure, these do not get sent out automatically as it could otherwise leak the test set, but here is a sample of the logs:

2021-08-20T12:48:48.529352337Z File "/opt/algorithm/detection.py", line 86, in process_image 2021-08-20T12:48:48.529361253Z refined_preds = self.verify_pred(pred_cents, image) 2021-08-20T12:48:48.529368727Z File "/opt/conda/lib/python3.7/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context 2021-08-20T12:48:48.529376833Z return func(*args, **kwargs) 2021-08-20T12:48:48.529383664Z File "/opt/algorithm/detection.py", line 49, in verify_pred 2021-08-20T12:48:48.529391146Z tile = cv2.resize(ver_patch, (self.ver_size, self.ver_size), interpolation=cv2.INTER_CUBIC) 2021-08-20T12:48:48.529398806Z cv2.error: OpenCV(4.5.3) /tmp/pip-req-build-l1r0y34w/opencv/modules/imgproc/src/resize.cpp:4051: error: (-215:Assertion failed) !ssize.empty() in function 'resize'

Thanks. We have fixed the problem.