Try-out log for debugging the docker container

Hi All,

I am getting the following error when use the Try-out algorithm option on the challenge website:

2022-08-23T23:29:07.377000+00:00 /opt/conda/lib/python3.7/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: libtorch_cuda_cu.so: cannot open shared object file: No such file or directory 2022-08-23T23:29:07.377000+00:00 warn(f"Failed to load image Python extension: {e}") 2022-08-23T23:29:07.377000+00:00 Could not find an int in the string 'endoscopic-robotic-surgery-video'.

Does anyone experienced such errors and know how to reslove them to make the submission possible? I am out of ideas and the deadline for Prelim evaluation is fast approaching. Any support/links to resolve these errors will be greatly appreciated.

Many thanks and

Kind regards, Bilal

It is strange the algorithms is running perfectly but is these errors which might be making the preliminary submission to fail.

2022-08-23T23:29:08.377000+00:00
2022-08-23T23:29:08.377000+00:00 TeamZERO prediction engine has started!. 2022-08-23T23:29:08.377000+00:00 -Loading key artefacts... 2022-08-23T23:29:08.377000+00:00 -3 mutli-class classification models have been detected & loaded. 2022-08-23T23:29:08.378000+00:00 -Segmntation model for image cropping is also loaded!. 2022-08-23T23:29:08.378000+00:00 -Tools dictionary loaded!. 2022-08-23T23:29:08.378000+00:00
2022-08-23T23:29:08.378000+00:00 /input/endoscopic-robotic-surgery-video.mp4 2022-08-23T23:29:08.378000+00:00 Processing the video file: /input/endoscopic-robotic-surgery-video.mp4 through TeamZERO prediction engine 2022-08-23T23:29:08.378000+00:00
2022-08-23T23:29:08.378000+00:00 -Image extraction started.. 2022-08-23T23:29:08.378000+00:00 --60 images from /input/endoscopic-robotic-surgery-video.mp4 are extracted in /images folder. Extraction done!. 2022-08-23T23:29:08.378000+00:00
2022-08-23T23:29:08.378000+00:00 -Image cropping begun for 60 images... 2022-08-23T23:29:18.380000+00:00 █ 2022-08-23T23:29:18.380000+00:00 2022-08-23T23:29:18.380000+00:00 |----------------------------------------| 0.00% [0/2 00:00<?] 2022-08-23T23:29:18.380000+00:00 2022-08-23T23:29:18.380000+00:00 |████████████████████--------------------| 50.00% [1/2 00:01<00:01] 2022-08-23T23:29:18.380000+00:00 2022-08-23T23:29:18.380000+00:00 |████████████████████████████████████████| 100.00% [2/2 00:01<00:00 0.0164] 2022-08-23T23:29:18.380000+00:00 2022-08-23T23:29:18.380000+00:00
2022-08-23T23:29:18.380000+00:00 2022-08-23T23:29:18.380000+00:00
2022-08-23T23:29:18.380000+00:00 --60 images have been croppped. Cropping done!. 2022-08-23T23:29:18.380000+00:00
2022-08-23T23:29:18.380000+00:00 -Tools presence detection started. 2022-08-23T23:29:18.380000+00:00 █ 2022-08-23T23:29:18.380000+00:00 epoch train_loss valid_loss usm1_loss usm2_loss usm3_loss usm4_loss usm1_err usm2_err usm3_err usm4_err combo_err time
2022-08-23T23:29:29.383000+00:00 █ 2022-08-23T23:29:29.383000+00:00 █ 2022-08-23T23:29:29.383000+00:00 2022-08-23T23:29:29.383000+00:00 Epoch 1/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:29.383000+00:00 2022-08-23T23:29:29.383000+00:00 Epoch 1/4 : |████████████████████████████████████████| 100.00% [1/1 00:02<00:00] 2022-08-23T23:29:29.383000+00:00 2022-08-23T23:29:29.383000+00:00 Epoch 1/4 :
2022-08-23T23:29:29.383000+00:00 2022-08-23T23:29:29.383000+00:00 Epoch 1/4 :
2022-08-23T23:29:29.383000+00:00 █ 2022-08-23T23:29:29.383000+00:00 2022-08-23T23:29:29.383000+00:00 Epoch 2/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:29.383000+00:00 2022-08-23T23:29:29.383000+00:00 Epoch 2/4 : |████████████████████████████████████████| 100.00% [1/1 00:02<00:00] 2022-08-23T23:29:29.383000+00:00 2022-08-23T23:29:29.383000+00:00 Epoch 2/4 :
2022-08-23T23:29:29.383000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 2/4 :
2022-08-23T23:29:29.384000+00:00 █ 2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 3/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 3/4 : |████████████████████████████████████████| 100.00% [1/1 00:02<00:00] 2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 3/4 :
2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 3/4 :
2022-08-23T23:29:29.384000+00:00 █ 2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 4/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 4/4 : |████████████████████████████████████████| 100.00% [1/1 00:02<00:00] 2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 4/4 :
2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 4/4 :
2022-08-23T23:29:29.384000+00:00 █ 2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 1/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 1/4 : |████████████████████████████████████████| 100.00% [1/1 00:02<00:00] 2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 1/4 :
2022-08-23T23:29:29.384000+00:00 2022-08-23T23:29:29.384000+00:00 Epoch 1/4 :
2022-08-23T23:29:29.384000+00:00 █ 2022-08-23T23:29:29.384000+00:00 epoch train_loss valid_loss usm1_loss usm2_loss usm3_loss usm4_loss usm1_err usm2_err usm3_err usm4_err combo_err time
2022-08-23T23:29:41.387000+00:00 █ 2022-08-23T23:29:41.387000+00:00 █ 2022-08-23T23:29:41.387000+00:00 2022-08-23T23:29:41.387000+00:00 Epoch 1/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:41.387000+00:00 2022-08-23T23:29:41.387000+00:00 Epoch 1/4 : |████████████████████████████████████████| 100.00% [1/1 00:02<00:00] 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 1/4 :
2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 1/4 :
2022-08-23T23:29:41.388000+00:00 █ 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 2/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 2/4 : |████████████████████████████████████████| 100.00% [1/1 00:02<00:00] 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 2/4 :
2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 2/4 :
2022-08-23T23:29:41.388000+00:00 █ 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 3/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 3/4 : |████████████████████████████████████████| 100.00% [1/1 00:02<00:00] 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 3/4 :
2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 3/4 :
2022-08-23T23:29:41.388000+00:00 █ 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 4/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 4/4 : |████████████████████████████████████████| 100.00% [1/1 00:02<00:00] 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 4/4 :
2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 4/4 :
2022-08-23T23:29:41.388000+00:00 █ 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 1/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 1/4 : |████████████████████████████████████████| 100.00% [1/1 00:02<00:00] 2022-08-23T23:29:41.388000+00:00 2022-08-23T23:29:41.388000+00:00 Epoch 1/4 :
2022-08-23T23:29:41.389000+00:00 2022-08-23T23:29:41.389000+00:00 Epoch 1/4 :
2022-08-23T23:29:41.389000+00:00 █ 2022-08-23T23:29:41.389000+00:00 epoch train_loss valid_loss usm1_loss usm2_loss usm3_loss usm4_loss usm1_err usm2_err usm3_err usm4_err combo_err time
2022-08-23T23:29:51.391000+00:00 █ 2022-08-23T23:29:51.391000+00:00 █ 2022-08-23T23:29:51.391000+00:00 2022-08-23T23:29:51.391000+00:00 Epoch 1/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:51.391000+00:00 2022-08-23T23:29:51.391000+00:00 Epoch 1/4 : |████████████████████████████████████████| 100.00% [1/1 00:01<00:00] 2022-08-23T23:29:51.391000+00:00 2022-08-23T23:29:51.391000+00:00 Epoch 1/4 :
2022-08-23T23:29:51.391000+00:00 2022-08-23T23:29:51.391000+00:00 Epoch 1/4 :
2022-08-23T23:29:51.391000+00:00 █ 2022-08-23T23:29:51.391000+00:00 2022-08-23T23:29:51.391000+00:00 Epoch 2/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:51.391000+00:00 2022-08-23T23:29:51.391000+00:00 Epoch 2/4 : |████████████████████████████████████████| 100.00% [1/1 00:02<00:00] 2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 2/4 :
2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 2/4 :
2022-08-23T23:29:51.392000+00:00 █ 2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 3/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 3/4 : |████████████████████████████████████████| 100.00% [1/1 00:01<00:00] 2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 3/4 :
2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 3/4 :
2022-08-23T23:29:51.392000+00:00 █ 2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 4/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 4/4 : |████████████████████████████████████████| 100.00% [1/1 00:01<00:00] 2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 4/4 :
2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 4/4 :
2022-08-23T23:29:51.392000+00:00 █ 2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 1/4 : |----------------------------------------| 0.00% [0/1 00:00<?] 2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 1/4 : |████████████████████████████████████████| 100.00% [1/1 00:01<00:00] 2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 1/4 :
2022-08-23T23:29:51.392000+00:00 2022-08-23T23:29:51.392000+00:00 Epoch 1/4 :
2022-08-23T23:29:51.392000+00:00 --Predictions from all models in the ensemble learner are obtained!. 2022-08-23T23:29:51.393000+00:00 --Translation of class probabilities to tool labels is done!. 2022-08-23T23:29:51.393000+00:00 --Following tools remained unaccounted for: {'nan'}. Please ensure if it is OK to skip these tools from the output. 2022-08-23T23:29:51.393000+00:00 --Output JSON file generated & returned!. 2022-08-23T23:29:51.393000+00:00
2022-08-23T23:29:51.393000+00:00 /input/endoscopic-robotic-surgery-video.mp4 has been successfully processed!. 2022-08-23T23:29:51.393000+00:00

Seems you are generating some 'nan' tool-type. This may be the cause of why evaluation is failing when you submit.

Best, The organizers

Hi kbot,

Thanks for the reply. The nan category never goes into the expected JSON output. I'm tracking all labels that don't end up in the JSON to make sure we not missing on the important labels due to disparity in the dataset tool names and the ones used in the sample JSON.

Would you please help me check the log for the submission with comment TeamZERO Test Run 1 to see if something else might be causing the error?

I will greatly appreciate your help on this.

Thanks and Kind Regards, Bilal

Hi @bilalUWE,

The bug causing this error may have been fixed as stated in this thread: https://grand-challenge.org/forums/forum/endoscopic-surgical-tool-localization-using-tool-presence-labels-663/topic/bug-fix-prelim-category-2-evaluation-container-relating-to-keyerror-suction-irrigator-1198/

If you still want to have the logs, please send me: 1 - Algorithm name. 2 - Category name. 3 - ID of the submission attempt for the algorithm. 4 - Team name. 5 - The username who submitted the algorithm for evaluation.

I appreciate your patience!

Best, SurgToolLoc 2022 Organizing Committee