Algorithm failed on one or more cases

Hi Team,

I'm wondering what the cause of the failure is. Will you be able to share logs? I've tried out the algorithm on a sample case and it seemed to work fine.

Thanks!

Hi,

The error is below:

2022-01-07T20:40:32+00:00 2022-01-07 20:40:32.298 | WARNING | ganslate.utils.environment:setup_threading:94 - 2022-01-07T20:40:32+00:00 Max threads for SimpleITK and Opencv set to 1 2022-01-07T20:40:32+00:00 For numpy etc. set OMP_NUM_THREADS=1 as an env var while running the training script, 2022-01-07T20:40:32+00:00 e.g., OMP_NUM_THREADS=1 python tools/train.py ... 2022-01-07T20:40:32+00:00
2022-01-07T20:40:35+00:00 2022-01-07 20:40:35.902 | INFO | ganslate.nn.gans.base:load_networks:260 - Loaded the checkpoint from /opt/algorithm/model/checkpoints/200000.pth 2022-01-07T20:40:43+00:00 Could not load nodules.json using . 2022-01-07T20:40:51+00:00 Traceback (most recent call last): 2022-01-07T20:40:51+00:00 File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main 2022-01-07T20:40:51+00:00 "main", mod_spec) 2022-01-07T20:40:51+00:00 File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code 2022-01-07T20:40:51+00:00 exec(code, run_globals) 2022-01-07T20:40:51+00:00 File "/opt/algorithm/process.py", line 110, in 2022-01-07T20:40:51+00:00 Nodulegeneration().process() 2022-01-07T20:40:51+00:00 File "/home/algorithm/.local/lib/python3.7/site-packages/evalutils/evalutils.py", line 183, in process 2022-01-07T20:40:51+00:00 self.process_cases() 2022-01-07T20:40:51+00:00 File "/home/algorithm/.local/lib/python3.7/site-packages/evalutils/evalutils.py", line 191, in process_cases 2022-01-07T20:40:51+00:00 self._case_results.append(self.process_case(idx=idx, case=case)) 2022-01-07T20:40:51+00:00 File "/home/algorithm/.local/lib/python3.7/site-packages/evalutils/evalutils.py", line 273, in process_case 2022-01-07T20:40:51+00:00 segmented_nodules = self.predict(input_image=input_image) 2022-01-07T20:40:51+00:00 File "/opt/algorithm/process.py", line 100, in predict 2022-01-07T20:40:51+00:00 out = self.api.infer(input).squeeze().detach().cpu().numpy().copy() 2022-01-07T20:40:51+00:00 File "/home/algorithm/.local/lib/python3.7/site-packages/ganslate/engines/base.py", line 39, in infer 2022-01-07T20:40:51+00:00 return self.model.infer(data, args, kwargs) 2022-01-07T20:40:51+00:00 File "/home/algorithm/.local/lib/python3.7/site-packages/ganslate/nn/gans/unpaired/cyclegan.py", line 221, in infer 2022-01-07T20:40:51+00:00 return self.networksf'G_{direction}' 2022-01-07T20:40:51+00:00 File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl 2022-01-07T20:40:51+00:00 return forward_call(input, kwargs) 2022-01-07T20:40:51+00:00 File "/home/algorithm/.local/lib/python3.7/site-packages/ganslate/nn/generators/vnet/vnet2d.py", line 135, in forward 2022-01-07T20:40:51+00:00 out = up(out, skip, inverse) 2022-01-07T20:40:51+00:00 File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl 2022-01-07T20:40:51+00:00 return forward_call(*input, kwargs) 2022-01-07T20:40:51+00:00 File "/home/algorithm/.local/lib/python3.7/site-packages/ganslate/nn/generators/vnet/vnet2d.py", line 222, in forward 2022-01-07T20:40:51+00:00 xcat = torch.cat((up, skipx), 1) 2022-01-07T20:40:51+00:00 RuntimeError: Sizes of tensors must match except in dimension 2. Got 5 and 4 (The offending index is 0)

Hi,

The algorithm failed again with the error below:

2022-01-13T02:32:45+00:00 2022-01-13 02:32:45.388 | WARNING | ganslate.utils.environment:setup_threading:94 - 2022-01-13T02:32:45+00:00 Max threads for SimpleITK and Opencv set to 1 2022-01-13T02:32:45+00:00 For numpy etc. set OMP_NUM_THREADS=1 as an env var while running the training script, 2022-01-13T02:32:45+00:00 e.g., OMP_NUM_THREADS=1 python tools/train.py ... 2022-01-13T02:32:45+00:00
2022-01-13T02:32:49+00:00 2022-01-13 02:32:49.064 | INFO | ganslate.nn.gans.base:load_networks:260 - Loaded the checkpoint from /opt/algorithm/model/checkpoints/200000.pth 2022-01-13T02:32:57+00:00 Could not load nodules.json using . 2022-01-13T02:38:28+00:00 /opt/algorithm/utils.py:82: RuntimeWarning: invalid value encountered in true_divide 2022-01-13T02:38:28+00:00 return (image_data-image_min)/(image_max-image_min) 2022-01-13T02:38:28+00:00 Traceback (most recent call last): 2022-01-13T02:38:28+00:00 File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main 2022-01-13T02:38:28+00:00 "main", mod_spec) 2022-01-13T02:38:28+00:00 File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code 2022-01-13T02:38:28+00:00 exec(code, run_globals) 2022-01-13T02:38:28+00:00 File "/opt/algorithm/process.py", line 119, in 2022-01-13T02:38:28+00:00 Nodulegeneration().process() 2022-01-13T02:38:28+00:00 File "/home/algorithm/.local/lib/python3.7/site-packages/evalutils/evalutils.py", line 183, in process 2022-01-13T02:38:28+00:00 self.process_cases() 2022-01-13T02:38:28+00:00 File "/home/algorithm/.local/lib/python3.7/site-packages/evalutils/evalutils.py", line 191, in process_cases 2022-01-13T02:38:28+00:00 self._case_results.append(self.process_case(idx=idx, case=case)) 2022-01-13T02:38:28+00:00 File "/home/algorithm/.local/lib/python3.7/site-packages/evalutils/evalutils.py", line 273, in process_case 2022-01-13T02:38:28+00:00 segmented_nodules = self.predict(input_image=input_image) 2022-01-13T02:38:28+00:00 File "/opt/algorithm/process.py", line 111, in predict 2022-01-13T02:38:28+00:00 result[x_min:x_max, y_min:y_max] = out 2022-01-13T02:38:28+00:00 ValueError: could not broadcast input array from shape (216,152) into shape (152,216)