Executing Failed

Hello, the algorithm I submitted failed a lot of times. Could you please check my submission, and I want to make sure the filename and suffix of output segmentation is output.tiff or subjectxxx_CTA.tif? Thank you very much!

Thank you for reaching out. I have checked your submission and reviewed the logs. It appears that your algorithm has successfully processed the input cases, but there are a few points to consider:

1. Filenames and Suffixes: According to the logs, the output segmentation file is saved with the suffix ".tiff" in the format subjectxxx.tiff. Ensure that this aligns with your expected filename and suffix requirements. Your requirement is for the output file to be named output.mha or subjectxxx_CTA.mha, please update your code accordingly to reflect this naming convention. Both your input and output should be in the mha format.

2. Postprocessing Step: There is a warning indicating that postprocessing could not be run because the postprocessing file is missing. To resolve this, ensure that you run the consolidate_folds script in the output folder of the model, specifically in the ./resources/ directory.

3. Output Location: The output is written to /output/images/aortic-branches/. Please note that you don’t need to explicitly define the output folder.

Please note that these are our assumptions based on the logs.

Additionally, please run the inference.py file locally using the instructions provided here, and then prepare the Docker image using the instructions given here. If you continue to experience issues, please provide more details about the errors you encountered, and I'll be happy to assist further.

Thank you.

Hi, I have fixed postprocess error and save each file as subjectxxx.mha as the original filename in /input/, and succeed locally, but still failed. Could you please check again, thanks a lot!

You don't need to define input and output folders as you define locally. Please follow the same directory structure as defined here.

Could you show me the error log info, I make sure I follow the structure and generate segmentation files, the error may occur in the official evaluation process. Thanks a lot!

It looks like you’re encountering a CUDA memory error. Please adjust your code to ensure it runs within the available resources. You have access to Nvidia T4 GPUs with 16GB of memory, 8 CPU cores, and 32GB of CPU memory.

CUDA out of memory, Time limit and Low DSC with nnU-Net are three main problems when we producing algorithm, here is our solution, and we hope to be helpful for the other teams:

• Q1 CUDA out of memory: When we test locally, the model size does not exceed 10GB, but CUDA out of memory after submission.

• A1: Make sure your input file and output file are the same as official, use input_files[0] and output.mha!!!

# input path setting
input_files = glob(str(location / "*.tiff")) + glob(str(location / "*.mha"))
result = SimpleITK.ReadImage(input_files[0])

#Output path setting
SimpleITK.WriteImage(
        image,
        location / f"output{suffix}", # i.e., location/output.mha
        useCompression=True,
    )

• Q2 Time Limit: During sliding window inference, the excessive number of windows causes an inference time of more than 5 minutes.

• A2: The aortic structure is distributed in the center of the image, which appropriately reduces the number of sliding windows in the coronal and sagittal axis direction, for example of nnU-Net V1.

@incollection{huang2023revisiting,
  title={Revisiting nnU-Net for Iterative Pseudo Labeling and Efficient Sliding Window Inference},
  author={Huang, Ziyan and Wang, Haoyu and Ye, Jin and Niu, Jingqi and Tu, Can and Yang, Yuncheng and Du, Shiyi and Deng, Zhongying and Gu, Lixu and He, Junjun},
  booktitle={Fast and Low-Resource Semi-supervised Abdominal Organ Segmentation: MICCAI 2022 Challenge, FLARE 2022, Held in Conjunction with MICCAI 2022, Singapore, September 22, 2022, Proceedings},
  pages={178--189},
  year={2023},
  publisher={Springer}
}

• Q3 Low DSC and other metric: The performance and metric are normal when testing locally, but obtaining nearly 0% DSC after submission.

• A3: May helpful for teams using nnU-Net, the issue of Direction of test images can be seen on issue, thanks a lot for Organizers and Team shipc1220 (NexToU-cbDice).

Flipping window data before entering in the network if not using mirroring TTA, for example of nnU-Net V1:

# x: [b, c, z, y, x] loading from SimpleITK
pred = self.inference_apply_nonlin(self(torch.flip(x, (4, 3))))
result_torch += 1 / num_results * torch.flip(pred, (4, 3))

Again, great thanks to Organizers, shipc1220 (NexToU-cbDice) and Sonwe1e (Hanglok_AortaSeg24), and more discussion can be seen on Forum.