IMPORTANT: Queries about ground truth

Hi mate,

I'm writing this mail to present some issues. **I'm sure there are some issues with the trachea in the ground truth. **

I used a post-processing to append the missing voxels of the trachea (the largest airway) and no other changes have been made. Here are the results: Before&After trachea post-process: DICE FN FP 0.9454877 ± 0.0228938 (5) 0.0532402 (8) 0.0544731 (15) 0.9407131 ± 0.0266018 (10) 0.0676380 (28) 0.0487194 (8) The reduction of FP indicates part of the trachea was overlapped with the ground truth and the increase of FN shows part of trachea was missing during the annotation (I've reviewed all the cases before and after post-processing and i'm sure all my post-processing results of the trachea are correct whereas other branches remain unchanged).

So here is the conclusion: the annotation of ground truth is not perfect, especially for the trachea, so should we calculate all the metrics without considering the main trachea? Or will you amend the ground truth labels of the trachea? The number of voxels of the trachea is large compared with that of other branches, therefore this is an important issue that should be addressed.

Best regards

Hi,

Thank you for your question.

We performed lung field extraction as our former works for the preprocess usage for our labeling aim, this will slightly influence the main trachea part, few slices as you found. When mapping back, the few slices uppermost we donot ask the clinicians to re-labeling. However, we have the lung box information. In the test phase, we will crop the prediction results and ground-truths via the lung box information. So don't worry about that.

What’s more, the trachea is very easy to segment, and this part don’t hurt the BD/TD/Precision, which this phase will be launched soon and you will receive the official e-mail.

Best Regards, On the behalf of Institute of Medical Robotics, Shanghai Jiao Tong University

Many thanks for your query. As mentioned above, you don’t have to worry much about the trachea part as we know that the number of voxels is quite large compared with the remaining part. We will measure the connectivity and introduce more challenging cases during the testing phase where the methods with high dice metrics cannot always perform well.

Enjoy the challenge! We are looking forward to your submission in next phase.

Regards