Clarification on the use of the patient outline mask and spacing inconsistencies across patient volumes

Dear SynthRAD2025 organizers,

I would like to clarify the intended role of the provided patient outline mask. According to the documentation, this mask is meant to define the region of interest for synthesis. However, after reviewing the evaluation code and analyzing the results, it appears that geometric metrics (Dice and Hausdorff) are computed over the entire volume, without being restricted to the region defined by the mask.

This suggests that, to optimize these metrics, the synthetic CT must sometimes extend beyond the mask. Could you confirm whether this behavior is intentional?

Additionally, we have observed inconsistencies in the spacing information across some volumes. Although the documentation states that all volumes have a uniform spacing of 1×1×3 mm, we noticed that volumes with very similar image dimensions have significantly different physical fields of view. This suggests that the spacing recorded in the metadata may not be accurate.

For example, there is a clear difference between cases 2ABA005 (size: [289, 309, 88]) and 2ABA077 (size: [313, 308, 80]), despite their similar voxel dimensions.

Thank you in advance for your clarification.

Dear Valentin,

Thank you for your questions and for reaching out to us!

Regarding the voxel spacing and field of view discrepancies: All volumes were resampled to a uniform voxel spacing of 1×1×3 mm across both tasks and the three anatomical regions. However, the variation in physical field of view arises from differences in image matrix sizes, which reflects common patient size variability. In particular, some cases may include pediatric patients or individuals with very small/large body sizes, resulting in noticeably larger/smaller fields of view despite similar voxel dimensions. Due to anonymization procedures, patient age and other identifying metadata were removed, so we cannot confirm specific demographic details for individual cases like 2ABA005 or 2ABA077.

My colleague will reply to your question regarding the metrics and patient outline mask.

I hope this clarifies (part of) your concerns. Please don’t hesitate to reach out if you have further questions.

Best regards, Adrian on behalf of the SynthRAD2025 organizing team

Dear Valentin,

Regarding your question about geometric fidelity:

The ground-truth anatomical structures are generated using the TotalSegmentator model. These structures are confined within the patient's body outline (i.e., a mask). To evaluate the geometric fidelity of the synthetic CTs (sCTs), the same segmentation model is applied to them, and comparisons are made structure by structure.

Instead of evaluating based on overall volume, the assessment focuses on metrics that capture the degree of overlap and boundary alignment (e.g., Dice score, Hausdorff distance). Because the structures are only defined within the body mask, the evaluation implicitly respects this boundary. If a predicted structure extends outside this mask, it negatively impacts the geometric fidelity.

I hope this clarifies the evaluation of geometric fidelity. Please don't hesitate to reach out if you have more questions!

Best regards, Maarten on behalf of the SynthRAD2025 organizing team

Dear Maarten,

Thank you for the detailed explanation regarding the geometric fidelity evaluation using the body mask and TotalSegmentator outputs. The methodology makes sense in principle, especially the use of metrics like Dice and Hausdorff distance to assess overlap and boundary alignment.

However, I’d like to point out an issue we've encountered during our own evaluation efforts. While it's stated that the anatomical structures are confined within the patient body mask, in practice this approximation does not always hold. Specifically, we’ve noticed that in several cases the body mask is truncated, which means it does not always encompass the full body region.

Some concrete examples of this truncation include: - 1ABB022 - 1ABB051 - 1HNC129 - 1THB018 - 1THB125

In these cases, the top and/or bottom of the body are cut off in the mask, which affects the consistency of evaluations. We ran tests comparing evaluations with and without applying the mask constraint and found meaningful differences in results.

This brings up an important nuance: if the body mask is not explicitly defined and used during segmentation generation, the model may reasonably predict anatomical structures outside the mask area. Additionally, whether the mask is applied before or after segmentation has a significant impact. If the mask is applied before segmentation to restrict the input region, this can penalize predictions near image boundaries.

Given this, we believe it's important to clarify whether the mask is intended as a strict spatial constraint, and if so, at which stage it should be applied: during segmentation prediction, only during evaluation, or both.

Best regards, Cédric