Segmentation artifacts and guidelines for tiger-example-algorithm ¶
By: mart.vanrijthoven on March 31, 2022, 10:13 a.m.
Dear Participants,
As mentioned in the major update, we have noticed that submissions results for the segmentation task include artifacts. In this post, I want to give you some additional info on how you might can deal with these artifacts, particularly some guidelines if you use the tiger-algorithm example as a skeleton.
Before I get to the details, I want to clarify that I use "approximately 0.5um/px" in the text below. What I mean by this is the slide spacing close to 0.5. Because slides can have a bit lower resolution, for example. 0.4987 or a slightly higher resolution, for example, 0.5123.
Due to the size of a whole slide image (WSI) at full resolution, inference on a WSI is often made via a sliding-window approach. We already implemented a basic loop in the tiger-algorithm-example.
However, based on your settings, preprocessing steps, type of model, and postprocessing steps, you might want or need to change the basic version of this loop. Below, I give some insight into how you can deal with specific settings.
The use of lower magnification/spacing.
The basic loop in the tiger-algorithm example loops over the slide at approximately 0.5um/px. This setting corresponds the level 0 in the images and is configured here. If your model, for example, processes patches at a spacing of approximately 2.0um/px, you will need to adjust the level setting and adjust the loop to make it consistent with the spacing you are processing. If you use a different spacing, you should retrieve the exact spacing present in the WSI and the shape of the slide at that particular spacing and loop according to that shape. Furthermore, the evaluation expects your model predictions at the lowest spacing of the slide =~0.5um/px. Therefore you also have to upsample your model's predictions if you work with a different spacing.
The use of "valid padding"
Another example where you probably want to change the basic loop somehow is when you use 'valid padding' or 'same padding' in your models. If you use 'valid padding,' your model output will be smaller than the input. Hence in the basic loop, you will have to extract a larger patch, and you might even have to do some extra cropping to make your output prediction consistent with your defined writing tile size. Furthermore, valid padding might also profit from additional cropping of the predictions to avoid artifacts at the borders due to stitching. One strategy is to crop the predictions to a shape that is divisible by 2**depths (i.e., the number of pooling layers in your model) such that pooling is aligned over the full WSI.
The use of "same padding"
If you use "same padding," your input and output will be the same. However, your model padded the input with artificial pixels, which often results in artifacts at the border, which might not be visible in the patch itself, but become quite evident in the full WSI prediction due to the stitching by the sliding window loop. One way to solve this is by modifying the basic loop to extract patches in an overlapping fashion. Whereafter you have to crop the borders of your prediction. So, for example, if you use a patch size of 1024, you can slide with 512 overlaps and center crop your model's prediction patch to 512. In this setting, you will have to set the writing tile size to 512. If you use a different size, you must change the writing tile size accordingly. Other, more sophisticated techniques like predicting multiple overlaps and combining the softmax values with a gaussian are also possible. Which strategy you will use is up to you, but one way or another, you will need to address the artifacts, and this most likely involves a change in the basic loop.
As the last point, I would like to mention that you can or need to change the writing tile size. You can change it to any multiplier of 16. But the most efficient way is if you would use a writing tile size 512 or 1024. Depending on your model settings, you might have to do additional stitching or cropping.
To conclude, if you use the tiger-algorithm example, you might need to change something in the basic loop based on your preprocessing steps, model configuration, and postprocessing steps. What exactly is specific to your method. Please check your predictions at full WSI. If you have any problems or questions, please don't hesitate to ask them here in the forum. We are happy to help you, and we can try to solve all the issues, together.
Best wishes, Mart, on behalf of the TIGER organization team
