My approach is a pipeline that contains a pseudo-labelling step. Can I manually select the cases with the best pseudo-labels?

No, this is not allowed. Indeed, manual selection is considered as a manual annotation step as it involves external knowledge to select the best cases. You must find an approach that automatically selects the best pseudo-labels.