[DENTEX] Data rules

Dear organizers of the DENTEX challenge,

Thank you for setting up this interesting challenge on the Grand Challenge platform!

We have some questions regarding the data:

1. Are we allowed to optimize the annotations of the provided scans (e.g. annotate implants)?
2. Are we allowed to use a non-OPG dataset for pretraining (e.g. ImageNet22k)?
3. Can you name any notable differences between the labelled and unlabelled scans (e.g. OPG scanner, patient cohort)?
4. Can you name any notable differences between the train and validation/test scans (e.g. class prevalence, class distribution)?

Thank you for your time and effort!

Kind regards,

Niels van Nistelrooij

Dear Niels van Nistelrooij,

Thank you for your participation in the DENTEX challenge and for reaching out with your questions. We appreciate your interest in the data provided. Allow me to address each of your inquiries:

1. Optimizing annotations of the provided scans: Yes, you are allowed to optimize the annotations of the provided scans, including annotating implants. The goal of the challenge is to improve the accuracy and performance of dental image analysis, and optimizing the annotations can contribute to achieving this objective. Feel free to enhance the provided annotations as necessary. However, it is crucial that you document these modifications extensively in your final short paper.

2. Using a non-OPG dataset for pretraining: Yes, you are allowed to use a non-OPG dataset, such as ImageNet22k, for pretraining your models. The challenge encourages participants to leverage any relevant data sources to enhance their models' performance. However, please ensure that you clearly document the use of external datasets in your submission, detailing the specific dataset and its source.

3. Notable differences between the labelled and unlabelled scans: The organizers of the DENTEX challenge have not provided specific information regarding notable differences between the labelled and unlabelled scans. However, it is important to note that the patient cohort and scanner type for the dataset are randomly assigned. Therefore, any differences that may arise between the labelled and unlabelled scans are a result of this random assignment.

4. Notable differences between the train and validation/test scans: Similarly, the train and validation/test scans are randomly assigned from the same scanner and patient cohort. This random assignment ensures that both sets represent the same distribution of data and avoids any intentional biases.

We appreciate your time and effort in participating in the DENTEX challenge. If you have any further questions or need additional clarification, please feel free to ask. Good luck with your endeavors!

Kind regards, DENTEX Organizers

Dear organizers of the DENTEX challenge,

Thank you for answering my questions!

I am still a bit confused concerning the second question. Under the 'Data' tab it states

To ensure a fair comparison of methods, participants are not permitted to use additional public and/or private data to extend the provided DENTEX data or pre-train models on such datasets. However, they may use additional public and/or private data for scientific publication purposes, as long as they report their results using only the DENTEX2023 dataset to discuss potential differences.

I interpreted this as meaning that we can only use the DENTEX data for challenge submissions. After the challenge is over, we can use additional data to improve the effectiveness further, where a model trained with only the DENTEX data is still included in the evaluation to fairly compare methods of different participants. This interpretation conflicts with your answer.

Could you elaborate when we can use additional data, i.e. for challenge submissions, challenge group article, own scientific article?

Thank you for your time and effort!

Best regards,

Niels van Nistelrooij

Dear Niels van Nistelrooij,

Thank you for your message and your interest in the DENTEX challenge. I am glad you reached out for further clarification, and I apologize for any confusion that we may have caused.

We have updated the related rules. According to the new guidelines, you are indeed permitted to use additional public data to either augment the provided DENTEX data or for pre-training models on such data to enhance your model's performance. This implies that even for challenge submissions, you can use additional public data, as long as it is documented transparently in your final short paper submission. This documentation should include details on the dataset used and its source.

Regarding your query about when you can use additional data, the answer is:

Challenge Submissions: Yes, you can use additional public data, but you must ensure that the data is publicly available and its use is clearly documented in your short paper.

Challenge Group Article: Yes, the use of additional data is permitted, provided you disclose it clearly, stating the details of the dataset and its source.

Own Scientific Article: Yes, additional public and/or private data can be used. It is important to note that if you use additional data for a scientific publication, you need to report results using only the DENTEX2023 dataset to discuss potential differences, as stated in the earlier guidelines.

Remember, the key point is that the use of any additional data should be transparently communicated and well-documented.

I hope this clarifies your questions. Please do not hesitate to get back to us if you need any further information.

Best regards, DENTEX Challenge Organizers

Dear organizers of the DENTEX challenge,

Thank you for clarifying the rules concerning external data use!

Best regards,

Niels van Nistelrooij