Going live
Test your reader study¶
You are advised to first test your reader study. To be able to do this you should first add yourself as a reader (this is not automatically done). Go to Readers → Add Readers and add yourself with your grand-challenge username. Once this is done, click 'Launch this Reader Study', which opens a session where you can test the study.
Editors are still able to modify the reader study after it has been launched. Some changes, like changing existing questions, are only possible when there are no existing answers or ground truth. You can clear existing answers by going to Users Progress → Delete Users Answers, and your ground truth by going to Ground Truth → Delete ground truth
Invite readers¶
When you are happy with your reader study, you can invite participants with the following steps:
- Each participant should register at grand-challenge.org and send you their username
- Go to the Readers → Add Readers and add their name
- Readers can then go to https://grand-challenge.org/reader-studies/your-reader-study/ to launch the Reader study
You can send direct messages to your readers, go to Readers and click Message User.
View results¶
To see reader study progress per participant go to Users Progress. To view a user's answers in the viewer, click the ' View User's Answers'. This allows you to navigate through all the cases and review the answers that that particular reader has given in a read-only view.
If you uploaded a ground truth you can see details on the responses per case and per question from Statistics. Also at Leaderboard a leaderboard is shown with all participants ranked based on their performance.
Export results¶
If you want to export the results go to Export to download the data. All information is stored in different csv files which all can be downloaded separately. The first file that can be downloaded contains all the characteristics of the reader study such as settings, title and hanging list. The second file contains all questions and their settings. Next, there are separate files containing all the images and answers. Note that these files only store up to 1000 images or answers. If the number of images or answers exceeds 1000, multiple CSV files will be created. Alternatively, you could use the Grand Challenge API Client to download your results through Python.