In this tutorial we will focus on grand-challenge Reader studies and show you how to upload cases to reader studies on our platform.

Remember that you need to request access prior to using a particular Reader study. You do not need to request permission if you are using your own reader study.

If you haven't installed gcapi yet, follow the instructions here.

Import necessary libraries:

import gcapi
import os


Authenticate to Grand Challenge using your personal API token.

# authorize with your personal token
token = 'my-personal-api-token'
client = gcapi.Client(token=token)


If you are working on a Reader study, you most likely need to upload the cases to your study directory on the platform. This can be easily done via GC-API.

For this particular example, the data on your machine should be structured as follows:

mainfolder/
├── patient1_folder
│       ├──file_for_a_single_series
├── patient2_folder
│       ├──file_for_a_single_series
├──patient3_folder
│       ├──file_for_a_single_series
...
├──patientN_folder
│       ├──file_for_a_single_series


Additionally, you will need to know the slug of the particular reader study you will work with. If the path your reader study on grand-challenge.org is https://grand-challenge.org/reader-studies/corads-score-exam/, the slug would be corads-score-exam.

Lets now list the files in the upload directory to check whether they can be identified:

#Specify the directory path which you want to upload from
#Specify the "slug" of the study you want to upload your data to

files = {}

#Loop through files in the specified directory and add their names to the list
for root, directories, filenames in os.walk(upload_from_dir):
for direc in directories:
temp = []
for file in os.listdir(os.path.join(root,direc)):
temp.append(os.path.join(root,direc,file))
files[direc] = temp


#### Create display sets

To create display sets for the readers to view, you need to provide an interface for each image. Within a display set, a interface needs to be unique. For example, one cannot have three generic-medical-images within a single display set.

To upload cases and create the display-sets, in this example two sets are created and the display set contains two interfaces: a ct-image and an airway-segmentation:

display_sets = [
{
"ct-image": files[0],
"airway-segmentation": files[1],
},
{
"ct-image": files[2],
"airway-segmentation": files[3],
},
]

display_set_pks = client.create_display_sets_from_images(
display_sets=display_sets
)
print("Uploaded cases and created display sets: " + display_set_pks)


#### Update the order of display sets

To update the order of a display set (in the example to 10), you can do the following:

client.reader_studies.display_sets.partial_update(pk="pk-of-display-set", **{'order': 10})


Again, you will need the slug of the reader study you wish to use to retrieve a reader study via the gc-api. The response will contain the questions, general information and the hanging list with images of the reader study.

rs = next(client.reader_studies.iterate_all(params={"slug": slug}))


my_answers = list(client.reader_studies.answers.mine.iterate_all(params={"question__reader_study": rs["pk"]}))


If you would like to get the answers per case, you can use the following code snippet. A case is defined by the images for that case, you can get those from the hanging list. Below example uses a simple index as key.

import collections

for index, case in enumerate(rs["hanging_list_images"]):
image_list = list(v for v in case.values())
a
if collections.Counter(a["images"]) == collections.Counter(image_list)
)


The answers for (multiple) choice type questions contain only the id of the chosen option, not the title. If you would like to add the title to the answers, you can do so by combining information from the reader study questions.

# get the relevant questions in a dictionary with the api_url of the question as the key, and the options for the
# question as the value. The options contain the readable title.
choice_questions = {q["api_url"]:q for q in rs["questions"] if q["answer_type"] in ("Choice", "Multiple choice")}

# multiple choice
else:
# choice

# you can create a list for just the (multiple) choice type questions


from pathlib import Path