Our challenge platform

In 2012 a team with members from five groups in medical image analysis decided to build a platform to easily set up websites for challenges in biomedical images analysis. We named our group the Consortium for Open Medical Image Computing (COMIC). The platform was developed in Python and Django, is open source, and hosted on github where you can download it and install it on your own server.

Grand-challenges.org is an instance of our platform. It currently runs on hardware of Fraunhofer MEVIS in Bremen, Germany. The tools we offer include an easy way to create a site, add and edit pages like a wiki, registration mechanisms for participants, secure ways for organizers to provide challenge data to participants and for participants to upload results, ways to tabulate, sort and visualize the results, and some more features.

COMIC is still under development, and we are working on integrated functionality to visualize medical data and results of algorithms interactively, directly in your browser; to allow you to run your evaluation code that processes new results immediately in the cloud, and to let participants upload their algorithms themselves so that other can use them and upload new scans to be processed by these algorithms. But COMIC is it is today is already developed far enough to use it for real challenges and several challenges listed on this site use it.

Host a challenge on grand-challenge.org

Sign in to display a button here to create a new challenge!

To host a challenge, click the button above (visible after logging in). Detailed instructions are provided in the support documentation.

If you have questions or comments, post a message on the forum. You are most welcome to help us further develop and extend the COMIC platform. You can use the bug/issue tracker and code repository on github, and create a new issue there.

If you just want to have your challenge listed on the overview on the All Challenges page, and you run your challenge on your own site, just post a message in the forum or mail to support@grand-challenge.org.

Alternatives

COMIC is by no means the only way to run a challenge. Of course you can create your own website, like it has been done for most challenges that have been organized to date. We doubt however, if that is the most efficient way to create a long running challenge.

Codalab is an alternative to COMIC, with a lot of similarities, such as the fact that it is also based on python and django and it also open source. Codalab has been used extensively for challanges in the area of machine learning and computer vision.

Kitware has developed Covalic, that combines several open source platforms to host and run challenges.

Sicas has developed the Virtual Skeleton platform that has been used for several medical image analysis challenges.

Kaggle is a company that organizes challenges. On its site you can find info how to run your competition with Kaggle.

If you know of any other tools or platforms we should list here, please let us know.