On, we provide tools for researchers to organize challenges in medical image analysis.

Challenges are meant to facilitate a fair and objective comparison of a set of machine learning (ML) solutions for a defined clinical problem. Such a fair and objective comparison requires a representative set of test images that the ML algorithms will be run on and the definition of evaluation standards against which those algorithms' results will be compared. provides a scalable, fast, and intuitive platform for hosting such challenges.

Specifically, we offer the following tools:

  • An easy way to create a site, add and edit pages like a wiki
  • Registration mechanisms for participants
  • Secure ways for organizers to provide challenge data to participants and for participants to upload results
  • Mechanisms for participants to submit Algorithm solutions directly as Docker containers
  • Automated and fully customizable evaluations of the results generated by the submitted algorithms
  • Automated leaderboard management, including ways to tabulate, sort, and visualize the results
  • The possibility to define multiple leaderboards, corresponding to different Phases for your Challenge

As a user, you can either participate in a Challenge or create your own Challenge.

For a more in-depth explanation of what a challenge is and why it is useful, listen to James Meakin during one of our internal workshops: