Project AIR

Multi use case, multi vendor, multicenter validation of AI software in radiology

Rationale

Currently more than 150 artificial intelligence (AI) based software products are on the European market for radiology. An overlap exists in use cases being addressed by the different vendors. For radiology departments it becomes increasingly difficult disseminating which software is optimal for their situation. Performance of the software is one of the factors taken into consideration in the procurement process. However, there is limited public evidence regarding the performance of these products. Our recent study showed that only 31 out of 100 CE marked products that were evaluated had peer reviewed evidence demonstrating diagnostic accuracy efficacy. When studies are available, usually different datasets are used by each vendor/research group making it difficult to directly compare products and vendors.

Objective

The aim of Project AIR is to increase transparency in the market of AI for radiology. Project AIR develops a physical and digital infrastructure to perform validation studies demonstrating the stand-alone performance of commercially available (CE marked or FDA cleared ) AI based software for radiology use cases on a hidden, multicenter dataset. Readers studies are peformed to provide context to the results.

Would you like to join as a vendor or have feedback on the protocols? Please, send an email to kicky.vanleeuwen@radboudumc.nl.

Study Protocols

Open to feedback (until August 9th 2021)
  • Project AIR summary for vendors Download
  • Project AIR summary for radiologists Download
  • Study protocol: General Download
  • Case protocol: Nodule detection chest X-ray Download
  • Case protocol: Bone age prediction hand X-ray Download
  • Case protocol: LVO detection brain CTA Download