Project AIR

Multi use case, multi vendor, multicenter validation of AI software in radiology

Rationale

Currently more than 150 artificial intelligence (AI) based software products are on the European market for radiology. An overlap exists in use cases being addressed by the different vendors. For radiology departments it becomes increasingly difficult disseminating which software is optimal for their situation. Performance of the software is one of the factors taken into consideration in the procurement process. However, there is limited public evidence regarding the performance of these products. Our recent study showed that only 31 out of 100 CE marked products that were evaluated had peer reviewed evidence demonstrating diagnostic accuracy efficacy. When studies are available, usually different datasets are used by each vendor/research group making it difficult to directly compare products and vendors.

Objective

The aim of Project AIR is to increase transparency in the market of AI for radiology. Project AIR develops a physical and digital infrastructure to perform validation studies demonstrating the stand-alone performance of commercially available (CE marked or FDA cleared ) AI based software for radiology use cases on a hidden, multicenter dataset. Readers studies are peformed to provide context to the results.

Would you like to join as a vendor or have feedback on the protocols? Please, send an email to kicky.vanleeuwen@radboudumc.nl.

Study Protocols

  • Project AIR summary for vendors Download
  • Project AIR summary for radiologists Download
  • Study protocol: General v2.0 Download
  • Case protocol: Nodule detection chest X-ray v2.0 Download
  • Case protocol: Bone age prediction hand X-ray v2.0 Download
  • Case protocol: LVO detection brain CTA v2.0 Download
  • Study protocol: General v1.0 Download
  • Case protocol: Nodule detection chest X-ray v1.0 Download
  • Case protocol: Bone age prediction hand X-ray v1.0 Download
  • Case protocol: LVO detection brain CTA v1.0 Download