Benchmarking Pathfinder

Benchmarking


    How do we know it works? The challenge we face when responsibly scaling digital health innovation is bringing together traditional and emerging benchmarks in medical science and AI/data science. The former refers to objective scientific methodologies such as RCTs and meta studies often arranged in a top-down pyramid.

    The latter often refers to relatively subjective notions such as “human-centeredness” and “trustworthiness” in the physician-patient relationship. These are beginning to be seen as essential prerequisites for Artificial Intelligence and data use in health alongside scientific evidence for safety and efficacy.

    I-DAIR will focus initially on what kind of human-centered benchmarks are needed, how they should be developed and deployed and what could be the attributes of trusted and neutral platforms that act as “social stock exchanges” for these benchmarks.

    To achieve this, we will :

    • Explore tiered approaches to benchmarks as per risk (mitigation) and agency (enhancement);
    • Consider the use of stories (micro-narratives) as a way to benchmark the impact of digital health and Artificial Intelligence solutions;
    • Develop a Validation Network, through our clinical and impact cohorts, for a distributed assessment of algorithms against local datasets and contexts. This would mirror the international round robin test system for diagnostics.