[ad_1]
ALeaders of federal agencies are moving quickly to regulate artificial intelligence in health care, a proposal that now looks too big to fail.
The proposal is to establish an AI Assurance Laboratory – where AI model developers can develop and test AI models based on standards defined by regulators.
Many of the biggest names and organizations in health artificial intelligence have embraced the concept, which was described at the annual meeting of the Office of the National Coordinator for Health Information Technology, published in a prestigious JAMA special newsletter, and reported in an exclusive STAT report introduced in . The proposal was proposed by leaders of the Consortium for Artificial Intelligence in Health (CHAI) and has the strong support of two top regulators – the National Coordinator for Health IT and the Director of the U.S. Food and Drug Administration’s Digital Health Center of Excellence. The proposal responds to President Biden’s recent executive order calling for the development of artificial intelligence assurance infrastructure. The JAMA proposal concludes by requesting funding for “a small number of assurance laboratories that trial these different approaches and gather evidence that the creation of such laboratories can achieve the goals set forth in the executive order.”
It makes sense that there’s so much energy behind this concept. AI Assurance Labs can solve a small subset of the AI challenges healthcare providers face. The proposed network of artificial intelligence assurance laboratories can distinguish products that perform well across sites from those that do not. For example, ensuring a network of labs can test dozens of sepsis prediction models to effectively identify the best-performing AI products across sites. Healthcare providers and patients can benefit from increased transparency and prevent the adoption of flawed AI products.
But the proposal leaves a huge void and could exacerbate the digital divide, preventing many low-resource health care organizations from using artificial intelligence safely and effectively. We say this as AI experts with direct experience reviewing and managing AI products in resource-rich environments.
First, the AI Assurance Lab proposal fails to address the unequal distribution of AI governance capabilities. STAT reports that the initial AI assurance labs will be located at Duke University, the Mayo Clinic and Stanford University, all large health systems and academic medical centers that already have significant AI expertise. While our well-resourced institutions may benefit from funding from the AI Assurance Lab, we recognize the need to shift attention and resources to institutions that are different from ours and that are unable to effectively conduct AI Assurance. In addition to investing in a small number of AI assurance laboratories, we urge federal regulators to boldly invest in AI capabilities, infrastructure, and technical assistance to promote the safe, effective, and equitable use of AI in low-resource settings.
The proposal makes the implausible argument that federal investments in AI-enabled labs at institutions like Duke University, the Mayo Clinic, and Stanford University advance health equity. “Another option would be for health systems to establish their own local assurance laboratories,” said the team of authors, which includes prominent regulatory agencies. While possible for larger health systems and academic medical centers, this alternative It won’t scale up…Having a lab like this also exacerbates inequalities at the health system level, where better-resourced systems are able to provide stronger protection.” In a country plagued by systemic health inequalities, the investment It seems disingenuous that a small number of elite medical institutions can do more to improve AI security in under-resourced areas than directly invest in capacity building there.
Our own work on collaborative governance shows that direct responsibility for the impact of the use of AI systems will remain with frontline healthcare providers, regardless of resource levels. They are ultimately responsible for the “last mile” assessment of artificial intelligence. Federal regulators and state partner agencies such as CHAI and HAIP must address the challenges of scale and develop plans to enable all health care providers to perform these critical functions.
The proposal also suffers from two major flaws. First, regulation and investment must take into account differences in the priorities and concerns of AI product developers, regulators, and implementers. In practice, the most complex challenges faced by healthcare providers are beyond consensus. For example, in interviews with nearly 90 stakeholders across 10 healthcare organizations, we found that most organizations prefer to validate AI products locally before clinical use. The analysis by Stanford University, the Mayo Clinic and Duke University failed to account for actual differences in resources, population and operations among the more than 6,000 hospitals in the United States. Even if the AI vendor provides the healthcare provider with validation data from the AI assurance lab, the healthcare provider will want to confirm the analysis locally.
We also found that the way health care providers evaluate AI products often falls outside the scope of FDA and Office of the National Coordinator for Health Information Technology regulations. Representatives from the two federal agencies are by far the most prominent supporters of the AI Assurance Lab. For example, many health care providers are developing internal processes to assess the impact of AI products on health inequalities in response to requests from state attorney general offices and local public health offices. Healthcare providers face competitive pressures from the public, developers and regulators at different levels and require tailored support.
A second reason for the limited impact of AI assurance labs on the healthcare frontline is that AI products cannot be meaningfully evaluated in well-controlled computerized environments. In his famous book Deep Medicine, Eric Topol distinguished between computer testing (like those performed in the proposed laboratory) and prospective clinical studies. Topol emphasized that in silico testing involves “analyzing existing data sets which is very different from collecting data in a real clinical setting.” Once AI technology is put into use on the front lines, it becomes a sociotechnical system that changes the way people behave and interact. The effectiveness of artificial intelligence solutions depends more on user behavior and changes in the work environment than on quantitative metrics that can be calculated by the artificial intelligence assurance laboratory.
Numerous studies have confirmed this: The Epic sepsis model failed to meet Michigan Medicine’s technical performance goals but reduced mortality in a prospective randomized trial at Ohio Metro Health; despite changes that worsened the model’s specificity, But incorporating expert heuristics into HIV models to improve clinician-user trust; Duke University’s effort to implement a peripheral artery disease model aimed to address health inequalities but failed to identify the factors that perpetuate inequalities Structural challenges. These studies often require embedding social scientists into clinical settings where AI is implemented. These types of learning are unlikely to surface without empowering healthcare providers to conduct AI assurance activities themselves.
Thankfully, there is a path forward that involves leveraging AI-enabled laboratories combined with significant investment in technology infrastructure and regional extension centers to provide on-the-ground technical assistance to resource-poor health care providers. This approach is inspired by federal investments totaling more than $30 billion in 2009 to support the implementation of electronic health records in nearly all healthcare organizations. While there are many valid criticisms of EHRs that we wouldn’t want to replicate in AI (e.g., poor design, burdensome workflow, limited improvements in patient outcomes), federal agencies get one big thing right, and that’s Ability to roll out quickly. Funds are used to purchase the technology and a network of 62 regional outreach centers to “provide on-the-ground technical assistance to individual and small health care practices, health care facilities that lack the resources to implement and maintain EHRs, and facilities that provide primary care, public and critical access hospitals.” , community health centers, and other agencies provide services primarily to people who lack adequate insurance or medical care. “
We should approach AI assurance today with the same ambition and urgency we had when implementing EHRs 15 years ago. Regulators must invest in a portfolio of programs that go beyond computer testing to support the use of AI in real, diverse clinical settings.
Mark Sendak, MD, MPP, is director of population health and data science at Duke University’s Institute for Health Innovation and co-director of the Health AI Partnership. Nicholson Price, J.D., Ph.D., is a professor of law at the University of Michigan. Karandeep Singh, MD, MPH, is chief artificial intelligence officer at UC San Diego Health. Suresh Balu is director of the Institute for Health Innovation at Duke University and co-director of the Health AI Partnership.
[ad_2]
Source link