Regulating Black-Box Medicine

Abstract

Data drive modern medicine. And our tools to analyze those data are growing ever more powerful. As health data are collected in greater and greater amounts, sophisticated algorithms based on those data can drive medical innovation, improve the process of care, and increase efficiency. Those algorithms, however, vary widely in quality. Some are accurate and powerful, while others may be riddled with errors or based on faulty science. When an opaque algorithm recommends an insulin dose to a diabetic patient, how do we know that dose is correct? Patients, providers, and insurers face substantial difficulties in identifying high-quality algorithms; they lack both expertise and proprietary information. How should we ensure that medical algorithms are safe and effective? Medical algorithms need regulatory oversight, but that oversight must be appropriately tailored. Unfortunately, the Food and Drug Administration (FDA) has suggested that it will regulate algorithms under its traditional framework, a relatively rigid system that is likely to stifle innovation and to block the development of more flexible, current algorithms. This Article draws upon ideas from the new governance movement to suggest a different path. FDA should pursue a more adaptive regulatory approach with requirements that developers disclose information underlying their algorithms. Disclosure would allow FDA oversight to be supplemented with evaluation by providers, hospitals, and insurers. This collaborative approach would supplement the agency’s review with ongoing real-world feedback from sophisticated market actors. Medical algorithms have tremendous potential, but ensuring that such potential is developed in high-quality ways demands a careful balancing between public and private oversight, and a role for FDA that mediates—but does not dominate—the rapidly developing industry.