R-2021-060-1E

Japanese medical-device regulators were quick to recognize the unique challenges posed by artificial intelligence and machine learning, yet they failed to avoid regulatory lag. What can be done to speed up the regulation and approval of innovative, life-saving products?

*     *     *

Technological innovation is one of humankind’s great hopes. The development and marketing of innovative products and services can spur business growth and contribute to the resolution of society’s intractable problems while making our lives easier in countless ways. The potential for such innovation is increasing exponentially as technological progress accelerates.

At the same time, innovative products face unique obstacles to regulatory approval. Precisely because of their novel features and functions, it can be difficult to assess their performance, safety, and efficacy under the existing regulatory framework. As a result, the authorities are obliged to either revise that framework or devise a completely new one. This can substantially delay the appearance of innovative products on the market—a phenomenon known as regulatory lag.

In Japan, a prime example of this phenomenon was the regulatory response to medical devices incorporating artificial intelligence and machine learning (AI/ML). In the following, I offer an overview of that response (Part 1), followed by a discussion of the structural causes of such delays and a possible solution to the problem (Part 2).

Unique Features of AI-Enabled Medical Devices

Artificial intelligence made a dramatic leap forward during the 2010s, in what is known in Japan as the third AI boom. By the middle of the decade, it was anticipated that AI/ML would quickly penetrate the field of medical devices, particularly diagnostic imaging systems. That prospect raised a number of regulatory concerns. Until then, medical products categorized as “software as a medical device,” or SaMD (previously referred to as “medical device programs”) were regulated on the assumption that the software would continue to perform according to the developer’s original specifications unless intentionally updated or modified.[1]

But ML-enabled systems are expected to learn continuously from the data they use and generate, meaning that their performance will change after deployment even without any active human intervention. While the presumption is that post-deployment changes will lead to improvement, there is always a possibility that they will degrade or compromise performance from the user’s viewpoint. Moreover, it is extremely difficult for either developers or regulators to anticipate exactly how the performance of an ML-enabled product will change over time.

The Pharmaceuticals and Medical Devices Agency (PMDA, the body responsible for authorizing new drugs and medical devices in Japan) was fairly quick to identify the challenges raised by such products in terms of regulatory review and approval, including the difficulty of predicting the sort of conditions that could lead to errors and the problem of evaluating a product against set performance standards if the performance changed with real-world use. Also mentioned were such issues as consent requirements for the use of patient data for machine-learning purposes, protection of personal data, and assignment of responsibility for performance degradation owing to machine learning.[2]

Mechanism of Regulatory Lag

It was clear, in other words, that there were aspects to AI-enabled medical software that defied the current definition of a medical device and made such products impossible to evaluate using existing criteria and methodology. When confronted by a similar challenge in the form of regenerative medicine (RM), the PMDA established a separate regulatory framework for RM products, alongside pharmaceuticals and medical devices. However, this was not the approach taken with AI. Eventually, the system for review and approval of medical devices was modified, incorporating new procedures and criteria appropriate to AI/ML-enabled devices, and adopted in September 2020 with the revised version of the Act on Securing Quality, Efficacy, and Safety of Pharmaceuticals, Medical Devices, and Similar Products (Pharmaceuticals and Medical Devices Act).

How, then, did Japan’s regulatory authorities approach the review and approval of AI-based medical systems prior to the 2020 revision? Since there were no assessment criteria or methodologies adapted to the unique features of AI-based medical devices, regulators had to apply the criteria and methodologies for assessment of conventional “software as a medical device,” in which no functional change in performance is anticipated after the software’s deployment. What this meant, in effect, was that a market-approved AI/ML-enabled product needed to obtain approval again—as a new device—whenever its performance changed, giving rise to an impossibly cumbersome system.

What was the basic cause of this regulatory lag? Simply put, it was the failure of regulators to keep up with the rapid advances in AI, including its potential applications in the field of medical devices and the dynamics of commercialization.

That said, it is no simple matter to ascertain in advance how and when emerging technologies will find their way into commercially viable products. To be sure, not every new technology will ultimately require brand-new product regulations, but identifying those that do is an ongoing challenge. Regulatory rule making requires an investment of resources. To make such an investment even before the target products and services have taken shape is a tough call for administrators.[3] On the other hand, once a technology has matured to the point where one can confidently anticipate the sorts of products and services that it is likely to spawn, it is already quite late for regulators to begin thinking about the standards and processes they need to evaluate them.[4]

The Process of Adopting a New System

The chronological chart below provides an outline of the key developments leading up to the adoption of a new system for the regulation and approval of AI-based medical devices in Japan.

Milestones in the Regulation of AI-based Medical Devices in Japan[5]

Source: Created by the author based on the PMDA’s annual report issued in December 2020.

An important milestone in this process was the December 2017 submission of a technical report by the PMDA Science Board’s Subcommittee on Artificial Intelligence and its Applications in the Medical Field.[6] The report, titled “Issues and Recommendations on AI-based Medical Diagnostic Systems and Medical Devices,” clearly articulated the challenges posed by AI-based medical devices from the standpoint of regulatory science, discussing the need for new assessment methods and systems adapted to AI/ML technology and urging the development of new assessment parameters and indicators for AI-enabled diagnostic imaging systems.[7] It was one of the first reports of its kind, and an English summary was later published in the international journal Advanced Biomedical Engineering. In terms of early development of regulatory policies, Japan was in the vanguard at this point.

With the PMDA report as a starting point, a working group in the MHLW’s National Institute of Health Sciences drew up draft review guidelines proposing basic evaluation criteria for next-generation medical devices and regenerative medical products, which it submitted in March 2019. With these guidelines as a basis, the MHLW drafted PSEHB/MDED Notification No. 0523-2, Release of Evaluation Indices for Next-Generation Medical Devices, and Japan’s regulatory authorities proceeded to develop a concrete framework for the review and approval of AI-based medical devices as tools for diagnostic imaging.

This process culminated with the adoption of a revamped review system under the revised Pharmaceuticals and Medical Devices Act, which came into effect in September 2020.

In Part 2, we will examine the structural factors contributing to regulatory lag and consider a possible solution.

[1] Under the Pharmaceuticals and Medical Devices Act and related statutes, “software as a medical device” is defined as “software (or software functionality) which is intended for use as a medical device and could impact the life or health of a patient (or user) if it fails to function as intended.” See https://www.mhlw.go.jp/stf/seisakunitsuite/bunya/0000179749_00004.html. On March 31, 2021, the MHLW issued guidance for the classification of devices as SaMD; see https://www.mhlw.go.jp/content/11120000/000764274.pdf.

[2] See https://www.pmda.go.jp/files/000218153.pdf.

[3] See Kiyohito Nakai, “AI o katsuyo shita iryo kiki no regyureshon” (Regulation of Medical Devices Using AI), Regulatory Science of Medical Products, vol. 8, no. 1 (2019), pp. 17–24.

[4] Over the past several years, the PMDA has been studying and incrementally incorporating models of “horizon scanning” to identify emerging technologies requiring a regulatory response. See https://www.pmda.go.jp/files/000236584.pdf.

[5] See https://www.pmda.go.jp/files/000238726.pdf.

[6] Mention should also be made of the pioneering research led by Shohei Nakano under a 2015 grant from the Japan Agency for Medical Research and Development, which identified and explained future issues surrounding the regulation of AI-based medical devices. Many of the conclusions were incorporated in the MHLW’s March 2016 guidance on regulator review of medical software for diagnostic purposes.

[7] https://www.pmda.go.jp/files/000225407.pdf. For an English-language summary, see Kiyoyuki Chinzei et al., “Regulatory Science on AI-based Medical Devices and Systems,” Advanced Biomedical Engineering, vol. 7 (2018), pp. 118–23, https://www.jstage.jst.go.jp/article/abe/7/0/7_7_118/_pdf/-char/en.