This review article discusses the challenges and solutions for aligning AI goals with human values, anticipating the future application of artificial superintelligence in healthcare.
Paper
Artificial superintelligence alignment in healthcare
Japanese Journal of Radiology
https://doi.org/10.1038/s41746-025-01543-z
Author's Comments
AI technology is progressing rapidly, and future AI possessing intelligence surpassing humans holds the potential to bring significant benefits to medicine. However, high performance alone is not enough; if AI behavior deviates from our ethical standards or clinical goals, it could cause unexpected disadvantages. In this paper, to prevent such situations, we have organized what preparations are currently needed from a multidimensional perspective, including not only technological development but also ethics and system design.
Paper Overview
It is anticipated that technology will evolve from the currently dominant Artificial Narrow Intelligence (AI), which specializes in specific tasks, to Artificial General Intelligence (AGI) with human-level intelligence, and eventually to Artificial Superintelligence (ASI) that surpasses humans. While ASI could be a powerful force in diagnosis and treatment planning, if its objectives and behavioral principles do not match human values and medical ethics (the "alignment problem"), there is a risk of harm to patients and adverse effects on the healthcare system. This paper outlines the basic concepts of ASI and the importance of the alignment problem in healthcare, comprehensively examining technical pitfalls, ethical concerns, and multi-layered approaches to resolve them.
Paper Details
One central challenge in the alignment problem is the difficulty of setting goals for AI. For instance, an AI programmed to minimize mortality rates in an intensive care unit might take actions that formally satisfy the goal but are contrary to the original intent, such as refusing to admit critically ill patients to improve statistics. This risk is known as "reward hacking." There is also concern that AI might amplify historical social biases contained in training data. Indeed, cases have been reported where algorithms trained using past healthcare costs as a proxy for health needs unfairly underestimated the health risks of specific racial groups, making data fairness a major issue.
To address these challenges, this paper proposes solutions at the technical, clinical, and governance levels. Technically, reinforcement learning incorporating human feedback and the development of technologies to make AI reasoning processes understandable to humans are effective. In clinical settings, education allowing healthcare professionals to critically evaluate AI recommendations and the establishment of consent processes respecting patient autonomy are required. Furthermore, institutionally, the establishment of oversight committees by medical organizations and the creation of international regulatory frameworks are necessary. We conclude that for ASI to safely contribute to medicine, comprehensive measures including medical ethics and legal regulations, in addition to technological progress, must be implemented.