Policy Brief

11th November 2024

Policy Brief on Potential risks of widespread adoption of AI in medicine

Context of the study
1. The integration of artificial intelligence (AI) into medicine promises significant advances in healthcare delivery, diagnosis and
patient care.
2. However, this widespread adoption also presents several risks that need to be addressed to ensure the safe, fair and efficient
use of AI technologies.
3. This policy paper outlines key risks associated with AI in medicine, including data privacy and security, bias and inequity,
clinical decision-making challenges, regulatory and ethical issues, economic impacts, and technological limitations. It also
provides recommendations to mitigate these risks.
Summary of result
1. Extensive adoption of AI in medicine offers significant benefits but also many risks that must be carefully managed.
2. By addressing data privacy and security concerns, reducing bias, ensuring a strong regulatory framework, and providing ongoing
training for healthcare professionals, we can improve health while minimizing potential hitches. Can couple the power of AI to
improve health care outcomes.
3. Policymakers, healthcare providers, and AI developers must team up to create safe, impartial, and effective healthcare environs.
Introduction Clinical Decision-Making
Artificial Intelligence (AI) has the potential to revolutionize the 1. Risk: Over-reliance on AI could undermine healthcare
medical field by improving diagnostic accuracy, personalizing professionals, clinical judgment, leading to patient care
treatment plans, and enhancing patient outcomes.[1],[3],[9] Despite errors, as conflicting AI recommendations have raised
these benefits, the rapid adoption of AI in healthcare raises several concerns about its reliability in critical decisions.[14]
concerns that must be carefully managed. [2] This paper aims to 2. Example: AI algorithms can inherit biases from their
identify and analyse the potential risks associated with the training data, leading to unequal patient treatment. For
widespread use of AI in medicine and propose policy example, an AI predicting patient outcomes might be
recommendations to mitigate these risks. [8] trained on data that underrepresents certain racial or
Problem Statement: The adoption of AI in medicine is accelerating, socioeconomic groups, resulting in biased predictions and
but it brings with it significant risks that could undermine patient exacerbating health disparities. A notable case showed an
safety, exacerbate healthcare inequalities, and create new ethical and AI tool systematically underestimating the needs of Black
regulatory challenges. [2],[3-7] Addressing these risks is crucial to patients compared to white patients with similar
ensuring that AI technologies are used responsibly and effectively in conditions.[13]
healthcare settings.[10] • Regulatory and Ethical Challenges
Summary of Research 1. Risk: The rapid development of AI can outpace regulatory
Policy Analysis frameworks, creating oversight gaps and potential misuse.
Data Privacy and Security The lack of comprehensive AI regulations in healthcare is a
1. Risk: AI systems need large datasets, raising the risk of data significant barrier to safe and ethical deployment. [15]
breaches and unauthorized access to sensitive patient 2. Example: IBM Watson for Oncology: Software designed to
information, as numerous incidents have shown the vulnerability assist oncologists in diagnosing and treating cancer, the
of patient data in digital systems.[11], [12] system faced criticism for unsafe and incorrect treatment
2. Example: Community Health Systems Data Breach (2014): It recommendations. It was trained mainly on hypothetical
affected 4.5 million patients as cybercriminals exploited a patients, leading to inaccurate suggestions and raising
software vulnerability using sophisticated malware, leading to ethical concerns about patient safety and AI reliability in
the theft of sensitive patient data such as names, birth dates, critical medical decisions.[16]
social security numbers, phone numbers, and addresses.[12] • Economic and Workforce Impacts
Bias and Inequality 1. Risk: AI adoption could displace certain jobs and increase
1. Risk: AI algorithms can perpetuate biases in healthcare, causing healthcare costs due to high implementation and
unequal treatment based on race, gender, or socioeconomic maintenance expenses. While AI can improve efficiency, it
status, as studies show biased data leads to discriminatory may also lead to significant workforce changes and cost
outcomes. implications.[17]
2. Example: Bias in Health Risk Prediction Algorithms: An 2. Example: Job Displacement and Role Changes: AI-
algorithm predicting extra medical care needs was biased powered diagnostic tools raise concerns about job
against Black patients, underestimating their health needs displacement among radiologists, as these systems can
compared to white patients with similar conditions. This bias accurately analyse medical images and detect
arose because the algorithm used healthcare costs as a proxy for abnormalities. A study by the American College of Radiology
health needs, and Black patients historically incur lower costs found that while AI enhances diagnostic accuracy, it also
due to systemic barriers to care. [13] raises concerns about radiologists future roles. [18]
 • Technical Limitations References:
1. Risk: AI systems can malfunction, causing serious consequences 1. Ahmad, Z., et al., Artificial intelligence (AI) in medicine, current
applications and future role with special emphasis on its
in medical settings. Reports of such failures highlight the need
potential and promise in pathology: present and future impact,
for robust technical safeguards.[19] obstacles including costs and acceptance among pathologists,
2. Example: Algorithmic Failures: An AI system for diagnosing practical and philosophical considerations. A comprehensive
diabetic retinopathy performed well in controlled environments review. Diagnostic pathology, 2021. 16: p. 1-16.
but struggled in real world settings due to image quality 2. Muley, A., et al., Risk of AI in Healthcare: A comprehensive
literature review and study framework. arXiv preprint
variations and different clinical workflows. This highlights the arXiv:2309.14530, 2023.
technical limitations of AI in diverse and unpredictable clinical 3. Yadav, N., et al., Data privacy in healthcare: In the era of Artificial
environments.[20] Intelligence. Indian Dermatology Online Journal, 2023. 14(6): p.
788-792.
Policy Recommendations 4. Agarwal, R., et al., Addressing algorithmic bias and the
Robust Data Security Measures perpetuation of health inequities: An AI bias aware framework.
Health Policy and Technology, 2023. 12(1): p. 100702.
5. De Panfilis, L., et al., AI-based clinical decision-making systems
- Implement strong data encryption and access controls to protect in palliative medicine: ethical challenges. BMJ Supportive &
patient information. Palliative Care, 2023. 13(2): p. 183-189.
6. Kasula, B.Y., Ethical and regulatory considerations in AI-Driven
- Regularly update security protocols to address emerging threats. healthcare solutions. International Meridian Journal, 2021. 3(3):
Bias Mitigation Strategies p. 1-8.
7. Afjal, M., Evolving trends, limitations, and ethical considerations
in AI-driven conversational interfaces: assessing ChatGPT's
- Use diverse and representative datasets for training AI systems. impact on healthcare, financial services, and educational
sectors. Technology Analysis & Strategic Management, 2024: p.
- Conduct regular audits to identify and correct biases in AI 1-20.
algorithms. 8. Larasati, R. AI in Healthcare: Impacts, Risks and Regulation to
Mitigate Adverse Impacts. in CEUR Workshop Proceedings of the
Clear Regulatory Frameworks
3rd Workshop on Adverse Impacts and Collateral Effects of
Artificial Intelligence Technologies, AiOfAi 2023. 2023. CEUR
- Develop comprehensive regulations that keep pace with AI Workshop Proceedings (CEUR-WS. org).
advancements. 9. Pattyam, S.P., Artificial Intelligence for Healthcare Diagnostics:
Techniques for Disease Prediction, Personalized Treatment, and
- Ensure transparency and accountability in AI decision-making Patient Monitoring. Journal of Bioinformatics and Artificial
Intelligence, 2021. 1(1): p. 309-343.
processes.
10. Shaheen, M.Y., Applications of Artificial Intelligence (AI) in
Continuous Training for Healthcare Professionals healthcare: A review. ScienceOpen Preprints, 2021.
11. Sen, R. and S. Borle, Estimating the contextual risk of data
- Provide ongoing education and training to ensure that healthcare breach: An empirical approach. Journal of Management
professionals can effectively use AI tools without over-reliance. Information Systems, 2015. 32(2): p. 314-341.
12. Floyd, T., M. Grieco, and E.F. Reid. Mining hospital data breach
records: Cyber threats to us hospitals. in 2016 IEEE Conference
- Encourage collaboration between AI developers and healthcare on Intelligence and Security Informatics (ISI). 2016. IEEE.
providers to improve AI system design and implementation. 13. Chin, M.H., et al., Guiding principles to address the impact of
Economic and Workforce Considerations algorithm bias on racial and ethnic disparities in health and
health care. JAMA network open, 2023. 6(12): p. e2345050-
e2345050.
- Develop strategies to manage job displacement and support
14. Magrabi, F., et al., Artificial intelligence in clinical decision
workforce transition. support: challenges for evaluating AI and practical implications.
Yearbook of medical informatics, 2019. 28(01): p. 128-134.
- Assess the cost-effectiveness of AI systems to ensure they provide 15. Mennella, C., et al., Ethical and regulatory challenges of AI
value without disproportionately increasing healthcare costs. technologies in healthcare: A narrative review. Heliyon, 2024.
Technical Safeguards 16. Ross, C. and I. Swetlitz, IBM’s Watson supercomputer
recommended ‘unsafe and incorrect’cancer treatments, internal
documents show. Stat, 2018. 25: p. 1-10.
- Implement robust testing and validation procedures for AI systems. 17. Tiwari, R., The impact of AI and machine learning on job
displacement and employment opportunities. International
- Ensure interoperability between AI systems and existing healthcare Journal of Engineering Technologies and Management Research,
infrastructure to minimize disruptions. 2023. 7(1).
18. Waite, S., et al., A review of perceptual expertise in radiology-how
it develops, how we can test it, and why humans still matter in the
era of artificial intelligence. Academic Radiology, 2020. 27(1): p.
26-38.
Dr M. Masoud ul Hassan 19. Khan, B., et al., Drawbacks of artificial intelligence and their
MSP 3rd semester potential solutions in the healthcare sector. Biomedical Materials
& Devices, 2023. 1(2): p. 731-738.
Weekend Batch 20. Zafar, S., et al., Artificial intelligence algorithms in diabetic
Roll No:- 00805/HSA/MSPH-2023 retinopathy screening. Current Diabetes Reports, 2022. 22(6): p.
Email address:- drmasoudhassan@yahoo.com 267-274.

Contact Number:- 0333-5969151