Demystifying AI in Healthcare – Part 2 | Transparency
As the application of Artificial Intelligence (AI) within the healthcare sector continues to accelerate – with 94 percent of health care companies reporting they already apply AI/machine learning in some capacity – healthcare organizations must ground their AI integration in transparency to break down stigmas surrounding the usage of AI and to build trust among key stakeholders.1 As it stands, misconceptions about the applications and reliability of AI can lead to confusion and apprehension. This may range from healthcare workers who fear loss of employment and patients who are skeptical about AI’s ability to provide accurate diagnoses or treatment recommendations,2 to lawmakers calling for increased regulatory pressure on AI applications.
Part two in a four-part series sharing perspectives on several of the most pertinent considerations related to the use of AI in healthcare – transparency, privacy and security, and sophistication. Our first installment, “Demystifying AI in Healthcare,” explored these topics at a high level and reinforced the opportunity for healthcare organizations to best leverage AI applications while also navigating increased political, regulatory, and public scrutiny.
Earlier this year, for example, over 100,000 members of National Nurses United (NNU) marched to demand the hospital industry ensure safe staffing levels and patient safeguards amidst the rapid application of AI technologies.3 Further, a 2024 survey found the majority of registered voters want transparency into how AI developers and the health sector are using AI within patients’ health care experiences.4 Meanwhile, legislative bodies and regulators are responding to the call for increased transparency, as evidenced by a new federal rule, effective January 2025, requiring electronic health record (EHR) companies to disclose information including risk mitigation strategies and whether a tool has been tested in the real world.5
Notably, however, the regulatory landscape is shifting under the Trump Administration, with President Donald Trump rolling back Biden-era policies that “[hinder] AI innovation and [impose] onerous and unnecessary government control over the development of AI,” as well as the recent release of an AI action plan to reduce regulations that slow the development and deployment of AI technology in the U.S.6,7
Transparency should not be seen as a hurdle to AI adoption but rather as a vital tool for easing concerns, fostering open dialogue, and helping healthcare organizations navigate the complex regulatory landscape surrounding AI. By embedding transparency into AI strategy, organizations can highlight their commitment to responsible AI usage — and to build on that commitment, they must also ensure that AI is deployed in a safe and ethical manner.
Establishing an AI Governance Framework
As more and more healthcare organizations integrate AI into their systems and processes, establishing robust AI governance frameworks is imperative to ensuring the transparent and responsible use of these powerful technologies.
A robust governance framework promotes both transparency and accountability by establishing mechanisms for determining and monitoring how and when AI is used, as well as auditing and addressing AI-related errors or biases. Sound AI governance should also establish clear and transparent policies, procedures, and guidelines for documenting and disclosing AI decision-making processes. Transparency within governance structures plays a critical role in providing insight into how AI systems operate, which in turn helps keep organizations accountable for the decisions made through AI modeling, while also helping to minimizing risk associated with algorithmic bias or potential inaccuracies.
While governance structures can look different depending on an organization’s size and AI use cases, many healthcare companies stand up a leadership or steering committee to provide executive oversight and direction for AI programs. Importantly, another best practice is to establish an internal review committee responsible for providing ongoing tactical and operational support for AI efforts. Whether these internal governing bodies are leadership-driven or organization-wide, they should encompass a diverse group of stakeholders from across an organization, including clinicians, data scientists, ethicists, legal, compliance, risk, marketing/public relations and patient representatives.8
FTI Consulting further explores the importance of an effective AI Governance framework in-depth in a 2024 piece entitled, “Overcoming AI Misconceptions: The Role of Governance in Healthcare.”
Best Communications Practices
Coupling AI governance with a transparent communications approach works to alleviate concerns about the application of AI by disclosing its intended purpose and demystifying its specific functions.
Proactive and effective engagement with internal and external stakeholders — including leadership, clinical teams, patients and caregivers, lawmakers, and regulators — can build trust between healthcare organizations and these audiences while facilitating accountability. These communications should prioritize the timely notification of AI adoption, the intended purpose of the tools, and the organization’s strategy for implementing AI safely, effectively, and ethically.
Perhaps most importantly, messages should be tailored to different audiences and delivered via accessible communications channels with built-in avenues for open discussion and feedback. For example, healthcare organizations may consider acknowledging concerns and misconceptions from employees that AI systems will replace their jobs, reiterating that AI tools are additive to their roles and intended as an enhancement — not a replacement — for human expertise. In addition, both providers and patients may benefit from information about how AI models are constructed, tested, validated, and applied to mitigate concerns about the accuracy of information generated by AI. At the same time, executive leadership may benefit from regular status updates regarding the organization’s compliance with state and federal guidelines.
Regardless of its application — whether in administrative tasks, clinical settings, or logistics — investing in training is essential to enhancing communications around AI and ensuring its transparent and effective use. Well-structured training not only ensures that professionals use AI correctly and ethically but also contributes to overcoming resistance by demystifying its functions and limitations.9 While the need for structured AI training varies across geographies and industries, certain sectors, particularly healthcare, require dedicated programs to equip professionals with the knowledge and skills to apply AI responsibly and effectively.
Defining and Measuring Transparent AI
Transparency enables healthcare organizations to identify trusted and effective AI tools before adopting and integrating them into their enterprises.
Transparency is just as critical for evaluating AI tools prior to adoption as it is for establishing AI governance frameworks and developing tailored communications. Defining transparency and establishing clear metrics for evaluating AI solutions against that definition can help healthcare organizations determine whether AI tools meet their ethical and operational standards. Establishing evaluation criteria is particularly critical when working with several AI vendors to determine which AI tools are the most safe and effective. Providers adopting AI technology should be equipped with the information needed to better vet these tools and determine whether they have been checked for bias or inaccuracies.10
To facilitate their own development of criteria for AI tools, healthcare organizations have an opportunity to partner with groups that are already developing and providing these best practice standards for transparency. As we anticipate the use of AI in healthcare to become more and more regulated, healthcare organizations should act now to develop and apply AI governance frameworks and processes before it ultimately becomes a requirement by industry groups setting the standards for, and overseeing, the proper use of AI.
AI in Practice: The American College of Radiology (ACR) – an FTI Consulting client – is empowering radiologists to implement practical AI solutions safely, effectively, and transparently. In an effort to promote transparency around AI products and algorithms, the ACR launched AI Central, the most complete and up-to-date online, searchable directory of commercially available Imaging AI products in the United States. “Through AI Central, the ACR is making transparency in AI more than a principle — it’s a practical resource,” an ACR spokesperson shared with FTI Consulting. “By centralizing information about Food and Drug Administration (FDA)-cleared models, intended use, and performance, we’re helping radiologists and health systems make confident, data-driven decisions about the AI tools they .”
Looking Ahead
In the next installment of the series, we will examine data privacy and security in AI-driven applications, particularly as it pertains to healthcare organizations’ handling of highly sensitive patient data.
Samantha Hardey, a former Senior Consultant at FTI Consulting, contributed to this article.
References
[1] “How Artificial Intelligence Could Reshape Health Care,” Morgan Stanley (August 15, 2023), https://www.morganstanley.com/ideas/ai-in-health-care-forecast-2023
[2] Tyson, Alec, et al., “60% of Americans Would Be Uncomfortable With Provider Relying on AI in Their Own Health Care,” Pew Research Center (February 22, 2023), https://www.pewresearch.org/science/2023/02/22/60-of-americans-would-be-uncomfortable-with-provider-relying-on-ai-in-their-own-health-care/
[3] “Nurses march nationwide in support of safe staffing, patient protections against A.I.,” National Nurses United (January 14, 2025) https://www.nationalnursesunited.org/press/nurses-march-nationwide-in-support-of-safe-staffing-and-patient-protections-against-ai
[4] “USofCare Poll: Americans Overwhelmingly Demand Oversight and Transparency on AI Usage in Health Care,” United States of Care (June 25, 2024), https://unitedstatesofcare.org/pr-ai-oversight-transparency-health-care/
[5] Aguilar, Mario, “Hospitals now know how some health AI tools were developed. Will that change anything?” Stat News (January 6, 2025), https://www.statnews.com/2025/01/06/health-care-artificial-intelligence-regulation-transparency-rule-hti1-model-card/
[6] “Fact Sheet: President Donald J. Trump Takes Action to Enhance America’s AI Leadership,” The White House (January 23, 2025), https://www.whitehouse.gov/briefings-statements/2025/01/fact-sheet-president-donald-j-trump-takes-action-to-enhance-americas-ai-leadership/
[7] “Winning the Race: America’s AI Action Plan,” The White House (July 2025), https://www.whitehouse.gov/wp-content/uploads/2025/07/Americas-AI-Action-Plan.pdf
[8] Hassan, Neha, et al., “Road map for clinicians to develop and evaluate AI predictive models to inform clinical decision-making,” BMJ Health & Care Informatics, Vol. 30, no.1 (August 9, 2023).
[9] Sharma, Raj, “Why transparency is key to unlocking AI’s full potential,” World Economic Forum (January 2, 2025), https://www.weforum.org/stories/2025/01/why-transparency-key-to-unlocking-ai-full-potential/
[10] “AI in Healthcare: Why Regulators Want More Transparency | WSJ Tech News Briefing,” WSJ Podcast (December 6, 2023), https://www.youtube.com/watch?v=SI4Lb041CME
Related Insight
Demystifying AI in Healthcare
Part one of a four-part series sharing perspectives on several of the most pertinent considerations related to the use of AI in healthcare – transparency,
Related Expertise
The views expressed herein are those of the author(s) and not necessarily the views of FTI Consulting, Inc., its management, its subsidiaries, its affiliates, or its other professionals.
FTI Consulting, Inc., including its subsidiaries and affiliates, is a consulting firm and is not a certified public accounting firm or a law firm.
FTI Consulting is an independent global business advisory firm dedicated to helping organizations manage change, mitigate risk and resolve disputes: financial, legal, operational, political and regulatory, reputational and transactional. FTI Consulting professionals, located in all major business centers throughout the world, work closely with clients to anticipate, illuminate and overcome complex business challenges and opportunities. ©2025 FTI Consulting, Inc.
All rights reserved. fticonsulting.com
