Join the discussion in the Almanac Community.

In this article, we summarize results of the first-ever survey of Alliance members’ usage of, attitudes toward and concerns about the use of generative AI in CME/CPD. This survey was conducted and analyzed in November–December 2023. In the few months since then, AI has continued to advance at a bewildering pace. Accordingly, we suspect that AI use among Alliance members has continued to evolve. To further assess trends in AI use in CME/CPD, we will be conducting a follow-up survey of the Alliance membership in the fall of 2024.

Methodology and Demographics

Approximately 200 Alliance members responded to the November 2023 survey. The majority (54%) were between 35 to 54 years old, with 29% between 55 to 64. Most respondents (71%) had over a decade of experience in the field, ensuring that our findings represent the perspectives of seasoned professionals. Based on this demographic spread, we are reasonably confident that our findings represent the perspectives of seasoned professionals in the field.

AI Adoption at a Crossroads

The most striking find was a near-even split in AI adoption: 51% of respondents reported using AI for professional purposes, while 49% did not. This even division suggests that our field was at a critical juncture in terms of AI integration at the time of the survey.

For non-users, common reasons included:

• Concerns about the technology
• Lack of familiarity
• Uncertainty about relevance to CME/CPD
• Organizational prohibitions (8% of respondents)

Among AI users, ChatGPT emerged as the dominant tool, with 89% of users reporting experience with it. Other platforms like Bing AI and Bard saw significantly lower adoption rates, at less than 20% each. It's worth noting that at the time of the survey, Microsoft had just begun integrating AI into Office and tools like Copilot were not yet widely available.

Significant Concerns Amid Optimism

While 64% of respondents were optimistic about AI's role in CME/CPD, there were widespread concerns about accuracy (72%), quality (60%) and potential biases (36%) in AI outputs. This highlights the need for guidance and best practices in AI implementation.

Organizational Policies Lagging

Only 21% of respondents reported that their organizations had guidelines for appropriate AI use, suggesting a significant gap between AI adoption and formal governance structures in the CME/CPD field.

Applications and Concerns

Respondents reported diverse applications of AI, including:

• Marketing and communication tasks, such as crafting campaign copy for ads and promotional email blasts
• Assistance with needs assessments, practice gap identification and learning objective development
• Educational content creation, including slides, patient cases, content outlines and assessment questions
• Data analysis and outcomes summarization, such as synthesizing learner data from post-activity surveys

However, concerns were widespread:

• 72% of respondents expressed worries about the accuracy of AI-generated results
• 60% had reservations about output quality
• 51% noted the issue of AI hallucinations
• 36% raised concerns about potential biases in AI outputs

Some respondents expressed more specific concerns. One noted, "There's a risk of over-reliance on AI, which could lead to a degradation of professional skills and expertise." Another worried about "the potential homogenization of educational content if multiple providers rely too heavily on AI for content direction."

Organizational Readiness

The survey revealed a significant gap between AI adoption and organizational preparedness. Only 21% of respondents reported that their organizations had guidelines for appropriate AI use. This discrepancy suggests a need for more robust governance structures as AI integration continues.

One respondent highlighted the challenges of organizational adoption: "We're in the early stages of understanding AI, and it takes time to learn new processes and integrate them into our workflows.”

Future Outlook

Despite the challenges, the overall sentiment toward AI in CME/CPD was predominantly positive. Sixty-four percent of respondents expressed optimism about AI's future role in the field, with 27% neutral and only 9% pessimistic.

Those expressing optimism cited potential benefits such as improved information aggregation, increased productivity and cost and time savings. One enthusiastic respondent noted, "AI has the potential to automate tasks and generate content, freeing us up to focus on higher-level strategic work."

Implications of These Findings

This inaugural survey provides a valuable benchmark for understanding AI's place in CME/CPD as of late 2023. It reveals a field actively engaging with AI, grappling with its potential benefits and challenges. The near even split in adoption, coupled with widespread optimism, suggests we may be on the cusp of more widespread AI integration. However, the concerns raised about accuracy, quality and ethical implications underscore the need for careful consideration as we move forward. The gap between adoption and organizational readiness also highlights an area requiring attention. In the field of CME/CPD, this highlights an urgent need for AI literacy programs for CME/CPD professionals and development of best practices and ethical guidelines for AI use. These are matters we suggest the Alliance is well-suited to address in support of the profession.

For full details on this survey, please review this PDF that comprehensively lays out most of the AI survey questions that were asked.

What’s Next in AI?

The pace of AI development has been astonishingly rapid over the past 6 months. Consider that at the time of the November 2023 survey:

• ChatGPT was the only large language model (LLM) that most of us had ever used, with 89% reporting use. Bing and Bard AI were a distant second and third (less than 20% usage for each). Today, ChatGPT still dominates, but some of us are trying out other LLMs such as Claude from Anthropic and apps like Elicit and Perplexity that attempt to bridge the gap between generative AI and scientific evidence.
• Microsoft had just started to integrate AI into Office. We didn’t even ask about Copilot AI in our survey. Today, many of us are getting Copilot access. Additionally, it seems like nearly every app, everywhere, is getting an AI integration.
• At the time of the survey, video rendering tools like Sora AI and Luna Dream Machine were not available. Today, anyone can generate a remarkably realistic video in minutes by typing in a simple text prompt (here’s our first attempt — a group of physicians interacting with an advanced human body simulation).

As we move forward, it's clear that the Alliance has a crucial role to play in supporting your AI journey. Survey respondents overwhelmingly endorsed the need for AI guidelines, best practices, workshops and case studies. In response, the Alliance has formed an AI Task Force to proactively address these needs and ensure responsible AI integration in our field.

With these advances in mind, we encourage you to participate in the next Alliance AI survey this fall. It’s crucial to understand how the use of generative AI is evolving in our community over time. Your insights will be crucial to characterizing the developments since the initial benchmark survey described in this article.

--

AI Disclosure

ChatGPT was used to process and summarize the write-in survey responses that are discussed in this article.

Acknowledgements

These insights on AI use in CME/CPD were made possible through the efforts of a broad array of Alliance members, including the authors of this article (Andrew Bowser, Andrew Crim, Dustin Ensign and Brian McGowan); members of the Research Committee (Kelsey Caras, Jas Chahal, Sally Haas, Esther Nyarko, Kevin Obholz and Board Liaison Allison Gardner); the PEERS member section (Bethany Banner, Emily Belcher, Tabitha Gai, Don Harting, Anita Misra-Press, R. Michelle [Tyner] Skidmore and Kim Vadas) and Alliance staffers Ashley Barr, Angela Belusik and Danielle Callen. We are especially indebted to Jan Schultz, Immediate Past President of the Alliance, for her leadership and support in the development and execution of this research effort.

---