Enhancing Medicare Support with AI
Written by Dan Adler
The Challenge
The Medicare Rights Center (Medicare Rights) is a U.S.-based nonprofit consumer service organization that for 35 years has worked to ensure access to affordable health care for older adults and people with disabilities through counseling and advocacy, educational programs, and public policy initiatives. Medicare Rights operates an independent national helpline through which expert staff and volunteers help callers navigate Medicare coverage options, enrollment decisions, benefits coordination, appeals, and other topics. In 2024, Medicare Rights staff and volunteers addressed nearly 40,000 questions via email and the national helpline.
As a nonprofit organization that depends on volunteers, Medicare Rights faces two key challenges with its national helpline: limited capacity to handle the volume of calls that it receives and a need for staff and volunteers to navigate complex online Medicare content– often written in technical and legal jargon–while serving callers. As a Siegel PiTech PhD Impact Fellow during the Summer of 2024, I explored how conversational AI tools (conversational AI) like Claude or ChatGPT could support helpline staff and volunteers by retrieving, summarizing, and communicating information.
Screenshot of the conversational AI tool. © 2025 Medicare Rights Center. All Rights Reserved.
The Project
During my 12-week fellowship, I worked with Medicare Rights to design and build a conversational AI tool to support helpline staff and volunteers. We wanted to understand if such a tool could increase helpline volume by enabling staff and volunteers to more quickly identify online content that addresses callers’ questions. We chose to develop the tool to assist helpline staff and volunteers–rather than callers–due to the tendency of conversational AI tools to generate hallucinations: authoritative but factually incorrect information. Since staff and volunteers undergo Medicare training, we believed they would be better equipped to recognize hallucinations and prevent the dissemination of incorrect information.
To design the tool, we held conversations with 14 helpline staff and volunteers. Through these conversations, we identified the source content the tool should use to generate its responses, as well as the need for generated responses to provide empathetic and actionable information to facilitate helpline conversations. After building a prototype, we showed the tool to staff and volunteers, gathered feedback to improve it, and collected baseline data to measure its perceived utility and ease of use.
Impact and Path Forward
Feedback sessions revealed that 84% of staff and volunteers we spoke to perceived the conversational AI tool as both useful and easy to use. We also evaluated the tool’s Medicare knowledge by having it complete the same 50-question test used to assess staff and volunteers’ Medicare knowledge before beginning to counsel clients. Our conversational AI tool achieved a 92% score, well above the 80% passing score required for helpline staff and volunteers.
“Our conversational AI tool achieved a 92% score, well above the 80% passing score required for helpline staff and volunteers.”
Dan Adler
Ph.D. Student, Information Science, Cornell Tech
Moving forward, we are updating the tool based on feedback from staff and volunteers, which will make it easier to link source content to generated responses and validate conversational AI outputs. We plan to then roll out the tool to the staff and volunteers we interviewed, and measure its impact on helpline call volume. Personally, I plan to continue studying how AI and other computational tools can support individuals in accessing and using their health benefits.