Use of Generative Artificial Intelligence for Development of Plain Language Summaries: A Blinded Assessment of Education Preferences of the UK Sickle Cell Disease Community

Authors

• Oliver Shastri, Pfizer Ltd, Tadworth, UKFollow
• Orlando Agrippa, Sanius Health, London, UK
• Valerie Moss, Prime, Knutsford, UK
• Eleanor Millard, Pfizer Ltd, Tadworth, UK
• Lianne More, Pfizer Ltd, Tadworth, UK
• Eleanor Rose Brown, Pfizer Ltd, Tadworth, UK
• Seraj Sharif, Pfizer Ltd, Tadworth, UK
• Chris Finch, Prime, Knutsford, UK

Abstract

Objective: Generative artificial intelligence (GenAI) supports medical content development, including plain language summaries (PLS), thereby supporting accessibility of education. We examined preferences of PLS format created from a study assessing UK sickle cell disease (SCD) community social media conversations. Methods: Three written PLS were developed, human-written, AI-generated, and hybrid AI-human (person with SCD edited AI version for understandability). Videos were developed of each written PLS using the AI tool Synthesia. Adults (n = 93) with SCD and carers completed a survey assessing understandability of each written PLS and preference for written versus video. Participants were blinded to PLS source. They were presented at random with a PLS, rating understandability on a 5-point scale. They then answered multiple-choice questions to assess understanding, rated the other two written PLS and ranked each in order of understandability. Finally, participants watched the video of their top-ranked written PLS and stated their preferred format. Results: There was no significant difference (p > 0.05) in participant-rated understandability for the first PLS viewed: mean (standard deviation) 3.9 ± 1.1 (human-written), 4.1 ± 0.9 (AI), and 3.9 ± 0.8 (AI-human). Answers were similar for the multiple-choice questions regardless of which PLS participants saw first. Forty-one participants (44%) ranked the human PLS in first place for understandability, 31 (33%) the AI, and 21 (23%) the AI-human. Fifty-four participants (58%) preferred the video PLS, 27 (29%) preferred the written PLS and 12 (13%) had no preference. Conclusion: GenAI generates PLS as informative as human-written PLS, achieving similar readability scores for people living with SCD.

Recommended Citation

Shastri O, Agrippa O, Moss V, Millard E, More L, Brown E, Sharif S, Finch C. Use of Generative Artificial Intelligence for Development of Plain Language Summaries: A Blinded Assessment of Education Preferences of the UK Sickle Cell Disease Community. Patient Experience Journal. ; 13(3):410-416. doi: 10.35680/2372-0247.2147.

Creative Commons License

This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.