Introducing AWS HealthScribe – automatically generate clinical notes from patient-clinician conversations using AWS HealthScribe

Introduction

Today, we are excited to announce AWS HealthScribe (preview), a new HIPAA-eligible service empowering healthcare software vendors to build clinical applications that automatically generate preliminary clinical notes by analyzing patient-clinician conversations.

At AWS, we have been investing in healthcare and life sciences with purpose-built services that reinvent how customers collaborate, make data driven clinical and operational decisions, enable precision medicine, and decrease the cost of care. AWS provides cutting edge capabilities that you can use to build high-performance, population-scale applications that can store, transform, analyze, and access healthcare and life science data. With purpose-built capabilities customers can manage and drive insights from multiple types of healthcare and scientific data including clinical records, genomic and other omics data, medical imaging, and unstructured medical text and speech.

Documentation requirements impede the patient consultation experience

Imagine a busy day at the clinic, where clinicians spend their day juggling appointments as they try to provide quality care to every patient. In addition to this back-to-back schedule with limited breaks, clinicians must maintain detailed documentation for every patient visit. The time and effort spent on this necessary administrative work often takes away from the invaluable face-to-face interactions with patients.

Industry requirements demand meticulous documentation. Clinicians often spend nearly twice as much time on administrative tasks instead of face-to-face interactions with patients¹. This creates a struggle between providing compassionate care and maintaining accurate records. This burden takes a toll on both clinicians and patients. Patients receive less attention from their healthcare providers while the clinicians faces a higher risk of burnout and decreased job satisfaction. Medical scribes have helped alleviate the documentation workload, but they can be costly to hire, train, and retain, and often face similar burnout due to the time-consuming nature of documentation.

A New Era: AI-Powered Medical Transcription Solutions

AI holds the potential to transform the clinical documentation process by significantly reducing clinician and medical scribe involvement in administrative tasks. Traditional assistive AI agents have been limited in their ability to understand the context of transcribed conversations. However, advancements in generative AI and large language models have significantly improved contextual understanding.

At its core, generative AI learns to identify and recreate complex patterns within the data it is trained on. This capability makes it uniquely suited for the challenges in healthcare industry where data complexity and diversity pose significant challenges. By accelerating tasks that were once laborious and time-consuming, generative AI unlocks new opportunities for building assistive tools that free up time for patient care to transform healthcare delivery.

Despite the promise of AI, healthcare application developers face several challenges when building and integrating AI into clinical applications.

1. Implementation Complexity: Training, optimizing, and integrating conversational and generative AI services can be time-consuming and expensive.
2. Security: Developers must ensure that AI-powered solutions meet stringent security, privacy, and healthcare compliance requirements, adding another layer of complexity to the development process.
3. Trust: Lack of trust in AI-generated clinical notes and potential use of patient data in model training can lead to hesitation in adopting an AI-based solution.

Introducing AWS HealthScribe

AWS HealthScribe is a purpose-built HIPAA-eligible service empowering healthcare software vendors to develop clinical applications that automatically generate clinical notes by transcribing and summarizing patient-clinician conversations. AWS HealthScribe combines conversational and generative AI to reduce clinical documentation burden and enhance consultation experience. With AWS HealthScribe, you can provide a robust suite of AI-powered features designed to accelerate clinical documentation in your clinical application. AWS HealthScribe analyzes the patient-clinician conversation audio to provide:

1. Rich Consultation Transcripts: AWS HealthScribe provides a comprehensive turn-by-turn transcript with word-level timestamps for each dialogue in the transcript.
2. Speaker Role Identification:  Individuals present in the exam room are uniquely identified in the transcript and dialogue is attributed to the doctor or patient. This enables a clear understanding of “who said what” in the doctor-patient interaction throughout the encounter.
3. Transcript Segmentation: AWS HealthScribe categorizes transcript dialogues and organizes the clinically relevant portions into appropriate summary sections such as subjective, objective, assessment, and plan. It also identifies small talk and periods of silence in the conversation which helps users locate specific portions of the transcript.
4. Summarized Clinical Notes: AWS HealthScribe analyzes consultation conversations to generate summarized clinical notes for sections like chief complaint, history of present illness, assessment, and plan. These summaries are easy to review, edit, and finalize, and can provide quick recap highlights of patient visits to clinicians and scribes.
5. Evidence Mapping: Every sentence used in the AI-generated clinical note includes references to the original consultation transcript, making it easier for users to validate the accuracy of the summary.
6. Structured Medical Terms: AWS HealthScribe extracts structured medical terms from the conversation transcript, such as medical conditions, medications, and treatments. These medical terms can be used to generate useful workflow suggestions and auto-suggest relevant entries for various fields in your clinical application.

Healthcare application builders can integrate AWS HealthScribe in their clinical application to provide key highlights of the patient visit to the medical practitioner.

Fig 1: Illustrative example of the application experience that healthcare developers can provide users with AWS HealthScribe.

By consolidating these capabilities, AWS HealthScribe reduces the need for training, optimizing, and integrating separate AI services and building custom models, allowing for faster implementation. Customers can focus on delivering value to their end users without worrying about optimizing individual AI components.

A great example of this is how healthcare software vendors like 3M, ScribeEMR, and Babylon are already using AWS HealthScribe to power their clinical applications.

3M Health Information Systems (HIS) is an industry leader whose various M*Modal speech understanding, conversational and ambient AI solutions are currently used by more than 300,000 clinicians.

“Machine learning on AWS enables 3M HIS to transform clinician workflows and laborious processes to help health care organizations streamline clinical documentation and billing,” said Garri Garrison, President, 3M HIS. “3M HIS is collaborating with AWS to bring conversational and generative AI directly into clinical documentation workflows. AWS HealthScribe will be a core component of our clinician applications to help expedite, refine and scale the delivery of 3M’s ambient clinical documentation and virtual assistant solutions.”

Babylon is an integrated digital-first primary care service that manages population health at scale.

“Integrating AI with human medical expertise can make quality healthcare more affordable and accessible, and alleviates burdens on providers,” said Saurabh Johri, chief science officer, Babylon. “Innovating in areas like clinical summarization is one example with the potential to improve healthcare outcomes. As a leader in AI innovation, Babylon looks forward to continuing our collaboration with AWS and exploring integrating AWS HealthScribe’s generative AI capabilities with our natural language processing solutions.”

ScribeEMR is a leading provider of virtual medical scribing, virtual medical coding, and virtual medical office services for hundreds of medical practices, hospitals, and health systems.

“ScribeEMR’s goal is to help increase practice efficiency, maximize revenue, and reduce clinician burnout in the healthcare industry,” said Daya Shankar, co-founder and general manager at ScribeEMR, Inc. “By harnessing the power of AWS HealthScribe, we can transform the process of healthcare documentation using generative AI. With AWS HealthScribe, our advanced processes can now capture and interpret patient visits more effectively and optimize EMR workflows, coding, and reimbursement processes.”

Fig 2: AWS HealthScribe is designed responsibly with each AI-generated summary sentence linked back to consultation transcript.

Built responsibly with a focus on security and privacy

AWS HealthScribe is a HIPAA-eligible service that prioritizes patient data security and privacy. AWS will not use inputs or outputs generated through the AWS HealthScribe service to train AWS HealthScribe. Users have full control over their data and determine where they prefer to store transcriptions and preliminary clinical notes.

AWS HealthScribe is designed to be used in assistive role with the goal of making documentation easier for medical practitioners. Each AI-generated summary sentence is linked back to the consultation transcript, allowing users to easily verify accuracy by cross-referencing the source and understand the context behind the AI-generated note. Providing traceability and transparency for AI-generated insights aligns with the Responsible AI principle of explainability and helps foster trust and safe use of AI in the clinical setting.

Conclusion

Together with healthcare software vendors, AWS is helping customers improve the clinician-patient consultation experience. In your clinical application, AWS HealthScribe automatically provides and segments rich conversation transcripts, identifies speaker roles for patients and clinicians, extracts medical terms, and generates preliminary clinical notes. AWS HealthScribe combines these capabilities to reduce the need to integrate and optimize separate AI services, expediting implementation. AWS HealthScribe helps healthcare software vendors use AI responsibly by including references to original patient transcripts for every sentence in the AI-generated clinical notes. Security and privacy are built-in to AWS HealthScribe to protect sensitive patient data.

AWS HealthScribe is available in preview in US East (N. Virginia). To access the service, visit the AWS HealthScribe sign-up page and product page.

[1] Physicians spend nearly twice as much time on EHR/desk work as patients