This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.
Healio - Healthcare Chatbot: Access information from different data sources and create its own plan using C# Semantic Kernel, planner and Azure OpenAI
In this article we will go through a business use case of developing Healio - Healthcare Chatbot: Access information from different data sources and create its own plan using C# Semantic Kernel, planner and Azure OpenAI. You might be familiar with the fact that Retrieval Augmented Generation (RAG) pattern is an easy and efficient way to allow Azure OpenAI to "talk to your data". This pattern lets you query your data with Azure Cognitive Search (a search engine), find pertinent pieces of information from your data, then include that extra information to your prompts to the Azure OpenAI service to reply in a natural language.
This isn’t always enough
However, the RAG pattern has some limitations when you examine how it is implemented. It relies on a single data source (Cognitive Search). Cognitive Search can index many kinds of data sources (PDFs, Word docs, etc). It excels at searching unstructured data. Also, I need to copy the results from the Cognitive Search and add them to the prompt by myself in code. I had to process the results of Cognitive Search by myself in code.
This leads to several questions:
- What if I want to use multiple data sources to retrieve the data necessary to answer the user’s question?
- What if I don’t know or don’t want to hard-code the order of operation for calling multiple data sources?
- What if I don’t want to write the parsing code for calling multiple data sources with disparate data formats?
- How can I let AI “orchestrate” the API calls to answer questions & pull together data that I couldn’t predict beforehand?
To solve these more complex issues, I need more than the RAG pattern.
More complex example
Pretend that you are building the healthcare chatbot to help answer common customer support questions about health-related information, advice, and support to patients, visitors, and staff. They can also help with booking appointments, checking symptoms, and accessing medical records. Here are some common data sources that customer support uses to answer customer questions.
- Administrative data: This data is generated from claims, encounter, enrollment, and providers systems. It includes information such as type of service, diagnosis and procedure codes, location of service, and amount billed and reimbursed. This data can be used to answer questions about billing, insurance, and utilization of services.
- Patient medical records: This data is documentation of a patient’s medical history and care. It contains rich clinical detail such as diagnoses, treatments, medications, lab results, and outcomes. This data can be used to answer questions about a patient’s health condition, treatment plan, and progress.
- Patient surveys: This data is collected from survey instruments that capture self-reported information from patients about their health care experiences. It covers aspects such as satisfaction, quality, access, and outcomes of care. This data can be used to answer questions about a patient’s feedback, preferences, and expectations.
- Hospital databases: This data is available from various sources, such as individual hospitals, Doctor’s schedule, hospital associations, state and regional data organizations, health departments, and federal agencies,. It includes information such as hospital characteristics, performance measures, quality indicators, and patient outcomes. This data can be used to answer questions about a hospital’s reputation, accreditation, services, and achievements.
Complex user questions could be answered by combining these data sources. These data sources could be used in different systems, each with varying authorization needs, varying languages, varying protocols, etc.
We have to tell the OpenAI service when it should use our external data sources, what type of input data to give them & what type of data to get back. These instructions need to be in terms that humans can understand, since that is what the big language model is using.
Example complex user query
“How can I schedule an appointment with Dr. Smith, who performed my surgery last month, for a follow-up checkup next week and look availability of doctor in nearest hospital in my network?”
Let’s break this question down.
We need to know the availability of Dr. Smith. Just looking the past record of patient (as a simple RAG implementation would do) couldn’t answer this question and there is no guarantee that Dr. Smith is still working in same shift or with same hospital. We need to look the hospital’s database and check his schedule and availability.
Next, we have to find out the patient's previous medical history. We already checked the doctor's schedule, so we have the patient and doctor information.
In this case, we want to know the type of care service, cost, billing and which hospital for next week. A good chatbot would ask for more information about where “exactly” the user was planning to visit e.g. nearest hospital in network.
The next step is to make AI come up with a plan to access the different data sources that we mentioned above. It has to access them in the correct order (connected all the dots), interpret the output (because each API will give different data and is not aware of the other APIs) and finally create a response.
Enter Semantic Kernel
Semantic Kernel is a free SDK (C#, Java, Python, JavaScript) that can help you integrate the Azure OpenAI service into your application. It lets you access the Azure OpenAI endpoints for “chat” and “embedding” functions. With Semantic Kernel, you can avoid making direct REST API calls to the service.
To go further, the Semantic Kernel SDK has a feature called a “planner”. The planner coordinates multiple calls to the Azure OpenAI service to create a plan to answer the user’s questions. You can then follow the plan.
With Semantic Kernel, you can also make "plugins", which are ways to invoke native code. These native plugins can perform mathematical calculations, read/write files or make API calls.
A lot of the effort in creating this kind of application will be in the “prompt engineering” of trying to tell the Semantic Kernel when to use your plugin and what arguments to give it.
In part 2, I will explain how I built a sample app that does this and implement our use case to develop Healthcare Chatbot – Healio.