This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .
Introduction
The previous version of Azure OpenAI Services Virtual Assistant was limited to being a client-side application, which required users to share two API keys, making it less secure and not suitable for production. Additionally, monitoring usage and preparing for course assessments on using OpenAI Chatgpt was challenging. Finally, cost-saving was also a concern.
To address these issues, we have developed a serverless solution using Azure Static Web Apps.
This new solution eliminates the need for users to share API keys, provides better monitoring of usage, and reduces costs. With Azure Static Website, users can access the virtual assistant through a secure and reliable serverless platform, making it an ideal choice for both students and professionals alike.
If you haven't seen our previous post of the AI Virtual Teaching Assistant see Build a Virtual Assistant with Azure Open AI and Azure Speech Service.
Demo In Cantonese
Demo In English
Walkthrough Demo from Login in Cantonese
Architecture of the Solution
The virtual assistant application is built on Azure Static Web Apps and Azure Function, and uses Active Directory Application to authenticate users, with email as their identity. Each API call is proxied through Azure Function, and before each call, the user's email address is extracted from the request header and checked against the "User" storage table to ensure authorization. User messages, Chatgpt responses and tokens usage are stored in the "Chat" storage table for future reference. For debugging purposes, the speed-to-text and text-to-speech Azure Function saves the WAV file to the "Voice" blob storage, with a lifecycle policy set to delete files after 7 days. Deployment is handled through a GitHub Repo with GitHub Action, streamlining the process, and ensuring consistent updates across all environments. Overall, this architecture provides a secure, reliable, and scalable solution for virtual assistant applications.
Costing Concern
The primary operating cost of ChatGPT is determined by the number of tokens consumed. We have established a daily token usage limit of 50,000 tokens per user, but you can customize this limit for individual users by including a "Limit" property in their user record.
Predefined Prompts
We have added an exciting new feature to our virtual assistant application - the awesome-chatgpt-prompts collection. This collection includes a wide range of prompt examples that can be used with the ChatGPT model, allowing users to easily customize their virtual assistant to do something interesting with just one click. Whether you want your virtual assistant to tell jokes, provide weather updates, or play games, the awesome-chatgpt-prompts collection has something for everyone. This new feature not only adds an element of fun and personalization to the virtual assistant, but also enhances its functionality and usability for users.
Deploying the solution
- First, go to the GitHub repository at https://github.com/wongcyrus/AzureOpenAILive2DChatbot/tree/AzureStaticSite.
- Create a CodeSpace for the project.
- Login to Azure CLI using the command "az login --use-device-code".
- Open a terminal and set the GitHub Token using the command "export GITHUB_TOKEN=xxxxx", replacing "xxxxx" with your actual GitHub Token.
- Run the script "./deploy.sh".
- Once the deployment is complete, a new repo named "AzureOpenAILive2DChatbotCICD" will be created.
- If any GitHub Actions failed during the deployment, rerun them.
- Open Microsoft Azure Storage Explorer and update the users table. The PartitionKey and RowKey should be set to the user's email address.
Deployment Demo
Technology Behind the scenes of deployment
The CDK for Terraform project is a powerful tool that allows you to create all Azure and GitHub resources needed for your virtual assistant application. It is written in TypeScript and provides a simple, yet powerful, interface for managing resources. With CDKTF, you can easily create and manage Azure resources such as cognitive accounts, storage account, function apps, and static web apps. Additionally, it enables you to set up GitHub resources such as repositories and actions to streamline your deployment process. Overall, the CDKTF project greatly simplifies the process of setting up and managing your virtual assistant application, providing a seamless and efficient experience for developers.
import { Construct } from "constructs";
import { App, TerraformOutput, TerraformStack } from "cdktf";
import { AzurermProvider } from "@cdktf/provider-azurerm/lib/provider";
import { ResourceGroup } from "@cdktf/provider-azurerm/lib/resource-group";
import { AzureadProvider } from "@cdktf/provider-azuread/lib/provider";
import { AzapiProvider } from "./.gen/providers/azapi/provider";
import { ChatStorageAccountConstruct } from "./components/chat-storage-account";
import { CognitiveAccountConstruct } from "./components/cognitive";
import { StaticSiteConstruct } from "./components/static-site";
import { GitHubConstruct } from "./components/github";
class AzureOpenAiLive2DChatbotStack extends TerraformStack {
constructor(scope: Construct, id: string) {
super(scope, id);
new AzurermProvider(this, "azure", {
features: {
resourceGroup: {
preventDeletionIfContainsResources: false
}
},
skipProviderRegistration: false
});
new AzureadProvider(this, "azuread", {});
new AzapiProvider(this, "azapi", {});
const repository = "AzureOpenAILive2DChatbotCICD";
const uniquePrefix = "ivechat";
const region = "eastasia";
const resourceGroup = new ResourceGroup(this, 'resourceGroup', {
name: `azure-openai-live2d-chatbot`,
location: region,
});
const chatStorageAccountConstruct = new ChatStorageAccountConstruct(this, "chatStorageAccount", {
uniquePrefix: uniquePrefix,
resourceGroup: resourceGroup
});
const cognitiveAccountConstruct = new CognitiveAccountConstruct(this, "cognitiveAccount", {
uniquePrefix: uniquePrefix,
resourceGroup: resourceGroup
});
const staticSiteConstruct = new StaticSiteConstruct(this, "staticSite", {
resourceGroup: resourceGroup,
chatStorageAccountConnectionString: chatStorageAccountConstruct.chatStorageAccount.primaryConnectionString,
openAiCognitiveAccount: cognitiveAccountConstruct.openAiCognitiveAccount.primaryAccessKey,
openAiCognitiveDeploymentName: cognitiveAccountConstruct.openAiCognitiveDeployment.name,
ttsApiKey: cognitiveAccountConstruct.ttsCognitiveAccount.primaryAccessKey,
speechRegion: "EastUS"
});
new GitHubConstruct(this, "github", {
repository: repository,
clientID: staticSiteConstruct.live2DApplication.id,
clientSecret: staticSiteConstruct.live2DApplicationPassword.value,
apiToken: staticSiteConstruct.live2DStaticSite.apiKey,
});
......
To create Cognitive Resources
this.ttsCognitiveAccount = new CognitiveAccount(this, "ttsCognitiveAccount", {
name: props.uniquePrefix + "TTSCognitiveServicesAccount",
resourceGroupName: props.resourceGroup.name,
location: "EastUS",
kind: "SpeechServices",
skuName: "S0",
});
this.openAiCognitiveAccount = new CognitiveAccount(this, "openAiCognitiveAccount", {
name: props.uniquePrefix + "OpenAiCognitiveServicesAccount",
resourceGroupName: props.resourceGroup.name,
location: "EastUS",
kind: "OpenAI",
skuName: "S0",
});
this.openAiCognitiveDeployment = new CognitiveDeployment(this, "openAiCognitiveDeployment", {
name: props.uniquePrefix + "OpenAiCognitiveServicesDeployment",
cognitiveAccountId: this.openAiCognitiveAccount.id,
model: {
name: "gpt-35-turbo",
format: "OpenAI",
version: "0301",
},
scale: {
type: "Standard"
}
});
GitHub Repo and Action secrets.
new GithubProvider(this, "GitHubProvider", {
token: process.env.GITHUB_TOKEN,
});
new Repository(this, "Repository", {
name: props.repository,
visibility: "public",
template:
{
repository: "AzureOpenAILive2DChatbotDemo",
owner: "wongcyrus",
includeAllBranches: true
}
});
new ActionsSecret(this, "ClientIdActionsSecret", {
repository: props.repository,
secretName: "AADB2C_PROVIDER_CLIENT_ID",
plaintextValue: props.clientID
});
new ActionsSecret(this, "ClientSecretActionsSecret", {
repository: props.repository,
secretName: "AADB2C_PROVIDER_CLIENT_SECRET",
plaintextValue: props.clientSecret
});
new ActionsSecret(this, "DeploymentTokenActionsSecret", {
repository: props.repository,
secretName: "AZURE_STATIC_WEB_APPS_API_TOKEN",
plaintextValue: props.apiToken
});
Azure Static Web Apps
this.live2DStaticSite = new StaticSite(this, "live2DStaticSite", {
location: props.resourceGroup.location,
name: "live2DStaticSite",
resourceGroupName: props.resourceGroup.name,
skuTier: "Free",
});
const live2DApplicationInsights = new ApplicationInsights(this, "live2DApplicationInsights", {
name: "live2DApplicationInsights",
resourceGroupName: props.resourceGroup.name,
location: props.resourceGroup.location,
applicationType: "web",
})
new ResourceAction(this, "live2DStaticSiteAction", {
type: "Microsoft.Web/staticSites/config@2022-03-01",
resourceId: this.live2DStaticSite.id + "/config/appsettings",
method: "PUT",
body: `${Fn.jsonencode({
"properties": {
"APPINSIGHTS_INSTRUMENTATIONKEY": `${live2DApplicationInsights.instrumentationKey}`,
"APPLICATIONINSIGHTS_CONNECTION_STRING": `${live2DApplicationInsights.connectionString}`,
"chatStorageAccountConnectionString": `${props.chatStorageAccountConnectionString}`,
"openAiCognitiveAccount": `${props.openAiCognitiveAccount}`,
"openAiCognitiveDeploymentName": `${props.openAiCognitiveDeploymentName}`,
"ttsApiKey": `${props.ttsApiKey}`,
"speechRegion": `${props.speechRegion}`,
},
"kind": "appsettings"
})}`
});
this.live2DApplication = new Application(this, "live2DApplication", {
displayName: "AzureOpenAiLive2DChatbot",
signInAudience: "AzureADMyOrg",
web: {
redirectUris: ["https://" + this.live2DStaticSite.defaultHostName + "/.auth/login/aadb2c/callback"],
implicitGrant: {
accessTokenIssuanceEnabled: true,
idTokenIssuanceEnabled: true
}
}
});
this.live2DApplicationPassword = new ApplicationPassword(this, "live2DApplicationPwd", {
applicationObjectId: this.live2DApplication.objectId,
displayName: "live2DApplication cred",
});
GitHub Repo
Project Source https://github.com/wongcyrus/AzureOpenAILive2DChatbot
Conclusion
With this update, all Higher Diploma in Cloud and Data Centre Administration can use a specialized Azure OpenAI Chatgpt for their school work. Our plan will teach and assess student problem solving abilities with the support of virtual assistant. We can know is the prompt technique of each student and setup the extremely complicated assignment or test on top of using AI. The grade will be based on the minimum number of prompts and token consumption required to achieve optimal solution or answer.
The update to the Azure OpenAI Chatgpt virtual assistant and the integration of the awesome-chatgpt-prompts collection will be highly beneficial for Higher Diploma in Cloud and Data Centre Administration students. It will provide them with a specialized tool to support their school work and charge up their problem-solving abilities, while also allowing for the assessment of their prompt technique through the use of AI. With the virtual assistant, students can access information and resources quickly and efficiently, enabling them to complete their assignments and tests in a more streamlined and effective manner.
Additionally, the use of the CDKTF project will simplify the creation and management of all Azure and GitHub resources, making it easier for schools to get started with the virtual assistant. Overall, this update will provide a highly valuable resource for students and educators alike, facilitating a more efficient and effective learning experience.
Project collaborators include Shing Seto, Stanley Leung, Ka Ka Leung, XU YUAN and Hang Ming (Leo) Kwok from the IT114115 Higher Diploma in Cloud and Data Centre Administration.
About the Author
Cyrus Wong is the senior lecturer of Department of Information Technology (IT) of the Hong Kong Institute of Vocational Education (Lee Wa... and he focuses on teaching public Cloud technologies. He is one of the Microsoft Learn for Educators Ambassador and Microsoft Azure MVP from Hong Kong.
