Deploy a Phi-3 model in Azure AI, and consume it with C# and Semantic Kernel

This post has been republished via RSS; it originally appeared at: New blog articles in Microsoft Community Hub.

Hi friends!

Let's keep writing about Phi-3, and today's post is a demo about how to deploy Phi-3 model to Azure AI, and consuming it using C# and Semantic Kernel. This guide will walk you through the process, covering chat completion text scenarios and vision scenarios.

Why Phi-3 and Semantic Kernel?

Phi-3 is a state-of-the-art model that brings advanced AI capabilities to your projects, enhancing both text and vision scenarios. Coupled with the Semantic Kernel, an orchestrator that handles authentication and interactions with the cloud, you have a robust setup for deploying and using AI models efficiently.

Bonus: Semantic Kernel allows literally zero code changes to move from a local Phi-3 deployment to a cloud Phi-3 deployment for your applications.

Step 1: Deploy Phi-3 on Azure AI

First things first, let's get Phi-3 up and running on Azure AI. Follow these steps:

Create an Azure AI Hub: Head over to the Azure AI Studio and create a new Azure AI Hub.
Deploy the Phi-3 model: In the deployments, deploy a new base model, and select a Phi-3 model.

01 Deploy model in Azure AI.png

Serverless API: select the server API option.

02 Select Content Filter.png

Content Filter (Optional): Enable Content Filter if you want to use this model with a CF.

03 enable content filter.png

Endpoint Data: After a short time the model will be deployed. In the Endpoint / API Routes section, copy the Chat Completion Name, and the Endpoint. Also copy the Key.

04 Model Details.png

Step 2: Let's code!

With Phi-3 deployed, we can write C# code. Below there are 2 samples on how to use the Azure model for text and vision scenarios. In both scenarios the Azure Model details are stored using User Secrets. In example, you need to setup your projects secrets with the following commands:

dotnet user-secrets init dotnet user-secrets set "AZURE_AI_PHI3_MODEL" "phi3-mini-4k" dotnet user-secrets set "AZURE_AI_PHI3_ENDPOINT" "< endpoint >" dotnet user-secrets set "AZURE_AI_PHI3_APIKEY" "< api key > "

Chat Completion Text Scenario

Here’s a simple example of how to use Phi-3 for generating chat responses.

#pragma warning disable SKEXP0010 using Microsoft.Extensions.Configuration; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.ChatCompletion; var config = new ConfigurationBuilder().AddUserSecrets<Program>().Build(); var modelId = config["AZURE_AI_PHI3_MODEL"]; var endPoint = config["AZURE_AI_PHI3_ENDPOINT"]; var apiKey = config["AZURE_AI_PHI3_APIKEY"]; // create kernel var builder = Kernel.CreateBuilder(); builder.AddOpenAIChatCompletion(modelId, new Uri(endPoint), apiKey); var kernel = builder.Build(); // create chat var chat = kernel.GetRequiredService<IChatCompletionService>(); var history = new ChatHistory(); // run chat while (true) { Console.Write("Q: "); var userQ = Console.ReadLine(); if (string.IsNullOrEmpty(userQ)) { break; } history.AddUserMessage(userQ); Console.Write($"Phi3: "); var response = ""; var result = chat.GetStreamingChatMessageContentsAsync(history); await foreach (var message in result) { Console.Write(message.Content); response += message.Content; } history.AddAssistantMessage(response); Console.WriteLine(""); }

Vision Scenario

For vision scenarios, Phi-3 can analyze images and provide insights. Here’s an example of how to achieve this.

#pragma warning disable SKEXP0010 using Microsoft.Extensions.Configuration; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.ChatCompletion; var config = new ConfigurationBuilder().AddUserSecrets<Program>().Build(); var modelId = config["AZURE_AI_PHI3V_MODEL"]; var endPoint = config["AZURE_AI_PHI3V_ENDPOINT"]; var apiKey = config["AZURE_AI_PHI3V_APIKEY"]; // create kernel var builder = Kernel.CreateBuilder(); builder.AddOpenAIChatCompletion(modelId, new Uri(endPoint), apiKey); var kernel = builder.Build(); // create chat var chat = kernel.GetRequiredService<IChatCompletionService>(); var history = new ChatHistory(); var petsMusicImagePath = Path.Combine(Directory.GetCurrentDirectory(), "imgs", "petsmusic.png"); // get the mime type from the image var mimeType = "image/png"; // create chat collection items var collectionItems = new ChatMessageContentItemCollection { new TextContent("What's in the image?"), new ImageContent(File.ReadAllBytes(petsMusicImagePath), mimeType) }; history.AddUserMessage(collectionItems); Console.Write($"Phi3: "); var result = await chat.GetChatMessageContentsAsync(history); Console.WriteLine(result[^1].Content); Console.WriteLine("");

Recording

The following 5 minute video shows the full process and running demos:

Wrapping Up

Deploying and using Phi-3 on Azure AI with C# and Semantic Kernel opens up a world of possibilities in a simple way. Whether you're working on chatbots, vision applications, or any other AI-powered solution, this setup provides a scalable and efficient way to harness the power of Phi-3.

For more detailed instructions and examples, check out the Phi-3 Cookbook.

Happy coding!

Bruno Capuano