This post has been republished via RSS; it originally appeared at: Microsoft Tech Community - Latest Blogs - .
Among my resolutions for this year, I've decided to embark on a delicious journey - baking. My love for sugary treats has inspired me to bring the bakery home. Why step outside when I can whip up my own delights? You might be wondering what any of this has to do with Azure or Open AI. Well, as with any journey, I need a roadmap on how to get started. So, I wanted to learn how to make, not just any cake, but this delicious red velvet cake I had a few months ago.
Using GPT-4 turbo with vision, I can upload an image of the cake and request for a recipe. GPT-4 with vision model allows images as prompts and the possibilities are endless, not only for your personal requests but also for your business.
Spoiler
Azure AI Studio is still in public preview and not recommended for production workloads.
Getting started with Azure AI Studio
Azure AI Studio is a multiplatform that enables developers to create generative AI applications and custom copilot experiences. Using Azure AI Studio, you not only get access to GPT-4 Turbo with vision, but you get additional Azure AI vision capabilities and functionality.
To get started with Azure AI Studio, follow the following steps:
2. In Azure AI Studio, go to the build tab and create a new project. Once you create a new project, change the project name and click on create a new resource.
3. Create a new Azure AI resource. GPT-4 Turbo with Vision is available in specific regions, therefore, create a resource in the regions: Switzerland Noth, Sweden Central, West US or Australia East. Once done, click next.
4. Click create project then give it a few minutes for your project to be created.
5. Once your project is created, on the menu, head over to deployments to deploy a new model in this case gpt-4.
6. Under deployment, select model version: vision-preview and click deploy.
7. Once done, head over to your playground. Once at the playground, ensure the mode is set to chat, the deployment is gpt-4 and switch on the Vision slider to ensure you get the Azure AI Vision capabilities in your model. Once done, you can go ahead and chat with the model using images.
Spoiler
Additional charges might be added for using the model with Azure AI Vision functionality.
Adding AI Vision to your chatbot
Earlier, we talked about additional capabilities in Azure AI Studio. Once you include vision capabilities to your model, you get additional capabilities such as Optical Character Recognition (OCR), object grounding, video prompts and adding your data with images. In the next section, I will cover what each capability does and how you can use it for your projects.
OCR: Build a bakery business
Optical Character Recognition (OCR), is the ability of the model to extract text from images and expand context of your prompts. Using OCR, gives the you more flexibility to upload even handwritten documents.
Along the year, maybe, I will fall in love with baking, and want to share my delights with the world. In this case, I might decide to start a business, a bakery business. As I got the idea, I decided to write down some ideas of what the business might be about.
Once done jotting down my ideas, I go ahead and request the model to refine my business description, tagline and maybe generate a logo for the business.
Bringing your data with images into Azure AI Studio
For the business to be successful, it needs products. The good news is, Azure AI Studio enables you to add your images and data to ground the model enabling your users to see what they will get when they order. Let's see how we can upload our products and successfully bring our data into Azure AI Studio.
Spoiler
I used GPT-4 to generate images and descriptions of the different products.
1. In the playground, select add your data. Under add your data, in select data source, select upload files. Next accept cross-origin resource sharing to be able to upload files to the Azure Blob storage created when creating your project. Next, select create a new Azure AI Search resource, to index your data.
2. When you click create a new Azure AI Search it will redirect you to Azure to create the resource.
3. Next, go back to Azure AI Studio and update your AI search resource. In addition, enter a name for the index you intend to create. Lastly, check the two boxes to acknowledge additional charges on the resource you just created.
4. Next, upload Files to your blob storage. Once your images are uploaded, add image descriptions and metadata to your data then click next.
5. Review the files you just uploaded then finish
6. Before you start chatting, ensure you data is added by checking the index and resources you just created are added.
7. Let's see if our data has successfully been added by asking bakery related questions. We'll go back to the red velvet cake and see how much we can get a for a similar cake from the store.
Video Prompts.
Lastly, Azure AI Studio enables you use video as prompts where the model retries the video frames relevant to the users' prompt. Maybe, we create a simple advertisement for the bakery, showcasing the bakery's products.
To ensure your advertisement communicates what was intended, you might want to request a breakdown of your video.
In addition, you can request the chatbot on ways to make it more interesting and appealing to your customers.
Clean Up
Once you are done, it's important to clean up and delete any resources you've created. This is a crucial step to prevent unnecessary depletion of your credits.
Conclusion
In summary, GPT-4 with vision gives you the power to add images to add images into your prompts. When you add Azure AI Vision, the possibility are endless in how you can now craft your prompt to generate better responses and push your business further. Sign up today to try out GPT-4 in Azure at: . If you have a startup or an idea you can also sign up to Founders Hub where once accepted, you will get free Azure credits to take your business to the next level.
Curious to learn more on GPT-4 with vision and Azure AI Studio? You can utilize the resources below:
- You can find all the prompts and images at: BethanyJep/Bethany-s-bakery: Using Azure AI Vision to enhance GPT-4 Turbo with Vision Responses (github.com)
- Get started using GPT-4 Turbo with Vision on your images and videos in Azure AI Studio - Azure AI Studio | Microsoft Learn
- Use your image data with Azure OpenAI Service - Azure OpenAI | Microsoft Learn
- How to use the GPT-4 Turbo with Vision model - Azure OpenAI Service | Microsoft Learn