How Bing Chat Enterprise works with your data using GPT-4

QUICK LINKS: 

00:00 — Can you use GPT-based generative AI with your business data?

00:35 — What is Bing Chat Enterprise? 

01:03 — Bing Chat demonstration using up-to-date information 

02:05 — Bing Image Creator demonstration using DALL-E 

02:37 — How to access Bing Chat Enterprise and bringing protected data into prompts 

04:26 — How protections work with Bing Chat Enterprise 

05:21 — Admin controls for Bing Chat Enterprise 

06:09 — How Bing Chat Enterprise compares with Microsoft 365 Copilot 

Link References: 

Check out detailed documentation at https://aka.ms/BCEDocs 

Unfamiliar with Microsoft Mechanics? 

As Microsoft’s official video series for IT and tech enthusiasts, you can watch and share valuable content and demos of current and upcoming tech from the people who build it at Microsoft. 

• Subscribe to our YouTube: https://www.youtube.com/c/MicrosoftMechanicsSeries
• Talk with other IT Pros, join us on the Microsoft Tech Community: https://techcommunity.microsoft.com/t5/microsoft-mechanics-blog/bg-p/MicrosoftMechanicsBlog
• Watch or listen from anywhere, subscribe to our podcast: https://microsoftmechanics.libsyn.com/podcast

Keep getting this insider knowledge, join us on social: 

• Follow us on Twitter: https://twitter.com/MSFTMechanics
• Share knowledge on LinkedIn: https://www.linkedin.com/company/microsoft-mechanics/
• Enjoy us on Instagram: https://www.instagram.com/msftmechanics/
• Loosen up with us on TikTok: https://www.tiktok.com/@msftmechanics 

Video Transcript:

-By now we’ve all started to experience the power of generative AI with large language models like GPT. Using a single prompt, you can create content from existing data. In fact, by giving the model access to more specific information in your prompts, it can generate more relevant content and responses in a fraction of the time it would take for you to do it yourself. These services provide a lot of productivity advantages, but can you harness them while still retaining control over your organization’s data? Today I will show you how your organization can safely use generative AI with a closer look at Bing Chat Enterprise and its built-in commercial data protection. It uses GPT-4, included already as part of the service, and because it’s included with your Microsoft 365 Business subscription, it’s available for small organizations up to the largest enterprises. 

-Let’s start by looking at what Bing Chat Enterprise is and is not, along with some of the fundamental differences to public services like Chat GPT that you might be familiar with, and how Bing Chat Enterprise protects your data. The consumer Bing Chat service is an AI-powered chat bot. One of the core things that differentiates Bing Chat is that it’s grounded in up-to-date public web data. There is no knowledge cutoff based on when the model was trained. I’ll prompt Bing Chat with something tied to an event from the last couple of months. Where is Lionel Messi playing football these days? As you can see, Bing’s public search index grounds to the large language model with up-to-the-minute results using knowledge from the internet. Notice how distinct to the Bing experience the responses followed by source citations that you can always review to spot hallucinations in the model’s output. 

-Let’s try something else by asking it to write a poem about Lionel Messi’s career. Here you can see a poem generated based on the information and context available to it, and it knew when I typed his, I was referring to Lionel Messi, so it’s not constrained by a lack of knowledge. The underlying tech platform is constantly optimized for search relevance to ground the large language model, and the experience is continuously evolving. For example, beyond text outputs, Bing Chat also provides access to other large language models like DALL-E for more visual content generation with its Bing Image Creator. Now, what I’ve shown you up until this point is the consumer Bing Chat experience and Bing Chat Enterprise builds on this. It is still a web-based experience, but with some key differentiations. 

-Most importantly, Bing Chat Enterprise addresses legitimate concerns as to where your prompts and responses are stored and how this information might be used in the future. First, you can get to the experience in three ways. You can access it from Bing.com, the Microsoft Edge sidebar, and from Windows Copilot. Let’s try the experience out on Bing.com. Notice it is an authenticated web experience. Access to the service is controlled using the same Microsoft Entra ID system used to provide secure access to Microsoft 365 for your work or school accounts. I’m signed in here with my Entra ID work account. On my other desktop, I have my notes with specs for an unreleased product. Let’s say I want to write a value prop for it, so I’ll copy all of the text. Now, I’ll prompt use this information to write me a short value proposition on the Contoso Thunderbolt eBike and paste in everything from the clipboard, and you’ll see that Bing Chat Enterprise knows how to write a value prop, generating a response based on the information that I provided. 

-Next, I’ll ask Bing Chat Enterprise to compare specs of the top three eBikes in the market in 2023, and after, introducing each eBike, it provides a detailed tabular summary of key features and components for each eBike along with the citations. And if we fast forward in time to when the value prop is finished as a multi-page PDF file, then I can use the Edge sidebar to parse all the information in this PDF to generate social posts for our product launch. While Bing Chat Enterprise works on that, let me show you the rest of this PDF file on the left and what it’s parsing through. Without doing this, I would have needed to copy and paste each of these sections in the PDF one by one, and now I don’t have to. After a few seconds, it’s saved me a lot of time and written me some great draft social posts to quickly get started. 

-Now, let’s review what we just saw. First, I allowed the model to compare the confidential specs of our unreleased product with information on competitive offerings publicly available on the internet, and even started preparing for our social campaign. And I can feel confident in doing this because the conversation history is not retained and there is no risk of the data being used to train the underlying large language model. I’ll close this browser tab with a PDF and then hit the new topic button in this window with my spec and initial value prop, and when I do, behind the scenes, the temporary information stored and processed in the Bing Chat service is deleted, including all memory of the PDF. It’s gone. 

-As a point of reference, let’s go back to the Bing Chat consumer experience. Whereas you can see on the left under recent activity my chat history was recorded, whereas with Bing Chat Enterprise on the right, we’ve started a new session and there is no recent activity displayed. Because Microsoft purges this information and it’s not stored anywhere, it also means that from Microsoft 365 admins, there is no reporting or auditing of that data available. 

-That said, you do have a few controls as a Microsoft 365 admin. Using the licensing controls at the user or group level, you can configure the on-off state of the Bing Chat Enterprise service for anyone in your tenant. For the Edge sidebar, as we used with our PDF, there are controls to enable or disable whether Edge can use browser context. As a user, and if your policy permits it, you can control this in your Edge browser settings. Under app and notification settings in Bing Chat, you can choose to allow access to any webpage or PDF. 

-And for administrators, these settings are also available in group policy or, as you can see here in the Intune settings catalog, and you’ll find them under Microsoft Edge settings. So that’s what the core Bing Chat Enterprise service experience is and how it works. A question we get a lot is how it compares with Microsoft 365 Copilot and there are three main differences. First, Microsoft 365 Copilot operates entirely inside your tenant boundary. Your data never leaves that boundary. Second, Microsoft 365 Copilot can access and orchestrate your work data, including your emails, documents, and meetings, via the Microsoft Graph. It automatically retrieves this information and presents it to the large language model to generate a relevant grounded response. 

-Lastly, those experiences are tailored to the specific Microsoft 365 apps you’re using. In contrast, Bing Chat Enterprise is a separate service from Microsoft 365 that operates outside that tenant boundary. Its main information source is the public internet. It has no access to your internal company data. As demonstrated, you can safely augment your prompts by pasting in organizational data, and the service can combine this with knowledge from the web. All data is processed in the Bing Chat Enterprise service and is presented to the underlying large language model to generate a response, which includes a system prompt to help frame responses based on responsible AI practices. Again, chat data is only temporarily held during your open session and purged with each new chat session. 

-So that was an overview of Bing Chat Enterprise and how it can safely let your organization reap the benefits of generative AI today. If you’re considering Microsoft 365 Copilot, while you prepare your environment, Bing Chat Enterprise gives you an immediate and safe solution for you to take advantage of generative AI and for Microsoft 365 business users, it is at no additional cost. To learn more, check out aka.ms/BCEdocs. Subscribe to Microsoft Mechanics for the latest in tech updates, and thank you for watching.