This post has been republished via RSS; it originally appeared at: Core Infrastructure and Security Blog articles.
Hi everyone, my name is Tobias Kathein and I’m a Senior Engineer in Microsoft’s Customer Success Unit. Together with my colleagues Victor Zeilinger, Serge Gourraud and Rodrigo Sanchez from Customer Service & Support we’re going to discuss a real-world scenario in which a customer was unable to live migrate Virtual Machines in his newly set up Hyper-V environment.
In our scenario the customer was trying to initiate a Live Migration for a Virtual Machine from a remote system. This is quite a common scenario, that administrators open the Hyper-V Management console on an administrative Remote Desktop Services server and initiate the Live Migration of a VM between two Hyper-V hosts. The customer got doubts whether this is even opposed to work. Just to rule this one out upfront. Yes, it is opposed to work.
The customer was complaining that this isn’t working for him in his environment even though he set up the delegation correctly and enabled Kerberos as authentication protocol for Live Migrations. The issue wasn’t with a particular Virtual Machine, as all, even newly created VMs failed to be moved to another host. No matter if the Hyper-V Management console or the PowerShell Cmdlet Move-VM is used both fail. The error message returned is “No credentials are available in the security package (0x8009030E)”. The full error message including some additional details is shown below.
Even though the red error message in PowerShell looks a little bit fancier, it is the same error message that is returned telling us that there are no suitable credentials available. So, you can be assured the issue is not with the Hyper-V Management console nor with the Move-VM Cmdlet, because neither of them is working.
There are multiple reasons why Live Migrations fail with the message “No Credentials are available in the security package (0x8009030E).”
The most known cause of this issue is the absence a correct Kerberos Constrained Delegation. Either Kerberos Delegation is missing completely or for single services like in this case for CIFS or the Microsoft Virtual System Migration Service. Also don’t mix up the Microsoft Virtual System Migration Service with the Microsoft Virtual Console Service which can happen quite easily when using ADUC to configure Constrained Delegation as you can see below. The default column size doesn’t show what’s what.
Finding out which Kerberos Delegation entries have been configured is a little bit unclear in the ADUC. An easier way to verify all required entries are present is to run the following PowerShell command.
Starting Windows Server 2016 there is the need to select “Use any authentication protocol” when setting up the Kerberos Delegation, instead of “Use Kerberos only”. This is due to some changes made in the operating system that require protocol transition. Protocol transition is only possible if the above-mentioned option is selected. On systems older than Windows Server 2016 selecting “Use Kerberos only” is sufficient. If “Use any authentication protocol” was not selected, Live Migration initiated from remote hosts will fail.
The error message also appears when trying to move a VM and the account that is being used to initiate the Live Migration is member of the Protected Users group. Members of this group automatically have non-configurable protections applied to their accounts. Among other things the user’s credentials are not allowed to be passed along and therefore Live Migration will not work when initiated from a remote system.
Another possibility why Live Migrations fail with this error message is when the user account being used to initiate the Live Migration has the option “Account is sensitive and cannot be delegated” set to enabled. This is sometimes configured to avoid highly privileged accounts to ensure that the credentials of these accounts cannot be forwarded by a trusted application to another computer or service. However, accounts having configured this setting cannot be used to initiate a Live Migration between two Hyper-V from a third machine.
And that’s it. We hope to have shed some light on this topic and the posting was helpful for you. Thanks for reading and never stop live migrating.
Disclaimer
The sample scripts are not supported under any Microsoft standard support program or service. The sample scripts are provided AS IS without warranty of any kind. Microsoft further disclaims all implied warranties including, without limitation, any implied warranties of merchantability or of fitness for a particular purpose. The entire risk arising out of the use or performance of the sample scripts and documentation remains with you. In no event shall Microsoft, its authors, or anyone else involved in the creation, production, or delivery of the scripts be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use the sample scripts or documentation, even if Microsoft has been advised of the possibility of such damages.