This post has been republished via RSS; it originally appeared at: Storage at Microsoft articles.
First published on TECHNET on Nov 26, 2012In this post, we will discuss the instrumentation available in Server for NFS in Windows Server 2012 and how it can be used to detect and diagnose any deployment and operational issues.
Event Viewer
There are quite few changes in the Server for NFS event model for Windows Server 2012. In the previous releases of Windows Server, Server for NFS logged events in to the System channel. In Windows Server 2012, Server for NFS logs the events into its own channel. The event IDs are unchanged; however event channels and provider GUIDs are different. The following figure displays the layout of the event channel, where Server for NFS logs events.
Activity Logging
Server for NFS logs the events for some of the NFS operations into Operational channel which includes:
- Read and Write
- Lock and Unlock
- Mount and Unmount
- Create and Delete
The activity logging can be enabled using the PowerShell cmdlet Set-NfsServerConfiguration.
For example, the following command enables the activity logging for mount, read, and write operations.
PS C:\> Set-NfsServerConfiguration –LogActivity mount,read,write
- Open Server Manager and then click Services for Network File System (NFS) from the Tools menu.
- In Services for Network File System, right-click on Server for NFS and select Properties.
- Switch to Activity Logging tab and select the activities you want to be logged.
Identity Mapping Events
Server for NFS logs identity mapping related events into the IdentityMapping channel. The following are some of the critical events to watch for when local file based identity mapping is configured as the identity mapping source.
|
Event ID |
Level |
Message |
Resolution |
|
4025 |
Error |
A duplicate ID value <UID/GID number> was found when loading <FileName>. The file will not be used as a mapping source |
Server for NFS performs some validations against the passwd and group files. This event is logged if multiple user accounts in the passwd file have the same user identifier (UID) or multiple group accounts in the group file have the same group identifier (GID). To resolve this issue, edit the passwd/group files to change the UID/GID on the conflicting user/group account having this issue. Use Get-NfsMappedIdentity PowerShell cmdlet to retrieve the list of users/groups having the UID/GID mentioned in the event. |
|
4026 |
Error |
A duplicate name <AccountName> was found when loading <FileName>. The file will not be used as a mapping source |
Edit the file specified in the event and remove the duplicate account name. |
|
4027 |
Error |
A syntax error was found on line <LineNumber> when loading <FileName>. The file will not be used as a mapping source |
The passwd/group file specified in the event is not following the correct syntax required by Server for NFS. Edit the file and check for any errors at the line number mentioned. |
|
4029 |
Warning |
Mapping source update detected. File <FileName> not found |
Server for NFS looks for the passwd and group files at following location, %windir%\system32\drivers\etc. Make sure that the files are present at this location and NfsService has permission to read these files. [MCJ1] |
|
4030 |
Error |
<FileName> has no data. The file will not be used as a mapping source. |
The passwd/group file is empty. Make sure that you have right files stored at location %windir%\system32\drivers\etc or remove the files from this location if it was not intended to use mapping files as identity mapping source. |
|
4032 |
Error |
<FileName>. Memory allocation failed when processing the file. It will not be used as a mapping source. |
The system is overloaded and there is not enough memory available to process the request. Close some of the applications that are not required to free the memory. |
|
4033 |
Error |
<FileName>. Failed to process the file. The file will not be used as a mapping source. |
Unexpected error encountered while opening the file specified in the event. Check the file for correct syntax. |
The following are some of the critical events when the identity mapping store is Active Directory, Active Directory Lightweight Directory Services or other RFC2307 compliant LDAP store.
|
Event ID |
Level |
Message |
Resolution |
|
4012 |
Error |
Active Directory Domain Services(R) contains multiple users which match attribute <AttributeName>. Only one Windows(R) user should be associated with each UNIX UID. With multiple Windows users associated with one UNIX UID, Server for NFS cannot determine which Windows user is requesting access to files. No Windows users associated with the same UNIX UID will be able to access files through Server for NFS. Try removing the duplicate UNIX UID entries. |
Event 4012 indicates that the configured identity mapping store contains multiple user accounts that have an identical value for attribute uidNumber (the value is given in the event message text). Run the following PowerShell command to find out the user accounts having identical value for attribute uidNumber. Get-NfsMappedIdentity –AccountType user –Uid <UIdNumber> Then correct the value of the uidNumber attribute of the user accounts using the following PowerShell command. Set-NfsMappedIdentity –UserName <sAMAccountName> -UId <UidNumber> |
|
4013 |
Error |
Active Directory Domain Services(R) contains multiple groups which match attribute <AttributeName>. Only one Windows(R) group should be associated with each UNIX GID. With multiple Windows groups associated with one UNIX GID, Server for NFS cannot determine which Windows group to use to grant access to files. Try removing the duplicate UNIX GID entries. |
Event 4013 indicates that the configured identity mapping store contains multiple group accounts that have an identical value for attribute gidNumber (the value is given in the event message text). Run following PowerShell command to find out the group accounts having identical value for attribute gidNumber. Get-NfsMappedIdentity –AccountType group –Gid <GIdNumber> Use following PowerShell command to correct the value of the gidNumber attribute of the group account. Set-NfsMappedIdentity –GroupName <sAMAccountName> -GId <GidNumber> |
|
4014 |
Error |
Active Directory Domain Services(R) contains multiple users which match attribute <AttributeName>. Only one Windows(R) user should be associated with each sAMAccountName. With multiple Windows users associated with one sAMAccountName, Server for NFS cannot determine which Windows user is requesting access to files. No Windows users associated with the same sAMAccountName will be able to access files through Server for NFS. Try removing the duplicate sAMAccountName entries. |
Event 4014 indicates that the configured identity mapping store contains multiple users that have an identical value for attribute sAMAccountName (the value is given in the event message text). Try removing the duplicate user accounts having identical sAMAccountName. |
|
4015 |
Error |
Active Directory Domain Services(R) contains multiple groups which match attribute <AttributeName>. Only one Windows(R) group should be associated with each sAMAccountName. With multiple Windows groups associated with one sAMAccountName, Server for NFS cannot determine which Windows group to use to grant access to files. Try removing the duplicate sAMAccountName entries. |
Event 4015 indicates that the configured identity mapping store contains multiple groups that have an identical value for attribute sAMAccountName (the value is given in the event message text). Try removing the duplicate group accounts having identical sAMAccountName. |
|
4016 |
Error |
Server for NFS could not connect to the Lightweight Directory Access Protocol (LDAP) server for domain <DomainName>. Without a connection to the LDAP server, Server for NFS cannot query for Windows-to-UNIX user account mappings and cannot grant file access to any user. Verify that Server for NFS is configured to use the appropriate LDAP server using the Nfsadmin command-line tool. |
Event 4016 indicates that Server for NFS is not configured to use either Active Directory Domain Services (AD DS) or any other LDAP store or User Name Mapping as a Windows-UNIX identity mapping source. Use Set-NfsMappingStore PowerShell cmdlet to set the identity mapping store for the Server for NFS. |
|
4017 |
Error |
Server for NFS could not find any Lightweight Directory Access Protocol (LDAP) accounts which match attribute <AttributeName>. Without attribute <AttributeName>, Server for NFS does not know the corresponding Windows user account for the Unix user and cannot grant file access to the UNIX user.%n%n Verify that the LDAP server is configured with the appropriate attributes. |
Event 4017 indicates that Server for NFS could not find any Lightweight Directory Access Protocol (LDAP) accounts that match the attribute specified in the event message text. Add the necessary account information to the LDAP store by using New-NfsMappedIdentity or set-NfsMappedIdentity cmdlet. Then use Resolve-NfsMappedIdentity cmdlet to verify that Server for NFS is able to resolve the user account using the attribute specified in the event text. |
The following are some of the critical events when the identity mapping store User Name Mapping (UNM) server.
|
Event ID |
Level |
Message |
Resolution |
|
1005 |
Error |
Server for NFS could not obtain mapping information from User Name Mapping. Server for NFS will make another attempt after <Duration> minutes. Without any mapping information, Server for NFS will not be able grant file access to users. Verify the User Name Mapping service is started on the User Name Mapping server, and User Name Mapping ports are open on firewalls. |
Event 1005 indicates that Server for NFS cannot obtain mapping information from User Name Mapping (UNM) server. Incorrect settings in User Name Mapping source could cause this. Use Set-NfsMappingStore PowerShell cmdlet to configure User Name Mapping server. Get-NfsMappingStore cmdlet can be used to retrieve the current configuration. Use Resolve-NfsMappedIdentity cmdlet to verify that Server for NFS can obtain the mapping information from UNM server. |
|
1006 |
Error |
Server for NFS is not configured for either Active Directory Lookup or User Name Mapping. Without either Active Directory Lookup or User Name Mapping configured for the server, or Unmapped UNIX User Access configured on all shares, Server for NFS cannot grant file access to users. Configure Server for NFS for either Active Directory Lookup or User Name Mapping using the Nfsadmin command-line tool, or Unmapped UNIX User Access using the Nfsshare command-line tool. |
Event 1006 indicates that Server for NFS is not configured for either Active Directory Lookup or User Name Mapping. If you have configured shares on the Server for NFS to use ‘Unmapped UNIX User Access’ mode, you may ignore this event. Otherwise, to solve this problem, configure Server for NFS to use an identity mapping source using Set-NfsMappingStore PowerShell cmdlet. To verify that the mapping store is configured correctly, use Get-NfsMappingStore cmdlet. |
|
1056 |
Error |
Server for NFS could not obtain updated mapping information from User Name Mapping. Server for NFS will continue to use the mapping information it has and make another attempt after <Duration> minutes. If this problem persists, Server for NFS mapping information may become significantly out-of-date and may not be able grant file access to users. Verify that the User Name Mapping service is started either locally or on the remote server, and that User Name Mapping ports are open on firewalls. |
Event 1056 indicates that Server for NFS cannot obtain mapping information from User Name Mapping (UNM) server. Incorrect settings in User Name Mapping source could cause this. Use Set-NfsMappingStore PowerShell cmdlet to configure User Name Mapping server. Get-NfsMappingStore cmdlet can be used to retrieve the current mapping store configuration. Use Resolve-NfsMappedIdentity cmdlet to verify that Server for NFS can obtain the mapping information from UNM server. |
Admin Channel Events
The Server for NFS logs critical events that need admin’s intervention into Admin channel. Following are some of the critical events and recommended resolution steps.
|
Event ID |
Level |
Message |
Resolution |
|
1059 |
Error |
Server for NFS could not register with RPC Port Mapper on all requested port/protocol combinations. Server for NFS will attempt to continue but some NFS clients may not function properly. Network File System (NFS) clients discover NFS servers by querying the port mapper for a remote server (also known as Portmap and Rpcbind). NFS clients may not be able to discover and communicate with Server for NFS on this computer. |
These events indicate that other programs might be using some of the TCP/IP ports that are required by Server for. Determine if Server for NFS has registered all protocols To determine the ports and transports that Server for NFS uses, at an elevated command prompt on the affected server, type rpcinfo. Server for NFS registers on port 2049 for udp, tcp, udp6, tcp6 Make this TCP/IP port available and restart Server for NFS. To make TCP/IP port 2049 available and restart Server for NFS, use the following procedure: 1. At an elevated command prompt, type “netstat -a -b –o” to display all connections with their associated executables and processes. 2. Resolve port allocations conflicting with the NFS ports identified in Step 1 by stopping conflicting services, or programs. 3. Type “nfsadmin server stop”. 4. Type “nfsadmin server start”.
|
|
1060 |
Error |
Server for NFS could not register the Network File System (NFS) protocol on the specified port (%5). Status: %6. Server for NFS is will attempt to continue. At least one successful NFS port registration is required to start Server for NFS but some NFS clients may not function properly without this specific port registration. Verify that no other programs have registered with RPC Port Mapper with the following parameters. Program Name:%1 Program Number%2 Version:%3 Protocol: %4 Port:%5 |
|
|
1064 |
Warning |
Server for NFS cannot initialize the volume with drive letter %1 for sharing. Network File System (NFS) shared resources on the volume will not be available to NFS clients. Windows(R) may be low on system resources. Try increasing available system resources by closing programs, then restart Server for NFS manually. |
Event 1064 indicates that Server for NFS cannot provision the volume for sharing; therefore, shared resources on the volume will not be available to NFS clients. The likely cause is that the computer is short of resources. To resolve this issue increase available system resources using the following procedure: 1. Close all programs and stop unnecessary services on the affected server. 2. At an elevated PowerShell prompt , type “nfsadmin server stop”. 3. Type “nfsadmin server start”. To verify Server for NFS is sharing files, use the following procedure: 1. On the affected server, type Get-NfsShare. 2. Verify that the list of shared resources is correct. |
|
1069 |
Error |
Server for NFS could not establish a connection with configured NIS server |
Event 1069 indicates that Server for NFS is unable to access the Network Information Service (NIS) store where the netgroup configuration is stored. The most likely causes are: • NFS server is not configured appropriately to access NIS based netgroups. • There is a network connectivity issue between the Server for NFS and the NIS server. If Server for NFS is unable to access the netgroup store, determine if the location of the NIS NetGroup Source is accurate by using the following procedure: 1. At PowerShell prompt on the affected server, type Get-NfsNetgroupStore. 2. Verify that the NISDomain, and NISServer are configured correctly. 3. Verify that network connectivity exists between the Server for NFS and the NIS server where netgroups are configured as follows: • Use the rpcinfo.exe tool to verify that the NIS server is accessible over the network. To check if the source computer is accessible and the NIS service is registered on the source computer, type the following command, where <computername> is the name of the NIS server: rpcinfo <computername>. • The NIS service should appear in the output of this command as RPC program number 100004 and protocol version 2. Verify Server for NFS is configured appropriately to access NIS server Verify that Server for NFS is correctly configured to access the NIS server as follows: 1. In PowerShell window, run Get-NfsServerConfiguration cmdlet. 2. Verify that Protocol for NIS is UDP, TCP, or both, and is compatible with the protocol allowed at the NIS source computer as determined from the output of the command rpcinfo.exe <computername>. To verify that issue is resolved, use Get-NfsNetGroup cmdlet. You should be able to retrieve the netgroups from the netgroup store. |
|
1071 |
Warning |
Server for NFS was unable to obtain security information for the configured UnmappedUnixUserUsername user account %1. Check that the user account %1 is valid and meets all configured security policies. There may be additional information in the Windows Security event log. Server for NFS will attempt to revert to the default anonymous account. MSV Status: %2, SubStatus: %3S4U Status: %4, SubStatus: %5 |
Event 1071 indicates that Server for NFS was unable to obtain a logon token for the account used to process anonymous logons or for UNIX UIDs that do not have an explicit mapping. The event message details the account that led to the problem report. Ensure that the account is valid and can be used to perform a successful logon.
|
|
1072 |
Warning |
Server for NFS was unable to obtain security information for the GSS user account %1. Check that the user account %1 is valid and meets all configured security policies. There may be additional information in the Windows Security event log. MSV Status: %2, SubStatus: %3 S4U Status: %4, SubStatus: %5 |
Event 1072 indicates Server for NFS was unable to obtain a logon token for the account used to access the NFS server when using an RPCSEC_GSS based identity. The event message details the account that led to the problem report. Ensure that the account is valid and can be used to perform a successful logon. |
|
1073 |
Warning |
Server for NFS was unable to obtain or refresh security information for the user account %1. Check that the user account %1 is valid and meets all configured security policies. There may be additional information in the Windows Security event log.%n%nMSV Status: %2, SubStatus: %3%nS4U Status: %4, SubStatus: %5 |
Event 1073 indicates Server for NFS was unable to refresh an access token. The event message details the account that led to the problem report. Ensure that the account is valid and can be used to perform a successful logon.
|
|
4021 |
Error |
The Server for NFS was unable to begin monitoring of NFS related cluster events (%1). The Server for NFS will continue in a non-clustered mode. |
These events indicate that either the Cluster Service is not running or the computer is low on resources. Determine if the Cluster Service is running as follows: 1. At command prompt on the affected server, type services.msc. 2. Check if Cluster Service is running.
|
|
4022 |
Error |
The Server for NFS thread monitoring NFS related cluster events ended unexpectedly (%1). The Server for NFS will continue in a non-clustered mode. |
|
|
4023 |
Warning |
Server for NFS encountered an error condition when checking for the presence of Failover Clustering (%1) and will continue to operate but in a non-clustered configuration only. To re-detect Failover Clustering and allow Server for NFS to operate in a clustered configuration, Server for NFS should be restarted using either the Services for Network File System (NFS) administrative tool or nfsadmin server stop and nfsadmin server start |
Performance Counters
Server for NFS-NFSv4 Statistics
This performance counter set includes performance counters related to compound requests processed by Server for NFS. It also includes a performance counter indicating the count of virtual servers hosted by Server for NFS.
|
Name |
Description |
|
Total Compound Requests |
Total number of compound requests processed by Server for NFS since startup |
|
Successful Compound Responses |
Total number of compound requests succeeded since Server for NFS started |
|
Failed Compound Responses |
Total number of compound requests failed since Server for NFS started |
|
Total Virtual Servers |
Current count of virtual servers hosted by the Server for NFS. This counter is incremented when virtual server is successfully started and decremented on virtual server stop. This counter will be set to one in non-cluster case. In case of cluster, there will be one instance of virtual server per Server for NFS resource. |
Server for NFS-Netgroup
|
Name |
Description |
|
Failures Communicating With NIS |
Number of time the Server for NFS failed to communicate with the NIS server. |
Server for NFS-User Mapping
LDAP refers to Active Directory, Active Directory Lightweight Directory Services or any other RFC 2307-based LDAP Store. UNM Server refers to User Name Mapping server.
|
Name |
Description |
|
Total LDAP Requests |
Number of LDAP query requests made by the Server for NFS since startup. |
|
Total LDAP successes |
Count of LDAP lookup requests which resulted in successful UID/GID to account name or account name to UID/GID lookup. |
|
Total LDAP Failures |
Count of LDAP lookup requests which failed to retrieve the identity mapping information from LDAP store. |
|
Total LDAP Requests Per Second |
Number of LDAP lookup requests performed per second by the Server for NFS. |
|
Total UNMP Requests |
Number of user name mapping lookup requests performed by the Server for NFS since startup. |
|
Total UNMP Failures |
Count of user name mapping lookup request issued by the Server for NFS which resulted in failure. The failure reason could be anything like mapping does not exist or communication failure with the UNM server. |
|
Total UNMP Successes |
Count of mapping lookup request made against UNM Server which resulted in successful mapping information. |
|
Total UNMP Requests Per Second |
Count of UNMP mapping lookup requests issued by the Server for NFS per second. |
|
Average LDAP Lookup Latency |
Average amount of time taken by Server for NFS to resolve UID/GID to account name from the LDAP mapping store and vice versa. It is the total time spent doing the lookup in the LDAP mapping store divided by the total number of mapping lookup requests made to the LDAP mapping store. |
|
Maximum LDAP Lookup Latency |
Maximum amount of time taken by Server for NFS to resolve the identity mapping in the LDAP mapping store. |
|
Average UNMP Lookup Latency |
Average amount of time taken by Server for NFS to resolve the UID/GID to account name from UNMP mapping store and vice versa. It is basically the total time spent doing the lookup in the UNMP mapping store divided by the total number of mapping lookup requests made by the server to the UNMP mapping store. |
|
Maximum UNMP Lookup Latency |
Maximum amount of time spent by Server for NFS to resolve the identity mapping from the UNMP mapping store. |
Server for NFS-NFSv4 Read Write Statistics
|
Name |
Description |
|
Total cached MDL Reads |
Number of times the read operation is performed using a memory descriptor list (MDL) from the system cache manager. |
|
Total Fast IO Reads |
Number of times the read operation is performed using buffered IO from the system cache manager. |
|
Total Unstable Writes |
Count of NFS unstable writes performed by Server for NFS. |
|
Average Fast IO Read Latency |
Average time taken by Server for NFS to perform read operation using buffered IO from the system cache manager. It is the total time taken by the server performing all buffered IO reads divided by the number of buffered IO reads performed so far. |
|
Average Non Fast IO Read Latency |
Average amount of time taken by Server for NFS to perform read operations using IRP based IO. |
Server for NFS-NFSv4 Request/Response Sizes
|
Name |
Description |
|
Maximum Size of NTFS Reads |
Maximum size in bytes of the read request performed by Server for NFS. |
|
Minimum Size of NTFS Reads |
Minimum size in bytes of the read request performed by Server for NFS. |
|
Maximum Size of NTFS Writes |
Maximum size in bytes of the write request performed by Server for NFS. |
|
Minimum Size of NTFS Writes |
Minimum size in bytes of the write request performed by Server for NFS. |
|
Maximum Compound Request Size |
Maximum size in bytes of the NFS compound request. |
|
Average Compound Request Size |
Average size in bytes of the NFS compound request. |
|
Maximum Compound Reply Size |
Maximum size in bytes of the NFS compound reply. |
|
Average Compound Reply Size |
Average size in bytes of the NFS compound reply. |
|
Maximum Compound Operations in Request |
Maximum number of operations in a single NFS compound request. |
|
Average Compound Operations in Request |
Average number of operations in NFS compound request. |
Server for NFS-NFSv4 Throughput
|
Name |
Description |
|
NFS Compounds Processed/Sec |
Number of NFS compound requests processed per second. |
Server for NFS- NFSv4 Operation Statistics
Server for NFS- NFSv4 Operation Statistics performance counter set is reported for each compound operation. There is one instance of the following of performance counters for each compound operation in the Server for NFS-NFS v4 implementation.
|
Name |
Description |
|
Count Of Operations Processed |
Count of this NFS4 compound operations processed by the Server for NFS so far. |
|
% Operations At Dispatch |
This counter is not used in the current release. |
|
Average Number of Times Operation requeued |
Average Number of times this compound operation was re-queued for processing by the worker thread. |
|
Reply Packet Not Cached Count |
Number of times reply packet was not cached when requested by the client. |
|
Average latency |
Average amount of time taken by the server to execute this compound operation. This includes time taken to decoding the request and executing the operation. |
Server for NFS - Session and Connection Statistics
|
Name |
Description |
|
Active Sessions Count |
Number of active sessions with Server for NFS. |
|
Active Connections Count |
Number of active connections with Server for NFS. |
|
Total Bad Session Requests |
Number of session requests (OP_CREATE_SESSION) received by the Server for NFS so far with invalid arguments to operation. |
|
KRB5 RPCSEC_GSS Requests Count |
Number of requests received by Server for NFS with krb5 RPCSEC_GSS authentication. |
|
KRB5I RPCSEC_GSS Requests Count |
Number of requests received by Server for NFS with krb5i RPCSEC_GSS authentication. |
|
KRB5P RPCSEC_GSS Requests Count |
Number of requests received by Server for NFS with krb5p RPCSEC_GSS authentication. |
|
AUTH_NONE Requests Count |
Number of requests received by Server for NFS with AUTH_NONE authentication. |
|
AUTH_UNIX Requests Count |
Number of requests received by Server for NFS with AUTH_UNIX authentication. |
|
Client With Sessions |
Current count of clients that have session established with Server for NFS. |
|
Total Client With Sessions |
Number of clients that have created a session to Server for NFS since startup. |
|
Number of times admin forcefully closed a session |
Number of sessions force-closed by an administrator (Disconnect-NfsSession cmdlet). |
|
Number Of Times Admin Revoked State |
Number of open/lock states force-closed by an administrator (Revoke-NfsOpenFile / Revoke-NfsClientLock cmdlets). |
|
Lease Expiry Revoke State Count |
Number of open/lock states revoked by Server for NFs due to the lease expiry. |
|
Client Sessions Using Back Channel |
Number of client sessions using back channel. |
|
Clients Requesting SP4_MACH State Protection |
Number of clients requesting SP4_MACH state protection. |
|
Clients Requesting SP4_NONE State Protection |
Number of clients requesting SP4_NONE state protection. |
|
Clients Requesting SP4_SSV State Protection |
Number of clients requesting SP4_SSV state protection. |
|
Clients Requesting Bind Principal To State |
Number of clients requesting bind principal to state. |
|
Clients Requesting Persistent Session |
Number of clients requesting a persistent session. |
|
Number of clients requesting READW_LT |
Count of requests for READW_LT. |
|
Number of clients requesting WRITEW_LT |
Count of requests for WRITEW_LT. |
|
Special Anonymous State ID Use Count |
Count of requests for special anonymous state ID. |
|
Special Read Bypass State ID Use Count |
Count of requests for special read bypass state ID. |
|
Special Current State ID Use Count |
Count of requests for special current state ID. |