Summary

Now that SharePoint 2019 supports SMTP authentication you have the option to relay e-mail from SharePoint to any SMTP server that supports SMTP authentication. However, if you are using a port that falls back to anonymous, how do you if it's authenticating and not just falling back to anonymous

Let's look at the traffic behind the covers

To answer the question, you really need to inspect the traffic on the wire. Below I have provided 4 scenarios which customers may see when attempting to configure SMTP Auth with on-prem Exchange Servers.

Scenario 1: Auth working with port 587 over Kerberos

Here you will find a working configuration where SharePoint is set to use port 587 and SMTP Auth over Kerberos. This worked well without errors and the outcome was expected.

Conversation on the wire:

Exchange Receive Connector Permissions:

SharePoint SMTP Auth Settings:

Scenario 2: Auth working with port 25 over NTLM

Here you will find a working configuration where SharePoint is set to use port 25 and SMTP Auth over NTLM. This worked well without errors and the outcome was expected.

Conversation on the wire:

SharePoint SMTP Auth Setting:

Exchange Receive Connector Permissions:

Take special note to this Exchange Receive Connector permission group settings. "Exchange Users" is enabled. SMTP Auth (as a user) requires the "Exchange Users" permission group, which is not on by default for the "Default Frontend SERVERNAME" receive connector, which listens on port 25.

Scenario 3: Somewhat working with port 25 over NTLM

Here you will find a "somewhat working" configuration where SharePoint is set to use port 25 over NTLM and SMTP Auth. However, the e-mail was sent, but anonymous was used instead of user authentication as expected. This failure occurred due to a configuration error, which is described below.

Conversation on the wire:

Errors Text:

Response: 535 5.7.3 Authentication unsuccessful\r\n

Response code: Authentication credentials invalid (535)

Response parameter: 5.7.3 Authentication unsuccessful

SharePoint SMTP Auth Setting:

Exchange Receive Connector Permissions:

Take special note to this Exchange Receive Connector permission group settings, as this is the default setting. SMTP Auth (as a user) requires the "Exchange Users" permission group, which is not on by default for the "Default Frontend EXCHANGE" receive connector, which listens on port 25.

Scenario 4: Not working with port 587 over Kerberos

Here you will find a "non-working" configuration where SharePoint is set to use port 587 over Kerberos and SMTP Auth. This resulted in failure after 3 attempts due to a configuration error, which is described below.

Conversation on the wire:

Errors Text:

Response: 535 5.7.3 Unable to proxy authenticated session because either the backend does not support it or failed to resolve the user\r\n

Response code: Authentication credentials invalid (535)

Response parameter: 5.7.3 Unable to proxy authenticated session because either the backend does not support it or failed to resolve the user

Response: 530 5.7.57 SMTP; Client was not authenticated to send anonymous mail during MAIL FROM\r\n

Response code: Authentication required (530)

Response parameter: 5.7.57 SMTP; Client was not authenticated to send anonymous mail during MAIL FROM

SharePoint SMTP Auth Setting:

Exchange Receive Connector Permissions:

Why is this happening?

There are several reasons why SharePoint SMTP authentication can fail, and I will list the reasons I'm aware of here.

1. Missing UPN: If the authenticating user does not have a UPN set, authentication will fail with error "5.7.3 Unable to proxy authenticated session because either the backend does not support it or failed to resolve the user". If you see this error, check the user configured in the SharePoint outing e-mail settings,  to determine if the UPN is set. There are many ways to accomplish this, but I will show you the PowerShell method.

Check with Get-ADUser:

Get-ADUser spadmin

DistinguishedName : CN=SpAdmin,OU=SpService,DC=contoso,DC=com
Enabled : True
GivenName :
Name : SpAdmin
ObjectClass : user
ObjectGUID : 070d91df-5240-4569-8c1c-415bc9c20872
SamAccountName : SpAdmin
SID : S-1-5-21-1111111111-111111111-111111111-1111
Surname :
UserPrincipalName :

Set the UPN for the SMTP Authenticating Account:

Set-ADUser spadmin -UserPrincipalName spadmin@contoso.com

Now Get it account one more time:

Get-ADUser spadmin

DistinguishedName : CN=SpAdmin,OU=SpService,DC=contoso,DC=com
Enabled : True
GivenName :
Name : SpAdmin
ObjectClass : user
ObjectGUID : 070d91df-5240-4569-8c1c-415bc9c20872
SamAccountName : SpAdmin
SID : S-1-5-21-1111111111-111111111-111111111-1111
Surname :
UserPrincipalName : spadmin@contoso.com

Note: Now the UPN is set

Example of a missing SPN:

setspn -l contoso\exchange

Registered ServicePrincipalNames for CN=EXCHANGE,CN=Computers,DC=contoso,DC=com:
TERMSRV/EXCHANGE
TERMSRV/exchange.contoso.com
WSMAN/exchange
WSMAN/exchange.contoso.com
RestrictedKrbHost/EXCHANGE
HOST/EXCHANGE
RestrictedKrbHost/exchange.contoso.com
HOST/exchange.contoso.com

Set the SPN with NETBIOS Name:

setspn -s SMTPSVC/exchange contoso\EXCHANGE

Checking domain DC=contoso,DC=com
Registering ServicePrincipalNames for CN=SPWFE,CN=Computers,DC=contoso,DC=com:
SMTPSVC/exchange
Updated object

Set the SPN with FQDN:

setspn -s SMTPSVC/exchange.contoso.com contoso\EXCHANGE

Checking domain DC=contoso,DC=com
Registering ServicePrincipalNames for CN=EXCHANGE,CN=Computers,DC=contoso,DC=com:
SMTPSVC/exchange.contoso.com
Updated object

Get the SPNs once again:

setspn -l contoso\exchange

Registered ServicePrincipalNames for CN=EXCHANGE,CN=Computers,DC=contoso,DC=com:
SMTPSVC/exchange.contoso.com
SMTPSVC/EXCHANGE

TERMSRV/EXCHANGE
TERMSRV/exchange.contoso.com
WSMAN/exchange
WSMAN/exchange.contoso.com
RestrictedKrbHost/EXCHANGE
HOST/EXCHANGE
RestrictedKrbHost/exchange.contoso.com
HOST/exchange.contoso.com

Conclusion

In summary don't set it and forget it! After configuring your SharePoint Server to use SMTP Authentication, check your network traffic to ensure its working as expected. You may need to work with your Active Directory (to set SPNs) and Exchange Team (to set connector properties) to ensure that E-mail is being sent using authentication. Also, if you came here because this blog matches the errors you are experiencing, I hope this helped you figure out the problem quickly.

Thanks for reading!