SMTP, ESMTP, and the BDAT baddie

I recently had to troubleshoot a problem with an external SMTP service which was having difficulty delivering mail to our corporate mail server.  The delivering service was running Windows 2003 Standard and using the built-in Simple Mail Transfer Protocol (SMTP) service from IIS 6.0.  The receiving service was running Windows Server 2008, but also MS Exchange Server 2007 SP2.

Basically messages were not being received reliably.  Some came through and some didn’t.  The Message Tracking logs on Exchange 2007 didn’t yield much useful information, but before I turned up the logging level for the transport role, I took a look at the sending mail system.

Within C:\Windows\System32\LogFiles\SMTPSVC1 I found the most recent log file which recorded the following basic data around the failed email transmission:

22:15:27 – – 0
22:15:27 EHLO – 0
22:15:27 – – 0
22:15:27 MAIL – 0
22:15:27 – – 0
22:15:27 RCPT – 0
22:15:27 – – 0
22:15:27 BDAT – 0

I already knew that many security appliances do not like the new ESMTP BDAT command, so I Googled around and found this JoeKiller article which shed a little light on the subject, and that it was possible to force the session to not use the BDAT command at all.

By telneting to the service ‘telnet localhost 25’ and typing ‘ehlo’, the SMTP will list ESMTP verbs that it supports:

I knew I needed to remove BINARYMIME and CHUNKING, however little was mentioned regarding the exact steps to take, which in turn prompted this post.

Fortunately, I already had the IIS6.0 Resource Kit installed so was quick to find the SmtpInboundCommandSupportOptions value by opening the IIS Metabase Explorer, and navigating to LM\SmtpSvc\1

Here the default value was 7697601.  I knew that I wanted to disable the BINARYMIME and CHUNKING verbs so using the table here I subtracted 2097152 (BINARYMIME) and 1048576 (CHUNKING) from 797601:

7697601-2097152-1048576 = 4551873

I then set the SmtpInboundCommandSupportOptions value to 4551873, closed the IIS Metabase Explorer and restarted the IIS Admin Service (which in turn restarts the Simple Mail Transfer Protocol (SMTP) service).  Now the server only advertises and uses the following verbs:

Next was to restrict the sending of SMTP mail to not use the BDAT command either.  Back to the IIS Metabase Explorer, and change the value of SmtpOutboundCommandSupportOptions from 7 to 5.

Job done. Now I have a more firewall friendly mail host.