In this article I will show you how to secure SMTP message flow between Exchange Server 2007 in different Exchange organizations. Securing SMTP traffic between different Exchange 2007 organizations is much simpler than in previous versions of Exchange.

Basics

Is it necessary to protect SMTP traffic between different Exchange Servers? Let’s try a simple test. Start a network trace with your favourite network traffic analyzer. In this example I used Microsoft Network Monitor 3.0. While the trace is running, start a Telnet session in your Exchange Server with port 25 and send a message over Telnet. Stop the network trace with Netmon and filter the captured traffic by the SMTP protocol. What do you see? Right, the whole authentication process of the SMTP session is clear text.

Figure 1: SMTP network trace with Netmon

Figure 2: Sending an SMTP message via Telnet

OK, we know that it is necessary to implement some kind of security between these Exchange Servers. What is the right solution to accomplish this? It is possible to use IPSEC between these Exchange servers but what does this mean in implementation work? At a minimum you have to use pre-shared keys to implement IPSEC between these servers. This should work well as long as you only have a few Exchange Servers. Another possible solution, when implementing IPSEC between more than a handful of servers, is certificates but if you want to implement certificates between Exchange Servers you will need a PKI (Public Key Infrastructure).

There is another new solution for securing SMTP traffic between these servers in Exchange Server 2007. You can use a built in function from Exchange Server 2007 to secure the SMTP traffic between Exchange 2007 servers in different Exchange Organizations.

Exchange Server 2007 uses several methods to ensure Message integrity and Message encryption.

• Mutual TLS
• Opportunistic TLS
• Direct Trust
• Domain Security

Mutual TLS

TLS (Transport Layer Security), the successor of Secure Sockets Layer (SSL), is used to encrypt message flow in Exchange Server 2007. The term Mutual means that both Exchange Servers that are involved in the message Transport process will check the TLS certificate before the connection is established. Mutual TLS is deployed in a configuration where both the sender and the receiver authenticate one another before they send the data.

Opportunistic TLS

Opportunistic TLS is new to Exchange Server 2007. Exchange Server 2007 tries to secure the Message flow with other Exchange Servers or foreign messaging systems. It also tries to enable a TLS session with the other messaging system in the form of an anonymous TLS request. This is different from Exchange Server 2003 where you must manually enable TLS between different Exchange Servers.

Direct Trust

All message traffic is automatically encrypted between Exchange Servers regardless whether a Hub Transport or Edge Transport role is used. Direct Trust doesn’t use the complex X.509 certificate validation mechanism; instead it uses a direct validation in the form of the presence of a certificate in Active Directory. It doesn’t matter if you use self signed certificates or an internal Certificate Authority.

Domain Security

Domain Security is a combination of different techniques and features such as certificate Management, Exchange Server connector functionality and the behaviour of messaging clients like Microsoft Outlook 2007. The design goal of Domain Security with Exchange Server 2007 is also to establish a secure connection with mutual TLS.

Implementing TLS security

For the purpose of securing mailflow with mutual TLS, you can use your Hub Transport servers or, if you have implemented it, you can use Exchange Servers with the Edge Server role.

As a first step you have to establish a cerfiticate cross Forest trust through the two Exchange organizations. At a minimum you have to add the Root CA certificate from the external Certification authority (CA) to the trusted Root CA certificate store on the Hub Transport or Edge Transport Server. If you have multiple Edge or Hub Transport Servers it may be better to implement cross CA certificate trust or to add the Root CA certificate to the Trusted Root CA store via Group Policies. The following screenshot shows the Root CA certificate of OrganizationB.

Figure 3: Root CA certificate from the other Exchange Organization

Subject name

Subject Names play an important part in certificates used by Exchange Server 2007. The subject Name of a TLS certificate is used by DNS aware services. A DNS aware service calls the subject name of a certificate and compares this name with a request. ISA Server is a good example of this, when publishing Outlook Web Access or Outlook Anywhere in an HTTPS bridging scenario where the common name on the certificate must exactly match the name in the URL that is used to access OWA or Outlook Anywhere. The Subject Name field in a certificate binds a certificate to a single server or a special domain name.

The following table gives you an overview of the frequently used Relative Distinguished Names also known as RDN.

Table 1: Commonly used Relative Distinguished Names

Request a certificate

The next step is to request a certificate via the Exchange Management Shell. The Certificate request file can be used to issue a certificate from the internal CA.

Figure 4: Request Exchange certificates

Open the CA web console and submit a certificate request by using a base-64 encoded CMC or PKCS#10 file.

Figure 5: Enable the certificate with the web console

The following picture shows an example of the certificate request file. If your browser doesn’t allow opening of files, you can copy and paste the entire text from the request file into the certificate request section of the web console.

Figure 6: The certificate request file

Submit the certificate request.

Figure 7: Submit the Certificate request

In the following screenshot you will see the issued certificates from the internal Certificate Authority.

Figure 8: Issued certificates

Import the Certificate

It is important that you use the Exchange Management Shell to import the Certificate.

Import-ExchangeCertificate -Path c:\certificates\import.pfx | Enable-ExchangeCertificate -Services SMTP

Figure 9: Import the Certiciate into Exchange

Allow the Domain domaene.tld as a secure Domain list with the Exchange Management Shell

Set-TransportConfig -TLSReceiveDomainSecureList domaene.tld

Figure 10: Enable Domain Secure List

Enabling Domain Security on the SMTP Send Connector named “Outbound”

Set-SendConnector Outbound -DomainSecureEnabled:$True

Figure 11: Enable Domain Security with TLS in the Exchange Management Console

Enabling Domain Security on the SMTP Receive Connector named “Inbound”

Set-ReceiveConnector Inbound -DomainSecureEnabled:$True -AuthMechanism TLS

Please note:
E-Mail messages that have been successfully delivered through the domain secured mail flow connection are displayed in Outlook 2007 as "Domain Secure" messages.

Conclusion

As you have seen in this article, implementing secure SMTP messaging between Exchange 2007 servers in different Exchange 2007 organizations isn’t complicated and you don’t need a complicated solution like implementing IPSEC between these Servers.

Related Links

Implementing Domain Security for Exchange Server 2007