Anti-Spam by Using Edge Transport Server (Part 1)

by Jaap Wesselius [Published on 7 Oct. 2010 / Last Updated on 7 Oct. 2010]

Using the Edge Transport Server as anti-spam.

If you would like to be notified of when Jaap Wesselius releases the next part in this article series please sign up to our MSExchange.org Real Time Article Update newsletter.

Introduction

To implement an anti-spam solution you can implement an Exchange Server 2010 Edge Transport Server. This server is typically located in the DMZ where it accepts the SMTP messages coming from the Internet. After processing the message stream, the ‘clean’ stream of SMTP messages is then delivered to the internal Hub Transport Server. Since the Edge Transport Server is located in the DMZ it is not a member of the internal Active Directory domain, so some additional configuration is needed.

Location of the Edge Transport Server

As I already said, the Edge Transport Server is typically located in the DMZ of the network, between the internet and the internal network.


Figure 1: The test environment with the Edge Server between the Internet and the (internal) Hub Transport Server

Messages are delivered from the internet to the Edge Transport Server and then sent to the Hub Transport Server. Outbound messages from the Hub Transport Server are forwarded to the Edge Transport Server and the Edge Transport Server in turn is responsible for the delivery to the Internet. A mechanism needs to be in place to keep the Edge Transport Server up-to-date with information about the internal Active Directory. Since the Edge Transport Server is not a member of the internal Active Directory, it has no knowledge about for example Accepted Domains or Recipients.

Installing the Edge Transport Server

Installing the Edge Transport Server is straight forward. Install Windows Server 2008 R2 (I always recommend R2 since it is more efficient and faster than Windows Server 2008) and bring it up to date with the latest hotfixes. Configure the external network interface and the internal network interface. Make sure that name resolution is fully functional. The Edge Transport Server needs to be able to resolve all domain names on the Internet, but should also be able to locate the internal Hub Transport Server.

An important aspect is that the Edge Transport Server needs to have a Fully Qualified Domain Name, including a DNS suffix. To configure a DNS suffix, open the Computer Properties. In the “Computer name, domain and workgroup settings” section click “Change Settings” and in the System Properties window, click the Change button. This is the part where you normally enter a domain name when you want to join a domain. Right now click the More button and enter the primary DNS Suffix. Click OK and reboot the computer.


Figure 2: Change the primary DNS suffix

After rebooting, install the prerequisite software:

  • .NET Framework 3.5.1 (via Server Manager);
  • HotfixKB982867;
  • HotfixKB979744;
  • Hotfix KB983440;
  • Hotfix KB977020;
  • Office 2010 Filter Pack

After installing these prerequisites you can startup the installation application. Follow the wizard en select a “custom setup” when installing the Edge Transport Server. In the Server Role selection window, select the “Edge Transport Role” and check the “Automatically install Windows Server Roles and features required for Exchange Server”.


Figure 3: This option will automatically install the correct Windows Roles and Features

This will make sure that the necessary parts of Internet Information Server (IIS) are installed, and it will also make sure the Active Directory Lightweight Directory Services (AD LDS) is installed. Since the Edge Transport Server is not a domain member it has no access to Exchange information stored in Active Directory. Edge Transport Server configuration information is therefore stored in the AD LDS database. After continuing the wizard, the Edge Transport Server will be installed. When finished, you have to reboot the server.

The server is now ready and when you logon after the reboot you can open the Exchange Management Console. You might not be surprised, as it is completely empty, nothing is configured. No Accepted Domains, no Send Connectors, nothing. The only thing is that the anti-spam functionality is enabled by default, while this functionality is disabled by default on a Hub Transport Server.


Figure 4: Although the Edge Server is installed, it doesn't have any configuration information

Edge Synchronization

So, the Edge Transport Servers needs some configuration information from the internal Exchange organization. This process is called “Edge Synchronization” or “Edge Sync”. Edge Synchronization is setup by creating a subscription file on the Edge Transport Server and importing this subscription file on the Hub Transport Server.

To create this subscription file, logon to the Edge Transport Server, open the Exchange Management Shell and enter the following command:

New-Edgesubscription –FileName C:\Temp\Edge.XML

Note:
This file is only valid for 24 hours (1440 minutes). When this time expires before you import the file on the Hub Transport Server, the file is no longer valid and you have to create a new file.

Copy the file Edge.XML to the Hub Transport Server and save it on the local hard disk, for example again in C:\Temp\Edge.XML. It is possible to import the file in the Exchange Management Shell, but doing it using the Exchange Management Console is much easier. Open the Exchange Management Console, navigate to the Organization Configuration and select the Hub Transport Option. In the Results Pane select the “Edge Subscriptions” tab.

In the Actions Pane select “New Edge Subscription” and in the New Edge Subscription Wizard select the Active Directory site the Edge Server will connect to and the subscription file. This is the file we just copied to the local hard disk.


Figure 5: The Edge Subscription will connect the Edge Transport Server to the Hub Transport Server in this Active Directory site

Click New to continue. After creation a warning will be displayed: The Hub Transport Server needs to be able to successfully resolve the IP address of the Edge Transport Server. Furthermore the Edge Transport Server will communicate with the Edge Transport Server on TCP port 50636. You have to make sure that this port is open on the firewall, but this port is only needed inside out. It is a push mechanism from the Hub Transport Server to the Edge Transport Server.

When you check the Edge Transport Server after creation of the Edge Subscription you’ll notice the configuration info is also available.


Figure 6: When the Edge Synchronization is active there's Exchange information available on the Edge Transport Server

If you have setup your public MX records to use the Edge Transport Server it’s time to test it. If all goes well you are now able to send and receive messages through the Edge Transport Server.

Anti-Spam Solutions

The Edge Transport Server is implemented for message hygiene purposes. This means that its primary purpose is offering anti-spam and anti-virus functionality. Anti-spam is by default enabled on the Edge Transport Server, anti-virus is implemented by means of installing the Forefront Security for Exchange product.

Anti-spam is separated into five different layers, which are processed in the following order:

  1. Connection Filtering;
  2. Sender Filtering;
  3. Recipient Filtering;
  4. Sender ID Filtering;
  5. Content Filtering.

Where both Connection Filtering and Content Filtering separated into layers as well:


Figure 7: Graphical representation of the anti-spam layering in the Edge Transport Server

A message arrives (from the Internet) and will be delivered at the Edge Transport Server. When the messages successfully pass all filters it is delivered to the internal Hub Transport Server.

The connection filtering is the 1st layer of defense, and the most important one. When an SMTP host from the Internet tries to deliver a message at the Edge Transport Server the connection is validated first. If the IP address of the sending SMTP host on the IP Allow List the connection is accepted and the sending host can start delivering the message. If the IP address of the sending host is either on the IP Block List or on the RBL (Real Time Blocklist) the connection is refused. This is a very important step, and most likely the most important step since most spam generating servers are on the RBL. And you don’t want these messages to be delivered at all since they consume quite a lot of resources of your Exchange servers. If you have your Edge Transport Server up-and-running for some time and start monitoring, you’ll see that 97% of all inbound SMTP connections are refused by the connection filtering!

The second step is Sender Filtering. An Exchange administrator can determine if there are any external senders that are allowed or denied sending messages to your Exchange environment. Related to this is Recipient Filtering. If Recipient Filtering is enabled senders can only send messages to existing recipients in your Exchange organization. So, messages to iliketoknowyou@inframan.nl will not be delivered.

Sender ID filtering is also known as the “Sender ID Framework” or the “SPF Framework”. This is a bit like a DNS reverse lookup. By performing a Sender ID check a special DNS query is performed and an SPF record is checked. This SPF record (which is a TXT record in DNS) is configured with the names and IP address of the sending SMTP server. So, when a message is processed, the Edge Transport Server will check this SPF record and will compare the registered IP address with the source IP address of the message. If it doesn’t match it is most likely that another, untrusted SMTP Server is sending the message on behalf of somebody else. And thus most likely spam.

The last step is content filtering. With content filtering the message is inspected for certain content, or keywords in the message, both in the subject line as well as the body of the message. You can configure specific keywords, such as “viagra”, “online casino”, “I like to know you” etc. The content filtering will use this list of keywords when filtering. It takes about a week of monitoring your Exchange environment for generating an exhausting list of keywords you have to use in the content filtering. The content filtering also generates a rating to messages, known as the SCL (Spam Confidence Level). This is rating between 1 and 9 where a level 1 means “most likely to be legitimate e-mail” while level 9 means “most likely to be spam e-mail”.

Part of the content filtering is a quarantaine mailbox. Depending on the SCL rating you can have suspicious e-mail delivered in a quarantaine mailbox. While monitoring the quarantaine mailbox (manual process!) you can decide to deliver the actual message (false positive) or finally delete the message (when it turns out to be spam).

Attachment filtering is not really a part of the anti-spam solution in the Edge Transport Server, although its functionality can be extremely important. Attachment filtering is enabled by manually activating the attachment filter agent. Open an Exchange Management Shell and enter the following command:

Enable-TransportAgent –Identity “Attachment Filter Agent”

In my next article I will explain how the configure the various anti-spam settings.

If you would like to be notified of when Jaap Wesselius releases the next part in this article series please sign up to our MSExchange.org Real Time Article Update newsletter.

Featured Links