Exchange 2013 Mail Flow (Part 1)

by [Published on 29 Nov. 2012 / Last Updated on 29 Nov. 2012]

In this article the author will explore Mail Flow in the new Exchange 2013 and how it differs from Exchange 2007 and 2010.

If you would like to read the other parts in this article series please go to:

Introduction

As you probably already know, Microsoft reduced the number of server roles in Exchange 2013 to just two in order to “increase simplicity of scale, hardware utilization and failure isolation”:

  • Mailbox Server role: which includes the Client Access protocols, Hub Transport service, Mailbox databases and Unified Messaging, and also handles all activity for a given mailbox;
  • Client Access Server role: which provides authentication, redirection and proxy services (it doesn't do any data rendering). The Client Access server is a thin and stateless server and there is never anything queued or stored on it. The Client Access server offers the usual client access protocols: HTTP (Outlook Anywhere, ActiveSync and Web Services), POP, IMAP and SMTP, but not MAPI (all MAPI connections are encapsulated using RPC-over-HTTPS)!

Note that the Edge Server is not present in Exchange 2013. This is because its services are now provided by the Client Access Server (although you can still use an Exchange 2007/2010 Edge server as a gateway with your Exchange 2013 environment).

In this new architecture, the Client Access server and the Mailbox server roles are not as dependent on one another as in previous version of Exchange because all processing and activity for mailboxes occurs on the Mailbox server where the active database copy of a mailbox resides. All data rendering and data transformation is performed local to the active database copy, eliminating concerns of version compatibility between the Client Access server and the Mailbox server.

This architecture change brings many other benefits, but in this article we are going to focus solely on how it affects the flow of e-mail messages.

Transport Pipeline

Mail flow takes place through the Transport Pipeline which is a collection of services, connections, components and queues that work together to route messages. With Exchange 2013, the transport pipeline is now made up of 3 different services:

  • Front End Transport service, which runs on all Client Access servers, acts as a stateless proxy for all inbound and outbound external SMTP traffic. This service does not inspect message content but it can perform basic e-mail filtering based on connections, domains, senders and recipients. It only communicates with the Transport service on a Mailbox server and does not queue any messages locally;
  • Transport service runs on all Mailbox servers and is almost identical to the Hub Transport server role in previous versions of Exchange. It handles all SMTP mail flow for the organization, performs message categorization and content inspection. Unlike previous versions of Exchange, the Transport service never communicates directly with a mailbox database, which is now handled by the Mailbox Transport service. The Transport service routes messages between the Mailbox Transport service, the Transport service and the Front End Transport service. This service does queue messages locally.
  • Mailbox Transport service also runs on all Mailbox servers and is made of two separate services:
    • The Mailbox Transport Delivery service receives SMTP messages from the Transport service and connects to the mailbox database using an Exchange Remote Procedure Call [RPC] to deliver the message.
    • The Mailbox Transport Submission service connects to the mailbox database using RPC to retrieve messages and submits them over SMTP to the Transport service. The Mailbox Transport service also does not queue any messages locally.

As Exchange 2013 no longer has an Edge Server role, e-mail messages from the Internet are received and sent to Exchange using a third-party e-mail gateway, an Exchange 2007/2010 Edge server or through the Exchange 2013 Client Access server as Microsoft intends it to be. In this last scenario, these e-mails enter the Exchange organization through a Receive connector in the Front End Transport service and are then routed to the Transport service on a Mailbox server.

While Exchange 2007/2010 Hub Transport servers were not configured out of the box to accept e-mails from the internet, the new Client Access server comes with a Receive Connector named “Default Frontend <server_name>” that is already configured to allow “Anonymous Users” to connect to it:

 


Figure 1.1:
Default Frontend Receive Connector

As to messages within the organization, they enter the Transport service on a Mailbox server through one of four ways:

  1. Through a Receive connector;
  2. Through the Pickup and Replay directories;
  3. Through the Mailbox Transport service;
  4. Through agent submission.

Before putting everything together in a nice diagram, we just need to have a deeper look into the Transport Service, which consists of the following components and processes:

  • SMTP Receive: when e-mails are received by the Transport service, content inspection is performed, transport rules are applied and anti-spam/anti-malware inspection is performed (if enabled). If the e-mail is not rejected after passing through SMTP Receive, it is put in the Submission queue;
  • Submission is the process of putting e-mails into the actual Submission queue and can happen in three different ways:
    • Through an SMTP Receive connector;
    • Through the Pickup and Replay directories which now reside on Mailbox servers;
    • Through an agent.
  • Categorizer picks up a message at a time from the Submission queue and completes the following steps:
    • Recipient Resolution: the recipient's e-mail address is resolved to determine whether the recipient has a mailbox in the Exchange organization or an external e-mail address;
    • Routing Resolution: after information regarding the recipient is resolved, the ultimate destination for the message, the route to that destination and the next hop are determined;
    • Content Conversion: occurs so that the message is sent in a format that is readable by the recipient by transforming e-mail messages from one format to another format such as MAPI to MIME, UUENCODE to base64 encoded, or for appropriate rendering that is specific to an e-mail client like HTML, rich text format or plain text.

Additionally, mail flow rules are applied. After messages have been categorized, they are put into a delivery queue that is based on the destination of the message (mailbox database, Database Availability Group [DAG], Active Directory [AD] site, AD forest or external domain).

  • SMTP Send: depending on the location of the message recipient, the message is routed to:
    • the Mailbox Transport service on the same Mailbox server;
    • the Mailbox Transport service on a different Mailbox server that is part of the same DAG;
    • the Transport service on a Mailbox server in a different DAG, AD site, or AD forest; or
    • the Front End Transport service on a Client Access server for delivery to the Internet.

As a picture is worth a thousand words, the following diagram (taken from TechNet) puts everything we talked so far together and shows the relationships between the components in the Exchange 2013 transport pipeline:

 


Figure 1.2: Exchange 2013 Transport Pipeline

In the next article, we will analyze the flow of an e-mail message and trace its path using the diagram above!

Transport Agents

Twice now we mentioned Transport Agents without going into detail on what these are. Transport Agents, as with previous versions of Exchange, allows administrators to install custom software on an Exchange server in order to process e-mail messages that pass through the transport pipeline. This software might be created by Microsoft, by third-party vendors or by your organization.

Once more Exchange provides extensibility through the Exchange Transport Agents SDK. This SDK is .NET based and allows third-parties to implement the classes SmtpReceiveAgent, RoutingAgent and DeliveryAgent. When compiled against libraries in the SDK, the resulting assemblies are registered with Exchange 2013, which loads the agents and invokes their event handlers during specific stages of SMTP sessions or message processing.

The Transport service fully supports all the predefined classes in the SDK, which means that any transport agents written for Hub Transport server role in Exchange 2010 should work in the Transport service in Exchange 2013. On the other hand, the Front End Transport service only supports the SmtpReceiveAgent class while the Mailbox Transport service does not support the SDK at all, so you can't use any agents in the Mailbox Transport service.

Exchange 2013 comes with built-in transport agents, with most of them invisible and unmanageable by the transport agent cmdlets. However, a few of the built-in transport agents in the Transport service are visible using the Get-TransportAgent cmdlet:

 


Figure 1.3:
Exchange 2013 Transport Agents

Conclusion

So far we looked at how the new design of Exchange 2013 changed the Mail Flow in general.

In the second article of this series, we will see how message routing happens in the new version of Exchange and in the third and final article, we will finish by exploring how each service routes messages and we will track an e-mail to see its exact flow across all the different services.

If you would like to read the other parts in this article series please go to:

Featured Links