How do you measure your service delivery for Exchange? One possible method is the use of an IT scorecard and if your organization is using MOM 2005 then you can make use of the MOM 2005 SLA Scorecard for Exchange.
Back in November 2005, Microsoft released its answer to the need for measuring the service delivery of Exchange, namely the MOM 2005 SLA Scorecard for Exchange. Essentially, this scorecard gives you a dashboard from which you can see the availability and key performance statistics of your Exchange organization. Obviously you need to be running MOM 2005 in order to use this scorecard and if you haven’t evaluated MOM 2005 yet, maybe this article will help you realize that it offers plenty to help with measuring service delivery of Exchange. I don’t want to focus too much on the configuration of MOM 2005 within this article, although it is worth quickly noting that you’ll need to implement the Exchange MOM management pack, SQL Reporting Services and the MOM 2005 data warehouse for the scorecard to operate.
You can find the MOM 2005 SLA Scorecard for Exchange at the Microsoft download center.
First I want to cover the installation of the scorecard but if you just want to see what the scorecard looks like, there are plenty of screen shots later on in this article so feel free to scroll down now.
The first thing that you need to do is to import the required and optional management packs listed below into MOM via the Administrator Console and since this is an article focused on Exchange rather than MOM, I won’t go into detail on how to do this; you’ll need to reference your MOM documentation if you are unsure. The relevant management packs are:
- Exchange management pack. Obviously this is a required management pack and has to be the Exchange 2000 or Exchange 2003 version, depending on which version of Exchange you are using.
- SLA Scorecard for Exchange management pack. Another required management pack that actually collects the correct data to ensure that your system metrics can be determined.
- Antigen management pack. If you want to report on your Antigen antivirus statistics, you’ll need this management pack although it is optional and therefore not strictly required for correct operation.
- Exchange Intelligent Message Filter (IMF) management pack. If you use the IMF and want to report on specific performance counters, you’ll need this management pack although, like the Sybari management pack, it is not strictly required for correct operation.
Also, if you are running Exchange 2003 SP1 or greater, you can get metrics on blocked senders, blocked connections and blocked recipients. Finally, if you are running Outlook 2003 you will be able to obtain client-side monitoring statistics which are essentially the percentage of successful RPC operations between Outlook 2003 and the Exchange 2003 Information Store.
Once you’ve imported the relevant management packs, the actual SLA Scorecard for Exchange application can be installed. This can either be installed onto the SQL Server currently functioning as the MOM Reporting Server, or it can be installed onto an entirely separate SQL Server if required. The installation package comes in the form of the SLAScorecard.msi file and there are a total of 6 steps to installation. These steps and the requirements within them are detailed below in Table 1.
Table 1: SLA Scorecard for Exchange Installation Steps Overview
License and Support Information
Review and accept the End User License Agreement
- Confirm that you have deployed the Exchange 2000/2003 and SLA Scorecard management packs
- Specify MOM reporting server
- Test system prerequisites
Disk Space and Installation Location
Specify the installation location. The default is C:\Program Files\Microsoft\Solutions\SLA
- Specify the SQL instance to create the databases in
- Specify the name of the Presentation database. This is hard-coded to be SLAReporting
- Specify the name of the Staging database. The default is SLAStaging
- Specify where, on the SQL server, to store the database and transaction log files
Database Installation, Reports, and Web Deployment
Install the application files
Configure the Exchange servers within the database
The final section of the installation wizard, the Exchange Configuration section, actually starts to involve configuration of the scorecard. The Exchange Configuration screen of the installation process is shown below in Figure 1.
Figure 1: Exchange Configuration
Clicking the Config Exchange button invokes the SLA Scorecard for Exchange dashboard although it will obviously be empty at present. Within the scorecard, the next step is to click the Server Maintenance link found within the Administration section; this takes you to the Server Maintenance page where you should see that the Exchange servers within your organization are already present. An example of what this may look like is shown below in Figure 2, although it should be noted that this screen shot is taken from an existing system and therefore already shows servers that have been configured.
Figure 2: Server Maintenance Screen
All that is required now is to edit the Server Type and # of Exchange Databases fields for each server. Note, from Figure 2, the In Service Date field used to indicate when the particular Exchange server entered service within your organization. By default, this will be set to 1st September 2004 and should therefore be edited accordingly. For each server, you will need to click the Edit button to amend the In Service Date, Server Type and # of Exchange Databases options. For the server type, the following options are available:
- Public Folder
- Internet Gateway
From the drop-down list, you need to select the role of this particular server. What if this server is both a Mailbox and a Public Folder server, as is the likely case? Or perhaps it’s a Mailbox, Public Folder and Internet Gateway server? Since you can only select a single role from the drop-down list, you must add the server again with the other roles that it performs. I’ll cover adding a new server later on; for now, pick the main role for the server. Once you’ve done this, move on to the # of Exchange Databases field and enter a number between 1 and 20 that corresponds to the number of databases on this server. Once you’ve completed this initial configuration, close the scorecard window and you’ll be returned to the Exchange Configuration screen shown above in Figure 1. You can then test that your configuration is correct by clicking the Test Configuration button. If all is well, you’ll be presented with a dialog box informing you of this fact. That completes the installation.
To access the scorecard after installation, the URL is http://servername/slasa.
Views – Metrics & Measures
Once you enter the scorecard, you’ll be presented with the Metrics Scorecard by Week view as shown below in Figure 3.
Figure 3: Metrics Scorecard by Week
This is a nice simple view of the availability figures for the various messaging services for the past and current weeks. You can see from Figure 3 that the metric names correspond to the various server roles that you configured previously when setting up the scorecard. The most obvious metric that stands out is the Public Folder Availability metric for the week of 12th February, since it is highlighted in red. This is because the figure of 98.123% is lower than the defined SLA target of 99.000%. Also, you can see that the Exchange Mailbox Server Availability metric for the week of 19th February is 100%, whereas it was 99.081% for the previous week. As a result, the Trend column shows a positive trend via the green upwards arrow. Where does this data actually come from? Well, in essence the calculation is performed by examining the System and Application Event logs for various events such as the mailbox store mount and dismount events like the one shown below in Figure 4. Additional data is obtained from event log IDs 9523, 6005, 6006 and 1001. The date and time of such events is recorded and used to work out how long a particular role has been unavailable as a percentage of time.
Figure 4: Mailbox Store Dismount Event
Additionally, data from the Outage Maintenance page of the scorecard is used to determine overall availability figures and I’ll cover this page later on. It’s also possible to show monthly figures for your messaging services by choosing the Metrics Scorecard by Month report, a sample of which is shown below in Figure 5. Here you can see the availability metrics of the messaging system for the months of February and March 2006.
Figure 5: Metrics Scorecard by Month
There are also Measures views, both by week and by month, which show metrics on a number of common messaging components that may be affecting the ability to meet the SLAs. These reports give you a graph followed by tabular information. For example, Figure 6 below shows the graph section of the Measures Scorecard by Week report, where you can see key statistics such as the number of Exchange messages delivered as well as the number of Internet messages received. If you are using the IMF, you’ll also get statistics such as the number of messages that have been filtered by the IMF. The sender, recipient and connection blocking have not been configured on this particular system, hence the zero values. Figure 7 shows you the corresponding data in table format. Also, this particular report goes on to give you statistics such as the percentage of messages filtered by the IMF, the total number of mailboxes within your organization, as well as key Antigen antivirus statistics such as the number of attachments removed.
Figure 6: Measures Scorecard by Week Graph
Figure 7: Measures Scorecard by Week Table
Trend Reports – Metrics & Measures
Trend reports are also available that can be viewed in weekly, monthly or year-to-date (YTD) mode. An example is the portion of the Metrics Trend Report shown below in Figure 8, which shows the monthly mode view but only for three of the six availability statistics. This allows IT Managers to view the overall trends of the availability metrics within the organization. Only three months are shown in the table below, but you can begin to imagine the benefits of such a table that shows you the availability trends over the last few years. Don’t forget that it’s possible to export these reports into a variety of formats, such as Web archive, PDF, Excel, TIFF, CSV and XML.
Figure 8: Metrics Trend Report by Month
The Outage Maintenance area is the first of the three administration areas that you can use to configure your system. The Outage Maintenance area is extremely useful in that it allows you to approve planned outages such that your SLA figures take account of these outages. Take the example shown below in Figure 9.
Figure 9: Outage Maintenance Screen
Here you can see a list of affected servers, the date and time when the outages started and finished, the overall duration of the outage and the number of databases affected. Additionally, you’ll see the Planning, Approval and Category fields. Here, you can determine whether the outage was planned or unplanned and also whether the outage has been approved or not approved. The main thing to note regarding the approval is that any outage not approved will be marked negatively in your overall SLA metrics. Finally, there is the option to edit the category of the outage. Available category options are Not Set, Application, Hardware, OS, Site Move, Network, Site Power, Unexplained, Ancillary and Maintenance.
A nice feature across all three administration areas is the ability to export the information to Excel format.
You’ve already seen the Server Maintenance screen in Figure 2 above, seen when configuring the scorecard for the first time during the installation. One of the things that you may need to do is to configure your Exchange servers for multiple roles, such as a Mailbox and Public Folder server. For example, imagine you’ve configured SERVER1 as a Mailbox server. To configure SERVER1 as a Public Folder server as well, you’ll need to click the Add New Server Type button and choose SERVER1 from the drop-down list. You can then configure the Server Type option as Public Folder, set the appropriate number of databases and additionally configure the In Service Date field with the date this server was first brought online. When you’ve configured these options, click Update to complete the provisioning process.
You can also delete an existing server that may have been decommissioned from service by selecting the check-box next to the server name and clicking the Delete Server Type button. The Out of Service Date field should now be updated to reflect today’s date as the decommissioned date and this server will no longer form part of your SLA metrics.
As you can see, it's simple to keep the scorecard up-to-date with your changing Exchange organization.
Clicking the SLA Configuration link, found within the Administration section, brings you to the SLA Configuration area as shown below in Figure 10. Here, it’s possible to define the SLA percentages for the various server types with the default for all server types being 99.000%. When any SLA metric drops below the defined SLA target, that metric is highlighted in red as you’ve seen previously in Figure 3 above.
Figure 10: SLA Configuration
The MOM 2005 SLA Scorecard for Exchange allows different audiences to obtain the information they require about the Exchange system. For example, administrators can see exactly which servers are operating correctly and also which ones are not, whilst IT managers can use the information to perhaps put together a business case to justify improvements to the messaging system. If you haven’t already checked it out, it’s well worth doing so.