Crisis Contingency

For IT professionals, the possibility of data loss looms large on the horizon like a nasty storm about to hit. To help ease their minds, they spend a significant amount of time preparing for disaster by checking and re-checking their backup and recovery plans. A sound disaster recovery plan can turn a serious job-threatening disaster into a minor setback (and make your team look golden in the process).

When formulating a disaster recovery plan for Exchange, it is important to plan a strategy, document the process, implement and maintain the plan and be familiar with the recovery options. Here are a few select hints, tips and best practices to get started in creating a good backup plan for your Exchange Server 2003 running on Windows Server 2003.

When coming up with any backup plan, first thing’s first — the plan itself. It is very important to determine what the company’s e-mail retention policies are, if they aren’t already established. In other words, how far back will e-mail need to be recoverable in the event of a failure? The last thing an administrator wants to tell the CEO is, “Sorry, that message you inadvertently deleted is a day too old to be recovered from backups.” Small companies that aren’t dependent on e-mail for operational records probably can get by with a two-week rotation of daily backup tapes. Others that use e-mail to document critical business processes or are subject to increasingly stringent legal record-keeping requirements should have no gaps in the backup cycle and should retire at least one set of backup media monthly for archiving indefinitely, preferably off-site.

Additionally, administrators should plan for all the foreseeable crisis scenarios. For example, the plans should account for a hardware failure of the Exchange server that could require an extended period to repair. What if the Exchange database becomes corrupted during the workday? Do plans allow users to continue to work
offline even though their e-mail store may not be available during the recovery period? (The answer should be a resounding “yes.”)

Once all the Exchange backup plans are in place, it is imperative to document them and to keep a copy of all that hard work stored at an off-site location. Set up a binder to store information such as screenshots of the Exchange System Manager, hardware information about the server, disk drives and partition information plus any other useful data pertaining to the recovery plan.

Back It Up

In any Exchange installation, there are three main components that require protection — the application code, the database and logs, and individual mailbox stores (sometimes referred to as “brick” backups because they allow the restoration of building blocks down to a single e-mail message as well as multiple mailboxes). Application protection includes protecting the server operating system through System State backups, protecting your Active Directory through regular backups, the Internet Information Services metabase on the Exchange server, the Exchange Key Management Service and any Site Replication Services you may have running. Your database and logs generally go hand-in-hand with any backup software. Since Exchange is a transaction-oriented database, its log files are extremely important to a successful recovery (unless your restored mailboxes are full of duplicates and omissions). Mailbox-level backups can be the most useful, however, they should not be the sole method of recovery.

Another important decision is how to back up the data. Windows has a handy, built-in backup program called NT Backup that’s adequate, but it doesn’t provide mailbox-level backup. There are several third-party backup software products that provide more features, including Symantec BackupExec (formerly Veritas) and BrightStore ARCServe, to name a few. The backup schedule should specify the types of backups (normal/full, incremental or differential) that correlate with the company’s retention policies.

Tools of the Trade

There are few feelings worse than knowing the Exchange Server has gone down. But, for administrators who have the proper backups and contingency plans in place, crisis can be averted. There are some important tools that every Exchange administrator should know how to use to facilitate a full recovery. Two such command line tools are ESEUTIL and ISINTEG. ESEUTIL is an extremely powerful tool that can both check for and repair corrupted data in your Exchange database. ISINTEG helps to rebuild what you have fixed using ESEUTIL.

Of course, the best way to avoid disaster is proper maintenance of the Exchange server. Make sure that store limits are set for users and that those limits are enforced from the top down. You can check out what your store limits are by going into Exchange System Administrator and clicking on your Administrative Group → Servers → Your Exchange Server → Your First Storage Group. Right-click on the Private Store you want to check, click Properties and look at the Limits tab. Here is where to place restrictions for all users in that store.

The Windows System File Checker tool (sfc.exe) is another tool that administrators can use if there are problems with the server OS but the machine can still boot. This command line tool will scan the most important system files in the Windows Server 2003 installation to make sure they are all intact and the correct version of those files are present. It will then replace any corrupted files with the correct ones from a cached source in the System32 folder.

The key to a successful backup strategy is a solid plan that is well thought out and well documented. When the time comes to use it, administrators will thank their lucky stars that they possessed the foresight to set up these processes. They can make a nerve-racking situation as painless as possible.