Posts

Town of Gilbert

The town of Gilbert Arizona has been a client of N2 Network Solutions for years.  In 2009 Gilbert needed an Exchange 2007 solution.  N2 discovered, designed, and implemented a turn key Exchange solution.  This solution included continuous cluster replication or CCR for redundancy in the mail environment.  This gives Gilbert the ability to perform routine maintenance on one server without disrupting the end users.  Once this is complete the cluster can be switched to the secondary server so maintenance can be performed on the primary.

Getting Started with Exchange 2010

As most know Exchange 2010 is the latest version of Microsoft’s email server.  I wanted to write a short description of the software and outline its features. 
Like its predecessor Exchange 2010 requires that you run it on an x64 platform.  32-bit processing is surely but slowly becoming a thing of the past.  In 2010 however you must also be running Windows 2008 SP2 or 2008 R2.  One of the major decisions you’ll have to make is whether to select the standard or enterprise edition.  This basically boils down to how many stores you need.  Standard supports 5 stores per server as to where Enterprise you can do 50+.   As far as the client side CAL’s are concerned you must purchase the 2008 enterprise CAL’s if you wish to do unified messaging.  There is not however a limitation in the software.  It is simply a licensing issue.  Which means you’ll still have the ability to access unified messaging but it will not be licensed correctly.   Another feature Microsoft has decided to keep is the JET EDB database.  It has been rumored in the past that Microsoft would start using SQL server to house the Exchange database.  This is not the case. 
If you ever worked with recovery storage groups in Exchange 2003 or 2007 you will no longer find those in 2010.  As well you will not be able to find routing groups.  All of the Exchange 2010’s routing is done through active directory sites and services.  So you must make sure that you have properly configured your sites before moving forward with Exchange.  It is essential to Exchange 2010 functioning properly.  As with Exchange 2007 Microsoft still is trying to deemphasize public folders.  Their goal is to eventually replace these with their Sharepoint product. 
Another major feature of Exchange 2007 and 2010 is their ability to reject email at the gateway.  The Edge transport server allows you to configure ADAM and active directory lightweight services to query AD.  This allows you to get a list of valid email address and push them out to the border of your network.  If the edge server detects that someone is trying to send email to the inside of your organization and the user does not exist it is dropped immediately.  This saves on memory and processing power internally so that you don’t have to deal with spam. 
Additionally with Exchange 2007 and 2010 you get the ability to create UNC direct file access paths.  This way in OWA when a user needs a file on a network share they can grab it without needing a cumbersome VPN client.  Outlook anywhere also remains widely the same in 2007 and 2010.  It basically encapsulates your RPC packets into https packets.  This allows you to traverse your firewall without opening any additional ports.  Therefore giving users access to their email from Outlook wherever they may travel. 
One of the greatest new features of Exchange 2010 in my opinion is database availability groups or DAG.  This is essentially the same thing as CCR in Exchange 2007.  Anyone who has tried to configure CCR, LCR, or SCR in Exchange 2007 knows that it can be quite the process.  Microsoft simplified this with DAG’s in 2010.  It allows you to keep 16 copies of a users mailbox for redundancy and disaster recovery.  It does this through a process called log shipping.  Where 1MB files are created and then played into the database.  This allows you to keep a backup of your server at another physical location for disaster recovery or have two Exchange servers running next to each other.
Another nice feature in 2010 is the fact that the client access server or CAS redirects your client to their database server that houses their mailbox.  You no longer need to specify the location of your server in Outlook.  The CAS parses AD and redirects them automatically.  Therefore there is no hard coding.  This makes the transition for failover a lot easier. 
As most of you know who have used Exchange 2007 the GUI is simply a front end to Microsofts command line utility called EMS or Exchange Mangement Shell.  Anything you do in the GUI is converted to a command and executed against your server.  I would personally say you have about 90% functionality in the GUI as opposed to EMS.  However, EMS definitely makes the process a lot easier if you need to apply a setting to multiple objects at the same time. 
As with Exchange 2007 you still have the same five roles edge transport, hub transport, client access server, mailbox, and unified messaging.  Inside of these five roles only the edge transport server must be installed separately from the rest of the servers.  Everything else can be ran on one box.  Although this is not recommend for performance reasons.  The reason why the edge server is standalone is it was meant to sit in your DMZ or on the border of your network.  Absorbing the hits so your internal servers are not affected.  It has features such as safelist aggregation where Outlook client rules are brough outside to it so that it can apply those rules before the message ever enters your internal network. 
The hub server still is the same as Exchange 2007 it routes your messages internally and holds compliancy rules.  You can also run a command against it to install antispam featureset.  This way if you don’t have an edge transport server you can use it to receive outside mail directly.  Although this is not recommended by Microsoft.
The CAS server or client access server is meant to interface with your internal and external clients.  As stated before it automatically redirects your Outlook clients so that you don’t need to hardcode their mailbox server.  It also accepts connections from smart phones, OWA, etc.  It is basically your clients interface to your Exchange infrastructure.
If you wish to monitor your Exchange 2010 infrastructure Microsoft has made a plugin for their SCOM or system center operations manager.  This is Microsoft’s MOM replacement that allows you to montior your servers.
In Exchange 2010 you will no longer see SCR, LCR, or CCR.  They have been superceded by DAG or database availability groups.  This makes configuring database replication a lot smoother.  DAG’s also allow for your data to reside across multiple servers.  You can also have multiple DAG’s.   This is a great feature because if half of your users are in one DAG group and it goes down the other half are not even affected.  Other benefits are reduced restore time since you’re not restoring all of your users’ data only the ones in that DAG.  You can also have separate exchange policies for different DAG’s.  So if your management is in one and your regular users are in another you can change the rules that apply to them.  This is a great way to mitigate risk by distributing your load. 
As far as the enterprise and standard software go they are both installed from the same media.  It is just different license keys that you input that determine what version you are installing.  It is also upgradable.  You can go from trial to standard to enterprise.  However, you cannot downgrade backwards from enterprise to standard or standard to trial. 
In order to install Exchange 2010 your domain and forest functional level must be at 2003.  Also each site which contains Exchange 2010 must also contain a 2003SP2 domain controller or 2008 domain controller.  We recommend you have your domain running 2008R2 domain controllers however. 
Exchange still uses EAS or exchange active sync for mobile devices.  This way your contacts, calendar, email, etc. are all tightly integrated with your Windows mobile devices.
One common misconception that people have is Exchange enterprise must be installed on server enterprise software.  Or that server enterprise software cannot have Exchange standard installed on it.  Both of these are fallacies. 
When you begin your Exchange installation you should give serious consideration to how you configure your arrays.  Exchange is a very read/write intensive application.  Therefore you should separate your OS, log files, and database all on separate arrays.  If this is not possible it is then recommended that you at least put yoru OS and log files on one array and your database files on another.  The reason for this is simple.  The log files are write intensive and the database files are read intensive.  Separate these two out can speed up your disk I/O.
Memory requirements in Exchange 2010 have pretty much gone unchanged.  Start your server with 2GB of memory and then 5MB for every mailbox user.  I would also personally recommend to have a minimum of 4GB.  Memory is cheap enough these days that the benefit of having more of it outway the cost.
Although the databases in Exchange can grow very large we do not recommend that you go over 100GB.  This can become cumbersome to work with and decrease performance on your server.
If you wish to remotely manage your Exchange server you can install the management tools.  They will install on Vista SP2 and higher or server 2008 SP2 or higher.  This way you do not have to remotely login to your Exchange server to make all of your changes.
As far as your site layout goes you should also plan on having a global catalog server in every location that contains a mailbox server.  This is recommended by Microsoft and will reduce WAN traffic. 
Exchange has also setup a new permissions setup which they refer to as RBAC or role based access control.  From this you get 5 roles to manage your exchange infrastructure.  They are Organization management, view only organization management, recipient management, records management, and GAL synchronization management.
Another thing you should consider before installing Exchange 2010 is to make sure your domain is setup properly.  You can use tools such as NETDIAG and DCDIAG to verify this.  In order to install Exchange 2010 you’re going to need to be a member of domain admins, enterprise admins, and schema admins.  You will also want to add connect.microsoft.com and download.microsoft.com to your trusted sites list in IE.  Other pieces of software that must be installed are .NET 3.5, Windows remote management 2.0, powershell v2, 2007 office converter microsoft filter packs.  If you are installing the mailbox role you must also have AD services remote management tools.
Before starting the install you must prepare your schema by running setup /ps if it fails delete the contents of c:windowstemp, copy the files from your CD to yoru hard drive and rerun setup /ps.  You must then run setup /prepareAD /OrganizationName:MyCompany where “MyCompany” can be replaced by your organization name.
You must then prepare the prerequisites by running the following commands.

  • ServerManagerCMD -install RSAT-ADDS
  • ServerManagerCMD -install Web-Server
  • ServerManagerCMD -install Web-ISAPI-Ext
  • ServerManagerCMD -install Web-Metabase
  • ServerManagerCMD -install Web-Lgcy-Mgmt-Console
  • ServerManagerCMD -install Web-Basic-Auth
  • ServerManagerCMD -install Web-Digest-Auth
  • ServerManagerCMD -install Web-Windows-Auth
  • ServerManagerCMD -install Web-Dyn-Compression
  • ServerManagerCMD -install Net-http-Activation
  • ServerManagerCMD -install RPC-over-http-Proxy
  • Once this is complete reboot your server.  You are now ready to run Setup.com /mode:install /roles:H,C,M the H,C,M install hub cas and mailbox roles.
    Once your install is complete run the Exchange BPA or best practice analyzer.
    In order to install the Edge server you’ll want to make sure you’re running 2008 standard with SP2.  You’ll need .NET 3.5, remote management 2.0, powershell v2, AD LDS (can be installed via servermanagerCMD -i ADLDS).  For the edge server to work in a DMZ you’ll need to open ports 50389-50636.  Then run new-EdgeSubscription -filename “c:tempEdgeSubscriptionInfo.xml”  Copy that generated file to your hub server you can import it in the GUI and run start-edgeSubscription from EMS.  You can test this once it is imported to verify it is working properly by using test-EdgeSubscription from EMS.
    I would personally recommend using a RBL provider to stop spam from entering your organization.  One example of this is SpamHaus.  This queries the connecting server to a black list of IP’s and blocks communcation if it is found on the list.  This one feature can drastically cut down on spam.
    Another item you have to address is purchasing a SAN certificate for your Exchange server.  Exchange has moved to a secure by default mentality.  You will find connecting to OWA or using activesync become very painful if you try to issue your own SSL certificates.
    Another security improvement in Exchange 2007 and 2010 is that all intracommunication is secure and encrypted.  TLS is used for all server to server communication internally.  RPC is used for your Outlook clients to communicate with your servers.  SSL is configured for all external client communication including, OWA, activesync, etc.
    Opportunistic TLS is a new feature where your Exchange server will no long try to send via SMTP by default.  It will first send a STARTTLS command to use TLS to encrypt external SMTP communication with other servers.  If the other server however does not support this it will revert to unsecure communications.
    Still included in Exchange 2010 is the ability to use a journaling mailbox to track all of your emails.  This is required by some organizations.  Keep in mind that this feature can increase your processor and memory usage by 25%.  So you should make sure your server has plenty of resources before turning on this feature. 
    One of the requirements as previously stated is that Exchange 2010 must be running active directory 2003.  Even though 2008 is recommended if you are running Cisco Unified Messaging 4.2(1) or lower it is NOT compatible with active directory 2008. 
    When you upgrade your active directory infrastructure it is recommended that you create a virtual machine using Microsoft Hyper-v or Vmware.  Make the virtual machine an additional domain controller and make it a global catalog.  This way if your upgrade takes  turn for the worst you have data that is intact if you have to downgrade.  Do not forget to unplug it from the network before doing the upgrade.  If you need to revert back you can use NTDSUTIL to seize the roles.
    If for whatever reason you need to create a scratch installation of a domain you can always use the ADMT utility to move users, groups, computers, service accounts, and trusts.
    To migrate from 2003 Exchange to 2010 the overview is as follows.  First you must be running Exchange 2003 with service pack 2.  Your active directory domain and forest functional levels must be 2003 and at least one global catalog has to be 2003 server with SP2.  Instal AD LDIFDE tools on 2008 to upgrade your schema.  Upgrade your Exchange Schema.  Transfer OWA, activesync, and Outlook anywhere to the CAS server.  Install/upgrade hub server.  Transfer the mail flow to the hub transport server.  Install mailbox servers and DAG if required.  Move your public folder replicas using pfmigrat.wsf or PFRecursive.PS1.  Move your mailboxes.  Rehome OAB.  Rehome public folder heirarchy.  Transfer public folder replicas.  Delete 2003 public and private stores.  Delete routing group connectors.  Delete RUS using ADSIEdit.  Uninstall Exchange 2003.
    To migrate from 2007 Exchange to 2010 the process is a little less.  Make sure your Exchange 2007 server is running SP2.  Make sure your domain and forest is at 2003 functional level.  Global catalog server is at 2003 SP2.  Use AD LDIFDE tools to upgrade your schema.  Prepare schema.  CAS server.  Transfer OWA.  Install hub transport.  Transfer mail to hub transport.  Use AddReplicatoPFRecursive.Ps1 to move your public folder replications.  Move your mailboxes.  Rehome OAB.  Transfer public folder replica.  Delete public and private stores.  Uninstall Exchange 2007.

    Send a daily html CCR report via Powershell

    Create a powershell script to generate the html file

    Test-ReplicationHealth | ConvertTo-Html -title “BackupReport CCR” -body “Backup status per Storage Group” | Set-Content E:StorageGroupBackupStatus.html
    note:

    test-systemhealth

     test-servicehealth
     

    Create a powershell script to e-mail log file

    $filename = “E:StorageGroupBackupStatus.html”
    $smtpServer = “RelayServer”
    $msg = new-object Net.Mail.MailMessage
    $att = new-object Net.Mail.Attachment($filename)
    $smtp = new-object Net.Mail.SmtpClient($smtpServer)
    $msg.From = “from@emailaddress.com
    $msg.To.Add(“to@emailaddress.com”)
    $msg.Subject = “CCR Status”
    $msg.Body = “Attached is the CCR status”
    $msg.Attachments.Add($att)
    $smtp.Send($msg)

    Go into task scheduler

    For the program type this
    C:WindowsSystem32WindowsPowerShellv1.0powershell.exe
    For the arguments use
    -PSConsoleFile “C:Program FilesMicrosoftExchange Serverbinexshell.psc1” -noexit -command “. ‘e:SendLogFile.ps1′”

    Upgrading Exchange 2007 CCR cluster to SP2

    Upgrade Order

    1. CAS
    2. UM
    3. HUB
    4. EDGE
    5. MAILBOX

    Upgrade passive node first
    Browse to the SP2 files setup.com /Mode:Upgrade
    restart and wait for server to come back online
    Login to the passive node and open Exchange Console
    Stop-ClusteredMailboxServer Mailbox –StopReason “Upgrade to SP2”
    Confirm the state of the cluster is Offline
    Now we want to move the cluster
    Move-ClusteredMailboxServer Mailbox –TargetMachine Cluster2 –MoveComment “Upgrade to SP2”
    Upgrade CMS
    Setup.com /UpgradeCMS
    Open Failover Cluster Management in the administrative tools and verify the status is online at this point
    Upgrade the primary node which is now the passive one
    Setup.com /Mode:Upgrade
    Restart the server you are now at SP2 for Exchange 2007
    Move the cluster back to your primary server
    Move-ClusteredMailboxServer Mailbox –TargetMachine Cluster1 –MoveComment “Upgrade to SP2”
     
     
     
     

    Exchange 2007 CCR useful commands

    Move Clustered mailbox server to another server
    Move-ClusteredMailboxServer -Identity:<CMSName> -TargetMachine:<PassiveNodeName> -MoveComment:<Comment>
    View Storage group configuration
    Get-StorageGroup -Server mbx01 | fl
    View the database configuration
    Get-PublicFolderDatabase -Identity “mbx01SG2Public Folder Store” | Fl
    View the status of all storage groups
    Get-StorageGroupCopyStatus -Server:<Server>
    Or to view the status of a particular storage group
    Get-StorageGroupCopyStatus -Identity:<ServerStorageGroupName>
    How to view the status of a clustered mailbox server
    Get-ClusteredMailboxServerStatus -Identity:<Name>
    How to verify the copy of the database
    Suspend replication on your passive node
    Suspend-StorageGroupCopy -Identity <ServerStorageGroup> -SuspendComment <Comment>
    From you %windows%system32 directory create a shadow  copy
    Vssadmin create shadow /for=<Volume>
    Run this command wherever your ESEUTIL is located to verify the database(default location:%ProgramFiles%MicrosoftExchange Serverbin)
    Eseutil /k /p20 <Path for VSS Shadow Copy of database>
    If ESEUTIL reports non corrupt datbase delete the shadow
    Vssadmin delete shadows /For=<Volume>
    Moving database location in CCR environment
    Suspend replication
    Suspend-StorageGroupCopy -Identity <ServerStorageGroupName>
    Dismount database
    Dismount-Database -Identity <ServerStorageGroupNameDatabaseName>
    Move the database in the configuration only
    Move-DatabasePath -Identity <ServerStorageGroupNameDatabaseName> -EdbFilePath <NewPath> -ConfigurationOnly
    Move the files to the new location on this disks
    Mount-database
    Resume replication
    Resume-StorageGroupCopy -Identity <ServerStorageGroupName>

    SCR(Standby Continous Replication) in conjunction with CCR(Cluster Continuous Replication) is your ultimate DR(Disaster Recovery) choice

    With the latest version of Exchange 2007 you get two new features that were not available before, SCR and CCR.  Today I’m going to discus why using SCR in conjuction with CCR is your ultimate solution to disaster recovery.  For more information on why CCR is the best HA(High Availability) option in Exchange 2007 see this link.
    First thing I want to talk about is when an organization would need SCR.  SCR is basically a warm standby server that maintains a copy of your Exchange database.  This is most commonly an offsite solution so that if something were to happen to your building where your primary server is located you can restore with losing email, it basically gives you site resilience.
    SCR has the ability to provide a diaster recovery option for the following types of Exchange implementation:

    • Standalone install
    • CCR(Continuous Cluster Replication)
    • SCC(Single Copy Cluster)

    How SCR differs from CCR and LCR

    • You can replicate your data to multiple SCR targets for further redundancy
    • You can specify the delay period on your replication.  This way if your CCR or LCR data becomes corrupt you have time to break the synchronizationccr-and-scr

    Remember with SCR as well as LCR(Local Continuous Replication) and CCR(Continuous Cluster Replication) you will also need the enterprise version of Exchange 2007.

    CCR(Continuous Cluster Replication) in Exchange 2007 and why it is your optimal disaster recovery solution

    With the design of Microsoft Exchange server 2007 Microsoft added some new HA or high availability options that were not available in the previous versions of Exchange.  I want to take the time to outline CCR or Continuous Cluster Replication and why it is one of the most anticipated features of Exchange 2007 and the best solution if you are looking for HA in your messaging infrastructure.
    To begin there are basically two types of Exchange HA that you can implement.  DR, disaster recovery, or HA, high availability.  When it comes to these you have two very popular options.  The traditional option is SCC(Single Copy Cluster).  The latest feature of Exchange 2007 replaces this with CCR(Continuous Cluster Replication).  I will outline why CCR is a far superior choice over SCC.

    1. There exists no single point of failure.  When you implement SCC it requires a shared SAN which becomes your single point of failure. 
    2. Shared storage or a SAN is not a prerequisite for CCR.  However, if you go with an SCC solution you’re going to need to spend tens of thousands of extra dollars on a SAN solution.
    3. Mailbox server can span two locations.  This is probably one of the greatest accomplishments of Exchange Server 2007.  It has the ability to have a mailbox store at two locations.  If something happens to site A site B can pick up where it left off.
    4. Better failover design.  With CCR you maintain two independent copies of the log file and database data.  This is a big improvement and decreases chance the same failure will affect both servers.
    5. Easier to install.  Installation becomes easier since there is no shared SAN and the hardware requirements do not have to be identical
    6. Reduce load on your servers during backup.  Since your passive node contains up to date data on your Exchange database you can create your backup jobs on your passive node.  This means you can run backups mid day when the entire organization is hammering your active Exchange node.  This is completely seamless to the end user as they will notice no degradation while accessing the server.

    How it works
     

         Exres.dll is gateway for clusters responsible for bringing resources online offline.

         Every 500 milliseconds UDP packet heartbeat

         Quorum resource is a shared disk that holds the configuration information all resources need access

    Ccr requires:

         2003 Server Enterprise

         Two network cards

         Three hard drives OS/Logs/Databases

         Hotfix 921181 – Support for file share witness – http://support.microsoft.com/kb/921181

         Recommend Hub Transport for the file share witness

    Install CCR

                    Install x64 OS normally

                    Join domain

                    Isolate private nics to private vlan

                    Hotfix 921181

    Install IIS

                    Install updated msdaps.dll

    File Witness Share

                    Create directory on hub transport

                    Give the account which is used to run the cluster service full control of sharing and ntfs perms

    Create cluster

                    Go to program files administrative tools cluster admin

    Create MNS Quorum

                    Cluster res “majority node set” /priv mnsfileshare=servershare

                    Cluster group “cluster group” move

                    Cluster res “majority node set” /priv

    Install Exchange 2007

                    Heartbeat ports on the switch same vlan can be accomplished through port tagging

                    Set-transportconfig enable configure transport dumpsters on hub transport server which is required

                    Maxdumpstersizeperstoragegroup specify the max size of the transport dumpsters queue for each storage                          group 1.25 times the size of maximum message that can be set.

                    Maxdumpstertime parameter is how long email message is in the transport dumpster so value of 7:00:00 7                        days means you can extended outage of 7 days and no loss of messages

    Enable transport dumpster

                    Set-transpotconfig –maxdumpstersizeperstorage group <size> -MaxDumpsterTime <timespan>

    When installing Exchange 2007 make sure you have at least 3 hard drive arrays for the OS, logs, and database files.