Understanding Exchange Server Updates and the Process to Patching
For all you IT Pros continuing to support an on-premises Exchange Server infrastructure (in any configuration), Microsoft has a vital message for you: It is extremely important to keep Exchange up to date.
Due to the number of customers that were unprepared to install last month’s (Mar ’21) Emergency security patches, many had to scramble to install all the necessary updates first. They weren’t running a supported Cumulative Update (CU) release. Now, just imagine if you happen to have 50 CAS/Mailbox servers and 6 Edge Transport servers. I’ll explain the headache many of you literally went through to get your environment secure. (Some of you are still working and planning on this. You know who you are!). Let’s see if we can come to a straightforward plan to get on the other side of this now, and into the future.
Microsoft’s Exchange engineering and product support teams have an ideal state in mind when their customers call into tech support queues: “Yes, we are running a supported Cumulative Update (CU) release and have installed the latest Security Update (SU).” What does that mean? MS supports your configuration when you are running either the latest (N) CU release or the one prior (N-1) to the current. So, for Exchange Server 2019, as of this writing, MS supports you if you’re running CU8 or CU9. These were released in Dec 2020 and Mar 2021, respectively (CUs are released quarterly so you have some semblance of a schedule when planning). Security Updates or Critical Product Updates are released, when necessary, on ‘Patch Tuesday’ (the 2nd Tuesday of the month). These are published as standalone downloads and are available via Windows Update. You can find the most recent Security Updates via this link (just filter the Product Family dropdown with ‘Exchange Server’).
Let’s go over some questions I’m sure you IT Pros have on your mind.
Passwords Haven’t Disappeared Yet
123456. Qwerty. Iloveyou. No, these are not exercises for people who are brand new to typing. Shockingly, they are among the most common passwords that end users choose in 2021. Research has found that the average business user must manually type out, or copy/paste, the credentials to 154 websites per month. We repeatedly got one question that surprised us: “Why would I ever trust a third party with control of my network?
“Why does Microsoft release updates so often?” That’s a great question! Essentially, Microsoft releases updates when they’ve resolved issues with features in Exchange Server, many identified by you IT Pros supporting your environments. New features are also added to various CU releases. And of course, security updates are released to keep your environments secured, especially with your Internet-facing servers! Here’s an example of a typical release flow for two CUs and two SUs:
- On a certain month (e.g. April), Microsoft releases CU6; CU6 is cumulative, meaning you’ll get all updates included up to its release
- Next month, May, they release SU1, a security update for CU6
- Then in June, they release CU6 SU2, the second security update for CU6. CU6 SU2 includes all updates from SU1. So, if you were unable to install SU1, you only need to install SU2 on your servers to be fully patched and protected
- Then, to conclude this scenario, in July, CU7 is released. Installing CU7 gets you up to date in one fell swoop; it includes all fixes from all prior releases in this scenario
Next, if you just updated your entire Exchange farm a few months ago, how could you be ‘out of support’ already? Well, everyone knows how nimble the software business is today. Everyone knows how quickly code issues can be identified, especially a broadly deployed and heavily used feature, not to mention security vulnerabilities, including zero-day exploits. Microsoft remains committed to keeping the codebase for Exchange (and other products) equally nimble. They need a mantra like this to stay secured and to sustain their reputation of developing quality and safe software. And, in their defense, would you rather maintain 4 or 5 software codebase repositories, needing to patch every one of them when a security issue is identified, or just two? :)
Exchange’s mainstream support policy includes supplying security fixes for the two latest CUs (as I stated above, ‘N’ and ‘N-1’). Again, for Exchange Server 2019, CU8 and CU9 are supported. When CU10 is released, CU9 and CU10 will be supported.
“Updating Exchange Server seems complicated, how do I start?” Let me explain it in these three steps:
- Take an inventory of your servers. You can use this Exchange Server Health Checker Script on GitHub to see if you’re up to date or not with updates.
- Install the latest updates. You can use this very handy wizard to choose your current state and your desired future state in terms of CU level. It will instantly give you a nice, concise list of steps that will get you to the Pot O’ Gold.
- Troubleshoot. If you run into issues during updating procedures, follow this Exchange Update Wizard and these best practices for installation of updates. Be careful as you proceed and always remember to run any updates (setup.exe, ExchangeCU9.msp as examples) from an administrative command prompt or PowerShell console. Also, if you hit a real bad snag, be sure to utilize this documentation to get you back above water.
Next question – how should IT Pros balance the dynamic of keeping cognizant of 3rd party applications that rely on Exchange in our environments, and/or vice versa? Well, that’s simple. Not necessarily easy by any stretch of the imagination, but simple. You need to be in contact with your 3rd party support departments and make sure their software is also on the nimble bandwagon. Your Exchange servers contain arguably the most important information to your company. Protecting it and keeping it stable are a top priority.
What if your company runs 24×7? You have shifts around the clock, executives at all hours working across the globe. How are you supposed to incorporate downtime into a company that tolerates no downtime? In actuality, the Exchange update process is designed around this exact scenario. As long as you utilize Database Availability Groups (DAGs) and put servers gracefully into Maintenance Mode, you’ll encounter a non-disruptive process for your end-users.
A good one – Let’s say you’re running in Hybrid mode and only have a handful of mailboxes on-premises the vast majority are in the cloud (Exchange Online). Do you still need to keep your on-premises servers patched? Basically, you still need to keep your on-premises servers current. As you still use those servers to manage Exchange-related objects, they need to stay secure. Also, as a side note, you do not need to run the Hybrid Configuration Wizard after updates are installed.
Another question – If you’ve already looked up the specific CVEs (Common Vulnerabilities and Exposures) for the new security patches and their severity was on the low side, why update? This is nuanced; it’s not always vital in a specific month to patch a low severity vulnerability, but no one knows how that specific ‘low’ issue could interact with a future disclosed security issue. An attack may trigger only specific low-impact features or functions. It may only target a remote machine for example. Then, next month, there may be a discovery of additional functionality impacted. This new item could ALSO be a low rating. However, if your software is behind, say, for two months, these two issues could combine into an attack vector or chain that could result in a Critical alert!
The final question – You encounter inefficiencies when needing to update the AD schema when some of these CUs require them. You need to engage another team to plan, assess, and execute the schema updates. However, it can be difficult to schedule updates like these as it involves two separate teams with competing priorities. What to do? Microsoft recommends you request schema changes when the CUs are released. This can be a separate process/update; it doesn’t require you to run the CU and/or SU updates. So, you can make the request that the AD schema is updated. Two weeks later, that is complete. Then, you’re free to schedule and complete your Exchange CU and SU updates!
Thank you for learning more about keeping Exchange up to date. Although the emergency/critical updates released this past March were vital to getting deployed, the main point here is that you keep this continual mindset in place, so your Exchange environment is always humming and available to you and your company’s employees. The last thing you want to do is drop everything and go into emergency mode, discovering that you not only have the critical updates to install, but you first need to upgrade your entire infrastructure before doing so. Proactive versus reactive.