Those Exchange Online Bottomless Archives Aren’t Quite There, Yet… Except in Some Places

archive-key-hero

The Promise of the Bottomless Archive

The June 3, 2015 announcement that Exchange Online would provide “new auto-expanding, highly scalable archiving. This allows you to take advantage of a truly bottomless archive…” naturally created a lot of customer interest, especially for Office 365 tenants who need to retain large amounts of email information for compliance purposes. Tenants who want to import legacy email in the form of user PSTs or extracts from legacy archive servers also anticipate the ability to store this data in archive mailboxes. Unfortunately, the anticipated rollout of expandable archives has not proceeded quite as smoothly as predicted.

 

 

Although Microsoft “began a phased roll-out of this offering in the first quarter of 2016,” many tenants still report that their archive mailboxes are limited to 170GB composed of an initial 100GB allocation and an extension of 70GB that can be requested from Microsoft Support. The published limit for archive mailboxes for E3, E4, and E5 enterprise plans is “No Limit”, so what’s going on?

The Complex Nature of Exchange Online

Office 365 and Exchange Online are both complex infrastructures. Office 365 extends over 12 separate data center regions and Exchange Online is organized into a larger number of forests. When tenants sign up for Office 365, they elect to join a specific region and their Exchange Online mailboxes are assigned into a forest associated with that region. Just like an on-premises deployment, mailboxes are held in mailbox databases within Database Availability Groups (DAGs). Unlike any on-premises deployment, Exchange Online extends over thousands of DAGs and millions of database copies. All of which means that rolling out a new mailbox feature inside Exchange Online is a complicated affair that has to be managed on a global basis.

Changing the Mailbox Structure

Things are even more complex when the change involved a fundamental change in mailbox structure. Archive mailboxes are connected to their primary mailboxes by a GUID pointing to a location in the database that holds the mailbox. The GUID allows Exchange to connect the primary mailbox to its archive when clients request access to a user’s data. Expandable archives work by dividing a logical archive mailbox into multiple 50GB “chunks,” each of which is identified by a GUID. The GUIDs form a chain. When a client wants to access an archive mailbox, the GUIDs in the chain are used to connect the different chunks to form the archive.

You can use the Get-Mailbox cmdlet to view the information about the GUIDs for a mailbox:

[PS] C:\> Get-Mailbox –Identity TRedmond | Format-List MailboxLocations

MailboxLocations : {1;0370f354-2752-4437-878d-cf0e5310a8d4;Primary;
eurprd04.prod.outlook.com;353ce1b5-5044-4974-93f0-7b6f4a54edf8,
1;afc1e472-0826-498e-b990-85de223e809d;MainArchive;eurprd04.prod.outlook.com;
353ce1b5-5044-4974-93f0-7b6f4a54edf8}

We find information for the primary and archive mailboxes in MailboxLocations. The GUIDs tell Exchange Online the database where the mailbox and the archive are hosted and how many chunks exist. We can also see that the mailbox belongs to the eurprd04.prod.outlook.com forest. Only one GUID is present for the archive (“MainArchive”), so we know that there is not enough data in that mailbox for Exchange Online to expand it into a number of chunks. If more chunks are present for the archive, they are listed in the archive section as mailbox 2, 3, 4, and so on.

By comparison, a traditional mailbox is composed of one “chunk” that is limited by the amount of storage quota assigned to the mailbox. By default, the storage quotas assigned within Exchange Online are 50GB to mailboxes and 100GB for archives.

The process of expanding an archive mailbox into multiple chunks is automatic and happens without user intervention or knowledge. The only trace that expansion has occurred is found if you examine the MailboxLocations property for a mailbox using the Get-Mailbox cmdlet.

Clients are not aware of the complexity under the hood because they continue to use the same API calls to open and access the archive. Code within the Exchange Information Store process manage the process of connecting the chunks to form the logical archive.

Where Are Those Expanding Archives?

No one wants to mess up when the storage of customer data is involved. It should therefore not be a surprise to learn that the development group has been quite cautious about how they have deployed auto-expanding archives. Some initial hitches were more difficult than anticipated to overcome, but the good news is that “flighting” of the necessary code updates within Office 365 is now picking up pace. Deployment is complete in the Australia and APAC data center regions. Microsoft is monitoring the use and performance of expandable archives to make sure that everything works as expected. The rollout will continue to the larger Office 365 data center regions over the coming months.

According to Microsoft, many thousands of archive mailboxes have successfully used the auto-expand capability. Although the company won’t be more precise about the exact figures, Microsoft told me that some archive mailboxes have expanded past the one terabyte mark – or more than twenty 50GB chunks.

Not every mailbox is archive-enabled and not every mailbox that has an archive will ever store more than the normal 100GB limit. However, those that need automatic expansion are typically important to a company, usually for a reason linked to compliance, and this is why people are bothered when they suddenly discover that their archive is restricted to 170GB. Of course, an easy workaround exists to split the data that needs to be retained across multiple archive mailboxes, but that does not meet the expectations set by Microsoft. It also doesn’t help those who want to use the Office 365 Import Service or third-party migration software to bring the data held in PSTs into Exchange Online and so eradicate these pesky files.

In summary, auto-expanding archives are not a figment of anyone’s imagination. The technology does work and progress is being made to roll out the code to eligible users across the totality of Office 365. It’s just taken longer than expected.

Follow Tony on Twitter @12Knocksinna.

Want to know more about how to manage Office 365? Find what you need to know in “Office 365 for IT Pros,” the most comprehensive eBook covering all aspects of Office 365. Available in PDF and EPUB formats (suitable for iBooks) or for Amazon Kindle.