Office 365 Alert Policies Automate Audit Monitoring

Alerts All Round

Activity alerts have been available in Office 365 since mid-2016. An activity alert watches the flow of events into the Office 365 audit log and fires when users perform a selected activity, like checking in a file into a document library. You can create an alert for any activity recorded in the audit log, that is, if you enable auditing for your tenant (don’t forget to enable mailbox auditing too).

Activity alerts are and straightforward and deliver the benefit that an administrator does not have to check the audit log for specific events. However, firing an alert does not need much intelligence. An event happens and an alert fires. Simple.

Alert Policies

Alert policies apply a lot more intelligence to check audit entries. A policy can trigger alerts for every instance of a certain activity, such as when an administrator grants another user elevated permissions, or can look for patterns of events, such as when a certain number of activities occur within a specific time. For instance, users download 100 files from a SharePoint library within an hour. That might be evidence of a hard-working user. On the other hand, it might be a sign that someone is grabbing some valuable intellectual property that they plan to take with them to another job.

More impressively, Office 365 can set the threshold for a trigger by analyzing up to a week’s worth of activities to understand what usually happens within a tenant. If something then happens that is outside the expected norm, Office 365 notifies the recipients set in the policy. Being able to create alert policies based on events, simple aggregation, or baseline thresholds can cover many of the circumstances that cause administrators to have grey hair.

Default Policies

To help administrators understand how alert policies function, Office 365 creates the six default alert policies described in Table 1. You can amend the recipients for alerts generated by default alert policies but you cannot change the other settings.

Activity Alert Conditions Severity
Elevation of Exchange Admin privilege Fires when an administrator assigns another user account administrative permissions for Exchange (for example, adding an account to the Organization Management role group). Low
Malware campaign detected after delivery Fires when an unusually (compared to baseline) number of messages with malware arrive in user mailboxes. High
Malware campaign detected and blocked Fires when Office 365 detects that attackers try to send an unusual volume of messages with known malware to the tenant. Low
Unusual external user file activity Fires when external users with access to SharePoint or OneDrive for Business sites in the tenant perform an unusual number of activities like downloading files. Medium
Unusual volume of external file sharing Fires when users share an unusual number of files in SharePoint or OneDrive for Business sites with external users. Medium
Unusual volume of file deletion Fires when users remove an unusual number of files from SharePoint or OneDrive for Business sites within a brief period Medium

Table 1: Default Office 365 alert policies

Creating Custom Alert Policies

Like any default policies, the default alert policies cover some generic situations that might or might not apply to your tenant, which is why you can define your own policies. To begin, go to the Alerts section of the Security and Compliance Center. As you can see in Figure 1, we already have three custom policies defined alongside the six default policies (bolded).

Office 365 Alert Policies 1
Figure 1: Alert policies in the Security and Compliance Center (image credit: Tony Redmond)

Alert Policy Settings

To create a new policy, click New alert policy and then enter the name of the policy, a free-form description, a category for the alert, which you can use to sort alerts in the View alerts page. The categories are:

  • Data governance.
  • Data loss prevention.
  • Threat management.
  • Anything else.

You assign the alerts fired by a policy to be Low, Medium, or High. Obviously, the more destructive a condition is, the higher its severity should be. After defining the basic elements of a policy, the next step is to set out what the policy will do, as shown in Figure 2.

Office 365 Alert Policies 2
Figure 2: Defining the conditions for an alert policy (image credit: Tony Redmond)

Alert policies do not yet cover all the activities recorded in the Office 365 audit log because the intention is that alert policies can deliver near real-time notifications about problem and not all workloads feed events into the audit log that quickly. You can only select a single activity per policy, like the “Granted mailbox permission” activity selected here.

For most activities, you can define conditions that must exist before Office 365 fires an alert. For example, a user downloads a file to a computer with a specific IP address. It is also possible to configure alerts to fire every time a user performs the activity or when users download files from a specific site. We are interested in the activities of three users, so they are conditions for the policy.

The Threshold defines how often an activity must occur within a period before a problem condition exists. The threshold can also be unusual activity, which is when Office 365 compares activity against the baseline for the tenant. Setting too low a threshold usually results in many notifications, which can hide real problems (the lowest threshold is 3 activities). Setting too high means that Office 365 might not send notifications for situations when administrators need to act. It is easy to tune a threshold after noting how many notifications administrators must process to reach a point where notifications arrive when real problems exist. You can also set a daily limit for an alert policy so that a policy can only ever generate a certain number of notifications in a single day.

Handling Alerts

When Office 365 recognizes that the threshold and conditions set in an alert policy occur, it triggers an email alert (Figure 3) to the recipient list defined in the policy.

Office 365 Alert Policies 3
Figure 3: Email notification for an alert policy (image credit: Tony Redmond)

The alert message has an Investigate button. The link takes you to a page to view the details of the triggered alert. You can then resolve or suppress the alert. When you resolve an alert, you record a status for the alert (Active, Investigating, Resolved, Dismissed) and any comments to justify the status. The new status and comments and your name become part of the history of the alert.

If you suppress an alert, you tell Office 365 how long you want to suppress email notifications for similar alerts and select from 1 day, 7 days, 30 days, or 365 days. Office 365 continues to trigger new alerts during the suppression period but you will not receive email about them.


The Alerts dashboard (Figure 4) has widgets for alert trends (number of incidents over the last two weeks), alerts by severity, recent alerts, alert policies, and other alerts. These widgets give an overview of what is happening in the tenant. More details of recent alerts are available through the View alerts choice.

Office 365 Alerts Policy 4
Figure 4: The Alerts dashboard (image credit: Tony Redmond)

Automation is Goodness

Automated checking of audit events is all goodness. No one wants to have to check events manually, especially at the volume that a busy tenant generates. Alert policies are not a reason by themselves to upgrade to Office 365 E5 and many will be happy with activity alerts, but it is nice to have the option.

Follow Tony on Twitter @12Knocksinna.

Want to know more about how to manage Office 365? Find what you need to know in “Office 365 for IT Pros”, the most comprehensive eBook covering all aspects of Office 365. Available in PDF and EPUB formats (suitable for iBooks) or for Amazon Kindle.