Office 365 Alert Policies Automate Audit Monitoring
Last Update: Sep 04, 2024 | Published: Jul 27, 2017
Alerts All Round
Activity alerts have been available in Office 365 since mid-2016. An activity alert watches the flow of events into the Office 365 audit log and fires when users perform a selected activity, like checking in a file into a document library. You can create an alert for any activity recorded in the audit log, that is, if you enable auditing for your tenant (don’t forget to enable mailbox auditing too).
Activity alerts are and straightforward and deliver the benefit that an administrator does not have to check the audit log for specific events. However, firing an alert does not need much intelligence. An event happens and an alert fires. Simple.
Alert Policies
Alert policies apply a lot more intelligence to check audit entries. A policy can trigger alerts for every instance of a certain activity, such as when an administrator grants another user elevated permissions, or can look for patterns of events, such as when a certain number of activities occur within a specific time. For instance, users download 100 files from a SharePoint library within an hour. That might be evidence of a hard-working user. On the other hand, it might be a sign that someone is grabbing some valuable intellectual property that they plan to take with them to another job.
More impressively, Office 365 can set the threshold for a trigger by analyzing up to a week’s worth of activities to understand what usually happens within a tenant. If something then happens that is outside the expected norm, Office 365 notifies the recipients set in the policy. Being able to create alert policies based on events, simple aggregation, or baseline thresholds can cover many of the circumstances that cause administrators to have grey hair.
Default Policies
To help administrators understand how alert policies function, Office 365 creates the six default alert policies described in Table 1. You can amend the recipients for alerts generated by default alert policies but you cannot change the other settings.
| Activity Alert | Conditions | Severity |
| Elevation of Exchange Admin privilege | Fires when an administrator assigns another user account administrative permissions for Exchange (for example, adding an account to the Organization Management role group). | Low |
| Malware campaign detected after delivery | Fires when an unusually (compared to baseline) number of messages with malware arrive in user mailboxes. | High |
| Malware campaign detected and blocked | Fires when Office 365 detects that attackers try to send an unusual volume of messages with known malware to the tenant. | Low |
| Unusual external user file activity | Fires when external users with access to SharePoint or OneDrive for Business sites in the tenant perform an unusual number of activities like downloading files. | Medium |
| Unusual volume of external file sharing | Fires when users share an unusual number of files in SharePoint or OneDrive for Business sites with external users. | Medium |
| Unusual volume of file deletion | Fires when users remove an unusual number of files from SharePoint or OneDrive for Business sites within a brief period | Medium |
Table 1: Default Office 365 alert policies
Creating Custom Alert Policies
Like any default policies, the default alert policies cover some generic situations that might or might not apply to your tenant, which is why you can define your own policies. To begin, go to the Alerts section of the Security and Compliance Center. As you can see in Figure 1, we already have three custom policies defined alongside the six default policies (bolded).
Alert Policy Settings
To create a new policy, click New alert policy and then enter the name of the policy, a free-form description, a category for the alert, which you can use to sort alerts in the View alerts page. The categories are:
- Data governance.
- Data loss prevention.
- Threat management.
- Anything else.
You assign the alerts fired by a policy to be Low, Medium, or High. Obviously, the more destructive a condition is, the higher its severity should be. After defining the basic elements of a policy, the next step is to set out what the policy will do, as shown in Figure 2.
Alert policies do not yet cover all the activities recorded in the Office 365 audit log because the intention is that alert policies can deliver near real-time notifications about problem and not all workloads feed events into the audit log that quickly. You can only select a single activity per policy, like the “Granted mailbox permission” activity selected here.
For most activities, you can define conditions that must exist before Office 365 fires an alert. For example, a user downloads a file to a computer with a specific IP address. It is also possible to configure alerts to fire every time a user performs the activity or when users download files from a specific site. We are interested in the activities of three users, so they are conditions for the policy.
The Threshold defines how often an activity must occur within a period before a problem condition exists. The threshold can also be unusual activity, which is when Office 365 compares activity against the baseline for the tenant. Setting too low a threshold usually results in many notifications, which can hide real problems (the lowest threshold is 3 activities). Setting too high means that Office 365 might not send notifications for situations when administrators need to act. It is easy to tune a threshold after noting how many notifications administrators must process to reach a point where notifications arrive when real problems exist. You can also set a daily limit for an alert policy so that a policy can only ever generate a certain number of notifications in a single day.
Handling Alerts
When Office 365 recognizes that the threshold and conditions set in an alert policy occur, it triggers an email alert (Figure 3) to the recipient list defined in the policy.
The alert message has an Investigate button. The link takes you to a page to view the details of the triggered alert. You can then resolve or suppress the alert. When you resolve an alert, you record a status for the alert (Active, Investigating, Resolved, Dismissed) and any comments to justify the status. The new status and comments and your name become part of the history of the alert.
If you suppress an alert, you tell Office 365 how long you want to suppress email notifications for similar alerts and select from 1 day, 7 days, 30 days, or 365 days. Office 365 continues to trigger new alerts during the suppression period but you will not receive email about them.
Dashboard
The Alerts dashboard (Figure 4) has widgets for alert trends (number of incidents over the last two weeks), alerts by severity, recent alerts, alert policies, and other alerts. These widgets give an overview of what is happening in the tenant. More details of recent alerts are available through the View alerts choice.
Automation is Goodness
Automated checking of audit events is all goodness. No one wants to have to check events manually, especially at the volume that a busy tenant generates. Alert policies are not a reason by themselves to upgrade to Office 365 E5 and many will be happy with activity alerts, but it is nice to have the option.
Follow Tony on Twitter @12Knocksinna.
Want to know more about how to manage Office 365? Find what you need to know in “Office 365 for IT Pros”, the most comprehensive eBook covering all aspects of Office 365. Available in PDF and EPUB formats (suitable for iBooks) or for Amazon Kindle.