- Alerts and Monitoring >
- Alerts >
- Manage System Alerts
Manage System Alerts¶
On this page
System alerts are internal health checks that monitor the health of Ops Manager itself, including the health of backing databases, Backup Daemons, and backed-up deployments. Ops Manager runs health checks every five minutes.
To view the list of system alerts:
- Click the Admin link at the top of the Ops Manager UI.
- Click the Alerts tab.
- Click the Open Alerts link under System Alerts.
Disabled system alerts are grayed out.
If you have the Global Roles or Global Roles roles, you can modify notification settings or disable a system alert.
System Alert Components¶
Each system alert consists of three components:
Component | Examples |
---|---|
A condition that triggers the alert |
|
A list of recipients of the alert |
|
A method by which the alert is sent |
|
When the alert is enabled and its trigger condition is met, Ops Manager sends an alert to the specified recipients using the specified medium for that alert. For a list of the notification options, see the Select the alert recipients and delivery methods step in the Modify Notification Settings for a System Alert procedure on this page.
By default, Ops Manager enables all alerts and sends the alerts to the email address specified in the Admin Email Address field in the Ops Manager configuration options.
Available System Alerts¶
Ops Manager provides the following system alerts:
Alert Processing¶
Alert Type | Alert Message | Description |
---|---|---|
ORDINARY_ALERT_PROCESSING_DISABLED |
|
|
ORDINARY_ALERT_PROCESSING_ENABLED |
|
Backup¶
Alert Type | Alert Message | Description |
---|---|---|
OPLOG_TTL_RESIZE |
|
Sent when the Backup Daemon has fallen so far behind in applying oplog entries that Ops Manager has extended the period of time it stores the oplog entries. By default, Ops Manager stores oplog entries in the Oplog Store for 24 hours. If the Daemon has not yet applied an oplog entry an hour before its expiration, Ops Manager extends the storage period for another three hours. Ops Manager can continue to extend the storage period up to 14 days. If you receive this alert:
|
THEFT_FAILED |
|
Sent when a backup job migration to a new Backup Daemon fails. The backup job continues to run on the original Backup Daemon. For more information on moving jobs, see Jobs. |
Backup Daemon¶
Alert Type | Alert Message | Description |
---|---|---|
DAEMON_DOWN |
|
Sent when the Backup Daemon has not pinged Ops Manager for more than 15 minutes. |
DAEMON_UP |
|
|
LOW_HEAD_FREE_SPACE |
|
Sent when the disk partition on which the local copy of a backed-up replica set is stored has less than 1 GB of free space remaining. Follow the Modify Notification Settings for a System Alert procedure to change this space limit. |
LOW_HEAD_FREE_SPACE_PERCENT |
|
Sent when the disk partition on which the local copy of a backed-up replica set is stored has less than 10 percent of free space remaining. Follow the Modify Notification Settings for a System Alert procedure to change this percentage. |
SUFFICIENT_HEAD_FREE_SPACE |
|
Blockstore¶
Alert Type | Alert Message | Description |
---|---|---|
BALANCER_OFF |
|
|
BALANCER_ON |
|
Sent when a sharded blockstore is running the sharded cluster balancer. You should disable the balancer on a sharded blockstore. To disable the balancer, see Disable the Balancer. |
INSIDE_SPACE_USED_THRESHOLD |
|
|
OUTSIDE_SPACE_USED_THRESHOLD |
|
Sent when the disk space the
blockstore uses exceeds
the configured threshold
setting. The default threshold is 85% of the total disk capacity
on which the blockstore is stored. You can change the
mms.alerts.OutsideSpaceUsedThreshold.maximumSpaceUsedPercent
value in the Ops Manager configuration. |
Cron Job¶
Alert Type | Alert Message | Description |
---|---|---|
CRON_JOB_COMPLETED |
|
|
CRON_JOB_FAILED |
|
Cron Job Status¶
Alert Type | Alert Message | Description |
---|---|---|
CRON_JOB_DISABLED |
|
|
CRON_JOB_ENABLED |
|
Database Process¶
Alert Type | Alert Message | Description |
---|---|---|
BACKING_DATABASE_PROCESS_DOWN |
|
Sent when Ops Manager cannot connect to a backing database and run the ping command. |
BACKING_DATABASE_PROCESS_NO_STARTUP_WARNINGS |
|
|
BACKING_DATABASE_PROCESS_STARTUP_WARNINGS |
|
Sent when the MongoDB process hosting a backing
database contains startupWarnings in its
log files. |
BACKING_DATABASE_PROCESS_UP |
|
Modify Notification Settings for a System Alert¶
Click the Admin link in the top right corner of Ops Manager.¶
Ops Manager displays the Admin link only if you have administrative privileges.
Select the Alerts tab and then the System Alert Settings page.¶
On the line for the alert, click the ellipsis icon and select Edit.¶
Set the desired alert trigger values (Optional).¶
You can edit the values that trigger certain system alerts. These values are found in the Alert If section of the following alerts:
Alert | Field | Default |
---|---|---|
Backup daemon has low free head space |
Backup daemon free head space is below | 1 GB |
Backup daemon has low free head space percentage |
Backup daemon free head space percentage is below | 10 |
Select the alert recipients and delivery methods.¶
In the Send to section, configure notifications. To add notifications or recipients, click Add and select from the options listed below. To test a notification, click the test link that appears after you configure the notification and ensure that the service you are testing receives the message.
Notification Option | For Alert Type | Description |
---|---|---|
Project | Project or Global only | Sends the alert by email or SMS to the project. If you select SMS, Ops Manager sends the text message to the number configured on each user’s Account page. To send only to specific roles, clear All Roles and select the desired roles. |
Ops Manager User | Project only | Sends the alert by email or SMS to a specified Ops Manager user. If you select SMS, Ops Manager sends the text message to the number configured on the user’s Account page. |
Ops Manager Team | Project only | Sends the alert by email or SMS to a specified Ops Manager team. If you select SMS, Ops Manager sends the text message to the number configured on each team user’s Account page. Note This option appears only after at least one team has been created. |
SNMP Host | Any | Specify the hostname that will receive the v2c trap on
standard port 162 . The
MIB file for
SNMP is
available for download. |
Any | Sends the alert to a specified email address. | |
SMS | Any | Sends the alert to a specified mobile number. Available only if Ops Manager is configured for Twilio integration. |
HipChat | Any | Sends the alert to a HipChat room message stream. Enter the HipChat room name and API token. |
Slack | Any | Sends the alert to a Slack channel. Enter the channel name and either an API token or a Bot token. To create an API token, see the API page in your Slack account. For information on Bot users in Slack, see Bot Users. |
FlowDock | Any | Sends the alert to a Flowdock account. Enter the following:
|
PagerDuty | Any | Sends the alert to a PagerDuty account. Enter only the PagerDuty service key. Define escalation rules and alert assignments directly in PagerDuty. This option is available only for alerts that require acknowledgement. Informational alerts, such as the alert that a replica set has elected a new primary, cannot use this notification method. Users can acknowledge PagerDuty alerts only from the PagerDuty dashboard. |
Webhook | Global or Project only | Sends an HTTP POST request to an endpoint for programmatic processing. The request body contains a JSON document that uses the same format as the Ops Manager API Alerts resource. This option is available only if you have configured Webhook settings on the Group Settings page. |
DataDog | Any | Sends the alert to a DataDog account. Enter the following:
|
Administrators | Global or System only | Sends the alert to the email address specified in the Admin Email Address field in the Ops Manager configuration options. |
Global Alerts Summary Email | Global only | Sends a summary email of all global alerts to the specified email address. |
Click Save.¶
Disable a System Alert¶
Click the Admin link in the top right corner of Ops Manager.¶
Ops Manager displays the Admin link only if you have administrative privileges.
Select the Alerts tab and then the System Alert Settings page.¶
Disable the alert.¶
On the line for the system alert that you want to disable, click the ellipsis icon and select Disable.