Navigation

Global Alerts

Note

Groups and projects are synonymous terms. Your {GROUP-ID} is the same as your project id. For existing groups, your group/project id remains the same. This page uses the more familiar term group when referring to descriptions. The endpoint remains as stated in the document.

The globalAlerts resource allows you to retrieve and acknowledge alerts that have been triggered by a global alert configuration. You must have the Global Monitoring Admin to use this resource.

Base URL: https://{OPSMANAGER-HOST}:{PORT}/api/public/v1.0

Alert Status

When Ops Manager detects an alert condition, it opens an alert. If the global alert configuration contains no notification delay, the alert status goes immediately to OPEN. If the configuration contains a delay, Ops Manager sets the alert to TRACKING until the delay period ends, after which Ops Manager sets the alert to OPEN if the condition persists.

If the global alert configuration has multiple notifications, each with its own notification delay, Ops Manager uses the smallest delay value to determine when to move an alert from TRACKING to OPEN.

Endpoints

Get All Global Alerts

GET /globalAlerts

To specify alert status, use the status query parameter with one of the following values:

  • TRACKING
  • OPEN
  • CLOSED
GET /globalAlerts?status=STATUS

The status parameter cannot retrieve CANCELLED global alerts.

Get a Specific Global Alert

GET /globalAlerts/ALERT-ID

Acknowledge an Alert

To acknowledge a global alert, use PATCH to update the alert’s acknowledgedUntil field. You can optionally update the acknowledgementComment field with a comment. Do not include any other field in the PATCH request.

PATCH /globalAlerts/GLOBAL-ALERT-ID

To acknowledge an alert “forever”, set the field value to 100 years in the future. To unacknowledge a previously acknowledged alert, set the field value to the past.

If you add a comment in the acknowledgementComment field, Ops Manager displays the comment next to the message that the alert has been acknowledged.

Sample Entity

The following is an example return document. The fields in a return document depend on the alert type.

{
  "id" : "56e81a12e4b82547f6cd3a7",
  "groupId" : "56cfafabe4b0391691d7c2a5",
  "alertConfigId" : "56a104e0e4b00664f588ac8",
  "created" : "2016-10-15T14:20:02Z",
  "eventTypeName" : "INCONSISTENT_BACKUP_CONFIGURATION",
  "lastNotified" : "2016-10-17T15:21:25Z",
  "replicaSetName" : "testSet",
  "resolved" : "2016-10-17T15:21:20Z",
  "sourceTypeName" : "REPLICA_SET",
  "status" : "CLOSED",
  "tags" : [ ],
  "typeName" : "BACKUP",
  "updated" : "2016-10-17T15:21:20Z"
  "links" : []
}

Entity Fields

Name Type Description
id string Unique identifier.
groupId string Unique identifier of the group that this alert was opened for.
alertConfigId string Unique identifier of the global alert configuration that triggered this alert.
created date When the alert was opened.
eventTypeName string

The name of the event that triggered the alert.

  • Host alerts support these values:
    • HOST_DOWN
    • HOST_RECOVERING
    • VERSION_BEHIND
    • HOST_EXPOSED
    • OUTSIDE_METRIC_THRESHOLD
  • Agent alerts support these values:
    • MONITORING_AGENT_DOWN
    • MONITORING_AGENT_VERSION_BEHIND
    • BACKUP_AGENT_DOWN
    • BACKUP_AGENT_VERSION_BEHIND
    • BACKUP_AGENT_CONF_CALL_FAILURE
  • Backup alerts support these values:
    • OPLOG_BEHIND
    • CLUSTER_MONGOS_IS_MISSING
    • RESYNC_REQUIRED
    • BAD_CLUSTERSHOTS
    • RS_BIND_ERROR (x alerts only)
    • BACKUP_TOO_MANY_RETRIES (global alerts only)
    • BACKUP_IN_UNEXPECTED_STATE (global alerts only)
    • LATE_SNAPSHOT (global alerts only)
    • SYNC_SLICE_HAS_NOT_PROGRESSED (global alerts only)
    • BACKUP_JOB_TOO_BUSY (global alerts only)
    • GROUP_TAGS_CHANGED (global alerts only)
  • Project alerts support these values:
    • USERS_AWAITING_APPROVAL
    • USERS_WITHOUT_MULTI_FACTOR_AUTH
  • Replica set alerts support these values:
    • CONFIGURATION_CHANGED
    • PRIMARY_ELECTED
    • TOO_FEW_HEALTHY_MEMBERS
    • TOO_MANY_UNHEALTHY_MEMBERS
    • NO_PRIMARY
  • Sharded cluster alerts support this value:
    • CLUSTER_MONGOS_IS_MISSING
  • User alerts support these values:
    • JOINED_GROUP
    • REMOVED_FROM_GROUP
lastNotified date When the last notification was sent for this alert. Only present if notifications have been sent.
replicaSetName string Name of the replica set. Only present for global alerts of type HOST, HOST_METRIC, BACKUP, and REPLICA_SET.
resolved date When the alert was closed. Only present if the status is CLOSED.
sourceTypeName string

For global alerts of the type BACKUP, the type of server being backed up. Possible values are:

  • REPLICA_SET
  • SHARDED_CLUSTER
  • CONFIG_SERVER
status string

The current state of the alert. Possible values are:

  • TRACKING
  • OPEN
  • CLOSED
  • CANCELLED
tags array of strings The tags associated with the alert.
typeName string This field is deprecated and will be ignored.
updated date When the alert was last updated.
acknowledgedUntil date The date through which the alert has been acknowledged. Only present if the alert has been acknowledged.
acknowledgementComment string The comment left by the user who acknowledged the alert. Only present if the alert has been acknowledged.
acknowledgingUsername string The username of the user who acknowledged the alert. Only present if the alert has been acknowledged.
hostnameAndPort string The hostname and port of each host to which the alert applies. Only present for alerts of type HOST, HOST_METRIC, and REPLICA_SET.
hostId string Unique identifier of the host to which the metric pertains. Only present for alerts of type HOST, HOST_METRIC, and REPLICA_SET.
metricName string

The name of the measurement whose value went outside the threshold. Only present if eventTypeName is set to OUTSIDE_METRIC_THRESHOLD.

For possible values, see Measurement Types for Global Alerts on this page.

currentValue object The current value of the metric that triggered the alert. Only present for alerts of type HOST_METRIC.
currentValue.number number The value of the metric.
currentValue.units string

The units for the value. Depends on the type of metric. For example, a metric that measures memory consumption would have a byte measurement, while a metric that measures time would have a time unit. Possible values are:

  • RAW
  • BITS
  • BYTES
  • KILOBITS
  • KILOBYTES
  • MEGABITS
  • MEGABYTES
  • GIGABITS
  • GIGABYTES
  • TERABYTES
  • PETABYTES
  • MILLISECONDS
  • SECONDS
  • MINUTES
  • HOURS
  • DAYS
clusterId string The Unique identifier of the cluster to which this alert applies. Only present for alerts of type BACKUP, REPLICA_SET, and CLUSTER.
clusterName string The name the cluster to which this alert applies. Only present for alerts of type BACKUP, REPLICA_SET, and CLUSTER.

Measurement Types for Global Alerts

The globalAlerts resource returns measurement types in the metricName field. The field is present only if eventTypeName is set to OUTSIDE_METRIC_THRESHOLD.

Host Measurements

  • ASSERT_REGULAR
  • ASSERT_WARNING
  • ASSERT_MSG
  • ASSERT_USER
Rate of asserts for a MongoDB process found in the asserts document that the serverStatus command generates.
  • BACKGROUND_FLUSH_AVG
Amount of data flushed in the background.
  • CACHE_BYTES_READ_INTO
  • CACHE_BYTES_WRITTEN_FROM
  • CACHE_DIRTY_BYTES
  • CACHE_USED_BYTES
  • TICKETS_AVAILABLE_READS
  • TICKETS_AVAILABLE_WRITE
Amount of bytes in the WiredTiger storage engine cache and tickets found in the wiredTiger.cache and wiredTiger.concurrentTransactions documents that the serverStatus command generates.
  • CONNECTIONS
Number of connections to a MongoDB process found in the connections document that the serverStatus command generates.
  • CURSORS_TOTAL_OPEN
  • CURSORS_TOTAL_TIMED_OUT
Number of cursors for a MongoDB process found in the metrics.cursor document that the serverStatus command generates.
  • EXTRA_INFO_PAGE_FAULTS
  • GLOBAL_ACCESSES_NOT_IN_MEMORY
  • GLOBAL_PAGE_FAULT_EXCEPTIONS_THROWN
Numbers of Memory Issues and Page Faults for a MongoDB process.
  • GLOBAL_LOCK_CURRENT_QUEUE_TOTAL
  • GLOBAL_LOCK_CURRENT_QUEUE_READERS
  • GLOBAL_LOCK_CURRENT_QUEUE_WRITERS
Number of operations waiting on locks for the MongoDB process that the serverStatus command generates. Ops Manager computes these values based on the type of storage engine.
  • INDEX_COUNTERS_BTREE_ACCESSES
  • INDEX_COUNTERS_BTREE_HITS
  • INDEX_COUNTERS_BTREE_MISSES
  • INDEX_COUNTERS_BTREE_MISS_RATIO
Number of index btree operations.
  • JOURNALING_COMMITS_IN_WRITE_LOCK
  • JOURNALING_MB
  • JOURNALING_WRITE_DATA_FILES_MB
Number of journaling operations.
  • MEMORY_RESIDENT
  • MEMORY_VIRTUAL
  • MEMORY_MAPPED
  • COMPUTED_MEMORY
Amount of memory for a MongoDB process found in the mem document that the serverStatus command collects.
  • NETWORK_BYTES_IN
  • NETWORK_BYTES_OUT
  • NETWORK_NUM_REQUESTS
Amount of throughput for MongoDB process found in the network document that the serverStatus command collects.
  • OPLOG_SLAVE_LAG_MASTER_TIME
  • OPLOG_MASTER_TIME
  • OPLOG_MASTER_LAG_TIME_DIFF
  • OPLOG_RATE_GB_PER_HOUR
Durations and throughput of the MongoDB process’ oplog.
  • DB_STORAGE_TOTAL
  • DB_DATA_SIZE_TOTAL
Number of database operations on a MongoDB process since the process last started.
  • OPCOUNTER_CMD
  • OPCOUNTER_QUERY
  • OPCOUNTER_UPDATE
  • OPCOUNTER_DELETE
  • OPCOUNTER_GETMORE
  • OPCOUNTER_INSERT
Rate of database operations on a MongoDB process since the process last started found in the opcounters document that the serverStatus command collects.
  • OPCOUNTER_REPL_CMD
  • OPCOUNTER_REPL_UPDATE
  • OPCOUNTER_REPL_DELETE
  • OPCOUNTER_REPL_INSERT
Rate of database operations on MongoDB secondaries found in the opcountersRepl document that the serverStatus command collects.
  • DOCUMENT_METRICS_RETURNED
  • DOCUMENT_METRICS_INSERTED
  • DOCUMENT_METRICS_UPDATED
  • DOCUMENT_METRICS_DELETED
Average rate of documents returned, inserted, updated, or deleted per second during a selected time period.
  • OPERATIONS_SCAN_AND_ORDER
Average rate for operations per second during a selected time period that perform a sort but cannot perform the sort using an index.
  • OP_EXECUTION_TIME_READS
  • OP_EXECUTION_TIME_WRITES
  • OP_EXECUTION_TIME_COMMANDS

Average execution time in milliseconds per read, write, or command operation during a selected time period.

Available to hosts running MongoDB 3.4 or later.

  • RESTARTS_IN_LAST_HOUR
Number of times the host restarted within the previous hour.
  • QUERY_EXECUTOR_SCANNED
Average rate per second to scan index items during queries and query-plan evaluations found in the value of totalKeysExamined from the explain command.
  • QUERY_EXECUTOR_SCANNED_OBJECTS
Average rate of documents scanned per second during queries and query-plan evaluations found in the value of totalDocsExamined from the explain command.
  • QUERY_TARGETING_SCANNED_PER_RETURNED
Ratio of the number of index items scanned to the number of documents returned.
  • QUERY_TARGETING_SCANNED_OBJECTS_PER_RETURNED
Ratio of the number of documents scanned to the number of documents returned.

Examples

Get All Global Alerts

Request

curl -i -u "username:apiKey" --digest "https://<ops-manager-host>/api/public/v1.0/globalAlerts"

Response

Response Header

HTTP/1.1 401 Unauthorized
Content-Type: application/json;charset=ISO-8859-1
Date: {dateInUnixFormat}
WWW-Authenticate: Digest realm="MMS Public API", domain="", nonce="{nonce}", algorithm=MD5, op="auth", stale=false
Content-Length: {requestLengthInBytes}
Connection: keep-alive
HTTP/1.1 200 OK
Vary: Accept-Encoding
Content-Type: application/json
Strict-Transport-Security: max-age=300
Date: {dateInUnixFormat}
Connection: keep-alive
Content-Length: {requestLengthInBytes}

Response Body

{
  "links" : [],
  "results" : [ {
    "alertConfigId" : "573b7d12e4b0979a262467c1",
    "created" : "2016-10-18T08:08:08Z",
    "currentValue" : {
      "number" : 143.4739833843463,
      "units" : "RAW"
    },
    "eventTypeName" : "OUTSIDE_METRIC_THRESHOLD",
    "groupId" : "5729fde7e4b0132d250fed5f",
    "hostId" : "63f42376fb735471fe40ec54a7",
    "hostnameAndPort" : "replicaset-shard-00-02:27017",
    "id" : "573b7d2de4b02fd2c93423a6",
    "links" : [],
    "metricName" : "OPCOUNTER_CMD",
    "lastNotified" : "2016-10-18T19:29:54Z",
    "replicaSetName" : "replicaSet-shard-0",
    "resolved" : "2016-10-18T21:30:04Z",
    "status" : "CLOSED",
    "tags" : [ ],
    "typeName" : "HOST_METRIC",
    "updated" : "2016-10-18T21:30:04Z"
  }, ... ],
  "totalCount" : 8
}

Get All Open Global Alerts

Request

curl -i -u "username:apiKey" --digest "https://<ops-manager-host>/api/public/v1.0/globalAlerts?status=OPEN"

Response

Response Header

HTTP/1.1 401 Unauthorized
Content-Type: application/json;charset=ISO-8859-1
Date: {dateInUnixFormat}
WWW-Authenticate: Digest realm="MMS Public API", domain="", nonce="{nonce}", algorithm=MD5, op="auth", stale=false
Content-Length: {requestLengthInBytes}
Connection: keep-alive
HTTP/1.1 200 OK
Vary: Accept-Encoding
Content-Type: application/json
Strict-Transport-Security: max-age=300
Date: {dateInUnixFormat}
Connection: keep-alive
Content-Length: {requestLengthInBytes}

Response Body

{
  "links" : [],
  "results" : [ {
    "alertConfigId" : "5730f5e1e4b030a9634a3f69",
    "clusterId" : "572a00f2e4b051814b144e90",
    "clusterName" : "shardedCluster",
    "created" : "2016-10-09T06:16:36Z",
    "eventTypeName" : "OPLOG_BEHIND",
    "groupId" : "5729fde7e4b0132d250fed5f",
    "id" : "3b7d2de0a4b02fd2c98146de",
    "links" : [],
    "lastNotified" : "2016-10-10T20:42:32Z",
    "replicaSetName" : "shardedCluster-shard-0",
    "sourceTypeName" : "REPLICA_SET",
    "status" : "OPEN",
    "tags" : [ ],
    "typeName" : "BACKUP",
    "updated" : "2016-10-10T20:42:32Z"
  }, ... ],
  "totalCount" : 3
}

Get a Specific Global Alert

Request

curl -i -u "username:apiKey" --digest "https://<ops-manager-host>/api/public/v1.0/globalAlerts/3b7d2de0a4b02fd2c98146de"

Response

Response Header

HTTP/1.1 401 Unauthorized
Content-Type: application/json;charset=ISO-8859-1
Date: {dateInUnixFormat}
WWW-Authenticate: Digest realm="MMS Public API", domain="", nonce="{nonce}", algorithm=MD5, op="auth", stale=false
Content-Length: {requestLengthInBytes}
Connection: keep-alive
HTTP/1.1 200 OK
Vary: Accept-Encoding
Content-Type: application/json
Strict-Transport-Security: max-age=300
Date: {dateInUnixFormat}
Connection: keep-alive
Content-Length: {requestLengthInBytes}

Response Body

{
  "alertConfigId" : "5730f5e1e4b030a9634a3f69",
  "clusterId" : "572a00f2e4b051814b144e90",
  "clusterName" : "shardedCluster",
  "created" : "2016-10-09T06:16:36Z",
  "eventTypeName" : "OPLOG_BEHIND",
  "groupId" : "5729fde7e4b0132d250fed5f",
  "id" : "3b7d2de0a4b02fd2c98146de",
  "links" : [],
  "lastNotified" : "2016-10-10T20:42:32Z",
  "replicaSetName" : "shardedCluster-shard-0",
  "sourceTypeName" : "REPLICA_SET",
  "status" : "OPEN",
  "tags" : [ ],
  "typeName" : "BACKUP",
  "updated" : "2016-10-10T20:42:32Z"
}

Acknowledge an Alert

Request

curl -i -u "username:apiKey" --digest -H "Content-Type: application/json" -X PATCH "https://<ops-manager-host>/api/public/v1.0/globalAlerts/3b7d2de0a4b02fd2c98146de" --data '
{
  "acknowledgedUntil": "2016-11-01T00:00:00-0400"
}'

Response

Response Header

HTTP/1.1 401 Unauthorized
Content-Type: application/json;charset=ISO-8859-1
Date: {dateInUnixFormat}
WWW-Authenticate: Digest realm="MMS Public API", domain="", nonce="{nonce}", algorithm=MD5, op="auth", stale=false
Content-Length: {requestLengthInBytes}
Connection: keep-alive
HTTP/1.1 200 OK
Vary: Accept-Encoding
Content-Type: application/json
Strict-Transport-Security: max-age=300
Date: {dateInUnixFormat}
Connection: keep-alive
Content-Length: {requestLengthInBytes}

Response Body

{
  "alertConfigId" : "5730f5e1e4b030a9634a3f69",
  "clusterId" : "572a00f2e4b051814b144e90",
  "clusterName" : "shardedCluster",
  "created" : "2016-10-09T06:16:36Z",
  "eventTypeName" : "OPLOG_BEHIND",
  "groupId" : "5729fde7e4b0132d250fed5f",
  "id" : "3b7d2de0a4b02fd2c98146de",
  "links" : [],
  "lastNotified" : "2016-10-10T20:42:32Z",
  "replicaSetName" : "shardedCluster-shard-0",
  "sourceTypeName" : "REPLICA_SET",
  "status" : "OPEN",
  "acknowledgedUntil" : "2016-11-01T00:00:00Z",
  "acknowledgingUsername" : "admin@example.com",
  "tags" : [ ],
  "typeName" : "BACKUP",
  "updated" : "2016-10-10T22:03:11Z"
}