Ops Manager backups, once started, are an ongoing and continuous process. Data is continually backed up as long as the backup remains synchronized with the database. This process works like replica set data synchronization
The backup process:
Performs an initial sync to back up all of your existing data in its current state. In sharded clusters, this occurs on each shard and on the config servers.
Takes snapshots of the
datadirectory in a deployment as often as your snapshot schedule specifies and then transfers the snapshots to a storage system.
Monitors the oplog constantly and adds new database operations to the latest backup to keep the local Ops Manager copy of the data current.
The backup process works in this manner regardless of how snapshots are stored.
Backup Definition and Operational States¶
Each backup is defined as a job. Each job defines how much and how often data is backed up. Backup jobs are defined on a per-group basis.
The following table lists the states of a backup job:
|State||Retain Old Snapshots||Create New Snapshots||Apply Oplogs|
Change Operational States¶
Once backup jobs are active for a group, they run without further intervention until they are stopped or terminated. The operator can change the state of a backup in the following ways:
|Initial State||Desired State||Method|
Terminate deletes all retained backups.
Backup Process Flows¶
Once created, a backup job goes through the following process flow:
- The Backup Agent connects to, and authenticates with, the databases associated with the backup job.
- The initial sync begins and enters its
startingphase. Initial sync is a transition state between Inactive and Active. Initial Sync goes through a series of phases that are displayed on the Backup page to show progress. The Backup Agent streams the existing data to Ops Manager in 10 MB compressed bundles of documents called slices. The Backup Agent creates slices at the point in time when the backup was created. Data inserted to the instance after that point in time is captured separately.
transferringphase begins as the slices are streamed and stored in the Oplog Store temporarily on the Backup Daemon’s behalf. The Backup Daemon service cannot dedicate itself to processing the large stream of initial sync slices at the expense of processing other backup jobs. The Oplog Store stores the slices until the Backup Daemon can fetch them. The Oplog Store is created when the first Snapshot Store is created.
- While the Backup Agent is streaming the data, it tails the oplog. This tailing collects any differences between the state of the deployment database when the backup began and the deployment database’s current state. The oplog entries are sent in 10 MB compressed bundles of documents called oplog slices. These two streams of slices are collected in parallel to reduce the time needed to construct a complete backup.
buildingphase begins once Ops Manager has received all of the slices. In this phase, Ops Manager creates a local version of the backed up database called a head database on the host running the Backup Daemon service.
- Ops Manager, through the Backup Daemon service, inserts the documents stored in the Oplog Store into the head database.
applying oplogsphase begins as Ops Manager applies the tailed oplog entries into the head database.
- During the
fetching missing documentsphase, Ops Manager queries the deployment database for documents missed during document insertion. Ops Manager inserts the missing documents found in the deployment database into the head database.
- After inserting the missing documents, the
creating indexesphase begins as Ops Manager creates all of the indexes found in the deployment databases in the head database. When the indexes finish, the initial sync ends and the phase changes to
- Depending upon which snapshot store you chose to store your backup, a snapshot can be written out:
The characteristics of each storage method is covered in Backup Configuration Options.
The head database works as a full copy of the deployment database. It needs oplogs applied to it on a regular basis to keep its data synchronized with the deployment database. Snapshots are generated from the data stored in the head database according to your snapshot schedule.
Once the first full backup is completed, each active backup job follows this process:
- The Backup Agent tails the deployment’s oplog.
- The Backup Agent routinely batches new oplog entries in oplog slices and transfers them to Ops Manager.
- Ops Manager stores the oplog entries in the Oplog Store.
- Ops Manager applies the new oplog entries from the oplog slices to the head database that stores the deployment backup.
- Ops Manager creates a new snapshot and stores it in the snapshot store as specified in your snapshot schedule.