• Back Up and Restore Deployments >
• Restore MongoDB Deployments >
• Restore a Sharded Cluster from a Backup

Restore a Sharded Cluster from a Backup

Overview

When you restore a cluster from a backup, Ops Manager provides you with a restore files for the selected restore point. For an overview of the restore process, please see Restore Overview.

Considerations

The BSON specification changed the default subtype for the BSON binary datatype (BinData) from 2 to 0. Some binary data stored in a snapshot may be BinData subtype 2. The Backup Agent automatically detects and converts snapshot data in BinData subtype 2 to BinData subtype 0. If your application code expects BinData subtype 2, you must update your application code to work with BinData subtype 0.

See also

The notes on the BSON specification explain the particular specifics of this change.

The backup restore file now includes a metadata file, restoreInfo.txt. This file captures the options the database used when the snapshot was taken. The database must be run with the listed options after it has been restored.

See also

The Considerations of Seed a New Secondary from Snapshot explains the use and contents of the restoreInfo.txt metadata file.

Prerequisites

Client Requests During Restoration

You must ensure that the MongoDB deployment does not receive client requests during restoration. You must either:

• Restore to new systems with new hostnames and reconfigure your application code once the new deployment is running, or
• Ensure that the MongoDB deployment will not receive client requests while you restore data.

Snapshots when Agent Cannot Stop Balancer

Ops Manager displays a warning next to cluster snapshots taken while the balancer is enabled. If you restore from such a snapshot, you run the risk of lost or orphaned data. For more information, see Snapshots when Agent Cannot Stop Balancer.

Secure Copy (SCP) Delivery

Important

SCP requires you to generate a key pair before attempting to transfer files. SCP provides faster file delivery than HTTP.

Note

Microsoft Windows computers do not include SCP and require additional setup outside the scope of this manual.

Important

When copying files individually, the MaxStartups value in sshd_config should be increased to:

(4 × (number of shards + number of config servers)) + 10

SCP is performed in parallel and, by default, Secure Shell Daemon (sshd) installations use a small number of concurrent connections. Changing this setting in sshd_config allows SCP to support sufficient connections to complete the restore.

Example

For a sharded cluster with 7 shards and 3 config servers, change MaxStartups to 50:

MaxStartups  50

Automatic Restore

To have Ops Manager automatically restore the backup, perform the select the snapshot procedure.

Manual Restore

To restore the backup manually, perform the following:

1. Select and Retrieve the Snapshot.
2. Restore Primary Replica Set Member for Each Shard.
3. Restore All Secondary Replica Set Members for Each Shard.
4. Restore Each Config Server.
5. Start the MongoDB Sharded Cluster.

Select and Retrieve the Snapshot

1

Select the Backup tab and then Overview page.

2

Click the deployment’s ellipsis icon and select Restore.

3

Select the restore point.

Choose the point from which you want to restore your backup.

Restore Type	Description	Action
Snapshot	Allows you to choose one stored snapshot.	Select an existing snapshot to restore.
Point In Time	Creates a custom snapshot that includes all operations up to but not including the selected time. Example If you select 12:00, the last operation in the restore is 11:59:59 or earlier.	Select a Date and Time.

Click Next.

4

Select whether to have Ops Manager restore the snapshot.

Yes	Ops Manager restores the snapshot for you. If you select this option, Ops Manager prompts to choose the cluster to which to restore. You can select an existing cluster or create a new one. Follow the prompts in the wizard. Important You can skip the remainder of this page.
No, I’ll do it myself	Ops Manager provides you with the restore files for you to perform the restore manually.

5

Choose how to receive the restore files.

Select the restore method, format and destination.

Pull Via Secure HTTP	Type a direct download link. Select from the following options: Pull Restore Usage Limit Select whether the link can be re-used or used just once. If you select No Limit, the link is re-usable until it expires. Restore Link Expiration (in hours) Select the number of hours until the link expires. Important You can skip the remainder of this procedure.
Push Via Secure Copy	Direct Ops Manager to copy the restore files to your server via SCP. Important SCP requires you to generate a key pair before attempting to transfer files. SCP provides faster file delivery than HTTP. Note Microsoft Windows computers do not include SCP and require additional setup outside the scope of this manual.
Format	Select the format in which you want to receive the restore files: Individual DB Files Transmits MongoDB data files produced by Ops Manager directly to the target directory. The data is compressed during transmission. Archive Delivers database files in a single archive (tar or tar.gz) that you must extract before restoring the databases to a working directory. This option displays only if the archive size can be calculated. With Archive delivery, you need sufficient space on the destination server for both the archive and the extracted files.
SCP Host	Type the hostname of the server to receive the files.
SCP Port	Type the port of the server to receive the files.
SCP User	Type the username used to access to the server.
Auth Method	Select whether to use a username and password or an SSH certificate to authenticate to the server.
Password	Type the user password used to access to the server.
Passphrase	Type the SSH passphrase used to access to the server.
Target Directory	Type the absolute path to the directory on the server to which to copy the restore files.

Click Finalize Request.

6

Retrieve the snapshot.

If you selected Pull Via Secure HTTP:

Ops Manager creates links to the snapshot. By default, these links are available for an hour and can be used just once. To download the snapshot:

1. Click Backup, then the Restore History tab.
2. When the restore job completes, a download link appears for each shard and for one of the config servers.
3. Click each link to download the restore archives.
4. For the config server, copy its config restore archive to each of its replica set members.
5. For each shard, copy its restore archive to each of its replica set members.
6. Extract the shard restore archive to the shard’s working database path.

If you selected Push Via Secure Copy:

Ops Manager copies the files to the server directory you specified. After the files transfer, you should:

1. Verify that the files are complete, see how to validate a secure copy restore.
2. Copy the config server restore archive to each of its replica set members.
3. Copy each shard restore archive to each shard’s replica set members.
4. Extract the shard restore archive to the shard’s working database path.

Restore Primary Replica Set Member for Each Shard

For all shards, restore the primary replica set member. You must have a copy of the snapshot on the server that host the primary replica set member.

1

Shut down the entire replica set.

Shut down the replica set’s mongod processes using one of the following methods, depending on your configuration:

Automated Deployment

If you use Ops Manager Automation to manage the replica set, you must shut down through the Ops Manager console. See Shut Down a MongoDB Process.

Non-Automated Deployment on MongoDB 2.6 or Later

Connect to each member of the set and issue the following:

copy

use admin
db.shutdownServer()

Non-Automated Deployment on MongoDB 2.4 or earlier

Connect to each member of the set and issue the following:

copy

use admin
db.shutdownServer( { force: true } )

2

Restore the snapshot data files to the primary.

Extract the data files to the location where the mongod process will access them through the dbpath setting. If you are restoring to existing hardware, use a different data directory than used previously. The following are example commands:

copy

mv <snapshot> /data
tar -xvf <snapshot>.tar.gz

3

Start the primary with the new dbpath.

For example:

copy

mongod --dbpath <pathToData> --replSet <replicaSetName> --logpath <pathToData>/mongodb.log --fork

4

Connect to the primary and initiate the replica set.

Connect using the mongo shell and issue the rs.initiate() command.

5

Restart the primary as a standalone, without the --replSet option.

Shut down the process using one of the following methods:

Automated Deployment

Shut down through the Ops Manager console. See Shut Down a MongoDB Process.

Non-Automated Deployment on MongoDB 2.6 or Later

copy

use admin
db.shutdownServer()

Non-Automated Deployment on MongoDB 2.4 or earlier

copy

use admin
db.shutdownServer( { force: true } )

Restart the process as a standalone:

copy

mongod --dbpath /<pathToData> --logpath /<pathToData>/mongodb.log --fork

6

Connect to the primary and drop the oplog.

For example, first issue the following to connect:

copy

mongo

And then issue rs.drop() to drop the oplog.

copy

use local
db.oplog.rs.drop()

7

Run the seedSecondary script on the primary.

Use the appropriate script for your operating system:

UNIX-based	seedSecondary.sh
Windows	seedSecondary.bat

This script recreates the oplog collection and seeds it with the timestamp of the snapshot’s creation. This tells the secondary to recreate all operations to the current time without requiring a full initial sync. Ops Manager customizes this script for this particular snapshot and includes it in the snapshot archive.

To run the script, issue the following command, where:

<alternatePort>	The port of the mongod process
<oplogSizeInGigabytes>	The size of the replica set’s oplog
<replicaSetName>	The name of the replica set
<primaryHost:primaryPort>	The hostname:port combination for the replica set’s primary

For UNIX-based systems:

copy

./seedSecondary.sh <alternatePort> <oplogSizeInGigabytes> <replicaSetName> <primaryHost:primaryPort>
./seedSecondary.sh 27018 2 rs-1 primaryHost.example.com:27017

For Windows-based systems:

copy

.\seedSecondary.bat <alternatePort> <oplogSizeInGigabytes> <replicaSetName> <primaryHost:primaryPort>
.\seedSecondary.bat 27018 2 rs-1 primaryHost.example.com:27017

8

Restart the primary as part of a replica set.

Use the following sequence:

1. Shut down the process using one of the following methods:

   • Automated Deployment:

     Shut down through the Ops Manager console. See Shut Down a MongoDB Process.

   • Non-Automated Deployment on MongoDB 2.6 or Later:

     copy

     use admin
     db.shutdownServer()
     

   • Non-Automated Deployment on MongoDB 2.4 or earlier:

     copy

     use admin
     db.shutdownServer( { force: true } )
     

2. Restart the process as part of a replica set:

   copy

   mongod --dbpath /<pathToData> --replSet <replicaSetName>
   

Restore All Secondary Replica Set Members for Each Shard

After you have restored the primary for a shard you can restore all secondary replica set members. You must have a copy of the snapshot on all servers that host a secondary replica set member.

1

Connect to the server where you will create the new secondary.

2

Restore the snapshot data files to the secondary.

Extract the data files to the location where the mongod process will access them through the dbpath setting. If you are restoring to existing hardware, use a different data directory than used previously. The following are example commands:

copy

mv <snapshot> /data
tar -xvf <snapshot>.tar.gz

3

Start the secondary as a standalone, without the --replSet option.

Shut down the process using one of the following methods:

Automated Deployment

Shut down through the Ops Manager console. See Shut Down a MongoDB Process.

Non-Automated Deployment on MongoDB 2.6 or Later

copy

use admin
db.shutdownServer()

Non-Automated Deployment on MongoDB 2.4 or earlier

copy

use admin
db.shutdownServer( { force: true } )

Restart the process as a standalone:

copy

mongod --dbpath /<pathToData> --logpath /<pathToData>/mongodb.log --fork

4

Run the seedSecondary.sh script on the secondary.

Use the appropriate script for your operating system:

UNIX-based	seedSecondary.sh
Windows	seedSecondary.bat

This script recreates the oplog collection and seeds it with the timestamp of the snapshot’s creation. This tells the secondary to recreate all operations to the current time without requiring a full initial sync. Ops Manager customizes this script for this particular snapshot and includes it in the snapshot archive.

To run the script, issue the following command, where:

<alternatePort>	The port of the mongod process
<oplogSizeInGigabytes>	The size of the replica set’s oplog
<replicaSetName>	The name of the replica set
<primaryHost:primaryPort>	The hostname:port combination for the replica set’s primary

For UNIX-based systems:

copy

./seedSecondary.sh <alternatePort> <oplogSizeInGigabytes> <replicaSetName> <primaryHost:primaryPort>
./seedSecondary.sh 27018 2 rs-1 primaryHost.example.com:27017

For Windows-based systems:

copy

.\seedSecondary.bat <alternatePort> <oplogSizeInGigabytes> <replicaSetName> <primaryHost:primaryPort>
.\seedSecondary.bat 27018 2 rs-1 primaryHost.example.com:27017

5

Restart the secondary as part of the replica set.

Use the following sequence:

1. Shut down the process using one of the following methods:

   • Automated Deployment:

     Shut down through the Ops Manager console. See Shut Down a MongoDB Process.

   • Non-Automated Deployment on MongoDB 2.6 or Later:

     copy

     use admin
     db.shutdownServer()
     

   • Non-Automated Deployment on MongoDB 2.4 or earlier:

     copy

     use admin
     db.shutdownServer( { force: true } )
     

2. Restart the process as part of a replica set:

   copy

   mongod --dbpath /<pathToData> --replSet <replicaSetName>
   

6

Connect to the primary and add the secondary to the replica set.

Connect to the primary and use rs.add() to add the secondary to the replica set.

copy

rs.add("<host>:<port>")

Repeat this operation for each member of the set.

Restore Each Config Server

Perform this procedure separately for each config server. Each config server must have a copy of the tar file with the config server data.

Use this procedure only for manual restores.

1

Restore the snapshot to the config server.

Extract the data files to the location where the config server’s mongod process will access them. This is the location you will specify as the storage.dbPath when running mongod for the config server.

copy

tar -xvf <backup-restore-name>.tar.gz
mv <backup-restore-name> /data

2

Start the config server.

The following example starts the config server using the new data:

copy

mongod --configsvr --dbpath /data

3

Update the sharded cluster metadata.

If the new shards do not have the same hostnames and ports as the original cluster, you must update the shard metadata. To do this, connect to each config server and update the data.

First connect to the config server with the mongo shell. For example:

copy

mongo

Then access the shards collection in the config database. For example:

copy

use config
db.shards.find().pretty()

The find() method returns the documents in the shards collection. The collection contains a document for each shard in the cluster. The host field for a shard displays the name of the shard’s replica set and then the hostname and port of the shard. For example:

copy

{ "_id" : "shard0000", "host" : "shard1/localhost:30000" }

To change a shard’s hostname and port, use the MongoDB update() command to modify the documents in the shards collection.

Start the mongos process

Start the cluster’s mongos bound to your new config servers.

←   Restore Flows Restore a Replica Set from a Backup  →

© MongoDB, Inc 2008-present. MongoDB, Mongo, and the leaf logo are registered trademarks of MongoDB, Inc.