Navigation

Manage Sharded Collections

Overview

Sharding distributes data across multiple machines. MongoDB uses sharding to support deployments with very large data sets and high throughput operations. Ops Manager can create sharded clusters and sharded collections on those clusters.

This page explains how Ops Manager can manage sharded collections including determining how documents are distributed within sharded collections.

Sharding involves defining a shard key which is then used to partition documents within a collection. See the MongoDB manual for a more detailed explanation of sharding.

A shard key consists of one or more indexed fields that exist in every document within a collection. A shard key on a compound index is known as a compound shard key. Each collection can only have one shard key. You cannot change the shard key once you shard a collection. A sharded cluster can support both sharded and unsharded collections. See the MongoDB manual for best practices on choosing a shard key.

The sharded cluster attempts to distribute the documents in a sharded collection evenly among the shards in the cluster. You can use sharding zones to manage the distribution of documents within the collection.

Zone sharding associates ranges of a collection’s shard key values to one or more shards in the cluster called a zone. MongoDB eventually routes documents within a given range to the associated zone. This allows for targeted data distribution. Ops Manager supports both zoned and default sharding. See the MongoDB manual for a more detailed explanation of zone sharding.

Note

Tag Aware Sharding and Zone Sharding are interchangeable. Tag Aware sharding transitions to Zone Sharding with the release of MongoDB 3.4.

The following procedures explain how Ops Manager can:

  • Manage your sharded collections
  • Create new sharded collections
  • Import your sharded collections into Ops Manager
  • Define zones for sharded clusters
  • Define ranges for sharded collections

Each procedure assumes you have clicked the Deployment button to display the Deployment page first.

Enable sharded collection management

You can use Ops Manager to manage sharded collections. If you want Ops Manager to manage sharded collections, you need to run the import process in Ops Manager first. This ensures that no collections have their configurations overriden accidentally.

1

Click the ellipsis menu.

2

Select Manage Sharding.

The Sharding tab displays.

3

Click Manage Sharding.

4

Click Import Sharding Info to manage sharded collections on existing sharded clusters.

Imports any existing sharded collections and zones.

5

If the import is successful, click Review and Deploy.

6

Troubleshoot any failed imports. (Optional)

There are a few possible errors that could happen when importing sharded collections.

Overlapping ranges

Ops Manager does not support overlapped defined ranges.

Example

A compound shard key may appear to have overlapped ranges when it does not. This example explains the difference.

A simple compound shard key comprises two integers with values between 1 and 10. The chunk ranges for a collection where each chunk is approximately 64 MB are:

min max
[$min, $min] [1, 8]
[1,8] [3,1]
[3,1] [5,2]
[5,2] [5,10]
[5,10] [7,3]
[7,3] [$max,$max]

The ranges are based on the two values combined (or compound) and not each value individually. The second value goes up and down in each chunk, but the combination always increases from minimum to maximum.

You can use the mongo shell to resolve this issue on the database directly.

  1. Check the status of the shard ranges.
  2. Review the ranges for possible overlap.
  3. Remove a Tag from the Shard Key Range.
  4. Add a new Tag to a Shard Key Range.
Data type mismatch

For each range, Ops Manager requires the minimum and maximum values of each field in a shard key to be the same BSON data type. A compound shard key in a range can use a different BSON type for each field in the key. Ops Manager verifies this when the sharded collections are imported and when ranges are created.

Note

The Min key and Max key are different data types and are the only exception to not mixing BSON data types in the range.

You can use the mongo shell to resolve this issue on the database directly.

  1. Check the status of the shard ranges.
  2. Check Types in the mongo Shell for the minimum and maximum values.
  3. Remove a Tag from the Shard Key Range.
  4. Add a new Tag to a Shard Key Range.
Data type invalid

The minimum and maximum values for a range can only use eight BSON data types:

  • String
  • Integer
  • Double
  • Long
  • Date
  • Timestamp
  • ObjectId
  • MinKey / MaxKey

See also

Review information on the data types in the BSON specification.

You can use the mongo shell to resolve this issue on the database directly.

  1. Check the status of the shard ranges.
  2. Check Types in the mongo Shell for the minimum and maximum values.
  3. Remove a Tag from the Shard Key Range.
  4. Add a new Tag to a Shard Key Range.
7

Review your Changes.

  • If the changes are acceptable, click Confirm and Deploy.
  • If the changes need to be revised, click Cancel and return to Step 1.

Create a new sharded collection

You can create a new sharded collecton using Ops Manager.

1

Click New Collection.

2

Type the Database and Collection names into the respective fields.

3

Type the name for the Shard Key into the Shard Key 1 field.

There are two mutually exclusive options for Shard Keys:

  • Check hashed if you want to use a hashed shard key.
  • Check Enforce Unique Key if you want to have unique key names.

A Shard Key cannot be unique and hashed.

Note

If the field or fields chosen as the shard key are not indexed, Ops Manager creates an index for the shard key field(s) during this procedure.

4

If you want to create a compound shard key, click + add another field.

  • A compound shard key cannot be hashed, so the hashed checkbox is grayed out.
  • Check Enforce Unique Key if you want to have unique key names.

A compound shard key cannot include more than three keys.

5

Click Set Up Ranges to zone shards. (Optional)

If you want to use zone sharding on this collection, follow the steps under Define how collections are sharded using ranges.

6

Click Review and Deploy.

7

Review your Changes.

  • If the changes are acceptable, click Confirm and Deploy.
  • If the changes need to be revised, click Cancel and return to Step 1.

Configure zoned sharding

Note

Follow the next two procedures in this section if you intend on using zoned sharding for your sharded collections. Otherwise, you may skip this section.

Group shards into zones

Zones are a named group of one or more shards. After creating one or more zones, you can assign a range of shard key values and their corresponding documents to a zone. MongoDB eventually routes documents within a given range to the associated zone. Each zone can include multiple ranges and multiple shards. Each shard can belong to more than one zone. Each shard displays its zone(s) to the right of its name under Deployment.

1

Click Configure Zones.

2

Click + add another zone to add a new Zone.

3

Type a name into the Zone Name field.

4

Select shards to place in a given zone in the Shard(s) dropdown menu.

5

Click Review and Deploy.

If you try to delete a shard zone that has a tagged range associated with it, it fails. If you try to remove the last shard from a zone that has ranges tagged to it, that also fails. You must move all tagged ranges to another zone before you can remove the last shard from that zone.

6

Review your Changes.

  • If the changes are acceptable, click Confirm and Deploy.
  • If the changes need to be revised, click Cancel and return to Step 1.

Define how collections are sharded using ranges

Ranges specify minimum and maximum values for each field in a shard key. Each defined range is associated to a single zone. MongoDB eventually routes documents within a given range to the associated zone. The minimum value is an inclusive lower bound of the shard key values. The maximum value is exclusive upper bound of the shard key values. A range can belong to only one zone, but a zone can have multiple ranges.

Documents are routed based on the configured zones and ranges once the balancer moves the range into the desired zone. Once that occurs, documents within a range are routed to the associated zone and those outside a range may be routed to any shard in the cluster.

1

Select the data type for the shard key.

Important

Once you automate sharding, the Automation Agent reverts any configuration changes not made through the Ops Manager interface.

The Automation Agent does not change anything in sharded collections that are not managed through Ops Manager.

2

For each shard key, enter the minimum and maximum values and select the associated zone.

Compound shard keys have one range per component shard key but together are associated with only one zone.

A range’s minimum value is inclusive and the maximum value is exclusive.

Example

The following two ranges do not overlap:

min max zone
1 10 A
10 20 B

Note

Min and Max are absolute values: the absolute minimum and maximum value of any range without explicitly listing a specific value.

Each range can be associated only to a single zone. You cannot assign the same range to more than one zone.

3

If you have another range to set, click + add another range.

You can add additional shard key ranges for the given collection.

You cannot assign the same range to another zone in this manner.

4

Click Review and Deploy.

5

Review your Changes.

  • If the changes are acceptable, click Confirm and Deploy.
  • If the changes need to be revised, click Cancel and return to Step 1.

Disable sharded collection management

Click Unmanage.

Important

When a sharded collection is unmanaged, your sharded collections and zones are not deleted. These collections and zones can no longer be managed from the Ops Manager interface.