### Start mongosync with Configuration File Source: https://www.mongodb.com/docs/mongosync/current/reference/mongosync-behavior Example of starting the `mongosync` process using a YAML configuration file. This is useful for managing multiple `mongosync` options in a centralized file. ```shell $ mongosync --config /etc/mongosync.conf ``` -------------------------------- ### Start Migration Verifier Source: https://www.mongodb.com/docs/mongosync/current/reference/verification/verifier Command to start the migration-verifier process. Ensure you replace example URIs with your actual cluster connection strings. ```bash migration-verifier --verifyAll \ --srcURI example.net:27020 \ --destURI example.net:27021 \ --metaURI example.net:27017 ``` -------------------------------- ### Start mongosync with Namespace Filters Source: https://www.mongodb.com/docs/mongosync/current/reference/collection-level-filtering Attach filter JSON to the `/start` API call to begin syncing. This example includes collections starting with `accounts_` from the `sales` database and all collections from the `marketing` database. ```bash curl -X POST "http://localhost:27182/api/v1/start" --data ' { "source": "cluster0", "destination": "cluster1", "includeNamespaces": [ { "database": "sales", "collectionsRegex": { "pattern": "^accounts_.+$", "options": "i" } }, { "database": "marketing" } ] } ' ``` -------------------------------- ### Start API - Copy All Databases and Collections Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Initiates a data migration using the /start API. This example demonstrates copying all databases and their collections in their natural order. ```APIDOC ## POST /api/v1/start ### Description Initiates a data migration to copy all databases and their collections in their natural order. ### Method POST ### Endpoint /api/v1/start ### Request Body - **source** (string) - Required - The source cluster identifier. - **destination** (string) - Required - The destination cluster identifier. - **copyInNaturalOrder** (array) - Required - An array containing a single object with `database` set to "." to indicate copying all databases. - **database** (string) - Required - Set to "." to signify copying all databases. ### Request Example ```json { "source": "cluster0", "destination": "cluster1", "copyInNaturalOrder": [ { "database": "." } ] } ``` ### Response #### Success Response (200) - **success** (boolean) - Indicates if the operation was successful. ``` -------------------------------- ### Start API - Copy Specific Databases and Collections Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Initiates a data migration using the /start API. This example demonstrates copying specific databases and collections in their natural order. ```APIDOC ## POST /api/v1/start ### Description Initiates a data migration, allowing for the specification of databases and collections to be copied in their natural order. ### Method POST ### Endpoint /api/v1/start ### Request Body - **source** (string) - Required - The source cluster identifier. - **destination** (string) - Required - The destination cluster identifier. - **copyInNaturalOrder** (array) - Required - An array of objects, where each object specifies a database and optionally a list of collections to copy. - **database** (string) - Required - The name of the database. - **collections** (array) - Optional - An array of collection names to copy within the specified database. ### Request Example ```json { "source": "cluster0", "destination": "cluster1", "copyInNaturalOrder": [ { "database": "sales", "collections": [ "accounts", "orders" ] }, { "database": "marketing", "collections": [ "offers" ] } ] } ``` ### Response #### Success Response (200) - **success** (boolean) - Indicates if the operation was successful. ``` -------------------------------- ### Start mongosync with Database and Collection Filtering Source: https://www.mongodb.com/docs/mongosync/current/reference/collection-level-filtering Example configuration for starting mongosync with a filter that includes specific databases and collections. This filter syncs all collections from 'marketing' and specified collections from 'sales'. ```json "includeNamespaces" : [ { "database" : "sales", "collections": [ "EMEA", "APAC" ] }, { "database" : "marketing" } ] ``` -------------------------------- ### Start API - Copy All Collections in Specific Databases Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Initiates a data migration using the /start API. This example demonstrates copying all collections within specified databases in their natural order. ```APIDOC ## POST /api/v1/start ### Description Initiates a data migration, allowing for the specification of databases to copy all their collections in their natural order. ### Method POST ### Endpoint /api/v1/start ### Request Body - **source** (string) - Required - The source cluster identifier. - **destination** (string) - Required - The destination cluster identifier. - **copyInNaturalOrder** (array) - Required - An array of objects, where each object specifies a database for which all collections should be copied. - **database** (string) - Required - The name of the database. ### Request Example ```json { "source": "cluster0", "destination": "cluster1", "copyInNaturalOrder": [ { "database": "sales" }, { "database": "marketing" } ] } ``` ### Response #### Success Response (200) - **success** (boolean) - Indicates if the operation was successful. ``` -------------------------------- ### mongosync Start Configuration Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Configuration options for starting a mongosync migration, including namespace filtering and natural order copying. ```APIDOC ## Request Body Parameters ### `detectRandomId` - **Type**: boolean - **Necessity**: Optional - **Description**: When `true`, `mongosync` checks for random `_id` values in collections that are not clustered or capped, and are larger than 20 GiB. If `mongosync` determines that the collection uses random `_id` values, it copies the collection in natural order instead of the default `_id` order. This reduces the time `mongosync` takes to complete the collection copy phase. By default, `detectRandomId` is `true`. ### `includeNamespaces` - **Type**: array - **Necessity**: Optional - **Description**: Filters the databases or collections to include in sync. If you configure a filter on a source cluster that has multiple databases, `mongosync` only syncs the databases specified in the filter definition. ### `excludeNamespaces` - **Type**: array - **Necessity**: Optional - **Description**: Filters the databases or collections to exclude from sync. If you configure a filter on a source cluster that has multiple databases, `mongosync` only syncs the databases specified in the filter definition. ### `copyInNaturalOrder` - **Type**: array of documents - **Necessity**: Optional - **Description**: Copies a list of databases and collections in their natural order to the destination cluster. The natural order is the order in which you previously inserted the documents into the database. You must pass in an array of documents that each represent a database and its collections. Only use this option for collections that explicitly set a random `_id` field and are larger than 30 GiB. ``` -------------------------------- ### Start a Filtered Sync Job Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Starts a sync job with a filter to include specific databases and collections. Use `includeNamespaces` to define the scope of the sync. This example filters the 'sales' database to include only 'EMEA' and 'APAC' collections, and includes the entire 'marketing' database. ```json "includeNamespaces" : [ { "database" : "sales", "collections": [ "EMEA", "APAC" ] }, { "database" : "marketing" } ] ``` ```shell curl -X POST "http://localhost:27182/api/v1/start" --data ' { "source": "cluster0", "destination": "cluster1", "includeNamespaces": [ { "database": "sales", "collectionsRegex": { "pattern": "^accounts_.+$", "options": "i" } }, { "database": "marketing" } ] } ' ``` ```json {"success":true} ``` -------------------------------- ### Example mongosync Connection Strings Source: https://www.mongodb.com/docs/mongosync/current/quickstart These are example connection strings for two clusters, 'cluster0' and 'cluster1'. Replace placeholders with your actual cluster details and credentials. ```shell cluster0: mongodb://clusterAdmin:superSecret@clusterOne01.fancyCorp.com:20020,clusterOne02.fancyCorp.com:20020,clusterOne03.fancyCorp.com:20020 cluster1: mongodb://clusterAdmin:superSecret@clusterTwo01.fancyCorp.com:20020,clusterTwo02.fancyCorp.com:20020,clusterTwo03.fancyCorp.com:20020 ``` -------------------------------- ### INITIALIZING State Source: https://www.mongodb.com/docs/mongosync/current/reference/mongosync-states In the INITIALIZING state, the sync process is setting up and is not yet ready to start. The only available operation is to get the sync progress. ```APIDOC ## GET /progress ### Description Retrieves the current progress of the mongosync process. ### Method GET ### Endpoint /progress ``` -------------------------------- ### Start mongosync with a Configuration File Source: https://www.mongodb.com/docs/mongosync/current/reference/configuration Use the --config option to specify the path to your mongosync configuration file when starting the binary. This allows mongosync to load settings from the file. ```shell mongosync --config ``` -------------------------------- ### Start Multiple mongosync Instances Source: https://www.mongodb.com/docs/mongosync/current/topologies/multiple-mongosyncs Use the mongosync API to start synchronization for each instance. Ensure the options are consistent across all instances. ```APIDOC ## Start Multiple `mongosync` Instances Use `curl` or another HTTP client to issue the [start](https://mongodbcom-cdn.staging.corp.mongodb.com/docs/mongosync/reference/api/start/#std-label-c2c-api-start) command to each of the `mongosync` instances. ```shell curl mongosync01Host:27601/api/v1/start -XPOST --data '{ "source": "cluster0", "destination": "cluster1", "reversible": false, "enableUserWriteBlocking": "none" }' curl mongosync02Host:27602/api/v1/start -XPOST --data '{ "source": "cluster0", "destination": "cluster1", "reversible": false, "enableUserWriteBlocking": "none" }' ``` The `start` command options must be the same for all of the `mongosync` instances. ``` -------------------------------- ### Start a Filtered Sync Job Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Initiates a sync job with specific database and collection filters using `includeNamespaces`. This example demonstrates filtering on specific collections within a database and including entire databases. ```APIDOC ## POST /api/v1/start ### Description Starts a sync job with specified source and destination clusters, applying filters to include only certain databases and collections. ### Method POST ### Endpoint /api/v1/start ### Parameters #### Request Body - **source** (string) - Required - The source cluster identifier. - **destination** (string) - Required - The destination cluster identifier. - **includeNamespaces** (array) - Required - An array of namespace filters. Each object can specify a database and an optional array of collections or a `collectionsRegex` object. - **database** (string) - Required - The name of the database. - **collections** (array) - Optional - An array of collection names to include. - **collectionsRegex** (object) - Optional - A regular expression to filter collection names. - **pattern** (string) - Required - The regular expression pattern. - **options** (string) - Optional - Regex options (e.g., 'i' for case-insensitive). ### Request Example ```json { "source": "cluster0", "destination": "cluster1", "includeNamespaces": [ { "database": "sales", "collectionsRegex": { "pattern": "^accounts_.$", "options": "i" } }, { "database": "marketing" } ] } ``` ### Response #### Success Response (200) - **success** (boolean) - Indicates if the operation was successful. ``` -------------------------------- ### Example Atlas Cluster Connection Strings Source: https://www.mongodb.com/docs/mongosync/current/connecting/atlas-to-atlas Example connection strings for two Atlas clusters, 'cluster0' and 'cluster1', using a database administrative user. ```shell cluster0: mongodb+srv://clusterAdmin:superSecret@cluster1Name.abc123.mongodb.net cluster1: mongodb+srv://clusterAdmin:superSecret@cluster2Name.abc123.mongodb.net ``` -------------------------------- ### Example Progress Response Source: https://www.mongodb.com/docs/mongosync/current/reference/api/commit This is an example response from the `progress` endpoint. It indicates the current state of the synchronization, including `canCommit` status and lag times. ```json { "progress": { "state":"RUNNING", "canCommit":true, "canWrite":false, "info":"change event application", "lagTimeSeconds":0, "collectionCopy": { "estimatedTotalBytes":694, "estimatedCopiedBytes":694 }, "directionMapping": { "Source":"cluster0: localhost:27017", "Destination":"cluster1: localhost:27018" }, "source": { "pingLatencyMs":250 }, "destination": { "pingLatencyMs":-1 }, "verification": { "source": { "estimatedDocumentCount": 42, "hashedDocumentCount": 42, "lagTimeSeconds": 2, "totalCollectionCount": 42, "scannedCollectionCount": 10, "phase": "stream hashing" }, "destination": { "estimatedDocumentCount": 42, "hashedDocumentCount": 42, "lagTimeSeconds": 2, "totalCollectionCount": 42, "scannedCollectionCount": 10, "phase": "stream hashing" } } }, "success": true } ``` -------------------------------- ### Example Source MD5 Hash Output Source: https://www.mongodb.com/docs/mongosync/current/reference/verification/hash This is an example of the MD5 hash output returned by the `dbHash` command for the source cluster. ```text d41d8cd98f00b204e9800998ecf8427e ``` -------------------------------- ### Start API Endpoint Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Initiates a mongosync data migration with specified parameters. ```APIDOC ## POST /start ### Description Initiates a data synchronization task. This endpoint allows for detailed configuration of the migration process, including options for handling pre-existing data, reversibility, sharding configurations, and verification settings. ### Method POST ### Endpoint /start ### Parameters #### Request Body - **preExistingDestinationData** (boolean) - Optional - Allows pre-existing namespaces on the destination cluster. Requires namespace filters if set to true. Defaults to `false`. - **reversible** (boolean) - Optional - Enables the sync operation to be reversed. Not supported for all configurations. Defaults to `false`. - **sharding** (document) - Optional - Configures sync between a replica set and sharded cluster. Required for sync from replica set to sharded cluster. - **verification** (Document) - Optional - Configures the embedded verifier. ### Request Example { "preExistingDestinationData": true, "includeNamespaces": ["mydb.mycollection"], "reversible": false, "sharding": { "clusterRole": "configsvr" }, "verification": { "mode": "embedded" } } ### Response #### Success Response (200) - **status** (string) - Indicates the success of the operation. - **message** (string) - Provides details about the migration status. ``` -------------------------------- ### Regex Filter Example with Options Source: https://www.mongodb.com/docs/mongosync/current/reference/collection-level-filtering/filter-regex This example demonstrates matching collections in the 'sales' database that start with 'accounts_'. It uses the 'm' option for multiline matching and 's' to allow the dot character to match newline characters. ```json "includeNamespaces": [ { "database": "sales", "collectionsRegex": { "pattern": "^accounts_.+?$", "options": "ms" } } ] ``` -------------------------------- ### Sharding Parameters for Start API Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Details on the sharding parameters that can be configured when starting a sync job to a sharded cluster. ```APIDOC ## Sharding Parameters for POST /api/v1/start ### Description When syncing to a sharded cluster, the `sharding` option can be used to configure how collections are sharded on the destination cluster. ### Parameters #### Request Body (within the main request body for POST /api/v1/start) - **sharding** (object) - Optional - Configuration for sharding collections on the destination cluster. - **createSupportingIndexes** (boolean) - Optional - Whether to create a supporting index for the shard key if none exists. Defaults to `false`. - **shardingEntries** (array of documents) - Required if `sharding` is used - Specifies collections to shard and their shard keys. - **collection** (string) - Required - The name of the collection to shard. - **database** (string) - Required - The database of the collection to shard. - **shardCollection** (document) - Required - Defines the shard key for the collection. - **key** (array) - Required - An array of fields to use for the shard key. ``` -------------------------------- ### mongosync Configuration File Example Source: https://www.mongodb.com/docs/mongosync/current/reference/configuration Specify mongosync settings like cluster connection strings and log paths in a YAML configuration file. This example shows basic cluster definitions and log configuration. ```yaml cluster0: "mongodb://192.0.2.10:27017" cluster1: "mongodb://192.0.2.20:27017" logPath: "/var/log/mongosync" verbosity: "WARN" ``` -------------------------------- ### Start a Sync Job Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Starts a sync job between two specified clusters. This is the basic command to initiate data synchronization. ```shell curl localhost:27182/api/v1/start -XPOST \ --data \ { "source": "cluster0", "destination": "cluster1" } ``` ```json {"success":true} ``` -------------------------------- ### Start Migration Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Initiates the data synchronization process between a source and destination cluster. This endpoint configures and starts the migration, allowing for various options related to index building. ```APIDOC ## POST /start ### Description Starts the data synchronization process between a source and destination cluster. ### Method POST ### Endpoint /start ### Parameters #### Request Body - **source** (string) - Required - Name of the source cluster. - **destination** (string) - Required - Name of the destination cluster. - **buildIndexes** (string) - Optional - Configures index builds during sync. Supported options include `afterDataCopy`, `beforeDataCopy`, `excludeHashed`, `excludeHashedAfterCopy`, and `never`. Defaults to building indexes on the destination cluster if not specified. ### Request Example ```json { "source": "mySourceCluster", "destination": "myDestinationCluster", "buildIndexes": "afterDataCopy" } ``` ### Response #### Success Response (200) (Response details not provided in source text) #### Response Example (Response example not provided in source text) ``` -------------------------------- ### Start mongosync copying all databases and collections Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Use this to copy all databases and their collections in their natural order. Specify "." for the database to indicate all databases. ```shell curl -X POST "http://localhost:27182/api/v1/start" --data ' { "source": "cluster0", "destination": "cluster1", "copyInNaturalOrder": [ { "database": ".", }, ] }' ``` -------------------------------- ### Start Synchronization Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Starts the synchronization between a source and destination cluster. Requires mongosync to be in the IDLE state and appropriate user permissions. ```APIDOC ## POST /api/v1/start ### Description Starts the synchronization between a source and destination cluster. ### Method POST ### Endpoint /api/v1/start ### Requirements - **State**: mongosync must be in the `IDLE` state. - **Permissions**: The user specified in the mongosync connection string must have the required permissions on the source and destination clusters. - **Multiple Instances**: When configuring multiple mongosync instances, send identical API endpoint commands to each instance. ``` -------------------------------- ### Start Data Synchronization with mongosync Source: https://www.mongodb.com/docs/mongosync/current/quickstart Initiates data synchronization between a source and destination cluster. Use this command to begin the migration process. ```shell curl localhost:27182/api/v1/start -XPOST \ --data \ { "source": "cluster0", "destination": "cluster1" } ``` -------------------------------- ### Formatted mongosync log output example Source: https://www.mongodb.com/docs/mongosync/current/reference/logging Example of mongosync log messages formatted using `jq`, showing structured JSON output. ```json { "level": "info", "mongosyncID": "shard02", "verbosity": "INFO", "id": "shard02", "port":27301, "time": "2022-06-21T11:15:33-04:00", "message": "Mongosync Options" } { "level": "info", "mongosyncID": "shard02", "time": "2022-06-21T11:15:33-04:00", "message": "Initialized client0 with URI: mongodb://192.0.2.1:27130 and client1 with URI: mongodb://192.0.2.2:27140." } { "level": "info", "mongosyncID": "shard02", "time": "2022-06-21T11:15:33-04:00", "message": "Preflight checks completed." } { "level": "info", "mongosyncID": "shard02", "time": "2022-06-21T11:15:33-04:00", "message": "Launch replication thread" } ``` -------------------------------- ### Constructing Connection String for On-Premises Cluster Source: https://www.mongodb.com/docs/mongosync/current/connecting/onprem-to-atlas Example of creating a connection string for a self-managed cluster named 'cluster0' using administrative credentials. ```text cluster0: mongodb://clusterAdmin:superSecret@clusterOne01.fancyCorp.com:20020,clusterOne02.fancyCorp.com:20020,clusterOne03.fancyCorp.com:20020 ``` -------------------------------- ### Start Synchronization Session Source: https://www.mongodb.com/docs/mongosync/current/reference/api Starts the synchronization session between a source and destination cluster. Returns either an updated status of the synchronization session or an error. ```APIDOC ## POST /start ### Description Starts the synchronization session between a source and destination cluster. ### Method POST ### Endpoint /start ### Response #### Success Response (200) - status (object) - The current status of the synchronization session. #### Error Response (4xx/5xx) - error (object) - Details about the error that occurred. ``` -------------------------------- ### Start a Reversible Sync Job Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Starts a sync job that can be reversed. Use this when you might need to change the direction of the sync later. Refer to the `reverse` API for details on reversing the sync. ```shell curl localhost:27182/api/v1/start -XPOST \ --data \ { "source": "cluster0", "destination": "cluster1", "reversible": true } ``` ```json {"success":true} ``` -------------------------------- ### Start Synchronization Source: https://www.mongodb.com/docs/mongosync/current/topologies/rs-to-sharded Initiate data synchronization from a replica set to a sharded cluster. Configure sharding options to shard collections on the destination cluster. ```APIDOC ## Start Synchronization Call the [start](https://mongodbcom-cdn.staging.corp.mongodb.com/docs/mongosync/reference/api/start/#std-label-c2c-api-start) endpoint to initiate data synchronization. To sync from a replica set to a sharded cluster, set the `sharding` option for the `start` command to shard collections on the destination cluster. For more information, see [Sharding Parameters](https://mongodbcom-cdn.staging.corp.mongodb.com/docs/mongosync/reference/api/start/#std-label-c2c-api-start-sharding). Use the `sharding.shardingEntries` parameter to specify the collections to shard. Collections that you do not list in this array replicate as unsharded. For more information, see [Shard Replica Sets](https://mongodbcom-cdn.staging.corp.mongodb.com/docs/mongosync/reference/api/start/#std-label-c2c-shard-replica-sets) and [Choose a Shard Key](https://www.mongodb.com/docs/manual/core/sharding-choose-a-shard-key/#std-label-sharding-shard-key-selection). The following example starts a sync from a replica set to a sharded cluster: ### Request ```curl localhost:27182/api/v1/start -XPOST \ --data \ '{ "source": "cluster0", "destination": "cluster1", "sharding": { "createSupportingIndexes": true, "shardingEntries": [ { "database": "accounts", "collection": "us_east", "shardCollection": { "key": [ { "location": 1 }, { "region": 1 } ] } } ] } }' ``` ### Response ```json {"success":true} ``` ``` -------------------------------- ### Start Sync from Replica Set to Sharded Cluster Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Starts a sync job from a source replica set to a destination sharded cluster. The `sharding` option configures how collections are sharded, including the shard key definition. Collections not listed in `shardingEntries` will replicate as unsharded. ```shell curl localhost:27182/api/v1/start -XPOST \ --data ' { "source": "cluster0", "destination": "cluster1", "sharding": { "createSupportingIndexes": true, "shardingEntries": [ { "database": "accounts", "collection": "us_east", "shardCollection": { "key": [ { "location": 1 }, { "region": 1 } ] } } ] } } ' ``` ```json {"success":true} ``` -------------------------------- ### Run mongosync with default stdout logging Source: https://www.mongodb.com/docs/mongosync/current/reference/logging Start mongosync with a configuration file to output log messages to standard output by default. ```shell $ mongosync --config /etc/mongosync.conf ``` -------------------------------- ### Start mongosync with specific databases and collections Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Use this to copy specific collections from specified databases. Ensure the source and destination clusters are correctly identified. ```shell curl -X POST "http://localhost:27182/api/v1/start" --data ' { "source": "cluster0", "destination": "cluster1", "copyInNaturalOrder": [ { "database": "sales", "collections": [ "accounts", "orders", ] }, { "database": "marketing", "collections": [ "offers", ] }, ] }' ``` -------------------------------- ### Start Synchronization for a mongosync Instance Source: https://www.mongodb.com/docs/mongosync/current/topologies/multiple-mongosyncs Use curl to send a POST request to the mongosync API to start the synchronization process for a specific instance. The 'source' and 'destination' parameters must match the cluster names used in the mongosync command. ```shell curl mongosync01Host:27601/api/v1/start -XPOST --data \ '{ "source": "cluster0", "destination": "cluster1", "reversible": false, "enableUserWriteBlocking": "none" }' ``` ```shell curl mongosync02Host:27602/api/v1/start -XPOST --data \ '{ "source": "cluster0", "destination": "cluster1", "reversible": false, "enableUserWriteBlocking": "none" }' ``` -------------------------------- ### Start a Sync Job Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Initiates a standard data synchronization job between a source and destination cluster. ```APIDOC ## POST /api/v1/start ### Description Initiates a data synchronization job between a source and destination cluster. ### Method POST ### Endpoint /api/v1/start ### Parameters #### Request Body - **source** (string) - Required - The identifier of the source cluster. - **destination** (string) - Required - The identifier of the destination cluster. ### Request Example ```shell curl localhost:27182/api/v1/start -XPOST \ --data ' { "source": "cluster0", "destination": "cluster1" } ' ``` ### Response #### Success Response (200) - **success** (boolean) - Indicates if the request was successful. #### Response Example ```json {"success":true} ``` ``` -------------------------------- ### Mongosync Filter Syntax Example Source: https://www.mongodb.com/docs/mongosync/current/reference/collection-level-filtering This JSON structure defines the syntax for `includeNamespaces` and `excludeNamespaces` fields used in mongosync's `start` API for filtered sync. It shows how to specify databases and collections individually or using regular expressions. ```json "includeNamespaces": [ { "database": "", "collections": [ "" ] "databaseRegex": { "pattern": "", "options": "" }, "collectionsRegex": { "pattern": "", "options": "" } } ], "excludeNamespaces": [ { "database": "", "collections": [ "" ] "databaseRegex": { "pattern": "", "options": "" }, "collectionsRegex": { "pattern": "", "options": "" } } ] ``` -------------------------------- ### Start a mongosync Instance for a Shard Source: https://www.mongodb.com/docs/mongosync/current/topologies/multiple-mongosyncs Launch a mongosync instance, specifying the source and destination cluster connection strings, a unique ID for the shard it will replicate, and the port it will use. Ensure the --id matches the shard identifier. ```bash mongosync \ --cluster0 "mongodb://user:password@cluster0host:27500" \ --cluster1 "mongodb://user:password@cluster1host:27500" \ --id shard01 --port 27601 ``` ```bash mongosync \ --cluster0 "mongodb://user:password@cluster0host:27500" \ --cluster1 "mongodb://user:password@cluster1host:27500" \ --id shard02 --port 27602 ``` -------------------------------- ### Initialize mongosync Process Source: https://www.mongodb.com/docs/mongosync/current/reference/verification/embedded Run this command to initialize the mongosync process, specifying log path and cluster connection strings. ```shell ./bin/mongosync \ --logPath /var/log/mongosync \ --cluster0 "mongodb://clusterAdmin:superSecret@clusterOne01.fancyCorp.com:20020,clusterOne02.fancyCorp.com:20020,clusterOne03.fancyCorp.com:20020" \ --cluster1 "mongodb://clusterAdmin:superSecret@clusterTwo01.fancyCorp.com:20020,clusterTwo02.fancyCorp.com:20020,clusterTwo03.fancyCorp.com:20020" ``` -------------------------------- ### mongosync stdout log output example Source: https://www.mongodb.com/docs/mongosync/current/reference/logging Example of structured JSON log messages outputted to stdout by mongosync. ```json {"level":"info","mongosyncID":"shard02","verbosity":"INFO","id":"shard02","port":27301,"time":"2022-06-21T11:15:33-04:00","message":"Mongosync Options"} {"level":"info","mongosyncID":"shard02","time":"2022-06-21T11:15:33-04:00","message":"Initialized client0 with URI: mongodb://192.0.2.1:27130 and client1 with URI: mongodb://192.0.2.2:27140."} {"level":"info","mongosyncID":"shard02","time":"2022-06-21T11:15:33-04:00","message":"Preflight checks completed."} {"level":"info","mongosyncID":"shard02","time":"2022-06-21T11:15:33-04:00","message":"Launch replication thread" ``` -------------------------------- ### Print Mongosync Usage Information Source: https://www.mongodb.com/docs/mongosync/current/reference/mongosync-binary Use the `--help` or `-h` flags to display usage information for the Mongosync binary. ```bash mongosync --help ``` ```bash mongosync -h ``` -------------------------------- ### Initialize mongosync with Cluster Connections Source: https://www.mongodb.com/docs/mongosync/current/quickstart This command initializes mongosync by connecting to the source and destination clusters using their respective connection strings. Ensure the log path is correctly set. ```shell ./bin/mongosync \ --logPath /var/log/mongosync \ --cluster0 "mongodb://clusterAdmin:superSecret@clusterOne01.fancyCorp.com:20020,clusterOne02.fancyCorp.com:20020,clusterOne03.fancyCorp.com:20020" \ --cluster1 "mongodb://clusterAdmin:superSecret@clusterTwo01.fancyCorp.com:20020,clusterTwo02.fancyCorp.com:20020,clusterTwo03.fancyCorp.com:20020" ``` -------------------------------- ### Get Oplog Window with db.getReplicationInfo() Source: https://www.mongodb.com/docs/mongosync/current/reference/oplog-sizing Run this command on the source cluster to determine the difference in seconds between the first and last entry in the oplog. This value represents the minimum oplog window. For sharded clusters, run on each shard and use the smallest number. ```javascript db.getReplicationInfo().timeDiff ``` -------------------------------- ### mongosync structured JSON log format example Source: https://www.mongodb.com/docs/mongosync/current/reference/logging An example of a single structured JSON log message from mongosync, detailing an event with key-value pairs. ```json { "level": "info", "mongosyncID": "shard01", "componentName": "Change Event Application", "time": "2022-06-21T09:31:42-04:00", "message": "Starting change stream reader." } ``` -------------------------------- ### Start Data Synchronization Source: https://www.mongodb.com/docs/mongosync/current/quickstart Initiates data synchronization between a source and destination cluster. This endpoint starts the sync process and applies subsequent writes from the source to the destination. ```APIDOC ## POST /api/v1/start ### Description Starts the synchronization between a source and destination cluster. ### Method POST ### Endpoint /api/v1/start ### Parameters #### Request Body - **source** (string) - Required - The identifier for the source cluster. - **destination** (string) - Required - The identifier for the destination cluster. ### Request Example ```json { "source": "cluster0", "destination": "cluster1" } ``` ### Response #### Success Response (200) - **success** (boolean) - Indicates if the operation was successful. ``` -------------------------------- ### Initialize mongosync with a Configuration File Source: https://www.mongodb.com/docs/mongosync/current/reference/configuration Use this command to initialize mongosync when you have updated a configuration file with new settings. Ensure the `--config` option points to the correct path of your modified configuration file. ```shell mongosync --config ``` -------------------------------- ### Get Synchronization Status Source: https://www.mongodb.com/docs/mongosync/current/reference/api/progress Use this command to retrieve the current status of the mongosync synchronization process. The API does not auto-refresh, so call this endpoint again to get updated status. ```shell curl localhost:27018/api/v1/progress -XGET ``` -------------------------------- ### Initialize mongosync with Updated Command-Line Options Source: https://www.mongodb.com/docs/mongosync/current/reference/configuration Initialize mongosync using updated command-line options if you did not use a configuration file previously. This command allows direct specification of connection strings and other parameters like `--loadLevel`. ```shell mongosync \ --cluster0 "" \ --cluster1 "" \ --loadLevel ``` -------------------------------- ### Start with the Verifier Disabled Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Starts a sync job with the embedded verifier explicitly disabled by setting `verification.enabled` to `false`. This is useful for scenarios where custom verification methods are preferred. ```APIDOC ## POST /api/v1/start ### Description Initiates a sync job while disabling the embedded verifier. ### Method POST ### Endpoint /api/v1/start ### Parameters #### Request Body - **source** (string) - Required - The source cluster identifier. - **destination** (string) - Required - The destination cluster identifier. - **verification** (object) - Optional - Configuration for the embedded verifier. - **enabled** (boolean) - Required - Set to `false` to disable the verifier. ### Request Example ```json { "source": "cluster0", "destination": "cluster1", "verification": { "enabled": false } } ``` ### Response #### Success Response (200) - **success** (boolean) - Indicates if the operation was successful. ``` -------------------------------- ### Start Data Sync with API Source: https://www.mongodb.com/docs/mongosync/current/reference/verification/embedded Use the /start API endpoint to initiate data synchronization from the source cluster to the destination cluster. This command assumes mongosync is running on localhost:27182. ```shell curl localhost:27182/api/v1/start -XPOST \ --data ' { "source": "cluster0", "destination": "cluster1" } ' ``` -------------------------------- ### Start with the Verifier Disabled Source: https://www.mongodb.com/docs/mongosync/current/reference/api/start Starts a sync job with the embedded verifier disabled. Set `verification.enabled` to `false` to disable the verifier. This is useful if you plan to use an alternative method for verifying sync. ```shell curl localhost:27182/api/v1/start -XPOST \ --data ' { "source": "cluster0", "destination": "cluster1", "verification": { "enabled": false } } ' ``` ```json {"success":true} ``` -------------------------------- ### mongosync Warning Example Source: https://www.mongodb.com/docs/mongosync/current/reference/api/progress This JSON snippet shows an example of a warning message that mongosync might generate, specifically related to insufficient oplog size on the source cluster. It includes a URL for more detailed information. ```json "warnings": [ "The amount of available oplog on the source cluster is too small for mongosync to complete successfully. For more details, see https://www.mongodb.com/docs/cluster-to-cluster-sync/current/reference/oplog-sizing/." ] ``` -------------------------------- ### Start Verification Checks Source: https://www.mongodb.com/docs/mongosync/current/reference/verification/verifier Initiate verification checks by sending a POST request to the /api/v1/check endpoint of the Migration Verifier. An empty JSON object is sent as the payload. ```bash curl -H "Content-Type: application/json" \ -X POST -d '{}' http://127.0.0.1:27020/api/v1/check ```