### Build pgremapper with Go Install Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Installs the pgremapper tool using the Go installation command. Requires a configured Go environment. ```bash go install github.com/digitalocean/pgremapper@latest ``` -------------------------------- ### pgremapper General Usage Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Demonstrates the general command-line usage of pgremapper, including global options for concurrency, applying changes, and verbose output. It also shows how commands are appended. ```bash ./pgremapper [--concurrency ] [--yes] [--verbose] ``` -------------------------------- ### Build pgremapper with Docker Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Builds the pgremapper tool using a Docker container, useful when a local Go environment is not available. It clones the repository and uses a specified Go version for building. ```bash docker run --rm -v $(pwd):/pgremapper -w /pgremapper golang:1.21.4 go build -o pgremapper . ``` -------------------------------- ### pgremapper balance-bucket Usage Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Shows how to use the `balance-bucket` command to rebalance PGs within a specified CRUSH bucket. It includes options for controlling backfills and device classes. ```bash ./pgremapper balance-bucket [--max-backfills ] [--target-spread ] ``` ```bash ./pgremapper balance-bucket data11 --max-backfills 10 --target-spread 3 ``` ```bash ./pgremapper balance-bucket data11 --device-class nvme ``` -------------------------------- ### Export OSD upmap configurations Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Exports upmap configurations for specified OSDs in JSON format, which can be used with the `import-mappings` command. This is useful for backing up or transferring mapping states during CRUSH changes. ```bash $ ./pgremapper export-mappings [ ...] [--output ] [--whole-pg] ``` -------------------------------- ### Import PG Mappings Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Imports PG upmap data from a JSON file or standard input into the cluster. This command applies the specified PG remappings. The JSON format includes 'pgid' and a 'mapping' object with 'from' and 'to' OSD IDs. ```json [ { "pgid": "1.1", "mapping": { "from": 100, "to": 42, } } ] ``` ```bash ./pgremapper import-mappings [] ``` -------------------------------- ### Undo Up-maps: Move PGs off a host after a swap-bucket Source: https://github.com/digitalocean/pgremapper/blob/main/README.md This command facilitates moving PGs off a host (e.g., data01) to its replacement (e.g., data04) after a bucket swap. It allows fine-grained control over backfill concurrency by setting maximum backfill reservations per source OSD or bucket. ```bash $ ./pgremapper undo-upmaps bucket:data01 --max-backfill-reservations 2,bucket:data04:3 --max-source-backfills 2 ``` -------------------------------- ### Generate CRUSH Change Mappings Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Generates PG upmaps in JSON format for CRUSHmap changes, allowing for pre-review and import. Useful for backward-incompatible changes like switching 'chooseleaf' types, preventing immediate rebalancing issues. Requires the CRUSHmap text and an output file path. ```bash ./pgremapper generate-crush-change-mappings --crushmap-text /tmp/crushmap.txt --output /tmp/upmap.json ``` -------------------------------- ### Export UpMap Mappings Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Exports upmap mappings for specified OSDs in a JSON format compatible with `import-mappings`. Useful for backing up or migrating CRUSH map states. ```APIDOC ## POST /pgremapper/export-mappings ### Description Exports upmap mappings for specified OSDs or OSD specifications. The output can be filtered to include only relevant portions of the mappings or the entire PG's mapping context. The exported JSON is suitable for use with the `import-mappings` command. ### Method POST ### Endpoint /pgremapper/export-mappings ### Parameters #### Path Parameters - ** ...** (string) - Required - One or more OSDs or OSD specs for which mappings will be exported (e.g., `osd.1`, `bucket:data04`). #### Query Parameters - **--output** (string) - Optional - Specifies a file path to write the output JSON to instead of standard output. - **--whole-pg** (boolean) - Optional - If set, exports all mappings for PGs that include the specified OSD(s), not just the portions directly involving those OSDs. ### Request Example ```json { "command": "pgremapper export-mappings osd.1 osd.5" } ``` ```json { "command": "pgremapper export-mappings bucket:data10 --output mappings.json --whole-pg" } ``` ### Response #### Success Response (200) - **mappings** (string) - A JSON string containing the exported upmap mappings. #### Response Example ```json { "mappings": "[{\"from_osd\": 1, \"to_osd\": 5, \"pgid\": \"0.12345678\"}, ...]" } ``` ``` -------------------------------- ### pgremapper cancel-backfill Usage Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Details the usage of the `cancel-backfill` command, which reverts PG backfills by adjusting the upmap exception table. It supports various flags to filter which backfills to cancel. ```bash ./pgremapper cancel-backfill [--exclude-backfilling] [--include-osds ,...] [--exclude-osds ,...] [--pgs-including ,...] ``` -------------------------------- ### Drain OSDs Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Remaps PGs off of specified source OSDs to target OSDs, with options to control the number of backfills and movement constraints. ```APIDOC ## POST /pgremapper/drain ### Description Remaps Placement Groups (PGs) off of specified source OSDs to target OSDs. This operation aims to move PGs with minimal impact by selecting less busy targets and PGs. It can also manage backfill reservations and concurrency. ### Method POST ### Endpoint /pgremapper/drain ### Parameters #### Path Parameters - **** (string) - Required - The OSD or OSD spec(s) from which PGs will be remapped (e.g., `osd.4`, `bucket:data12`). Multiple OSD specs can be provided. #### Query Parameters - **--target-osds** (string) - Required - Comma-separated list of OSDs or OSD specs that will serve as backfill targets (e.g., `osd.21`, `bucket:data12`). - **--allow-movement-across** (string) - Optional - Constrains data movements across CRUSH bucket types (e.g., `host`). If not provided, movements are only allowed within the source OSD's direct CRUSH bucket. - **--max-backfill-reservations** (string) - Optional - Sets maximum reservation limits for backfill operations. Can be a default value or per-OSD spec (e.g., `2`, `osd.4:1`). - **--max-source-backfills** (integer) - Optional - Specifies the maximum number of backfills allowed for the source OSD. For replicated systems, this controls source concurrency via `--max-backfill-reservations`. ### Request Example ```json { "command": "pgremapper drain 4 --target-osds 21 --max-source-backfills 5" } ``` ```json { "command": "pgremapper drain 15 --target-osds bucket:data12 --allow-movement-across host --max-backfill-reservations 2 --max-source-backfills 8" } ``` ### Response #### Success Response (200) - **status** (string) - Indicates the success of the drain operation. #### Response Example ```json { "status": "PG remapping initiated successfully." } ``` ``` -------------------------------- ### Undo Up-maps: Move PGs back after OSD recreate Source: https://github.com/digitalocean/pgremapper/blob/main/README.md This command is used to move placement groups (PGs) back to an OSD after it has been reformatted or recreated. It allows specifying the target OSD and optionally limits the number of concurrent backfills from source OSDs. ```bash $ ./pgremapper undo-upmaps 21 --max-source-backfills 5 ``` ```bash $ ./pgremapper undo-upmaps 4 --target --max-source-backfills 5 ``` -------------------------------- ### Cancel all backfill in the system for augment Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Disables backfill and rebalancing to control PG movement during system augmentations. It temporarily disables the PG autoscaler if active, performs CRUSH map changes, cancels all backfill operations, and then re-enables the autoscaler and balance settings. Finally, it suggests enabling the upmap balancer for gentle data movements. ```bash # Make sure no data movement occurs when manipulating the CRUSH map. $ ceph osd set nobackfill $ ceph osd set norebalance # If you are using PG autoscaler (Nautilus+), also disable it. $ ceph osd pool set noautoscale $ ./pgremapper cancel-backfill --yes $ ceph osd unset noautoscale $ ceph osd unset norebalance $ ceph osd unset nobackfill ``` -------------------------------- ### Cancel Backfill Operations Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Commands to cancel or modify ongoing backfill operations for PGs based on various criteria like source/target OSDs, specific PGs, or host inclusion/exclusion. ```APIDOC ## POST /pgremapper/cancel-backfill ### Description Cancels backfill operations based on specified criteria. This is particularly useful during system augmentations or when specific OSDs need to be excluded from backfill. ### Method POST ### Endpoint /pgremapper/cancel-backfill ### Parameters #### Query Parameters - **--yes** (boolean) - Required - If set, skips the confirmation prompt before canceling backfill. - **--include-osds** (string) - Optional - Comma-separated list of OSDs or OSD specs to include in the cancellation criteria (e.g., `osd.1`, `bucket:data04`). - **--exclude-osds** (string) - Optional - Comma-separated list of OSDs or OSD specs to exclude from the cancellation criteria (e.g., `osd.21`, `osd.34`). - **--pgs-including** (string) - Optional - Specifies to cancel backfill for PGs that include the given OSD spec (e.g., `bucket:data10`). ### Request Example ```json { "command": "pgremapper cancel-backfill --yes" } ``` ```json { "command": "pgremapper cancel-backfill --include-osds osd.4 --exclude-osds osd.21,osd.34" } ``` ```json { "command": "pgremapper cancel-backfill --pgs-including bucket:data10" } ``` ### Response #### Success Response (200) - **status** (string) - Indicates the success of the operation. #### Response Example ```json { "status": "Backfill cancellation initiated successfully." } ``` ``` -------------------------------- ### Cancel backfill including specific OSDs or buckets Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Cancels backfill operations involving specific OSDs or CRUSH buckets as sources or targets, with options to exclude certain OSDs from this cancellation. This is useful for reducing backfill load on particular hosts or OSDs. ```bash $ ./pgremapper cancel-backfill --include-osds bucket:data04 --exclude-osds 21,34 ``` ```bash $ ./pgremapper cancel-backfill --pgs-including bucket:data10 ``` -------------------------------- ### Remap PG Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Modifies the upmap exception table for a specific PG by adding or updating a mapping from a source OSD to a target OSD. This is a safer alternative to directly using 'ceph osd pg-upmap-items'. ```bash ./pgremapper remap ``` -------------------------------- ### Drain PGs from OSDs Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Remaps PGs from specified source OSDs to target OSDs, with options to limit the number of concurrent backfills and control movement across CRUSH buckets. This command is used to offload PGs from one or more OSDs. ```bash $ ./pgremapper drain [ ...] --target-osds [,] [--allow-movement-across ] [--max-backfill-reservations default_max[,osdspec:max]] [--max-source-backfills ] ``` ```bash # Schedule backfills to move 5 PGs from OSD 4 to OSD 21: $ ./pgremapper drain 4 --target-osds 21 --max-source-backfills 5 ``` ```bash # Move PGs off-host: Schedule backfills to move 8 PGs from OSD 15 to any combination of OSDs on host data12, ensuring we don't exceed 2 backfill reservations anywhere: $ ./pgremapper drain 15 --target-osds bucket:data12 --allow-movement-across host --max-backfill-reservations 2 --max-source-backfills 8 ``` -------------------------------- ### Undo Upmap Operations Source: https://github.com/digitalocean/pgremapper/blob/main/README.md Removes or modifies upmap items for a list of OSDs, allowing them to become sources or targets for backfill operations. This is useful for draining OSDs or achieving higher concurrency during migrations. It supports options to control backfill reservations and concurrency limits. ```bash ./pgremapper undo-upmaps [ ...] [--max-backfill-reservations default_max[,osdspec:max]] [--max-source-backfills ] [--target] ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.