### Run Cruise Control with Gradle and start script Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/README.md Build the project and run the start script with optional JAR paths and port. ```bash ./gradlew jar copyDependantLibs ``` ```bash ./kafka-cruise-control-start.sh [-jars PATH_TO_YOUR_JAR_1,PATH_TO_YOUR_JAR_2] config/cruisecontrol.properties [port] ``` -------------------------------- ### Install cruise-control-client with pip Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/Getting-Started Installs the cruise-control-client package from PyPI. After installation, the cccli command is available. ```bash pip install cruise-control-client ``` -------------------------------- ### Query partition resource utilization with GET /kafkacruisecontrol/partition_load Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/REST-APIs Retrieves partition load sorted by the utilization of a specified resource. Supported parameters include resource, start, end, entries, json, allow_capacity_estimation, max_load, avg_load, topic, partition, min_valid_partition_ratio, brokerid, doAs, and reason. Defaults: resource=DISK, start=earliest monitored time, end=current time, entries=all partitions. ```http GET /kafkacruisecontrol/partition_load ``` -------------------------------- ### Create and activate a virtual environment Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/Getting-Started Creates a new virtual environment in the current directory and activates it. This is the recommended first step before installing cruise-control-client. ```bash python3.7 -m venv . . bin/activate ``` -------------------------------- ### POST /kafkacruisecontrol/rebootstrap Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/REST-APIs Trigger a workload balance. Starts a workload balance operation. ```APIDOC ## POST /kafkacruisecontrol/rebootstrap ### Description Trigger a workload balance. ### Method POST ### Endpoint /kafkacruisecontrol/rebootstrap ### Response #### Success Response (200) Starts the workload balance operation. ``` -------------------------------- ### Response from topic_configuration endpoint Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/Change-topic-replication-factor-through-Cruise-Control Example response showing the optimization moves and cluster load after updating replication factor. The response includes a full list of topics changed. ```text Optimization has 64 inter-broker replica(1216 MB) moves, 0 intra-broker replica(0 MB) moves and 83 leadership moves with a cluster model of 168 recent windows and 100.000% of the partitions covered. ... Cluster load after updating replication factor of topics [__KafkaCruiseControlPartitionMetricSamples] to 4 ... ``` -------------------------------- ### GET /kafkacruisecontrol/proposals Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/REST-APIs Get optimization proposals. Returns optimization proposals for the cluster. ```APIDOC ## GET /kafkacruisecontrol/proposals ### Description Get optimization proposals. ### Method GET ### Endpoint /kafkacruisecontrol/proposals ### Response #### Success Response (200) Returns optimization proposals. ``` -------------------------------- ### Query cluster load with GET /kafkacruisecontrol/load Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/REST-APIs HTTP GET request to query the current cluster load from Cruise Control. Requires the Load Monitor to be in the RUNNING state. Supports parameters such as start, end, time, allow_capacity_estimation, populate_disk_info, capacity_only, json, verbose, doAs, and reason. ```http GET /kafkacruisecontrol/load ``` -------------------------------- ### GET /kafkacruisecontrol/partition_load Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/REST-APIs Query partition resource utilization. Returns resource utilization for partitions. ```APIDOC ## GET /kafkacruisecontrol/partition_load ### Description Query partition resource utilization. ### Method GET ### Endpoint /kafkacruisecontrol/partition_load ### Response #### Success Response (200) Returns partition resource utilization. ``` -------------------------------- ### GET /kafkacruisecontrol/partition_state Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/REST-APIs Query partition and replica state. Returns the state of partitions and replicas. ```APIDOC ## GET /kafkacruisecontrol/partition_state ### Description Query partition and replica state. ### Method GET ### Endpoint /kafkacruisecontrol/partition_state ### Response #### Success Response (200) Returns partition and replica state. ``` -------------------------------- ### Get cluster load with cccli load Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/docs/wiki/Python Client/cccli-Command-Line-Usage.md Use the `load` command to retrieve the current cluster load from the Cruise Control REST API. The command starts a long-running poll and returns a JSON response with host and broker metrics. The sample response shows the output format with version, hosts, and brokers sections. ```bash cccli -a someCruiseControlAddress:9090 load ``` ```text Starting long-running poll of http://someCruiseControlAddress:9090/kafkacruisecontrol/load?allow_capacity_estimation=False&json=true 'version': 1 'hosts': CpuPct DiskMB DiskPct FollowerNwInRate LeaderNwInRate Leaders NwOutRate PnwOutRate Replicas Host some-kafka-host-1425 16.08 420,937.78 2.21 346.75 273.61 4578 563.29 1,681.31 18730 some-kafka-host-1427 12.88 433,809.72 2.27 163.00 104.87 9412 677.65 1,026.96 16805 some-kafka-host-1429 19.75 428,663.50 2.25 423.13 295.61 8983 790.29 2,010.03 19177 some-kafka-host-1433 20.22 414,232.50 2.17 102.44 155.51 11495 672.79 1,322.92 17532 some-kafka-host-2187 11.71 629,250.62 2.20 354.76 230.98 6765 714.39 1,825.22 18710 some-kafka-host-2192 12.34 623,767.00 2.18 280.24 187.04 10431 353.13 1,154.15 19190 some-kafka-host-2199 16.93 646,967.81 2.26 252.68 345.10 11215 1,124.97 1,822.09 18799 'brokers': BrokerState CpuPct DiskMB DiskPct FollowerNwInRate Host LeaderNwInRate Leaders NwOutRate PnwOutRate Replicas Broker 1425 ALIVE 16.08 420,937.78 2.21 346.75 some-kafka-host-1425 273.61 4578 563.29 1,681.31 18730 1427 ALIVE 12.88 433,809.72 2.27 163.00 some-kafka-host-1427 104.87 9412 677.65 1,026.96 16805 1429 ALIVE 19.75 428,663.50 2.25 423.13 some-kafka-host-1429 295.61 8983 790.29 2,010.03 19177 1433 ALIVE 20.22 414,232.50 2.17 102.44 some-kafka-host-1433 155.51 11495 672.79 1,322.92 17532 2187 ALIVE 11.71 629,250.62 2.20 354.76 some-kafka-host-2187 230.98 6765 714.39 1,825.22 18710 2192 ALIVE 12.34 623,767.00 2.18 280.24 some-kafka-host-2192 187.04 10431 353.13 1,154.15 19190 2199 ALIVE 16.93 646,967.81 2.26 252.68 some-kafka-host-2199 345.10 11215 1,124.97 1,822.09 18799 ``` -------------------------------- ### GET /kafkacruisecontrol/proposals Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/REST-APIs Returns optimization proposals generated based on the workload model of the given timestamp. Also returns workload summary before and after optimization. ```APIDOC ## GET /kafkacruisecontrol/proposals ### Description Returns optimization proposals generated based on the workload model of the given timestamp. Also returns workload summary before and after optimization. ### Method GET ### Endpoint /kafkacruisecontrol/proposals ### Parameters #### Query Parameters - **ignore_proposal_cache** (boolean) - Optional - Whether to ignore the cached proposal or not. Default: false - **data_from** (string) - Optional - Whether to calculate proposal from available valid partitions or valid windows. Default: `VALID_WINDOWS` - **goals** (list) - Optional - List of goals used to generate proposal. Default: default goals - **kafka_assigner** (boolean) - Optional - Whether to use Kafka assigner mode to generate proposals. Default: false - **allow_capacity_estimation** (boolean) - Optional - Whether to allow broker capacity to be estimated. Default: true - **excluded_topics** (regex) - Optional - Regular expression to specify topics excluded from replica and leadership movement. Default: null - **use_ready_default_goals** (boolean) - Optional - Whether to use only ready goals to generate proposals. Default: false - **exclude_recently_demoted_brokers** (boolean) - Optional - Whether to allow leader replicas to be moved to recently demoted brokers. Default: false - **exclude_recently_removed_brokers** (boolean) - Optional - Whether allow replicas to be moved to recently removed broker. Default: false - **destination_broker_ids** (list) - Optional - Specify brokers to move replicas to. Default: available brokers - **rebalance_disk** (boolean) - Optional - Whether to balance load between disks within brokers (requires JBOD Kafka deployment). Default: false - **json** (boolean) - Optional - Return in JSON format or not. Default: false - **verbose** (boolean) - Optional - Return detailed state information. Default: false - **doAs** (string) - Optional - Propagated user by the trusted proxy service. Default: null - **fast_mode** (boolean) - Optional - True to compute proposals in fast mode, false otherwise. Default: true - **reason** (string) - Optional - Reason for the request. Default: "No reason provided" ### Response #### Success Response (200) Returns optimization proposals and workload summary before and after optimization. Proposals can be generated based on **valid_window** or **valid_partitions**. ``` -------------------------------- ### Get optimization proposals Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/REST-APIs Returns optimization proposals generated based on the workload model of the given timestamp. The workload summary before and after the optimization will also be returned. Supported parameters include ignore_proposal_cache, data_from, goals, kafka_assigner, allow_capacity_estimation, excluded_topics, use_ready_default_goals, exclude_recently_demoted_brokers, exclude_recently_removed_brokers, destination_broker_ids, rebalance_disk, json, verbose, doAs, fast_mode, and reason. ```http GET /kafkacruisecontrol/proposals ``` -------------------------------- ### curl POST to topic_configuration endpoint Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/Change-topic-replication-factor-through-Cruise-Control Example request to change replication factor for a specific topic using query parameters. The topic parameter is treated as a regular expression. ```bash curl -X POST -c cookie "localhost:2540/kafkacruisecontrol/topic_configuration?topic=__KafkaCruiseControlPartitionMetricSamples&replication_factor=4" ``` -------------------------------- ### Add-broker with cruise-control-client (Python) and equivalent cccli command Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/cruise-control-client-Usage-in-Python-Applications Starts a broker addition on a cruise-control instance. The cccli command presumes allow_capacity_estimation=False, json=True, and dryrun=False. The Python code uses AddBrokerEndpoint with a list of broker IDs and CruiseControlResponder to send a POST request to /kafkacruisecontrol/add_broker. ```bash cccli -a someCruiseControlAddress:9090 add-broker broker,ids,to,add # Note that cccli presumes: # allow_capacity_estimation=False # json=True # dryrun=False ``` ```python from cruisecontrolclient.client.Endpoint import AddBrokerEndpoint from cruisecontrolclient.client.Responder import CruiseControlResponder # 1) Generate or define the socket address for the desired cruise-control instance cc_socket_address = 'someCruiseControlAddress:9090' # 2) Select which endpoint and parameters to use endpoint = AddBrokerEndpoint(['broker', 'ids', 'to', 'add']) endpoint.add_param(parameter_name="allow_capacity_estimation", value=False) endpoint.add_param(parameter_name="json", value=True) # 3) Instantiate a Responder json_responder = CruiseControlResponder() # 4) Start a long-running poll to retrieve a Requests.Response object response = json_responder.retrieve_response_from_Endpoint(cc_socket_address, endpoint) # 5) Process the response, likely by JSONifying it json_response = response.json() ``` -------------------------------- ### Rebalance with cruise-control-client (Python) and equivalent cccli command Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/cruise-control-client-Usage-in-Python-Applications Starts a rebalance on a cruise-control instance. The cccli command presumes allow_capacity_estimation=False, json=True, and dryrun=False. The Python code uses RebalanceEndpoint and CruiseControlResponder to send a POST request to /kafkacruisecontrol/rebalance. ```bash cccli -a someCruiseControlAddress:9090 rebalance # Note that cccli presumes: # allow_capacity_estimation=False # json=True # dryrun=False ``` ```python from cruisecontrolclient.client.Endpoint import RebalanceEndpoint from cruisecontrolclient.client.Responder import CruiseControlResponder # 1) Generate or define the socket address for the desired cruise-control instance cc_socket_address = 'someCruiseControlAddress:9090' # 2) Select which endpoint and parameters to use endpoint = RebalanceEndpoint() endpoint.add_param(parameter_name="allow_capacity_estimation", value=False) endpoint.add_param(parameter_name="json", value=True) # 3) Instantiate a Responder json_responder = CruiseControlResponder() # 4) Start a long-running poll to retrieve a Requests.Response object response = json_responder.retrieve_response_from_Endpoint(cc_socket_address, endpoint) # 5) Process the response, likely by JSONifying it json_response = response.json() ``` -------------------------------- ### Set JMX port for Cruise Control Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/README.md Export the JMX_PORT environment variable to emit JMX metrics on a specific port before starting Cruise Control. ```bash export JMX_PORT=56666 ``` -------------------------------- ### Execute approved POST request with review_id Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/2-step-verification-for-POST-requests Executes an approved POST request by sending a request to the reviewed endpoint with the approved review_id. For example, a rebalance request with reviewId=42 is executed via this command. The response is an OptimizationResult or progress response for async endpoints. ```http POST /kafkacruisecontrol/rebalance?review_id=42 ``` -------------------------------- ### Remove disks from brokers with remove_disks endpoint Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/docs/wiki/User Guide/REST-APIs.md The first request shows the general format for the remove_disks endpoint, which moves all replicas from specified disks to remaining disks on the same broker. The second example removes two disks from broker 101 with dryrun set to false. The brokerid_and_logdirs parameter uses commas between pairs and hyphens between broker id and logdir name. ```http POST /kafkacruisecontrol/remove_disks?brokerid_and_logdirs=[id1-logdir1,id1-logdir2...] ``` ```http POST /kafkacruisecontrol/remove_disks?brokerid_and_logdirs=101-/tmp/kafka-logs-1,101-/tmp/kafka-logs-2&dryrun=false ``` -------------------------------- ### Get all user tasks with cccli Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/docs/wiki/Python Client/cccli-Command-Line-Usage.md Retrieves all user tasks from Cruise Control. The -a flag specifies the Cruise Control address and port. The command polls the /kafkacruisecontrol/user_tasks endpoint and returns a list of tasks with their status and request URLs. ```bash cccli -a someCruiseControlAddress:9090 user-tasks ``` -------------------------------- ### GET /kafkacruisecontrol/partition_load Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/REST-APIs This endpoint returns partition load sorted by utilization of a specified resource. It supports filtering by topic, partition, and broker ID, and can report maximum or average load values. ```APIDOC ## GET /kafkacruisecontrol/partition_load ### Description This endpoint returns the partition load sorted by the utilization of a given resource. It supports filtering by topic, partition, and broker ID, and can report maximum or average load values. ### Method GET ### Endpoint /kafkacruisecontrol/partition_load ### Parameters #### Query Parameters - **resource** (string) - Optional - Resource type to sort partition load. Available resources are `DISK`, `CPU`, `NW_IN`, `NW_OUT`. Default: `DISK`. - **start** (long) - Optional - Timestamp in milliseconds of the earliest metric sample used to generate load. Default: time of earliest valid window. - **end** (long) - Optional - Timestamp in milliseconds of the latest metric sample used to generate load. Default: current system time. - **entries** (integer) - Optional - Number of partition load entries to report in response. Default: `MAX_INT`. - **json** (boolean) - Optional - Return in JSON format or not. Default: false. - **allow_capacity_estimation** (boolean) - Optional - Whether to allow capacity estimation when cruise-control is unable to obtain all per-broker capacity information. Default: true. - **max_load** (boolean) - Optional - Whether to report the max load for partition in windows. Default: false. - **avg_load** (boolean) - Optional - Whether to report the average load for partition in windows. Default: false. - **topic** (regex) - Optional - Regular expression to filter partition load to report based on partition's topic. Default: null. - **partition** (integer/range) - Optional - Partition number (e.g. 10) or range (e.g. 1-10) to filter partition load to report. Default: null. - **min_valid_partition_ratio** (double) - Optional - Minimal valid partition ratio requirement for cluster model. Default: null. - **brokerid** (int) - Optional - Broker id to filter partition load to report. Default: null. - **doAs** (string) - Optional - Propagated user by the trusted proxy service. Default: null. - **reason** (string) - Optional - Reason for the request. Default: "No reason provided". ### Response #### Success Response (200) Returns a partition list sorted by the utilization of the specified resource in the time range specified by `start` and `end`. The resource can be `CPU`, `NW_IN`, `NW_OUT`, and `DISK`. By default, `start` is the earliest monitored time, `end` is current wall clock time, `resource` is `DISK`, and `entries` is all partitions in the cluster. If `max_load` and `avg_load` are both not specified or false, for `DISK` resource, latest value is reported; for `NW_IN`/`NW_OUT`/`CPU` resource, average value is reported. ``` -------------------------------- ### Remove-broker with cruise-control-client (Python) and equivalent cccli command Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/cruise-control-client-Usage-in-Python-Applications Starts a broker removal on a cruise-control instance. The cccli command presumes allow_capacity_estimation=False, json=True, and dryrun=False. The Python code uses RemoveBrokerEndpoint with a list of broker IDs and CruiseControlResponder to send a POST request to /kafkacruisecontrol/remove_broker. ```bash cccli -a someCruiseControlAddress:9090 remove-broker broker,ids,to,remove # Note that cccli presumes: # allow_capacity_estimation=False # json=True # dryrun=False ``` ```python from cruisecontrolclient.client.Endpoint import RemoveBrokerEndpoint from cruisecontrolclient.client.Responder import CruiseControlResponder # 1) Generate or define the socket address for the desired cruise-control instance cc_socket_address = 'someCruiseControlAddress:9090' # 2) Select which endpoint and parameters to use endpoint = RemoveBrokerEndpoint(['broker', 'ids', 'to', 'remove']) endpoint.add_param(parameter_name="allow_capacity_estimation", value=False) endpoint.add_param(parameter_name="json", value=True) # 3) Instantiate a Responder json_responder = CruiseControlResponder() # 4) Start a long-running poll to retrieve a Requests.Response object response = json_responder.retrieve_response_from_Endpoint(cc_socket_address, endpoint) # 5) Process the response, likely by JSONifying it json_response = response.json() ``` -------------------------------- ### Build API Wiki with Gradle Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/cruise-control/src/main/resources/yaml/README.md Run this command from the base directory to generate the API wiki. Requires npm and redoc-cli (npm module) as prerequisites. The output is placed in target/api_wiki. ```bash ./gradlew buildApiWiki ``` -------------------------------- ### Sample response from cccli topic-configuration Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/docs/wiki/Python Client/cccli-Command-Line-Usage.md Sample output from the topic-configuration command, showing the long-running poll URL and the summary and goal summary sections. The summary includes metrics like dataToMoveMB, numLeaderMovements, and numReplicaMovements, while the goal summary shows per-goal metrics for CPU, disk, and network. ```text Starting long-running poll of http://someCruiseControlAddress:9090/kafkacruisecontrol/topic_configuration?skip_rack_awareness_check=False&replication_factor=3&topic=some-topic-regex&json=true 'summary': {'dataToMoveMB': 0, 'excludedBrokersForLeadership': [], 'excludedBrokersForReplicaMove': [], 'excludedTopics': [''], 'intraBrokerDataToMoveMB': 0, 'monitoredPartitionsPercentage': 98.95486235618591, 'numIntraBrokerReplicaMovements': 0, 'numLeaderMovements': 353, 'numReplicaMovements': 10, 'recentWindows': 73} 'goalSummary': RackAwareGoal: NO-ACTION AVG STD MIN MAX cpu 17.58 1.66 15.63 20.51 disk 841,522.70 29,585.51 787,726.56 874,522.56 leaderReplicas 3,061.80 635.85 2,576.00 4,290.00 networkInbound 5,907.75 585.13 5,170.29 6,600.34 networkOutbound 2,444.02 667.00 1,630.82 3,467.63 potentialNwOut 5,726.13 687.42 4,616.13 6,547.50 replicas 7,070.80 7.65 7,064.00 7,082.00 topicReplicas 7.09 0.05 0.00 410.00 ReplicaCapacityGoal: NO-ACTION AVG STD MIN MAX cpu 17.58 1.66 15.63 20.51 disk 841,522.70 29,585.51 787,726.56 874,522.56 leaderReplicas 3,061.80 635.85 2,576.00 4,290.00 networkInbound 5,907.75 585.13 5,170.29 6,600.34 networkOutbound 2,444.02 667.00 1,630.82 3,467.63 potentialNwOut 5,726.13 687.42 4,616.13 6,547.50 replicas 7,070.80 7.65 7,064.00 7,082.00 topicReplicas 7.09 0.05 0.00 410.00 DiskCapacityGoal: NO-ACTION AVG STD MIN MAX cpu 17.58 1.66 15.63 20.51 disk 841,522.70 29,585.51 787,726.56 874,522.56 leaderReplicas 3,061.80 635.85 2,576.00 4,290.00 networkInbound 5,907.75 585.13 5,170.29 6,600.34 networkOutbound 2,444.02 667.00 1,630.82 3,467.63 potentialNwOut 5,726.13 687.42 4,616.13 6,547.50 replicas 7,070.80 7.65 7,064.00 7,082.00 topicReplicas 7.09 0.05 0.00 410.00 NetworkInboundCapacityGoal: NO-ACTION AVG STD MIN MAX cpu 17.58 1.66 15.63 20.51 disk 841,522.70 29,585.51 787,726.56 874,522.56 leaderReplicas 3,061.80 635.85 2,576.00 4,290.00 networkInbound 5,907.75 585.13 5,170.29 6,600.34 networkOutbound 2,444.02 667.00 1,630.82 3,467.63 potentialNwOut 5,726.13 687.42 4,616.13 6,547.50 replicas 7,070.80 7.65 7,064.00 7,082.00 topicReplicas 7.09 0.05 0.00 410.00 NetworkOutboundCapacityGoal: NO-ACTION AVG STD MIN MAX cpu 17.58 1.66 15.63 20.51 disk 841,522.70 29,585.51 787,726.56 874,522.56 leaderReplicas 3,061.80 635.85 2,576.00 4,290.00 networkInbound 5,907.75 585.13 5,170.29 6,600.34 networkOutbound 2,444.02 667.00 1,630.82 3,467.63 potentialNwOut 5,726.13 687.42 4,616.13 6,547.50 replicas 7,070.80 7.65 7,064.00 7,082.00 topicReplicas 7.09 0.05 0.00 410.00 CpuCapacityGoal: NO-ACTION AVG STD MIN MAX cpu 17.58 1.66 15.63 20.51 disk 841,522.70 29,585.51 787,726.56 874,522.56 leaderReplicas 3,061.80 635.85 2,576.00 4,290.00 networkInbound 5,907.75 585.13 5,170.29 6,600.34 networkOutbound 2,444.02 667.00 1,630.82 3,467.63 potentialNwOut 5,726.13 687.42 4,616.13 6,547.50 replicas 7,070.80 7.65 7,064.00 7,082.00 topicReplicas 7.09 0.05 0.00 410.00 ReplicaDistributionGoal: NO-ACTION AVG STD MIN MAX cpu 17.58 1.66 15.63 20.51 disk 841,522.70 29,585.51 787,726.56 874,522.56 leaderReplicas 3,061.80 635.85 2,576.00 4,290.00 networkInbound 5,907.75 585.13 5,170.29 6,600.34 networkOutbound 2,444.02 667.00 1,630.82 3,467.63 potentialNwOut 5,726.13 687.42 4,616.13 6,547.50 replicas 7,070.80 7.65 7,064.00 7,082.00 topicReplicas 7.09 0.05 0.00 410.00 ``` -------------------------------- ### Download and extract a release (e.g., 0.1.10) Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/README.md Download a specific release tarball, extract it, and initialize a local git repository with a tag. ```bash wget https://github.com/linkedin/cruise-control/archive/0.1.10.tar.gz && tar zxvf 0.1.10.tar.gz && cd cruise-control-0.1.10/ ``` ```bash git init && git add . && git commit -m "Init local repo." && git tag -a 0.1.10 -m "Init local version." ``` -------------------------------- ### GET /kafkacruisecontrol/permissions Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/docs/wiki/User Guide/REST-APIs.md This snippet documents the GET endpoint for querying user permissions. ```APIDOC ## GET /kafkacruisecontrol/permissions ### Description Returns the currently authenticated user's permissions as a list of assigned user roles. The role list can include `"ADMIN"`, `"USER"`, and `"VIEWER"`. ### Method GET ### Endpoint /kafkacruisecontrol/permissions ### Parameters #### Query Parameters - **doAs** (string) - Optional - Propagated user by the trusted proxy service. Default: null - **json** (boolean) - Optional - Return in JSON format or not. Default: false - **get_response_schema** (boolean) - Optional - Return JSON schema in the response header or not. Default: false - **reason** (string) - Optional - Reason for the request. Default: "No reason provided" ### Response #### Success Response (200) Returns a list of user roles. #### Error Response (400) If no security is enabled in the Kafka Cluster, returns the message: `"Unable to retrieve privilege information for an unsecure connection"`. ``` -------------------------------- ### Sample response for add-broker dry-run Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/cccli-Command-Line-Usage Shows the output of the add-broker dry-run, including the polled URL, summary metrics, and goal summaries with NO-ACTION status. ```text Starting long-running poll of http://someCruiseControlAddress:9090/kafkacruisecontrol/add_broker?brokerid=1500&allow_capacity_estimation=False&dryrun=True&json=true 'summary': {'dataToMoveMB': 156, 'excludedBrokersForLeadership': [], 'excludedBrokersForReplicaMove': [], 'excludedTopics': [], 'intraBrokerDataToMoveMB': 0, 'monitoredPartitionsPercentage': 98.13196063041687, 'numIntraBrokerReplicaMovements': 0, 'numLeaderMovements': 0, 'numReplicaMovements': 8, 'recentWindows': 163} 'goalSummary': RackAwareGoal: NO-ACTION AVG STD MIN MAX cpu 17.67 1.71 15.82 19.75 disk 705,834.50 7,766.46 698,927.44 720,480.56 leaderReplicas 2,997.80 637.82 2,508.00 4,228.00 networkInbound 5,316.30 605.01 4,689.12 6,296.78 networkOutbound 1,935.27 441.51 1,184.77 2,519.02 potentialNwOut 4,478.07 323.65 3,994.05 4,795.87 replicas 6,876.80 7.86 6,869.00 6,889.00 topicReplicas 7.13 0.05 0.00 410.00 ReplicaCapacityGoal: NO-ACTION AVG STD MIN MAX cpu 17.67 1.71 15.82 19.75 disk 705,834.50 7,766.46 698,927.44 720,480.56 leaderReplicas 2,997.80 637.82 2,508.00 4,228.00 networkInbound 5,316.30 605.01 4,689.12 6,296.78 networkOutbound 1,935.27 441.51 1,184.77 2,519.02 potentialNwOut 4,478.07 323.65 3,994.05 4,795.87 replicas 6,876.80 7.86 6,869.00 6,889.00 topicReplicas 7.13 0.05 0.00 410.00 DiskCapacityGoal: NO-ACTION AVG STD MIN MAX cpu 17.67 1.71 15.82 19.75 disk 705,834.50 7,766.46 698,927.44 720,480.56 leaderReplicas 2,997.80 637.82 2,508.00 4,228.00 networkInbound 5,316.30 605.01 4,689.12 6,296.78 networkOutbound 1,935.27 441.51 1,184.77 2,519.02 potentialNwOut 4,478.07 323.65 3,994.05 4,795.87 replicas 6,876.80 7.86 6,869.00 6,889.00 topicReplicas 7.13 0.05 0.00 410.00 NetworkInboundCapacityGoal: NO-ACTION AVG STD MIN MAX cpu 17.67 1.71 15.82 19.75 disk 705,834.50 7,766.46 698,927.44 720,480.56 leaderReplicas 2,997.80 637.82 2,508.00 4,228.00 networkInbound 5,316.30 605.01 4,689.12 6,296.78 networkOutbound 1,935.27 441.51 1,184.77 2,519.02 potentialNwOut 4,478.07 323.65 3,994.05 4,795.87 replicas 6,876.80 7.86 6,869.00 6,889.00 topicReplicas 7.13 0.05 0.00 410.00 NetworkOutboundCapacityGoal: NO-ACTION AVG STD MIN MAX cpu 17.67 1.71 15.82 19.75 disk 705,834.50 7,766.46 698,927.44 720,480.56 leaderReplicas 2,997.80 637.82 2,508.00 4,228.00 networkInbound 5,316.30 605.01 4,689.12 6,296.78 networkOutbound 1,935.27 441.51 1,184.77 2,519.02 potentialNwOut 4,478.07 323.65 3,994.05 4,795.87 replicas 6,876.80 7.86 6,869.00 6,889.00 topicReplicas 7.13 0.05 0.00 410.00 ``` -------------------------------- ### GET /kafkacruisecontrol/cluster_load Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/docs/wiki/User Guide/REST-APIs.md Query the current cluster load. ```APIDOC ## GET /kafkacruisecontrol/cluster_load ### Description Query the current cluster load. ### Method GET ### Endpoint /kafkacruisecontrol/cluster_load ``` -------------------------------- ### Sample response for cccli rebalance --dry-run Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/docs/wiki/Python Client/cccli-Command-Line-Usage.md This is the sample response from the rebalance command, showing the progress of the rebalance operation and the summary of the planned changes. It includes the operation steps, completion percentages, and the summary with data to move and replica movements. ```text Starting long-running poll of http://someCruiseControlAddress:9090/kafkacruisecontrol/rebalance?allow_capacity_estimation=False&dryrun=True&json=true 'version': 1 'progress': operation: Rebalance completionPercentage description time-in-ms step WAITING_FOR_CLUSTER_MODEL 100.00 The job requires a cluster model and it is waiting to get the cluster model lock. 65 AGGREGATING_METRICS 100.00 Retrieve the metrics of all the partitions from the aggregated metrics. 7813 GENERATING_CLUSTER_MODEL 0.00 Generating the cluster model for the cluster. 1732 ......... 'version': 1 'progress': operation: Rebalance completionPercentage description time-in-ms step WAITING_FOR_CLUSTER_MODEL 100.00 The job requires a cluster model and it is waiting to get the cluster model lock. 65 AGGREGATING_METRICS 100.00 Retrieve the metrics of all the partitions from the aggregated metrics. 7813 GENERATING_CLUSTER_MODEL 0.00 Generating the cluster model for the cluster. 5451 OPTIMIZING RackAwareGoal 100.00 Optimizing goal RackAwareGoal 460 OPTIMIZING ReplicaCapacityGoal 100.00 Optimizing goal ReplicaCapacityGoal 319 OPTIMIZING DiskCapacityGoal 100.00 Optimizing goal DiskCapacityGoal 186 OPTIMIZING NetworkInboundCapacityGoal 100.00 Optimizing goal NetworkInboundCapacityGoal 187 OPTIMIZING NetworkOutboundCapacityGoal 100.00 Optimizing goal NetworkOutboundCapacityGoal 284 OPTIMIZING CpuCapacityGoal 100.00 Optimizing goal CpuCapacityGoal 183 OPTIMIZING ReplicaDistributionGoal 100.00 Optimizing goal ReplicaDistributionGoal 672 OPTIMIZING PotentialNwOutGoal 100.00 Optimizing goal PotentialNwOutGoal 188 OPTIMIZING DiskUsageDistributionGoal 100.00 Optimizing goal DiskUsageDistributionGoal 455 OPTIMIZING NetworkInboundUsageDistributionGoal 100.00 Optimizing goal NetworkInboundUsageDistributionGoal 2219 OPTIMIZING NetworkOutboundUsageDistributionGoal 100.00 Optimizing goal NetworkOutboundUsageDistributionGoal 521 OPTIMIZING CpuUsageDistributionGoal 0.00 Optimizing goal CpuUsageDistributionGoal 712 ......... 'summary': {'dataToMoveMB': 87290, 'excludedBrokersForLeadership': [], 'excludedBrokersForReplicaMove': [], 'excludedTopics': ['__some-topic-we-excluded', '__some-other-topic-we-excluded'], 'intraBrokerDataToMoveMB': 0, 'monitoredPartitionsPercentage': 99.24935102462769, 'numIntraBrokerReplicaMovements': 0, 'numLeaderMovements': 501, 'numReplicaMovements': 85, 'recentWindows': 168} 'goalSummary': RackAwareGoal: NO-ACTION AVG STD MIN MAX cpu 0.16 0.03 0.12 0.20 disk 513,930.64 103,743.40 414,232.50 646,967.81 networkInbound 502.25 165.94 257.96 718.74 networkOutbound 699.50 216.86 353.13 1,124.97 potentialNwOut 1,548.96 350.61 1,026.96 2,010.03 replicas 18,420.43 835.51 16,805.00 19,190.00 topicReplicas 4.87 1.31 0.00 342.00 ``` -------------------------------- ### Build Cruise Control and copy metrics reporter jar Source: https://github.com/cruise-control-for-kafka/cruise-control/blob/main/README.md Build the project with Gradle and copy the metrics reporter jar to the Kafka server's dependency folder. ```bash ./gradlew jar ``` ```bash cp ./cruise-control-metrics-reporter/build/libs/cruise-control-metrics-reporter-A.B.C.jar ``` -------------------------------- ### View cccli help Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/Getting-Started Displays help for the cccli command-line interface, showing available endpoints and options. ```bash cccli --help ``` -------------------------------- ### GET /kafkacruisecontrol/review_board Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/2-step-verification-for-POST-requests Retrieves the requests in the Review Board, optionally filtered by review IDs. ```APIDOC ## [GET] /kafkacruisecontrol/review_board ### Description Returns the requests in the Review Board, optionally filtered by review IDs. If review_ids is not provided, all requests are returned. ### Method GET ### Endpoint /kafkacruisecontrol/review_board ### Parameters #### Query Parameters - **json** (boolean) - Optional - Whether to return the response in JSON format. - **review_ids** (list of IDs) - Optional - Comma-separated list of review IDs to filter the response. ### Request Example GET /kafkacruisecontrol/review_board?json=true&review_ids=1,2 ### Response #### Success Response (200) - Returns the list of requests in the Review Board with their details (ID, SUBMITTER_ADDRESS, SUBMISSION_TIME_MS, STATUS, ENDPOINT_WITH_PARAMS). #### Response Example { "requests": [ { "id": 1, "status": "PENDING_REVIEW" } ] } ``` -------------------------------- ### GET /kafkacruisecontrol/load Source: https://github.com/cruise-control-for-kafka/cruise-control/wiki/REST-APIs Query the current cluster load. Returns the current load of the Kafka cluster. ```APIDOC ## GET /kafkacruisecontrol/load ### Description Query the current cluster load. ### Method GET ### Endpoint /kafkacruisecontrol/load ### Response #### Success Response (200) Returns the current cluster load. ```