### Nested Property Processor Backwards Compatible Addition Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/multilang/configuring-credential-providers.md Example of adding a new nested key processor method to a class implementing NestedPropertyProcessor. This method is designed to be backwards-compatible by doing nothing if the key is not recognized. ```java default void acceptFoo(...) { // do nothing } ``` -------------------------------- ### Build KCL from Source, Skipping Unit Tests Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/README.md Build the Amazon Kinesis Client Library from source using Maven, while skipping unit tests. ```bash mvn clean install -DskipUTs=true ``` -------------------------------- ### Run All Integration Tests Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/README.md Execute all integration tests for the KCL. Ensure you have valid AWS credentials configured. ```bash mvn verify -DskipITs=false ``` -------------------------------- ### Build KCL from Source with Maven Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/README.md Build the Amazon Kinesis Client Library from source using Maven. This command disables GPG signing and does not run integration tests. ```bash mvn clean install -Dgpg.skip=true ``` -------------------------------- ### Lease Rebalancing Threshold Calculation Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/kcl_3x_deep-dive.md Demonstrates the calculation of upper and lower limits for triggering lease rebalancing based on worker metrics and a threshold percentage. ```Java double upperLimit = averageWorkerMetric * (1.0 + (double) this.reBalanceThresholdPercentage / 100.0); double lowerLimit = averageWorkerMetric * (1.0 - (double) this.reBalanceThresholdPercentage / 100.0); ``` -------------------------------- ### Run Integration Tests with Custom AWS Profile Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/README.md Execute integration tests using a specified AWS profile. Ensure the profile is configured in your local .aws/credentials. ```bash mvn -DskipITs=false -DawsProfile="" verify ``` -------------------------------- ### Run Specific Integration Test Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/README.md Execute a single integration test by specifying its class name. Requires valid AWS credentials. ```bash mvn -Dit.test="BasicStreamConsumerIntegrationTest" -DskipITs=false verify ``` -------------------------------- ### DefaultCredentialsProvider Configuration Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/multilang/configuring-credential-providers.md Configure a default AwsCredentialsProvider by specifying 'DefaultCredentialsProvider' in the properties file. ```properties AwsCredentialsProvider = DefaultCredentialsProvider ``` -------------------------------- ### Lease Management Configuration Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/kcl-configurations.md Configuration options for managing leases in the Kinesis client. ```APIDOC ## Lease Management Configuration Options ### Description This section details the configuration parameters for lease management within the Amazon Kinesis Client. These settings control how the client manages leases for processing shards, including failover behavior and synchronization intervals. ### Parameters #### LeaseManagementConfig Parameters - **workerIdentifier** (string) - Required - A unique identifier that represents this instantiation of the application processor. This must be unique. - **failoverTimeMillis** (long) - Optional - The number of milliseconds that must pass before a lease owner is considered failed. Default: 10000 (10 seconds). - **shardSyncIntervalMillis** (long) - Optional - The time interval between shard sync calls. Default: 60000 (60 seconds). - **cleanupLeasesUponShardCompletion** (boolean) - Optional - When set to TRUE, leases are removed as soon as the child leases have started processing. Default: TRUE. - **ignoreUnexpectedChildShards** (boolean) - Optional - When set to TRUE, child shards that have an open shard are ignored. This is primarily for DynamoDB Streams. Default: FALSE. ### Request Example ```json { "workerIdentifier": "unique-worker-id-123", "failoverTimeMillis": 15000, "shardSyncIntervalMillis": 30000, "cleanupLeasesUponShardCompletion": false, "ignoreUnexpectedChildShards": true } ``` ### Response #### Success Response (200) This configuration does not directly return a response in the traditional sense. It is applied during the client's initialization. #### Response Example N/A ``` -------------------------------- ### Set Custom Coordinator State Table Name in KCL Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/kcl-configurations.md Configure a custom name for the coordinator state table by accessing `coordinatorStateTableConfig` via `coordinatorConfig` and then invoking the `tableName` method. This should be done after the `configsBuilder` is initialized. ```java CoordinatorConfig coordinatorConfig = configsBuilder.coordinatorConfig(); coordinatorConfig .coordinatorStateTableConfig() .tableName("CustomTableNameForCoordinatorStateTable"); ``` -------------------------------- ### Lease Rebalancing Dampening Calculation Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/kcl_3x_deep-dive.md Applies a dampening percentage to the calculated load to take from an over-utilized worker, preventing excessive rebalancing. ```Java double loadToTake = currentWorkerMetric - averageWorkerMetric; double dampenedLoadToTake = loadToTake * this.dampeningPercentageValue / 100.0; ``` -------------------------------- ### Lease Assignment Logic in KCL 3.x Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/kcl_3x_deep-dive.md This code is part of the LeaseAssignmentManager and handles lease assignments. Non-leader workers will not perform assignments. ```Java public void leaseAssignment() { if (!this.leader) { return; } // ... assignment logic ... } ``` -------------------------------- ### KclStsAssumeRoleCredentialsProvider with Nested Properties Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/multilang/configuring-credential-providers.md This configuration uses nested properties to specify additional parameters like endpointRegion and externalId for the KclStsAssumeRoleCredentialsProvider. Backslashes are optional for multi-line readability. ```properties AwsCredentialsProvider = KclStsAssumeRoleCredentialsProvider||\ |endpointRegion=us-east-1|externalId=spartacus ``` -------------------------------- ### KCL 3.x Leader Failure Handling Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/kcl_3x_deep-dive.md KCL 3.x extends DynamoDBLockClient's failure handling. It automatically enables another worker to claim leadership if a worker fails to maintain its heartbeat. Additionally, it detects and releases leadership after three consecutive lease assignment failures. ```Java if (leaseAssignmentManager.hasConsecutiveFailures(leader, 3)) { lease.release(); // Release leadership to enable other workers to take the leadership and perform lease assignments. } ``` -------------------------------- ### StsAssumeRoleCredentialsProvider Configuration Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/multilang/configuring-credential-providers.md Configure an StsAssumeRoleCredentialsProvider with an ARN and session name in a properties file. The endpointRegion can be specified; otherwise, it defaults to the region in the AWS config file. ```properties AwsCredentialsProvider = StsAssumeRoleCredentialsProvider|||endpointRegion=us-east-1 ``` -------------------------------- ### Set Custom Lease Table Name in KCL Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/kcl-configurations.md Use the `tableName` parameter within `ConfigsBuilder` to specify a custom name for the KCL lease table. This is done during the initialization of the `ConfigsBuilder`. ```java ConfigsBuilder configsBuilder = new ConfigsBuilder( streamName, applicationName, kinesisClient, dynamoDbAsyncClient, cloudWatchClient, UUID.randomUUID().toString(), new SampleRecordProcessorFactory() ).tableName("CustomNameForLeaseTable"); ``` -------------------------------- ### Calculate CPU utilization percentage in ECS Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/kcl_3x_deep-dive.md This formula is used to calculate the CPU utilization percentage for a container within an ECS task. It requires previous and current total CPU usage deltas and the system CPU usage deltas. ```text cpuUtilization = (cpuDelta / SystemDelta) * online_cpus * 100.0 where: cpuDelta = total_usage - prev_total_usage systemDelta = system_cpu_usage - prev_system_cpu_usage ``` -------------------------------- ### Leader Election in KCL 3.x Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/kcl_3x_deep-dive.md KCL 3.x uses DynamoDBLockClient for leader election. It reads a lock entry, validates the owner's heartbeat, and claims leadership if the previous owner's heartbeat has expired. ```Java // Reads a single lock entry from the DynamoDB lock table // Validates the active status of the current lock owner by checking if the lock item gets constant heartbeat // Claims leadership if the previous lock owner's heartbeat has expired, following a first-worker-wins model ``` -------------------------------- ### Set Custom Worker Metrics Table Name in KCL Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/kcl-configurations.md To set a custom name for the worker metrics table, access `workerMetricsTableConfig` through `leaseManagementConfig` and then call the `tableName` method. This configuration should follow the `configsBuilder` creation. ```java LeaseManagementConfig leaseManagementConfig = configsBuilder.leaseManagementConfig(); leaseManagementConfig .workerUtilizationAwareAssignmentConfig() .workerMetricsTableConfig() .tableName("CustomTableNameForWorkerMetricsTableTable"); ``` -------------------------------- ### KCL 3.x Maven Dependency Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/README.md Add the KCL 3.x dependency to your Maven project. This version is recommended for new projects. ```xml software.amazon.kinesis amazon-kinesis-client 3.4.3 ``` -------------------------------- ### KCL 1.x Maven Dependency Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/README.md Add the KCL 1.x dependency to your Maven project. This version is for legacy projects still on the 1.x branch. ```xml com.amazonaws amazon-kinesis-client 1.14.1 ``` -------------------------------- ### Configure KCL STS Assume Role Credentials Provider Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/docs/multilang/configuring-credential-providers.md Use this configuration in your KCL multilang application to specify the KclStsAssumeRoleCredentialsProvider. This provider allows assuming an IAM role with a specified ARN and session name, optionally including an endpoint region. ```properties AwsCredentialsProvider = KclStsAssumeRoleCredentialsProvider|||endpointRegion=us-east-1 ``` -------------------------------- ### KCL 2.x Maven Dependency Source: https://github.com/awslabs/amazon-kinesis-client/blob/master/README.md Add the KCL 2.x dependency to your Maven project. Use this version if you are maintaining a project on the 2.x branch. ```xml software.amazon.kinesis amazon-kinesis-client 2.7.0 ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.