### Dependency Installation and Generator Execution Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt Provides instructions on how to set up the Python development environment by installing necessary dependencies and running the protocol generator script. ```APIDOC ## Installing Dependencies and Running Generator ### Description This guide outlines the steps required to prepare your environment for generating Hazelcast client protocol code, including installing Python dependencies and executing the generator script. ### Installation Steps 1. **Install Required Python Packages**: Use pip to install all packages listed in the `requirements.txt` file. ```bash pip3 install -r requirements.txt ``` 2. **Verify Installation**: Run a simple Python command to confirm that the essential libraries (YAML, JSON Schema, Jinja2) are installed correctly. ```bash python3 -c "import yaml, jsonschema, jinja2; print('Dependencies installed')" ``` This command should output `Dependencies installed` if successful. ### Running the Generator Execute the generator script with appropriate flags. For instance, to skip binary generation and ID checks: ```bash ./generator.py --no-binary --no-id-check ``` This command initiates the code generation process based on the protocol definitions and specified configurations. ``` -------------------------------- ### Python Setup for Hazelcast Protocol Generator Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/README.md Command to install Python dependencies required for the Hazelcast protocol code generator. This ensures all necessary libraries are available to run the generator after cloning the repository. It requires Python 3 to be installed and configured. ```bash pip3 install -r requirements.txt ``` -------------------------------- ### Install Python Dependencies and Run Generator (Bash) Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt This bash script outlines the steps for setting up the development environment. It includes installing necessary Python packages via `pip3`, verifying the installation by importing key libraries, and running the code generator script `./generator.py` with specific flags to control its behavior, such as disabling binary generation and ID checks. ```bash # Install required Python packages pip3 install -r requirements.txt # Verify installation python3 -c "import yaml, jsonschema, jinja2; print('Dependencies installed')" # Run generator with validation ./generator.py --no-binary --no-id-check ``` -------------------------------- ### Service Definition in YAML Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt Example of a service definition in YAML format for the Hazelcast protocol. This structure defines a 'Map' service with a 'put' method, specifying its parameters, types, retryability, and response. It adheres to the protocol schema for describing remote method calls and distributed operations. ```yaml id: 1 name: Map methods: - id: 1 name: put since: 2.0 doc: | Puts an entry into this map with a given ttl (time to live) value. request: retryable: false partitionIdentifier: key params: - name: name type: String nullable: false since: 2.0 doc: | Name of the map. - name: key type: Data nullable: false since: 2.0 doc: | Key for the map entry. - name: value type: Data nullable: false since: 2.0 doc: | Value for the map entry. - name: threadId type: long nullable: false since: 2.0 - name: ttl type: long nullable: false since: 2.0 response: params: - name: response type: Data nullable: true since: 2.0 doc: | Old value of the entry ``` -------------------------------- ### Custom Type Definition in YAML Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt Example of defining custom complex types in YAML for the Hazelcast protocol. This illustrates how to specify custom data structures like 'ServiceAConfig' and 'ServiceBConfig', including their parameters, types, and nullability. The 'returnWithFactory' option is also shown for specific custom types. ```yaml customTypes: - name: ServiceAConfig since: 2.7 params: - name: address type: String nullable: false since: 2.7 - name: port type: int nullable: false since: 2.7 - name: ServiceBConfig since: 2.7 returnWithFactory: true params: - name: address type: String nullable: false since: 2.7 - name: addressParser type: Data nullable: false since: 2.7 ``` -------------------------------- ### Java Getter Pattern for Custom Type Parameters Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/README.md Illustrates the expected getter patterns in Java for accessing parameters of a custom type within its codec. Boolean parameters use the 'is' prefix, while other types use the 'get' prefix, adhering to Java bean conventions. This ensures the generated codec can correctly retrieve parameter values from custom type instances. ```java // For boolean parameters: customType1.isParamName1() // For other types: customType1.getParamName2() ``` -------------------------------- ### YAML Method Definition for Hazelcast Protocol Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/README.md Illustrates the structure of a single method within a Hazelcast service definition. It includes method identifiers, names, documentation, request details (like retryability and parameters), response parameters, and optional event definitions. This serves as a blueprint for client-server interactions. ```yaml - id: METHOD-ID-1 (1-255) name: METHOD-NAME-1 since: 2.0 doc: | Documentation of the method call request: retryable: false partitionIdentifier: None params: - name: parameter1 type: String nullable: false since: 2.0 doc: | Documentation of the parameter 1 - name: parameter1 type: Data nullable: false since: 2.0 doc: | Documentation of the parameter 2 response: params: - name: response parameter 1 type: Data nullable: true since: 2.0 doc: | the response parameter 1 #Optional events section events: - name: Event-1 params: - name: event-param-1 type: Data nullable: true since: 2.0 doc: | Documentation of the event parameter 1 - name: value type: Data nullable: true since: 2.0 doc: | Documentation of the event parameter 2 ``` -------------------------------- ### Code Generation CLI Commands Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt Command-line interface commands for the Hazelcast protocol code generator. These commands allow users to specify the target language, output directory, and other options for generating client protocol codecs. Different flags control language selection, repository path, binary file generation, and namespace customization. ```bash # Generate Java codecs (default) to a custom repository ./generator.py -r ~/git/hazelcast/ # Generate Python codecs with custom output directory ./generator.py -l py -r ~/projects/hazelcast-python -o client/protocol/codec # Generate C++ codecs without binary compatibility files ./generator.py -l cpp -r ~/dev/hazelcast-cpp --no-binary # Generate TypeScript codecs with custom namespace ./generator.py -l ts -n "@hazelcast/client" -o src/codec # Generate documentation in Markdown format ./generator.py -l md -o docs/protocol ``` -------------------------------- ### Generate Hazelcast Client Protocol Codecs Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/README.md This command-line utility generates client protocol codecs for different languages based on protocol definitions. It supports specifying root directories, target languages, protocol definition paths, output directories, namespaces, and flags for controlling binary and ID checks. Language support includes Java, C++, C#, Python, TypeScript, and Markdown. ```bash ./generator.py [-r ROOT_DIRECTORY] [-l LANGUAGE] [-p PROTOCOL_DEFS_PATH] [-o OUTPUT_DIRECTORY] [-n NAMESPACE] [-b BINARY_OUTPUT_DIR] [-t TEST_OUTPUT_DIR] [--no-binary] [--no-id-check] ``` ```bash ./generator.py -r ~/git/hazelcast/ ``` ```bash ./generator.py -r ~/git/hazelcast/ -o custom/out ``` -------------------------------- ### Java Factory Method Instantiation for Custom Type Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/README.md Demonstrates how to instantiate a custom type in Java using a factory method when the default constructor is not suitable or when `returnWithFactory` is set to true. This involves defining a specific method in a factory class (e.g., `CustomTypeFactory`) that accepts the type's parameters and returns an instance of the custom type. ```java // Factory method signature required in CustomTypeFactory: CustomType1 createCustomType1(boolean paramName1, String paramName2) ``` -------------------------------- ### Java Constructor Instantiation for Custom Type Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/README.md Shows the default mechanism for creating an instance of a custom type in Java using its constructor. The constructor parameters must match the order and types defined in the custom type's protocol definition. This is the primary way custom types are instantiated by the generated codecs if `returnWithFactory` is false. ```java new CustomType1(paramName1, paramName2) ``` -------------------------------- ### Protocol Version Management Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt Demonstrates how to manage protocol evolution by filtering parameters based on version compatibility, ensuring backward compatibility. ```APIDOC ## Protocol Version Management ### Description This section details how the Hazelcast protocol handles versioning, allowing for the addition of new parameters in newer versions while maintaining backward compatibility with older clients. ### Key Concepts - **Version-Based Parameter Filtering**: Functions like `filter_new_params` allow developers to determine which parameters are available for a specific protocol version. - **Numerical Version Comparison**: Helper functions like `get_version_as_number` enable direct comparison of version strings (e.g., "2.0" vs. "2.5"), facilitating version checks. ### Example Usage ```python from util import get_version_as_number, filter_new_params parameter_list = [ {"name": "name", "type": "String", "since": "2.0"}, {"name": "config", "type": "Config", "since": "2.5"}, {"name": "options", "type": "Data", "since": "2.7"} ] # Filter parameters available in version 2.5 available_params = filter_new_params(parameter_list, "2.5") # available_params will contain parameters with 'since' <= "2.5" # Compare versions numerically v1 = get_version_as_number("2.0") # Returns 20000 v2 = get_version_as_number("2.5") # Returns 20500 v3 = get_version_as_number("2.7.1") # Returns 20701 ``` ``` -------------------------------- ### Add Useful Config API Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/custom-types-quick-guide.md This section details the 'addUsefulConfig' API, including its parameters, request structure, and response. ```APIDOC ## POST /hazelcast/hazelcast-client-protocol/addUsefulConfig ### Description Adds a useful configuration to the Hazelcast client. ### Method POST ### Endpoint `/hazelcast/hazelcast-client-protocol/addUsefulConfig` ### Parameters #### Request Body - **name** (String) - Required - The configuration name. - **serviceAConfig** (ServiceAConfig) - Required - Configuration for Service A. - **serviceBConfig** (ServiceBConfig) - Required - Configuration for Service B. ### Request Example ```json { "name": "myConfig", "serviceAConfig": { "address": "localhost", "port": 8080 }, "serviceBConfig": { "address": "192.168.1.1", "addressParser": { /* Data type representing framework specific types */ } } } ``` ### Response #### Success Response (200) - **status** (String) - Indicates success or failure of the operation. #### Response Example ```json { "status": "success" } ``` ``` -------------------------------- ### YAML Service Definition for Hazelcast Protocol Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/README.md Defines the structure for a Hazelcast service, including its ID, name, and a list of methods. Each method is detailed with its own ID, name, request parameters, and response structure, along with optional event definitions. This format is used for protocol communication. ```yaml id: Service Id (0-255) name: Service Name methods: - id: METHOD-ID-1 (1-255) name: METHOD-NAME-1 ... - id: METHOD-ID-2 (1-255) name: METHOD-NAME-2 ... ``` -------------------------------- ### Map Reference Object Names for Compatibility Tests (Python) Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/custom-types-quick-guide.md This Python snippet demonstrates adding entries to the `reference_objects_dict` in `binary/util.py`. These mappings link string names used by generated tests to actual object instances of custom configuration types. ```python 'ServiceAConfig': 'aServiceAConfig', 'ServiceBConfig': 'aServiceBConfig' ``` -------------------------------- ### Generate Binary Compatibility Files (Java) Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt This script generates binary compatibility files for a specified version of the Hazelcast protocol. It loads service definitions, retrieves binary templates, and saves generated files to designated output directories for both binary compatibility and test cases. It depends on Python scripts like `load_services`, `get_binary_templates`, and `save_binary_files`. ```python lang = SupportedLanguages.JAVA version = "2.7" services = load_services("protocol-definitions") binary_templates = get_binary_templates(lang) save_binary_files( binary_output_dir="output/java/hazelcast/src/test/resources", protocol_defs_path="protocol-definitions", version=version, services=services ) save_test_files( test_output_dir="output/java/hazelcast/src/test/java/com/hazelcast/client/protocol/compatibility", lang=lang, version=version, services=services, templates=binary_templates ) ``` -------------------------------- ### Language-Specific Type Mapping Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt Explains how to configure type conversions between protocol-defined types and the corresponding types in a target programming language, such as Java. ```APIDOC ## Language-Specific Type Mapping ### Description This section describes the mechanism for mapping generic protocol types to language-specific types within the Hazelcast client protocol generation process. This is crucial for generating correct client code for different languages. ### Common Type Mappings (Java Example) Language modules, like the Java one, typically define a mapping from protocol type names to fully qualified Java class names. ```java // Example mapping in a Java language module java_types_common = { "String": "java.lang.String", "boolean": "boolean", "int": "int", "long": "long", "UUID": "java.util.UUID", "Data": "com.hazelcast.internal.serialization.Data", "ServiceAConfig": "com.hazelcast.config.ServiceAConfig", "ServiceBConfig": "com.hazelcast.config.ServiceBConfigHolder" } ``` ### Custom Type Registration For complex or custom types not covered by common mappings, mechanisms exist to extend the type registration. ```python # Example of extending custom type mappings from binary import CustomConfigTypes CustomConfigTypes.extend([ "ServiceAConfig", "ServiceBConfig" ]) ``` ``` -------------------------------- ### Binary Compatibility Generation Script Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt Python script snippet for generating binary compatibility test files. It imports functions from a 'binary_generator' module to retrieve binary templates and save both binary files and test files, facilitating protocol version verification. ```python from binary_generator import get_binary_templates, save_binary_files, save_test_files # Example usage (assuming functions are available and imported) # binary_templates = get_binary_templates() # save_binary_files(binary_templates, 'output/binary_tests') # save_test_files(binary_templates, 'output/compatibility_tests') ``` -------------------------------- ### Filter Protocol Parameters by Version (Python) Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt This Python snippet demonstrates version management for protocol evolution, allowing parameters to be added across different versions while maintaining backward compatibility. It includes utility functions `get_version_as_number` for numerical version comparison and `filter_new_params` to filter parameters based on a specified version. This is crucial for handling API changes gracefully. ```python from util import get_version_as_number, filter_new_params # Check if a method version supports specific parameters method_version = "2.0" parameter_list = [ {"name": "name", "type": "String", "since": "2.0"}, {"name": "config", "type": "Config", "since": "2.5"}, {"name": "options", "type": "Data", "since": "2.7"} ] # Filter parameters available in version 2.5 available_params = filter_new_params(parameter_list, "2.5") # Returns: [{"name": "name", ...}, {"name": "config", ...}] # Compare versions numerically v1 = get_version_as_number("2.0") # 20000 v2 = get_version_as_number("2.5") # 20500 v3 = get_version_as_number("2.7.1") # 20701 ``` -------------------------------- ### Add Entry Listener API Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt Adds an entry listener for a map, allowing clients to receive real-time updates on map entries. This method supports typed event parameters for push notifications. ```APIDOC ## POST /hazelcast/hazelcast-client-protocol/addEntryListener ### Description Adds an entry listener for this map. Clients can subscribe to receive notifications about entry changes. ### Method POST ### Endpoint /hazelcast/hazelcast-client-protocol/addEntryListener ### Parameters #### Path Parameters None #### Query Parameters None #### Request Body - **name** (String) - Required - The name of the map to attach the listener to. - **includeValue** (boolean) - Required - Whether to include the entry value in the event. - **listenerFlags** (int) - Required - Flags to specify the types of events to listen for. - **localOnly** (boolean) - Required - Whether the listener should only process local events. ### Request Example ```json { "name": "myMap", "includeValue": true, "listenerFlags": 15, "localOnly": false } ``` ### Response #### Success Response (200) - **response** (UUID) - The unique identifier for the registered listener. #### Response Example ```json { "response": "a1b2c3d4-e5f6-7890-1234-567890abcdef" } ``` ### Events #### Entry Event - **key** (Data) - Optional - The key of the entry that changed. - **value** (Data) - Optional - The new value of the entry. - **oldValue** (Data) - Optional - The old value of the entry. - **mergingValue** (Data) - Optional - The value used during merging. - **eventType** (int) - Required - The type of entry event. - **uuid** (UUID) - Required - The UUID of the listener. ``` -------------------------------- ### Protocol Schema Validation in Python Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt Python script demonstrating how to validate a service definition file against a JSON schema using the 'jsonschema' and 'PyYAML' libraries. It loads both the service definition and the schema, then uses jsonschema.validate to check for compliance, printing success or error messages. ```python import jsonschema import yaml import json # Load and validate a service definition with open('protocol-definitions/Map.yaml', 'r') as f: service_def = yaml.safe_load(f) with open('schema/protocol-schema.json', 'r') as f: schema = json.load(f) try: jsonschema.validate(instance=service_def, schema=schema) print("Service definition is valid") except jsonschema.ValidationError as e: print(f"Validation error: {e.message}") ``` -------------------------------- ### Add Custom Config Types to Binary Compatibility Tests (Python) Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/custom-types-quick-guide.md This Python snippet shows how to add custom configuration types to the `CustomConfigTypes` list in `binary/__init__.py`. This is necessary for the generated binary compatibility tests to recognize custom types like ServiceAConfig and ServiceBConfig. ```python "ServiceAConfig", "ServiceBConfig" ``` -------------------------------- ### Define Method with Server-Side Events (YAML) Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt This YAML snippet defines a method `addEntryListener` that triggers server-side events with typed parameters for push notifications. It includes request parameters, response details, and a comprehensive list of event parameters like key, value, oldValue, mergingValue, eventType, and uuid. This is used for implementing real-time updates in the Hazelcast client. ```yaml - id: 5 name: addEntryListener since: 2.0 doc: | Adds an entry listener for this map. request: retryable: false partitionIdentifier: None params: - name: name type: String nullable: false since: 2.0 - name: includeValue type: boolean nullable: false since: 2.0 - name: listenerFlags type: int nullable: false since: 2.0 - name: localOnly type: boolean nullable: false since: 2.0 response: params: - name: response type: UUID nullable: false since: 2.0 events: - name: Entry params: - name: key type: Data nullable: true since: 2.0 - name: value type: Data nullable: true since: 2.0 - name: oldValue type: Data nullable: true since: 2.0 - name: mergingValue type: Data nullable: true since: 2.0 - name: eventType type: int nullable: false since: 2.0 - name: uuid type: UUID nullable: false since: 2.0 ``` -------------------------------- ### Custom Type Definition in YAML Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/README.md Defines the structure for a custom type within the Hazelcast client protocol using YAML. This includes the type's name, version, whether to return with a factory, and a list of its parameters with their types, nullability, and version information. This definition is used by the code generator to create custom codecs. ```yaml customTypes: - name: CustomType1 since: 2.0 returnWithFactory: true # optional params: - name: paramName1 type: boolean nullable: false since: 2.0 - name: paramName2 type: String nullable: true since: 2.0 ``` -------------------------------- ### Java Classes for Service Configurations Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/custom-types-quick-guide.md These Java classes define the domain types for 'ServiceAConfig' and 'ServiceBConfig'. 'ServiceAConfig' uses standard Java types, while 'ServiceBConfig' uses an opaque 'Data' type for its 'parser' field to accommodate framework-specific implementations. ```java public final class ServiceAConfig { private final String address; private final int port; public ServiceAConfig(String address, int port) { this.address = address; this.port = port; } public String getAddress() { return address; } public int getPort() { return port; } } public final class ServiceBConfig { private final String address; private final AddressParser parser; public ServiceBConfig(String address, AddressParser parser) { this.address = address; this.parser = parser; } public String getAddress() { return address; } public AddressParser getParser() { return parser; } } ``` -------------------------------- ### Map Protocol Types to Java Types (Python) Source: https://context7.com/hazelcast/hazelcast-client-protocol/llms.txt This Python code defines common type mappings from Hazelcast protocol types to their corresponding Java types. It includes a dictionary `java_types_common` for standard conversions and a mechanism using `CustomConfigTypes` to extend this mapping for custom configuration types, ensuring correct serialization and deserialization in the Java client. ```python # In java/__init__.py java_types_common = { "String": "java.lang.String", "boolean": "boolean", "int": "int", "long": "long", "UUID": "java.util.UUID", "Data": "com.hazelcast.internal.serialization.Data", "ServiceAConfig": "com.hazelcast.config.ServiceAConfig", "ServiceBConfig": "com.hazelcast.config.ServiceBConfigHolder" } # Custom codec registration for complex types from binary import CustomConfigTypes CustomConfigTypes.extend([ "ServiceAConfig", "ServiceBConfig" ]) ``` -------------------------------- ### Define New Service API in YAML Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/custom-types-quick-guide.md This YAML snippet defines a new service API endpoint 'addUsefulConfig' within the DynamicConfig.yaml file. It specifies the request parameters, including custom types 'ServiceAConfig' and 'ServiceBConfig', and their respective documentation and version information. ```yaml - id: 21 name: addUsefulConfig since: 2.7 doc: | Adds a useful configuration. request: retryable: false partitionIdentifier: -1 params: - name: name type: String nullable: false since: 2.7 doc: | The configuration name. - name: serviceAConfig type: ServiceAConfig nullable: false since: 2.7 doc: | Configuration for Service A. - name: serviceBConfig type: ServiceBConfig nullable: false since: 2.7 doc: | Configuration for Service B. response: {} ``` -------------------------------- ### Define Custom Types in YAML Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/custom-types-quick-guide.md This YAML snippet defines two custom types, 'ServiceAConfig' and 'ServiceBConfig', in the Custom.yaml file. It outlines the parameters and their types for each custom configuration, essential for the protocol schema. ```yaml - name: ServiceAConfig since: 2.7 params: - name: address type: String nullable: false since: 2.7 - name: port type: int nullable: false since: 2.7 - name: ServiceBConfig since: 2.7 params: - name: address type: String nullable: false since: 2.7 - name: addressParser type: Data nullable: false since: 2.7 ``` -------------------------------- ### Define Reference Objects for Compatibility Tests (Java) Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/custom-types-quick-guide.md This Java code illustrates how to define static final reference objects within the `ReferenceObjects` class. These objects are used by the client compatibility tests to represent instances of custom configuration types, such as ServiceAConfig and ServiceBConfig, and their associated data. ```java // ... public static final ServiceAConfig aServiceAConfig = new ServiceAConfig(aString, anInt); public static final ServiceBConfigHolder aServiceBConfig = new ServiceBConfigHolder(aString, aData); ``` -------------------------------- ### Java Holder Class for ServiceBConfig Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/custom-types-quick-guide.md This Java class, 'ServiceBConfigHolder', acts as an intermediate domain type for 'ServiceBConfig'. It replaces the framework-specific 'AddressParser' with the generic 'Data' type, allowing it to be represented in the client protocol. ```java public final class ServiceBConfigHolder { private final String address; private final Data addressParser; public ServiceBConfigHolder(String address, Data addressParser) { this.address = address; this.addressParser = addressParser; } public String getAddress() { return address; } public Data getAddressParser() { return addressParser; } } ``` -------------------------------- ### Map Custom Types to Java Classes Source: https://github.com/hazelcast/hazelcast-client-protocol/blob/master/custom-types-quick-guide.md This Python snippet shows how to map the custom protocol types 'ServiceAConfig' and 'ServiceBConfig' to their corresponding Java classes within the '_java_types_common' dictionary in '__init__.py'. This mapping is crucial for the code generator. ```python "ServiceAConfig": "com.hazelcast.config.ServiceAConfig", "ServiceBConfig": "com.hazelcast.config.ServiceBConfigHolder" ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.