### Build and Run Validator Tool Source: https://github.com/allegro/json-avro-converter/blob/master/README.md Commands to build the validator JAR and display help information. ```bash ./gradlew :validator:shadowJar java -jar validator/build/libs/json2avro-validator-{version}.jar --help ``` -------------------------------- ### Perform Basic JSON to Avro Conversions Source: https://github.com/allegro/json-avro-converter/blob/master/README.md Demonstrates converting JSON to binary Avro and GenericData.Record, as well as handling conversion exceptions and converting Avro back to JSON. ```java import tech.allegro.schema.json2avro.converter.AvroConversionException; import tech.allegro.schema.json2avro.converter.JsonAvroConverter; import org.apache.avro.generic.GenericData; import org.apache.avro.Schema; // Avro schema with one string field: username String schema = "{" + " \"type\" : \"record\", ``` -------------------------------- ### Validate and Convert via Command Line Source: https://context7.com/allegro/json-avro-converter/llms.txt Executes various conversion and validation tasks using the standalone validator JAR. ```bash # Build the validator JAR ./gradlew :validator:shadowJar # Validate JSON against Avro schema (JSON to Avro conversion) java -jar validator/build/libs/json2avro-validator-*.jar \ -s schema.avsc \ -i input.json # Convert Avro to JSON java -jar validator/build/libs/json2avro-validator-*.jar \ -s schema.avsc \ -i input.avro \ -m avro2json # Round-trip conversion (JSON -> Avro -> JSON) java -jar validator/build/libs/json2avro-validator-*.jar \ -s schema.avsc \ -i input.json \ -m json2avro2json # Save output to file java -jar validator/build/libs/json2avro-validator-*.jar \ -s schema.avsc \ -i input.json \ -o output.avro # Enable debug logging java -jar validator/build/libs/json2avro-validator-*.jar \ -s schema.avsc \ -i input.json \ -d # Show help java -jar validator/build/libs/json2avro-validator-*.jar --help # Example schema file (user.avsc): # { # "namespace": "com.example", # "type": "record", # "name": "User", # "fields": [ # {"name": "name", "type": "string"}, # {"name": "age", "type": "int"}, # {"name": "email", "type": ["null", "string"], "default": null} # ] # } # Example input JSON (user.json): # {"name": "Bob", "age": 50, "email": "bob@example.com"} ``` -------------------------------- ### Convert Complex Avro Types in Java Source: https://context7.com/allegro/json-avro-converter/llms.txt Shows how to process Avro schemas containing arrays, maps, and nested records using the converter. ```java import tech.allegro.schema.json2avro.converter.JsonAvroConverter; import tech.allegro.schema.json2avro.converter.AvroJsonConverter; String schema = """ { "type": "record", "name": "Order", "fields": [ {"name": "orderId", "type": "string"}, { "name": "items", "type": { "type": "array", "items": { "type": "record", "name": "OrderItem", "fields": [ {"name": "productId", "type": "string"}, {"name": "quantity", "type": "int"}, {"name": "price", "type": "double"} ] } } }, { "name": "metadata", "type": {"type": "map", "values": "string"} }, { "name": "shipping", "type": { "type": "record", "name": "ShippingInfo", "fields": [ {"name": "method", "type": "string"}, {"name": "address", "type": "string"} ] } } ] } """; String json = """ { "orderId": "ORD-12345", "items": [ {"productId": "PROD-001", "quantity": 2, "price": 29.99}, {"productId": "PROD-002", "quantity": 1, "price": 49.99} ], "metadata": { "source": "web", "campaign": "summer-sale" }, "shipping": { "method": "express", "address": "456 Oak Ave, Boston, MA" } } """; JsonAvroConverter converter = new JsonAvroConverter(); AvroJsonConverter jsonConverter = new AvroJsonConverter(); byte[] avro = converter.convertToAvro(json.getBytes(), schema); byte[] result = jsonConverter.convertToJson(avro, schema); System.out.println(new String(result)); ``` -------------------------------- ### Initialize Converter with Custom Logic Source: https://github.com/allegro/json-avro-converter/blob/master/README.md Register custom converters during the initialization of the JsonAvroConverter. ```java new JsonAvroConverter(new CompositeJsonToAvroReader(new CustomFieldConverter())) new AvroJsonConverter() ``` -------------------------------- ### Convert JSON to Avro using JsonAvroConverter Source: https://context7.com/allegro/json-avro-converter/llms.txt Use JsonAvroConverter for converting JSON data to binary Avro, GenericData.Record, or SpecificRecord. Ensure correct schema and JSON format for successful conversion. Handle AvroConversionException for errors. ```java import tech.allegro.schema.json2avro.converter.JsonAvroConverter; import tech.allegro.schema.json2avro.converter.AvroConversionException; import org.apache.avro.Schema; import org.apache.avro.generic.GenericData; // Define an Avro schema String schema = """ { "type": "record", "name": "User", "namespace": "com.example", "fields": [ {"name": "name", "type": "string"}, {"name": "age", "type": "int"}, {"name": "email", "type": ["null", "string"], "default": null}, {"name": "tags", "type": {"type": "array", "items": "string"}} ] } """; String json = """ { "name": "John Doe", "age": 30, "email": "john@example.com", "tags": ["developer", "java"] } """; JsonAvroConverter converter = new JsonAvroConverter(); // Convert JSON to binary Avro byte[] avroBytes = converter.convertToAvro(json.getBytes(), schema); // Convert JSON to GenericData.Record Schema parsedSchema = new Schema.Parser().parse(schema); GenericData.Record record = converter.convertToGenericDataRecord(json.getBytes(), parsedSchema); System.out.println("Name: " + record.get("name")); // Output: Name: John Doe System.out.println("Age: " + record.get("age")); // Output: Age: 30 // Handle conversion errors try { String invalidJson = """{"name": 123, "age": "not a number"}""" ; converter.convertToAvro(invalidJson.getBytes(), schema); } catch (AvroConversionException e) { System.err.println("Conversion failed: " + e.getMessage()); // Output: Conversion failed: Failed to convert JSON to Avro: Field age is expected to be type: java.lang.Number } ``` -------------------------------- ### Convert JSON with Optional Fields and Unions in Java Source: https://context7.com/allegro/json-avro-converter/llms.txt Demonstrates handling optional fields and union types in Avro schemas without requiring JSON type declarations. Optional fields can be omitted or provided directly. ```java import tech.allegro.schema.json2avro.converter.JsonAvroConverter; import tech.allegro.schema.json2avro.converter.AvroJsonConverter; import org.apache.avro.generic.GenericData; import org.apache.avro.Schema; String schema = """ { "type": "record", "name": "Person", "fields": [ {"name": "name", "type": "string"}, {"name": "nickname", "type": ["null", "string"], "default": null}, {"name": "age", "type": ["null", "int"], "default": null}, { "name": "address", "type": ["null", { "type": "record", "name": "Address", "fields": [ {"name": "street", "type": "string"}, {"name": "city", "type": "string"} ] }], "default": null } ] } """; JsonAvroConverter converter = new JsonAvroConverter(); AvroJsonConverter jsonConverter = new AvroJsonConverter(); Schema parsedSchema = new Schema.Parser().parse(schema); // Optional fields can be omitted entirely String minimalJson = """{"name": "Alice"}"""; GenericData.Record record = converter.convertToGenericDataRecord(minimalJson.getBytes(), parsedSchema); System.out.println("Nickname: " + record.get("nickname")); // Output: Nickname: null // Or provided directly without type wrapping String fullJson = """ { "name": "Bob", "nickname": "Bobby", "age": 25, "address": {"street": "123 Main St", "city": "Springfield"} } """; byte[] avro = converter.convertToAvro(fullJson.getBytes(), schema); byte[] result = jsonConverter.convertToJson(avro, schema); System.out.println(new String(result)); // Output: {"name":"Bob","nickname":"Bobby","age":25,"address":{"street":"123 Main St","city":"Springfield"}} ``` -------------------------------- ### Implement Custom AvroTypeConverter in Java Source: https://context7.com/allegro/json-avro-converter/llms.txt Define custom conversion logic by implementing the AvroTypeConverter interface. Use canManage to specify which fields or logical types the converter should handle. ```java import tech.allegro.schema.json2avro.converter.JsonAvroConverter; import tech.allegro.schema.json2avro.converter.CompositeJsonToAvroReader; import tech.allegro.schema.json2avro.converter.types.AvroTypeConverter; import org.apache.avro.Schema; import org.apache.avro.generic.GenericData; import com.fasterxml.jackson.databind.ObjectMapper; import java.util.Deque; // Custom converter for a specific field public class PrefixConverter implements AvroTypeConverter { private final String prefix; public PrefixConverter(String prefix) { this.prefix = prefix; } @Override public Object convert(Schema.Field field, Schema schema, Object jsonValue, Deque path, boolean silently) { return prefix + jsonValue.toString(); } @Override public boolean canManage(Schema schema, Deque path) { // Apply to fields named "code" return "code".equals(path.getLast()); } } // Custom converter for a logical type public class CustomTimestampConverter implements AvroTypeConverter { @Override public Object convert(Schema.Field field, Schema schema, Object jsonValue, Deque path, boolean silently) { // Convert string "now" to current timestamp if ("now".equals(jsonValue)) { return System.currentTimeMillis(); } return Long.parseLong(jsonValue.toString()); } @Override public boolean canManage(Schema schema, Deque path) { return schema.getType() == Schema.Type.LONG && schema.getLogicalType() != null && "timestamp-millis".equals(schema.getLogicalType().getName()); } } // Usage String schema = """ { "type": "record", "name": "Record", "fields": [ {"name": "code", "type": "string"}, {"name": "timestamp", "type": {"type": "long", "logicalType": "timestamp-millis"}} ] } """; String json = """{"code": "ABC123", "timestamp": "now"}"""; // Create converter with custom type converters JsonAvroConverter converter = new JsonAvroConverter( new ObjectMapper(), new CompositeJsonToAvroReader( new PrefixConverter("PREFIX-"), new CustomTimestampConverter() ) ); Schema parsedSchema = new Schema.Parser().parse(schema); GenericData.Record record = converter.convertToGenericDataRecord(json.getBytes(), parsedSchema); System.out.println("Code: " + record.get("code")); // Output: Code: PREFIX-ABC123 System.out.println("Timestamp: " + record.get("timestamp")); // Output: Timestamp: ``` -------------------------------- ### Handle Unknown JSON Fields in Java Source: https://context7.com/allegro/json-avro-converter/llms.txt Configure how the converter reacts to JSON fields not present in the Avro schema using FailOnUnknownField or a custom UnknownFieldListener. ```java import tech.allegro.schema.json2avro.converter.JsonAvroConverter; import tech.allegro.schema.json2avro.converter.UnknownFieldListener; import tech.allegro.schema.json2avro.converter.FailOnUnknownField; import tech.allegro.schema.json2avro.converter.CompositeJsonToAvroReader; import tech.allegro.schema.json2avro.converter.AvroConversionException; import com.fasterxml.jackson.databind.ObjectMapper; import java.util.ArrayList; import java.util.List; String schema = """ { "type": "record", "name": "User", "fields": [ {"name": "name", "type": "string"} ] } """; // JSON with extra fields not in schema String json = """ { "name": "John", "extraField": "ignored", "anotherExtra": 123 } """; // Default behavior: silently ignore unknown fields JsonAvroConverter defaultConverter = new JsonAvroConverter(); byte[] avro = defaultConverter.convertToAvro(json.getBytes(), schema); // Works fine, extraField and anotherExtra are ignored // Fail on unknown fields JsonAvroConverter strictConverter = new JsonAvroConverter( new ObjectMapper(), new CompositeJsonToAvroReader(List.of(), new FailOnUnknownField()) ); try { strictConverter.convertToAvro(json.getBytes(), schema); } catch (AvroConversionException e) { System.err.println("Error: " + e.getCause().getMessage()); // Output: Error: Field .extraField is unknown } // Log unknown fields without failing List unknownFields = new ArrayList<>(); UnknownFieldListener loggingListener = (name, value, path) -> { unknownFields.add(path + "." + name); System.out.println("Warning: Unknown field " + path + "." + name + " = " + value); }; JsonAvroConverter loggingConverter = new JsonAvroConverter( new ObjectMapper(), new CompositeJsonToAvroReader(List.of(), loggingListener) ); loggingConverter.convertToAvro(json.getBytes(), schema); System.out.println("Unknown fields found: " + unknownFields); // Output: Unknown fields found: [.extraField, .anotherExtra] ``` -------------------------------- ### Validate JSON to Avro to JSON conversion Source: https://github.com/allegro/json-avro-converter/blob/master/README.md Use this command to verify the round-trip conversion process between JSON and Avro formats using a schema file. ```bash java -jar json2avro-validator.jar -s user.avcs -i user.json -m json2avro2json ``` -------------------------------- ### Convert JSON to SpecificRecord in Java Source: https://context7.com/allegro/json-avro-converter/llms.txt Uses JsonAvroConverter and AvroJsonConverter to map JSON data to Avro-generated classes and back. ```java import tech.allegro.schema.json2avro.converter.JsonAvroConverter; import tech.allegro.schema.json2avro.converter.AvroJsonConverter; import org.apache.avro.Schema; // Assuming you have an Avro-generated class: com.example.User // Generated from schema with: name (string), age (int), email (string) String json = """ { "name": "Jane Doe", "age": 28, "email": "jane@example.com" } """; JsonAvroConverter converter = new JsonAvroConverter(); AavroJsonConverter jsonConverter = new AvroJsonConverter(); // Get schema from generated class Schema schema = User.getClassSchema(); // Convert JSON to SpecificRecord User user = converter.convertToSpecificRecord(json.getBytes(), User.class, schema); System.out.println("Name: " + user.getName()); // Output: Name: Jane Doe System.out.println("Age: " + user.getAge()); // Output: Age: 28 System.out.println("Email: " + user.getEmail()); // Output: Email: jane@example.com // Convert SpecificRecord back to JSON byte[] jsonOutput = jsonConverter.convertToJson(user); System.out.println(new String(jsonOutput)); // Output: {"name":"Jane Doe","age":28,"email":"jane@example.com"} ``` -------------------------------- ### Validate JSON to Avro Conversion Source: https://github.com/allegro/json-avro-converter/blob/master/README.md Use the validator tool to verify JSON against an Avro schema. ```bash java -jar json2avro-validator.jar -s user.avcs -i user.json ``` -------------------------------- ### Implement Custom AvroTypeConverter Source: https://github.com/allegro/json-avro-converter/blob/master/README.md Extend the library by implementing AvroTypeConverter to handle custom types or specific field paths. ```java public class CustomFieldConverter implements AvroTypeConverter { @Override public Object convert(Schema.Field field, Schema schema, Object jsonValue, Deque path, boolean silently) { return "custom-" + jsonValue; } @Override public boolean canManage(Schema schema, Deque path) { return "customField".equals(path.getLast()); } } ``` -------------------------------- ### Convert Avro to JSON via Validator Source: https://github.com/allegro/json-avro-converter/blob/master/README.md Use the validator tool to convert Avro binary data back to JSON format. ```bash java -jar json2avro-validator.jar -s user.avcs -i user.avro -m avro2json ``` -------------------------------- ### Add Converter Dependency to Gradle Source: https://context7.com/allegro/json-avro-converter/llms.txt Include this dependency in your Gradle project to use the JSON Avro Converter library. ```groovy dependencies { implementation 'tech.allegro.schema.json2avro:converter:0.3.0' } ``` -------------------------------- ### Add Dependency for json-avro-converter Source: https://github.com/allegro/json-avro-converter/blob/master/README.md Include this dependency in your Gradle build file to use the converter library. ```groovy dependencies { compile group: 'tech.allegro.schema.json2avro', name: 'converter', version: '0.3.0' } ``` -------------------------------- ### Convert JSON to Avro with Logical Types (Dates/Timestamps) Source: https://context7.com/allegro/json-avro-converter/llms.txt This snippet shows how to convert JSON to Avro using the converter, specifically for Avro logical types like date, time-millis, timestamp-millis, and timestamp-micros. It accepts ISO-8601 formatted strings or numeric values for these types. ```java import tech.allegro.schema.json2avro.converter.JsonAvroConverter; import tech.allegro.schema.json2avro.converter.AvroJsonConverter; import org.apache.avro.generic.GenericData; import org.apache.avro.Schema; String schema = """ { "type": "record", "name": "Event", "fields": [ {"name": "eventId", "type": "string"}, { "name": "eventDate", "type": {"type": "int", "logicalType": "date"} }, { "name": "eventTime", "type": {"type": "int", "logicalType": "time-millis"} }, { "name": "createdAt", "type": {"type": "long", "logicalType": "timestamp-millis"} }, { "name": "updatedAt", "type": {"type": "long", "logicalType": "timestamp-micros"} } ] } """; // Logical types accept ISO-8601 formatted strings or numeric values String json = """ { "eventId": "EVT-001", "eventDate": "2024-01-15", "eventTime": "14:30:00", "createdAt": "2024-01-15T14:30:00Z", "updatedAt": "2024-01-15T14:30:00.123456Z" } """; JsonAvroConverter converter = new JsonAvroConverter(); Schema parsedSchema = new Schema.Parser().parse(schema); GenericData.Record record = converter.convertToGenericDataRecord(json.getBytes(), parsedSchema); System.out.println("Event Date (days since epoch): " + record.get("eventDate")); System.out.println("Created At (millis since epoch): " + record.get("createdAt")); ``` -------------------------------- ### Add Converter Dependency to Maven Source: https://context7.com/allegro/json-avro-converter/llms.txt Include this dependency in your Maven project to use the JSON Avro Converter library. ```xml tech.allegro.schema.json2avro converter 0.3.0 ``` -------------------------------- ### Convert Avro to JSON using AvroJsonConverter Source: https://context7.com/allegro/json-avro-converter/llms.txt Utilize AvroJsonConverter to transform binary Avro data or GenericRecord instances back into JSON format. This supports round-trip conversions and custom logical type handling. ```java import tech.allegro.schema.json2avro.converter.AvroJsonConverter; import tech.allegro.schema.json2avro.converter.JsonAvroConverter; import org.apache.avro.Conversion; String schema = """ { "type": "record", "name": "Event", "fields": [ {"name": "id", "type": "string"}, {"name": "timestamp", "type": "long"}, {"name": "data", "type": {"type": "map", "values": "string"}} ] } """; String json = """ { "id": "evt-123", "timestamp": 1699900000000, "data": {"key1": "value1", "key2": "value2"} } """; JsonAvroConverter jsonToAvro = new JsonAvroConverter(); AvroJsonConverter avroToJson = new AvroJsonConverter(); // Round-trip conversion: JSON -> Avro -> JSON byte[] avroBytes = jsonToAvro.convertToAvro(json.getBytes(), schema); byte[] jsonOutput = avroToJson.convertToJson(avroBytes, schema); System.out.println(new String(jsonOutput)); // Output: {"id":"evt-123","timestamp":1699900000000,"data":{"key1":"value1","key2":"value2"}} // Convert GenericRecord directly to JSON GenericData.Record record = jsonToAvro.convertToGenericDataRecord(json.getBytes(), new Schema.Parser().parse(schema)); byte[] jsonFromRecord = avroToJson.convertToJson(record); ``` -------------------------------- ### Convert JSON to Avro with Enum Validation Source: https://context7.com/allegro/json-avro-converter/llms.txt Use this snippet to convert JSON strings to Avro byte arrays, ensuring that enum values in the JSON conform to the defined Avro schema. Invalid enum values will result in an AvroConversionException. ```java import tech.allegro.schema.json2avro.converter.JsonAvroConverter; import tech.allegro.schema.json2avro.converter.AvroConversionException; String schema = """ { "type": "record", "name": "Task", "fields": [ {"name": "title", "type": "string"}, { "name": "status", "type": { "type": "enum", "name": "TaskStatus", "symbols": ["PENDING", "IN_PROGRESS", "COMPLETED", "CANCELLED"] } }, { "name": "priority", "type": { "type": "enum", "name": "Priority", "symbols": ["LOW", "MEDIUM", "HIGH", "CRITICAL"] } } ] } """; JsonAvroConverter converter = new JsonAvroConverter(); // Valid enum values String validJson = """ { "title": "Implement feature X", "status": "IN_PROGRESS", "priority": "HIGH" } """; byte[] avro = converter.convertToAvro(validJson.getBytes(), schema); // Invalid enum value throws exception try { String invalidJson = """ { "title": "Test task", "status": "INVALID_STATUS", "priority": "HIGH" } """; converter.convertToAvro(invalidJson.getBytes(), schema); } catch (AvroConversionException e) { System.err.println("Error: " + e.getCause().getMessage()); // Output: Error: Field status is expected to be of enum type and be one of PENDING, IN_PROGRESS, COMPLETED, CANCELLED } ``` === COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.