### Complete Avro Object Container Read/Write Example Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-object-container.md A full example demonstrating how to write SensorReading data to an Avro Object Container file with compression and metadata, and then read it back. ```kotlin import com.github.avrokotlin.avro4k.* import kotlinx.serialization.Serializable import org.apache.avro.file.CodecFactory import java.io.FileInputStream import java.io.FileOutputStream @Serializable data class SensorReading( val timestamp: Long, val sensorId: String, val temperature: Double, val humidity: Double ) fun main() { // Writing data val outputFile = FileOutputStream("sensor_data.avro") val schema = Avro.schema() val serializer = Avro.serializersModule.serializer() val writer = AvroObjectContainer.Default.openWriter( schema, serializer, outputFile ) { codec(CodecFactory.deflateCodec(9)) metadata("source", "weather_station_01") metadata("location", "Building A") } // Write multiple readings repeat(1000) { val reading = SensorReading( timestamp = System.currentTimeMillis() + it * 1000, sensorId = "SENSOR-001", temperature = 20.0 + (Math.random() * 5), humidity = 50.0 + (Math.random() * 20) ) writer.writeValue(reading) } writer.close() outputFile.close() // Reading data val inputFile = FileInputStream("sensor_data.avro") val deserializer = Avro.serializersModule.serializer() val readings = AvroObjectContainer.Default.decodeFromStream( deserializer, inputFile ) { metadata("source")?.let { println("Data from: ${it.asString()}") } } // Process readings for ((index, reading) in readings.withIndex()) { println("Reading $index: $reading") if (index >= 10) break // Only read first 10 } inputFile.close() } ``` -------------------------------- ### Example: UUID Serializer Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-serializer.md An example demonstrating the creation of a custom serializer for the UUID type. ```kotlin object UUIDSerializer : AvroSerializer("UUID") ``` -------------------------------- ### Complete Example Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-object-container.md Demonstrates writing and reading Avro data using AvroObjectContainer, including setting and accessing metadata. ```APIDOC ## Complete Example ```kotlin import com.github.avrokotlin.avro4k.* import kotlinx.serialization.Serializable import org.apache.avro.file.CodecFactory import java.io.FileInputStream import java.io.FileOutputStream @Serializable data class SensorReading( val timestamp: Long, val sensorId: String, val temperature: Double, val humidity: Double ) fun main() { // Writing data val outputFile = FileOutputStream("sensor_data.avro") val schema = Avro.schema() val serializer = Avro.serializersModule.serializer() val writer = AvroObjectContainer.Default.openWriter( schema, serializer, outputFile ) { codec(CodecFactory.deflateCodec(9)) metadata("source", "weather_station_01") metadata("location", "Building A") } // Write multiple readings repeat(1000) { val reading = SensorReading( timestamp = System.currentTimeMillis() + it * 1000, sensorId = "SENSOR-001", temperature = 20.0 + (Math.random() * 5), humidity = 50.0 + (Math.random() * 20) ) writer.writeValue(reading) } writer.close() outputFile.close() // Reading data val inputFile = FileInputStream("sensor_data.avro") val deserializer = Avro.serializersModule.serializer() val readings = AvroObjectContainer.Default.decodeFromStream( deserializer, inputFile ) { metadata("source")?.let { println("Data from: ${it.asString()}") } } // Process readings for ((index, reading) in readings.withIndex()) { println("Reading $index: $reading") if (index >= 10) break // Only read first 10 } inputFile.close() } ``` ``` -------------------------------- ### Gradle Kotlin DSL Setup Source: https://github.com/avro-kotlin/avro4k/blob/main/README.md Add the avro4k-core dependency to your project using the Gradle Kotlin DSL. ```kotlin plugins { kotlin("jvm") version kotlinVersion kotlin("plugin.serialization") version kotlinVersion } dependencies { implementation("com.github.avro-kotlin.avro4k:avro4k-core:$avro4kVersion") } ``` -------------------------------- ### Gradle Groovy DSL Setup Source: https://github.com/avro-kotlin/avro4k/blob/main/README.md Add the avro4k-core dependency to your project using the Gradle Groovy DSL. ```groovy plugins { id 'org.jetbrains.kotlin.multiplatform' version kotlinVersion id 'org.jetbrains.kotlin.plugin.serialization' version kotlinVersion } dependencies { implementation "com.github.avro-kotlin.avro4k:avro4k-core:$avro4kVersion" } ``` -------------------------------- ### Custom logical type serializer implementation example Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/configuration.md Example of implementing a custom KSerializer for a logical type. This allows for custom encoding and decoding logic. ```kotlin object CustomLogicalTypeSerializer : KSerializer { // Implementation... } val avro = Avro { setLogicalTypeSerializer("custom-logical-type", CustomLogicalTypeSerializer()) } ``` -------------------------------- ### Registering multiple custom serializers Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/configuration.md Example demonstrating how to register multiple custom serializers for various types using serializersModule. ```kotlin val avro = Avro { serializersModule = SerializersModule { contextual(UUIDv7Serializer()) contextual(MoneySerializer()) contextual(JsonNodeSerializer()) } } ``` -------------------------------- ### Complete Avro4k Single Object Example Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-single-object.md Demonstrates the end-to-end process of setting up a schema registry, encoding a data class to a byte array, and decoding it back using AvroSingleObject. ```kotlin import com.github.avrokotlin.avro4k.* import kotlinx.serialization.Serializable import org.apache.avro.SchemaNormalization @Serializable data class Purchase( val orderId: String, val customerId: Int, val amount: Double, val timestamp: Long ) fun main() { // Setup schema registry val schema = Avro.schema() val fingerprint = SchemaNormalization.parsingFingerprint64(schema) val schemaRegistry = mapOf(fingerprint to schema) // Create single object encoder/decoder val singleObject = AvroSingleObject { fp -> schemaRegistry[fp] } // Encode val purchase = Purchase("ORD-001", 123, 99.99, System.currentTimeMillis()) val encoded = singleObject.encodeToByteArray(purchase) println("Encoded ${encoded.size} bytes") // Decode val decoded = singleObject.decodeFromByteArray(encoded) println("Decoded: $decoded") } ``` -------------------------------- ### Runtime Contextual Type Serialization Configuration Source: https://github.com/avro-kotlin/avro4k/blob/main/gradle-plugin/README.md Example of configuring Avro instance with serializers for contextual types at runtime. ```kotlin val yourConfiguredInstance = Avro { serializersModule = SerializersModule { contextual(YourUuidSerializer) } } ``` -------------------------------- ### Example Usage of OriginalElementName Strategy Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-configuration.md Demonstrates how the OriginalElementName strategy results in the schema field name matching the data class field name. ```kotlin @Serializable data class MyData(val myField: String) // Schema field name: "myField" ``` -------------------------------- ### Maven Setup for Kotlin Serialization Plugin Source: https://github.com/avro-kotlin/avro4k/blob/main/README.md Configure the Kotlin Maven plugin to include the serialization compiler plugin. ```xml org.jetbrains.kotlin kotlin-maven-plugin ${kotlin.version} compile compile compile kotlinx-serialization org.jetbrains.kotlin kotlin-maven-serialization ${kotlin.version} ``` -------------------------------- ### Maven Setup for Avro4k Dependency Source: https://github.com/avro-kotlin/avro4k/blob/main/README.md Add the avro4k-core dependency to your Maven project. ```xml com.github.avro-kotlin.avro4k avro4k-core ${avro4k.version} ``` -------------------------------- ### Implement a Custom Avro Serializer Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/README.md Provides an example of creating a custom AvroSerializer for a CustomType. This involves defining the schema, serialization, and deserialization logic. ```kotlin object CustomTypeSerializer : AvroSerializer("CustomType") { override fun getSchema(context: SchemaSupplierContext): Schema { return Schema.create(Schema.Type.STRING) } override fun serializeAvro(encoder: AvroEncoder, value: CustomType) { encoder.encodeString(value.toString()) } override fun deserializeAvro(decoder: AvroDecoder): CustomType { return CustomType(decoder.decodeString()) } } ``` -------------------------------- ### Configuration Inheritance Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/configuration.md Create specialized avro4k configurations by extending a base configuration. This example shows a strict variant and a validated variant. ```kotlin // Base configuration val baseConfig = Avro { fieldNamingStrategy = FieldNamingStrategy.Builtins.SnakeCase implicitNulls = true } // Strict variant val strictConfig = Avro(baseConfig) { implicitEmptyCollections = false validateSerialization = true } // Validated variant val validateConfig = Avro(baseConfig) { validateSerialization = true } ``` -------------------------------- ### Example Generated Avro Schema for Pizza Source: https://github.com/avro-kotlin/avro4k/blob/main/README.md This is an example of the Avro schema generated for the `Pizza` data class. It details the record structure, fields, types, and nested records. ```json { "type": "record", "name": "Pizza", "namespace": "com.github.avrokotlin.avro4k.example", "fields": [ { "name": "name", "type": "string" }, { "name": "ingredients", "type": { "type": "array", "items": { "type": "record", "name": "Ingredient", "fields": [ { "name": "name", "type": "string" }, { "name": "sugar", "type": "double" } ] } } }, { "name": "topping", "type": [ "null", { "type": "record", "name": "Ingredient" } ], "default": null }, { "name": "vegetarian", "type": "boolean" } ] } ``` -------------------------------- ### Define Schema Evolution with Avro Annotations Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/README.md Shows how to use Avro annotations like @AvroAlias and @AvroDoc to manage schema evolution and provide documentation for fields. This example defines a User data class with version 1 schema details. ```kotlin @Serializable @AvroAlias("UserV1") data class User( val id: Int, @AvroAlias("full_name") val name: String, @AvroDoc("User email") val email: String? = null ) ``` -------------------------------- ### Decode from Stream with Inferred Type and Example Usage Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-object-container.md Decodes values from an object container file with the type inferred. This example shows taking the first 100 events from a stream. ```kotlin @Serializable data class Event(val id: String, val type: String) val inputStream = FileInputStream("events.avro") val events = AvroObjectContainer.Default.decodeFromStream(inputStream) // Take first 100 events val first100 = events.take(100).toList() ``` -------------------------------- ### Example of Writing Values and Closing Writer Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-object-container.md Demonstrates writing multiple log entries to an Avro object container and properly closing the writer. The `finally` block ensures the writer is closed even if errors occur. ```kotlin @Serializable data class Log(val level: String, val message: String) val writer = container.openWriter(outputStream) writer.writeValue(Log("INFO", "Application started")) writer.writeValue(Log("ERROR", "Something failed")) writer.close() ``` -------------------------------- ### Example Usage of @AvroFixed Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/annotations.md Demonstrates using @AvroFixed with ByteArray, String, and BigDecimal properties to define fixed-size Avro types. ```kotlin import kotlinx.serialization.Serializable import com.github.avrokotlin.avro4k.annotations.AvroFixed import com.github.avrokotlin.avro4k.annotations.AvroDecimal import java.math.BigDecimal @Serializable data class Fingerprint( @AvroFixed(16) val data: ByteArray ) @Serializable data class Hash( @AvroFixed(32) val sha256: String // 32 bytes of UTF-8 encoded string ) @Serializable data class Money( @AvroDecimal(scale = 2, precision = 18) @AvroFixed(16) val amount: BigDecimal ) ``` -------------------------------- ### Example: Catch MissingFieldsEncodingException Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/errors.md Demonstrates how to catch MissingFieldsEncodingException when encoding incomplete data. This occurs when a record is missing required fields. ```kotlin @Serializable data class User(val id: Int, val name: String) try { // Encoding with custom schema that requires both fields val schema = Avro.schema() val incompleteData = mapOf("id" to 1) // Missing "name" // This would throw MissingFieldsEncodingException } catch (e: MissingFieldsEncodingException) { println("Missing fields: ${e.message}") } ``` -------------------------------- ### Example Usage of @AvroDefault Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/annotations.md Illustrates using @AvroDefault with various types including String, Int, Boolean, nullable String, List, and Map to specify default values. ```kotlin import kotlinx.serialization.Serializable import com.github.avrokotlin.avro4k.annotations.AvroDefault @Serializable data class Config( @AvroDefault("\"localhost\"") val host: String, @AvroDefault("8080") val port: Int, @AvroDefault("true") val debug: Boolean, @AvroDefault("null") val comment: String?, @AvroDefault("[]") val tags: List, @AvroDefault("{\"key\":\"value\"}") val metadata: Map ) ``` -------------------------------- ### Implement Avro Schema Generation Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-serializer.md Implement getSchema to define the Avro schema for a custom type. This example creates a fixed schema for a ByteArray. ```kotlin fun getSchema(context: SchemaSupplierContext): Schema ``` ```kotlin object FixedBytesSerializer : AvroSerializer("fixed") { override fun getSchema(context: SchemaSupplierContext): Schema { return Schema.createFixed("FixedBytes", null, null, 16) } override fun serializeAvro(encoder: AvroEncoder, value: ByteArray) { encoder.encodeFixed(value) } override fun deserializeAvro(decoder: AvroDecoder): ByteArray { return decoder.decodeFixed().bytes() } } ``` -------------------------------- ### Define a Custom AvroSerializer Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-serializer.md Create a custom serializer by extending AvroSerializer and providing a descriptor name. This is the basic setup for any custom type. ```kotlin object MyTypeSerializer : AvroSerializer("MyType") ``` -------------------------------- ### Example: Fixed size mismatch with @AvroFixed Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/errors.md Illustrates the scenario leading to a 'Fixed size mismatch' SerializationException, where a ByteArray's size does not match the expected size defined by @AvroFixed. ```kotlin @Serializable data class Fingerprint( @AvroFixed(32) val hash: ByteArray // Must be exactly 32 bytes ) ``` -------------------------------- ### Example Usage of @AvroEnumDefault Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/annotations.md Shows how to use @AvroEnumDefault on an enum value (UNKNOWN) to handle unknown enum symbols during deserialization, as demonstrated in the Task data class. ```kotlin import kotlinx.serialization.Serializable import com.github.avrokotlin.avro4k.annotations.AvroEnumDefault @Serializable enum class Status { PENDING, ACTIVE, @AvroEnumDefault UNKNOWN, COMPLETED } @Serializable data class Task( val id: String, val status: Status // Defaults to UNKNOWN if unknown value in data ) ``` -------------------------------- ### Avro Default Instance Usage Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro.md Demonstrates basic usage of the default Avro instance for schema generation, encoding, and decoding. ```kotlin val schema = Avro.schema() val bytes = Avro.encodeToByteArray(myInstance) val decoded = Avro.decodeFromByteArray(bytes) ``` -------------------------------- ### Basic Avro Instance Configuration Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/configuration.md Configure Avro instances with basic settings like field naming strategy and implicit nulls/collections. ```kotlin val avro = Avro { fieldNamingStrategy = FieldNamingStrategy.Builtins.SnakeCase implicitNulls = false implicitEmptyCollections = false } ``` -------------------------------- ### Configure Avro Instance Source: https://github.com/avro-kotlin/avro4k/blob/main/Migrating-from-v1.md Demonstrates the new builder syntax for configuring Avro instances. The constructor with AvroConfiguration is replaced by a lambda-based configuration. ```kotlin // Previously val avro = Avro( AvroConfiguration( namingStrategy = FieldNamingStrategy.SnackCase, implicitNulls = true, ), SerializersModule { contextual(CustomSerializer()) } ) // Now val avro = Avro { namingStrategy = FieldNamingStrategy.SnackCase implicitNulls = true serializersModule = SerializersModule { contextual(CustomSerializer()) } } ``` -------------------------------- ### Container File Pattern for Writing Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/extension-functions.md Demonstrates opening a writer for Avro Object Container files, specifying a codec, writing multiple records, and closing the writer and output stream. ```kotlin @Serializable data class Record(val id: Int, val data: String) // Write val output = FileOutputStream("data.avro") val writer = AvroObjectContainer.Default.openWriter(output) { codec(CodecFactory.snappyCodec()) } records.forEach { writer.writeValue(it) } writer.close() output.close() ``` -------------------------------- ### Create Custom Avro Instance with Configuration Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro.md Builds a custom Avro instance with specific configuration options like field naming strategy, null handling, and logical type serializers. Use this for tailored Avro processing. ```kotlin val customAvro = Avro { fieldNamingStrategy = FieldNamingStrategy.Builtins.SnakeCase implicitNulls = false implicitEmptyCollections = false validateSerialization = true setLogicalTypeSerializer("custom-type", CustomSerializer()) } val schema = customAvro.schema() ``` -------------------------------- ### Run Benchmark Locally Source: https://github.com/avro-kotlin/avro4k/blob/main/benchmark/README.md Execute the benchmark tests locally using the provided Gradle wrapper command. Results will be generated in the specified directory. ```shell ../gradlew benchmark ``` -------------------------------- ### Get Serializer and Deserializer from Serde Source: https://github.com/avro-kotlin/avro4k/blob/main/confluent-kafka-serializer/README.md Retrieve the serializer and deserializer instances from a configured Avro4k Kafka serde object. ```kotlin val serializer = serde.serializer() val deserializer = serde.deserializer() ``` -------------------------------- ### Using Loose Configuration for Testing Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/errors.md Configure avro4k with lenient settings for implicit nulls and empty collections during testing to focus on core logic rather than strict schema adherence. ```kotlin val testAvro = Avro { implicitNulls = true implicitEmptyCollections = true } // More forgiving - catches actual bugs, not schema mismatches ``` -------------------------------- ### Implement Generic Deserialization Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-serializer.md Override deserializeGeneric to support deserialization with non-Avro formats. This example demonstrates deserializing a Duration from a string. ```kotlin open fun deserializeGeneric(decoder: Decoder): T { throw UnsupportedOperationException(...) } ``` ```kotlin object DurationSerializer : AvroSerializer("duration") { override fun deserializeAvro(decoder: AvroDecoder): Duration { return decoder.decodeFixed().let { Duration.ofMillis(it.bytes().asLong()) } } override fun deserializeGeneric(decoder: Decoder): Duration { return Duration.parse(decoder.decodeString()) } } ``` -------------------------------- ### Custom Avro Instance Configuration Source: https://github.com/avro-kotlin/avro4k/blob/main/README.md Shows how to create a custom Avro4k instance with specific configurations for field naming, null handling, collection handling, serialization validation, and logical type serializers. This custom instance is then used for encoding, decoding, and schema generation. ```kotlin val yourAvroInstance = Avro { fieldNamingStrategy = FieldNamingStrategy.Builtins.SnakeCase implicitNulls = false implicitEmptyCollections = false validateSerialization = true setLogicalTypeSerializer("your-logical-type", YourSerializer()) } yourAvroInstance.encodeToByteArray(MyData("value")) yourAvroInstance.decodeFromByteArray(bytes) yourAvroInstance.schema() ``` -------------------------------- ### Create Configured Reflect Avro4kKafkaSerde Directly Source: https://github.com/avro-kotlin/avro4k/blob/main/confluent-kafka-serializer/README.md Instantiate a ReflectAvro4kKafkaSerde with configuration parameters directly, avoiding a separate .configure call. ```kotlin val serde = ReflectAvro4kKafkaSerde( isKey = false, props = mapOf("schema.registry.url" to "http://the-url.com") ) ``` -------------------------------- ### Shared Avro Schema Definition Source: https://github.com/avro-kotlin/avro4k/blob/main/gradle-plugin/README.md Example of a shared Avro schema defining an enum type 'Country' within the 'shared' namespace. ```json { "type": "enum", "name": "Country", "namespace": "shared", "symbols": ["FR", "GB", "IT"] } ``` -------------------------------- ### Avro Schema with Type Reference Source: https://github.com/avro-kotlin/avro4k/blob/main/gradle-plugin/README.md Example of an Avro schema defining a 'Profile' record that references a 'Country' type from a shared schema. ```json { "type": "record", "name": "Profile", "fields": [ { "name": "nickname", "type": "string" }, { "name": "country", "type": "shared.Country" } ] } ``` -------------------------------- ### Default Avro Configuration Usage Source: https://github.com/avro-kotlin/avro4k/blob/main/README.md Demonstrates the usage of Avro4k with its default configuration for encoding, decoding, and schema generation. ```kotlin Avro.encodeToByteArray(MyData("value")) Avro.decodeFromByteArray(bytes) Avro.schema() ``` -------------------------------- ### Recommended Performance Configuration Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/configuration.md Configure avro4k for optimal performance by using original element name strategy and disabling validation. ```kotlin val fastAvro = Avro { fieldNamingStrategy = FieldNamingStrategy.Builtins.OriginalElementName validateSerialization = false implicitNulls = true implicitEmptyCollections = true } ``` -------------------------------- ### Basic Avro Encoding and Decoding Source: https://github.com/avro-kotlin/avro4k/blob/main/README.md Demonstrates how to generate Avro schemas, serialize Kotlin data classes to byte arrays, and deserialize them back. This is for 'pure' Avro format without schema prefix. ```kotlin package myapp import com.github.avrokotlin.avro4k.* import kotlinx.serialization.* @Serializable data class Project(val name: String, val language: String) fun main() { // Generating schemas val schema = Avro.schema() println(schema.toString()) // {"type":"record","name":"Project","namespace":"myapp","fields":[{"name":"name","type":"string"},{"name":"language","type":"string"}]} // Serializing objects val data = Project("kotlinx.serialization", "Kotlin") val bytes = Avro.encodeToByteArray(data) // Deserializing objects val obj = Avro.decodeFromByteArray(bytes) println(obj) // Project(name=kotlinx.serialization, language=Kotlin) } ``` -------------------------------- ### Implement Generic Serialization Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-serializer.md Override serializeGeneric to support serialization with non-Avro formats like JSON. This example shows serialization of an Instant to a string. ```kotlin open fun serializeGeneric(encoder: Encoder, value: T) { throw UnsupportedOperationException(...) } ``` ```kotlin object TimestampSerializer : AvroSerializer("timestamp") { override fun serializeAvro(encoder: AvroEncoder, value: Instant) { encoder.encodeLong(value.toEpochMilli()) } override fun serializeGeneric(encoder: Encoder, value: Instant) { encoder.encodeString(value.toString()) } } ``` -------------------------------- ### Configure Avro Instance for Schema Generation Source: https://github.com/avro-kotlin/avro4k/blob/main/README.md Create a custom `Avro` instance to apply specific configurations before generating schemas. This allows for customization of the schema generation process. ```kotlin val yourAvroInstance = Avro { // your configuration } yourAvroInstance.schema() ``` -------------------------------- ### Custom Avro Serializer for WebURL Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-serializer.md Demonstrates how to create a custom Avro serializer for a data class like WebURL. This includes defining the schema, serialization, and deserialization logic for both Avro and generic formats. Registration is shown via a contextual serializer module. ```kotlin import com.github.avrokotlin.avro4k.* import com.github.avrokotlin.avro4k.serializer.AvroSerializer import kotlinx.serialization.KSerializer import kotlinx.serialization.descriptors.SerialDescriptor import kotlinx.serialization.encoding.Decoder import kotlinx.serialization.encoding.Encoder import org.apache.avro.Schema import java.net.URL // Custom type data class WebURL(val url: String) { init { require(url.startsWith("http")) { "URL must start with http" } } } // Custom serializer object WebURLSerializer : AvroSerializer("URL") { override fun getSchema(context: SchemaSupplierContext): Schema { return Schema.create(Schema.Type.STRING) } override fun serializeAvro(encoder: AvroEncoder, value: WebURL) { encoder.encodeString(value.url) } override fun deserializeAvro(decoder: AvroDecoder): WebURL { return WebURL(decoder.decodeString()) } override fun serializeGeneric(encoder: Encoder, value: WebURL) { encoder.encodeString(value.url) } override fun deserializeGeneric(decoder: Decoder): WebURL { return WebURL(decoder.decodeString()) } } // Registration and usage fun main() { val avro = Avro { serializersModule = SerializersModule { contextual(WebURLSerializer) } } @Serializable data class Website( val name: String, @Contextual val homepage: WebURL ) val site = Website("My Site", WebURL("https://example.com")) val bytes = avro.encodeToByteArray(site) val decoded = avro.decodeFromByteArray(bytes) println(decoded) } ``` -------------------------------- ### SerializationException: Not all bytes consumed Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/errors.md Example of catching a SerializationException when extra bytes remain after deserialization. This can happen with decodeFromByteArray if the byte array is larger than expected. ```kotlin val extraBytes = byteArrayOf(1, 2, 3, 4, 5) try { // If only first 3 bytes are valid data Avro.decodeFromByteArray(extraBytes) } catch (e: SerializationException) { println("Error: ${e.message}") } ``` -------------------------------- ### Configure Specific Serde with Custom Avro Instance Source: https://github.com/avro-kotlin/avro4k/blob/main/confluent-kafka-serializer/README.md Create a SpecificAvro4kKafkaSerde instance, passing a custom Avro object and reified type for advanced configuration. ```kotlin val avro = Avro { // your custom configuration } val serde = SpecificAvro4kKafkaSerde( avro = avro, isKey = false, props = mapOf("schema.registry.url" to "http://the-url.com") ) ``` -------------------------------- ### Custom Union Type Serializer Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-encoder-decoder.md Example of a custom serializer for a union type, demonstrating how to use encodeUnionIndex to select the correct branch before encoding the value. ```kotlin object UnionTypeSerializer : AvroSerializer("union") { override fun serializeAvro(encoder: AvroEncoder, value: Any) { when (value) { is String -> { encoder.encodeUnionIndex(0) // String branch encoder.encodeString(value) } is Int -> { encoder.encodeUnionIndex(1) // Int branch encoder.encodeInt(value) } } } } ``` -------------------------------- ### Apply @AvroStringable Annotation to Properties Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/annotations.md Example of using @AvroStringable to serialize Int, Boolean, and Double properties as Avro strings. This requires the type to be stringable. ```kotlin @Serializable data class Settings( @AvroStringable val timeout: Int, @AvroStringable val enabled: Boolean, @AvroStringable val version: Double ) ``` -------------------------------- ### Create Configured Reflect Serializer and Deserializer Directly Source: https://github.com/avro-kotlin/avro4k/blob/main/confluent-kafka-serializer/README.md Instantiate ReflectAvro4kKafkaSerializer and ReflectAvro4kKafkaDeserializer with configuration parameters directly. ```kotlin val serializer = ReflectAvro4kKafkaSerializer( isKey = false, props = mapOf("schema.registry.url" to "http://the-url.com") ) val deserializer = ReflectAvro4kKafkaDeserializer( isKey = false, props = mapOf("schema.registry.url" to "http://the-url.com") ) ``` -------------------------------- ### Get Current Writer Schema Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-encoder-decoder.md Retrieves the Avro schema used for encoding the current value being decoded. This schema corresponds to the actual value, not a union. ```kotlin val schema = decoder.currentWriterSchema ``` -------------------------------- ### SerializationException: Unsupported schema type Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/errors.md Example of a SerializationException due to type incompatibility between the value and the schema. This occurs when trying to encode a type that is not supported by the target schema. ```kotlin SerializationException: Unsupported schema 'string' for encoded type of int, and also not matching to any compatible type (one of long, float, double). Actual schema: [schema definition] ``` -------------------------------- ### Configure Reflect Avro4kKafkaSerializer and Deserializer Source: https://github.com/avro-kotlin/avro4k/blob/main/confluent-kafka-serializer/README.md Create and configure separate ReflectAvro4kKafkaSerializer and ReflectAvro4kKafkaDeserializer instances. Ensure to call .configure before use. ```kotlin val serializer = ReflectAvro4kKafkaSerializer() serializer.configure(mapOf("schema.registry.url" to "http://the-url.com"), isKey = false) val deserializer = ReflectAvro4kKafkaDeserializer() deserializer.configure(mapOf("schema.registry.url" to "http://the-url.com"), isKey = false) ``` -------------------------------- ### Reading Binary Encoded Files Source: https://github.com/avro-kotlin/avro4k/blob/main/Migrating-from-v1.md Illustrates how to read binary encoded Avro files. The new approach requires explicit schema handling when the writer schema does not correspond to the expected type. ```kotlin // Previously Avro.default.openInputStream(serializer) { decodeFormat = AvroDecodeFormat.Binary(schema) } .from(data).use { avroInputStream -> return avroInputStream.nextOrThrow() } ``` ```kotlin // Now val inputStream = ByteArrayInputStream(data) while (inputStream.remaining() > 0) { // If the writer schema corresponds to the specified type val element = Avro.decodeFromStream(inputStream) // If the writer schema does not correspond to the specified type val element = Avro.decodeFromStream(writerSchema, inputStream) // With explicit writer schema and serializer val element = Avro.decodeFromStream(writerSchema, serializer, inputStream) } ``` -------------------------------- ### SerializationException: Named schema not found in union Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/errors.md Example illustrating a SerializationException where a named schema is not found within a union. This occurs when the schema being encoded is not present in the union definition. ```kotlin @Serializable sealed class Event { @Serializable data class UserCreated(val id: Int) : Event() } // If union schema doesn't include UserCreated ``` -------------------------------- ### Create Configured Specific Avro4kKafkaSerde Directly Source: https://github.com/avro-kotlin/avro4k/blob/main/confluent-kafka-serializer/README.md Instantiate SpecificAvro4kKafkaSerde, serializer, and deserializer with configuration parameters directly, inferring type with reified methods. ```kotlin val serde = SpecificAvro4kKafkaSerde( isKey = false, props = mapOf("schema.registry.url" to "http://the-url.com") ) val serializer = SpecificAvro4kKafkaSerializer( isKey = false, props = mapOf("schema.registry.url" to "http://the-url.com") ) val deserializer = SpecificAvro4kKafkaDeserializer( isKey = false, props = mapOf("schema.registry.url" to "http://the-url.com") ) ``` -------------------------------- ### Apply @AvroDecimal and @AvroFixed Annotations Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/annotations.md Example of using @AvroDecimal to set scale and precision for a BigDecimal, and combining it with @AvroFixed for fixed-size encoding. Precision must be greater than scale. ```kotlin @Serializable data class Price( @AvroDecimal(scale = 2, precision = 10) val amount: BigDecimal, @AvroDecimal(scale = 4, precision = 12) @AvroFixed(8) val exchangeRate: BigDecimal ) ``` -------------------------------- ### Apply @AvroProp Annotation to Data Class Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/annotations.md Example of applying @AvroProp to a data class and its fields to add custom schema properties. Supports both string and JSON values. ```kotlin @Serializable @AvroProp("custom_property", "custom_value") @AvroProp("json_property", "{\"nested\":\"value\"}") data class MyData( @AvroProp("field_prop", "42") val myField: String ) ``` -------------------------------- ### Configure Reflect Avro4kKafkaSerde Source: https://github.com/avro-kotlin/avro4k/blob/main/confluent-kafka-serializer/README.md Create and configure a ReflectAvro4kKafkaSerde instance. Ensure to call .configure before use. ```kotlin val serde = ReflectAvro4kKafkaSerde() serde.configure(mapOf("schema.registry.url" to "http://the-url.com"), isKey = false) ``` -------------------------------- ### Kafka Avro Deserializer Implementation Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-single-object.md Implements the Kafka Deserializer interface to decode Avro byte arrays back into Kotlin objects. Requires a schema registry setup. ```kotlin class KafkaAvroDeserializer : org.apache.kafka.common.serialization.Deserializer { private val singleObject = AvroSingleObject(schemaRegistry) override fun deserialize(topic: String?, data: ByteArray?): MyData? { return data?.let { singleObject.decodeFromByteArray(it) } } } ``` -------------------------------- ### ListRecord Data Class (Deprecated) Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/types.md An immutable record implementation backed by a list, also deprecated in favor of org.apache.avro.generic.GenericData.Record. It provides methods to get schema and field values. ```kotlin @Deprecated("Use GenericData.Record instead") @ExperimentalAvro4kApi public data class ListRecord( private val s: Schema, private val values: List, ) : Record ``` -------------------------------- ### Basic Encode and Decode User Data Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/README.md Demonstrates how to encode a User data class to a byte array and then decode it back. Ensure the User class is annotated with @Serializable. ```kotlin @Serializable data class User(val id: Int, val name: String) // Encode val user = User(1, "Alice") val bytes = Avro.encodeToByteArray(user) // Decode val decoded = Avro.decodeFromByteArray(bytes) ``` -------------------------------- ### Basic Encoding and Decoding Pattern Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/extension-functions.md Demonstrates the basic pattern for encoding a serializable data class to ByteArray and then decoding it back. ```kotlin @Serializable data class Message(val id: String, val text: String) // Create schema once val schema = Avro.schema() // Encode val message = Message("msg-1", "Hello") val bytes = Avro.encodeToByteArray(schema, message) // Decode val decoded = Avro.decodeFromByteArray(schema, bytes) ``` -------------------------------- ### Kotlin Property with SerialName Annotation Source: https://github.com/avro-kotlin/avro4k/blob/main/gradle-plugin/README.md Example of a Kotlin property generated with the 'useKotlinConventionForFieldNames' flag enabled, showing the '@SerialName' annotation to preserve the original Avro field name for serialization. ```kotlin @SerialName("user_id") public val userId: Int ``` -------------------------------- ### Kafka-Friendly Configuration Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/configuration.md Configuration suitable for Kafka topics, using SnakeCase for field naming and enabling implicit nulls and empty collections. This allows for gradual schema evolution. ```kotlin val kafkaAvro = Avro { fieldNamingStrategy = FieldNamingStrategy.Builtins.SnakeCase implicitNulls = true // Compatible with schema changes implicitEmptyCollections = true } // Allows gradual schema evolution in Kafka topics ``` -------------------------------- ### Integrate with Message Brokers using AvroSingleObject Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/README.md Implement a message broker integration using AvroSingleObject, requiring a schema registry lookup by fingerprint. ```kotlin val singleObject = AvroSingleObject { schemaRegistry[fingerprint] } ``` -------------------------------- ### Apply @AvroDoc Annotation to Data Class and Enum Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/annotations.md Example of using @AvroDoc to add documentation to a data class, its fields, and an enum type. Documentation is ignored in value classes. ```kotlin @Serializable @AvroDoc("Represents a user account") data class User( @AvroDoc("Unique user identifier") val id: Int, @AvroDoc("User's full name") val name: String ) @Serializable @AvroDoc("Status of a request") enum class Status { PENDING, ACTIVE, COMPLETED } ``` -------------------------------- ### Customize Avro Serialization Behavior Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/README.md Illustrates how to create a custom Avro instance with specific configurations, such as snake_case field naming, disabling implicit nulls, and enabling validation. ```kotlin val customAvro = Avro { fieldNamingStrategy = FieldNamingStrategy.Builtins.SnakeCase implicitNulls = false validateSerialization = true } ``` -------------------------------- ### Strict Validation Configuration Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/configuration.md Configuration for strict validation, disabling implicit nulls and empty collections, and enabling serialization validation. This setup throws exceptions for any missing or incompatible data. ```kotlin val strictAvro = Avro { implicitNulls = false implicitEmptyCollections = false validateSerialization = true } // Throws exceptions for any missing or incompatible data ``` -------------------------------- ### Container File Pattern for Reading Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/extension-functions.md Illustrates opening a reader for Avro Object Container files and iterating through the decoded records. Ensure the input stream is closed. ```kotlin // Read val input = FileInputStream("data.avro") val records = AvroObjectContainer.Default.decodeFromStream(input) records.forEach { println(it) } input.close() ``` -------------------------------- ### Schema Evolution with Aliases Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-configuration.md Demonstrates how to handle schema evolution by using @AvroAlias to map old field names to new ones. This allows for backward compatibility when renaming fields. ```kotlin val avro = Avro { fieldNamingStrategy = FieldNamingStrategy.Builtins.SnakeCase } @Serializable data class UserV2( val id: Int, @AvroAlias("name") val fullName: String, val email: String? = null ) val schema = avro.schema() ``` -------------------------------- ### Kafka Avro Serializer Implementation Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-single-object.md Implements the Kafka Serializer interface to encode Kotlin objects to Avro byte arrays for Kafka messages. Requires a schema registry setup. ```kotlin class KafkaAvroSerializer : org.apache.kafka.common.serialization.Serializer { private val singleObject = AvroSingleObject(schemaRegistry) override fun serialize(topic: String?, data: Any?): ByteArray? { return data?.let { singleObject.encodeToByteArray(it) } } } ``` -------------------------------- ### Configure Reflect Serde with Custom Avro Instance Source: https://github.com/avro-kotlin/avro4k/blob/main/confluent-kafka-serializer/README.md Create a ReflectAvro4kKafkaSerde instance, passing a custom Avro object for advanced configuration like logical types or serializer modules. ```kotlin val avro = Avro { // your custom configuration setLogicalTypeSerializer("my-logical-type", MyLogicalTypeSerializer()) implicitNulls = false } val serde = ReflectAvro4kKafkaSerde( avro = avro, isKey = false, props = mapOf("schema.registry.url" to "http://the-url.com") ) ``` -------------------------------- ### Example of Closing Writer in Finally Block Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-object-container.md Ensures the Avro object container writer is closed properly, even if exceptions occur during the writing process. This pattern is recommended for resource management. ```kotlin val writer = container.openWriter(outputStream) try { for (item in items) { writer.writeValue(item) } } finally { writer.close() } ``` -------------------------------- ### Custom Serializer Using Current Writer Schema Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-encoder-decoder.md Example of a custom serializer that checks the current writer schema to determine how to encode a value. It encodes as a string if the schema type is STRING. ```kotlin object CustomSerializer : AvroSerializer("MyType") { override fun serializeAvro(encoder: AvroEncoder, value: MyType) { val schema = encoder.currentWriterSchema if (schema.type == Schema.Type.STRING) { encoder.encodeString(value.toString()) } } } ``` -------------------------------- ### Apply @AvroAlias Annotation to Data Class Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/annotations.md Example of using @AvroAlias to provide backward compatibility by mapping old field names to new ones. Aliases are not affected by naming strategies. ```kotlin @Serializable @AvroAlias("UserV1", "LegacyUser") data class User( val id: Int, @AvroAlias("full_name", "name") val fullName: String, val email: String ) ``` -------------------------------- ### Extended Avro Instance Configuration Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/configuration.md Configure Avro instances with advanced options including validation, custom logical types, and custom serializers modules. ```kotlin val avro = Avro { fieldNamingStrategy = FieldNamingStrategy.Builtins.SnakeCase implicitNulls = false implicitEmptyCollections = false validateSerialization = true setLogicalTypeSerializer("custom-type", CustomSerializer()) serializersModule = SerializersModule { contextual(CustomTypeSerializer()) } } ``` -------------------------------- ### SerializationException: Fixed size mismatch Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/errors.md Example of a SerializationException caused by a ByteArray size mismatch with a fixed schema size. This occurs when the provided byte array length does not match the size specified in the @AvroFixed annotation. ```kotlin SerializationException: Fixed size mismatch for actual size of 16. Actual schema: fixed(32) ``` -------------------------------- ### Database-Friendly Configuration Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/configuration.md Configuration optimized for database schemas, using SnakeCase for field naming and disabling implicit nulls and empty collections. This requires explicit definition of nullable fields and empty collections. ```kotlin val dbAvro = Avro { fieldNamingStrategy = FieldNamingStrategy.Builtins.SnakeCase implicitNulls = false // Require nullable fields in data implicitEmptyCollections = false // Require explicit empty collections } // Matches typical database schema patterns ``` -------------------------------- ### Set Compression Codec Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-object-container.md Configures the compression codec for the Avro object container file. Choose from available codecs like snappy, deflate, or null (no compression). ```kotlin builder.codec(CodecFactory.snappyCodec()) ``` ```kotlin builder.codec(CodecFactory.deflateCodec(9)) ``` ```kotlin builder.codec(CodecFactory.nullCodec()) ``` -------------------------------- ### Custom Avro Serializer for Logical Type Date Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-serializer.md Example of implementing a custom Avro serializer for a logical type, specifically 'date'. This involves defining the Avro schema with the 'logicalType' property and implementing the serialization and deserialization logic for the date. ```kotlin object DateSerializer : AvroSerializer("date") { override fun getSchema(context: SchemaSupplierContext): Schema { val schema = Schema.create(Schema.Type.INT) schema.addProp("logicalType", "date") return schema } override fun serializeAvro(encoder: AvroEncoder, value: LocalDate) { encoder.encodeInt(value.toEpochDay().toInt()) } override fun deserializeAvro(decoder: AvroDecoder): LocalDate { return LocalDate.ofEpochDay(decoder.decodeInt().toLong()) } } ``` -------------------------------- ### Usage of Generated Classes Source: https://github.com/avro-kotlin/avro4k/blob/main/gradle-plugin/README.md Demonstrates how to use the Kotlin data classes generated by the Avro4k plugin for encoding and decoding byte arrays. ```kotlin Avro.encodeToByteArray(YourGeneratedClass(...)) Avro.decodeFromByteArray(bytes) ``` -------------------------------- ### Custom AvroObjectContainer with Strict Configuration Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/avro-object-container.md Instantiate a custom AvroObjectContainer with strict settings for nulls and empty collections. ```kotlin val strictContainer = AvroObjectContainer { implicitNulls = false implicitEmptyCollections = false } ``` -------------------------------- ### Extending Existing Avro Configuration Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/configuration.md Create a new Avro instance by extending an existing configuration, adding or overriding specific settings. ```kotlin val baseAvro = Avro { fieldNamingStrategy = FieldNamingStrategy.Builtins.SnakeCase } val extendedAvro = Avro(baseAvro) { validateSerialization = true serializersModule = SerializersModule { contextual(AdditionalSerializer()) } } ``` -------------------------------- ### Recovery: Use decodeFromSource for streaming Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/errors.md Illustrates the recovery strategy for 'Not all bytes were consumed' SerializationException by using decodeFromSource for streaming data. ```kotlin // Use decodeFromSource for streaming to avoid this issue val source = Buffer(bytes) val data = Avro.decodeFromSource(source) ``` -------------------------------- ### Serializing BigDecimal as String Source: https://github.com/avro-kotlin/avro4k/blob/main/Migrating-from-v1.md Demonstrates the change in serializing BigDecimal as a string. The previous method used a specific serializer, while the new method utilizes @Contextual and @AvroStringable. ```kotlin // Previously @Serializable data class MyData( @Serializable(with = BigDecimalAsStringSerializer::class) val bigDecimalAsString: BigDecimal, ) // Now @Serializable data class MyData( @Contextual @AvroStringable val bigDecimalAsString: BigDecimal, ) ``` -------------------------------- ### Stream Processing Pattern for Encoding Source: https://github.com/avro-kotlin/avro4k/blob/main/_autodocs/api-reference/extension-functions.md Illustrates how to encode multiple records to a stream using `encodeToSink` and buffered output. Ensure to close the sink and output stream. ```kotlin @Serializable data class LogEntry(val timestamp: Long, val level: String, val message: String) // Write multiple entries val outputFile = FileOutputStream("log.bin") val sink = outputFile.asSink().buffered() entries.forEach { entry -> Avro.encodeToSink(entry, sink) } sink.close() outputFile.close() ```