### Install Avro from Source

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/1.12.0/Getting started (Python)/_index.md

Install the Avro Python library by building from source. This involves downloading the source archive, navigating to the Python directory, and running the setup script.

```shell
tar xvf avro-1.12.0.tar.gz
cd avro-1.12.0
python setup.py install
python
>>> import avro # should not raise ImportError
```

--------------------------------

### Install Avro Python from Source

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/Getting started (Python)/_index.md

Install the Avro Python library by building from source. This involves downloading the source archive, extracting it, and running the setup script.

```shell
tar xvf avro-{{< avro_version >}}.tar.gz
cd avro-{{< avro_version >}}
python setup.py install
python
>>> import avro # should not raise ImportError
```

--------------------------------

### Compile and Run Avro Java Example

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/1.12.0/Getting started (Java)/_index.md

Use these Maven commands to compile the Java example code and then run it. Ensure you are in the 'examples/java-example' directory.

```shell
$ mvn compile
$ mvn -q exec:java -Dexec.mainClass=example.GenericMain
```

--------------------------------

### Run Avro Java Example

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/Getting started (Java)/_index.md

Execute this command to run the generic Avro Java example. Ensure you are in the 'examples/java-example' directory.

```shell
$ mvn -q exec:java -Dexec.mainClass=example.GenericMain
```

--------------------------------

### Install Apache.Avro NuGet Package

Source: https://github.com/apache/avro/blob/main/lang/csharp/README.md

Use this command to install the Apache.Avro package from NuGet into your C# project.

```powershell
Install-Package Apache.Avro
```

--------------------------------

### Complete Example: protocol.avdl

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/IDL Language/_index.md

An example Avro IDL protocol file demonstrating imports, errors, and various method definitions including oneway and keyword usage.

```APIDOC
## Complete Example: protocol.avdl

```java
/*
 * Header with license information.
 */

/**
 * An example protocol in Avro IDL
 */
@namespace("org.apache.avro.test")
protocol Simple {
  // Import the example file above
  import idl "schema.avdl";

  /** Errors are records that can be thrown from a method */
  error TestError {
    string message;
  }

  string hello(string greeting);
  /** Return what was given. Demonstrates the use of backticks to name types/fields/messages/parameters after keywords */
  TestRecord echo(TestRecord `record`);
  int add(int arg1, int arg2);
  bytes echoBytes(bytes data);
  void `error`() throws TestError;
  // The oneway keyword forces the method to return null.
  void ping() oneway;
}
```
```

--------------------------------

### Install Avro Python Package

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/Getting started (Python)/_index.md

Install the Avro Python library from PyPI using pip. This is the recommended method for most users.

```shell
python3 -m pip install avro
```

--------------------------------

### Compile and Run Java Example

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/Getting started (Java)/_index.md

Builds and executes the Avro Java example code using Maven. This includes code generation via the Avro Maven plugin.

```shell
$ mvn compile # includes code generation via Avro Maven plugin
$ mvn -q exec:java -Dexec.mainClass=example.SpecificMain
```

--------------------------------

### Complete Example: schema.avdl

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/IDL Language/_index.md

An example Avro IDL file demonstrating various features including default namespace, schema definition, enum, fixed, record, union, and aliases.

```APIDOC
## Complete Example: schema.avdl

```java
/*
 * Header with license information.
 */
// Optional default namespace (if absent, the default namespace is the null namespace).
namespace org.apache.avro.test;
// Optional main schema definition; if used, the IDL file is equivalent to a .avsc file.
schema TestRecord;

/** Documentation for the enum type Kind */
@aliases(["org.foo.KindOf"])
enum Kind {
  FOO,
  BAR, // the bar enum value
  BAZ
} = FOO; // For schema evolution purposes, unmatched values do not throw an error, but are resolved to FOO.

/** MD5 hash; good enough to avoid most collisions, and smaller than (for example) SHA256. */
fixed MD5(16);

record TestRecord {
  /** Record name; has no intrinsic order */
  string @order("ignore") name;

  Kind @order("descending") kind;

  MD5 hash;

  /*
  Note that 'null' is the first union type. Just like .avsc / .avpr files, the default value must be of the first union type.
  */
  union { null, MD5 } /** Optional field */ @aliases(["hash"]) nullableHash = null;
  // Shorthand syntax; the null in this union is placed based on the default value (or first is there's no default).
  MD5? anotherNullableHash = null;

  array<long> arrayOfLongs;
}
```
```

--------------------------------

### Initialize Submodules

Source: https://github.com/apache/avro/blob/main/doc/README.md

Run this command to get and update all necessary Hugo modules before building the website.

```sh
hugo mod get -u
```

--------------------------------

### Compile Avro Java Example

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/Getting started (Java)/_index.md

Use this command to compile the Java example code within the Avro documentation's Maven project.

```shell
$ mvn compile
```

--------------------------------

### Include Directories Setup

Source: https://github.com/apache/avro/blob/main/lang/c++/CMakeLists.txt

Sets up include directories for the Avro library. It includes the 'include/avro' directory from the source and the binary directory for generated files.

```cmake
include_directories (include/avro ${CMAKE_CURRENT_BINARY_DIR})
```

--------------------------------

### Include Avro Autoloader (Manual Install)

Source: https://github.com/apache/avro/blob/main/lang/php/README.md

Require the autoload.php file after manually installing the Avro PHP library.

```php
<?php
require_once('avro-php/autoload.php');
```

--------------------------------

### Install Avro Python from Source (Editable)

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/Getting started (Python)/_index.md

Install the Avro Python library from source in editable mode. This is useful for development and allows changes to the source code to be reflected immediately.

```shell
cd lang/py/
python3 -m pip install -e .
python
```

--------------------------------

### Install Avro-js

Source: https://github.com/apache/avro/blob/main/lang/js/README.md

Install the avro-js package using npm. This command is used in a Node.js environment.

```bash
$ npm install avro-js
```

--------------------------------

### Build and Run Docker Container Simultaneously

Source: https://github.com/apache/avro/blob/main/doc/content/en/project/How to contribute/_index.md

A shortcut to build the Docker image and start the Avro website container in one command.

```shell
docker-compose up --build
```

--------------------------------

### Example User JSON Output

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/MapReduce guide/_index.md

Example JSON representations of Users generated by the GenerateData class.

```json
{"name": "user", "favorite_number": null, "favorite_color": "red"}
{"name": "user", "favorite_number": null, "favorite_color": "green"}
{"name": "user", "favorite_number": null, "favorite_color": "purple"}
{"name": "user", "favorite_number": null, "favorite_color": null}
...
```

--------------------------------

### Add CMake Test

Source: https://github.com/apache/avro/blob/main/lang/c/examples/CMakeLists.txt

Configures a test named 'quickstop' that changes directory and then executes the compiled program. This is useful for verifying example functionality.

```cmake
add_test(quickstop
    ${CMAKE_COMMAND} -E chdir ${AvroC_SOURCE_DIR}/examples
    ${exec_name}
)
```

--------------------------------

### Install ReportGenerator Tool

Source: https://github.com/apache/avro/blob/main/lang/csharp/CODECOVERAGE.md

Installs the ReportGenerator tool globally using the .NET CLI. Ensure you are using the specified version or check Nuget for the latest.

```bash
dotnet tool install --global dotnet-reportgenerator-globaltool --version 5.1.4 --add-source https://www.nuget.org/packages/
```

--------------------------------

### Avro Datum Reference Counting Example

Source: https://github.com/apache/avro/blob/main/lang/c/docs/index.txt

Illustrates reference counting for Avro string datums. Shows how to create a string datum and correctly dereference it.

```c
avro_datum_t string = avro_string("This is my string");

...
avro_datum_decref(string);
```

--------------------------------

### Example Grammar and LL(1) Parsing Table

Source: https://github.com/apache/avro/blob/main/lang/java/avro/src/main/java/org/apache/avro/io/parsing/doc-files/parsing.html

Illustrates an Avro schema for an array of ints and its corresponding traditional LL(1) parsing table.

```plaintext
n0 ::= `arraystart` rint `arrayend`
rint ::= int rint | ε
```

```plaintext
T(n0,`arraystart`) = n0 ::= `arraystart` rint `arrayend`
T(rint,int) = rint ::= int rint
T(rint,`arrayend`) = ε
T(A,y) = Error -- if (A,y) is none of the above
```

--------------------------------

### Add Subdirectories for Build

Source: https://github.com/apache/avro/blob/main/lang/c/CMakeLists.txt

Adds subdirectories for source, examples, tests, and documentation to the build process.

```cmake
add_subdirectory(src)
add_subdirectory(examples)
add_subdirectory(tests)
add_subdirectory(docs)
```

--------------------------------

### Serve Hugo Site Locally

Source: https://github.com/apache/avro/blob/main/doc/content/en/project/How to contribute/_index.md

Starts a local Hugo server for development. Changes to .md and .html files in the content/ folder will be reflected in the browser after saving.

```shell
hugo server --navigateToChanged
```

--------------------------------

### Install Avro PHP via Composer

Source: https://github.com/apache/avro/blob/main/lang/php/README.md

Add the apache/avro package to your project's composer.json for development.

```json
{
    "require-dev": {
        "apache/avro": "dev-main"
    }
}
```

--------------------------------

### Naming: Instance fields

Source: https://github.com/apache/avro/blob/main/lang/csharp/STYLING.md

Instance fields should be named using camelCase and start with an underscore '_'.

```csharp
private int _myInt;

internal string _myString;
```

--------------------------------

### String Encoding Example

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/Specification/_index.md

Demonstrates the binary encoding of a three-character string 'foo'. It includes the length encoded as a long followed by the UTF-8 bytes of the string.

```text
06 66 6f 6f
```

--------------------------------

### Enable Faster Code Generation (Beta)

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/Getting started (Java)/_index.md

Runs the Avro Java example with the beta feature for faster decoding and encoding enabled. This is controlled by a system property at runtime.

```shell
$ mvn -q exec:java -Dexec.mainClass=example.SpecificMain \
    -Dorg.apache.avro.specific.use_custom_coders=true
```

--------------------------------

### Serve Website Locally with Docker

Source: https://github.com/apache/avro/blob/main/doc/README.md

Preview the website locally without installing Hugo by using a Docker container. Mounts the current directory and maps port 1313.

```sh
# You can do the same thing without installing hugo via docker.
# From the Avro root directory:
docker run --rm -v $(pwd):/src -p 1313:1313 jakejarvis/hugo-extended:latest --source doc/ server \
    --buildDrafts --buildFuture --bind 0.0.0.0 --navigateToChanged
```

--------------------------------

### Union Encoding Example (String)

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/Specification/_index.md

Demonstrates the binary encoding for the string 'a' within a union schema of ['null', 'string']. It includes the union index for string (1) and the encoded string.

```text
02 02 61
```

--------------------------------

### Run Ruby Unit Tests

Source: https://github.com/apache/avro/blob/main/doc/content/en/project/How to contribute/_index.md

Navigate to the Ruby module directory, install the 'echoe' gem, and then run rake to clean and test.

```shell
cd avro-trunk/lang/ruby
gem install echoe
rake clean test
```

--------------------------------

### Build Executable and Link Library

Source: https://github.com/apache/avro/blob/main/lang/c/examples/CMakeLists.txt

Defines an executable target and links it against a static library. This is a common setup for C projects using CMake.

```cmake
add_executable(quickstop quickstop.c)
target_link_libraries(quickstop avro-static)
```

--------------------------------

### Get Bytes Value Length

Source: https://github.com/apache/avro/blob/main/lang/c/docs/index.txt

Example demonstrating how to get the size of a bytes Avro value when only the size is needed, by passing NULL for the destination pointer.

```c
avro_value_t  *value = /* from somewhere */;
size_t  size;
avro_value_get_bytes(value, NULL, &size);
```

--------------------------------

### Troubleshooting Hugo Build Error

Source: https://github.com/apache/avro/blob/main/doc/content/en/project/How to contribute/_index.md

Example of a common error encountered when the extended version of Hugo is not installed. This indicates a problem with SCSS transformation.

```shell
➜ hugo server

INFO 2021/01/21 21:07:55 Using config file: 
Building sites … INFO 2021/01/21 21:07:55 syncing static files to /
Built in 288 ms
Error: Error building site: TOCSS: failed to transform "scss/main.scss" (text/x-scss): resource "scss/scss/main.scss_9fadf33d895a46083cdd64396b57ef68" not found in file cache
```

--------------------------------

### Union Encoding Example

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/1.12.0/Specification/_index.md

Demonstrates the binary encoding of a union. It starts with an int indicating the schema's position in the union, followed by the value encoded per that schema.

```json
["null","string"]
```

```text
00
```

```text
02 02 61
```

--------------------------------

### Run Python Unit Tests

Source: https://github.com/apache/avro/blob/main/doc/content/en/project/How to contribute/_index.md

Navigate to the Python module directory and use setup.py to build and run tests.

```shell
cd avro-trunk/lang/py
./setup.py build test
```

--------------------------------

### Build and Test All Languages

Source: https://github.com/apache/avro/blob/main/BUILD.md

Run the './build.sh test' command to execute all language-specific tests. This is a primary command for verifying the build.

```bash
./build.sh test

```

--------------------------------

### Run C Unit Tests

Source: https://github.com/apache/avro/blob/main/doc/content/en/project/How to contribute/_index.md

Navigate to the C module directory and execute the build script to clean and then test.

```shell
cd avro-trunk/lang/c
./build.sh clean
./build.sh test
```

--------------------------------

### Avro IDL Protocol Example

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/1.12.0/IDL Language/_index.md

An example of an Avro IDL protocol definition (`.avdl` file) that imports a schema and defines RPC messages.

```APIDOC
## protocol protocol.avdl

### Description
This protocol defines RPC services using Avro IDL. It imports a previously defined schema (`schema.avdl`) and declares messages for various operations, including a custom error type.

### Protocol Definition
```java
/*
 * Header with license information.
 */

/**
 * An example protocol in Avro IDL
 */
@namespace("org.apache.avro.test")
protocol Simple {
  // Import the example file above
  import idl "schema.avdl";

  /** Errors are records that can be thrown from a method */
  error TestError {
    string message;
  }

  string hello(string greeting);
  /** Return what was given. Demonstrates the use of backticks to name types/fields/messages/parameters after keywords */
  TestRecord echo(TestRecord `record`);
  int add(int arg1, int arg2);
  bytes echoBytes(bytes data);
  void `error`() throws TestError;
  // The oneway keyword forces the method to return null.
  void ping() oneway;
}
```
```

--------------------------------

### Create Release Distribution Files

Source: https://github.com/apache/avro/blob/main/BUILD.md

Use the './build.sh dist' command to generate all release distribution files. These files are typically placed in the 'dist/' directory.

```bash
./build.sh dist

```

--------------------------------

### Avro Schema Example

Source: https://github.com/apache/avro/blob/main/lang/java/avro/src/main/java/org/apache/avro/io/parsing/doc-files/parsing.html

An example Avro schema defining a record with fields including a double, an array of strings, and a map with varied value types.

```json
{
  "type":"record", "name":"foo",
  "fields":[
    {"name":"bar","type":"double"},
    {"name":"baz","type":{"type":"array", "items":"string"}},
    {"name":"zip",
     "type":{"type":"map",
     "values":["null",{"type":"array", "items":"bytes"},"foo"]
    }},
  ]
}
```

--------------------------------

### Include Avro Autoloader (From Source)

Source: https://github.com/apache/avro/blob/main/lang/php/README.md

If using from source, add 'lib/' to your include path and require 'lib/avro.php'.

```php
<?php
require_once('lib/autoload.php');
```

--------------------------------

### Avro IDL Schema Example

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/1.12.0/IDL Language/_index.md

An example of an Avro IDL schema definition (`.avdl` file) demonstrating various features like enums, fixed types, records, unions, and annotations.

```APIDOC
## schema schema.avdl

### Description
This schema defines a `TestRecord` with various fields including a string, an enum, a fixed type, unions, and an array. It also demonstrates the use of annotations for ordering, aliases, and documentation.

### Schema Definition
```java
/*
 * Header with license information.
 */
// Optional default namespace (if absent, the default namespace is the null namespace).
namespace org.apache.avro.test;
// Optional main schema definition; if used, the IDL file is equivalent to a .avsc file.
schema TestRecord;

/** Documentation for the enum type Kind */
@aliases(["org.foo.KindOf"])
enum Kind {
  FOO,
  BAR, // the bar enum value
  BAZ
} = FOO; // For schema evolution purposes, unmatched values do not throw an error, but are resolved to FOO.

/** MD5 hash; good enough to avoid most collisions, and smaller than (for example) SHA256. */
fixed MD5(16);

record TestRecord {
  /** Record name; has no intrinsic order */
  string @order("ignore") name;

  Kind @order("descending") kind;

  MD5 hash;

  /*
  Note that 'null' is the first union type. Just like .avsc / .avpr files, the default value must be of the first union type.
  */
  union { null, MD5 } /** Optional field */ @aliases(["hash"]) nullableHash = null;
  // Shorthand syntax; the null in this union is placed based on the default value (or first is there's no default).
  MD5? anotherNullableHash = null;

  array<long> arrayOfLongs;
}
```
```

--------------------------------

### Run PHP Unit Tests

Source: https://github.com/apache/avro/blob/main/doc/content/en/project/How to contribute/_index.md

Navigate to the PHP module directory and execute the build script to clean and then test.

```shell
cd avro-trunk/lang/php
./build.sh clean
./build.sh test
```

--------------------------------

### JavaScript Number Precision Issue Example

Source: https://github.com/apache/avro/blob/main/lang/js/doc/Advanced-usage.md

Demonstrates how JavaScript's internal number representation can lead to precision loss with large integers. This example shows that parsing a large number string results in an incorrect value.

```javascript
Number.parseInt('9007199254740995') === 9007199254740996 // true
```

--------------------------------

### Create Persistent RPC Server with Node Net

Source: https://github.com/apache/avro/blob/main/lang/js/doc/Advanced-usage.md

Sets up an RPC server using Node.js net module to listen for incoming persistent stream connections.

```javascript
var net = require('net');

net.createServer()
  .on('connection', function (con) { protocol.createListener(con); })
  .listen(8000);
```

--------------------------------

### Deserialized User Output

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/1.12.0/Getting started (Java)/_index.md

Example output of deserialized Avro records.

```json
{"name": "Alyssa", "favorite_number": 256, "favorite_color": null}
{"name": "Ben", "favorite_number": 7, "favorite_color": "red"}
```

--------------------------------

### Example: Working with Generic Avro Long Values

Source: https://github.com/apache/avro/blob/main/lang/c/docs/index.txt

Demonstrates creating a generic Avro value implementation for a long schema, creating and setting values, and freeing the resources. This snippet shows how to generate Avro longs within a specified range.

```c
avro_schema_t  schema = avro_schema_long();
avro_value_iface_t  *iface = avro_generic_class_from_schema(schema);

avro_value_t  val;
avro_generic_value_new(iface, &val);

/* Generate Avro longs from 0-499 */
int  i;
for (i = 0; i < 500; i++) {
    avro_value_reset(&val);
    avro_value_set_long(&val, i);
    /* do something with the value */
}

avro_generic_value_free(&val);
avro_value_iface_decref(iface);
avro_schema_decref(schema);

```

--------------------------------

### C struct initialization on Win32

Source: https://github.com/apache/avro/blob/main/lang/c/README.maintaining_win32.txt

Illustrates the correct and incorrect syntax for initializing C structs on Win32, highlighting the limitation of '.element' notation.

```c
typedef struct
{
   int a;
   int b;
} test_t;

test_t t1 = { 0, 0 };

test_t t2 = { .a = 0, . b = 0 };
```

--------------------------------

### MapType Methods

Source: https://github.com/apache/avro/blob/main/lang/js/doc/API.md

Provides a method to get the type of the values within a map.

```APIDOC
## MapType

### Description
Represents a map type in Avro.

### Methods
#### `getValuesType()`

- **Description**: Returns the type of the map's values.
- **Returns**: The type of the map values.
```

--------------------------------

### Naming: Static fields

Source: https://github.com/apache/avro/blob/main/lang/csharp/STYLING.md

Static fields should be named using camelCase and start with 's_'.

```csharp
private static int s_myInt;
```

--------------------------------

### Avro Schema with Named Types and Namespaces

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/1.12.0/Specification/_index.md

Demonstrates how to define records, enums, and fixed types with explicit and inherited namespaces, illustrating the construction of fullnames.

```json
{
  "type": "record",
  "name": "Example",
  "doc": "A simple name (attribute) and no namespace attribute: use the null namespace (\")", the fullname is 'Example'.",
  "fields": [
    {
      "name": "inheritNull",
      "type": {
        "type": "enum",
        "name": "Simple",
        "doc": "A simple name (attribute) and no namespace attribute: inherit the null namespace of the enclosing type 'Example'. The fullname is 'Simple'.",
        "symbols": ["a", "b"]
      }
    }, {
      "name": "explicitNamespace",
      "type": {
        "type": "fixed",
        "name": "Simple",
        "namespace": "explicit",
        "doc": "A simple name (attribute) and a namespace (attribute); the fullname is 'explicit.Simple' (this is a different type than of the 'inheritNull' field).",
        "size": 12
      }
    }, {
      "name": "fullName",
      "type": {
        "type": "record",
        "name": "a.full.Name",
        "namespace": "ignored",
        "doc": "A name attribute with a fullname, so the namespace attribute is ignored. The fullname is 'a.full.Name', and the namespace is 'a.full'.",
        "fields": [
          {
            "name": "inheritNamespace",
            "type": {
              "type": "enum",
              "name": "Understanding",
              "doc": "A simple name (attribute) and no namespace attribute: inherit the namespace of the enclosing type 'a.full.Name'. The fullname is 'a.full.Understanding'.",
              "symbols": ["d", "e"]
            }
          }
        ]
      }
    }
  ]
}
```

--------------------------------

### Custom TypedFieldConverter Implementation

Source: https://github.com/apache/avro/blob/main/lang/csharp/src/apache/main/Reflect/README.md

Example implementation of a TypedFieldConverter for converting between byte[] and GenericFixed Avro types.

```csharp
        public class GenericFixedConverter : TypedFieldConverter<byte[],GenericFixed>
        {
            public override GenericFixed From(byte[] o, Schema s)
            {
                return new GenericFixed(s as FixedSchema, o);
            }

            public override byte[] To(GenericFixed o, Schema s)
            {
                return o.Value;
            }
        }
```

--------------------------------

### Copy C API Documentation

Source: https://github.com/apache/avro/blob/main/doc/content/en/project/How to contribute/_index.md

Copies generated C API documentation from the build output to the website's static content folder.

```shell
cp ../avro/build/avro-doc-1.12.0-SNAPSHOT/api/c/* content/en/docs/external/c/
```

--------------------------------

### ArrayType Methods

Source: https://github.com/apache/avro/blob/main/lang/js/doc/API.md

Provides methods to get information about an array type, specifically the type of its items.

```APIDOC
## ArrayType

### Description
Represents an array type in Avro.

### Methods
#### `getItemsType()`

- **Description**: Returns the type of the array's items.
- **Returns**: The type of the array items.
```

--------------------------------

### Verify PGP Signatures with GPG

Source: https://github.com/apache/avro/blob/main/doc/content/en/project/Download/_index.md

Use GPG to import the KEYS file and verify the signature of a downloaded file. Ensure KEYS and the .asc file are from the main distribution directory.

```shell
% gpg --import KEYS
% gpg --verify downloaded_file.asc downloaded_file
```

--------------------------------

### Generated Avro Grammar

Source: https://github.com/apache/avro/blob/main/lang/java/avro/src/main/java/org/apache/avro/io/parsing/doc-files/parsing.html

The LL(1) grammar derived from the example Avro schema, showing non-terminals and their productions.

```plaintext
  n0 ::= double n1 n2
  r1 ::= string r1 | ε
  n1 ::= arraystart r1 arrayend
  r2 ::= string n3 r2 | ε
  n2 ::= mapstart r2 mapend
  u3 ::= 1 null | 2 n4 | 3 n0
  n3 ::= union u3
  r4 ::= bytes r4 | ε
  n4 ::= arraystart r4 arrayend

The symbol "n0" is the start-symbol for this grammar.
```

--------------------------------

### ColorCount Output JSON

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/MapReduce guide/_index.md

Example JSON output from the ColorCount MapReduce job, showing color counts.

```json
{"value": 3, "key": "blue"}
{"value": 7, "key": "green"}
{"value": 1, "key": "none"}
{"value": 2, "key": "orange"}
{"value": 3, "key": "purple"}
{"value": 2, "key": "red"}
{"value": 2, "key": "yellow"}
```

--------------------------------

### Find Documentation Tools

Source: https://github.com/apache/avro/blob/main/lang/c/docs/CMakeLists.txt

Searches for the 'asciidoc' and 'source-highlight' executables required for HTML documentation generation. If not found, a warning is issued.

```cmake
# TODO(dln): Use FindAsciidoc script instead.
message(STATUS "Searching for asciidoc...")
find_program(ASCIIDOC_EXECUTABLE asciidoc)
find_program(SOURCE_HIGHLIGHT_EXECUTABLE source-highlight)
```

--------------------------------

### FixedType Methods

Source: https://github.com/apache/avro/blob/main/lang/js/doc/API.md

Provides methods to get optional type aliases and the size in bytes for a fixed type.

```APIDOC
## FixedType

### Description
Represents a fixed type in Avro.

### Methods
#### `getAliases()`

- **Description**: Returns optional type aliases.
- **Returns**: An object containing type aliases.

#### `getSize()`

- **Description**: Returns the size in bytes of instances of this type.
- **Returns**: The size in bytes.
```

--------------------------------

### LogicalType Methods

Source: https://github.com/apache/avro/blob/main/lang/js/doc/API.md

Provides methods to get the underlying Avro type and implement custom logical type conversions.

```APIDOC
## LogicalType

### Description
Abstract class used to implement custom native types in Avro.

### Methods
#### `getUnderlyingType()`

- **Description**: Gets the underlying Avro type.
- **Returns**: The underlying Avro type.

#### `_fromValue(val)`

- **Description**: Converts a deserialized value to the final, wrapped value.
- **Parameters**:
  - `val` (...): A value deserialized by the underlying type.
- **Returns**: The wrapped value.

#### `_toValue(any)`

- **Description**: Converts a wrapped value to a value that can be serialized by the underlying type.
- **Parameters**:
  - `any` (...): A wrapped value.
- **Returns**: A serializable value.

#### `_resolve(type)`

- **Description**: Resolves the writer's type to convert its values to the current type's wrapped value.
- **Parameters**:
  - `type` ({Type}): The writer's type.
- **Returns**: `undefined` if conversion is not possible, otherwise a conversion function.
```

--------------------------------

### ColorCount Example (New API)

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/MapReduce guide/_index.md

Implements a MapReduce job using the newer `org.apache.hadoop.mapreduce` API for counting colors from Avro User records. Requires Avro and Hadoop mapreduce libraries.

```java
package example;

import java.io.IOException;

import org.apache.avro.Schema;
import org.apache.avro.mapred.AvroKey;
import org.apache.avro.mapred.AvroValue;
import org.apache.avro.mapreduce.AvroJob;
import org.apache.avro.mapreduce.AvroKeyInputFormat;
import org.apache.avro.mapreduce.AvroKeyValueOutputFormat;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.NullWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;

import example.avro.User;

public class MapReduceColorCount extends Configured implements Tool {

  public static class ColorCountMapper extends
      Mapper<AvroKey<User>, NullWritable, Text, IntWritable> {

    @Override
    public void map(AvroKey<User> key, NullWritable value, Context context)
        throws IOException, InterruptedException {

      CharSequence color = key.datum().getFavoriteColor();
      if (color == null) {
        color = "none";
      }
      context.write(new Text(color.toString()), new IntWritable(1));
    }
  }

  public static class ColorCountReducer extends
      Reducer<Text, IntWritable, AvroKey<CharSequence>, AvroValue<Integer>> {

    @Override
    public void reduce(Text key, Iterable<IntWritable> values,
        Context context) throws IOException, InterruptedException {

      int sum = 0;
      for (IntWritable value : values) {
        sum += value.get();
      }
      context.write(new AvroKey<CharSequence>(key.toString()), new AvroValue<Integer>(sum));
    }
  }


```

--------------------------------

### RecordType Methods

Source: https://github.com/apache/avro/blob/main/lang/js/doc/API.md

Provides methods to get optional type aliases, fields, and the record constructor for a record type.

```APIDOC
## RecordType

### Description
Represents a record type in Avro.

### Methods
#### `getAliases()`

- **Description**: Returns optional type aliases.
- **Returns**: An object containing type aliases.

#### `getFields()`

- **Description**: Returns a copy of the array of fields contained in this record.
- **Returns**: An array of field objects, each with `getAliases()`, `getDefault()`, `getName()`, `getOrder()`, and `getType()` methods.

#### `getRecordConstructor()`

- **Description**: Returns the `Record` constructor for instances of this type.
- **Returns**: The `Record` constructor.
```

--------------------------------

### Enable Snappy Codec with ZLIB

Source: https://github.com/apache/avro/blob/main/lang/c/CMakeLists.txt

Checks for Snappy and ZLIB libraries to enable the Snappy codec. Includes Snappy headers and sets variables.

```cmake
find_package(Snappy)
if (SNAPPY_FOUND AND ZLIB_FOUND)  # Snappy borrows crc32 from zlib
    set(SNAPPY_PKG snappy)
    add_definitions(-DSNAPPY_CODEC)
    include_directories(${SNAPPY_INCLUDE_DIRS})
    message("Enabled snappy codec")
else (SNAPPY_FOUND AND ZLIB_FOUND)
    set(SNAPPY_PKG "")
    set(SNAPPY_LIBRARIES "")
    message("Disabled snappy codec. libsnappy not found or zlib not found.")
endif (SNAPPY_FOUND AND ZLIB_FOUND)
```

--------------------------------

### Namespace matches folder structure

Source: https://github.com/apache/avro/blob/main/lang/csharp/STYLING.md

Determines if the namespace should match the folder structure. The example shows both true and false configurations.

```csharp
// dotnet_style_namespace_match_folder = true
// file path: Example/Convention/C.cs
using System;

namespace Example.Convention
{
    class C
    {
    }
}

// dotnet_style_namespace_match_folder = false
// file path: Example/Convention/C.cs
using System;

namespace Example
{
    class C
    {
    }
}
```

--------------------------------

### Parsing Avro Schema from File

Source: https://github.com/apache/avro/blob/main/lang/js/doc/Home.md

Shows how to load an Avro schema directly from a file path using avro.parse.

```javascript
var couponType = avro.parse('./Coupon.avsc');
```

--------------------------------

### Record Encoding Example

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/Specification/_index.md

Shows the binary encoding of a record with 'long' and 'string' fields. The encoding is a concatenation of the field values.

```json
{
  "type": "record",
  "name": "test",
  "fields" : [
    {"name": "a", "type": "long"},
    {"name": "b", "type": "string"}
  ]
}
```

```text
36 06 66 6f 6f
```

--------------------------------

### Avro Union Schema Example

Source: https://github.com/apache/avro/blob/main/lang/csharp/src/apache/main/Reflect/README.md

Represents a union in Avro schema, allowing a field to be of multiple types, including null.

```json
["null", { "type": "record", "name": "X"}]
```

--------------------------------

### Create Transient RPC Server with Express

Source: https://github.com/apache/avro/blob/main/lang/js/doc/Advanced-usage.md

Sets up an RPC server using the Express framework to handle incoming transient stream requests.

```javascript
var app = require('express')();

app.post('/', function (req, res) {
  protocol.createListener(function (cb) { cb(res); return req; });
});

app.listen(3000);
```

--------------------------------

### Sort System.* using directives first

Source: https://github.com/apache/avro/blob/main/lang/csharp/STYLING.md

Ensures that System.* using directives are sorted alphabetically and placed before other using directives.

```csharp
using System.Collections.Generic;
using System.Threading.Tasks;
using Avro;
```

--------------------------------

### Access Avro File Metadata

Source: https://github.com/apache/avro/blob/main/lang/js/doc/Home.md

Listen for the 'metadata' event on the file decoder stream to get the record type and codec.

```javascript
personStream.on('metadata', function (type, codec) { /* Something useful. */ });
```

--------------------------------

### Run Avro Website in Docker Container

Source: https://github.com/apache/avro/blob/main/doc/content/en/project/How to contribute/_index.md

Starts the Avro website within a Docker container. Access it at http://localhost:1313.

```shell
docker-compose up
```

--------------------------------

### Create Avro User Objects in Java

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/Getting started (Java)/_index.md

Demonstrates creating Avro User objects using direct constructors, an alternate constructor, and the builder pattern. Builders automatically handle default values and perform data validation.

```java
User user1 = new User();
user1.setName("Alyssa");
user1.setFavoriteNumber(256);
// Leave favorite color null

// Alternate constructor
User user2 = new User("Ben", 7, "red");

// Construct via builder
User user3 = User.newBuilder()
             .setName("Charlie")
             .setFavoriteColor("blue")
             .setFavoriteNumber(null)
             .build();
```

--------------------------------

### Using Class Cache with Reflect

Source: https://github.com/apache/avro/blob/main/lang/csharp/src/apache/main/Reflect/README.md

Demonstrates creating and reusing a ClassCache with ReflectWriter and ReflectReader to improve performance by caching reflection details.

```csharp
    var cache = new ClassCache();
    var writer = new ReflectWriter<MultiList>(schema, cache);
    var reader = new ReflectReader<MultiList>(schema, schema, cache);
```

--------------------------------

### Clean Generated Artifacts

Source: https://github.com/apache/avro/blob/main/BUILD.md

Execute './build.sh clean' to remove all generated build artifacts. This is useful for starting a fresh build.

```bash
./build.sh clean

```

--------------------------------

### ColorCount Example (Old API)

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/MapReduce guide/_index.md

Implements a MapReduce job using the older `org.apache.hadoop.mapred` API for counting colors from Avro User records. Requires Avro and Hadoop mapred libraries.

```java
package example;

import java.io.IOException;

import org.apache.avro.*;
import org.apache.avro.Schema.Type;
import org.apache.avro.mapred.*;
import org.apache.hadoop.conf.*;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.mapred.*;
import org.apache.hadoop.util.*;

import example.avro.User;

public class MapredColorCount extends Configured implements Tool {

  public static class ColorCountMapper extends AvroMapper<User, Pair<CharSequence, Integer>> {
    @Override
    public void map(User user, AvroCollector<Pair<CharSequence, Integer>> collector, Reporter reporter)
        throws IOException {
      CharSequence color = user.getFavoriteColor();
      // We need this check because the User.favorite_color field has type ["string", "null"]
      if (color == null) {
        color = "none";
      }
      collector.collect(new Pair<CharSequence, Integer>(color, 1));
    }
  }

  public static class ColorCountReducer extends AvroReducer<CharSequence, Integer,
                                                            Pair<CharSequence, Integer>> {
    @Override
    public void reduce(CharSequence key, Iterable<Integer> values,
                       AvroCollector<Pair<CharSequence, Integer>> collector,
                       Reporter reporter)
        throws IOException {
      int sum = 0;
      for (Integer value : values) {
        sum += value;
      }
      collector.collect(new Pair<CharSequence, Integer>(key, sum));
    }
  }

  public int run(String[] args) throws Exception {
    if (args.length != 2) {
      System.err.println("Usage: MapredColorCount <input path> <output path>");
      return -1;
    }

    JobConf conf = new JobConf(getConf(), MapredColorCount.class);
    conf.setJobName("colorcount");

    FileInputFormat.setInputPaths(conf, new Path(args[0]));
    FileOutputFormat.setOutputPath(conf, new Path(args[1]));

    AvroJob.setMapperClass(conf, ColorCountMapper.class);
    AvroJob.setReducerClass(conf, ColorCountReducer.class);

    // Note that AvroJob.setInputSchema and AvroJob.setOutputSchema set
    // relevant config options such as input/output format, map output
    // classes, and output key class.
    AvroJob.setInputSchema(conf, User.getClassSchema());
    AvroJob.setOutputSchema(conf, Pair.getPairSchema(Schema.create(Type.STRING),
        Schema.create(Type.INT)));

    JobClient.runJob(conf);
    return 0;
  }

  public static void main(String[] args) throws Exception {
    int res = ToolRunner.run(new Configuration(), new MapredColorCount(), args);
    System.exit(res);
  }
}

```

--------------------------------

### AvroReducer Implementation (mapreduce API)

Source: https://github.com/apache/avro/blob/main/doc/content/en/docs/++version++/MapReduce guide/_index.md

Example of an AvroReducer for the newer mapreduce API. It extends the standard Reducer and uses AvroKey/AvroValue for input/output.

```java
public static class ColorCountReducer extends
      Reducer<Text, IntWritable, AvroKey<CharSequence>, AvroValue<Integer>> {

    @Override
    public void reduce(Text key, Iterable<IntWritable> values,
        Context context) throws IOException, InterruptedException {

      int sum = 0;
      for (IntWritable value : values) {
        sum += value.get();
      }
      context.write(new AvroKey<CharSequence>(key.toString()), new AvroValue<Integer>(sum));
    }
  }
```

--------------------------------

### Run C++ Unit Tests

Source: https://github.com/apache/avro/blob/main/doc/content/en/project/How to contribute/_index.md

Navigate to the C++ module directory and execute the build script to clean and test.

```shell
cd avro-trunk/lang/c++
./build.sh clean test
```