### Run Scio Examples Locally

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Getting-Started.md

Executes Scio example pipelines using the DirectRunner and local filesystem. Pipelines are run via SBT, specifying the example main class and input/output parameters.

```bash
neville@localhost scio $ sbt
[info] ...
> project scio-examples
[info] ...
> runMain com.spotify.scio.examples.WordCount --input=<FILE PATTERN> --output=<DIRECTORY>
```

--------------------------------

### Run Scio Examples on Dataflow

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Getting-Started.md

Executes Scio example pipelines on Google Cloud Dataflow. This involves setting the DataflowRunner in SBT and providing GCP project and region details.

```bash
neville@localhost scio $ sbt
[info] ...
> project scio-examples
[info] ...
> set beamRunners := "DataflowRunner"
[info] ...
> runMain com.spotify.scio.examples.WordCount
--project=<PROJECT ID>
--region=<GCE AVAILABILITY ZONE> --runner=DataflowRunner
--input=<FILE PATTERN> --output=<DIRECTORY>
```

--------------------------------

### Compile Scio Project

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Getting-Started.md

Ensures the Scio project loads and builds successfully by compiling both the main and test sources using SBT.

```sbt
sbt compile Test/compile
```

--------------------------------

### Build Scio with SBT

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Getting-Started.md

Clones the Scio repository and builds the project using SBT. It includes options for cross-building across Scala versions and publishing local artifacts.

```bash
git clone git@github.com:spotify/scio.git
cd scio
# 'sbt +publishLocal' to cross build for all Scala versions
# 'sbt ++$SCALA_VERSION publishLocal' to build for a specific Scala version
sbt publishLocal
```

--------------------------------

### Authenticate Google Cloud

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Getting-Started.md

Sets up Google Cloud's application default credentials for Scio to access services like BigQuery. This command initiates an interactive login process.

```bash
gcloud auth application-default login
```

--------------------------------

### BigQuery Dataflow Pipeline Setup

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Scio-REPL.md

Illustrates starting the Scio REPL with Dataflow runner options and specifying a BigQuery project ID for processing data from BigQuery.

```shell
$ java -jar -Dbigquery.project=<project-id> scio-repl-0.7.0.jar \
> --project=<project-id> \
> --stagingLocation=<staging-dir> \
> --tempLocation=<temp-dir> \
> --runner=DataflowRunner
```

--------------------------------

### Dataflow Service Pipeline Setup

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Scio-REPL.md

Demonstrates how to start the Scio REPL with options for running pipelines on Google Cloud Dataflow. It includes necessary command-line arguments for project, staging, and temporary locations.

```shell
$ java -jar scio-repl-0.7.0.jar \
> --project=<project-id> \
> --stagingLocation=<staging-dir> \
> --tempLocation=<temp-dir> \
> --runner=DataflowRunner
```

--------------------------------

### Add Scio Dependencies to build.sbt

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Getting-Started.md

Configures SBT project dependencies for Scio, including core, testing, and specific Apache Beam runners. It also shows how to add the paradise compiler plugin for type-safe BigQuery API.

```scala
libraryDependencies ++= Seq(
  "com.spotify" %% "scio-core" % scioVersion,
  "com.spotify" %% "scio-test" % scioVersion % Test,
  "org.apache.beam" % "beam-runners-direct-java" % beamVersion % Runtime,
  "org.apache.beam" % "beam-runners-google-cloud-dataflow-java" % beamVersion % Runtime
)

addCompilerPlugin("org.scalamacros" % "paradise" % "2.1.1" cross CrossVersion.full)
```

--------------------------------

### Configure BigQuery Project in SBT

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Getting-Started.md

Sets the Google Cloud project ID for BigQuery requests during compile time via SBT system properties. It also shows how to specify a JSON secret file for credentials.

```bash
sbt -Dbigquery.project=<PROJECT-ID>
sbt -Dbigquery.secret=secret.json
```

--------------------------------

### Scio Dataflow Job Configuration Options

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Getting-Started.md

Configuration parameters for running Apache Beam pipelines using Scio on Google Cloud Dataflow. These options control project settings, worker resources, network configurations, and experimental features to optimize job execution and performance.

```APIDOC
DataflowJobConfiguration:
  --project: The project ID for your Google Cloud Project. Required for running pipelines on the Cloud Dataflow managed service.
  --region: The Compute Engine regional endpoint for launching worker instances. See https://cloud.google.com/dataflow/docs/resources/locations for a list of regions.
  --workerMachineType: Specifies the machine type for worker instances. Start with smaller types like 'n1-standard-1' and increase if memory issues arise. 'n1-standard-4' is suitable for memory-intensive jobs.
  --maxNumWorkers: Controls the maximum number of worker instances. Avoid setting this too high (e.g., 1000) to prevent resource contention and manage costs.
  --diskSizeGb: The size of the boot disk for worker instances in gigabytes. Increase this if shuffle operations encounter disk space issues. Alternatively, optimize code by replacing 'groupByKey' with 'reduceByKey' or 'sumByKey'.
  --workerDiskType: Specifies the type of disk for worker instances. Use SSD for jobs with computationally expensive shuffles. See https://cloud.google.com/compute/docs/reference/latest/diskTypes for disk types and https://cloud.google.com/compute/docs/disks/performance for performance details.
  --network: Specifies the network to use for worker instances. Use this if your pipeline needs to communicate with external services via VPN, such as on-premise HDFS.
  --experiments: A comma-separated list of experimental features to enable. Examples include:
    - shuffle_mode=service: Utilizes the external shuffle service instead of local disk.
    - enable_custom_bigquery_sink: Enables a custom sink to work around BigQuery writing limitations.
    - worker_region=<REGION>: Uses a specified Google Cloud region for workers instead of the default zone, allowing for more flexible capacity.

Related Documentation:
- Specifying Pipeline Execution Parameters: https://cloud.google.com/dataflow/pipelines/specifying-exec-params
- Service Optimization and Execution: https://cloud.google.com/dataflow/service/dataflow-service-desc
- DataflowPipelineOptions Javadoc: org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
- DataflowPipelineWorkerPoolOptions Javadoc: org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
- DataflowWorkerHarnessOptions Javadoc (for side input performance): org.apache.beam.runners.dataflow.options.DataflowWorkerHarnessOptions#getWorkerCacheMb
- FAQ on Side Input Performance: FAQ.md#how-do-i-improve-side-input-performance-
```

--------------------------------

### Basic PTransform Override Setup

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Scio-Unit-Tests.md

Demonstrates the initial setup for overriding a named PTransform in a Scio test. This is useful for mocking transforms that interact with external services.

```scala
@@snip [JobTestTest.scala](/scio-test/core/src/test/scala/com/spotify/scio/testing/JobTestTest.scala) { #JobTestTest_example_kv }
```

--------------------------------

### Install Scio REPL via Homebrew

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Scio-REPL.md

Installs the Scio REPL using Homebrew on macOS. This command taps the spotify/public repository and then installs the scio package.

```bash
brew tap spotify/public
brew install scio
scio-repl
```

--------------------------------

### Scio Pipeline Test Data Setup

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Scio-Unit-Tests.md

Defines input data and expected output for a Scio pipeline test. This Scala code snippet illustrates how to prepare test data for a word count scenario.

```scala
val inData = Seq("a b c d e", "a b a b", "")
val expected = Seq("a: 3", "b: 3", "c: 1", "d: 1", "e: 1")
```

--------------------------------

### Scio REPL Startup with BigQuery Client

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Scio-REPL.md

Starts the Scio REPL with the BigQuery client enabled, displaying the welcome message and indicating available clients like 'bq' and 'sc'.

```shell
$ java -jar -Dbigquery.project=<project-id> scio-repl-0.7.0.jar
Welcome to
                 _____
    ________________(_)_____
    __  ___/  ___/_  /_  __ \
    _(__  )/ /__ _  / / /_/ /
    /____/ \___/ /_/  \____/   version 0.7.0

Using Scala version 2.12.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_144)
Type in expressions to have them evaluated.
Type :help for more information.

BigQuery client available as 'bq'
Scio context available as 'sc'
```

--------------------------------

### Test Scio Pipeline with Side Input

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Scio-Unit-Tests.md

Illustrates testing a Scio pipeline that utilizes side inputs. This example shows how to configure multiple input sources for JobTest when one serves as a side input.

```scala
@@snip [JoinExampleTest.scala](/scio-examples/src/test/scala/com/spotify/scio/examples/cookbook/JoinExamplesTest.scala#L73) { #JoinExamplesTest_example }
```

--------------------------------

### Run Word Count Example

Source: https://github.com/spotify/scio/blob/main/README.md

Executes the included word count example job. This command runs the staged Scio application, specifying an output directory for the results.

```bash
target/universal/stage/bin/scio-job --output=wc
```

--------------------------------

### Basic Dataflow Pipeline Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Scio-REPL.md

An example Scala code snippet showing how to create a ScioContext, read text files from Google Cloud Storage, perform a word count, and save the results back to GCS.

```scala
import com.spotify.scio.

def sc: ScioContext = ???

val shakespeare = sc.textFile("gs://dataflow-samples/shakespeare/*")

val wordCount = shakespeare
    .flatMap(_.split("[^a-zA-Z']+").filter(_.nonEmpty))
    .countByValue
    .map(_.toString)
    .saveAsTextFile("gs://<gcs-output-dir>")

val result = sc
    .run()
    .waitUntilDone()
    .tap(wordCount)
    .value
    .take(3)
```

--------------------------------

### BigQuery and Dataflow Integration Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Scio-REPL.md

A Scala example demonstrating reading data from a BigQuery table using Scio, performing transformations (counting tornadoes per month), and materializing a subset of results locally.

```scala
import com.spotify.scio._
import com.spotify.scio.bigquery._

def sc: ScioContext = ???

val tornadoes = sc.bigQuerySelect(Query("SELECT tornado, month FROM [apache-beam-testing:samples.weather_stations]"))
 
val counts = tornadoes
    .flatMap(r => if (r.getBoolean("tornado")) Seq(r.getLong("month")) else Nil)
    .countByValue
    .map(kv => TableRow("month" -> kv._1, "tornado_count" -> kv._2))
    .take(3)
    .materialize

val result = sc
    .run()
    .waitUntilDone()
    .tap(counts)
    .value
```

--------------------------------

### Start Scio REPL from SBT Console

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Scio-REPL.md

Clones the Scio repository and runs the REPL using SBT. This method is suitable for development or testing directly from the source code.

```bash
git clone git@github.com:spotify/scio.git
cd scio
sbt scio-repl/run
```

--------------------------------

### Run Scio REPL from SBT Project Template

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Scio-REPL.md

Starts the Scio REPL for projects generated from the scio-template.g8 template. Run this command from the project's root directory.

```bash
sbt repl/run
```

--------------------------------

### Apply Scalafix Refactoring

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/dev/Style-Guide.md

Command to enable and apply Scalafix refactoring rules to the codebase. This command helps automate code improvements and linting.

```bash
sbt scalafixEnable scalafixAll
```

--------------------------------

### Scalafix Rules Configuration

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/dev/Style-Guide.md

Configuration snippet for Scalafix, specifying the refactoring and linting rules to be applied. These rules help maintain code quality and tidiness.

```scalafix
rules = [
  RemoveUnused,
  LeakingImplicitClassVal
]
```

--------------------------------

### Mock Client with ListenableFuture

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/AsyncDoFn.md

Defines a mock client that returns Guava's ListenableFuture for asynchronous operations. This serves as an example for integrating with GuavaAsyncDoFn.

```scala
import com.google.common.util.concurrent.{ListenableFuture, Futures}

case class MyClient(value: String) {
  def request(i: Int): ListenableFuture[String] = Futures.immediateFuture(s"$value$i")
}
```

--------------------------------

### Subtraction Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Builtin.md

An example demonstrating how to get elements from one SCollection that are not present in another.

```scala
import com.spotify.scio.values.SCollection

val a: SCollection[String] = ???
val b: SCollection[String] = ???
val notInB: SCollection[String] = a.subtract(b)
```

--------------------------------

### Find Top N Elements Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Builtin.md

An example showing how to get the top 10 integer elements from an SCollection.

```scala
import com.spotify.scio.values.SCollection

val elements: SCollection[Int] = ???
val top10: SCollection[Iterable[Int]] = elements.top(10)
```

--------------------------------

### Scio REPL Startup with Increased Heap Size

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Scio-REPL.md

Starts the Scio REPL with a specified heap size (e.g., 2GiB) to mitigate OutOfMemory errors, showing the welcome message and available clients.

```shell
$ java -Xmx2g -jar scio-repl-0.7.0.jar
Welcome to
                 _____
    ________________(_)_____
    __  ___/  ___/_  /_  __ \
    _(__  )/ /__ _  / / /_/ /
    /____/ \___/ /_/  \____/   version 0.7.0

Using Scala version 2.12.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_144)

Type in expressions to have them evaluated.
Type :help for more information.

BigQuery client available as 'bq'
Scio context available as 'sc'
```

--------------------------------

### Scala Class Header Indentation

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/dev/Style-Guide.md

Example of Scala class formatting for headers that exceed a single line. Subsequent lines are aligned, and a blank line separates the header from the class body.

```scala
class Foo(val param1: String,
          val param2: String,
          val param3: Array[Byte])
  extends FooInterface  // 2 space indent here
  with Logging {

  def firstMethod(): Unit = { /* body */ }  // blank line above
}
```

--------------------------------

### Scala Method Declaration Indentation

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/dev/Style-Guide.md

Example of Scala method declaration formatting, showing how parameters are aligned when they exceed a single line. Return types are placed on the same line as the last parameter.

```scala
  def saveAsBigQuery(
    table: TableReference,
    schema: TableSchema,
    writeDisposition: WriteDisposition,
    createDisposition: CreateDisposition,
    tableDescription: String,
    timePartitioning: TimePartitioning)(implicit ev: T <:< TableRow): ClosedTap[TableRow] = {
   // method body
  }

  def saveAsTypedBigQuery(
    tableSpec: String,
    writeDisposition: WriteDisposition = TableWriteParam.DefaultWriteDisposition,
    createDisposition: CreateDisposition = TableWriteParam.DefaultCreateDisposition,
    timePartitioning: TimePartitioning = TableWriteParam.DefaultTimePartitioning)(
    implicit tt: TypeTag[T],
    ev: T <:< HasAnnotation,
    coder: Coder[T]): ClosedTap[T] = {
      // method body
  }
```

--------------------------------

### Run Scio REPL from Pre-built JAR

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Scio-REPL.md

Starts the Scio REPL by executing a downloaded JAR file. The REPL provides interactive Scala evaluation and makes Scio contexts available.

```bash
java -jar scio-repl-<version>.jar
```

--------------------------------

### Setup OverrideTypeProvider via System Property

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/internals/OverrideTypeProvider.md

Configures Scio to use a custom OverrideTypeProvider by setting a JVM system property. This is typically done at initialization time, for example, in a build.sbt file or application startup.

```scala
System.setProperty(
  "override.type.provider",
  "com.spotify.scio.bigquery.validation.SampleOverrideTypeProvider")
```

```sbt
initialize in Test ~= { _ => System.setProperty(
  "override.type.provider",
  "com.spotify.scio.bigquery.validation.SampleOverrideTypeProvider")
}
```

--------------------------------

### Format Codebase with SBT Plugins

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/dev/Style-Guide.md

Command to format the entire codebase using sbt plugins for Scala, Java, and header generation. Ensures consistent code style across the project.

```bash
sbt scalafmtAll javafmtAll headerCreateAll
```

--------------------------------

### Create New Scio Project

Source: https://github.com/spotify/scio/blob/main/README.md

Uses the giter8 template to quickly create a new Scio job repository. This command initiates the project scaffolding process.

```bash
sbt new spotify/scio.g8
```

--------------------------------

### Build and Stage Scio Project

Source: https://github.com/spotify/scio/blob/main/README.md

Builds the Scio project and stages it for execution. This command prepares the project for running jobs, typically by packaging dependencies.

```bash
cd scio-job
sbt stage
```

--------------------------------

### Scio Job: Read and Write SMB Data

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Sort-Merge-Bucket.md

Demonstrates a Scio job that reads from and writes to Sort-Merge-Bucket (SMB) data sources using `ParquetAvroSortedBucketIO`. This example shows the setup for reading grouped data and saving results as sorted buckets, requiring `Account` records and an integer SMB key.

```Scala
import org.apache.beam.sdk.extensions.smb.ParquetAvroSortedBucketIO
import org.apache.beam.sdk.values.TupleTag
import com.spotify.scio._
import com.spotify.scio.avro.Account
import com.spotify.scio.values.SCollection
import com.spotify.scio.smb._

object SmbJob {
    def main(cmdLineArgs: Array[String]): Unit = {
        val (sc, args) = ContextAndArgs(cmdLineArgs)
        
        // Read
        sc.sortMergeGroupByKey(
            classOf[Integer],
            ParquetAvroSortedBucketIO
                .read(new TupleTag[Account](), classOf[Account])
                .from(args("input"))
        )
        
        // Write
        val writeData: SCollection[Account] = ???
        writeData.saveAsSortedBucket(
            ParquetAvroSortedBucketIO
              .write(classOf[Integer], "id", classOf[Account])
              .to(args("output"))
        )
        
        sc.run().waitUntilDone()
    }
}
```

--------------------------------

### Intersection Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Builtin.md

An example demonstrating how to find common elements between two SCollections of strings.

```scala
import com.spotify.scio.values.SCollection

val a: SCollection[String] = ???
val b: SCollection[String] = ???
val common: SCollection[String] = a.intersection(b)
```

--------------------------------

### Using ScioIO Directly

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/internals/ScioIO.md

Introduces the `ScioContext#read` and `SCollection#write` methods, which allow direct usage of `ScioIO[T]` implementations. This bypasses the need for syntactic sugar methods like `ScioContext#textFile`.

```APIDOC
Direct ScioIO Usage:

Methods added to leverage ScioIO[T] directly:
  - ScioContext#read[T](io: ScioIO[T]): SCollection[T]
  - SCollection[T]#write(io: ScioIO[T]): ClosedWrite[T]

These methods enable using ScioIO implementations without relying on the convenience methods like `ScioContext.textFile` or `SCollection.saveAsTextFile`.
```

--------------------------------

### Sample SCollection Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Builtin.md

An example showing how to randomly sample elements from an SCollection with replacement.

```scala
import com.spotify.scio.values.SCollection

val elements: SCollection[String] = ???
val result: SCollection[String] = elements.sample(withReplacement = true, fraction = 0.01)
```

--------------------------------

### Batch SCollection Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Builtin.md

An example demonstrating how to batch elements of an SCollection by a specified size.

```scala
import com.spotify.scio.values.SCollection

val elements: SCollection[String] = ???
val batchedElements: SCollection[Iterable[String]] = elements.batch(10)
```

--------------------------------

### Intersect By Key Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Builtin.md

An example showing how to find common keys between a keyed SCollection and a collection of keys.

```scala
import com.spotify.scio.values.SCollection

val a: SCollection[(String, Int)] = ???
val b: SCollection[String] = ???
val common: SCollection[(String, Int)] = a.intersectByKey(b)
```

--------------------------------

### Randomly Split SCollection Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Builtin.md

An example demonstrating how to split an SCollection into three parts with specified weights.

```scala
import com.spotify.scio.values.SCollection

val elements: SCollection[Int] = ???
val weights: Array[Double] = Array(0.2, 0.6, 0.2)
val splits: Array[SCollection[Int]] = elements.randomSplit(weights)
```

--------------------------------

### Proto-Tools for Protobuf Schema and JSON Conversion

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/io/Protobuf.md

Provides command-line instructions for using `proto-tools` from `gcs-tools` to extract Protobuf schemas from Avro files and convert Avro-stored Protobuf data to JSON. This requires tapping the Spotify homebrew repository.

```Bash
brew tap spotify/public
brew install gcs-proto-tools
proto-tools getschema data.protobuf.avro
proto-tools tojson data.protobuf.avro
```

--------------------------------

### Format Code with sbt

Source: https://github.com/spotify/scio/wiki/Style-Guide

Command to format the entire codebase using scalafmt, including Scala code, test code, and sbt build files.

```sbt
sbt scalafmt test:scalafmt scalafmtSbt
```

--------------------------------

### Process Markdown Documentation with mdoc

Source: https://github.com/spotify/scio/blob/main/CONTRIBUTING.md

Compiles markdown files containing Scala code examples using mdoc. This is used for generating documentation and ensuring examples are up-to-date.

```sbt
site/mdoc
```

--------------------------------

### Kryo Tuning Command Line Options

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/FAQ.md

Pass Kryo tuning options via the command line to configure buffer sizes and reference tracking. These options are useful for optimizing serialization performance.

```shell
--kryoBufferSize=1024 --kryoMaxBufferSize=8192 --kryoReferenceTracking=false --kryoRegistrationRequired=true
```

--------------------------------

### Scio Typed BigQuery Pipeline Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/FAQ.md

Example of using typed BigQuery classes within a Scio pipeline. This demonstrates how to read data using `sc.typedBigQuery` with a custom case class.

```scala
import com.spotify.scio._
import com.spotify.scio.values._
import com.spotify.scio.bigquery._

def main(cmdlineArgs: Array[String]): Unit = {
  val (sc, args) = ContextAndArgs(cmdlineArgs)

  val data: SCollection[Tornado] = sc.typedBigQuery[Tornado]()
  // ... process data ...
}
```

--------------------------------

### Kryo Registration Error Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/internals/Kryo.md

This is an example of an error message encountered when a class is not registered with Kryo. The exception indicates the missing class and provides a hint on how to register it, typically by adding `classOf[MissingClass]` to your KryoRegistrar.

```bash
[info]   java.lang.IllegalArgumentException: Class is not registered: org.apache.avro.generic.GenericData$Record
[info] Note: To register this class use: kryo.register(org.apache.avro.generic.GenericData$Record.class);
```

--------------------------------

### Authenticate GCP for Scio Tests (Bash/SBT)

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/dev/build.md

To enable Google Cloud Platform-dependent examples, authenticate your application with default credentials. This involves using `gcloud auth application-default login` followed by an sbt test run to ensure GCP resources can be accessed.

```bash
gcloud auth application-default login
sbt test
```

--------------------------------

### Avro String Key ClassCastException Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/extras/Sort-Merge-Bucket.md

An example of a ClassCastException that may occur when using Avro CharSequence keys with older Scio versions (prior to 0.14.0), where Utf8 was used instead of String.

```bash
Cause: java.lang.ClassCastException: class org.apache.avro.util.Utf8 cannot be cast to class java.lang.String
  at org.apache.beam.sdk.coders.StringUtf8Coder.encode(StringUtf8Coder.java:37)
  at org.apache.beam.sdk.extensions.smb.BucketMetadata.encodeKeyBytes(BucketMetadata.java:222)
```

--------------------------------

### Release Notes Structure

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/dev/How-to-Release.md

Recommended structure for organizing release notes, categorizing changes for clarity into Enhancements, Bug Fixes, Documentation, Build Improvements, and Dependency Updates.

```markdown
## 🚀 Enhancements
## 🐛 Bug Fixes
## 📗 Documentation
## 🏗️ Build Improvements
## 🌱 Dependency Updates
```

--------------------------------

### Scio IO Documentation

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/index.md

Details how to use Scio with various input/output formats and services, including type-safe integration with BigQuery, Avro, Parquet, Bigtable, and Protobuf.

```apidoc
Scio IO Connectors:

Avro:
  io/Avro.md
  Description: Using Scio with Avro files for data serialization and deserialization.

BigQuery:
  io/BigQuery.md
  Description: Type-safe integration with Google BigQuery for reading and writing data.

Bigtable:
  io/Bigtable.md
  Description: Using Scio with Google Cloud Bigtable for NoSQL data storage.

Parquet:
  io/Parquet.md
  Description: Using Scio with Parquet files for efficient columnar data storage.

Protobuf:
  io/Protobuf.md
  Description: Using Scio with Protobuf for structured data serialization.
```

--------------------------------

### SQL: Syntax Error Example

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/releases/migrations/v0.8.0-Migration-Guide.md

Demonstrates a SQL query with a syntax error, specifically a misspelled keyword. The example includes the Scala code with the error and the corresponding Bash error output indicating the parsing failure.

```Scala
tsql"select username, age fom $users".as[(String, Int)]
```

```Bash
ParseException: Encountered "users" at line 1, column 27.
Was expecting one of:
    <EOF>
    "ORDER" ...
    "LIMIT" ...
    "OFFSET" ...
    "FETCH" ...
    "FROM" ...
    "," ...
    "UNION" ...
    "INTERSECT" ...
    "EXCEPT" ...
    "MINUS" ...


Query:
select username, age fom  users


schema of users:
┌──────────────────────────────────────────┬──────────────────────┬──────────┐
│ NAME                                     │ TYPE                 │ NULLABLE │
├──────────────────────────────────────────┼──────────────────────┼──────────┤
│ username                                 │ STRING               │ NO       │
│ email                                    │ STRING               │ NO       │
│ age                                      │ INT32                │ NO       │
└──────────────────────────────────────────┴──────────────────────┴──────────┘

Query result schema (inferred) is unknown.
Expected schema:
┌──────────────────────────────────────────┬──────────────────────┬──────────┐
│ NAME                                     │ TYPE                 │ NULLABLE │
├──────────────────────────────────────────┼──────────────────────┼──────────┤
│ _1                                       │ STRING               │ NO       │
│ _2                                       │ INT32                │ NO       │
└──────────────────────────────────────────┴──────────────────────┴──────────┘
```

--------------------------------

### Scio: Sparse Join Example (Scala)

Source: https://github.com/spotify/scio/blob/main/site/src/main/paradox/Joins.md

Demonstrates the usage of Scio's `sparseJoin` method. This example shows how to join two SCollections, `a` and `b`, using a sparse join optimization, which is beneficial when `b` is large but has sparse keys.

```scala
import com.spotify.scio.values.SCollection
import magnolify.guava.auto._

val a: SCollection[(String, Int)] = ???
val b: SCollection[(String, Boolean)] = ???

val bNumKeys: Int = ???
val joined = a.sparseJoin(b, bNumKeys)
```