### Running Examples with Jaeger

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/CONTRIBUTING.md

A three-terminal workflow to test tracing: start Jaeger, run an example server, and then run an HTTP client to send requests and verify traces in Jaeger.

```bash
# Terminal 1: Start Jaeger
mise run run-jaeger
```

```bash
# Terminal 2: Start example server
mise run run-example-axum-otlp-server
```

```bash
# Terminal 3: Send requests and check traces
mise run run-example-http-client
# Then check traces in Jaeger UI at http://localhost:16686
```

--------------------------------

### Run Axum Application with Tracing

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/examples/axum-otlp/README.md

Navigate to the example directory and run the Axum application. This will compile and start the server, demonstrating the integrated tracing and logging output.

```sh
❯ cd examples/axum-otlp
❯ cargo run
```

--------------------------------

### Initialization Examples

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/init-tracing-opentelemetry.md

Examples demonstrating how to initialize the tracing subscriber with different configurations.

```APIDOC
## Initialization Examples

**Example:**
```rust
use init_tracing_opentelemetry::TracingConfig;
use tracing::Level;

// Development preset
let _guard = TracingConfig::development().init_subscriber()?;

// Custom configuration
let _guard = TracingConfig::default()
    .with_json_format()
    .with_stderr()
    .with_log_directives("debug")
    .with_otel(true)
    .init_subscriber()?;

// Non-global subscriber
let guard = TracingConfig::development()
    .with_global_subscriber(false)
    .init_subscriber()?;
```
```

--------------------------------

### Run Container and Example Tasks

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/CONTRIBUTING.md

Start the Jaeger all-in-one container for local development or run various example applications, including gRPC servers/clients and Axum HTTP servers.

```bash
# Start Jaeger all-in-one for local development
mise run run-jaeger
```

```bash
# Run example applications
mise run run-example-grpc-server     # gRPC server example
```

```bash
mise run run-example-grpc-client     # gRPC client example
```

```bash
mise run run-example-axum-otlp-server # Axum HTTP server
```

```bash
mise run run-example-http-client     # HTTP client test
```

```bash
mise run run-example-load            # Load testing example
```

--------------------------------

### Local Jaeger Setup and UI Access

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/CONTRIBUTING.md

Start a local Jaeger instance using 'mise run run-jaeger' and access its UI via http://localhost:16686. Jaeger listens on ports like 4317/4318 for OTLP.

```bash
# Start Jaeger (runs on various ports including 16686 for UI, 4317/4318 for OTLP)
mise run run-jaeger
```

```bash
# Open Jaeger UI
open http://localhost:16686
```

--------------------------------

### Clone Repository and Install Dependencies

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/CONTRIBUTING.md

Clone the project repository and install all necessary development tools and dependencies defined in .mise.toml.

```bash
git clone https://github.com/davidB/tracing-opentelemetry-instrumentation-sdk.git
cd tracing-opentelemetry-instrumentation-sdk
```

```bash
# Install all tools defined in .mise.toml (Rust, protoc, grpcurl, etc.)
mise install

# Activate the environment (or use 'mise use' for shell integration)
mise activate
```

--------------------------------

### Axum Application Setup with Tracing and OpenTelemetry

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/axum-tracing-opentelemetry/README.md

Set up an Axum application with OpenTelemetry tracing middleware. Ensure tracing is initialized before running the application. The `OtelInResponseLayer` adds trace context to responses, while `OtelAxumLayer` starts traces for incoming requests.

```rust
use axum_tracing_opentelemetry::middleware::{OtelAxumLayer, OtelInResponseLayer};

#[tokio::main]
async fn main() -> Result<(), axum::BoxError> {
    // very opinionated init of tracing, look as is source to make your own
    let _guard = init_tracing_opentelemetry::TracingConfig::production().init_subscriber()?;

    let app = app();
    // run it
    let addr = &"0.0.0.0:3000".parse::<SocketAddr>()?;
    tracing::warn!("listening on {}", addr);
    let listener = tokio::net::TcpListener::bind(addr).await?;
    axum::serve(listener, app.into_make_service()).await?;
    Ok(())
}

fn app() -> Router {
    Router::new()
        .route("/", get(index)) // request processed inside span
        // include trace context as header into the response
        .layer(OtelInResponseLayer::default())
        //start OpenTelemetry trace on incoming request
        .layer(OtelAxumLayer::default())
        .route("/health", get(health)) // request processed without span / trace
}
```

--------------------------------

### Kubernetes Deployment with Manual Environment Variable Setup

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/init-tracing-opentelemetry/README.md

Example Kubernetes Deployment manifest showing manual configuration of environment variables for the OpenTelemetry SDK within a container.

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    metadata:
      containers:
        - name: app
          env:
            - name: OTEL_SERVICE_NAME
              value: "app"
            - name: OTEL_EXPORTER_OTLP_PROTOCOL
              value: "grpc"
            # for otel collector in `deployment` mode, use the name of the service
            # - name: OTEL_EXPORTER_OTLP_ENDPOINT
            #   value: "http://opentelemetry-collector.opentelemetry-collector:4317"
            # for otel collector in sidecar mode (imply to deploy a sidecar CR per namespace)
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "http://localhost:4317"
            # for `daemonset` mode: need to use the local daemonset (value interpolated by k8s: `$(...)`)
            # - name: OTEL_EXPORTER_OTLP_ENDPOINT
            #   value: "http://$(HOST_IP):4317"
            # - name: HOST_IP
            #   valueFrom:
            #     fieldRef:
            #       fieldPath: status.hostIP
```

--------------------------------

### Install cargo-binstall Manually

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/CONTRIBUTING.md

Manually install cargo-binstall using a provided script if automatic tool installation fails, then retry the 'mise run' task.

```bash
# Install cargo-binstall manually
curl -L --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/cargo-bins/cargo-binstall/main/install-from-binstall-release.sh | bash

# Then retry the task
mise run <task-name>
```

--------------------------------

### Check Container Runtime Installation

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/CONTRIBUTING.md

Verify that one of the supported container runtimes (podman, nerdctl, docker) is installed and accessible in your PATH.

```bash
# Make sure one of these is installed and available
which podman || which nerdctl || which docker
```

--------------------------------

### Install mise

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/CONTRIBUTING.md

Install mise using the provided script or a package manager. This is a prerequisite for managing development tools.

```bash
curl https://mise.run | sh
```

```bash
# macOS: brew install mise
# Arch: pacman -S mise
# Ubuntu/Debian: see https://mise.jdx.dev/installing-mise.html
```

--------------------------------

### Install Individual Tools

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/CONTRIBUTING.md

Manually install specific tools like cargo-hack, cargo-nextest, cargo-insta, or cargo-deny using 'mise run install:<tool-name>'.

```bash
# Individual tool installation (usually handled automatically by other tasks)
mise run install:cargo-hack
```

```bash
mise run install:cargo-nextest
```

```bash
mise run install:cargo-insta
```

```bash
mise run install:cargo-deny
```

--------------------------------

### Initialize with Production Preset

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/init-tracing-opentelemetry/README.md

Use the `production()` preset for a standard OpenTelemetry setup. The returned guard ensures pending traces are sent on drop.

```rust
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Simple preset
    let _guard = init_tracing_opentelemetry::TracingConfig::production().init_subscriber()?;

    //...

    Ok(())
}
```

--------------------------------

### Start FakeCollectorServer

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/fake-opentelemetry-collector.md

Starts a new fake collector server on a random available port. Use this to initialize the mock collector for testing.

```rust
use fake_opentelemetry_collector::FakeCollectorServer;

let server = FakeCollectorServer::start().await?;
let endpoint = server.endpoint();  // Get OTLP endpoint
```

--------------------------------

### Axum Application Log Output

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/examples/axum-otlp/README.md

Example log output from the Axum application after starting. Shows initialization of logging and tracing, resource attributes, OTLP exporter configuration, and endpoint listening information.

```text
   Compiling examples-axum-otlp v0.1.0 (/home/david/src/github.com/davidB/axum-tracing-opentelemetry/examples/axum-otlp)
    Finished dev [unoptimized + debuginfo] target(s) in 3.60s
     Running `/home/david/src/github.com/davidB/axum-tracing-opentelemetry/target/debug/examples-axum-otlp`
     0.000041809s  INFO init_tracing_opentelemetry::tracing_subscriber_ext: init logging & tracing
    at init-tracing-opentelemetry/src/tracing_subscriber_ext.rs:82 on main

     0.000221695s DEBUG otel::setup::resource: key: service.name, value: unknown_service
    at init-tracing-opentelemetry/src/resource.rs:63 on main

     0.000242183s DEBUG otel::setup::resource: key: os.type, value: linux
    at init-tracing-opentelemetry/src/resource.rs:63 on main

     0.000280946s DEBUG otel::setup: OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: "http://localhost:4317"
    at init-tracing-opentelemetry/src/otlp.rs:22 on main

     0.000293128s DEBUG otel::setup: OTEL_EXPORTER_OTLP_TRACES_PROTOCOL: "grpc"
    at init-tracing-opentelemetry/src/otlp.rs:23 on main

     0.000377897s DEBUG otel::setup: OTEL_TRACES_SAMPLER: "always_on"
    at init-tracing-opentelemetry/src/otlp.rs:80 on main

     0.000561931s DEBUG otel::setup: OTEL_PROPAGATORS: "tracecontext,baggage"
    at init-tracing-opentelemetry/src/lib.rs:97 on main

     0.000134291s  WARN examples_axum_otlp: listening on 0.0.0.0:3003
    at examples/axum-otlp/src/main.rs:15 on main

     0.000150401s  INFO examples_axum_otlp: try to call `curl -i http://127.0.0.1:3003/` (with trace)
    at examples/axum-otlp/src/main.rs:16 on main

     0.000159659s  INFO examples_axum_otlp: try to call `curl -i http://127.0.0.1:3003/health` (with NO trace)
    at examples/axum-otlp/src/main.rs:17 on main
```

--------------------------------

### Complete Tracing and Logging Configuration

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/configuration.md

A comprehensive example demonstrating how to configure tracing and logging with custom resource detection, various formatting and output options, level directives, OpenTelemetry integration, and global subscriber initialization.

```rust
use init_tracing_opentelemetry::{
    TracingConfig,
    LogFormat,
    LogTimer,
    WriterConfig,
    resource::DetectResource,
};
use tracing::Level;

let resource = DetectResource::default()
    .with_fallback_service_name(env!("CARGO_PKG_NAME"))
    .with_fallback_service_version(env!("CARGO_PKG_VERSION"))
    .build();

let _guard = TracingConfig::default()
    // Format
    .with_json_format()
    // Output
    .with_stderr()
    // Levels
    .with_log_directives("myapp=debug,tower=info")
    .with_default_level(Level::INFO.into())
    .with_otel_trace_level(Level::TRACE.into())
    // Features
    .with_file_names(true)
    .with_line_numbers(true)
    .with_thread_names(false)
    .with_timer(LogTimer::Time)
    .with_span_events(tracing_subscriber::fmt::format::FmtSpan::NEW |
                      tracing_subscriber::fmt::format::FmtSpan::CLOSE)
    // OpenTelemetry
    .with_otel(true)
    .with_logs(true)
    .with_metrics(true)
    .with_resource_config(resource)
    .with_otel_tracer_name("myapp-tracer")
    // Subscriber
    .with_global_subscriber(true)
    .init_subscriber()?;
```

--------------------------------

### TracingConfig Builder Methods

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/init-tracing-opentelemetry.md

Demonstrates how to use the TracingConfig builder methods to configure tracing settings and initialize a subscriber. Includes examples for development presets, custom configurations, and non-global subscriber initialization.

```rust
use init_tracing_opentelemetry::TracingConfig;
use tracing::Level;

// Development preset
let _guard = TracingConfig::development().init_subscriber()?;

// Custom configuration
let _guard = TracingConfig::default()
    .with_json_format()
    .with_stderr()
    .with_log_directives("debug")
    .with_otel(true)
    .init_subscriber()?;

// Non-global subscriber
let guard = TracingConfig::development()
    .with_global_subscriber(false)
    .init_subscriber()?;

```

--------------------------------

### Run Jaeger All-in-One with Docker

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/README.md

Starts a Jaeger all-in-one instance using Docker, which supports OpenTelemetry, Jaeger, and Zipkin protocols. This is useful for local trace visualization. Ensure you have a container runtime installed.

```sh
# launch Jaeger with OpenTelemetry, Jaeger, Zipking,... mode.
# see https://www.jaegertracing.io/docs/1.49/getting-started/#all-in-one

# docker/nerdctl/podman or any container runner
docker run --rm --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 5778:5778 \
  -p 9411:9411 \
  cr.jaegertracing.io/jaegertracing/jaeger:latest

open http://localhost:16686
```

--------------------------------

### Enable OpenTelemetry Setup Debug Logs

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/configuration.md

Set the RUST_LOG environment variable to 'otel::setup=debug' to enable debug logs for OpenTelemetry setup.

```rust
std::env::set_var("RUST_LOG", "otel::setup=debug");
```

--------------------------------

### Initialize Tracing with Defaults

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/README.md

Basic setup for initializing the tracing subscriber with development defaults. This should be called early in your application's lifecycle.

```rust
use init_tracing_opentelemetry::TracingConfig;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize tracing with defaults
    let _guard = TracingConfig::development().init_subscriber()?;
    
    tracing::info!("Application started");
    Ok(())
}
```

--------------------------------

### Example: HTTP Flavor Conversion

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/http-module.md

Demonstrates the conversion of different HTTP versions to their semantic convention flavor strings using the `http_flavor` function.

```rust
use http::Version;
use tracing_opentelemetry_instrumentation_sdk::http::http_flavor;

assert_eq!(http_flavor(Version::HTTP_11), "1.1");
assert_eq!(http_flavor(Version::HTTP_2), "2.0");
assert_eq!(http_flavor(Version::HTTP_3), "3.0");
```

--------------------------------

### Rust: Demo Fake Tracer and Collector with Snapshot Testing

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/fake-opentelemetry-collector/README.md

This example demonstrates setting up a fake OpenTelemetry collector, initializing a tracer provider, creating and ending a span with an event, and then collecting and snapshot testing the exported spans using `insta`. It highlights the typical workflow for testing OpenTelemetry instrumentation.

```rust
use std::time::Duration;

use fake_opentelemetry_collector::{setup_tracer_provider, FakeCollectorServer};
use opentelemetry::trace::TracerProvider;
use opentelemetry::trace::{Span, SpanKind, Tracer};
use tracing::debug;

#[tokio::test(flavor = "multi_thread")]
async fn demo_fake_tracer_and_collector() {
    debug!("Start the fake collector");
    let mut fake_collector = FakeCollectorServer::start()
        .await
        .expect("fake collector setup and started");

    debug!("Init the 'application' & tracer provider");
    let tracer_provider = setup_tracer_provider(&fake_collector).await;
    let tracer = tracer_provider.tracer("test");

    debug!("Run the 'application' & sending span...");
    let mut span = tracer
        .span_builder("my-test-span")
        .with_kind(SpanKind::Server)
        .start(&tracer);
    span.add_event("my-test-event", vec![]);
    span.end();

    debug!("Shutdown the 'application' & tracer provider and force flush the spans");
    let _ = tracer_provider.force_flush();
    tracer_provider
        .shutdown()
        .expect("no error during shutdown");
    drop(tracer_provider);

    debug!("Collect & check the spans");
    let otel_spans = fake_collector
        .exported_spans(1, Duration::from_secs(20))
        .await;
    //insta::assert_debug_snapshot!(otel_spans);
    insta::assert_yaml_snapshot!(otel_spans, {
        "[].start_time_unix_nano" => "[timestamp]",
        "[].end_time_unix_nano" => "[timestamp]",
        "[].events[].time_unix_nano" => "[timestamp]",
        "[].trace_id" => insta::dynamic_redaction(|value, _path| {
            assert2::assert!(let Some(trace_id) = value.as_str());
            format!("[trace_id:lg{}]", trace_id.len())
        }),
        "[].span_id" => insta::dynamic_redaction(|value, _path| {
            assert2::assert!(let Some(span_id) = value.as_str());
            format!("[span_id:lg{}]", span_id.len())
        }),
        "[].links[].trace_id" => insta::dynamic_redaction(|value, _path| {
            assert2::assert!(let Some(trace_id) = value.as_str());
            format!("[trace_id:lg{}]", trace_id.len())
        }),
        "[].links[].span_id" => insta::dynamic_redaction(|value, _path| {
            assert2::assert!(let Some(span_id) = value.as_str());
            format!("[span_id:lg{}]", span_id.len())
        }),
    });
}
```

--------------------------------

### Setup Logger Provider

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/fake-opentelemetry-collector.md

Creates a logger provider configured to export logs to the fake collector. This function is useful for setting up log export during testing.

```rust
pub async fn setup_logger_provider(
    fake_server: &FakeCollectorServer,
) -> opentelemetry_sdk::logs::SdkLoggerProvider
```

--------------------------------

### Setup Meter Provider

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/fake-opentelemetry-collector.md

Creates a meter provider configured to export metrics to the fake collector. This function is used to set up metric export for testing purposes.

```rust
pub async fn setup_meter_provider(
    fake_server: &FakeCollectorServer,
) -> opentelemetry_sdk::metrics::SdkMeterProvider
```

--------------------------------

### Initialize Subscriber with Development Configuration

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/init-tracing-opentelemetry.md

Initializes a tracing subscriber with development-friendly settings, including a guard for proper cleanup. The guard must be held for the duration of the tracing setup.

```rust
use init_tracing_opentelemetry::TracingConfig;

let _guard = TracingConfig::development().init_subscriber()?;
// Guard must be held for proper cleanup on drop
```

--------------------------------

### Setup Tracer Provider

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/fake-opentelemetry-collector.md

Creates a tracer provider configured to export traces to the fake collector. It clears the OTLP traces endpoint environment variable, configures a batch span exporter, and points it to the fake server endpoint.

```rust
pub async fn setup_tracer_provider(
    fake_server: &FakeCollectorServer,
) -> opentelemetry_sdk::trace::SdkTracerProvider
```

```rust
let server = FakeCollectorServer::start().await?;
let tracer_provider = setup_tracer_provider(&server).await;
```

--------------------------------

### TracingConfig Configuration

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/init-tracing-opentelemetry.md

Main configuration builder for tracing setup, providing a builder pattern for flexible configuration of format, writer, level, features, and OpenTelemetry settings.

```APIDOC
## TracingConfig Struct

Main configuration builder for tracing setup. Provides builder pattern for flexible configuration.

### Fields

- **`format`** (`LogFormat`) - Required - Output format. Default: `LogFormat::default()`
- **`writer`** (`WriterConfig`) - Required - Output destination. Default: `WriterConfig::Stdout`
- **`level_config`** (`LevelConfig`) - Required - Level filtering. Default: `LevelConfig::default()`
- **`features`** (`FeatureSet`) - Required - Optional features. Default: `FeatureSet::default()`
- **`otel_config`** (`OtelConfig`) - Required - OpenTelemetry settings. Default: `OtelConfig::default()`
- **`global_subscriber`** (`bool`) - Required - Set as global default. Default: `true`
- **`tracer_name`** (`String`) - Required - Name for the tracer. Default: `""`
```

--------------------------------

### Run Otel Desktop Viewer with Docker

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/README.md

Launches the Otel Desktop Viewer using Docker. This tool is useful for collecting and visualizing OpenTelemetry traces locally. Ensure you have a container runtime like Docker, nerdctl, or Podman installed.

```sh
# also available via `brew install --cask ctrlspice/tap/otel-desktop-viewer`
# For AMD64 (most common)
# docker/nerdctl/podman or any container runner
docker run -p 8000:8000 -p 4317:4317 -p 4318:4318 ghcr.io/ctrlspice/otel-desktop-viewer:latest-amd64

open http://localhost:8000
```

--------------------------------

### FakeCollectorServer::start

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/fake-opentelemetry-collector.md

Starts a new fake collector server on a random available port. It binds to 127.0.0.1:0 and launches a Tonic gRPC server in a background task to accept OTLP exports.

```APIDOC
## FakeCollectorServer::start

### Description
Starts a new fake collector server on a random available port.

### Method
`async fn start() -> Result<Self, Box<dyn std::error::Error>>`

### Parameters
None

### Returns
`Result<FakeCollectorServer, Box<dyn Error>>` - Server instance or error

### Behavior
- Binds to `127.0.0.1:0` (random port)
- Launches Tonic gRPC server in background task
- Accepts trace, log, and metrics OTLP exports

### Example
```rust
use fake_opentelemetry_collector::FakeCollectorServer;

let server = FakeCollectorServer::start().await?;
let endpoint = server.endpoint();  // Get OTLP endpoint
```
```

--------------------------------

### Initialize Global Tracing Subscriber Once

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/errors.md

Avoid 'default Trace subscriber already installed' errors by ensuring the global subscriber is initialized only once using `once_cell::sync::OnceCell`.

```rust
use once_cell::sync::OnceCell;

static TRACING: OnceCell<init_tracing_opentelemetry::Guard> = OnceCell::new();

fn init_tracing() -> Result<(), Box<dyn std::error::Error>> {
    let guard = TracingConfig::development().init_subscriber()?;
    TRACING.set(guard).ok();
    Ok(())
}
```

--------------------------------

### Valid Log Directive Examples

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/errors.md

Demonstrates various ways to configure log directives for filtering. Supports global levels, specific crate levels, and multiple directives.

```rust
.with_log_directives("debug")                           // All debug
```

```rust
.with_log_directives("myapp=trace")                     // Specific crate
```

```rust
.with_log_directives("myapp=debug,tower=info")          // Multiple directives
```

```rust
.with_log_directives("debug,tower=warn")                // Multiple levels
```

--------------------------------

### Early Tracing Initialization in Main Function

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/errors.md

Initialize tracing as early as possible in the `main` function to ensure all subsequent logs and traces are captured. This pattern is crucial for comprehensive application monitoring from the start.

```rust
fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Initialize as early as possible
    let _guard = TracingConfig::development().init_subscriber()?;
    
    // Now all logging and tracing is available
    tracing::info!("Application started");
    
    run_application().await?;
    Ok(())
}
```

--------------------------------

### Kubernetes Deployment with OpenTelemetry Operator Injection

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/init-tracing-opentelemetry/README.md

Example Kubernetes Deployment manifest demonstrating how to use annotations to inject the OpenTelemetry SDK via the OpenTelemetry operator.

```yaml
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    metadata:
      annotations:
        # to inject environment variables only by opentelemetry-operator
        instrumentation.opentelemetry.io/inject-sdk: "opentelemetry-operator/instrumentation"
        instrumentation.opentelemetry.io/container-names: "app"
      containers:
        - name: app
```

--------------------------------

### Configure Tonic gRPC Client Middleware

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/configuration.md

Use OtelGrpcLayer for Tonic gRPC clients to add OpenTelemetry tracing. It allows filtering requests, for example, to exclude reflection services.

```rust
use tonic_tracing_opentelemetry::middleware::client::OtelGrpcLayer;

OtelGrpcLayer::default()
    .filter(|path| !path.starts_with("/grpc.reflection"))
```

--------------------------------

### Verify OTLP Collector Status

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/errors.md

Troubleshoot 'connection refused' OTLP export errors by ensuring your OpenTelemetry collector is running. Use Docker to start a collector or a fake collector for testing.

```bash
# Verify collector is running
docker run -p 4317:4317 -p 4318:4318 \
  otel/opentelemetry-collector:latest

# Or use fake collector in tests
```

--------------------------------

### Typical Tracing Usage Pattern

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/fake-opentelemetry-collector.md

Demonstrates a typical usage pattern for tracing with the fake OpenTelemetry collector in a Tokio test. It covers starting the collector, configuring the tracer provider, running application code that generates traces, and then collecting and verifying these traces.

```rust
use fake_opentelemetry_collector::FakeCollectorServer;
use std::time::Duration;

#[tokio::test]
async fn test_tracing() -> Result<(), Box<dyn std::error::Error>> {
    // Start fake collector
    let mut server = FakeCollectorServer::start().await?;
    
    // Configure tracer provider to send to collector
    let tracer_provider = setup_tracer_provider(&server).await;
    opentelemetry::global::set_tracer_provider(tracer_provider);
    
    // Run application code...
    // Traces will be sent to fake collector
    
    // Collect and verify traces
    let spans = server.exported_spans(1, Duration::from_millis(500)).await;
    assert!(!spans.is_empty());
    assert_eq!(spans[0].name, "expected_span_name");
    
    Ok(())
}
```

--------------------------------

### Instrument SQL Query Execution

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/tracing-opentelemetry-instrumentation-sdk/README.md

Applies a handmade OpenTelemetry span to a SQL query execution using the `.instrument()` method from the `tracing` crate. This example demonstrates how to wrap a `sqlx` query with a custom span created by `make_otel_span`.

```rust
// Insert or update
        sqlx::query!(
                "INSERT INTO ...",
                id,
                sub_key,
                result,
            )
            .execute(&*self.pool)
            .instrument(make_otel_span("INSERT"))
            .await
            .map_err(...)?;
```

--------------------------------

### Development Workflow Steps

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/CONTRIBUTING.md

Sequence of commands for setting up the environment, formatting/linting code, running tests, and performing full validation before submitting a PR.

```bash
# Setup environment (first time only):
mise install
```

```bash
# Before making changes:

# Format and check code
mise run format
mise run lint
mise run check
```

```bash
# While developing:

# Run tests frequently
mise run test

# Test with examples if relevant
mise run run-jaeger &  # Start Jaeger in background
mise run run-example-axum-otlp-server
```

```bash
# Before submitting PR:
# Full validation
mise run format
mise run lint
mise run check
mise run test
mise run deny
```

--------------------------------

### Equivalent Configuration for Production Preset

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/configuration.md

This demonstrates the configuration options equivalent to `TracingConfig::production()`, featuring JSON format, stderr output, and disabled span events.

```rust
TracingConfig::default()
    .with_json_format()
    .with_stderr()
    .with_otel(true)
    .with_log_directives("info")
    .with_line_numbers(false)
    .with_thread_names(false)
    .without_span_events()
```

--------------------------------

### Using SpanType Enum

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/tracing-opentelemetry-instrumentation-sdk.md

Example of creating an info span with the 'span.type' field set using the SpanType enum.

```rust
use tracing_opentelemetry_instrumentation_sdk::span_type::SpanType;

let span = tracing::info_span!("query", "span.type" = %SpanType::Sql);

```

--------------------------------

### Get Server Endpoint URL

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/fake-opentelemetry-collector.md

Provides the OTLP endpoint URL for the fake collector server. This can be configured in OpenTelemetry exporters.

```rust
pub fn endpoint(&self) -> String
```

--------------------------------

### Initialize with Custom Configuration

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/init-tracing-opentelemetry/README.md

Configure tracing with JSON format, stderr output, and specific log directives. The `init_subscriber()` function returns a guard that handles trace flushing.

```rust
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // custom configuration
    let _guard = init_tracing_opentelemetry::TracingConfig::default()
        .with_json_format()
        .with_stderr()
        .with_log_directives("debug")
        .init_subscriber()?;

    //...

    Ok(())
}
```

--------------------------------

### Get Server Address

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/fake-opentelemetry-collector.md

Retrieves the listening socket address of the fake collector server. Useful for debugging or manual inspection.

```rust
pub fn address(&self) -> SocketAddr
```

--------------------------------

### Equivalent Configuration for Development Preset

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/configuration.md

This shows the equivalent configuration options for the `TracingConfig::development()` preset, including pretty format, stdout, OpenTelemetry enabled, and debug log directives.

```rust
TracingConfig::default()
    .with_pretty_format()
    .with_stdout()
    .with_otel(true)
    .with_log_directives("debug")
    .with_line_numbers(true)
    .with_thread_names(true)
    .with_span_events(FmtSpan::NEW | FmtSpan::CLOSE)
```

--------------------------------

### FakeCollectorServer::endpoint

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/fake-opentelemetry-collector.md

Gets the server's OTLP endpoint URL, formatted as a string suitable for use with OpenTelemetry client configurations.

```APIDOC
## FakeCollectorServer::endpoint

### Description
Gets the server's OTLP endpoint URL.

### Method
`pub fn endpoint(&self) -> String`

### Parameters
None

### Returns
`String` - The server's OTLP endpoint URL (e.g., `http://127.0.0.1:4317`)

### Example
```rust
let endpoint = server.endpoint();  // e.g., "http://127.0.0.1:4317"
// Use as OTEL_EXPORTER_OTLP_ENDPOINT
```
```

--------------------------------

### Equivalent Configuration for Minimal Preset

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/configuration.md

This illustrates the configuration options that match `TracingConfig::minimal()`, using compact format and disabling OpenTelemetry.

```rust
TracingConfig::default()
    .with_compact_format()
    .with_otel(false)
```

--------------------------------

### Build Methods

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/init-tracing-opentelemetry.md

Methods for constructing layers and initializing the subscriber.

```APIDOC
## Build Methods

```rust
pub fn build_layer<S>(&self) -> Result<Box<dyn Layer<S> + Send + Sync + 'static>, Error>
where
    S: Subscriber + for<'a> LookupSpan<'a>

pub fn build_filter_layer(&self) -> Result<EnvFilter, Error>

pub fn init_subscriber(self) -> Result<Guard, Error>

pub fn init_subscriber_ext<F, SOut>(self, transform: F) -> Result<Guard, Error>
where
    SOut: Subscriber + for<'a> LookupSpan<'a> + Send + Sync,
    F: FnOnce(Registry) -> SOut
```

Constructs layers and initializes the subscriber.
```

--------------------------------

### FakeCollectorServer::address

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/fake-opentelemetry-collector.md

Gets the server's listening socket address, useful for determining the exact network interface and port the server is bound to.

```APIDOC
## FakeCollectorServer::address

### Description
Gets the server's listening socket address.

### Method
`pub fn address(&self) -> SocketAddr`

### Parameters
None

### Returns
`SocketAddr` - The server's listening socket address (e.g., `127.0.0.1:12345`)

### Example
```rust
let addr = server.address();
println!("Server listening on: {}", addr);
```
```

--------------------------------

### Run Full Test Suite

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/CONTRIBUTING.md

Execute the complete test suite, including linting and security checks, before submitting changes. Ensure all commands complete successfully.

```bash
mise run lint && mise run test && mise run deny
```

--------------------------------

### LogTimer Enum

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/types.md

Controls the timestamp format in log output. Options include no timestamp, wall-clock time, or elapsed time since process start.

```rust
pub enum LogTimer {
    None,
    Time,
    Uptime,
}
```

--------------------------------

### LogTimer Enum

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/init-tracing-opentelemetry.md

Controls the timestamp format in log output. Options include no timestamp, wall-clock time, or elapsed time since process start.

```APIDOC
## LogTimer Enum

### Description
Controls the timestamp format in log output.

### Variants
- `None` - No timestamp
- `Time` - Wall-clock time
- `Uptime` - Elapsed time since process start

### Default
`Uptime` in debug builds, `Time` in release builds
```

--------------------------------

### Initialize OpenTelemetry Tracing

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/configuration.md

Recommended initialization sequence for OpenTelemetry tracing, including initializing propagators and the tracing subscriber.

```rust
use init_tracing_opentelemetry::{init_propagator, TracingConfig};

// 1. Initialize propagators
init_propagator()?;

// 2. Initialize tracing with configuration
let _guard = TracingConfig::development()
    .init_subscriber()?;

// 3. Application code (traces are now collected)
```

--------------------------------

### LogTimer Enum

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/types.md

Specifies how timestamps are generated for log output. Users can choose to include no timestamp, wall-clock time, or elapsed time since the process started.

```APIDOC
## LogTimer Enum

### Description
Specifies how timestamps are generated for log output. Users can choose to include no timestamp, wall-clock time, or elapsed time since the process started.

### Enum Variants
- **None**: No timestamp.
- **Time**: Wall-clock time.
- **Uptime**: Elapsed time since process start.

### Source
`init-tracing-opentelemetry/src/config.rs`

### Default
`Uptime` in debug builds, `Time` in release builds.
```

--------------------------------

### Typical Usage with Axum Router

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/axum-tracing-opentelemetry.md

Demonstrates how to apply the `OtelAxumLayer` and `OtelInResponseLayer` to an Axum router for tracing requests and responses.

```APIDOC
## Integration Pattern

Typical usage with Axum router:

```rust
use axum::{Router, routing::get};
use axum_tracing_opentelemetry::middleware::{OtelAxumLayer, OtelInResponseLayer};

async fn handler() -> &'static str {
    "OK"
}

let app = Router::new()
    .route("/", get(handler))
    .layer(OtelAxumLayer::default())
    .layer(OtelInResponseLayer::default());

// Bind and serve...
```
```

--------------------------------

### Configure Sampling via SDK Builder

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/configuration.md

Configure sampling using the TracingConfig builder pattern. Ensure to call init_subscriber() to apply the configuration.

```rust
TracingConfig::default()
    // ... other config
    .init_subscriber()?;
```

--------------------------------

### Axum Integration with OpenTelemetry Middleware

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/README.md

Integrates OpenTelemetry tracing middleware into an Axum application. This setup includes layers for request span creation and response header injection.

```rust
use axum::{Router, routing::get};
use axum_tracing_opentelemetry::middleware::{OtelAxumLayer, OtelInResponseLayer};
use init_tracing_opentelemetry::TracingConfig;

async fn handler() -> &'static str {
    "OK"
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let _guard = TracingConfig::development().init_subscriber()?;
    
    let app = Router::new()
        .route("/", get(handler))
        .layer(OtelAxumLayer::default())
        .layer(OtelInResponseLayer::default());
    
    let listener = tokio::net::TcpListener::bind("127.0.0.1:3000").await?;
    axum::serve(listener, app).await?;
    Ok(())
}
```

--------------------------------

### Handle OTelSdkError During Initialization

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/errors.md

Catch OTelSdkError which can occur during OpenTelemetry SDK initialization, tracer provider construction, or export pipeline setup. Verify environment variables and collector accessibility.

```rust
let result = TracingConfig::default()
    .with_otel(true)
    .init_subscriber();

// May fail if OTLP endpoint unreachable during initialization
```

--------------------------------

### Advanced Subscriber Modification with Custom Layer

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/init-tracing-opentelemetry/README.md

Extend the tracing subscriber by adding custom layers and modifying its behavior before application configuration. This example adds a TokioBlockedLayer and customizes log directives.

```rust
use init_tracing_opentelemetry::TracingConfig;
use tokio_blocked::TokioBlockedLayer;
use tracing::info;
use tracing_subscriber::layer::SubscriberExt;

#[tokio::main]
async fn main() {
    let blocked = TokioBlockedLayer::new()
        .with_warn_busy_single_poll(Some(std::time::Duration::from_micros(150)));

    let _guard = TracingConfig::default()
        .with_log_directives("info,tokio::task=trace,tokio::task::waker=warn")
        .with_span_events(tracing_subscriber::fmt::format::FmtSpan::NONE)
        .init_subscriber_ext(|subscriber| subscriber.with(blocked))
        .unwrap();

    tokio::task::spawn(async {
        // BAD!
        // This produces a warning log message.
        info!("blocking!");
        std::thread::sleep(std::time::Duration::from_secs(1));
    })
    .await
    .unwrap();

    tokio::time::sleep(tokio::time::Duration::from_secs(5)).await;
}
```

--------------------------------

### Axum Router Integration with Tracing Layers

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/axum-tracing-opentelemetry.md

Demonstrates how to apply OtelAxumLayer and OtelInResponseLayer to an Axum router for OpenTelemetry tracing. Ensure Axum and Tower dependencies are met.

```rust
use axum::{Router, routing::get};
use axum_tracing_opentelemetry::middleware::{OtelAxumLayer, OtelInResponseLayer};

async fn handler() -> &'static str {
    "OK"
}

let app = Router::new()
    .route("/", get(handler))
    .layer(OtelAxumLayer::default())
    .layer(OtelInResponseLayer::default());

// Bind and serve...
```

--------------------------------

### Configure Tonic Server with OpenTelemetry Layer

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/tonic-tracing-opentelemetry/README.md

Add the OtelGrpcLayer to a tonic server builder to trace incoming requests. A filter can be applied to exclude specific services like health checks.

```rust
    Server::builder()
        // create trace for every request including health_service
        .layer(server::OtelGrpcLayer::default().filter(filters::reject_healthcheck))
        .add_service(health_service)
        .add_service(reflection_service)
        //.add_service(GreeterServer::new(greeter))
        .add_service(GreeterServer::new(greeter))
        .serve_with_shutdown(addr, shutdown_signal())
        .await?;
```

--------------------------------

### Server Integration with OpenTelemetry

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/api-reference/tonic-tracing-opentelemetry.md

Add the `OtelGrpcLayer` to your Tonic server builder to automatically instrument incoming gRPC requests with OpenTelemetry spans. Ensure necessary imports are included.

```rust
use tonic::{transport::Server, Response, Status};
use tonic_tracing_opentelemetry::middleware::server::OtelGrpcLayer;

async fn start_server() -> Result<(), Box<dyn std::error::Error>> {
    let greeter = MyGreeterService::default();
    
    Server::builder()
        .layer(OtelGrpcLayer::default())
        .add_service(GreeterServer::new(greeter))
        .serve("127.0.0.1:50051".parse()?)
        .await?;
    
    Ok(())
}
```

--------------------------------

### Configure Resource Detection Builder

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/configuration.md

Use `DetectResource::default()` to create a builder for resource detection. Fallback service name and version can be provided.

```rust
DetectResource::default()
    .with_fallback_service_name("myapp")
    .with_fallback_service_version("1.0.0")
    .build()
```

--------------------------------

### Development Tracing Configuration

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/README.md

Use `TracingConfig::development()` for pretty-formatted output to stdout with debug logging, including file names and line numbers, and with OpenTelemetry enabled.

```rust
TracingConfig::development()
```

--------------------------------

### Test Tracing with Fake Collector

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/README.md

This test setup uses `FakeCollectorServer` to simulate an OpenTelemetry collector, allowing you to verify that spans are exported correctly. Ensure the tracer is configured to use the fake server's endpoint.

```rust
use fake_opentelemetry_collector::FakeCollectorServer;
use std::time::Duration;

#[tokio::test] async fn test_tracing() -> Result<(), Box<dyn std::error::Error>> {
    let mut server = FakeCollectorServer::start().await?;
    // Configure tracer to use server.endpoint()
    // Run application
    let spans = server.exported_spans(1, Duration::from_millis(500)).await;
    assert!(!spans.is_empty());
    Ok(())
}
```

--------------------------------

### Check for Duplicate OpenTelemetry Versions

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/init-tracing-opentelemetry/README.md

Use `cargo tree` to identify and resolve version conflicts in OpenTelemetry dependencies, which can cause issues with global setup. The recommended solution is to rely on re-exports from this crate to ensure version alignment.

```sh
# Check only one version of opentelemetry should be used
# else issue with setup of global (static variable)
cargo tree -i opentelemetry
```

--------------------------------

### Configure Tonic Client with OpenTelemetry Layer

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/tonic-tracing-opentelemetry/README.md

Apply the OtelGrpcLayer to a tonic channel to enable OpenTelemetry tracing for outgoing requests. Ensure global tracer provider shutdown.

```rust
    let channel = Channel::from_static("http://127.0.0.1:50051")
        .connect()
        .await?; //Devskim: ignore DS137138
    let channel = ServiceBuilder::new()
        .layer(OtelGrpcLayer::default())
        .service(channel);

    let mut client = GreeterClient::new(channel);

    //... 

    opentelemetry::global::shutdown_tracer_provider();
```

--------------------------------

### Set Project Version

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/CONTRIBUTING.md

Use 'mise run set-version <version>' to set a specific version across all workspace crates.

```bash
# Set version across all workspace crates
mise run set-version 0.1.0
```

--------------------------------

### Enable OpenTelemetry Logs Feature

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/init-tracing-opentelemetry/README.md

Add the `init-tracing-opentelemetry` crate with the `otlp` and `logs` features enabled to your Cargo.toml to configure OpenTelemetry log export.

```toml
[dependencies]
init-tracing-opentelemetry = { version = "*", features = ["otlp", "logs"] }
```

--------------------------------

### Handle SetupError for Unknown Propagator

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/errors.md

Catch SetupError when an unknown propagator name is specified in the `OTEL_PROPAGATORS` environment variable. Use only supported propagator names.

```rust
std::env::set_var("OTEL_PROPAGATORS", "unknown");
let result = init_propagator();
// Error: SetupError("unsupported propagators form env OTEL_PROPAGATORS: 'unknown'")
```

--------------------------------

### Configure Log Level Directives

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/configuration.md

Set log level directives, environment fallbacks, and default/OTEL trace levels. The resolution order is directives, then environment variables, then the default level.

```rust
TracingConfig {
    level_config: LevelConfig {
        directives: String,
        env_fallbacks: Vec<String>,
        default_level: LevelFilter,
        otel_trace_level: LevelFilter,
    },
}

.with_log_directives("debug,myapp=trace")
.with_default_level(LevelFilter::INFO)
.with_env_fallback("MY_LOG_LEVEL")
.with_otel_trace_level(LevelFilter::DEBUG)
```

--------------------------------

### Configure OTLP Exporter Environment Variables

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/examples/axum-otlp/README.md

Set these environment variables to configure the OpenTelemetry OTLP exporter for traces. Supports both gRPC and HTTP/protobuf protocols. Ensure the sampler is set appropriately for your needs.

```sh
# For GRPC:
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="http://localhost:4317"
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL="grpc"
export OTEL_TRACES_SAMPLER="always_on"

# For HTTP:
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT="http://127.0.0.1:4318/v1/traces"
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL="http/protobuf"
export OTEL_TRACES_SAMPLER="always_on"
```

--------------------------------

### Test Utilities for Tracing Initialization

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/errors.md

This pattern uses `#[cfg(test)]` and `#[ctor::ctor]` to ensure tracing is initialized for tests without affecting the main application's configuration. It's ideal for enabling tracing in test environments.

```rust
#[cfg(test)]
mod tests {
    use init_tracing_opentelemetry::TracingConfig;
    
    #[ctor::ctor]
    fn init_tracing() {
        let _ = TracingConfig::default()
            .with_global_subscriber(false)
            .init_subscriber();
    }
    
    #[test]
    fn test_something() {
        tracing::info!("Test log visible");
    }
}
```

--------------------------------

### Handle SetupError for Unsupported Propagator

Source: https://github.com/davidb/tracing-opentelemetry-instrumentation-sdk/blob/main/_autodocs/errors.md

Catch SetupError when an unsupported propagator is requested via `init_propagator()`. Ensure the necessary feature (e.g., 'zipkin') is enabled in Cargo.toml.

```rust
use init_tracing_opentelemetry::init_propagator;

// Requires 'zipkin' feature
std::env::set_var("OTEL_PROPAGATORS", "b3");
let result = init_propagator();
// Error: SetupError("unsupported propagators form env OTEL_PROPAGATORS: 'b3', ...")
```