Jaeger

https://www.jaegertracing.io/docs/
Admin
Jaeger is an open-source, end-to-end distributed tracing platform designed to monitor, troubleshoot,...
Tokens:17,456
Snippets:273
Trust Score:10
Update:2 months ago
Show doc for...
Tokens:
Raw
### Install and Start Jaeger as a Windows Service

Source: https://www.jaegertracing.io/docs/2.20/deployment/windows

Commands to register the Jaeger executable with nssm, configure log output paths, set a service description, and start the service.

```bash
nssm install Jaeger C:\Jaeger\jaeger.exe --config=C:\Jaeger\config.yaml

nssm set Jaeger AppStdout C:\Jaeger\jaeger.out.log
nssm set Jaeger AppStderr C:\Jaeger\jaeger.err.log
nssm set Jaeger Description Jaeger All-in-One

nssm start Jaeger
```

--------------------------------

### Example UI Configuration File

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

A comprehensive example of the JSON structure used to configure various UI features, including dependencies, themes, tracking, and custom menu links.

```json
{
  "dependencies": {
    "dagMaxNumServices": 200,
    "menuEnabled": true
  },
  "monitor": {
    "menuEnabled": true
  },
  "themes": {
    "enabled": true
  },
  "archiveEnabled": true,
  "tracking": {
    "gaID": "UA-000000-2",
    "trackErrors": true
  },
  "menu": [
    {
      "label": "About Jaeger",
      "items": [
        {
          "label": "GitHub",
          "url": "https://github.com/jaegertracing/jaeger"
        },
        {
          "label": "Docs",
          "url": "https://www.jaegertracing.io/docs/latest/"
        }
      ]
    }
  ],
  "search": {
    "maxLookback": {
      "label": "2 Days",
      "value": "2d"
    },
    "maxLimit": 1500
  },
  "linkPatterns": [{
    "type": "process",
    "key": "jaeger.version",
    "url": "https://github.com/jaegertracing/jaeger-client-java/releases/tag/#{jaeger.version}",
    "text": "Information about Jaeger release #{jaeger.version}"
  },
  {
    "type": "tags",
    "key": "uniqueId",
    "url": "https://mykibana.com/uniqueId=#{uniqueId}&traceId=#{trace.traceID}",
    "text": "Redirect to kibana to view log"
  },
  {
    "type": "traces",
    "url": "https://my-logs.server?from=#{startTime | add -60000000 | epoch_micros_to_date_iso}&to=#{endTime | add 60000000 | epoch_micros_to_date_iso}'",
    "text": "Redirect to kibana to view log with formatted dates"
  }],
  "traceIdDisplayLength": 20,
  "criticalPathEnabled": true,
  "topTagPrefixes": ["http.", "db."]
}
```

--------------------------------

### Example metric output with labels

Source: https://www.jaegertracing.io/docs/2.20/operations/monitoring

A formatted example of a metric showing specific labels for receivers and service metadata.

```text
otelcol_receiver_accepted_spans{
    receiver="otlp",
    service_instance_id="f91d66c2-0445-42bf-a062-32aaed09facf",
    service_name="jaeger",
    service_version="2.0.0",
    transport="http"
} 44
```

--------------------------------

### Run custom Jaeger binary

Source: https://www.jaegertracing.io/docs/2.20/deployment/custom-distribution

Starts the generated binary using a custom configuration file.

```bash
./build/jaeger --config config.yaml
```

--------------------------------

### Example Successful Metric Output

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

Sample output showing successful metric store requests and latency buckets.

```text
jaeger_metricstore_requests_total{operation="get_call_rates",result="ok"} 18
jaeger_metricstore_requests_total{operation="get_error_rates",result="ok"} 18
jaeger_metricstore_requests_total{operation="get_latencies",result="ok"} 36

jaeger_metricstore_latency_bucket{operation="get_call_rates",result="ok",le="0.005"} 5
jaeger_metricstore_latency_bucket{operation="get_call_rates",result="ok",le="0.01"} 13
jaeger_metricstore_latency_bucket{operation="get_call_rates",result="ok",le="0.025"} 18

jaeger_metricstore_latency_bucket{operation="get_error_rates",result="ok",le="0.005"} 7
jaeger_metricstore_latency_bucket{operation="get_error_rates",result="ok",le="0.01"} 13
jaeger_metricstore_latency_bucket{operation="get_error_rates",result="ok",le="0.025"} 18

jaeger_metricstore_latency_bucket{operation="get_latencies",result="ok",le="0.005"} 7
jaeger_metricstore_latency_bucket{operation="get_latencies",result="ok",le="0.01"} 25
jaeger_metricstore_latency_bucket{operation="get_latencies",result="ok",le="0.025"} 36
```

--------------------------------

### Start ClickHouse Server

Source: https://www.jaegertracing.io/docs/2.20/storage/clickhouse

Use Docker Compose to launch a ClickHouse instance with the necessary configuration for Jaeger.

```bash
docker compose -f docker-compose/clickhouse/docker-compose.yml up -d
```

--------------------------------

### Configure Cassandra Storage in Jaeger

Source: https://www.jaegertracing.io/docs/2.20/storage/cassandra

Example configuration for the Jaeger storage extension using a Cassandra backend.

```yaml
extensions:
  jaeger_storage:
    backends:
      some_storage:
        cassandra:
          schema:
            create: true
            keyspace: jaeger_dc1
            datacenter: dc1
            trace_ttl: 48h
            dependencies_ttl: 48d
            compaction_window: 2h
            replication_factor: 1
```

--------------------------------

### Configure jaeger_query extension

Source: https://www.jaegertracing.io/docs/2.20/deployment/configuration

Example configuration for the jaeger_query extension, including storage backends, UI settings, and server endpoints.

```yaml

jaeger_query:
  storage:
    traces: some_trace_storage
    metrics: some_metrics_storage
  base_path: /
  ui:
    config_file: /etc/jaeger/ui-config.json
    log_access: true
  grpc:
    endpoint: 0.0.0.0:16685
  http:
    endpoint: 0.0.0.0:16686

```

--------------------------------

### Run Jaeger with Custom Configuration

Source: https://www.jaegertracing.io/docs/2.20/getting-started

Starts the Jaeger container using a local configuration file mapped into the container filesystem.

```bash
docker run --rm --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 5778:5778 \
  -p 9411:9411 \
  -v /path/to/local/config.yaml:/jaeger/config.yaml \
  cr.jaegertracing.io/jaegertracing/jaeger:2.20.0 \
  --config /jaeger/config.yaml
```

--------------------------------

### Configure remote_sampling extension

Source: https://www.jaegertracing.io/docs/2.20/deployment/configuration

Example configuration for the remote_sampling extension, supporting either file-based or adaptive sampling strategies.

```yaml

remote_sampling:
  # You can either use file or adaptive sampling strategy in remote_sampling
  # file:
  #   path: ./cmd/jaeger/sampling-strategies.json
  adaptive:
    sampling_store: some_store
    initial_sampling_probability: 0.1
  http:
  grpc:

```

--------------------------------

### Run HotROD Demo via Docker Compose

Source: https://www.jaegertracing.io/docs/2.20/getting-started

Clones the Jaeger repository and starts the HotROD demo application using Docker Compose.

```bash
export JAEGER_VERSION=2.7.0 #Pick the newest version
git clone https://github.com/jaegertracing/jaeger.git jaeger
cd jaeger/examples/hotrod
docker compose up
# press Ctrl-C to exit
```

--------------------------------

### Example Prometheus Query Log Output

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

A sample log entry showing the query structure used by Jaeger to fetch metrics from Prometheus.

```json
2024-11-26T19:09:43.152Z debug metricsstore/reader.go:258 Prometheus query results
{
  "kind": "extension",
  "name": "jaeger_storage",
  "results": "",
  "query": "sum(rate(traces_span_metrics_calls_total{service_name =~ \"redis\", span_kind =~ \"SPAN_KIND_SERVER\"}[10m])) by (service_name,span_name)",
  "range": {
    "Start": "2024-11-26T19:04:43.14Z",
    "End": "2024-11-26T19:09:43.14Z",
    "Step": 60000000000
  }
}
```

--------------------------------

### Example Failed Metric Output

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

Sample output showing incremented error counts when reading metrics from Prometheus fails.

```text
jaeger_metricstore_requests_total{operation="get_call_rates",result="err"} 4
jaeger_metricstore_requests_total{operation="get_error_rates",result="err"} 4
jaeger_metricstore_requests_total{operation="get_latencies",result="err"} 8
```

--------------------------------

### GET /api/dependencies

Source: https://www.jaegertracing.io/docs/2.20/architecture/apis

Retrieves the service dependencies graph as a list of edges.

```APIDOC
## GET /api/dependencies

### Description
Retrieves the service dependencies graph, returned as a list of edges represented as tuples (caller, callee, count).

### Method
GET

### Endpoint
/api/dependencies

### Parameters
#### Query Parameters
- **endTs** (number) - Required - The end of the time interval in milliseconds since epoch.
- **lookback** (number) - Required - The length of the time interval in milliseconds.
```

--------------------------------

### Run Jaeger All-in-One via Docker

Source: https://www.jaegertracing.io/docs/2.20/getting-started

Starts the Jaeger all-in-one container with default ports exposed for trace data collection and UI access.

```bash
docker run --rm --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 5778:5778 \
  -p 9411:9411 \
  cr.jaegertracing.io/jaegertracing/jaeger:2.20.0
```

--------------------------------

### Initialize Elasticsearch for Rollover

Source: https://www.jaegertracing.io/docs/2.20/storage/elasticsearch

Prepares Elasticsearch by creating index templates, the first rollover index, and read/write aliases. Add -e ARCHIVE=true to initialize archive storage.

```bash
docker run -it --rm --net=host \
  jaegertracing/jaeger-es-rollover:latest \
  init http://localhost:9200 # <1>
```

--------------------------------

### Configure UI via Query Extension

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

Specify the path to the UI configuration file within the Jaeger query extension settings.

```yaml
extensions:
  jaeger_query:
    ui:
      config_file: /path/to/config-ui.json
```

--------------------------------

### Initialize Cassandra Schema Manually

Source: https://www.jaegertracing.io/docs/2.20/storage/cassandra

Commands to initialize the Cassandra schema using the provided script and cqlsh or a Docker container.

```bash
MODE=test sh ./internal/storage/v1/cassandra/schema/create.sh | cqlsh
```

```bash
docker run \
  -e CQLSH_HOST={server IP address}  \
  -e MODE=prod \
  jaegertracing/jaeger-cassandra-schema:2.20.0
```

--------------------------------

### List gRPC services via reflection

Source: https://www.jaegertracing.io/docs/2.20/architecture/apis

Use the grpc_cli tool to list available gRPC services on a Jaeger instance. Note that due to protobuf compatibility issues, only the list command is fully supported.

```bash
grpc_cli ls localhost:16685
  grpc.health.v1.Health
  grpc.reflection.v1.ServerReflection
  grpc.reflection.v1alpha.ServerReflection
  jaeger.api_v2.QueryService
  jaeger.api_v2.metrics.MetricsQueryService
  jaeger.api_v3.QueryService
```

--------------------------------

### Run Jaeger with a configuration file

Source: https://www.jaegertracing.io/docs/2.20/deployment/configuration

Execute the Jaeger binary using a specified YAML configuration file.

```bash
jaeger --config config.yaml
```

--------------------------------

### Build custom distribution with ocb

Source: https://www.jaegertracing.io/docs/2.20/deployment/custom-distribution

Executes the builder with the specified configuration file. The --skip-strict-versioning flag is necessary to bypass version mismatches with upstream dependencies.

```bash
ocb --config builder.yaml --skip-strict-versioning
```

--------------------------------

### Override configuration values via command line

Source: https://www.jaegertracing.io/docs/2.20/deployment/configuration

Use the --set flag to override specific configuration properties at runtime, useful for all-in-one deployments.

```bash
--set=receivers.otlp.protocols.grpc.endpoint=0.0.0.0:4317
```

--------------------------------

### Configure Elasticsearch/OpenSearch Metric Backends

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

Points the metric backend to the same storage configuration used for traces in Elasticsearch or OpenSearch.

```yaml
extensions:
  jaeger_storage:
    backends:
      elasticsearch_trace_storage: &elasticsearch_config
        elasticsearch:
          ...
    metric_backends:
      elasticsearch_trace_storage: *elasticsearch_config # This points the metric backend to the same Elasticsearch trace storage configuration
```

```yaml
extensions:
  jaeger_storage:
    backends:
      opensearch_trace_storage: &opensearch_config
        opensearch:
          ...
    metric_backends:
      opensearch_trace_storage: *opensearch_config # This points the metric backend to the same OpenSearch trace storage configuration
```

--------------------------------

### Cross-compile custom distribution

Source: https://www.jaegertracing.io/docs/2.20/deployment/custom-distribution

Builds the distribution for a specific target architecture and operating system using environment variables.

```bash
GOOS=linux GOARCH=arm64 ocb --config builder.yaml
```

--------------------------------

### Enable UI Themes

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

Toggle the theme selection button in the UI navigation bar.

```json
{
  "themes": {
    "enabled": true
  }
}
```

--------------------------------

### Run tracegen utility

Source: https://www.jaegertracing.io/docs/2.20/operations/tools

Executes the tracegen utility via Docker to generate a continuous stream of spans for stress testing.

```bash
docker run \
  --rm \
  jaegertracing/jaeger-tracegen:2.20.0 
```

--------------------------------

### Enable Critical Path Visualization

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

Toggle the critical path highlighting feature in the trace view.

```json
{
  "criticalPathEnabled": true
}
```

--------------------------------

### Generate Test Data

Source: https://www.jaegertracing.io/docs/2.20/storage/clickhouse

Use the trace generator tool to populate the ClickHouse database with sample trace data.

```bash
go run ./cmd/tracegen -duration 10s -workers 3 -pause 250ms
```

--------------------------------

### Configure Custom Menu Links

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

Define individual links or grouped dropdown menus for the global navigation bar.

```json
{
  "label": "Some text here",
  "url": "https://example.com"
}
```

```json
{
  "label": "Dropdown button",
  "items": [ ]
}
```

--------------------------------

### Configure jaeger_storage extension

Source: https://www.jaegertracing.io/docs/2.20/deployment/configuration

Define tracing and metric backends within the jaeger_storage extension block.

```yaml
jaeger_storage:
  backends:
    some_trace_storage:
      memory:
        max_traces: 100000
  metric_backends:
    some_metrics_storage:
      prometheus:
        endpoint: http://prometheus:9090
        normalize_calls: true
        normalize_duration: true
```

--------------------------------

### Define Metric Backends for PromQL

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

Configures a remote PromQL-compatible storage backend within the jaeger_storage extension.

```yaml
extensions:
  jaeger_storage:
    backends:
      some_trace_storage:
        ...
    metric_backends:
      some_metrics_storage:
        prometheus:
          endpoint: http://prometheus:9090
```

--------------------------------

### Run Jaeger with ClickHouse

Source: https://www.jaegertracing.io/docs/2.20/storage/clickhouse

Execute the Jaeger binary with the ClickHouse configuration and the required feature gate enabled.

```bash
go run ./cmd/jaeger --config cmd/jaeger/config-clickhouse.yaml --feature-gates=storage.clickhouse
```

--------------------------------

### Define multiple storage backends

Source: https://www.jaegertracing.io/docs/2.20/storage

Configure multiple storage backends by defining them under the jaeger_storage backends section.

```yaml
  jaeger_storage:
    backends:
      some_store:
        memory:
          max_traces: 100000
      another_store:
        memory:
          max_traces: 100000
```

--------------------------------

### Configure cqlshrc for TLS

Source: https://www.jaegertracing.io/docs/2.20/storage/cassandra

Configuration file for cqlsh to enable TLS client-to-node connections.

```ini
# Creating schema in a cassandra cluster requiring client TLS certificates.
#
# Create a volume for the schema docker container containing four files:
# cqlshrc: this file
# ca-cert: the cert authority for your keys
# client-key: the keyfile for your client
# client-cert: the cert file matching client-key
#
# if there is any sort of DNS mismatch and you want to ignore server validation
# issues, then uncomment validate = false below.
#
# When running the container, map this volume to /root/.cassandra and set the
# environment variable CQLSH_SSL=--ssl
[ssl]
certfile = ~/.cassandra/ca-cert
userkey = ~/.cassandra/client-key
usercert = ~/.cassandra/client-cert
# validate = false
```

--------------------------------

### Configure UI base_path

Source: https://www.jaegertracing.io/docs/2.20/deployment/configuration

Sets the internal URL prefix under which Jaeger registers its HTTP routes.

```yaml

extensions:
  jaeger_query:
    base_path: /jaeger

```

--------------------------------

### Minimal builder.yaml manifest

Source: https://www.jaegertracing.io/docs/2.20/deployment/custom-distribution

A configuration file for the OpenTelemetry Collector Builder to create a custom Jaeger binary with core components.

```yaml
dist:
  module: github.com/example/my-jaeger
  name: jaeger
  description: My custom Jaeger distribution
  output_path: ./build

telemetry:
  gomod: github.com/jaegertracing/jaeger v2.19.0
  import: github.com/jaegertracing/jaeger/components/telemetry

extensions:
  - gomod: github.com/jaegertracing/jaeger v2.19.0
    import: github.com/jaegertracing/jaeger/components/extension/jaegerstorage
  - gomod: github.com/jaegertracing/jaeger v2.19.0
    import: github.com/jaegertracing/jaeger/components/extension/jaegerquery
  - gomod: github.com/jaegertracing/jaeger v2.19.0
    import: github.com/jaegertracing/jaeger/components/ext/extension/healthcheckv2extension

receivers:
  - gomod: github.com/jaegertracing/jaeger v2.19.0
    import: github.com/jaegertracing/jaeger/components/ext/receiver/otlpreceiver

exporters:
  - gomod: github.com/jaegertracing/jaeger v2.19.0
    import: github.com/jaegertracing/jaeger/components/exporter/storageexporter

processors:
  - gomod: github.com/jaegertracing/jaeger v2.19.0
    import: github.com/jaegertracing/jaeger/components/ext/processor/batchprocessor
```

--------------------------------

### Integrate Trace Page in Embedded Mode

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

Configure the trace page layout using specific query parameters for collapsing headers or hiding UI elements.

```text
http://localhost:16686/trace/{trace-id}?uiEmbed=v0
```

```text
http://localhost:16686/trace/{trace-id}?
    uiEmbed=v0&
    uiTimelineCollapseTitle=1
```

```text
http://localhost:16686/trace/{trace-id}?
    uiEmbed=v0&
    uiTimelineHideMinimap=1
```

```text
http://localhost:16686/trace/{trace-id}?
    uiEmbed=v0&
    uiTimelineHideSummary=1
```

```text
http://localhost:16686/trace/{trace-id}?
    uiEmbed=v0&
    uiTimelineHideMinimap=1&
    uiTimelineHideSummary=1
```

--------------------------------

### Add custom processor to manifest

Source: https://www.jaegertracing.io/docs/2.20/deployment/custom-distribution

Include a new processor in the manifest by specifying its gomod path and import path.

```yaml
processors:
  # ... existing processors ...
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor v0.123.0
    import: github.com/open-telemetry/opentelemetry-collector-contrib/processor/resourcedetectionprocessor
```

--------------------------------

### Configure Monitor Tab Empty State

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

Customize the display content shown in the Monitor tab when no monitoring data is available.

```json
{
  "monitor": {
    "menuEnabled": true,
    "docsLink": "https://www.jaegertracing.io/docs/latest/spm/",
    "emptyState": {
      "mainTitle": "Get started with Service Performance Monitoring",
      "subTitle": "No data available",
      "description": "Configure your services to emit RED metrics.",
      "info": "Additional information text",
      "alert": {
        "message": "SPM requires metrics storage backend",
        "type": "info"
      }
    }
  }
}
```

--------------------------------

### Access Trace in Embedded Mode

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

Use the uiEmbed=v0 query parameter to enable the embedded layout for a specific trace.

```text
http://localhost:16686/trace/abc123?uiEmbed=v0
```

--------------------------------

### Configure SpanMetrics Connector for PromQL

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

Defines the exporter, connector, and pipeline configuration required to bridge traces to metrics for PromQL-compatible backends.

```yaml
# Declare an exporter for metrics produced by the connector.
# For example, a Prometheus server may be configured to scrape
# the metrics from this endpoint.
exporters:
  prometheus:
    endpoint: "0.0.0.0:8889"

# Declare spanmetrics connector.
connectors:
  spanmetrics:
    # any connector configuration options
    ...

# Enable the spanmetrics connector to bridge
# the traces pipeline into the metrics pipeline.
service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger_storage_exporter, spanmetrics]
    metrics/spanmetrics:
      receivers: [spanmetrics]
      exporters: [prometheus]
```

--------------------------------

### Run Jaeger ES Rollover Initializer with ILM

Source: https://www.jaegertracing.io/docs/2.20/storage/elasticsearch

Executes the rollover initializer container with ILM support enabled via environment variables.

```bash
docker run -it --rm --net=host\
  -e ES_USE_ILM=true \
  jaegertracing/jaeger-es-rollover:latest \
  init http://localhost:9200 # <1>
```

--------------------------------

### Integrate Search Page in Embedded Mode

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

Configure the search page layout by appending query parameters to the URL.

```text
http://localhost:16686/search?
    service=my-service&
    start=1543917759557000&
    end=1543921359557000&
    limit=20&
    lookback=1h&
    maxDuration&
    minDuration&
    uiEmbed=v0
```

```text
http://localhost:16686/search?
    service=my-service&
    start=1543917759557000&
    end=1543921359557000&
    limit=20&
    lookback=1h&
    maxDuration&
    minDuration&
    uiEmbed=v0&
    uiSearchHideGraph=1
```

--------------------------------

### Execute Index Rollover

Source: https://www.jaegertracing.io/docs/2.20/storage/elasticsearch

Triggers the rollover API to create a new write index based on specified conditions like age or size.

```bash
docker run -it --rm --net=host \
  -e CONDITIONS='{"max_age": "2d"}' \
  jaegertracing/jaeger-es-rollover:latest \
  rollover  http://localhost:9200 # <1>
```

--------------------------------

### Configure Bearer Token for Prometheus Queries

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

Provide a token file path to authenticate Jaeger queries against a Prometheus server requiring a bearer token.

```yaml
extensions:
  jaeger_storage:
    backends:
      ...
    metric_backends:
      some_metrics_storage:
        prometheus:
          endpoint: http://prometheus:9090
          token_file_path: /path/to/token/file
          token_override_from_context: true
```

--------------------------------

### Configure max_clock_skew_adjust

Source: https://www.jaegertracing.io/docs/2.20/deployment/configuration

Sets the maximum allowed clock skew adjustment for traces. Setting this to 0s disables the adjustment.

```yaml

extensions:
  jaeger_query:
    max_clock_skew_adjust: 30s

```

--------------------------------

### Run anonymizer utility

Source: https://www.jaegertracing.io/docs/2.20/operations/tools

Executes the anonymizer utility via Docker to generate a sanitized JSON version of a specific trace.

```bash
docker run \
  --rm \
  --volume /tmp:/tmp \
  jaegertracing/jaeger-anonymizer:2.20.0 \
  --trace-id <TRACE_ID> \
  --query-host-port <JAEGER_QUERY_HOST_PORT>
```

--------------------------------

### Configure remote sampling file path

Source: https://www.jaegertracing.io/docs/2.20/architecture/sampling

Specify the path to the JSON file containing sampling strategies within the Jaeger configuration.

```yaml
remote_sampling:
  file:
    path: ./cmd/jaeger/sampling-strategies.json
```

--------------------------------

### Configure Jaeger storage exporter

Source: https://www.jaegertracing.io/docs/2.20/deployment/configuration

Defines the trace storage backend and queue parameters for the jaeger_storage_exporter.

```yaml
jaeger_storage_exporter:
  trace_storage: some_trace_storage
  queue:
    num_consumers: 10
    queue_size: 100
```

--------------------------------

### Configure Jaeger Internal Telemetry

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

Enable the metrics endpoint in the Jaeger configuration file to expose internal telemetry.

```yaml
service:
  ...
  telemetry:
    resource:
      service.name: jaeger
    metrics:
      level: detailed
      address: 0.0.0.0:8888
```

--------------------------------

### Configure Trace Graph Memory

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

Set the memory limit for the GraphViz module used in trace graph rendering.

```json
{
  "traceGraph": {
    "layoutManagerMemory": 33554432
  }
}
```

--------------------------------

### Verify Metric Store Connectivity

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

Use curl to inspect Jaeger internal metrics for successful metric store requests.

```bash
curl -s http://jaeger:8888/metrics | grep jaeger_metricstore
```

--------------------------------

### POST /api/traces

Source: https://www.jaegertracing.io/docs/2.20/architecture/apis

Endpoint for submitting legacy Thrift-encoded trace batches.

```APIDOC
## POST /api/traces

### Description
Submit a legacy Thrift-encoded Batch struct to the Jaeger collector.

### Method
POST

### Endpoint
/api/traces

### Request Body
- **Batch** (Thrift binary) - Required - The trace batch encoded using Thrift's binary encoding.

### Headers
- **Content-Type**: application/vnd.apache.thrift.binary
```

--------------------------------

### Configure Visual Studio Code for remote debugging

Source: https://www.jaegertracing.io/docs/2.20/operations/troubleshooting

Use this launch configuration in your local Jaeger source clone to attach to a remote debug session on port 12345.

```json
$ cat .vscode/launch.json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Launch remote",
            "type": "go",
            "request": "attach",
            "mode": "remote",
            "remotePath": "",
            "port": 12345,
            "host": "127.0.0.1",
            "cwd": "${workspaceRoot}",
        }
    ]
}
```

--------------------------------

### Reference Metrics Store in Jaeger Query

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

Links the defined metrics storage to the Jaeger query extension.

```yaml
extensions:
  jaeger_query:
    traces: some_trace_storage
    metrics_storage: some_metrics_storage
```

--------------------------------

### Configure Trace ID Display Length

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

Set the number of characters to display for trace IDs in the UI.

```json
{
  "traceIdDisplayLength": 12
}
```

--------------------------------

### Configure Prometheus for Jaeger v2

Source: https://www.jaegertracing.io/docs/2.20/operations/monitoring

Use this configuration to enable Prometheus to scrape metrics from the Jaeger v2 internal telemetry port.

```yaml
scrape_configs:
  - job_name: 'jaeger-v2'
    static_configs:
      - targets: ['jaeger:8888']  # Internal telemetry port
    scrape_interval: 15s
    metrics_path: /metrics
```

--------------------------------

### Verify Remote Sampling Strategy

Source: https://www.jaegertracing.io/docs/2.20/operations/troubleshooting

Use curl to query the Jaeger collector API and verify the sampling strategy returned for a specific service.

```bash
$ curl "jaeger-collector:14268/api/sampling?service=foobar"
{"strategyType":"PROBABILISTIC","probabilisticSampling":{"samplingRate":0.001}}
```

--------------------------------

### Set Storage in Jaeger Query for Elasticsearch/OpenSearch

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

Configures the Jaeger query extension to use the same backend for both traces and metrics.

```yaml
extensions:
  jaeger_query:
    storage:
      traces: elasticsearch_trace_storage # Must match the backend defined in jaeger_storage extension
      metrics: elasticsearch_trace_storage # Use the same storage for metrics
```

```yaml
extensions:
  jaeger_query:
    storage:
      traces: opensearch_trace_storage # Must match the backend defined in jaeger_storage extension
      metrics: opensearch_trace_storage # Use the same storage for metrics
```

--------------------------------

### Enable Debug Logging for Jaeger

Source: https://www.jaegertracing.io/docs/2.20/architecture/spm

Set the log level to debug to inspect the queries being sent to Prometheus.

```yaml
service:
  telemetry:
    ...
    logs:
      level: debug
```

--------------------------------

### Define Top Tag Prefixes

Source: https://www.jaegertracing.io/docs/2.20/deployment/frontend-ui

Specify prefixes for span tags to prioritize their display order in span details.

```json
{
  "topTagPrefixes": ["http.", "db.", "rpc."]
}
```

--------------------------------

### Define sampling strategies JSON

Source: https://www.jaegertracing.io/docs/2.20/architecture/sampling

Structure for defining service-specific and default sampling strategies, including probabilistic and rate-limiting types.

```json
{
  "service_strategies": [
    {
      "service": "foo",
      "type": "probabilistic",
      "param": 0.8,
      "operation_strategies": [
        {
          "operation": "op1",
          "type": "probabilistic",
          "param": 0.2
        },
        {
          "operation": "op2",
          "type": "probabilistic",
          "param": 0.4
        }
      ]
    },
    {
      "service": "bar",
      "type": "ratelimiting",
      "param": 5
    }
  ],
  "default_strategy": {
    "type": "probabilistic",
    "param": 0.5,
    "operation_strategies": [
      {
        "operation": "/health",
        "type": "probabilistic",
        "param": 0.0
      },
      {
        "operation": "/metrics",
        "type": "probabilistic",
        "param": 0.0
      }
    ]
  }
}
```

--------------------------------

### Configure environment variable interpolation in YAML

Source: https://www.jaegertracing.io/docs/2.20/deployment/configuration

Use the ${env:VAR:-default} syntax within the YAML configuration to allow environment variable overrides for settings like endpoints.

```yaml
receivers:
  otlp:
    protocols:
      grpc:
        endpoint: "${env:JAEGER_LISTEN_HOST:-localhost}:4317"
      http:
        endpoint: "${env:JAEGER_LISTEN_HOST:-localhost}:4318"
```

--------------------------------

### POST /api/v1/spans

Source: https://www.jaegertracing.io/docs/2.20/architecture/apis

Endpoint for submitting Zipkin v1 format spans.

```APIDOC
## POST /api/v1/spans

### Description
Submit spans in Zipkin v1 JSON or Thrift formats.

### Method
POST

### Endpoint
/api/v1/spans
```

--------------------------------

### Run Jaeger in a Docker Container

Source: https://www.jaegertracing.io/docs/2.20/deployment

Executes the Jaeger container with mapped ports and a mounted configuration file.

```bash
docker run --rm --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  -p 5778:5778 \
  -p 9411:9411 \
  -v /path/to/local/config.yaml:/jaeger/config.yaml \
  jaegertracing/jaeger:2.20.0 \
  --config /jaeger/config.yaml
```

--------------------------------

### Reference archive storage in query configuration

Source: https://www.jaegertracing.io/docs/2.20/storage

Assign a specific backend to the archive storage by referencing its name in the jaeger_query configuration.

```yaml
  jaeger_query:
    storage:
      traces: some_store
      traces_archive: another_store
```

--------------------------------

### Configure memory storage max_traces

Source: https://www.jaegertracing.io/docs/2.20/storage/memory

Sets a limit on the number of traces stored in memory to prevent unbounded memory usage.

```yaml

  jaeger_storage:
    backends:
      some_store:
        memory:
          max_traces: 100000

```

--------------------------------

### POST /api/v2/spans

Source: https://www.jaegertracing.io/docs/2.20/architecture/apis

Endpoint for submitting Zipkin v2 format spans.

```APIDOC
## POST /api/v2/spans

### Description
Submit spans in Zipkin v2 JSON or Protobuf formats.

### Method
POST

### Endpoint
/api/v2/spans
```

--------------------------------

### Retrieve Jaeger metrics via curl

Source: https://www.jaegertracing.io/docs/2.20/operations/monitoring

Use this command to fetch metrics from the Jaeger collector endpoint.

```bash
curl -s http://jaeger-collector:8888/metrics
```

--------------------------------

### Create Elasticsearch ILM Policy

Source: https://www.jaegertracing.io/docs/2.20/storage/elasticsearch

Defines an ILM policy named jaeger-ilm-policy to manage index rollover and deletion.

```bash
curl -X PUT \
http://localhost:9200/_ilm/policy/jaeger-ilm-policy \
-H 'Content-Type: application/json; charset=utf-8' \
--data-binary @- << EOF
{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "rollover": {
            "max_age": "1m"
          },
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "2m",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}
EOF
```

--------------------------------

### Key Jaeger metrics

Source: https://www.jaegertracing.io/docs/2.20/operations/monitoring

These metrics track span ingestion and export status within the collector.

```text
otelcol_receiver_accepted_spans
otelcol_receiver_refused_spans

otelcol_exporter_sent_spans
otelcol_exporter_send_failed_spans
```
Tokens:
Raw