### Access Log Example

Source: https://github.com/istio/ztunnel/blob/master/README.md

An example of an access log entry, showing connection details. Note that the real logs are on a single line and the format is considered unstable.

```text
2024-04-11T15:38:42.182974Z  INFO access: connection complete
    src.addr=10.244.0.24:46238 src.workload="shell-6d8bcd654d-t88gp" src.namespace="default" src.identity="spiffe://cluster.local/ns/default/sa/default"
    dst.addr=10.244.0.42:15008 dst.hbone_addr=10.96.108.116:80 dst.service="echo.default.svc.cluster.local"
    direction="outbound" bytes_sent=67 bytes_recv=490 duration="13ms"
```

--------------------------------

### Set Up In-Pod Mode Network Namespace

Source: https://github.com/istio/ztunnel/blob/master/Development.md

These commands create a network namespace for a simulated pod and configure the necessary veth devices for in-pod testing. This setup is required before running the fake server or ztunnel in in-pod mode.

```shell
ip netns add pod1
ip -n pod1 link set lo up

# veth device
ip link add pod1 type veth peer name pod1-eth0
# move one end to the pod
ip link set pod1-eth0 netns pod1
# configure the veth devices
ip link set pod1 up
ip -n pod1 link set pod1-eth0 up
ip addr add dev pod1 10.0.0.1/24
ip -n pod1 addr add dev pod1-eth0 10.0.0.2/24
```

--------------------------------

### Run ztunnel with Local Kind Setup

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Executes ztunnel in a local Kind cluster environment. This involves port-forwarding Istiod, copying the service account token, and running the ztunnel proxy with specific environment variables for debugging and local development.

```shell
kubectl port-forward -n istio-system svc/istiod 15012:15012 &
mkdir -p ./var/run/secrets/tokens/
kubectl exec -n istio-system fake-tunnel-worker1 -- cat /var/run/secrets/tokens/istio-token > ./var/run/secrets/tokens/istio-token
xargs env <<EOF
INPOD_UDS=/tmp/worker1-ztunnel/ztunnel.sock
CLUSTER_ID=Kubernetes
RUST_LOG=debug
PROXY_MODE="shared"
ISTIO_META_DNS_CAPTURE="true"
ISTIO_META_DNS_PROXY_ADDR="127.0.0.1:15053"
SERVICE_ACCOUNT=ztunnel
POD_NAMESPACE=istio-system
POD_NAME=ztunnel-worker1
CA_ROOT_CA=/tmp/istio-root.pem
XDS_ROOT_CA=/tmp/istio-root.pem
CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUNNER="sudo -E"
cargo run proxy ztunnel
EOF
```

--------------------------------

### Setup iptables redirection for ztunnel

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Use this script to set up iptables rules for redirecting traffic from a specified user to a given port. Ensure the local.sh script is sourced and ZTUNNEL_REDIRECT_USER is exported before running.

```shell
source ./scripts/local.sh
export ZTUNNEL_REDIRECT_USER="iptables1"
redirect-user-setup
redirect-to 15001
```

--------------------------------

### Get Rust Version for Building

Source: https://github.com/istio/ztunnel/blob/master/README.md

Run this command to determine the Rust version required by the build-tools image for Ztunnel development.

```shell
$ BUILD_WITH_CONTAINER=1 make rust-version
```

--------------------------------

### Clean Up Stale Network Namespaces

Source: https://github.com/istio/ztunnel/blob/master/tests/README.md

This shell command lists all network namespaces, filters for those starting with 'test_', and deletes them. Use this to clean up any broken namespace states during testing.

```shell
ip -j netns ls | jq -r '.[].name' | grep '^test_' | xargs -n1 sudo ip netns del
```

--------------------------------

### Set Istiod Custom Host Environment Variable

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Sets an environment variable on the Istiod deployment to specify a custom host, which is necessary for local development setups where Istiod might be running on localhost.

```shell
kubectl set env -n istio-system deploy/istiod ISTIOD_CUSTOM_HOST=localhost
```

--------------------------------

### Run ztunnel in In-Pod Shared Mode

Source: https://github.com/istio/ztunnel/blob/master/Development.md

This command runs ztunnel in shared proxy mode within an in-pod setup. It requires specific environment variables like `INPOD_UDS`, `FAKE_CA`, `XDS_ADDRESS`, and `LOCAL_XDS_PATH`. Consider setting `CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUNNER` for sudo execution.

```shell
RUST_LOG=debug PROXY_MODE=shared INPOD_UDS=/tmp/ztunnel FAKE_CA="true" XDS_ADDRESS="" LOCAL_XDS_PATH=./examples/localhost.yaml cargo run --features testing
```

--------------------------------

### Run Ztunnel Benchmarks

Source: https://github.com/istio/ztunnel/blob/master/benches/README.md

Use `cargo bench` to execute benchmarks. Options include running quickly or with a specified number of samples. You can also save and compare against a baseline.

```shell
cargo bench # Just run benchmarks
cargo bench -- --quick # Just run benchmarks, with less samples
cargo bench -- --profile-time 10 # run benchmarks with cpu profile; results will be in out/rust/criterion/<group>/<test>/profile/profile.pb
# Compare to a baseline
cargo bench -- --save-baseline <name> # save baseline
# ...change something...
cargo bench -- --baseline <name> # compare against it
```

--------------------------------

### Run Workload Commands in a Network Namespace

Source: https://github.com/istio/ztunnel/blob/master/tests/README.md

Use the `run` method on a workload builder to execute commands within a dedicated network namespace. This simulates a single node in Kubernetes for testing network-dependent scenarios.

```rust
manager
    .workload_builder("client")
    .on_local_node()
    .register()? 
    .run(|| { ... commands run here are in a network namespace ...})
```

--------------------------------

### Build ztunnel Binary

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Build the ztunnel binary using `make build` with specified features. The output binary can then be executed directly.

```shell
make build FEATURES="--features testing"
./out/rust/debug/ztunnel
```

--------------------------------

### Bootstrap ztunnel for Local Istiod Connection

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Fetches a pod-bound Service Account token required for ztunnel authentication with a CA when running locally and connecting to a real Istiod instance. This command assumes a ztunnel pod is already running.

```shell
source ./scripts/local.sh
ztunnel-local-bootstrap
```

--------------------------------

### Run Fake Server in In-Pod Mode

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Execute this command to run the fake server within the specified network namespace. The `INPOD_UDS` environment variable must be set to the path of the Unix domain socket.

```shell
INPOD_UDS=/tmp/ztunnel cargo run --example inpodserver -- pod1
```

--------------------------------

### Create a Pod for Fetching ztunnel Credentials

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Creates a temporary pod on a specific node with the 'ztunnel' service account to obtain necessary CA certificates and the service account token for local ztunnel execution.

```yaml
apiVersion: v1
kind: Pod
metadata:
  name: fake-tunnel-worker1
  namespace: istio-system
spec:
  nodeName: ambient-worker
  terminationGracePeriodSeconds: 1
  serviceAccountName: ztunnel
  containers:
  - name: cat-token
    image: ubuntu:22.04
    command:
    - bash
    - -c
    args:
    - "sleep 10000"
    ports:
    - containerPort: 80
    volumeMounts:
    - mountPath: /var/run/secrets/tokens
      name: istio-token
  volumes:
  - name: istio-token
    projected:
      defaultMode: 420
      sources:
      - serviceAccountToken:
          audience: istio-ca
          expirationSeconds: 43200
          path: istio-token
```

--------------------------------

### Build Ztunnel with FIPS BoringSSL

Source: https://github.com/istio/ztunnel/blob/master/README.md

After configuring the paths in .cargo/config.toml, use this command to build Ztunnel.

```shell
cargo build
```

--------------------------------

### Analyze memory profile with pprof

Source: https://github.com/istio/ztunnel/blob/master/PROFILING.md

Use the `pprof` tool to analyze the collected memory profile data. Ensure `PPROF_BINARY_PATH` is set to the directory containing the copied binaries for accurate symbol resolution.

```sh
PPROF_BINARY_PATH=../../ztunnel-libs-pprof pprof -http=:8080 mem.pb.gz
```

--------------------------------

### Create Kind Cluster with Extra Mounts

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Creates a Kind Kubernetes cluster with specific node configurations, including an extra mount for the worker node to facilitate ztunnel's connection to the cni-socket.

```shell
kind create cluster --config=-
<<EOF
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: ambient
nodes:
- role: control-plane
- role: worker
  extraMounts:
  - hostPath: /tmp/worker1-ztunnel/
    containerPath: /var/run/ztunnel/
- role: worker
containerdConfigPatches:
- |-
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
    endpoint = ["http://${KIND_REGISTRY_NAME}:5000"]
EOF
```

--------------------------------

### Run ztunnel Locally Without Dependencies

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Use this command to run ztunnel entirely locally, bypassing Kubernetes and Istiod dependencies. It configures fake certificates, disables the XDS client, and specifies a local XDS path. Ensure `CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUNNER` is set if running with sudo.

```shell
FAKE_CA="true" \
XDS_ADDRESS="" \
LOCAL_XDS_PATH=./examples/localhost.yaml \
CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUNNER="sudo -E" \
PROXY_MODE=dedicated \
PROXY_WORKLOAD_INFO=default/local/default \
cargo run --features testing
```

--------------------------------

### Configure FIPS BoringSSL for Linux ARM64

Source: https://github.com/istio/ztunnel/blob/master/README.md

Manually edit .cargo/config.toml to specify the path to vendored FIPS BoringSSL libraries for linux/arm64 builds.

```toml
BORING_BSSL_FIPS_PATH = { value = "vendor/boringssl-fips/linux_arm64", force = true, relative = true }
BORING_BSSL_FIPS_INCLUDE_PATH = { value = "vendor/boringssl-fips/include/", force = true, relative = true }
```

--------------------------------

### Build Release with BoringSSL

Source: https://github.com/istio/ztunnel/blob/master/README.md

Use this command to build a release version of Ztunnel with the BoringSSL TLS mode enabled.

```shell
TLS_MODE=boring ./scripts/release.sh
```

--------------------------------

### Configure FIPS BoringSSL for Linux x86_64

Source: https://github.com/istio/ztunnel/blob/master/README.md

Manually edit .cargo/config.toml to specify the path to vendored FIPS BoringSSL libraries for linux/x86_64 builds.

```toml
BORING_BSSL_FIPS_PATH = { value = "vendor/boringssl-fips/linux_x86_64", force = true, relative = true }
BORING_BSSL_FIPS_INCLUDE_PATH = { value = "vendor/boringssl-fips/include/", force = true, relative = true }
```

--------------------------------

### Copy ztunnel binaries for symbol resolution

Source: https://github.com/istio/ztunnel/blob/master/PROFILING.md

Copy the ztunnel main binary and standard libraries from the container to a local path. This is necessary for symbol resolution when analyzing profiles with tools like pprof.

```sh
# ztunnel main binary
kubectl cp istio-system/ztunnel-qkvdj:/usr/local/bin/ztunnel ../../ztunnel-libs-pprof/ztunnel
# stdlibs (optional)
kubectl cp istio-system/ztunnel-qkvdj:/usr/lib/$BINARY_COMPILED_ARCH/ ../../ztunnel-libs-pprof/
```

--------------------------------

### Run command as redirected user

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Execute commands as the user configured for traffic redirection. This is useful for sending requests through ztunnel after setting up the redirection rules.

```shell
redirect-run curl localhost:8080
```

--------------------------------

### Envoy Server iperf strace Output

Source: https://github.com/istio/ztunnel/blob/master/benches/README.md

Displays syscall statistics for an Envoy server under iperf load. Highlights recvfrom and writev calls.

```text
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 65.24    1.199264           1    757275         8 recvfrom
 34.73    0.638315          26     23670           writev
```

--------------------------------

### Curl CPU profile from ztunnel

Source: https://github.com/istio/ztunnel/blob/master/PROFILING.md

Fetch the CPU profile data from the ztunnel admin port. This data can then be analyzed using profiling tools.

```sh
curl localhost:15000/debug/pprof/profile > profile.prof
```

--------------------------------

### Check ztunnel Sockets in Pod Network Namespace

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Use this command to inspect the network sockets within the 'pod1' network namespace, specifically looking for processes related to ztunnel.

```shell
ip netns exec pod1 ss -ntlp | grep ztunnel
```

--------------------------------

### Envoy Client iperf strace Output

Source: https://github.com/istio/ztunnel/blob/master/benches/README.md

Shows syscall statistics for an Envoy client under iperf load. Focuses on sendto and readv calls.

```text
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 68.24    1.363149           3    440440         1 sendto
 31.72    0.633584          11     55114        31 readv
```

--------------------------------

### TCP to TCP Request Flow Pseudocode

Source: https://github.com/istio/ztunnel/blob/master/benches/README.md

Illustrates the basic bidirectional byte copying mechanism between two TCP sockets, highlighting the use of dynamically sized buffers.

```pseudocode
copy_bidi():
    loop {
        data = tcp_in.read(up to 256k) # based on dynamic buffer size
        h2.write(data)
    }
h2::write(data):
    Buffer data as a DATA frame, up to a max of `max_send_buffer_size`. We configure this to 256k.
    Asyncronously, the connection driver will pick up this data and call `rustls.write_vectored([256bytes, rest of data])`.
rustls::write(data):
    data=encrypt(data)
    # TLS records are at most 16k
    # In practice I have observed at most 4 chunks; unclear where this is configured.
    tcp_out.write_vectored([chunks of 16k])
```

--------------------------------

### Redirect Traffic to ztunnel in Pod Network Namespace

Source: https://github.com/istio/ztunnel/blob/master/Development.md

This script is used to redirect network traffic to ztunnel within the 'pod1' network namespace. Adjustments to `localhost.yaml` and process startup within the pod's netns might be necessary for traffic to function correctly.

```shell
ip netns exec pod1 ./scripts/ztunnel-redirect-inpod.sh
```

--------------------------------

### TCP to TCP Strace Output

Source: https://github.com/istio/ztunnel/blob/master/benches/README.md

Shows typical `strace` output for TCP-to-TCP traffic, indicating the syscalls and their frequency related to data transfer.

```text
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 55.21    0.841290           5    140711           writev
 44.78    0.682359          17     38481           recvfrom
```

--------------------------------

### HBONE to TCP Strace Output

Source: https://github.com/istio/ztunnel/blob/master/benches/README.md

Presents `strace` output for HBONE-to-TCP traffic, detailing `sendto` and `recvfrom` calls with their respective sizes and frequencies.

```text
% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 61.08    1.253541          50     24703           sendto
 38.19    0.783733           2    360707         8 recvfrom
```

--------------------------------

### HBONE to TCP Request Flow Pseudocode

Source: https://github.com/istio/ztunnel/blob/master/benches/README.md

Details the HBONE to TCP request flow, involving HTTP2 decoding, TLS encryption, and underlying TCP reads. It references specific buffer sizes and configurations.

```pseudocode
The receive flow is driven by `h2`. Under the hood this uses a [`LengthDelimitedCodec`](https://docs.rs/tokio-util/latest/tokio_util/codec/length_delimited/struct.LengthDelimitedCodec.html).
`h2` will attempt to decode 1 frame at a time, using an internal buffer.
This buffer starts at [`8kb`](https://github.com/tokio-rs/tokio/blob/ed4ddf443d93c3e14ae23699a5a2f81902ad1e66/tokio-util/src/codec/framed_impl.rs#L26) but will grow to meet the size of frames.
We allow up to a max of `1mb` frame sizes (`config.frame_size`).

Ultimately, this will call `rustls.read(buf)`.
This goes through a few indirections, but ultimately ends up in `rustls.deframer_buffer`.
This is what calls `read()` on the underlying IO, in our case the TCP connection.
This buffer is configured to do [`4kb`](https://github.com/rustls/rustls/blob/8a8023addb9ae311f66b16e272e85654c9588eeb/rustls/src/msgs/deframer.rs#L724) reads generally.

Upon reading the frame from the wire, these get [buffered up by `h2`](https://github.com/hyperium/h2/blob/4617f49b266d560a773372a90be283ba8b2400a9/src/proto/streams/stream.rs#L100).
We read these in [`recv_stream.poll_data`](../src/proxy/h2.rs), trigger by the `copy_bidirectional`.
Ultimately, this will write out 1 DATA frame worth of data to the upstream TCP connection
```

--------------------------------

### Configure ztunnel Node Label and DaemonSet Patch

Source: https://github.com/istio/ztunnel/blob/master/Development.md

Labels a specific Kubernetes node to exclude it from ztunnel scheduling and patches the ztunnel DaemonSet to respect this label, effectively removing ztunnel from the designated node.

```shell
kubectl label node ambient-worker ztunnel=no
kubectl patch daemonset -n istio-system ztunnel --type=merge -p='{"spec":{"template":{"spec":{"affinity":{"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"ztunnel","operator":"NotIn","values":["no"]}]}}]}}}}}}}'
```

--------------------------------

### Curl memory profile from ztunnel

Source: https://github.com/istio/ztunnel/blob/master/PROFILING.md

Fetch the memory profile data (heap) from the ztunnel admin port. This data is useful for diagnosing memory leaks or high memory consumption.

```sh
curl localhost:15000/debug/pprof/heap > mem.pb.gz
```

--------------------------------

### Port-forward ztunnel admin port

Source: https://github.com/istio/ztunnel/blob/master/PROFILING.md

Use this command to forward the ztunnel admin port (15000) from the cluster to your local machine. This is a prerequisite for accessing profiling data.

```sh
kubectl port-forward -n istio-system ztunnel-qkvdj 15000:15000
```

=== COMPLETE CONTENT === This response contains all available snippets from this library. No additional content exists. Do not make further requests.