### Install the tail Package Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/logfile/tail/README.md Install the tail package using the go get command. This command fetches and installs the package and its dependencies. ```Go go get github.com/hpcloud/tail/... ``` -------------------------------- ### Install CloudWatch Agent from RPM Package Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/README.md Installs a previously built CloudWatch Agent RPM package on a Linux system. ```bash rpm -Uvh amazon-cloudwatch-agent.rpm ``` -------------------------------- ### Install CloudWatch Agent from Windows Package Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/README.md Installs the CloudWatch Agent from a ZIP package on a Windows system. Requires unzipping the package and running the install script. ```powershell unzip amazon-cloudwatch-agent.zip ./install.ps1 ``` -------------------------------- ### Install rpm-build Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/README.md Installs the rpm-build package required for building RPMs on Linux systems. ```bash sudo yum install -y rpmdevtools rpm-build ``` -------------------------------- ### Example Transformation: mem.cached.localhost Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/statsd/README.md Demonstrates the transformation of a 'mem.cached.localhost' Statsd bucket into InfluxDB line protocol, showcasing the application of a filtered template. ```plaintext mem.cached.localhost:256|g => mem_cached,host=localhost 256 ``` -------------------------------- ### Example NVIDIA SMI Output Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/nvidia_smi/README.md This is an example of the output generated by the nvidia_smi input plugin. It shows metrics for multiple GPUs. ```text nvidia_smi,compute_mode=Default,host=8218cf,index=0,name=GeForce\ GTX\ 1070,pstate=P2,uuid=GPU-823bc202-6279-6f2c-d729-868a30f14d96 fan_speed=100i,memory_free=7563i,memory_total=8112i,memory_used=549i,temperature_gpu=53i,utilization_gpu=100i,utilization_memory=90i 1523991122000000000 nvidia_smi,compute_mode=Default,host=8218cf,index=1,name=GeForce\ GTX\ 1080,pstate=P2,uuid=GPU-f9ba66fc-a7f5-94c5-da19-019ef2f9c665 fan_speed=100i,memory_free=7557i,memory_total=8114i,memory_used=557i,temperature_gpu=50i,utilization_gpu=100i,utilization_memory=85i 1523991122000000000 nvidia_smi,compute_mode=Default,host=8218cf,index=2,name=GeForce\ GTX\ 1080,pstate=P2,uuid=GPU-d4cfc28d-0481-8d07-b81a-ddfc63d74adf fan_speed=100i,memory_free=7557i,memory_total=8114i,memory_used=557i,temperature_gpu=58i,utilization_gpu=100i,utilization_memory=86i 1523991122000000000 ``` -------------------------------- ### CloudWatch Agent Log Filtering Example Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/README.md Illustrates log entries and how they are processed by the agent based on the defined filters. Entries containing 'Firefox' are excluded, 'POST' requests are included, and 'GET' requests are excluded as they do not match the include regex. ```text 2021-09-27T19:36:35Z I! [logagent] Firefox Detected // Agent excludes this 2021-09-27T19:36:35Z POST (StatusCode: 200). // Agent would push this to CloudWatch 2021-09-27T19:36:35Z GET (StatusCode: 400). // doesn't match regex, will be excluded ``` -------------------------------- ### Install CloudWatch Agent from DEB Package Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/README.md Installs a previously built CloudWatch Agent DEB package on a Debian-based Linux system. ```bash dpkg -i -E ./amazon-cloudwatch-agent.deb ``` -------------------------------- ### ECS Service Discovery Configuration Example Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/internal/ecsservicediscovery/README.md Sample configuration in TOML format for enabling ECS service discovery. Specifies cluster region, frequency, result file, target cluster, and Docker label mappings for Prometheus metrics. ```toml [inputs.prometheus.ecs_service_discovery] sd_cluster_region = "us-east-2" sd_frequency = "15s" sd_result_file = "/opt/aws/amazon-cloudwatch-agent/etc/ecs_sd_targets.yaml" sd_target_cluster = "EC2-Justin-Testing" [inputs.prometheus.ecs_service_discovery.docker_label] sd_job_name_label = "ECS_PROMETHEUS_JOB_NAME" sd_metrics_path_label = "ECS_PROMETHEUS_METRICS_PATH" sd_port_label = "ECS_PROMETHEUS_EXPORTER_PORT_SUBSET_A" [[inputs.prometheus.ecs_service_discovery.task_definition_list]] sd_job_name = "task_def_1" sd_metrics_path = "/stats/metrics" sd_metrics_ports = "9901;9404;9406" sd_task_definition_arn_pattern = ".*:task-definition/bugbash-java-fargate-awsvpc-task-def-only:[0-9]+" [[inputs.prometheus.ecs_service_discovery.task_definition_list]] sd_container_name_pattern = "^bugbash-jar.*$" sd_metrics_ports = "9902" sd_task_definition_arn_pattern = ".*:task-definition/nginx:[0-9]+" ``` -------------------------------- ### Example Transformation: cpu.load.us-west Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/statsd/README.md Illustrates the transformation of a specific Statsd bucket ('cpu.load.us-west') into InfluxDB line protocol using a defined template. This shows the resulting measurement and tag. ```plaintext cpu.load.us-west:100|g => cpu_load,region=us-west 100 ``` -------------------------------- ### Define Performance Object and Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Configure a specific performance object, its instances, and the counters to collect. This example collects '% Idle Time', '% Disk Read Time', and '% Disk Write Time' for the 'C:' and 'D:' instances of the 'LogicalDisk' object. ```toml [[inputs.win_perf_counters.object]] ObjectName = "LogicalDisk" Instances = ["C:","D:"] Counters = [ "% Idle Time", "% Disk Read Time", "% Disk Write Time", ] ``` -------------------------------- ### Example ECS Service Discovery Result Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/internal/ecsservicediscovery/README.md Example output file generated by ECS service discovery, listing targets and their associated labels for Prometheus. ```yaml - targets: - 10.6.1.95:32785 labels: __metrics_path__: /metrics ECS_PROMETHEUS_EXPORTER_PORT_SUBSET_B: "9406" ECS_PROMETHEUS_JOB_NAME: demo-jar-ec2-bridge-subset-b-dynamic ECS_PROMETHEUS_METRICS_PATH: /metrics InstanceType: t3.medium LaunchType: EC2 SubnetId: subnet-0347624eeea6c5969 TaskDefinitionFamily: demo-jar-ec2-bridge-dynamic-port-subset-b TaskGroup: family:demo-jar-ec2-bridge-dynamic-port-subset-b TaskRevision: "7" VpcId: vpc-033b021cd7ecbcedb container_name: demo-jar-ec2-bridge-dynamic-port-subset-b job: task_def_2 - targets: - 10.6.1.95:32783 labels: __metrics_path__: /metrics ECS_PROMETHEUS_EXPORTER_PORT_SUBSET_B: "9406" ECS_PROMETHEUS_JOB_NAME: demo-jar-ec2-bridge-subset-b-dynamic ECS_PROMETHEUS_METRICS_PATH: /metrics InstanceType: t3.medium LaunchType: EC2 SubnetId: subnet-0347624eeea6c5969 TaskDefinitionFamily: demo-jar-ec2-bridge-dynamic-port-subset-b TaskGroup: family:demo-jar-ec2-bridge-dynamic-port-subset-b TaskRevision: "7" VpcId: vpc-033b021cd7ecbcedb container_name: demo-jar-ec2-bridge-dynamic-port-subset-b job: task_def_2 ``` -------------------------------- ### Statsd Mixed Type Metric Example Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/statsd/README.md Shows how to combine different metric types (counter and timing) in a single line. ```plaintext foo:1|c:200|ms ``` -------------------------------- ### Install CloudWatch Agent from Darwin Package Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/README.md Installs the CloudWatch Agent from a tar.gz package on macOS. This involves extracting the archive, copying files to system directories, and setting up launch daemons. ```bash tar -xvf amazon-cloudwatch-agent.tar.gz cp -rf ./opt/aws /opt cp -rf ./Library/LaunchDaemons/com.amazon.cloudwatch.agent.plist /Library/LaunchDaemons/ ``` -------------------------------- ### Statsd Metric Examples Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/statsd/README.md Examples of different statsd metric types including gauges, counters, sets, and timings/histograms. ```plaintext users.current.den001.myapp:32|g users.current.den001.myapp:+10|g users.current.den001.myapp:-10|g ``` ```plaintext deploys.test.myservice:1|c deploys.test.myservice:101|c deploys.test.myservice:1|c|@0.1 ``` ```plaintext users.unique:101|s users.unique:101|s users.unique:102|s ``` ```plaintext load.time:320|ms load.time.nanoseconds:1|h load.time:200|ms|@0.1 ``` -------------------------------- ### Troubleshooting: Windows nvidia-smi Output Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/nvidia_smi/README.md Run this command on Windows to get the full output of the nvidia-smi binary for troubleshooting. Ensure the path to nvidia-smi.exe is correctly specified. ```sh "C:\Program Files\NVIDIA Corporation\NVSMI\nvidia-smi.exe" -q -x ``` -------------------------------- ### AWS AppSignals Processor Configuration Example Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/processors/awsapplicationsignals/README.md Configure rules for the AWS AppSignals processor to manage telemetry data. Supports 'keep', 'drop', and 'replace' actions with specific selectors and replacements. ```yaml awsapplicationsignals: resolvers: ["eks"] rules: - selectors: - dimension: Operation match: "POST *" - dimension: RemoteService match: "*" action: keep rule_name: "keep01" - selectors: - dimension: Operation match: "GET *" - dimension: RemoteService match: "*" action: keep rule_name: "keep02" - selectors: - dimension: Operation match: "POST *" action: drop rule_name: "drop01" - selectors: - dimension: Operation match: "*" replacements: - target_dimension: RemoteOperation value: "This is a test string" action: replace rule_name: "replace01" ``` -------------------------------- ### Sample Java Metadata Result Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/internal/detector/java/README.md This is an example of the metadata structure returned by the Java detector for a classified process. ```json { "categories": ["JVM"], "name": "com.example.DemoApplication", "telemetry_port": 1234, "status": "READY" } ``` -------------------------------- ### Statsd Merged Metric Examples Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/statsd/README.md Demonstrates merging multiple stats into a single line by separating them with additional colons. ```plaintext users.current.den001.myapp:32|g:+10|g:-10|g ``` ```plaintext deploys.test.myservice:1|c:101|c:1|c|@0.1 ``` ```plaintext users.unique:101|s:101|s:102|s ``` ```plaintext load.time:320|ms:200|ms|@0.1 ``` -------------------------------- ### Troubleshooting: Linux nvidia-smi Output Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/nvidia_smi/README.md Run this command on Linux to get the full output of the nvidia-smi binary for troubleshooting. It requires sudo privileges and runs as the telegraf user. ```sh sudo -u telegraf -- /usr/bin/nvidia-smi -q -x ``` -------------------------------- ### Sample Tomcat Metadata Result Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/internal/detector/tomcat/README.md This JSON structure represents the metadata collected for a detected Tomcat instance. It includes categories and the detected installation or runtime directory. ```json { "categories": ["TOMCAT"], "name": "/opt/tomcat/instance1" } ``` -------------------------------- ### CloudWatch Agent Configuration for Application Signals Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/processors/awsapplicationsignals/README.md This JSON configuration enables application signals collection for traces and logs. It specifies rules for keeping specific service and remote service combinations and dropping operations that match 'GET *'. ```json { "agent": { "region": "us-west-2", "debug": true }, "traces": { "traces_collected": { "app_signals": {} } }, "logs": { "metrics_collected": { "app_signals": { "rules": [ { "selectors": [ { "dimension": "Service", "match": "pet-clinic-frontend" }, { "dimension": "RemoteService", "match": "customers-service" } ], "action": "keep", "rule_name": "keep01" }, { "selectors": [ { "dimension": "Operation", "match": "GET *" } ], "action": "drop", "rule_name": "drop01" } } } } } ``` -------------------------------- ### Configure Log Filtering in CloudWatch Agent Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/README.md Specify filters to process log messages, publishing only events that pass all filters to CloudWatch Logs. This example excludes logs containing 'Firefox' and includes logs with 'PUT' or 'POST' requests. ```json { "collect_list": [ { "file_path": "/opt/aws/amazon-cloudwatch-agent/logs/test.log", "log_group_name": "test.log", "log_stream_name": "test.log", "filters": [ { "type": "exclude", "expression": "Firefox" }, { "type": "include", "expression": "P(UT|OST)" } ] } ] } ``` -------------------------------- ### Nvidia SMI Plugin Configuration Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/nvidia_smi/README.md Configure the nvidia-smi plugin to specify the binary path, startup error behavior, and polling timeout. The bin_path defaults to \"/usr/bin/nvidia-smi\". startup_error_behavior can be \"error\" or \"ignore\". ```toml # Pulls statistics from nvidia GPUs attached to the host [[inputs.nvidia_smi]] ## Optional: path to nvidia-smi binary, defaults "/usr/bin/nvidia-smi" ## We will first try to locate the nvidia-smi binary with the explicitly specified value (or default value), ## if it is not found, we will try to locate it on PATH(exec.LookPath), if it is still not found, an error will be returned # bin_path = "/usr/bin/nvidia-smi" ## Optional: specifies plugin behavior regarding missing nvidia-smi binary ## Available choices: ## - error: telegraf will return an error on startup ## - ignore: telegraf will ignore this plugin # startup_error_behavior = "error" ## Optional: timeout for GPU polling # timeout = "5s" ``` -------------------------------- ### Build Multi-Arch Image with Docker Buildx Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/amazon-cloudwatch-container-insights/cloudwatch-agent-dockerfile/README.md Use `docker buildx` to create multi-architecture Docker images. Ensure you have a builder instance created and selected for multi-arch support. This command builds the image for specified platforms and can push it to a registry. ```bash docker buildx create --name multi-builder docker buildx use multi-builder # Add proper tag and --push if you want to publish it docker buildx build --platform linux/amd64,linux/arm64 . # To build multi arch image from source code, run the following at project root docker buildx build --platform linux/amd64,linux/arm64 -f amazon-cloudwatch-container-insights/cloudwatch-agent-dockerfile/Dockerfile . ``` -------------------------------- ### Push Multi-Arch Image with Manifest-Tool Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/amazon-cloudwatch-container-insights/cloudwatch-agent-dockerfile/README.md Use `manifest-tool` to push a multi-architecture image manifest. This tool does not require a Docker daemon and is useful for managing manifests, especially when integrating with ECR. Ensure you have built `manifest-tool` from source if using an older version. ```bash # NOTE: the released version of manifest-tool is a bit outdated, you need to build it from source manifest-tool push from-spec multi-arch-agent.yaml ``` -------------------------------- ### Build and Package CloudWatch Agent Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/README.md Builds the CloudWatch Agent and packages it into RPM, DEB, and ZIP formats for release. ```bash make release ``` -------------------------------- ### Enable Pre-Vista Performance Counter Support Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Set `PreVistaSupport` to `true` to use the localized performance counter interface available in versions of Windows prior to Vista. This is recommended for older systems like Windows Server 2003. ```toml [[inputs.win_perf_counters]] PreVistaSupport = true ``` -------------------------------- ### InfluxDB Tagged Statsd Metric Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/statsd/README.md Example of specifying tags in a statsd metric similar to InfluxDB's line protocol. ```plaintext users.current,service=payroll,region=us-west:32|g ``` -------------------------------- ### Configure .NET CLR Loading Monitoring Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects .NET CLR class loading metrics for the 'w3wp' process. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = ".NET CLR Loading" Counters = ["% Time Loading"] Instances = ["w3wp"] Measurement = "win_dotnet_loading" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Configure .NET CLR Jit Monitoring Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects .NET CLR Jit compilation metrics for the 'w3wp' process. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = ".NET CLR Jit" Counters = ["% Time in Jit","IL Bytes Jitted / sec"] Instances = ["w3wp"] Measurement = "win_dotnet_jit" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Build CloudWatch Agent Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/README.md Builds the CloudWatch Agent for Linux, Debian, and Windows environments. Use 'make release' to also package it into RPM, DEB, and ZIP formats. ```bash make build ``` -------------------------------- ### Include '_Total' Instances Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Set `IncludeTotal` to `true` when `Instances` is set to `"*"` to include performance data for `_Total` instances. This is useful for aggregated metrics, such as total processor time. ```toml [[inputs.win_perf_counters.object]] ObjectName = "Processor Information" Instances = ["*"] Counters = [ "% Processor Time", ] IncludeTotal = true ``` -------------------------------- ### Tail a Log File in Go Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/logfile/tail/README.md Use this snippet to tail a log file and process its lines in real-time. Ensure the file exists and the program has read permissions. ```Go t, err := tail.TailFile("/var/log/nginx.log", tail.Config{Follow: true}) for line := range t.Lines { fmt.Println(line.Text) } ``` -------------------------------- ### Specify Measurement Name Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Assign a custom measurement name for the collected performance data. This helps in organizing metrics, for example, separating disk performance from processor performance. ```toml [[inputs.win_perf_counters.object]] ObjectName = "LogicalDisk" Instances = ["C:"] Counters = [ "% Idle Time", ] Measurement = "win_disk" ``` -------------------------------- ### Create Multi-Arch Image Manifest with Docker Manifest Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/amazon-cloudwatch-container-insights/cloudwatch-agent-dockerfile/README.md This command creates a manifest list for a multi-architecture image by referencing existing single-architecture images. Ensure experimental features are enabled in your Docker configuration. This is useful if you build images on different machines. ```bash # NOTE: manifest is a experimental command, docker versions released after mid 2018 should have it # enable experimental in your ~/.docker/config.json with: # { # "experimental": "enabled" # } docker manifest create cloudwatch-agent:foo --amend cloudwatch-agent:foo-arm64 --amend cloudwatch-agent:foo-amd64 docker manifest push cloudwatch-agent:foo ``` -------------------------------- ### Configure System Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects system-wide performance metrics such as context switches and processor queue length. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = "System" Counters = ["Context Switches/sec","System Calls/sec", "Processor Queue Length"] Instances = ["------"] Measurement = "win_system" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Configure Logical Disk Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects disk performance metrics including idle time, busy time, and queue length. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = "LogicalDisk" Instances = ["*"] Counters = ["% Idle Time", "% Disk Time","% Disk Read Time", "% Disk Write Time", "% User Time", "Current Disk Queue Length"] Measurement = "win_disk" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Basic tail command simulation Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/logfile/README.md Simulates the behavior of the unix tail -F --lines=0 command for following log files. ```bash tail -F --lines=0 myfile.log ``` -------------------------------- ### Configure Processor Usage Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects processor usage statistics per core. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = "Processor" Instances = ["*"] Counters = ["% Idle Time", "% Interrupt Time", "% Privileged Time", "% User Time", "% Processor Time"] Measurement = "win_cpu" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Configure DNS Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collect DNS performance metrics by specifying the 'DNS' object and relevant counters. Ensure the 'Instances' are set correctly, typically to '------' for aggregate data. ```toml [[inputs.win_perf_counters.object]] ObjectName = "DNS" Counters = ["Dynamic Update Received","Dynamic Update Rejected","Recursive Queries","Recursive Queries Failure","Secure Update Failure","Secure Update Received","TCP Query Received","TCP Response Sent","UDP Query Received","UDP Response Sent","Total Query Received","Total Response Sent"] Instances = ["------"] Measurement = "win_dns" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Collect All Instances of a Counter Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Use `Instances = ["*"]` to collect data for all available instances of a specified counter. This is a common way to gather metrics for all disks or all processors. ```toml [[inputs.win_perf_counters.object]] ObjectName = "LogicalDisk" Instances = ["*"] Counters = ["% Free Space"] Measurement = "win_disk" ``` -------------------------------- ### Configure DFS Namespace Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects performance metrics for DFS Namespace, useful for servers hosting DFS Namespaces or acting as Domain Controllers. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = "DFS Namespace Service Referrals" Instances = ["*"] Counters = ["Requests Processed","Requests Failed","Avg. Response Time"] Measurement = "win_dfsn" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). #WarnOnMissing = false # Print out when the performance counter is missing, either of object, counter or instance. ``` -------------------------------- ### Sample NVIDIA GPU Metadata Result Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/internal/detector/nvidia/README.md This JSON structure represents a successful detection of NVIDIA GPUs with drivers loaded. ```json { "categories": ["NVIDIA_GPU"], "status": "READY" } ``` -------------------------------- ### Configure .NET CLR LocksAndThreads Monitoring Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects .NET CLR thread and lock metrics for the 'w3wp' process. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = ".NET CLR LocksAndThreads" Counters = ["# of current logical Threads","# of current physical Threads","# of current recognized threads","# of total recognized threads","Queue Length / sec","Total # of Contentions","Current Queue Length"] Instances = ["w3wp"] Measurement = "win_dotnet_locks" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Configure Process Performance Counters for IIS Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collect specific process performance metrics, such as processor time, handle count, and memory usage, for the 'w3wp' instance, which typically represents IIS worker processes. ```toml [[inputs.win_perf_counters.object]] # Process metrics, in this case for IIS only ObjectName = "Process" Counters = ["% Processor Time","Handle Count","Private Bytes","Thread Count","Virtual Bytes","Working Set"] Instances = ["w3wp"] Measurement = "win_proc" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Configure DFS Replication Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects performance metrics for DFS Replication, useful for servers hosting DFS Replication folders or acting as Domain Controllers. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = "DFS Replication Service Volumes" Instances = ["*"] Counters = ["Data Lookups","Database Commits"] Measurement = "win_dfsr" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). #WarnOnMissing = false # Print out when the performance counter is missing, either of object, counter or instance. ``` -------------------------------- ### Enable Optional Metric Configuration Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/receiver/awsnvmereceiver/documentation.md Use this configuration to enable an optional metric. Replace `` with the specific metric to enable. ```yaml metrics: : enabled: true ``` -------------------------------- ### Manifest-Tool Configuration for Multi-Arch Image Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/amazon-cloudwatch-container-insights/cloudwatch-agent-dockerfile/README.md This YAML file defines the structure for a multi-architecture image manifest to be used with `manifest-tool`. It specifies the main image tag and lists the individual platform-specific images with their respective architectures and operating systems. ```yaml # multi-arch-agent.yaml image: {{account_id}}.dkr.ecr.{{aws_region}}.amazonaws.com/cloudwatch-agent:foo manifests: - image: {{account_id}}.dkr.ecr.{{aws_region}}.amazonaws.com/cloudwatch-agent:foo-amd64 platform: architecture: amd64 os: linux - image: {{account_id}}.dkr.ecr.{{aws_region}}.amazonaws.com/cloudwatch-agent:foo-arm64 platform: architecture: arm64 os: linux ``` -------------------------------- ### Collect Data for Objects Without Instances Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md For performance objects that do not have distinct instances, specify `Instances = ["------"]` to retrieve data. This is a required setting when no other instance selection is applicable. ```toml [[inputs.win_perf_counters.object]] ObjectName = "System" Instances = ["------"] Counters = ["System Up Time"] Measurement = "win_system" ``` -------------------------------- ### Configure .NET CLR Security Monitoring Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects .NET CLR security-related metrics for the 'w3wp' process. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = ".NET CLR Security" Counters = ["% Time in RT checks","Stack Walk Depth","Total Runtime Checks"] Instances = ["w3wp"] Measurement = "win_dotnet_security" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Configure IIS ASP.NET Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Monitor general ASP.NET performance metrics such as application restarts, request wait times, and queued requests. The '*' instance setting applies to all ASP.NET instances. ```toml [[inputs.win_perf_counters.object]] # IIS, ASP.NET ObjectName = "ASP.NET" Counters = ["Application Restarts","Request Wait Time","Requests Current","Requests Queued","Requests Rejected"] Instances = ["*"] Measurement = "win_aspnet" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Sample SQL Query for GPU Temperature Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/nvidia_smi/README.md This SQL query can be used to alert on the average temperature of GPUs over the last minute. It groups the data by time, index, name, and host. ```sql SELECT mean("temperature_gpu") FROM "nvidia_smi" WHERE time > now() - 5m GROUP BY time(1m), "index", "name", "host" ``` -------------------------------- ### Configure Kubernetes Metadata Extension Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/extension/k8smetadata/README.md Enable the k8smetadata extension by listing 'endpointslices' and 'services' under 'extensions.k8smetadata.objects'. Ensure the extension is also included in the main 'service.extensions' list. ```yaml extensions: k8smetadata: objects: - endpointslices - services service: extensions: [k8smetadata] ``` -------------------------------- ### Configure Network Interface Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects network interface statistics including bytes and packets sent/received per second. Use '*' for Instances to gather all interfaces. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = "Network Interface" Counters = ["Bytes Received/sec","Bytes Sent/sec","Packets Received/sec","Packets Sent/sec"] Instances = ["*"] # Use 6 x - to remove the Instance bit from the query. Measurement = "win_net" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Configure IIS ASP.NET Applications Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collect performance metrics for ASP.NET Applications, including cache statistics, compilations, errors, and request queues. Use '*' for instances to monitor all applications. ```toml [[inputs.win_perf_counters.object]] # IIS, ASP.NET Applications ObjectName = "ASP.NET Applications" Counters = ["Cache Total Entries","Cache Total Hit Ratio","Cache Total Turnover Rate","Output Cache Entries","Output Cache Hits","Output Cache Hit Ratio","Output Cache Turnover Rate","Compilations Total","Errors Total/Sec","Pipeline Instance Count","Requests Executing","Requests in Application Queue","Requests/Sec"] Instances = ["*"] Measurement = "win_aspnet_app" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Configure .NET CLR Memory Monitoring Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects .NET CLR memory management metrics for the 'w3wp' process. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = ".NET CLR Memory" Counters = ["% Time in GC","# Bytes in all Heaps","# Gen 0 Collections","# Gen 1 Collections","# Gen 2 Collections","# Induced GC","Allocated Bytes/sec","Finalization Survivors","Gen 0 heap size","Gen 1 heap size","Gen 2 heap size","Large Object Heap size","# of Pinned Objects"] Instances = ["w3wp"] Measurement = "win_dotnet_mem" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Configure .NET CLR Exceptions Monitoring Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects .NET CLR Exception metrics for the 'w3wp' process. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = ".NET CLR Exceptions" Counters = ["# of Exceps Thrown / sec"] Instances = ["w3wp"] Measurement = "win_dotnet_exceptions" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Docker Compose Configuration for NVIDIA SMI Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/nvidia_smi/README.md This Docker Compose configuration demonstrates how to set up a Telegraf container to use the NVIDIA SMI input plugin. It includes necessary device mappings, volume mounts for binaries and libraries, and environment variables for NVIDIA. ```yaml telegraf: image: telegraf runtime: nvidia devices: - /dev/nvidiactl:/dev/nvidiactl - /dev/nvidia0:/dev/nvidia0 volumes: - ./telegraf/etc/telegraf.conf:/etc/telegraf/telegraf.conf:ro - /usr/bin/nvidia-smi:/usr/bin/nvidia-smi:ro - /usr/lib/x86_64-linux-gnu/nvidia:/usr/lib/x86_64-linux-gnu/nvidia:ro environment: - LD_PRELOAD=/usr/lib/x86_64-linux-gnu/nvidia/current/libnvidia-ml.so ``` -------------------------------- ### Configure Memory Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects memory usage statistics like available bytes and page faults. Use 6 hyphens for Instances to remove the Instance bit from the query. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = "Memory" Counters = ["Available Bytes","Cache Faults/sec","Demand Zero Faults/sec","Page Faults/sec","Pages/sec","Transition Faults/sec","Pool Nonpaged Bytes","Pool Paged Bytes"] Instances = ["------"] # Use 6 x - to remove the Instance bit from the query. Measurement = "win_mem" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Configure IIS Web Service Cache Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Monitor the Web Service Cache performance, focusing on cache hit percentages for URIs and files. The '*' instance setting applies to all relevant cache instances. ```toml [[inputs.win_perf_counters.object]] # Web Service Cache / IIS ObjectName = "Web Service Cache" Counters = ["URI Cache Hits %","Kernel: URI Cache Hits %","File Cache Hits %"] Instances = ["*"] Measurement = "win_websvc_cache" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Statsd Input Plugin Configuration Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/statsd/README.md Configure the statsd plugin to listen on a specific address and port, and control cache clearing behavior. ```toml # Statsd Server [[inputs.statsd]] ## Address and port to host UDP listener on service_address = ":8125" ## The following configuration options control when telegraf clears it's cache ## of previous values. If set to false, then telegraf will only clear it's ## cache when the daemon is restarted. ## Reset gauges every interval (default=true) delete_gauges = true ## Reset counters every interval (default=true) delete_counters = true ## Reset sets every interval (default=true) delete_sets = true ## Reset timings & histograms every interval (default=true) delete_timings = true ## separator to use between elements of a statsd metric metric_separator = "_" ## Parses tags in the datadog statsd format ## http://docs.datadoghq.com/guides/dogstatsd/ parse_data_dog_tags = false ## Statsd data translation templates, more info can be read here: ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#graphite # templates = [ # "cpu.* measurement*" # ] ## Number of UDP messages allowed to queue up, once filled, ## the statsd server will start dropping packets allowed_pending_messages = 10000 ``` -------------------------------- ### Configure IIS Web Service Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collect metrics for the IIS Web Service, including request rates, connection attempts, and current connections. Use '*' for instances to monitor all web services. ```toml [[inputs.win_perf_counters.object]] # IIS, Web Service ObjectName = "Web Service" Counters = ["Get Requests/sec","Post Requests/sec","Connection Attempts/sec","Current Connections","ISAPI Extension Requests/sec"] Instances = ["*"] Measurement = "win_websvc" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Configure IIS HTTP Service Request Queues Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Monitor the HTTP Service request queues in the Kernel. This configuration collects metrics like 'CurrentQueueSize' and 'RejectedRequests' for all instances. ```toml [[inputs.win_perf_counters.object]] # HTTP Service request queues in the Kernel before being handed over to User Mode. ObjectName = "HTTP Service Request Queues" Instances = ["*"] Counters = ["CurrentQueueSize","RejectedRequests"] Measurement = "win_http_queues" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Statsd Bucket to InfluxDB Measurement Transformation Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/statsd/README.md Demonstrates how Statsd buckets are transformed into InfluxDB measurements and tags using a simple template. This is useful for organizing and querying time-series data. ```go templates = [ "measurement.measurement.region" ] ``` -------------------------------- ### Filtered Statsd Bucket to InfluxDB Measurement Transformation Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/statsd/README.md Shows how to use glob matching in templates to filter Statsd buckets and apply different transformation rules based on the bucket name. This allows for more granular control over data mapping. ```go templates = [ "cpu.* measurement.measurement.region", "mem.* measurement.measurement.host" ] ``` -------------------------------- ### Configure Active Directory Domain Controller Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Collects performance metrics specific to Active Directory Domain Controllers, including LDAP and authentication statistics. Set IncludeTotal to true to include the _Total instance. ```toml [[inputs.win_perf_counters.object]] ObjectName = "DirectoryServices" Instances = ["*"] Counters = ["Base Searches/sec","Database adds/sec","Database deletes/sec","Database modifys/sec","Database recycles/sec","LDAP Client Sessions","LDAP Searches/sec","LDAP Writes/sec"] Measurement = "win_ad" # Set an alternative measurement to win_perf_counters if wanted. #Instances = [""] # Gathers all instances by default, specify to only gather these #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` ```toml [[inputs.win_perf_counters.object]] ObjectName = "Security System-Wide Statistics" Instances = ["*"] Counters = ["NTLM Authentications","Kerberos Authentications","Digest Authentications"] Measurement = "win_ad" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` ```toml [[inputs.win_perf_counters.object]] ObjectName = "Database" Instances = ["*"] Counters = ["Database Cache % Hit","Database Cache Page Fault Stalls/sec","Database Cache Page Faults/sec","Database Cache Size"] Measurement = "win_db" #IncludeTotal=false #Set to true to include _Total instance when querying for all (*). ``` -------------------------------- ### Logs Input Plugin Configuration Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/logfile/README.md Configuration for the logs input plugin to stream log files. It supports glob matching for file paths and specifies destination, state folder, and individual file configurations. ```toml [[inputs.logs]] ## files to tail. ## These accept standard unix glob matching rules, but with the addition of ## ** as a "super asterisk". ie: ## "/var/log/**.log" -> recursively find all .log files in /var/log ## "/var/log/*/*.log" -> find all .log files with a parent dir in /var/log ## "/var/log/apache.log" -> just tail the apache log file ## ## See https://github.com/gobwas/glob for more examples ## ## Default log output destination name for all file_configs ## each file_config can override its own destination if needed destination = "cloudwatchlogs" ## folder path where state of how much of a file has been transferred is stored file_state_folder = "/tmp/logfile/state" [[inputs.logs.file_config]] file_path = "/tmp/logfile.log*" log_group_name = "logfile.log" log_stream_name = "" timestamp_regex = "^(\d{2} \w{3} \d{4} \d{2}:\d{2}:\d{2}).*$" timestamp_layout = ["_2 Jan 2006 15:04:05"] timezone = "UTC" trim_timestamp = false multi_line_start_pattern = "{timestamp_regex}" ## Read file from beginning. from_beginning = false ## Whether file is a named pipe pipe = false retention_in_days = -1 destination = "cloudwatchlogs" [[inputs.logs.file_config]] file_path = "/var/log/*.log" ## Regular expression for log files to ignore blacklist = "journal|syslog" ## Publish all log files that match file_path publish_multi_logs = true log_group_name = "varlog" log_stream_name = "" timestamp_regex = "^(\d{2} \w{3} \d{4} \d{2}:\d{2}:\d{2}).*$" timestamp_layout = ["_2 Jan 2006 15:04:05"] timezone = "UTC" trim_timestamp = true multi_line_start_pattern = "{timestamp_regex}" ## Read file from beginning. from_beginning = false ## Whether file is a named pipe pipe = false retention_in_days = -1 destination = "cloudwatchlogs" ## Max size of each log event, defaults to 262144 (256KB) max_event_size = 262144 ## Suffix to be added to truncated logline to indicate its truncation, defaults to "[Truncated...]" truncate_suffix = "[Truncated...]" ``` -------------------------------- ### Sample Kafka Client Metadata Result Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/internal/detector/kafkaclient/README.md This JSON structure represents the metadata output when a Kafka client process is detected. It indicates the presence of a Kafka client within the 'KAFKA/CLIENT' category. ```json { "categories": ["KAFKA/CLIENT"] } ``` -------------------------------- ### Enable Printing Valid Performance Objects Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Set `PrintValid` to `true` to display all performance objects that match the configuration during plugin startup. This is useful for debugging and verifying object names. ```toml [[inputs.win_perf_counters]] PrintValid = true ``` -------------------------------- ### Warn on Missing Performance Counters Source: https://github.com/aws/amazon-cloudwatch-agent/blob/main/plugins/inputs/win_perf_counters/README.md Enable `WarnOnMissing` to log warnings during the first plugin run if any specified ObjectName/Instance/Counter combinations are not found. This aids in debugging configuration issues. ```toml [[inputs.win_perf_counters.object]] ObjectName = "DirectoryServices" Instances = ["*"] Counters = ["some counter"] WarnOnMissing = true ```