### Get Resource File Details Response Example Source: https://github.com/ccfos/nightingale/blob/main/doc/api/ai-skill.md Example response for retrieving resource file details. Includes file metadata and content. ```json { "dat": { "id": 10, "skill_id": 1, "name": "references/common/llm.md", "content": "# LLM Reference\n文件完整内容...", "size": 1024, "created_at": 1710000000, "created_by": "admin", "updated_at": 1710000000, "updated_by": "admin" }, "err": "" } ``` -------------------------------- ### Import Skill Response Example Source: https://github.com/ccfos/nightingale/blob/main/doc/api/ai-skill.md Example of a successful response when importing or updating a Skill. Returns the updated Skill ID. ```json { "dat": 1, "err": "" } ``` -------------------------------- ### YAML Configuration File Example Source: https://github.com/ccfos/nightingale/blob/main/doc/design/cli/README.md Example of a YAML configuration file (~/.n9e-cli.yaml) for setting API URL, token, and default output. This file is read by Viper. ```yaml # ~/.n9e-cli.yaml api_url: http://localhost:17000 token: your-api-token default_output: json ``` -------------------------------- ### Example Smart Device Metrics Output Source: https://github.com/ccfos/nightingale/blob/main/integrations/SMART/markdown/README.md This is an example of the output format for smart device health and temperature metrics. ```text smart_device_health_ok agent_hostname=1.2.3.4 device=nvme0 model=INTEL_SSDPE2KX040T8 serial_no=PHLJ830200CH4P0DGN 1 smart_device_temp_c agent_hostname=1.2.3.4 device=nvme0 model=INTEL_SSDPE2KX040T8 serial_no=PHLJ830200CH4P0DGN 53 ``` -------------------------------- ### Common LogQL Examples Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-datasource/datasources/loki.md A collection of common LogQL query patterns for filtering, searching, and analyzing logs. These examples demonstrate basic and advanced LogQL syntax. ```logql {job="myapp"} ``` ```logql {job="myapp"} |= "error" ``` ```logql {job="myapp"} != "debug" ``` ```logql {job="myapp"} |~ "timeout|refused" ``` ```logql count_over_time({job="myapp"} |= "error" [5m]) ``` ```logql {job="myapp", level="error"} ``` ```logql {job="myapp"} | json | status >= 500 ``` ```logql rate({job="myapp"} |= "error" [1m]) ``` -------------------------------- ### Full Net Response Configuration Example Source: https://github.com/ccfos/nightingale/blob/main/integrations/Net_Response/markdown/README.md A comprehensive example of the net_response configuration file, including mappings, instance targets, labels, interval settings, protocol selection, and string matching for UDP checks. ```toml [mappings] # "127.0.0.1:22"= {region="local",ssh="test"} # "127.0.0.1:22"= {region="local",ssh="redis"} [[instances]] targets = [ # "127.0.0.1:22", # "localhost:6379", # ":9090" ] # # append some labels for series # labels = { region="cloud", product="n9e" } # # interval = global.interval * interval_times # interval_times = 1 ## Protocol, must be "tcp" or "udp" ## NOTE: because the "udp" protocol does not respond to requests, it requires ## a send/expect string pair (see below). # protocol = "tcp" ## Set timeout # timeout = "1s" ## Set read timeout (only used if expecting a response) # read_timeout = "1s" ## The following options are required for UDP checks. For TCP, they are ## optional. The plugin will send the given string to the server and then ## expect to receive the given 'expect' string back. ## string sent to the server # send = "ssh" ## expected string in answer # expect = "ssh" ``` -------------------------------- ### Install and Manage Categraf on Windows as a Service Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/categraf-deploy-guide/SKILL.md Commands to install, start, stop, and uninstall Categraf as a Windows service. Configuration changes require stopping and starting the service. ```cmd categraf.exe --win-service-install categraf.exe --win-service-start categraf.exe --win-service-stop categraf.exe --win-service-uninstall ``` -------------------------------- ### Test Script Output Example Source: https://github.com/ccfos/nightingale/blob/main/integrations/Exec/markdown/README.md Example output from a test script (`collect_cert_expiretime.sh`) in InfluxDB format, showing metric, labels, and fields. ```text cert,cloud=huaweicloud,region=huabei-beijing-4,azone=az1,product=cert,domain_name=www.baidu.com expire_days=163 cert,cloud=huaweicloud,region=huabei-beijing-4,azone=az1,product=cert,domain_name=www.weibo.com expire_days=85 cert,cloud=huaweicloud,region=huabei-beijing-4,azone=az1,product=cert,domain_name=www.csdn.net expire_days=281 ``` -------------------------------- ### List Metrics Example Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/create-alert-rule/SKILL.md Use `list_metrics` to check for Categraf metric data before creating a rule. This example searches for metrics related to CPU usage. ```text list_metrics(datasource_id=, keyword="cpu_usage") ``` -------------------------------- ### Install and Manage Categraf on Linux with systemd Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/categraf-deploy-guide/SKILL.md Use these commands to register, start, check status, and stop Categraf as a systemd service on Linux. For non-root users, use the `--user` flag. ```bash cd /path/to/categraf sudo ./categraf --install # register as a system service sudo ./categraf --start # start sudo ./categraf --status # check status sudo ./categraf --stop # stop ``` ```bash ./categraf --user --install ./categraf --user --start ``` -------------------------------- ### Install Categraf as a Windows Service Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/categraf-deploy-guide/SKILL.md Use this command to install Categraf as a Windows service for background operation. Ensure you are in the directory containing `categraf.exe`. ```bash categraf.exe --win-service-install ``` -------------------------------- ### Prometheus Output Format Example Source: https://github.com/ccfos/nightingale/blob/main/integrations/Exec/markdown/README.md A shell script demonstrating how to output monitoring data in Prometheus format. Categraf ignores lines starting with '#'. ```shell #!/bin/sh echo '# HELP demo_http_requests_total Total number of http api requests' echo '# TYPE demo_http_requests_total counter' echo 'demo_http_requests_total{api="add_product"} 4633433' ``` -------------------------------- ### Example PromQL for Alerting Source: https://github.com/ccfos/nightingale/wiki/faq This example demonstrates a PromQL query for an alert rule. It checks if the idle CPU usage is less than 20%. The alert will trigger if this condition persists for the specified duration (e.g., 60 seconds), with checks performed every 10 seconds. ```promql cpu_usage_idle{cpu="cpu-total"} < 20 ``` -------------------------------- ### IPVS Configuration Example Source: https://github.com/ccfos/nightingale/blob/main/integrations/IPVS/markdown/README.md This is a minimal configuration for collecting IPVS metrics. No specific configuration options are required for basic collection. ```toml # Collect virtual and real server stats from Linux IPVS # no configuration ``` -------------------------------- ### Example PromQL Query Generation Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/promql-generator/SKILL.md This example demonstrates generating a PromQL query to find machines with CPU usage exceeding 80%. It involves calculating CPU usage from the `node_cpu_seconds_total` metric. ```json { "query": "100 - avg by(instance)(rate(node_cpu_seconds_total{mode=\"idle\"}[5m])) * 100 > 80", "explanation": "Compute each machine's CPU usage (100% minus the idle proportion), filtering for instances exceeding 80%" } ``` -------------------------------- ### Delete Resource File Response Example Source: https://github.com/ccfos/nightingale/blob/main/doc/api/ai-skill.md Example of a successful response when deleting a resource file. Returns an empty data field. ```json { "dat": "", "err": "" } ``` -------------------------------- ### Complete VictoriaLogs Alert Rule Example Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/create-alert-rule/datasources/victorialogs.md A full example of a VictoriaLogs alert rule, demonstrating the integration of query, trigger, and notification settings. ```json [{ "name": "Application error log alert", "note": "More than 20 error logs for the payment service in the last 1 minute", "prod": "logging", "cate": "victorialogs", "datasource_ids": [11], "datasource_queries": [{"match_type": 0, "op": "in", "values": [11]}], "disabled": 0, "prom_eval_interval": 30, "prom_for_duration": 60, "rule_config": { "queries": [ { "ref": "A", "query": "service:payment AND _msg:error | stats count() as value", "interval": 60 } ], "triggers": [ { "mode": 1, "exp": "$A.value > 20", "severity": 2, "recover_config": {"judge_type": 0} } ] }, "notify_version": 1 }] ``` -------------------------------- ### Common LogsQL Query Examples Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-datasource/datasources/victorialogs.md Examples of LogsQL queries for various search requirements, including simple keyword searches, AND conditions, filtering by stream, excluding keywords, and using regex. ```logs-ql _msg:error ``` ```logs-ql _msg:error AND service:payment ``` ```logs-ql _stream:{job="myapp"} AND _msg:error ``` ```logs-ql _msg:error NOT _msg:debug ``` ```logs-ql _msg:~/"timeout|refused"/ ``` -------------------------------- ### Complete Loki Alert Rule Example Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/create-alert-rule/datasources/loki.md A full example of a Loki alert rule configuration, including name, note, datasource details, evaluation intervals, and notification settings. This demonstrates how to combine query and trigger configurations. ```json [{ "name": "Too many application error logs", "note": "More than 10 error logs within 5 minutes", "prod": "logging", "cate": "loki", "datasource_ids": [2], "datasource_queries": [{"match_type": 0, "op": "in", "values": [2]}], "disabled": 0, "prom_eval_interval": 30, "prom_for_duration": 60, "rule_config": { "queries": [ {"prom_ql": "count_over_time({job=\"myapp\"} |= \"error\" [5m]) > 10", "severity": 2} ], "triggers": [ { "mode": 1, "exp": "$A > 10", "severity": 2, "recover_config": {"judge_type": 0} } ] }, "enable_in_bg": 0, "enable_days_of_weeks": [["0","1","2","3","4","5","6"]], "enable_stimes": ["00:00"], "enable_etimes": ["00:00"], "notify_recovered": 1, "notify_repeat_step": 60, "notify_max_number": 0, "callbacks": [], "append_tags": [], "annotations": {}, "extra_config": {}, "notify_version": 1, "notify_rule_ids": [] }] ``` -------------------------------- ### PromQL Example with Business Group Tagging Source: https://github.com/ccfos/nightingale/wiki/faq Example PromQL query demonstrating how to filter alerts by a business group tag ('bg'). This is recommended for segmenting alerts across different business units. ```promql cpu_usage_idle{bg="a"}<15 ``` -------------------------------- ### Example cURL Request with Basic Authentication Source: https://github.com/ccfos/nightingale/blob/main/doc/api/pushgw-proxy-write.md This example demonstrates how to send a remote_write payload to the /proxy/v1/write endpoint using cURL, including necessary headers and basic authentication. ```bash curl -u myuser:mypass \ -H 'Content-Type: application/x-protobuf' \ -H 'Content-Encoding: snappy' \ -H 'X-Prometheus-Remote-Write-Version: 0.1.0' \ --data-binary @payload.snappy \ "http://pushgw:17000/proxy/v1/write" ``` -------------------------------- ### ClickHouse Time Handling Example Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/sql-generator/SKILL.md Illustrates ClickHouse functions for date conversion and time intervals. ```sql toDate(column), now() - INTERVAL 7 DAY ``` -------------------------------- ### Configure CloudWatch Plugin Instance Source: https://github.com/ccfos/nightingale/blob/main/integrations/CloudWatch/markdown/README.md Example TOML configuration for a plugin instance, specifying metric names and dimensions. ```toml [[instances]] period = "1m" interval = "5m" [[instances.metrics]] names = ["Latency"] ## Dimension filters for Metric (optional) [[instances.metrics.dimensions]] name = "LoadBalancerName" value = "p-example" [[instances.metrics.dimensions]] name = "AvailabilityZone" value = "*" ``` -------------------------------- ### Go Template Example for Alert Event Display Source: https://github.com/ccfos/nightingale/wiki/faq Example of using Go template syntax within alert rule notes to customize the display of alert event information. This allows for dynamic inclusion of labels like metric name and business group. ```go-template 监控指标:{{$labels.__name__}} 所属业务组:{{$labels.busigroup}} ``` -------------------------------- ### MySQL Time Handling Example Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/sql-generator/SKILL.md Shows MySQL-specific functions for date manipulation and current time. ```sql DATE(column), DATE_SUB(NOW(), INTERVAL 7 DAY) ``` -------------------------------- ### SQL Join Query Example Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/sql-generator/SKILL.md Demonstrates a LEFT JOIN operation to combine data from two tables. ```sql SELECT a.*, b.name FROM table_a a LEFT JOIN table_b b ON a.id = b.a_id; ``` -------------------------------- ### Example AMD ROCm SMI Output Source: https://github.com/ccfos/nightingale/blob/main/integrations/AMD_ROCm_SMI/markdown/README.md Example of the metrics collected by the amd_rocm_smi plugin, formatted for InfluxDB. ```text amd_rocm_smi,gpu_id=0x6861,gpu_unique_id=0x2150e7d042a1124,host=ali47xl,name=card0 clocks_current_memory=167i,clocks_current_sm=852i,driver_version=51114i,fan_speed=14i,memory_free=17145282560i,memory_total=17163091968i,memory_used=17809408i,power_draw=7,temperature_sensor_edge=28,temperature_sensor_junction=29,temperature_sensor_memory=92,utilization_gpu=0i 1630572551000000000 amd_rocm_smi,gpu_id=0x6861,gpu_unique_id=0x2150e7d042a1124,host=ali47xl,name=card0 clocks_current_memory=167i,clocks_current_sm=852i,driver_version=51114i,fan_speed=14i,memory_free=17145282560i,memory_total=17163091968i,memory_used=17809408i,power_draw=7,temperature_sensor_edge=29,temperature_sensor_junction=30,temperature_sensor_memory=91,utilization_gpu=0i 1630572701000000000 amd_rocm_smi,gpu_id=0x6861,gpu_unique_id=0x2150e7d042a1124,host=ali47xl,name=card0 clocks_current_memory=167i,clocks_current_sm=852i,driver_version=51114i,fan_speed=14i,memory_free=17145282560i,memory_total=17163091968i,memory_used=17809408i,power_draw=7,temperature_sensor_edge=29,temperature_sensor_junction=29,temperature_sensor_memory=92,utilization_gpu=0i 1630572749000000000 ``` -------------------------------- ### IPVS Real Server Metrics (fwmark) Source: https://github.com/ccfos/nightingale/blob/main/integrations/IPVS/markdown/README.md This example shows IPVS real server metrics when the virtual server is configured using fwmark. It includes connection and packet statistics. ```shell ipvs_real_server,address=172.18.64.220,address_family=inet,port=9000,virtual_fwmark=47 inactive_connections=0i,pkts_out=0i,bytes_out=0i,pps_in=0i,cps=0i,active_connections=0i,pkts_in=0i,bytes_in=0i,pps_out=0i,connections=0i 1541019340000000000 ``` ```shell ipvs_real_server,address=172.18.64.219,address_family=inet,port=9000,virtual_fwmark=47 cps=0i,active_connections=0i,inactive_connections=0i,connections=0i,pkts_in=0i,bytes_out=0i,pkts_out=0i,bytes_in=0i,pps_in=0i,pps_out=0i 1541019340000000000 ``` -------------------------------- ### Diagnose Host Onboarding Pipeline Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/host-onboard-diagnose/SKILL.md Use these commands to gather evidence at different stages of the host onboarding pipeline. This helps pinpoint where the process is failing. ```bash grep -E 'heartbeat|omit_hostname' /etc/categraf/conf/config.toml ``` ```bash curl -s http://:17000/api/n9e/self-metrics | grep ``` -------------------------------- ### Environment Variables for Configuration Source: https://github.com/ccfos/nightingale/blob/main/doc/design/cli/README.md Demonstrates how to override configuration settings using environment variables. These variables take precedence over the configuration file. ```bash export N9E_API_URL=http://localhost:17000 export N9E_TOKEN=your-api-token ``` -------------------------------- ### URL Routing by Severity Example Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/notify-channel-copilot/SKILL.md Demonstrates how to dynamically route an HTTP request URL based on the event's severity level using Go template syntax. ```go http://bot/notify?level={{$event.Severity}} ``` -------------------------------- ### PostgreSQL Multi-Schema Query Example Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/create-alert-rule/datasources/pgsql.md Example of a PostgreSQL query targeting a specific schema other than the default 'public'. The `database.schema.table` format is crucial. ```json { "ref": "A", "sql": "SELECT count(*) AS value FROM testdb.monitoring.events WHERE created_at >= NOW() - INTERVAL '5 minutes'", "keys": {"valueKey": "value"}, "interval": 60 } ``` -------------------------------- ### IPVS Virtual Server Metrics (fwmark) Source: https://github.com/ccfos/nightingale/blob/main/integrations/IPVS/markdown/README.md This example shows IPVS virtual server metrics when configured using fwmark. It includes connection and packet statistics. ```shell ipvs_virtual_server,address_family=inet,fwmark=47,netmask=32,sched=rr cps=0i,connections=0i,pkts_in=0i,pkts_out=0i,bytes_in=0i,bytes_out=0i,pps_in=0i,pps_out=0i 1541019340000000000 ``` -------------------------------- ### Get Index List Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-datasource/datasources/elasticsearch.md Retrieves a list of available Elasticsearch indices. This can be done via a direct POST request or through a proxy GET request. ```APIDOC ## Get Index List ### Description Retrieves a list of available Elasticsearch indices. ### Method POST ### Endpoint /api/n9e/indices ### Request Body - **cate** (string) - Required - Datasource category, should be `elasticsearch`. - **datasource_id** (integer) - Required - The ID of the Elasticsearch datasource. ### Request Example ```json { "cate": "elasticsearch", "datasource_id": 1 } ``` --- ## Get Index List (Proxy) ### Description An alternative method to retrieve a list of indices using a proxy GET request. ### Method GET ### Endpoint /api/n9e/proxy//_cat/indices?format=json&s=index ### Parameters #### Path Parameters - **datasource_id** (integer) - Required - The ID of the Elasticsearch datasource. #### Query Parameters - **format** (string) - Optional - Specifies the output format, e.g., `json`. - **s** (string) - Optional - Specifies sorting, e.g., `index`. ``` -------------------------------- ### Example SQL Query Generation Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/sql-generator/SKILL.md Demonstrates generating a SQL query to calculate daily order amounts for the last 7 days, including date formatting, aggregation, and filtering. ```json { "query": "SELECT DATE(created_at) as date, SUM(amount) as total_amount FROM business.orders WHERE created_at >= DATE_SUB(CURDATE(), INTERVAL 7 DAY) GROUP BY DATE(created_at) ORDER BY date", "explanation": "Group by day and sum the order amounts over the last 7 days, sorted by date" } ``` -------------------------------- ### Call Example for create_dashboard Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/create-dashboard/SKILL.md Use this JSON structure to call the `create_dashboard` tool. It includes essential fields like group ID, name, datasource ID, tags, variables, and panels. Note that complex configurations beyond these simplified fields are not supported. ```json { "group_id": 1, "name": "Linux Host Monitoring", "datasource_id": 1, "tags": "linux host", "variables": "[\"{\\\"name\\\":\\\"ident\\\",\\\"label\\\":\\\"Host\\\",\\\"definition\\\":\\\"label_values(cpu_usage_idle, ident)\\\"}\"]", "panels": "[{\"name\": \"CPU Usage\", \"type\": \"stat\", \"queries\":[{\"promql\": \"avg(cpu_usage_active{cpu=\"cpu-total\",ident=~\"$ident\"})\", \"legend\": \"CPU\"}], \"unit\": \"percent\"},{\"name\": \"CPU Usage Trend\", \"type\": \"timeseries\", \"queries\":[{\"promql\": \"cpu_usage_active{cpu=\"cpu-total\",ident=~\"$ident\"}\", \"legend\": \"{{ident}}\"}], \"unit\": \"percent\"}]" } ``` -------------------------------- ### Get Datasources for Alert Events API Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-alert-events/SKILL.md Use this GET request to fetch a list of datasources associated with alert events. Requires an Authorization token. ```http GET /api/n9e/alert-cur-events-datasources Authorization: Bearer ``` -------------------------------- ### List Integration Components Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/create-dashboard/SKILL.md Use `list_files` to discover available integration components. This helps in identifying the base directory for integration templates. ```shell list_files(base="integrations") ``` -------------------------------- ### API Response for Successful Deletion Start Source: https://github.com/ccfos/nightingale/blob/main/doc/api/alert-mute-batch-delete.md This JSON response indicates that the server has accepted the request and started the asynchronous deletion process for alert mutes. ```json { "dat": "Alert mutes deletion started", "err": "" } ``` -------------------------------- ### Pattern and Action Examples Source: https://github.com/ccfos/nightingale/blob/main/integrations/Mtail/markdown/README.md Shows various ways to combine patterns and actions in mtail, including simple regex matching, variable comparisons, and logical operators. ```mtail /foo/ { ACTION1 } variable > 0 { ACTION2 } /foo/ && variable > 0 { ACTION3 } ``` -------------------------------- ### ClickHouse SQL Example: Group statistics by field Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-datasource/datasources/clickhouse.md Example SQL query for ClickHouse to group statistics by a specific field. Supports ordering and limiting results. ```sql SELECT service, count() AS value FROM logs WHERE ts >= $from AND ts < $to GROUP BY service ORDER BY value DESC LIMIT 10 ``` -------------------------------- ### Basic MySQL Configuration Source: https://github.com/ccfos/nightingale/blob/main/integrations/MySQL/markdown/README.md Configure connection details for a MySQL instance. Ensure username and password are provided for authentication. ```toml [[instances]] address = "127.0.0.1:3306" username = "root" password = "1234" # # set tls=custom to enable tls # parameters = "tls=false" # 通过 show global status监控mysql,默认抓取一些基础指标, # 如果想抓取更多global status的指标,把下面的配置设置为true extra_status_metrics = true # 通过show global variables监控mysql的全局变量,默认抓取一些常规的 # 常规的基本够用了,扩展的部分,默认不采集,下面的配置设置为false extra_innodb_metrics = false # 监控processlist,关注较少,默认不采集 gather_processlist_processes_by_state = false gather_processlist_processes_by_user = false # 监控各个数据库的磁盘占用大小 gather_schema_size = false # 监控所有的table的磁盘占用大小 gather_table_size = false # 是否采集系统表的大小,通过不用,所以默认设置为false gather_system_table_size = false # 通过 show slave status监控slave的情况,比较关键,所以默认采集 gather_slave_status = true # # timeout # timeout_seconds = 3 # # interval = global.interval * interval_times # interval_times = 1 # 为mysql实例附一个instance的标签,因为通过address=127.0.0.1:3306不好区分 # important! use global unique string to specify instance # labels = { instance="n9e-10.2.3.4:3306" } ## Optional TLS Config # use_tls = false # tls_min_version = "1.2" # tls_ca = "/etc/categraf/ca.pem" # tls_cert = "/etc/categraf/cert.pem" # tls_key = "/etc/categraf/key.pem" ## Use TLS but skip chain & host verification # insecure_skip_verify = true # 自定义SQL,指定SQL、返回的各个列那些是作为metric,哪些是作为label # [[instances.queries]] # measurement = "users" # metric_fields = [ "total" ] # label_fields = [ "service" ] # # field_to_append = "" # timeout = "3s" # request = ''' # select 'n9e' as service, count(*) as total from n9e_v5.users # ''' ``` -------------------------------- ### Get Business Group List API Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-alert-events/SKILL.md Use this GET request to retrieve the business group list, which can be used for filtering by bgid. Requires an Authorization token. ```http GET /api/n9e/busi-groups Authorization: Bearer ``` -------------------------------- ### Get Tag Values API Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-alert-events/SKILL.md Use this GET request to retrieve the top 20 most frequent values for a specified tag key. Requires an Authorization token. ```http GET /api/n9e/event-tagvalues?key= Authorization: Bearer ``` -------------------------------- ### Call search_n9e_docs with keywords Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/doc-qa/SKILL.md Use `search_n9e_docs` to find relevant information by breaking out keywords (2-4 words, space-separated, using official terminology). Try synonyms if initial searches yield no results. ```action Call search_n9e_docs(keywords, top_n=3) ``` -------------------------------- ### ClickHouse SQL Example: Count aggregated per minute Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-datasource/datasources/clickhouse.md Example SQL query for ClickHouse to count aggregated data per minute. Uses $from and $to time variables. ```sql SELECT toStartOfMinute(ts) AS t, count() AS value FROM logs WHERE ts >= $from AND ts < $to GROUP BY t ORDER BY t ``` -------------------------------- ### Example Smart Attribute Metrics Output Source: https://github.com/ccfos/nightingale/blob/main/integrations/SMART/markdown/README.md This is an example of the output format for smart attribute metrics such as program fail count, erase fail count, and wear leveling count. ```text smart_attribute_program_fail_count agent_hostname=1.2.3.4 device=nvme0 model= name=Program_Fail_Count serial_no=PHLJ830200CH4P0DGN 0 smart_attribute_erase_fail_count agent_hostname=1.2.3.4 device=nvme0 model= name=Erase_Fail_Count serial_no=PHLJ830200CH4P0DGN 0 smart_attribute_wear_leveling_count agent_hostname=1.2.3.4 device=nvme0 model= name=Wear_Leveling_Count serial_no=PHLJ830200CH4P0DGN 34360328200 ``` -------------------------------- ### Basic MySQL Configuration Source: https://github.com/ccfos/nightingale/blob/main/integrations/MySQL/markdown/mysql.md Configure connection details for a MySQL instance, including address, username, and password. Optional parameters like TLS and custom queries can also be set. ```toml # # collect interval # interval = 15 # 要监控 MySQL,首先要给出要监控的MySQL的连接地址、用户名、密码 [[instances]] address = "127.0.0.1:3306" username = "root" password = "1234" # # set tls=custom to enable tls # parameters = "tls=false" # 通过 show global status监控mysql,默认抓取一些基础指标, # 如果想抓取更多global status的指标,把下面的配置设置为true extra_status_metrics = true # 通过show global variables监控mysql的全局变量,默认抓取一些常规的 # 常规的基本够用了,扩展的部分,默认不采集,下面的配置设置为false extra_innodb_metrics = false # 监控processlist,关注较少,默认不采集 gather_processlist_processes_by_state = false gather_processlist_processes_by_state = false # 监控各个数据库的磁盘占用大小 gather_schema_size = false # 监控所有的table的磁盘占用大小 gather_table_size = false # 是否采集系统表的大小,通过不用,所以默认设置为false gather_system_table_size = false # 通过 show slave status监控slave的情况,比较关键,所以默认采集 gather_slave_status = true # # timeout # timeout_seconds = 3 # # interval = global.interval * interval_times # interval_times = 1 # 为mysql实例附一个instance的标签,因为通过address=127.0.0.1:3306不好区分 # important! use global unique string to specify instance # labels = { instance="n9e-10.2.3.4:3306" } ## Optional TLS Config # use_tls = false # tls_min_version = "1.2" # tls_ca = "/etc/categraf/ca.pem" # tls_cert = "/etc/categraf/cert.pem" # tls_key = "/etc/categraf/key.pem" ## Use TLS but skip chain & host verification # insecure_skip_verify = true # 自定义SQL,指定SQL、返回的各个列那些是作为metric,哪些是作为label # [[instances.queries]] # measurement = "users" # metric_fields = [ "total" ] # label_fields = [ "service" ] # # field_to_append = "" # timeout = "3s" # request = ''' # select 'n9e' as service, count(*) as total from n9e_v5.users # ''' ``` -------------------------------- ### Get Available Event Tag Keys API Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-alert-events/SKILL.md Use this GET request to retrieve a list of tag keys available for filtering alert events. Requires an Authorization token. ```http GET /api/n9e/event-tagkeys Authorization: Bearer ``` -------------------------------- ### Create Alert Rule - Prometheus Simplified Path Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/create-alert-rule/SKILL.md This example demonstrates creating an alert rule for Prometheus using the simplified path. The threshold is passed separately and should not be included in `prom_ql`. ```python create_alert_rule( group_id=1, name="CPU too high", datasource_id=1, prom_ql="avg by (ident) (100 - cpu_usage_idle{cpu=\"cpu-total\"})", threshold=80, operator=">", severity=2 ) ``` -------------------------------- ### ClickHouse SQL Example: Search logs Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-datasource/datasources/clickhouse.md Example SQL query for ClickHouse to search logs based on a message pattern. Supports time filtering, pattern matching, and result limiting. ```sql SELECT * FROM logs WHERE ts >= $from AND ts < $to AND message LIKE '%error%' ORDER BY ts DESC LIMIT 100 ``` -------------------------------- ### Injecting Authentication Header Example Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/notify-channel-copilot/SKILL.md Shows how to inject an authentication token into the Authorization header of an HTTP request using Go template syntax. ```go Authorization: Bearer {{$params.token}} ``` -------------------------------- ### Get Elasticsearch Index List via Proxy Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-datasource/datasources/elasticsearch.md An alternative method to get the index list using the proxy endpoint. Useful for direct access to Elasticsearch's _cat API. ```http GET /api/n9e/proxy//_cat/indices?format=json&s=index Authorization: Bearer ``` -------------------------------- ### HTML Escaping in Body Example Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/notify-channel-copilot/SKILL.md Provides an example of how to use the `unescaped` function within Go templates to prevent HTML tag escaping when the Body field requires raw HTML content. ```go {{unescaped "..."}} ``` -------------------------------- ### Run Categraf with Docker (Host Metrics) Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/categraf-deploy-guide/SKILL.md Deploy Categraf using Docker to collect host CPU, memory, and disk metrics. This requires mounting host system directories and using host networking. ```bash docker run -td \ --name categraf \ --restart=always \ --network=host \ -e TZ=Asia/Shanghai \ -e HOST_PROC=/hostfs/proc \ -e HOST_SYS=/hostfs/sys \ -e HOST_MOUNT_PREFIX=/hostfs \ -v /home/flashcat/categraf/conf/:/etc/categraf/conf \ -v /proc:/hostfs/proc:ro \ -v /sys:/hostfs/sys:ro \ flashcatcloud/categraf:latest ``` -------------------------------- ### MongoDB User Authorization Example Source: https://github.com/ccfos/nightingale/blob/main/integrations/MongoDB/markdown/README.md This shell command demonstrates how to create a MongoDB user with the necessary roles for Ccfgraf monitoring. It connects to the MongoDB instance and uses the db.createUser command. ```shell mongo -h xxx -u xxx -p xxx --authenticationDatabase admin > use admin > db.createUser({user:"categraf",pwd:"categraf",roles: [{role:"read",db:"local"},{"role":"clusterMonitor","db":"admin"}]}) ``` -------------------------------- ### SQL Grouping and Sorting Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/sql-generator/SKILL.md Demonstrates how to group results, apply aggregate functions, filter groups, and order the output. ```sql SELECT column, COUNT(*) as cnt FROM table GROUP BY column HAVING cnt > 10 ORDER BY cnt DESC LIMIT 100; ``` -------------------------------- ### Complete Elasticsearch Alert Rule Example Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/create-alert-rule/datasources/elasticsearch.md A full example of an Elasticsearch alert rule designed to detect 'Too many ES error logs'. It specifies alert conditions, notification settings, and data source query details. ```json [{ "name": "Too many ES error logs", "note": "More than 100 error logs within 5 minutes", "prod": "logging", "cate": "elasticsearch", "datasource_ids": [2], "datasource_queries": [{"match_type": 0, "op": "in", "values": [2]}], "disabled": 0, "prom_eval_interval": 60, "prom_for_duration": 0, "rule_config": { "queries": [ { "ref": "A", "index_type": "index", "index": "logs-*", "filter": "level:ERROR", "date_field": "@timestamp", "interval": 300, "value": {"func": "count"} } ], "triggers": [ { "mode": 1, "exp": "$A > 100", "severity": 2, "recover_config": {"judge_type": 0} } ] }, "enable_in_bg": 0, "enable_days_of_weeks": [["0","1","2","3","4","5","6"]], "enable_stimes": ["00:00"], "enable_etimes": ["00:00"], "notify_recovered": 1, "notify_repeat_step": 60, "notify_max_number": 0, "callbacks": [], "append_tags": [], "annotations": {}, "extra_config": {}, "notify_version": 1, "notify_rule_ids": [] }] ``` -------------------------------- ### Get Label Values Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/query-datasource/datasources/prometheus.md Retrieve all values for a specific label name. ```APIDOC ## GET /api/n9e/proxy//api/v1/label//values ### Description Retrieves all unique values for a given label name from the datasource. ### Method GET ### Endpoint /api/n9e/proxy//api/v1/label//values ### Parameters #### Path Parameters - **label_name** (string) - Required - The name of the label to retrieve values for. #### Query Parameters - **Authorization** (string) - Required - Bearer ``` -------------------------------- ### Bring NIC up (whitelist) Source: https://github.com/ccfos/nightingale/blob/main/aiagent/skill/embedded/builtin/modify-task-tpl/SKILL.md Restarts a network interface if it's down. Includes a whitelist to prevent accidental modification of container or VPN interfaces. ```bash #!/bin/bash set -euo pipefail PAYLOAD=$(cat) IFACE=$(echo "$PAYLOAD" | jq -r '.iface // empty') [ -z "$IFACE" ] && { echo "no iface"; exit 2; } # whitelist: only allow eth* / ens* / eno*, to avoid touching container/VPN NICs [[ "$IFACE" =~ ^(eth|ens|eno) ]] || { echo "iface $IFACE not in whitelist"; exit 0; } ip link show "$IFACE" ip link set "$IFACE" up sleep 2 ip link show "$IFACE" ```