IBM Kubecost Self Hosted

https://www.ibm.com/docs/en/kubecost/self-hosted/2.x
Admin
IBM Kubecost Self Hosted provides cost management and optimization for Kubernetes environments,...
Tokens:170,847
Snippets:1,810
Trust Score:10
Update:3 months ago
Show doc for...
Tokens:
Raw
### Install Kubecost bundle

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installations-installing-kubecost-plural

Installs the specified Kubecost bundle into your workspace. This example shows the AWS configuration.

```bash
plural bundle install kubecost kubecost-aws
```

--------------------------------

### Install Kubecost Primary

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-aws-distro-open-telemetry

Install the primary Kubecost instance using Helm.

```bash
aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
helm install YOUR_NAMESPACE \
  oci://public.ecr.aws/kubecost/cost-analyzer \
  -f values-kubecost-primary.yaml
  -f https://raw.githubusercontent.com/kubecost/cost-analyzer-helm-chart/v2.6/cost-analyzer/values-eks-cost-monitoring.yaml
```

--------------------------------

### Install Kubecost Agent

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-aws-distro-open-telemetry

Install the Kubecost agent for multi-account monitoring.

```bash
aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
helm install YOUR_NAMESPACE \
  oci://public.ecr.aws/kubecost/cost-analyzer \
  -f values-kubecost-agent.yaml
  -f https://raw.githubusercontent.com/kubecost/cost-analyzer-helm-chart/v2.7/cost-analyzer/values-eks-cost-monitoring.yaml
```

--------------------------------

### Install and deploy Plural Console

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installations-installing-kubecost-plural

Installs the Plural Console to manage resources and upgrades, followed by the build and deploy commands.

```bash
plural bundle install console console-aws
plural build
plural deploy --commit"deploying the console too"
```

--------------------------------

### Install Kubecost via Helm

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installations-installing-kubecost-alibaba

Use this command to install the Kubecost cost-analyzer chart into the kubecost namespace.

```bash
helm install kubecost/cost-analyzer -n kubecost -f values.yaml
```

--------------------------------

### Install KSM via Helm

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=overview-kube-state-metrics-ksm-emission

Use this command to install the kube-state-metrics chart from the Prometheus Community repository into a dedicated namespace.

```bash
helm install kube-state-metrics \
  --repo https://prometheus-community.github.io/helm-charts kube-state-metrics \
  --namespace kube-state-metrics --create-namespace
```

--------------------------------

### Install Kubecost via Helm

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-aws-agentless-amp

Authenticates with ECR and installs the Kubecost Helm chart using specific configuration files.

```bash
aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
helm install $KUBECOST_NAMESPACE -n $KUBECOST_NAMESPACE \
  oci://public.ecr.aws/kubecost/cost-analyzer \
  -f https://raw.githubusercontent.com/kubecost/cost-analyzer-helm-chart/v2.6/cost-analyzer/values-eks-cost-monitoring.yaml \
  -f values-kubecost-primary.yaml
```

```bash
aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
helm install $KUBECOST_NAMESPACE -n $KUBECOST_NAMESPACE \
  oci://public.ecr.aws/kubecost/cost-analyzer \
  -f https://raw.githubusercontent.com/kubecost/cost-analyzer-helm-chart/v2.6/cost-analyzer/values-eks-cost-monitoring.yaml \
  -f values-kubecost-agent.yaml
```

--------------------------------

### Install Kubecost via Helm

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=configuration-parallel-install-upgrading

Commands for installing the primary and parallel Kubecost instances using distinct namespaces and release names.

```bash
helm install kubecost --namespace kubecost ...
```

```bash
helm install kubecost2 --namespace kubecost2 ...
```

--------------------------------

### Example output for describe-addon command

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installations-amazon-eks-integration

The expected JSON response when the add-on is successfully active.

```json
{
  "addon": {
    "addonName": "kubecost_kubecost",
    "clusterName": "$YOUR_CLUSTER_NAME",
    "status": "ACTIVE",
    "addonVersion": "v1.97.0-eksbuild.1",
    "health": {
      "issues": []
    },
    "addonArn": "arn:aws:eks:$AWS_REGION:xxxxxxxxxxxx:addon/$YOUR_CLUSTER_NAME/kubecost_kubecost/90c23198-cdd3-b295-c410-xxxxxxxxxxxx",
    "createdAt": "2022-12-01T12:18:26.497000-08:00",
    "modifiedAt": "2022-11-10T12:53:21.140000-08:00",
    "tags": {}
  }
}
```

--------------------------------

### Example output for create-addon command

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installations-amazon-eks-integration

The expected JSON response after initiating the Kubecost add-on creation.

```json
{
  "addon": {
    "addonName": "kubecost_kubecost",
    "clusterName": "$YOUR_CLUSTER_NAME",
    "status": "CREATING",
    "addonVersion": "v1.97.0-eksbuild.1",
    "health": {
      "issues": []
    },
    "addonArn": "arn:aws:eks:$AWS_REGION:xxxxxxxxxxxx:addon/$YOUR_CLUSTER_NAME/kubecost_kubecost/90c23198-cdd3-b295-c410-xxxxxxxxxxxx",
    "createdAt": "2022-12-01T12:18:26.497000-08:00",
    "modifiedAt": "2022-12-01T12:50:52.222000-08:00",
    "tags": {}
  }
}
```

--------------------------------

### Install Kubecost with Dedicated Service Account

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=integration-aws-cloud-using-irsaeks-pod-identities

Installs Kubecost via Helm using the dedicated service account and cloud integration secret.

```bash
helm install kubecost \
 --repo https://kubecost.github.io/cost-analyzer/ cost-analyzer \
 --namespace kubecost \
 --setserviceAccount.name=kubecost-serviceaccount \
 --setserviceAccount.create=false\
 --setkubecostProductConfigs.cloudIntegrationSecret=cloud-integration
```

--------------------------------

### Query total costs example

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-query-collection-total-costs

Example request and response for retrieving total costs of a collection without group details.

```http
GET /model/collection/query/total?id=8a939ba1-ff15-4600-b711-2cb109114914&window=30d
```

```json
{
  "id": "8a939ba1-ff15-4600-b711-2cb109114914",
  "name": "Infrastructure",
  "category": "Team",
  "window": {
    "start": "2023-01-01T00:00:00Z",
    "end": "2023-02-01T00:00:00Z"
  },
  "data": {
    "cost": {
      "totalCost": 60.0,
      "idleCost": 10.0,
      "domainCosts": {
        "kubernetesCost": {
          "totalCost": 40.0,
          "idleCost": 10.0
        },
        "cloudCost": {
          "totalCost": 20.0
        }
      }
    }
  }
}
```

--------------------------------

### Install DCGM Exporter

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=configuration-nvidia-gpu-monitoring-configurations

Deploy the DCGM Exporter using the specified configuration file.

```bash
helm upgrade -i dcgm dcgm-exporter \
  --repo https://nvidia.github.io/dcgm-exporter/helm-charts \
  -n dcgm-exporter --create-namespace \
  -f values-dcgm.yaml
```

--------------------------------

### Query total costs with groups example

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-query-collection-total-costs

Example request and response for retrieving total costs including group-level breakdowns and overlap data by setting groups=true.

```http
GET /model/collection/query/total?id=8a939ba1-ff15-4600-b711-2cb109114914&window=30d&groups=true
```

```json
{
  "id": "8a939ba1-ff15-4600-b711-2cb109114914",
  "name": "Infrastructure",
  "category": "Team",
  "window": {
    "start": "2023-01-01T00:00:00Z",
    "end": "2023-02-01T00:00:00Z"
  },
  "data": {
    "cost": {
      "totalCost": 60.0,
      "idleCost": 10.0,
      "domainCosts": {
        "kubernetesCost": {
          "totalCost": 40.0,
          "idleCost": 10.0
        },
        "cloudCost": {
          "totalCost": 20.0
        }
      }
    },
    "overlap": -10.0,
    "groups": [
      {
        "selectors": {
          "kubernetes": {
            "kind": "container",
            "filterString": "namespace:\"infrastructure\""
          }
        },
        "cost": {
          "totalCost": 30.0,
          "idleCost": 5.0,
          "domainCosts": {
            "kubernetesCost": {
              "totalCost": 30.0,
              "idleCost": 5.0
            }
          }
        }
      },
      {
        "selectors": {
          "kubernetes": {
            "kind": "container",
            "filterString": "node:\"monitoring\""
          }
        },
        "cost": {
          "totalCost": 20.0,
          "idleCost": 5.0,
          "domainCosts": {
            "kubernetesCost": {
              "totalCost": 20.0,
              "idleCost": 5.0
            }
          }
        }
      },
      {
        "selectors": {
          "cloud": {
            "filterString": "service:\"s3\"+label[team]:\"infrastructure\""
          }
        },
        "cost": {
          "totalCost": 20.0,
          "domainCosts": {
            "cloudCost": {
              "totalCost": 20.0
            }
          }
        }
      }
    ]
  }
}
```

--------------------------------

### Trend Calculation Output Example

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-cloud-cost-trends-api

Example output showing a calculated trend value for a specific project, indicating a percentage change in spending.

```json
"trends": {
  "project-123": {
    "isInfinite": false,
    "isNaN": false,
    "value": -0.1472170691451784
  }
},
"window": {
  "start": "2023-11-29T00:00:00Z",
  "end": "2023-12-06T00:00:00Z"
},
"comparisonWindow": {
  "start": "2023-11-22T00:00:00Z",
  "end": "2023-11-29T00:00:00Z"
}
```

--------------------------------

### Install Datadog Agent via Helm

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=configuration-integrating-kubecost-datadog

Add the Datadog Helm repository and install the agent with the necessary Prometheus scraping flags enabled.

```bash
helm repo add datadog https://helm.datadoghq.com
helm upgrade -i datadog-agent datadog/datadog \
--setdatadog.site='us5.datadoghq.com'\
--setdatadog.apiKey=$<DATADOG_KEY_ID> \
--setdatadog.prometheusScrape.enabled=‘true’ \
--setdatadog.prometheusScrape.serviceEndpoints=‘true’
```

--------------------------------

### Create Kubecost Namespace

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-aws-distro-open-telemetry

Create a dedicated namespace for the Kubecost installation.

```bash
kubectl create ns YOUR_NAMESPACE
```

--------------------------------

### Example cost metric response

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-amazon-managed-service-prometheus

Expected JSON output when verifying cost metrics.

```json
{
  "status": "success",
  "data": {
    "resultType": "vector",
    "result": [
      {
        "metric": {
          "__name__": "node_total_hourly_cost",
          "cluster_id": "qa-eks1",
          "alpha_eksctl_io_cluster_name": "qa-eks1",
          "arch": "amd64",
          "instance": "ip-192-168-47-226.us-east-2.compute.internal",
          "instance_type": "t3.medium",
          "job": "kubecost"
        },
        "value": [
          1697630306,
          "0.04160104542160034"
        ]
      }
    ]
  }
}
```

--------------------------------

### Initialize Plural configuration

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installations-installing-kubecost-plural

Run this command to start the configuration wizard for your Git repository and cloud provider.

```bash
plural init
```

--------------------------------

### Initialize Namespace and Environment Variables

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=integration-aws-cloud-using-irsaeks-pod-identities

Sets up the Kubernetes namespace and required environment variables for cluster configuration.

```bash
kubectl create ns kubecostexportCLUSTER_NAME=YOUR_CLUSTERexportAWS_REGION=YOUR_REGION
```

--------------------------------

### Helm installation error due to missing PodSecurityPolicy

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=troubleshoot-install

Example of the error message encountered when installing Kubecost on Kubernetes v1.25+ due to the removal of PodSecurityPolicy.

```bash
helm install kubecost kubecost/cost-analyzer
Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: [
    resource mapping not found for name: "kubecost-grafana" namespace: "" from "": no matches for kind "PodSecurityPolicy" in version "policy/v1beta1" ensure CRDs are installed first,
    resource mapping not found for name: "kubecost-cost-analyzer-psp" namespace: "" from "": no matches for kind "PodSecurityPolicy" in version "policy/v1beta1" ensure CRDs are installed first
]
```

--------------------------------

### Retrieve cluster cost events via GET request

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-cost-events-audit-api

Example JSON response structure for a successful 200 OK request to the audit events endpoint.

```json
{
  "code": 200,
  "data": [
    {
      "eventType": "add",
      "resourceType": "deployment",
      "timestamp": "2023-02-28T13:58:44.552788548Z",
      "properties": {
        "cluster": "cluster-one",
        "name": "sample-deployment",
        "namespace": "kubecost"
      },
      "totalRequests": {
        "cpu": "100m",
        "memory": "32Mi"
      },
      "resourceCosts": {
        "costPerCPUCoreHour": 0.031616438356164386,
        "costPerRAMByteHour": 3.94217363775593e-12,
        "costPerGPUHour": 0.95
      },
      "costPrediction": {
        "totalMonthlyRate": 2.4045625000000004,
        "cpuMonthlyRate": 2.3080000000000003,
        "ramMonthlyRate": 0.0965625,
        "gpuMonthlyRate": 0
      }
    },
    ...
  ]
}
```

--------------------------------

### Upgrade Kubecost Installation

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=federation-kubecost-aggregator

Upgrades or installs the Kubecost release using the specified aggregator configuration file.

```bash
helm upgrade \
  --install"kubecost"\
  --repo https://kubecost.github.io/cost-analyzer/ cost-analyzer \
  --namespace kubecost \
  -f aggregator.yaml
```

--------------------------------

### Cloud Cost Aggregation Example

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=explorer-cloud-cost-metrics

Demonstrates how KubernetesPercent values are calculated when aggregating costs across nodes with different pricing models.

```text
Node1 {
    ListCost: {
        Cost: 2,
        KubernetesPercent: 1.0,
    },
    AmortizedNetCost: {
        Cost:  1,
        KubernetesPercent: 1.0
    },
}

Node2 {
    ListCost: {
        Cost: 2,
        KubernetesPercent: 0.0,
    },
    AmortizedNetCost: {
        Cost:  2,
        KubernetesPercent: 0.0
    },
}

agg {
    ListCost: {
        Cost: 4,
        KubernetesPercent: 0.5,
    },
    AmortizedNetCost: {
        Cost:  3,
        KubernetesPercent: 0.33
    },

}
```

--------------------------------

### Install Kubecost with Datadog configuration

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=configuration-integrating-kubecost-datadog

Uses Helm to deploy Kubecost with a predefined Datadog values file and a specific token.

```bash
helm upgrade --install kubecost --namespace kubecost --create-namespace \
 --repo https://kubecost.github.io/cost-analyzer/ cost-analyzer \
 -f https://raw.githubusercontent.com/kubecost/poc-common-configurations/main/datadog/datadog-values.yaml \
 --setkubecostToken="aGVsbUBrdWJlY29zdC5jb20=xm343yadf98"
```

--------------------------------

### Predict API Response Example

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-spec-cost-prediction-api

Example JSON response structure returned by the /speccost endpoint upon a successful request.

```json
{
  "code": 200,
  "data": [
    {
      "namespace": "kubecost",
      "controllerKind": "deployment",
      "controllerName": "kubecost-cost-analyzer",
      "costBefore": {
        "totalMonthlyRate": 3.5397661399108418,
        "cpuMonthlyRate": 2.3273929838395513,
        "ramMonthlyRate": 1.2123731560712905,
        "gpuMonthlyRate": 0,
        "monthlyCPUCoreHours": 73,
        "monthlyRAMByteHours": 304653271040,
        "monthlyGPUHours": 0
      },
      "costAfter": {
        "totalMonthlyRate": 2.623504800996625,
        "cpuMonthlyRate": 0.6283961056366789,
        "ramMonthlyRate": 1.9951086953599462,
        "gpuMonthlyRate": 0,
        "monthlyCPUCoreHours": 19.71,
        "monthlyRAMByteHours": 501344315550,
        "monthlyGPUHours": 0
      },
      "costChange": {
        "totalMonthlyRate": -0.9162613389142167,
        "cpuMonthlyRate": -1.6989968782028724,
        "ramMonthlyRate": 0.7827355392886557,
        "gpuMonthlyRate": 0,
        "monthlyCPUCoreHours": -53.29,
        "monthlyRAMByteHours": 196691044510,
        "monthlyGPUHours": 0
      }
    }
  ]
}
```

--------------------------------

### Execute GKE service account setup script

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=configuration-cluster-controller

Run this command to automatically create the required service account for GKE clusters.

```bash
/bin/bash -c "$(curl -fsSL https://github.com/kubecost/cluster-turndown/releases/latest/download/gke-create-service-key.sh)" -- <Project ID> <Service Account Name> <Namespace> cluster-controller-service-key
```

--------------------------------

### Install Kubecost with Helm

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-grafana-cloud-integration-kubecost

Use this command to install or upgrade the Kubecost Helm release with the necessary Grafana Cloud Prometheus configuration.

```bash
helm upgrade -i -n kubecost kubecost kubecost/cost-analyzer \
  --namespace kubecost \
  --set global.prometheus.fqdn=https://<REPLACE-WITH-GRAFANA-PROM-QUERY-ENDPOINT> \
  --set global.prometheus.enabled=false \
  --set global.prometheus.queryServiceBasicAuthSecretName=dbsecret
```

--------------------------------

### Clone Configuration Repository

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-aws-distro-open-telemetry

Download the necessary configuration files for deploying Kubecost with ADOT.

```bash
git clone https://github.com/kubecost/poc-common-configurations.git
cd poc-common-configurations/aws-amp/adot
```

--------------------------------

### Build and deploy Kubecost

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installations-installing-kubecost-plural

Generates the configuration and deploys the infrastructure to your cloud provider.

```bash
plural build
plural deploy --commit"deploying kubecost"
```

--------------------------------

### Metadata Connection Error Example

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=troubleshoot-install

Example of a timeout error occurring when the Kubecost pod cannot reach the cloud provider metadata server.

```text
gcpprovider.go Error loading metadata cluster-name: Get "http://169.254.169.254/computeMetadata/v1/instance/attributes/cluster-name": dial tcp 169.254.169.254:80: i/o timeout
```

--------------------------------

### Install Kubecost via Flat Manifest

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installation

Deploys Kubecost using a static manifest file. This method is not recommended due to limited flexibility for future upgrades.

```bash
kubectl apply -f https://raw.githubusercontent.com/kubecost/cost-analyzer-helm-chart/v2.8/kubecost.yaml
```

--------------------------------

### Query collection time series costs example

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-query-collection-time-series-costs

Example request and response for querying time series costs over a 30-day window.

```http
GET /model/collection/query/timeseries?id=8a939ba1-ff15-4600-b711-2cb109114914&window=30d
```

```json
{
  "id": "8a939ba1-ff15-4600-b711-2cb109114914",
  "name": "Infrastructure",
  "category": "Team",
  "window": {
    "start": "2023-01-01T00:00:00Z",
    "end": "2023-02-01T00:00:00Z"
  },
  "data": {
    "timeseries": [
      {
        "window": {
          "start": "2023-01-01T00:00:00Z",
          "end": "2023-01-02T00:00:00Z"
        },
        "cost": {
          "totalCost": 20.0,
          "idleCost": 5.0,
          "domainCosts": {
            "cloudCost": {
              "totalCost": 10.0
            },
            "kubernetesCost": {
              "totalCost": 10.0,
              "idleCost": 5.0
            }
          }
        }
      },
      {
        "window": {
          "start": "2023-01-02T00:00:00Z",
          "end": "2023-01-03T00:00:00Z"
        },
        "cost": {
          "totalCost": 20.0,
          "idleCost": 5.0,
          "domainCosts": {
            "cloudCost": {
              "totalCost": 10.0
            },
            "kubernetesCost": {
              "totalCost": 10.0,
              "idleCost": 5.0
            }
          }
        }
      },
      ...
      {
        "window": {
          "start": "2023-01-31T00:00:00Z",
          "end": "2023-02-01T00:00:00Z"
        },
        "cost": {
          "totalCost": 20.0,
          "idleCost": 5.0,
          "domainCosts": {
            "cloudCost": {
              "totalCost": 10.0
            },
            "kubernetesCost": {
              "totalCost": 10.0,
              "idleCost": 5.0
            }
          }
        }
      }
    ]
  }
}
```

--------------------------------

### Analyze the CPU profile

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=install-profile-http-endpoint

Use the Go pprof tool to visualize the captured CPU profile data.

```bash
go tool pprof -http 127.0.0.1:9367 /tmp/profile.cpu.out
```

--------------------------------

### Install Amazon EBS CSI add-on

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installations-amazon-eks-integration

Installs the Amazon EBS CSI add-on for EKS using the previously created IAM role.

```bash
eksctl create addon \
  --name aws-ebs-csi-driver \
  --cluster $CLUSTER_NAME \
  --service-account-role-arn $SERVICE_ACCOUNT_ROLE_ARN \
  --force
```

--------------------------------

### Query category time series costs example

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-query-category-time-series-costs

Example request and response for querying time series costs for a category over a 30-day window.

```http
GET /model/collections/query/timeseries?category=Team&window=30d
```

```json
{
  "category": "Team",
  "window": {
    "start": "2023-01-01T00:00:00Z",
    "end": "2023-02-01T00:00:00Z"
  },
  "collections": [
    {
      "id": "8a939ba1-ff15-4600-b711-2cb109114914",
      "name": "Infrastructure"
    },
    {
      "id": "5aaaab83-f721-41da-ad90-8dca4f7c0f45",
      "name": "Core"
    },
    {
      "id": "ba8bda2f-2cce-4f9d-a9d7-655b3ca52cbd",
      "name": "Front End"
    }
  ],
  "data": {
    "timeseries": [
      {
        "window": {
          "start": "2023-01-01T00:00:00Z",
          "end": "2023-01-02T00:00:00Z"
        },
        "cost": {
          "totalCost": 30.0,
          "idleCost": 5.0,
          "domainCosts": {
            "cloudCost": {
              "totalCost": 10.0
            },
            "kubernetesCost": {
              "totalCost": 20.0,
              "idleCost": 5.0
            }
          }
        }
      },
      {
        "window": {
          "start": "2023-01-02T00:00:00Z",
          "end": "2023-01-03T00:00:00Z"
        },
        "cost": {
          "totalCost": 30.0,
          "idleCost": 5.0,
          "domainCosts": {
            "cloudCost": {
              "totalCost": 10.0
            },
            "kubernetesCost": {
              "totalCost": 20.0,
              "idleCost": 5.0
            }
          }
        }
      },
      ...
      {
        "window": {
          "start": "2023-01-31T00:00:00Z",
          "end": "2023-02-01T00:00:00Z"
        },
        "cost": {
          "totalCost": 30.0,
          "idleCost": 5.0,
          "domainCosts": {
            "cloudCost": {
              "totalCost": 10.0
            },
            "kubernetesCost": {
              "totalCost": 20.0,
              "idleCost": 5.0
            }
          }
        }
      }
    ]
  }
}
```

--------------------------------

### Download Kubecost Datadog Dashboard

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=configuration-integrating-kubecost-datadog

Use this command to download the example dashboard JSON file from the official Kubecost repository.

```bash
wget https://raw.githubusercontent.com/kubecost/poc-common-configurations/main/datadog/Kubecostdashboard.json
```

--------------------------------

### Download the CPU profile

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=install-profile-http-endpoint

Save the generated CPU profile to a local file for further analysis.

```bash
curl -G 'localhost:9090/model/httpprofile/cpu' \
  -o '/tmp/profile.cpu.out'
```

--------------------------------

### Update collection example

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-update-existing-collection

Example of a PUT request targeting a specific collection ID and the corresponding JSON body containing the updated collection specification.

```http
PUT /model/collection?id=8a939ba1-ff15-4600-b711-2cb109114914
```

```json
{
  "name": "Infrastructure",
  "category": "Team",
  "groups": [
    {
      "selectors": {
        "kubernetes": {
          "kind": "container",
          "filterString": "namespace:\"infrastructure\""
        }
      }
    },
    {
      "selectors": {
        "kubernetes": {
          "kind": "container",
          "filterString": "label[app]:\"infrastructure\""
        }
      }
    },
    {
      "selectors": {
        "cloud": {
          "filterString": "service:\"AmazonEC2\"+label[team]:\"infrastructure\""
        }
      }
    }
  ]
}
```

--------------------------------

### Install Kubecost with Helm

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-google-cloud-managed-service-prometheus

Execute the Helm upgrade command to deploy Kubecost using the configured values.yaml file.

```bash
helm upgrade -i kubecost cost-analyzer/ \
  --namespace kubecost \
  --create-namespace \
  -f values.yaml
```

--------------------------------

### Install Kubecost via Helm

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installation

Deploys Kubecost directly using the Helm chart. This method does not generate a product token for tier management or trials.

```bash
helm upgrade --install kubecost \
  --repo https://kubecost.github.io/cost-analyzer/ cost-analyzer \
  --namespace kubecost --create-namespace
```

--------------------------------

### Install Grafana Agent for Kubecost

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-grafana-cloud-integration-kubecost

Configures and installs the Grafana Agent to scrape Kubecost metrics. Replace placeholders with your specific Grafana Cloud credentials and cluster details.

```bash
cat <<'EOF' |

kind: ConfigMap
metadata:
  name: grafana-agent
apiVersion: v1
data:
  agent.yaml: |
    metrics:
      wal_directory: /var/lib/agent/wal
      global:
        scrape_interval: 60s
        external_labels:
          cluster: <REPLACE-WITH-YOUR-CLUSTER-NAME>
      configs:
      - name: integrations
        remote_write:
        - url: https://<REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-ENDPOINT>
          basic_auth:
            username: <REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-USERNAME>
            password: <REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-API-KEY>
        scrape_configs: #Need further scrape config update
        - job_name: kubecost
          honor_labels: true
          scrape_interval: 1m
          scrape_timeout: 10s
          metrics_path: /metrics
          scheme: http
          dns_sd_configs:
          - names:
            - kubecost-cost-analyzer.kubecost
            type: 'A'
            port: 9003
        - job_name: kubecost-networking
          kubernetes_sd_configs:
            - role: pod
          relabel_configs:
          # Scrape only the the targets matching the following metadata
            - source_labels: [__meta_kubernetes_pod_label_app]
              action: keep
              regex:  'kubecost-network-costs'

EOF
(export NAMESPACE=kubecost && kubectl apply -n $NAMESPACE -f -)

MANIFEST_URL=https://raw.githubusercontent.com/grafana/agent/v0.24.2/production/kubernetes/agent-bare.yaml NAMESPACE=kubecost /bin/sh -c "$(curl -fsSL https://raw.githubusercontent.com/grafana/agent/v0.24.2/production/kubernetes/install-bare.sh)" | kubectl apply -f -
```

--------------------------------

### Create Kubecost Scraper

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-aws-agentless-amp

Initialize the AMP scraper for Kubecost using the specified configuration blob.

```bash
KUBECOST_SCRAPER_OUTPUT=$(aws amp create-scraper --output json \
  --alias kubecost-scraper \
  --source eksConfiguration="{clusterArn=$CLUSTER_ARN, securityGroupIds=[$SECURITY_GROUP_IDS],subnetIds=[$SUBNET_IDS]}" \
  --scrape-configuration configurationBlob="$(base64 scraper-kubecost-with-networking.yaml|tr -d '\n')" \
  --destination ampConfiguration="{workspaceArn=$WORKSPACE_ARN}")
echo $KUBECOST_SCRAPER_OUTPUT
KUBECOST_SCRAPER_ID=$(echo $KUBECOST_SCRAPER_OUTPUT|jq -r .scraperId)
echo $KUBECOST_SCRAPER_ID
```

--------------------------------

### GET /model/logs/level

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=troubleshoot-install

Retrieves the current log level configuration.

```APIDOC
## GET /model/logs/level

### Description
Retrieves the current log level currently in use by the system.

### Method
GET

### Endpoint
/model/logs/level
```

--------------------------------

### Clone Configuration Repository

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-aws-agentless-amp

Download the required configuration files for deploying Kubecost with AWS Agentless AMP.

```bash
git clone https://github.com/kubecost/poc-common-configurations.git
cd poc-common-configurations/aws/amp-agentless
```

--------------------------------

### GET /model/savings/turbonomic/suspendContainerPods

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-turbonomic-actions

Returns virtual machines that Turbonomic recommends for suspension.

```APIDOC
## GET /model/savings/turbonomic/suspendContainerPods

### Description
The Suspend Container Pods API returns virtual machines that Turbonomic recommends for suspension. The list of results returned should align with those in the Turbonomic Actions Center.

### Method
GET

### Endpoint
http://<kubecost-address>/model/savings/turbonomic/suspendContainerPods

### Parameters
#### Query Parameters
- **filter** (string) - Optional - Filter your results by cluster, namespace and/or controller.

### Response
#### Success Response (200)
- **code** (integer) - Status code
- **data** (object) - Response data containing numResults, totalSavings, and actions list

#### Response Example
{
"code": 200,
"data": {
"numResults": 1,
"totalSavings": 9.03,
"actions": [
{
 "action": {
   "cluster": "standard-cluster-1",
   "node": "gke-standard-cluster-1-spotpool-b4a02c44-1001",
   "available": true,
   "targetId": "11111111111111"
 },
 "currentMonthlyRate": 9.03,
 "predictedMonthlyRate": 0,
 "predictedSavings": 9.03
}
]
}
}
```

--------------------------------

### Create and configure cross-project service account

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=integrations-gcp-cloud-billing-integration

Execute these commands to create the service account and assign the required IAM roles for BigQuery access across projects.

```bash
gcloud config set project $KUBECOST_PROJECT_ID
        
gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME \
  --display-name "Cross Project CUR" \
  --format json
        
gcloud projects add-iam-policy-binding $BIG_QUERY_PROJECT_ID \
  --member serviceAccount:$SERVICE_ACCOUNT_NAME@$KUBECOST_PROJECT_ID.iam.gserviceaccount.com \
  --role roles/compute.viewer
        
gcloud projects add-iam-policy-binding $BIG_QUERY_PROJECT_ID \
  --member serviceAccount:$SERVICE_ACCOUNT_NAME@$KUBECOST_PROJECT_ID.iam.gserviceaccount.com \
  --role roles/bigquery.user
        
gcloud projects add-iam-policy-binding $BIG_QUERY_PROJECT_ID \
  --member serviceAccount:$SERVICE_ACCOUNT_NAME@$KUBECOST_PROJECT_ID.iam.gserviceaccount.com \
  --role roles/bigquery.dataViewer
        
gcloud projects add-iam-policy-binding $BIG_QUERY_PROJECT_ID \
  --member serviceAccount:$SERVICE_ACCOUNT_NAME@$KUBECOST_PROJECT_ID.iam.gserviceaccount.com \
  --role roles/bigquery.jobUser
```

--------------------------------

### GET /model/budgets

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-budget-api

Retrieves a list of all existing recurring budget rules.

```APIDOC
## GET /model/budgets

### Description
Lists all existing recurring budget rules.

### Method
GET

### Endpoint
http://<your-kubecost-address>/model/budgets

### Response
#### Success Response (200)
- **code** (int) - Status code
- **data** (array) - List of existing budget rules
```

--------------------------------

### Open the database in DuckDB

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=federation-kubecost-aggregator

Starts the DuckDB REPL interface using the copied database file.

```bash
duckdb kubecost-example.duckdb.read.kubecost.copy
```

--------------------------------

### Create a product key JSON file

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=configuration-adding-product-key

Define the product key in a JSON file to be used for creating a Kubernetes secret.

```json
{
  "key": "<YOUR_PRODUCT_KEY>"
}
```

--------------------------------

### Example Policy ARN Output

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=integration-aws-cloud-using-irsaeks-pod-identities

The ARN generated after creating the IAM policy.

```text
arn:aws:iam::SUB_ACCOUNT_222222222:policy/kubecost-access-cur-in-payer-account
```

--------------------------------

### Create cAdvisor and Node Exporter Scraper

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=guide-aws-agentless-amp

Initialize the AMP scraper for cAdvisor and node exporter.

```bash
CADVSIOR_SCRAPER_OUTPUT=$(aws amp create-scraper --output json \
  --alias cadvisor-scraper \
  --source eksConfiguration="{clusterArn=$CLUSTER_ARN, securityGroupIds=[$SECURITY_GROUP_IDS],subnetIds=[$SUBNET_IDS]}" \
  --scrape-configuration configurationBlob="$(base64 scraper-cadvisor-node-exporter.yaml|tr -d '\n')" \
  --destination ampConfiguration="{workspaceArn=$WORKSPACE_ARN}")
echo $CADVSIOR_SCRAPER_OUTPUT
CADVSIOR_SCRAPER_ID=$(echo $CADVSIOR_SCRAPER_OUTPUT|jq -r .scraperId)
echo $CADVSIOR_SCRAPER_ID
```

--------------------------------

### Partial API Response

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-spec-cost-prediction-api

A truncated example of the JSON response returned by the API.

```json
[
  {
    "namespace": "kubecost",
    "controllerKind": "deployment",
    "controllerName": "kubecost-cost-analyzer",
    "costBefore": {
      "totalMonthlyRate": 3.5397661399108418,
      "cpuMonthlyRate": 2.3273929838395513,
      "ramMonthlyRate": 1.2123731560712905,
      "gpuMonthlyRate": 0,
      "monthlyCPUCoreHours": 73,
```

--------------------------------

### Install Kubecost on Amazon EKS

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installations-amazon-eks-integration

Deploys Kubecost using Helm. Replace <$VERSION> with the desired chart version.

```bash
helm upgrade -i kubecost \
  oci://public.ecr.aws/kubecost/cost-analyzer \
  --version <$VERSION> \
  --namespace kubecost \
  --create-namespace \
  -f https://raw.githubusercontent.com/kubecost/cost-analyzer-helm-chart/<$VERSION>/cost-analyzer/values-eks-cost-monitoring.yaml
```

```bash
helm upgrade -i kubecost \
  oci://public.ecr.aws/kubecost/cost-analyzer \
  --version <$VERSION> \
  --namespace kubecost \
  --create-namespace \
  -f https://raw.githubusercontent.com/kubecost/cost-analyzer-helm-chart/<$VERSION>/cost-analyzer/values-eks-cost-monitoring.yaml \
```

--------------------------------

### GET /model/savings/turbonomic/resizeWorkloadControllers

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-turbonomic-actions

Returns a list of workloads for which request resizing has been recommended by Turbonomic.

```APIDOC
## GET /model/savings/turbonomic/resizeWorkloadControllers

### Description
The Resize Workload Controllers API returns workloads for which request resizing has been recommended by Turbonomic. The list of results returned should align with those in the Turbonomic Actions Center.

### Method
GET

### Endpoint
http://<kubecost-address>/model/savings/turbonomic/resizeWorkloadControllers

### Parameters
#### Query Parameters
- **filter** (string) - Optional - Filter your results by cluster, namespace and/or controller.

### Response
#### Success Response (200)
- **code** (integer) - Status code
- **data** (object) - Response data containing numResults, totalSavings, and actions list

#### Response Example
{
"code": 200,
"data": {
"numResults": 1,
"totalSavings": 2.00,
"actions": [
{
 "action": {
   "cluster": "standard-cluster-1",
   "namespace": "kubecost",
   "controller": "kubecost-cost-analyzer",
   "replicaCount": 1,
   "compoundActions": {
     "cost-model": [
       {
         "target": "VCPURequest",
         "unit": "mCores",
         "oldValue": 200,
         "newValue": 100
       }
     ]
   },
   "available": true,
   "targetId": "11111111111111"
 },
 "currentMonthlyRate": 4.00,
 "predictedMonthlyRate": 2.00,
 "predictedSavings": 2.00
}
]
}
}
```

--------------------------------

### GET /model/diagnostic/tableWindowCount

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=apis-aggregator-diagnostic

Determines the number of unique WindowStart/WindowEnd pairs in a specified table.

```APIDOC
## GET /model/diagnostic/tableWindowCount

### Description
Used to determine the number of unique WindowStart/WindowEnd pairs exist in the table.

### Parameters
#### Query Parameters
- **table** (string) - Required - The table containing container data to consider. Try `container_1d`.
```

--------------------------------

### Configure SAML and RBAC via Helm

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=configuration-user-management-saml

Example configuration for enabling SAML authentication and mapping identity provider groups to Kubecost roles.

```yaml
# EXAMPLE CONFIGURATION
# View setup guides below, for full list of Helm configuration values
saml:
  enabled: true
  secretName: "kubecost-okta"
  idpMetadataURL: "https://your.idp.subdomain.okta.com/app/exk4h09oysB785123/sso/saml/metadata"
  appRootURL: "https://kubecost.your.com"
  authTimeout: 1440
  audienceURI: "https://kubecost.your.com"
  nameIDFormat: "urn:oasis:names:tc:SAML:2.0:attrname-format:basic"
  rbac:
    enabled: true
    groups:
      - name: admin
        enabled: true
        assertionName: "kubecost_group"
        assertionValues:
          - "kubecost_admin"
          - "kubecost_superusers"
      - name: readonly
        enabled: true
        assertionName: "kubecost_group"
        assertionvalues:
          - "kubecost_users"
    customGroups:
      - assertionName: "kubecost_group"
```

--------------------------------

### Describe Kubecost add-on status

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=installations-amazon-eks-integration

Check the current status of the Kubecost add-on installation.

```bash
aws eks describe-addon \
  --addon-name kubecost_kubecost \
  --cluster-name $YOUR_CLUSTER_NAME \
  --region $AWS_REGION
```

--------------------------------

### Configure Postgres Integration in Helm values

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=configuration-kubecost-postgres-integration

Example YAML configuration for enabling and setting up Postgres integration within the Kubecost Helm values file.

```yaml
global:
  integrations:
    postgres:
      enabled: true
      runInterval: "12h" # How frequently to run the integration.
      databaseHost: "" # REQUIRED. ex: my.postgres.database.azure.com
      databasePort: "" # REQUIRED. ex: 5432
      databaseName: "" # REQUIRED. ex: postgres
      databaseUser: "" # REQUIRED. ex: myusername
      databasePassword: "" # REQUIRED. ex: mypassword
      databaseSecretName: "" # OPTIONAL. Specify your own k8s secret containing the above credentials. Must have key "creds.json".
      queryConfigs:
        allocations:
          - databaseTable: "kubecost_allocation_data"
            window: "7d"
            aggregate: "namespace"
            idle: "true"
            shareIdle: "true"
            shareNamespaces: "kubecost,kube-system"
            shareLabels: ""
          - databaseTable: "kubecost_allocation_data_by_cluster"
            window: "10d"
            aggregate: "cluster"
            idle: "true"
            shareIdle: "false"
            shareNamespaces: ""
            shareLabels: ""
        assets:
          - databaseTable: "kubecost_assets_data"
            window: "7d"
            aggregate: "cluster"
        cloudCosts:
          - databaseTable: "kubecost_cloudcosts_data"
            window: "7d"
            aggregate: "service"
# REQUIRED. Aggregator must be enabled and running as a statefulset.
kubecostAggregator:
  deployMethod: "statefulset"
kubecostModel:
  federatedStorageConfigSecret: "federated-store"
prometheus:
  server:
    global:
      external_labels:
        cluster_id: "primary-cluster-1"
# REQUIRED. A Kubecost Enterprise license key.
kubecostProductConfigs:
  productKey:
    enabled: true
    key: "my-enterprise-key-here"
```

--------------------------------

### Install DCGM Exporter via Helm

Source: https://www.ibm.com/docs/en/kubecost/self-hosted/2.x?topic=configuration-nvidia-gpu-monitoring-configurations

Deploy the DCGM Exporter chart using the previously defined values file.

```bash
helm upgrade \
  -i dcgm dcgm-exporter \
  --repo https://nvidia.github.io/dcgm-exporter/helm-charts \
  -n dcgm-exporter \
  --create-namespace \
  -f values-dcgm.yaml
```
Tokens:
Raw