Ingress NGINX

https://kubernetes.github.io/ingress-nginx/
Admin
Ingress NGINX is an Ingress controller for Kubernetes using NGINX as a reverse proxy and load...
Tokens:98,193
Snippets:797
Trust Score:9.7
Update:1 month ago
Show doc for...
Tokens:
Raw
### Install libcap and check capabilities

Source: https://kubernetes.github.io/ingress-nginx/troubleshooting

Install the libcap package and check the capabilities of the controller binary to verify it has the net_bind_service capability.

```bash
$ apk add libcap
(1/1) Installing libcap (2.50-r0)
Executing busybox-1.33.1-r7.trigger
OK: 26 MiB in 41 packages
$ getcap /nginx-ingress-controller
/nginx-ingress-controller cap_net_bind_service=ep

```

--------------------------------

### Install Kubernetes Dashboard

Source: https://kubernetes.github.io/ingress-nginx/examples/auth/oauth-external-auth

Command to install the Kubernetes Dashboard using kubectl. This is a prerequisite for both the OAuth2 Proxy and Vouch Proxy examples.

```bash
kubectl create -f https://raw.githubusercontent.com/kubernetes/kops/master/addons/kubernetes-dashboard/v1.10.1.yaml
```

--------------------------------

### Install ingress-nginx on OVHcloud using Helm

Source: https://kubernetes.github.io/ingress-nginx/deploy

Installs the ingress controller on OVHcloud using Helm. Adds the repository, updates it, and installs the chart in the ingress-nginx namespace.

```bash
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
helm -n ingress-nginx install ingress-nginx ingress-nginx/ingress-nginx --create-namespace

```

--------------------------------

### Create session affinity example Ingress

Source: https://kubernetes.github.io/ingress-nginx/examples/affinity/cookie

Use this command to create the session affinity example Ingress resource from the file 'ingress.yaml'. This is the only command provided in the documentation.

```bash
kubectl create -f ingress.yaml

```

--------------------------------

### Install ingress-nginx on Scaleway

Source: https://kubernetes.github.io/ingress-nginx/deploy

Installs the ingress controller on Scaleway. Refer to the Scaleway documentation for proxy protocol configuration.

```bash
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.15.1/deploy/static/provider/scw/deploy.yaml

```

--------------------------------

### Install ingress-nginx on Exoscale

Source: https://kubernetes.github.io/ingress-nginx/deploy

Installs the ingress controller on Exoscale. The full list of supported annotations is in the Exoscale Cloud Controller Manager documentation.

```bash
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/exoscale/deploy.yaml

```

--------------------------------

### Deployment resource (partial example) for ingress-nginx controller

Source: https://kubernetes.github.io/ingress-nginx/deploy/upgrade

Partial example of the controller Deployment resource showing the image field with a specific version tag and digest. Used as a reference for upgrading by changing the image tag.

```yaml
kind: Deployment
metadata:
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  replicas: 1
  selector: ...
  template:
    metadata: ...
    spec:
      containers:
        - name: ingress-nginx-controller
          image: registry.k8s.io/ingress-nginx/controller:v1.0.4@sha256:545cff00370f28363dad31e3b59a94ba377854d3a11f18988f5f9e56841ef9ef
          args: ...
```

--------------------------------

### Install ingress-nginx on Digital Ocean

Source: https://kubernetes.github.io/ingress-nginx/deploy

Installs the ingress controller on Digital Ocean. The default service only configures one annotation; additional annotations may be needed for load balancer graphs.

```bash
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.15.1/deploy/static/provider/do/deploy.yaml

```

--------------------------------

### Strace the controller to debug system calls

Source: https://kubernetes.github.io/ingress-nginx/troubleshooting

Install strace and trace the controller to see the system calls executed when it fails.

```bash
$ apk add strace
(1/1) Installing strace (5.12-r0)
Executing busybox-1.33.1-r7.trigger
OK: 28 MiB in 42 packages
$ strace /nginx-ingress-controller
execve("/nginx-ingress-controller", ["/nginx-ingress-controller"], 0x7ffeb9eb3240 /* 131 vars */) = 0
arch_prctl(ARCH_SET_FS, 0x29ea690)      = 0
...

```

--------------------------------

### Promote IP to Static with gcloud

Source: https://kubernetes.github.io/ingress-nginx/examples/static-ip

Promotes the IP to static using gcloud compute addresses create. The example is for GKE/GCE and shows the created address details.

```bash
$ gcloud compute addresses create ingress-nginx-lb --addresses 104.154.109.191 --region us-central1
Created [https://www.googleapis.com/compute/v1/projects/kubernetesdev/regions/us-central1/addresses/ingress-nginx-lb].
---
address: 104.154.109.191
creationTimestamp: '2017-01-31T16:34:50.089-08:00'
description: ''
id: '5208037144487826373'
kind: compute#address
name: ingress-nginx-lb
region: us-central1
selfLink: https://www.googleapis.com/compute/v1/projects/kubernetesdev/regions/us-central1/addresses/ingress-nginx-lb
status: IN_USE
users:
- us-central1/forwardingRules/a09f6913ae80e11e6a8c542010af0000

```

--------------------------------

### Check Ingress Resource Events with kubectl get and describe

Source: https://kubernetes.github.io/ingress-nginx/troubleshooting

Use kubectl get ing to list Ingress resources and kubectl describe ing to view detailed events, rules, and annotations. The output shows the cafe-ingress resource with rules for cafe.com and backend services tea-svc and coffee-svc.

```bash
$ kubectl get ing -n <namespace-of-ingress-resource>
NAME           HOSTS      ADDRESS     PORTS     AGE
cafe-ingress   cafe.com   10.0.2.15   80        25s

$ kubectl describe ing <ingress-resource-name> -n <namespace-of-ingress-resource>
Name:             cafe-ingress
Namespace:        default
Address:          10.0.2.15
Default backend:  default-http-backend:80 (172.17.0.5:8080)
Rules:
  Host      Path  Backends
  ----      ----  --------
  cafe.com
            /tea      tea-svc:80 (<none>)
            /coffee   coffee-svc:80 (<none>)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"networking.k8s.io/v1","kind":"Ingress","metadata":{"annotations":{},"name":"cafe-ingress","namespace":"default","selfLink":"/apis/networking/v1/namespaces/default/ingresses/cafe-ingress"},"spec":{"rules":[{"host":"cafe.com","http":{"paths":[{"backend":{"serviceName":"tea-svc","servicePort":80},"path":"/tea"},{"backend":{"serviceName":"coffee-svc","servicePort":80},"path":"/coffee"}]}}]},"status":{"loadBalancer":{"ingress":[{"ip":"169.48.142.110"}]}}}

Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  CREATE  1m    ingress-nginx-controller  Ingress default/cafe-ingress
  Normal  UPDATE  58s   ingress-nginx-controller  Ingress default/cafe-ingress
```

--------------------------------

### Create Service from manifest file

Source: https://kubernetes.github.io/ingress-nginx/examples/grpc

Use this command to create the Service resource from the saved manifest file 'service.go-grpc-greeter-server.yaml'.

```bash
$ kubectl create -f service.go-grpc-greeter-server.yaml

```

--------------------------------

### Get values of prometheus release

Source: https://kubernetes.github.io/ingress-nginx/user-guide/monitoring

Displays the values of the installed prometheus release to validate that Prometheus has been reconfigured. The output shows the podMonitorSelectorNilUsesHelmValues and serviceMonitorSelectorNilUsesHelmValues flags set to false.

```bash
helm get values prometheus --namespace prometheus

```

```yaml
prometheus:
  prometheusSpec:
    podMonitorSelectorNilUsesHelmValues: false
    serviceMonitorSelectorNilUsesHelmValues: false

```

--------------------------------

### FastCGI Service (example-service)

Source: https://kubernetes.github.io/ingress-nginx/user-guide/fcgi-services

Service that selects the example-app pod and exposes port 9000, matching the FastCGI port defined in the pod.

```yaml
apiVersion: v1
kind: Service
metadata:
  name: example-service
spec:
  selector:
    app: example-app
  ports:
  - port: 9000
    targetPort: 9000
    name: fastcgi

```

--------------------------------

### Get values of ingress-nginx release

Source: https://kubernetes.github.io/ingress-nginx/user-guide/monitoring

Displays the values of the installed ingress-nginx release to validate that the controller has been reconfigured to export metrics. The output shows the metrics and serviceMonitor settings.

```bash
helm get values ingress-nginx --namespace ingress-nginx

```

```yaml
controller:
  metrics:
    enabled: true
    serviceMonitor:
      additionalLabels:
        release: prometheus
      enabled: true

```

--------------------------------

### Build and deploy demo app

Source: https://kubernetes.github.io/ingress-nginx/user-guide/third-party-addons/opentelemetry

Commands to build images and deploy the demo application using make targets.

```bash
# build images
make images

# deploy demo app:
make deploy-app

```

--------------------------------

### List nodes with external IPs

Source: https://kubernetes.github.io/ingress-nginx/deploy/baremetal

Shows the output of `kubectl get node` for a 3-node cluster, where the external IP is added as an example (in most bare-metal environments this value is <None>).

```bash
$ kubectl get node
NAME     STATUS   ROLES    EXTERNAL-IP
host-1   Ready    master   203.0.113.1
host-2   Ready    node     203.0.113.2
host-3   Ready    node     203.0.113.3

```

--------------------------------

### Validate metrics configuration with helm get values

Source: https://kubernetes.github.io/ingress-nginx/user-guide/monitoring

Use this command to verify that the controller is configured for metrics. It displays the values of the installed release, and you should see the metrics and podAnnotations settings as shown in the expected output.

```bash
helm get values ingress-nginx --namespace ingress-nginx
```

--------------------------------

### Build and deploy the ingress controller with kind

Source: https://kubernetes.github.io/ingress-nginx/developer-guide/getting-started

Starts a local Kubernetes cluster using kind, builds the ingress controller, and deploys it. For v1.x.x versions, you may need to configure a custom kind node image for Kubernetes 1.22.

```bash
make dev-env
```

--------------------------------

### Deploy controller and services with kubectl

Source: https://kubernetes.github.io/ingress-nginx/examples/customization/external-auth-headers

Deploys the authentication service, echo service, and ingress objects. The output shows the created resources and the running pods and ingresses.

```bash
$ kubectl create -f deploy/
deployment "demo-auth-service" created
service "demo-auth-service" created
ingress "demo-auth-service" created
deployment "demo-echo-service" created
service "demo-echo-service" created
ingress "public-demo-echo-service" created
ingress "secure-demo-echo-service" created

$ kubectl get po
NAME                                        READY     STATUS    RESTARTS   AGE
demo-auth-service-2769076528-7g9mh          1/1       Running            0          30s
demo-echo-service-3636052215-3vw8c          1/1       Running            0          29s

kubectl get ing
NAME                       HOSTS                                 ADDRESS   PORTS     AGE
public-demo-echo-service   public-demo-echo-service.kube.local             80        1m
secure-demo-echo-service   secure-demo-echo-service.kube.local             80        1m
```

--------------------------------

### Create main deployment and service

Source: https://kubernetes.github.io/ingress-nginx/examples/canary

This command creates the main deployment and service for the application. It applies a Kubernetes manifest that defines a Deployment named 'production' and a Service named 'production'. The deployment uses the image 'registry.k8s.io/ingress-nginx/e2e-test-echo:v1.2.9@sha256:9920d084b452b38ee663005a455aa7ed12c15afa512741ea9596e206a189bdf0' and sets environment variables for node name, pod name, namespace, and IP. The service exposes port 80 and selects pods with label 'app: production'.

```bash
echo "
---
# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: production
  labels:
    app: production
spec:
  replicas: 1
  selector:
    matchLabels:
      app: production
  template:
    metadata:
      labels:
        app: production
    spec:
      containers:
      - name: production
        image: registry.k8s.io/ingress-nginx/e2e-test-echo:v1.2.9@sha256:9920d084b452b38ee663005a455aa7ed12c15afa512741ea9596e206a189bdf0
        ports:
        - containerPort: 80
        env:
          - name: NODE_NAME
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: POD_IP
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
---
# Service
apiVersion: v1
kind: Service
metadata:
  name: production
  labels:
    app: production
spec:
  ports:
  - port: 80
    targetPort: 80
    protocol: TCP
    name: http
  selector:
    app: production
" | kubectl apply -f -
```

--------------------------------

### Install ingress-nginx on GCE-GKE

Source: https://kubernetes.github.io/ingress-nginx/deploy

Installs the ingress controller on GCE-GKE. Note that the default GKE LoadBalancer does not support the PROXY protocol.

```bash
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.15.1/deploy/static/provider/cloud/deploy.yaml

```

--------------------------------

### index.php test script for FastCGI

Source: https://kubernetes.github.io/ingress-nginx/user-guide/fcgi-services

A simple PHP script to be placed at /var/www/html/index.php inside the example-app pod. Use kubectl exec to create this file; it is not persistent if the pod is deleted.

```html
<!DOCTYPE html>
<html>
    <head>
        <title>PHP Test</title>
    </head>
    <body>
        <?php echo '<p>FastCGI Test Worked!</p>'; ?>
    </body>
</html>

```

--------------------------------

### Server Snippet Annotation Example

Source: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations

Example of using the nginx.ingress.kubernetes.io/server-snippet annotation to add custom configuration in the server block. This annotation can only be used once per host.

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/server-snippet: |
      set $agentflag 0;

      if ($http_user_agent ~* "(Mobile)" ){
        set $agentflag 1;
      }

      if ( $agentflag = 1 ) {
        return 301 https://m.example.com;
      }

```

--------------------------------

### Install ingress-nginx on bare metal clusters

Source: https://kubernetes.github.io/ingress-nginx/deploy

Installs the ingress controller on bare metal clusters using NodePort. Typically uses a port in the 30000-32767 range.

```bash
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.15.1/deploy/static/provider/baremetal/deploy.yaml

```

--------------------------------

### Get cluster nodes to show external IPs

Source: https://kubernetes.github.io/ingress-nginx/deploy/baremetal

Shows the output of 'kubectl get node' for a 3-node cluster. In bare-metal environments, external IPs are often <None>.

```text
$ kubectl get node
NAME     STATUS   ROLES    EXTERNAL-IP
host-1   Ready    master   203.0.113.1
host-2   Ready    node     203.0.113.2
host-3   Ready    node     203.0.113.3


```

--------------------------------

### Deploy otel-collector, grafana and Jaeger backend

Source: https://kubernetes.github.io/ingress-nginx/user-guide/third-party-addons/opentelemetry

Commands to add Helm charts, install cert-manager, create observability namespace, deploy OpenTelemetry collector operator and collector, Jaeger, Zipkin, Tempo, and Grafana.

```bash
# add helm charts needed for grafana and OpenTelemetry collector
helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
# deploy cert-manager needed for OpenTelemetry collector operator
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.15.3/cert-manager.yaml
# create observability namespace
kubectl apply -f https://raw.githubusercontent.com/esigo/nginx-example/main/observability/namespace.yaml
# install OpenTelemetry collector operator
helm upgrade --install otel-collector-operator -n otel --create-namespace open-telemetry/opentelemetry-operator
# deploy OpenTelemetry collector
kubectl apply -f https://raw.githubusercontent.com/esigo/nginx-example/main/observability/collector.yaml
# deploy Jaeger all-in-one
kubectl apply -f https://github.com/jaegertracing/jaeger-operator/releases/download/v1.37.0/jaeger-operator.yaml -n observability
kubectl apply -f https://raw.githubusercontent.com/esigo/nginx-example/main/observability/jaeger.yaml -n observability
# deploy zipkin
kubectl apply -f https://raw.githubusercontent.com/esigo/nginx-example/main/observability/zipkin.yaml -n observability
# deploy tempo and grafana
helm upgrade --install tempo grafana/tempo --create-namespace -n observability
helm upgrade -f https://raw.githubusercontent.com/esigo/nginx-example/main/observability/grafana/grafana-values.yaml --install grafana grafana/grafana --create-namespace -n observability

```

--------------------------------

### Check Prometheus installation with helm ls

Source: https://kubernetes.github.io/ingress-nginx/user-guide/monitoring

Lists all Helm releases across namespaces to verify Prometheus is installed. The output shows the prometheus release in the prometheus namespace with status deployed.

```bash
helm ls -A

```

```text
NAME          NAMESPACE       REVISION    UPDATED                                 STATUS      CHART                           APP VERSION

```

```text
ingress-nginx ingress-nginx   10          2022-01-20 18:08:55.267373 -0800 PST    deployed    ingress-nginx-4.0.16            1.1.1

```

```text
prometheus    prometheus      1          2022-01-20 16:07:25.086828 -0800 PST    deployed    kube-prometheus-stack-30.1.0    0.53.1

```

--------------------------------

### Get ingress-nginx NodePort Service

Source: https://kubernetes.github.io/ingress-nginx/deploy/baremetal

Shows the output of 'kubectl -n ingress-nginx get svc' displaying the NodePort service with ports 80 and 443 mapped to NodePorts 30100 and 30101.

```text
$ kubectl -n ingress-nginx get svc
NAME                   TYPE        CLUSTER-IP     PORT(S)
ingress-nginx          NodePort    10.0.220.217   80:30100/TCP,443:30101/TCP


```

--------------------------------

### Set GO111MODULE and download dependencies

Source: https://kubernetes.github.io/ingress-nginx/developer-guide/getting-started

Sets the GO111MODULE environment variable to 'on' and downloads Go module dependencies. Required before building the project.

```bash
export GO111MODULE=on
```

```bash
go mod download
```

--------------------------------

### FastCGI ConfigMap (example-cm)

Source: https://kubernetes.github.io/ingress-nginx/user-guide/fcgi-services

ConfigMap required to set FastCGI directive parameters. Must be created before the Ingress object; otherwise the Ingress Controller pods need to be restarted.

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: example-cm
data:
  SCRIPT_FILENAME: "/var/www/html/index.php"

```
Tokens:
Raw