#Tasks for Envoy Gateway

#Overview

This document extends the main configuration path (operator → gateway → route → policy) and introduces advanced tasks beyond the standard resource configurations covered in the previous documents.

When applying configuration changes in the Gateway API, there are three primary approaches available:

1. Modify standard Gateway API resources: Directly edit Gateway, HTTPRoute, TCPRoute, UDPRoute, and other core resources.
2. Attach policies via PolicyAttachment: Use SecurityPolicy, ClientTrafficPolicy, BackendTrafficPolicy, and other policy resources to extend Gateway and Route behavior.
3. Configure global settings: Modify the EnvoyGatewayCtl to change envoy-gateway instance behavior or other global settings that affect all gateways.

This document focuses on advanced tasks and special scenarios that are not part of the main configuration path, including cross-namespace routing, observability setup, deployment customization, and troubleshooting.

#Prerequisites

1. Configure EnvoyGatewayCtl
2. Configure Gateway
3. Configure Route
4. Configure GatewayAPI Policy

#Advanced Tasks

#OpenTelemetry (OTel)

Please follow instructions in OpenTelemetry Integration, but use EnvoyGatewayCtl to modify the envoy-gateway-config.

#How To Attach to Listener Created In Other Namespace

In the Gateway's listener configuration, you need to specify which namespaces are allowed to attach Routes to it.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: example-gateway
spec:
  listeners:
    - name: http-80
      protocol: HTTP
      port: 80
      allowedRoutes:
        namespaces:
          from: All # without limit
    - name: http-81
      protocol: HTTP
      port: 81
      allowedRoutes:
        namespaces:
          from: Same # only allow routes in the same namespace
    - name: http-82
      protocol: HTTP
      port: 82
      allowedRoutes:
        namespaces:
          from: Selector
          selector:
            matchLabels:
              team: frontend # only allow routes in the namespace with label team=frontend

Please refer to Cross-Namespace routing for more details.

#How To Use a Certificate Created In Another Namespace

To use a certificate created in another namespace, create a ReferenceGrant in the namespace where the certificate is stored. Please follow instructions in cross-namespace-certificate-references and referencegrant.

#How To Use SSL Passthrough

Please follow instructions in

• tls
• tls-passthrough

#How To Change the Minimum TLS Version

Please follow instructions in customize-gateway-tls-parameters

cat <<EOF | kubectl apply -f -
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: ClientTrafficPolicy
metadata:
  name: enforce-tls-13
  namespace: default
spec:
  targetRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: eg
  tls:
    minVersion: "1.3"
EOF

The .spec.tls field in ClientTrafficPolicy is clienttlssettings. If you need to customize cipher suites in addition to the minimum TLS version, refer to the same upstream task and API reference.

#How To Specify NodePort When Using NodePort Service

When using a NodePort service, Kubernetes assigns a NodePort value to each service port. When accessing the service through a node IP, use the assigned NodePort instead of the service port.

There are two approaches:

Manually retrieve the NodePort assignment by following get nodeport from svc port.

Manually specify the NodePort in the EnvoyProxy configuration instead of letting Kubernetes automatically assign it.

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: demo
spec:
  ipFamily: DualStack
  provider:
    kubernetes:
      envoyDeployment:
        container:
          imageRepository: registry.alauda.cn:60080/acp/envoyproxy/envoy
      envoyService:
        patch:
          type: StrategicMerge
          value:
            spec:
              ports:
                - nodeport: 31888
                  port: 80
        type: NodePort
    type: Kubernetes

#How To Add Pod Annotations In Envoy Gateway

Add pod annotation

#How To Set NodeSelector And Tolerations For envoy-gateway-operator

Update the Subscription resource.

# example of nodeSelector and tolerations
kubectl patch subscription envoy-gateway-operator  -n envoy-gateway-operator   --type='merge' -p '
{
  "spec": {
    "config": {
      "nodeSelector": {
        "node-role.kubernetes.io/infra": ""
      },
      "tolerations": [
        {
          "effect": "NoSchedule",
          "key": "node-role.kubernetes.io/infra",
          "operator": "Equal",
          "value": "reserved"
        }
      ]
    }
  }
}'

#How To Set NodeSelector And Tolerations For envoy-gateway

Update the EnvoyGatewayCtl resource.

# in default $NAME=cpaas-default and $NS=envoy-gateway-operator
kubectl patch envoygatewayctl $NAME -n $NS --type='merge' -p '
{
  "spec": {
    "deployment": {
      "pod": {
        "nodeSelector": {
          "node-role.kubernetes.io/infra": ""
        },
        "tolerations": [
          {
            "effect": "NoSchedule",
            "key": "node-role.kubernetes.io/infra",
            "operator": "Equal",
            "value": "reserved"
          }
        ]
      }
    }
  }
}'

#How To Set NodeSelector And Tolerations For envoy-proxy

Update the EnvoyProxy resource.

kubectl patch envoyproxy $NAME -n $NS --type='merge' -p '
{
  "spec": {
    "provider": {
      "kubernetes": {
        "envoyDeployment": {
          "pod": {
            "nodeSelector": {
              "node-role.kubernetes.io/infra": ""
            },
            "tolerations": [
              {
                "effect": "NoSchedule",
                "key": "node-role.kubernetes.io/infra",
                "operator": "Equal",
                "value": "reserved"
              }
            ]
          }
        }
      }
    }
  }
}'

#How To Use hostNetwork In envoy-proxy

Using hostNetwork: true allows the Envoy proxy pods to use the host network namespace directly. This can be useful for:

• achieving better network performance
• accessing the gateway through the node IP directly

Considerations:

• Pods using hostNetwork will bind directly to the host's network interfaces
• Port conflicts may occur if multiple pods try to use the same port on the same node
• Security isolation is reduced as pods share the host's network namespace
• You should use nodeSelector or affinity rules to control pod placement and avoid port conflicts

There are two approaches to configure hostNetwork, depending on whether you want to use privileged ports (< 1024) directly:

#Approach 1: Using Port Offset (Default, Recommended)

This is the default and recommended approach. Envoy Gateway automatically adds an offset of 10000 to privileged ports to avoid requiring special permissions.

Pros:

• No special permissions or capabilities required
• More secure because it runs as a non-root user without additional privileges
• Simpler configuration
• Works out of the box

Cons:

• Clients must use offset ports (10080, 10443)

Configuration:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: demo
  namespace: demo
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyDeployment:
        patch:
          type: StrategicMerge
          value:
            spec:
              template:
                spec:
                  hostNetwork: true                    # Enable host network mode
                  dnsPolicy: ClusterFirstWithHostNet   # Required for proper DNS resolution
        pod:
          nodeSelector:                                # Recommended: control pod placement to avoid conflict
            kubernetes.io/hostname: "demo"

Access:

• Port 80 → Access via http://<node-ip>:10080
• Port 443 → Access via https://<node-ip>:10443

#Approach 2: Using Privileged Ports Directly with useListenerPortAsContainerPort

This approach allows Envoy to bind to privileged ports (< 1024) directly, such as port 80 and 443.

Pros:

• Can use standard ports (80 and 443) directly
• Better compatibility with clients expecting standard ports

Cons:

• Requires NET_BIND_SERVICE capability
• Slightly reduced security compared to port offset approach
• More complex configuration

Configuration:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: demo
  namespace: demo
spec:
  provider:
    type: Kubernetes
    kubernetes:
      useListenerPortAsContainerPort: true  # Disable port offset
      envoyDeployment:
        patch:
          type: StrategicMerge
          value:
            spec:
              template:
                spec:
                  hostNetwork: true
                  dnsPolicy: ClusterFirstWithHostNet
                  containers:
                  - name: envoy
                    command:
                      - /usr/local/bin/envoy-with-cap  # use envoy with filecap
                    securityContext:
                      capabilities:
                        add:
                        - NET_BIND_SERVICE  # Required for binding to privileged ports
        pod:
          nodeSelector:                                # Recommended: control pod placement to avoid conflict
            kubernetes.io/hostname: "demo"

Access:

• Port 80 → Access via http://<node-ip>:80
• Port 443 → Access via https://<node-ip>:443

#How To Access a LoadBalancer VIP from Within the Cluster

By default, Envoy Gateway creates LoadBalancer services with externalTrafficPolicy: Local. This policy preserves client source IP addresses but has an important limitation: requests from cluster nodes without Envoy Gateway pods will fail because traffic is not forwarded to other nodes.

Solution 1: Use Service ClusterIP (Recommended for in-cluster access)

For applications running inside the cluster, use the service ClusterIP instead of the LoadBalancer VIP. This avoids the routing limitation entirely.

Solution 2: Change to Cluster Traffic Policy

If you need to access the LoadBalancer VIP from any cluster node, change externalTrafficPolicy to Cluster:

kubectl patch envoyproxy $GATEWAY_NAME -n $GATEWAY_NS --type='json' -p='[  
  {"op": "replace", "path": "/spec/provider/kubernetes/envoyService/externalTrafficPolicy", "value": "Cluster"}  
]'

#Related Documentation

• Configure EnvoyGatewayCtl
• Configure Gateway
• Configure Route
• Configure GatewayAPI Policy

#More Configuration

Please refer to EnvoyGateway Tasks