Gateway API
Gateway API는 K8s 네트워크 인프라에서 외부 및 내부의 트래픽을 관리하고 제어하기 위한 새로운 표준으로 작성되었습니다. 기존의 Ingress API를 확장, 개선한 형태로 다양한 시나리오를 지원하며 TCP, UDP등의 다양한 프로토콜 또한 지원합니다.
주요 개념
- Gateway 트래픽을 받아들이는 리소스로 물리/논리 네트워크 장치에 대한 추상화를 제공하고, 여러 리스닝 포트의 정의를 통해 다양한 서비스로 트래픽 전달이 가능합니다.
- GatewayClass Gateway의 동작을 정의하는 클래스로 네트워크 또는 클러스터 관리자는 여러개의 클래스를 정의하여 다양한 유형의 트래픽 처리 방식을 설정할수 있습니다.
- HTTPRoute, TCPRoute, UDPRoute 트래픽을 특정 서비스로 라우팅하는 규칙을 정의하며, 각각의 프로토콜에 맞추어서 정의할수 있습니다.
- Listener HTTP, HTTPS, TCP, UDP 등의 프로토콜을 수신할수 있도록 정의하는 구성요소입니다.
주요 기능
- 개선된 리소스 모델 새로운 사용자 정의 리소스를 도입하여 라우팅 규칙을 정의하는 방식에 대해 보다 세부적이고 표현력있는 방법을 제공합니다.
- 프로토콜 독립적 TCP, UDP, TLS을 포함한 여러 프로토콜들을 지원합니다.
- 강화된 보안 TLS 구성 및 더 세부적인 액세스 제어 제공합니다.
- 네임스페이스 교차 지원 서로 다른 네임스페이스의 서비스로의 트래픽을 라우팅합니다.
- 확장성 사용자 정의 리소스와 정책으로 쉽게 확장 가능합니다.
- 역할 지향 운영자, 개발자, 보안팀간의 역할에 따른 권한을 부여할수 있습니다.
흐름도
Client => Gateway - HTTPRoute - Service => POD
실습(Gloo Gateway)
KinD 구성
cat <<EOT> kind-1node.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 30000
hostPort: 30000
- containerPort: 30001
hostPort: 30001
- containerPort: 30002
hostPort: 30002
EOT
Install KinD Cluster
kind create cluster --image kindest/node:v1.30.0 --config kind-1node.yaml --name myk8s
노드에 기본 툴 설치
docker exec -it myk8s-control-plane sh -c 'apt update && apt install tree psmisc lsof wget bsdmainutils bridge-utils net-tools dnsutils tcpdump ngrep iputils-ping git vim -y'
노드/파드 확인
k get nodes -o wide
k get pod -A
---
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
myk8s-control-plane NotReady control-plane 21s v1.30.0 172.18.0.2 <none> Debian GNU/Linux 12 (bookworm) 6.10.4-linuxkit containerd://1.7.15
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-7db6d8ff4d-fqbxt 0/1 Pending 0 3s
kube-system coredns-7db6d8ff4d-zpsgs 0/1 Pending 0 3s
kube-system etcd-myk8s-control-plane 1/1 Running 0 19s
kube-system kindnet-kmcmk 1/1 Running 0 3s
kube-system kube-apiserver-myk8s-control-plane 1/1 Running 0 21s
kube-system kube-controller-manager-myk8s-control-plane 1/1 Running 0 19s
kube-system kube-proxy-f65qg 1/1 Running 0 3s
kube-system kube-scheduler-myk8s-control-plane 1/1 Running 0 19s
local-path-storage local-path-provisioner-988d74bc-hlbkb 0/1 Pending 0 3s
---
GatewayAPI CRDs설치
k apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.0.0/standard-install.yaml
k get crds
---
NAME CREATED AT
gatewayclasses.gateway.networking.k8s.io 2024-10-12T13:44:52Z
gateways.gateway.networking.k8s.io 2024-10-12T13:44:52Z
httproutes.gateway.networking.k8s.io 2024-10-12T13:44:52Z
referencegrants.gateway.networking.k8s.io 2024-10-12T13:44:52Z
---
Cluster 접속 및 Glooctl 설치
클러스터 접속
docker exec -it myk8s-control-plane bash
설치
curl -sL https://run.solo.io/gloo/install | GLOO_VERSION=v1.17.7 sh
export PATH=$HOME/.gloo/bin:$PATH
설치 확인
glooctl version
---
Server: version undefined, could not find any version of gloo running
{
"client": {
"version": "1.17.7"
},
"kubernetesCluster": {
"major": "1",
"minor": "30",
"gitVersion": "v1.30.0",
"buildDate": "2024-05-13T22:00:36Z",
"platform": "linux/amd64"
}
---
Gloo Gateway 설치
모니터링
watch -d kubectl get pod,svc,endpointslices,ep -n gloo-system
Helm을 통한 설치
helm repo add gloo https://storage.googleapis.com/solo-public-helm
helm repo update
helm install -n gloo-system gloo-gateway gloo/gloo \
--create-namespace \
--version 1.17.7 \
--set kubeGateway.enabled=true \
--set gloo.disableLeaderElection=true \
--set discovery.enabled=false
배포 확인
k get crd | grep 'networking.k8s.io'
---
gatewayclasses.gateway.networking.k8s.io 2024-10-12T13:44:52Z
gateways.gateway.networking.k8s.io 2024-10-12T13:44:52Z
httproutes.gateway.networking.k8s.io 2024-10-12T13:44:52Z
referencegrants.gateway.networking.k8s.io 2024-10-12T13:44:52Z
---
k get crd | grep -v 'networking.k8s.io'
---
NAME CREATED AT
authconfigs.enterprise.gloo.solo.io 2024-10-12T13:49:46Z
gatewayparameters.gateway.gloo.solo.io 2024-10-12T13:49:47Z
gateways.gateway.solo.io 2024-10-12T13:49:47Z
graphqlapis.graphql.gloo.solo.io 2024-10-12T13:49:48Z
httpgateways.gateway.solo.io 2024-10-12T13:49:47Z
httplisteneroptions.gateway.solo.io 2024-10-12T13:49:47Z
listeneroptions.gateway.solo.io 2024-10-12T13:49:47Z
proxies.gloo.solo.io 2024-10-12T13:49:47Z
ratelimitconfigs.ratelimit.solo.io 2024-10-12T13:49:48Z
routeoptions.gateway.solo.io 2024-10-12T13:49:47Z
routetables.gateway.solo.io 2024-10-12T13:49:47Z
settings.gloo.solo.io 2024-10-12T13:49:47Z
tcpgateways.gateway.solo.io 2024-10-12T13:49:47Z
upstreamgroups.gloo.solo.io 2024-10-12T13:49:48Z
upstreams.gloo.solo.io 2024-10-12T13:49:47Z
virtualhostoptions.gateway.solo.io 2024-10-12T13:49:47Z
virtualservices.gateway.solo.io 2024-10-12T13:49:47Z
---
k get pod,svc,endpointslices -n gloo-system
---
NAME READY STATUS RESTARTS AGE
pod/gateway-proxy-57c49d4f48-np5pq 1/1 Running 0 79s
pod/gloo-748d877c4-h8rq9 1/1 Running 0 79s
pod/gloo-resource-rollout-check-7fwrs 0/1 Completed 0 79s
pod/gloo-resource-rollout-pw9sx 0/1 Completed 0 79s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/gateway-proxy LoadBalancer 10.96.100.162 <pending> 80:30978/TCP,443:32508/TCP 79s
service/gloo ClusterIP 10.96.106.50 <none> 9977/TCP,9976/TCP,9988/TCP,9966/TCP,9979/TCP,443/TCP 79s
NAME ADDRESSTYPE PORTS ENDPOINTS AGE
endpointslice.discovery.k8s.io/gateway-proxy-zmgjz IPv4 8080,8443 10.244.0.8 79s
endpointslice.discovery.k8s.io/gloo-zmqzc IPv4 9979,9988,9966 + 3 more... 10.244.0.6 79s
---
통신 테스트
모니터링
watch -d kubectl get pod,svc,endpointslices,ep -n httpbin
httpbin application 배포
k apply -f https://raw.githubusercontent.com/solo-io/solo-blog/main/gateway-api-tutorial/01-httpbin-svc.yaml
배포 확인
k get deploy,pod,svc,endpointslices,sa -n httpbin
---
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/httpbin 0/1 1 0 10s
NAME READY STATUS RESTARTS AGE
pod/httpbin-5855dc8bdd-kl8g9 0/1 ContainerCreating 0 10s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/httpbin ClusterIP 10.96.122.251 <none> 8000/TCP 10s
NAME ADDRESSTYPE PORTS ENDPOINTS AGE
endpointslice.discovery.k8s.io/httpbin-bk848 IPv4 <unset> <unset> 10s
NAME SECRETS AGE
serviceaccount/default 0 10s
serviceaccount/httpbin 0 10s
---
NodePort 설정
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
labels:
app: httpbin
service: httpbin
name: httpbin
namespace: httpbin
spec:
type: NodePort
ports:
- name: http
port: 8000
targetPort: 80
nodePort: 30000
selector:
app: httpbin
EOF
접속확인
http://localhost:30000"
Gateway Listener 설정
배포
k apply -f https://raw.githubusercontent.com/solo-io/gloo-gateway-use-cases/main/gateway-api-tutorial/02-gateway.yaml
확인
k get gateway -n gloo-system
k get gateway -n gloo-system -o yaml
---
apiVersion: v1
items:
- apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"gateway.networking.k8s.io/v1","kind":"Gateway","metadata":{"annotations":{},"name":"http","namespace":"gloo-system"},"spec":{"gatewayClassName":"gloo-gateway","listeners":[{"allowedRoutes":{"namespaces":{"from":"All"}},"name":"http","port":8080,"protocol":"HTTP"}]}}
creationTimestamp: "2024-10-12T13:56:54Z"
generation: 1
name: http
namespace: gloo-system
resourceVersion: "1785"
uid: 921da8d7-5b96-4d69-80f9-7dd3ecd6f8ad
spec:
gatewayClassName: gloo-gateway
listeners:
- allowedRoutes:
namespaces:
from: All
name: http
port: 8080
protocol: HTTP
---
Proxy 인스턴스 확인
k get deployment gloo-proxy-http -n gloo-system
---
NAME READY UP-TO-DATE AVAILABLE AGE
gloo-proxy-http 1/1 1 1 118s
---
Envoy 사용 확인
k describe pod -n gloo-system |grep Image:
---
Image: quay.io/solo-io/gloo-envoy-wrapper:1.17.7
Image: quay.io/solo-io/gloo:1.17.7
Image: quay.io/solo-io/gloo-envoy-wrapper:1.17.7
---
서비스 상태 확인
k get svc -n gloo-system gloo-proxy-http
---
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
gloo-proxy-http LoadBalancer 10.96.253.152 <pending> 8080:30342/TCP 2m55s
---
노드포트를 통한 접속 설정
cat <<EOF | k apply -f -
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/instance: http
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: gloo-proxy-http
app.kubernetes.io/version: 1.17.7
gateway.networking.k8s.io/gateway-name: http
gloo: kube-gateway
helm.sh/chart: gloo-gateway-1.17.7
name: gloo-proxy-http
namespace: gloo-system
spec:
ports:
- name: http
nodePort: 30001
port: 8080
selector:
app.kubernetes.io/instance: http
app.kubernetes.io/name: gloo-proxy-http
gateway.networking.k8s.io/gateway-name: http
type: LoadBalancer
EOF
서비스 확인
k get svc -n gloo-system gloo-proxy-http
---
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
gloo-proxy-http LoadBalancer 10.96.253.152 <pending> 8080:30001/TCP 4m3s
---
HTTPRoute를 사용한 간단한 라우팅
설정
cat << EOF | k apply -f -
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: httpbin
namespace: httpbin
labels:
example: httpbin-route
spec:
parentRefs:
- name: http
namespace: gloo-system
hostnames:
- "api.example.com"
rules:
- matches:
- path:
type: Exact
value: /get
backendRefs:
- name: httpbin
port: 8000
EOF
설정 확인
k get httproute -n httpbin
---
NAME HOSTNAMES AGE
httpbin ["api.example.com"] 21s
---
테스트
k port-forward deployment/gloo-proxy-http -n gloo-system 8080:8080 &
curl -is -H "Host: api.example.com" http://localhost:8080/get
---
HTTP/1.1 200 OK
server: envoy
date: Sat, 12 Oct 2024 14:05:49 GMT
content-type: application/json
content-length: 239
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 2
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "api.example.com",
"User-Agent": "curl/8.7.1",
"X-Envoy-Expected-Rq-Timeout-Ms": "15000"
},
"origin": "10.244.0.11",
"url": "http://api.example.com/get"
}
---
다른 테스트
curl -is -H "Host: api.example.com" http://localhost:8080/delay/1
---
HTTP/1.1 404 Not Found
date: Sat, 12 Oct 2024 14:07:52 GMT
server: envoy
content-length: 0
---
오류나는 이유는 route에 대해서 /get 만등록했기 때문이다.
정규식 패턴 매칭
몇가지 설정 추가
Here are the modifications we’ll apply to our HTTPRoute:
- matches:
# Switch from an Exact Matcher(정확한 매팅) to a PathPrefix (경로 매팅) Matcher
- path:
type: PathPrefix
value: /api/httpbin/
filters:
# Replace(변경) the /api/httpbin matched prefix with /
- type: URLRewrite
urlRewrite:
path:
type: ReplacePrefixMatch
replacePrefixMatch: /
적용
k apply -f https://raw.githubusercontent.com/solo-io/gloo-gateway-use-cases/main/gateway-api-tutorial/04-httpbin-rewrite.yaml
정규식 패턴 추가 확인
k describe httproute -n httpbin
---
Filters:
Type: URLRewrite
URL Rewrite:
Path:
Replace Prefix Match: /
Type: ReplacePrefixMatch
Matches:
Path:
Type: PathPrefix
Value: /api/httpbin/
---
테스트
curl -is -H "Host: api.example.com" http://localhost:8080/api/httpbin/get
---
HTTP/1.1 200 OK
server: envoy
date: Sat, 12 Oct 2024 14:10:58 GMT
content-type: application/json
content-length: 289
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 1
{
"args": {},
"headers": {
"Accept": "*/*",
"Host": "api.example.com",
"User-Agent": "curl/8.7.1",
"X-Envoy-Expected-Rq-Timeout-Ms": "15000",
"X-Envoy-Original-Path": "/api/httpbin/get"
},
"origin": "10.244.0.11",
"url": "http://api.example.com/get"
}
---
curl -is -H "Host: api.example.com" http://localhost:8080/api/httpbin/delay/1 # 응답 시간 조정 1초 지연
---
HTTP/1.1 200 OK
server: envoy
date: Sat, 12 Oct 2024 14:11:21 GMT
content-type: application/json
content-length: 343
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 1002
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Host": "api.example.com",
"User-Agent": "curl/8.7.1",
"X-Envoy-Expected-Rq-Timeout-Ms": "15000",
"X-Envoy-Original-Path": "/api/httpbin/delay/1"
},
"origin": "10.244.0.11",
"url": "http://api.example.com/delay/1"
}
---
Upstream Bearer Tokens를 활용한 변환
설정 변경
filters:
- type: URLRewrite
urlRewrite:
path:
type: ReplacePrefixMatch
replacePrefixMatch: /
# Add a Bearer token to supply a static API key when routing to backend system
- type: RequestHeaderModifier
requestHeaderModifier:
add:
- name: Authorization
value: Bearer my-api-key
설정 적용
k apply -f https://raw.githubusercontent.com/solo-io/gloo-gateway-use-cases/main/gateway-api-tutorial/05-httpbin-rewrite-xform.yaml
설정 확인
k describe httproute -n httpbin
---
Backend Refs:
Group:
Kind: Service
Name: httpbin
Port: 8000
Weight: 1
Filters:
Type: URLRewrite
URL Rewrite:
Path:
Replace Prefix Match: /
Type: ReplacePrefixMatch
Request Header Modifier:
Add:
Name: Authorization
Value: Bearer my-api-key
Type: RequestHeaderModifier
Matches:
Path:
Type: PathPrefix
Value: /api/httpbin/
---
테스트
curl -is -H "Host: api.example.com" http://localhost:8080/api/httpbin/get
---
HTTP/1.1 200 OK
server: envoy
date: Sat, 12 Oct 2024 14:13:33 GMT
content-type: application/json
content-length: 332
access-control-allow-origin: *
access-control-allow-credentials: true
x-envoy-upstream-service-time: 1
{
"args": {},
"headers": {
"Accept": "*/*",
"Authorization": "Bearer my-api-key",
"Host": "api.example.com",
"User-Agent": "curl/8.7.1",
"X-Envoy-Expected-Rq-Timeout-Ms": "15000",
"X-Envoy-Original-Path": "/api/httpbin/get"
},
"origin": "10.244.0.11",
"url": "http://api.example.com/get"
}
---
Migrate
배포
k apply -f https://raw.githubusercontent.com/solo-io/gloo-gateway-use-cases/main/gateway-api-tutorial/06-workload-svcs.yaml
배포 확인
k get deploy,pod,svc,endpointslices -n my-workload
---
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/my-workload-v1 1/1 1 1 37s
deployment.apps/my-workload-v2 1/1 1 1 37s
NAME READY STATUS RESTARTS AGE
pod/my-workload-v1-644f98bbd9-hjcmk 1/1 Running 0 37s
pod/my-workload-v2-5bb5fcfcbc-vkvr9 1/1 Running 0 37s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/my-workload-v1 ClusterIP 10.96.187.21 <none> 8080/TCP 37s
service/my-workload-v2 ClusterIP 10.96.253.125 <none> 8080/TCP 37s
NAME ADDRESSTYPE PORTS ENDPOINTS AGE
endpointslice.discovery.k8s.io/my-workload-v1-pmkpg IPv4 8080 10.244.0.12 37s
endpointslice.discovery.k8s.io/my-workload-v2-5xnl9 IPv4 8080 10.244.0.13 37s
---
라우팅 배포
k apply -f https://raw.githubusercontent.com/solo-io/gloo-gateway-use-cases/main/gateway-api-tutorial/07-workload-route.yaml
확인
k get httproute -A
---
NAMESPACE NAME HOSTNAMES AGE
httpbin httpbin ["api.example.com"] 11m
my-workload my-workload ["api.example.com"] 5s
---
my-workload-v1 라우팅 확인
k describe httproute -n my-workload
---
Hostnames:
api.example.com
Parent Refs:
Group: gateway.networking.k8s.io
Kind: Gateway
Name: http
Namespace: gloo-system
Rules:
Backend Refs:
Group:
Kind: Service
Name: my-workload-v1
Namespace: my-workload
Port: 8080
Weight: 1
Matches:
Path:
Type: PathPrefix
Value: /api/my-workload
---
접속 테스트
curl -is -H "Host: api.example.com" http://localhost:8080/api/my-workload
---
HTTP/1.1 200 OK
vary: Origin
date: Sat, 12 Oct 2024 14:16:43 GMT
content-length: 296
content-type: text/plain; charset=utf-8
x-envoy-upstream-service-time: 1
server: envoy
{
"name": "my-workload-v1",
"uri": "/api/my-workload",
"type": "HTTP",
"ip_addresses": [
"10.244.0.12"
],
"start_time": "2024-10-12T14:16:43.438057",
"end_time": "2024-10-12T14:16:43.438450",
"duration": "393.228µs",
"body": "Hello From My Workload (v1)!",
"code": 200
}
---
버전 변경을 위한 1:1 비율 라우팅으로 변경
k apply -f https://raw.githubusercontent.com/solo-io/gloo-gateway-use-cases/main/gateway-api-tutorial/08-workload-route-header.yaml
k describe httproute -n my-workload
---
Rules:
Backend Refs:
Group:
Kind: Service
Name: my-workload-v2
Namespace: my-workload
Port: 8080
Weight: 1
Matches:
Headers:
Name: version
Type: Exact
Value: v2
Path:
Type: PathPrefix
Value: /api/my-workload
Backend Refs:
Group:
Kind: Service
Name: my-workload-v1
Namespace: my-workload
Port: 8080
Weight: 1
---
헤더 변경하며 테스트
v1 테스트
curl -is -H "Host: api.example.com" http://localhost:8080/api/my-workload
HTTP/1.1 200 OK
vary: Origin
date: Sat, 12 Oct 2024 14:18:10 GMT
content-length: 295
content-type: text/plain; charset=utf-8
x-envoy-upstream-service-time: 0
server: envoy
{
"name": "my-workload-v1",
"uri": "/api/my-workload",
"type": "HTTP",
"ip_addresses": [
"10.244.0.12"
],
"start_time": "2024-10-12T14:18:10.221498",
"end_time": "2024-10-12T14:18:10.221575",
"duration": "76.684µs",
"body": "Hello From My Workload (v1)!",
"code": 200
}
v2 테스트
curl -is -H "Host: api.example.com" -H "version: v2" http://localhost:8080/api/my-workload
HTTP/1.1 200 OK
vary: Origin
date: Sat, 12 Oct 2024 14:18:18 GMT
content-length: 296
content-type: text/plain; charset=utf-8
x-envoy-upstream-service-time: 1
server: envoy
{
"name": "my-workload-v2",
"uri": "/api/my-workload",
"type": "HTTP",
"ip_addresses": [
"10.244.0.13"
],
"start_time": "2024-10-12T14:18:18.286022",
"end_time": "2024-10-12T14:18:18.286788",
"duration": "766.002µs",
"body": "Hello From My Workload (v2)!",
"code": 200
}
1:1 비율로 트래픽 흐르도록 설정 변경
k apply -f https://raw.githubusercontent.com/solo-io/gloo-gateway-use-cases/main/gateway-api-tutorial/09-workload-route-split.yaml
k describe httproute -n my-workload
---
Rules:
Backend Refs:
Group:
Kind: Service
Name: my-workload-v1
Namespace: my-workload
Port: 8080
Weight: 50
Group:
Kind: Service
Name: my-workload-v2
Namespace: my-workload
Port: 8080
Weight: 50
---
접속 테스트
for i in {1..100}; do curl -s -H "Host: api.example.com" http://localhost:8080/api/my-workload/ | grep body; done | sort | uniq -c | sort -nr
---
50 "body": "Hello From My Workload (v2)!",
50 "body": "Hello From My Workload (v1)!",
---
디버깅
구성 문제에 대해 확인하고 조치하는 방법에 대해 glooctl을 사용해서 확인하는 방법입니다.
my-bad-workload-v2 업스트림에 대한 타겟을 올바르지 않은 것으로 변경
k apply -f https://raw.githubusercontent.com/solo-io/gloo-gateway-use-cases/main/gateway-api-tutorial/10-workload-route-split-bad-dest.yaml
라우팅 확인
k describe httproute -n my-workload
Rules:
Backend Refs:
Group:
Kind: Service
Name: my-workload-v1
Namespace: my-workload
Port: 8080
Weight: 50
Group:
Kind: Service
Name: my-bad-workload-v2
Namespace: my-workload
Port: 8080
Weight: 50
Matches:
Path:
Type: PathPrefix
Value: /api/my-workload
Status:
Parents:
Conditions:
Last Transition Time: 2024-10-12T14:21:49Z
Message: Service "my-bad-workload-v2" not found
Observed Generation: 4
Reason: BackendNotFound
Status: False
Type: ResolvedRefs
Last Transition Time: 2024-10-12T14:15:27Z
접속 테스트시 1:1 비율로 500 에러 발생
curl -is -H "Host: api.example.com" http://localhost:8080/api/my-workload
---
HTTP/1.1 200 OK
vary: Origin
date: Sat, 12 Oct 2024 14:22:49 GMT
content-length: 295
content-type: text/plain; charset=utf-8
x-envoy-upstream-service-time: 0
server: envoy
{
"name": "my-workload-v1",
"uri": "/api/my-workload",
"type": "HTTP",
"ip_addresses": [
"10.244.0.12"
],
"start_time": "2024-10-12T14:22:49.713025",
"end_time": "2024-10-12T14:22:49.713084",
"duration": "59.269µs",
"body": "Hello From My Workload (v1)!",
"code": 200
}
---
curl -is -H "Host: api.example.com" http://localhost:8080/api/my-workload
---
HTTP/1.1 500 Internal Server Error
date: Sat, 12 Oct 2024 14:22:50 GMT
server: envoy
content-length: 0
---
KinD 클러스터 접속
docker exec -it myk8s-control-plane bash
디버깅 진행
glooctl check
---
...
Checking VirtualServices... OK
Checking Gateways... OK
Checking Proxies... 1 Errors!
...
Checking Kubernetes HTTPRoutes... 1 Errors!
...
Skipping Gloo Instance check -- Gloo Federation not detected.
Error: 2 errors occurred:
* Found proxy with warnings by 'gloo-system': gloo-system gloo-system-http
Reason: warning:
Route Warning: InvalidDestinationWarning. Reason: invalid destination in weighted destination list: *v1.Upstream { blackhole_ns.kube-svc:blackhole-ns-blackhole-cluster-8080 } not found
* HTTPRoute my-workload.my-workload.http status (ResolvedRefs) is not set to expected (True). Reason: BackendNotFound, Message: Service "my-bad-workload-v2" not found
---
정상화 진행
k apply -f https://raw.githubusercontent.com/solo-io/gloo-gateway-use-cases/main/gateway-api-tutorial/09-workload-route-split.yaml
디버깅 진행
glooctl check
---
Checking Deployments... OK
Checking Pods... OK
Checking Upstreams... OK
Checking UpstreamGroups... OK
Checking AuthConfigs... OK
Checking RateLimitConfigs... OK
Checking VirtualHostOptions... OK
Checking RouteOptions... OK
Checking Secrets... OK
Checking VirtualServices... OK
Checking Gateways... OK
Checking Proxies... OK
Detected Kubernetes Gateway integration!
Checking Kubernetes GatewayClasses... OK
Checking Kubernetes Gateways... OK
Checking Kubernetes HTTPRoutes... OK
Skipping Gloo Instance check -- Gloo Federation not detected.
No problems detected.
---
모니터링
포트포워드 설정
k -n gloo-system port-forward deployment/gloo-proxy-http 19000 &
접속
http://localhost:19000/
호출된 결과에 대한 http code 확인
curl -s http://localhost:19000/stats | grep -E "(^cluster.kube-svc_httpbin-httpbin-8000_httpbin.upstream.*(2xx|5xx))"
---
cluster.kube-svc_httpbin-httpbin-8000_httpbin.upstream_rq_2xx: 4
---
의도적으로 500 발생
curl -is -H "Host: api.example.com" http://localhost:8080/api/httpbin/status/500
값 확인
curl -s http://localhost:19000/stats | grep -E "(^cluster.kube-svc_httpbin-httpbin-8000_httpbin.upstream.*(2xx|5xx))"
---
cluster.kube-svc_httpbin-httpbin-8000_httpbin.upstream_rq_2xx: 4
cluster.kube-svc_httpbin-httpbin-8000_httpbin.upstream_rq_5xx: 1
---
클러스터 정리
kind delete cluster --name myk8s
후기
Ingress의 경우 전 직장에서도 사용해보았고, 지금은 클라우드 사업자가 만든 방식으로 사용해서 어느정도는 익숙했기 때문에 크게 어려울것은 없었지만, 그동안 여러 요구사항에 의해 Gateway API가 새롭게 추가되었다는것을 새롭게 알게 되었고, 알게된 시점부터 배워보고 싶었는데 이번 스터디에서 드디어 Gateway에 대한 부분을 공부하게 되어 매우 좋았습니다. 남은 3번의 스터디도 잘 참석하고 배워서 지금보다는 조금 더 잘알고 잘 사용하는 엔지니어가 되었으면 합니다.