Overview
Istio를 통해 Traffic을 제어 할 수 있다.
다음은 Istio를 통해서도 Canary를 구현 할 수 있다. Canary 또한 Traffic을 통제하여 신버전으로 Traffic을 적게 가져가는 식으로 하면 된다.
Weighted 기반 라우팅
Demo Application 구성
apiVersion: apps/v1 kind: Deployment metadata: name: position-simulator spec: selector: matchLabels: app: position-simulator replicas: 1 template: metadata: labels: app: position-simulator spec: containers: - name: position-simulator image: richardchesterwood/istio-fleetman-position-simulator:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice command: ["java","-Xmx50m","-jar","webapp.jar"] imagePullPolicy: Always --- apiVersion: apps/v1 kind: Deployment metadata: name: position-tracker spec: selector: matchLabels: app: position-tracker replicas: 1 template: metadata: labels: app: position-tracker spec: containers: - name: position-tracker image: richardchesterwood/istio-fleetman-position-tracker:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice command: ["java","-Xmx50m","-jar","webapp.jar"] imagePullPolicy: Always --- apiVersion: apps/v1 kind: Deployment metadata: name: api-gateway spec: selector: matchLabels: app: api-gateway replicas: 1 template: metadata: labels: app: api-gateway spec: containers: - name: api-gateway image: richardchesterwood/istio-fleetman-api-gateway:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice command: ["java","-Xmx50m","-jar","webapp.jar"] imagePullPolicy: Always --- apiVersion: apps/v1 kind: Deployment metadata: name: webapp spec: selector: matchLabels: app: webapp replicas: 1 template: metadata: labels: app: webapp spec: containers: - name: webapp image: richardchesterwood/istio-fleetman-webapp-angular:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice imagePullPolicy: Always --- apiVersion: apps/v1 kind: Deployment metadata: name: vehicle-telemetry spec: selector: matchLabels: app: vehicle-telemetry replicas: 1 template: metadata: labels: app: vehicle-telemetry spec: containers: - name: vehicle-telemtry image: richardchesterwood/istio-fleetman-vehicle-telemetry:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice imagePullPolicy: Always --- apiVersion: apps/v1 kind: Deployment metadata: name: staff-service spec: selector: matchLabels: app: staff-service replicas: 1 template: metadata: labels: app: staff-service version: safe # 구버전에 대한 Application Label이다. Safe가 기존 운영하던 Stable한 버전이라고 친다. spec: containers: - image: richardchesterwood/istio-fleetman-staff-service:6-placeholder name: staff-service env: - name: SPRING_PROFILES_ACTIVE value: production-microservice imagePullPolicy: Always ports: - containerPort: 8080 --- apiVersion: apps/v1 kind: Deployment metadata: name: staff-service-risky-version spec: selector: matchLabels: app: staff-service replicas: 1 template: metadata: labels: app: staff-service version: risky # 신버전에 대한 Application에 대한 Deploy이다. 신버전에 대해서 version: risky 라는 Label로 지정하였다. spec: containers: - name: staff-service image: richardchesterwood/istio-fleetman-staff-service:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice imagePullPolicy: Always ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: fleetman-webapp spec: selector: app: webapp ports: - appProtocol: http name: http port: 80 protocol: TCP targetPort: 80 type: ClusterIP --- apiVersion: v1 kind: Service metadata: name: fleetman-position-tracker spec: selector: app: position-tracker ports: - name: http port: 8080 type: ClusterIP --- apiVersion: v1 kind: Service metadata: name: fleetman-api-gateway spec: selector: app: api-gateway ports: - name: http port: 8080 type: ClusterIP --- apiVersion: v1 kind: Service metadata: name: fleetman-vehicle-telemetry spec: selector: app: vehicle-telemetry ports: - name: http port: 8080 type: ClusterIP --- apiVersion: v1 kind: Service metadata: name: fleetman-staff-service spec: selector: app: staff-service ports: - name: http port: 8080 type: ClusterIP
Demo Application 웹 접속을 위한 GW, VS 구성
apiVersion: networking.istio.io/v1beta1 kind: Gateway metadata: name: fleetman-gw spec: selector: istio: ingressgateway servers: - hosts: - "kiali-mng-dev.test.co.kr" port: number: 80 name: http protocol: HTTP
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: fleetman-vs
spec:
hosts:"kiali-mng-dev.test.co.kr"
gateways:fleetman-gw
http:match:
- port: 80
route: - destination:
host: fleetman-webapp
port:number: 80
Kiali 에서보면 구조는 아래와 같다.
staff-service에 보면 Risky 버전과 Safe 버전 2가지가 공존하는 것을 볼 수 있다.
- Demo Web 접속
- 해당 웹어플리케이션은 배송 추적 시스템이다.
- port: 80
신버전과 구버전간의 트래픽 비율 조정
kind: VirtualService apiVersion: networking.istio.io/v1alpha3 metadata: name: canary-vs # 이 부분은 그냥 이름 넣어주면 된다. namespace: default spec: hosts: - fleetman-staff-service.default.svc.cluster.local # 라우팅 규칙을 적용할 서비스 DNS 이름 입니다. (내부 용 FQDN으로 처리) http: - route: - destination: host: fleetman-staff-service.default.svc.cluster.local # 타겟 DNS 이름 subset: safe-group # Destination Rule의 이름 weight: 90 # 비율 지정 - destination: host: fleetman-staff-service.default.svc.cluster.local # 타겟 DNS 이름 subset: risky-group # Destination Rule의 이름 weight: 10 --- kind: DestinationRule # 각 Subset에 포함될 파드 정의 apiVersion: networking.istio.io/v1alpha3 metadata: name: canary-dr namespace: default spec: host: fleetman-staff-service # 서비스 subsets: - labels: # Selector version: safe # 파드를 Label과 함께 찾는다. name: safe-group - labels: version: risky name: risky-group
확인
실제 City Truck의 운전자 사진이 Risk 버전에서는 제대로 표시가 안되는 것을 확인
브라우저에서는 Caching 때문에 제대로 확인이 안될 수 있는데 curl로 확인 해보자
root # while true; do curl http://kiali-mng-dev.test.co.kr/api/vehicles/driver/City%20Truck; echo; sleep 0.5; done {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/1.jpg"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/1.jpg"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/1.jpg"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/1.jpg"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"} {"name":"Pam Parry","photo":"https://rac-istio-course-images.s3.amazonaws.com/placeholder.png"}
"1.jpg" 가 10% 확율 치고는 자주 나온거 같긴하지만 얼추 그래도 맞는 것 같다.
Session Affinity
Istio에서 Consistent Hash 라는 것을 이용하여 Session을 유지 해주는 기능을 제공한다.
LB에서 Hash 알고리즘을 이용하여 Client로부터 받은 데이터를 Hashing 한 후 데이터를 파드에 전송하는데 Hash 값을 이용하여 Sticky Session을 유지한다
Session 관리로 여러가지 방식을 지원한다. Header, Cookie, SourceIP 등
여기서는 SourceIP를 통해 테스트 진행 해 보았다.
위에 테스트한 데모 어플리케이션이 Version에 따라 safe와 risky 버전을 나눠서 90:10 비율로 처리 했는데, Consistent Hash를 사용하여 처음 접근 했던 세션이 그대로 유지 되는지 테스트
kind: VirtualService
apiVersion: networking.istio.io/v1alpha3
metadata:
name: session-vs
namespace: default
spec:
hosts:
- fleetman-staff-service.default.svc.cluster.local
http:
- route:
- destination:
host: fleetman-staff-service.default.svc.cluster.local
subset: all-staff-service-pods
# weight: weight 옵션을 같이 사용 할 수 없다.
---
kind: DestinationRule
apiVersion: networking.istio.io/v1alpha3
metadata:
name: session-dr
namespace: default
spec:
host: fleetman-staff-service
trafficPolicy:
loadBalancer:
consistentHash:
useSourceIp: true # useSourceIp: true로 설정하면 접속한 SourceIP으로 세션을 유지 해준다.
subsets:
- labels:
app: staff-service
name: all-staff-service-pods
처음 접속 했을 때 Stable 버전으로 접속되서 다른 Truck들도 사진이 잘 나온다. 반면에 Risky 버전으로 접속이 됐으면 계속 사진이 없는 Truck만 조회 된다.
Prefix기반 라우팅
Demo Application 구성
apiVersion: apps/v1 kind: Deployment metadata: name: position-simulator spec: selector: matchLabels: app: position-simulator replicas: 1 template: # template for the pods metadata: labels: app: position-simulator spec: containers: - name: position-simulator image: richardchesterwood/istio-fleetman-position-simulator:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice command: ["java","-Xmx50m","-jar","webapp.jar"] imagePullPolicy: Always --- apiVersion: apps/v1 kind: Deployment metadata: name: position-tracker spec: selector: matchLabels: app: position-tracker replicas: 1 template: # template for the pods metadata: labels: app: position-tracker spec: containers: - name: position-tracker image: richardchesterwood/istio-fleetman-position-tracker:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice command: ["java","-Xmx50m","-jar","webapp.jar"] imagePullPolicy: Always --- apiVersion: apps/v1 kind: Deployment metadata: name: api-gateway spec: selector: matchLabels: app: api-gateway replicas: 1 template: # template for the pods metadata: labels: app: api-gateway spec: containers: - name: api-gateway image: richardchesterwood/istio-fleetman-api-gateway:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice command: ["java","-Xmx50m","-jar","webapp.jar"] imagePullPolicy: Always --- apiVersion: apps/v1 kind: Deployment metadata: name: webapp spec: selector: matchLabels: app: webapp replicas: 1 template: # template for the pods metadata: labels: app: webapp version: original spec: containers: - name: webapp image: richardchesterwood/istio-fleetman-webapp-angular:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice imagePullPolicy: Always --- apiVersion: apps/v1 kind: Deployment metadata: name: webapp-experimental spec: selector: matchLabels: app: webapp replicas: 1 template: # template for the pods metadata: labels: app: webapp version: experimental # labels로 version: experimental 이름으로 canary와 동일한 신규버전을 만들었다. spec: containers: - name: webapp image: richardchesterwood/istio-fleetman-webapp-angular:6-experimental env: - name: SPRING_PROFILES_ACTIVE value: production-microservice imagePullPolicy: Always --- apiVersion: apps/v1 kind: Deployment metadata: name: vehicle-telemetry spec: selector: matchLabels: app: vehicle-telemetry replicas: 1 template: # template for the pods metadata: labels: app: vehicle-telemetry spec: containers: - name: vehicle-telemtry image: richardchesterwood/istio-fleetman-vehicle-telemetry:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice imagePullPolicy: Always --- apiVersion: apps/v1 kind: Deployment metadata: name: staff-service spec: selector: matchLabels: app: staff-service replicas: 0 template: # template for the pods metadata: labels: app: staff-service version: safe # version: safe로 기존 stable 버전이다. spec: containers: - name: staff-service image: richardchesterwood/istio-fleetman-staff-service:6-placeholder env: - name: SPRING_PROFILES_ACTIVE value: production-microservice imagePullPolicy: Always ports: - containerPort: 8080 --- apiVersion: apps/v1 kind: Deployment metadata: name: staff-service-risky-version spec: selector: matchLabels: app: staff-service replicas: 1 template: # template for the pods metadata: labels: app: staff-service version: risky # version: risky가 신규 버전의 Canry 이다. spec: containers: - name: staff-service image: richardchesterwood/istio-fleetman-staff-service:6 env: - name: SPRING_PROFILES_ACTIVE value: production-microservice imagePullPolicy: Always ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: fleetman-webapp spec: # This defines which pods are going to be represented by this Service # The service becomes a network endpoint for either other services # or maybe external users to connect to (eg browser) selector: app: webapp ports: - name: http port: 80 nodePort: 30080 type: NodePort --- apiVersion: v1 kind: Service metadata: name: fleetman-position-tracker spec: # This defines which pods are going to be represented by this Service # The service becomes a network endpoint for either other services # or maybe external users to connect to (eg browser) selector: app: position-tracker ports: - name: http port: 8080 type: ClusterIP --- apiVersion: v1 kind: Service metadata: name: fleetman-api-gateway spec: selector: app: api-gateway ports: - name: http port: 8080 type: ClusterIP --- apiVersion: v1 kind: Service metadata: name: fleetman-vehicle-telemetry spec: selector: app: vehicle-telemetry ports: - name: http port: 8080 type: ClusterIP --- apiVersion: v1 kind: Service metadata: name: fleetman-staff-service spec: selector: app: staff-service ports: - name: http port: 8080 type: ClusterIP
Virtaul Service, Gateway, Destination Rule
apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: ingress-gateway-configuration spec: selector: istio: ingressgateway servers: - port: number: 80 name: http protocol: HTTP hosts: - "*" --- kind: VirtualService apiVersion: networking.istio.io/v1alpha3 metadata: name: fleetman-webapp namespace: default spec: hosts: - "*" gateways: - ingress-gateway-configuration http: - match: # Prefix에 대한 설정 - uri: # IF /experimental 이거나 prefix: "/experimental" - uri: # OR /canary 이면 prefix: "/canary" route: # THEN destination rule의 experimental로 라우팅 해라. - destination: host: fleetman-webapp subset: experimental - match: # / Prefix면 DR의 original로 기존 버전으로 라우팅 - uri : prefix: "/" route: - destination: host: fleetman-webapp subset: original --- kind: DestinationRule apiVersion: networking.istio.io/v1alpha3 metadata: name: fleetman-webapp namespace: default spec: host: fleetman-webapp subsets: - labels: version: original # labels.version: original로 라우팅 name: original # Subset Name이 Original이면 - labels: version: experimental # labels.version: experimental로 라우팅 name: experimental # Subset Name이 experimental이면
테스트
root # while true; do curl -s http://kiali-mng-dev.test.co.kr/experimental | grep title; sleep 0.5; done <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management Istio Premium Enterprise Edition</title>
/experimental로 하면 신규버전으로 "Fleet Management Istio Premium Enterprise Edition" 만 표시되어야 하는데 라운드로빈으로 왔다갔다 호출 되는거 같다.
이 부분은 Istio-Ingressgateway로 트래픽을 제대로 타지 못했거나 K8s 서비스 오브젝트를 통해 접근 했을 경우 또는 다른 이슈가 있는 듯하다.
Haproxy와 Ingressgateway service의 설정은 딱히 문제가 없는 듯 하다.
확인 해보니 기존에 테스트로 생성해둔 GW, VS가 동일한 호스트와 포트로 사용중인게 있어서 문제였다 (관련해서 kiali를 통해 발견했다. 스크린샷은 찍지못함)
관련해서 제거 한 후 다시 테스트
root # while true; do curl -s http://kiali-mng-dev.test.co.kr/experimental | grep title; sleep 0.5; done <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management Istio Premium Enterprise Edition</title> <title>Fleet Management Istio Premium Enterprise Edition</title>
root # while true; do curl -s http://kiali-mng-dev.test.co.kr/ | grep title; sleep 0.5; done <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title> <title>Fleet Management</title>
신규 버전과 구 버전의 Prefix로 라우팅이 잘 되는듯 하다.
SubDomain으로 라우팅
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: ingress-gateway-configuration
spec:
selector:
istio: ingressgateway #
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "kiali-mng-dev.test.co.kr"
- "prometheus-mng-dev.test.co.kr"
#- "*.kiali-mng-dev.test.co.kr" 이런식으로 와일드카드로도 설정 가능
#- "*.test.co.kr"
---
kind: VirtualService
apiVersion: networking.istio.io/v1alpha3
metadata:
name: fleetman-webapp
namespace: default
spec:
hosts:
- "kiali-mng-dev.test.co.kr"
gateways:
- ingress-gateway-configuration
http:
- route:
- destination:
host: fleetman-webapp
subset: original
---
kind: VirtualService
apiVersion: networking.istio.io/v1alpha3
metadata:
name: fleetman-webapp-experiment
namespace: default
spec:
hosts:
- "prometheus-mng-dev.test.co.kr"
gateways:
- ingress-gateway-configuration
http:
- route:
- destination:
host: fleetman-webapp
subset: experimental
---
kind: DestinationRule
apiVersion: networking.istio.io/v1alpha3
metadata:
name: fleetman-webapp
namespace: default
spec:
host: fleetman-webapp
subsets:
- labels:
version: original
name: original
- labels:
version: experimental
name: experimental
JWT 기반 라우팅은 아직 알파단계 조만간 나올 것으로 보인다.
'IT > Istio' 카테고리의 다른 글
Mutual TLS(mTLS) with Istio (1) | 2024.01.02 |
---|---|
서킷 브레이킹 (0) | 2023.11.10 |