Kubernetes集群——（k8s）service（二）+ingress配置+NodeProt实现负载均衡+session实现会话保持(daemonset)+TLS加密认证

技术2022-07-11 121

一、ingress简介一种全局的、为了代理不同后端 Service 而设置的负载均衡服务，就是 Kubernetes 里的 Ingress 服务。 • Ingress由两部分组成：Ingress controller和Ingress服务。 • Ingress Controller 会根据你定义的 Ingress 对象，提供对应的代理能力。业界常用的各种反向代理项目，比如 Nginx、HAProxy、Envoy、Traefik 等，都已经为Kubernetes 专门维护了对应的 Ingress Controller。官网：https://kubernetes.github.io/ingress-nginx/

二、使用nodprot方式实现ingress的负载均衡应用ingress controller定义文件：

获取镜像vim deploy.yaml

镜像下载完成并导入仓库更改文件里面的镜像格式

vim deploy.yaml

2.2应用：

[root@server2 manifest]# kubectl -n ingress-nginx describe svc ingress-nginx-controller

直接访问ingress端口

访问成功：404表示默认发布页面不存在 [root@client Desktop]# curl 172.25.254.3:30549 <html> <head><title>404 Not Found</title></head> <body> <center><h1>404 Not Found</h1></center> <hr><center>nginx/1.19.0</center> </body> </html>

创建ingress服务参考官网：https://kubernetes.github.io/ingress-nginx/user-guide/basic-usage/

[root@server2 manifest]# vim ingress.yaml [root@server2 manifest]# cat ingress.yaml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: ingress-myservicea annotations: # use the shared ingress-nginx kubernetes.io/ingress.class: "nginx" spec: rules: - host: www1.westos.org http: paths: - path: / backend: serviceName: myservice servicePort: 80 [root@server2 manifest]# kubectl apply -f ingress.yaml ingress.networking.k8s.io/ingress-myservicea created

添加 www1.westos.org到外部主机解析文件中，当前添加到server3上（所有节点都可以添加域名解析）每个节点都有30549端口都可以访问

[root@server2 manifest]# netstat -tnpl | grep 30549 tcp 0 0 0.0.0.0:30549 0.0.0.0:* LISTEN 17859/kube-proxy [root@server3 net.d]# netstat -tnpl | grep 30549 tcp 0 0 0.0.0.0:30549 0.0.0.0:* LISTEN 18321/kube-proxy [root@server4 net.d]# netstat -tnpl | grep 30549 tcp 0 0 0.0.0.0:30549 0.0.0.0:* LISTEN 18326/kube-proxy

补充：定义多个域名+service+ingress

vim ingress.yaml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: ingress1 annotations: kubernetes.io/ingress.class: "nginx" spec: rules: - host: www1.westos.org http: paths: - path: / backend: serviceName: myservice servicePort: 80 - host: www2.westos.org http: paths: - path: / backend: serviceName: myservice2 servicePort: 80 [root@server2 manifest]# cat service.yml kind: Service apiVersion: v1 metadata: name: myservice spec: ports: - protocol: TCP port: 80 targetPort: 80 selector: app: myapp 标签 type: NodePort --- kind: Service apiVersion: v1 metadata: name: myservice2 spec: ports: - protocol: TCP port: 80 targetPort: 80 selector: app: myappv2 标签 type: NodePort [root@server2 manifest]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 21h myservice NodePort 10.103.19.178 <none> 80:31511/TCP 8s myservice2 NodePort 10.100.55.226 <none> 80:31663/TCP 8s [root@server2 manifest]# cat deployment.yml apiVersion: apps/v1 kind: Deployment metadata: name: deployment-myapp-v1 spec: replicas: 2 selector: matchLabels: app: myapp 标签 template: metadata: labels: app: myapp 标签 spec: containers: - name: myapp image: myapp:v1 --- apiVersion: apps/v1 kind: Deployment metadata: name: deployment-myapp-v2 spec: replicas: 2 selector: matchLabels: app: myappv2 template: metadata: labels: app: myappv2 spec: containers: - name: myappv2 image: myapp:v2

以上全部运行启动

添加域名解析外部主机测试访问

当重新创建ingress后，ingress里面的pod会访问k8s的api在/etc/下重新读取Nginx，制定新的规则。

[root@server2 manifest]# kubectl -n ingress-nginx get pod NAME READY STATUS RESTARTS AGE ingress-nginx-admission-create-5lsbv 0/1 Completed 0 98m ingress-nginx-admission-patch-74qn4 0/1 Completed 0 98m ingress-nginx-controller-77b5fc5746-mjvvr 1/1 Running 0 98m 登录查看信息 [root@server2 manifest]# kubectl -n ingress-nginx exec -it ingress-nginx-controller-77b5fc5746-mjvvr -- sh

三、用DaemonSet结合nodeselector来部署ingress-controller到特定的node上，然后使用 HostNetwork直接把该pod与宿主机node的网络打通，直接使用宿主机的80/433端口就能访问服务。优点是整个请求链路最简单，性能相对NodePort模式更好。 • 缺点是由于直接利用宿主机节点的网络和端口，一个node只能部署一个ingress- controller pod。 • 比较适合大并发的生产环境使用。 3.1修改ingress controller部署文件

[root@server2 manifest]# vim deploy.yaml

删除之前的信息 [root@server2 manifest]# kubectl -n ingress-nginx delete deployments.apps ingress-nginx-controller deployment.apps "ingress-nginx-controller" deleted [root@server2 manifest]# kubectl -n ingress-nginx delete svc ingress-nginx-controller service "ingress-nginx-controller" deleted [root@server2 manifest]# kubectl -n ingress-nginx delete svc ingress-nginx-controller-admission service "ingress-nginx-controller-admission" deleted

测试访问

查看server3上的端口（其他节点上没有这两个端口，因为就绑定了server3） [root@server3 net.d]# cd [root@server3 ~]# netstat -tnpl | grep 80 tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 93848/nginx: master tcp6 0 0 :::80 :::* LISTEN 93848/nginx: master [root@server3 ~]# netstat -tnpl | grep 443 tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 93848/nginx: master tcp6 0 0 :::8443 :::* LISTEN 93827/nginx-ingress tcp6 0 0 :::443 :::* LISTEN 93848/nginx: master

会话保持参考官网地址：https://kubernetes.github.io/ingress-nginx/examples/auth/basic/

[root@server2 manifest]# vim ingress.yaml [root@server2 manifest]# kubectl apply -f ingress.yaml ingress.networking.k8s.io/ingress1 configured ingress.networking.k8s.io/ingress2 configured

在浏览器测试访问：四、Ingress+ TLS 配置参考官网信息

4.1生成证书和key

[root@server2 ~]# openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginxsvc/O=nginxsvc" Generating a 2048 bit RSA private key .........................................................................................................................+++ .....................+++ writing new private key to 'tls.key' ----- [root@server2 ~]# ls tls.crt tls.key [root@server2 ~]# kubectl create secret tls tls-secret --key tls.key --cert tls.crt secret/tls-secret created [root@server2 ~]# kubectl get secrets 查看 NAME TYPE DATA AGE default-token-754fk kubernetes.io/service-account-token 3 23h tls-secret kubernetes.io/tls 2 94s

4.2把证书生效到Nginx调度上也就是ingress-nginx-controller 参考官网文档

[root@server2 ~]# vim tls.yml apiVersion: networking.k8s.io/v1beta1 kind: Ingress metadata: name: nginx-tls spec: tls: - hosts: - www1.westos.org # This assumes tls-secret exists and the SSL # certificate contains a CN for foo.bar.com secretName: tls-secret rules: - host: www1.westos.org http: paths: - path: / backend: # This assumes http-svc exists and routes to healthy endpoints serviceName: myservice servicePort: 80 [root@server2 ~]# kubectl apply -f tls.yml ingress.networking.k8s.io/nginx-tls created [root@server2 manifest]# vim deploy.yaml

[root@server2 ~]# kubectl get ingress NAME CLASS HOSTS ADDRESS PORTS AGE ingress-myservicea <none> www1.westos.org 172.25.254.3 80 3h5m ingress1 <none> www1.westos.org 172.25.254.3 80 79m ingress2 <none> www2.westos.org 172.25.254.3 80 79m nginx-tls <none> www1.westos.org 172.25.254.3 80, 443 6m5s

在浏览器测试访问当访问80端口直接跳转到443加密访问

[root@server2 ~]# kubectl -n ingress-nginx exec -it ingress-nginx-controller-7jqmm -- sh /etc/nginx $

Processed: 0.012, SQL: 9