使用kube-router网络最简部署k8s集群方案(附因被墙问题所需的软件包) 知识

7 4月前 462

一、软件版本

     docker-ce-18.09.6-3.el7.x86_64

     docker-ce-cli-18.09.6-3.el7.x86_64

     kubelet-1.14.2-0.x86_64

     kubernetes-cni-0.7.5-0.x86_64

     kubeadm-1.14.2-0.x86_64

     kubectl-1.14.2-0.x86_64


二、集群部署

1. 准备3台服务器:

    master

    node1

    node2

2. 在3台机器上分别安装 docker kubelet kubectl kubeadm

此处省略安装步骤,如网络通信正常均使用官网yum源即可,文章最后会提供所有本文章使用到的软件包

3. 使用最建配置初始化master节点:

# cat kubeadm-config.yaml 

apiVersion: kubeadm.k8s.io/v1beta1
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: "master"
  bindPort: 6443
nodeRegistration:
  kubeletExtraArgs:
    root-dir: "/etc/kubernetes/kubelet"
---
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: stable
controlPlaneEndpoint: "master:6443"
etcd:
    external:
        endpoints:
        - http://etcd_node:2379
networking:
  podSubnet: "192.168.0.0/16"
  serviceSubnet: "172.16.0.0/16"
imageRepository: "k8s.gcr.io"
apiServer:
  extraArgs:
    insecure-bind-address: "0.0.0.0"
    insecure-port: "3218

参数说明:

#使用外部etcd节点
etcd:
    external:
        endpoints:
        - http://etcd_node:2379
#----------------------------------------
#自定义pod和service网段
networking:
  podSubnet: "192.168.0.0/16"
  serviceSubnet: "172.16.0.0/16"
#----------------------------------------
#开启apiserver http端口
apiServer:
  extraArgs:
    insecure-bind-address: "0.0.0.0"
    insecure-port: "3218"

初始化master节点:

如果上面配置文件报错请参考:链接: https://pan.baidu.com/s/1qK1-Ei7Q_LhLzNqdD6Cm7A 提取码: 9grx 

官方配置参:https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta2

kubeadm init --config kubeadm-config.yaml

4. 根据初始化完成的master节点提供的join信息将node1 和 node2 加入到集群中

命令省略

三、配置kube-router

1. 删除kube-proxy

kubectl -n kube-system delete ds kube-proxy

2. 清理kube-proxy产生的iptables规则(有待确认这一步是否真的生效)

docker run --privileged -v /lib/modules:/lib/modules --net=host  k8s.gcr.io/kube-proxy:v1.14.2 kube-proxy --cleanup

3. 部署kube-router

kubectl apply -f kube-router-all-service-daemonset.yaml

修改kube-router-all-service-daemonset.yaml中kubeconfig所在路径

# cat kube-router-all-service-daemonset.yaml 

apiVersion: v1
kind: ConfigMap
metadata:
  name: kube-router-cfg
  namespace: kube-system
  labels:
    tier: node
    k8s-app: kube-router
data:
  cni-conf.json: |
    {
       "cniVersion":"0.3.0",
       "name":"mynet",
       "plugins":[
          {
             "name":"kubernetes",
             "type":"bridge",
             "bridge":"kube-bridge",
             "isDefaultGateway":true,
             "ipam":{
                "type":"host-local"
             }
          }
       ]
    }
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: kube-router
  namespace: kube-system
  labels:
    k8s-app: kube-router
spec:
  template:
    metadata:
      labels:
        k8s-app: kube-router
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ''
    spec:
      containers:
      - name: kube-router
        image: docker.io/cloudnativelabs/kube-router
        args: ["--run-router=true", "--run-firewall=true", "--run-service-proxy=true", "--kubeconfig=/root/.kube/config"] #修改kubeconfig路径
        securityContext:
          privileged: true
        imagePullPolicy: Always
        env:
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        - name: KUBE_ROUTER_CNI_CONF_FILE
          value: /etc/cni/net.d/10-kuberouter.conflist
        livenessProbe:
          httpGet:
            path: /healthz
            port: 20244
          initialDelaySeconds: 10
          periodSeconds: 3
        volumeMounts:
        - name: lib-modules
          mountPath: /lib/modules
          readOnly: true
        - name: cni-conf-dir
          mountPath: /etc/cni/net.d
        - name: kubeconfig
          mountPath: /root/.kube/config #修改kubeconfig路径
          readOnly: true
      initContainers:
      - name: install-cni
        image: busybox
        imagePullPolicy: Always
        command:
        - /bin/sh
        - -c
        - set -e -x;
          if [ ! -f /etc/cni/net.d/10-kuberouter.conflist ]; then
            if [ -f /etc/cni/net.d/*.conf ]; then
              rm -f /etc/cni/net.d/*.conf;
            fi;
            TMP=/etc/cni/net.d/.tmp-kuberouter-cfg;
            cp /etc/kube-router/cni-conf.json ${TMP};
            mv ${TMP} /etc/cni/net.d/10-kuberouter.conflist;
          fi
        volumeMounts:
        - name: cni-conf-dir
          mountPath: /etc/cni/net.d
        - name: kube-router-cfg
          mountPath: /etc/kube-router
      hostNetwork: true
      tolerations:
      - key: CriticalAddonsOnly
        operator: Exists
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
        operator: Exists
      - effect: NoSchedule
        key: node.kubernetes.io/not-ready
        operator: Exists
      volumes:
      - name: lib-modules
        hostPath:
          path: /lib/modules
      - name: cni-conf-dir
        hostPath:
          path: /etc/cni/net.d
      - name: kube-router-cfg
        configMap:
          name: kube-router-cfg
      - name: kubeconfig
        hostPath:
          path: /root/.kube/config  #修改kubeconfig路径

启动kube-router:

kubectl apply -f kube-router-all-service-daemonset.yaml

观察启动是否成功:

# kubectl  get po --all-namespaces -o wide
NAMESPACE     NAME                                            READY   STATUS    RESTARTS   AGE     IP             NODE                     NOMINATED NODE   READINESS GATE
kube-system   kube-router-d7dzp                               1/1     Running   0          5m59s   node1_ip       node1                    <none>           <none>
kube-system   kube-router-gxl24                               1/1     Running   0          5m59s   node2_ip       node2                    <none>           <none>
kube-system   kube-router-rlr66                               1/1     Running   0          5m59s   master_ip      master                   <none>           <none

参考文档:

https://github.com/cloudnativelabs/kube-router/blob/master/docs/kubeadm.md

四、验证网络通信是否正常:

kubectl apply -f whats-my-ip.yaml
# cat whats-my-ip.yaml 

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  generation: 1
  labels:
    run: whats-my-ip
  name: whats-my-ip
  namespace: default
  resourceVersion: "1147342"
  selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/whats-my-ip
spec:
  progressDeadlineSeconds: 600
  replicas: 2
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      run: whats-my-ip
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        run: whats-my-ip
    spec:
      containers:
      - image: cloudnativelabs/whats-my-ip
        imagePullPolicy: Always
        name: whats-my-ip
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status: {}
---
apiVersion: v1
kind: Service
metadata:
  labels:
    run: whats-my-ip
  name: whats-my-ip
  namespace: default
  resourceVersion: "1110563"
  selfLink: /api/v1/namespaces/default/services/whats-my-ip
spec:
  clusterIP: 172.16.111.1
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
    nodePort: 30080
  selector:
    run: whats-my-ip
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

1.查看pod

# kubectl  get po -o wide
NAMESPACE     NAME                                            READY   STATUS    RESTARTS   AGE     IP             NODE                     NOMINATED NODE   READINESS GATES
default       whats-my-ip-5b9588465b-59s55                    1/1     Running   0          12s     192.168.2.6    node1                    <none>           <none>
default       whats-my-ip-5b9588465b-tnbf6                    1/1     Running   0          12s     192.168.1.3    node2                    <none>           <none>

2.查看service

# kubectl get svc
NAMESPACE     NAME          TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                  AGE
default       kubernetes    ClusterIP   172.16.0.1     <none>        443/TCP                  74m
default       whats-my-ip   NodePort    172.16.111.1   <none>        8080:30080/TCP           3m3

3. curl service ip:port  是否正常

#在任意node节点上:
# curl  172.16.111.1:8080
HOSTNAME:whats-my-ip-5b9588465b-qqpnd IP:192.168.2.6
# curl  172.16.111.1:8080
HOSTNAME:whats-my-ip-5b9588465b-hkvq2 IP:192.168.1.3

4. curl node_ip:port 是否正常

# curl node1:30080
HOSTNAME:whats-my-ip-5b9588465b-jmdw4 IP:192.168.1.3
# curl node1:30080
HOSTNAME:whats-my-ip-5b9588465b-79pk5 IP:192.168.2.6

# curl node2:30080
HOSTNAME:whats-my-ip-5b9588465b-jmdw4 IP:192.168.1.3
# curl node2:30080
HOSTNAME:whats-my-ip-5b9588465b-79pk5 IP:192.168.2.6

五、附件

1. rpm包

链接: https://pan.baidu.com/s/1s6s4IcPQ9CQ6chRq3UC6rA 提取码: ha97 复制这段内容后打开百度网盘手机App,操作更方便哦

2. 镜像包,以下镜像包含文章中所有使用的镜像:

docker images
REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
busybox                              latest              e4db68de4ff2        5 days ago          1.22MB
k8s.gcr.io/kube-proxy                v1.14.2             5c24210246bb        4 weeks ago         82.1MB
k8s.gcr.io/kube-apiserver            v1.14.2             5eeff402b659        4 weeks ago         210MB
k8s.gcr.io/kube-controller-manager   v1.14.2             8be94bdae139        4 weeks ago         158MB
k8s.gcr.io/kube-scheduler            v1.14.2             ee18f350636d        4 weeks ago         81.6MB
cloudnativelabs/kube-router          latest              4b7497e883ee        5 weeks ago         105MB
k8s.gcr.io/coredns                   1.3.1               eb516548c180        5 months ago        40.3MB
k8s.gcr.io/etcd                      3.3.10              2c4adeb21b4f        6 months ago        258MB
k8s.gcr.io/pause                     3.1                 da86e6ba6ca1        18 months ago       742kB
cloudnativelabs/whats-my-ip          latest              7be89ad1e6ef        2 years ago         5.95MB

链接: https://pan.baidu.com/s/13qkHs_L_vx7t_zguYok7sw 提取码: x6iw 复制这段内容后打开百度网盘手机App,操作更方便哦

最后于 4月前 被7编辑 ,原因:
最新回复 (4)
  • 愚人乙 4月前
    0 引用 2
    给你调整了下排版
  • lawrence_lx 4月前
    0 引用 3
    这个文档部署下来,能正常运行吗
  • lawrence_lx 4月前
    0 引用 4
    这文档从初始化master就报错了
    error converting YAML to JSON: yaml: line 17: found unexpected end of stream
  • 7 4月前
    0 引用 5
    lawrence_lx 这文档从初始化master就报错了 error converting YAML to JSON: yaml: line 17: found unexpected end of stream

    配置文件上传到百度云盘,文章都是复制粘贴 难免有格式会出错,自己多研究下    链接: https://pan.baidu.com/s/1qK1-Ei7Q_LhLzNqdD6Cm7A 提取码: 9grx 复制这段内容后打开百度网盘手机App,操作更方便哦   

    • 运维开源项目互助社区—致敬开源
      6
        立即登录 立即注册 
返回