将jenkins部署到minikube上，并提供外网访问

相关minikube安装和操作参考：

• https://blog.iwiki.ink/archives/ubuntu%E5%AE%89%E8%A3%85minikubekubernetes
• https://blog.iwiki.ink/archives/minikube-qi-dong-dashboard-bing-ke-gong-wai-wang-fang-wen

jenkins部署到minikube参考官方文档（跟官方走总不会翻车🤣）：

• https://www.jenkins.io/doc/book/installing/kubernetes/

1.安装jenkins

官方已经提供了jenkins部署所需的主要yaml文件，在：https://github.com/scriptcamp/kubernetes-jenkins
这里主要讲解下每个文件的作用

1.1 创建namespace

kubectl create namespace devops-tools

此步非必需，为了方便管理，用namespace区分开

1.2 配置集群权限 serviceAccount.yaml

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: jenkins-admin
rules:
  - apiGroups: [""]
    resources: ["*"]
    verbs: ["*"]
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins-admin
  namespace: devops-tools
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: jenkins-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: jenkins-admin
subjects:
- kind: ServiceAccount
  name: jenkins-admin
  namespace: devops-tools

这个文件主要是服务于集群权限控制，这个直接照官方来好了。注意namespace一致即可

# 执行如下命令生效
kubectl apply -f serviceAccount.yaml

1.3 配置挂载数据卷 volume.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: jenkins-pv-volume
  labels:
    type: local
spec:
  storageClassName: local-storage
  claimRef:
    name: jenkins-pv-claim
    namespace: devops-tools
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  local:
    path: /mnt
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - worker-node01
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: jenkins-pv-claim
  namespace: devops-tools
spec:
  storageClassName: local-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 3Gi

注意下node，worker-node01这个值需要跟着实际的node走，minikube部署那这个值就是minikube，k8s的话，需要执行

kubectl get nodes

才能知道node信息

这里的path: /mnt一定要重点注意，它挂载是挂载到minikube虚拟机环境中的地址，我这里是用的docker

myPc@myPc:/mnt$ docker ps
CONTAINER ID   IMAGE                                                                 COMMAND                  CREATED        STATUS          PORTS                                                                                                                                  NAMES
f066afd33376   registry.cn-hangzhou.aliyuncs.com/google_containers/kicbase:v0.0.36   "/usr/local/bin/entr…"   27 hours ago   Up 21 minutes   127.0.0.1:49157->22/tcp, 127.0.0.1:49156->2376/tcp, 127.0.0.1:49155->5000/tcp, 127.0.0.1:49154->8443/tcp, 127.0.0.1:49153->32443/tcp   minikube
myPc@myPc:/mnt$ docker exec -it minikube /bin/bash
root@minikube:/# ls
Release.key  bin  boot  data  dev  docker.key  etc  home  kic.txt  kind  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var
root@minikube:/# cd mnt/
root@minikube:/mnt# ls
config.xml               hudson.model.UpdateCenter.xml     jobs              nodes    queue.xml.bak  secret.key.not-so-secret  updates      users
copy_reference_file.log  jenkins.telemetry.Correlator.xml  nodeMonitors.xml  plugins  secret.key     secrets                   userContent  war

看看上面的输出，你应该懂了吧😂
你可能在想，要怎么才能把数据挂载到宿主机上呢？可以使用minikube mount命令

# 例如
minkube mount /mnt:/mnt

但是！但是！到生产集群环境下，每台机器都挂载到自己的宿主机上，想想也太扯了，所以要统一数据文件持久化到一台固定的机器上，方便维护。这里提示下可以考虑NFS（网络文件系统）技术

执行如下命令生效

kubectl create -f volume.yaml

1.4 部署 deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jenkins
  namespace: devops-tools
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jenkins-server
  template:
    metadata:
      labels:
        app: jenkins-server
    spec:
      securityContext:
            fsGroup: 1000
            runAsUser: 1000
      serviceAccountName: jenkins-admin
      containers:
        - name: jenkins
          image: jenkins/jenkins:lts
          resources:
            limits:
              memory: "2Gi"
              cpu: "1000m"
            requests:
              memory: "500Mi"
              cpu: "500m"
          ports:
            - name: httpport
              containerPort: 8080
            - name: jnlpport
              containerPort: 50000
          livenessProbe:
            httpGet:
              path: "/login"
              port: 8080
            initialDelaySeconds: 90
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 5
          readinessProbe:
            httpGet:
              path: "/login"
              port: 8080
            initialDelaySeconds: 60
            periodSeconds: 10
            timeoutSeconds: 5
            failureThreshold: 3
          volumeMounts:
            - name: jenkins-data
              mountPath: /var/jenkins_home
      volumes:
        - name: jenkins-data
          persistentVolumeClaim:
              claimName: jenkins-pv-claim

这里的volumeMounts就是关联到volume.yaml的配置,意思是讲jenkins容器中的 /var/jenkins_home地址挂载到volume.yaml中配置的/mnt地址，类似于docker的-v数据参数，只不过这里挂载也只是挂载到minikube虚拟环境中

执行如下命令生效

kubectl apply -f deployment.yaml

1.5 对外暴露服务 service.yaml

apiVersion: v1
kind: Service
metadata:
  name: jenkins-service
  namespace: devops-tools
  annotations:
      prometheus.io/scrape: 'true'
      prometheus.io/path:   /
      prometheus.io/port:   '8080'
spec:
  selector:
    app: jenkins-server
  type: NodePort
  ports:
    - port: 8080
      targetPort: 8080
      nodePort: 32000      

执行如下命令生效

kubectl apply -f service.yaml

该文件是以nodePort的形式向外暴露服务的，效果等于

kubectl expose deployment jenkins --port=32000 --type=NodePort

执行完成后你就可以以任一node的ip地址+32000端口去访问jenkins了，这个32000端口可自行修改
如果不知道端口，可以执行

minikube service jenkins-service -n=devops-tools

查看URL信息，这个命令也会自动打开浏览器，ssh下会报错

2.配置外网可达

执行完上面所有的步骤后外网依旧没法访问，只能宿主机上访问。官方也提供了相应的解决方案，不过我这里是用的是nginx反向代理，安装nginx

sudo apt-get install nginx

然后

sudo vim /etc/nginx/conf.d/jenkins.conf

jenkins.conf内容如下

upstream jenkins {
    server 192.168.49.2:32000;
}

server {
  listen    8080;
  server_name 192.168.43.18;

  #charset koi8-r;
  #access_log /var/log/nginx/log/showdoc.access.log main;

  location / {
    proxy_pass http://jenkins;
    add_header Access-Control-Allow-Origin *;        #允许跨域
    add_header Access-Control-Allow-Headers X-Requested-With;
    add_header Access-Control-Allow-Methods GET,POST,OPTIONS;
  }
}

upstream中的ip和端口为NodePort的地址
server中配置宿主机的ip和端口，配置完后执行

sudo service nginx restart

重启nginx即可，然后就能外网访问到jenkins了

补充

任何时候踩坑都没有一蹴而就的情况🐶
附上删除命令，等执行错了的时候可以回滚

关于数据卷的

# 用于删除volume.yaml中配置的StorageClass
kubectl delete sc local-storage -n=devops-tools 

# 用于删除volume.yaml中配置的PersistentVolume
kubectl delete pv jenkins-pv-volume -n=devops-tools 

# 用于删除volume.yaml中配置的PersistentVolume
kubectl delete pvc jenkins-pv-claim -n=devops-tools 

关于service的

# 用于删除service ，svc等价于service，缩写
kubectl delete svc jenkins-service -n=devops-tools 

关于deployment的

# 用于删除service ，svc等价于service，缩写
kubectl delete deployment jenkins -n=devops-tools 

具体的删除可以参考这篇文章：https://www.modb.pro/db/381010