Sign inSign up

weiyigeek/nfs-subdir-external-provisioner

By weiyigeek

•Updated almost 3 years ago

Image
0

2.0K

weiyigeek/nfs-subdir-external-provisioner repository overview

关注「WeiyiGeek」公众号⁠

设为「特别关注」每天带你玩转网络安全运维、应用开发、物联网IOT学习！

希望各位看友【关注、点赞、评论、收藏、投币】，助力每一个梦想。

[-] 作者主页：https://www.weiyigeek.top⁠

[-] 作者博客：https://blog.weiyigeek.top⁠

⁠1.集群中基于nfs的provisioner的动态持卷环境部署

描述: 在K8S集群中我们常常会使用动态持久卷对应用数据进行持久化保存，例如应用配置、依赖插件、应用数据以及应用访问日志等，所以动态持久卷的重要性不言而喻，此小节将讲解如何搭建以及nfs存储服务以及使用kubernetes-sigs提供的[nfs-subdir-external-provisioner] (https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner)项目针对已存在的⁠ nfs 服务器进行 provisioner 部署。


Q: 什么是nfs-subdir-external-provisioner?

NFS subdir external provisioner 是一个自动配置器，它使用您现有的和已配置的 NFS 服务器来支持通过 Persistent Volume Claims 动态配置 Kubernetes Persistent Volumes。 持久卷配置为 ${namespace}-${pvcName}-${pvName}。

温馨提示：在进行此环境搭建时，请注意您必须已经有配置好的 NFS 服务器，为了保持与早期部署文件的向后兼容性，NFS Client Provisioner 的命名在部署 YAML 中保留为 nfs-storage-provisioner。

项目地址: https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner⁠


步骤 01.我在一台Ubuntu 20.04主机上配置 NFS 存储服务器(IP地址为10.20.176.102)，如果你已有存储服务器提供的NFS服务器则可以略过此步骤，其它操作系统安装部署请看出我的这篇博客文章[NFS网络文件系统基础配置与使用] (https://blog.weiyigeek.top/2019/5-16-104.html⁠)

# Step1.Install Required Packages（Server）
$ apt-get update
$ apt-get install nfs-kernel-server rpcbind

# Step2.Configure NFS Server
# [根据需求配置] 设置允许连接的ip, Example Allow 192.168.1.0/24 to be Accessed on Network
$ nano /etc/hosts.allow
portmap: 192.168.1.0/24   

# [根据需求配置] idmapd 配置, 此处配置端口为40000
$ nano /etc/default/nfs-common
NEED_IDMAPD=YES  
STATDOPTS="--port 40000"

# [根据需求配置] 内核服务配置,此处mountd服务端口为40001
$ nano /etc/default/nfs-kernel-server
RPCMOUNTDOPTS="--manage-gids --port 40001"

# NFS 服务器共享路径配置
$ nano /etc/exports
# Example for NFSv2 and NFSv3:
# /srv/homes       hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
# Example for NFSv4:
# /srv/nfs4        gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes  gss/krb5i(rw,sync,no_subtree_check)
/app/storage/nfs *(rw,sync,no_root_squash,no_subtree_check)

# Step3.创建NFS配置需要共享的目录
mkdir -vp /app/storage/nfs

# Step4.自启动RPC服务于nfs共享服务
systemctl enable rpcbind nfs-server.service
systemctl start rpcbind nfs-server.service

# Step5.更新nfs配置文件 (nano /etc/exports)
  exportfs -a

# Step6.Ubuntu防火墙规则配置 （PS: 通过rpcinfo命令可以查看 NFS 相关的端口）
ufw allow 111,2049,40000,40001/tcp
ufw reload

步骤 02.【所有节点】在客户端尝试挂载搭建的NFS服务器，并分别在节点机器挂载到/storage/nfs目录中

# 查看 NFS 挂载目录
showmount -e 127.0.0.1
  # Export list for 127.0.0.1:
  # /app/storage/nfs *

# 挂载 NFS 服务器到指定节点目录的/storage/nfs目录
mkdir -vp /storage/nfs
# 临时挂载
mount -t nfs 10.20.176.102:/app/storage/nfs /storage/nfs
# 永久挂载将挂载配置写入到/etc/fstab中
tee -a /etc/fstab <<"EOF"
10.20.176.102:/app/storage/nfs /storage/nfs nfs defaults 0 0
EOF
mount -a 

# 查看挂载信息
$ mount -l
$ df -Th | grep "/storage/nfs"
10.20.176.102:/app/storage/nfs nfs4   97G   11G   82G  12% /storage/nfs

# 如果不使用了则可执行如下命令进行卸载挂载点
umount /storage/nfs

WeiyiGeek.挂载NFS服务到各个节点


步骤 03.helm3 部署工具快速安装

# helm3 安装
$ wget https://get.helm.sh/helm-v3.9.0-linux-amd64.tar.gz && tar -zxf helm-v3.9.0-linux-amd64.tar.gz
$ tar -zxf helm-v3.9.0-linux-amd64.tar.gz
$ cp linux-amd64/helm /usr/local/bin/helm3

步骤 04.在Github中下载nfs-subdir-external-provisioner的release压缩包或者使用helm3指定仓库进行安装。

  • 方式1.release 压缩包
$ wget https://github.com/kubernetes-sigs/nfs-subdir-external-provisioner/releases/download/nfs-subdir-external-provisioner-4.0.16/nfs-subdir-external-provisioner-4.0.16.tgz
$ tar -zxvf nfs-subdir-external-provisioner-4.0.16.tgz
# 按照指定需求编辑如下values值
$ egrep -v '^$|#' values.yaml
replicaCount: 1
strategyType: Recreate
image:
  repository: registry.cn-hangzhou.aliyuncs.com/weiyigeek/nfs-subdir-external-provisioner
  tag: v4.0.2
  pullPolicy: IfNotPresent
imagePullSecrets: []
nfs:
  server: 10.20.176.102
  path: /app/storage/nfs
  mountOptions:
  volumeName: nfs-subdir-external-provisioner-root
  reclaimPolicy: Retain
storageClass:
  create: true
  provisionerName: cluster.local/nfs-storage-subdir-provisioner
  defaultClass: false
  name: nfs-storage
  allowVolumeExpansion: true
  reclaimPolicy: Delete
  archiveOnDelete: true
  onDelete:
  pathPattern:
  accessModes: ReadWriteOnce
  annotations: {}
leaderElection:
  enabled: true
rbac:
  create: true
podSecurityPolicy:
  enabled: false
podAnnotations: {}
podSecurityContext: {}
securityContext: {}
serviceAccount:
  create: true
  annotations: {}
  name:
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
labels: {}

# 指定下载解压目录进行chat图表安装
$ helm3 install nfs-storage nfs-subdir-external-provisioner/
  NAME: nfs-storage
  LAST DEPLOYED: Mon Jun 20 15:41:50 2022
  NAMESPACE: default
  STATUS: deployed
  REVISION: 1
  TEST SUITE: None
  • 方式2.helm3指定仓库
$ helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/
$ helm nfs-storage nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
  --set nfs.server=10.20.176.102 \
  --set nfs.path=/app/storage/nfs

# 更新
$ helm3 upgrade nfs-storage nfs-subdir-external-provisioner/

温馨提示: 由于nfs-subdir-external-provisioner镜像是k8s.gcr.io域名下国内可能无法拉取，此处我已经拉取到阿里云镜像仓库中registry.cn-hangzhou.aliyuncs.com/weiyigeek/nfs-subdir-external-provisioner:v4.0.2，如果你需要自行拉取请参考此篇使用Aliyun容器镜像服务对海外镜像仓库中镜像进行拉取构建⁠ ()


步骤 05.查看部署的nfs-storage动态持久卷以及其pod状态

$ helm3 list
  NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                                    APP VERSION
  nfs-storage     default         1               2022-06-20 15:41:50.960642233 +0800 CST deployed        nfs-subdir-external-provisioner-4.0.16   4.0.2

$ kubectl get storageclasses.storage.k8s.io
  NAME          PROVISIONER                                    RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
  nfs-storage   cluster.local/nfs-storage-subdir-provisioner   Delete          Immediate           true                   17s

$ kubectl get pod
  NAME                                                           READY   STATUS    RESTARTS   AGE
  nfs-storage-nfs-subdir-external-provisioner-77f4d85f74-kljck   1/1     Running   0          33s

步骤 06.如想卸载helm3安装的nfs-storage动态卷执行如下命令即可。

$ helm3 uninstall nfs-storage
release "nfs-storage" uninstalled

至此，基于NFS的provisioner 动态持久卷的安装部署到此结束。

温馨提示: 如果需要部署多个nfs持久卷, 则我们需要更改values.yaml中如下字段，并使用不同的名称。

# values.yaml
nfs:
  server: 10.20.176.102
  path: /app/storage/nfs/pvc/local
  volumeName: nfs-subdir-external-provisioner-local
  ....
storageClass:
  ....
  provisionerName: cluster.local/nfs-local-subdir-provision  # 关键点(提供者不能一样)
  name: nfs-local

# 例如,此处创建两个nfs动态卷即nfs-local与nfs-dev。
helm3 install nfs-local nfs-subdir-external-provisioner/
helm3 install nfs-dev nfs-subdir-external-provisioner-dev/

# helm 部署结果查看
helm3 list
  # NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                                   APP VERSION
  # nfs-dev         default         1               2022-07-14 11:27:33.997601363 +0800 CST deployed        nfs-subdir-external-provisioner-4.0.16  4.0.2
  # nfs-local       default         1               2022-07-14 11:28:02.49579496 +0800 CST  deployed        nfs-subdir-external-provisioner-4.0.16  4.0.2

# storageclasses 查看
kubectl get storageclasses.storage.k8s.io
  # NAME                PROVISIONER                                  RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
  # nfs-dev (default)   cluster.local/nfs-dev-subdir-provisioner     Delete          Immediate           true                   4m36s
  # nfs-local           cluster.local/nfs-local-subdir-provisioner   Delete          Immediate           true                   4m8s

# nfs-subdir-external-provisioner pod 查看
kubectl get pod
  # NAME                                                         READY   STATUS    RESTARTS   AGE
  # nfs-dev-nfs-subdir-external-provisioner-cf7684f8b-fzl9h      1/1     Running   0          5m4s
  # nfs-local-nfs-subdir-external-provisioner-6f97d44bb8-424tk   1/1     Running   0          4m36s

正常情况下显示出的日志信息:

$ kubectl logs -f nfs-dev-nfs-subdir-external-provisioner-cf7684f8b-fzl9h
  # I0714 03:27:35.390909       1 leaderelection.go:242] attempting to acquire leader lease  default/cluster.local-nfs-dev-subdir-provisioner...
  # I0714 03:27:35.400777       1 leaderelection.go:252] successfully acquired lease default/cluster.local-nfs-dev-subdir-provisioner
  # I0714 03:27:35.400856       1 event.go:278] Event(v1.ObjectReference{Kind:"Endpoints", Namespace:"default", Name:"cluster.local-nfs-dev-subdir-provisioner", UID:"34019115-d09e-433d-85d6-c254c5492ce5", APIVersion:"v1", ResourceVersion:"5510886", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' nfs-dev-nfs-subdir-external-provisioner-cf7684f8b-fzl9h_b0aaa93a-f7fc-4931-b0cf-e04f42cefd9a became leader
  # I0714 03:27:35.400943       1 controller.go:820] Starting provisioner controller cluster.local/nfs-dev-subdir-provisioner_nfs-dev-nfs-subdir-external-provisioner-cf7684f8b-fzl9h_b0aaa93a-f7fc-4931-b0cf-e04f42cefd9a!
  # I0714 03:27:35.501130       1 controller.go:869] Started provisioner controller cluster.local/nfs-dev-subdir-provisioner_nfs-dev-nfs-subdir-external-provisioner-cf7684f8b-fzl9h_b0aaa93a-f7fc-4931-b0cf-e04f42cefd9a!

【WeiyiGeek Blog 个人博客 - 为了能到远方，脚下的每一步都不能少 】

欢迎各位志同道合的朋友一起学习交流【点击加入交流群⁠】，如文章有误请在下方留下您宝贵的经验知识!

作者主页: 【 https://weiyigeek.top⁠】 博客地址: 【 https://blog.weiyigeek.top⁠ 】

WeiyiGeek Blog 博客 - 为了能到远方，脚下的每一步都不能少

专栏书写不易，如果您觉得这个专栏还不错的，请给这篇专栏 【点个赞、投个币、收个藏、关个注，转个发，留个言】(人间六大情)，这将对我的肯定，谢谢！。

  • echo "【点个赞】，动动你那粗壮的拇指或者芊芊玉手，亲！"

  • printf("%s", "【投个币】，万水千山总是情，投个硬币行不行，亲！")

  • fmt.Printf("【收个藏】，阅后即焚不吃灰，亲！")

  • console.info("【转个发】，让更多的志同道合的朋友一起学习交流，亲！")

  • System.out.println("【关个注】，后续浏览查看不迷路哟，亲！")

  • cout << "【留个言】，文章写得好不好、有没有错误，一定要留言哟，亲! " << endl;

更多网络安全、系统运维、应用开发、物联网实践、网络工程、全栈文章，尽在 https://blog.weiyigeek.top⁠ 之中，谢谢各位看又支持！

Tag summary

Content type

Image

Digest

sha256:4dc0d27b8…

Size

17.1 MB

Last updated

about 4 years ago

docker pull weiyigeek/nfs-subdir-external-provisioner