kubernetes编排文件示例

编排文件生成网址：https://www.kubebiz.com/

mysql单机

需要一个配置文件,内容不会就用默认的即可

my.cnf

[mysqld]pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
datadir         = /var/lib/mysqlsymbolic-links=0
sql-mode=ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION

mysql-deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:name: my-mysqlnamespace: default
spec:replicas: 1selector:matchLabels:app: my-mysqltemplate:metadata:labels:app: my-mysqlspec:containers:- name: my-mysqlimage: 'mysql:8.0.30'imagePullPolicy: IfNotPresentenv:- name: MYSQL_ROOT_PASSWORDvalue: Jeol@1201- name: MYSQL_USERvalue: test- name: MYSQL_PASSWORDvalue: Jeol@1201ports:- containerPort: 3306protocol: TCPname: httpvolumeMounts:- name: my-mysql-datamountPath: /var/lib/mysql- name: mysql-confmountPath: /etc/mysql/mysql.conf.dvolumes:- name: my-mysql-datahostPath:# 确保文件所在目录成功创建。path: /app/kube/mysql/datatype: DirectoryOrCreate- name: mysql-confhostPath:path: /app/kube/mysql/conf/mysql.conf.dtype: DirectoryOrCreate

mysql-svc.yaml

apiVersion: v1
kind: Service
metadata:name: mysql-exportnamespace: default
spec:type: NodePortselector:app: my-mysqlports:- port: 3306targetPort: 3306nodePort: 32306

nacos集群

官方：https://nacos.io/zh-cn/docs/v2/quickstart/quick-start-kubernetes.html

其他参考：https://www.likecs.com/show-305484280.html https://blog.csdn.net/warrah/article/details/106488733

1、nfs服务端安装

nfs服务端安装完成，编辑好规则，共享目录为/data/nfs-volume nfs服务的ip为 k8s-node1

2、部署NFS StorageClass

1、项目地址(有部分yaml文件从此项目获取)，可以手动下载，再上传，下面的部分命令是在下载的项目下执行

git clone https://github.com/nacos-group/nacos-k8s.git

2、角色设置

kubectl create -f deploy/nfs/rbac.yaml

如果您的K8S命名空间不是默认的，请在创建RBAC之前执行以下脚本（不执行脚本，你直接再rbac.yaml中加上namesapce配置，效果一致，脚本更快）

kubectl create ns nacos

sed -i "s/namespace:.*/namespace: nacos/g" ./deploy/nfs/rbac.yaml

3.NFS-Client Provisioner部署，用于动态生成nfs类型的pv

kubectl create -f deploy/nfs/deployment.yaml -n nacos

# 执行前修改参照部署的nfs服务，修改如下
apiVersion: v1
kind: ServiceAccount
metadata:name: nfs-client-provisioner
---
kind: Deployment
apiVersion: apps/v1
metadata:name: nfs-client-provisioner
spec:replicas: 1strategy:type: Recreateselector:matchLabels:app: nfs-client-provisionertemplate:metadata:labels:app: nfs-client-provisionerspec:serviceAccount: nfs-client-provisionercontainers:- name: nfs-client-provisionerimage: quay.io/external_storage/nfs-client-provisioner:latestvolumeMounts:- name: nfs-client-rootmountPath: /persistentvolumesenv:- name: PROVISIONER_NAMEvalue: fuseim.pri/ifs- name: NFS_SERVERvalue: k8s-node1  #修改- name: NFS_PATHvalue: /data/nfs-volume/nacos/nfs-share #修改volumes:- name: nfs-client-rootnfs:server: k8s-node1  #修改path: /data/nfs-volume/nacos/nfs-share # 修改

4、storageclass创建，用于配合nfs-client-provisioner动态创建持久卷

kubectl create -f deploy/nfs/class.yaml -n nacos

记录一下内容

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: managed-nfs-storage
provisioner: fuseim.pri/ifs
parameters:archiveOnDelete: "false"

5、验证（根据标签获取对应pod的详情）

kubectl get pod -l app=nfs-client-provisioner -n nacos

3、数据库

可以只部署一个数据库，也可以部署主从数据库，需要说明的是，如果你使用先有的数据库，那么建表语句你可以再安装包的conf目录下找到

一个数据库部署参考，注意修改共享服务端地址和共享目录

kubectl create -f deploy/mysql/mysql-nfs.yaml -n nacos

主从数据库，主要是镜像不同

主

apiVersion: v1
kind: ReplicationController
metadata:name: mysql-masterlabels:name: mysql-master
spec:replicas: 1selector:name: mysql-mastertemplate:metadata:labels:name: mysql-masterspec:containers:- name: mysql-masterimage: nacos/nacos-mysql-master:latestports:- containerPort: 3306volumeMounts:- name: mysql-master-datamountPath: /var/lib/mysqlenv:- name: MYSQL_ROOT_PASSWORDvalue: "root"- name: MYSQL_DATABASEvalue: "nacos_devtest"- name: MYSQL_USERvalue: "nacos"- name: MYSQL_PASSWORDvalue: "nacos"- name: MYSQL_REPLICATION_USERvalue: 'nacos_ru'- name: MYSQL_REPLICATION_PASSWORDvalue: 'nacos_ru'          volumes:- name: mysql-master-datanfs:server: k8s-node1path: /data/nfs-volume/nacos/mysql-master
---
apiVersion: v1
kind: Service
metadata:name: mysql-masterlabels:name: mysql-master
spec:type: NodePortports:- port: 3306targetPort: 3306nodePort: 32308selector:name: mysql-master

从

apiVersion: v1
kind: ReplicationController
metadata:name: mysql-slavelabels:name: mysql-slave
spec:replicas: 1selector:name: mysql-slavetemplate:metadata:labels:name: mysql-slavespec:containers:- name: mysql-slaveimage: nacos/nacos-mysql-slave:latestports:- containerPort: 3307volumeMounts:- name: mysql-slave-datamountPath: /var/lib/mysqlenv:- name: MYSQL_ROOT_PASSWORDvalue: "root"- name: MYSQL_USERvalue: "nacos"- name: MYSQL_PASSWORDvalue: "nacos" - name: MYSQL_REPLICATION_USERvalue: 'nacos_ru'- name: MYSQL_REPLICATION_PASSWORDvalue: 'nacos_ru'volumes:- name: mysql-slave-datanfs:server: k8s-node1path: /data/nfs-volume/nacos/mysql-slave
---
apiVersion: v1
kind: Service
metadata:name: mysql-slavelabels:name: mysql-slave
spec:type: NodePortports:- port: 3307targetPort: 3307nodePort: 32307selector:name: mysql-slave

应用生效,需要说明的是，从库无法连接，可能需要进入从库容器内部设置权限，还有就是镜像内部相关的表是已经创建好的，不需要在创建

kubectl create -f deploy/mysql/mysql-nfs-master.yaml -n nacos
kubectl create -f deploy/mysql/mysql-nfs-slave.yaml -n nacos

4、部署nacos

修改deploy/nacos/nacos-pvc-nfs.yaml和数据库配置文件保持一致即可！

主从数据可配置，

data:mysql.master.db.name: "主库名称"mysql.master.port: "主库端口"mysql.slave.port: "从库端口"mysql.master.user: "主库用户名"mysql.master.password: "主库密码"

kubectl create -f deploy/nacos/nacos-pvc-nfs.yaml -n nacos

pvc一直处于pending状态，

#  pvc日志Normal  ExternalProvisioning  4s (x22 over 5m19s)  persistentvolume-controller  waiting for a volume to be created, either by external provisioner "fuseim.pri/ifs" or manually created by system administrator# pvc日志
provision "nacos/data-nacos-0" class "managed-nfs-storage": unexpected error getting claim reference: selfLink was empty, can't make reference

​ 解决方式一

​ 这样之后，我的kubectl命令直接启动执行不了了，放弃

# /etc/kubernetes/manifests/kube-apiserver.yamlspec:containers:- command:- kube-apiserver...- --feature-gates=RemoveSelfLink=false # 增加

解决方式二

使用不基于SelfLink的provisioner

参考：https://blog.csdn.net/echizao1839/article/details/125766826

# 国外镜像
gcr.io/k8s-staging-sig-storage/nfs-subdir-external-provisioner:v4.0.0
# 使用替换的国内镜像
registry.cn-beijing.aliyuncs.com/pylixm/nfs-subdir-external-provisioner:v4.0.0
# 替换nfs deploy中的镜像

内存不足

0/3 nodes are available: 1 Insufficient memory, 1 node(s) didn't match pod anti-affinity rules, 1 node(s) had untolerated taint {node-role.kubernetes.io/control-plane: }. preemption: 0/3 nodes are available: 1 Preemption is not helpful for scheduling, 2 No preemption victims found for incoming pod.

# 其中一台机器内存不足，所以这一块调小一点，无语，并且k8s的节点需要有三个，当然又时候并不是这个的原因，机器内存不足，该启动不了还是启动不了requests:memory: "500Mi"cpu: "300m"

# 调小之后，其中一致pending的nacos终于进入了初始化阶段，但是一直无法进入就绪状态，可以强制部署在内存充足的机器上，在deploy上添加spec:nodeName: k8s-node1

成功截图

5、通过ingress 访问

$ vim nacos-ingress.yaml
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:name: nacos-headlessnamespace: nacos
spec:rules:- host: nacos-web.nacos-demo.comhttp:paths:- path: /pathType: Prefixbackend:service:name: nacos-headlessport:number: 8848

我ingress似乎有问题，天翼云端口问题，总之访问不了，配置仅供参考

6、尝试直接通过service暴露端口

添加nodePort访问，访问无响应

nacos pod内部日志报错,这个似乎不行，应该和StatefulSet的模式有关吧，改成nodePort就不行，不改的话日志还是正常的，无语，所以还是需要通过ingress访问

service 访问StatefulSet 似乎需要设置clusterIP: None， 然后service会生成StatefulSet 对应的DNS地址，然后sevice改了模式之后，导致nacos pod之间互相之间访问不了，构成不了集群

2022/11/05 10:42:11 Have not found myself in list yet.
My Hostname: nacos-0.nacos-headless.nacos.svc.cluster.local
Hosts in list: nacos-headless.nacos.svc.cluster.loca

nacos单机

yaml文件如下，注意，持久卷使用的是nfs,你可以替换成集群，部分配置可能多余，毕竟是通过集群的yaml文件改的，数据库也换了，初始化sql从安装包中获取

---
apiVersion: v1
kind: Service
metadata:name: nacos-headlesslabels:app: nacos-headless
spec:type: ClusterIPclusterIP: Noneports:- port: 8848name: servertargetPort: 8848- port: 9848name: client-rpctargetPort: 9848- port: 9849name: raft-rpctargetPort: 9849## 兼容1.4.x版本的选举端口- port: 7848name: old-raft-rpctargetPort: 7848selector:app: nacos
---
apiVersion: v1
kind: ConfigMap
metadata:name: nacos-cm
data:mysql.host: "10.127.1.12"mysql.db.name: "nacos_devtest"mysql.port: "3306"mysql.user: "nacos"mysql.password: "passwd"
---
apiVersion: apps/v1
kind: StatefulSet
metadata:name: nacos
spec:serviceName: nacos-headlessreplicas: 3template:metadata:labels:app: nacosannotations:pod.alpha.kubernetes.io/initialized: "true"spec:affinity:podAntiAffinity:requiredDuringSchedulingIgnoredDuringExecution:- labelSelector:matchExpressions:- key: "app"operator: Invalues:- nacostopologyKey: "kubernetes.io/hostname"containers:- name: k8snacosimagePullPolicy: Alwaysimage: nacos/nacos-server:latestresources:requests:memory: "2Gi"cpu: "500m"ports:- containerPort: 8848name: client- containerPort: 9848name: client-rpc- containerPort: 9849name: raft-rpc- containerPort: 7848name: old-raft-rpcenv:- name: NACOS_REPLICASvalue: "3"- name: MYSQL_SERVICE_HOSTvalueFrom:configMapKeyRef:name: nacos-cmkey: mysql.host- name: MYSQL_SERVICE_DB_NAMEvalueFrom:configMapKeyRef:name: nacos-cmkey: mysql.db.name- name: MYSQL_SERVICE_PORTvalueFrom:configMapKeyRef:name: nacos-cmkey: mysql.port- name: MYSQL_SERVICE_USERvalueFrom:configMapKeyRef:name: nacos-cmkey: mysql.user- name: MYSQL_SERVICE_PASSWORDvalueFrom:configMapKeyRef:name: nacos-cmkey: mysql.password- name: MODEvalue: "cluster"- name: NACOS_SERVER_PORTvalue: "8848"- name: PREFER_HOST_MODEvalue: "hostname"- name: NACOS_SERVERSvalue: "nacos-0.nacos-headless.default.svc.cluster.local:8848 nacos-1.nacos-headless.default.svc.cluster.local:8848 nacos-2.nacos-headless.default.svc.cluster.local:8848"selector:matchLabels:app: nacos

访问如图

开启权限认证

默认是允许所有客户端 进行操作，无需密码，添加环境变量

            - name: NACOS_AUTH_ENABLEvalue: "true"

dubbo-admin

访问是成功了，nacos也连接上了，但是查询不到服务列表，可能版本不对，需要下载源码本地跑一下试试，为啥版本是0.3.0， 这个0.5.0的不好使，而且其默认的最后一个版本就是0.3.0

apiVersion: apps/v1
kind: Deployment
metadata:name: dubbo-adminnamespace: default
spec:replicas: 1selector:matchLabels:app: dubbo-admintemplate:metadata:labels:app: dubbo-adminspec:containers:- name: dubbo-adminimage: apache/dubbo-adminimagePullPolicy: IfNotPresentcommand: [ "/bin/bash", "-ce", "java -Dadmin.registry.address=nacos://nacos:nacos@61.171.5.6:30848 -Dadmin.config-center=nacos://nacos:nacos@61.171.5.6:30848 -Dadmin.metadata-report.address=nacos://nacos:nacos@61.171.5.6:30848 -jar /app.jar"]readinessProbe:tcpSocket:port: 8080initialDelaySeconds: 60 periodSeconds: 20
---
apiVersion: v1
kind: Service
metadata:name: dubbo-admin-servicenamespace: default
spec:type: NodePortselector:app: dubbo-adminports:- port: 8089targetPort: 8080nodePort: 31990

关联信息

• 关联的主题：
• 上一篇：
• 下一篇：
• image: 20221021/1
• 转载自：