云原生应用也就是面向“云”而设计的应用，在使用云原生技术后，开发者无需考虑底层的技术实现，可以充分发挥云平台的弹性和分布式优势，实现快速部署、按需伸缩、不停机交付等。 今天我们通过把wordpress 和Mysql部署到IBM公有云的托管K8S上，来实践如何部署一个应用到k8s的环境，从而为我们后面的云原生开发打下实践的基础。
本实验与k8s官方教程的主要有以下区别：
本实验iaas层基于ibm cloud vpc gen2，k8s官方教程基于本地的minikube。
本实验持久化基于 IBM vpc block storage, 而k8s官网用的是本地storage作为持久化。
相同点：
- 本实验同样使用 dynamic PVC 自动创建相对应的PV。
1. 准备工作
- 由于本实验的iaas层基于ibm vpc gen2， 所以需要先创建对应的vpc实例, 可以使用ibm cloud gui console 快速创建vpc。
- Push image 到 IBM Registry 使用以下命令
ibmcloud cr login # 登录 IBM Registry。
ibmcloud cr namespace-add spark_test # 添加一个namespace
ibmcloud cr namespace-list -v # list namespace
docker pull wordpress:4.8-apache # 从dockerhub下载image 到本地
docker pull mysql:5.6
docker tag wordpress:4.8-apache us.icr.io/spark_test/wordpress:4.8-apache # tag imag
docker tag mysql:5.6 us.icr.io/spark_test/mysql:5.6
docker push wordpress:4.8-apache us.icr.io/spark_test/wordpress:4.8-apache # 推送image到仓库对应的namespace下
docker push mysql:5.6 us.icr.io/spark_test/mysql:5.6远端仓库的格式为 {zone}.us.icr.io,当登录Registry 会提示远端仓库的信息，这里只有做相应的替换就可以了。
查看image是否上传成功
localhost:IKS spark$ ibmcloud cr image-list
Listing images...
Repository Tag Digest Namespace Created Size Security status
us.icr.io/spark_test/busybox 1.28.4 74f634b1bc1b spark_test 2 years ago 723 kB Unsupported OS
us.icr.io/spark_test/mysql 5.6 574b626bb891 spark_test 1 week ago 103 MB 4 Issues
us.icr.io/spark_test/wordpress 4.8-apache b40c224a95cd spark_test 3 years ago 170 MB 366 Issues- 登录 ibm cloud ibmcloud login -a cloud.ibm.com -r us-south -g friday-you --apikey xxxxx
2. 创建 IKS cluster
- 使用IBM console 创建 cluster cluster 创建好后，进入cluster overview页面获取 access 信息。
- 设置cluster 环境
localhost:IKS spark$ ibmcloud ks cluster config --cluster bvn8bkad0kms4uq0eij0
localhost:IKS spark$ kubectl config current-context
spark-iks-us-south-1-bx2.4x16/bvn8bkad0kms4uq0eij0- 查看cluster nodes，确认环境已经可用
localhost:IKS spark$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
10.240.0.4 Ready <none> 6h43m v1.18.13+IKS3. 创建Secret
https://kubernetes.io/zh/docs/concepts/configuration/secret/
#config/mysql-pass.yaml
apiVersion: v1
kind: Secret
metadata:
name: mysql-pass
type: kubernetes.io/basic-auth
stringData:
password: t0p-Secret #这个mysql db root password
localhost:IKS spark$ kubectl apply -f config/mysql-pass.yaml
secret/mysql-pass created
localhost:IKS spark$ kubectl get secret mysql-pass
NAME TYPE DATA AGE
mysql-pass kubernetes.io/basic-auth 1 65s
localhost:IKS spark$ kubectl describe secret mysql-pass
Name: mysql-pass
Namespace: default
Labels: <none>
Annotations:
Type: kubernetes.io/basic-auth
Data
====
password: 10 bytes
localhost:IKS spark$4. （可选） 创建自己的 storageclasses
free plan 默认没有定义好的storage classes，需要自己定义。
localhost:IKS spark$ kubectl apply -f config/storage-classes.yaml
storageclass.storage.k8s.io/my-ibmc-vpc-block-retain-5iops-tier created
localhost:IKS spark$ kubectl get storageclasses
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
ibmc-vpc-block-10iops-tier (default) vpc.block.csi.ibm.io Delete Immediate false 41m
ibmc-vpc-block-5iops-tier vpc.block.csi.ibm.io Delete Immediate false 41m
ibmc-vpc-block-custom vpc.block.csi.ibm.io Delete Immediate false 41m
ibmc-vpc-block-general-purpose vpc.block.csi.ibm.io Delete Immediate false 41m
ibmc-vpc-block-metro-10iops-tier vpc.block.csi.ibm.io Delete WaitForFirstConsumer false 41m
ibmc-vpc-block-metro-5iops-tier vpc.block.csi.ibm.io Delete WaitForFirstConsumer false 41m
ibmc-vpc-block-metro-custom vpc.block.csi.ibm.io Delete WaitForFirstConsumer false 41m
ibmc-vpc-block-metro-general-purpose vpc.block.csi.ibm.io Delete WaitForFirstConsumer false 41m
ibmc-vpc-block-metro-retain-10iops-tier vpc.block.csi.ibm.io Retain WaitForFirstConsumer false 41m
ibmc-vpc-block-metro-retain-5iops-tier vpc.block.csi.ibm.io Retain WaitForFirstConsumer false 41m
ibmc-vpc-block-metro-retain-custom vpc.block.csi.ibm.io Retain WaitForFirstConsumer false 41m
ibmc-vpc-block-metro-retain-general-purpose vpc.block.csi.ibm.io Retain WaitForFirstConsumer false 41m
ibmc-vpc-block-retain-10iops-tier vpc.block.csi.ibm.io Retain Immediate false 41m
ibmc-vpc-block-retain-5iops-tier vpc.block.csi.ibm.io Retain Immediate false 41m
ibmc-vpc-block-retain-custom vpc.block.csi.ibm.io Retain Immediate false 41m
ibmc-vpc-block-retain-general-purpose vpc.block.csi.ibm.io Retain Immediate false 41m
my-ibmc-vpc-block-retain-5iops-tier vpc.block.csi.ibm.io Retain Immediate false 12s5. Create dynamic PVC
动态pvc，通过指定storageClassName来调用storage plugin，由vpc gen2的storage 组件自动分配符合pvc要求的pv资源，并绑定pv到pvc下。
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: my-pvc
spec:
accessModes:
- ReadWriteOnce #读写一次，只能mount到一个pod, 所有的block storage只支持 `ReadWriteOnce` 这一种模式, NFS 支持读写多次（不同的pod mount到一个NFS ）
resources:
requests:
storage: 10Gi # 10G大小
storageClassName: ibmc-vpc-block-retain-5iops-tier # 由于我们指定了pvc 的storage classes， 所以pv会自动创建。 如果需要手动创建pv，这个字段设置为空字符串。
localhost:IKS spark$ kubectl apply -f config/my_pvc.yaml
persistentvolumeclaim/my-pvc created
localhost:IKS spark$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
my-pvc Bound pvc-c09494f1-bfbc-45d7-a20b-1469819644df 10Gi RWO ibmc-vpc-block-retain-5iops-tier 100s
localhost:IKS spark$
localhost:IKS spark$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
my-pvc Pending ibmc-vpc-block-retain-5iops-tier 8s
localhost:IKS spark$ kubectl describe pvc my-pvc
Name: my-pvc
Namespace: default
StorageClass: ibmc-vpc-block-retain-5iops-tier
Status: Bound
Volume: pvc-c09494f1-bfbc-45d7-a20b-1469819644df
Labels: <none>
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
volume.beta.kubernetes.io/storage-provisioner: vpc.block.csi.ibm.io
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 10Gi
Access Modes: RWO
VolumeMode: Filesystem
Mounted By: <none>
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Provisioning 37s vpc.block.csi.ibm.io_ibm-vpc-block-csi-controller-0_538d6bfe-8958-4c9a-af17-0bcc35bf0e0b External provisioner is provisioning volume for claim "default/my-pvc"
Normal ExternalProvisioning 24s (x2 over 37s) persistentvolume-controller waiting for a volume to be created, either by external provisioner "vpc.block.csi.ibm.io" or manually created by system administrator
Normal ProvisioningSucceeded 21s vpc.block.csi.ibm.io_ibm-vpc-block-csi-controller-0_538d6bfe-8958-4c9a-af17-0bcc35bf0e0b Successfully provisioned volume pvc-c09494f1-bfbc-45d7-a20b-1469819644df检查PV 已经被绑定
localhost:IKS spark$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-c09494f1-bfbc-45d7-a20b-1469819644df 10Gi RWO Retain Bound default/my-pvc ibmc-vpc-block-retain-5iops-tier 40s
localhost:IKS spark$6. 创建mysql
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: wordpress-mysql
labels:
app: wordpress
spec:
selector:
matchLabels:
app: wordpress
tier: mysql
strategy:
type: Recreate
template:
metadata:
labels:
app: wordpress
tier: mysql
spec:
s:
- image: us.icr.io/spark_test/mysql:5.6
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef: # 引用secret password
name: mysql-pass
key: password
ports:
-Port: 3306
name: mysql
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: my-pvc # 通过指定pvc， mount到指定的pv7. 创建mysql service
由于mysql是内部使用并且只有一个instance，这里使用headless的方式创建mysql headless 不会为服务分配cluster-ip，同时dns会为每个pod instance分配对应的dns域名
apiVersion: v1
kind: Service
metadata:
name: wordpress-mysql
labels:
app: wordpress
spec:
ports:
- port: 3306
selector:
app: wordpress
tier: mysql
clusterIP: None # 设置cluster ip 为None，表明为headless servicelocalhost:single_instance_wordpress spark$ kubectl exec -it my-deployment-6d78f74d89-kw2dt -- nslookup wordpress-mysql
Server: 172.21.0.10
Address 1: 172.21.0.10 kube-dns.kube-system.svc.cluster.local
Name: 1wordpress-mysq
Address 1: 172.17.59.87 172-17-59-87.wordpress-mysql.default.svc.cluster.local
# ^- headless 不会为服务分配cluster-ip，同时dns会为每个pod instance分配对应的dns域名8. 同理创建 wordpress pvc 和 deployment
kubectl apply -f config/wordpress-pvc.yaml
kubectl apply -f config/wordpress-deployment.yaml9. 通过loadbalance方式暴露 wordpress service
kubectl expose deploy wordpress --port=80 --target-port=80 --type=LoadBalancer --name wordpress-svc
localhost:single_instance_wordpress spark$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 172.21.0.1 <none> 443/TCP 9h
mysql-svc LoadBalancer 172.21.151.23 f9d4dc4c-us-south.lb.appdomain.cloud 3306:30863/TCP 3h4m
wordpress-mysql ClusterIP None <none> 3306/TCP 7h25m
wordpress-svc LoadBalancer 172.21.12.204 09cdb3f4-us-south.lb.appdomain.cloud 80:31380/TCP 5h18m通过分配的域名 f9d4dc4c-us-south.lb.appdomain.cloud 可以直接在浏览器里访问wordpress 如果访问不了，需要在vpc下的default security group 下允许端口到 30000到 32767的tcp流量通过。
10. （option）暴露mysql
本地通过mysql client 连接暴露的mysql svc， 确认wordpress 已经连接到mysql svc，并产生数据表项
kubectl expose deploy wordpress-mysql --port=3306 --target-port=3306 --type=LoadBalancer --name mysql-svclocalhost:IKS spark$ mysql -h f9d4dc4c-us-south.lb.appdomain.cloud -u root --password=t0p-Secret
mysql> SHOW DATABASES;
+---------------------+
| Database |
+---------------------+
| information_schema |
| #mysql50#lost+found |
| mysql |
| performance_schema |
| wordpress |
+---------------------+
5 rows in set (0.27 sec)
mysql> use wordpress;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> tables;
... #省略
mysql> select * from wp_users;
.... #省略
1 row in set (0.32 sec)
mysql> exit;11 查看所有的资源
localhost:single_instance_wordpress spark$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 172.21.0.1 <none> 443/TCP 9h
mysql-svc LoadBalancer 172.21.151.23 f9d4dc4c-us-south.lb.appdomain.cloud 3306:30863/TCP 3h13m
wordpress-mysql ClusterIP None <none> 3306/TCP 7h34m
wordpress-svc LoadBalancer 172.21.12.204 09cdb3f4-us-south.lb.appdomain.cloud 80:31380/TCP 5h27m
localhost:single_instance_wordpress spark$
localhost:single_instance_wordpress spark$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
my-pvc Bound pvc-c09494f1-bfbc-45d7-a20b-1469819644df 10Gi RWO ibmc-vpc-block-retain-5iops-tier 8h
wordpress-pvc Bound pvc-519a59f8-286a-400b-9d4a-76ba3ed93bc8 10Gi RWO ibmc-vpc-block-general-purpose 5h34m
localhost:single_instance_wordpress spark$
localhost:single_instance_wordpress spark$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-519a59f8-286a-400b-9d4a-76ba3ed93bc8 10Gi RWO Delete Bound default/wordpress-pvc ibmc-vpc-block-general-purpose 5h34m
pvc-c09494f1-bfbc-45d7-a20b-1469819644df 10Gi RWO Retain Bound default/my-pvc ibmc-vpc-block-retain-5iops-tier 8h
localhost:single_instance_wordpress spark$
localhost:single_instance_wordpress spark$ kubectl get pods
NAME READY STATUS RESTARTS AGE
my-deployment-6d78f74d89-kw2dt 1/1 Running 5 5h20m
wordpress-77cb7cb6cc-zqbq5 1/1 Running 0 5h8m
wordpress-mysql-84b6cc4c6c-n4hg6 1/1 Running 0 7h42m
localhost:single_instance_wordpress spark$
localhost:single_instance_wordpress spark$ kubectl get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
my-deployment 1/1 1 1 5h20m
wordpress 1/1 1 1 5h9m
wordpress-mysql 1/1 1 1 7h43m
localhost:single_instance_wordpress spark$一键部署
可以指定文件夹来一键部署以上所有资源
kubectl apply -f ./config/