当前位置：首页>编程语言>正文

etcd容器启动失败 etcd operator

编程语言2024-04-28 18:26:44

基于velero对etcd的备份和恢复

Velero简介

Velero 地址：https://github.com/vmware-tanzu/velero

Velero 是vmware开源的一个云原生的灾难恢复和迁移工具，它本身也是开源的,采用Go语言编写，可以安全的备份、恢复和迁移Kubernetes集群资源数据，https://velero.io/。
Velero 是西班牙语意思是帆船，非常符合Kubernetes社区的命名风格，Velero的开发公司Heptio，已被VMware收购。
Velero 支持标准的K8S集群，既可以是私有云平台也可以是公有云，除了灾备之外它还能做资源移转，支持把容器应用从一个集群迁移到另一个集群。
Velero 的工作方式就是把kubernetes中的数据备份到对象存储以实现高可用和持久化，默认的备份保存时间为720小时，并在需要的时候进行下载和恢复。

Velero与etcd快照备份的区别

etcd 快照是全局备份，在即使一个资源对象需要恢复，也需要做全局恢复到备份的状态，即会影响其它namespace中pod运行服务。
Velero可以有针对性的备份，比如按照namespace单独备份、只备份单独的资源对象等，在恢复的时候只恢复单独的namespace或资源对象，而不影响其它namespace中pod运行服务。
velero支持ceph、oss等对象存储，etcd 快照是一个为本地文件。
velero支持任务计划实现周期备份，但etcd 快照也可以基于cronjob实现。

Velero工作流程

流程图

etcd容器启动失败 etcd operator,etcd容器启动失败 etcd operator_客户端,第1张

本地 Velero 客户端发送备份指令。
Kubernetes 集群内就会创建一个 Backup 对象。
BackupController 监测 Backup 对象并开始备份过程。
BackupController 会向 API Server 查询相关数据。
BackupController 将查询到的数据备份到远端的对象存储。

Velero特性

Velero 目前包含以下特性：

支持 Kubernetes 集群数据备份和恢复
支持复制当前 Kubernetes 集群的资源到其它 Kubernetes 集群
支持复制生产环境到开发以及测试环境

Velero组件

Velero 组件一共分两部分，分别是服务端和客户端。

服务端：运行在你 Kubernetes 的集群中
客户端：是一些运行在本地的命令行的工具，需要已配置好 kubectl 及集群 kubeconfig 的机器上

部署minio

#创建数据目录
root@k8s-deploy-01:~# mkdir /opt/data/minio -p
#下载镜像
root@k8s-deploy-01:~# docker pull minio/minio:RELEASE.2022-04-12T06-55-35Z
#创建minio容器,如果不指定,则默认用户名与密码为 minioadmin/minioadmin,可以通过环境变量自定义。
#密码长度最低8位：Access key length should be at least 3, and secret key length at least 8 characters
root@k8s-deploy-01:~# docker run --name minio \
> -p 9000:9000 \
> -p 9999:9999 \
> -d --restart=always \
> -e "MINIO_ROOT_USER=admin" \
> -e "MINIO_ROOT_PASSWORD=12345678" \
> -v /opt/data/minio:/data \
> minio/minio:RELEASE.2022-04-12T06-55-35Z server /data\
--console-address '0.0.0.0:9999'

创建bucket:

etcd容器启动失败 etcd operator,etcd容器启动失败 etcd operator_json_03,第3张

验证bucket:

etcd容器启动失败 etcd operator,etcd容器启动失败 etcd operator_etcd容器启动失败_04,第4张

部署velero

#下载软件包
wget https://github.com/vmware-tanzu/velero/releases/download/v1.8.1/velero-v1.8.1-linux-amd64.tar.gz
#解压
root@k8s-master-01:/opt# tar xf velero-v1.8.1-linux-amd64.tar.gz
root@k8s-master-01:/opt# cd velero-v1.8.1-linux-amd64/
root@k8s-master-01:/opt/velero-v1.8.1-linux-amd64# cp velero /usr/local/bin/velero
#验证
root@k8s-master-01:~# velero version
Client:
	Version: v1.8.1
	Git commit: 18ee078dffd9345df610e0ca9f61b31124e93f50
<error getting server version: no matches for kind "ServerStatusRequest" in version "velero.io/v1">

配置velero认证环境：

#工作目录
root@k8s-master-01:~# mkdir /opt/data/velero -p
#认证文件
root@k8s-master-01:/opt/data/velero# cat velero-auth.txt 
[default]
aws_access_key_id = admin
aws_secret_access_key = 12345678
#准备user-csr授权文件
root@k8s-master-01:/opt/data/velero# cat awsuser-csr.json 
{
  "CN": "awsuser",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "BeiJing",
      "L": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}

配置证书签发

#下载
wget https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssl_1.6.1_linux_amd64 
wget https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssljson_1.6.1_linux_amd64 
wget https://github.com/cloudflare/cfssl/releases/download/v1.6.1/cfssl-certinfo_1.6.1_linux_amd64
root@k8s-master-01:/opt/soft/cfssl# mv cfssl_1.6.1_linux_amd64 cfssl
root@k8s-master-01:/opt/soft/cfssl# mv cfssl-certinfo_1.6.1_linux_amd64 cfssl-certinfo
root@k8s-master-01:/opt/soft/cfssl# mv cfssljson_1.6.1_linux_amd64 cfssljson
root@k8s-master-01:/opt/soft/cfssl# chmod  a+x  cfssl*
root@k8s-master-01:/opt/soft/cfssl# cp cfssl* /usr/local/bin/
#执行证书签发
root@k8s-master-01:/opt/data/velero# cfssl gencert -ca=/etc/kubernetes/ssl/ca.pem -ca-key=/etc/kubernetes/ssl/ca-key.pem -config=/opt/data/velero/ca-config.json -profile=kubernetes ./awsuser-csr.json | cfssljson -bare awsuser
2022/08/01 21:00:11 [INFO] generate received request
2022/08/01 21:00:11 [INFO] received CSR
2022/08/01 21:00:11 [INFO] generating key: rsa-2048
2022/08/01 21:00:11 [INFO] encoded CSR
2022/08/01 21:00:11 [INFO] signed certificate with serial number 600603671237595968234312721954258747098708764648
2022/08/01 21:00:11 [WARNING] This certificate lacks a "hosts" field. This makes it unsuitable for
websites. For more information see the Baseline Requirements for the Issuance and Management
of Publicly-Trusted Certificates, v.1.1.6, from the CA/Browser Forum (https://cabforum.org);
specifically, section 10.2.3 ("Information Requirements").
#分发证书到api-server证书路径
root@k8s-master-01:/opt/data/velero# cp awsuser-key.pem /etc/kubernetes/ssl/
root@k8s-master-01:/opt/data/velero# cp awsuser.pem /etc/kubernetes/ssl/
#生成集群认证config文件
root@k8s-master-01:/opt/data/velero# export KUBE_APISERVER="https://172.31.3.110:6443"
root@k8s-master-01:/opt/data/velero# kubectl config set-cluster kubernetes \
> --certificate-authority=/etc/kubernetes/ssl/ca.pem \  #指定ca公钥信息
> --embed-certs=true \  #加密
> --server=${KUBE_APISERVER} \  #apiserver地址
> --kubeconfig=./awsuser.kubeconfig  #写到当前目录下
Cluster "kubernetes" set.
#设置客户端证书认证
root@k8s-master-01:/opt/data/velero# kubectl config set-credentials awsuser \
> --client-certificate=/etc/kubernetes/ssl/awsuser.pem \ #客户端私钥
> --client-key=/etc/kubernetes/ssl/awsuser-key.pem \  #客户端公钥
> --embed-certs=true \  #加密
> --kubeconfig=./awsuser.kubeconfig   #写入到当前文件
User "awsuser" set.
#设置上下文参数
root@k8s-master-01:/opt/data/velero# kubectl config set-context kubernetes \
> --cluster=kubernetes \
> --user=awsuser \
> --namespace=velero-system \
> --kubeconfig=./awsuser.kubeconfig
Context "kubernetes" created.
#设置默认上下文
root@k8s-master-01:/opt/data/velero# kubectl config use-context kubernetes --kubeconfig=awsuser.kubeconfig
Switched to context "kubernetes".
#k8s集群中创建awsuser账户，设置位集群admin权限
root@k8s-master-01:/opt/data/velero# kubectl create clusterrolebinding awsuser --clusterrole=cluster-admin --user=awsuser
clusterrolebinding.rbac.authorization.k8s.io/awsuser created

安装

#创建namespace
root@k8s-master-01:/opt/data/velero# kubectl create ns myserver
#安装
root@k8s-master-01:/opt/data/velero# velero --kubeconfig  ./awsuser.kubeconfig \
> install \
>     --provider aws \  #提供者
>     --plugins velero/velero-plugin-for-aws:v1.3.1 \  #镜像
>     --bucket velerodata  \   #bucket就是在minio里面创建的bucket
>     --secret-file ./velero-auth.txt \   #认证文件，提供了账号密码
>     --use-volume-snapshots=false \  #是否用快照
> --namespace velero-system \  #命名空间
> --backup-location-config region=minio,s3ForcePathStyle="true",s3Url=http://172.31.3.141:9000  #ip指向minio
#验证
root@k8s-master-01:/opt/data/velero# kubectl get pod -n velero-system 
NAME                      READY   STATUS    RESTARTS   AGE
velero-858b9459f9-jprrr   1/1     Running   0          3m28s
root@k8s-master-01:/opt/data/velero# kubectl describe pod velero-858b9459f9-jprrr -n velero-system

备份

#备份命令
root@k8s-master-01:/opt/data/velero# velero backup --help
Work with backups

Usage:
  velero backup [command]

Available Commands:
  create      Create a backup
  delete      Delete backups
  describe    Describe backups
  download    Download all Kubernetes manifests for a backup
  get         Get backups
  logs        Get backup logs 
#备份myserver
root@k8s-master-01:~# DATE=`date +%Y%m%d%H%M%S`
root@k8s-master-01:~# velero backup create myserver-backup-${DATE} \
> --include-cluster-resources=true \   #是否备份全局资源
> --include-namespaces myserver \   #指定备份的namespaces
> --kubeconfig=/opt/data/velero/awsuser.kubeconfig \  #认证文件
> --namespace velero-system   #velero的namespace
Backup request "myserver-backup-20220801213554" submitted successfully.
#验证
root@k8s-master-01:~# velero backup describe myserver-backup-20220801213554 --kubeconfig=/opt/data/velero/awsuser.kubeconfig --namespace velero-system
Name:         myserver-backup-20220801213554
Namespace:    velero-system
Labels:       velero.io/storage-location=default
Annotations:  velero.io/source-cluster-k8s-gitversion=v1.24.3
              velero.io/source-cluster-k8s-major-version=1
              velero.io/source-cluster-k8s-minor-version=24

Phase:  Completed

minio验证备份数据

etcd容器启动失败 etcd operator,etcd容器启动失败 etcd operator_linux_05,第5张

恢复测试

#查看现有资源
root@k8s-master-01:~# kubectl get pod -n myserver 
NAME        READY   STATUS    RESTARTS       AGE
net-test1   1/1     Running   1 (147m ago)   3d6h
#删除
root@k8s-master-01:~# kubectl delete pod net-test1 -n myserver 
pod "net-test1" deleted
root@k8s-master-01:~# kubectl get pod -n myserver 
No resources found in myserver namespace
#恢复
root@k8s-master-01:~#  velero restore create --from-backup myserver-backup-20220801213554 --wait --kubeconfig=/opt/data/velero/awsuser.kubeconfig --namespace velero-system
Restore request "myserver-backup-20220801213554-20220801214809" submitted successfully.
Waiting for restore to complete. You may safely press ctrl-c to stop waiting - your restore will continue in the background.
...................
Restore completed with status: Completed. You may check for more information using the commands `velero restore describe myserver-backup-20220801213554-20220801214809` and `velero restore logs myserver-backup-20220801213554-20220801214809`.
root@k8s-master-01:~# kubectl get pod -n myserver 
NAME        READY   STATUS    RESTARTS   AGE
net-test1   1/1     Running   0          4m5s

备份指定资源对象

#使用--ordered-resources参数指定备份的资源对象
velero backup create pod-backup-202207222335 --include-cluster-
resources=true --ordered-resources 'pods=myserver/net-test1,defafut/net-test1' --namespace velero-system -
-include-namespaces=myserver,defafut

root@k8s-master-01:~# DATE=`date +%Y%m%d%H%M%S`
root@k8s-master-01:~# velero backup create pods-backup-${DATE} \
>  --include-cluster-resources=true \
>  --ordered-resources 'pods=myserver/net-test1,velero-system/velero-858b9459f9-jprrr' \  #多个pod用逗号分隔
>  --namespace velero-system \
>  --include-namespaces=myserver,velero-system  #资源所属的namespaces
Backup request "pods-backup-20220801215429" submitted successfully.

批量备份所有namespace脚本

root@k8s-master-01:/opt/data/velero# vim ns-back.sh
#!/bin/bash
NS_NAME=`kubectl get ns | awk '{if (NR>1){print}}' | awk '{print }'`
DATE=`date +%Y%m%d%H%M%S`
cd /opt/data/velero/
for i in $NS_NAME;do
velero backup create ${i}-ns-backup-${DATE} \
--include-cluster-resources=true \
--include-namespaces ${i} \
--kubeconfig=/opt/data/velero/awsuser.kubeconfig \  #替换为自己的认证文件，也可以使用 /root/.kube/config
--namespace velero-system
done
#测试
root@k8s-master-01:/opt/data/velero# sh ns-back.sh 
Backup request "default-ns-backup-20220801220432" submitted successfully.
Run `velero backup describe default-ns-backup-20220801220432` or `velero backup logs default-ns-backup-20220801220432` for more details.
Backup request "kube-node-lease-ns-backup-20220801220432" submitted successfully.
Run `velero backup describe kube-node-lease-ns-backup-20220801220432` or `velero backup logs kube-node-lease-ns-backup-20220801220432` for more details.
Backup request "kube-public-ns-backup-20220801220432" submitted successfully.
Run `velero backup describe kube-public-ns-backup-20220801220432` or `velero backup logs kube-public-ns-backup-20220801220432` for more details.
Backup request "kube-system-ns-backup-20220801220432" submitted successfully.
Run `velero backup describe kube-system-ns-backup-20220801220432` or `velero backup logs kube-system-ns-backup-20220801220432` for more details.
Backup request "kubernetes-dashboard-ns-backup-20220801220432" submitted successfully.
Run `velero backup describe kubernetes-dashboard-ns-backup-20220801220432` or `velero backup logs kubernetes-dashboard-ns-backup-20220801220432` for more details.
Backup request "myserver-ns-backup-20220801220432" submitted successfully.
Run `velero backup describe myserver-ns-backup-20220801220432` or `velero backup logs myserver-ns-backup-20220801220432` for more details.
Backup request "velero-system-ns-backup-20220801220432" submitted successfully.
Run `velero backup describe velero-system-ns-backup-20220801220432` or `velero backup logs velero-system-ns-backup-20220801220432` for more details.

查看全文

https://www.xamrdz.com/lan/5pu1937159.html