OpenStack对象存储(swift)是一个多租户的对象存储系统,它支持大规模扩展,可以以低成本来管理大型的非结构化数据。
二、安装并配置控制器节点
获得 admin 凭证来获取只有管理员能执行的命令的访问权限
# source /home/admin-openrc.sh
要创建身份认证服务的凭证有这几个步骤:创建 swift 用户,给 swift 用户添加 admin 角色,创建 swift 服务条目,创建对象存储服务 API 端点。
# openstack user create --domain default --password-prompt swift
# openstack role add --project service --user swift admin
# openstack service create --name swift --description "OpenStack Object Storage" object-store
# openstack endpoint create --region RegionOne object-store public http://172.16.5.135:8080/v1/AUTH_%\(tenant_id\)s
# openstack endpoint create --region RegionOne object-store internal http://172.16.5.135:8080/v1/AUTH_%\(tenant_id\)s
# openstack endpoint create --region RegionOne object-store admin http://172.16.5.135:8080/v1
三、配置组件
安装软件包
# yum install openstack-swift-proxy python-swiftclient python-keystoneclient python-keystonemiddleware memcached
从对象存储的仓库源中获取代理服务的配置文件
# curl -o /etc/swift/proxy-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/proxy-server.conf-sample?h=stable/ocata
编辑文件 /etc/swift/proxy-server.conf
# vi /etc/swift/proxy-server.conf
[DEFAULT]
bind_port = 8080
user = swift
swift_dir = /etc/swift
[pipeline:main]
pipeline = catch_errors gatekeeper healthcheck proxy-logging cache_sync bulk ratelimit authtoken keystoneauth copy-quotas account-quotas slo dlo versioned_writes proxy-logging proxy-server
[app:proxy-server]
use = egg:swift#proxy
account_autocreate = True
[filter:keystoneauth]
use = egg:swift#keystoneauth
operator_roles = admin,user
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
auth_uri = http://172.16.5.135:5000
auth_url = http://172.16.5.13535357
memcached_servers = 172.16.5.135:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = swift
password = 123qwe
delay_auth_decision = True
[filter:cache]
use = egg:swift#memcache
memcache_servers = controller:11211
四、安装和配置存储节点(每个存储节点都要执行)
安装支持的工具包:
# yum install xfsprogs rsync
使用XFS格式化/dev/sdb
和/dev/sdc
设备:
# mkfs.xfs /dev/sdb
# mkfs.xfs /dev/sdc
创建挂载点目录结构:
# mkdir -p /srv/node/sdb
# mkdir -p /srv/node/sdc
编辑/etc/fstab
文件并添加以下内容:
/dev/sdb /srv/node/sdb xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
/dev/sdc /srv/node/sdc xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
挂载设备:
# mount /srv/node/sdb
# mount /srv/node/sdc
创建并编辑/etc/rsyncd.conf
文件并包含以下内容:
uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 172.16.5.135
[account]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/account.lock
[container]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/container.lock
[object]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/object.lock
启动 “rsyncd” 服务和配置它随系统启动:
# systemctl enable rsyncd.service
# systemctl start rsyncd.service
五、配置组件(每个存储节点都要执行,bind_ip是本机IP)
安装软件包:
# yum install openstack-swift-account openstack-swift-container openstack-swift-object
从对象存储源仓库中获取accounting,以及object服务配置文件
# curl -o /etc/swift/account-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/account-server.conf-sample?h=stable/ocata
# curl -o /etc/swift/container-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/container-server.conf-sample?h=stable/ocata
# curl -o /etc/swift/object-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/object-server.conf-sample?h=stable/ocata
编辑 /etc/swift/account-server.conf 文件:
#vi /etc/swift/account-server.conf
[DEFAULT]
bind_ip = 172.16.5.135
bind_port = 6202
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True
[pipeline:main]
pipeline = healthcheck recon account-server
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
编辑/etc/swift/container-server.conf文件:
# vi /etc/swift/container-server.conf
[DEFAULT]
bind_ip = 172.16.5.135
bind_port = 6201
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True
[pipeline:main]
pipeline = healthcheck recon-server
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
编辑/etc/swift/object-server.conf文件:
# vi /etc/swift/object-server.conf
[DEFAULT]
bind_ip = 172.16.5.135
bind_port = 6200
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True
[pipeline:main]
pipeline = healthcheck recon object-server
[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
recon_lock_path = /var/lock
确认挂载点目录结构是否有合适的所有权:
# chown -R swift:swift /srv/node
创建 “recon” 目录和确保它有合适的所有权:
# mkdir -p /var/cache/swift
# chown -R root:swift /var/cache/swift
# chmod -R 775 /var/cache/swift
六、创建,分发并初始化rings(每个存储节点都要执行)
创建账户ring,切换到 /etc/swift目录创建基本 account.builder 文件。
# swift-ring-builder account.builder create 10 3 1
添加每个节点到 ring 中:
# swift-ring-builder account.builder add --region 1 --zone 1 --ip 172.16.5.135 --port 6202 --device sdb --weight 100
# swift-ring-builder account.builder add --region 1 --zone 1 --ip 172.16.5.135 --port 6202 --device sdc --weight 100
# swift-ring-builder account.builder add --region 1 --zone 2 --ip 172.16.5.136 --port 6202 --device sdb --weight 100
# swift-ring-builder account.builder add --region 1 --zone 2 --ip 172.16.5.136 --port 6202 --device sdc --weight 100
平衡 ring:
# swift-ring-builder account.builder rebalance
验证 ring 的内容:
# swift-ring-builder account.builder
account.builder, build version 4
1024 partitions, 3.000000 replicas, 1 regions, 2 zones, 4 devices, 100.00 balance, 0.00 dispersion
The minimum number of hours before a partition can be reassigned is 1
The overload factor is 0.00% (0.000000)
Devices: id region zone ip address port replication ip replication port name weight partitions balance meta
0 1 1 172.16.5.135 6202 10.0.0.51 6202 sdb 100.00 0 -100.00
1 1 1 172.16.5.135 6202 10.0.0.51 6202 sdc 100.00 0 -100.00
2 1 2 172.16.5.136 6202 10.0.0.52 6202 sdb 100.00 0 -100.00
3 1 2 172.16.5.136 6202 10.0.0.52 6202 sdc 100.00 0 -100.00
创建容器ring,切换到 /etc/swift
目录创建基本container.builder
文件:
# swift-ring-builder.builder create 10 3 1
添加每个节点到 ring 中:
# swift-ring-builder.builder add \
--region 1 --zone 1 --ip 172.16.5.135 --port 6201 --device sdb --weight 100
# swift-ring-builder.builder add \
--region 1 --zone 1 --ip 172.16.5.135 --port 6201 --device sdc --weight 100
# swift-ring-builder.builder add \
--region 1 --zone 2 --ip 172.16.5.136 --port 6201 --device sdb --weight 100
# swift-ring-builder.builder add \
--region 1 --zone 2 --ip 172.16.5.136 --port 6201 --device sdc --weight 100
平衡 ring:
# swift-ring-builder.builder rebalance
验证 ring 的内容:
# swift-ring-builder.builder
container.builder, build version 4
1024 partitions, 3.000000 replicas, 1 regions, 2 zones, 4 devices, 100.00 balance, 0.00 dispersion
The minimum number of hours before a partition can be reassigned is 1
The overload factor is 0.00% (0.000000)
Devices: id region zone ip address port replication ip replication port name weight partitions balance meta
0 1 1 172.16.5.135 6201 10.0.0.51 6201 sdb 100.00 0 -100.00
1 1 1 172.16.5.135 6201 10.0.0.51 6201 sdc 100.00 0 -100.00
2 1 2 172.16.5.136 6201 10.0.0.52 6201 sdb 100.00 0 -100.00
3 1 2 172.16.5.136 6201 10.0.0.52 6201 sdc 100.00 0 -100.00
创建对象ring,切换到 /etc/swift
目录创建基本object.builder
文件:
# swift-ring-builder object.builder create 10 3 1
添加每个节点到 ring 中:
# swift-ring-builder object.builder add \
--region 1 --zone 1 --ip 172.16.5.135 --port 6200 --device sdb --weight 100
# swift-ring-builder object.builder add \
--region 1 --zone 1 --ip 172.16.5.135 --port 6200 --device sdc --weight 100
# swift-ring-builder object.builder add \
--region 1 --zone 2 --ip 172.16.5.136 --port 6200 --device sdb --weight 100
# swift-ring-builder object.builder add \
--region 1 --zone 2 --ip 172.16.5.136 --port 6200 --device sdc --weight 100
平衡 ring:
# swift-ring-builder object.builder rebalance
验证 ring 的内容:
# swift-ring-builder object.builder
object.builder, build version 4
1024 partitions, 3.000000 replicas, 1 regions, 2 zones, 4 devices, 100.00 balance, 0.00 dispersion
The minimum number of hours before a partition can be reassigned is 1
The overload factor is 0.00% (0.000000)
Devices: id region zone ip address port replication ip replication port name weight partitions balance meta
0 1 1 172.16.5.135 6200 10.0.0.51 6200 sdb 100.00 0 -100.00
1 1 1 172.16.5.135 6200 10.0.0.51 6200 sdc 100.00 0 -100.00
2 1 2 172.16.5.136 6200 10.0.0.52 6200 sdb 100.00 0 -100.00
3 1 2 172.16.5.136 6200 10.0.0.52 6200 sdc 100.00 0 -100.00
七、完成安装
从对象存储源仓库中获取 /etc/swift/swift.conf 文件:
# curl -o /etc/swift/swift.conf \
https://git.openstack.org/cgit/openstack/swift/plain/etc/swift.conf-sample?h=stable/ocata
编辑 /etc/swift/swift.conf 文件
[swift-hash]
swift_hash_path_suffix = 123
swift_hash_path_prefix = qwe
[storage-policy:0]
name = Policy-0
default = yes
复制swift.conf
文件到每个存储节点和其他允许了代理服务的额外节点的 /etc/swift 目录。
在所有节点上,确认配置文件目录是否有合适的所有权:
# chown -R root:swift /etc/swift
在控制节点和其他运行了代理服务的节点上,启动对象存储代理服务及其依赖服务,并将它们配置为随系统启动:
# systemctl enable openstack-swift-proxy.service memcached.service
# systemctl start openstack-swift-proxy.service memcached.service
在存储节点上,启动对象存储服务,并将其设置为随系统启动:
# systemctl enable openstack-swift-account.service openstack-swift-account-auditor.service \
openstack-swift-account-reaper.service openstack-swift-account-replicator.service
# systemctl start openstack-swift-account.service openstack-swift-account-auditor.service \
openstack-swift-account-reaper.service openstack-swift-account-replicator.service
# systemctl enable openstack-swift-container.service \
openstack-swift-container-auditor.service openstack-swift-container-replicator.service \
openstack-swift-container-updater.service
# systemctl start openstack-swift-container.service \
openstack-swift-container-auditor.service openstack-swift-container-replicator.service \
openstack-swift-container-updater.service
# systemctl enable openstack-swift-object.service openstack-swift-object-auditor.service \
openstack-swift-object-replicator.service openstack-swift-object-updater.service
# systemctl start openstack-swift-object.service openstack-swift-object-auditor.service \
openstack-swift-object-replicator.service openstack-swift-object-updater.service
八、 验证操作
如果其中的一项或多项步骤没有正确执行,请在/var/log/audit/audit.log
文件中检查SELinux的关于禁止swift
过程的信息。如果该文件存在的话,将/srv/node
目录下swift_data_t
type, object_r role 和the system_u user关于安全等级的信息设置成最低安全等级(s0)
# chcon -R system_u:object_r:swift_data_t:s0 /srv/node
导入demo
凭证:
# source /home/demo-openrc.sh
显示服务状态:
# swift --debug stat
进入仪表盘(控制界面),去上传文件试一下吧。
注:
打开swift日志
# vi /etc/rsyslog.d/openstack-swift.conf
local0.*;local2.* /var/log/swift/swift.log
#& stop
# systemctl restart rsyslog.service
openstack swift做glance后端存储
# vim /etc/glance/glance-api.conf
[glance_store]
default_store = swift 默认为file 改为swift
stores = file, http 默认不用动
swift_store_auth_version = 2 默认版本为2
stores=glance.store.swift.Store,glance.store.filesystem.Store 此项一定要加上,不然无法上传
swift_store_auth_address = http://controller:5000/v2.0 controller的keystone认证
swift_store_user = service:swift 使用swift用户
swift_store_key = swift 密码
swift_store_container = glance 将要被创建的容器
swift_store_create_container_on_put = True 上传开
swift_store_large_object_size = 5120 最大5G限制,但与glance结合后限制无效
swift_store_large_object_chunk_size = 200 最大200个容器
swift_enable_snet = False
# systemctl restart openstack-glance-api
九、排查错误
如果你很不幸遇到了503错误,并且根据日志的报错内容觉得是代理服务可能出错了,但是无论如何修改依旧报503错误。亲,你创建挂载点目录结构要修改成下面的:
# mkdir -p /srv/node/sdb
# mkdir -p /srv/node/sdc
并且ring的时候:–device sdb –device sdc。