I had an RDO IceHouse installation of openstack on Fedora 20. It had no swift installed, so I was trying to install it. The installation was done ~6-8 month ago, and since then some additional configs were made, so running packstack again was a bit of risk.
Following mainly that document I succeeded to have swift running.
But the problem is that when I create a container (from a dashboard, for example), in 5-30 seconds it disappears.
$ swift stat
Account: AUTH_df715cfea8e240e3be22ba7bd56d148a
Containers: 1
Objects: 0
Bytes: 0
Accept-Ranges: bytes
X-Timestamp: 1418907580.14515
X-Trans-Id: tx811daed7a0d846d8b7ad7-005492cfcb
Content-Type: text/plain; charset=utf-8
$ swift stat
Account: AUTH_df715cfea8e240e3be22ba7bd56d148a
Containers: 0
Objects: 0
Bytes: 0
X-Put-Timestamp: 1418907615.35912
X-Timestamp: 1418907615.35912
X-Trans-Id: tx62147245b42340419681f-005492cfdf
Content-Type: text/plain; charset=utf-8
Possible reason: in parallel with services (openstack-swift-…), i run swift-init (which run the same processes) and the configuration changes I did were not taken into account.
stop things with swift-init: swift-init kill all (Note: I tried to fix it by doing all the following steps without this one. It did not work, so it was crucial)
stop all services (I run all on one node):for service in openstack-swift-object openstack-swift-object-replicator openstack-swift-object-updater openstack-swift-object-auditor openstack-swift-container openstack-swift-container-replicator openstack-swift-container-updater openstack-swift-container-auditor openstack-swift-account openstack-swift-account-replicator openstack-swift-account-reaper openstack-swift-account-auditor openstack-swift-proxy openstack-swift-account; do service $service stop; done
remove all files from the node (I had it at /srv/node/partition1)
in /etc/swift, removed {account,container,object}{.builder,.ring.gz} (also removed things from /etc/swift/backup)
# swift upload C1 admin-openrc.sh
Warning: failed to create container ‘C1’: 404 Not Found:
Not Found
The resource could not be found.<
Object PUT failed: http://176.204.66.102:8080/v1/AUTH_0341849e239042fba0fce28f32e541b0/C1/admin-openrc.sh 404 Not Found [first 60 chars of response]
A 404 on a PUT means the “group of things” above the request failed the existence check. So if you get a 404 on object PUT it’s because the proxy failed the container existence check. If you get a 404 on a container PUT it’s because the proxy failed the account existence check.
Check the following:
1) are you running account_autocreate? You should, it is normally on by default [1] 2) are you running with 0 recheck_account_existence? ’cause that’s broken [2]
I’m curious if when you list/stat the account does it have any containers in it? Swift will lazy provision authorized accounts, and that code has been worked on in the last development cycle. If you have containers/objects in then the account definitely has a real physical data file and then it definitely exists – but if there’s no data in the account it can mean different things from a troubleshooting perspective. There may be some backend error talking to the account servers – if you’re unable to resolve the issue thoroughly examine any ERROR lines from /var/log/syslog for messages relating to problems connecting to account servers.
# systemctl status openstack-swift-object.service openstack-swift-object-auditor.service openstack-swift-object-replicator.service openstack-swift-object-updater.service -l
● openstack-swift-object.service – OpenStack Object Storage (swift) – Object Server
Loaded: loaded (/usr/lib/systemd/system/openstack-swift-object.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2019-02-19 00:20:53 EST; 3s ago
Main PID: 24340 (swift-object-se)
Tasks: 4
CGroup: /system.slice/openstack-swift-object.service
├─24340 /usr/bin/python2 /usr/bin/swift-object-server /etc/swift/object-server.conf
├─24376 /usr/bin/python2 /usr/bin/swift-object-server /etc/swift/object-server.conf
├─24378 /usr/bin/python2 /usr/bin/swift-object-server /etc/swift/object-server.conf
└─24379 /usr/bin/python2 /usr/bin/swift-object-server /etc/swift/object-server.conf
Feb 19 00:20:53 object1 systemd[1]: Started OpenStack Object Storage (swift) – Object Server.
Feb 19 00:20:53 object1 object-server[24340]: Started child 24376
Feb 19 00:20:53 object1 object-server[24340]: Started child 24378
Feb 19 00:20:53 object1 object-server[24340]: Started child 24379
● openstack-swift-object-auditor.service – OpenStack Object Storage (swift) – Object Auditor
Loaded: loaded (/usr/lib/systemd/system/openstack-swift-object-auditor.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2019-02-19 00:20:53 EST; 3s ago
Main PID: 24337 (swift-object-au)
Tasks: 2
CGroup: /system.slice/openstack-swift-object-auditor.service
└─24337 /usr/bin/python2 /usr/bin/swift-object-auditor /etc/swift/object-server.conf
Feb 19 00:20:53 object1 systemd[1]: Started OpenStack Object Storage (swift) – Object Auditor.
Feb 19 00:20:53 object1 object-auditor[24337]: Exception dumping recon cache: #012Traceback (most recent call last):#012 File “/usr/lib/python2.7/site-packages/swift/common/utils.py”, line 2597, in dump_recon_cache#012 with lock_file(cache_file, lock_timeout, unlink=False) as cf:#012 File “/usr/lib64/python2.7/contextlib.py”, line 17, in enter#012 return self.gen.next()#012 File “/usr/lib/python2.7/site-packages/swift/common/utils.py”, line 1877, in lock_file#012 fd = os.open(filename, flags)#012OSError: [Errno 13] Permission denied: ‘/var/cache/swift/object.recon’
Feb 19 00:20:53 object1 object-auditor[24337]: Exception dumping recon cache: #012Traceback (most recent call last):#012 File “/usr/lib/python2.7/site-packages/swift/common/utils.py”, line 2597, in dump_recon_cache#012 with lock_file(cache_file, lock_timeout, unlink=False) as cf:#012 File “/usr/lib64/python2.7/contextlib.py”, line 17, in enter#012 return self.gen.next()#012 File “/usr/lib/python2.7/site-packages/swift/common/utils.py”, line 1877, in lock_file#012 fd = os.open(filename, flags)#012OSError: [Errno 13] Permission denied: ‘/var/cache/swift/object.recon’
Feb 19 00:20:53 object1 object-auditor[24372]: Begin object audit “forever” mode (ZBF)
Feb 19 00:20:53 object1 object-auditor[24372]: Object audit (ZBF) “forever” mode completed: 0.00s. Total quarantined: 0, Total errors: 0, Total files/sec: 0.00, Total bytes/sec: 0.00, Auditing time: 0.00, Rate: 0.00
Feb 19 00:20:53 object1 object-auditor[24373]: Begin object audit “forever” mode (ALL)
Feb 19 00:20:53 object1 object-auditor[24373]: Object audit (ALL) “forever” mode completed: 0.00s. Total quarantined: 0, Total errors: 0, Total files/sec: 0.00, Total bytes/sec: 0.00, Auditing time: 0.00, Rate: 0.00
● openstack-swift-object-replicator.service – OpenStack Object Storage (swift) – Object Replicator
Loaded: loaded (/usr/lib/systemd/system/openstack-swift-object-replicator.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2019-02-19 00:20:53 EST; 3s ago
Main PID: 24338 (swift-object-re)
Tasks: 1
CGroup: /system.slice/openstack-swift-object-replicator.service
└─24338 /usr/bin/python2 /usr/bin/swift-object-replicator /etc/swift/object-server.conf
Feb 19 00:20:53 object1 systemd[1]: Started OpenStack Object Storage (swift) – Object Replicator.
Feb 19 00:20:53 object1 object-replicator[24338]: Starting object replicator in daemon mode.
Feb 19 00:20:53 object1 object-replicator[24338]: Starting object replication pass.
Feb 19 00:20:53 object1 object-replicator[24338]: Nothing replicated for 0.00165200233459 seconds.
Feb 19 00:20:53 object1 object-replicator[24338]: Object replication complete. (0.00 minutes)
Feb 19 00:20:53 object1 object-replicator[24338]: Exception dumping recon cache: #012Traceback (most recent call last):#012 File “/usr/lib/python2.7/site-packages/swift/common/utils.py”, line 2597, in dump_recon_cache#012 with lock_file(cache_file, lock_timeout, unlink=False) as cf:#012 File “/usr/lib64/python2.7/contextlib.py”, line 17, in enter#012 return self.gen.next()#012 File “/usr/lib/python2.7/site-packages/swift/common/utils.py”, line 1877, in lock_file#012 fd = os.open(filename, flags)#012OSError: [Errno 13] Permission denied: ‘/var/cache/swift/object.recon’
● openstack-swift-object-updater.service – OpenStack Object Storage (swift) – Object Updater
Loaded: loaded (/usr/lib/systemd/system/openstack-swift-object-updater.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2019-02-19 00:20:53 EST; 3s ago
Main PID: 24342 (swift-object-up)
Tasks: 1
CGroup: /system.slice/openstack-swift-object-updater.service
└─24342 /usr/bin/python2 /usr/bin/swift-object-updater /etc/swift/object-server.conf
Feb 19 00:20:53 object1 systemd[1]: Started OpenStack Object Storage (swift) – Object Updater.
at the place of the command “chown -R root:swift /var/cache/swift”, we need also execute the command “chmod -R 775 /var/cache/swift”, as when we make dir (as root user) with command “mkdir -p /var/cache/swift”, the access permission of dir /var/cache/swift will be 755(as rwxr-xr-x), which means “swift” user can not write file in this dir. so the exception like below will occur in the log /var/log/messages.”[Errno 13] Permission denied: ‘/var/cache/swift/object.recon'” ————————— Apr 13 20:36:27 kvm-sz-004-003 object-auditor: Exception dumping recon cache: #012Traceback (most recent call last):#012 File “/usr/lib/python2.7/site-packages/swift/common/utils.py”, line 2597, in dump_recon_cache#012 with lock_file(cache_file, lock_timeout, unlink=False) as cf:#012 File “/usr/lib64/python2.7/contextlib.py”, line 17, in __enter__#012 return self.gen.next()#012 File “/usr/lib/python2.7/site-packages/swift/common/utils.py”, line 1877, in lock_file#012 fd = os.open(filename, flags)#012OSError: [Errno 13] Permission denied: ‘/var/cache/swift/object.recon’ —————————- if we execute the command “chmod -R 775 /var/cache/swift”, this kind of exception will not occur.
brctl addbr br0 #创建一个网桥
brctl show #显示网桥信息
brctl addif br0 eth0 && ip addr del dev eth0 192.168.2.200/24 && ifconfig br0 192.168.2.200/24 up
brctl show #查看结果
ifconfig br0 #验证br0是否成功取代了eth0的IP
注意: 这里的IP地址为 宿主机ip
修改虚拟机桥接到br0网卡，在宿主机修改
virsh list --all
ps aux |grep kvm
virsh stop Centos-6.6-x68_64
virsh list --all
1 start request repeated too quickly for xxx.service
This error is for preventing many repetition of service restart in system error.
If the service restart exceeds the value of StartLimitBurst within the time specified by the value of StartLimitInterval, the service startup will fail. StartLimitInterval is 10 seconds by default and StartLimitBurst is 5 by default.
For example, dhcpd is as below. The service startup will fail with the 6th “systemctl restart dhcpd”.
$ i=1
$ while : ; do
echo ${i}
i=$(expr ${i} + 1);
sudo systemctl restart dhcpd || break
done
1
2
3
4
5
6
Job for dhcpd.service failed because start of the service was
attempted too often. See "systemctl status dhcpd.service" and
"journalctl -xe" for details.
To force a start use "systemctl reset-failed dhcpd.service" followed
by "systemctl start dhcpd.service" again.
“start request repeated too quickly for dhcpd.service” is in journal.
$ sudo journalctl -xeu dhcpd
<snip>
systemd[1]: start request repeated too quickly for dhcpd.service
<snip>
2 Use systemctl reload
Use “systemctl reload” if the service can reload.
$ systemctl show -p CanReload named
CanReload=yes
Running “systemctl reload” more than 6 times within 10 seconds will not fail.
$ sudo systemctl start named
$ i=1
$ while : ; do
echo ${i}
i=$(expr ${i} + 1);
sudo systemctl restart named || break
done
1
2
3
4
5
6
7
8
<snip>
3 Use StartLimitBurst=0 if the service cannot reload
There is a service which cannot reload. The dhcpd community seems not to have the resource for implementation and maintenance of reload.
$ systemctl show -p CanReload dhcpd
CanReload=no
For a service which cannot reload, disable checker of service start with making StartLimitBurst be 0.
Check systemd’s file with the following command.
$ systemctl show -p FragmentPath dhcpd
FragmentPath=/usr/lib/systemd/system/dhcpd.service
