I've been messing with the S3 backup functionality for a while now. I randomly noticed that the backups had stopped working properly after I started getting email messages about deleting old backups not working properly:
[snip]
Creating TAR file of home directory ..
.. done
Uploading archive to Amazon's S3 service ..
.. done
.. completed in 9 seconds
20 servers backed up successfully, 0 had errors.
Deleting backups from backups/servers/example.com/daily/%d-%b-%Y_%H-%M in Amazon S3 bucket example_bucket older than 7 days ..
.. failed to list S3 files :
But hey, the files had been uploaded, it just couldn't delete the old backups, right?
After that had been occurring for a few days, I decided to look into the problem since I was tired of receiving emails every day. Then I noticed no new backups had been created since March 31st. So 7 days of backups had never taken place!
I then tried to delete the scheduled backup as well as the "Storage Provider", and configuring everything again. At the same time I gave the IAM user maximum permissions for anything S3 related:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::*",
"arn:aws:s3:::*/*"
]
}
]
}
After running the backup, everything looks like it's running smoothly, but the actual files aren't created on S3. No files anywhere, nothing. No errors in the log output, which happily reports: "Uploading archive to Amazon's S3 service .... done"
Questions:
I tried to upgrade botocore which VirtualMin apparently uses, and I get this error:
# pip install --upgrade botocore
Cleaning up...
Exception:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/usr/lib/python2.7/dist-packages/pip/commands/install.py", line 290, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/usr/lib/python2.7/dist-packages/pip/req.py", line 1097, in prepare_files
req_to_install, self.upgrade)
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 194, in find_requirement
page = self._get_page(main_index_url, req)
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 568, in _get_page
session=self.session,
File "/usr/lib/python2.7/dist-packages/pip/index.py", line 694, in get_page
req, link, "connection error: %s" % exc, url,
TypeError: __str__ returned non-string (type Error)
Storing debug log for failure in /root/.pip/pip.log
That lead me on another journey, and as it turns out, this was caused by a borked /etc/ssl directory; after running
update-ca-certificates --fresh, things started working again, including pip and s3 backups.