How to backup WHM/cPanel accounts to DigitalOcean Spaces
For the past several years I have been using Amazon S3 to backup my WHM/cPanel accounts. It works quite good but the problem is Amazon’s complex pricing model (that charges you for things like diskspace, bandwidth, requests) making price for the service is quite unpredictable. I’ve read on many places that people often got surprised at the end of the month receiving the bill from Amazon.
Recently I have started testing DigitalOcean Spaces, a service similar to Amazon’s S3 (S3 compatible alternative) but with a pricing structure anyone can understand: for $5/month you get 250 GB of storage and 1 TB of outbound transfer (inbound transfer is free like on S3). If you need more than that it will cost you 2cents per every additional GB of storage and 1cent for every additional of outbound GB. Also first two months for every new space are free!
Back to Amazon: my last Amazon S3 bill was $48.60 and less than $20 bucks from that was storage and the remaining about $30 was mostly request fees, some transfers that I have no idea how are made since I only backup there (never actually needed to retrieve anything) and fees for files already deleted (but are still kept in some kind of “recycle bin”). My December bill was $53.60.
Currently, I have around 1.1 TB of data that I back up at Amazon S3. Moving that to Spaces will cost me 1.11 x 1024 x $0.02 = $22.5. Plus I get first two months for free so that will save me another roughly $100 for the first two months. As a bonus I won’t have to think anymore about requests or storage classes or anything. The only thing I need to know is amount of data I need to store (and additionally amount of data that I need to retrieve from the storage over 1 TB).
Now, I know that Spaces might not be the best solution for mission-critical backups (because it probably offers lower redundancy compared to S3) but I’m quite sure that it’s probably good enough for my needs.
Since Spaces use same API as Amazon S3 it should be compatible with (almost) all tools that you can use to manage S3 meaning that it should be compatible with WHM backups. But no matter how I tried to set it up – it just wouldn’t work!
There is a cPanel feature request asking to make it possible to use Spaces as backup destination, it’s already 3 months old and nothing is done about that already. I hope that cPanel folks will understand that this is something that needs to be done, and you’ll see later in this post, it’s quite simple too. Also, a quick note here, I’d like that cPanel allows you to use any service S3 compatible object storage, not just Spaces since there are other alternatives that some people might use (like Wasabi, OVH object storage, DreamObjects, ExoScale …)
November 2018 Update: WHM now naively supports DigitalOcean
Recently WHM have introduced native support for all S3 compliant object storage providers including DigitalOcean. This helps a lot since editing of WHM/cPanel core files is no longer needed and using DigitalOcean Spaces for backups is just couple clicks away.
Here’s how to set it up
1. First go to Backups -> Backup Configuration and at Backup Settings tab click on Enable Backups checkbox to enable the backups
2. Now click on Additional Destinations tab and pick S3 Compatible and click on Create New Destination button:
Destination Name: could be anything you want… make it descriptive like DigitalOcean Spaces Server Backup
Folder: folder within the Space where all your backups will be. You can leave it if you want backups to be stored at root of your space and not in some folder. If you put folder that doesn’t exist on Space – it will be created automatically and all backups will be stored in that folder.
Bucket: name of the DigitalOcean Space
Access Key ID: Spaces Access Key
Secret Access Key: Spaces Secret Key
Now click on Save and Validate
If validation was a success that means that your server can now access DigitalOcean space and that it can copy server backups there. To finish the setup is you can switch back to Backup Settings tab and configure all other things related to your backups (what to backup, how and when).
NOTE: The information below is obsolete since the introduction of S3 compatible backups in WHM and you should not use it.
Setting up WHM to use DigitalOcean Spaces for backups Please make sure that you have made a backups of files and accounts before making any changes !!! After making these changes you can only use DigitalOcean spaces for backup (can’t use S3 and Spaces – only Spaces !!!) You can easily revert the changes and use Amazon S3 again You can backup only to Spaces in ONE Region !!! You are doing this at your own risk. I can’t be held responsible for any damages or data loss this may cause !!! Before we start, if you already have a WHM backup running on Amazon S3 go to WHM > Backup Configuration then scroll down and disable the S3 backup clicking on action Disable next to the S3 backup. To allow WHM to store backups at S3 compatible DigitalOcean Spaces we will have to make a one very small change to the file on the path below:
Make a backup of this file before making any changes !
Open the file for editing and search for my $s3 = Amazon::S3->new it should be at line 111 now before retry add a line:
<code>'host' => 'ams3.digitaloceanspaces.com',</code>
It should look like this: Make sure that you have put the correct Spaces url , depending on the Region of your Space. Save the changes and go back to WHM.
Creating a new DigitalOcean Spaces backup Destination in WHM Amazon S3 go to WHM > Backup Configuration then scroll down and select Amazon S3 and click on Create a new destination: And setup it up as follows: Destination Name : could be anything you want… make it descriptive… Folder : folder within the Space where all your backups will be. You can leave it if you want backups to be stored at root of your space and not in some folder. If you put folder that doesn’t exist on Space – it will be created automatically and all backups will be stored in that folder. Bucket : name of the DigitalOcean Space Access Key ID : Spaces Access Key Secret Access Key : Spaces Secret Key Now click on Save and Validate If validation was a success the setup is done and your WHM accounts should be backed up to DigitalOcean Space on next backup run. If you don’t want to wait and want the start the backup right now, login to SSH and execute
I know this is a hack but I sure hope that cPanel guys will soon make the necessary changes so any S3 compatible service can be used for backups – not just Amazon S3. For the end, here’s a screenshot of my Space with my WHM backups just to prove that this actually works 🙂
February 16, 2018 - 7:06 am
This tutorial worked for me, thanks for your effort.
Mostafiz R Rahman
February 20, 2018 - 3:58 am
February 27, 2018 - 10:02 am
This Article helped me too, now I have next step that how to restore backup from DigitalOcean Spaces.
Waiting for your reply. I’m happy to help or work on this issue.
We have an option of s3cmd.
May 11, 2018 - 12:38 am
I really appreciate this post!
My only concern is that after an auto update yesterday, I got an email this morning that backup transport had failed, turns out that the update had cleared these changes.
I’m going to try copy this script (with the fix) into a custom script, and will update tomorrow how that goes.
May 11, 2018 - 12:42 am
Hmmm, update, I can’t get the custom script right, and it’s too late to continue tonight.