Backup to Amazon S3 Bucket from CentOS 6.4

In this tutorial I’ll make mysql db backup and then upload it to AMAZON S3 Bucket using S3CMD tool and bash script. The script will also create the separate folder automatically for each day inside the bucket.

Before starting this tutorial, we need to install the EPEL for s3cmd tools:

wget http://ftp.riken.jp/Linux/fedora/epel/RPM-GPG-KEY-EPEL-6 
wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
rpm --import RPM-GPG-KEY-EPEL-6
rpm -ivh epel-release-6-8.noarch.rpm
yum repolist

Switch to the root user:

sudo su -

1

Install the s3cmd tools:

yum install s3cmd

2

After installation, run the following command in order to configure the s3cmd tools using your AMAZON ACCESS KEY and SECRET KEY.

s3cmd --configure

3

It is always a good practice to create a separate scripts directory:

mkdir /script
cd /script
nano mysqlbackuptos3.sh

4

Paste the below code into the script (Please modify the following script to suit your purposes):

#!/bin/bash
#############################################################
# Created by: Arbab Nazar (tendo.linux{at}gmail(dot)com)    #
#                                                           #                                                                                                                    #
#############################################################
## Specify the name of the database that you want to backup
DATABASE="tendodb"

## USERNAME and PASSWORD to take the backup
USER="arbab"
PASSWORD="PASSWORD"
## Defination of some necessary variables
S3_BUCKET=tendodbbackup
DATE=`date +%A-"(%d.%m.%Y)".%H`
BACKUP_LOC=/tmp/backups
S3BDIR=`date +%d.%A-"(%d.%m.%Y)"`
mysql_backup(){
/usr/bin/mysqldump -u $USER -p$PASSWORD --databases $DATABASE | gzip -9 > $BACKUP_LOC/tendodb_$DATE.sql.gz
 s3cmd ls s3://$S3_BUCKET/$S3BDIR > /tmp/log.txt
 grep -lr "$S3BDIR" /tmp/log.txt
 if [ $? -ne 0 ]
 then
 mkdir /tmp/$S3BDIR
 s3cmd put -r /tmp/$S3BDIR s3://$S3_BUCKET/
s3cmd sync -r $BACKUP_LOC/ s3://$S3_BUCKET/$S3BDIR/
else
 s3cmd sync -r $BACKUP_LOC/ s3://$S3_BUCKET/$S3BDIR/
fi
}
mysql_backup
echo "----------------------------------------------------------------------------------------------" >> /scripts/dbbackup.log
echo "Backup Successfully uploading to the S3 Bucket at `date +%A.%Y%m%d-%H.%M`" >> /script/dbbackup.log
echo "##############################################END#############################################" >> /scripts/dbbackup.log
rm -rf $BACKUP_LOC/* /tmp/$S3BDIR
exit 0

5

Create the backups directory inside the tmp directory (or anywhere else, as per your desire) to hold the mysql backup temporarily before uploading it to the s3 bucket:

mkdir /tmp/backups
chmod 0777 /tmp/backups

6

Verify the access to the bucket, where you want to upload the backup using the following command:

s3cmd ls

7

Give the execute right to the script and run it:

cd /script
chmod +x mysqlbackuptos3.sh
ls
./mysqlbackuptos3.sh

8

This bash script also creates a log file inside the /script directory:

cd /script
ls
cat dbbackup.log

9

Inside the S3 Bucket, it will create the mysql backup like this:

10

To automate this process using the cronjob, enter crontab -e at the shell prompt:

crontab -e

11

Add this line to run the script at midnight every day:

0 0 * * * /script/mysqlbackuptos3.sh > /dev/null

12

Enjoy :-)

Hope this will help you!

Please Remember me in your prayers!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: