HowTo

Rsync Backup Server

Backing-up data can be frustrating and time consuming. Dumping data to tape is slow and fairly expensive. A DLT or ATO tape can set you back some serious bucks. If you have multiple *nix servers on a LAN or across a WAN, the situation becomes more complicated, time consuming and expensive.

One great way to simplify backup for multiple hosts is to use tar over ssh to do a push backup to a central storage node. In this scenario, you would use a storage node with a lot of disk space to stage the data before dumping it to tape. That is a great solution, but managing a bunch of ssh scripts and keys on a bunch of servers can become a hassle. Rsyncd helps simplify your backup nightmares by a file share from which your storage node can perform a pull backup with security options and without keys or passwords. The last statement assumes you are operating on an internal network where you trust the storage node administrators. 

There are multiple steps to this HowTo, but you do it in two phases; client and server. The client is the computer that has the data you want to backup and the server is the storage node. The naming convention is sort of backwards because the client is running the Rsyncd server and the storage node uses the Rsync client to sync with the source.

You'll want to setup an Rsync share on each server (client) that has data you want to backup. In the example below, I am sharing the backup snapshot directory from my LAMP Rotating Backup HowTo. The advantage of using this solution in conjunction with the LAMP Rotating backup is that the storage node does not have to do any of the rotating because the client lampbackup.sh backup script is self-rotating.

Setting up an Rsync share on the client:

On Fedora, RHEL or CentOS, use yum to install rsync and rsyncd

yum install rsync

You will need to add an Rsyncd startup script to /etc/init.d

#!/bin/bash
#
# chkconfig:   2345 50 50
# description: The rsync daemon

# source function library
. /etc/rc.d/init.d/functions

PROG='/usr/bin/rsync'
BASE=${0##*/}

# Adapt the --config parameter to point to your rsync daemon configuration
# The config file must contain following line:
# pid file = /var/run/<filename>.pid
# Where <filename> is the filename of the init script (= this file)
OPTIONS="--daemon --config=/etc/rsyncd.conf"

case "$1" in
 start)
   echo -n $"Starting $BASE: "
   daemon --check $BASE $PROG $OPTIONS
   RETVAL=$?
   [ $RETVAL -eq 0 ] && touch /var/lock/subsys/$BASE
   echo
   ;;
 stop)
   echo -n $"Shutting down $BASE: "
   killproc $BASE
   RETVAL=$?
   [ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/$BASE
   echo
   ;;
 restart|reload)
   $0 stop
   sleep 1
   $0 start
   ;;
 *)
   echo "Usage: $0 {start|stop|restart|reload}" >&2
   exit 1
   ;;
esac

exit 0
 

Start Rsyncd on boot by issuing the command below

chkconfig rsyncd on

Start the Rsyncd daemon on Fedora, RHEL and CentOS

service rsyncd start

Now setup your Rsyncd share and restrict access to your storage node. In this case, your storage node has an IP address of 192.168.1.254
Notice the trailing slash on the path to the share source.


log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
use chroot = no 
# Your shared directory goes here
[backup]
path = /home/backup/snapshots/
comment = Rsync backup share
read only = no
list = yes
# auth users = username
# secrets file = /etc/rsyncd.scrt
auth hosts = 192.168.1.254, 127.0.0.1

As you can see above, you can easily share the snapshots directory without having to deal with a username and password or ssh keys if you use theauth hosts option. Again, this is assuming that you are operating on a secured, internal network and you trust the systems administrator managing the storage node. You can further secure the access to the client system and the storage node by using an iptables rule like the one below

iptables -A INPUT -p tcp --dport 873 -s 192.168.1.1/24 -j ACCEPT

Setup the storage node

You do not need to run Rsyncd on the storage node, but you do need to have Rsync installed. Now that your have one or more clients sharing data for backup, just setup a simple shell script that syncs with the various sources and run it as a cron job. No need to worry about rotating or cleaning the backups since the source is doing that for you.

#!/bin/bash
# Simple Rsync script to sync with Rsyncd servers
# Notice the trailing slash in the source and none in the destination


rsync -aurv --progress --delete serverx.yourdomain.com::backup/ /storage 

rsync -aurv --progress --delete servery.yourdomain.com::backup/ /storage 

rsync -aurv --progress --delete serverz.yourdomain.com::backup/ /storage 

exit 0

I like to use one big volume to store and share my backups. I name the archives using the current date and hostname which makes it easy to find the backup you need when it comes time to do a restore; for example, you would see something like the list below if you did an ls -la on the backup storage volume.

[root@nas storage]# ls -la
total 5336
drwxr-xr-x 2 root root    4096 2009-09-06 22:57 .
drwxr-xr-x 3 root root    4096 2009-09-06 15:27 ..
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-06.serverx.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-07.serverx.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-08.serverx.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-09.serverx.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-10.serverx.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-06.servery.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-07.servery.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-08.servery.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-09.servery.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-10.servery.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-06.serverz.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-07.serverz.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-08.serverz.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-09.serverz.yourdomain.net.tgz
-rw-r--r-- 1 root root 2722026 2009-09-06 23:00 2009-09-10.serverz.yourdomain.net.tgz

Archive the backups

If you still need to archive to tape, the storage volume can be dumped to tape using dump or tar and rotated off-site. You will find that you will use less tapes when using a central storage solution verses a staggard multi-node solution.

Share the storage node

You may want to share the storage node using NFSSFTPFTPCIFS or DAV to make it easy for you or others to resore data. I use the tgz archives because it makes it easy to preserve permissions on the files and directories of the source data, but you could apply the same concept to folders archived using rsync or cp on the client.

 

 

Back To Articles