Using RSYNC in Data Backup Solution

Novell Cool Solutions: Tip
By Gary Childers

Digg This - Slashdot This Posted: 17 Nov 2004

I needed to use RSYNC outside of Nterprise Branch Office for a data backup solution, so that I no longer would need to perform tape backups at remote office locations. Here is my solution. The data is copied daily using rsync from a remote server to a central office server, which is then centrally backed up to tape.

My Central Office server runs NW6.5 with the RSYNC component installed. My remote office server has rsync v.2.6.2, downloaded from the Novell developer site.

My REMOTE server runs a syncusrs.ncf file (scheduled to run daily via Remote Manager) to push its data to the RSYNC server:

rsync -rav --volume=DATA: users/ RSYNC1::USERS  --delete --exclude-from=SYS:rsync/excludes.txt

This mirrors the DATA:\USERS directory from the REMOTE server to the specified path on the RSYNC server.

The -rav means -r for recursive, -a for archive, -v for verbose. The source string is "--volume=DATA: users/" (note the trailing "/"). The destination string is "RSYNC1::REMOTEUSERS" (the server name can be resolved via the HOSTS file, and the REMOTEUSERS corresponds to the section specified in the rsyncd.conf file on the RSYNC server). The double-colon "::" specifies that rsync communication will occur via TCP, default port 873. The -- delete switch will delete files on the target that have been deleted on the source (use carefully). The --exclude-from switch will exclude copying files that are listed in the specified text file (like, *.mp3).

See the rsync man pages for a full list of rsync options (google rsync man page).

Since rsync does not save the NetWare attributes on the files, I run this trustusers.ncf one minute before the rsync:

TRUSTEE SAVE DATA:USERS DATA:USERS\trustees.txt

Then the trustees.txt file gets synched with the data. Thus if I have to restore, I can also later restore all the NetWare permissions. TRUSTEE.nlm can be downloaded here.

In the event that I want to perform a full data restore of the entire USERS directory, I can pull data back from the RSYNC server to the source server with a restusrs.ncf file:

rsync -rav RSYNC1::USERS --volume=DATA: users --update

This will restore any files deleted since the last backup, by reversing the source and target strings. The --update switch will ensure that any files modified since the last backup will not be overwritten.

This is a "brute force" restore, I realize. If used, it would also restore deleted files in subdirectories that weren't meant to be restored.

If I need to perform a more selective restore of data, say, down to the directory level, I can execute the following command on the REMOTE server:

rsync -rav RSYNC1::REMOTEUSERS/admin/foo/bar/ --volume=DATA: users/admin/foo/bar --update

This will restore all files in the data:/users/admin/foo/bar directory, while preserving any that were updated since the last backup (rsync). The trailing "/" in the source string is important -- meaning all files/folders in that directory, rather than the directory itself.

Example

My RSYNC server has this section in the rsyncd.conf file:

[REMOTEUSERS]
   path = DATA:/rsync/REMOTE/USERS
   comment = USERS backup area for REMOTE server
   read only = no
   use chroot = no 
   timeout = 3600
   transfer logging = yes
   use lfs=yes
   hosts allow=10.1.1.203
   hosts deny=*

(This is easily configured from Remote Manager on the RSYSNC server)

My REMOTE server has the excludes.txt file to exclude certain file types from the transfer:

#EXCLUDES.TXT
*.mp3
*.wma
*.rma
*.mov
*.vid

If you have any questions you may contact Gary at gchilders@alphanumeric.com

Reader Comments

• Excellent. Already using Rsync outside BO to backup remote database over the BM VPN. First 1GB database backup took 2 hrs, delta updates takes 10 minutes.
• not realy clearly
• Extra tools!
• Helped me make sense of the config files - got the job done!
• Also look at www.nasbackup.com for a gui based Windows solution in combination with rsync on netware.

Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com