Synchronize Directories with rsync

Novell Cool Solutions: Trench
By Kirk Coombs

Digg This - Slashdot This Posted: 29 Sep 2005

Applies to:

• SUSE Linux
• SUSE Linux Enterprise Server
• Open Enterprise Server
• Novell Linux Desktop

Disclaimer: rsync is a very robust program with many features. This tip only touches on a small subset of them, and is only intended to be an introduction.

Why Use rsync?

There are many cases when it is useful to keep two directories in sync with each other. Examples include mirroring a FTP server and performing backups. rsync is a great utility for this purpose. Some of the major features include:

• Transferring only the changed portions of files when syncing, decreasing the amount of data transfered.
• Compression can be employed to further reduce the transfered data.
• Supports preserving permissions, owners, groups, links, etc.
• Can use a remote shell (rsh or ssh) or the rsync daemon (rsyncd).
• Access can be authenticated or anonymous.

A Simple rsync Example

For those who have used rcp or scp before, rsync will seem very familiar. Suppose you own a server called voyager which has a data directory /data. You would like to set-up a rsync process on another computer, enterprise which creates a backup of voyager's /data directory. The /data directory is quite large and must be synced daily, so only the changes should transmitted. It is also important to preserve links, users, permissions, etc. The access should not be anonymous.

Because voyager is running the SSH daemon (sshd), there is no configuration required. Simply make sure that both voyager and enterprise have the rsync package installed. To sync the data, simply issue these commands on enterprise:

# mkdir /voyager_data/
# rsync -avz user@voyager:/data/ /voyager_data/

The command syntax is simple. rsync takes any options (avz), then the source sever and directory, followed by the destination. By using a ':' after the source server we are telling rsync to use the remote shell protocol. If the rsync daemon is desired, use a '::'.

The options used were:

-a   Archive mode. Preserve timestamps, users, permissions, etc.

-v    Verbose mode.

-z    Enable compression.

Some rsync Daemon Examples

Now suppose that the data on voyager should be accessed anonymously. This case works well for FTP servers, making it easier for others to mirror the server. 

The first step is to configure and start the rsync daemon (rsyncd). Settings for the daemon are stored in /etc/rsyncd.conf. The default configuration file contains some good defaults. For more information on the options available in the configuration file, see the rsyncd.conf(5) man page. To configure anonymous access, edit rsyncd.conf to have the following:

gid = users
read only = true
use chroot = true
transfer logging = true
log format = %h %o %f %l %b
log file = /var/log/rsyncd.log

[Data]
        path = /data
        comment = An Example of Anonymous Access

Then install the daemon into the default run level and start it.

# insserv rsyncd
# rcrsyncd start

Now, on enterprise the data can be synced with:

> rsync -avz voyager::Data voyager_data/

If the connection needs to be authenticated, the changes are minor. Users need to be added to /etc/rsyncd.secrets in the following format:

username:password

For example:

kirk:asdf
joe:qwerty

Then, modify rsyncd.conf to have the following:

gid = users
read only = true
use chroot = true
transfer logging = true
log format = %h %o %f %l %b
log file = /var/log/rsyncd.log

[Data]
        path = /data
        comment = An Example of Authenticated Access
        auth users = kirk joe
        secrets file = /etc/rsyncd.secrets

Now, sync the data with:

rsync -avz kirk@137.65.208.2::Data voyager_data/

The password is prompted for, and the data is synced. Prompting for the password can be avoided by placing it in the RSYNC_PASSWORD environment variable, or by using the--password-file option.

Reader Comments

• very good, could be even more detailed.
• gold star. gives users something to start with.. and tells them how to learn more.. man pages..
• I thought it will help me to sync _only_ directories. Without content.
• A good explanation. Links to more detailed pages on rsync or a more detailed explanation here would be good.

Novell Cool Solutions (corporate web communities) are produced by WebWise Solutions. www.webwiseone.com