rsync

What is rsync?

Rysnc helps you transfer data from one location to another in an efficient manner. It is one of those tools that you learn to use and wonder how you lived without it. Rsync is the de facto standard in backup solutions because of its flexibility and power.

Rsync checks each file and transfers only what has changed. What does this mean exactly? Rsync will actually look and see what in the file has changed and upload only the part of the file that has changed. Unlike ftp and other transfer solutions rsync doesn’t simply re-upload the entire file.

The difference in the files are then compressed (an optionally encrypted through ssh) then sent so the transfer uses the minimal amount of bandwidth. Rsync is often used by Amazon S3 users as they must pay for bandwidth. When you are paying bandwidth bit by bit you can’t afford anything but rsync.

Rsync is Efficient but What Else Can it Do?

As you can probably tell by the name rsync is really good at syncing files across a network. If a file has changed it can detect the change and transfer only that change. This makes rsync a perfect candidate for doing incremental backups or mirroring a website.

For example to mirror one folder to another you could do:

rsync –av /path/to/source /home/nixtutor/rsync/daily

You can also use the same technique to sync from one computer to another:

rsync –av /path/to/source user@nixutor.com:/home/nixtutor/rsync/daily

By default this will only upload new files and changes but not delete or remove files that no longer exist. To do this you can add the –delete flag. This is rsync’s way of protecting yourself from mirroring a blank directory.

rsync –av –delete /path/to/source user@nixutor.com:/home/nixtutor/rsync/daily

Want to Sync Specific Files?

In this example we only sync .iso files.

rsync -zrv –include=”*.iso” host:/home/nixtutor /home/

Things to Keep in Mind

  • Rsync is powerful but unforgiving
  • Rsync follows the unix methodology, do one thing and do it well. Thus it doesn’t provide encryption only efficient file transferring. Run rsync through SSH if you need encryption.
  • Rsync will not delete files that have been removed unless you supply the –delete flag.
  • Windows doesn’t keep file modification times to better than two seconds. Use the –modify-window=2 option to get around this when syncing to Windows file shares.

Most backup senarios can be done with rsync and cron. For an example on how to create a daily weekly backup scheme check out, Use Rsync for Daily, Weekly and Full Monthly Backups.

Subscribe via Email

Subscribe to Nixtutor via Email

Enter your email address:

Was this information useful?

3 Responses

  1. pavium

    8-16-2009

    Yes, you’ll need -modify-window=2 for everyday backups of Windows shares, but when Daylight Saving kicks in or out, expect backup times to increase alarmingly as Windows changes the timestamp of every file by an hour.

  2. Yang

    9-6-2009

    I’m told that rsnapshot relies on rsync to trasmit files on SSH from and to remote servers? Is that true?

    So if I am using rsnapshot to back things up already, there’s no need to use rsync right? I mean I’m pretty new to Debian and with rsnapshot I save all the trouble to set up cron and things like that that would otherwise have to be set up with rsync.

  3. jpaugh64

    10-12-2009

    @Yang: I wouldn’t be surprised if rsnapshot uses rsync. That’s how a lot of graphical programs came to be in the Linux world: they often don’t have anything more than a graphical interface to the user, and an interface to some command-line program.

    GUIs are nice for new people; but, eventually, it becomes far easier to use the command line, where all options and settings are only a switch away.