PROGRAMS

# rsync

## Overview

rsync is a file copying utility for Linux which is similar to scp (secure copy), but with more functionality. It can be used to copy/sync files locally or between computers connected on a network. Unlike scp, it can compare the source and destination and only copy over the differences, reducing the amount of data transferred and the transfer time. It even supports the resuming of partially transferred files. For this reason it is very popular for backup purposes and other situations where either large files or a large amount of files need to be transferred.

rsync can copy/sync locally or across a network.

It has the following basic syntax:

ln -nfs $HOME/Backups/back-$date $HOME/Backups/current Another interesting side-note for those interested is that at the core of the compare algorithm is a MD5 checksum and a rolling checksum. This speeds up the checking, rather than comparing the files bit-by-bit. However, this can lead to errors. The probability of rsync believing two files are in sync, but actually are not, requires both a collision (a collision is when different inputs product the same output) in the MD5 and rolling checksum, which sits around the 2^-160 mark (very unlikely). ## How To Incrementally Copy A Large File From Server To Local Folder If you want to copy a large file, you may want to do it incrementally because of disconnection issues, or to resume copying at a different time. rsync supports this with the --append option. $ rsync username@ip_address:/directory_on_server/file_on_server.big --progress --append

--progress is useful as it continuously prints the percentage progress of the operation.

## Including And Excluding Files

rsync supports a glob-like syntax for matching against files/directories you wish to include/exclude.

### Excluding Files

You can use the --exclude-from flag to pass in a file containing a list of files/directories to exclude.

* matches any partial substring of a single directory. ** matches any substring of the path.

If you want to match against a .git directory at the root level of your directory tree:

$rsync --exclude=".git" If you want to match against __cache__ directories that could be at any level in your directory tree: $ rsync --exclude="**/__cache__"

*** can be used with --include so that when matching against directories, all sub-directories and files below it will also be included.

An alternative to using rsync’s include/exclude rules is to combine rsync with the popular file-finding utility find.

### Including Files

You can choose to only include specific files by first matching them with --include, and then excluding everything with --exclude='*':

For example, to copy all *.png files from ~/source/ to ~/destination/:

\$ rsync -r --include="*.png" --exclude='8' ~/source/ ~/destination/

## Combining rsync With find

find source_dir -iname '*.jpg' -print0 |  rsync -0 -v --files-from=- . destination_dir/