Rsync Examples to Transfer Files in Linux

On Linux operating system, the 'rsync' stands for Remote Sync. It's an utility used to synchronize (copy) files and directories from a source (SRC) to a destination (DEST).

Files and directories could be synced on a local host or to/from a remote host. On the SRC, files and directories will be synchronized. Whereas, on the DEST, the synced files and directories will be taken place.

In addition, rsync command can recursively copy files and directories, copy symbolic links, preserve (permissions, group, modification time and ownership) file identity.

How to unzip zip files in Linux

To view this video please enable JavaScript, and consider upgrading to a web browser that supports HTML5 video

How to unzip zip files in Linux

Rsync can use two different ways for syncing/copying files or directories:

• Use remote shell: ssh or rsh
• Use rsync daemon directly through TCP

Even more, rsync uses delta-transfer algorithm which only copy/sync the different data between SRC and DEST.

This tutorial will go through many ways to use rsync command through hands-on practices and examples. We will also explain the detailed of the most common options of rsync command.

Rsync command syntax

Before deep-diving into the ways of using rsync command, let's take a look at its syntax:

rsync [options] <SRC_PATH> <DEST_PATH>

1. Copy/Sync file on the local machine

Rsync command can be used to copy/sync file on your local machine.

Assuming that you want to copy a file from home directory of user 'linoxide' named '/home/linoxide' to folder '/opt/test'.

$ sudo rsync -zvh /home/linoxide/computer-networking.pdf /opt/test/
computer-networking.pdf

sent 1.01K bytes  received 35 bytes  2.10K bytes/sec
total size is 3.03K  speedup is 2.89

Where:

-z: compress

-v: verbose the output

-h: human-readable

2. Copy/Sync directory on the local machine

To transfer all the files from SRC local directory to DEST directory, run the rsync command with option -zavh.

Assuming that you want to copy home directory of user 'linoxide' named '/home/linoxide' to folder '/opt/backup'.

$ sudo rsync -zavh /home/linoxide/ /opt/backup/

sending incremental file list
./
.bash_history
.bash_logout
.bashrc
.lesshst
.profile
.sudo_as_admin_successful
.viminfo
computer-networking.pdf
.ssh/
.ssh/known_hosts

sent 4.85K bytes  received 198 bytes  10.10K bytes/sec
total size is 9.59K  speedup is 1.90

Where:

-a: archive data during the copy/sync process

You can verify whether the rsync command was successful or not? Run the below command:

$ ls /opt/backup/ -la
total 40
drwxr-xr-x 3 linoxide linoxide 4096 Thg 6 15 16:02 .
drwxr-xr-x 4 root     root     4096 Thg 6 15 16:25 ..
-rw------- 1 linoxide linoxide  817 Thg 6 15 16:01 .bash_history
-rw-r--r-- 1 linoxide linoxide  220 Thg 6 11 15:58 .bash_logout
-rw-r--r-- 1 linoxide linoxide 3771 Thg 6 11 15:58 .bashrc
-rw-rw-r-- 1 linoxide linoxide 3028 Thg 2 25  2017 computer-networking.pdf
-rw------- 1 linoxide linoxide   40 Thg 6 15 15:51 .lesshst
-rw-r--r-- 1 linoxide linoxide  807 Thg 6 11 15:58 .profile
drwx------ 2 linoxide linoxide 4096 Thg 6 13 17:58 .ssh
-rw-r--r-- 1 linoxide linoxide    0 Thg 6 15 16:02 .sudo_as_admin_successful
-rw------- 1 linoxide linoxide  680 Thg 6 14 16:50 .viminfo

3. Rsync over ssh with specific Port

You can use ssh (secure shell) to copy/sync data, your data will be transferred in a secure connection with encryption. Nobody can read your data while it is being transferred on the Internet. When you use rsync command, you need to provide the user/root password to accomplish the particular task. Using the SSH option will send your logins in an encrypted manner so that your password will be safe.

Copy a file from a remote to local with a specific ssh port. In order to specify a protocol with rsync command, you need to use -e option with protocol name you want to use.

For example, the following rsync command will copy file 'deploy.yaml' via a remote shell from a remote machine to home directory of user 'linoxide'.

$ rsync -avz -e "ssh -p 22" [email protected]:/home/cas/deploy.yaml /home/linoxide/
[email protected]'s password: 
receiving incremental file list

sent 20 bytes  received 60 bytes  32.00 bytes/sec
total size is 239  speedup is 2.99

4. Copy/Sync files and directories from local machine to remote machine

Supposing that you want to copy/sync the folder 'test-data' in home directory of user 'linoxide' in local host to home directory of a remote machine '[email protected]', run command:

$ rsync -azrvh /home/linoxide/computer-networking.pdf [email protected]: 
[email protected]'s password: 
sending incremental file list
computer-networking.pdf

sent 1.04K bytes  received 35 bytes  430.80 bytes/sec
total size is 3.03K  speedup is 2.81

Where:

-r: recursive copy

5. Copy/Sync files and directories from remote machine to local machine

The rsync command helps you transfer a remote directory 'document' to be copied in your local host.

Assuming that you want to copy/sync files/directories from a remote host '[email protected]' to home directory of user 'linoxide'.

For example:

$ rsync -zavhr [email protected]:document /home/linoxide
[email protected]'s password: 
receiving incremental file list
document/
document/kubernetes.txt

sent 47 bytes  received 139 bytes  74.40 bytes/sec
total size is 0  speedup is 0.00 

6. Rsync -include and -exclude options

These two options allow us to include and exclude files by specifying parameters with these option helps us to specify those files or directories which you want to include in your sync and exclude files and folders with you don’t want to be transferred.

Here in this example, rsync command will include those files and directory only which starts with 'k' and exclude all other files and directory.

$ rsync -avhz -e "ssh -p 22" --include 'k*' --exclude '*' [email protected]: /home/linoxide

[email protected]'s password: 
receiving incremental file list
./
k8s.yaml
kubernetes.yaml
kuber/

sent 88 bytes  received 280 bytes  147.20 bytes/sec
total size is 73  speedup is 0.20

7. Set max size of files

You can specify the max file size to be transferred or sync. You can do it with '--max-size' option.

In the following example, max file size is 200k, so this command will transfer only those files which are equal or smaller than 200k.

$ rsync -zavhr --max-size='200k' [email protected]: /home/linoxide
[email protected]'s password: 
receiving incremental file list
.bash_history
.bash_logout
.bashrc
.profile
.sudo_as_admin_successful
.viminfo
computer-networking.pdf

sent 2.58K bytes  received 51.60K bytes  21.67K bytes/sec
total size is 113.24K  speedup is 2.09

8. Delete source files after successful transfer

Now, suppose you have a web server and a data backup server, you created a daily backup and synced it with your backup server, now you don’t want to keep that local copy of backup in your web server. So, will you wait for the transfer to complete and then delete that local backup file manually? Of Course NO. This automatic deletion can be done using '--remove-source-files' option.

$ ls /home/directory
computer-networking.pdf  deploy.yaml  document  k8s.yaml  kuber  kubernetes.yaml

------
$ rsync -azvh --remove-source-files /home/linoxide/deploy.yaml [email protected]:
[email protected]'s password: 
sending incremental file list

sent 70 bytes  received 20 bytes  36.00 bytes/sec
total size is 239  speedup is 2.66

Run the 'ls' command again, you can see the file 'deploy.yaml' was deleted on SRC.

$ ls
computer-networking.pdf  document  k8s.yaml  kuber  kubernetes.yaml

9. Find the difference in files and directories between SRC and DEST

In order to find any difference in the files of directories between SRC and DEST, you can run rsync command with  '-i' option.

$ rsync -avzi /home/linoxide/cloud [email protected]:
[email protected]'s password: 
sending incremental file list
cd+++++++++ cloud/
<f+++++++++ cloud/computer-networking.pdf
<f+++++++++ cloud/k8s.yaml
<f+++++++++ cloud/kubernetes.yaml
cd+++++++++ cloud/document/
<f+++++++++ cloud/document/kubernetes.txt
cd+++++++++ cloud/kuber/
<f+++++++++ cloud/kuber/test.txt

sent 1,472 bytes  received 127 bytes  1,066.00 bytes/sec
total size is 3,101  speedup is 1.94

10. Limit the bandwidth

You can set the limitation of the bandwidth when running rsync command with '--bwlimit=<KB/s>' option.

Assuming that you want to limit the data transfer rate to 300KB/s, run the command:

$ rsync -avzh --bwlimit=300 /home/linoxide/coursera.png [email protected]:

[email protected]'s password: 
sending incremental file list
coursera.png

sent 2.93M bytes  received 35 bytes  234.48K bytes/sec
total size is 3.00M  speedup is 1.02

11. Resume the transferring when failed in scp

Sometimes, you have to transfer a very large file via scp command but got error while copying and could not start transferring again using scp command due to the large file size and time consumption.

In this situation, you can use rsync command to start copying the file from where it got the error. For example, copying a large ISO file in home directory to remote host '[email protected]'

$ scp /home/linoxide/ubuntu-18.04.4-live-server-amd64.iso [email protected]:

[email protected]'s password: 
ubuntu-18.04.4-live-server-amd64.iso                                                                                                                                              32%  278MB  86.7MB/s   00:06 ETA

The process has been killed by user. At that time, 32% of file was transferred. Now, you can continue the current work by running rsync:

$ rsync -P -avzh ubuntu-18.04.4-live-server-amd64.iso [email protected]:

[email protected]'s password: 
sending incremental file list
ubuntu-18.04.4-live-server-amd64.iso
        912.26M 100%   15.89MB/s    0:00:54 (xfr#1, to-chk=0/1)

sent 541.25M bytes  received 132.44K bytes  9.25M bytes/sec
total size is 912.26M  speedup is 1.69

rsync command to resume the transferring file 12. Display progress while transferring data

If you want to see the transferred progress when running rsync command, you can use '--progress' option.

It will show the files and the time remaining to complete the copy.

For example:

$ rsync -avh --progress /home/linoxide/cloud/ [email protected]:
[email protected]'s password: 
sending incremental file list
./
computer-networking.pdf
          3.03K 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=8/10)
coursera.png
          3.00M 100%  114.38MB/s    0:00:00 (xfr#2, to-chk=7/10)
k8s.yaml
             39 100%    1.52kB/s    0:00:00 (xfr#3, to-chk=6/10)
kubernetes.yaml
             34 100%    1.33kB/s    0:00:00 (xfr#4, to-chk=5/10)
ubuntu-18.04.4-live-server-amd64.iso
        912.26M 100%   89.60MB/s    0:00:09 (xfr#5, to-chk=4/10)
document/
document/kubernetes.txt
              0 100%    0.00kB/s    0:00:00 (xfr#6, to-chk=1/10)
kuber/
kuber/test.txt
              0 100%    0.00kB/s    0:00:00 (xfr#7, to-chk=0/10)

sent 915.49M bytes  received 164 bytes  79.61M bytes/sec
total size is 915.26M  speedup is 1.00

rsync command with progress Conclusion

In this tutorial, we learned how to use rsync command with some examples for copying or syncing files/directories.

Thanks for reading and please leave your suggestion in the below comment section.