Rsync usage and ‘Argument list too long’ error

Rsync beats using SCP if you’re trying to copy files from one directory/machine to another. Definitely use rsync if you already have a partial copy and want to copy only new/changed files. This will save you having to copy any files that already exist and are the same.

E.g.

rsync -avz user1@machine1.fqdn:/source/directory/with/all/the/files/* /target/directory/

The -avz options are for:
-a archive mode, which includes a lot of things:

-r recursive (assess/copy subdirectories)
-l copy symlinks as symlinks
-p preserve pemissions
-t preserve modification times
-g preserve group
-o preserve owner (but this will only work if you are super-user)
-D (try and transfer block and special devices - only works as super-user)

-v verbose output
-z compress if possible

There’s also the -n switch which basically executes a dry run and shows you what would happen if you were to run the command.
The -e switch allows you to specify the shell you want to use - if for some reason it’s not picking the one you want. The default is SSH unless you have an environment variable RSYNC_RSH set.

However, sometimes when you use * to say all files in a certain directory e.g.

rsync -e ssh -avz user1@machine1.fqdn:/source/directory/with/all/the/files/* .

rsync will respond with: Argument list too long. This is because the *nix shell expansion creates a list of every single file which can be quite a few!

So, the solution is this:
rsync -e ssh -avz user1@machine1.fqdn:/source/directory/with/all/the/files/. .

Watch out for those trailing slashes - they decide if rsync will create a directory into which to put all the files…

Tags: argument, common, errors, list, long, options, rsync, too