All Linux HowTo's

Rsync and the “Argument list too long” problem

Share This:

If you try to rsync a subset of many files from a single directory, you might get the error “Argument list too long”. Actually, you can get this error with many bash commands. This article explains how to work around it.

rsync -avz /images/* cdn.example.com:/images/
-bash: /usr/bin/rsync: Argument list too long

The solution is to either remove the “*” from the source as in the following example:

rsync -avz /images/ cdn.example.com:/images/

Or do things a little smarter. Here’s the two step process. Here’s how to do it.

Create a list of the files (in this case we want pictures) you want to copy:

find /images/ -name *.png > /tmp/my_image_list.txt

And now we feed that into rsync:

rsync -avz --files-from=/tmp/my_image_list.txt / cdn.example.com:/images/

The last command needs an explanation. The “rsync -avz” says to use archive mode, be verbose and compress the session. The second part “–files-from=/tmp/my_image_list.txt” is how we feed the source list into rsync. Then we have the “/” by its self. This is the starting point (or reference point) and can be changed if you wanted to change where to get the files from. Finally we have the target of “cdn.example.com:/images/” on the remote machine.

Leave a Reply

Your email address will not be published. Required fields are marked *