using rsync with files/folders using embedded blanks in names
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
using rsync with files/folders using embedded blanks in names
How do I use rsync to read and write when the file and folder names use embedded blanks?
DISCLAIMER -- This issues is typographically awkward. I'll try my best but please bear with me.
I have a flock of external drives, SD cards, and thumb drives that were written using embedded blanks in file and folder names. I'm tasked to consolidate these files and folders onto a linux file server. A simple copy using:
Code:
prompt$ cp -av "some\ Source" "some\ Target"
One issue,
(or similar) does not seem to be good enough.
One tangle comes from the "volume name" or "volume label" that typically results in a mount point like /media/username/some sort of string with blanks
Now when you add folder and file names you quickly run out of allowed characters on a command line in scripts or internal to utilities like rsync.
While running, rsync reports "I/O error writing..." for some blank-embedded files. It succeeds with others. The failing files can be moved manually using:
I have a lot of files and directories with spaces in the names, but haven't seen that error before.
Can you show the exact parameters you are using with rsync?
I have a lot of files and directories with spaces in the names, but haven't seen that error before.
Can you show the exact parameters you are using with rsync?
First, this is a local copy -- disk to disk on the same workstation. Okay, one is an internal (typ. source) and one is an external (typ. target)
I pull files from EXT4, without embedded blanks, without issue using:
Code:
prompt$ rsync -av /someSource/ /someTarget
Since its a local copy, I don't need or want compression.
I only get the errors when I have embedded blanks. I'll try --protect-args like *michaelk* suggested.
I have scanned and checked and tested both the source and target file systems and find no errors. When I'm doing this, I limit the workload so that the disk I/O gets most of the workstation (an 8GB, i5 Thinkpad™ laptop).
I really suspect that the embedded blanks and long path+name character counts are part of the troubles but I've no evidence ... I'll have to do a test and submit a bug report if I get proof.
the i/o error means probably the filename was split into parts and rsync wanted to write to (or read from) an invalid device/path - which was only a part of the real filename.
You need to use quotations to protect filenames against this.
you can use rsync -v to have verbose output or --itemize-changes to get more info.
You would also need to give a better example, we can only guess without seeing the actual filenames/error messages.
Looks like I may have misinterpreted the problem.
...
Let me restate things to clarify:
I start a local run using rsync. Either SOURCE or TARGET might be an internal HDD or an external usb-connected store or a media device.
Somewhere along the way, the SOURCE folder names or file names or both, start to use embedded blanks. The resulting path+filename is usually long, but nowhere near the PATH_MAX=4095 oro 4096 value.
rsync now must construct a path+filename to use against the TARGET
rsync reports "I/O error" while writing the target; there errors are consistently reported for blank-embedded path+filename on the TARGET
there are no reported I/O errors with the same operation where there are zero, none, nada, path+filename using embedded blanks
Thanks in advanced because I'm thorougly stumped,
~~~ 0;-/ Dan
I agree with pan64, find a minimal 'broken' rsync example, use -v option and show us the exact cmd, and o/p you get.
This is one of those qns where we need to be as if we are sitting in front of the terminal ourselves.
I'd also agree with TenTenths: spaces considered harmful.
Although the Unix spec says spaces are allowed in filenames, practically every *nix cmd assumes spaces means param separation, unless you take special precautions.
If these are your files, rename them without spaces - you'll thank me later as they will continue to cause problems forever otherwise.
Even if they are not, you may be able to get away with temp renaming them, then processing them, then renaming back - I've had to to that when processing MSWin sourced files - it makes *nix cmds/tools etc 'just work'
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.