LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   How do I scp all files except for ... ? (https://www.linuxquestions.org/questions/linux-newbie-8/how-do-i-scp-all-files-except-for-866905/)

ccornchip 03-07-2011 03:19 AM

How do I scp all files except for ... ?
 
Hi, don't really know where to ask this question, so I'm putting it here (after all, I am fairly new).

Ok, so. I want to `scp' a whole bunch of files to another server :

$ scp * username@otherserver:/dir/dir/dir/

... which works fine.
Next, I want to do the same thing but for all files EXCEPT for the files with 'foo' or 'bar' in the name. After sniffing around a while I found several different solutions one of them was to pipe ls into grep -v ... twice:

$ ls | grep -v 'foo' | grep -v 'bar'
file.1
lala.2
...
yadda.3

Ok, that worked great! Now let's see if I have this `command substitution` thing worked out:

$ echo `ls | grep -v 'foo' | grep -v 'bar'`
file.1 lala.2 ... yadda.3

Hooray! So now I can do my scp ...

$ scp `ls | grep -v 'foo' | grep -v 'bar'` username@otherserver:/dir/dir/dir/
file.1: No such file or directory
lala.2: No such file or directory
...
yadda.3: No such file or directory
: No such file or directory

So, what went wrong?
How can I copy a bunch of files except for those with foo or bar in the name?
Is scp just annoying like this and I should be using something else??

At the moment I'm moving the foos and bars into a temporary directory, scp-ing everything, then moving them back, but after the tenth time it starts getting a bit tedious.

repo 03-07-2011 03:37 AM

I don't think scp alone can do what you ask.
Perhaps you can use rsync

Kind regards

colucix 03-07-2011 03:45 AM

Hi and welcome to LinuxQuestions!

I tried you code and it works for me. Anyway you can try some alternatives:
Code:

find . -maxdepth 1 -type f -not \( -name \*foo\* -o -name \*bar\* \) -exec scp {} username@otherserver: \;
in this case the scp command is executed once for each file, so that you should have set up a passwordless connection with otherserver, otherwise you will be prompted for password every time. To execute only one scp command you can pass the output of find to xargs:
Code:

find . -maxdepth 1 -type f -not \( -name \*foo\* -o -name \*bar\* \) | xargs -I{} scp {} username@otherserver:
Using find you have the advantage to refine the search criteria at your pleasure. Moreover you can easily manage files with spaces in their name if you add the -print0 option to find and -0 to xargs. In this way different items are separated by a null character and the spaces in file names (if any) are treated as part of the names themselves.

You can also try your ls + grep commands:
Code:

ls | grep -v 'foo' | grep -v 'bar' | xargs -I{} scp {} username@otherserver:
Hope this helps.

ccornchip 03-07-2011 04:12 AM

For some reason, xargs didn't like the -I option ... which is odd since it exists in the usage.
Quote:

xargs: invalid option -- I
Usage: xargs [-0prtx] [-E eof-str] [-e[eof-str]] [-I replace-str]
[-i[replace-str]] [-L max-lines] [-l[max-lines]] [-n max-args]
[-s max-chars] [-P max-procs] [--null] [--eof[=eof-str]]
[--replace[=replace-str]] [--max-lines[=max-lines]] [--interactive]
[--max-chars=max-chars] [--verbose] [--exit] [--max-procs=max-procs]
[--max-args=max-args] [--no-run-if-empty] [--version] [--help]
[command [initial-arguments]]
However, `command-substituting` the output of find into scp worked! (apologies if I'm mis-using lingo)
$ scp `find . -maxdepth 1 -type f -not \( -name \*foo\* -o -name \*bar\* \)` username@otherserver:/dir/dir/dir/

SUCCESS!
Maybe it was the ./ before the filesnames that did the trick.

Thank you!

colucix 03-07-2011 04:28 AM

Actually the -I option is lowercase in your version of xargs. Are you running a Linux OS or some othe *nix flavor?

Regarding the command substitution using find in place of ls + grep, it should not make any difference. Indeed the ls + grep worked on my system. FYI a more tricky find command to allow -exec with multiple arguments (only one scp command executed) is:
Code:

find . -maxdepth 1 -type f -not \( -name \*foo\* -o -name \*bar\* \) -exec sh -c 'scp "$@" username@otherserver:/dir/dir/dir/' sh {} +
Take in mind that -exec has two different syntax:
Code:

-exec command \;
-exec command {} +

the latter works only if the argument list {} is immediately before the plus sign. Since in the scp command the last argument must be the remote destination directory, we invoke a shell using the -c option to specify the command to execute and we retrieve the file names as a list of arguments "$@". Maybe this is a bit more difficult to remember. On the other end the result is quite the same as your command line. Not to mention that if you don't have GNU find, the -exec version with multiple arguments might not be available.


All times are GMT -5. The time now is 02:46 PM.