LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-07-2011, 04:19 AM   #1
ccornchip
LQ Newbie
 
Registered: Nov 2010
Posts: 5

Rep: Reputation: 1
How do I scp all files except for ... ?


Hi, don't really know where to ask this question, so I'm putting it here (after all, I am fairly new).

Ok, so. I want to `scp' a whole bunch of files to another server :

$ scp * username@otherserver:/dir/dir/dir/

... which works fine.
Next, I want to do the same thing but for all files EXCEPT for the files with 'foo' or 'bar' in the name. After sniffing around a while I found several different solutions one of them was to pipe ls into grep -v ... twice:

$ ls | grep -v 'foo' | grep -v 'bar'
file.1
lala.2
...
yadda.3

Ok, that worked great! Now let's see if I have this `command substitution` thing worked out:

$ echo `ls | grep -v 'foo' | grep -v 'bar'`
file.1 lala.2 ... yadda.3

Hooray! So now I can do my scp ...

$ scp `ls | grep -v 'foo' | grep -v 'bar'` username@otherserver:/dir/dir/dir/
file.1: No such file or directory
lala.2: No such file or directory
...
yadda.3: No such file or directory
: No such file or directory

So, what went wrong?
How can I copy a bunch of files except for those with foo or bar in the name?
Is scp just annoying like this and I should be using something else??

At the moment I'm moving the foos and bars into a temporary directory, scp-ing everything, then moving them back, but after the tenth time it starts getting a bit tedious.
 
Old 03-07-2011, 04:37 AM   #2
repo
LQ 5k Club
 
Registered: May 2001
Location: Belgium
Distribution: Arch
Posts: 8,527

Rep: Reputation: 898Reputation: 898Reputation: 898Reputation: 898Reputation: 898Reputation: 898Reputation: 898
I don't think scp alone can do what you ask.
Perhaps you can use rsync

Kind regards
 
Old 03-07-2011, 04:45 AM   #3
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
Hi and welcome to LinuxQuestions!

I tried you code and it works for me. Anyway you can try some alternatives:
Code:
find . -maxdepth 1 -type f -not \( -name \*foo\* -o -name \*bar\* \) -exec scp {} username@otherserver: \;
in this case the scp command is executed once for each file, so that you should have set up a passwordless connection with otherserver, otherwise you will be prompted for password every time. To execute only one scp command you can pass the output of find to xargs:
Code:
find . -maxdepth 1 -type f -not \( -name \*foo\* -o -name \*bar\* \) | xargs -I{} scp {} username@otherserver:
Using find you have the advantage to refine the search criteria at your pleasure. Moreover you can easily manage files with spaces in their name if you add the -print0 option to find and -0 to xargs. In this way different items are separated by a null character and the spaces in file names (if any) are treated as part of the names themselves.

You can also try your ls + grep commands:
Code:
ls | grep -v 'foo' | grep -v 'bar' | xargs -I{} scp {} username@otherserver:
Hope this helps.
 
Old 03-07-2011, 05:12 AM   #4
ccornchip
LQ Newbie
 
Registered: Nov 2010
Posts: 5

Original Poster
Rep: Reputation: 1
For some reason, xargs didn't like the -I option ... which is odd since it exists in the usage.
Quote:
xargs: invalid option -- I
Usage: xargs [-0prtx] [-E eof-str] [-e[eof-str]] [-I replace-str]
[-i[replace-str]] [-L max-lines] [-l[max-lines]] [-n max-args]
[-s max-chars] [-P max-procs] [--null] [--eof[=eof-str]]
[--replace[=replace-str]] [--max-lines[=max-lines]] [--interactive]
[--max-chars=max-chars] [--verbose] [--exit] [--max-procs=max-procs]
[--max-args=max-args] [--no-run-if-empty] [--version] [--help]
[command [initial-arguments]]
However, `command-substituting` the output of find into scp worked! (apologies if I'm mis-using lingo)
$ scp `find . -maxdepth 1 -type f -not \( -name \*foo\* -o -name \*bar\* \)` username@otherserver:/dir/dir/dir/

SUCCESS!
Maybe it was the ./ before the filesnames that did the trick.

Thank you!

Last edited by ccornchip; 03-07-2011 at 05:29 AM.
 
Old 03-07-2011, 05:28 AM   #5
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
Actually the -I option is lowercase in your version of xargs. Are you running a Linux OS or some othe *nix flavor?

Regarding the command substitution using find in place of ls + grep, it should not make any difference. Indeed the ls + grep worked on my system. FYI a more tricky find command to allow -exec with multiple arguments (only one scp command executed) is:
Code:
find . -maxdepth 1 -type f -not \( -name \*foo\* -o -name \*bar\* \) -exec sh -c 'scp "$@" username@otherserver:/dir/dir/dir/' sh {} +
Take in mind that -exec has two different syntax:
Code:
-exec command \;
-exec command {} +
the latter works only if the argument list {} is immediately before the plus sign. Since in the scp command the last argument must be the remote destination directory, we invoke a shell using the -c option to specify the command to execute and we retrieve the file names as a list of arguments "$@". Maybe this is a bit more difficult to remember. On the other end the result is quite the same as your command line. Not to mention that if you don't have GNU find, the -exec version with multiple arguments might not be available.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] scp -r copies files repeatedly MorayJ Linux - Software 2 07-30-2009 06:26 PM
scp - Two computers, can only transfer files one way jqweezy Linux - Networking 2 02-13-2009 03:12 PM
SCP files from one server to another rev1976 Linux - Newbie 4 09-18-2008 09:21 PM
copy files....using scp help me suguname Mandriva 2 01-26-2005 02:14 AM
copy files....using scp help me suguname Mandriva 5 01-25-2005 06:42 AM


All times are GMT -5. The time now is 05:31 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration