Trying to copy file output by read while command

00mpa · 01-15-2015, 03:07 PM

The point of this is to look for certain types of files from a ton of directories and consolidate those files into one directory. What further complicates things is the fact that there are files with different version numbers and I can only copy over the latest one.

Below is how I tried to do that:

if [ -d ~/directory/*${tid}* ]; then
cd ~/directory/*${tid}*;
ls -rt file_L1b*.csv | tail -1

while read -r line; do
cp $line ~/test_dir/;
done
fi

The output that I get is :

usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file target_file
cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file ... target_directory

Therefore, what do I need to change so that I can use the output of the read -r line which is a file name and copy it to the ~/test_dir/ directory. This was the only way that I could come up with so I'm not sure if there is an easier way to accomplish my goal.

Thanks!

grail · 01-15-2015, 07:13 PM

Ok ... a few things:

1. Please place data/code inside [code][/code] tags to make it more legible

2. Your use of wildcards is not going to workout so well, for example:

Code:

if [ -d ~/directory/*${tid}* ]; then

Now I will assume that 'tid' is being successfully set elsewhere in your code. This line says are there any directories below the path ~/directory that contain 'tid'.
This is quite different from saying is there a single directory in the path that exists and contains 'tid'

3. Just as there may be multiple directories tested, the cd may also get confused if there are multiple directories to change to

Now we get to the section causing your error:

4. The while loop has nothing feeding it so it would be waiting for user input on standard in (in your case the keyboard). Assuming you hit the enter key without typing anything, this would then fill the
'line' variable with nothing, hence your cp command would look like :- cp ~/test_dir/
As you can see this is an incomplete copy command and hence your error

The resolution is that you need to feed the file names to the while command. To have the output of a command sent to while it needs to be of the form:

Code:

while ...
do
  ...
done< <(your command here)

Please note there is a space between the 2 < symbols

michaelk · 01-15-2015, 11:23 PM

You do not need the while loop if you assign the output of the ls pipe command to a variable.

Code:

$latest_file=$( ls -rt file_L1b*.csv | tail -1 )
cp $latest_file ~/test_dir/

mina86 · 01-16-2015, 06:40 AM

Quote:

Originally Posted by grail

2. Your use of wildcards is not going to workout so well, for example:

Code:

if [ -d ~/directory/*${tid}* ]; then

[…] This line says are there any directories below the path ~/directory that contain 'tid'.

Just to make it clear, this line is actually broken if there is more than one directory matching the wildcard. Observe:

Code:

$ mkdir /tmp/a && cd /tmp/a
$ mkdir foo1 foo2
$ tid=foo
$ test -d *${tid}*
-bash: test: foo1: binary operator expected
$ echo $?
2

Quote:

Originally Posted by grail

The resolution is that you need to feed the file names to the while command. To have the output of a command sent to while it needs to be of the form:

Code:

while ...
do
  ...
done< <(your command here)

A more portable way is to do:

Code:

your command here | while …; do
    …
done

but beware of setting variables inside while loop and using it outside. And of course, if file names are read from a file it’s enough to do:

Code:

while …; do
    …
done <filename