LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   little tricks to speed things up... (https://www.linuxquestions.org/questions/linux-newbie-8/little-tricks-to-speed-things-up-886770/)

AndrewJS 06-16-2011 04:53 PM

little tricks to speed things up...
 
Hi there

I quite new to using UNIX, however I have a certain amount of experience using a UNIX-based astronomy package called IRAF (Image Reduction and Analysis Facility).

In IRAF, there are certain commands that are useful when copying and otherwise manipulating files....for example, if I have a list of file names stored in list.txt such as:

file1.fit
file2.fit
file3.fit
etc..
#note that .fit files are the image file format used in IRAF

I can then copy (and rename) the images by the following:

Code:

imcopy @list.txt @list.txt//copied
where the "@" is a symbol which means do it in a batch, ie. look at each .fit file in the list individually and copy it but this time with the name as specified by the "@list.txt//copied". to clarify, in the above command, the first parameter (@list.txt) is the input list and the second parameter (@list.txt//copied) is the output list.

so to clarify what the input and output would be for the above line of code:

input output
file1.fit file1copied.fit
file2.fit file2copied.fit
file3.fit file2copied.fit
etc.. etc...

and this brings me to my second part of the question:

it is hopefully clear from the input and output as I just stated it, that the "//" command appends an additional substring to the names contained within the file. This is very useful and a variation of this is that you can also prepend ie. if it were copied//@list.txt instead of @list.txt//copied the result would be:

input output
file1.fit copiedfile1.fit
file2.fit copiedfile2.fit
file3.fit copiedfile2.fit
etc.. etc...

The .fit file format is specific to astronomy and what I am seeking to find out is if UNIX has operators equivelant to "@" and "//" as I think it will allow me to transfer text files (or any other kind of file much more quickly that I currently know how to in UNIX), ie. if that above operators worked with the unix cp command (which it doesn't), to transfer from files from one directory to the other I could use the following command:

Code:

cp @list.txt "pathofnewdirectory/"//@list.txt
so in the above example if I had files in my list.txt such as:

file.txt
file2.txt
file3.txt

the cp command would copy these into the path of my chosen directory...ie:

pathofnewdirectory/file.txt
pathofnewdirectory/file2.txt
pathofnewdirectory/file3.txt

to summarise: are there UNIX operators equivelant to "@" and "//" as I specified above?
thakns

chrism01 06-16-2011 08:06 PM

Here's a quick n dirty soln
Code:

for file in `cat f.list`
do
    cp $file tdir/str${file}
done

You could make that more flexible by passing in 'str' as a param if you want; something like
Code:

cp $file tdir/${1}${file}
Here are some good links:
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/

MTK358 06-16-2011 08:07 PM

Use $(command) instead of backticks.

It can't be confused with sungle quotes, nests easily, and doesn't treat backslashes differently inside.

chrism01 06-16-2011 08:12 PM

:) old habits die hard

Valery Reznic 06-17-2011 12:35 AM

Quote:

Originally Posted by chrism01 (Post 4388081)
:) old habits die hard

I personally prefer `command` and not $(command)
- $(command) is bash specific and can't be used with /bin/sh, that is not bash (on Solaris for sure, may be Debian's dash too)
- `command` can not be nested. When it's come to scripting I am paranoid and testing exit status of each command, so instead of
Code:

for file in `cat f.list`
do
    cp $file tdir/str${file}
done

I will do
Code:

res=`cat f.list` || exit
for file in $res
do
    cp $file tdir/str${file}
done

So, nesting (for me) is never needed. And using `command` help to avoid temptation :)

The only drawback is that "`" can be confused with "'".

Well... Each choice has a price

MTK358 06-17-2011 09:36 AM

Quote:

Originally Posted by Valery Reznic (Post 4388196)
The only drawback is that "`" can be confused with "'".

There are more drawbacks. See my previous post.

Valery Reznic 06-17-2011 10:34 AM

Quote:

Originally Posted by MTK358 (Post 4388632)

It can't be confused with sungle quotes, nests easily, and doesn't treat backslashes differently inside.

1. Agreed
2. I never use nested $(command), because after each command I'd like to test exit status
3. doesn't treat backslashes differently inside - can you explain it, please

MTK358 06-17-2011 11:05 AM

Quote:

Originally Posted by Valery Reznic (Post 4388682)
3. doesn't treat backslashes differently inside - can you explain it, please

http://wiki.bash-hackers.org/syntax/expansion/cmdsubst

http://www.linuxquestions.org/questi...kticks-131463/

selfprogrammed 06-17-2011 03:57 PM

Writing little bash scripts like this can take an entire day (or two), such that one is tempted to just write C code (which one then finds takes 3 or 4 days, by time you add all the extra features that you initially did not plan on).

Off the top of my head, untested, but close to what I usually do.
Probably will have fix that $2".fit" somehow.

mkdir tmptmptmp
cp $1 tmptmptmp
rename ".fit" $2".fit" tmptmptmp/*
mv tmptmptmp/* $3
rmdir tmptmptmp

I have some faint memory of some utility having a copy and rename, but cannot remember where. Maybe midnight commander (mc), or something similar.

Reuti 06-17-2011 04:32 PM

Back to the original question:

If you just want to copy the files to a new directory without renaming (as you mention at the end of your post), there is nothing special to do:
Code:

$ cp *.fit path_to_dir
will copy each file to the destination directory. If you want to rename them in the same step, mcp can do it:
Code:

$ mcp "*.fit" "path_to_dir/#1.copied.fit"

AndrewJS 06-18-2011 08:56 AM

that looks very straight forward, thanks Reuti!

chrism01 06-19-2011 11:07 PM

Quote:

Writing little bash scripts like this can take an entire day (or two)
You are kidding right? ;)


All times are GMT -5. The time now is 07:27 AM.