LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-16-2011, 05:53 PM   #1
AndrewJS
LQ Newbie
 
Registered: Dec 2010
Posts: 16

Rep: Reputation: 0
little tricks to speed things up...


Hi there

I quite new to using UNIX, however I have a certain amount of experience using a UNIX-based astronomy package called IRAF (Image Reduction and Analysis Facility).

In IRAF, there are certain commands that are useful when copying and otherwise manipulating files....for example, if I have a list of file names stored in list.txt such as:

file1.fit
file2.fit
file3.fit
etc..
#note that .fit files are the image file format used in IRAF

I can then copy (and rename) the images by the following:

Code:
imcopy @list.txt @list.txt//copied
where the "@" is a symbol which means do it in a batch, ie. look at each .fit file in the list individually and copy it but this time with the name as specified by the "@list.txt//copied". to clarify, in the above command, the first parameter (@list.txt) is the input list and the second parameter (@list.txt//copied) is the output list.

so to clarify what the input and output would be for the above line of code:

input output
file1.fit file1copied.fit
file2.fit file2copied.fit
file3.fit file2copied.fit
etc.. etc...

and this brings me to my second part of the question:

it is hopefully clear from the input and output as I just stated it, that the "//" command appends an additional substring to the names contained within the file. This is very useful and a variation of this is that you can also prepend ie. if it were copied//@list.txt instead of @list.txt//copied the result would be:

input output
file1.fit copiedfile1.fit
file2.fit copiedfile2.fit
file3.fit copiedfile2.fit
etc.. etc...

The .fit file format is specific to astronomy and what I am seeking to find out is if UNIX has operators equivelant to "@" and "//" as I think it will allow me to transfer text files (or any other kind of file much more quickly that I currently know how to in UNIX), ie. if that above operators worked with the unix cp command (which it doesn't), to transfer from files from one directory to the other I could use the following command:

Code:
cp @list.txt "pathofnewdirectory/"//@list.txt
so in the above example if I had files in my list.txt such as:

file.txt
file2.txt
file3.txt

the cp command would copy these into the path of my chosen directory...ie:

pathofnewdirectory/file.txt
pathofnewdirectory/file2.txt
pathofnewdirectory/file3.txt

to summarise: are there UNIX operators equivelant to "@" and "//" as I specified above?
thakns
 
Old 06-16-2011, 09:06 PM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,240

Rep: Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324
Here's a quick n dirty soln
Code:
for file in `cat f.list`
do
    cp $file tdir/str${file}
done
You could make that more flexible by passing in 'str' as a param if you want; something like
Code:
cp $file tdir/${1}${file}
Here are some good links:
http://rute.2038bug.com/index.html.gz
http://tldp.org/LDP/Bash-Beginners-G...tml/index.html
http://www.tldp.org/LDP/abs/html/
 
Old 06-16-2011, 09:07 PM   #3
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Rep: Reputation: 720Reputation: 720Reputation: 720Reputation: 720Reputation: 720Reputation: 720Reputation: 720
Use $(command) instead of backticks.

It can't be confused with sungle quotes, nests easily, and doesn't treat backslashes differently inside.
 
1 members found this post helpful.
Old 06-16-2011, 09:12 PM   #4
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,240

Rep: Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324
old habits die hard
 
Old 06-17-2011, 01:35 AM   #5
Valery Reznic
ELF Statifier author
 
Registered: Oct 2007
Posts: 675

Rep: Reputation: 136Reputation: 136
Quote:
Originally Posted by chrism01 View Post
old habits die hard
I personally prefer `command` and not $(command)
- $(command) is bash specific and can't be used with /bin/sh, that is not bash (on Solaris for sure, may be Debian's dash too)
- `command` can not be nested. When it's come to scripting I am paranoid and testing exit status of each command, so instead of
Code:
for file in `cat f.list`
do
    cp $file tdir/str${file}
done
I will do
Code:
res=`cat f.list` || exit
for file in $res
do
    cp $file tdir/str${file}
done
So, nesting (for me) is never needed. And using `command` help to avoid temptation

The only drawback is that "`" can be confused with "'".

Well... Each choice has a price
 
Old 06-17-2011, 10:36 AM   #6
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Rep: Reputation: 720Reputation: 720Reputation: 720Reputation: 720Reputation: 720Reputation: 720Reputation: 720
Quote:
Originally Posted by Valery Reznic View Post
The only drawback is that "`" can be confused with "'".
There are more drawbacks. See my previous post.
 
Old 06-17-2011, 11:34 AM   #7
Valery Reznic
ELF Statifier author
 
Registered: Oct 2007
Posts: 675

Rep: Reputation: 136Reputation: 136
Quote:
Originally Posted by MTK358 View Post

It can't be confused with sungle quotes, nests easily, and doesn't treat backslashes differently inside.
1. Agreed
2. I never use nested $(command), because after each command I'd like to test exit status
3. doesn't treat backslashes differently inside - can you explain it, please
 
Old 06-17-2011, 12:05 PM   #8
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Rep: Reputation: 720Reputation: 720Reputation: 720Reputation: 720Reputation: 720Reputation: 720Reputation: 720
Quote:
Originally Posted by Valery Reznic View Post
3. doesn't treat backslashes differently inside - can you explain it, please
http://wiki.bash-hackers.org/syntax/expansion/cmdsubst

http://www.linuxquestions.org/questi...kticks-131463/
 
Old 06-17-2011, 04:57 PM   #9
selfprogrammed
Member
 
Registered: Jan 2010
Location: Minnesota, USA
Distribution: Slackware 13.37
Posts: 290

Rep: Reputation: 55
Writing little bash scripts like this can take an entire day (or two), such that one is tempted to just write C code (which one then finds takes 3 or 4 days, by time you add all the extra features that you initially did not plan on).

Off the top of my head, untested, but close to what I usually do.
Probably will have fix that $2".fit" somehow.

mkdir tmptmptmp
cp $1 tmptmptmp
rename ".fit" $2".fit" tmptmptmp/*
mv tmptmptmp/* $3
rmdir tmptmptmp

I have some faint memory of some utility having a copy and rename, but cannot remember where. Maybe midnight commander (mc), or something similar.

Last edited by selfprogrammed; 06-17-2011 at 04:59 PM.
 
Old 06-17-2011, 05:32 PM   #10
Reuti
Senior Member
 
Registered: Dec 2004
Location: Marburg, Germany
Distribution: openSUSE 13.1
Posts: 1,326

Rep: Reputation: 253Reputation: 253Reputation: 253
Back to the original question:

If you just want to copy the files to a new directory without renaming (as you mention at the end of your post), there is nothing special to do:
Code:
$ cp *.fit path_to_dir
will copy each file to the destination directory. If you want to rename them in the same step, mcp can do it:
Code:
$ mcp "*.fit" "path_to_dir/#1.copied.fit"
 
Old 06-18-2011, 09:56 AM   #11
AndrewJS
LQ Newbie
 
Registered: Dec 2010
Posts: 16

Original Poster
Rep: Reputation: 0
that looks very straight forward, thanks Reuti!
 
Old 06-20-2011, 12:07 AM   #12
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,240

Rep: Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324
Quote:
Writing little bash scripts like this can take an entire day (or two)
You are kidding right?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Mozilla designer says Google Chrome uses speed tricks LXer Syndicated Linux News 0 06-21-2010 11:00 PM
LXer: Tip: OpenSSH Speed Tips and Tricks LXer Syndicated Linux News 0 11-22-2008 02:30 PM
Speed things up? brunog Fedora 2 11-25-2005 12:26 PM
??X11 speed tricks?? binidiot *BSD 6 07-30-2004 09:12 AM
I want to speed up Red Hat things up but how? jimdaworm Red Hat 10 09-08-2003 02:52 PM


All times are GMT -5. The time now is 04:27 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration