LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 11-18-2009, 06:36 AM   #1
psynce_friction
LQ Newbie
 
Registered: Jan 2009
Posts: 9

Rep: Reputation: 0
shell script question


hello all

I have a tiny script which I am trying to get working. It takes a list of file names e.g XXXXXXXXXXXXXX_right, XXXXXXXXXXXXXX_left

The script reads through the list strips the _left and _right identifies uniq names and prints a list of unique names with the _left, _right reattached
Do to this i have

for i in $(cat input.file|sed 's/_[a-z]*//' |sort |uniq -c |grep -v '2 ');
do grep $i input.file ;done

this should produce a list of approx 14600 file names however what i actually get is a list of 223512606 file names

can anyone see where i have gone wrong?

Thanks in advance
 
Old 11-18-2009, 07:43 AM   #2
indiajoe
Member
 
Registered: Jan 2009
Location: India
Distribution: Porteus atma
Posts: 84

Rep: Reputation: 21
Hi
Hint: You did the loop for each 14600 files 14600 times. The number of files you got is square of the number of files.
The problem is in how piping works.
-Cheers
indiajoe
 
Old 11-18-2009, 07:49 AM   #3
Disillusionist
Senior Member
 
Registered: Aug 2004
Location: England
Distribution: Ubuntu
Posts: 1,013

Rep: Reputation: 83
Have you thought about using a while loop?

Code:
$(cat input.file|sed 's/_[a-z]*//' |sort |uniq -c |grep -v '2 ')|while read i
do
 grep $i input.file
done
 
Old 11-18-2009, 08:00 AM   #4
indiajoe
Member
 
Registered: Jan 2009
Location: India
Distribution: Porteus atma
Posts: 84

Rep: Reputation: 21
Hi
Just check whether the following script will work.
Code:
IFSOLD=$IFS 
IFS=\n 
for i in $(cat input.file|sed 's/_[a-z]*//' |sort |uniq -c |grep -v '2 '| cut -b 9-); do grep $i input.file ;done 
IFS=$IFSOLD
Replace 9- with the number of characters you want remove to get the file name without any white spaces or number.
Somebody pls point out a neater way of doing the above step without all pipings. There should be a better way.
-Cheers
indiajoe

Last edited by indiajoe; 11-18-2009 at 08:47 AM. Reason: Made a terrible mistake.
 
Old 11-18-2009, 08:18 AM   #5
psynce_friction
LQ Newbie
 
Registered: Jan 2009
Posts: 9

Original Poster
Rep: Reputation: 0
Hi indiajoe

ran your script and it says
zsh: argument list too long: grep

doing some error checking and i ran

head step2b.out|sed 's/_[a-z]*//' |sort |uniq -c |sed 's/\t1//' |grep -v '2 '

which produces an out put

1 FE4758201C23O2
1 FE4758201CV20O
1 FE4758201D3YER
1 FE4758201DCUH9
1 FE4758201DN2UI
1 FE4758201EFEIT
1 FE4758201EKY4O
1 FE4758201ENBE3
1 FE4758201EQESX
1 FE4758201EUG4B

is the space and one before the file name causing a problem. I have tried to add another grep cmd to remove it but can't get it to work
 
Old 11-18-2009, 08:28 AM   #6
psynce_friction
LQ Newbie
 
Registered: Jan 2009
Posts: 9

Original Poster
Rep: Reputation: 0
yep that was it thanks all much appreciated
 
Old 11-18-2009, 08:48 AM   #7
indiajoe
Member
 
Registered: Jan 2009
Location: India
Distribution: Porteus atma
Posts: 84

Rep: Reputation: 21
Hi
Sorry for the mistake.
I have corrected it. Please post back if you get a better method.
Thanks
indiajoe
 
Old 11-18-2009, 09:12 AM   #8
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,695
Blog Entries: 5

Rep: Reputation: 240Reputation: 240Reputation: 240
how does your input file look like? what should your output be?
 
Old 11-18-2009, 11:32 AM   #9
Disillusionist
Senior Member
 
Registered: Aug 2004
Location: England
Distribution: Ubuntu
Posts: 1,013

Rep: Reputation: 83
use sed:

Code:
sed 's/[1-9] //'
EDIT:-

use the -u option on sort.
Code:
IFSOLD=$IFS 
IFS=\n 
for i in $(cat input.file|sed 's/_[a-z]*//' |sort -u| cut -b 9-); do grep $i input.file ;done 
IFS=$IFSOLD

Last edited by Disillusionist; 11-18-2009 at 11:42 AM.
 
Old 11-20-2009, 08:27 AM   #10
psynce_friction
LQ Newbie
 
Registered: Jan 2009
Posts: 9

Original Poster
Rep: Reputation: 0
as i mentioned there was some space and a 1 followed by a spave before the file name i.e. -----1-filename and the cript was being run for each word in the line. I included sed 's/ 1 //'to get rid of this and it worked fine

for i in $(cat file.in|sed 's/_[a-z]*//' |sort |uniq -c |sed 's/ 1 //' |grep -v '2 '); do grep $i file.in ;done

cheers

Brian

Last edited by psynce_friction; 11-20-2009 at 08:38 AM.
 
Old 11-21-2009, 04:26 AM   #11
Disillusionist
Senior Member
 
Registered: Aug 2004
Location: England
Distribution: Ubuntu
Posts: 1,013

Rep: Reputation: 83
The spaces followed by the number and then a space is because you are using uniq -c

If you removed the uniq -c and used sort -u, you would not then need to change the output.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell Script Question ytnexus Linux - Newbie 3 06-28-2009 09:22 PM
Shell Script Question abdul_zu Programming 7 10-06-2005 03:33 AM
shell script question Whiteghost Programming 17 09-10-2005 06:52 PM
shell script question yoderp Programming 6 06-16-2005 04:47 PM
Shell Script Question swinchen Programming 1 08-20-2004 02:09 PM


All times are GMT -5. The time now is 06:05 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration