LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-26-2012, 12:57 PM   #31
atjurhs
Member
 
Registered: Aug 2012
Posts: 190

Original Poster
Rep: Reputation: Disabled

here's the link

http://www.linuxquestions.org/questi...57#post4789957

I called the thread "deleting every other row, sometimes"
 
Old 09-28-2012, 09:50 AM   #32
atjurhs
Member
 
Registered: Aug 2012
Posts: 190

Original Poster
Rep: Reputation: Disabled
for some reason the output file from the first script on post 26 was not accepted by MatLab scripts. I'll go back and try the second script on post 26 and see if the MatLab script likes it any better. Otherwise I'll have to find one of the guys to help me debug the the problem, but I can't really work on this till Monday as my server is down

Tabby

Last edited by atjurhs; 09-28-2012 at 09:52 AM.
 
Old 09-28-2012, 07:11 PM   #33
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
@atjurhs: Try injecting a raw file first. When it works, see what differs it from the output of the script.
 
Old 10-01-2012, 03:34 PM   #34
atjurhs
Member
 
Registered: Aug 2012
Posts: 190

Original Poster
Rep: Reputation: Disabled
yipeeee, I found the problem

in the raw input file there is never a row that does not contain a value in at least one of the columns of data.

in the output file (from 1st of the script shown on post 26, (which runs much faster than the second)) there are several rows that will contain nothing but comas. so this output file is not the right input format for that MatLab code.

wrong format
Code:
header_3, header_5, header_8, header10_  , header11_0_0_    , header12, header13     , header14_, header15_
123 , 123456789 , 1234567 , 1234567 , 1 , 12345678 , 1 , 1 , 123456789
   ,          ,     , , ,        ,  ,  
1234567,        , 12345, , ,        ,  ,  
   ,          ,     , , ,        ,  ,  
   ,          ,     , , ,        ,  ,  
   ,          ,     , , ,        ,  ,  
   ,          ,     , , ,        , 1, 1 
   ,          ,     , , ,        ,  ,  
1234567,        ,     , , ,        ,  ,  
1234567,        , 12345, , ,        ,  ,  , 12345
123 , 123456789 , 1234567 , 1234567 , 1 , 12345678 , 1 , 1 , 123456789
   ,          ,     , , ,        ,  ,  
1234567,        , 12345, , ,        ,  ,
right format
Code:
header_3, header_5, header_8, header10_  , header11_0_0_    , header12, header13     , header14_, header15_
123 , 123456789 , 1234567 , 1234567 , 1 , 12345678 , 1 , 1 , 123456789 
1234567,        , 12345, , ,        ,  ,  
   ,          ,     , , ,        , 1, 1 
1234567,        ,     , , ,        ,  ,  
1234567,        , 12345, , ,        ,  ,  , 12345
123 , 123456789 , 1234567 , 1234567 , 1 , 12345678 , 1 , 1 , 123456789 
1234567,        , 12345, , ,        ,  ,
so all that needs to be added is to strip out those rows that only contain all comas and no values

Tabby

Last edited by atjurhs; 10-01-2012 at 03:35 PM.
 
Old 10-01-2012, 08:54 PM   #35
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
Then you could just add a test to that:
Code:
#!/bin/bash

# Change columns here. This is repetitive within the loop, but just for convenience.
REMOVE=(1 2 4 6 7 9)

# Extension of output file's name.
OUTPUTEXT='out'

IFS=','

for FILE; do
	while read LINE; do
		FIELDS=() # just to be safe
		read -a FIELDS <<< "${LINE//, /,}"
		for I in "${REMOVE[@]}"; do
			unset "FIELDS[$I - 1]"
		done
		LINE="${FIELDS[*]}"
		[[ $LINE == *[^\ ,]* ]] && echo "${LINE//,/, }"
	done < "$FILE" > "$FILE.$OUTPUTEXT"
done
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
merge columns from multiple files vijay_babu1981 Linux - Newbie 21 06-24-2014 06:59 AM
Averaging columns from multiple files carlr Programming 3 03-18-2012 01:24 AM
extracting columns from multiple files with awk orcaja Linux - Newbie 7 02-14-2012 10:24 PM
merge multiple files each with two columns. 11st col same but may have difft values newbie271 Linux - Newbie 2 01-10-2012 06:03 PM
[SOLVED] AWK (or TCL/TK): Matching rows and columns between multiple files Euler2 Programming 6 05-30-2011 06:31 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 10:07 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration