LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 11-23-2010, 07:53 PM   #16
sycamorex
LQ Veteran
 
Registered: Nov 2005
Location: London
Distribution: Slackware64-current
Posts: 5,829
Blog Entries: 1

Rep: Reputation: 1227Reputation: 1227Reputation: 1227Reputation: 1227Reputation: 1227Reputation: 1227Reputation: 1227Reputation: 1227Reputation: 1227

See post#12
 
Old 11-23-2010, 08:47 PM   #17
satimis
Senior Member
 
Registered: Apr 2003
Posts: 3,673

Original Poster
Rep: Reputation: 55
Quote:
Originally Posted by sycamorex View Post
See post#12
Hi sycamorex,

Wonderful suggestion. Thanks.

It saves me lot of time dragging along on this small problem. NEdit supports copy and paste column. vi also can copy column but I haven't find a way to paste the same to another file.

Anyway, lot of thanks to folks providing me assistance on my problem.

B.R.
satimis
 
Old 11-23-2010, 10:33 PM   #18
fordeck
Member
 
Registered: Oct 2006
Location: Utah
Posts: 520

Rep: Reputation: 61
If you don't need column #1 then you might try this:


Code:
my_machine:~$ sed -e 's/^  //' -e 's/^[0-9]* //' test.txt | awk '{print $2 "\t" $3}'
Yield	Rainfall
60	8
50	10
70	11
70	10
80	9
50	9
60	12
40	11
Use sed to strip off column #1 and pipe its output to awk and print the columns you need.

Regards,

Fordeck
 
Old 11-23-2010, 10:53 PM   #19
satimis
Senior Member
 
Registered: Apr 2003
Posts: 3,673

Original Poster
Rep: Reputation: 55
Quote:
Originally Posted by fordeck View Post
If you don't need column #1 then you might try this:


Code:
my_machine:~$ sed -e 's/^  //' -e 's/^[0-9]* //' test.txt | awk '{print $2 "\t" $3}'
Yield	Rainfall
60	8
50	10
70	11
70	10
80	9
50	9
60	12
40	11
Use sed to strip off column #1 and pipe its output to awk and print the columns you need.

Regards,

Fordeck
Hi Fordeck,

Thanks.

What are the function of ^ and // on the streamline editor?

B.R.
satimis
 
Old 11-24-2010, 06:02 AM   #20
fordeck
Member
 
Registered: Oct 2006
Location: Utah
Posts: 520

Rep: Reputation: 61
We have two "sed" search and replace commands each begins with -e.

In the sed command 's/^ //' is a search and replace function where the search is contained between the first foward slash and the second forward slash.

In this case "^ " which searches for a line that begins (^) with two spaces.

Between the second forward slash and the third forward slash are what you want to replace the two spaces with. In this case nothing.

In the second search and replace command 's/^[0-9]* //' we are searching for a line that begins (^) with 0 or more numbers followed by a space and again will replace it with (//) nothing.

So essentially what it would do based on the data set you provided is strip off the first field of information. Now "Year" becomes column/field #1 "Yield" becomes field #2 etc. etc.. These are then piped to awk and $2 followed by a tab followed by $3 are printed to your screen or could be redirected to another file.

I hope this helps explain what the command did.

Regards,

Fordeck
 
Old 11-24-2010, 08:36 AM   #21
satimis
Senior Member
 
Registered: Apr 2003
Posts: 3,673

Original Poster
Rep: Reputation: 55
Quote:
Originally Posted by fordeck View Post
We have two "sed" search and replace commands each begins with -e.

In the sed command 's/^ //' is a search and replace function where the search is contained between the first foward slash and the second forward slash.

In this case "^ " which searches for a line that begins (^) with two spaces.

Between the second forward slash and the third forward slash are what you want to replace the two spaces with. In this case nothing.

In the second search and replace command 's/^[0-9]* //' we are searching for a line that begins (^) with 0 or more numbers followed by a space and again will replace it with (//) nothing.

So essentially what it would do based on the data set you provided is strip off the first field of information. Now "Year" becomes column/field #1 "Yield" becomes field #2 etc. etc.. These are then piped to awk and $2 followed by a tab followed by $3 are printed to your screen or could be redirected to another file.

I hope this helps explain what the command did.
Hi Fordeck,

Thanks for your detail explanation and time.

My understanding is as follows: (If I'm wrong pls correct me)

1)
The 1st "search and replace" is to search the line beginning with "2 spaces", i.e. the 1st line and replace "2 spaces" with nothing, i.e. stripping it away

2)
The 2nd "search and replace" is to search the lines beginning with "0-9" and replace them with nothing, i.e. stripping them away.


What is the function * after [0-9] ?

What is the function of "\t" ?


A side question is it possible printing the output as a line separated with a) a space or b) with a comma ",", i.e. column becoming row?

TIA

B.R.
satimis
 
Old 11-24-2010, 08:48 AM   #22
fordeck
Member
 
Registered: Oct 2006
Location: Utah
Posts: 520

Rep: Reputation: 61
The "*" in this regular expression 's/^[0-9]* //' basically means zero or more of the preceding character.

The "\t" is used to insert a tab between fields in the output. So you could instead replace it with " " for a space or "," to use a comma.

Regarding your question "column becoming a row?" could you show me an example of what you are talking about?

Regards,

Fordeck
 
Old 11-24-2010, 09:26 AM   #23
satimis
Senior Member
 
Registered: Apr 2003
Posts: 3,673

Original Poster
Rep: Reputation: 55
Quote:
Originally Posted by fordeck View Post
- snip -

Regarding your question "column becoming a row?" could you show me an example of what you are talking about?
Hi,

Example:
Code:
Yield 60 50 70 70 80 50 60 40
Rainfall 8 10 11 10 9 9 12 11

Yield,60,50,70,70,80,50,60,40
Rainfall,8,10,11,10,9,9,12,11
Others note with thanks

B.R.
satimis
 
Old 11-24-2010, 09:29 AM   #24
fordeck
Member
 
Registered: Oct 2006
Location: Utah
Posts: 520

Rep: Reputation: 61
You can use the "paste" command to transpose a column to a row. Check out man paste.


Regards,

Fordeck

Last edited by fordeck; 11-24-2010 at 09:30 AM.
 
Old 11-24-2010, 09:36 AM   #25
lumak
Member
 
Registered: Aug 2008
Location: Phoenix
Distribution: Arch
Posts: 799
Blog Entries: 32

Rep: Reputation: 111Reputation: 111
... If this is a homework assignment and you are learning about scripting, then this answer might be more complicated and un ethical to answer here.

For example, each number is right justified with the title of each column. This means the spacing has to be real spaces and not a tab character.

The only way to transpose the columns with the same formatting would be a complicated script that will count how wide the column is supposed to be, then re align each column individually as it prints each line.

For example:
Code:
  Year Yield Rainfall Temperature
1 1963	  60        8          56
to trans pose Rainfall with Yield, I would count each columns width by its title:

COL1=Year COL2=Rainfall COL3=Yield COL4=Temperature
COLW1=4 COLW2=8 COLW3=5 COLW4=11

then reprint the first column as needed by determining what needed to be swapped

echo " $COL1 $COL2 $COL3 $COL4"

then some complicated function to re print each line to columns:

# Loop all lines

echo "$LINE_NUMBER "
# Loop this line
# Don't forget to swap it as you are doing this
DATA=`awk "{print \$$COL_NUMBER}"`
CURWIDTH=<current data width>
let "SPACE_WIDTH = COLW1 - CURWIDTH"
SPACES=< some conversion from SPACE_WIDTH to actual spaces >
# Note '-n' to prevent a new line
echo -n "$SPACES $DATA "
# Some terminating check
# then echo a new line
echo

# Some terminating check


Enjoy

Last edited by lumak; 11-24-2010 at 09:38 AM.
 
Old 11-25-2010, 05:17 AM   #26
satimis
Senior Member
 
Registered: Apr 2003
Posts: 3,673

Original Poster
Rep: Reputation: 55
Hi lumak,

Thanks for your advice.

Quote:
Originally Posted by lumak View Post
... If this is a homework assignment and you are learning about scripting, then this answer might be more complicated and un ethical to answer here.
Non of abovementioned applies to my need.

I'm learning R (R-Project) and need many datasets to plot graphs. If I can't find the dataset needed on R database I have to build it/them myself. I can find many sample dataset on Internet but some of them need modified. That is the cause starting this thread.

Quote:
For example, each number is right justified with the title of each column. This means the spacing has to be real spaces and not a tab character.

The only way to transpose the columns with the same formatting would be a complicated script that will count how wide the column is supposed to be, then re align each column individually as it prints each line.

For example:
Code:
  Year Yield Rainfall Temperature
1 1963	  60        8          56
to trans pose Rainfall with Yield, I would count each columns width by its title:

COL1=Year COL2=Rainfall COL3=Yield COL4=Temperature
COLW1=4 COLW2=8 COLW3=5 COLW4=11

then reprint the first column as needed by determining what needed to be swapped

echo " $COL1 $COL2 $COL3 $COL4"

then some complicated function to re print each line to columns:

# Loop all lines

echo "$LINE_NUMBER "
# Loop this line
# Don't forget to swap it as you are doing this
DATA=`awk "{print \$$COL_NUMBER}"`
CURWIDTH=<current data width>
let "SPACE_WIDTH = COLW1 - CURWIDTH"
SPACES=< some conversion from SPACE_WIDTH to actual spaces >
# Note '-n' to prevent a new line
echo -n "$SPACES $DATA "
# Some terminating check
# then echo a new line
echo

# Some terminating check
Whether run following commands:-
Code:
sed -e 's/^  //' -e 's/^[0-9]* //' test.txt \
echo " $COL1 $COL2 $COL3 $COL4" \
echo "$LINE_NUMBER " \
DATA=`awk "{print \$$COL_NUMBER}"` \
CURWIDTH=<current data width> \ # what shall I replace "current data width"
let "SPACE_WIDTH = COLW1 - CURWIDTH" \
SPACES=< some conversion from SPACE_WIDTH to actual spaces > \ # what shall I replace On < some ..... space >
echo -n "$SPACES $DATA " \
echo
to print the output on terminal
???

TIA


B.R.
satimis
 
Old 11-25-2010, 06:31 AM   #27
satimis
Senior Member
 
Registered: Apr 2003
Posts: 3,673

Original Poster
Rep: Reputation: 55
Quote:
Originally Posted by fordeck View Post
You can use the "paste" command to transpose a column to a row. Check out man paste.
Hi Fordeck,

I got it done with "paste"
Code:
$ sed -e 's/^  //' -e 's/^[0-9]* //' test.txt | awk '{print $1}' | paste -s -d .
Year.1963.1964.1965.1966.1967.1968.1969.1970.

$ sed -e 's/^  //' -e 's/^[0-9]* //' test.txt | awk '{print $1}' | paste -s -d /
Year/1963/1964/1965/1966/1967/1968/1969/1970/

$ sed -e 's/^  //' -e 's/^[0-9]* //' test.txt | awk '{print $1}' | paste -s -d " "
Year 1963 1964 1965 1966 1967 1968 1969 1970
Thanks

Edit:

Reference:-
UNIX BASH scripting
http://unstableme.blogspot.com/2009/...ples-uses.html


B.R.
satimis

Last edited by satimis; 11-25-2010 at 06:34 AM.
 
Old 11-25-2010, 02:05 PM   #28
lumak
Member
 
Registered: Aug 2008
Location: Phoenix
Distribution: Arch
Posts: 799
Blog Entries: 32

Rep: Reputation: 111Reputation: 111
If it's a script and you use 'echo' inside of it, then you can send the output of the script to a file.

Code:
sh myscript.sh > create_and_or_overwright.txt
sh myscript.sh >> create_and_or_append.txt
My pseudo type code was aimed at keeping the format the same as the example data sheet while transposing the columns. That is, right justified numbers with each column. If this is for personal use, you could probably just save it as a csv file. I'm sure anything dealing with data points is going to understand a text file with "Comma Separated Values". The \t tab character would probably be good enough too.

It all depends then on if you need the related data points in rows or columns and all the other options mentioned above would suffice.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
awk multiple column into single column ilukacevic Programming 49 07-19-2010 07:23 PM
Read text file column by column RVF16 Programming 11 05-31-2009 07:16 AM
Concatenate column 1 and column 2 of related lines cgcamal Programming 4 11-20-2008 10:43 AM
sql command to copy column from a table tooparam Programming 1 06-19-2007 05:21 AM
boot error after using Copy Commander to copy debian system onto larger new drive Interceptor Linux - Hardware 7 05-04-2003 12:40 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 08:13 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration