How to copy a column

sycamorex · 11-23-2010, 07:53 PM

See post#12

satimis · 11-23-2010, 08:47 PM

Quote:

Originally Posted by sycamorex

See post#12

Hi sycamorex,

Wonderful suggestion. Thanks.

It saves me lot of time dragging along on this small problem. NEdit supports copy and paste column. vi also can copy column but I haven't find a way to paste the same to another file.

Anyway, lot of thanks to folks providing me assistance on my problem.

B.R.
satimis

fordeck · 11-23-2010, 10:33 PM

If you don't need column #1 then you might try this:

Code:

my_machine:~$ sed -e 's/^  //' -e 's/^[0-9]* //' test.txt | awk '{print $2 "\t" $3}'
Yield	Rainfall
60	8
50	10
70	11
70	10
80	9
50	9
60	12
40	11

Use sed to strip off column #1 and pipe its output to awk and print the columns you need.

Regards,

Fordeck

satimis · 11-23-2010, 10:53 PM

Quote:

Originally Posted by fordeck

If you don't need column #1 then you might try this:

Code:

my_machine:~$ sed -e 's/^  //' -e 's/^[0-9]* //' test.txt | awk '{print $2 "\t" $3}'
Yield	Rainfall
60	8
50	10
70	11
70	10
80	9
50	9
60	12
40	11

Use sed to strip off column #1 and pipe its output to awk and print the columns you need.

Regards,

Fordeck

Hi Fordeck,

Thanks.

What are the function of ^ and // on the streamline editor?

B.R.
satimis

fordeck · 11-24-2010, 06:02 AM

We have two "sed" search and replace commands each begins with -e.

In the sed command 's/^ //' is a search and replace function where the search is contained between the first foward slash and the second forward slash.

In this case "^ " which searches for a line that begins (^) with two spaces.

Between the second forward slash and the third forward slash are what you want to replace the two spaces with. In this case nothing.

In the second search and replace command 's/^[0-9]* //' we are searching for a line that begins (^) with 0 or more numbers followed by a space and again will replace it with (//) nothing.

So essentially what it would do based on the data set you provided is strip off the first field of information. Now "Year" becomes column/field #1 "Yield" becomes field #2 etc. etc.. These are then piped to awk and $2 followed by a tab followed by $3 are printed to your screen or could be redirected to another file.

I hope this helps explain what the command did.

Regards,

Fordeck

satimis · 11-24-2010, 08:36 AM

Quote:

Originally Posted by fordeck

We have two "sed" search and replace commands each begins with -e.

In the sed command 's/^ //' is a search and replace function where the search is contained between the first foward slash and the second forward slash.

In this case "^ " which searches for a line that begins (^) with two spaces.

Between the second forward slash and the third forward slash are what you want to replace the two spaces with. In this case nothing.

In the second search and replace command 's/^[0-9]* //' we are searching for a line that begins (^) with 0 or more numbers followed by a space and again will replace it with (//) nothing.

So essentially what it would do based on the data set you provided is strip off the first field of information. Now "Year" becomes column/field #1 "Yield" becomes field #2 etc. etc.. These are then piped to awk and $2 followed by a tab followed by $3 are printed to your screen or could be redirected to another file.

I hope this helps explain what the command did.

Hi Fordeck,

Thanks for your detail explanation and time.

My understanding is as follows: (If I'm wrong pls correct me)

1)
The 1st "search and replace" is to search the line beginning with "2 spaces", i.e. the 1st line and replace "2 spaces" with nothing, i.e. stripping it away

2)
The 2nd "search and replace" is to search the lines beginning with "0-9" and replace them with nothing, i.e. stripping them away.

What is the function * after [0-9] ?

What is the function of "\t" ?

A side question is it possible printing the output as a line separated with a) a space or b) with a comma ",", i.e. column becoming row?

TIA

B.R.
satimis

fordeck · 11-24-2010, 08:48 AM

The "*" in this regular expression 's/^[0-9]* //' basically means zero or more of the preceding character.

The "\t" is used to insert a tab between fields in the output. So you could instead replace it with " " for a space or "," to use a comma.

Regarding your question "column becoming a row?" could you show me an example of what you are talking about?

Regards,

Fordeck

satimis · 11-24-2010, 09:26 AM

Quote:

Originally Posted by fordeck

- snip -

Regarding your question "column becoming a row?" could you show me an example of what you are talking about?

Hi,

Example:

Code:

Yield 60 50 70 70 80 50 60 40
Rainfall 8 10 11 10 9 9 12 11

Yield,60,50,70,70,80,50,60,40
Rainfall,8,10,11,10,9,9,12,11

Others note with thanks

B.R.
satimis

fordeck · 11-24-2010, 09:29 AM

You can use the "paste" command to transpose a column to a row. Check out man paste.

Regards,

Fordeck

lumak · 11-24-2010, 09:36 AM

... If this is a homework assignment and you are learning about scripting, then this answer might be more complicated and un ethical to answer here.

For example, each number is right justified with the title of each column. This means the spacing has to be real spaces and not a tab character.

The only way to transpose the columns with the same formatting would be a complicated script that will count how wide the column is supposed to be, then re align each column individually as it prints each line.

For example:

Code:

  Year Yield Rainfall Temperature
1 1963	  60        8          56

to trans pose Rainfall with Yield, I would count each columns width by its title:

COL1=Year COL2=Rainfall COL3=Yield COL4=Temperature
COLW1=4 COLW2=8 COLW3=5 COLW4=11

then reprint the first column as needed by determining what needed to be swapped

echo " $COL1 $COL2 $COL3 $COL4"

then some complicated function to re print each line to columns:

# Loop all lines

echo "$LINE_NUMBER "
# Loop this line
# Don't forget to swap it as you are doing this
DATA=`awk "{print \$$COL_NUMBER}"`
CURWIDTH=<current data width>
let "SPACE_WIDTH = COLW1 - CURWIDTH"
SPACES=< some conversion from SPACE_WIDTH to actual spaces >
# Note '-n' to prevent a new line
echo -n "$SPACES $DATA "
# Some terminating check
# then echo a new line
echo

# Some terminating check

Enjoy

satimis · 11-25-2010, 05:17 AM

Hi lumak,

Thanks for your advice.

Quote:

Originally Posted by lumak

... If this is a homework assignment and you are learning about scripting, then this answer might be more complicated and un ethical to answer here.

Non of abovementioned applies to my need.

I'm learning R (R-Project) and need many datasets to plot graphs. If I can't find the dataset needed on R database I have to build it/them myself. I can find many sample dataset on Internet but some of them need modified. That is the cause starting this thread.

Quote:

For example, each number is right justified with the title of each column. This means the spacing has to be real spaces and not a tab character.

The only way to transpose the columns with the same formatting would be a complicated script that will count how wide the column is supposed to be, then re align each column individually as it prints each line.

For example:

Code:

  Year Yield Rainfall Temperature
1 1963	  60        8          56

to trans pose Rainfall with Yield, I would count each columns width by its title:

COL1=Year COL2=Rainfall COL3=Yield COL4=Temperature
COLW1=4 COLW2=8 COLW3=5 COLW4=11

then reprint the first column as needed by determining what needed to be swapped

echo " $COL1 $COL2 $COL3 $COL4"

then some complicated function to re print each line to columns:

# Loop all lines

echo "$LINE_NUMBER "
# Loop this line
# Don't forget to swap it as you are doing this
DATA=`awk "{print \$$COL_NUMBER}"`
CURWIDTH=<current data width>
let "SPACE_WIDTH = COLW1 - CURWIDTH"
SPACES=< some conversion from SPACE_WIDTH to actual spaces >
# Note '-n' to prevent a new line
echo -n "$SPACES $DATA "
# Some terminating check
# then echo a new line
echo

# Some terminating check

Whether run following commands:-

Code:

sed -e 's/^  //' -e 's/^[0-9]* //' test.txt \
echo " $COL1 $COL2 $COL3 $COL4" \
echo "$LINE_NUMBER " \
DATA=`awk "{print \$$COL_NUMBER}"` \
CURWIDTH=<current data width> \ # what shall I replace "current data width"
let "SPACE_WIDTH = COLW1 - CURWIDTH" \
SPACES=< some conversion from SPACE_WIDTH to actual spaces > \ # what shall I replace On < some ..... space >
echo -n "$SPACES $DATA " \
echo

to print the output on terminal
???

TIA

B.R.
satimis

satimis · 11-25-2010, 06:31 AM

Quote:

Originally Posted by fordeck

You can use the "paste" command to transpose a column to a row. Check out man paste.

Hi Fordeck,

I got it done with "paste"

Code:

$ sed -e 's/^  //' -e 's/^[0-9]* //' test.txt | awk '{print $1}' | paste -s -d .
Year.1963.1964.1965.1966.1967.1968.1969.1970.

$ sed -e 's/^  //' -e 's/^[0-9]* //' test.txt | awk '{print $1}' | paste -s -d /
Year/1963/1964/1965/1966/1967/1968/1969/1970/

$ sed -e 's/^  //' -e 's/^[0-9]* //' test.txt | awk '{print $1}' | paste -s -d " "
Year 1963 1964 1965 1966 1967 1968 1969 1970

Thanks

Edit:

Reference:-
UNIX BASH scripting
http://unstableme.blogspot.com/2009/...ples-uses.html

B.R.
satimis

lumak · 11-25-2010, 02:05 PM

If it's a script and you use 'echo' inside of it, then you can send the output of the script to a file.

Code:

sh myscript.sh > create_and_or_overwright.txt
sh myscript.sh >> create_and_or_append.txt

My pseudo type code was aimed at keeping the format the same as the example data sheet while transposing the columns. That is, right justified numbers with each column. If this is for personal use, you could probably just save it as a csv file. I'm sure anything dealing with data points is going to understand a text file with "Comma Separated Values". The \t tab character would probably be good enough too.

It all depends then on if you need the related data points in rows or columns and all the other options mentioned above would suffice.