awk multiple column into single column

grail · 07-17-2010, 01:39 AM

Quote:

Originally Posted by colucix

I find the comma more elegant and quick to type.

Couldn't agree more ... mainly just got focused on solution and forgot to look at that

ilukacevic · 07-19-2010, 03:55 AM

Quote:

Originally Posted by grail

Edit: Also, maybe you could show us some actual input? (changed to protect the data of course if necessary)

Sure. Here it is (a part of it at least - but it's all the same to the end)

0.000000D+00 0.000000D+00 0.000000D+00 0.122461D-02 0.122462D-02 2.379873D-03
0.499338D-04 0.751159D-04 0.106910D-03 0.122764D-02 0.122801D-02 0.237831D-02
0.996762D-04 0.148457D-03 0.212526D-03 0.123686D-02 0.123769D-02 0.237359D-02
0.149133D-03 0.218288D-03 0.315631D-03 0.125227D-02 0.125257D-02 0.236556D-02
0.198325D-03 0.283038D-03 0.415135D-03 0.126999D-02 0.127492D-02 0.235407D-02
0.247335D-03 0.341512D-03 0.510068D-03 0.128913D-02 0.130346D-02 0.233902D-02
0.296235D-03 0.393110D-03 0.599544D-03 0.130851D-02 0.133691D-02 0.232040D-02

and so on...

And I would need

1 0.000000D+00 |
2 0.000000D+00 |
3 0.000000D+00 | -> 1st row
4 0.122461D-02 |
5 0.122461D-02 |
6 2.379873D-03 |
1 0.499338D-04
2 0.751159D-04
. .
. .
. .

and so on.

Igor

ilukacevic · 07-19-2010, 04:15 AM

Quote:

Originally Posted by colucix

Awk works in double precision but it's not aware of the fortran D notation. You should first convert D to E. Following my previous suggestion (but it can be applied for all the others) you can try something like:

Code:

eval cat $(seq -f "<(awk '{sub(/D/,\"E\"); print $%.0f''*''219474.6306726}' file)" 1 6) | nl

Edit: a little more explicit:

Code:

for i in $(seq 1 6)
do
  awk '{sub(/D/,"E"); print $'$i'*219474.6306726}' file
done | nl

Hi!

Unfortunately, these do not give the thing I need. Instead I get

522322
52197.9
52094.3
51918
51665.9
51335.6
50926.9
50441.9
49883.9
49257.6
48565.6
47809.7
46990.2
46106.4
45156.5
44137.9
43044.2

and so on...

These are not multiplied correctly and are too few of them - should be 6*141 of them.
Thnx for the tip anyway.

Igor

grail · 07-19-2010, 04:16 AM

hmmmm .... your current output based on the new input is not the same as what you requested previously??

Surely the first 3 items of row one should be spaced out at intervals of 6?

My current output for your input is:

Code:

ilukacevic · 07-19-2010, 04:20 AM

Quote:

Originally Posted by grail

Well I was hoping you would work the line number issue out, but it you were on the right track as to use NR:

Code:

awk '{for(i=1;i<=NF;i++)if(arr[i] ~ /./)arr[i]=arr[i]"\n"NR" "$i;else arr[i]=NR" "$i}END{for(x=1;x<=length(arr);x++)printf("%s\n",arr[x])}' in_file

I am not sure how this affect your double precision.

It doesn't. But this is it - with this line I got exactly what I needed.

thnx

All I need to do now is to convert data format.

Igor

grail · 07-19-2010, 04:26 AM

If you try my next one from post #22 it includes the factor parameter

ilukacevic · 07-19-2010, 05:50 AM

Quote:

Originally Posted by grail

Yeah but a little ++counter fixed that (mainly cause I couldn't workout how to use the sequence numbers again as the counter

)

Code:

eval cat $(seq -f "<(awk '{sub(/D/,\"E\"); print ++n\" \"$%.0f''*''219474.6306726}' file)" 1 6)

thnx again! This one set a single mulitplied column with line numbers, but the multiplication is wrong again. And only the first column is printed.
Now I have the feeling that I'm bugging you all too much.

Igor

ilukacevic · 07-19-2010, 05:54 AM

Quote:

Originally Posted by grail

If you try my next one from post #22 it includes the factor parameter

Hi!
I'm sorry, but I don't understand two things.
1] Is this a shell script or a perl script?
2] Where is the line in which the names of the input and output files should be stated?

thnx

Igor

grail · 07-19-2010, 06:46 AM

No it is an awk script and the files are used as follows:

Code:

./script input_file > output_file

Or you can have the script print to a file instead of back to standard out.
Change the last line to read:

Code:

print arr[x] > "output_file"

MTK358 · 07-19-2010, 07:33 AM

Quote:

Originally Posted by ilukacevic

Sure. Here it is (a part of it at least - but it's all the same to the end)

0.000000D+00 0.000000D+00 0.000000D+00 0.122461D-02 0.122462D-02 2.379873D-03

And I would need

1 0.000000D+00 |
2 0.000000D+00 |
3 0.000000D+00 | -> 1st row
4 0.122461D-02 |
5 0.122461D-02 |
6 2.379873D-03

Shouldn't my first Perl script (http://www.linuxquestions.org/questi...1/#post4035240) work then?

colucix · 07-19-2010, 10:14 AM

The history of the requirements of this thread:

post #1 - input table must be printed out in column order (row number added)
post #10 - fields in the input table are separated by two spaces
post #13 - data are numbers in scientific notation and need to be multiplied by a constant factor
post #32 - input table must be printed out in row order (column number added)
post #35 - input table must be printed out in column order
post #37 - the result of the multiplication by the given factor 219474.6306726 is wrong.
post #37 - maybe input changed again (since the assertion "only the first column is printed" is false).

Quote:

Now I have the feeling that I'm bugging you all too much.

Just a little!

Please Igor, we only need to know two simple things at this point:
1) Which is the actual input (two or three rows of data are enough)
2) What should be the exact and final output (data already multiplied and written in the desired format)

And eventually what is the scripting language of your choice (one for which you're already skilled or one you feel it's worth to learn). I'm trying only to bring this challenging but confused thread to a more straight direction.

ilukacevic · 07-19-2010, 01:08 PM

Quote:

Originally Posted by MTK358

Shouldn't my first Perl script (http://www.linuxquestions.org/questi...1/#post4035240) work then?

Hi!
thnx for the suggestion, but where do I input the incoming and output file?
Using your perl script unmodified, I get

Could not open file at mtk.pl line 3.

Igor

MTK358 · 07-19-2010, 01:33 PM

It expects an input filename as an argument. Otherwise, if you want it to read from STDIN, replace these two lines:

Code:

open my $file, $ARGV[1] or die "Could not open file";
while (<$file>) # iterate over the lines

with this one line:

Code:

while (<STDIN>) # iterate over the lines

ilukacevic · 07-19-2010, 01:51 PM

Quote:

Originally Posted by colucix

The history of the requirements of this thread:

post #1 - input table must be printed out in column order (row number added)
post #10 - fields in the input table are separated by two spaces
post #13 - data are numbers in scientific notation and need to be multiplied by a constant factor
post #32 - input table must be printed out in row order (column number added)
post #35 - input table must be printed out in column order
post #37 - the result of the multiplication by the given factor 219474.6306726 is wrong.
post #37 - maybe input changed again (since the assertion "only the first column is printed" is false).

Just a little!

Please Igor, we only need to know two simple things at this point:
1) Which is the actual input (two or three rows of data are enough)
2) What should be the exact and final output (data already multiplied and written in the desired format)

And eventually what is the scripting language of your choice (one for which you're already skilled or one you feel it's worth to learn). I'm trying only to bring this challenging but confused thread to a more straight direction.

Hi!
I appreciate your effort to clarify the "problem". At one moment today, I was also confused by numerous suggestions, I kind of got lost in answering them all. I think that I even hadn't answered them all, and I should have

The scripting language is not important...I'll accept everything

I'm not skilled at any one. I just found awk command very useful to perform operations on column data, so I started with it.

post#32 I see that my scribling on the side of the 2nd column messed things up. Post #35 is correct.

1)
These are the first three input lines (6 columns & 141 rows)

0.000000D+00 0.000000D+00 0.000000D+00 0.122461D-02 0.122462D-02 2.379873D-03
0.499338D-04 0.751159D-04 0.106910D-03 0.122764D-02 0.122801D-02 0.237831D-02
0.996762D-04 0.148457D-03 0.212526D-03 0.123686D-02 0.123769D-02 0.237359D-02

2)
These should be the first several lines of the output (2 columns & 846 rows)

1 0
2 10,959202
3 21,876397
. .
. .
. .
141 .
1 0
2 16,486034
3 32,582545
. .
. .
. .
141 .
1 0
2 23,464033
3 46,644065
. .
. .
. .
141 .
1 268,77083
2 269,43584
3 271,45939
. .
. .
. .
141 .
1 268,77302
2 269,51704
3 271,64156
. .
. .
. .
141 .
1 522,32175
2 521,97871
3 520,94279
. .
. .
. .
141 .

I remembered that someone already posted that one can change D into E with sed command. Then, would awk recognize numbers with E and multiply correctly? Grail got the multiplication ok in his post from 11:35 today, but the line numbers were not.

Thank you all again for the trouble!

Yours faithfully,

Igor

MTK358 · 07-19-2010, 02:06 PM

I don't understand the output format. What exactly is happening and what are those dots doing?