LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-30-2015, 07:50 PM   #1
Drigo
Member
 
Registered: Jun 2009
Posts: 111

Rep: Reputation: 17
Talking Modify rows to columns in bash script??


I have this file.txt:
0 0 0 1 2 3
0 0 0 4 5 6
0 0 0 7 8 9

I want it to look like file2.txt:
0 0 0
0 0 0
0 0 0
1 4 7
2 5 8
3 6 9

What is the easiest way to do it? Prob. a single command line?
 
Old 07-30-2015, 08:24 PM   #2
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_10{.0|.1|.2}
Posts: 3,887
Blog Entries: 1

Rep: Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007
I don't have a simple command, but awk would be the tool. Shouldn't be too difficult.
 
Old 07-30-2015, 09:15 PM   #3
mralk3
Member
 
Registered: May 2015
Posts: 760

Rep: Reputation: 232Reputation: 232Reputation: 232
Modify rows to columns in bash script??

You could read each line into a loop and make each line an array. Then format the second file accordingly.
 
Old 07-31-2015, 02:52 AM   #4
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,256

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
It would be nice to see what you have tried first, otherwise it is all academic and you can use any number of tools for the job.
 
Old 07-31-2015, 09:26 AM   #5
ondoho
Senior Member
 
Registered: Dec 2013
Posts: 4,176

Rep: Reputation: 837Reputation: 837Reputation: 837Reputation: 837Reputation: 837Reputation: 837Reputation: 837
i recently tried something similar with text.
it requires 2 nested for-loops, if you get them right it's a really short script.
 
Old 07-31-2015, 11:05 AM   #6
allend
Senior Member
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware-current
Posts: 4,437

Rep: Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354Reputation: 1354
Using bash looks like homework. In real life I would read into a spreadsheet, then copy and paste with the transpose option.
 
Old 07-31-2015, 12:09 PM   #7
Drigo
Member
 
Registered: Jun 2009
Posts: 111

Original Poster
Rep: Reputation: 17
Not homework, but I'll give it a try. I wasn't looking for a for loop but seems that might do the trick...
 
Old 07-31-2015, 03:22 PM   #8
Drigo
Member
 
Registered: Jun 2009
Posts: 111

Original Poster
Rep: Reputation: 17
Not a homework, but here's the solution:
#!/bin/bash
#Read in the bvecs file
BVECFILE=$1
BVECNEWFILE="bvecs_new"
#Store it as an array
readarray BVEC < $BVECFILE
BVECNUM=${#BVEC[@]}
#Check the number of directions
GRADNUM=$(echo $BVEC | wc -w )
#Indexing at i=1 so we get rid of the first vol0 indexed as 0
if [ -f $BVECNEWFILE ] ; then
echo "$BVECNEWFILE already exists. Removing and replacing..."
rm $BVECNEWFILE
fi
echo "New bvecs are generated in "$BVECNEWFILE" "
#echo "GRADX is $GRADX"
for ((i=2; i<=${GRADNUM}; i++ ));
do
gx=$(echo ${BVEC[0]} | awk -v x=$i '{print $x}')
gy=$(echo ${BVEC[1]} | awk -v x=$i '{print $x}')
gz=$(echo ${BVEC[2]} | awk -v x=$i '{print $x}')
echo "$gx $gy $gz" >> $BVECNEWFILE
done
 
1 members found this post helpful.
Old 07-31-2015, 04:22 PM   #9
HMW
Member
 
Registered: Aug 2013
Location: Sweden
Distribution: Debian, Arch, Xubuntu, Manjaro
Posts: 745
Blog Entries: 3

Rep: Reputation: 347Reputation: 347Reputation: 347Reputation: 347
^ Nice! I never seem to have the time to learn awk properly. I found this to be an entertaining problem, so I did my own variant in Python. Looks like this:

Code:
#!/usr/bin/env python3

# Declare our matrix 3 chars wide, 6 lines
matrix = [[ 0 for i in range(3)] for j in range(6) ]

x = 0
y = 0
count = 0

# Open the original file
with open("f1.txt") as f1:
    txt = f1.read()

# Read the file, char by char.
# Insert chars at correct place in matrix.
for char in txt:
    if count > 5:
        count = 0
        x = 0
        y += 1
    if char != "\n":
        matrix[x][y] = char
        x += 1
        count+=1


# Now let's open a file to write the result to, and create it if not 
# already existing
outfile = open("f2.txt", "w")

x = 0
y = 0

while True:
    outfile.write(matrix[x][y])
    y += 1
    if y > 2:
        outfile.write("\n")
        y = 0
        x += 1
    if x > 5:
        break

The original file:
Code:
$ cat f1.txt
000123
000456
000789
The output (outfile) from my program:
Code:
$ cat f2.txt 
000
000
000
147
258
369
Thanks for a good exercise!

HMW
 
1 members found this post helpful.
Old 07-31-2015, 06:02 PM   #10
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_10{.0|.1|.2}
Posts: 3,887
Blog Entries: 1

Rep: Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007
OK, I couldn't NOT come up with a minimalist awk example since no one else did (actually I had my son do it).

Because we are turning the corner, let's call it corner.awk, with the following lines:

Code:
{
        for(f=1;f<=NF;++f) arr[NR ":" f]=$f;
}
END{
        for(f=1;f<=NF;++f){
                for(n=1;n<=NR;++n)
                        printf("%s ",arr[n ":" f]);
                printf("\n");
        }
}
Assuming your data file is file.txt, you would invoke it like this...

Code:
# awk -f corner.awk file.txt
0 0 0
0 0 0
0 0 0
1 4 7
2 5 8
3 6 9

OR save to file...

# awk -f corner.awk file.txt >file2.txt
Also works for any width of field, alpha or numeric, and any number of columns and lines so long as all rows have same number of space separated columns!

Gotta love awk!

And thanks for the exercise as well!

Last edited by astrogeek; 07-31-2015 at 08:48 PM. Reason: Fixed percent signs, additions, and more...
 
1 members found this post helpful.
Old 07-31-2015, 10:51 PM   #11
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 14,848

Rep: Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823
Not minimalist, but mine was similarly done (in awk) but using an array of arrays - I find them easy to "walk" as you can use "for i in array" followed by "for j in array[i]" ...
No need to know the bounds in advance.

Edit: - not relevant here as the prints needed to be ordered - I did similar to @astrogeek for the printing.

Just a nit - hanging whitespace on lines makes regex using the "$" anchor more complex than necessary.

Last edited by syg00; 07-31-2015 at 10:56 PM.
 
1 members found this post helpful.
Old 07-31-2015, 11:26 PM   #12
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_10{.0|.1|.2}
Posts: 3,887
Blog Entries: 1

Rep: Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007
Quote:
Originally Posted by syg00 View Post
Not minimalist, but mine was similarly done (in awk) but using an array of arrays - I find them easy to "walk" as you can use "for i in array" followed by "for j in array[i]" ...
No need to know the bounds in advance.

Edit: - not relevant here as the prints needed to be ordered - I did similar to @astrogeek for the printing.

Just a nit - hanging whitespace on lines makes regex using the "$" anchor more complex than necessary.
HAHA! Yes I was aware of the trailing space but was out of time - hoped no one would notice!

Here is one without trailing whitespace and two lines shorter!

Code:
{
        for(f=1;f<=NF;++f) arr[NR ":" f]=$f;
}
END{
        for(f=1;f<=NF;++f)
                for(n=1;n<=NR;++n)
                        printf("%s%s",arr[n ":" f],(n<NR ? " " : "\n"));
}
Thanks! These are fun when you have the time and the right mindset, fortunate superposition today!

EDIT ****
Although I use a linear array the associative keys are effectively compound to produce the same logic as would an array of arrays, allowing same use of (i,j), or (n,f) in my own case.

Last edited by astrogeek; 08-01-2015 at 12:12 AM. Reason: typos, addition, pesky percent signs!
 
1 members found this post helpful.
Old 08-01-2015, 12:17 AM   #13
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 14,848

Rep: Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823Reputation: 1823
Cute - I did it as follows, but needs an extra print statement for the newline. I might steal that idea ...
Code:
printf(n==NR?"%d" : "%d ",_[n][m])

Last edited by syg00; 08-01-2015 at 12:25 AM. Reason: dodgy � rendering
 
1 members found this post helpful.
Old 08-01-2015, 12:43 AM   #14
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_10{.0|.1|.2}
Posts: 3,887
Blog Entries: 1

Rep: Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007Reputation: 2007
Quote:
Originally Posted by syg00 View Post
Cute - I did it as follows, but needs an extra print statement for the newline. I might steal that idea ...
Code:
printf(n==NR?"d" : "%d ",_[n][m])
Thanks for the warm fuzzy feeling of satisfaction!

When my son produced his first cut, he had a proper BEGIN block and used print(...) statements.

He also gave us the linear array idea which is good! (My own first attempt used multiple arrays as well.)

In order to get the newlines right with print he had to define ORS="" to override the default (\n), among other things.

I wanted to rely on defaults without a BEGIN block to keep it minimal so I changed to use of printf(...), and hastily introduced the trailing space because I was somewhat rushed, but knew a ternary would fix it - just didn't do it.

No need to "steal" the idea, you would have thought of it yourself in another minute or two!

Anyway, sharing and immitation are both good things - and I have FREELY shared the thought! I like to never miss an opportunity to make that point, so let's make it "official" with a universal statement of copy rights...

Code:
# Acknowledgment of right to use, copy, modify and distribute:
#
# You already have the right to use, modify and distribute this
# or any other thought or idea, and need no license or other
# permission from anyone to do so!
#
# Exercise it freely and never concede it to anyone!

Last edited by astrogeek; 08-01-2015 at 12:56 AM.
 
Old 08-01-2015, 02:36 AM   #15
HMW
Member
 
Registered: Aug 2013
Location: Sweden
Distribution: Debian, Arch, Xubuntu, Manjaro
Posts: 745
Blog Entries: 3

Rep: Reputation: 347Reputation: 347Reputation: 347Reputation: 347
^ Awesome, all of the above!

I toyed with only using loops for a while, but found it easier to, eh, "visualize" using a matrix. As I keep saying, one of these days, I might find the time and energy to learn awk properly.

Last edited by HMW; 08-01-2015 at 02:41 AM. Reason: Clarification
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Converting a file with Rows and Columns to just Columns mphillips67 Linux - Newbie 14 03-05-2014 11:31 AM
[SOLVED] Perl script for manipulating rows and columns captainentropy Programming 10 08-26-2013 03:45 AM
Script to convert words in a text file [columns to rows] naveenchandar Linux - Newbie 2 11-08-2012 02:00 AM
[SOLVED] transform rows into columns in bash xeon123 Linux - Newbie 4 07-30-2011 08:19 AM
Script to convert logs columns to rows fono Linux - Software 10 05-19-2009 09:29 PM


All times are GMT -5. The time now is 04:51 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration