LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 06-19-2012, 02:27 AM   #1
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Rep: Reputation: Disabled
sum up values from each columns (awk)


I have different files with variable numbers of columns.

What I try to do is to output the values of each columns:
- with the single value if it is the same in the entire column,
- or the values separated by "," if it exist different ones in the same column

example 1:
input:
1|2|3|4
1|1|3|4
1|2|3|3

output:
1|1,2|3|3,4

example 2:
input:
1|2
1|3
7|2

output:
1,7|2,3

(the order of numbers separated by "," doesn't matter)

Thanks for your help !
 
Old 06-19-2012, 03:26 AM   #2
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 4,658

Rep: Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252
you can define associative arrays for every column: col1, col2, col3, col4. line is automatically split into $1, $2, $3 ... (-F\| is used to define separator). col1[$1] = 1; col2[$2] = 2 ... will define the element of arrays. finally (using END) you need to print the defined elements of those arrays







_____________________________________
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.
Happy with solution ... mark as SOLVED
(located in the "thread tools")
 
Old 06-19-2012, 04:10 AM   #3
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Original Poster
Rep: Reputation: Disabled
Hi Pan 64, thanks for your help !

I am not sure to understand the term "associative array".
Is it the same as an array that you define when using the split function?

If we look at the first input I wrote:
Code:
1|2|3
1|1|3
1|2|3
Are these values equivalent to these elements below?
Code:
col1[$1]|col2[$1]|col3[$1]
col1[$2]|col2[$2]|col3[$2]
col1[$3]|col2[$3]|col3[$3]
 
Old 06-19-2012, 04:14 AM   #4
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 4,658

Rep: Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252
see here, you will find some useful tips





_____________________________________
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.
Happy with solution ... mark as SOLVED
(located in the "thread tools")
 
Old 06-19-2012, 08:09 AM   #5
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Original Poster
Rep: Reputation: Disabled
ok, associative arrays are still not very clear for me, but maybe something that would look like that:
Code:
BEGIN{FS=OFS="|"}

{
    for (i=1; i<=NF; i++){
        coli[$i]++
            if(coli[$i] <is unique????>){
                print $i}

            else
                print coli[$i] <with a "," somewhere ?????>
}
}
 
Old 06-19-2012, 10:38 AM   #6
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 4,658

Rep: Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252
that's fine, you are almost ready.
Code:
function printarray (a)
{
    for (i=1; i<=length(a); i++)
    {
        print a[i];
        if ( i<length(a) ) printf ",";
    }  
}
this will print an array. Hope this helps


_____________________________________
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.
Happy with solution ... mark as SOLVED
(located in the "thread tools")
 
Old 06-19-2012, 06:59 PM   #7
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,261

Rep: Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028
An associative array is often referred to as a hash table, although strictly speaking one is an implementation of the other

https://en.wikipedia.org/wiki/Associative_array
Quote:
In computer science, an associative array, map, or dictionary is an abstract data type composed of a collection of (key,value) pairs, such that each possible key appears at most once in the collection.
.
.
.
The dictionary problem is the task of designing a data structure that implements an associative array. A standard solution to the dictionary problem is a hash table;
https://en.wikipedia.org/wiki/Hash_table
Quote:
Thus, a hash table implements an associative array.
 
Old 06-20-2012, 01:18 AM   #8
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Original Poster
Rep: Reputation: Disabled
Quote:
Code:
function printarray (a)
{
    for (i=1; i<=length(a); i++)
    {
        print a[i];
        if ( i<length(a) ) printf ",";
    }
}

- What is the point of "function printarray (a)" ?
- Why we don't mention fields?
- Does "length" refers to the length of array a, in other words the number of lines in the column?
 
Old 06-20-2012, 02:19 AM   #9
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 4,658

Rep: Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252
this function will print the list of the values of the array passed to the func (separated by a comma). We do not need fields, records and other funny things, just a for loop. length is a built-in function (of awk), it will return the length of a string or the length of an array (see man page).





_____________________________________
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.
Happy with solution ... mark as SOLVED
 
Old 06-20-2012, 04:03 AM   #10
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Original Poster
Rep: Reputation: Disabled
I don't really see where I am going with that.
I need to learn a bit more I guess.

Thanks anyway !
 
Old 06-20-2012, 04:16 AM   #11
pan64
Senior Member
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 4,658

Rep: Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252Reputation: 1252
the function is used to print col1, col2, col3 and col4, the four arrays where you collected your data







_____________________________________
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.
Happy with solution ... mark as SOLVED
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Sum up value of columns in 4 files using shell script reach.sree@gmail.com Programming 6 06-07-2012 07:29 PM
[SOLVED] AWK: add columns while keep format for other columns cristalp Programming 3 10-13-2011 06:14 AM
awk's sum not working? grob115 Linux - Newbie 4 04-29-2010 12:44 PM
mysql sum from one of two columns secretlydead Programming 1 12-16-2009 01:11 AM
awk merge and sum lines problem lalo4080 Programming 4 08-12-2008 10:21 AM


All times are GMT -5. The time now is 12:29 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration