LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 06-19-2012, 03:27 AM   #1
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Rep: Reputation: Disabled
sum up values from each columns (awk)


I have different files with variable numbers of columns.

What I try to do is to output the values of each columns:
- with the single value if it is the same in the entire column,
- or the values separated by "," if it exist different ones in the same column

example 1:
input:
1|2|3|4
1|1|3|4
1|2|3|3

output:
1|1,2|3|3,4

example 2:
input:
1|2
1|3
7|2

output:
1,7|2,3

(the order of numbers separated by "," doesn't matter)

Thanks for your help !
 
Old 06-19-2012, 04:26 AM   #2
pan64
Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 5,142

Rep: Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364
you can define associative arrays for every column: col1, col2, col3, col4. line is automatically split into $1, $2, $3 ... (-F\| is used to define separator). col1[$1] = 1; col2[$2] = 2 ... will define the element of arrays. finally (using END) you need to print the defined elements of those arrays







_____________________________________
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.
Happy with solution ... mark as SOLVED
(located in the "thread tools")
 
Old 06-19-2012, 05:10 AM   #3
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Original Poster
Rep: Reputation: Disabled
Hi Pan 64, thanks for your help !

I am not sure to understand the term "associative array".
Is it the same as an array that you define when using the split function?

If we look at the first input I wrote:
Code:
1|2|3
1|1|3
1|2|3
Are these values equivalent to these elements below?
Code:
col1[$1]|col2[$1]|col3[$1]
col1[$2]|col2[$2]|col3[$2]
col1[$3]|col2[$3]|col3[$3]
 
Old 06-19-2012, 05:14 AM   #4
pan64
Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 5,142

Rep: Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364
see here, you will find some useful tips





_____________________________________
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.
Happy with solution ... mark as SOLVED
(located in the "thread tools")
 
Old 06-19-2012, 09:09 AM   #5
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Original Poster
Rep: Reputation: Disabled
ok, associative arrays are still not very clear for me, but maybe something that would look like that:
Code:
BEGIN{FS=OFS="|"}

{
    for (i=1; i<=NF; i++){
        coli[$i]++
            if(coli[$i] <is unique????>){
                print $i}

            else
                print coli[$i] <with a "," somewhere ?????>
}
}
 
Old 06-19-2012, 11:38 AM   #6
pan64
Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 5,142

Rep: Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364
that's fine, you are almost ready.
Code:
function printarray (a)
{
    for (i=1; i<=length(a); i++)
    {
        print a[i];
        if ( i<length(a) ) printf ",";
    }  
}
this will print an array. Hope this helps


_____________________________________
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.
Happy with solution ... mark as SOLVED
(located in the "thread tools")
 
Old 06-19-2012, 07:59 PM   #7
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.6, Centos 5.10
Posts: 16,324

Rep: Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041
An associative array is often referred to as a hash table, although strictly speaking one is an implementation of the other

https://en.wikipedia.org/wiki/Associative_array
Quote:
In computer science, an associative array, map, or dictionary is an abstract data type composed of a collection of (key,value) pairs, such that each possible key appears at most once in the collection.
.
.
.
The dictionary problem is the task of designing a data structure that implements an associative array. A standard solution to the dictionary problem is a hash table;
https://en.wikipedia.org/wiki/Hash_table
Quote:
Thus, a hash table implements an associative array.
 
Old 06-20-2012, 02:18 AM   #8
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Original Poster
Rep: Reputation: Disabled
Quote:
Code:
function printarray (a)
{
    for (i=1; i<=length(a); i++)
    {
        print a[i];
        if ( i<length(a) ) printf ",";
    }
}

- What is the point of "function printarray (a)" ?
- Why we don't mention fields?
- Does "length" refers to the length of array a, in other words the number of lines in the column?
 
Old 06-20-2012, 03:19 AM   #9
pan64
Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 5,142

Rep: Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364
this function will print the list of the values of the array passed to the func (separated by a comma). We do not need fields, records and other funny things, just a for loop. length is a built-in function (of awk), it will return the length of a string or the length of an array (see man page).





_____________________________________
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.
Happy with solution ... mark as SOLVED
 
Old 06-20-2012, 05:03 AM   #10
lcvs
LQ Newbie
 
Registered: Jun 2012
Posts: 16

Original Poster
Rep: Reputation: Disabled
I don't really see where I am going with that.
I need to learn a bit more I guess.

Thanks anyway !
 
Old 06-20-2012, 05:16 AM   #11
pan64
Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 5,142

Rep: Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364Reputation: 1364
the function is used to print col1, col2, col3 and col4, the four arrays where you collected your data







_____________________________________
If someone helps you, or you approve of what's posted, click the "Add to Reputation" button, on the left of the post.
Happy with solution ... mark as SOLVED
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Sum up value of columns in 4 files using shell script reach.sree@gmail.com Programming 6 06-07-2012 08:29 PM
[SOLVED] AWK: add columns while keep format for other columns cristalp Programming 3 10-13-2011 07:14 AM
awk's sum not working? grob115 Linux - Newbie 4 04-29-2010 01:44 PM
mysql sum from one of two columns secretlydead Programming 1 12-16-2009 02:11 AM
awk merge and sum lines problem lalo4080 Programming 4 08-12-2008 11:21 AM


All times are GMT -5. The time now is 09:22 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration