help with comma separated values and what should be a simple script.

zaber · 03-05-2008, 02:19 PM

I need help on a very simple script, or what should be. I have two sets of values, with each value separated by a comma. I need a way to make a third set of values that consists of values that are the same from each list.

In other words I am running getent group groupa, and getent group groupb. I need to have a list (same format) of the members that are in both groups.

Thanks for any help. I have not been able to figure this one out.

Tinkster · 03-05-2008, 02:34 PM

Hi,

And welcome to LQ!

Would this suffice?

Code:

echo $(getent group groupa) $(getent group groupb)|tr ' ' '\n'|tee newfile

Cheers,
Tink

zaber · 03-05-2008, 02:51 PM

No I am afraid that did not help, but thank you for giving me some new commands to look at. What I am trying to do is to make a CSV of the that are the same in two groups. right now I have:

Code:

#!/bin/bash
g1=`getent group g1 | cut -f 4 -d :`
echo "g1 Group is: "$g1
echo ""
g2=`getent group g2 | cut -f 4 -d :`
echo "g2 Group is: "$g2
exit

I am able to display groups 1 and 2 but just not the members common to both.

Thank you again.

Tinkster · 03-05-2008, 03:03 PM

Ooops ... I missed the part about the identical.

Can you give me a sample of the expected output?
I don't quite see how cutting the last field of the output
will result in a CSV file ...

Cheers,
Tink

ghostdog74 · 03-05-2008, 10:08 PM

Code:

getent group groupa groupb | awk 'BEGIN{FS=":";OFS=","}
{  s=$4 OFS s
}END {
  n=split(s,a,",")
 for (i=1;i<=n;i++) b[a[i]]++
 for (usr in b ) if ( b[usr] > 1 ) print usr " is duplicate"
}'

jschiwal · 03-05-2008, 10:31 PM

If the output is useful when sorted, you could use the "comm" command.
comm -12 <(sort file1.csv) <(sort file2.csv)

Code:

cat file1.csv 
abc, def, geh
def, abc, xxx
swe, 123, 456
$ cat file2.csv 
abc, def, geh
def, abc, xxx
swe, 123, 456
qwe, qwe, rty
uio, asd, vbn
poi, fhg, vnb
$ comm -12 <(sort file1.csv) <(sort file2.csv)
abc, def, geh
def, abc, xxx
swe, 123, 456

The <(command) form allows you to use the output of a command where a filename is expected.

If you have a list of users in a group, you could use "tr" to convert spaces to newlines and sort the result:
comm -12 <(getent group groupa | tr ' ' '\n' | sort) <(getent group groupb | tr ' ' '\n')
This will give you a list of users common to both groups. I don't know what you want to do with the list. Usually csv files represent different types of values in respective fields.

angrybanana · 03-06-2008, 02:35 AM

Code:

getent group groupa groupb|perl -F':' -lane 'for(split ",", $F[3]) {print if ++$seen{$_}>1}'

Quote:

Originally Posted by zaber

I need to have a list (same format) of the members that are in both groups.

what do you mean by that? can you give an example of the output format?

zaber · 03-06-2008, 08:35 AM

I am sorry I am not being clear. The output I am looking for is user1,user2,user3. when all belong to both groups being queried.

angrybanana · 03-06-2008, 10:01 AM

Code:

getent group groupa groupb|perl -F':' -lane '
for(split ",", $F[3]) {push @a, $_ if $seen{$_}++}
END{print join ",", @a}'

That should work, hope the code is clear.

zaber · 03-06-2008, 10:46 AM

Quote:

Originally Posted by angrybanana

That should work, hope the code is clear.

Works perfectly. Thank you.
Now I just need to figure out how it works, i still have a lot to learn. lol

thank you again and thank you to everyone who answered.

angrybanana · 03-06-2008, 01:58 PM

Found out that perl has its own function for getent.
I fiddled with it a bit, this is what I have currently.

Code:

use strict;
use warnings;

my @grps = qw/optical storage/;
my (@users, %seen, @dup);

for my $grp (@grps){
  @dup = (@dup, grep {$seen{$_}++ } (split / /, +(getgrnam($grp))[3]));
}
print join (",", @dup), "\n";

This might have sloppy code in it, but it should be slightly easier to understand then the other one. I'll write a detailed explanation of the code later today when I have more time.

EDIT:
code explanation

Code:

getent group groupa groupb|perl -F':' -lane '
for(split ",", $F[3]) {push @a, $_ if $seen{$_}++}
END{print join ",", @a}'

Code:

-F':' -lane

basically this sets the field separator to ':' and puts the split values in an array @F

Code:

for(split ",", $F[3]) {push @a, $_ if $seen{$_}++}

for(split ",", $F[3]) {..}: We take the forth field (which is the user list) and we split it on ",". For each user run {} with the username stored in $_ (default)
code if condition : Run code, if the condition is true.
condition:
$seen{$_}++: for each username we add the variable as a key to a hash (%seen), and increment the value (after the first time it'll be 1, then 2 and so on). we end up with a hash of ($seen{username} = count).
The if statement is kind of tricky, because the ++ comes after the variable, it is incremented after the if statement is processed. Basically the first time we see "bob" ($seen{"bob"} = undef) the if condition is false, and now "bob" value is set 1 ($seen{"bob"}=1). So the statement will only be true if the key has already been added to the hash. (every time after the first time)
code:
push(@a, $_): this just adds the username ($_, which is a duplicate) to the array (@a).
The last line is just joining the array with "," and printing it.

This is only good for comparing 2 lists of users (2 groups). Of course this could be fixed with (if $seen{$_}++ = number of groups).

PS: Hope my explanation makes sense (and is correct), I'm pretty bad with perl, so this might not be the best way of doing things..but it works. (took me less then 1/2 the time to code this in python... actually, a lot less then 1/2)