LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   use awk to read and update FileA and FileB at the same time (https://www.linuxquestions.org/questions/linux-newbie-8/use-awk-to-read-and-update-filea-and-fileb-at-the-same-time-736801/)

magische_vogel 06-30-2009 06:20 PM

use awk to read and update FileA and FileB at the same time
 
Hi everybody,

Here is my awk script :
controle_accounts.awk contents:
#! /usr/bin/awk -f
BEGIN {
FS=";"; # Changing the field separator
"date +%Y/%m/%d" | getline now;
}{
#if account is expired and account not disabled then disable account
if ($6<now && $4==1) {$4=0; //disable the account in the usersDB.conf only.
How to disable the corresponding account in the accesslist.conf file should go here also.}
print ($1 ";" $2 ";" $3 ";" $4 ";" $5 ";" $6 ";" $7) > "/var/etc/usersDB.conf"; //write changes to input file.
}

usersDB.conf description :
username;name;role;account_status;registration_date;expiration_date;note
usersDB.conf contents:
user1;geroge;U;1;2009/03/01;2009/06/01;none
user2;piere;U;1;2009/04/15;2009/07/15;none
user3;albert;U;1;2009/06/01;2009/07/30;none
user4;philipe;U;1;2009/06/01;2009/06/25;none
user5;michael;U;1;2009/06/01;2009/09/30;none
user6;jack;U;1;2009/06/01;2009/08/30;none
user7;ronald;U;1;2009/06/01;2009/05/30;none
user8;patrick;U;1;2009/06/01;2009/04/19;none

accesslist.conf description :
username;password
accesslist.conf contents:
U: user1 kenav7wr
U: user2 cxq978fk
U: user3 rkl749w2
U: user4 krvv56of
U: user5 879yrw55
U: user6 49452ayd
U: user7 37adck6r
U: user8 cxyo1gah

note : the primary key between the usersDB.conf and the accesslist.conf is the username

My question is how to disable the expired account in the second file named accesslist.conf by adding # in front of U: (same

question for enabling the account) using the above awk programme.

expected results :
usersDB.conf contents:
user1;geroge;U;0;2009/03/01;2009/06/01;none
user2;piere;U;1;2009/04/15;2009/07/15;none
user3;albert;U;1;2009/06/01;2009/07/30;none
user4;philipe;U;0;2009/06/01;2009/06/25;none
user5;michael;U;1;2009/06/01;2009/09/30;none
user6;jack;U;1;2009/06/01;2009/08/30;none
user7;ronald;U;0;2009/06/01;2009/05/30;none
user8;patrick;U;0;2009/06/01;2009/04/19;none

accesslist.conf contents:
#U: user1 kenav7wr
U: user2 cxq978fk
U: user3 rkl749w2
#U: user4 krvv56of
U: user5 879yrw55
U: user6 49452ayd
#U: user7 37adck6r
#U: user8 cxyo1gah

/var/etc/controle_accounts.awk usersDB.conf

Any help will be appreciated.

osor 06-30-2009 08:51 PM

How about something like this:
Code:

#!/usr/bin/gawk -f
BEGIN {
        FS=OFS=";"
        "date +%Y/%m/%d" | getline now
}
ARGIND==1 {
        if ($6<now && $4==1)
                $4=0
        enabled[$1]=$4
        print $0
}
ARGIND==2 && FNR==1 {FS=OFS=" "}
ARGIND==2 { print (enabled[$2]?"":"#")$0 }


magische_vogel 06-30-2009 09:32 PM

Quote:

Originally Posted by osor (Post 3592305)
How about something like this:
Code:

#!/usr/bin/awk -f
BEGIN {
        FS=OFS=";"
        "date +%Y/%m/%d" | getline now
}
ARGIND==1 {
        if ($6<now && $4==1)
                $4=0
        enabled[$1]=$4
        print $0
}
ARGIND==2 && FNR=1 {FS=OFS=" "}
ARGIND==2 { print (enabled[$2]?"":"#")$0 }


hi osor,

i make copy paste your script into then i excute it that way :

./controle_accounts.awk usersDB.conf accesslist.conf
i don't get any results even the script was executed without errors.
can you give more details for each statement in your script. where the changes are saved ? in each input file or to the standard output ?

thanx.

ghostdog74 06-30-2009 09:37 PM

Code:

#!/bin/bash
awk 'BEGIN{
    OFS=FS=";"
    now=systime()       
    tempfile = "/var/etc/temp"
    tempaccessfile = "/var/etc/tempaccess"
}
FNR==NR{
    access[++e]=$0
    afile=FILENAME
    next   
}
$4==1{
    m=split($6,d,"/") #split date on /
    strd = d[1]" "d[2]" "d[3]" 0 0 0"
    t=mktime(strd) #get seconds
    if ( t < now ){
        $4=0
        print $0 > tempfile
        for(o=1;o<=e;o++){
            if ( access[o] ~ $1){
                access[o] = "#"access[o]
            }
        }       
    }   
}
END{   
    cmd = "mv "tempfile" "FILENAME
    # write changes to userDB.conf
    #system(cmd) #uncomment to use
    for(o=1;o<=e;o++){
        print access[o] > tempaccessfile
    }       
    cmd = "mv "tempaccessfile" "afile   
    # write changes to accesslist.conf
    # system(cmd) #uncomment to use
   
}
' accesslist.conf userDB.conf


magische_vogel 06-30-2009 09:39 PM

Quote:

Originally Posted by osor (Post 3592305)
How about something like this:
Code:

#!/usr/bin/awk -f
BEGIN {
        FS=OFS=";"
        "date +%Y/%m/%d" | getline now
}
ARGIND==1 {
        if ($6<now && $4==1)
                $4=0
        enabled[$1]=$4
        print $0
}
ARGIND==2 && FNR=1 {FS=OFS=" "}
ARGIND==2 { print (enabled[$2]?"":"#")$0 }


Hi osor,

When i run your script (./controle_accounts.awk usersDB.conf accesslist.conf) after i copy and past it to controle_accounts.awk file i don't get any results and any errors. Can you give more details on each statement you use in your script. did your script update the usersDB.conf and accesslist.conf files to reflect the last changes before it exit.

best regards.

magische_vogel 06-30-2009 09:53 PM

Quote:

Originally Posted by ghostdog74 (Post 3592346)
Code:

#!/bin/bash
awk 'BEGIN{
    OFS=FS=";"
    now=systime()       
    tempfile = "/var/etc/temp"
    tempaccessfile = "/var/etc/tempaccess"
}
FNR==NR{
    access[++e]=$0
    afile=FILENAME
    next   
}
$4==1{
    m=split($6,d,"/") #split date on /
    strd = d[1]" "d[2]" "d[3]" 0 0 0"
    t=mktime(strd) #get seconds
    if ( t < now ){
        $4=0
        print $0 > tempfile
        for(o=1;o<=e;o++){
            if ( access[o] ~ $1){
                access[o] = "#"access[o]
            }
        }       
    }   
}
END{   
    cmd = "mv "tempfile" "FILENAME
    # write changes to userDB.conf
    #system(cmd) #uncomment to use
    for(o=1;o<=e;o++){
        print access[o] > tempaccessfile
    }       
    cmd = "mv "tempaccessfile" "afile   
    # write changes to accesslist.conf
    # system(cmd) #uncomment to use
   
}
' accesslist.conf userDB.conf


Hi ghostdog74,

I get this error when i run your script:
awk: line 39: function mktime never defined
Can you explain how your script process the files.

Thanks.

ghostdog74 06-30-2009 09:59 PM

mktime for GNU awk (gawk).

magische_vogel 06-30-2009 10:28 PM

Quote:

Originally Posted by ghostdog74 (Post 3592366)
mktime for GNU awk (gawk).

now your script is working good, but i can't find the rest of my users in the usersDB.conf file, it seem that your script save only the disabled users to the temp file.is it possible to add the already enabled users to the temp file.

thanks

ghostdog74 06-30-2009 10:32 PM

Quote:

Originally Posted by magische_vogel (Post 3592394)
now your script is working good, but i can't find the rest of my users in the usersDB.conf file, it seem that your script save only the disabled users to the temp file.is it possible to add the already enabled users to the temp file.

thanks

move the "print $0 > tempfile" after the if (t<now) block.

magische_vogel 06-30-2009 10:47 PM

Quote:

Originally Posted by ghostdog74 (Post 3592401)
move the "print $0 > tempfile" after the if (t<now) block.

when i move the "print $0 > tempfile" after the if (t<now) block i get all the disbaled users duplicated in the usersDB.conf file, but already enabled users are ok. can you test that please.
thanx

$4==1{
m=split($6,d,"/") #split date on /
strd = d[1]" "d[2]" "d[3]" 0 0 0"
t=mktime(strd) #get seconds
if ( t < now ){
$4=0
print $0 > tempfile
for(o=1;o<=e;o++){
if ( access[o] ~ $1){
access[o] = "#"access[o]
}
}
}
print $0 > tempfile
}

magische_vogel 06-30-2009 10:59 PM

Quote:

Originally Posted by magische_vogel (Post 3592412)
when i move the "print $0 > tempfile" after the if (t<now) block i get all the disbaled users duplicated in the usersDB.conf file, but already enabled users are ok. can you test that please.
thanx

$4==1{
m=split($6,d,"/") #split date on /
strd = d[1]" "d[2]" "d[3]" 0 0 0"
t=mktime(strd) #get seconds
if ( t < now ){
$4=0
print $0 > tempfile
for(o=1;o<=e;o++){
if ( access[o] ~ $1){
access[o] = "#"access[o]
}
}
}
print $0 > tempfile
}

I'am sorry, i forget to remove the "print $0 > tempfile" from the if ( t < now ) block.

osor 06-30-2009 11:01 PM

Quote:

Originally Posted by magische_vogel (Post 3592341)
i don't get any results even the script was executed without errors.

Probably because I forgot to specify gawk (which has the ARGIND variable). There was also a typo I fixed.

If you want to use nawk, the following should work (I put in some comments):
Code:

#!/usr/bin/awk -f

FNR==1 { # when we encounter line 1 of a file
        FS=OFS= arg++ ? " " : ";" # arg = 1 for first file, 2 for second
        "date +%Y/%m/%d" | getline now
}
arg==1 {
        if ($6<now && $4==1)
                $4 = 0
        enabled[$1] = $4 # record the enabled status of this username
        print $0 > usersDB.conf
}
arg==2 { print (enabled[$2] ? "" : "#") $0  > access.conf }

You were correct in how to run it:
Code:

./scritpt.awk usersDB.conf access.conf

magische_vogel 06-30-2009 11:05 PM

Quote:

Originally Posted by magische_vogel (Post 3592417)
I'am sorry, i forget to remove the "print $0 > tempfile" from the if ( t < now ) block.

I get a strange behaviour when i run the script twice. All the disabled users are deleted from the usersDB.conf, can you check how your script process disabled users ($4==0).

thanx

magische_vogel 06-30-2009 11:24 PM

Quote:

Originally Posted by osor (Post 3592418)
Probably because I forgot to specify gawk (which has the ARGIND variable). There was also a typo I fixed.

If you want to use nawk, the following should work (I put in some comments):
Code:

#!/usr/bin/awk

FNR==1 { # when we encounter line 1 of a file
        FS=OFS= arg++ ? " " : ";" # arg = 1 for first file, 2 for second
        "date +%Y/%m/%d" | getline now
}
arg==1 {
        if ($6<now && $4==1)
                $4 = 0
        enabled[$1] = $4 # record the enabled status of this username
        print $0 > usersDB.conf
}
arg==2 { print (enabled[$2] ? "" : "#") $0  > access.conf }

You were correct in how to run it:
Code:

./scritpt.awk usersDB.conf access.conf


when i run the script i get the folowing errors:

debiansrv:/var/etc# ./test2 usersDB.conf accesslist.conf
awk: ./test2
awk: ^ syntax error
awk: ./test2
awk: ^ regular expression not terminated

can you check what happens please.

thanks

ghostdog74 06-30-2009 11:53 PM

Quote:

Originally Posted by magische_vogel (Post 3592422)
I get a strange behaviour when i run the script twice. All the disabled users are deleted from the usersDB.conf, can you check how your script process disabled users ($4==0).

thanx

by right, you are the one supposed to check, as i can't anticipate what you do at your end. Remove the > tempfile and just print to the screen and see the results. Insert print statements where applicable to see values of variables at different points in your code. That's one of the way to troubleshoot your program. If you are not familiar yet with (g)awk, learn it. See my sig for link to learning gawk

ghostdog74 06-30-2009 11:56 PM

Quote:

Originally Posted by osor (Post 3592418)
comments):
Code:

..
        if ($6<now && $4==1)
        ...


i really doubt 2 dates can be compared when in this format YYYY/MM/DD.

magische_vogel 07-01-2009 06:05 AM

Quote:

Originally Posted by osor (Post 3592418)
Probably because I forgot to specify gawk (which has the ARGIND variable). There was also a typo I fixed.

If you want to use nawk, the following should work (I put in some comments):
Code:

#!/usr/bin/awk

FNR==1 { # when we encounter line 1 of a file
        FS=OFS= arg++ ? " " : ";" # arg = 1 for first file, 2 for second
        "date +%Y/%m/%d" | getline now
}
arg==1 {
        if ($6<now && $4==1)
                $4 = 0
        enabled[$1] = $4 # record the enabled status of this username
        print $0 > usersDB.conf
}
arg==2 { print (enabled[$2] ? "" : "#") $0  > access.conf }

You were correct in how to run it:
Code:

./scritpt.awk usersDB.conf access.conf

Your script is working fine except when i run it twice it deactivate already deactivated users i mean it add # to "#F: username" in the access.conf file. is it possible to ignore the already deactivated users in the access.conf file.

#!/usr/bin/gawk -f

FNR==1 { # when we encounter line 1 of a file
FS=OFS= arg++ ? " " : ";" # arg = 1 for first file, 2 for second
"date +%Y/%m/%d" | getline now
}
arg==1 {
if ($6<now && $4==1)
$4 = 0
enabled[$1] = $4 # record the enabled status of this username
print $0 > "usersDB.conf"
}
arg==2 { print (enabled[$2] ? "" : "#") $0 > "access.conf" }

thanks a lot.

osor 07-01-2009 12:21 PM

Quote:

Originally Posted by magische_vogel (Post 3592436)
when i run the script i get the folowing errors:

Another typo (seems to be a theme with me in this thread)! I of course meant
Code:

#!/usr/bin/awk -f

osor 07-01-2009 12:21 PM

Quote:

Originally Posted by ghostdog74 (Post 3592469)
i really doubt 2 dates can be compared when in this format YYYY/MM/DD.

Sure they can (at least in ASCII and its supersets including UTF-8 and all 15 parts of ISO-8859).

String comparison is done as you would expect, and most charsets (including ASCII) are created so that digits follow the normal rules for succession (including 0 < 1). As long as all fields are padded (which in this case they are), the string comparison should suffice for dates.


All times are GMT -5. The time now is 08:22 PM.