LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-12-2012, 03:58 PM   #1
piyush128k
Member
 
Registered: Jun 2012
Posts: 68
Blog Entries: 1

Rep: Reputation: Disabled
diff of two files not working thru script


My bash script:-

#!/bin/bash
exec &> newfile
cat /tmp/query.txt | grep Inactive |awk -F: '{ print $1 }';
cat /home/piyush/linuxusers.txt |awk -F: '{ print $2 }';
if ! diff /tmp/query.txt linuxusers.txt >/dev/null; then
echo ids are different
else echo ids are same
comm -12 query.txt linuxusers.txt > newfile
fi

*******************************************************I have tried few commands from http://forums.opensuse.org/english/o...sh-script.html
*******************************************************

My table 1:- (the actual table does not have the field names). This table is my normal user list for my linux user accounts I have.
SNo. ID Names Status
1 11 Dave Active
2 12 Matt1 Active
3 13 Matt2 Active
4 14 Matt3 Inactive
5 15 Kevin Active
6 16 Joanne Active
7 17 Richard Inactive
8 18 Piyush Active
9 19 Robert Active
10 20 Ghost Active

My table 2:- (the actual table does not have the field names). This table is actually the out from a mysql query that filters the user names as in who is active and who is inactive
SNo. ID Names Status
1 11 Dave Active
2 12 Matt1 Active
3 13 Matt2 Active
4 14 Matt3 Inactive
5 15 Kevin Active
6 16 Joanne Active
7 17 Richard Inactive
_______________________________________________________

My objective is, through my script have a third file created which will only show the list of users who are either "Inactive" and who do not exit in table 2. Both conditions being met. Based on the changes I have made from the link so mentioned above, I get a mixed result but not what I want. I am failing in this and need to correct it. I have searched and found many commands probably it has confused me further because of which...I am not finding my mistake.

Help.
 
Old 09-12-2012, 04:52 PM   #2
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,353

Rep: Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989
You should ask yourself what you expect each line in that script to do, especially the exec and the two cat lines, and whether or not that's what you want.

For example, what is the exec doing? And "cat file | grep string" is one of the many useless uses of cat. You can just use grep directly with "grep string file". You're then piping this to awk and printing a certain column, however you're just printing this to the screen (the semicolons are unnecessary here BTW). You then do a diff on the original files, which means your previous grep and awk haven't actually done anything useful. The diff should be run separately, and the exit code can then be checked with an "if [ $? != 0 ]" or the like.

I'm not really following your description of what the script is supposed to do, so I can't help much beyond that. Maybe given your two input files you can provide us with what the output should look like?

Last edited by suicidaleggroll; 09-12-2012 at 04:58 PM.
 
Old 09-12-2012, 09:54 PM   #3
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,252

Rep: Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328Reputation: 2328
Everything suicidaleggroll said AND you are using 2 different linuxusers.txt files.
Also
Quote:
either "Inactive" and who do not exit in table 2
does not make sense.
In English its 'blah either blah OR blah' and 'exit' != 'exist'.

Quote:
Both conditions being met
Do you mean you WANT both conditions to be true ie 'AND' or do you mean you think both ARE true?

Last but not least, you do know you could do what you (seem) to want all in the SQL, instead of dumping the output, then fiddling around in bash?
 
Old 09-12-2012, 10:21 PM   #4
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 670Reputation: 670Reputation: 670Reputation: 670Reputation: 670Reputation: 670
There is a great book "Crash Course in MySQL" that I'd highly recommend.
 
Old 09-13-2012, 12:56 PM   #5
piyush128k
Member
 
Registered: Jun 2012
Posts: 68
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
This is what I want for output.
This is basically the list of users who are inactive and do not exist in the MySQL database when we compare both files for differences.

I should get Matt3 and Richard as listed not because they are listed like that in both tables but because they are inactive as per table 2, which is the MySQL table file.
I should also get Piyush, Robert and Ghost as well because they do not exist in table2. This will help me delete user accounts from my linux box.


Output:-
4 14 Matt3 Inactive
7 17 Richard Inactive
8 18 Piyush Active
9 19 Robert Active
10 20 Ghost Active
 
Old 09-14-2012, 10:32 AM   #6
piyush128k
Member
 
Registered: Jun 2012
Posts: 68
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
#!/bin/bash
exec &> newfile
cat /tmp/table2 file.txt|grep Inactive |awk -F: '{ print $1 }'
diff /tmp/table2file.txt /home/piyush/table1file.txt |grep ">"

Now this creates a 3rd file and give me exactly what I want.
Is this the right way to do it or....
 
Old 09-14-2012, 10:39 AM   #7
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,353

Rep: Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989Reputation: 1989
Two of those three lines are doing nothing. As I said before:
Quote:
You should ask yourself what you expect each line in that script to do, especially the exec and the two cat lines, and whether or not that's what you want.

For example, what is the exec doing? And "cat file | grep string" is one of the many useless uses of cat. You can just use grep directly with "grep string file". You're then piping this to awk and printing a certain column, however you're just printing this to the screen (the semicolons are unnecessary here BTW). You then do a diff on the original files, which means your previous grep and awk haven't actually done anything useful.
 
Old 09-16-2012, 07:41 AM   #8
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958
Please use ***[code][/code] tags*** around your code and data, to preserve formatting and to improve readability. Please do not use quote tags, bolding, colors, or other fancy formatting.

Quote:
And "cat file | grep string" is one of the many useless uses of cat. You can just use grep directly with "grep string file".
Not only this, but it's also a useless use of grep. You can do pattern matching directly in awk.

Code:
awk '/pattern/ { print $1 }' inputfile.txt >outputfile.txt
It always helps to learn how to use the tools you're calling on effectively.


Some more advice. Don't directly use filenames in your code. Store them (and any other repeating data) in variables at the top of the script, and use those throughout your code. Not only does it help avoid errors, but it makes it easier to change them later if needed. Don't forget to quote the variables.
 
Old 10-18-2012, 03:38 PM   #9
piyush128k
Member
 
Registered: Jun 2012
Posts: 68
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
Okay re-opening it because, the answers i got were not helping me as i thought they would

---------- Post added 10-18-12 at 02:39 PM ----------

#! /bin/bash
CMD="show databases; use metadata; show tables; select * from usernames; select * from personaldata; describe usernames; describe personaldata; select usernames.SNo, usernames.DataTelid, usernames.UName, personaldata.ActiveInactive from usernames, personaldata where usernames.DataTelid=personaldata.DataTelid into outfile '/tmp/querydb';"
mysql -u root -pnew-password -e "$CMD"
exec &> /tmp/final
cat /tmp/querydb|grep Inactive
awk -F':' '{print $1}' /etc/passwd | sort > /tmp/userlist
diff /tmp/userlist /tmp/querydb

My question:- I am stuck and need immediate help. I guess I dont have to explain much by looking at the code. However what i want is that: My querydb file has SNo, Company id, Full name and Active or Inactive as the status. My userlist file is a sorted list of user in /etc/passwd. I need to compare userlist file with querydb file to list the un comon user names and mention inactive beside their name.
 
Old 10-19-2012, 07:56 AM   #10
piyush128k
Member
 
Registered: Jun 2012
Posts: 68
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
No response ?
 
Old 10-19-2012, 09:01 AM   #11
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 670Reputation: 670Reputation: 670Reputation: 670Reputation: 670Reputation: 670
Look at the comm command to filter lines unique to one file:

comm -23 <(sort file1) <(sort file2)
will list lines unique to file 1.

The numbers determine which column not to print.
 
Old 10-19-2012, 09:47 AM   #12
piyush128k
Member
 
Registered: Jun 2012
Posts: 68
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
@jschiwal i believe you mean it will filter the lines as per the user names?
Because more than just the lines, the usernames is what i am concerned about.

trying your suggestions
 
Old 10-19-2012, 10:01 AM   #13
piyush128k
Member
 
Registered: Jun 2012
Posts: 68
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
diff is the name of my bash script in SLES 11x
when i do ./diff , i get display of both tables and their properties on the terminal window.
in the /tmp directory it creates querydb, userlist and final however.....
when i open the final file, it gives me the inactive username from the querydb file because of the "cat" command but....
at the bottom it says querydb file or directory does not exist.
here is the actual message:
" sort: open failed: querydb: No such file or directory" Does this mean that by the time the script executes the "comm" command the querydb file is not generated.

Your suggestion on using "comm" looks like this in my bash query:
comm -23 <(sort querydb) <(sort userlist)


Any suggestions
 
Old 10-19-2012, 10:08 AM   #14
piyush128k
Member
 
Registered: Jun 2012
Posts: 68
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
What i changed right now is:- comm -23 <(sort /tmp/querydb) <(sort /tmp/userlist)
and when i run ./diff (this is my actual script)

all three files are created (querydb, userlist and final)
now the final instead of an error message shows me the list as :-
the inactive user because of the "cat" command and it shows to me a common name that exists in the userlist and in the querydb
 
Old 10-19-2012, 01:29 PM   #15
piyush128k
Member
 
Registered: Jun 2012
Posts: 68
Blog Entries: 1

Original Poster
Rep: Reputation: Disabled
no response
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Perl Script to count files and directories is not working. user_28 Programming 15 08-29-2010 08:35 PM
Script to isolate files with 1 instance of each of 4 diff. words (grep) kmkocot Linux - Newbie 3 08-24-2009 09:54 AM
diff files elainelaw Linux - Software 2 07-23-2009 06:52 AM
diff two files noir911 Linux - Server 3 03-25-2009 06:00 PM
is it possible to diff ps files? markhod Linux - General 8 09-05-2005 01:17 AM


All times are GMT -5. The time now is 01:50 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration