LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-20-2011, 01:47 PM   #1
smritisingh03
Member
 
Registered: Nov 2010
Posts: 43

Rep: Reputation: 0
checking if table name is repeated in file and if yes ,remove it


Hi All

I have an output file that looks like this:

Quote:
status for ACCOUNT_MISSING_FRM_RCIS_LINK- mismatch
status for ACCOUNT_MISSING_FRM_RCIS_LINK is ACCOUNT_MISSING_FRM_RCIS_LINK- does not exist in DB
status for ADP_COMMENT- mismatch
status for ADP_CONFIG- match
status for ADP_FIELD- mismatch
status for ADP_HEADER- match
status for ADP_INDEX- mismatch
status for ADP_JOIN- mismatch
status for ADP_LANGUAGE- match
status for ADP_NATIVE_SQL- match
status for ADP_OBJECT- match
status for ADP_OBJECT_NEW- mismatch
status for ADP_OBJECT_NEW is ADP_OBJECT_NEW- does not exist in DB
now if you see the first line:

status for ACCOUNT_MISSING_FRM_RCIS_LINK- mismatch
status for ACCOUNT_MISSING_FRM_RCIS_LINK is ACCOUNT_MISSING_FRM_RCIS_LINK- does not exist in DB


this should appear just once as :

Quote:
status for ACCOUNT_MISSING_FRM_RCIS_LINK- does not exist in DB
the same goes for last line.

for further information the ACCOUNT_MISSING_FRM_RCIS_LINK is a table name and it row count is taken from a log and then Database checked for the rowcount to see if it is a match,mismatch,or the table does not exist!!

I am getting the desird output just that i need to do something to this output file.

Any help will be greatly appricited.Thankyou!!!!

Last edited by smritisingh03; 01-20-2011 at 01:50 PM.
 
Old 01-20-2011, 01:53 PM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
Hi,

This could work, depending on the rest of the file:

Code:
sed 's/status for .* is /status for /' infile
Hope this helps.
 
Old 01-20-2011, 02:24 PM   #3
smritisingh03
Member
 
Registered: Nov 2010
Posts: 43

Original Poster
Rep: Reputation: 0
Hi druuna

i tried ur sed but it is doing only part of the job.i am posting the output.

Quote:
status for TABLE_X_UDP_LVL1_VW_BASE_TMP- mismatch

status for TABLE_X_UDP_LVL2_VW_BASE_TMP- mismatch
status for TABLE_X_UDP_LVL2_VW_BASE_TMP- does not exist in DB

status for TABLE_X_UDP_LVL3_VW_BASE_TMP- mismatch
status for TABLE_X_UDP_LVL4_VW_BASE_TMP- mismatch
status for TABLE_X_UDP_LVL4_VW_BASE_TMP- does not exist in DB

status for TABLE_X_UDP_LVL5_VW_BASE_TMP- mismatch
status for TABLE_X_UDP_LVL5_VW_BASE_TMP- does not exist in DB
I want the output file to look like:-


status for TABLE_X_UDP_LVL2_VW_BASE_TMP- does not exist in DB
status for TABLE_X_UDP_LVL4_VW_BASE_TMP- does not exist in DB
status for TABLE_X_UDP_LVL5_VW_BASE_TMP- does not exist in DB

which meand that the first line in case where the table name is same should be deleted.

thankyou so much.I really really appreciate!!!
 
Old 01-20-2011, 02:47 PM   #4
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 19,258

Rep: Reputation: 4436Reputation: 4436Reputation: 4436Reputation: 4436Reputation: 4436Reputation: 4436Reputation: 4436Reputation: 4436Reputation: 4436Reputation: 4436Reputation: 4436
Quote:
Originally Posted by smritisingh03 View Post
Hi druuna
i tried ur sed but it is doing only part of the job.i am posting the output. I want the output file to look like:-

status for TABLE_X_UDP_LVL2_VW_BASE_TMP- does not exist in DB
status for TABLE_X_UDP_LVL4_VW_BASE_TMP- does not exist in DB
status for TABLE_X_UDP_LVL5_VW_BASE_TMP- does not exist in DB

which meand that the first line in case where the table name is same should be deleted.

thankyou so much.I really really appreciate!!!
Spell your words out, please. And now that we know what you want, can you post what you've written/tried?
 
Old 01-20-2011, 02:58 PM   #5
smritisingh03
Member
 
Registered: Nov 2010
Posts: 43

Original Poster
Rep: Reputation: 0
hi TBone

the input file to this script is:

Quote:
status for ACCOUNT_MISSING_FRM_RCIS_LINK- mismatch
status for ACCOUNT_MISSING_FRM_RCIS_LINK is ACCOUNT_MISSING_FRM_RCIS_LINK- does not exist in DB

status for ADP_COMMENT- mismatch
status for ADP_CONFIG- match
status for ADP_FIELD- mismatch
status for ADP_HEADER- match
status for ADP_INDEX- mismatch
status for ADP_JOIN- mismatch
status for ADP_LANGUAGE- match
status for ADP_NATIVE_SQL- match
status for ADP_OBJECT- match
status for ADP_OBJECT_NEW- mismatch
status for ADP_OBJECT_NEW is ADP_OBJECT_NEW- does not exist in DB

status for ADP_RELATION- match
status for ADP_TBL_OID- mismatch
status for ADP_TBL_OID_UNUSED- match
status for ADP_UPGRADE_OPS- match
status for ADP_VIEW_FIELD- mismatch
status for AUTHEN_NE_CON_BUS_PROD- mismatch
status for AUTHEN_NE_CON_BUS_PROD is AUTHEN_NE_CON_BUS_PROD- does not exist in DB
and what i tried is:

Quote:
#!/bin/sh



#sed '/is/d' statusOP


#awk '{
#if ($0 in stored_lines)
# x=1
# else
# print
# stored_lines[$0]=1
# }' statusOP

# #i > fileout

#awk '!x[$0]++' statusOP > statusOPfile.new

sed 's/status for .* is /status for /' statusOP
the output of this looks like:

Quote:
status for TABLE_X_UDP_LVL2_VW_BASE_TMP- mismatch
status for TABLE_X_UDP_LVL2_VW_BASE_TMP- does not exist in DB

status for TABLE_X_UDP_LVL3_VW_BASE_TMP- mismatch
status for TABLE_X_UDP_LVL4_VW_BASE_TMP- mismatch
status for TABLE_X_UDP_LVL4_VW_BASE_TMP- does not exist in DB

status for TABLE_X_UDP_LVL5_VW_BASE_TMP- mismatch
status for TABLE_X_UDP_LVL5_VW_BASE_TMP- does not exist in DB

status for TEMP_ADDRESS_WO_ACC- mismatch
status for TEMP_ADDRESS_WO_ACC- does not exist in DB
so what I want is:

status for TABLE_X_UDP_LVL2_VW_BASE_TMP- does not exist in DB[/B]
status for TABLE_X_UDP_LVL3_VW_BASE_TMP- mismatch
status for TABLE_X_UDP_LVL4_VW_BASE_TMP- does not exist in DB
status for TABLE_X_UDP_LVL5_VW_BASE_TMP- does not exist in DB
status for TEMP_ADDRESS_WO_ACC- does not exist in DB[/QUOTE]

ANY HELP WOULD BE GREATLY APPRECIATED!!!!

Last edited by smritisingh03; 01-20-2011 at 02:59 PM.
 
Old 01-20-2011, 03:05 PM   #6
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
Hi,

Would it be an idea to remove all the status for XYZ - does not exist in DB lines (trying to keep it simple)?
Or are there lines like that that do _not_ have a previous line with the same table name?

This should work, assuming that the duplicate tables are right after eachother:
Code:
#!/bin/bash

inFile="$1"

awk -F"[ -]" '
BEGIN { seen="" }
{ if( $3 == seen ){
    oldseen = $3
  } else { print }
  seen = $3
}' $inFile
Run it and provide an infile (somename.sh name.of.infile).

Hope this helps.
 
Old 01-20-2011, 03:37 PM   #7
smritisingh03
Member
 
Registered: Nov 2010
Posts: 43

Original Poster
Rep: Reputation: 0
Quote:
Would it be an idea to remove all the status for XYZ - does not exist in DB lines (trying to keep it simple)?
Or are there lines like that that do _not_ have a previous line with the same table name?
actually no as I have to bring out the status between the rowcount in logfile as well as databse.so basically when i say
ABC match

it means that thae rowcount for table ABC is same in log file as well as in database

ABC mismatch

means that the rowcount for ABC is different in database

ABC does not exist means that the table ABC does not exist in databse.So there is a difference in MISMATCH and DOES NOT EXIST which is why we cannot remove all the status for XYZ - does not exist in DB lines .

Quote:
Or are there lines like that that do _not_ have a previous line with the same table name?
yes it is the same pattern throughout i.e,

status for table ABC mismatch
status for table ABC does not exist in database

so i just want to get rid of status for table ABC mismatch

what i was thinking is to write a sed liner which finds the pattern does not exist in database and deletes the line above.I think this should work and I tried

awk -v RS='[^\n]*\n*pattern\n[^\n]*' '{print}' ORS="" infile

but it is not wkng.please help.
 
Old 01-20-2011, 03:43 PM   #8
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
Hi,

Ignore, see my next post.

My previous answer had 2 parts, the second (the awk shell script) should do what you want. It is not related to the previous paragraph.

I'm talking about this part:

Code:
#!/bin/bash

inFile="$1"

awk -F"[ -]" '
BEGIN { seen="" }
{ if( $3 == seen ){
    oldseen = $3
  } else { print }
  seen = $3
}' $inFile
Run it and provide an infile (somename.sh name.of.infile).


EDIT: Hold on..... I seem to have made misread something. Be right back

Last edited by druuna; 01-20-2011 at 03:55 PM. Reason: 2 conflicting rules posted by OP
 
Old 01-20-2011, 03:54 PM   #9
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
Hi,

Please don't mix your wanted output.....
From post #1
Quote:
this should appear just once as :

status for ACCOUNT_MISSING_FRM_RCIS_LINK- does not exist in DB
From post #7
Quote:
so i just want to get rid of status for table ABC mismatch
Assuming that post #7 is correct:
Code:
#!/bin/bash

inFile="$1"

awk -F"[ -]" '
BEGIN { seen="" }
{ if( $3 != seen ){
    oldseen = $3 ; print
  }
  seen = $3
}' $inFile
Assuming post #1 is what you want:
Code:
#!/bin/bash

inFile="$1"

awk -F"[ -]" '
BEGIN { seen="" }
{ if( $3 == seen ){
    oldseen = $3
  } else { print }
  seen = $3
}' $inFile
 
Old 01-20-2011, 04:30 PM   #10
smritisingh03
Member
 
Registered: Nov 2010
Posts: 43

Original Poster
Rep: Reputation: 0
it doesnt work!!!
 
Old 01-20-2011, 04:36 PM   #11
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
"It" does work on my side.........
 
Old 01-21-2011, 03:08 AM   #12
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Maybe something like:
Code:
awk -F"[ -]+" 'NR == 1{x = $0;y = $3;next}{if(y != $3){print x;x = $0;y = $3}else{$3=$4="";x = $0}}' file
 
Old 01-24-2011, 04:53 PM   #13
smritisingh03
Member
 
Registered: Nov 2010
Posts: 43

Original Poster
Rep: Reputation: 0
Thankyoy all for helping me solve this one...hats off to this forum!!!!!!!!
 
Old 01-25-2011, 04:27 AM   #14
sumeet inani
Member
 
Registered: Oct 2008
Posts: 902
Blog Entries: 26

Rep: Reputation: 49
Don't forget to thank members for their useful reply.

Last edited by sumeet inani; 01-25-2011 at 04:28 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
File recovery after partition table damage - XFS file system gracecourt Linux - General 4 09-28-2017 06:29 PM
[SOLVED] Bash. Remove from text table Algis Programming 1 11-07-2010 03:13 AM
Any log file for checking file permission change history in RHEL 5.1? bilalcochin Linux - Newbie 3 04-02-2010 10:57 AM
External USB disk: repeated file system corruption Sven Sandberg Linux - Hardware 1 08-12-2007 06:02 AM
-- MARK -- : Why is this repeated in my log file? scarr3d Debian 6 11-19-2003 05:33 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:55 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration