LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 02-27-2018, 05:34 AM   #31
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 10,883

Rep: Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237

there is no need to write script, grep will do it by itself.
as I told you already: grep --file=lookup-file input-file > output-file
just you need to use the lookup file I posted. see man grep about details (of --file)
 
1 members found this post helpful.
Old 02-27-2018, 05:47 AM   #32
keefaz
LQ Guru
 
Registered: Mar 2004
Distribution: Slackware
Posts: 6,154

Rep: Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698Reputation: 698
Depending on how large the files sizes are, perl language could be a good candidate too
But I agree with pan64, grep and creation of patterns files looks appealing with 4 lines of code to solve the whole problem
 
Old 02-27-2018, 05:52 AM   #33
bishnumnnit2006
LQ Newbie
 
Registered: Feb 2018
Posts: 16

Original Poster
Rep: Reputation: Disabled
Hey sorry but I am not sure how grep will work when we will getting different codes on every run, suppose we have 100 codes in a main file are we grepping all 100 codes in script, I am not sure if it works.
 
Old 02-27-2018, 05:55 AM   #34
bishnumnnit2006
LQ Newbie
 
Registered: Feb 2018
Posts: 16

Original Poster
Rep: Reputation: Disabled
This is what my actual code look like and its working fine.

#!/bin/bash



. /xmretl/Scripts/setTLMEtlEnv.ksh



. $XmrEtlScripts/TLMCmnFuncs.ksh



CurrDt=`date +%Y%m%d_%H%M%S`



LogFile=SplitTLMGrpRefFiles_Test_$CurrDt.log



echo "Start the execution of the script" >> $XmrEtlScriptLogs/$LogFile



echo "Client code in source file is "$CODE"" >> $XmrEtlScriptLogs/$LogFile

touch ${XmrEtlSrcFiles}/AMR1_CODE.TXT

touch ${XmrEtlSrcFiles}/AMR2_CODE.TXT

chmod 777 ${XmrEtlSrcFiles}/*CODE.TXT*

cat ${XmrEtlSrcFiles}/AMR1.TXT | awk 'BEGIN{RS=" "}{$1=$1}1' > ${XmrEtlSrcFiles}/AMR1_CODE.TXT

cat ${XmrEtlSrcFiles}/AMR2.TXT | awk 'BEGIN{RS=" "}{$1=$1}1' > ${XmrEtlSrcFiles}/AMR2_CODE.TXT





while read -r LINE

do



#CODE=`echo $LINE |cut -d'|' -f4|cut -d'-' -f1|grep -v AcctID`

CODE=`echo $LINE |awk -F '|' '{ split($4,a,"-"); print a[1]}'`

#echo "Client code in source file is "$CODE"" >> $XmrEtlScriptLogs/$LogFile

#touch ${XmrEtlSrcFiles}/AMR1_CODE.TXT

#touch ${XmrEtlSrcFiles}/AMR2_CODE.TXT

#chmod 777 ${XmrEtlSrcFiles}/*CODE.TXT*

#cat ${XmrEtlSrcFiles}/AMR1.TXT | awk 'BEGIN{RS=" "}{$1=$1}1' > ${XmrEtlSrcFiles}/AMR1_CODE.TXT

#cat ${XmrEtlSrcFiles}/AMR2.TXT | awk 'BEGIN{RS=" "}{$1=$1}1' > ${XmrEtlSrcFiles}/AMR2_CODE.TXT



while read -r LINE1

do

LKP_AMR1=`echo $LINE1`

if [[ $CODE = $LKP_AMR1 ]]

then

`touch /xmretl/TLMLandingZone/AMR1_ERU_6_GROUPREFERENCE_12012017_3.TXT` >> $XmrEtlScriptLogs/$LogFile

`chmod 775 /xmretl/TLMLandingZone/AMR1_ERU_6_GROUPREFERENCE_12012017_3.TXT`

`echo $LINE >> /xmretl/TLMLandingZone/AMR1_ERU_6_GROUPREFERENCE_12012017_3.TXT`

fi

done < ${XmrEtlSrcFiles}/AMR1_CODE.TXT





while read -r LINE2

do

LKP_AMR2=`echo $LINE2`

if [[ $CODE = $LKP_AMR2 ]]

then

`touch /xmretl/TLMLandingZone/AMR2_ERU_6_GROUPREFERENCE_12012017_3.TXT`

`chmod 775 /xmretl/TLMLandingZone/AMR2_ERU_6_GROUPREFERENCE_12012017_3.TXT`

`echo $LINE >> /xmretl/TLMLandingZone/ERU_6_GROUPREFERENCE_12012017_3.TXT`

fi



done < ${XmrEtlSrcFiles}/AMR2_CODE.TXT



done < ${XmrEtlSrcFiles}/ERU_6_GROUPREFERENCE_02262018_1.TXT
 
Old 02-27-2018, 05:56 AM   #35
bishnumnnit2006
LQ Newbie
 
Registered: Feb 2018
Posts: 16

Original Poster
Rep: Reputation: Disabled
The only issue is with the performance, so if you have any alternate solution please come out with the script, Many Thanks.
 
Old 02-27-2018, 06:01 AM   #36
rtmistler
Moderator
 
Registered: Mar 2011
Location: MA, USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu, Debian
Posts: 6,709
Blog Entries: 12

Rep: Reputation: 2360Reputation: 2360Reputation: 2360Reputation: 2360Reputation: 2360Reputation: 2360Reputation: 2360Reputation: 2360Reputation: 2360Reputation: 2360Reputation: 2360
Quote:
Originally Posted by bishnumnnit2006 View Post
Hey sorry but I am not sure how grep will work when we will getting different codes on every run, suppose we have 100 codes in a main file are we grepping all 100 codes in script, I am not sure if it works.
You've been shown quite a few examples by various members. Please understand that you stand to raise the frustration level of them if you seemingly don't appear to be trying any of their suggestions. There are plenty of things you can do to test and debug, much of which I imagine you have done a bit of. I'd suggest that you update your fellow members with some progress with a few attempt updates. People are not here to do these tasks literally for you, we are here to help you to learn and do for yourself. You can use a smaller set of data to test a few tactics and then run it on your larger files. Bear in mind that I don't know everything about awk, it is one of my weakest "non-talents" in the Unix/Linux world. However when I choose to use it, or the tr command, I perform a web search saying exactly what I wish to do with the command. And if I've been too particular, I wean it down to be "how do I split a line with awk", "how do I print the 5th field with awk", and so forth. As you get to use LQ more and eventually offer advice to fellow members, you'll appreciate those who, while they have questions, they also work with some diligence at attaining their solution. Also, you'll feel much more satisfied if you partake an active role in solving your problem, you will own part of that solution, if not most of it.
Quote:
Originally Posted by bishnumnnit2006 View Post
The only issue is with the performance, so if you have any alternate solution please come out with the script, Many Thanks.
I wrote all of the above from your earlier reply. This that you just wrote is now bordering on rudeness towards forum members. Please consider what I've written above. If you feel that LQ members are here to hand you solutions, then this is not a forum where you can attain the help you are seeking.

You also persist with failing to put your code within [code] tags to properly separate it and retain formatting.
 
1 members found this post helpful.
Old 02-27-2018, 06:05 AM   #37
bishnumnnit2006
LQ Newbie
 
Registered: Feb 2018
Posts: 16

Original Poster
Rep: Reputation: Disabled
That's fine, thanks everyone for your help, you can closed this discussion .

Have a nice time!
 
Old 02-27-2018, 06:19 AM   #38
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 10,883

Rep: Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237Reputation: 3237
I have already posted some improvements and already mentioned that....
 
Old 02-27-2018, 08:18 AM   #39
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: MID-SOUTH USA
Distribution: Slackware 14.2 / Slackware 14.2 current / Manjaro
Posts: 6,347

Rep: Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228
Quote:
Originally Posted by bishnumnnit2006 View Post
Hey sorry but I am not sure how grep will work when we will getting different codes on every run, suppose we have 100 codes in a main file are we grepping all 100 codes in script, I am not sure if it works.
this is why one should and some do, write the code, in a command line or script, does not matter, then test it with some actual data pulled out of the chuck of it all and placed into a secure location so it does not effect the normal process.

even just that one grep line that is said that is all you need, then place it into a file by whatever name so you do not have to keep rewriting the entire line all of the time, just the filename to get it to do its work.


one shouldn't do anything like this on production runs without first testing it.
 
Old 02-27-2018, 08:52 AM   #40
BW-userx
LQ Guru
 
Registered: Sep 2013
Location: MID-SOUTH USA
Distribution: Slackware 14.2 / Slackware 14.2 current / Manjaro
Posts: 6,347

Rep: Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228Reputation: 1228
this is knowing what you are looking for that the information held within the two files have.
Code:
$ egrep 'GHU|TRP|ABC|AAR|TIA' source
6|ERU_MCSD_POS|ERU_MCSD_POS|AAR-IAS-AR8F92001T2-C
6|ERU6_ARCGEN|generic Accounts|ABC-IAS-AR8F92001T2-C
6|ERU6_ARCGEN|Archive Accounts|TRP-IAS-AR8F92001T2-C
6|ERU6_ARCGEN|Archive Generic|TIA-IAS-AR8F92001T2-C
 
Old 02-27-2018, 03:59 PM   #41
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_10{.0|.1|.2}
Posts: 4,664
Blog Entries: 6

Rep: Reputation: 2525Reputation: 2525Reputation: 2525Reputation: 2525Reputation: 2525Reputation: 2525Reputation: 2525Reputation: 2525Reputation: 2525Reputation: 2525Reputation: 2525
@bishnumnnit2006:

As you have been аsked, please place your code snippets inside [CODE]...[/CODE] tags for better readability. You may type those yourself or click the "#" button in the edit controls.

You have received much good advice, including complete examples of how to approach your problem, but have not shown any effort of your own to use, or even consider the help provided. You persist in asking for others to provide complete solutions for you. This is not how LQ works.

Please review the Site FAQ for guidance in posting your questions and general forum usage.

We are happy to help, but if you are looking for others to do your work for you, this is not the place.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Script help to name a file from 1st column in a file SmurfGGM Linux - General 2 08-26-2014 08:07 AM
LXer: Monster RPG 2 source code release on Dec. 1st LXer Syndicated Linux News 0 11-09-2012 01:02 AM
[SOLVED] substring match in specific column udiubu Linux - Newbie 2 06-05-2012 05:29 AM
Printing a column in a file.. slight confusion with my code pdklinux79 Linux - Newbie 3 06-17-2008 06:19 PM
MySQL splitting a string with a delimiter and taking the 1st value and update Lantzvillian Programming 7 01-31-2008 11:57 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 06:53 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration