LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-10-2013, 09:22 AM   #16
castor0troy
LQ Newbie
 
Registered: Apr 2013
Posts: 27

Original Poster
Rep: Reputation: Disabled

the keyword in the main file should 'begin with' the keyword in input.txt
123goldcoast doesnt begin with 'goldcoast'
 
Old 04-10-2013, 11:21 AM   #17
millgates
Member
 
Registered: Feb 2009
Location: 192.168.x.x
Distribution: Slackware
Posts: 840

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
With regexp, you can match the begining of the line with ^. So, for example in awk, it could look like this:
Code:
keyword="^goldcoast";
if ($0 ~ keyword) { ... }
or with grep:

Code:
grep "^goldcoast" file
 
Old 04-10-2013, 12:52 PM   #18
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,245

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
As the keyword is to be at the start of the line, is there any definitive delimiter at the end of the word that would allow us to set the first field equal to all words potentially in the input file?
 
Old 04-10-2013, 10:29 PM   #19
castor0troy
LQ Newbie
 
Registered: Apr 2013
Posts: 27

Original Poster
Rep: Reputation: Disabled
thanks for the support.
i am trying to learn how to execute an awk script.
how do i do this?

should i save it as file.sh on the server?
 
Old 04-10-2013, 11:19 PM   #20
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.8, Centos 5.10
Posts: 17,240

Rep: Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324Reputation: 2324
Save it as file.awk and make it executable (chmod u+x file.awk)
 
Old 04-11-2013, 12:06 AM   #21
castor0troy
LQ Newbie
 
Registered: Apr 2013
Posts: 27

Original Poster
Rep: Reputation: Disabled
thanks
i saved this as ss.awk and executed the script echo | awk -f ss.awk but got this error.
can someone check if there is an error in the awk file.



awk: ss.awk:1: awk '
awk: ss.awk:1: ^ invalid char ''' in expression


==========================

awk '
NR == FNR { kw[$0]=0; }
NR != FNR {
for (w in kw) {
split($0,a,w);
kw[w]+= (length(a)-1);
}
}
END {
for(w in kw) {print w, kw[w];}
}
' input.txt file.txt >output.txt
 
Old 04-11-2013, 01:21 AM   #22
millgates
Member
 
Registered: Feb 2009
Location: 192.168.x.x
Distribution: Slackware
Posts: 840

Rep: Reputation: 380Reputation: 380Reputation: 380Reputation: 380
In this form, it's actually a shell script which calls awk. To make it an awk script, it should look like this:

Code:
#!/usr/bin/awk -f

NR == FNR { kw[$0]=0; }
NR != FNR {
    for (w in kw) {
        split($0,a,w);
        kw[w]+= (length(a)-1);
    }
}
END {
    for(w in kw) {print w, kw[w];}
}
 
Old 04-11-2013, 01:28 AM   #23
castor0troy
LQ Newbie
 
Registered: Apr 2013
Posts: 27

Original Poster
Rep: Reputation: Disabled
thanks.
you have removed this line.
' input.txt file.txt >output.txt


how do i define the files?
 
Old 04-11-2013, 02:14 AM   #24
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,104

Rep: Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267
you just execute the script:
<scriptname> input.txt file.txt > output.txt
 
Old 04-11-2013, 06:34 PM   #25
w1k0
Senior Member
 
Registered: May 2008
Location: Poland
Distribution: Slackware, Mint
Posts: 1,252

Rep: Reputation: 205Reputation: 205Reputation: 205
1. Prepare cat file:

Code:
cat1
1cat
cat2
2cat
cat3
3cat
cat4
4cat
cat5
5cat
2. Prepare cat.sh script:

Code:
#!/bin/bash
mkdir cat\?
cat="grep -c '^cat'"
eval "$cat cat" > cat\?/cat.cat
3. You will find the result in cat\?/cat.cat file.

(The answer is 5.)
 
Old 04-12-2013, 04:19 AM   #26
castor0troy
LQ Newbie
 
Registered: Apr 2013
Posts: 27

Original Poster
Rep: Reputation: Disabled
ok guys.
after 2 hrs of struggling i managed to run the script successfully.
echo | awk -f ss.awk input.txt file.txt >output.txt

it worked for smaller files.
however its not giving results for bigger files.
input.txt is 20mb and file.txt is 2gb.

is there anything besides awk that i could use for bigger files?
 
Old 04-12-2013, 04:39 AM   #27
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,104

Rep: Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267
that echo | at the beginning should be removed.
awk should be able to handle that file also.
Have you got any (error) message?
 
Old 04-12-2013, 06:05 AM   #28
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,245

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
Agreed, very hard to diagnose:
Quote:
however its not giving results for bigger files.
 
Old 04-12-2013, 07:22 AM   #29
castor0troy
LQ Newbie
 
Registered: Apr 2013
Posts: 27

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
that echo | at the beginning should be removed.
awk should be able to handle that file also.
Have you got any (error) message?
no error message.script just keeps on running but no results.

i tried the grep command on the same files and got this error
]# grep -of input.txt file.txt |sort|uniq -c >output.txt
grep: file.txt: Input/output error

any idea?
 
Old 04-12-2013, 08:05 AM   #30
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,104

Rep: Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267Reputation: 2267
probably in /var/log you will see some interesting messages...
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Creating an alias in ksh that uses grep and includes 'grep -v grep' doug248 Linux - Newbie 2 08-05-2012 03:07 PM
[SOLVED] run ps|grep command by script/command line ... ERROR: Unsupported option (BSD syntax) masuch Programming 4 05-23-2012 05:13 AM
How to pass the result of a command to another command (like grep) desb01 Programming 4 06-25-2009 01:09 PM
Help me in Grep Command + cd command in single line JeiPrakash Linux - Newbie 3 05-27-2008 05:16 AM
grep command itz2000 Linux - Newbie 2 09-21-2005 08:06 PM


All times are GMT -5. The time now is 03:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration