LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-05-2007, 05:34 AM   #1
ziggy25
Member
 
Registered: Aug 2005
Distribution: Debian 5.2
Posts: 56

Rep: Reputation: 15
Regular Expressions


Could someone please explain to me what is going on here..

File1
Code:
/oracle/syn/ccs/ccsora/admin/ccs/bin/mkSQLLdr: No such file or directory
merlin|oraccs|CCS> more mkSQLLdr
#!/bin/sh

files=`(ls data/*.dat)`
for file in $files
do
   if [ -f $file ] ; then
       ex -s $file< file2
   fi
done
file2
Code:
g/^$/d
g/^[ ]*$/d
g/--/d
g/rows selected/d
%s/  */ /g
//g
v/\~\~/j!
v/\~\~/j!
v/\~\~/j!
v/\~\~/j!
/g\~\~ /s/\~\~ /\~\~
v/\~\~/j!
v/\~\~/j!
v/\~\~/j!
g/@NULL@/s/@NULL@/NULL/g
g/NULL/s/NULL//g
g/"/s/"/""/g
g/@/s/@/"/g
g/\~\~/s/\~\~//g
g/^$/d
w
q
 
Old 11-05-2007, 05:40 AM   #2
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974
a list of files are processed by ex for regular expression matches. we can't really tell you much more than that an i expect you already know that. if you want to know what each regex does, then just google for a 101 about regular expressions. it's really hard to give you a meaning that you'll care about without any contextual data to relate them to.
 
Old 11-05-2007, 05:48 AM   #3
ziggy25
Member
 
Registered: Aug 2005
Distribution: Debian 5.2
Posts: 56

Original Poster
Rep: Reputation: 15
Here is a sample of the data that is being processed (Two rows from a database table).

Code:
@2344@,@james @,@NULL@,@John@,@05-JUL-04@,@A@,@702723344@,@XXXX@test.
com@,@vngixHkkilG6y23iiQ4lRc6CvewyI=@,@05-JUL-04@~~

@1323@,@23212 @,@NULL@,@Paul@,@05-JUL-04@,@A@,@803295023@,@James
@test.com@,@vngixHkkil32G6yiiQ4lRc6CvewyI=@,@05-JUL-04@~~
What is happening is that it replaces the @ sign with a " sign. Here is the converted data after i ran the script

Code:
"53509","12323",,"John","05-JUL-04","A","234356734","John"test.com","vngixHkkilG6yiiQ4lRc6CvewyI=","05-JUL-04"
"10357","1233",,"2132","05-JUL-04","A","45354734","James"test.com","vngixHkkilG6yiiQ4lRc6CvewyI=","05-JUL-04
"
As you can see the @ sign in the email address (Columns 7) was also replaced with the " sign. I need to understand how it works so that i can prevent it from replacing the @ sign on the email address.

Thanks

Last edited by ziggy25; 12-02-2009 at 08:48 AM.
 
Old 11-05-2007, 05:57 AM   #4
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974
well that's one of them.... g/@/s/@/"/g. this reads, "for all lines (g/) which contain an @ (@), substitute (/s/) the @ (@) for a (/) Double quote (") in every instance (/g)"

if you want to avoid the email being battered, you'd need to improve the regex to be fussier. you would probably be best breaking it into multiple ones or it'll get a lot messier. so rather than replacing @ with ", replace '@,@' with '","', '^@' (@ at start of line) with ',' and '@$' (@ at end of line) with '"' in three separate goes. you *could* do it in one, but i doubt you'll really care to optimize like that.

way to get BT spammed btw...

Last edited by acid_kewpie; 11-05-2007 at 06:02 AM.
 
Old 11-05-2007, 06:10 AM   #5
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976Reputation: 1976
Quote:
Originally Posted by acid_kewpie View Post
and '@$' (@ at end of line) with '"' in three separate goes.
Actually in the examples @ is not at the end of line, but @\~\~$ is, which will be replaced by "\~\~. Note the escaped ~.
 
Old 11-05-2007, 06:31 AM   #6
ziggy25
Member
 
Registered: Aug 2005
Distribution: Debian 5.2
Posts: 56

Original Poster
Rep: Reputation: 15
Hi thanks for your reply

I understood the bit about replacing the @,@ with ",". I think i can do this by replacing g/@/s/@/"/g with /g/@,@/s/","/"/g

Could you explain in a bit more detail on how i can replace the @ sign at the begning of the line and the @~~ at the end of the line. The ~ character seem to be used all over the place and im not sure why.

And also you mentioned that /g means every instance. What do the other values mean? v,d, s, j, w and q.

Thanks

ps. i've edited the post to remove the company name. Could you also edit your post to remove reference to the company name.
 
Old 11-05-2007, 06:52 AM   #7
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974
your regex is wrong, but more becuase you don't really *need* the g/ part anyway, as you're just replicating what you already know. so just try '%s/@,@/","/g', without the g/@,@/ stuff at all to replace all lines (the % sign just means all lines). (check a vi cheat sheet for more knowledge about what this is doing)

and BT deserve all they get.. they make my working life hell, the chimps.
 
Old 11-05-2007, 07:57 AM   #8
ziggy25
Member
 
Registered: Aug 2005
Distribution: Debian 5.2
Posts: 56

Original Poster
Rep: Reputation: 15
What is this line doing (v/\~\~/j!) and why is it repeated seven times in the script?

Thanks
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Regular expressions bhuwan Programming 5 02-26-2006 12:07 AM
regular expressions. stomach Linux - Software 1 02-10-2006 07:41 AM
Regular Expressions markjuggles Programming 2 05-05-2005 12:39 PM
Regular Expressions overbored Linux - Software 3 06-24-2004 03:34 PM
regular expressions? alaios Linux - General 2 06-11-2003 04:51 PM


All times are GMT -5. The time now is 10:38 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration