Regular Expressions

ziggy25 · 11-05-2007, 05:34 AM

Could someone please explain to me what is going on here..

File1

Code:

/oracle/syn/ccs/ccsora/admin/ccs/bin/mkSQLLdr: No such file or directory
merlin|oraccs|CCS> more mkSQLLdr
#!/bin/sh

files=`(ls data/*.dat)`
for file in $files
do
   if [ -f $file ] ; then
       ex -s $file< file2
   fi
done

file2

Code:

g/^$/d
g/^[ ]*$/d
g/--/d
g/rows selected/d
%s/  */ /g
//g
v/\~\~/j!
v/\~\~/j!
v/\~\~/j!
v/\~\~/j!
/g\~\~ /s/\~\~ /\~\~
v/\~\~/j!
v/\~\~/j!
v/\~\~/j!
g/@NULL@/s/@NULL@/NULL/g
g/NULL/s/NULL//g
g/"/s/"/""/g
g/@/s/@/"/g
g/\~\~/s/\~\~//g
g/^$/d
w
q

acid_kewpie · 11-05-2007, 05:40 AM

a list of files are processed by ex for regular expression matches. we can't really tell you much more than that an i expect you already know that. if you want to know what each regex does, then just google for a 101 about regular expressions. it's really hard to give you a meaning that you'll care about without any contextual data to relate them to.

ziggy25 · 11-05-2007, 05:48 AM

Here is a sample of the data that is being processed (Two rows from a database table).

Code:

@2344@,@james @,@NULL@,@John@,@05-JUL-04@,@A@,@702723344@,@XXXX@test.
com@,@vngixHkkilG6y23iiQ4lRc6CvewyI=@,@05-JUL-04@~~

@1323@,@23212 @,@NULL@,@Paul@,@05-JUL-04@,@A@,@803295023@,@James
@test.com@,@vngixHkkil32G6yiiQ4lRc6CvewyI=@,@05-JUL-04@~~

What is happening is that it replaces the @ sign with a " sign. Here is the converted data after i ran the script

Code:

"53509","12323",,"John","05-JUL-04","A","234356734","John"test.com","vngixHkkilG6yiiQ4lRc6CvewyI=","05-JUL-04"
"10357","1233",,"2132","05-JUL-04","A","45354734","James"test.com","vngixHkkilG6yiiQ4lRc6CvewyI=","05-JUL-04
"

As you can see the @ sign in the email address (Columns 7) was also replaced with the " sign. I need to understand how it works so that i can prevent it from replacing the @ sign on the email address.

Thanks

acid_kewpie · 11-05-2007, 05:57 AM

well that's one of them.... g/@/s/@/"/g. this reads, "for all lines (g/) which contain an @ (@), substitute (/s/) the @ (@) for a (/) Double quote (") in every instance (/g)"

if you want to avoid the email being battered, you'd need to improve the regex to be fussier. you would probably be best breaking it into multiple ones or it'll get a lot messier. so rather than replacing @ with ", replace '@,@' with '","', '^@' (@ at start of line) with ',' and '@$' (@ at end of line) with '"' in three separate goes. you *could* do it in one, but i doubt you'll really care to optimize like that.

way to get BT spammed btw...

colucix · 11-05-2007, 06:10 AM

Quote:

Originally Posted by acid_kewpie

and '@$' (@ at end of line) with '"' in three separate goes.

Actually in the examples @ is not at the end of line, but @\~\~$ is, which will be replaced by "\~\~. Note the escaped ~.

ziggy25 · 11-05-2007, 06:31 AM

Hi thanks for your reply

I understood the bit about replacing the @,@ with ",". I think i can do this by replacing g/@/s/@/"/g with /g/@,@/s/","/"/g

Could you explain in a bit more detail on how i can replace the @ sign at the begning of the line and the @~~ at the end of the line. The ~ character seem to be used all over the place and im not sure why.

And also you mentioned that /g means every instance. What do the other values mean? v,d, s, j, w and q.

Thanks

ps. i've edited the post to remove the company name. Could you also edit your post to remove reference to the company name.

acid_kewpie · 11-05-2007, 06:52 AM

your regex is wrong, but more becuase you don't really *need* the g/ part anyway, as you're just replicating what you already know. so just try '%s/@,@/","/g', without the g/@,@/ stuff at all to replace all lines (the % sign just means all lines). (check a vi cheat sheet for more knowledge about what this is doing)

and BT deserve all they get.. they make my working life hell, the chimps.

ziggy25 · 11-05-2007, 07:57 AM

What is this line doing (v/\~\~/j!) and why is it repeated seven times in the script?

Thanks