LinuxQuestions.org - [SOLVED] Text manipulation using sed

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - Text manipulation using sed (https://www.linuxquestions.org/questions/linux-newbie-8/text-manipulation-using-sed-806531/)

Text manipulation using sed

I have a number of text files (26 per database x 100+ databases) which need 'correcting' in order to import into postgresql.

I think that I have identified all the problem characters and I need to automate the process as much as possible.

I have a script to convert the characters and I do them one by one (not effecient but easier to understand).

What I cannot understand is why some conversions work in the script :

eg : sed 's/|/","/g' < file.in > file.out
sed 's/$/");/' < file1.in > file2.out

but others will not :

eg sed 's/"/-/g

If I vi the individual data file, I can use the following :

eg :%s/"/-/g or
:%s'/'-'g and these work.

My problem characters are
1. ' to - (:%s/'/-/g)
2. " to - (:%s/"/-/g)
3. / to - (:%s'/'-'g)
4. \ to (:%s'\' 'g)
5. \\ to -(:%s'/\\'-'g)
6. ` to - (:%s/`/-/g)
7. & to a (:%s/&/a/g)
8. @ to a (:%s/@/a/g)

Maybe it depends from the order by which it makes substitutions. For example you have to change \\ before \ otherwise the former will become -\. Regarding the syntax I checked and the following works for me:

Code:

sed "s/'/-/"

sed 's/"/-/'

sed 's/\//-/'

sed 's/\\\\/-/'

sed 's/\\/-/'

sed 's/`/-/'

sed 's/&/a/'

sed 's/@/a/'

Eventually, can you post some lines of an input file where you encounter problems?

How about placing each group into a variable and then character class:

Code:

change_dash="\"'\`/"

change_a='&@'

change_empty='\\'



sed -e "s@[$change_dash]@-@g" -e 's@\\\\@-@g'

sed "s/[$change_empty]//g" $1

sed "s/[$change_a]/a/g" $1