Download your favorite Linux distribution at LQ ISO.
Go Back > Forums > Linux Forums > Linux - Newbie
User Name
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!


  Search this Thread
Old 05-07-2010, 01:22 PM   #1
LQ Newbie
Registered: Aug 2008
Location: Mauritius
Distribution: Mandriva 2008 Powerpack
Posts: 12

Rep: Reputation: 0
Text manipulation using sed

I have a number of text files (26 per database x 100+ databases) which need 'correcting' in order to import into postgresql.

I think that I have identified all the problem characters and I need to automate the process as much as possible.

I have a script to convert the characters and I do them one by one (not effecient but easier to understand).

What I cannot understand is why some conversions work in the script :

eg : sed 's/|/","/g' < > file.out
sed 's/$/");/' < > file2.out

but others will not :

eg sed 's/"/-/g

If I vi the individual data file, I can use the following :

eg :%s/"/-/g or
:%s'/'-'g and these work.

My problem characters are
1. ' to - (:%s/'/-/g)
2. " to - (:%s/"/-/g)
3. / to - (:%s'/'-'g)
4. \ to (:%s'\' 'g)
5. \\ to -(:%s'/\\'-'g)
6. ` to - (:%s/`/-/g)
7. & to a (:%s/&/a/g)
8. @ to a (:%s/@/a/g)
Old 05-07-2010, 01:36 PM   #2
LQ Guru
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978Reputation: 1978
Maybe it depends from the order by which it makes substitutions. For example you have to change \\ before \ otherwise the former will become -\. Regarding the syntax I checked and the following works for me:
sed "s/'/-/"
sed 's/"/-/'
sed 's/\//-/'
sed 's/\\\\/-/'
sed 's/\\/-/'
sed 's/`/-/'
sed 's/&/a/'
sed 's/@/a/'
Eventually, can you post some lines of an input file where you encounter problems?
Old 05-08-2010, 12:17 AM   #3
LQ Guru
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,530

Rep: Reputation: 2897Reputation: 2897Reputation: 2897Reputation: 2897Reputation: 2897Reputation: 2897Reputation: 2897Reputation: 2897Reputation: 2897Reputation: 2897Reputation: 2897
How about placing each group into a variable and then character class:

sed -e "s@[$change_dash]@-@g" -e 's@\\\\@-@g'
sed "s/[$change_empty]//g" $1
sed "s/[$change_a]/a/g" $1


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Challenging text manipulation using using sed or alternative phugoid Linux - Software 1 03-24-2009 01:41 AM
CSV | Text manipulation lmedland Programming 26 07-31-2008 10:44 AM
Easy string/text manipulation/indentation for restructured text brianmcgee Linux - Software 1 04-22-2008 08:27 PM
need help with text manipulation pcorajr Programming 12 12-15-2006 07:33 AM
More text manipulation ice_hockey Linux - General 2 05-28-2005 01:43 AM > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:12 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration