LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-03-2014, 02:32 PM   #1
Sunray74
LQ Newbie
 
Registered: Dec 2014
Posts: 11

Rep: Reputation: Disabled
Script to replace Control characters with a number, if not with Space


Hi Guys,

I'm new to Linux and i have a project where my script need to read the text files and replace the Control-L character in the First Byte with a number and if there is no Control-L character present then
the script should append/insert 'space' in the 1st byte of every line in the file.

I have created the script till replacing the Control-L character but unable to add the Decode statement to the SED, like if there is no Control-L then insert space in the first byte.

My Script:

** Quote **

#!/bin/bash

outdir='outdir'
indir=`pwd`

if [ $# -eq 1 ]
then

filename=$1

>$outdir/$filename.txt

files=`ls $indir/$filename[a-z].txt`

for i in $files
do
i=`basename $i`

sed s/^L/1/g $indir/$i > $outdir/$i

cat $outdir/$i >> $outdir/$filename.txt

rm $outdir/$i

done
else

echo "Invalid args"

exit 1
fi

** UnQuote **

Any help would be highly appreciated.

Regards.
 
Old 12-03-2014, 08:21 PM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,254

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
Couple of points:

1. Please use [code][/code] tags around code or data to protect formatting

2. 'Invalid args' whilst semi-informative it does not advise what the correct input may have been

3. $() is clearer and more versatile than ``

4. [[]] is preferred over [] and (()) preferred for arithmetic

5. Use your globbing directly in the for loop and try not to use ls. This example may not have issues but should your files names have unusual characters, such as spaces, the for loop will not behave as expected
Code:
for i in $indir/$filename[a-z].txt
6. Use meaningful variable names, like the 'i' in for loop above as in a large script the start of the loop may be many lines back so when you come across $i you have no real idea what it refers to

7. An alternative to basename is using the ## bash construct

8. Quote all variables, again to preserve any unusual characters ... of course the caveat is you may well want the unusual characters to expand

9. Quote sed statements (I have also shown quoting for variables below)
Code:
sed 's/^L/1/g' "$indir/$i" > "$outdir/$i"
10. You go from indir_i to outdir_i to outdir_filename and then remove outdir_i. Why not cut out the middle man and simply redirect the sed immediately into outdir_filename. This also allows you to remove the basename
line as well as we can now simply use our for loop variable:
Code:
sed 's/^L/1/g' "$i" >> "$outdir/$filename"
11. sed pattern flag, 'g', makes no sense based on your requirement that we are looking at the start of the line. Currently you will replace '^L' anywhere in the line and not specifically at the start.
Here is the alternative:
Code:
sed 's/^^L/1/' "$i" >> "$outdir/$filename"
Above are merely suggestions

Ok, on to your actual question. You are already replacing the '^L' so now you need to tell sed what to do for the non-^L starting lines.
Important tip here is that you need to process the non-^L lines first (try placing solution below in the alternate order and see what happens:
Code:
sed -re 's/^([^^L])/ \1/' -e 's/^^L/1/' "$i" >> "$outdir/$filename"
 
Old 12-04-2014, 12:30 AM   #3
Sunray74
LQ Newbie
 
Registered: Dec 2014
Posts: 11

Original Poster
Rep: Reputation: Disabled
Thanks a lot "grail". i would test the functionality and would update you accordingly.

Thanks once again for your suggestions and would make sure that i follow the standards.
 
Old 12-04-2014, 11:40 AM   #4
Sunray74
LQ Newbie
 
Registered: Dec 2014
Posts: 11

Original Poster
Rep: Reputation: Disabled
Thanks a Ton !! The code has worked for me and i have taken your suggestions in to consideration to modify the code accordingly.

Thanks once again.

Regards
 
Old 12-04-2014, 09:27 PM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,254

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
No probs ... glad we got there Please mark as SOLVED once you have your solution.
 
Old 12-08-2014, 11:55 AM   #6
Sunray74
LQ Newbie
 
Registered: Dec 2014
Posts: 11

Original Poster
Rep: Reputation: Disabled
Hi Grail,

Need your help in finding a way to read "*" (asterisk) in the file name, like for ex: test*123, and based on that i have to include
a character (any of the 4 alphabets - F, P, R or X) in place of the "*" and write the file name as "testF123" (replace asterisk with "F" in this case) to the output file.

Based on the file name, if it has a * then there should be a condition to write any of the 4 alphabets mentioned above in place of * and write the new file name (TESTF123)
to the output on the top left, precisely 3rd or 4th line and if there is no "*" in the file name then the earlier logic should be applied as is.

makes sense ?

Please let me know if you need more clarifications, so that i can get back to you with more information.

Thanks for your help.
 
Old 12-08-2014, 07:26 PM   #7
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,254

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
Quoting ... quoting ... quoting

Double quotes around variables will protect them from expanding unusual characters, such as *, and single quotes force any character to be seen as is, hence
using these two concepts you should easily be able to get bash to work out which files you need to update.
 
Old 12-10-2014, 11:34 AM   #8
Sunray74
LQ Newbie
 
Registered: Dec 2014
Posts: 11

Original Poster
Rep: Reputation: Disabled
Thanks for providing the tips. i was able to achieve the requirement which i have posted couple days back with the help of SED command.

also, can you guide me on how to search a specific string in a file and add a new string below the searched string ?

Ex: using my above script, i need to search a word "search_word" in the files where my script is reading and exactly below that i need to insert
a new word "new_word" and write that to the output. "search_word" would be in many places but would be starting only from the first byte.

Appreciate if you could throw some light.

Regards
 
Old 12-10-2014, 11:42 AM   #9
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,128

Rep: Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271
Quote:
Originally Posted by Sunray74 View Post
also, can you guide me on how to search a specific string in a file and add a new string below the searched string ?
in general:
sed 's/<search string>/<replace string>/' inputfile > outputfile works, you need to only specify those strings. For example beginning of line is ^, so:
sed 's/^search_word/new_wordsearch_word/' will do something like you described. see man page of sed and tutorials, examples about usage to find the most convenient way....
 
Old 12-10-2014, 12:51 PM   #10
Sunray74
LQ Newbie
 
Registered: Dec 2014
Posts: 11

Original Poster
Rep: Reputation: Disabled
Hi Pan64,

Thanks for the idea but this would actually replace the existing string which i do not want.
i tried that option to search for a 'search_word' and replace with 'search_word new_word", this way
i can achieve it but my requirement is as below..

File:

xxxxxxxxxxxxxxxxxxx
search_wordxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
search_wordxxxxxxxx
xxxxxxxxxxxxxxxxxxx

New Output should be like this...

xxxxxxxxxxxxxxxxxxx
search_wordxxxxxxxx
new_wordxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
search_wordxxxxxxxx
new_wordxxxxxxxxxxx

Appreciate if you could guide me in achieving the required output.

Thanks a lot for all your help.

Regards,
 
Old 12-10-2014, 01:00 PM   #11
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,128

Rep: Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271
so that means you want to print the original line and the modified version too?
sed '/search_word/{p;s/search_word/replace_word/}'
 
Old 12-10-2014, 01:05 PM   #12
Sunray74
LQ Newbie
 
Registered: Dec 2014
Posts: 11

Original Poster
Rep: Reputation: Disabled
Awesome, thanks a lot.

i have tested the logic on the command line and it worked. i would include this logic in to my script to test the functionality and would update you the results.

Thanks once again Pan64.
 
Old 12-10-2014, 01:09 PM   #13
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian i686 (solaris)
Posts: 8,128

Rep: Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271Reputation: 2271
you are welcome
(if you really want to say thanks just press YES)
 
1 members found this post helpful.
Old 12-10-2014, 02:37 PM   #14
Sunray74
LQ Newbie
 
Registered: Dec 2014
Posts: 11

Original Poster
Rep: Reputation: Disabled
Hi Pan64,

I have made a mistake here, the original file has empty lines after the "search_word" and the "new_word" only should insert below the "search_word" but it has taken the whole line with the command.

File:

xxxxxxxxxxxxxxxxxxx
search_wordxxxxxxxx

xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
search_wordxxxxxxxx

xxxxxxxxxxxxxxxxxxx

New Output should be like this...

xxxxxxxxxxxxxxxxxxx
search_wordxxxxxxxx
new_word

xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
search_wordxxxxxxxx
new_word

xxxxxxxxxxxxxxxxxxx

Sorry for the confusion and request you to provide me the correct syntax.
 
Old 12-10-2014, 08:02 PM   #15
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,254

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
So try:
Code:
sed '/search/a\new_word' file
Also, you can look here for more information on how to use sed.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Using sed in bash script to serach and replace special characters, and text PoltoX Programming 6 08-14-2013 11:25 PM
Remove control characters in Script output Dalavich Linux - Newbie 2 02-26-2013 04:05 PM
Bash script to accept password and replace characters with * as they are typed krisdames Programming 16 10-03-2011 12:22 AM
sed: replace same number of characters between tags unihiekka Linux - Newbie 6 12-30-2008 04:51 AM
How do I replace special characters in a string within a bash script? rhaup0317 Linux - Newbie 2 06-03-2008 12:56 PM


All times are GMT -5. The time now is 04:27 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration