LinuxQuestions.org
Latest LQ Deal: Linux Power User Bundle
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 04-10-2014, 08:51 PM   #1
boblc123
LQ Newbie
 
Registered: Feb 2013
Posts: 13

Rep: Reputation: Disabled
bash script to remove line from file that contains special character at the end


Hey all, It's me again.

I have a directory with about 200,000 files in it. There's 1 specific line that I need to remove from each file and that line contains a special character at the end (I think the octal value of the character is 023). Any suggestions on how to do it? The special character really throws me off.

Here's how I'm finding the line that I need to remove:

for i in `ls -f -1'
do
myvar=`strings $i | head -25 | grep -i REMOVEME`

#Here is where I need code to remove the above line from the file without creating a backup (I don't have enough space to backup 200k files). Remember the above line contains a special character at the end.

NOTE: I'm using bash. I've heard perl might be better to use for this, but I don't know any perl.

Last edited by boblc123; 04-10-2014 at 08:53 PM. Reason: adding info.
 
Old 04-10-2014, 09:25 PM   #2
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290
Hi,

you should be able to do this with a simple sed oneliner for example
Code:
sed '/.*X$/d' foo.txt
This will delete all lines that end with "X" and output to the terminal. Replace X with whatever the character is. Once you are convinced the sed command does what you want you can make it edit the file inplace by using the -i option. I strongly recommend that make backups before doing any of the following
Eg
Code:
sed -i '/.*X$/d' foo.txt
In principle you can pass multiple files. Eg
Code:
sed -i '/.*X$/d' foo.txt bar.txt baz.txt
Etc. Not sure if 200000 files will work though. In that case you could do something like
Code:
for f in * ; do sed -i '/.*X$/d' $f ; done
HTH,

Evo2.

PS. Test carefully before running any commands on your files and be sure to have working/tested backups.
 
Old 04-11-2014, 09:53 AM   #3
boblc123
LQ Newbie
 
Registered: Feb 2013
Posts: 13

Original Poster
Rep: Reputation: Disabled
Well, I don't want to delete the special character everywhere in the file. The special character appears in multiple places, but I only want to delete it from one specific line.
 
Old 04-11-2014, 10:51 AM   #4
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,629

Rep: Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944Reputation: 2944
So match against whatever makes that line stand out.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Script to remove character and malicious line from any file. sinjed Linux - Newbie 10 04-18-2013 08:51 PM
Shell script/Perl Script to remove the string until it finds special character '_' pooppp Programming 10 07-17-2012 10:36 AM
Shell script/Perl Script to remove the string until it finds special character '_' pooppp Programming 1 07-13-2012 02:03 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 02:54 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration