LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 03-16-2013, 01:13 PM   #1
CaptainDerp
LQ Newbie
 
Registered: Mar 2013
Posts: 12

Rep: Reputation: Disabled
Question How to remove all objects BEFORE first whitespace using sed or awk?


How to remove all objects BEFORE first white space using sed or awk?

source sample

derf3eef3fd34rf/rf3/4f34f/4/stuffudontwant stuffIwant.here

desired output

stuffiwant.here


I also need a script or one liner that will remove a list of objects from a text file,that is provided from a list in a seperate text file.

Last edited by CaptainDerp; 03-16-2013 at 01:19 PM.
 
Old 03-16-2013, 01:26 PM   #2
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Ubuntu
Posts: 1,099

Rep: Reputation: 288Reputation: 288Reputation: 288
Quote:
Originally Posted by CaptainDerp View Post
How to remove all objects BEFORE first white space using sed or awk?

source sample

derf3eef3fd34rf/rf3/4f34f/4/stuffudontwant stuffIwant.here

desired output

stuffiwant.here
cut will do this job. This code ...

Code:
echo derf3eef3fd34rf/rf3/4f34f/4/stuffudontwant stuffIwant.here
| cut -d ' ' -f2
... produced this result ...
Code:
stuffIwant.here
Daniel B. Martin
 
Old 03-16-2013, 01:28 PM   #3
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Ubuntu
Posts: 1,099

Rep: Reputation: 288Reputation: 288Reputation: 288
Quote:
Originally Posted by CaptainDerp View Post
I also need a script or one liner that will remove a list of objects from a text file,that is provided from a list in a separate text file.
Help us to help you. Provide a sample input file (10-15 lines will do). Construct a sample output file which corresponds to your sample input and post both samples here. With "Before and After" examples we can better understand your needs and also judge if our proposed solution fills those needs.

Daniel B. Martin
 
Old 03-16-2013, 03:22 PM   #4
CaptainDerp
LQ Newbie
 
Registered: Mar 2013
Posts: 12

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by danielbmartin View Post
Help us to help you. Provide a sample input file (10-15 lines will do). Construct a sample output file which corresponds to your sample input and post both samples here. With "Before and After" examples we can better understand your needs and also judge if our proposed solution fills those needs.

Daniel B. Martin

Sure, I have a large list1 of line by line items, and I have another list2 of items that are in that large list1, and I needed to remove every line from large list1 contained in list2


Large list 1

site.com
url.com
stuff.com
porno.com
msn.com
rap.com
crap.com
aol.com
morecrap.com
shizzo.com
meatspin.com
blah.com
microsoft.com
eatpussy.com

And list2 contains a list of items to be removed from large list1

site.com
url.com
stuff.com
porno.com
rap.com
crap.com
morecrap.com
shizzo.com
meatspin.com

In this example the desired output should be

aol.com
msn.com
microsoft.com

Last edited by CaptainDerp; 03-16-2013 at 03:24 PM.
 
Old 03-16-2013, 05:29 PM   #5
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Ubuntu
Posts: 1,099

Rep: Reputation: 288Reputation: 288Reputation: 288
Quote:
Originally Posted by CaptainDerp View Post
Sure, I have a large list1 of line by line items, and I have another list2 of items that are in that large list1, and I needed to remove every line from large list1 contained in list2.
I'm away from home now and have no access to a Linux machine...

I think the comm command will do the job, after sorting both files. Check into it.

Daniel B. Martin
 
Old 03-16-2013, 09:20 PM   #6
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian
Posts: 2,455

Rep: Reputation: 843Reputation: 843Reputation: 843Reputation: 843Reputation: 843Reputation: 843Reputation: 843
Quote:
Originally Posted by CaptainDerp View Post
Sure, I have a large list1 of line by line items, and I have another list2 of items that are in that large list1, and I needed to remove every line from large list1 contained in list2
I would suggest grep for this:
Code:
grep -Fvf remove-these.txt items.txt > remaining-items.txt
 
1 members found this post helpful.
Old 03-17-2013, 11:15 AM   #7
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Ubuntu
Posts: 1,099

Rep: Reputation: 288Reputation: 288Reputation: 288
Quote:
Originally Posted by ntubski View Post
Code:
grep -Fvf remove-these.txt items.txt > remaining-items.txt
Short and sweet! Love it!

Daniel B. Martin
 
Old 03-24-2013, 08:21 PM   #8
CaptainDerp
LQ Newbie
 
Registered: Mar 2013
Posts: 12

Original Poster
Rep: Reputation: Disabled
:(

Quote:
Originally Posted by danielbmartin View Post
cut will do this job. This code ...

Code:
echo derf3eef3fd34rf/rf3/4f34f/4/stuffudontwant stuffIwant.here
| cut -d ' ' -f2
... produced this result ...
Code:
stuffIwant.here
Daniel B. Martin

This isnt going to work for me because I have a huge list of lines like this.
 
Old 03-24-2013, 09:28 PM   #9
ntubski
Senior Member
 
Registered: Nov 2005
Distribution: Debian
Posts: 2,455

Rep: Reputation: 843Reputation: 843Reputation: 843Reputation: 843Reputation: 843Reputation: 843Reputation: 843
Quote:
Originally Posted by CaptainDerp View Post
This isnt going to work for me because I have a huge list of lines like this.
How is that a problem?
 
Old 03-24-2013, 09:53 PM   #10
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Ubuntu
Posts: 1,099

Rep: Reputation: 288Reputation: 288Reputation: 288
Quote:
Originally Posted by CaptainDerp View Post
This isnt going to work for me because I have a huge list of lines like this.
Perhaps you want ...
Code:
cut -d ' ' -f2 $InFile > $OutFile
Your script would specify the complete file identifiers for InFile and OutFile.
In my programs they are usually named thusly:
Code:
# File identification
   Path=$(readlink -f $0 | cut -d'.' -f1)
OutFile=$Path"out.txt"
 InFile=$Path"inp.txt"
This assumes you will have all three files (the script, InFile, and OutFile) in the same folder.

Daniel B. Martin
 
  


Reply

Tags
awk, grep, sed


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Remove All Objects after the first whitespace on each line using Sed? Help Please. CaptainDerp Programming 7 03-05-2013 08:04 AM
SED or AWK - remove every 4 of 5 new lines Mallardle Linux - Newbie 6 08-30-2010 07:44 AM
Remove everything up to the last numbers of a string w/ sed or awk OutThere Linux - General 4 04-23-2009 07:01 PM
How to remove everything before the first space in Sed or Awk OutThere Linux - General 1 04-05-2009 10:45 PM
Using sed in bash to remove whitespace jimieee Programming 3 01-28-2004 10:33 AM


All times are GMT -5. The time now is 11:15 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration