LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-16-2006, 04:27 AM   #1
wgato
Member
 
Registered: Jun 2006
Distribution: ubuntu slax dsl
Posts: 40

Rep: Reputation: 15
command to compare lines in a file?


i want to go through a file, line by line to see if any lines are the same. what command does this?


i'm reading through diff and compare but they seem to be for comparing 2 different files.

thanks!
 
Old 12-16-2006, 06:16 AM   #2
makyo
Member
 
Registered: Aug 2006
Location: Saint Paul, MN, USA
Distribution: {Free,Open}BSD, CentOS, Debian, Fedora, Solaris, SuSE
Posts: 728

Rep: Reputation: 74
Hi.

Look at man uniq ... cheers, makyo
 
Old 12-16-2006, 06:41 AM   #3
titopoquito
Senior Member
 
Registered: Jul 2004
Location: Lower Rhine region, Germany
Distribution: Slackware 14.1 (32 and 64 bit)
Posts: 1,594

Rep: Reputation: 124Reputation: 124
You could use sort/uniq/diff to find those lines. There are probably simpler solutions

Code:
sort textfile.txt > sorted.txt
sort textfile.txt | uniq > sorted-and-uniq.txt
diff sorted.txt sorted-and-uniq.txt
will give you all lines that are duplicates.
 
Old 12-16-2006, 12:22 PM   #4
linuxone
LQ Newbie
 
Registered: May 2006
Posts: 15

Rep: Reputation: 0
read the man file on uniq, the -d option only prints duplicate lines.

sort textfile.txt|uniq -d

should get you what you want and it's a little more elegant.


Quote:
Originally Posted by titopoquito
You could use sort/uniq/diff to find those lines. There are probably simpler solutions

Code:
sort textfile.txt > sorted.txt
sort textfile.txt | uniq > sorted-and-uniq.txt
diff sorted.txt sorted-and-uniq.txt
will give you all lines that are duplicates.
 
Old 12-17-2006, 08:55 AM   #5
wgato
Member
 
Registered: Jun 2006
Distribution: ubuntu slax dsl
Posts: 40

Original Poster
Rep: Reputation: 15
thanks all!
i hadnt heard of uniq before and didnt find it with apropos.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Random file lines directed to a new file. In script an error. In command line no err leventis Programming 1 09-28-2006 07:16 AM
command to count how many lines in a file? dr_zayus69 Linux - Software 3 01-06-2005 10:42 AM
Using diff to compare file with common lines, but at different line numbers jimieee Linux - Newbie 3 05-10-2004 07:26 AM
use file data in command lines karlis Linux - General 3 08-29-2003 07:27 PM
file compare program Nyc0n Linux - General 4 08-18-2001 09:08 PM


All times are GMT -5. The time now is 01:41 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration