LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Ubuntu
User Name
Password
Ubuntu This forum is for the discussion of Ubuntu Linux.

Notices



Reply
 
Search this Thread
Old 08-13-2013, 08:56 AM   #1
herakles_14
Member
 
Registered: Sep 2010
Location: I currently reside in Glendale, AZ
Distribution: Manjaro & PCLinuxOS
Posts: 286

Rep: Reputation: 0
removing duplicate files


Hello to anyone out there;

I have a minor dilemma in that after updating I find I have some duplicates and would like to get rid of them. Here hey are:

Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ raring/main i386 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_main_binary-i386_Packages)
W: Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ raring/universe i386 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_universe_binary-i386_Packages)
W: Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ raring/multiverse i386 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_multiverse_binary-i386_Packages)
W: Duplicate sources.list entry http://us.archive.ubuntu.com/ubuntu/ raring/restricted i386 Packages (/var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_restricted_binary-i386_Packages)

Is there a simple comman I could use with the Terminal to eliminate the duplicates without causing any harm to the system?

herakles_14
 
Old 08-13-2013, 09:21 AM   #2
goumba
Member
 
Registered: Dec 2009
Location: New Jersey, USA
Distribution: Debian Wheezy (7.0), Jessie (future 8) and Sid (unstable)
Posts: 572
Blog Entries: 3

Rep: Reputation: 78
This means there are duplicate lines in your sources.list file.

Edit /etc/apt/sources.list (as root) and remove the duplicate lines, leaving only one.

You could also do sort -u, but it will mess up the order of lines in the file (and that doesn't really matter, except if you care, i.e. it is commented). Copy sources.list to another file - sort will overwrite the original and you'll end up with an empty file if you try to do use sources.list as a source and destination, then do (as root)

Code:
sort -u file > sources.list
 
Old 08-13-2013, 08:41 PM   #3
herakles_14
Member
 
Registered: Sep 2010
Location: I currently reside in Glendale, AZ
Distribution: Manjaro & PCLinuxOS
Posts: 286

Original Poster
Rep: Reputation: 0
goumba;

Unfortunately command line code: (sort -u file > sources.list) did not work. Basically I was informed that there was no such file or directory.

I followed the trail /var/lib/apt/lists an came across a long list, some of which had matches with the four duplicates. Here I came into a problem. How do I delete the duplicates?

On the Code I am going to enter it again but as:

sort -u file > sources.list.d

Then see if that makes any difference?
 
Old 08-13-2013, 08:44 PM   #4
herakles_14
Member
 
Registered: Sep 2010
Location: I currently reside in Glendale, AZ
Distribution: Manjaro & PCLinuxOS
Posts: 286

Original Poster
Rep: Reputation: 0
It did not make a difference.
 
Old 08-15-2013, 04:08 PM   #5
rsleventhal
Member
 
Registered: Apr 2007
Location: Sunny Florida, USA
Distribution: CentOS, RHEL, U/X/Kubuntu
Posts: 36

Rep: Reputation: 15
Quote:
Originally Posted by herakles_14 View Post
It did not make a difference.
I think we're having a communications and/or path problem

try this:

Code:
sudo su -
<enter your password when promted>

Code:
cd /etc/apt
Now, let's make a backup copy of your sources.list file just so we have a square one to go back to if we need to.
Code:
cp -p ./sources.list ./working.sources.list
OK..the suggestion earlier in this thread (by goumba) is a good one, but I think you got confused by his example. Here's a more explicit instruction set which should address your particular issue.

As goumba suggested, let's use the sort -u command (sort unique) to output to a new file:

Quote:
sort -u ./sources.list > ./sorted.sources.list
A quick check to see if the new file (sorted.sources.list) is smaller than the original (which it should be if we eliminated duplicate entries)

Code:
ls -lah |grep sources
This should output at least three lines, one each for sources.list, working.sources.list, and sorted.sources.list

IF sorted.sources.list is smaller in size than sources.list, we're probably OK to overwrite and try it (remembering that we have a backup copy of the original)

So...do this:
Code:
mv ./sorted.sources.list ./sources.list
Then try to do an
Code:
apt-get update
or

Code:
aptitude update
If that works without getting the errors you were getting before, you've done it.

Good luck!

-R

Last edited by rsleventhal; 08-15-2013 at 04:10 PM.
 
Old 08-16-2013, 08:47 PM   #6
herakles_14
Member
 
Registered: Sep 2010
Location: I currently reside in Glendale, AZ
Distribution: Manjaro & PCLinuxOS
Posts: 286

Original Poster
Rep: Reputation: 0
rsleventhal;

I do appreciate your suggestions in how I could alleviate my duplicate file problem. I was able to resolve the issue but not in the way I would have liked. Before seeing your post, I was trying all sorts if things one of which let me log in but could not go any further. I had to install Ubuntu 13.04 again from the get go. For obvious reasons that solved my duplicate file problem.
It may be a wise move if I were to follow your suggestion an make a make a back up copy of my Sources.list file just in case? What do you think?
 
Old 08-18-2013, 08:48 AM   #7
goumba
Member
 
Registered: Dec 2009
Location: New Jersey, USA
Distribution: Debian Wheezy (7.0), Jessie (future 8) and Sid (unstable)
Posts: 572
Blog Entries: 3

Rep: Reputation: 78
Quote:
Originally Posted by rsleventhal View Post
I think we're having a communications and/or path problem

OK..the suggestion earlier in this thread (by goumba) is a good one, but I think you got confused by his example. Here's a more explicit instruction set which should address your particular issue.
Yeah s/he made no mention as to being unfamiliar, so I took it for granted s/he had basic knowledge of what they were doing. Sorry herakles_14 for the confusion.
 
Old 08-18-2013, 04:52 PM   #8
herakles_14
Member
 
Registered: Sep 2010
Location: I currently reside in Glendale, AZ
Distribution: Manjaro & PCLinuxOS
Posts: 286

Original Poster
Rep: Reputation: 0
Wink

goumba;

That's okay. What would life be without some confusion.
 
Old 08-18-2013, 06:20 PM   #9
widget
Senior Member
 
Registered: Oct 2008
Location: S.E. Montana
Distribution: Debian Testing, Stable, Sid and Manjaro, Mageia 3, LMDE
Posts: 2,348

Rep: Reputation: 400Reputation: 400Reputation: 400Reputation: 400Reputation: 400
Quote:
Originally Posted by herakles_14 View Post
goumba;

That's okay. What would life be without some confusion.
One of the finest teachers I had said that confusion was the foundation of learning. I find that to be true for me. Hope it is for you.
 
Old 08-19-2013, 11:11 AM   #10
herakles_14
Member
 
Registered: Sep 2010
Location: I currently reside in Glendale, AZ
Distribution: Manjaro & PCLinuxOS
Posts: 286

Original Poster
Rep: Reputation: 0
widget,

That teacher of yours may have had a point?? Since switching from Windows to Linux some three years back I have often been confused but with perserverance and having places like this to ask questions I've been able to learn more an be less confused.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell script to concatinate two text files and removing the duplicate entries punky Programming 13 06-17-2012 09:30 PM
File Mgmt. Removing duplicate files. heimbichner Linux - General 3 12-29-2011 11:11 PM
does tar or bzip2 squash duplicate or near-duplicate files? garydale Linux - Software 6 11-19-2009 05:43 PM
removing duplicate entries shabev Linux - Enterprise 3 03-25-2008 11:36 AM
Removing Duplicate Hardware Suse 9.1 oldtrout Linux - Hardware 0 12-08-2004 03:58 PM


All times are GMT -5. The time now is 10:50 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration