LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 02-10-2013, 07:57 PM   #1
bennetfox
LQ Newbie
 
Registered: Apr 2011
Posts: 13

Rep: Reputation: 0
Building a Debian Wheezy/Ubuntu Precise Mirror using apt-mirror


Hello everyone!

I'm trying to build a mirror of Debian Wheezy 7.0 and Ubuntu Precise 12.04.1 on my local area network and I am using apt-mirror to do all of the downloading. I've done the mandatory reading and research, but I'm having a problem with the configuration file of apt-mirror. when I run an apt-get update on the local client, apt-get responds with:

Code:
$ sudo apt-get update

(...cut for length...)

W: Failed to fetch http://greenfox.foxden.lan/debian/dists/wheezy/updates/main/source/Sources  404  Not Found

W: Failed to fetch http://greenfox.foxden.lan/debian/dists/wheezy/updates/contrib/source/Sources  404  Not Found

W: Failed to fetch http://greenfox.foxden.lan/debian/dists/wheezy/updates/non-free/source/Sources  404  Not Found

W: Failed to fetch http://greenfox.foxden.lan/debian/dists/wheezy/updates/main/binary-i386/Packages  404  Not Found

W: Failed to fetch http://greenfox.foxden.lan/debian/dists/wheezy/updates/contrib/binary-i386/Packages  404  Not Found

W: Failed to fetch http://greenfox.foxden.lan/debian/dists/wheezy/updates/non-free/binary-i386/Packages  404  Not Found

W: Failed to fetch http://greenfox.foxden.lan/debian/dists/wheezy/contrib/i18n/Translation-en  404  Not Found

W: Failed to fetch http://greenfox.foxden.lan/debian/dists/wheezy/main/i18n/Translation-en  404  Not Found

W: Failed to fetch http://greenfox.foxden.lan/debian/dists/wheezy/non-free/i18n/Translation-en  404  Not Found

E: Some index files failed to download. They have been ignored, or old ones used instead.
This is my /etc/apt/mirror.list file:

Code:
$ /etc/apt/mirror.list

$ cat /etc/apt/mirror.list
############# config ##################

set base_path    /home/apt-mirror
set mirror_path  $base_path/mirror
set skel_path    $base_path/skel
set var_path     $base_path/var
set cleanscript $var_path/clean.sh
set defaultarch  i386
set postmirror_script $var_path/postmirror.sh
set run_postmirror 0
set nthreads     20
set _tilde 0

############# end config ##############

############# Debian 7.0 Wheezy ################

deb http://debian.osuosl.org/debian/ wheezy main contrib non-free
deb-src http://debian.osuosl.org/debian/ wheezy main contrib non-free
deb http://security.debian.org/ wheezy/updates main contrib non-free
deb-src http://security.debian.org/ wheezy/updates main contrib non-free
deb http://debian.osuosl.org/debian/ wheezy-updates contrib non-free main
This is the sources.list file I am using on the local machine:

Code:
$ cat /etc/apt/sources.list
deb http://greenfox.foxden.lan/debian/ wheezy main contrib non-free
deb-src http://greenfox.foxden.lan/debian/ wheezy main contrib non-free

deb http://greenfox.foxden.lan/debian/ wheezy/updates main contrib non-free
deb-src http://greenfox.foxden.lan/debian/ wheezy/updates main contrib non-free
deb http://greenfox.foxden.lan/debian/ wheezy-updates contrib non-free main
Did I overlook something in my configurations or do I have a typo somewhere that I'm just not seeing?

Any help would be greatly appreciated!

Last edited by bennetfox; 02-10-2013 at 07:59 PM.
 
Old 02-11-2013, 02:19 PM   #2
lykwydchykyn
Member
 
Registered: Mar 2006
Location: Tennessee, USA
Distribution: Debian, Ubuntu
Posts: 135

Rep: Reputation: 36
Not sure about all of the problem, but the translation files do not automatically download; this is a known bug in apt-mirror.

I added the following to my postmirror script:
Code:
cd /srv/apt-mirror/mirror/ftp.us.debian.org/debian/dists/wheezy

for directory in contrib main non-free; do                                                                                                                     
    cd $directory                                                                                                                                              
    mkdir i18n 2>/dev/null                                                                                                                                     
    cd i18n                                                                                                                                                    
    rm Translation-en*                                                                                                                                         
    wget http://ftp.us.debian.org/debian/dists/wheezy/$directory/i18n/Translation-en.bz2                                                                       
    cd ../../                                                                                                                                                  
done
You'll want to adapt it to your install, of course (my stuff is in a different directory than yours). Not sure about the other errors since I don't mirror sources. I assume the apt-mirror script has run at least once?
 
Old 02-11-2013, 05:13 PM   #3
bennetfox
LQ Newbie
 
Registered: Apr 2011
Posts: 13

Original Poster
Rep: Reputation: 0
It's funny you say that because yesterday in amongst my Googling I ran across a page that talked about my error and how the author of apt-mirror has been informed of it, but hasn't fixed the issue yet. I saw a few solutions on how to fix the issue by inserting code into the source but that's just a bit past my level of expertise (I'm not an uber-techie-geek yet, I'm still learning! ). Your solution is something I can implement and I do greatly appreciate it!

Since apt-mirror isn't functioning fully to my expectations, I might take a look at the ftp scripts using rsync to keep my local mirrors updated instead.

Thank you again for your help!
 
Old 02-15-2013, 07:19 PM   #4
bennetfox
LQ Newbie
 
Registered: Apr 2011
Posts: 13

Original Poster
Rep: Reputation: 0
I'm still having troubles mirroring the wheezy/updates locally. Every time I run apt-get update on my local debian machines, I'm presented with this:

Code:
Err http://greenfox wheezy/updates/main Sources
  404  Not Found
Err http://greenfox wheezy/updates/contrib Sources
  404  Not Found
Err http://greenfox wheezy/updates/non-free Sources
  404  Not Found
Err http://greenfox wheezy/updates/main i386 Packages
  404  Not Found
Err http://greenfox wheezy/updates/contrib i386 Packages
  404  Not Found
Err http://greenfox wheezy/updates/non-free i386 Packages
  404  Not Found
Any help would be greatly appreciated!
 
Old 02-15-2013, 08:15 PM   #5
lykwydchykyn
Member
 
Registered: Mar 2006
Location: Tennessee, USA
Distribution: Debian, Ubuntu
Posts: 135

Rep: Reputation: 36
Maybe the mirror itself is having trouble synchronizing?

Have you tried running:
Code:
su apt-mirror -c apt-mirror
As root in a terminal to see if everything mirrors without error?

Also, can you post the apache configuration for the web server?

Last edited by lykwydchykyn; 02-15-2013 at 08:18 PM.
 
Old 02-16-2013, 03:59 PM   #6
bennetfox
LQ Newbie
 
Registered: Apr 2011
Posts: 13

Original Poster
Rep: Reputation: 0
Hmm... Now this is interesting. I tried the command you suggested and this is the response I get:

Code:
# su - apt-mirror -c apt-mirror
Downloading 195 index files using 20 threads...
apt-mirror: can't write to intermediate file (index-urls.0) at /usr/bin/apt-mirror line 210.
When I normally run apt-mirror, I usually do it from the command line like this:

Code:
$ sudo apt-mirror
Hopefully soon we can get to the bottom of this! Any help would be appreciated!
 
Old 02-16-2013, 07:55 PM   #7
lykwydchykyn
Member
 
Registered: Mar 2006
Location: Tennessee, USA
Distribution: Debian, Ubuntu
Posts: 135

Rep: Reputation: 36
If you run:
Code:
sudo apt-mirror
The apt-mirror runs as root and all the files will be owned by root. The cron job it ships with runs as "apt-mirror", so it won't update properly because "apt-mirror" (the user) can't overwrite root's files.

You probably need to recursively chown all the files in the apt-mirror repository to apt-mirror, then try the su version of the command again.

can you list the contents of your /home/apt-mirror/mirrors directory?
 
Old 02-17-2013, 04:06 PM   #8
bennetfox
LQ Newbie
 
Registered: Apr 2011
Posts: 13

Original Poster
Rep: Reputation: 0
Directory listing of /home/apt-mirror:

Code:
:/home/apt-mirror/mirror$ ls -la
total 36
drwxr-xr-x 9 apt-mirror apt-mirror 4096 Feb 10 15:21 .
drwxr-xr-x 5 apt-mirror apt-mirror 4096 Feb  6 18:23 ..
drwxr-xr-x 3 root       root       4096 Feb  9 14:35 archive.canonical.com
drwxr-xr-x 3 root       root       4096 Feb  6 18:29 debian.osuosl.org
drwxr-xr-x 3 root       root       4096 Feb  9 15:27 dl.google.com
drwxr-xr-x 3 root       root       4096 Feb  9 15:27 download.virtualbox.org
drwxr-xr-x 3 root       root       4096 Feb  7 09:20 extras.ubuntu.com
drwxr-xr-x 3 root       root       4096 Feb 10 15:21 security.debian.org
drwxr-xr-x 3 root       root       4096 Feb  6 18:29 ubuntu.osuosl.org
I also changed the ownerships to reflect the apt-mirror user:

Code:
$ sudo chown -R apt:mirror:apt-mirror apt-mirror/
Now my apt-mirror/mirrors directory looks like this:

Code:
$ ls -la
total 36
drwxr-xr-x 9 apt-mirror apt-mirror 4096 Feb 10 15:21 .
drwxr-xr-x 5 apt-mirror apt-mirror 4096 Feb  6 18:23 ..
drwxr-xr-x 3 apt-mirror apt-mirror 4096 Feb  9 14:35 archive.canonical.com
drwxr-xr-x 3 apt-mirror apt-mirror 4096 Feb  6 18:29 debian.osuosl.org
drwxr-xr-x 3 apt-mirror apt-mirror 4096 Feb  9 15:27 dl.google.com
drwxr-xr-x 3 apt-mirror apt-mirror 4096 Feb  9 15:27 download.virtualbox.org
drwxr-xr-x 3 apt-mirror apt-mirror 4096 Feb  7 09:20 extras.ubuntu.com
drwxr-xr-x 3 apt-mirror apt-mirror 4096 Feb 10 15:21 security.debian.org
drwxr-xr-x 3 apt-mirror apt-mirror 4096 Feb  6 18:29 ubuntu.osuosl.org
I just ran apt-mirror with the "su apt-mirror -c apt-mirror" command and everything completed successfully.

However I am still having the same error about apt-get update not being able to find the wheezy/updates repositories.

Code:
Err http://greenfox wheezy/updates/main Sources        
  404  Not Found
Err http://greenfox wheezy/updates/contrib Sources
  404  Not Found
Err http://greenfox wheezy/updates/non-free Sources
  404  Not Found
Err http://greenfox wheezy/updates/main i386 Packages
  404  Not Found
Err http://greenfox wheezy/updates/contrib i386 Packages
  404  Not Found
Err http://greenfox wheezy/updates/non-free i386 Packages
  404  Not Found

Last edited by bennetfox; 02-17-2013 at 04:08 PM.
 
Old 02-18-2013, 01:09 AM   #9
lykwydchykyn
Member
 
Registered: Mar 2006
Location: Tennessee, USA
Distribution: Debian, Ubuntu
Posts: 135

Rep: Reputation: 36
Ok, I think I see the problem here.

"wheezy/updates" comes from a separate server than the other wheezy packages (security.debian.org). Because of this, it's in a different folder under the "mirrors" folder.

I don't know how you are sharing these out with your webserver, if it's directly or with aliases or symlinks or what, but the (webroot)/debian folder cannot point to *both* the main debian repository directory and the security server directory.

You need to make the security.debian.org folder available in it's own path on your web server, then adjust your client's sources.list file accordingly.

I don't know if I explained that clearly, let me know.
 
Old 02-18-2013, 04:52 PM   #10
bennetfox
LQ Newbie
 
Registered: Apr 2011
Posts: 13

Original Poster
Rep: Reputation: 0
THANK YOU for explaining that to me! After reading your message, I looked at my /etc/apt/sources.list and noticed the slight differences. I've got the mirror directories linked to the /var/www directory so it's all accessable via http. I dug around a little bit in the directory structure and saw the different directories and I created a new symlink and the problem was solved!

Code:
$ sudo ln -s /home/apt-mirror/mirror/security.debian.org/ /var/www/security
apt-get update and apt-get upgrade went off without a hitch!

Now my next question is this: how do I add the debian/netinst mirror to it so I can mirror the network installation of Debian and Ubuntu? I would like to have the ability to install and upgrade a new installation of Debian/Ubuntu using local mirror resources instead of downloading software or updates from the Internets.
 
Old 02-18-2013, 11:24 PM   #11
lykwydchykyn
Member
 
Registered: Mar 2006
Location: Tennessee, USA
Distribution: Debian, Ubuntu
Posts: 135

Rep: Reputation: 36
Just add:
Code:
deb http://ftp.us.debian.org/debian/ wheezy main/debian-installer
To your sources. I have this on a server at work along with the netboot installer, works great. You have to manually keep the netboot installer up-to-date whenever there's a new point release, or else you get an error during install. Otherwise it works great.
 
Old 02-19-2013, 05:05 PM   #12
bennetfox
LQ Newbie
 
Registered: Apr 2011
Posts: 13

Original Poster
Rep: Reputation: 0
Added the aforementioned line to my mirrors.list, resynched, and tried a network install and it went off without any issues! Thank you so much for your help!

What would I add to the mirrir.list file to accomplish the same thing with a Ubuntu network installation? I've been googling around all morning and haven't been able to find my answer!
 
Old 02-20-2013, 11:32 AM   #13
lykwydchykyn
Member
 
Registered: Mar 2006
Location: Tennessee, USA
Distribution: Debian, Ubuntu
Posts: 135

Rep: Reputation: 36
Here's what I've got on my mirror for Precise:
Code:
#Precise32
deb http://us.archive.ubuntu.com/ubuntu/ precise main restricted universe multiverse
deb http://us.archive.ubuntu.com/ubuntu/ precise-updates main restricted universe multiverse
deb http://us.archive.ubuntu.com/ubuntu/ precise-backports main restricted universe multiverse
deb http://us.archive.ubuntu.com/ubuntu/ precise-security main restricted universe multiverse
deb http://us.archive.ubuntu.com/ubuntu/ precise main/debian-installer 


#Precise64

deb-amd64 http://us.archive.ubuntu.com/ubuntu/ precise main restricted universe multiverse
deb-amd64 http://us.archive.ubuntu.com/ubuntu/ precise-updates main restricted universe multiverse
deb-amd64 http://us.archive.ubuntu.com/ubuntu/ precise-backports main restricted universe multiverse
deb-amd64 http://us.archive.ubuntu.com/ubuntu/ precise-security main restricted universe multiverse
deb-amd64 http://us.archive.ubuntu.com/ubuntu/ precise main/debian-installer
Keep in mind if you want to install 64-bit, multiarch is configured by default so it needs both the 64-bit and 32-bit repos. Kind of a pain, unfortunately.
 
Old 02-21-2013, 08:09 PM   #14
bennetfox
LQ Newbie
 
Registered: Apr 2011
Posts: 13

Original Poster
Rep: Reputation: 0
I haven't forgotten about you, I've just been swamped with getting apt-mirror to function correctly as well as working on other projects as well!

Thank you so much for the mirror.list file. I'm curious if you also have one for Debian?

Let me see if I got this right, you have to mirror both 1386 and 64 bit distributions even if you only use just one?
 
Old 02-22-2013, 04:56 PM   #15
lykwydchykyn
Member
 
Registered: Mar 2006
Location: Tennessee, USA
Distribution: Debian, Ubuntu
Posts: 135

Rep: Reputation: 36
Quote:
Originally Posted by bennetfox View Post
I haven't forgotten about you, I've just been swamped with getting apt-mirror to function correctly as well as working on other projects as well!

Thank you so much for the mirror.list file. I'm curious if you also have one for Debian?
Yeah, but I just mirror squeeze, and I'm pulling updates from the web:
Code:
#Debian Squeeze 
deb http://ftp.us.debian.org/debian/ squeeze main contrib non-free
deb http://ftp.us.debian.org/debian/ squeeze main/debian-installer 
deb http://ftp.us.debian.org/debian/ squeeze-updates main contrib non-free
Quote:
Let me see if I got this right, you have to mirror both 1386 and 64 bit distributions even if you only use just one?
If you are using 64 bit with multiarch enabled (I think that's default) then you need the 32bit repos or you'll get errors. If you are just running 32bit or if you turn off multiarch, you can just do whichever one.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Problems with Debootstrap and apt-mirror-based Debian mirror lil_drummaboy Debian 3 06-22-2010 07:12 PM
apt-mirror doesn't create appropriate mirror ??? abd_bela Debian 1 09-30-2009 05:23 PM
LXer: Create a Local Ubuntu Repository using Apt-Mirror and Apt-Cacher LXer Syndicated Linux News 0 08-04-2009 11:13 PM
LXer: netselect-apt - tip to find the fastest Debian mirror LXer Syndicated Linux News 0 05-23-2007 10:01 AM
LXer: How To Create A Local Debian/Ubuntu Mirror With apt-mirror LXer Syndicated Linux News 0 01-04-2007 06:33 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 08:14 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration