LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
LinkBack Search this Thread
Old 05-15-2012, 12:45 AM   #1
memilanuk
Member
 
Registered: Sep 2010
Location: Washington state, USA
Distribution: Ubuntu among others
Posts: 66

Rep: Reputation: 2
Can squid be used to cache OS updates/upgrades?


So... I have a virtual LAN running in Virtualbox (currently on Ubuntu 11.10) where I frequently install, update, tinker with, blow away and then reinstall various distros. In the interest of not having to re-download a lot of the same files over and over again, I'm looking at setting up squid on the 'gateway' VM server. Actually, it's already installed, using squid-deb-proxy which acts as a caching proxy for Ubuntu updates and such. That particular variant uses its own config, log, and cache files separate from the main squid program, and is pretty much zero config out of the box (very slick).

I have some questions, though, as I had normally thought of squid mainly as something for accelerating (or blocking) web page viewing, not speeding up file downloads. Please bear with me as I ask some potentially silly questions...

If I have other VMs 'behind' this gateway server with squid, and say box A has CentOS 6.2 on it and downloads various RPMs for an update, will those be cached by squid? If I box B also has CentOS 6.2 on it and requests the same updates later, will it get them from the local cache or from the website? (think I know the answer to this one, but figured I'd ask anyways)

Same scenario, but box A and B have different versions of the same OS... CentOS 5.8 and 6.2... will box B get its updates from the local cache, or from the upstream site?

Same scenario, but box A and B have the same version of very *similar* OSes... like CentOS 6.2 and Scientific Linux 6.2. What happens?

Similar scenario, but box A and B have similar versions of the same file/package from related but different distros - say Ubuntu & Mint, or Debian and Crunchbang. What happens?

If I'm understanding things right, the only time I'd actually *want* the client VMs to share the files from the cache would be in the case of where there are multiple installs of the same OS i.e. both the same 'brand' (CentOS) and the same version number (6.2). In all the others, the machines should download their own file/rpm, and any subsequent request for the exact same file from the same place would then use the cached version. Am I understanding this correctly?

Do I need to do anything special - besides setting up a pretty big cache, with long retention time, to make this happen?

Anything else I'm overlooking?

TIA,

Monte
 
Old 05-15-2012, 02:28 AM   #2
UndiFineD
LQ Newbie
 
Registered: Jun 2004
Location: Netherlands
Distribution: adistro
Posts: 24

Rep: Reputation: 0
squid does cache any files you retrieve from the internet
another method you may like for multiple machines is this
http://apt-mirror.sourceforge.net/
 
Old 05-15-2012, 06:15 AM   #3
devilboy09
Member
 
Registered: Nov 2011
Location: Iran
Distribution: Debian, CentOS, LFS
Posts: 351

Rep: Reputation: 7
i don't think so.
 
Old 05-15-2012, 09:09 AM   #4
memilanuk
Member
 
Registered: Sep 2010
Location: Washington state, USA
Distribution: Ubuntu among others
Posts: 66

Original Poster
Rep: Reputation: 2
UndiFineD,

Already got squid-deb-proxy working, which is pretty slick/amazing for a 'zeroconf' system. Nothing to setup, beyond installing the client software - though I believe that just uses avahi to automatically find the right proxy in the background and a person *could* manually specify the same info as far as host ip and port number if they wanted. I probably *will* set up a small mirror at some point in the future... 'just because I can'

devilboy09,

Could you be a little more vague? You don't think so... what?
 
Old 05-15-2012, 09:23 AM   #5
TobiSGD
Moderator
 
Registered: Dec 2009
Location: Hanover, Germany
Distribution: Slackware
Posts: 14,868
Blog Entries: 2

Rep: Reputation: 3804Reputation: 3804Reputation: 3804Reputation: 3804Reputation: 3804Reputation: 3804Reputation: 3804Reputation: 3804Reputation: 3804Reputation: 3804Reputation: 3804
I normally use apt-cacher-ng for those tasks, easy and fast.
 
Old 05-15-2012, 09:27 AM   #6
memilanuk
Member
 
Registered: Sep 2010
Location: Washington state, USA
Distribution: Ubuntu among others
Posts: 66

Original Poster
Rep: Reputation: 2
Again... while I appreciate the debian/ubuntu related suggestions... please bear in mind I'm looking for something to handle multiple *non Debian* distros as well. I've got the deb angle covered, thanks. Please stick to the topic at hand - squid.
 
Old 05-15-2012, 01:01 PM   #7
memilanuk
Member
 
Registered: Sep 2010
Location: Washington state, USA
Distribution: Ubuntu among others
Posts: 66

Original Poster
Rep: Reputation: 2
Okay... looks like the RH-based distros (CentOS, SL, Fedora) *can* update thru a proxy by simply adding a line or two in /etc/yum.conf... now it just looks like my squid config (stock, out of the box) needs some work to make everything work.
 
Old 05-16-2012, 11:33 PM   #8
memilanuk
Member
 
Registered: Sep 2010
Location: Washington state, USA
Distribution: Ubuntu among others
Posts: 66

Original Poster
Rep: Reputation: 2
Well... turned out the stock setup for squid3, at least on Ubuntu 12.04 LTS, doesn't have any ACLs defined for the local LAN. Once I did that by adding the lines:

Code:
acl lan src 192.168.1.0/24

http_access allow lan
It also doesn't setup a disk-based cache either, for some reason. So I had to add a line like this:

Code:
cache_dir ufs /var/spool/squid3 1000 16 256
and added this line to /etc/yum.conf immediately after [MAIN]:

Code:
prox=http://192.168.1.1:3128
Then the RH-based distros started happily using the squid caching proxy server. There's probably some more fine-tuning that could be done, judging by the config file for squid-deb-proxy, but it got things working for now.
 
Old 02-15-2013, 04:08 PM   #9
workent
LQ Newbie
 
Registered: May 2012
Posts: 6

Rep: Reputation: Disabled
Optimal Squid repo

Hi guys,

[Been a while since last post]
[Similar query - please excuse he cross-post from another forum]

I've been using apt-cacher-ng quite successfully for a number of years now, as I have a number of Ubuntu & Debian (*.deb) systems on my network - baremetal & VM's.

So far, so good, but it fails to address the RedHat (*.rpm @ yum) & other distro's quite as well.

I've now provisioned a VM running Squid3 & it seems to be holding pretty well so far, based largely on the work found here & here.

What I'd really like to do is not filter/cache *all* my traffic, but dedicate the instance to only do so for POSIX packages & images (.deb, .rpm, .img, .iso, etc), and in doing so also sort then into some appropriate mounted NFS shares & to store those files indefinately, unless there are newer versions of same files: ie. ISO @ nas:/mnt/iso , DEB @ nas:/mnt/apt , RPM @ nas:/mnt/yum

What's nice re apt-cacher-ng, is that it'll store (& expire) packages that have passed through it indefinitely, & has some very handy tools to check if those files are still cool, and if not, allows me to purge unreferenced, damaged or bogus .deb's

Could anyone with ninja squid-fu skills please offer some advice on how to achieve such an outcome?
 
  


Reply

Tags
cache, squid, update


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Using Active directory Authentication of parent non-squid cache to child squid cache espiya7 Linux - Server 0 05-05-2009 08:04 AM
Security Upgrades, fc-cache Problems tronayne Slackware 2 04-20-2007 09:03 AM
Is /home ever touched by upgrades or updates? Haz2a Linux - General 5 10-18-2006 09:44 AM
How to cache windows updates using Squid ngwasuma Linux - Networking 7 06-21-2006 05:44 AM
Managing security updates and upgrades arobinson74 Slackware 8 10-19-2004 07:16 AM


All times are GMT -5. The time now is 11:06 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration