LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 05-02-2009, 07:32 PM   #1
RagnaKaos
LQ Newbie
 
Registered: May 2009
Posts: 2

Rep: Reputation: 0
Smart mirror for distribution repositories


Hi,
I've been googling a lot to try and find some piece of software to allow mirroring a directory on an HTTP or FTP server without having to keep it up to date manually or by cron jobs and keeping it from downloading files that are never used on the mirror.

The basic idea is that it acts a sort of proxy between a local system and a software repository, for example the Ubuntu apt sources, which are over 200 GB large. The software would, upon a regular request, begin downloading the file and save it locally while also redirecting it to the requesting system. If another system then requests the same file, the software would then be able to deliver it directly from its local cache.

I know this sounds very similar to a normal caching proxy, but they usually only store small files and it would require a different setup than just setting the mirror site to the local server, which would allow a central setup of the best mirror.

I really hope I've explained this well enough, as I haven't been able to find anything through Google, which may be due to me not using the right terms for the problem.

So my question is: has anyone heard of any application like this?

Thanks,
- Marcus
 
Old 05-02-2009, 10:21 PM   #2
lazlow
Senior Member
 
Registered: Jan 2006
Posts: 4,363

Rep: Reputation: 172Reputation: 172
If you are using a RH based system you can use the createrepo package, set yum priorities for the local repo to be first, and set yum so it downloads to your local repo directory.
 
Old 05-03-2009, 03:39 AM   #3
RagnaKaos
LQ Newbie
 
Registered: May 2009
Posts: 2

Original Poster
Rep: Reputation: 0
Well, I was looking for a general solution, so that I would be able to implement it in different scenarios.
But I do have one scenario, which is the most important one:

An Arch Linux server is supposed to run a mirror for a bunch of Ubuntu hosts that are wasting a lot of bandwidth on downloading the same packages over and over again. But maintaining a full Ubuntu mirror with its continually expanding repositories of more than 200 GB also seems like wasting bandwidth, as we only use a tiny fraction of those packages.

- Marcus
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Mirror repositories Red Squirrel Linux - Server 1 01-03-2009 09:19 PM
LXer: Burning Debian packages and repositories to disc with APTonCD and apt-mirror LXer Syndicated Linux News 0 07-03-2007 06:01 PM
smart repositories? alaios SUSE / openSUSE 2 11-16-2006 05:23 AM
any distribution with smart battery support? Shioni Linux - General 0 06-14-2006 02:38 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 07:52 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration