LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > General
User Name
Password
General This forum is for non-technical general discussion which can include both Linux and non-Linux topics. Have fun!

Notices

Reply
 
Search this Thread
Old 04-03-2014, 06:46 AM   #1
Vinter
Member
 
Registered: Feb 2005
Location: Germany
Distribution: Aptosid
Posts: 142

Rep: Reputation: 19
Best file hosting service for automatic upload + direct download?


Hi!

Currently, I'm developing an application that would allow people to subscribe to shows from German and French television, similar to mediathek3, but optimized for cron and with smaller bandwidth usage by providing diffs. (And written in Python. Fuck Java. Sorry.) In order to distribute the show listings, I'm looking for a service that lets me

a) upload any changes automatically,

b) provide direct links to users, and

c) (ideally) check HTTP headers etc. like if the files were hosted on a standard web server.

I've looked around a bit already, but the available options are confusingly many. So, which service would you recommend? Free would of course be preferred if it's >95% reliable, but a relatively minor charge would also be acceptable. To be as clear as possible, the storage and distribution work as follows (right now, could be adapted a bit):

- New users or those with corrupted files have to download ~2 MByte max per station, once
- Seven-day history of diffs is kept on server (at the moment - ~20-100k each day, each station) so people can update whole days from outdated data
- About ten times a day, a "today" diff is generated; if the program is run though cron, every user will download all of those.
- Naturally, I want all of these changes published (i. e. uploaded and downloadable) without human interaction, i. e. a scriptable upload API and direct file access of some kind is imperative.
- Depending on the way the links are generated, I'd maybe also need some kind of storage that would be accessible via a static link, so that people could download the currently available links to diffs, maybe modification times etc.

So typical usage would be like 500k every two hours per user, plus a few outliers if people subscribe to new stations and have to download whole databases. All in all, laughable storage requirements and rather small bandwidth requirements - all I need is the direct access, really. Because the files only change every two hours or so, it's not even a really problem if the speed limit is at a few kBps.

I hope there's something out there for these needs - all I can find are huge storage spaces with rather complicated sharing APIs, or web hosting services that focus on actual websites which I don't need at all.

For reference, the script is available here: https://github.com/Pikaro/mediathek-cli
DO note that this is alpha software and still uses my tiny personal VPS as a data source, so please be careful with it! However, I'm very happy about any kind of comment, this is my first software project that I think has some potential to be useful to the public.

Thanks for any recommendations in advance!
Regards,
V

Last edited by Vinter; 04-03-2014 at 06:48 AM.
 
Old 04-29-2014, 08:50 AM   #2
cin_
Member
 
Registered: Dec 2010
Posts: 264

Rep: Reputation: 23
tarsnap

have you seen or heard of tarsnap?

http://www.tarsnap.com/

Last edited by cin_; 04-29-2014 at 08:50 AM. Reason: gramm`err
 
Old 05-19-2014, 11:08 AM   #3
dugan
Senior Member
 
Registered: Nov 2003
Location: Canada
Distribution: distro hopper
Posts: 4,632

Rep: Reputation: 1429Reputation: 1429Reputation: 1429Reputation: 1429Reputation: 1429Reputation: 1429Reputation: 1429Reputation: 1429Reputation: 1429Reputation: 1429
Any hosting service that supports Python would work for you. This is a good place to start shopping: http://djangofriendly.com/hosts/

Last edited by dugan; 05-19-2014 at 11:22 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Does somebody know an internet free file upload service? stf92 General 3 09-14-2013 04:46 PM
user can upload and download file but not delete file in ftp on thel 6 kukusuku Linux - Server 1 08-21-2011 02:50 AM
How to upload file with tftp service? lzgonline Linux - Software 11 06-30-2011 09:27 PM
PHP file download without direct file access prasepretep Programming 2 02-14-2008 05:34 AM


All times are GMT -5. The time now is 05:56 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration