LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   General (https://www.linuxquestions.org/questions/general-10/)
-   -   Best file hosting service for automatic upload + direct download? (https://www.linuxquestions.org/questions/general-10/best-file-hosting-service-for-automatic-upload-direct-download-4175500434/)

Vinter 04-03-2014 06:46 AM

Best file hosting service for automatic upload + direct download?
 
Hi!

Currently, I'm developing an application that would allow people to subscribe to shows from German and French television, similar to mediathek3, but optimized for cron and with smaller bandwidth usage by providing diffs. (And written in Python. Fuck Java. Sorry.) In order to distribute the show listings, I'm looking for a service that lets me

a) upload any changes automatically,

b) provide direct links to users, and

c) (ideally) check HTTP headers etc. like if the files were hosted on a standard web server.

I've looked around a bit already, but the available options are confusingly many. So, which service would you recommend? Free would of course be preferred if it's >95% reliable, but a relatively minor charge would also be acceptable. To be as clear as possible, the storage and distribution work as follows (right now, could be adapted a bit):

- New users or those with corrupted files have to download ~2 MByte max per station, once
- Seven-day history of diffs is kept on server (at the moment - ~20-100k each day, each station) so people can update whole days from outdated data
- About ten times a day, a "today" diff is generated; if the program is run though cron, every user will download all of those.
- Naturally, I want all of these changes published (i. e. uploaded and downloadable) without human interaction, i. e. a scriptable upload API and direct file access of some kind is imperative.
- Depending on the way the links are generated, I'd maybe also need some kind of storage that would be accessible via a static link, so that people could download the currently available links to diffs, maybe modification times etc.

So typical usage would be like 500k every two hours per user, plus a few outliers if people subscribe to new stations and have to download whole databases. All in all, laughable storage requirements and rather small bandwidth requirements - all I need is the direct access, really. Because the files only change every two hours or so, it's not even a really problem if the speed limit is at a few kBps.

I hope there's something out there for these needs - all I can find are huge storage spaces with rather complicated sharing APIs, or web hosting services that focus on actual websites which I don't need at all.

For reference, the script is available here: https://github.com/Pikaro/mediathek-cli
DO note that this is alpha software and still uses my tiny personal VPS as a data source, so please be careful with it! However, I'm very happy about any kind of comment, this is my first software project that I think has some potential to be useful to the public.

Thanks for any recommendations in advance!
Regards,
V

cin_ 04-29-2014 08:50 AM

tarsnap
 
have you seen or heard of tarsnap?

http://www.tarsnap.com/

dugan 05-19-2014 11:08 AM

Any hosting service that supports Python would work for you. This is a good place to start shopping: http://djangofriendly.com/hosts/

Parshan 05-19-2023 12:22 AM

Quote:

Originally Posted by cin_ (Post 5161229)
have you seen or heard of tarsnap?

http://www.tarsnap.com/

No, Sir. May you share some of your reviews or thoughts on it? I read about it after reading your post and get some idea about it from others post. But i would like to hear your thoughts on it. Are you using it now?


All times are GMT -5. The time now is 07:25 PM.