Download "real Time" Data -> Modifie it -> Upload to my Webhoster?!
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Download "real Time" Data -> Modifie it -> Upload to my Webhoster?!
Hi
I would say that my problem is very simple.
Now I work hard on my project to make an Webpage for Travellers who have no hidden Ads like the other well known.
Here the Gov offer me some "Open Data" who get updated frequently.
The major problem is I have to use the Data how the get Distributed that is in some case really weird.
Like Weather Data who get Distributed as CSV File.
The next problem like the Weather Data is there are different Source for them.
Depend on the District there are 3 Source for the Weather Data who are different in the way how the are build.
So what I am looking for is a way to run an Server on my ESXi who do there Job 24/7 unattended. The Data must be equalized and than uploadet into the DB from my Hoster to get Published on my Page.
The source can be a CSV File, XML or many other weird Formats.
Sadly the Page is just in German but here are all Open Data who I want use some of: https://www.data.gv.at/
Before someone ask why I want just the open Data that's simple because I can do with what every I want. I relay hate 3rd Party stuff who show Ads, Track User and other ssshhttt...!
Hi
I would say that my problem is very simple.
Now I work hard on my project to make an Webpage for Travellers who have no hidden Ads like the other well known. Here the Gov offer me some "Open Data" who get updated frequently. The major problem is I have to use the Data how the get Distributed that is in some case really weird. Like Weather Data who get Distributed as CSV File.
The next problem like the Weather Data is there are different Source for them. Depend on the District there are 3 Source for the Weather Data who are different in the way how the are build.
So what I am looking for is a way to run an Server on my ESXi who do there Job 24/7 unattended. The Data must be equalized and than uploadet into the DB from my Hoster to get Published on my Page. The source can be a CSV File, XML or many other weird Formats. Sadly the Page is just in German but here are all Open Data who I want use some of: Before someone ask why I want just the open Data that's simple because I can do with what every I want. I relay hate 3rd Party stuff who show Ads, Track User and other ssshhttt...!
So you're wanting a script of some type to load data into a database?? Shouldn't be a problem at all....so post what you've written/done/tried so far, and tell us where you're stuck. We're happy to help, but we will NOT write your scripts for you.
If you're looking for a starting point, you need to also provide details. Does that input data file have a way to be downloaded automatically, or are you going to do it manually? What format is that data in? What database are you using? How are you loading/doing this procedure now? Version/distro of Linux you're using? Language you want to write this in? As with your other thread, you need to provide details.
Sry i am in Rush and on my phone now.
I just install Suse on my Acer Laptop that's all I have no idea which language i should choose. If some explane bow to do maybe I can start and adjust myself.
I need to know where the limits are. What doesn't a Scripting language can do?
When I am back in the Hotel I will show the data. I think it's an ordinary SQL database.
My Hoster use cpanel.
Some people from the "how to build a Webpage" Community say I should show the table on the page... Omg how dies it look? Horrible... I will make an nice page for foreign Traveller with live data such as weather who look more professional.
There are other thinks why I want grabb like public Transport Dada when I get permission for but that is a different topic.
Again, doesn't matter if you are in a rush...DO NOT use text speak. Again, it's "sorry", not "sry"
Quote:
I just install Suse on my Acer Laptop that's all I have no idea which language i should choose. If some explane bow to do maybe I can start and adjust myself.
I need to know where the limits are. What doesn't a Scripting language can do?
Since you are the one writing these programs, it is you who needs to pick the language you're comfortable with. There are MANY, including scripting languages, perl, ruby, python, C, etc. The 'limits' are whatever you're willing to write.
Quote:
When I am back in the Hotel I will show the data. I think it's an ordinary SQL database.
The INPUT data that you're downloading from that page is an entire SQL database? Great....then you (again) need to tell us what kind of SQL database it is.
Quote:
My Hoster use cpanel.
...which has no bearing on anything here. If you want to load a database/run a program, we need to know the version/distro of Linux.
Quote:
Some people from the "how to build a Webpage" Community say I should show the table on the page... Omg how dies it look? Horrible... I will make an nice page for foreign Traveller with live data such as weather who look more professional. There are other thinks why I want grabb like public Transport Dada when I get permission for but that is a different topic.
Great...so again, can you show us ANY of your efforts to date?
Again, doesn't matter if you are in a rush...DO NOT use text speak. Again, it's "sorry", not "sry"
Since you are the one writing these programs, it is you who needs to pick the language you're comfortable with. There are MANY, including scripting languages, perl, ruby, python, C, etc. The 'limits' are whatever you're willing to write.
The INPUT data that you're downloading from that page is an entire SQL database? Great....then you (again) need to tell us what kind of SQL database it is.
...which has no bearing on anything here. If you want to load a database/run a program, we need to know the version/distro of Linux.
Great...so again, can you show us ANY of your efforts to date?
So you are downloading the data manually. That's fine, but you should investigate using wget or curl to try to automate this. Read the man pages on those commands.
Quote:
for example this 3 Data should be connected together and than equalised to have an DB where all available Data are in one DB.
Right...again we UNDERSTAND WHAT YOU WANT. But again, we are not seeing anything that you have done/tried to actually accomplish it.
Quote:
What did you mean with Linux Distribution? On my own Server? I dont know any so I dont pre fare any.
Simple question; if you're using Linux NOW, what kind of Linux? Red Hat 7.2? CentOS 7.4? Fedora 26? Ubuntu 16.04?? What did you install???
Quote:
Are there any Question who I dont replay with the Data you need?
AGAIN: ***YOU*** need to look at the utilities/commands that have been mentioned before, read up on their use, and START to write your own scripts. AGAIN, we are happy to help, but we aren't going to write it for you. Show some effort, and we'll help you out.
i had a look at e.g. this: https://data.wien.gv.at/daten/geo?ve...etCapabilities
right-click on the page in your browser, select "View Source".
it's XML.
i'd use a bash script and xmllint to parse it and get at the data i need (see my weather forecast in my github).
but all this talk about databases is way beyond me.
i had a look at e.g. this: https://data.wien.gv.at/daten/geo?ve...etCapabilities
right-click on the page in your browser, select "View Source". it's XML. i'd use a bash script and xmllint to parse it and get at the data i need (see my weather forecast in my github).
but all this talk about databases is way beyond me.
That part can be fairly straight forward. If you're already parsing XML, convert it to a CSV file, and MySQL has utilities to import it already.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.