Download "real Time" Data -> Modifie it -> Upload to my Webhoster?!
 

Hi
I would say that my problem is very simple.
Now I work hard on my project to make an Webpage for Travellers who have no hidden Ads like the other well known.
Here the Gov offer me some "Open Data" who get updated frequently.
The major problem is I have to use the Data how the get Distributed that is in some case really weird.
Like Weather Data who get Distributed as CSV File.
The next problem like the Weather Data is there are different Source for them.
Depend on the District there are 3 Source for the Weather Data who are different in the way how the are build.

So what I am looking for is a way to run an Server on my ESXi who do there Job 24/7 unattended. The Data must be equalized and than uploadet into the DB from my Hoster to get Published on my Page.
The source can be a CSV File, XML or many other weird Formats.
Sadly the Page is just in German but here are all Open Data who I want use some of:
https://www.data.gv.at/

Before someone ask why I want just the open Data that's simple because I can do with what every I want. I relay hate 3rd Party stuff who show Ads, Track User and other ssshhttt...!

So you're wanting a script of some type to load data into a database?? Shouldn't be a problem at all....so post what you've written/done/tried so far, and tell us where you're stuck. We're happy to help, but we will NOT write your scripts for you.

If you're looking for a starting point, you need to also provide details. Does that input data file have a way to be downloaded automatically, or are you going to do it manually? What format is that data in? What database are you using? How are you loading/doing this procedure now? Version/distro of Linux you're using? Language you want to write this in? As with your other thread, you need to provide details.

Sry i am in Rush and on my phone now.
I just install Suse on my Acer Laptop that's all I have no idea which language i should choose. If some explane bow to do maybe I can start and adjust myself.
I need to know where the limits are. What doesn't a Scripting language can do?
When I am back in the Hotel I will show the data. I think it's an ordinary SQL database.
My Hoster use cpanel.
Some people from the "how to build a Webpage" Community say I should show the table on the page... Omg how dies it look? Horrible... I will make an nice page for foreign Traveller with live data such as weather who look more professional.
There are other thinks why I want grabb like public Transport Dada when I get permission for but that is a different topic.

Again, doesn't matter if you are in a rush...DO NOT use text speak. Again, it's "sorry", not "sry"
Since you are the one writing these programs, it is you who needs to pick the language you're comfortable with. There are MANY, including scripting languages, perl, ruby, python, C, etc. The 'limits' are whatever you're willing to write.
The INPUT data that you're downloading from that page is an entire SQL database? Great....then you (again) need to tell us what kind of SQL database it is.
...which has no bearing on anything here. If you want to load a database/run a program, we need to know the version/distro of Linux.
Great...so again, can you show us ANY of your efforts to date?

So I dont know any scripting languages. Which one is easy to understand and could done the Job?
To see the Data you have click on "Entdecke -> zur Resource"
Here are the source:
https://www.data.gv.at/katalog/datas...4-2d9b3defbedc -> Temp and Air Pollution for Vienna (only)
https://www.data.gv.at/katalog/datas...b-c2b3061bfe5a -> another Temp and Air Pollution Measurements
https://www.data.gv.at/katalog/datas...4-2ea8f40c746f -> Weather for selected City in Austria

for example this 3 Data should be connected together and than equalised to have an DB where all available Data are in one DB.

What did you mean with Linux Distribution? On my own Server? I dont know any so I dont pre fare any.

Are there any Question who I dont replay with the Data you need?

They are all about as easy (and hard) as each other. And ANY of them can do it.
So you are downloading the data manually. That's fine, but you should investigate using wget or curl to try to automate this. Read the man pages on those commands.
Right...again we UNDERSTAND WHAT YOU WANT. But again, we are not seeing anything that you have done/tried to actually accomplish it.
Simple question; if you're using Linux NOW, what kind of Linux? Red Hat 7.2? CentOS 7.4? Fedora 26? Ubuntu 16.04?? What did you install???
AGAIN: ***YOU*** need to look at the utilities/commands that have been mentioned before, read up on their use, and START to write your own scripts. AGAIN, we are happy to help, but we aren't going to write it for you. Show some effort, and we'll help you out.

i had a look at e.g. this: https://data.wien.gv.at/daten/geo?ve...etCapabilities
right-click on the page in your browser, select "View Source".
it's XML.
i'd use a bash script and xmllint to parse it and get at the data i need (see my weather forecast in my github).

but all this talk about databases is way beyond me.

That part can be fairly straight forward. If you're already parsing XML, convert it to a CSV file, and MySQL has utilities to import it already.

Thanks tomorrow morning my flight Home goes and I will read it later.

How to import the Data every 30min or so?

Have a CRON job run the script.

if your system is already running systemd, a systemd timer would be more appropriate.