LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 05-25-2014, 12:02 PM   #1
sayhello_to_the_world
Member
 
Registered: May 2013
Posts: 229

Rep: Reputation: Disabled
using XML::Twig and DBI for storing a xml-file into a myql-db


good eveing dear linux-experts.



i have a xml-file that comes out of a osm-parser - it is formatted in xml: which method fits if i want to store the output of this request to a mysql database:

the dataset i have gathered from this site: http://overpass-turbo.eu

see the output here

Code:
<node id="2064639440" lat="49.4873181" lon="8.4710548"> 
    <tag k="amenity" v="restaurant"/> 
    <tag k="cuisine" v="turkish"/> 
    <tag k="email" v="info@lynso.de"/> 
    <tag k="name" v="Kilim  - Café und Bar Restaurant"/> 
    <tag k="opening_hours" v="Su-Th 17:00-1:00; Fr, Sa 17:00-3:00"/> 
    <tag k="operator" v="Cengiz Kaya"/> 
    <tag k="phone" v="06 21 - 43 755 371"/> 
    <tag k="website" v="http://www.kilim-mannheim.de/"/> 
  </node> 
  <node id="2126473801" lat="49.4851170" lon="8.4756295"> 
    <tag k="amenity" v="restaurant"/> 
    <tag k="cuisine" v="italian"/> 
    <tag k="email" v="mannheim1@vapiano.de"/> 
    <tag k="fax" v="+49 621 1259 779"/> 
    <tag k="name" v="Vapiano"/> 
    <tag k="opening_hours" v="Su-Th 10:00-24:00; Fr-Sa 10:00-01:00"/> 
    <tag k="operator" v="Vapiano"/> 
    <tag k="phone" v="+49 621 1259 777"/> 
    <tag k="website" v="http://www.vapiano.de/newsroom/?store=29"/> 
    <tag k="wheelchair" v="yes"/> 
  </node>

question;

how can this be done? note we have the fild names and all the set of the datbase is clear. the question is - how to put the data into the database?




Code:
 
#!/usr/bin/perl  
use strict ; 
use DBI; 
use XML::Twig; 
 
# prepare database 
my $dbh=dbh(); # connect 
init(); 
$dbh->do('USE db123'); 
#$dbh->do('DELETE FROM pois'); 
 
# sql 
my $sql = 'REPLACE INTO pois VALUES (?,?,?,?,?,?)'; 
my $sth = $dbh->prepare($sql); 
 
# set up handler 
my $t = XML::Twig->new(  
  twig_handlers => { 'node' => \&node } 
); 
 
# parse xml 
my $xml = do { local $/; <DATA> }; 
$t->parse($xml); 
#$t->parsefile('.osm'); 
 
sub node { 
  my ($t,$elt) = @_; 
   
  my %data=( 
   'id'  => $elt->att('id'), 
   'lat' => $elt->att('lat'), 
   'lon' => $elt->att('lon'), 
   ); 
  for my $tag ( $elt->children() ){ 
    $data{$tag->att('k')} = $tag->att('v'); 
    #print $tag->att('k').' = '.$tag->att('v')."\n"; 
  } 
  
  # update database 
  my @f = map{ $data{$_} }('id','lat','lon','name','amenity','operator'); 
  if ($f[3] ne '' && $f[4] ne '' && $f[5] ne ''){ 
    print "-- INSERT --\n". 
    (join "\n",@f). 
    "\n\n"; 
    $sth->execute(@f); 
  } 
} 
 
 
sub init { 
  $dbh-> do('CREATE DATABASE IF NOT EXISTS db123  
             DEFAULT CHARACTER SET latin1  
             COLLATE latin1_german2_ci'); 
  $dbh->do('USE db123'); 
  $dbh->do('CREATE TABLE IF NOT EXISTS pois ( 
           id       BIGINT(20) UNSIGNED NOT NULL, 
           lat      FLOAT(10,7) NOT NULL, 
           lon      FLOAT(10,7) NOT NULL, 
           name     VARCHAR(255) COLLATE utf8_bin NOT NULL, 
           amenity  VARCHAR(255) COLLATE utf8_bin NOT NULL, 
           operator VARCHAR(255) COLLATE utf8_bin NOT NULL, 
           PRIMARY KEY  (id) 
          ) ENGINE=InnoDB DEFAULT  
            CHARSET=utf8  
            COLLATE=utf8_bin'); 
} 
 
sub dbh { 
  my $dsn = "DBI:mysql:database=;host=localhost"; 
  my $dbh = DBI->connect($dsn, 'user', 'pwd', 
            {RaiseError => 1, PrintError => 1})  
            or die (Error connecting " $DBI::errstr"); 
}
- how to do it best way?
 
Old 05-25-2014, 12:14 PM   #2
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,617

Rep: Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963
Quote:
Originally Posted by sayhello_to_the_world View Post
good eveing dear linux-experts.
i have a xml-file that comes out of a osm-parser - it is formatted in xml: which method fits if i want to store the output of this request to a mysql database the dataset i have gathered from this site: http://overpass-turbo.eu see the output here
question;
how can this be done? note we have the fild names and all the set of the datbase is clear. the question is - how to put the data into the database?

- how to do it best way?
You seem to answer your own question. You posted XML data, then posted a Perl program that reads XML data and posts it into a MySQL database. The sample data and the program was directly lifted from: http://www.perlmonks.org/?node_id=1087321

The only question here is: what's the problem? Does that program give you an error? If so, what is it? What results are you getting? What's actually wrong/not-working???? The 'best way' is the way YOU decide is best; if that program works, and you're happy with the results, then you have the 'best way'. If you're not happy, you need to identify what the problem(s) are, and fix them.

Again, searching for a solution on your own would lead you to possible solutions and much knowledge to let you solve your problem; you've been given this advice in numerous other threads of yours:
http://gis.stackexchange.com/questio...-to-a-mysql-db
http://forum.openstreetmap.org/viewtopic.php?pid=152695
http://mysqlserverteam.com/mysql-5-7...is-an-example/
 
Old 05-25-2014, 12:55 PM   #3
sayhello_to_the_world
Member
 
Registered: May 2013
Posts: 229

Original Poster
Rep: Reputation: Disabled
hello dear TBone

many many thanks for the hints - you do a great job and i have learned alot form you.


Quote:
The only question here is: what's the problem? Does that program give you an error? If so, what is it? What results are you getting? What's actually wrong/not-working???? The 'best way' is the way YOU decide is best; if that program works, and you're happy with the results, then you have the 'best way'. If you're not happy, you need to identify what the problem(s) are, and fix them.
TBone - youre right and you convince me with your ideas.




one quick question: i do lots of work with PHP
therfore i have installed an
Apache
Mysql
PHPMyadmin on my opensuse 13.1

all works great!!
[ and the phpmyadmin is allso available http://localhost/phpMyAdmin/ ]

Question - the allready installed mysql database can be usef for the perl-job too, can ´t it!?



i will do the further tests. And then i come bak and report all the findings.


Thanks for all your work here - you are so supportive and the community benefits very very much.

i do as adviced and come back and report all the findings.

greeting

Last edited by sayhello_to_the_world; 05-25-2014 at 01:18 PM.
 
Old 05-26-2014, 10:54 AM   #4
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,617

Rep: Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963Reputation: 7963
Quote:
Originally Posted by sayhello_to_the_world View Post
hello dear TBone
many many thanks for the hints - you do a great job and i have learned alot form you. TBone - youre right and you convince me with your ideas.
Yes, you keep saying that, but you CONTINUE to ignore the advice, and keep posting questions that show you've put very little (if any) effort forward of your own. You posted sample data that wasn't your own, and a program that does what you're looking for, but didn't actually TRY that program, to see what results you got. If it didn't work, the errors would have told you what needed to be changed to give you the results you're after.
Quote:
one quick question: i do lots of work with PHP therfore i have installed an Apache Mysql PHPMyadmin on my opensuse 13.1 all works great!!
[ and the phpmyadmin is allso available ]

Question - the allready installed mysql database can be usef for the perl-job too, can ´t it!?
Obviously, yes. A database is just that, and it operates independently from the web engine. Your systems administrators can easily explain the differences between such things in detail. PHPMyadmin does not enter into things, since it's only a web front-end to tasks you can already accomplish at the mysql command prompt.
Quote:
i will do the further tests. And then i come bak and report all the findings. Thanks for all your work here - you are so supportive and the community benefits very very much. i do as adviced and come back and report all the findings.
Will you??? There are many threads where you never responded to people asking for further information, nor did you go back and post the solutions.

Last edited by TB0ne; 05-26-2014 at 12:21 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
MySQL to XML including data, DTD and XML Schema rm_-rf_windows Linux - General 3 07-06-2015 07:43 AM
[SOLVED] openbox- autostart.sh, rc.xml, menu.xml fail to load new-here Slackware 2 08-07-2012 07:15 PM
how to add xml-stylesheet tag in a XML File using libxml2 ? peacemission Programming 6 05-26-2012 02:20 AM
no xml, convert tvtime stationlist to xml for mythtv/freevo... frenchn00b Linux - General 8 11-03-2007 11:35 PM
configure: error: could not find DocBook XML DTD V4.1.2 in XML catalog Fadoksi Linux - Software 1 07-16-2006 06:41 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 07:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration