using XML::Twig and DBI for storing a xml-file into a myql-db
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
using XML::Twig and DBI for storing a xml-file into a myql-db
good eveing dear linux-experts.
i have a xml-file that comes out of a osm-parser - it is formatted in xml: which method fits if i want to store the output of this request to a mysql database:
how can this be done? note we have the fild names and all the set of the datbase is clear. the question is - how to put the data into the database?
Code:
#!/usr/bin/perl
use strict ;
use DBI;
use XML::Twig;
# prepare database
my $dbh=dbh(); # connect
init();
$dbh->do('USE db123');
#$dbh->do('DELETE FROM pois');
# sql
my $sql = 'REPLACE INTO pois VALUES (?,?,?,?,?,?)';
my $sth = $dbh->prepare($sql);
# set up handler
my $t = XML::Twig->new(
twig_handlers => { 'node' => \&node }
);
# parse xml
my $xml = do { local $/; <DATA> };
$t->parse($xml);
#$t->parsefile('.osm');
sub node {
my ($t,$elt) = @_;
my %data=(
'id' => $elt->att('id'),
'lat' => $elt->att('lat'),
'lon' => $elt->att('lon'),
);
for my $tag ( $elt->children() ){
$data{$tag->att('k')} = $tag->att('v');
#print $tag->att('k').' = '.$tag->att('v')."\n";
}
# update database
my @f = map{ $data{$_} }('id','lat','lon','name','amenity','operator');
if ($f[3] ne '' && $f[4] ne '' && $f[5] ne ''){
print "-- INSERT --\n".
(join "\n",@f).
"\n\n";
$sth->execute(@f);
}
}
sub init {
$dbh-> do('CREATE DATABASE IF NOT EXISTS db123
DEFAULT CHARACTER SET latin1
COLLATE latin1_german2_ci');
$dbh->do('USE db123');
$dbh->do('CREATE TABLE IF NOT EXISTS pois (
id BIGINT(20) UNSIGNED NOT NULL,
lat FLOAT(10,7) NOT NULL,
lon FLOAT(10,7) NOT NULL,
name VARCHAR(255) COLLATE utf8_bin NOT NULL,
amenity VARCHAR(255) COLLATE utf8_bin NOT NULL,
operator VARCHAR(255) COLLATE utf8_bin NOT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB DEFAULT
CHARSET=utf8
COLLATE=utf8_bin');
}
sub dbh {
my $dsn = "DBI:mysql:database=;host=localhost";
my $dbh = DBI->connect($dsn, 'user', 'pwd',
{RaiseError => 1, PrintError => 1})
or die (Error connecting " $DBI::errstr");
}
good eveing dear linux-experts.
i have a xml-file that comes out of a osm-parser - it is formatted in xml: which method fits if i want to store the output of this request to a mysql database the dataset i have gathered from this site: http://overpass-turbo.eu see the output here question;
how can this be done? note we have the fild names and all the set of the datbase is clear. the question is - how to put the data into the database?
- how to do it best way?
You seem to answer your own question. You posted XML data, then posted a Perl program that reads XML data and posts it into a MySQL database. The sample data and the program was directly lifted from: http://www.perlmonks.org/?node_id=1087321
The only question here is: what's the problem? Does that program give you an error? If so, what is it? What results are you getting? What's actually wrong/not-working???? The 'best way' is the way YOU decide is best; if that program works, and you're happy with the results, then you have the 'best way'. If you're not happy, you need to identify what the problem(s) are, and fix them.
many many thanks for the hints - you do a great job and i have learned alot form you.
Quote:
The only question here is: what's the problem? Does that program give you an error? If so, what is it? What results are you getting? What's actually wrong/not-working???? The 'best way' is the way YOU decide is best; if that program works, and you're happy with the results, then you have the 'best way'. If you're not happy, you need to identify what the problem(s) are, and fix them.
TBone - youre right and you convince me with your ideas.
one quick question: i do lots of work with PHP
therfore i have installed an
Apache
Mysql
PHPMyadmin on my opensuse 13.1
hello dear TBone
many many thanks for the hints - you do a great job and i have learned alot form you. TBone - youre right and you convince me with your ideas.
Yes, you keep saying that, but you CONTINUE to ignore the advice, and keep posting questions that show you've put very little (if any) effort forward of your own. You posted sample data that wasn't your own, and a program that does what you're looking for, but didn't actually TRY that program, to see what results you got. If it didn't work, the errors would have told you what needed to be changed to give you the results you're after.
Quote:
one quick question: i do lots of work with PHP therfore i have installed an Apache Mysql PHPMyadmin on my opensuse 13.1 all works great!!
[ and the phpmyadmin is allso available ]
Question - the allready installed mysql database can be usef for the perl-job too, can ´t it!?
Obviously, yes. A database is just that, and it operates independently from the web engine. Your systems administrators can easily explain the differences between such things in detail. PHPMyadmin does not enter into things, since it's only a web front-end to tasks you can already accomplish at the mysql command prompt.
Quote:
i will do the further tests. And then i come bak and report all the findings. Thanks for all your work here - you are so supportive and the community benefits very very much. i do as adviced and come back and report all the findings.
Will you??? There are many threads where you never responded to people asking for further information, nor did you go back and post the solutions.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.