[SOLVED] grabbing mysql content from a remote server

qwertyjjj · 08-13-2009, 03:12 PM

Rather than leave a mysql port open and push content to it, is it more secure to get a database to pull content into it from a remote connection (therefore only allowing output on the mysql port) or do both have to be open?
If so, I'll have to do something with restricting the times and/or IP addresses.

Can bash scripts run sql code in them?

centosboy · 08-13-2009, 03:30 PM

Quote:

Originally Posted by qwertyjjj

Rather than leave a mysql port open and push content to it, is it more secure to get a database to pull content into it from a remote connection (therefore only allowing output on the mysql port) or do both have to be open?
If so, I'll have to do something with restricting the times and/or IP addresses.

Can bash scripts run sql code in them?

yes.
or from command line remotely.

Code:

mysql -h hostname -u user -ppassword -e'command goes here'

qwertyjjj · 08-13-2009, 04:24 PM

Quote:

Originally Posted by centosboy

yes.
or from command line remotely.

Code:

mysql -h hostname -u user -ppassword -e'command goes here'

Would I only need an output port for this?
If the mysql command is connecting to another server is the password passed in clear text?
TO then move data from 1 server to the other do you state all db connections in the mysql code?
remote to local or is it 2 different mysql connections?

qwertyjjj · 08-13-2009, 04:32 PM

Actually I don't need to put anything into the local database I think, just drag data across.
So, the bash script can loop through result sets and then run htpasswd access for squid?
Is there anyway I can encrypt the password?

I'm guessing bash can't loop through sql records and do something depending on each result or can it?
Or do I have to put the output of the SQL query into a file and then process the file?

I basically want to either:
(a) read a dataset and run a htpasswd command on each relevant line to add or delete a user from the squid passwd file or
(b) read a dataset and import it into a local table that squid can use for mysql authentication

With (b), I'm not sure how to make 2 mysql connections at the same time as the data needs to go from a remote database to a local database.

chrism01 · 08-13-2009, 07:28 PM

Personally, I'd do that in Perl (in fact I have done multiple DBs, not squid though). It can have multiple DB cxns open at one time, so you SELECT from one DB, mangle the results, INSERT into target DB.
Your choice as to which box the program runs on.

http://perldoc.perl.org/
http://www.perlmonks.org/?node=Tutorials
http://search.cpan.org/~timb/DBI-1.609/DBI.pm

centosboy · 08-14-2009, 02:20 AM

Quote:

Originally Posted by qwertyjjj

Actually I don't need to put anything into the local database I think, just drag data across.
So, the bash script can loop through result sets and then run htpasswd access for squid?
Is there anyway I can encrypt the password?

I'm guessing bash can't loop through sql records and do something depending on each result or can it?
Or do I have to put the output of the SQL query into a file and then process the file?

I basically want to either:
(a) read a dataset and run a htpasswd command on each relevant line to add or delete a user from the squid passwd file or
(b) read a dataset and import it into a local table that squid can use for mysql authentication

With (b), I'm not sure how to make 2 mysql connections at the same time as the data needs to go from a remote database to a local database.

Yeah, this can be done in perl, php or shell.
Depends what is easiest for yourself

qwertyjjj · 08-14-2009, 05:17 AM

Quote:

Originally Posted by centosboy

Yeah, this can be done in perl, php or shell.
Depends what is easiest for yourself

If I use PHP, should the script be hidden in the root? I assume it has to be on the webserver root or linux won't read it?
So, /var/www but not /var/www/html ?

qwertyjjj · 08-14-2009, 12:21 PM

Quote:

Originally Posted by qwertyjjj

If I use PHP, should the script be hidden in the root? I assume it has to be on the webserver root or linux won't read it?
So, /var/www but not /var/www/html ?

If I issue a htpasswd command with PHP or Perl, and the user already exists, it updates the user. Would this break an existing connection for that user in the password file if they were connected to the squid server?

Is there a way to scan through htpasswd files for a username before adding or updating it? I don't see much point in updating users every few minutes if they're already there? If not, I'll have to do the back end work in SQL I think.

How big can a htpasswd grow before performance suffers?

qwertyjjj · 08-14-2009, 01:30 PM

I'll use Perl since I don't see any need to mess about with safe mode for PHP to allow it ti run the system command.
I am getting the following errors with this code?

Code:

[root@localhost bin]# vi /etc/myscripts/add_squid_users.pl
[root@localhost bin]# /etc/myscripts/add_squid_users.pl
Can't locate mysql.pm in @INC (@INC contains: /usr/lib/perl5/site_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.7/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.6/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/site_perl/5.8.8 /usr/lib/perl5/site_perl/5.8.7 /usr/lib/perl5/site_perl/5.8.6 /usr/lib/perl5/site_perl/5.8.5 /usr/lib/perl5/site_perl /usr/lib/perl5/vendor_perl/5.8.8/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.7/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.6/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.5/i386-linux-thread-multi /usr/lib/perl5/vendor_perl/5.8.8 /usr/lib/perl5/vendor_perl/5.8.7 /usr/lib/perl5/vendor_perl/5.8.6 /usr/lib/perl5/vendor_perl/5.8.5 /usr/lib/perl5/vendor_perl /usr/lib/perl5/5.8.8/i386-linux-thread-multi /usr/lib/perl5/5.8.8 .) at /etc/myscripts/add_squid_users.pl line 4.
BEGIN failed--compilation aborted at /etc/myscripts/add_squid_users.pl line 4.
[root@localhost bin]# vi /etc/myscripts/add_squid_users.pl
[root@localhost bin]# /etc/myscripts/add_squid_users.pl
Can't locate object method "connect" via package "Mysql" (perhaps you forgot to load "Mysql"?) at /etc/myscripts/add_squid_users.pl line 15.
[root@localhost bin]#

Code:

#!/usr/bin/perl

# PERL MODULE
#use mysql;
use DBD::mysql;

# CONFIG VARIABLES
$host = "remote-server";
$database = "db";
$tablename = "tbl";
$user = "usern";
$pw = "pwd";

# PERL MYSQL CONNECT
$connect = Mysql->connect($host, $database, $user, $pw);

# DEFINE A MySQL QUERY
$myquery = "SELECT user, password, enabled, customers_id FROM passwd WHERE UpdateOnNextCycle=1";

# EXECUTE THE QUERY FUNCTION
$execute = $connect->query($myquery);

while (@results = $execute->fetchrow()) {

        #if enabled =1 then add the user
        if ($results[2] == 1 )
        {
                #htpasswd command add
                system("htpasswd -b /etc/squid/squid_passwd ".$results[0]." ".$results[1]);

        }

        #if enabled =0 then delete the user
        if ($results[2] == 0 )
        {
                #htpasswd command del
                system("htpasswd -D ".$results[0]);
        }

        #after which we update the UpdateOnNextCycle flag to 0 and set UsedDemo to 1
        $execute_upd = $connect->query("
                                 UPDATE passwd
                                 SET UpdateOnNextCycle = 0,
                                 UsedDemo = 1
                                 WHERE customers_id = ".$results[3]."
                                        ");

}

chrism01 · 08-16-2009, 11:19 PM

Although uou will have DBD::MySQL installed, that's just the DB driver module specific to MySQL. What you actually put in your program is

use DBI;

which is the DB Interface module for (nearly) all DB modules called from perl.
Its basically a front-end to eg DBD::MySQL

http://search.cpan.org/~timb/DBI-1.609/DBI.pm (my post #5 above)

and do check for errors in each query eg 'Simple Example' on that page.