Named pipes through the internet?

usao · 04-10-2015, 04:44 PM

I have a need to backup a database on a system where there is not enough filesystem storage for the backup. The idea now is to dump the database to a named pipe on host1, have that cross the internet to host2 where I can read from another named pipe and write to a filesystem.

So, I basically need to write to a named pipe on one host and read from a named pipe on the second host.

Looking for ways/methods/examples of how I can do this.

Nemesiz · 04-10-2015, 05:12 PM

Not sure about pipes but I prefer to use remote fs mount over ssh with sshfs

usao · 04-10-2015, 05:29 PM

Open to options, but the 2 hosts are seperated by distance and the link is not great, so I would be worried about begin able to sustain a mount.
Anyhow, I could use instruction on how to do this method also, since I don't know.

suicidaleggroll · 04-10-2015, 05:54 PM

If it can't sustain a mount, how is it going to sustain a pipe (tcp/ip socket?)?

Your best bet is probably just to rsync over ssh.

Code:

rsync -av /path/to/source/dir/ user@IP:/path/to/backup/dir/

If the connection hangs/drops, just run it again and it'll pick up where it left off.

unSpawn · 04-10-2015, 07:38 PM

Quote:

Originally Posted by usao

So, I basically need to write to a named pipe on one host and read from a named pipe on the second host.

As in 'mysql destination.sql < <(ssh host "mysqldump source")'?

kmhuntly · 04-10-2015, 09:05 PM

not sure what you're trying to do, but if its mysql you could just SSH to the host with a tunnel -

ssh -qtl username -L localport:localhost:remoteport hostname

and then

mysql -h localhost -p localport -u user -P

Nemesiz · 04-11-2015, 03:45 AM

As for mount FS other ssh: sshfs user@remote_host:/path /path

thegwer · 04-11-2015, 02:19 PM

For stability is there a way you could add a cheap box or external HDD to give you enough space to run mysqldump and then use rsync to the offsite location?

travis

usao · 04-11-2015, 02:51 PM

Quote:

Originally Posted by suicidaleggroll

If it can't sustain a mount, how is it going to sustain a pipe (tcp/ip socket?)?

Your best bet is probably just to rsync over ssh.

Code:

rsync -av /path/to/source/dir/ user@IP:/path/to/backup/dir/

If the connection hangs/drops, just run it again and it'll pick up where it left off.

The point is that the source system doesn't have enough space to dump the DB locally, so I have to dump directly to a remote host.

usao · 04-11-2015, 02:58 PM

Quote:

Originally Posted by unSpawn

As in 'mysql destination.sql < <(ssh host "mysqldump source")'?

It a Sybase IQ database. Sybase has the ability to dump to a device or a file, and I have used named-pipes previously as a way to dump directly to 'gzip', but we don't have the option to install a external drive or add storage since the data contains sensitive personal information which wouldn't pass an audit. We can use encrypted channels though, which is why I thought dumping through a named pipe would work because we can layer encryption through the pipe easily enough.

Nemesiz · 04-12-2015, 05:29 AM

I suggest you to use SSH. SSH is already encrypted and mounted you can directly dump to it as locally.