LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-09-2017, 11:43 AM   #1
NobleOne
LQ Newbie
 
Registered: Jul 2017
Posts: 29

Rep: Reputation: Disabled
Batching Copying Files Across Network


I've read a lot of posts on multiple sites about this, but not one post that seems to combine BOTH the ability to BATCH copy files over a network (i.e. scp) with a secure solution.

The goal is to routinely backup files from Linux box A to Linux box B via cron job. So obviously, entering a password isn't an option, and using a password-less login is of course not secure. So how best to get this done?
 
Old 07-09-2017, 12:13 PM   #2
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,614
Blog Entries: 3

Rep: Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860
Usually rsync is a good way to go for transferring multiple files. The sftp program works, too, but cannot easily resume if interrupted. Both work over SSH so that means you can use keys.

Code:
rsync -a -v -H -e "ssh -i ~/.ssh/somekey.rsa" \
        /source/dir/ user@remote.example.com:/dest/dir/

Usually if you do not use a passphrase on your key you should lock it down substantially with a forced command on the server. Otherwise, with a little bit of configuration, you can use an agent to hold the key for that connection.
 
Old 07-09-2017, 12:28 PM   #3
NobleOne
LQ Newbie
 
Registered: Jul 2017
Posts: 29

Original Poster
Rep: Reputation: Disabled
Thanks. Since your post, I have been reading up on rsync and rsync_t and my eyes are glazing over. Seems pretty complicated.
 
Old 07-09-2017, 12:33 PM   #4
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,614
Blog Entries: 3

Rep: Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860
You can go quite deep, but that is not really necessary. With password authentication, via SSH, you would run it like this:

Code:
rsync -a -v -H \
        /source/dir/ user@remote.example.com:/dest/dir/
And to use a key see, the previous example in post #2.

Can you give more details about the transfer you have in mind and how you have used rsync so far. Also, how have you been using keys for regular SSH connections?
 
Old 07-09-2017, 12:36 PM   #5
NobleOne
LQ Newbie
 
Registered: Jul 2017
Posts: 29

Original Poster
Rep: Reputation: Disabled
Thanks! I'll give it a try.
 
Old 07-09-2017, 12:38 PM   #6
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,237

Rep: Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656Reputation: 1656
Passwordless SSH keys are not insecure. Its the standard way of unattended copying to remote hosts.

Set up your passwordless SSH keys under your preferred user: http://www.linuxproblem.org/art_9.html

That will probably save your key as .ssh/id_rsa.pub unless you specify otherwise.

Open up your cron as that user:

Code:
su - user1
crontab -e
add your rsync command to the cron and save (for example every 30 minutes):

Code:
*/30 * * * * rsync -varh -e "ssh -i ~/.ssh/id_rsa.pub" /local/source/dir/ remoteuser@remotemachine:/remote/dest/dir/
Keep in mind that the user you are setting SSH keys up for has to have permissions to the local files you are trying to copy.
 
Old 07-09-2017, 12:41 PM   #7
NobleOne
LQ Newbie
 
Registered: Jul 2017
Posts: 29

Original Poster
Rep: Reputation: Disabled
Super, sz. I think I can handle this. I love this forum!
 
Old 07-09-2017, 12:45 PM   #8
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,614
Blog Entries: 3

Rep: Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860Reputation: 2860
Quote:
Originally Posted by szboardstretcher View Post
That will probably save your key as .ssh/id_rsa.pub unless you specify otherwise.
I'd recommend specifying a unique filename for the key pair so that later when you have more than one it will be easier to keep track of them. Likewise embedding a comment in the public key helps, too, once you have several public keys together. The -f and -C options do that. See "man sshkey-gen"
 
1 members found this post helpful.
Old 07-09-2017, 12:47 PM   #9
NobleOne
LQ Newbie
 
Registered: Jul 2017
Posts: 29

Original Poster
Rep: Reputation: Disabled
Got it, thanks again!
 
  


Reply

Tags
batch, scp, secure


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Copying files with scp - from a network to another network githin Linux - Networking 4 05-05-2011 08:57 PM
Copying files over a network - Strange behaviour jsteel Linux - Networking 5 01-11-2010 01:32 PM
Copying Large Files on the Network bichonfrise74 Linux - Newbie 7 02-20-2009 12:48 PM
Copying files through a network. Denwar Mandriva 1 04-13-2004 07:30 AM
copying files across the network jcwilliams Linux - Networking 2 01-12-2001 12:03 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:13 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration