LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 09-25-2017, 01:55 PM   #1
tapanksaha
LQ Newbie
 
Registered: Sep 2017
Posts: 3

Rep: Reputation: Disabled
rsync to establish and copy (pull) files from remote using encrypted passphrase


My requirement is
  1. I have 2 servers 101 (non-prod) and 901 (prod)
  2. I need to copy a directory on every Sunday from 901 to 101 using RSYNC
  3. I need to copy the delta of the files in the directory every day from Monday through Saturday from 901 to 101 using RSYNC
  4. I can only pull data from 901 to 101. That means script will run on 101 and can pull files from 901
  5. I cannot set the password less ssh setup (.ssh/id_rsa.pub)
  6. The above scripts need to run through cron, so passwords cannot be provided manually as they will be scheduled jobs.
  7. So, I need to pass the password for the prod (901) server specific user when running the script in non-prod (101)
  8. I need this password to be highly secured using encryption or cipher and stored in 101 so that none can access or view it
  9. The scripts will be scheduled in 101 (non-prod) cron as a specific user

Please suggest how should I proceed or it will be really helpful if someone can provide me a working code.

Thanks in Advance.

Thanks,

Tapan
 
Old 09-25-2017, 02:06 PM   #2
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,215
Blog Entries: 3

Rep: Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703
Welcome. We can help you figure out how to do it, but the effort will have to be from you.

One question I have is about this:

Quote:
Originally Posted by tapanksaha View Post
5. I cannot set the password less ssh setup (.ssh/id_rsa.pub)
What obstacles do you have in setting up public key encryption there? Public key encryption is the established best practice for authenticating SSH and Rsync (which goes over SSH).
 
Old 09-25-2017, 02:57 PM   #3
tapanksaha
LQ Newbie
 
Registered: Sep 2017
Posts: 3

Original Poster
Rep: Reputation: Disabled
Hello Sir,

Reason being, in our organization, as per role segregation, many associates are given non-production access and not the production one. And the service account which will be used to copy directory is accessible to many resources using SUDO privileges.
Now, if the ssh password less is setup, they can do an SSH authentication to the production servers and leave us in high risk.

So, my requirement is little weird.
Thank you for your understanding. If you have any better suggestion, please help.

Thanks,
Tapan
 
Old 09-25-2017, 05:07 PM   #4
IsaacKuo
Senior Member
 
Registered: Apr 2004
Location: Baton Rouge, Louisiana, USA
Distribution: Debian Stable
Posts: 2,546
Blog Entries: 8

Rep: Reputation: 465Reputation: 465Reputation: 465Reputation: 465Reputation: 465
You need another service account, specifically for this, that is NOT accessible to "many resources". If you do not already know how to lock down and restrict SUDO privileges, learn it. Sudo is an excellent tool for letting various users have limited access to ONLY the elevated access they precisely need. Sudo is an awful tool for providing "admin" superuser access.

If this is too much disruption to for the non-prod developers, then probably you just can't do what you want using a "pull" strategy. A "push" strategy with password-less ssh...or even just a simple nfs share could be the best approach.

By a simple nfs share, I mean a share on the non-prod box which is restricted to the prod box's IP address.

Another possibility is to do a "pull" from a read only nfs share on the prod box. This means that anyone with pretty much any access to the non-prod box will be able to read that directory on the prod box. But since this will get copied to the non-prod box, and those users can access the contents on the non-prod box, is this a problem?

If it were me, I would not be copying any sort of sensitive data from prod to non-prod, so I'd be okay with a read-only nfs share on the prod box. However, I know things are not always ideal in real world setups...

Anyway, that's my tentative recommendation. Set up a read-only nfs share on the prod box, and use rsync to copy from that share to the local box. No need for any sort of ssh login at all.
 
Old 09-25-2017, 06:35 PM   #5
tapanksaha
LQ Newbie
 
Registered: Sep 2017
Posts: 3

Original Poster
Rep: Reputation: Disabled
Hello Isaac,

Thank you for the valuable suggestions. I think the option below from your suggestion should be possible in my case.

"Another possibility is to do a "pull" from a read only nfs share on the prod box. This means that anyone with pretty much any access to the non-prod box will be able to read that directory on the prod box. But since this will get copied to the non-prod box, and those users can access the contents on the non-prod box, is this a problem?"

We do not have any issue in reading the content. The only concern is that none should be able to login to Production servers. So, I will talk to Client IT department and see if I can get a read only NFS in production. Another concern they might raise is that copying data from the original directory to the read-only NFS will again consume some of the hardware resources and don't know if it will impact in any way.
However, i will give it a try.

Thanks,
Tapan
 
Old 09-25-2017, 06:50 PM   #6
IsaacKuo
Senior Member
 
Registered: Apr 2004
Location: Baton Rouge, Louisiana, USA
Distribution: Debian Stable
Posts: 2,546
Blog Entries: 8

Rep: Reputation: 465Reputation: 465Reputation: 465Reputation: 465Reputation: 465
You can make the original directory the nfs read only share. On the prod server itself, there's no need to copy the data to another location, assuming everything in the original directory is okay to be in the share.

If it is not acceptable to directly share the original directory, consider using cp with hardlinks. This takes extremely little hard drive space and is very fast. The target directory must be on the same partition as the source for hardlinks to work, but this makes it easy to only "copy" the files that you want to be available.
 
Old 09-25-2017, 11:30 PM   #7
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,215
Blog Entries: 3

Rep: Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703
Quote:
Originally Posted by tapanksaha View Post
Now, if the ssh password less is setup, they can do an SSH authentication to the production servers and leave us in high risk.
You could use have the production server initiate the rsync transfer to the non-production server. But if you can only initiate the script from the non-production server, then you must learn the proper way to use SSH keys and, maybe, sudo. From the short description it sounds like you really need to read about how to apply to concept of least privilege to both sudo and SSH keys. Both can be locked down to allow only a single function.

Unfortunately, with the vague details, we can only give vague pointers in the right direction. For sudo, you'll want to pay attention to the configuration file. See:

Once the configurations in /etc/sudoers are locked down for the one transfer account, then look at tightening down the SSH key or keys for that one account. For that look a the manual pages on your system for "sshd" and "sshd_config". In the latter, pay special attention to the directives Match, PasswordAuthentication, and AuthorizedKeysFile. You'll need to set AuthorizedKeysFile to a location that the account can read but not write. In the former, pay special attention to the section "AUTHORIZED_KEYS FILE FORMAT" and the option command=.

You are using a separate account for these transfers, right?

If you have more details, we can be more precise.

Last edited by Turbocapitalist; 09-25-2017 at 11:48 PM. Reason: PasswordAuthentication
 
Old 09-26-2017, 01:53 AM   #8
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,497

Rep: Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240
If I understand well you can use rsyncd to "share" that directory and use rsync to copy. Do not need to use ssh at all.
 
Old 09-26-2017, 02:40 AM   #9
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,215
Blog Entries: 3

Rep: Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703Reputation: 3703
Quote:
Originally Posted by pan64 View Post
If I understand well you can use rsyncd to "share" that directory and use rsync to copy. Do not need to use ssh at all.
From what I gather, that method requires an extra port open and the data is transfered unencrypted:

Quote:
"Also note that the rsync daemon protocol does not currently provide any encryption of the data that is transferred over the connection. Only authentication is provided. Use ssh as the transport if you want encryption."
Source: https://linux.die.net/man/5/rsyncd.conf

Regular rsync goes over SSH so the port for that is probably already open plus you can use all the usual SSH tricks like read-only single-purpose keys.
 
Old 09-26-2017, 03:07 AM   #10
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,497

Rep: Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240Reputation: 7240
yes, I know that, but as it was explained: We do not have any issue in reading the content. And as it was also told: almost anyone can log in and use sudo... I think the original problem is to do that copy automatically, without user interaction and without entering password (but without using passwordless ssh too). rsyncd fulfils all these requirements.
But OP will give us more details to make it clear
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Files seem to take up more space in destination after rsync copy Karderio Linux - Newbie 6 09-22-2014 08:41 PM
Script to copy files, rsync? Ruffe42 Linux - Newbie 5 04-30-2014 03:55 PM
[SOLVED] copy files with rsync or dd vnc Linux - Newbie 4 02-07-2013 06:51 AM
encrypted home passphrase ? minty33 Linux - Newbie 2 09-24-2012 06:41 PM
rsync copying everything not only changed files with luks encrypted drive changcheh Linux - General 1 09-10-2011 10:44 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 01:09 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration