LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-04-2013, 04:22 PM   #1
sigint-ninja
Member
 
Registered: Feb 2011
Location: Republic Of Ireland
Distribution: Debian,Centos,Slackware
Posts: 463

Rep: Reputation: 29
Wink Some more Rsync questions


ok so i am a total noob at linux
i managed to copy a file using rsync from one of my linux boxes
to another one over the internet...i just have a few more questions and would appreciate it if you could just give me a push in the right direction...

this was a simple command from my terminal...but i realise if i want to backup a server with many directories i will need to create a bash script right? i need the rsync script to run at a certain time, i will use cron to do this...this i understand. what i dont understand is this...

how can i get the script to login with the remote linux box automatically...obviously i dont want to have to type in a password every time the backup runs.

are bash scripts simply text files? do they have to be saved with a .sh extension...where do they get placed? does it matter? does the cron just find the script and execute it?

lastly i dont think i have ssh configured properly between the machines as it gives an error about keys or something...whats the most secure way to implement ssh?

thanks for all your reading
 
Old 11-04-2013, 04:30 PM   #2
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974
rsync is, imho, best run over an ssh tunnel. Here the automatic login is achieved via the use of a preshared ssh key (utterly unrelated to rsync here)... plenty of guides showing this setup - https://www.digitalocean.com/communi...rsync-over-ssh

a script can be called anything and placed anywhere. the extension doesn't matter. any executed file will be presumed to be a shell script, but placing a shebang:
Code:
#!/bin/bash
as the very first line will make it explicitly use that interpreter instead of the default sh (so, python, perl, bash, ruby etc...) You'd just usually set it as an executable file
Code:
chmod +x /where/ever/that/file/is/script.sh
After that, you can just run "/where/ever/that/file/is/script.sh" in a prompt and it will run.

the cron job finds it by the fact that you would usually provide the absolute path of it. Whilst you can modify the environment that cron uses to look for files, just specifying the full path is much clearer an simpler.
 
Old 11-04-2013, 07:17 PM   #3
sigint-ninja
Member
 
Registered: Feb 2011
Location: Republic Of Ireland
Distribution: Debian,Centos,Slackware
Posts: 463

Original Poster
Rep: Reputation: 29
Thanks for that info...i havent been this excited about computing for a long time now...really enjoying linux and doing stuff like this. I was first messing about with centos and now have debian installed too. Are the distros really very different to use/work with in a work environment? Am i better off sticking to a specific distro? Thanks again for the help...
 
Old 11-04-2013, 08:30 PM   #4
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,630

Rep: Reputation: 495Reputation: 495Reputation: 495Reputation: 495Reputation: 495
check out the links in my sig for howto create the ssh keys you will need, there is also some remote backup scripts out there that use rsync that are easy enough to manipulate.
 
1 members found this post helpful.
Old 11-05-2013, 04:55 PM   #5
paladin.michael
Member
 
Registered: Jun 2011
Location: Danbury, CT, USA
Distribution: Kubuntu, Slackware, Debian, FreePBX
Posts: 72

Rep: Reputation: 22
Quote:
...Are the distros really very different to use/work with in a work environment? Am i better off sticking to a specific distro?...
Concerning distribution differences, while the base system is usually very similar there can be very important differences between different distributions, not the least of which are program versions available in the package management repository (if it has one), binary vs compile at install time package management, different default paths, installation methodologies, etc. I recommend figuring out which distribution meets your needs at work and stick with it while learning others on non-production or non-mission-critical applications.

Just to give a few arbitrary examples of major differences between distributions:

Gentoo's Portage package management system is incredibly expansive, compiles everything (and I mean everything) you install on the system using it from source by default (though I think there may be a way to force it to use binary packages too), and uses a "use flags" system which allows you to customize everything down to the smallest degree, even if your configurations conflict with each other, so long as you write the flags properly. This allows it to use bleeding edge versions of software specifically configured for your system (A gentoo installation is closer to a custom personal distribution than a universal structure), but means you're waiting until tomorrow to see that X, KDE environment you just entered the install command for and are going to be climbing that learning curve and doing lots of reading, potentially getting lost a little on the way.

Debian, on the other hand, uses apt which does binary distribution by default (e.g. no compilation, fast, but not as custom built to your system and potentially slower and an older version due to availability in the repository) While derivatives use this system, they tend to vastly differ in default software available. Additionally, Debian doesn't necessarily do things the same way other distros do, meaning different default paths for certain things. However it's an easier to learn package management system compared to Gentoo and the binary packages tend to give instant gratification when you need to install something which is one reason why so many distros designed for people used to windows are based on it (ubuntu, mint, etc.) as it allows them to retain a lot of the expectations they have from Windows (binary packages, not a lot of loading time for installations, etc.)

Slackware has no automatic global package manager to handle dependency management for you. You learn about the software and install the prerequisites yourself (after compiling in many cases, but not necessarily) But you'll learn quite a lot, know exactly what is on the system (because you put it there personally) and it'll run until the hardware breaks once it's set up properly. It also tends to be very traditional in how things are done and upgrades aren't as obvious without the package management other distros rely on.

But they all utilize the same general folder structure for the base system and can be classified as GNU/Linux operating systems.
 
Old 11-06-2013, 04:21 PM   #6
sigint-ninja
Member
 
Registered: Feb 2011
Location: Republic Of Ireland
Distribution: Debian,Centos,Slackware
Posts: 463

Original Poster
Rep: Reputation: 29
thanks for that information i guess they really arent kidding when they say open source offers choice.
i noticed you didnt mention red hat at all...isnt it true that most corporate systems are red hat/red hat based?

what i really ment to ask is, despite the differences you mentioned, if i learned say slackware backwards, would i pretty much be able to administer a wide range of linux servers etc in the real world.

again thanks for the reply
 
Old 11-06-2013, 04:35 PM   #7
paladin.michael
Member
 
Registered: Jun 2011
Location: Danbury, CT, USA
Distribution: Kubuntu, Slackware, Debian, FreePBX
Posts: 72

Rep: Reputation: 22
The primary reason I didn't mention RedHat is because I don't have as much experience with it compared to some of the others. My employer utilizes an almost exclusively Microsoft environment so I'm due to go back and play with it a bit. The last time I messed with it was back before I was doing things at work using Linux and I didn't like it as much for my personal machines as some others at that time.

That being said, even with the differences involved between distributions, yes, the majority of skills you learn in administrating a linux system do carry over between them. In spite of the plethora of choice out there, the core of the system is still GNU/Linux and while there are a couple which don't stick to traditional paths or file placement, they still keep things in the same place as everyone else for the most part and it's still a Linux kernel in there and the GNU software collection providing the basic tools.
 
Old 11-13-2013, 04:16 PM   #8
sigint-ninja
Member
 
Registered: Feb 2011
Location: Republic Of Ireland
Distribution: Debian,Centos,Slackware
Posts: 463

Original Poster
Rep: Reputation: 29
thanks for that info paladin,

lieb...there is a ssh guide you have posted at the beginning it says "Unless otherwise specified all of these tasks are performed on server A."

which is server A??? the server that is sending data...or receiving data from a remote host...sorry if it seems a silly question

sn
 
Old 11-14-2013, 03:11 AM   #9
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974
"Server A" is the client, but yeah... doesn't actually say that does it??!
 
Old 11-14-2013, 04:47 AM   #10
AwesomeMachine
Senior Member
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian testing/sid; OpenSuSE; Fedora
Posts: 1,896

Rep: Reputation: 267Reputation: 267Reputation: 267
Arrow Linux Distros

Quote:
Originally Posted by sigint-ninja View Post
thanks for that information i guess they really arent kidding when they say open source offers choice.
i noticed you didnt mention red hat at all...isnt it true that most corporate systems are red hat/red hat based?

what i really ment to ask is, despite the differences you mentioned, if i learned say slackware backwards, would i pretty much be able to administer a wide range of linux servers etc in the real world.

again thanks for the reply
Nearly every Linux distro is a debian fork. Some notable exceptions are Red Hat/Fedora, Gentoo/Sabayan, Slackware and SuSE/opensuse. Linux Mint, Arch Linux, Knoppix, Mepis, Ubuntu, Kubuntu, etc. are all debian forks. If debian didn't exist, none of those would either.

If you want to learn server operation, you probably want to do away with the X server, because server-based machines don't require an X server. They're not designed to be fun or anything like that.

And, computer operators follow corporate policies on automation and documentation of it. Every so often, especially when a new system is installed, the company (any company) will hunt down a premiere operator to write all kinds of automation scripts.

The new guy is thinking, 'Two more months of this and I can just hit one key every five minutes and everything will work by itself. Won't that be great!'

The company is thinking, 'As soon as this guy finishes the difficult stuff, we can make a severance agreement with him and hire grunts to push one key every five minutes.'

So, you just can't win. But programmers tend to get abused less, because they're more dangerous. They can destroy an entire corporation. They're not ideologically motivated to do that, but you don't want to push them too far, because it has happened!

And, it isn't worth learning server administration for your home-based network. It's like becoming an auto mechanic to fix your own vehicle.
 
Old 11-14-2013, 07:40 AM   #11
acid_kewpie
Moderator
 
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974Reputation: 1974
Quote:
Originally Posted by AwesomeMachine View Post
Nearly every Linux distro is a debian fork.
That's really quite an overstatement.

Quote:
Originally Posted by AwesomeMachine View Post
If you want to learn server operation, you probably want to do away with the X server, because server-based machines don't require an X server. They're not designed to be fun or anything like that.
Reality says far far far too many enterprise linux servers run X constantly, and is engrained in their operation.
Quote:

And, computer operators follow corporate policies on automation and documentation of it. Every so often, especially when a new system is installed, the company (any company) will hunt down a premiere operator to write all kinds of automation scripts.

The new guy is thinking, 'Two more months of this and I can just hit one key every five minutes and everything will work by itself. Won't that be great!'

The company is thinking, 'As soon as this guy finishes the difficult stuff, we can make a severance agreement with him and hire grunts to push one key every five minutes.'
In my experience it's more "Changing stuff is expensive! All the project management and cross charging required would invalidate any long term savings. Let's just keep doing things as they are."
Quote:

So, you just can't win. But programmers tend to get abused less, because they're more dangerous. They can destroy an entire corporation. They're not ideologically motivated to do that, but you don't want to push them too far, because it has happened!

And, it isn't worth learning server administration for your home-based network. It's like becoming an auto mechanic to fix your own vehicle.

Last edited by acid_kewpie; 11-14-2013 at 07:41 AM.
 
Old 11-14-2013, 05:50 PM   #12
paladin.michael
Member
 
Registered: Jun 2011
Location: Danbury, CT, USA
Distribution: Kubuntu, Slackware, Debian, FreePBX
Posts: 72

Rep: Reputation: 22
While the discussion about differences between distributions and how prolific different distributions are can go on for a while, one of my favorite sources for this stuff has become the Linux Documentation Project.

http://www.tldp.org

There is a ton of information here which spans from beginner stuff through to the really arcane. One of the sections I'd recommend to get a good solid base in Linux which will be usable with pretty much any distribution would be the Introduction to Linux Guide. Just go to the Guides section and scroll down a little and you'll find it.

I have a pdf version of it on my kindle that I'm reading through to re-learn a lot of basics and I think it would be a good thing for a beginner who really wants to get a good understanding of the basics.
 
Old 11-17-2013, 11:38 AM   #13
sigint-ninja
Member
 
Registered: Feb 2011
Location: Republic Of Ireland
Distribution: Debian,Centos,Slackware
Posts: 463

Original Poster
Rep: Reputation: 29
ok thanks to everyone who posted...i have learned a lot over the last few weeks...i just have a last question regarding rsync...here goes

if i have server A and Server B located 20km's away...and im using Rsync to backup +- 200GB of data from A to B (only backing up changes) , should I also have a 2TB hdd connected to server B that takes a local full backup everyday??? cause surely if its only backing up the changes and a file becomes corrupt (ie it cant be opened) the changes will copy over to my only copy of the data? whats the best practice?

thanks again
 
Old 11-17-2013, 06:31 PM   #14
paladin.michael
Member
 
Registered: Jun 2011
Location: Danbury, CT, USA
Distribution: Kubuntu, Slackware, Debian, FreePBX
Posts: 72

Rep: Reputation: 22
Some more Rsync questions

as far as good disaster recovery, you should have historical backups of some kind to protect against this. whether you perform a local backup at the site of server a or do weekly full backups which transfer to server b and are synced to a long term storage location to be retained for some reasonable amount of time or some other option you think it's better for you is partially dependant on what is available as a storage location.

I would recommend the second option. additionally, if you do weekly full backups it could also make your incremental backups smaller as they would be the difference from a more recent full backup, hence fewer differences.
 
Old 11-18-2013, 01:19 AM   #15
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,630

Rep: Reputation: 495Reputation: 495Reputation: 495Reputation: 495Reputation: 495
Quote:
Originally Posted by sigint-ninja View Post
thanks for that info paladin,

lieb...there is a ssh guide you have posted at the beginning it says "Unless otherwise specified all of these tasks are performed on server A."

which is server A??? the server that is sending data...or receiving data from a remote host...sorry if it seems a silly question

sn
wow, thank you for pointing that out. Ill fix that ASAP. Server A is the local server, not the remote. server B is typically remote. I apologize and thank you for pointing out my oversight.

It has been adjusted. Again please forgive my oversight. I try to help where and when I can.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
rsync questions jsteel Linux - General 2 07-05-2010 10:52 AM
rsync questions crazy8 Linux - Newbie 11 04-05-2008 01:59 PM
Rsync Questions progrock Linux - Networking 2 11-14-2007 12:41 PM
rsync questions slack66 Linux - General 4 11-05-2005 01:31 AM
rsync questions csross Linux - General 3 06-25-2004 11:10 AM


All times are GMT -5. The time now is 11:42 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration