LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 01-03-2018, 10:36 AM   #1
howarddavidp
LQ Newbie
 
Registered: Jan 2018
Location: DC/Alexandria
Distribution: All of them
Posts: 3

Rep: Reputation: Disabled
Need help with a backup script and mounting a volume under AWS Linux


Hello,

I am new, want to first say hello. Yes, I read the newbie forum and the 2 posts, I guessed this was the best place to post. Not to offer a bounty, but if the folks here help me get my butt out of a sling, count me in for an annual membership

So, my question revolves around a backup script for Tenable Security Center. The script is not really the problem, but rather the volume and the placement of the tarball backup file. The script gerernates a tarball with this command:

tar -Pzvcf sc_backup.tar.gz /opt/sc

However, the application is installed to the root of /dev/xvdf, which maps to /opt/sc. (2TB of space for 234GB worth of data files)

When I do a "ls" of /opt/sc, the files and directories are installed there (I did not do the install, it is AWS linux install. I assume the virtual volume I was given is called XVDF, from reading the /dev folder.) This volumes is 2 TB in size, and it is using 234GB.

I ran a "df -h" command, and it returned:

Filesystem Size Used Avail Use% Mounted On
/dev/xvda2 100G 2.0G 99G 2% /
tmpfs 32G 0 32G 0% /dev
"a few other temps"
/dev/xvdf 2.2T 234G 1.8T 12% /opt/sc

So my dilema is I do not have the space to run this backup, because it seems the app is installed into the root of the volume with the space. No, I cannot re-install the app into the proper directory easily (it would require me to involve several others and it will take lots of effort).

I though maybe I could modify the "tar" command to exclude a directory that I would make under /opt/sc, so I make something like /opt/sc/backup, then tell tar to backup everything under /opt/sc EXCEPT the /opt/sc/backup folder. This might be the easiest way.

My other thought was to make an /opt/sc/backup folder, mount it under /backup, then again exclude it from the tar command. I am not sure what this gets me if I can figure out how to exclude /opt/sc/backup folder?

Of course, if there are other, better options, that is why I came here.

The complete script, in case anyone want to see it, is as follows:

# Tenable Console backup script
# This will make the backup file in the /backup.space #
# directory
service SecurityCenter stop
ps -fu tns
killall -u tns
killall httpd
mkdir /backup.space
cd /backup.space
tar -Pzvcf sc_backup.tar.gz /opt/sc
tar -tvf sc_backup.tar.gz



Many thanks in advance, I hope I found the best place to get some help. I have used Linux for years, on and off, then on, then off again, etc. Looks like I will be doing more and more, so I am looking for some help going forward.

TY VVM,

Dave Howard
 
Old 01-03-2018, 02:43 PM   #2
dpajares
LQ Newbie
 
Registered: Apr 2004
Location: uruguay
Distribution: debian
Posts: 26
Blog Entries: 1

Rep: Reputation: 15
Hi all. Do you try with -C option (man tar). I recommend to read this post, maybe is a permission problem

https://stackoverflow.com/questions/...ory-permission

Can you give us the output of the -vv tar command.

Regards,
 
Old 01-03-2018, 02:52 PM   #3
_roman_
Member
 
Registered: Dec 2017
Location: _Austro_Bavaria_
Distribution: gentoo / linux mint
Posts: 433

Rep: Reputation: 28
I used for a very short time rdiffbackup or what it is called. (you may use any rsync based software for that, there should be many choices available)
it allows to exclude certain folders which makes sense

I only backup from a dead system. Rephrased I boot up sysrescue live-cd to handle my not in use /
I think backup from a live running system is a bit problematic.
 
Old 01-05-2018, 09:03 AM   #4
howarddavidp
LQ Newbie
 
Registered: Jan 2018
Location: DC/Alexandria
Distribution: All of them
Posts: 3

Original Poster
Rep: Reputation: Disabled
Not found an answer yet

Hello,

We do not have the option to boot to something else and then run a backup, this is AWS linux and we are limited.

The best I have found using the man pages, is to use the --exclude='folder' option in the beginning of the tar command. I am going to test it on another folder with less data first to see if it works.

The other option, which I prefer, would be to mount the xvdf volumes at /opt, then makes a /opt/sc folder and a /opt/backup folder. I could then backup one to the other and not have to exclude anything. Problem is getting the admin to help do this, and I would also need to change how the mounting would work when it reboots, which I am not sure how to do?

Any other suggestions, other than RTFM?

Thanks,
Dave

Last edited by howarddavidp; 01-05-2018 at 09:05 AM.
 
Old 01-05-2018, 09:27 AM   #5
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Yawnstown, Ohio
Distribution: Mojave
Posts: 9,279
Blog Entries: 36

Rep: Reputation: Disabled
https://www.linuxquestions.org/quest...nd-aws-809188/ may offer some help.

I utilize an s3 mount for my backups using s3fs
Code:
s3fs           fuse.s3fs  256T     0  256T   0% /backups
Notoriously s.l.o..w., But...it works.

my /etc/fstab:
Code:
s3fs#internal		  /backups	fuse	allow_other,nonempty	0	0
https://cloud.netapp.com/blog/amazon...-a-file-system discusses the technique for implementing s3fs.
and https://www.interserver.net/tips/kb/...tu-using-s3fs/

Good Luck, and Happy New Year!

Last edited by Habitual; 01-06-2018 at 03:34 PM.
 
Old 01-06-2018, 06:45 AM   #6
voleg
Member
 
Registered: Oct 2013
Distribution: RedHat CentOS Fedora SuSE
Posts: 328

Rep: Reputation: 50
As you use AWS, do backup directly to S3, it will be cheaper.
Code:
# aws s3 sync --delete /path/to/directory/to/backup s3://your-bucket-name/
You can make it in very secure way, even encrypted at rest.
You can configure bucket for versioning, this will simulate snapshots.
In this case, do not forget to configure expiration, otherwise you will billed for extra space.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Mounting a NSS volume in Linux AwakenLie Linux - Networking 5 03-31-2016 03:22 PM
LXer: AWS console breach leads to demise of service with “proven” backup plan LXer Syndicated Linux News 0 06-19-2014 05:20 AM
LXer: Perfect Backup Solution with Amazon S3 and aws LXer Syndicated Linux News 1 05-25-2010 07:06 AM
Mounting a NetApp volume on Linux machine durdenr Linux - General 1 04-30-2009 02:47 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 05:46 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration