LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 03-07-2012, 06:38 PM   #1
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,581

Rep: Reputation: 480Reputation: 480Reputation: 480Reputation: 480Reputation: 480
HOWTO encrypt a tar file on the fly


The company has placed a requirement that all backup files written to USB devices be encrypted. I would like to use a FOSS tool for encrypting the tarballs created with

tar -cjvpf file_path.tar.bz /path/to/director/*

Ideally the encryption can be done on the fly instead of having to make the tarball, encrypt it, copy the tarball, check that it copied correctly, then rm the original to save space on the HDD.

I have a nice little script atm that is using LVM and nothing fancy atm for the tarball, it is just that:

Code:
#!/bin/bash

###########################################################
### Created by Ray Brunkow with help from Bryan Smith
###
# Copyright (C) 2012 Raymond L. Brunkow.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 or version 3 of the
# license, at your option.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
###
##########################################################

### Checking for rsync argument
#####################################
# if statement will go here


### Setting Variables
#####################################

dtstamp="`date +%Y-%m-%d-%H:%M:%S `"
dow=`date +%a`
log=${dtstamp}-vgtar.log

### Create LVM Directions.
# [1] Create your volume group (vgusb), logical volume (backup) in the volume group (vgusb) and filesystem on it (vgusb-backup):
# NOTE***  BTW, when you do your "vgcreate" -- make sure nothing is on /dev/sdb1 that you care about.  ;)
# This is in fdisk:  Also, use the slice ID for LVM (8E hex) instead of Ext2/3/4 (83 hex) for LVM.
#
# fdisk /dev/sdX were X is the drive letter you discover via dmesg or tail -f /var/log/messages
# d to delete all partitions on the USB device before you start.
# n for new partition.
# p for primary
# 1 for 1 partition
# t to change flag as to what type of partition we are creating.
# Command (m for help): t
# Selected partition 1
# Hex code (type L to list codes): 8e
# p to view that you have the correct file type for the partition:
# Command (m for help): p
#
# Disk /dev/sda: 8084 MB, 8084520960 bytes
# 249 heads, 62 sectors/track, 1022 cylinders
# Units = cylinders of 15438 * 512 = 7904256 bytes
#
#    Device Boot      Start         End      Blocks   Id  System
#  /dev/sda1               1        1022     7888787   8e  Linux LVM
#
### Now that the USB Device is partitioned correctly we can continue creating the LVM.
### NOTE  From this point forward I will use /dev/sdb and /dev/sdb1 as example device/partition.
#
# pvcreate /dev/sdb1
# vgcreate vgusb /dev/sdb1
# vgchange -ay vgusb   # NEVER HURTS
# lvcreate -l 100%FREE -n backup vgusb    # See below if you have problems here.
# lvchange -ay /dev/mapper/vgusb-backup   # NEVER HURTS
# mkfs.ext3 -j /dev/mapper/vgusb-backup
# tune2fs -c 0 /dev/mapper/vgusb-backup
# vgchange -ay vgusb
#
### This will create both the VG, LV, format the drive, and turn off file system checking.
#
# 
# If the lvcreate -l 100%FREE -n backup vgusb gives you fit do the following:
# We will use the -L option but first we must find the exact number of PEs "free" in the VG
# run "vgdisplay" and you should see something like below:
#	[root@rx30 ~]# vgdisplay
#	  WARNING: Ignoring duplicate config node: umask (seeking umask)
#	  --- Volume group ---
#	  VG Name               vgusb
#	  System ID             
#	  Format                lvm2
#	  Metadata Areas        1
#	  Metadata Sequence No  2
#	  VG Access             read/write
#	  VG Status             resizable
#	  MAX LV                0
#	  Cur LV                1
#	  Open LV               1
#	  Max PV                0
#	  Cur PV                1
#	  Act PV                1
#	  VG Size               7.54 GB
#	  PE Size               4.00 MB
#	  Total PE              1931     #  This is the line you are looking for.
#	  Alloc PE / Size       1931 / 7.54 GB
#	  Free  PE / Size       0 / 0   
#	  VG UUID               d0qGoQ-DGjl-BcjA-IzTo-4mk1-SG71-9kcTrr
#
# Now you can try the lvcreate this way
# lvcreate -L 1931 -n backup vgusb     #### NOTE remember this is the example, use the correct Total PE from your device.
# Follow the rest of the directions above to complete the creation of the LVM.
######################################


### SCAN / ON-LINE
######################################

#	umount anything already mounted as /mnt/backup
umount -f /mnt/backup >> $log 
lvchange -an /dev/vgusb/backup >> $log  # Making offline to prevent issues
vgchange -an vgusb >> $log              # Making offline to prevent issues

#	Scan

pvscan >> $log    # Never hurts
vgscan >> $log    # Never hurts
vgchange -ay vgusb >> $log 
lvchange -ay /dev/vgusb/backup >> $log 
sync


### Fail if the logical volume "backup" is not available
######################################

if [ ! -e "/dev/mapper/vgusb-backup" ] ; then
  echo  "[Backup] USB Backup Disk Not Connected" >> $log 
  exit 1
fi

### MOUNT ATTEMPT
#######################################

mount -t ext3 /dev/mapper/vgusb-backup /mnt/backup >> $log 
rc=$?
if [ $rc -ne 0 ]; then
  echo "[Backup] Unable to mount (rc=${rc}) USB Backup Disk" >> $log 
  exit 2
fi

### BACKUP
########################################

tar -cjvpf /mnt/backup/${dow}-${dtstamp}.tar.bz /usr/rx30/* >> $log  2>&1
sync ; sync

### UMOUNT / OFF-LINE
########################################

umount -f /mnt/backup >> $log 
lvchange -an /dev/vgusb/backup >> $log 
vgchange -an vgusb >> $log 
sync
echo  "[Backup] Completed backup ${dtstamp} at `date`" >> $log 
exit 0
still have a lot to do on this script, but its a work in progress.

Thanks in advance for the help and guidance, also thanks to those who helped me with switching from trying to use the /dev v LVM. I still have loads to learn, but I am liking how powerful the LVM is over the unreliability of the /dev.

FYI, yes this is for very very very low end users in the field. we set it up, and hope they dont break it.
 
Old 03-07-2012, 08:57 PM   #2
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
The openssl enc(1) program can read from stdin. You'll have to hard-code the key in your script if you want symmetric encryption. (Or you could use GnuPG for asymmetric, as long as a big performance hit is OK.)

For instance:
Code:
$ tar -cj special-dir | 
  openssl enc -aes128 -salt -out special-dir.tar.bz2.enc -e -a -k 'foo%my%pass'
That will produce a bzip2'd tarball that has been encrypted with the AES128 block cipher, and then base64-encoded.

Last edited by anomie; 03-07-2012 at 09:02 PM. Reason: removed superfluous option.
 
1 members found this post helpful.
Old 03-07-2012, 10:50 PM   #3
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,581

Original Poster
Rep: Reputation: 480Reputation: 480Reputation: 480Reputation: 480Reputation: 480
Quote:
Originally Posted by anomie View Post
The openssl enc(1) program can read from stdin. You'll have to hard-code the key in your script if you want symmetric encryption. (Or you could use GnuPG for asymmetric, as long as a big performance hit is OK.)

For instance:
Code:
$ tar -cj special-dir | 
  openssl enc -aes128 -salt -out special-dir.tar.bz2.enc -e -a -k 'foo%my%pass'
That will produce a bzip2'd tarball that has been encrypted with the AES128 block cipher, and then base64-encoded.
just to make sure i understand the 'foo...' portion, is this the passcode that is hard coded to decrypt the file?

also what different commands would i need to decrypt the file so i could untar it?
 
Old 03-07-2012, 11:07 PM   #4
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
The quoted 'foo%my%pass' is your encryption key (read: password used to encrypt the file). To decrypt (so that you're left with a bzip2'd tarball), you'll use:
Code:
$ openssl enc -aes128 -in special-dir.tar.bz2.enc -out special-dir.tar.bz2 -d -a
Be careful that you don't specify the same file for -in and -out. (The enc(1) program assumes you know what you're doing, and will overwrite your encrypted archive without a second thought if that's what you tell it to do.)
 
1 members found this post helpful.
Old 03-07-2012, 11:41 PM   #5
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,581

Original Poster
Rep: Reputation: 480Reputation: 480Reputation: 480Reputation: 480Reputation: 480
Quote:
Originally Posted by anomie View Post
The quoted 'foo%my%pass' is your encryption key (read: password used to encrypt the file). To decrypt (so that you're left with a bzip2'd tarball), you'll use:
Code:
$ openssl enc -aes128 -in special-dir.tar.bz2.enc -out special-dir.tar.bz2 -d -a
Be careful that you don't specify the same file for -in and -out. (The enc(1) program assumes you know what you're doing, and will overwrite your encrypted archive without a second thought if that's what you tell it to do.)
got ya on the foo and the decrypting. many thanks. i will play with this tomorrow.

side question, are their tools in Windows that can also decrypt this file?
 
Old 03-08-2012, 08:26 AM   #6
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,581

Original Poster
Rep: Reputation: 480Reputation: 480Reputation: 480Reputation: 480Reputation: 480
i must be missing something. my lack of understanding the tar process and Linux in general. im slowly learning...


Code:
tar -cjvpf /usr/rx30/rx.dat | openssl enc -aes128 -salt -out /mnt/backup/foo.tar.bz2.enc -e -a -k 'TDSrx30'
tar: Cowardly refusing to create an empty archive
Try `tar --help' or `tar --usage' for more information.
so am i getting the directory portion backwards?
 
Old 03-08-2012, 10:19 AM   #7
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
Take a look at the tar(1) command options you're using, and compare them to what I posted.

You can either copy my exact command, or you can (at least) remove the -f option from your tar(1) invocation. That's causing a problem.
 
1 members found this post helpful.
Old 03-08-2012, 10:28 AM   #8
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,581

Original Poster
Rep: Reputation: 480Reputation: 480Reputation: 480Reputation: 480Reputation: 480
figured it out, had to add - in front of the path to the file

tar -cjvpf - /usr/rx30/rx.dat | openssl enc -aes128 -salt -out /mnt/backup/foo.tar.bz2.enc -e -a -k 'TDSrx30'
 
Old 03-08-2012, 10:36 AM   #9
anomie
Senior Member
 
Registered: Nov 2004
Location: Texas
Distribution: RHEL, Scientific Linux, Debian, Fedora, Lubuntu, FreeBSD
Posts: 3,930
Blog Entries: 5

Rep: Reputation: Disabled
(If you posted your real encryption key, please change it now.)
 
Old 03-09-2012, 08:47 PM   #10
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,581

Original Poster
Rep: Reputation: 480Reputation: 480Reputation: 480Reputation: 480Reputation: 480
Quote:
Originally Posted by anomie View Post
(If you posted your real encryption key, please change it now.)
thanks, no that is an example one. only used internally for testing on beta projects that contain no live data. its all fake made up junk data. names like harry potter with a Dr. eye write scripts, etc...
 
Old 03-12-2012, 04:30 PM   #11
lleb
Senior Member
 
Registered: Dec 2005
Location: Florida
Distribution: CentOS/Fedora
Posts: 2,581

Original Poster
Rep: Reputation: 480Reputation: 480Reputation: 480Reputation: 480Reputation: 480
Also please mark this as Solved. Thank you.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Howto Encrypt & Run Bash script File nabeeliumattack Linux - General 7 05-17-2008 05:58 PM
how to do a tar extract | create pipe on the fly? jjalocha Linux - General 5 12-29-2007 09:50 PM
tar get on the fly nifflerX Linux - General 3 07-27-2005 11:09 AM
tar and ftp put on the fly nifflerX Linux - General 6 06-14-2005 12:45 PM


All times are GMT -5. The time now is 06:42 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration