[SOLVED] HOWTO encrypt a tar file on the fly

lleb · 03-07-2012, 05:38 PM

The company has placed a requirement that all backup files written to USB devices be encrypted. I would like to use a FOSS tool for encrypting the tarballs created with

tar -cjvpf file_path.tar.bz /path/to/director/*

Ideally the encryption can be done on the fly instead of having to make the tarball, encrypt it, copy the tarball, check that it copied correctly, then rm the original to save space on the HDD.

I have a nice little script atm that is using LVM and nothing fancy atm for the tarball, it is just that:

Code:

#!/bin/bash

###########################################################
### Created by Ray Brunkow with help from Bryan Smith
###
# Copyright (C) 2012 Raymond L. Brunkow.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 or version 3 of the
# license, at your option.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
###
##########################################################

### Checking for rsync argument
#####################################
# if statement will go here


### Setting Variables
#####################################

dtstamp="`date +%Y-%m-%d-%H:%M:%S `"
dow=`date +%a`
log=${dtstamp}-vgtar.log

### Create LVM Directions.
# [1] Create your volume group (vgusb), logical volume (backup) in the volume group (vgusb) and filesystem on it (vgusb-backup):
# NOTE***  BTW, when you do your "vgcreate" -- make sure nothing is on /dev/sdb1 that you care about.  ;)
# This is in fdisk:  Also, use the slice ID for LVM (8E hex) instead of Ext2/3/4 (83 hex) for LVM.
#
# fdisk /dev/sdX were X is the drive letter you discover via dmesg or tail -f /var/log/messages
# d to delete all partitions on the USB device before you start.
# n for new partition.
# p for primary
# 1 for 1 partition
# t to change flag as to what type of partition we are creating.
# Command (m for help): t
# Selected partition 1
# Hex code (type L to list codes): 8e
# p to view that you have the correct file type for the partition:
# Command (m for help): p
#
# Disk /dev/sda: 8084 MB, 8084520960 bytes
# 249 heads, 62 sectors/track, 1022 cylinders
# Units = cylinders of 15438 * 512 = 7904256 bytes
#
#    Device Boot      Start         End      Blocks   Id  System
#  /dev/sda1               1        1022     7888787   8e  Linux LVM
#
### Now that the USB Device is partitioned correctly we can continue creating the LVM.
### NOTE  From this point forward I will use /dev/sdb and /dev/sdb1 as example device/partition.
#
# pvcreate /dev/sdb1
# vgcreate vgusb /dev/sdb1
# vgchange -ay vgusb   # NEVER HURTS
# lvcreate -l 100%FREE -n backup vgusb    # See below if you have problems here.
# lvchange -ay /dev/mapper/vgusb-backup   # NEVER HURTS
# mkfs.ext3 -j /dev/mapper/vgusb-backup
# tune2fs -c 0 /dev/mapper/vgusb-backup
# vgchange -ay vgusb
#
### This will create both the VG, LV, format the drive, and turn off file system checking.
#
# 
# If the lvcreate -l 100%FREE -n backup vgusb gives you fit do the following:
# We will use the -L option but first we must find the exact number of PEs "free" in the VG
# run "vgdisplay" and you should see something like below:
#	[root@rx30 ~]# vgdisplay
#	  WARNING: Ignoring duplicate config node: umask (seeking umask)
#	  --- Volume group ---
#	  VG Name               vgusb
#	  System ID             
#	  Format                lvm2
#	  Metadata Areas        1
#	  Metadata Sequence No  2
#	  VG Access             read/write
#	  VG Status             resizable
#	  MAX LV                0
#	  Cur LV                1
#	  Open LV               1
#	  Max PV                0
#	  Cur PV                1
#	  Act PV                1
#	  VG Size               7.54 GB
#	  PE Size               4.00 MB
#	  Total PE              1931     #  This is the line you are looking for.
#	  Alloc PE / Size       1931 / 7.54 GB
#	  Free  PE / Size       0 / 0   
#	  VG UUID               d0qGoQ-DGjl-BcjA-IzTo-4mk1-SG71-9kcTrr
#
# Now you can try the lvcreate this way
# lvcreate -L 1931 -n backup vgusb     #### NOTE remember this is the example, use the correct Total PE from your device.
# Follow the rest of the directions above to complete the creation of the LVM.
######################################


### SCAN / ON-LINE
######################################

#	umount anything already mounted as /mnt/backup
umount -f /mnt/backup >> $log 
lvchange -an /dev/vgusb/backup >> $log  # Making offline to prevent issues
vgchange -an vgusb >> $log              # Making offline to prevent issues

#	Scan

pvscan >> $log    # Never hurts
vgscan >> $log    # Never hurts
vgchange -ay vgusb >> $log 
lvchange -ay /dev/vgusb/backup >> $log 
sync


### Fail if the logical volume "backup" is not available
######################################

if [ ! -e "/dev/mapper/vgusb-backup" ] ; then
  echo  "[Backup] USB Backup Disk Not Connected" >> $log 
  exit 1
fi

### MOUNT ATTEMPT
#######################################

mount -t ext3 /dev/mapper/vgusb-backup /mnt/backup >> $log 
rc=$?
if [ $rc -ne 0 ]; then
  echo "[Backup] Unable to mount (rc=${rc}) USB Backup Disk" >> $log 
  exit 2
fi

### BACKUP
########################################

tar -cjvpf /mnt/backup/${dow}-${dtstamp}.tar.bz /usr/rx30/* >> $log  2>&1
sync ; sync

### UMOUNT / OFF-LINE
########################################

umount -f /mnt/backup >> $log 
lvchange -an /dev/vgusb/backup >> $log 
vgchange -an vgusb >> $log 
sync
echo  "[Backup] Completed backup ${dtstamp} at `date`" >> $log 
exit 0

still have a lot to do on this script, but its a work in progress.

Thanks in advance for the help and guidance, also thanks to those who helped me with switching from trying to use the /dev v LVM. I still have loads to learn, but I am liking how powerful the LVM is over the unreliability of the /dev.

FYI, yes this is for very very very low end users in the field. we set it up, and hope they dont break it.

anomie · 03-07-2012, 07:57 PM

The openssl enc(1) program can read from stdin. You'll have to hard-code the key in your script if you want symmetric encryption. (Or you could use GnuPG for asymmetric, as long as a big performance hit is OK.)

For instance:

Code:

$ tar -cj special-dir | 
  openssl enc -aes128 -salt -out special-dir.tar.bz2.enc -e -a -k 'foo%my%pass'

That will produce a bzip2'd tarball that has been encrypted with the AES128 block cipher, and then base64-encoded.

lleb · 03-07-2012, 09:50 PM

Quote:

Originally Posted by anomie

The openssl enc(1) program can read from stdin. You'll have to hard-code the key in your script if you want symmetric encryption. (Or you could use GnuPG for asymmetric, as long as a big performance hit is OK.)

For instance:

Code:

$ tar -cj special-dir | 
  openssl enc -aes128 -salt -out special-dir.tar.bz2.enc -e -a -k 'foo%my%pass'

That will produce a bzip2'd tarball that has been encrypted with the AES128 block cipher, and then base64-encoded.

just to make sure i understand the 'foo...' portion, is this the passcode that is hard coded to decrypt the file?

also what different commands would i need to decrypt the file so i could untar it?

anomie · 03-07-2012, 10:07 PM

The quoted 'foo%my%pass' is your encryption key (read: password used to encrypt the file). To decrypt (so that you're left with a bzip2'd tarball), you'll use:

Code:

$ openssl enc -aes128 -in special-dir.tar.bz2.enc -out special-dir.tar.bz2 -d -a

Be careful that you don't specify the same file for -in and -out. (The enc(1) program assumes you know what you're doing, and will overwrite your encrypted archive without a second thought if that's what you tell it to do.)

lleb · 03-07-2012, 10:41 PM

Quote:

Originally Posted by anomie

The quoted 'foo%my%pass' is your encryption key (read: password used to encrypt the file). To decrypt (so that you're left with a bzip2'd tarball), you'll use:

Code:

$ openssl enc -aes128 -in special-dir.tar.bz2.enc -out special-dir.tar.bz2 -d -a

Be careful that you don't specify the same file for -in and -out. (The enc(1) program assumes you know what you're doing, and will overwrite your encrypted archive without a second thought if that's what you tell it to do.)

got ya on the foo and the decrypting. many thanks. i will play with this tomorrow.

side question, are their tools in Windows that can also decrypt this file?

lleb · 03-08-2012, 07:26 AM

i must be missing something. my lack of understanding the tar process and Linux in general. im slowly learning...

Code:

tar -cjvpf /usr/rx30/rx.dat | openssl enc -aes128 -salt -out /mnt/backup/foo.tar.bz2.enc -e -a -k 'TDSrx30'
tar: Cowardly refusing to create an empty archive
Try `tar --help' or `tar --usage' for more information.

so am i getting the directory portion backwards?

anomie · 03-08-2012, 09:19 AM

Take a look at the tar(1) command options you're using, and compare them to what I posted.

You can either copy my exact command, or you can (at least) remove the -f option from your tar(1) invocation. That's causing a problem.

lleb · 03-08-2012, 09:28 AM

figured it out, had to add - in front of the path to the file

tar -cjvpf - /usr/rx30/rx.dat | openssl enc -aes128 -salt -out /mnt/backup/foo.tar.bz2.enc -e -a -k 'TDSrx30'

anomie · 03-08-2012, 09:36 AM

(If you posted your real encryption key, please change it now.)

lleb · 03-09-2012, 07:47 PM

Quote:

Originally Posted by anomie

(If you posted your real encryption key, please change it now.)

thanks, no that is an example one. only used internally for testing on beta projects that contain no live data. its all fake made up junk data. names like harry potter with a Dr. eye write scripts, etc...

lleb · 03-12-2012, 03:30 PM

Also please mark this as Solved. Thank you.