[SOLVED] Grabbing the last field of a delimited entry without using "awk" or "sed"

drandre · 04-18-2012, 04:30 PM

Using a do-while loop, I am processing entries contained in a file, each on a separate line. Each entry is a path to a file. Each file will be encrypted using openssl, and the encrypted file will be saved in directory called "/home/backup". Since I am not replicating the path of each file in the "/home/backup" directory, I only need the last field which contains the file name.

I need a command, or series of commands, that can grab the last field of a delimited entry without using the "awk" or "sed" commands.

Below is the file "/root/CONFIGS" containing paths to files (the exception is the "/etc/xinetd.d directory"), as well as the code that I have so far. The openssl code won't work until I give it a valid file output (the last field).

Code:

/boot/grub/menu.lst
/etc/cups/printers.conf
/etc/fstab
/etc/group
/etc/hosts
/etc/inittab
/etc/issue
/etc/issue.net
/etc/modprobe.conf
/etc/named.conf
/etc/ntp.conf
/etc/passwd
/etc/postfix/main.cf
/etc/resolv.conf
/etc/samba/smb.conf
/etc/services
/etc/shells
/etc/vmware/vmnet8/dhcpd/dhcpd.conf
/etc/xinetd.conf
/etc/xinetd.d
/var/cron

Code:

function1()
{
while read configName
do 
	if [ -e "$configName" ]
	then
		openssl enc -aes-256-cbc -salt -in $configName -out /home/backup$configName.enc -pass pass:$secondKey #This does not work
	else
		echo -e "ERROR: File \"$configName\" does not exist."
	fi
done < "/root/CONFIGS"
}

echo -en "\nSpecify encryption key: "
read firstKey
echo -en "\nConfirm encryption key: "
read secondKey
if [ "$firstKey" = "$secondKey" ]
then
	function1
else
	echo -e "\nERROR: Encryption key does not match. \n"
	exit 2
fi
}

Tinkster · 04-18-2012, 04:32 PM

Had a look at basename?
Or bash string substitution?

Code:

$ tmp=/boot/grub/menu.lst
$ echo ${tmp##*/}
menu.lst

Cheers,
Tink

CTM · 04-18-2012, 04:34 PM

Any particular reason you can't use sed or awk? Anyway, here's a creative use of cut and rev:

Code:

$ cat CONFIGS | rev | cut -d'/' -f1 | rev
menu.lst
printers.conf
fstab
group
hosts
inittab
issue
issue.net
modprobe.conf
named.conf
ntp.conf
passwd
main.cf
resolv.conf
smb.conf
services
shells
dhcpd.conf
xinetd.conf
xinetd.d
cron

drandre · 04-18-2012, 04:37 PM

Quote:

Originally Posted by Tinkster

Had a look at basename?
Or bash string substitution?

Looked at, yes.

I am working on this as part of a scripting project and I have been told that I "should use the cut command".

Quote:

Originally Posted by CTM

Any particular reason you can't use sed or awk? Anyway, here's a creative use of cut and rev:

Code:

$ cat CONFIGS | rev | cut -d'/' -f1 | rev
menu.lst
printers.conf
fstab
group
hosts
inittab
issue
issue.net
modprobe.conf
named.conf
ntp.conf
passwd
main.cf
resolv.conf
smb.conf
services
shells
dhcpd.conf
xinetd.conf
xinetd.d
cron

I shall try this code...

drandre · 04-18-2012, 06:06 PM

Quote:

Originally Posted by CTM

Code:

cat CONFIGS | rev | cut -d'/' -f1 | rev

This code works perfectly. Thank you for your help!

Tinkster · 04-18-2012, 06:23 PM

Quote:

Originally Posted by drandre

Looked at, yes.

I am working on this as part of a scripting project and I have been told that I "should use the cut command".

I shall try this code...

Fair enough

If you can get away w/ out it - here's a version of your original function which I
believe I modified in a way that would let you use the original file unmodified,
and w/o use of external commands:

Code:

function1()
{
while read configName
configName=${configName##*/}
do 
	if [ -e "$configName" ]
	then
		openssl enc -aes-256-cbc -salt -in $configName -out /home/backup$configName.enc -pass pass:$secondKey #This does not work
	else
		echo -e "ERROR: File \"$configName\" does not exist."
	fi
done < "/root/CONFIGS"
}

echo -en "\nSpecify encryption key: "
read firstKey
echo -en "\nConfirm encryption key: "
read secondKey
if [ "$firstKey" = "$secondKey" ]
then
	function1
else
	echo -e "\nERROR: Encryption key does not match. \n"
	exit 2
fi
}

Cheers,
Tink

drandre · 04-18-2012, 07:29 PM

Quote:

Originally Posted by Tinkster

Code:

configName=${configName##*/}

I see what you did there. Clever, very clever...

As it so happens, I have a somewhat related question that have run into.

In addition to creating encrypted copies of the files, I also need to checksum each file (I'm using sha256sum), creating a list of checksums in the process, and then creating an encrypted file containing that list (using openssl again). The catch is that the list of checksums cannot be stored in a temporary file.

This presents two issues. First, appending multiple lines of data to something other than a temp file - I'm thinking a variable would be suitable. Based on what I have Googled, it is possible, but I do not understand how. Second, to encrypt a file with openssl you have to specify an input file. I've tried using a variable instead of an input file, but openssl spits errors.

Thoughts?

Tinkster · 04-18-2012, 07:43 PM

You can make openssl read from STDIN, and output to STDOUT.

Cheers,
Tink

drandre · 04-18-2012, 07:49 PM

Quote:

Originally Posted by Tinkster

You can make openssl read from STDIN, and output to STDOUT.

Cheers,
Tink

That was simple enough. The first issue, storing multiple lines of data in something other than a temp file, still stumps me. I'm thinking that appending data to a variable somehow, or concatenating, would be the theoretical answer - although the practical application has me guessing.

Tinkster · 04-18-2012, 07:58 PM

W/o trying to replicate your set-up for testing: you should be able
to do the processing in a loop within a sub-shell, and not have to
store anything, not even in RAM (I would have thought).

Cheers,
Tink

drandre · 04-18-2012, 08:13 PM

Quote:

Originally Posted by Tinkster

W/o trying to replicate your set-up for testing: you should be able
to do the processing in a loop within a sub-shell, and not have to
store anything, not even in RAM (I would have thought).

Cheers,
Tink

I'm not sure that I understand?

Concatenation, as I have now tried, also works well.

Code:

file="abc.txt"
tempvar=$(echo "$tempvar $file")
echo $tempvar
abc.txt
file="def.txt"
tempvar=$(echo "$tempvar $file")
echo $tempvar
abc.txt def.txt

The only downside is that the data is not separated into unique lines; but as far as I know, variables only contain data as a stream.

Tinkster · 04-18-2012, 08:29 PM

Maybe I didn't full understand:

Code:

 for i in $(cat files ); do sha256sum $i; done
cb63fb113fecd6ddfdded66056d53ffcd84dcc60373b233aa9d428193fe86454  /etc/fstab
8e7ae18adb8cf63f4a16b972a9ad6c006f26e030ac96261a1bcf103bfdac9ed8  /etc/group
7dd02593eb46aa52b234d3a99b20cfd4730e05e21421f66a42db12fc6727b109  /etc/hosts
a4544d6f0d115de70d5f28bfaa7fa087c1bf1762426958f799e3b0b7b4086062  /etc/inittab
dcb805cb05d78dea3198c43628f7feb80a53b06117450c483d0c01809bee4158  /etc/issue
a7932e9fb9a0c51d71522cc6ba37034f2f2f97e68c5c4fcfed77b82590bd402c  /etc/issue.net
03f7f652376a6c037d6052690a59c741461065a96fee6a6f275ee81e9cba8277  /etc/ntp.conf
220e7d11f4959bce181a45fd0a8519ec0af9db276eaf0f2b98e33586dcaad660  /etc/resolv.conf
41ecfecc953fe2c2d9d5a31d509a318655c120214814f8478465d3522cae9882  /etc/samba/smb.conf
95f8369b7708713f7265cf80e8a2ef312dff455d4dd2d04cd1f81b9c85d90881  /etc/services
d94d37a8f7ae7bf3faffb74bd2a4ecb7e625234b30d6b7caa8edda399fc400f5  /etc/shells

If you slapped that in front of a pipe to openssl for encryption you shouldn't
need to store anything? Obviously the loop would happen over the lines
you read from your files ....

Cheers,
Tink

drandre · 04-18-2012, 08:53 PM

Quote:

Originally Posted by Tinkster

Maybe I didn't full understand:

Code:

 for i in $(cat files ); do sha256sum $i; done
cb63fb113fecd6ddfdded66056d53ffcd84dcc60373b233aa9d428193fe86454  /etc/fstab
8e7ae18adb8cf63f4a16b972a9ad6c006f26e030ac96261a1bcf103bfdac9ed8  /etc/group
7dd02593eb46aa52b234d3a99b20cfd4730e05e21421f66a42db12fc6727b109  /etc/hosts
a4544d6f0d115de70d5f28bfaa7fa087c1bf1762426958f799e3b0b7b4086062  /etc/inittab
dcb805cb05d78dea3198c43628f7feb80a53b06117450c483d0c01809bee4158  /etc/issue
a7932e9fb9a0c51d71522cc6ba37034f2f2f97e68c5c4fcfed77b82590bd402c  /etc/issue.net
03f7f652376a6c037d6052690a59c741461065a96fee6a6f275ee81e9cba8277  /etc/ntp.conf
220e7d11f4959bce181a45fd0a8519ec0af9db276eaf0f2b98e33586dcaad660  /etc/resolv.conf
41ecfecc953fe2c2d9d5a31d509a318655c120214814f8478465d3522cae9882  /etc/samba/smb.conf
95f8369b7708713f7265cf80e8a2ef312dff455d4dd2d04cd1f81b9c85d90881  /etc/services
d94d37a8f7ae7bf3faffb74bd2a4ecb7e625234b30d6b7caa8edda399fc400f5  /etc/shells

If you slapped that in front of a pipe to openssl for encryption you shouldn't
need to store anything? Obviously the loop would happen over the lines
you read from your files ....

Cheers,
Tink

I think that is on the right track - but for each loop, the checksum data has to be stored incrementally in something other than a temp file, without echoing the checksums back to the screen. Then, once all of the checksums are collected, it is put into openssl and saved as an encrypted file.

Does that make sense? I might be confusing more than I am helping!

Tinkster · 04-18-2012, 09:05 PM

Quote:

Originally Posted by drandre

I think that is on the right track - but for each loop, the checksum data has to be stored incrementally in something other than a variable, without echoing the checksums back to the screen. Then, once all of the checksums are collected, it is put into openssl and saved as an encrypted file.

Does that make sense? I might be confusing more than I am helping!

Sorry, doesn't make sense to me :}

You already have that loop working, right? I don't understand why you think
you need to store the lines w/ the sha's ... can you give us a larger snippet
of your code?

If the desired result is to have ALL checksummed file names into an encrypted
file I can't see any need for storing any output in a variable (or any other
way, for that matter) - just pipe it to openssl, encrypt it from stdin, and
write the results to your (then encrypted) file - voila.

Cheers,
Tink

drandre · 04-18-2012, 09:19 PM

Quote:

Originally Posted by Tinkster

If the desired result is to have ALL checksummed file names into an encrypted
file I can't see any need for storing any output in a variable (or any other
way, for that matter) - just pipe it to openssl, encrypt it from stdin, and
write the results to your (then encrypted) file - voila

That is exactly what I was looking to do. Can you show me how?

My code is included below. Bear with me as I have changed some variable names. The variable containing checksums of files is shown in red, but based on what you have told me, this variable is unnecessary. The code that encrypts the checksums in the variable is shown in blue, but it produces error messages and does not work. The contents of the CONFIGS file is posted earlier in this thread.

Code:

while read filePath #Read file paths from file
do 
	if [ -e "$filePath" ] #If file path exists
	then
		fileName=${filePath##*/} #Variable contains file name from last field
		openssl enc -aes-256-cbc -salt -in $filePath -out /home/backup/$fileName.enc -pass pass:Anonymous 2>/dev/null #Encrypt file, save output to backup directory, and suppress errors
		checksumList=$(echo "$checksumList " $(sha256sum $filePath 2>/dev/null)) #Variable contains checksums of files
	else
		echo -e "ERROR: File \"$filePath\" does not exist." #Error
	fi
done < "/root/CONFIGS" #Read input from file

openssl enc -aes-256-cbc -salt -in $checksumList -out EncryptedChecksums.enc -pass pass:Anonymous ;; #Encrypt checksums