LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Grabbing the last field of a delimited entry without using "awk" or "sed" (https://www.linuxquestions.org/questions/linux-newbie-8/grabbing-the-last-field-of-a-delimited-entry-without-using-awk-or-sed-940535/)

drandre 04-18-2012 04:30 PM
Grabbing the last field of a delimited entry without using "awk" or "sed"
 
Using a do-while loop, I am processing entries contained in a file, each on a separate line. Each entry is a path to a file. Each file will be encrypted using openssl, and the encrypted file will be saved in directory called "/home/backup". Since I am not replicating the path of each file in the "/home/backup" directory, I only need the last field which contains the file name.

I need a command, or series of commands, that can grab the last field of a delimited entry without using the "awk" or "sed" commands.

Below is the file "/root/CONFIGS" containing paths to files (the exception is the "/etc/xinetd.d directory"), as well as the code that I have so far. The openssl code won't work until I give it a valid file output (the last field).

Code:
/boot/grub/menu.lst
/etc/cups/printers.conf
/etc/fstab
/etc/group
/etc/hosts
/etc/inittab
/etc/issue
/etc/issue.net
/etc/modprobe.conf
/etc/named.conf
/etc/ntp.conf
/etc/passwd
/etc/postfix/main.cf
/etc/resolv.conf
/etc/samba/smb.conf
/etc/services
/etc/shells
/etc/vmware/vmnet8/dhcpd/dhcpd.conf
/etc/xinetd.conf
/etc/xinetd.d
/var/cron
Code:
function1()
{
while read configName
do
        if [ -e "$configName" ]
        then
                openssl enc -aes-256-cbc -salt -in $configName -out /home/backup$configName.enc -pass pass:$secondKey #This does not work
        else
                echo -e "ERROR: File \"$configName\" does not exist."
        fi
done < "/root/CONFIGS"
}

echo -en "\nSpecify encryption key: "
read firstKey
echo -en "\nConfirm encryption key: "
read secondKey
if [ "$firstKey" = "$secondKey" ]
then
        function1
else
        echo -e "\nERROR: Encryption key does not match. \n"
        exit 2
fi
}

Tinkster 04-18-2012 04:32 PM
Had a look at basename?
Or bash string substitution?
Code:
$ tmp=/boot/grub/menu.lst
$ echo ${tmp##*/}
menu.lst

Cheers,
Tink

CTM 04-18-2012 04:34 PM
Any particular reason you can't use sed or awk? Anyway, here's a creative use of cut and rev:

Code:
$ cat CONFIGS | rev | cut -d'/' -f1 | rev
menu.lst
printers.conf
fstab
group
hosts
inittab
issue
issue.net
modprobe.conf
named.conf
ntp.conf
passwd
main.cf
resolv.conf
smb.conf
services
shells
dhcpd.conf
xinetd.conf
xinetd.d
cron

drandre 04-18-2012 04:37 PM
Quote:
Originally Posted by Tinkster (Post 4656580)
Had a look at basename?
Or bash string substitution?
Looked at, yes.

I am working on this as part of a scripting project and I have been told that I "should use the cut command".

Quote:
Originally Posted by CTM (Post 4656581)
Any particular reason you can't use sed or awk? Anyway, here's a creative use of cut and rev:

Code:
$ cat CONFIGS | rev | cut -d'/' -f1 | rev
menu.lst
printers.conf
fstab
group
hosts
inittab
issue
issue.net
modprobe.conf
named.conf
ntp.conf
passwd
main.cf
resolv.conf
smb.conf
services
shells
dhcpd.conf
xinetd.conf
xinetd.d
cron
I shall try this code...

drandre 04-18-2012 06:06 PM
Quote:
Originally Posted by CTM (Post 4656581)
Code:
cat CONFIGS | rev | cut -d'/' -f1 | rev
This code works perfectly. Thank you for your help!

Tinkster 04-18-2012 06:23 PM
Quote:
Originally Posted by drandre (Post 4656586)
Looked at, yes.

I am working on this as part of a scripting project and I have been told that I "should use the cut command".



I shall try this code...
Fair enough :)

If you can get away w/ out it - here's a version of your original function which I
believe I modified in a way that would let you use the original file unmodified,
and w/o use of external commands:
Code:
function1()
{
while read configName
configName=${configName##*/}
do
        if [ -e "$configName" ]
        then
                openssl enc -aes-256-cbc -salt -in $configName -out /home/backup$configName.enc -pass pass:$secondKey #This does not work
        else
                echo -e "ERROR: File \"$configName\" does not exist."
        fi
done < "/root/CONFIGS"
}

echo -en "\nSpecify encryption key: "
read firstKey
echo -en "\nConfirm encryption key: "
read secondKey
if [ "$firstKey" = "$secondKey" ]
then
        function1
else
        echo -e "\nERROR: Encryption key does not match. \n"
        exit 2
fi
}



Cheers,
Tink

drandre 04-18-2012 07:29 PM
Quote:
Originally Posted by Tinkster (Post 4656626)
Code:
configName=${configName##*/}
I see what you did there. Clever, very clever...

As it so happens, I have a somewhat related question that have run into.

In addition to creating encrypted copies of the files, I also need to checksum each file (I'm using sha256sum), creating a list of checksums in the process, and then creating an encrypted file containing that list (using openssl again). The catch is that the list of checksums cannot be stored in a temporary file.

This presents two issues. First, appending multiple lines of data to something other than a temp file - I'm thinking a variable would be suitable. Based on what I have Googled, it is possible, but I do not understand how. Second, to encrypt a file with openssl you have to specify an input file. I've tried using a variable instead of an input file, but openssl spits errors.

Thoughts?

Tinkster 04-18-2012 07:43 PM
You can make openssl read from STDIN, and output to STDOUT.


Cheers,
Tink

drandre 04-18-2012 07:49 PM
Quote:
Originally Posted by Tinkster (Post 4656672)
You can make openssl read from STDIN, and output to STDOUT.


Cheers,
Tink
That was simple enough. The first issue, storing multiple lines of data in something other than a temp file, still stumps me. I'm thinking that appending data to a variable somehow, or concatenating, would be the theoretical answer - although the practical application has me guessing.

Tinkster 04-18-2012 07:58 PM
W/o trying to replicate your set-up for testing: you should be able
to do the processing in a loop within a sub-shell, and not have to
store anything, not even in RAM (I would have thought).


Cheers,
Tink

drandre 04-18-2012 08:13 PM
Quote:
Originally Posted by Tinkster (Post 4656682)
W/o trying to replicate your set-up for testing: you should be able
to do the processing in a loop within a sub-shell, and not have to
store anything, not even in RAM (I would have thought).


Cheers,
Tink
I'm not sure that I understand?

Concatenation, as I have now tried, also works well.

Code:
file="abc.txt"
tempvar=$(echo "$tempvar $file")
echo $tempvar
abc.txt
file="def.txt"
tempvar=$(echo "$tempvar $file")
echo $tempvar
abc.txt def.txt
The only downside is that the data is not separated into unique lines; but as far as I know, variables only contain data as a stream.

Tinkster 04-18-2012 08:29 PM
Maybe I didn't full understand:
Code:
for i in $(cat files ); do sha256sum $i; done
cb63fb113fecd6ddfdded66056d53ffcd84dcc60373b233aa9d428193fe86454  /etc/fstab
8e7ae18adb8cf63f4a16b972a9ad6c006f26e030ac96261a1bcf103bfdac9ed8  /etc/group
7dd02593eb46aa52b234d3a99b20cfd4730e05e21421f66a42db12fc6727b109  /etc/hosts
a4544d6f0d115de70d5f28bfaa7fa087c1bf1762426958f799e3b0b7b4086062  /etc/inittab
dcb805cb05d78dea3198c43628f7feb80a53b06117450c483d0c01809bee4158  /etc/issue
a7932e9fb9a0c51d71522cc6ba37034f2f2f97e68c5c4fcfed77b82590bd402c  /etc/issue.net
03f7f652376a6c037d6052690a59c741461065a96fee6a6f275ee81e9cba8277  /etc/ntp.conf
220e7d11f4959bce181a45fd0a8519ec0af9db276eaf0f2b98e33586dcaad660  /etc/resolv.conf
41ecfecc953fe2c2d9d5a31d509a318655c120214814f8478465d3522cae9882  /etc/samba/smb.conf
95f8369b7708713f7265cf80e8a2ef312dff455d4dd2d04cd1f81b9c85d90881  /etc/services
d94d37a8f7ae7bf3faffb74bd2a4ecb7e625234b30d6b7caa8edda399fc400f5  /etc/shells

If you slapped that in front of a pipe to openssl for encryption you shouldn't
need to store anything? Obviously the loop would happen over the lines
you read from your files ....



Cheers,
Tink

drandre 04-18-2012 08:53 PM
Quote:
Originally Posted by Tinkster (Post 4656691)
Maybe I didn't full understand:
Code:
for i in $(cat files ); do sha256sum $i; done
cb63fb113fecd6ddfdded66056d53ffcd84dcc60373b233aa9d428193fe86454  /etc/fstab
8e7ae18adb8cf63f4a16b972a9ad6c006f26e030ac96261a1bcf103bfdac9ed8  /etc/group
7dd02593eb46aa52b234d3a99b20cfd4730e05e21421f66a42db12fc6727b109  /etc/hosts
a4544d6f0d115de70d5f28bfaa7fa087c1bf1762426958f799e3b0b7b4086062  /etc/inittab
dcb805cb05d78dea3198c43628f7feb80a53b06117450c483d0c01809bee4158  /etc/issue
a7932e9fb9a0c51d71522cc6ba37034f2f2f97e68c5c4fcfed77b82590bd402c  /etc/issue.net
03f7f652376a6c037d6052690a59c741461065a96fee6a6f275ee81e9cba8277  /etc/ntp.conf
220e7d11f4959bce181a45fd0a8519ec0af9db276eaf0f2b98e33586dcaad660  /etc/resolv.conf
41ecfecc953fe2c2d9d5a31d509a318655c120214814f8478465d3522cae9882  /etc/samba/smb.conf
95f8369b7708713f7265cf80e8a2ef312dff455d4dd2d04cd1f81b9c85d90881  /etc/services
d94d37a8f7ae7bf3faffb74bd2a4ecb7e625234b30d6b7caa8edda399fc400f5  /etc/shells

If you slapped that in front of a pipe to openssl for encryption you shouldn't
need to store anything? Obviously the loop would happen over the lines
you read from your files ....



Cheers,
Tink
I think that is on the right track - but for each loop, the checksum data has to be stored incrementally in something other than a temp file, without echoing the checksums back to the screen. Then, once all of the checksums are collected, it is put into openssl and saved as an encrypted file.

Does that make sense? I might be confusing more than I am helping!

Tinkster 04-18-2012 09:05 PM
Quote:
Originally Posted by drandre (Post 4656707)
I think that is on the right track - but for each loop, the checksum data has to be stored incrementally in something other than a variable, without echoing the checksums back to the screen. Then, once all of the checksums are collected, it is put into openssl and saved as an encrypted file.

Does that make sense? I might be confusing more than I am helping!
Sorry, doesn't make sense to me :}

You already have that loop working, right? I don't understand why you think
you need to store the lines w/ the sha's ... can you give us a larger snippet
of your code?

If the desired result is to have ALL checksummed file names into an encrypted
file I can't see any need for storing any output in a variable (or any other
way, for that matter) - just pipe it to openssl, encrypt it from stdin, and
write the results to your (then encrypted) file - voila.



Cheers,
Tink

drandre 04-18-2012 09:19 PM
Quote:
Originally Posted by Tinkster (Post 4656714)
If the desired result is to have ALL checksummed file names into an encrypted
file I can't see any need for storing any output in a variable (or any other
way, for that matter) - just pipe it to openssl, encrypt it from stdin, and
write the results to your (then encrypted) file - voila
That is exactly what I was looking to do. Can you show me how?

My code is included below. Bear with me as I have changed some variable names. The variable containing checksums of files is shown in red, but based on what you have told me, this variable is unnecessary. The code that encrypts the checksums in the variable is shown in blue, but it produces error messages and does not work. The contents of the CONFIGS file is posted earlier in this thread.

Code:
while read filePath #Read file paths from file
do
        if [ -e "$filePath" ] #If file path exists
        then
                fileName=${filePath##*/} #Variable contains file name from last field
                openssl enc -aes-256-cbc -salt -in $filePath -out /home/backup/$fileName.enc -pass pass:Anonymous 2>/dev/null #Encrypt file, save output to backup directory, and suppress errors
                checksumList=$(echo "$checksumList " $(sha256sum $filePath 2>/dev/null)) #Variable contains checksums of files
        else
                echo -e "ERROR: File \"$filePath\" does not exist." #Error
        fi
done < "/root/CONFIGS" #Read input from file

openssl enc -aes-256-cbc -salt -in $checksumList -out EncryptedChecksums.enc -pass pass:Anonymous ;; #Encrypt checksums

Tinkster 04-18-2012 09:34 PM
If I haven't overlooked anything major:
Code:
while read filePath #Read file paths from file
do
        if [ -e "$filePath" ] #If file path exists
        then
                fileName=${filePath##*/} #Variable contains file name from last field
                openssl enc -aes-256-cbc -salt -in $filePath -out /home/backup/$fileName.enc -pass pass:Anonymous 2>/dev/null 1>>~/enc.log #Encrypt file, save output to backup directory, and suppress errors
                # checksumList=$(echo "$checksumList " $(sha256sum $filePath 2>/dev/null)) #Variable contains checksums of files
                sha256sum $filePath 2>/dev/null
        else
                echo -e "ERROR: File \"$filePath\" does not exist." #Error
                exit 5
        fi
done < "/root/CONFIGS" | openssl enc -aes-256-cbc -salt -out EncryptedChecksums.enc -pass pass:Anonymous

drandre 04-18-2012 09:59 PM
Quote:
Originally Posted by Tinkster (Post 4656722)
If I haven't overlooked anything major:
Code:
while read filePath #Read file paths from file
do
        if [ -e "$filePath" ] #If file path exists
        then
                fileName=${filePath##*/} #Variable contains file name from last field
                openssl enc -aes-256-cbc -salt -in $filePath -out /home/backup/$fileName.enc -pass pass:Anonymous 2>/dev/null 1>>~/enc.log #Encrypt file, save output to backup directory, and suppress errors
                # checksumList=$(echo "$checksumList " $(sha256sum $filePath 2>/dev/null)) #Variable contains checksums of files
                sha256sum $filePath 2>/dev/null
        else
                echo -e "ERROR: File \"$filePath\" does not exist." #Error
                exit 5
        fi
done < "/root/CONFIGS" | openssl enc -aes-256-cbc -salt -out EncryptedChecksums.enc -pass pass:Anonymous
Problem solved! (again)

Is there anything that Linux gurus can't do?

Thank you, again!

Tinkster 04-18-2012 10:54 PM
Sweet, and welcome.


All times are GMT -5. The time now is 01:16 AM.