LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-13-2012, 01:12 PM   #1
dotran
LQ Newbie
 
Registered: Apr 2012
Posts: 27
Blog Entries: 2

Rep: Reputation: Disabled
Decypting file if see bad file then move to bad dir


Please help with this syntax if or case

Here is the good log then decypted file and create trigger file. If see the bad log need to move files to /export/home/mytmp/test1/Bad_Files and send out email then exit not do anything. So far I have no luck yet, cause always have error “gpg: WARNING: using insecure memory!” when decrypting files

Good log:
gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/faq.html for more information
gpg: encrypted with 2048-bit RSA key, ID 4C0F8A06, created 2012-02-03
"TEST <dsec-1@test.com>"

Bad log:
gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/faq.html for more information
gpg: encrypted with 2048-bit RSA key, ID 4C0F8A06, created 2012-02-03
"TEST <dsec-1@test.com>"

gunzip: Rx_test_20110702001.dat.gz: unexpected end of file
gpg: block_filter 1093f0: read error (size=10513,a->size=10513)
gpg: block_filter 1093a0: read error (size=15835,a->size=15835)
gpg: mdc_packet with invalid encoding
gpg: decryption failed: invalid packet
gpg: block_filter: pending bytes!
gpg: block_filter: pending bytes!


Quote:
KEY=`cat /usr/local/apps/secured/test_passphrase`
for Dotran in Rx_test*dat.gz
do

OUT_FILE=Decrypted_${Dotran}

gunzip -c ${Dotran} | \
gpg --decrypt --no-tty --passphrase ${KEY} | \
gzip -c > ${OUT_FILE}

if [[ $? != 0 ]] ; then
#if [[ $? == 0 ]] ; then
#case $Dotran in
echo "[ FAILED ] -- File had move to to /export/home/mytmp/test1/Bad_Files." | mailx -s "[ FAILED ] -- File{s} [${Dotran}] corruption or still transfer." abc@test.com
mv -f Rx_test*dat.gz /export/home/mytmp/test1/Bad_Files
#rm -rf Decrypted_*
exit 0
fi
done

touch TRIG_FILE

exit 0
 
Old 11-13-2012, 01:29 PM   #2
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,258

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
I'm guessing it's your gunzip that's crashing? How big are these files and how time-critical is the processing?

You could always throw a "gunzip -t" in the script before the actual gunzip | gpg | gzip command. If the gunzip -t fails, it's a bad file, and you can throw it in the bad directory before doing the processing on it.

Also, why are you moving all of Rx_test*dat.gz to the bad directory when one bad file is encountered? Wouldn't you want to just move $Dotran to the bad directory instead?

Last edited by suicidaleggroll; 11-13-2012 at 01:31 PM.
 
Old 11-13-2012, 01:41 PM   #3
dotran
LQ Newbie
 
Registered: Apr 2012
Posts: 27
Blog Entries: 2

Original Poster
Rep: Reputation: Disabled
Thank for reply. Code below work perfect. I just want add more loop if bad file then move to bad dir.

I'm guessing it's your gunzip that's crashing? (No) How big are these files and how time-critical is the processing? (Could be multiple files)

Also, why are you moving all of Rx_test*dat.gz to the bad directory when one bad file is encountered? Wouldn't you want to just move $Dotran to the bad directory instead? Yes...I just typo on that when test the files. And the files really big 3gb or 4gb. Don't want gunzip -t check first and want go head decrypt if see the error then failed. Thanks


Quote:
KEY=`cat /usr/local/apps/secured/test_passphrase`
for Dotran in Rx_test*dat.gz
do

OUT_FILE=Decrypted_${Dotran}

gunzip -c ${Dotran} | \
gpg --decrypt --no-tty --passphrase ${KEY} | \
gzip -c > ${OUT_FILE}

Last edited by dotran; 11-13-2012 at 01:49 PM.
 
Old 11-13-2012, 01:50 PM   #4
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,258

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
Quote:
Originally Posted by dotran View Post
I'm guessing it's your gunzip that's crashing? (No)
It looks like it to me, from your bad log file:
Code:
gunzip: Rx_test_20110702001.dat.gz: unexpected end of file
A gunzip -t would catch that problem before you enter into your processing. Something like:
Code:
gunzip -t $Dotran
if [[ $? != 0 ]]; then
   echo "[ FAILED ] -- File had move to to /export/home/mytmp/test1/Bad_Files." | mailx -s "[ FAILED ] -- File{s} [${Dotran}] corruption or still transfer." abc@test.com
   mv -f $Dotran /export/home/mytmp/test1/Bad_Files
else
   gunzip -c ${Dotran} | gpg --decrypt --no-tty --passphrase ${KEY} | gzip -c > ${OUT_FILE}
fi

Quote:
Originally Posted by dotran View Post
How big are these files and how time-critical is the processing? (Could be multiple files)
That didn't answer my question. I'm asking if the added processing time caused by the insertion of a gunzip -t before the gunzip | gpg | gzip would adversely affect the operation of the system. If these are huge files and you're already pushing the system to its limits to keep up with some real time deadline, then this could cause a problem. If it's just a few small files then it shouldn't matter.

Checking the exit status of a gunzip -t before running your processing is a much cleaner way to handle this problem than parsing a log file in my opinion.

Last edited by suicidaleggroll; 11-13-2012 at 01:53 PM.
 
Old 11-13-2012, 02:01 PM   #5
dotran
LQ Newbie
 
Registered: Apr 2012
Posts: 27
Blog Entries: 2

Original Poster
Rep: Reputation: Disabled
Thank for your reply again. And this code will loop for multiple files if see one good file then decrypt and bad file then move? I will test the code more.

Quote:
gunzip -t $Dotran
if [[ $? != 0 ]]; then
echo "[ FAILED ] -- File had move to to /export/home/mytmp/test1/Bad_Files." | mailx -s "[ FAILED ] -- File{s} [${Dotran}] corruption or still transfer." abc@test.com
mv -f $Dotran /export/home/mytmp/test1/Bad_Files
else
gunzip -c ${Dotran} | gpg --decrypt --no-tty --passphrase ${KEY} | gzip -c > ${OUT_FILE}
fi
 
Old 11-13-2012, 02:05 PM   #6
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,258

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
The code I posted is intended to be placed inside your for loop, replacing your current gunzip | gpg | gzip and error checking.

Code:
KEY=`cat /usr/local/apps/secured/test_passphrase`
for Dotran in Rx_test*dat.gz
do
   OUT_FILE=Decrypted_"$Dotran"

   gunzip -t "$Dotran"
   if [[ $? != 0 ]]; then
      echo "[ FAILED ] -- File has been moved to /export/home/mytmp/test1/Bad_Files." | mailx -s "[ FAILED ] -- File{s} [$Dotran] corruption or still transfer." abc@test.com
      mv -f "$Dotran" /export/home/mytmp/test1/Bad_Files
   else
      gunzip -c "$Dotran" | gpg --decrypt --no-tty --passphrase $KEY | gzip -c > "$OUT_FILE"
   fi
done

touch TRIG_FILE

exit 0

FYI - you don't need {} around your variable names unless there are other characters nearby that could be misinterpreted as part of the variable name. Eg:
Code:
var=1
echo ${var}  # Prints "1"
echo $var    # Also prints "1"
echo ${var}a # Prints "1a"
echo $vara   # Prints nothing, since "vara" is the name of a variable that has not been declared yet
You don't do this in your code, which means the {} are unnecessary (but harmless). You should quote the variable substitutions though, in case there are spaces or other "bad" characters in the file names.

Last edited by suicidaleggroll; 11-13-2012 at 02:13 PM.
 
Old 11-14-2012, 02:05 PM   #7
dotran
LQ Newbie
 
Registered: Apr 2012
Posts: 27
Blog Entries: 2

Original Poster
Rep: Reputation: Disabled
Thanks. After a lot the test and I reallly like this code work great, but if we have the wrong password then gunzip -t think is good file and start do decrypting files and create trigger file too. How do we check this syntax password "$KEY" ? Please help or any input on this?

Quote:
KEY=`cat /usr/local/apps/secured/test_passphrase`
for Dotran in Rx_test*dat.gz
do
OUT_FILE=Decrypted_"$Dotran"

gunzip -t "$Dotran"
if [[ $? != 0 ]]; then
echo "[ FAILED ] -- File has been moved to /export/home/mytmp/test1/Bad_Files." | mailx -s "[ FAILED ] -- File{s} [$Dotran] corruption or still transfer." abc@test.com
mv -f "$Dotran" /export/home/mytmp/test1/Bad_Files
exit 0
else
gunzip -c "$Dotran" | gpg --decrypt --no-tty --passphrase $KEY | gzip -c > "$OUT_FILE"
fi
done

touch TRIG_FILE

exit 0
 
Old 11-14-2012, 02:20 PM   #8
suicidaleggroll
LQ Guru
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 5,258

Rep: Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947Reputation: 1947
In that case, it may be best to just split up your gunzip | gpg | gzip command into three separate parts. Run them each individually, and check the exit status at each step along the way.
 
Old 11-14-2012, 03:35 PM   #9
dotran
LQ Newbie
 
Registered: Apr 2012
Posts: 27
Blog Entries: 2

Original Poster
Rep: Reputation: Disabled
Thanks for your input, but don't want to split up your gunzip | gpg | gzip command into three separate parts, cause waste a mount of time to gunzip first (file over 3gb). Is way simple syntac check at the end decypting if see error "bad passphrase" then move to bad directory too? Thanks

gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/faq.html for more information
gpg: encrypted with 2048-bit RSA key, ID 4C0F8A06, created 2012-02-03
"TEST <dsec-1@test.com>"
gpg: public key decryption failed: bad passphrase
gpg: decryption failed: secret key not available


Quote:
else
gunzip -c "$Dotran" | gpg --decrypt --no-tty --passphrase $KEY | gzip -c > "$OUT_FILE"
fi
 
Old 11-15-2012, 02:46 PM   #10
dotran
LQ Newbie
 
Registered: Apr 2012
Posts: 27
Blog Entries: 2

Original Poster
Rep: Reputation: Disabled
After a lot tested and I see the decypted_* file always empty. I come with another check highlight red and see the script work really great. Any different solution or any put on this. Thank

-rw-rw-r-- 1 ca7prod ftpusers 20 Nov 15 12:43 Rx_test12323.dat.gz

gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/faq.html for more information
gpg: encrypted with 2048-bit RSA key, ID 4C0F8A06, created 2012-02-03
"Rx_test12323.dat.gz <dsec-1@test.com>"
gpg: public key decryption failed: bad passphrase
gpg: decryption failed: secret key not available


Quote:
KEY=`cat /usr/local/apps/secured/test_passphrase`
for Dotran in Rx_test*dat.gz
do
OUT_FILE=Decrypted_"$Dotran"

gunzip -t "$Dotran"
if [[ $? != 0 ]]; then
echo "[ FAILED ] -- File has been moved to /export/home/mytmp/test1/Bad_Files." | mailx -s "[ FAILED ] -- File{s} [$Dotran] corruption or still transfer." abc@test.com
mv -f "$Dotran" /export/home/mytmp/test1/Bad_Files
else
gunzip -c "$Dotran" | gpg --decrypt --no-tty --passphrase $KEY | gzip -c > "$OUT_FILE"
#gunzip -c "$Dotran" | gpg --decrypt --no-tty --passphrase '${KEY}' | gzip -c > "$OUT_FILE"
if [ `zcat "$OUT_FILE" | head -n 1 | wc -c ` -eq 0 ]; then
rm -rf "$OUT_FILE" && mv -f "$Dotran" /export/home/mytmp/test1/Bad_Files
echo "[ FAILED ] -- File has been moved to /export/home/mytmp/test1/Bad_Files." | mailx -s "[ FAILED ] -- File{s} [$Dotran] gpg: bad passphrase key or secret key not available." abc@test.com
exit 0
else
echo "[$Dotran decrypting is good file.]"
fi

fi
done

touch TRIG_FILE

exit 0

Last edited by dotran; 11-15-2012 at 03:01 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Bad, Bad, BAD! (Firefox is basically ditching html5 video support) smeezekitty General 11 05-05-2010 07:29 PM
"Mount: wrong file system type, bad option, bad superblock" User Name. Linux - Newbie 9 09-03-2008 04:40 PM
apt-file returns nothing; 'bad file descriptor' overbored Debian 3 10-03-2004 10:13 PM
wrong fs type ,bad option, bad superblock on /dev/hda3, or too many mounted file syst Vencalator Linux - Newbie 4 09-20-2003 02:58 PM
wrong fs type, bad option, bad superblock on /dev/cdrom, or too many mounted file sy TheCyberDude Linux - Software 1 02-11-2002 01:44 PM


All times are GMT -5. The time now is 02:39 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration