LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 07-22-2010, 05:06 PM   #1
SilversleevesX
Member
 
Registered: May 2009
Posts: 181
Blog Entries: 9

Rep: Reputation: 15
BASH read text list zip files into a single archive --possible?


AKA "zipping on the fly .. the slow-as-molasses way."

The list includes full pathnames to each file, and they're all in subfolders of the same parent folder (which, unfortunately, is not the root folder of the drive or system on which the files reside).

A cleaned-up and radio-ready portion of the list looks like
Quote:
.../taiga/ahqr-va-choyvp/bv0884-073.jpg
.../taiga/ahqr-va-choyvp/bv2635-073.jpg
.../taiga/ahqr-va-choyvp/bv3067-175.jpg
.../taiga/ahqr-va-choyvp/bv3867-229.jpg
.../taiga/ahqr-va-choyvp/bv5828-071.jpg
.../taiga/ahqr-va-choyvp/bv5878-072.jpg
What I'd like to be able to do is zip all the files in the list into a single archive, to avoid the step of having to copy them to the same location (presumably another folder on the HD) and then zip that folder. I'm more inclined to make provisions about extracting to a single folder at some other time.

Is this possible in BASH, or would I have to consider a faster, more robust scripting language such as python or perl?

BZT
 
Click here to see the post LQ members have rated as the most helpful post in this thread.
Old 07-22-2010, 05:35 PM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Well.. suppose you create the list using the find command.. an option might be:
Code:
find . -name \*.jpg | zip archive -@
To extract all the files in the current directory (despite the relative paths) you can try the -j option of unzip. To extract in a different directory use the -d option, e.g.
Code:
unzip -j archive.zip -d /path/to/some/dir
Please, see man zip and man unzip for details.
 
Old 07-22-2010, 06:31 PM   #3
SilversleevesX
Member
 
Registered: May 2009
Posts: 181
Blog Entries: 9

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by colucix View Post
Well.. suppose you create the list using the find command.. an option might be:
Code:
find . -name \*.jpg | zip archive -@
To extract all the files in the current directory (despite the relative paths) you can try the -j option of unzip. To extract in a different directory use the -d option, e.g.
Code:
unzip -j archive.zip -d /path/to/some/dir
Please, see man zip and man unzip for details.
I have the lists. I want zip to read a given list and compress the files specified in it into a single zip archive. I'm not looking to zip whole directories of a certain file type, but rather particular ones the lists of which have been generated and re-encoded (as far as LF/CR and other finer details) in a text editor.

I've been trying a few different things since my OP to this thread. One resulted in zip (as a matter of fact) zipping every JPEG in every sub-folder of the parent folder. The second one that worked zipped the files that were on the list, but did so preserving the sub-directory paths (better than 48 folders). This got me closer to what I wanted, so I tried the -j option: zip insisted on creating empty archives. If I can get over this obstacle I'll have the script I intended to have.

BZT
 
Old 07-22-2010, 09:18 PM   #4
SilversleevesX
Member
 
Registered: May 2009
Posts: 181
Blog Entries: 9

Original Poster
Rep: Reputation: 15
This seems to work:
Code:
while read 'line'
do
t="$line"
#cp $t /cygdrive/c/tmp/tempfolder/
n=$(basename "$t")
#echo "Copying file $n to temporary folder"

zip -jg0q /cygdrive/c/FTP/lite1280 $t
echo "Zipping file $n."
cd ../
#rm -r tempfolder/*.jpg
#echo "Deleting copy of $n in temporary folder."
done<lite1280-wp.txt
About the only thing it skips are files with spaces in the names, even though I have them escaped in both Unix-friendly edited lists. But five out of 140 (3⅙ %) is really not too much of a loss. For the time being, considering it's incidentally such a small part of the whole, I'll chalk it up to a weakness in either bash or the .zip executable.

BZT
 
Old 07-23-2010, 02:08 AM   #5
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,424

Rep: Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876
Hey BZT

I know your new to some of this, but I also think some of what I am about to say has been covered before:
Code:
while read 'line'
Quotes not required.
Code:
t="$line"
#and
n=$(basename "$t")
Why bother if you do not do anything else to 'line'
Code:
cd ../
Why? or was this just a leftover before removing your temp folder?

There may be better options than the one you have chosen, but what you have, as you say it works, could just be:
Code:
while read line
do
    zip -jg0q /cygdrive/c/FTP/lite1280 $line
    echo "Zipping file ${line##*/}."
done<lite1280-wp.txt
 
1 members found this post helpful.
Old 07-23-2010, 02:26 AM   #6
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Quote:
Originally Posted by SilversleevesX View Post
This got me closer to what I wanted, so I tried the -j option: zip insisted on creating empty archives. If I can get over this obstacle I'll have the script I intended to have.
Indeed the -j option should work even for archive creation. You tell us something is wrong, but you don't show the command nor the exact warning/error/output messages so that it's difficult to have a clue. For example on my system the following works:
Code:
$ cat filelist
./taiga/ahqr-va-choyvp/bv0884-073.jpg
./taiga/ahqr-va-choyvp/bv2635-073.jpg
./taiga/ahqr-va-choyvp/bv3067-175.jpg
./taiga/ahqr-va-choyvp/bv3867-229.jpg
./taiga/ahqr-va-choyvp/bv5828-071.jpg
./taiga/ahqr-va-choyvp/bv5878-072.jpg
./file.jpg
./file with spaces.jpg
$ zip -j archive -@ < filelist
  adding: bv5878-072.jpg (stored 0%)
  adding: bv0884-073.jpg (stored 0%)
  adding: bv2635-073.jpg (stored 0%)
  adding: bv3067-175.jpg (stored 0%)
  adding: bv3867-229.jpg (stored 0%)
  adding: bv5828-071.jpg (stored 0%)
  adding: file.jpg (stored 0%)
  adding: file with spaces.jpg (stored 0%)
Note the compression ratio is 0 because in my test the files are empty.
 
2 members found this post helpful.
Old 07-23-2010, 02:31 AM   #7
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Quote:
Originally Posted by SilversleevesX View Post
Code:
zip -jg0q /cygdrive/c/FTP/lite1280 $t
Quote:
Originally Posted by grail
Code:
zip -jg0q /cygdrive/c/FTP/lite1280 $line
Don't forget to use double quotes around $t or $line to manage names with blank spaces.
 
1 members found this post helpful.
Old 07-25-2010, 06:59 AM   #8
SilversleevesX
Member
 
Registered: May 2009
Posts: 181
Blog Entries: 9

Original Poster
Rep: Reputation: 15
Here's the code I puzzled out on my own that worked. (In so doing, I found out how much the zip binary has a serious tooth against any zip it doesn't create itself. I should learn 'touch' a little better, maybe?)

Code:
while read 'line'
do
t="$line"
n=$(basename "$t")

zip -jg0q /cygdrive/c/FTP/lite1280 $t
echo "Zipping file $n."
cd ../
done<lite1280-wp.txt
By coincidence (or maybe not?) it's the spitting image of grail's script from 3:08 PM.

And here's a snippet of lite1280-wp.txt :
Quote:
/cygdrive/c/taiga/beach/bv0326-018.jpg
/cygdrive/c/taiga/beach/bv0468-023.jpg
/cygdrive/c/taiga/beach/bv0994-216.jpg
/cygdrive/c/taiga/beach/bv2039-068.jpg
/cygdrive/c/taiga/beach/bv3259-218.jpg
/cygdrive/c/taiga/beach/bv5807-096.jpg
/cygdrive/c/taiga/beach/bv8348-123.jpg
/cygdrive/c/taiga/beach/bv8622-190.jpg
/cygdrive/c/taiga/beach/bv9241-032.jpg
/cygdrive/c/taiga/beach/gae52-5205-044-006.jpg
/cygdrive/c/taiga/beach/gae52-5205-081-006.jpg
/cygdrive/c/taiga/beach/gae54-5418-950-001.jpg
/cygdrive/c/taiga/beach/gae59-5927-039-005.jpg
/cygdrive/c/taiga/beach/gae61-6139-035-006.jpg
/cygdrive/c/taiga/beach/gallery5adds022.jpg
/cygdrive/c/taiga/beach/gallery5adds023.jpg
Sorry I didn't get myself more involved in the process here on LQ. It wasn't until I'd run the above script, moved the product zip file/s over to my laptop and unzipped them that it occurred to me there might be some answers to my OP.

BZT
PS: On my redundant "Christmas wrapping" of variable values ---
Quote:
Originally Posted by grail
Hey BZT

I know you're new to some of this, but I also think some of what I am about to say has been covered before:
Code:
while read 'line'
Quotes not required.
Code:
t="$line"
#and
n=$(basename "$t")
Why bother if you do not do anything else to 'line'
I could argue adaptability with earlier BASH builds, but there I'd be dancing in the dark I know. Does it slow a script down considerably for the shell to have to look back one more instance to find the real, original value assigned to a variable? If so, I'll wean myself off this double-dipping ASAP. If not, then I hardly see anything harmful in continuing.
*-bee-zed-*

Last edited by SilversleevesX; 07-25-2010 at 07:10 AM. Reason: Some comments/commentary added.
 
Old 07-25-2010, 07:46 AM   #9
grail
Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 7,424

Rep: Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876Reputation: 1876
Quote:
If not, then I hardly see anything harmful in continuing
Harmful?? Not at all, but I guess once you get onto larger scripts It can make it difficult if you have to track back yourself manually through
2 or 3 layers of variable setting to only find nothing is being changed or used by the extra variables.
 
1 members found this post helpful.
Old 03-10-2014, 02:43 AM   #10
SilversleevesX
Member
 
Registered: May 2009
Posts: 181
Blog Entries: 9

Original Poster
Rep: Reputation: 15
And the BASH 4.x solution...

cat your ASCII text file (the list) and add
zip -@ foo.zip

This may have been supported in previous BASH versions. I took it from the man page for the zip version running in my current install of Cygwin, which is running 4.1.10(4)-release (i686-pc-cygwin).

BZT
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Archive manager not recognising ZIP files - suggests it's a text file. AnanthaP Ubuntu 4 12-27-2007 07:10 AM
Script to Add files to a Zip archive with out unzipping it !! prashix Programming 12 10-23-2007 09:19 AM
Zip gives error "-bash: /usr/bin/zip: Argument list too long" konathamsrinu Programming 3 07-06-2006 10:08 AM
How to Unzip 2 Zip files through one single command sumitarun Linux - Newbie 2 03-23-2005 08:00 AM
What Are archive.zip files? kajensen Linux - General 1 09-21-2003 04:00 AM


All times are GMT -5. The time now is 09:00 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration