LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-14-2016, 10:11 AM   #1
JockVSJock
Senior Member
 
Registered: Jan 2004
Location: SATX
Distribution: RHEL/CentOS
Posts: 1,226
Blog Entries: 4

Rep: Reputation: 132Reputation: 132
Possible to have Tar Command run on multiple files in a directory at one time?


I'm monitoring a script that I wrote to tar up files on one server and then move them to an logging server. Want to make sure the script captures all output and we have match on both each day before I know its ok.

Tar can only be ran on one file at a time, correct?

Code:
[root@server jun]$ tar -tf *.tar.gz | wc -l
tar: server.aud-2016-06-06.tar.gz: Not found in archive
tar: server.aud-2016-06-07.tar.gz: Not found in archive
tar: server.aud-2016-06-08.tar.gz: Not found in archive
tar: server.aud-2016-06-09.tar.gz: Not found in archive
tar: server.aud-2016-06-10.tar.gz: Not found in archive
tar: server.aud-2016-06-11.tar.gz: Not found in archive
tar: server.aud-2016-06-12.tar.gz: Not found in archive
tar: server.aud-2016-06-13.tar.gz: Not found in archive
tar: 0
Error exit delayed from previous errors

I've actually coded up a for loop in bash where it will go thru all the files in the current directory and do the count and output to a txt file in /tmp.

Code:
#!/bin/bash

for i in *.tar.gz;
 do tar -tf $i | wc -l > /tmp/tar_server_output.txt ;
done
Here is the output...

Code:
64 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-05.tar.gz
60 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-06.tar.gz
55 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-07.tar.gz
56 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-08.tar.gz
48 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-09.tar.gz
42 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-10.tar.gz
41 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-11.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-12.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-13.tar.gz
6 /logdata/linux_servers/server/2016/jun/tar_count.sh

The numbers on the left hand side don't match if I were to run tar -tf (file name here) | wc -l.

So I'm not sure what I'm doing incorrectly here with the script Vs the command line.
 
Old 06-14-2016, 11:13 AM   #2
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 3,580

Rep: Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566
Quote:
Originally Posted by JockVSJock View Post
Code:
#!/bin/bash

for i in *.tar.gz;
 do tar -tf $i | wc -l > /tmp/tar_server_output.txt ;
done
Here is the output...

Code:
64 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-05.tar.gz
60 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-06.tar.gz
55 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-07.tar.gz
56 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-08.tar.gz
48 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-09.tar.gz
42 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-10.tar.gz
41 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-11.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-12.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-13.tar.gz
6 /logdata/linux_servers/server/2016/jun/tar_count.sh
That is not the output from that script. First, when reading from stdin "wc -l" will produce a line with just a single number. Second, that script overwrites the output file on each iteration of the loop, so all you would see is a single line with the last number.

To answer your original question, yes, tar can work with only a single archive file at a time. Any additional arguments will be taken as names to search for in that archive.
 
Old 06-14-2016, 11:59 AM   #3
JockVSJock
Senior Member
 
Registered: Jan 2004
Location: SATX
Distribution: RHEL/CentOS
Posts: 1,226
Blog Entries: 4

Original Poster
Rep: Reputation: 132Reputation: 132
So even from a for loop, I can't have loop on the tar -tf command and piping that output thru wc -l for each file?

I'm not familiar with any other way to do this either...
 
Old 06-14-2016, 12:21 PM   #4
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Yes you can use it in the for loop, but you need to append to a file and not overwrite it each time.

I would add that another reason that the output shown does not come from the script provided is the last file does not even match the glob used (*.tar.gz), so even if it had worked that file
should never have been listed or tested with wc.
 
Old 06-14-2016, 12:38 PM   #5
dab1414
Member
 
Registered: May 2011
Location: OK, USA
Distribution: Slackware 14.1_64
Posts: 70

Rep: Reputation: 50
Quote:
Originally Posted by JockVSJock View Post
Code:
#!/bin/bash

for i in *.tar.gz;
 do tar -tf $i | wc -l > /tmp/tar_server_output.txt ;
done
.
Try something like this
Code:
for i in `ls -1 *.tar.gz`;
do tar -tvf $i | wc -l >> /tmp/tar_server_output.txt;
done
 
Old 06-14-2016, 01:01 PM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Quote:
Originally Posted by dab1414 View Post
Try something like this
Code:
for i in `ls -1 *.tar.gz`;
do tar -tvf $i | wc -l >> /tmp/tar_server_output.txt;
done
Actually, please do not try something like the above. ls should never be used to feed a for loop and the already utilised globbing is already superior.
Also, as explained rknichols, the output from piped data to wc -l will return only a number so the addition of verbose output will not help the required file name
appearing in the output
 
Old 06-14-2016, 01:54 PM   #7
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 3,580

Rep: Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566
Code:
for i in *.tar.gz
do echo "$(tar -tf "$i" | wc -l) $i"
done >/tmp/tar_server_output.txt
 
1 members found this post helpful.
Old 06-14-2016, 03:11 PM   #8
JockVSJock
Senior Member
 
Registered: Jan 2004
Location: SATX
Distribution: RHEL/CentOS
Posts: 1,226
Blog Entries: 4

Original Poster
Rep: Reputation: 132Reputation: 132
Quote:
Originally Posted by grail View Post
Actually, please do not try something like the above. ls should never be used to feed a for loop and the already utilised globbing is already superior.
Too late...

Code:

[root@server jun]$ ./tar_count3.sh
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 50899: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 7: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
10: Invalid argument
tar: 10\:30: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 49663: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 7: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
10: Invalid argument
tar: 10\:30: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 58694: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 8: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
11: Invalid argument
tar: 11\:38: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 48311: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 9: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
12: Invalid argument
tar: 12\:48: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 47484: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 10: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
 
Old 06-14-2016, 04:04 PM   #9
JockVSJock
Senior Member
 
Registered: Jan 2004
Location: SATX
Distribution: RHEL/CentOS
Posts: 1,226
Blog Entries: 4

Original Poster
Rep: Reputation: 132Reputation: 132
Quote:
Originally Posted by rknichols View Post
Code:
for i in *.tar.gz
do echo "$(tar -tf "$i" | wc -l) $i"
done >/tmp/tar_server_output.txt

I take it the echo command will instead of doing one line, it will echo back all of the lines that the tar -tf "$i" is output for, correct?
 
Old 06-14-2016, 05:31 PM   #10
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 3,580

Rep: Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566Reputation: 1566
Each invocation of the echo command writes one line, and that line consists of the number produced by the tar and wc pipeline followed by the filename that was used. Run the script with "bash -x ..." to see what it is doing.
 
Old 06-14-2016, 06:58 PM   #11
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
I will add that the reason you got so many errors from the ls version was you used -l (el) as opposed to -1 (one) ... but it still not the recommended way because of word splitting.
 
  


Reply

Tags
bash loop, tar, wc -l


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] seeking safe way to run long (clock time) 'tar' command SaintDanBert Linux - Server 6 06-29-2016 05:24 AM
Tar multiple folder to multiple files to another directory and delete original folder zeroize Linux - Newbie 6 03-31-2016 08:36 PM
SCP - multiple files at a time and run in background jack2 Linux - General 1 03-31-2016 11:37 AM
tar doesn't compress .hidden files in the same directory it is run in? BassKozz Linux - Newbie 2 06-02-2009 05:46 PM
How do I decompress and desarchive multiple .tar.gz files to a single directory? Baldorg Linux - Software 7 11-27-2003 01:42 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 12:03 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration