[SOLVED] Possible to have Tar Command run on multiple files in a directory at one time?

JockVSJock · 06-14-2016, 09:11 AM

I'm monitoring a script that I wrote to tar up files on one server and then move them to an logging server. Want to make sure the script captures all output and we have match on both each day before I know its ok.

Tar can only be ran on one file at a time, correct?

Code:

[root@server jun]$ tar -tf *.tar.gz | wc -l
tar: server.aud-2016-06-06.tar.gz: Not found in archive
tar: server.aud-2016-06-07.tar.gz: Not found in archive
tar: server.aud-2016-06-08.tar.gz: Not found in archive
tar: server.aud-2016-06-09.tar.gz: Not found in archive
tar: server.aud-2016-06-10.tar.gz: Not found in archive
tar: server.aud-2016-06-11.tar.gz: Not found in archive
tar: server.aud-2016-06-12.tar.gz: Not found in archive
tar: server.aud-2016-06-13.tar.gz: Not found in archive
tar: 0
Error exit delayed from previous errors

I've actually coded up a for loop in bash where it will go thru all the files in the current directory and do the count and output to a txt file in /tmp.

Code:

#!/bin/bash

for i in *.tar.gz;
 do tar -tf $i | wc -l > /tmp/tar_server_output.txt ;
done

Here is the output...

Code:

64 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-05.tar.gz
60 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-06.tar.gz
55 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-07.tar.gz
56 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-08.tar.gz
48 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-09.tar.gz
42 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-10.tar.gz
41 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-11.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-12.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-13.tar.gz
6 /logdata/linux_servers/server/2016/jun/tar_count.sh

The numbers on the left hand side don't match if I were to run tar -tf (file name here) | wc -l.

So I'm not sure what I'm doing incorrectly here with the script Vs the command line.

rknichols · 06-14-2016, 10:13 AM

Quote:

Originally Posted by JockVSJock

Code:

#!/bin/bash

for i in *.tar.gz;
 do tar -tf $i | wc -l > /tmp/tar_server_output.txt ;
done

Here is the output...

Code:

64 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-05.tar.gz
60 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-06.tar.gz
55 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-07.tar.gz
56 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-08.tar.gz
48 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-09.tar.gz
42 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-10.tar.gz
41 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-11.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-12.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-13.tar.gz
6 /logdata/linux_servers/server/2016/jun/tar_count.sh

That is not the output from that script. First, when reading from stdin "wc -l" will produce a line with just a single number. Second, that script overwrites the output file on each iteration of the loop, so all you would see is a single line with the last number.

To answer your original question, yes, tar can work with only a single archive file at a time. Any additional arguments will be taken as names to search for in that archive.

JockVSJock · 06-14-2016, 10:59 AM

So even from a for loop, I can't have loop on the tar -tf command and piping that output thru wc -l for each file?

I'm not familiar with any other way to do this either...

grail · 06-14-2016, 11:21 AM

Yes you can use it in the for loop, but you need to append to a file and not overwrite it each time.

I would add that another reason that the output shown does not come from the script provided is the last file does not even match the glob used (*.tar.gz), so even if it had worked that file
should never have been listed or tested with wc.

dab1414 · 06-14-2016, 11:38 AM

Quote:

Originally Posted by JockVSJock

Code:

#!/bin/bash

for i in *.tar.gz;
 do tar -tf $i | wc -l > /tmp/tar_server_output.txt ;
done

.

Try something like this

Code:

for i in `ls -1 *.tar.gz`;
do tar -tvf $i | wc -l >> /tmp/tar_server_output.txt;
done

grail · 06-14-2016, 12:01 PM

Quote:

Originally Posted by dab1414

Try something like this

Code:

for i in `ls -1 *.tar.gz`;
do tar -tvf $i | wc -l >> /tmp/tar_server_output.txt;
done

Actually, please do not try something like the above. ls should never be used to feed a for loop and the already utilised globbing is already superior.
Also, as explained rknichols, the output from piped data to wc -l will return only a number so the addition of verbose output will not help the required file name
appearing in the output

rknichols · 06-14-2016, 12:54 PM

Code:

for i in *.tar.gz
do echo "$(tar -tf "$i" | wc -l) $i"
done >/tmp/tar_server_output.txt

JockVSJock · 06-14-2016, 02:11 PM

Quote:

Originally Posted by grail

Actually, please do not try something like the above. ls should never be used to feed a for loop and the already utilised globbing is already superior.

Too late...

Code:


[root@server jun]$ ./tar_count3.sh
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 50899: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 7: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
10: Invalid argument
tar: 10\:30: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 49663: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 7: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
10: Invalid argument
tar: 10\:30: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 58694: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 8: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
11: Invalid argument
tar: 11\:38: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 48311: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 9: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
12: Invalid argument
tar: 12\:48: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 47484: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 10: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now

JockVSJock · 06-14-2016, 03:04 PM

Quote:

Originally Posted by rknichols

Code:

for i in *.tar.gz
do echo "$(tar -tf "$i" | wc -l) $i"
done >/tmp/tar_server_output.txt

I take it the echo command will instead of doing one line, it will echo back all of the lines that the tar -tf "$i" is output for, correct?

rknichols · 06-14-2016, 04:31 PM

Each invocation of the echo command writes one line, and that line consists of the number produced by the tar and wc pipeline followed by the filename that was used. Run the script with "bash -x ..." to see what it is doing.

grail · 06-14-2016, 05:58 PM

I will add that the reason you got so many errors from the ls version was you used -l (el) as opposed to -1 (one) ... but it still not the recommended way because of word splitting.