LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Possible to have Tar Command run on multiple files in a directory at one time? (https://www.linuxquestions.org/questions/linux-newbie-8/possible-to-have-tar-command-run-on-multiple-files-in-a-directory-at-one-time-4175582243/)

JockVSJock 06-14-2016 09:11 AM

Possible to have Tar Command run on multiple files in a directory at one time?
 
I'm monitoring a script that I wrote to tar up files on one server and then move them to an logging server. Want to make sure the script captures all output and we have match on both each day before I know its ok.

Tar can only be ran on one file at a time, correct?

Code:


[root@server jun]$ tar -tf *.tar.gz | wc -l
tar: server.aud-2016-06-06.tar.gz: Not found in archive
tar: server.aud-2016-06-07.tar.gz: Not found in archive
tar: server.aud-2016-06-08.tar.gz: Not found in archive
tar: server.aud-2016-06-09.tar.gz: Not found in archive
tar: server.aud-2016-06-10.tar.gz: Not found in archive
tar: server.aud-2016-06-11.tar.gz: Not found in archive
tar: server.aud-2016-06-12.tar.gz: Not found in archive
tar: server.aud-2016-06-13.tar.gz: Not found in archive
tar: 0
Error exit delayed from previous errors


I've actually coded up a for loop in bash where it will go thru all the files in the current directory and do the count and output to a txt file in /tmp.

Code:


#!/bin/bash

for i in *.tar.gz;
 do tar -tf $i | wc -l > /tmp/tar_server_output.txt ;
done

Here is the output...

Code:


64 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-05.tar.gz
60 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-06.tar.gz
55 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-07.tar.gz
56 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-08.tar.gz
48 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-09.tar.gz
42 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-10.tar.gz
41 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-11.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-12.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-13.tar.gz
6 /logdata/linux_servers/server/2016/jun/tar_count.sh


The numbers on the left hand side don't match if I were to run tar -tf (file name here) | wc -l.

So I'm not sure what I'm doing incorrectly here with the script Vs the command line.

rknichols 06-14-2016 10:13 AM

Quote:

Originally Posted by JockVSJock (Post 5560741)
Code:


#!/bin/bash

for i in *.tar.gz;
 do tar -tf $i | wc -l > /tmp/tar_server_output.txt ;
done

Here is the output...

Code:


64 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-05.tar.gz
60 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-06.tar.gz
55 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-07.tar.gz
56 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-08.tar.gz
48 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-09.tar.gz
42 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-10.tar.gz
41 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-11.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-12.tar.gz
38 /logdata/linux_servers/server/2016/jun/server.aud-2016-06-13.tar.gz
6 /logdata/linux_servers/server/2016/jun/tar_count.sh


That is not the output from that script. First, when reading from stdin "wc -l" will produce a line with just a single number. Second, that script overwrites the output file on each iteration of the loop, so all you would see is a single line with the last number.

To answer your original question, yes, tar can work with only a single archive file at a time. Any additional arguments will be taken as names to search for in that archive.

JockVSJock 06-14-2016 10:59 AM

So even from a for loop, I can't have loop on the tar -tf command and piping that output thru wc -l for each file?

I'm not familiar with any other way to do this either...

grail 06-14-2016 11:21 AM

Yes you can use it in the for loop, but you need to append to a file and not overwrite it each time.

I would add that another reason that the output shown does not come from the script provided is the last file does not even match the glob used (*.tar.gz), so even if it had worked that file
should never have been listed or tested with wc.

dab1414 06-14-2016 11:38 AM

Quote:

Originally Posted by JockVSJock (Post 5560741)
Code:


#!/bin/bash

for i in *.tar.gz;
 do tar -tf $i | wc -l > /tmp/tar_server_output.txt ;
done

.

Try something like this
Code:

for i in `ls -1 *.tar.gz`;
do tar -tvf $i | wc -l >> /tmp/tar_server_output.txt;
done


grail 06-14-2016 12:01 PM

Quote:

Originally Posted by dab1414 (Post 5560803)
Try something like this
Code:

for i in `ls -1 *.tar.gz`;
do tar -tvf $i | wc -l >> /tmp/tar_server_output.txt;
done


Actually, please do not try something like the above. ls should never be used to feed a for loop and the already utilised globbing is already superior.
Also, as explained rknichols, the output from piped data to wc -l will return only a number so the addition of verbose output will not help the required file name
appearing in the output

rknichols 06-14-2016 12:54 PM

Code:

for i in *.tar.gz
do echo "$(tar -tf "$i" | wc -l) $i"
done >/tmp/tar_server_output.txt


JockVSJock 06-14-2016 02:11 PM

Quote:

Originally Posted by grail (Post 5560810)
Actually, please do not try something like the above. ls should never be used to feed a for loop and the already utilised globbing is already superior.

Too late...

Code:



[root@server jun]$ ./tar_count3.sh
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 50899: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 7: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
10: Invalid argument
tar: 10\:30: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 49663: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 7: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
10: Invalid argument
tar: 10\:30: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 58694: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 8: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
11: Invalid argument
tar: 11\:38: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 48311: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 9: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
12: Invalid argument
tar: 12\:48: Cannot open: Input/output error
tar: Error is not recoverable: exiting now
tar: -rw-------: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 1: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: oracle: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 47484: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: Jun: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
tar: 10: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now


JockVSJock 06-14-2016 03:04 PM

Quote:

Originally Posted by rknichols (Post 5560835)
Code:

for i in *.tar.gz
do echo "$(tar -tf "$i" | wc -l) $i"
done >/tmp/tar_server_output.txt



I take it the echo command will instead of doing one line, it will echo back all of the lines that the tar -tf "$i" is output for, correct?

rknichols 06-14-2016 04:31 PM

Each invocation of the echo command writes one line, and that line consists of the number produced by the tar and wc pipeline followed by the filename that was used. Run the script with "bash -x ..." to see what it is doing.

grail 06-14-2016 05:58 PM

I will add that the reason you got so many errors from the ls version was you used -l (el) as opposed to -1 (one) ... but it still not the recommended way because of word splitting.


All times are GMT -5. The time now is 06:57 PM.