[SOLVED] bash script -What am I doing wrong?

steponas · 12-28-2022, 02:56 PM

Hello everyone, I have a task :

Quote:

Create a directory "work". Copy all files from the "/etc" directory to the directory with the suffix ".conf" (including files that
are in subdirectories). Print the 5 filenames that have the most lines and archive them in the "work" directory.

I have written half of the bash script, but I do not know how to write the other half corectly. I do not know what is wrong. Maybe you could help me?

Code:

#!/bin/bash
rm -rf work
mkdir work
cd work
for x in $(find /etc -name "*.conf")
do 
cp $x work
done

and this is the other half which I do not understand how to write, I tried to write it, but that do not work:

Code:

wc -l work | sort -n | tail -5 
tar -czvf work.tar.gz work

goumba · 12-28-2022, 05:23 PM

Code:

wc -l work

You should be getting an error there.

Code:

wc: work: Is a directory

This is what you want:

Code:

wc -l work/*

Also, this is just an aesthetic thing, and as I would prefer it:

Code:

sort -rn | head -n 5

This will sort in reverse numeric order, then give the five files with the most lines, just as your example. However, this will display in descending order, with the file with the most lines at the top.

Also:

Code:

for x in $(find /etc -name "*.conf")
do 
cp $x work
done

can be shortened:

Code:

find /etc -name '*.conf' -exec cp -t work '{}' \+

or

Code:

find /etc -name '*.conf' -exec cp '{}' work \;

MadeInGermany · 12-29-2022, 05:01 AM

The find -exec is not only shorter, it is safe regarding special characters in file names.
for x in $( ) splits on whitespace; an embedded space is not recognized as such. Also the shell tries to expand on special characters like *

jmgibson1981 · 12-29-2022, 07:39 AM

One other aesthetic thing. Maybe it's just me but I like to be able to clearly see the sections in my scripts.

Code:

#!/bin/bash

rm -rf work
mkdir work
cd work
for x in $(find /etc -name "*.conf") ; do 
  cp $x work
done

Indentation makes things far more readable in general. At a glance you can see the loop instead of having to find it line by line. Again... maybe it's just me. The bigger the script the more helpful it is to keep track of where you are.

steponas · 12-29-2022, 08:27 AM

Thank you very much!

rnturn · 12-29-2022, 10:40 AM

Quote:

Originally Posted by steponas

Hello everyone, I have a task :

I have written half of the bash script, but I do not know how to write the other half corectly. I do not know what is wrong. Maybe you could help me?

Code:

#!/bin/bash
rm -rf work
mkdir work
cd work
for x in $(find /etc -name "*.conf")
do 
cp $x work
done

and this is the other half which I do not understand how to write, I tried to write it, but that do not work:

Code:

wc -l work | sort -n | tail -5 
tar -czvf work.tar.gz work

What are you intending to do with the wc|sort|tail pipeline? All that does is display five files to STDOUT. Plus... you should be getting an error message when executing "wc -l work" (i.e., "wc: work is a directory"). Try appending a slash ("/"). But you'll still have to find a way to use those five filenames---'tar' isn't going limit what it adds to the archive the way you've written whose two lines.

I'm not sure if what you've done is exactly a solution to the problem statement. You're grabbing ALL of the '.conf' files under /etc and copying them to the work/ directory. That leaves a lot of extra files laying around afterward. What I was thinking of was:

Code:

$ mkdir WORK
$ cp $( find /etc -type f -iname '*.conf' -exec wc -l {} 2>/dev/null \; | sort -n | tail -5 | awk '{print $2}' ) WORK/

That leaves just the five files in the WORK directory. (Note: I redirect error messages generated as part of the '-exec' command to /dev/null---running "sudo find ..." would make that unnecessary.) Then your tar command will only include the five files. Again, a trailing slash after the work directory name is (probably) never a bad idea.

HTH...

Tip 'o' the hat to goumba. We seem to have been thinking along the same lines (I failed to click Submit last night).

computersavvy · 12-29-2022, 11:44 AM

That is a good solution, but has one flaw in my testing.
What happens if there are multiple files with the same name in different subdirectories?

Code:

$ cp $( find /etc -type f -iname '*.conf' -exec wc -l {} 2>/dev/null \; | sort -n | awk '{print $2}' ) WORK/
cp: will not overwrite just-created 'WORK/netlib.conf' with '/etc/flexiblasrc.d/netlib.conf'
cp: will not overwrite just-created 'WORK/bacula.conf' with '/etc/logwatch/conf/services/bacula.conf'
cp: will not overwrite just-created 'WORK/dnf.conf' with '/etc/dnf/dnf.conf'
cp: will not overwrite just-created 'WORK/im-multipress.conf' with '/etc/gtk-3.0/im-multipress.conf'
cp: will not overwrite just-created 'WORK/fedora-workstation.conf' with '/etc/anaconda/profile.d/fedora-workstation.conf'
cp: will not overwrite just-created 'WORK/mlx4.conf' with '/etc/rdma/mlx4.conf'
cp: will not overwrite just-created 'WORK/client.conf' with '/etc/pulse/client.conf'
cp: will not overwrite just-created 'WORK/java.conf' with '/etc/abrt/plugins/java.conf'
cp: will not overwrite just-created 'WORK/ipsec.conf' with '/etc/ipsec.conf'
cp: will not overwrite just-created 'WORK/openssl.conf' with '/etc/xrdp/openssl.conf'
cp: will not overwrite just-created 'WORK/system.conf' with '/etc/systemd/system.conf'
cp: will not overwrite just-created 'WORK/user.conf' with '/etc/clamav-unofficial-sigs/user.conf'
cp: will not overwrite just-created 'WORK/dnsmasq.conf' with '/etc/dnsmasq.conf'

Would it not be better to store the full path (minus the leading / or /etc) as the file name saved so the output shows exactly which files are meeting the selected criteria and where they are located. The find command returns the full path so saving it would be relatively simple.

This may not be obvious if 2 different files with the same name are of differing sizes and one shows up in the output of tail without the full path to that file.

Note that the only thing I removed from your pipeline was the tail command.

Also, since this command was run without root privileges it will never return the .conf files that are in protected areas such as /etc/wireguard or /etc/sssd or others that are only accessible by root. It cannot even read or copy the many .conf files in /etc that are only root read-write such as /etc/aide.conf or /etc/freshclam.conf. Thus the command (both cp and find) must be run as root or with sudo to even have a chance of being 100% accurate.