[SOLVED] bash script problem -- pass complex *one-liner* to a variable

RandyTech · 01-25-2013, 06:15 AM

I found a one-liner on another forum:
http://superuser.com/questions/32183...es-recursively
This one-liner used here to illustrate the syntax problem I encounter with many so-called one-liners ... it does exactly what I need (almost). First problem I encountered (and not so significant to me per say), it does not work for me at line command or inside a bash script as a "one-liner" but works fine inside a script (where I need it) if I break it down as seen in the following code sample.

The real main problem I encounter is I cannot get the results *into a variable*. Tried every angle I can think of and googled it to death. I'm stuck. Can anyone offer proper syntax to place output from this snippit into a variable:

Code:

find /etc/sysconfig/ -type d | while read dir
do
  echo "$dir" : $(find "$dir" -type f | wc -l)
done | sort -k2 -t ':' -n |head -n 1

If it would work, I imagine the one-liner would look something like this:

Code:

MyVar=`find /etc/sysconfig/ -type d | while read dir ; do ; echo "$dir" : $(find "$dir" -type f | wc -l) ; done | sort -k2 -t ':' -n |head -n 1`

In the above one-liner (and in a perfect world), the "MyVar" variable would give me the path to the directory with the fewest file count. I don't need the one-liner to work as long as I can get the same results into "MyVar".

The true objective to this post is looking for insight on this *complex one-liner* issue -- not being able to figure out proper syntax to get the results into a variable. I encounter this same problem with many so called one-liners -- they give the desired results but how to get those results into a variable?

PS: Yes I know the single path returned in my sample snippet is not the *only* empty directory. That is not a concern. The point to the snippit is to find *at least one* path with the fewest files, regardless how many other paths may have the same file count.

pan64 · 01-25-2013, 06:27 AM

this worked for me as one-liner and script too. bash 3.2.51(1). Which shell did you try?

RandyTech · 01-25-2013, 07:28 AM

Hi pan64 and thanks much for your reply.
This is running in a Centos 4.x machine using /bin/bash

regardless... any idea on syntax, how to get the results into a variable?

konsolebox · 01-25-2013, 07:29 AM

It's not even a one-liner IMO. It's just a condensed set of multiple instructions.

Anyway I think you could change that to:

Code:

MyVar=$(find /etc/sysconfig/ -type d | while read dir; do echo "$dir:$(find "$dir" -type f | wc -l)"; done | sort -k2 -t: -n | head -n1 | cut -f1 -d:)

Certainly that could be simplified but it depends on what you really want to do noting /etc/sysconfig seems specific.

pan64 · 01-25-2013, 07:30 AM

instead, I would use a simple perl script.

RandyTech · 01-25-2013, 07:45 AM

bash script is a big stretch for me, but mostly just same as linux line command so I get by. I know nothing about perl.

RandyTech · 01-25-2013, 08:05 AM

Hi konsolebox and thanks many times over for your example!!!

Quote:

Originally Posted by konsolebox

It's not even a one-liner IMO. It's just a condensed set of multiple instructions.

Anyway I think you could change that to:

Code:

MyVar=$(find /etc/sysconfig/ -type d | while read dir; do echo "$dir:$(find "$dir" -type f | wc -l)"; done | sort -k2 -t: -n | head -n1 | cut -f1 -d:)

Certainly that could be simplified but it depends on what you really want to do noting /etc/sysconfig seems specific.

With that I was able to spot possible syntax errors in the original one-liner and get it working as line command too:

Code:

find /etc/sysconfig/ -type d | while read dir; do echo "$dir:$(find "$dir" -type f | wc -l)" ; done | sort -k2 -t ':' -n |head -n 1

And with that, ultimately get the original one-liner working inside my script file to pass results into the variable:

Code:

MyVar=$(find /etc/sysconfig/ -type d | while read dir; do echo "$dir:$(find "$dir" -type f | wc -l)" ; done | sort -k2 -t ':' -n |head -n 1)

Believe me, I tried putting the whole line inside $(code) but no joy there. I tried $((code)) and ($(code)), and even replace with variatins using square brackets.

Any, marking this issue [SOLVED] and thanks again for your help!!

RandyTech · 01-25-2013, 08:16 AM

Quote:

Originally Posted by konsolebox

It's not even a one-liner IMO. It's just a condensed set of multiple instructions.

Its a grey area I guess, and I get what you are saying so I have to agree in some part. If it can legitimately go onto a single command line at the system prompt and complete a task (simple or complex) without further user action then I have to classify that as a one-liner, even if it is composed of multiple instructions.

RandyTech · 01-25-2013, 08:22 AM

Quote:

Originally Posted by konsolebox

... it depends on what you really want to do noting /etc/sysconfig seems specific.

In case you are curious, I knew that path was generic to all *nux systems with multiple sub-directories, so safe to use it in my sample code and expect anyone would get something more illustrative than file not found error.

konsolebox · 01-25-2013, 09:36 AM

I see. Anyhow welcome

David the H. · 01-27-2013, 12:29 PM

1) When you have complex commands it might help to set them up in a function first, and capture the results of that.

2) $(..) is highly recommended over `..`. It's easier to read and nests better. As far as I'm concerned, nobody using a modern shell should ever need to touch the backtick key.

3) In bash, process substitution is usually preferred over piping something into a loop, as it avoids the variable scoping issue.

4) When operating on file lists produced by find, you should generally use the -print0 null separator option (and the corresponding input options in the receiving commands).

4) Also, for commands that produce multiple lines of output, you should usually consider storing them in an array instead. The new mapfile built-in makes this easy, although it doesn't work with null separators. That would require another read loop.

5) The second find in the sample code could be replaced with a simple subloop (run in a subshell to avoid having to reset the variables).

Code:

dircount() {

    local dname fname fnum
    while IFS='' read -r -d '' dname; do
        (
          cd "$dname"
	  for fname in * ; do
	      [[ -f "$fname" ]] && (( fnum++ ))
          done
	  printf '%d : %s\n' "$fnum" "$dname"
        )
    done < <( find "$1" -type d -print0 ) | sort -k1,1n

}

mapfile -t myArray < <( dircount /etc/sysconfig )

echo "${myArray[0]}"             #show the smallest entry
echo "${myArray[-1]}"            #show the largest entry
printf '%s\n' "${myArray[@]}"    #list all entries, one per line

It's a bit more code, but I think it's cleaner overall. If I was really up to it I could've also replaced the sort command with a sorting loop as well, but why bother reinventing the wheel?

RandyTech · 01-27-2013, 04:35 PM

>>> ENTER THE MASTER >>>

Quote:

Originally Posted by David the H.

1) ... might help to set them up in a function first ...

Wow David -- They don't label you "guru" for nothing -- Thanks!!

You lost me starting with the first comment and stayed pretty consistent that way to the end. I might have a chance spotting a syntax issue, and I can usually break these so called one-liners down to individual command lines to help figure them out (sometimes not), but total reconstruction with alternative command set and and and ... whaaaah. You are sooo over my head.

That's ok though. I appreciate the effort behind the added commentary even if I don't understand. Mostly because I know my subject line should be luring enough to bring users more advanced than myself into the thread to feast on your illustrious post. Well, that is, hopefully they don't get thrown by the "go use perl instead" comment and lose all interest.

Oh, I did understand most the comments tucked away there in your link regarding $(...) so that should come in handy to know. Really like the way you took time to provide topical comments and links. Thanks again!! Thumbs up

David the H. · 01-27-2013, 05:01 PM

Heheh. Sorry if I threw too much at you at once. Just take it slow and work your way through it at your own pace. Read the links I gave. It really isn't that difficult once you start to become familiar with it.

If you haven't seen it yet, take some time to read through the http://mywiki.wooledge.org/BashGuide from the same site. It should give you a good grounding in all the basics of shell syntax.

And if you have any specific questions, just ask!