LinuxQuestions.org - `wc`, `xargs`

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - `wc`, `xargs` (https://www.linuxquestions.org/questions/programming-9/%60wc%60-%60xargs%60-588318/)

h/w	09-30-2007 12:05 AM

`wc`, `xargs`

why are the following two different -

Code:

find . -type d | xargs -t -n1 ls

and

Code:

find . -type d | xargs -t -n1 wc -c

The first one runs the `ls` on each argument passed (in this case a line comprising a string).
The `wc` however does not run as I expected.

So, I tried something simpler like

Code:

wc -m 'foo bar'

I don't get the character count as I expected. What am I getting wrong here?
The `wc` manpage notes that stdin is read in the absence of a filename, so this must be something real minor that I'm getting wrong?
Thanks.

colucix

09-30-2007 12:31 AM

Code:

find . -type d | xargs -t -n1 wc -c

It seems you're looking for directories, here (-type d). Indeed, doing a word count of directories may bring to unexpected results! ;)

h/w	09-30-2007 12:58 AM

Quote:

Originally Posted by colucix (Post 2908030)

Code:

find . -type d | xargs -t -n1 wc -c

It seems you're looking for directories, here (-type d). Indeed, doing a word count of directories may bring to unexpected results! ;)

I'm trying to get the number of characters in each line the find returns. I thought using -n1 with xargs would pass wc each line, and it would do the rest as it's piped.

Which brings me to a related question. How can we use wc to run on a string on the cli? For eg: wc -c "foo bar"? I know we could do a wc -c << EOF, and then terminate it for the result, but is there another way?
This thing's so basic, I'm surprised at my inability. heh.

colucix

09-30-2007 02:13 AM

Ok, sorry. Now it is clear what you are trying to accomplish. The wc command is suited to work on files, not strings. You may workaround this behaviour in different ways, but it would be better using shell, as in

Code:

find . -type d | xargs -n1 expr length

carl.waldbieser

09-30-2007 08:11 AM

Quote:

Originally Posted by colucix (Post 2908065)

Code:

find . -type d | xargs -n1 expr length

I found it to be faster for many directories to use awk:

Code:

$ find . -type d | wc -l

9807

$ time find . -type d | awk -F '' '{print NF;}'

real    0m0.759s

user    0m0.252s

sys    0m0.448s

$ time find . -type d | xargs -n1 expr length

real    0m13.898s

user    0m2.680s

sys    0m9.049s

I am guessing this is because xargs is starting a new "expr" subprocess for every directory name.

h/w	09-30-2007 09:17 AM

Quote:

Originally Posted by colucix (Post 2908065)

Ok, sorry.

Oh, no. There wasn't anything you said earlier that you had to say sorry for. :) Thanks for your inputs.

Quote:

Originally Posted by colucix (Post 2908065)

Now it is clear what you are trying to accomplish. The wc command is suited to work on files, not strings. You may workaround this behaviour in different ways, but it would be better using shell, as in

Code:

find . -type d | xargs -n1 expr length

`expr` won't cut it. I need to use `wc` ONLY because I can't think of any other way to go ahead after it.

The manpage for `echo` has an example for finding the depth of a file using `echo $PWD | tr / ' ' | wc -w`. This is precisely what I want to do, but on a dir tree.
So, I tried `find . -type d | tr / ' '`. This would give us, in a line of words, the list of all dirs.

The commands I'm trying on are POSIX versions, and don't have the same kind of options as many of us might have. For eg: there's a %d directive in `find` in my version, that lists the depth. There's the print0 switch to null-terminate. But I don't have these on the version I'm trying to work on. :)

h/w	09-30-2007 09:19 AM

Quote:

Originally Posted by carl.waldbieser (Post 2908251)

I found it to be faster for many directories to use awk:
I am guessing this is because xargs is starting a new "expr" subprocess for every directory name.

Right. Going the xargs way is indeed going to take more time. But I can't think of another way to pipe a set of words to tr or wc.

h/w	09-30-2007 12:53 PM

so, this is what I'm trying to do -

Code:

echo /foo/bar/moo/cows/ | tr / ' ' | wc -w

I'd like to do that on a whole directory. Something like:

Code:

find /some/dir -type f | tr / ' ' | xargs -i wc -w {}

Well, that's the idea, at least.
Anyone?

colucix

09-30-2007 03:15 PM

I think there are not many chances to achieve the result by means of the wc command. When you pipe the output from echo, or from any other command, wc treats the text as a whole, but if you pass a list of items wc interpret them as files to be processed. The wanted result can be obtained by cycling over the output of find, e.g.

Code:

for string in `find /some/dir -type f`

do

  echo $string | tr / " " | wc -w

done

The purpose to parse a text one line at a time is better achieved by awk, hence - following the suggestion from carl - you can obtain the depth of every file by

Code:

find /some/dir -type f | awk -F/ '{print NF - 1}'

By the way, I still have the feeling this is not the wanted result, but I'm a little confused now.

carl.waldbieser

09-30-2007 03:35 PM

If I am reading your correctly, you want a count of all the directory components in a given hierarchy?

Code:

$ find . -print | awk -F '/' 'BEGIN {sum=0;} {sum += NF} END {print sum}'

I think that should be POSIX compatible.

h/w	09-30-2007 04:20 PM

Quote:

Originally Posted by colucix (Post 2908598)

Code:

for string in `find /some/dir -type f`

do

  echo $string | tr / " " | wc -w

done

The purpose to parse a text one line at a time is better achieved by awk, hence - following the suggestion from carl - you can obtain the depth of every file by

Code:

find /some/dir -type f | awk -F/ '{print NF - 1}'

By the way, I still have the feeling this is not the wanted result, but I'm a little confused now.

Thanks colucix.
I decided to go with -

Code:

for i in `find /some/path/ -type f`; do echo $i | tr / ' ' | wc -w; done

I still am unsure why I wasn't able to achieve the same with xargs. For eg:, if you use the -t switch, you can see the command you expect to run, but that wc call doesn't, for some reason, run as I wanted.

Thanks anyway.

h/w	09-30-2007 04:22 PM

Quote:

Originally Posted by carl.waldbieser (Post 2908605)

If I am reading your correctly, you want a count of all the directory components in a given hierarchy?

Code:

$ find . -print | awk -F '/' 'BEGIN {sum=0;} {sum += NF} END {print sum}'

I think that should be POSIX compatible.

Thanks, Carl. Your line's POSIX compatible, yes. The problem was to get the depth of each file under a directory. With some help, I've found the way to do it as in my above post.

Thanks again.

All times are GMT -5. The time now is 11:51 AM.