LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 09-30-2007, 12:05 AM   #1
h/w
Senior Member
 
Registered: Mar 2003
Location: New York, NY
Distribution: Debian Testing
Posts: 1,286

Rep: Reputation: 45
`wc`, `xargs`


why are the following two different -

Code:
find . -type d | xargs -t -n1 ls
and

Code:
find . -type d | xargs -t -n1 wc -c
The first one runs the `ls` on each argument passed (in this case a line comprising a string).
The `wc` however does not run as I expected.

So, I tried something simpler like
Code:
wc -m 'foo bar'
I don't get the character count as I expected. What am I getting wrong here?
The `wc` manpage notes that stdin is read in the absence of a filename, so this must be something real minor that I'm getting wrong?
Thanks.
 
Old 09-30-2007, 12:31 AM   #2
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Code:
find . -type d | xargs -t -n1 wc -c
It seems you're looking for directories, here (-type d). Indeed, doing a word count of directories may bring to unexpected results!
 
Old 09-30-2007, 12:58 AM   #3
h/w
Senior Member
 
Registered: Mar 2003
Location: New York, NY
Distribution: Debian Testing
Posts: 1,286

Original Poster
Rep: Reputation: 45
Quote:
Originally Posted by colucix View Post
Code:
find . -type d | xargs -t -n1 wc -c
It seems you're looking for directories, here (-type d). Indeed, doing a word count of directories may bring to unexpected results!
I'm trying to get the number of characters in each line the find returns. I thought using -n1 with xargs would pass wc each line, and it would do the rest as it's piped.

Which brings me to a related question. How can we use wc to run on a string on the cli? For eg: wc -c "foo bar"? I know we could do a wc -c << EOF, and then terminate it for the result, but is there another way?
This thing's so basic, I'm surprised at my inability. heh.
 
Old 09-30-2007, 02:13 AM   #4
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
Ok, sorry. Now it is clear what you are trying to accomplish. The wc command is suited to work on files, not strings. You may workaround this behaviour in different ways, but it would be better using shell, as in
Code:
find . -type d | xargs -n1 expr length
 
Old 09-30-2007, 08:11 AM   #5
carl.waldbieser
Member
 
Registered: Jun 2005
Location: Pennsylvania
Distribution: Kubuntu
Posts: 197

Rep: Reputation: 32
Quote:
Originally Posted by colucix View Post
Ok, sorry. Now it is clear what you are trying to accomplish. The wc command is suited to work on files, not strings. You may workaround this behaviour in different ways, but it would be better using shell, as in
Code:
find . -type d | xargs -n1 expr length
I found it to be faster for many directories to use awk:
Code:
$ find . -type d | wc -l
9807
$ time find . -type d | awk -F '' '{print NF;}'
real    0m0.759s
user    0m0.252s
sys     0m0.448s
$ time find . -type d | xargs -n1 expr length
real    0m13.898s
user    0m2.680s
sys     0m9.049s
I am guessing this is because xargs is starting a new "expr" subprocess for every directory name.
 
Old 09-30-2007, 09:17 AM   #6
h/w
Senior Member
 
Registered: Mar 2003
Location: New York, NY
Distribution: Debian Testing
Posts: 1,286

Original Poster
Rep: Reputation: 45
Quote:
Originally Posted by colucix View Post
Ok, sorry.
Oh, no. There wasn't anything you said earlier that you had to say sorry for. Thanks for your inputs.
Quote:
Originally Posted by colucix View Post
Now it is clear what you are trying to accomplish. The wc command is suited to work on files, not strings. You may workaround this behaviour in different ways, but it would be better using shell, as in
Code:
find . -type d | xargs -n1 expr length
`expr` won't cut it. I need to use `wc` ONLY because I can't think of any other way to go ahead after it.

The manpage for `echo` has an example for finding the depth of a file using `echo $PWD | tr / ' ' | wc -w`. This is precisely what I want to do, but on a dir tree.
So, I tried `find . -type d | tr / ' '`. This would give us, in a line of words, the list of all dirs.

The commands I'm trying on are POSIX versions, and don't have the same kind of options as many of us might have. For eg: there's a %d directive in `find` in my version, that lists the depth. There's the print0 switch to null-terminate. But I don't have these on the version I'm trying to work on.
 
Old 09-30-2007, 09:19 AM   #7
h/w
Senior Member
 
Registered: Mar 2003
Location: New York, NY
Distribution: Debian Testing
Posts: 1,286

Original Poster
Rep: Reputation: 45
Quote:
Originally Posted by carl.waldbieser View Post
I found it to be faster for many directories to use awk:
I am guessing this is because xargs is starting a new "expr" subprocess for every directory name.
Right. Going the xargs way is indeed going to take more time. But I can't think of another way to pipe a set of words to tr or wc.
 
Old 09-30-2007, 12:53 PM   #8
h/w
Senior Member
 
Registered: Mar 2003
Location: New York, NY
Distribution: Debian Testing
Posts: 1,286

Original Poster
Rep: Reputation: 45
so, this is what I'm trying to do -
Code:
echo /foo/bar/moo/cows/ | tr / ' ' | wc -w
I'd like to do that on a whole directory. Something like:
Code:
find /some/dir -type f | tr / ' ' | xargs -i wc -w {}
Well, that's the idea, at least.
Anyone?
 
Old 09-30-2007, 03:15 PM   #9
colucix
Moderator
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,458

Rep: Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941Reputation: 1941
I think there are not many chances to achieve the result by means of the wc command. When you pipe the output from echo, or from any other command, wc treats the text as a whole, but if you pass a list of items wc interpret them as files to be processed. The wanted result can be obtained by cycling over the output of find, e.g.
Code:
for string in `find /some/dir -type f`
do
  echo $string | tr / " " | wc -w
done
The purpose to parse a text one line at a time is better achieved by awk, hence - following the suggestion from carl - you can obtain the depth of every file by
Code:
find /some/dir -type f | awk -F/ '{print NF - 1}'
By the way, I still have the feeling this is not the wanted result, but I'm a little confused now.
 
Old 09-30-2007, 03:35 PM   #10
carl.waldbieser
Member
 
Registered: Jun 2005
Location: Pennsylvania
Distribution: Kubuntu
Posts: 197

Rep: Reputation: 32
If I am reading your correctly, you want a count of all the directory components in a given hierarchy?

Code:
$ find . -print | awk -F '/' 'BEGIN {sum=0;} {sum += NF} END {print sum}'
I think that should be POSIX compatible.
 
Old 09-30-2007, 04:20 PM   #11
h/w
Senior Member
 
Registered: Mar 2003
Location: New York, NY
Distribution: Debian Testing
Posts: 1,286

Original Poster
Rep: Reputation: 45
Quote:
Originally Posted by colucix View Post
I think there are not many chances to achieve the result by means of the wc command. When you pipe the output from echo, or from any other command, wc treats the text as a whole, but if you pass a list of items wc interpret them as files to be processed. The wanted result can be obtained by cycling over the output of find, e.g.
Code:
for string in `find /some/dir -type f`
do
  echo $string | tr / " " | wc -w
done
The purpose to parse a text one line at a time is better achieved by awk, hence - following the suggestion from carl - you can obtain the depth of every file by
Code:
find /some/dir -type f | awk -F/ '{print NF - 1}'
By the way, I still have the feeling this is not the wanted result, but I'm a little confused now.
Thanks colucix.
I decided to go with -
Code:
for i in `find /some/path/ -type f`; do echo $i | tr / ' ' | wc -w; done
I still am unsure why I wasn't able to achieve the same with xargs. For eg:, if you use the -t switch, you can see the command you expect to run, but that wc call doesn't, for some reason, run as I wanted.

Thanks anyway.
 
Old 09-30-2007, 04:22 PM   #12
h/w
Senior Member
 
Registered: Mar 2003
Location: New York, NY
Distribution: Debian Testing
Posts: 1,286

Original Poster
Rep: Reputation: 45
Quote:
Originally Posted by carl.waldbieser View Post
If I am reading your correctly, you want a count of all the directory components in a given hierarchy?

Code:
$ find . -print | awk -F '/' 'BEGIN {sum=0;} {sum += NF} END {print sum}'
I think that should be POSIX compatible.
Thanks, Carl. Your line's POSIX compatible, yes. The problem was to get the depth of each file under a directory. With some help, I've found the way to do it as in my above post.

Thanks again.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
xargs yawe_frek Linux - Software 1 04-15-2007 04:26 PM
Xargs: Missing quote mellowjen Programming 5 02-27-2007 05:29 PM
xargs and my own script harmster Linux - Software 4 05-07-2005 10:35 AM
error with find . | xargs cbonar Linux - Newbie 7 12-09-2004 11:22 AM
xargs question reitzell Linux - Newbie 4 07-04-2004 02:36 AM


All times are GMT -5. The time now is 01:35 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration