SlackBuilds.org buildscript templates use find | xargs, not find -exec. Why?

Lockywolf · 05-01-2019, 09:46 PM

I am not sure who to address this question, but using find -exec removes the necessity to use -print 0, also saves one pipe, and is generally considered more robust.

Is there a particular reason why the template suggests find | xargs ?

https://slackbuilds.org/templates/cmake-template.SlackBuild

Code:

find $PKG -print0 | xargs -0 file | grep -e "executable" -e "shared object" | grep ELF \
  | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null || true

dugan · 05-01-2019, 09:57 PM

I assume the specific reason is that it's what Slackware's build scripts used before SBo became active.

ponce · 05-01-2019, 11:52 PM

Quote:

Originally Posted by dugan

I assume the specific reason is that it's what Slackware's build scripts used before SBo became active.

I also suppose it's because of this...

Quote:

Originally Posted by Lockywolf

I am not sure who to address this question, but using find -exec removes the necessity to use -print 0, also saves one pipe, and is generally considered more robust.

Is there a particular reason why the template suggests find | xargs ?

https://slackbuilds.org/templates/cmake-template.SlackBuild

Code:

find $PKG -print0 | xargs -0 file | grep -e "executable" -e "shared object" | grep ELF \
  | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null || true

I guess the reason this syntax is used is because way it can process both executables and shared libraries in one run.

Lockywolf · 05-02-2019, 12:00 AM

What does find have to do with executables?

Code:

find $pkg -exec file {}\; | grep ...

Should be able to do that too.

ponce · 05-02-2019, 12:03 AM

Quote:

Originally Posted by Lockywolf

What does find have to do with executables?

Code:

find $pkg -exec file {}\; | grep ...

Should be able to do that too.

libraries not necessarily have the executable flag set.

Lockywolf · 05-02-2019, 12:06 AM

Where did you find any mentions of the execute flag?

Quote:

-exec command ;
Execute command; true if 0 status is returned. All following arguments to find are taken to be arguments to the command until an argument consisting of ';' is encountered. The string '{}' is replaced by the current file name being processed everywhere it occurs in the arguments to the command, not just in arguments where it is alone, as in some versions of find. Both of these constructions might need to be escaped (with a '\') or quoted to protect them from expansion by the shell. See the EXAMPLES section for examples of the use of the -exec option. The specified command is run once for each matched file. The command is executed in the starting directory. There are unavoidable security problems surrounding use of the -exec action; you should use the -execdir option instead.

ponce · 05-02-2019, 12:10 AM

sorry, ignore my above posts, I just woke up and I'm still sleeping...

gus3 · 05-02-2019, 12:27 AM

The "find ... -exec something '{}' \;" idiom runs the "something" command once for every item found. Yet another gotcha has to do with arcane quoting rules concerning braces and the semicolon, which always have meanings for shell scripts/commands.

Using the "find ... -print0 | xargs -0 something" idiom is much more considerate to the system. The "xargs" command knows the maximum (safe) length of a command line, so it can execute the "something" command, via fork()/exec(), the minimum number of times to process all the results from the "find" command. Put simply, is it more efficient to delete a hundred files one at a time, or 20 at a time? One at a time involves 100 invocations of fork()/exec(); 20 at a time involves just 5.

--------

Astute readers may note that the "find" command has an option to use a plus-sign instead of a semicolon in the command line passed to the "-exec" option. That is, "find ... -exec something '{}' \+". It's supposed to build up a compatible list of command arguments, in the style of xargs. Besides the arcane quoting rules cited above, there's also the fact that the "\+" option was added to "-exec" in 2005. The implementation was imperfect; AFAICT, it hasn't been fixed yet. (See findutils-4.6.0-exec-args.patch.gz in the Slackware-current source tree.)

Other astute readers may point out that "xargs permits similar brace-pair substitutions!". But "xargs" builds a command's argument list by default, which is its whole raison d'être; the "-exec" option to "find" is exactly that, an option.

--------

Lockywolf, the example you quote in your original post is a good demonstration of Unix philosophy's benefits. "Do one thing, and do it well." That pipe sequence of six commands clearly defines each of the six steps for finding the files containing executable object code, then peeling off the unneeded ELF stuff, in order to build a usable (but terse) Slackware package. It saves space, in both installed Slackware systems, and Slackware package mirrors. The pipe sequence builds up the list of files to pass to "strip --strip-unneeded", in the most efficient way currently possible.