LinuxQuestions.org - Script to get/set dependencies/required packages in a slackbuild

- Slackware (https://www.linuxquestions.org/questions/slackware-14/)

- - Script to get/set dependencies/required packages in a slackbuild (https://www.linuxquestions.org/questions/slackware-14/script-to-get-set-dependencies-required-packages-in-a-slackbuild-4175563496/)

Quote:

Originally Posted by franzen (Post 5481371)

cxxlibs seem to be removed from -current, these libs seem to be added aaa_elflibs.
One of the ideas of aaa_elflibs is to avoid upgrade-issues, so old libraries will have
to there, also if newer versions gonna be added. It will grow if is has to contain "everything", which isn't very suitable for a minimal install i think.

thanks for the info
if cxxlibs will not exist anymore since they are merged into aaa_elflibs than they will not be detected

and for elflibs, I think I will add a --ignoredep and possible --adddep option the the command line options,
than the user can decide if aaa_elflibs, or something else, is wanted or not.

Hi franzen,

if you are still intrested in improving your get_deps.sh script... i have tested it against 1000 packages and have a few comments:

From your script:

Code:

# find all files relevant for hard elf-dependencies

find . | xargs file | fgrep -w ELF | cut -d : -f1 > $internal

I found only one package (maybe there will be others) where one header file (real ASCII file) was false detected as ELF dependency because the magic bytes matched a ELF library and the file contained the word ELF in the first text line. It was a header file from the LLVM package and XARGS thought it was a ELF library.

I fixed that by using the same search method like some Slackbuild scripts are using to strip debug info from libraries.

Code:

# find all files relevant for hard elf-dependencies

find . | xargs file | grep -e "ELF .*executable" -e "ELF .*shared object" | cut -d : -f1 > $internal

Some packages have files with spaces/blanks in the file name which will cause xargs fail to find the file. I'm now using this to fix it:

Code:

# find all files relevant for hard elf-dependencies

find . -printf "\"%p\"\n" | xargs file | grep -e "ELF .*executable" -e "ELF .*shared object" | cut -d : -f1 > $internal

This will change the filename piped to xargs using "/path/to/file".

If the package is not yet installed and some libraries depended on symlinks that point to regular file in the package you will also get some errors about missing dependencies. I added some extra code to search for symlinks that point to real files inside of the package and add these to the internal solved dependencies:

Code:

find . -type l > "$tmpfile"

  while read elflink; do

    [ x"$(readlink -f "$elflink" | xargs file 2>/dev/null | grep -e "ELF .*executable" -e "ELF .*shared object" | cut -d ':' -f1)" != x"" ] && echo "$elflink" >> "$internal"

  done < "$tmpfile" && rm "$tmpfile"

If a dependency is not found in the /etc/ld.so.conf search path the script will also give some error messages about missing dependencies. This happened for a few rare binary packaged packages that have internal library directories. This is a little bit more complex. For myself i fixed that by using a two staged search for libraries:
Stage1 -> Fast -> Use your code and check the directories in /etc/ld.so.conf
If library wasn't found in any search path:
Stage2 -> Slow -> using grep "libname" /var/log/packages

I also did a test with about 260 Xorg related packages just to compare the speed of the three methods (getdeps, sbbdep and requiredbuilder):
getdeps -> 260 packages: 74min
sbbdep -> 260 packages: 79min
requiredbuilder -> 260 packages: 79min

That might look like not a big difference, but a lot of Xorg related packages do not have many dependencies. At least it shows that is really faster. Right now i'm doing another automated test with 1000 packages and it's not much slower (less then 1%) compared to building a package without checking for dependencies.

I think i made some more (maybe more cosmetic) changes, but would have to compare with your latest version to check what is different.

Quote:

Originally Posted by DarkVision (Post 5487216)

I also did a test with about 260 Xorg related packages just to compare the speed of the three methods (getdeps, sbbdep and requiredbuilder):
getdeps -> 260 packages: 74min
sbbdep -> 260 packages: 79min
requiredbuilder -> 260 packages: 79min

not possible, resolving all dependencies on a fully installed Slackware system takes for sbbdep just a few seconds/minutes, depending on the type of HD/SSD.
and with an existing cache it's nearly instant time, so the 79 minutes you write for sbbdep is totally bogus.
I don't know what you measured/did but it does definitely not look right.

Hi DarkVision,

thanks for your testing.
I fixed the things you mentioned, exept searching more than /etc/ld.so.conf, i'll do that later. I also want to implement that already installed packages may get calculated their deps.

Quote:

Originally Posted by a4z (Post 5487425)

The measured time was the time building the packages and checking for dependencies (and for sbbdep including building the cache). With more packages creating the cache might not be that relevant but you have to do it at least once. For a single package on a new installed system sbbdep including creating cache is much slower... for example while compiling sbbdep and check for required files sbbdep takes 73seconds (including creating the cache). Using getdeps this will take 26seconds only. So maybe sbbdep might be as fast as getdeps but creating the cache might be a time consuming step. And for 260 packages it may make a difference.

I really wanted to be sure how much faster getdeps can be compared to sbbdep and requiredbuilder and therefore i can not just do a single checking command. So i used sbbdep, requiredbuilder and getdeps while building those 260 XOrg related packages from source and checking for dependencies right before the makepkg command to include the slack-required file. The measured difference between these three tests is just related to the used dependency checker.

So.. No. sbbdep does not take that much time... but while building 260 packages from source and check for dependencies on my system it takes 5mins longer then getdeps.

The test was done using virtualbox with the same setup before i started to compile those 260 packages. VirtualBox was using a AMD FX(tm)-6100 Six-Core Processor with 1400MHz and max. 5 cores used.

BTW: I just upgraded to sbbdep/current. The new ignore feature looks interesting :) Could it be that when compiling sbbdep that CMakeLists.txt does not check for installed libsl3? If not installed builing sbbdep will fail at compile time. Of course with libsl3 installed everything works fine.

Quote:

Originally Posted by DarkVision (Post 5490613)

creating the cache is a one time action, and therefore irrelevant.
depending how you use it, even the synchronisation might be irrelevant.
but creating the cache is irrelevant for published performance number. in general for the tool, and especially for the dependency check.

Quote:

Originally Posted by DarkVision (Post 5490613)

I really wanted to be sure how much faster getdeps can be compared to sbbdep and requiredbuilder and therefore i can not just do a single checking command. So i used sbbdep, requiredbuilder and getdeps while building those 260 XOrg related packages from source and checking for dependencies right before the makepkg command to include the slack-required file. The measured difference between these three tests is just related to the used dependency checker.

do sbbdep /var/adm/packages/gimp-2.8.6-x86_64-1
with an existing cache and compare it to whatever,
use it with firefox and libreoffice
this is speed a measurement of the tool.

notice than I did not mention the correctness of the check ;-)

Quote:

Originally Posted by DarkVision (Post 5490613)

So.. No. sbbdep does not take that much time... but while building 260 packages from source and check for dependencies on my system it takes 5mins longer then getdeps.

The test was done using virtualbox with the same setup before i started to compile those 260 packages. VirtualBox was using a AMD FX(tm)-6100 Six-Core Processor with 1400MHz and max. 5 cores used.

so it is not sbbdep that takes longer, it is what ever you do.
sorry but I have to be pedantic.
sbbdep does the dependency check faster and more correct. and I am proud that this is the case, it was a hell lot of work to make it work like that. not mentioning all other features that are just a side effect of that.
I have therefore to tell if publishing some fantasy numbers happens.

Quote:

Originally Posted by DarkVision (Post 5490613)

BTW: I just upgraded to sbbdep/current. The new ignore feature looks interesting :) Could it be that when compiling sbbdep that CMakeLists.txt does not check for installed libsl3? If not installed builing sbbdep will fail at compile time. Of course with libsl3 installed everything works fine.

nice, did the commit yesterday, I am glad and surprised that you notice it.
ad libsl3. when I do a release the libsl3 will be included in the source distro, and no one notice that it exists,
as it was in previous version (under a different name)
you use developer version so you need to care for this your self, for now, this might change in future.
the tech detail: it was included as a hg sub repo in previouse versions, but libsl3 is now in a git repo, it has at least one different user than me on more than one platform. it looks really stable well tested and feature complete and therefore I might do an extra release/deploy at some point of time in future, but I will see how I will handle this, sbbdep releases might possible never be influenced by this,

Quote:

Originally Posted by a4z (Post 5490678)

creating the cache is a one time action, and therefore irrelevant.

Maybe... if you do it many many many times... for a one time shot it will make a difference. Anyway.. who cares. It is just a dependency checker... maybe more or less better then requiredbuilder or the get_deps.sh script. it is not a package everyone needs ;)

Quote:

Originally Posted by DarkVision (Post 5490771)

of course sbbdep its not a package everyone needs, it's a developer tool. and not good comparable with the other you mention since they fulfil just a subset of the functionality and they do this not even correct. ;)