How to rebuild all SBo packages installed on the system

After installation/setup of misc/hoorex, the command:

hoorex -g all -I -r

will make a list of all installed SBo SlackBuilds and their build dependencies in a suitable build order (you may debate whether or not build order is important).

Originally Posted by elcore I will address your concerns, but I'm sure you think I haven't read the license or any of the build scripts, and that is why I disagreed with your opinion. Truth is, I disagree because I don't like being told what to do, and if that don't fit your hive mentality it's not my problem.

What opinion did I have about the build scripts that you disagree with (other than the fact that SlackBuilds can be used for repackaging or compiling)? As for being told what to do, when you use someone else's work, you're bound by how they provide it. Sure, some will allow you to fork it and make your own rules (to a certain extent, some licenses won't let you switch to other licenses), but if you get stuff from SBo, you are limited to how they provide it (until you decide to make changes). This is no different than buying a new car and being told you can't order it without the radio. It doesn't mean you can't remove the radio once you buy it, but you can only buy it with a radio.

Originally Posted by elcore 1. It is obvious that current is not stable, and there is a current branch available, official or not. You argue about semantics.

The unstable branch is not for 14.2. There is no unstable branch you can do 14.2 work in (other than a locally forked/mirrored copy). I don't run -current on my machines because I need the stability (and I don't like keeping up with the update regimen required for -current). That is why I ensure updates to my SlackBuilds are stable on my system.

Originally Posted by elcore 2. If you maintain, and introduce 50 deps, with 50 security issues, and 550 other bugs, that's your problem. If you push all that into SBo which targets stable, it becomes a problem for everyone downstream, including me. Hence, you're breaking stable systems for no good reason.

And you're assuming I'm pushing bug and security ridden software? When I do updates to my packages on SBo, I test what I can to verify it's as stable as possible. I can't recall reverting any of my updates once pushed to SBo due to buggy or security issues (but many times I've not pushed updates due to there being additional bugs). In regards to pitivi, I did introduce deps to that app, but all were already available on SBo and should've already been vetted. I did verify that pitivi worked as expected once it was compiled from source.

Originally Posted by elcore 3. If you note that you're in charge of what you maintain, but may decide to introduce 650 bugs.. where does that leave a sysadmin? Fork or deal with 650 bugs?

Do you just assume everyone is introducing bugs with any change to the SBo repo?

Originally Posted by elcore "Simply upgrading a package does not constitute breaking stable." Opinion. Fact is, package upgrades can break a stable system beyond all recognition.

It can break stable, but it isn't guaranteed to. So, no, upgrading a package doesn't guarantee you're breaking stable. In fact, most times there are no issues with updates. Especially if I test the packages locally before I update them on SBo.

Originally Posted by elcore "So we can only follow the rules they've put forth." Opinion. You can fork and make your own rules, license is permissive enough.

That is semantics, as you put it earlier. If we use what they provide, we follow their rules. Of course you can fork it and change things as you'd like. Later on, I even mentioned forking it.

Originally Posted by elcore "That's not how SBo admins feel" Crystal ball? What if they say one thing and feel something different? WTF how do you know what they feel?

Wow, you're reaching at this point. They've repeatedly stated their view and have repeatedly stated they have no intention to change how maintainers maintain their SlackBuilds. No, I don't have telepathy and don't technically know how they individually feel, but how the SBo admins feel, as a group, can be seen in the policies presented on the site. You flat out knew what I meant with that and decided to twist it to your liking.

Originally Posted by elcore That's kinda hypocritical, coming from you.

This went from a simple, "dependencies can change" to you attacking me. You've called into question my updating practice with my SlackBuilds on SBo with no indication that my updating practice is problematic. I've never heard anything from any of my SlackBuilds other than occasional requests to update them (and sometimes I refuse due to not wanting to introduce things like qt5 -- other times, I'll test them and see if they cause any problems).

Many people on the forum have expressed desires to have packages not included in Slackware so they can be on SBo and be updated more often. Are you against any upgrading of packages on SBo?

Originally Posted by elcore Go patronize someone else man, you've written nothing here that I don't already know, and presented your opinion as fact. You just talk and talk, but never get to the point, like some soccer mom, or politician in distress.

Don't be my handle.

Originally Posted by bassmadrigal What opinion did I have about the build scripts that you disagree with (other than the fact that SlackBuilds can be used for repackaging or compiling)?

It's where you, as maintainer, may introduce additional deps which can change the build order. Which I called sabotage.
So I suggested there might be unstable branch for testing stuff like that, but you still prefer doing it on stable and create additional work for everyone downstream.

Originally Posted by bassmadrigal The unstable branch is not for 14.2.

But unstable upgrades are to be pushed into 14.2 regardless, in your opinion? I already do 14.2-stable and 14.2-unstable separation locally, so can you.

Originally Posted by bassmadrigal Do you just assume everyone is introducing bugs with any change to the SBo repo?

Yes. Assuming the worst is something I do all the time.

Originally Posted by bassmadrigal This went from a simple, "dependencies can change" to you attacking me.

Well, I did say I wanted to back off and not derail the thread, but you insisted.

just to clarify a couple of points:
- there's no "unstable" branch in the SBo's repository;
- I maintain (with welcomed contributions) a fork of the SBo repository, not related or endorsed by SBo, with fixes to let the stuff build on Slackware current: not everything I publish there (also some occasional upgrades that I use on my own installations) will go on SBo when the next Slackware version will be out, just the needed fixes (I let others decide what to cherry pick);
- it may happen that additional dependencies are added/changed to scripts on SBo, they are not set in stone: the maintainer may just follow what upstream decides as mandatory (for example, qt5 for vlc), or himself decides that to have a functional program you need other dependencies, or whatever, remember that it's the maintainer call;
- everybody is free to fork the SBo repo and do whatever he wants with it (like I said, I do it too!).

Originally Posted by chris.willing After installation/setup of misc/hoorex, the command: hoorex -g all -I -r will make a list of all installed SBo SlackBuilds and their build dependencies in a suitable build order (you may debate whether or not build order is important).

Thank you Chris!
This seems useful.

Anyway it doesn't seem enough, because it lists all SBo dependencies, even if some of them are already installed from an other repo, different than SBo.

I tried the following check:

root@master:~# for i in $(hoorex -s /var/lib/sbopkg/SBo/14.2/ -g all -I -r);do echo $i;done|wc -l 
282
# for i in $(hoorex -s /var/lib/sbopkg/SBo/14.2/ -g all -I -r);do echo $i;done|grep ffmpeg
gst0-ffmpeg
ffmpeg

# find /var/log/packages/ -name "*ffmpeg*"
/var/log/packages/gst0-ffmpeg-0.10.13-x86_64-1_SBo
/var/log/packages/ffmpeg-3.4.2-x86_64-1alien

So I want to rebuild gst0-ffmpeg, but ignore ffmpeg. I don't want to rebuild ffmpeg.
On the other hand, I should find 256 SBo packages installed on my system, while hoorex finds 282... Likely 26 packages are installed on my system from repos other than SBo.
I have to check these numbers a little more...

Originally Posted by elcore Truth is, I disagree because I don't like being told what to do, and if that don't fit your hive mentality it's not my problem.

So you are arguing just for the hell of it and because it makes you feel different?

Originally Posted by joenew Thank you Chris! This seems useful. Anyway it doesn't seem enough, because it lists all SBo dependencies, even if some of them are already installed from an other repo, different than SBo.

Yes that would be right - it assumes they're all from SBo (can't determine whether they're from somewhere else).

I tried the following check: root@master:~# for i in $(hoorex -s /var/lib/sbopkg/SBo/14.2/ -g all -I -r);do echo $i;done|wc -l 282

FYI, after the initial hoorex setup, you don't need to keep using the '-s /path/to/SlackBuilds' part of the command any more; hoorex remembers the location from the setup (unless you've moved location of the SBo tree, of course). Also, you don't need the loop to output all entries on separate lines to pipe to wc. You could achieve the same result with:

hoorex -g all -I -r -1 | wc -l

The -1 (number 1) option tells hoorex to output any results in a single column.

chris

Originally Posted by orbea So you are arguing just for the hell of it and because it makes you feel different?

No, I just contradict him in hope that you will come along and pull something out of context, even though it wasn't directed at you.
It is certain that I'm being wrong for even considering a well respected member and maintainer shouldn't have authority over me and what I do or don't do.
And I did not want to argue, before he assumed I know nothing and that I was born in 2014 because I happened to create an account in 2014 and my english is not exactly what native speaker expects.

Anyway, to keep on topic: @joenew I'd change tags in a SlackBuild if I were still having that problem, so the things that need ordering have something other than "*_SBo" (jut one of many ways to fix it)

No one other than you ever claimed that any SBo admin or maintainer has power or authority over you, not being a native speaker is not an excuse to spread misinformation and slander other users. The SlackBuild scripts are all free software and you are welcome to audit them, report issues to the maintainers, the mailing list or even fork them and host them in your own repo (Assuming you respect the license). I'll assume you know this already and you are willingly being contrary out of some sort of cognitive dissonance.

If you have any specific issues to point out you know where to do so, but can you please take your toxic attitude somewhere else?

Originally Posted by elcore It's where you, as maintainer, may introduce additional deps which can change the build order. Which I called sabotage. So I suggested there might be unstable branch for testing stuff like that, but you still prefer doing it on stable and create additional work for everyone downstream.

You must live in an amazing world to think that software that is not upgraded is stable. Changing a build order is not sabotage. I switched from a binary release to a compiled release so users could know what's on their system. I think that is considered sabotage in only your mind.

And as I said above, I test everything locally on my own system before I push them up, and the SBo admins verify there's no glaring issues with the package when submitted. That's about the best one can do with how SBo is currently set up. Would you rather I keep old, stale, possibly buggy, or insecure versions in SBo just so they don't change? (I won't be doing this, but I'm curious if that's what you want.)

Originally Posted by elcore But unstable upgrades are to be pushed into 14.2 regardless, in your opinion? I already do 14.2-stable and 14.2-unstable separation locally, so can you.

I test my upgrades before they are pushed, and I've refrained from pushing upgrades when they aren't stable.

Originally Posted by elcore Yes. Assuming the worst is something I do all the time.

You must be a happy person. /s

Originally Posted by elcore Well, I did say I wanted to back off and not derail the thread, but you insisted.

You attacked me first and then say you don't want to derail this thread? Did you expect I'd just sit there in silence? Dream on! If you've paid any attention on this forum, you know I'm one (for better or worse... probably usually worse) that doesn't let that stuff slide.

Originally Posted by elcore No, I just contradict him in hope that you will come along and pull something out of context, even though it wasn't directed at you. It is certain that I'm being wrong for even considering a well respected member and maintainer shouldn't have authority over me and what I do or don't do. And I did not want to argue, before he assumed I know nothing and that I was born in 2014 because I happened to create an account in 2014 and my english is not exactly what native speaker expects.

As orbea stated, nobody on SBo has any authority over you. All my SlackBuilds use the standard license allowing you to make whatever changes you want. SBo keeps a git history so if you don't like how a package is updated, you're free to use an older version. You're even free to use versions from other Slackware releases, although, with the lack of updates, it's very possible that those are broken (coming between you and your beloved "stable", even though they haven't been upgraded in years).

I never said you know nothing and I honestly don't care when you registered for this forum (it's impossible to judge someone's knowledge based on when they registered for a site, their post count, and even their reputation count), but I did correct you when you say the build order doesn't change. It may not always change, but it certainly can depending on what package maintainers on SBo do. If you expect the build order will never change and you write a program or script based on that assumption, you may run into issues. Sure, you may call changing dependencies sabotage, but it is still a very real possibility during routine upgrades on SBo (off the top of my head, I've only changed dependencies when switching pitivi from a binary repackaging to a source compile, but there might've been some I've forgotten about over the years).

Originally Posted by orbea If you have any specific issues to point out you know where to do so, but can you please take your toxic attitude somewhere else?

Sure thing, I can back off. I've honestly already tried to, but then he'd returned with more questioning like.. right after I say don't bother me find someone else..
All that aside, I'm more interested in what I can learn from the thread actual subject, as crafting automation such as this one without stepping on toes is difficult.
We may disagree on what's toxic here apart from my attitude, cause for me it's probably 'do it by the book' thing which kills both competition and innovation.
But I'll respect your wish and will go away now.

Originally Posted by chris.willing Yes that would be right - it assumes they're all from SBo (can't determine whether they're from somewhere else). FYI, after the initial hoorex setup, you don't need to keep using the '-s /path/to/SlackBuilds' part of the command any more; hoorex remembers the location from the setup (unless you've moved location of the SBo tree, of course). Also, you don't need the loop to output all entries on separate lines to pipe to wc. You could achieve the same result with: hoorex -g all -I -r -1 | wc -l The -1 (number 1) option tells hoorex to output any results in a single column. chris

Thanks for your infos. Various checks on number of packages seems to confirm congruent results.

I tried to clean-up the hoorex packages list, so that it is generated a new queue containing:
- installed packages from SBo repo
- new packages not installed at all (could be new dependencies added to some installed SBo packages)

I wrote a little script to do that called "torebuild.sh".
It returns also a check report with files generated and their number of lines contained.

An other check done comparing the above hoorex cleaned queue, with my old script sbo-rebuild.sh output queue.
The two queue contain the same packages (excluded hoorex package, just added today and not listed in the old sbo-rebuild queue).

Here we go:

# hoorex -g all -I -r -1 > hoorex-detected

# cat torebuild.sh

#!/bin/bash


ALL=all_installed_packages
HOOREXLIST=$1
REB=to_rebuild.log
NOTREB=not_to_rebuild.log
NEW=new_to_build.log

[ -e $REB ] && rm $REB
[ -e $NOTREB ] && rm $NOTREB
[ -e $NEW ] && rm $NEW

ls -1 /var/log/packages > $ALL

for i in $(cat $HOOREXLIST); do
        if grep -q "^$i-[^-]*-[^-]*-[^-]*$" $ALL; then
                if grep -q "^$i-[^-]*-[^-]*-[^-]*_SBo$" $ALL; then
                        echo $i >> $REB
                else
                        echo >> $NOTREB
                fi
        else
                echo >> $REB
                echo >> $NEW
        fi
done

cat <<EOF
Results:

$ALL: $(wc -l < $ALL)
$HOOREXLIST: $(wc -l < $HOOREXLIST)
$REB: $(wc -l < $REB)
$NOTREB: $([ -e $NOTREB ] && wc -l < $NOTREB || echo "not exixsts")
$NEW: $([ -e $NEW ] && wc -l < $NEW || echo "not exists")
EOF

# ./torebuild.sh hoorex-detected                                                                           
Results:

all_installed_packages: 1698
hoorex-detected: 282
to_rebuild.log: 256
not_to_rebuild.log: 26
new_to_build.log: not exists

# sort to_rebuild.log > a && sort /var/log/sbo-rebuild/2018.06.14-18.35-sborebuild.queue.log> b && diff a b
141d140
< hoorex

Notes:
In the two queues, the new one generated with hoorex based script and the other one derived from the old sgq based script, packages are sorted in different ways, anyway they should be equally congruent with dependency hierarchy.
Moreover the above check, confirms the lists to rebuild contain the same packages.

I report below the old script sbo-rebuild.sh (sqg based) already shown in a previous message.
https://www.linuxquestions.org/quest...3/#post5869803

Any comments are well come!