LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 11-30-2017, 05:54 AM   #1
FlinchX
Member
 
Registered: Nov 2017
Distribution: Slackware Linux
Posts: 666

Rep: Reputation: Disabled
A question about sbopkg queues


I have started to use sbopkg to learn it, since before I just manually downloaded buildscripts from SBo and ran them. I have noticed that when I do Queue->Add all installed packages, then View, the queue looks like it's sorted in alphabetical order. I would expect it to be sorted dependency-wise, packages with zero dependencies at the top etc, so packages don't appear in the queue before any of their dependencies. Can sbopkg do that and build just one queue containing all packages (no references to other queues), or this kind of queue management still has to be done manually?
 
Old 11-30-2017, 06:57 AM   #2
willysr
Senior Member
 
Registered: Jul 2004
Location: Jogja, Indonesia
Distribution: Slackware-Current
Posts: 4,661

Rep: Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784
queue file is used to build a list of dependencies needed to build a package, so it will be generated if there's a dependencies for a script. If there's no deps, no queue file will be generated by sqg.

You can use sqg -a to generate dependency list for all scripts in the repository (it takes some time to complete, but it will be a lot faster in the next version if you have parallel installed).

In the next version, you can use sqg to generate multiple queue for many scripts at once and even use a custom name for it. That should be very useful if you have to install the same application in another machine.
 
1 members found this post helpful.
Old 11-30-2017, 06:58 AM   #3
bassmadrigal
LQ Guru
 
Registered: Nov 2003
Location: West Jordan, UT, USA
Distribution: Slackware
Posts: 8,792

Rep: Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656
sbopkg itself is not aware of any dependencies. Any packages you add to queues would need to be manually sorted by you.

However, sbopkg comes with a companion script called sqg (slackbuild queue generator) that can create queues for you based on dependencies. You run it using sqg -p $PACKAGENAME (replacing $PACKAGENAME with the name of the package you're trying to build). It will add the dependencies needed and sort them to the proper build order and save the file in /var/lib/sbopkg/queues/$PACKAGENAME.sqf. Then you can run sbopkg -i $PACKAGENAME and it will ask if you want to build just the package or the queue file (if a queue file exists). If you select the queue file, it will process that file in order, building and installing each package before moving to the next.

Check out the following links for more details and feel free to ask any other questions you might have:
https://docs.slackware.com/howtos:sl...es_with_sbopkg
https://www.sbopkg.org/queues.php
https://htmlpreview.github.io/?https...ools/sqg8.html
https://slackblogs.blogspot.com/2014...es-easily.html
 
1 members found this post helpful.
Old 11-30-2017, 07:28 AM   #4
FlinchX
Member
 
Registered: Nov 2017
Distribution: Slackware Linux
Posts: 666

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by willysr View Post
That should be very useful if you have to install the same application in another machine.
My question is about the queue that sbopkg generates for all SBo packages installed on a system and exactly this situation. I want to install some packages on a computer, and have sbopkg build the queue for all of them. I just don't know if that queue could be used to install the same set of SBo packages on another computer without any additional manual intervention for changing the order of packages in that queue.

Just in case if I failed to explain my question well, I'll give a trivial hypothetical example below.

There is app_x and app_y. app_x requires app_y. So if I try to install app_x first, it will complain that app_y is missing. I can use sbopkg to install app_y, then install app_x. After that, if I do Queue->Add all installed packages to the queue, sbopkg will build a queue that's sorted in alphabetical order.

So I end manually installing packages in this order:

- app_y
- app_x

But the queue will have a different order (because it is sorted in alphabetical order):

- app_x
- app_y

So my question is, can I use this queue on a second computer? Will sbopkg be smart enough to process dependencies of packages in the queue and figure out that it should install app_y first?
 
Old 11-30-2017, 10:23 AM   #5
bassmadrigal
LQ Guru
 
Registered: Nov 2003
Location: West Jordan, UT, USA
Distribution: Slackware
Posts: 8,792

Rep: Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656
A queue can be saved and loaded from sbopkg. It is just a text file with the packages listed and the order to build them in. In my testing, my queue wasn't being sorted alphabetically, but in the order I added items to it. I tested it by first adding virtualbox, then armadillo, then pitivi, and my queue looks like:

Code:
virtualbox
armadilla
pitivi
However, if your queue needs to be adjusted, you can do that under Main Screen -> Queues -> Sort. Once your queue is the way you like it, you can then save it and then load it up on another computer.

But I feel like it is probably easier to just use sqg to build the queue for the items you need and then just keep that locally on the computer and then do the same on another computer.
 
1 members found this post helpful.
Old 11-30-2017, 12:19 PM   #6
FlinchX
Member
 
Registered: Nov 2017
Distribution: Slackware Linux
Posts: 666

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by bassmadrigal View Post
In my testing, my queue wasn't being sorted alphabetically, but in the order I added items to it.
Again, I was not trying to find details about a custom queue that I can build by manually adding items to it, but about the queue of all installed packages that sbopkg builds.


Quote:
Originally Posted by bassmadrigal View Post
However, if your queue needs to be adjusted, you can do that under Main Screen -> Queues -> Sort. Once your queue is the way you like it, you can then save it and then load it up on another computer.
In case of a big queue of all installed packages (that is sorted in alphabetical order) there might be the need to do a lot of sorting. That's why instead of randomly installing packages whenever I feel like and then just getting the queue (I hoped I could cut a corner here, but apparently I'll need to adjust it manually), I think I'll just use my notes about dependencies and just convert them to queues, then have one global queue that sources all others. Thanks for clearing the things up.
 
Old 11-30-2017, 12:27 PM   #7
BCarey
Senior Member
 
Registered: Oct 2005
Location: New Mexico
Distribution: Slackware
Posts: 1,639

Rep: Reputation: Disabled
Quote:
Originally Posted by FlinchX View Post
So my question is, can I use this queue on a second computer? Will sbopkg be smart enough to process dependencies of packages in the queue and figure out that it should install app_y first?
It will process them in the order listed.
 
1 members found this post helpful.
Old 11-30-2017, 12:42 PM   #8
BCarey
Senior Member
 
Registered: Oct 2005
Location: New Mexico
Distribution: Slackware
Posts: 1,639

Rep: Reputation: Disabled
Quote:
Originally Posted by FlinchX View Post
Again, I was not trying to find details about a custom queue that I can build by manually adding items to it, but about the queue of all installed packages that sbopkg builds.




In case of a big queue of all installed packages (that is sorted in alphabetical order) there might be the need to do a lot of sorting. That's why instead of randomly installing packages whenever I feel like and then just getting the queue (I hoped I could cut a corner here, but apparently I'll need to adjust it manually), I think I'll just use my notes about dependencies and just convert them to queues, then have one global queue that sources all others. Thanks for clearing the things up.
You could make use of sqg to make your manual process easier, though, since it creates individual queue files for all programs you could concatonate the files you need and then weed out duplicates.
 
1 members found this post helpful.
Old 11-30-2017, 12:55 PM   #9
bassmadrigal
LQ Guru
 
Registered: Nov 2003
Location: West Jordan, UT, USA
Distribution: Slackware
Posts: 8,792

Rep: Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656
Quote:
Originally Posted by FlinchX View Post
Again, I was not trying to find details about a custom queue that I can build by manually adding items to it, but about the queue of all installed packages that sbopkg builds.
sbopkg doesn't generate queues in any other way but manually. The only automated way to generate queues is by using the sqg script.

Quote:
Originally Posted by FlinchX View Post
In case of a big queue of all installed packages (that is sorted in alphabetical order) there might be the need to do a lot of sorting. That's why instead of randomly installing packages whenever I feel like and then just getting the queue (I hoped I could cut a corner here, but apparently I'll need to adjust it manually), I think I'll just use my notes about dependencies and just convert them to queues, then have one global queue that sources all others. Thanks for clearing the things up.
I can't replicate queues being sorted alphabetically on my system, and it's never worked that way for me. What you might be looking at is the queue files that already exist (after having run sqg -a) and those will be sorted alphabetically so you can find them, but those are each queue files to build that specific program, not a list of software and the order to build them in.

But, based on your second to last sentence, I wonder if something other than sbopkg would be a better option for you. Earlier this year, I switched from sbopkg (which I had used for quite a few years) to slackrepo. The biggest benefit in my eyes for slackrepo is you document any changes from the stock SlackBuild in what are called hint files. This includes passing options, adding additional optional dependencies, or even overriding versions and downloads (in case you want to run a different version than what's in the repo). This is basically semi-permanent documentation of how you built the program and will continue to be used whenever any updates are pushed through SBo.

One of the other big benefits is it tracks all dependencies and will recompile software relying on updated parent dependencies when it's determined it's needed. This can be extremely handy if you update something like x264, which may require rebuilding ffmpeg, which may require building a number of different programs that rely on ffmpeg. It helps minimize the possibilities of breakages when packages are upgraded.

The only downside with slackrepo (actually, difficulty might be a better word than downside) is it prefers to reside on a clean Slackware installation. It does this because it uses a clean chroot every time it builds a piece of software. It will install all required dependencies, build the software, store the package, and then destroy the chroot, restoring the system back to stock. This ensures you don't have any undocumented software included in your packages and makes them highly portable. I use a VM for mine rather than a dedicated machine.

You can read about it more on the slackrepo website. The software is developed by forum member 55020.
 
2 members found this post helpful.
Old 12-01-2017, 06:53 PM   #10
FlinchX
Member
 
Registered: Nov 2017
Distribution: Slackware Linux
Posts: 666

Original Poster
Rep: Reputation: Disabled
Will ask a couple more ontopic questions here to avoid creating a new thread.

I have noticed that if I try to reinstall a package that is already installed, sbopkg finds it in cache and reinstalls the same version. I don't understand why, if versions match. Shouldn't it just skip reinstalling?

After deleting the package from cache and trying to install it, sbopkg does even more work - it rebuilds the package, then reinstalls the same version. Same question as in previous scenario.

So now I wonder, if sbopkg does this redundant work for single packages, will it do even more redundant work for queues? For example, if a queue for a program is big, all those packages get installed, then a second big queue for another program contains many of those already installed dependencies as well, will sbopkg do a lot of additional work for each package, like reinstalling them at least (if they exist in cache), or even rebuilding them (if cache is empty)?

I know I was told sbopkg doesn't know about dependencies, but this is a different situation. sbopkg certainly knows about installed packages, since it can show a list of them, so why does it try to reinstall them if versions match?
 
Old 12-01-2017, 08:38 PM   #11
willysr
Senior Member
 
Registered: Jul 2004
Location: Jogja, Indonesia
Distribution: Slackware-Current
Posts: 4,661

Rep: Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784Reputation: 1784
A good reason is to make sure your package is rebuilt against the correct libraries installed on your system
 
1 members found this post helpful.
Old 12-01-2017, 09:36 PM   #12
bassmadrigal
LQ Guru
 
Registered: Nov 2003
Location: West Jordan, UT, USA
Distribution: Slackware
Posts: 8,792

Rep: Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656Reputation: 6656
Quote:
Originally Posted by FlinchX View Post
Will ask a couple more ontopic questions here to avoid creating a new thread.

I have noticed that if I try to reinstall a package that is already installed, sbopkg finds it in cache and reinstalls the same version. I don't understand why, if versions match. Shouldn't it just skip reinstalling?

After deleting the package from cache and trying to install it, sbopkg does even more work - it rebuilds the package, then reinstalls the same version. Same question as in previous scenario.

So now I wonder, if sbopkg does this redundant work for single packages, will it do even more redundant work for queues? For example, if a queue for a program is big, all those packages get installed, then a second big queue for another program contains many of those already installed dependencies as well, will sbopkg do a lot of additional work for each package, like reinstalling them at least (if they exist in cache), or even rebuilding them (if cache is empty)?
Based on my testing, sbopkg doesn't reinstall previously built packages. However, it will reuse previously downloaded tarballs to compile the package (assuming the md5sum matches). But then I mainly used the commandline and flags and not the gui, so maybe there's some differences in how those are processed/handled.

I'm not sure if sbopkg doing this is by design or some unintended side effect from how it handles queues. I know Willy is the current maintainer of the program, but it was originally developed by Chess Griffin and had two other people who took up maintenance until Willy did.

Personally, I would generate a queue file for a program I needed to build, then I would go and edit the queue file to remove any existing programs from it (although, this would be after I read the READMEs of the program and all dependencies to make sure I didn't need to compile it different than I originally did), then build the queue. It did take some extra time in the preparation, but I had an old computer, and depending on what was being compiled, it could save me hours.

This issue is another reason I love slackrepo. It is fully aware of what it has built and will install those previously compiled programs into that chroot before compiling the main program. It is also highly aware of the dependencies and will automatically rebuild child dependencies if the parents changed. sbopkg can't do this in its current state. That means if sbopkg pops up an upgrade for only h264, it may break ffmpeg and any other programs that rely on ffmpeg without ever notifying you. A single program can have a cascading effect on your system and could leave many programs broken. It's why I almost never used sbopkg's upgrade process, because I was worried on what it might break and then the time it could take to fix it (especially with how long it took to compile software on that system). I don't have that worry when using slackrepo. When it detects an update for a program, it will automatically recompile any programs that depend on that updated one.
 
2 members found this post helpful.
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Threads checking for space two times in queues VS keeping the queues in sorted order Aquarius_Girl Programming 1 12-23-2015 08:05 PM
[SOLVED] Sbopkg Slackware queues, still exist? a4z Slackware 2 06-11-2015 06:13 AM
[Solved] SBopkg queues git download Mark Pettit Slackware 7 04-19-2015 12:31 PM
[SOLVED] How to update the Sbopkg Slackware queues? DrCR Slackware 3 12-26-2012 12:48 AM
POSIX Queues with message type facility of System V queues ehsan_haq98@yahoo.com Programming 2 12-27-2009 11:45 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 11:16 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration