LinuxQuestions.org - [SOLVED] Correct universal way of building/installing SBo package in batch mode?

- Slackware (https://www.linuxquestions.org/questions/slackware-14/)

- - Correct universal way of building/installing SBo package in batch mode? (https://www.linuxquestions.org/questions/slackware-14/correct-universal-way-of-building-installing-sbo-package-in-batch-mode-4175672375/)

FlinchX

04-01-2020 05:29 PM

Correct universal way of building/installing SBo package in batch mode?

I am trying to avoid doing sqg -a for the whole SBo repository, because it's time consuming.

Some packages have dependencies, others don't.

My goal is to install a single package in batch mode, so my input data is an "APPNAME" string.

Do I just need to do

Code:

sqg -p APPNAME

sbopkg -i APPNAME

and it'll be smart enough to generate the queue if there are dependencies, then use the queue rather than the package name if there's one?

Richard Cranium

04-01-2020 07:45 PM

Use slapt-get to install, if that's what you want.

montagdude

04-01-2020 08:34 PM

There's no universal way. There are different tools that have different methods.

FlinchX

04-02-2020 01:09 AM

I want to use sbopkg, as my example shows, perhaps I should've added that explicitly to the title.

giomat

04-02-2020 03:47 AM

if you pass to sqg a package with no dependencies it will simply print a complaint that the queue file is empty.
If a queue is generated, then building with "sbopkg -i" will ask you if you want to use the queue or just build and install the package.
Anyway queue files are really just simple plain text files where every line contains the name of a package to build, so it's
really easy to modify the existing ones or roll your own if you want to add some optional dependency to a package.
Queue files are stored in /var/lib/sbopkg/queues,

FlinchX

04-02-2020 03:57 AM

Quote:

Originally Posted by giomat (Post 6106931)

So, I can just do sqg -p APP for anything I want to install, then do sbopkg -B -i APP and it will pick the queue if there is one in QUEUEDIR, otherwise it'll treat APP as packagename and will do everything without asking anything interactively?

giomat

04-02-2020 04:36 AM

Quote:

Originally Posted by FlinchX (Post 6106935)

well regarding this last bit, by default sbopkg will ask you explicitly if you want to build everything or just the package, because it will find both a queue file (.sqf) and an sbo package having the same name. I'm not sure but perhaps passing the -B option will force sbopkg to always prefer the queue file if found. If it doesn't, then perhaps you could ask the sbopkg developer to add a new flag for your use case.

bassmadrigal

04-02-2020 11:10 AM

As far as I know, there is no feature in sbopkg to automatically choose the queuefile. Adding -B will still ask if you want to process the queue or program.

For what it's worth, there has been an update to sqg (that hasn't been pushed out to a released package yet and likely won't until 15.0 is released) allowing you to generate a custom queuefile name. This would allow you to do something like below, which would allow you to specify a custom name for your queue and then run sbopkg -i against that custom name, and there will only be the queuefile to use, so -B should then work.

Code:

sqg -p $PRGNAM -o $PRGNAM-q

sbopkg -B -i $PRGNAM-q

Unfortunately, you can't just grab the latest since there's been a lot of changes to deal with the package database being moved from /var/log/packages/ to /var/lib/pkgtools/packages/ and it won't work on 14.2. Plus, this is just a hack to get what you want done and doesn't allow you to prefer the queuefile or package automatically. It may not be a bad idea to request a new option (or two) to prefer either the queuefile or package automatically when building.

FlinchX

04-02-2020 04:28 PM

Quote:

As far as I know, there is no feature in sbopkg to automatically choose the queuefile. Adding -B will still ask if you want to process the queue or program.

I have just tested and indeed, this is what sbopkg-0.38.1 does.

Quote:

there has been an update to sqg (that hasn't been pushed out to a released package yet and likely won't until 15.0 is released) allowing you to generate a custom queuefile name

Indeed, another way to get rid of the interactive prompt would be a command line switch to use the queue when there's both a queue and a package.

bassmadrigal

04-02-2020 05:04 PM

Quote:

Originally Posted by FlinchX (Post 6107177)

Indeed, another way to get rid of the interactive prompt would be a command line switch to use the queue when there's both a queue and a package.

I did have a look at the sbopkg script earlier, and it shouldn't be too hard to add an option to prefer the queue or package, but what I would expect the preferred options to be were already taken. -p and -q are both used for other features. -P is also used, but -Q is available. If you have any suggestions on what letters could be used, let me know and I might be able to throw together a patch and submit a pull request to Willy for it over the weekend. I'd also be able to supply a patch for the sbopkg script that is used for 14.2 (since the pull request would be based on the one used for -current and may not work for our version).

FlinchX

04-02-2020 05:40 PM

I think -Q as in "use the queue when there's both a queue and a package" makes sense.

So far the "prefer package when there's both queue and package" scenario escapes me, I can't think of a situation when this can be useful. It's either "use package" when a package has no dependencies (even if using sqg then, no queue would be generated) or "use queue" when there's both, because a package has dependencies and a queue was generated for it.

Quote:

I'd also be able to supply a patch for the sbopkg script that is used for 14.2

Having a new sbopkg release with such a feature that works with 14.2 would be great, because I'm using 14.2 as playground (didn't get to fiddle with -current yet).

bassmadrigal

04-02-2020 06:10 PM

Quote:

Originally Posted by FlinchX (Post 6107197)

So far the "prefer package when there's both queue and package" scenario escapes me, I can't think of a situation when this can be useful. It's either "use package" when a package has no dependencies (even if using sqg then, no queue would be generated) or "use queue" when there's both, because a package has dependencies and a queue was generated for it.

What if you had created a queue for ffmpeg and built all the packages for it and now you need to rebuild just ffmpeg? You would either need to remove the ffmpeg.sqf file or couldn't run sbopkg without needing to enter 'p' for package.

This could happen if a package has a lot of dependencies and if that package happens to be updated, you may not need to rebuild all the dependencies (especially if it contains something like qt5).

I'm not sure how often I would need to automate sbopkg to even need to specify to use the queue or package, but it would be nice to support both scenarios if we're already supporting one.

FlinchX

04-02-2020 06:25 PM

Now I see what you mean.

Perhaps a sbopkg developer could share some wisdom about the design choices that were made in the beginning. As in, why certain sbopkg options were implemented to accept both packages/queues.

bassmadrigal

04-02-2020 06:59 PM

Quote:

Originally Posted by FlinchX (Post 6107208)

As far as I know, forum member (and SBo admin) willysr is the active developer of sbopkg right now, but I don't know if he is one of the creators or just who is maintaining it now.

willysr

04-02-2020 07:51 PM

I'm just maintaining it. The original author is Chess Griffin, Slakmagik, and Mauro

FlinchX

04-03-2020 01:14 AM

I have submitted a formal issue on github regarding this https://github.com/sbopkg/sbopkg/issues/60

willysr

04-03-2020 01:23 AM

I think you missed the origianl idea of how -B is designed for

Quote:

-B Process the packages or queues without prompting for confirmation first.

So basically you still need to decide whether you want to install a single package or from queue. If you choose queue, it will directly install based on queue without prompting, so it's already as it's designed.

willysr

04-03-2020 01:25 AM

Here's what i mean

without -B:

Code:

sbopkg -i filezilla

Both a queuefile and a package were found with the name "filezilla".



Use (Q)ueuefile, (P)ackage, or (A)bort?: q



###########################################

      New queue process started on:

      Fri Apr  3 13:24:00 WIB 2020

###########################################



+++++++++++++++++++++++++++++++++++++++++++

PRE-CHECK LOG

Using the SBo repository for Slackware 14.2

Queue Process:  Download, build, and install



libfilezilla:

  Checking GPG for libfilezilla.tar.gz ... OK

  Processing libfilezilla 0.16.0-1

  Using original .info file

  Using original SlackBuild file

  No build options selected.



wxGTK3:

  Checking GPG for wxGTK3.tar.gz ... OK

  Processing wxGTK3 3.0.4-2

  Using original .info file

  Using original SlackBuild file

  No build options selected.



filezilla:

  Checking GPG for filezilla.tar.gz ... OK

  Processing filezilla 3.42.1-1

  Using original .info file

  Using original SlackBuild file

  No build options selected.



+++++++++++++++++++++++++++++++++++++++++++



Pre-check complete.



Do you wish to proceed based on the search results above? Packages not

found will be skipped during the process.



(P)roceed or (Q)uit?:

with -B:

Code:

sbopkg -B -i filezilla

Both a queuefile and a package were found with the name "filezilla".



Use (Q)ueuefile, (P)ackage, or (A)bort?: q



###########################################

      New queue process started on:

      Fri Apr  3 13:24:35 WIB 2020

###########################################



+++++++++++++++++++++++++++++++++++++++++++

PRE-CHECK LOG

Using the SBo repository for Slackware 14.2

Queue Process:  Download, build, and install



libfilezilla:

  Checking GPG for libfilezilla.tar.gz ... OK

  Processing libfilezilla 0.16.0-1

  Using original .info file

  Using original SlackBuild file

  No build options selected.



wxGTK3:

  Checking GPG for wxGTK3.tar.gz ... OK

  Processing wxGTK3 3.0.4-2

  Using original .info file

  Using original SlackBuild file

  No build options selected.



filezilla:

  Checking GPG for filezilla.tar.gz ... OK

  Processing filezilla 3.42.1-1

  Using original .info file

  Using original SlackBuild file

  No build options selected.



+++++++++++++++++++++++++++++++++++++++++++



Pre-check complete.







Processing libfilezilla

FlinchX

04-03-2020 01:35 AM

We're having the discussion split between LQ and GitHub, but since you closed the issue there, I'll provide a more detailed answer here.

I'm not contesting how -B works. The problem is that in your example `sbopkg -B -i filezilla`, sbopkg still stops prompting the user for interactive input. Am I missing some less obvious way to make sbopkg fully non-interactive in this situation, for automation purposes?

willysr

04-03-2020 01:48 AM

It still give prompt since it gives the users an option to use queue or directly use a package
if the package doesn't have a queue, it will directly install the package without asking

Code:

sbopkg -B -i libfilezilla



###########################################

      New queue process started on:

      Fri Apr  3 13:47:14 WIB 2020

###########################################



+++++++++++++++++++++++++++++++++++++++++++

PRE-CHECK LOG

Using the SBo repository for Slackware 14.2

Queue Process:  Download, build, and install



libfilezilla:

  Checking GPG for libfilezilla.tar.gz ... OK

  Processing libfilezilla 0.16.0-1

  Using original .info file

  Using original SlackBuild file

  No build options selected.



+++++++++++++++++++++++++++++++++++++++++++



Pre-check complete.







Processing libfilezilla

FlinchX

04-03-2020 01:55 AM

Quote:

Originally Posted by willysr (Post 6107302)

It still give prompt

And this is exactly my problem, I don't want to fill any prompts manually, because this way I can't automate installing stuff with sbopkg. I'd like to be able to tell sbopkg to prefer queues or packages when it finds both with the same name, via command line argument.

willysr

04-03-2020 04:20 AM

Feel free to send a PR

FlinchX

04-03-2020 04:26 AM

@bassmadrigal

What about something like

Code:

-n type When there are queues and packages with same name, treat the names passed to -b/-d/-i

        as of this type. Valid options are: queue, pkg. When this option is set, sbopkg

        running in batch mode won't prompt for a choice.

The -n option isn't taken and there would be just one additional option to handle both cases.

FlinchX

04-03-2020 07:32 PM

This is what I have so far.

Note that this is a patch against 0.38.1, not against the latest git master. As I said above, I'm not using -current, so I don't have a sandbox to test that scenario.

I did a couple of tests in 14.2 and it kind of works, but I'm a shell scripting noob, so feedback is necessary.

@bassmadrigal : if you are attempting to do the same and are planning to provide patches for both 14.2 and -current, you are welcome to use the code.

Code:

--- sbopkg.orig        2016-09-01 16:45:16.000000000 +0000

+++ sbopkg        2020-04-04 00:00:00.000000000 +0000

@@ -4515,6 +4515,8 @@

 unset ALLOW_MULTI    # If set, allow more that one instance of sbopkg running

 unset BUILDLIST      # List of packages to build/install (from CLI)

 unset PIDFILE        # PID

+unset NAME_TYPE      # The policy used to treat names: either queue or package

+                      # (see "-n")

 

 unset BUILD CHK_UPDATES GENSEARCH CHANGELOG OBSOLETESRC GETPKGS

 unset RSYNC SEARCH UPDATE VERSION CUSTOMVER SKIP_INSTALLED VIEW_READMES

@@ -4560,7 +4562,7 @@

 fi

 

 # This is the command line options and help.

-while getopts ":b:BcD:d:e:f:g:hi:kloPpqRrs:uV:v" OPT; do

+while getopts ":b:BcD:d:e:f:g:hi:kln:oPpqRrs:uV:v" OPT; do

    case $OPT in

        b ) # Download, build

            set_type build

@@ -4607,6 +4609,11 @@

            CHANGELOG=1

            unset DIAG

            ;;

+        n ) # How to treat names when there are both queues and packages

+            # with the same name

+            NAME_TYPE=$OPTARG

+            unset DIAG

+            ;;

        o ) # Show obsolete sources

            OBSOLETESRC=1

            unset DIAG

@@ -4666,6 +4673,10 @@

  -i pkg/queue(s) Like '-b', but also install built packages.

  -k              Skip installed packages when building.

  -l              Display the repo's ChangeLog.txt and then quit.

+  -n name_type    When there are queues and packages with the same name, treat

+                  the names passed to -b/-d/-i as of this type. Valid options

+                  are: queue, pkg. When this option is set, sbopkg running in

+                  batch mode won't prompt for a choice.

  -o              Display the obsolete source files & prompt for deletion.

  -P              List uninstalled cached package files & prompt for deletion.

  -p              List installed packages from active repo.

@@ -4713,6 +4724,16 @@

    readonly ON_ERROR

 fi

 

+if [[ -n $NAME_TYPE && \

+      $NAME_TYPE != queue && \

+      $NAME_TYPE != pkg ]]; then

+    echo "Unknown -n specifier -- \"$NAME_TYPE\"" >&2

+    echo "Valid values are: queue, pkg" >&2

+    exit 1

+else

+    readonly NAME_TYPE

+fi

+

 # Check for a good config file and set initial variables

 config_check

 

@@ -4796,19 +4817,27 @@

            fi

            if [[ -r $QUEUEDIR/$PKGBUILD.sqf ]] &&

                    search_package $PKGBUILD; then

-                crunch_fmt "Both a queuefile and a package were found with \

-                    the name \"$PKGBUILD\"."

-                echo

-                while :; do

-                    read $NFLAG \

-                        -ep "Use (Q)ueuefile, (P)ackage, or (A)bort?: "

-                    case $REPLY in

-                        Q|q) parse_queue $QUEUEDIR/$PKGBUILD.sqf; break ;;

-                        P|p) parse_arguments "$PKGBUILD"; break ;;

-                        A|a) exit 1 ;;

-                        *) unknown_response ;;

-                    esac

-                done

+                if [[ -z $NAME_TYPE ]]; then

+                    crunch_fmt "Both a queuefile and a package were found with \

+                        the name \"$PKGBUILD\"."

+                    echo

+                    while :; do

+                        read $NFLAG \

+                            -ep "Use (Q)ueuefile, (P)ackage, or (A)bort?: "

+                        case $REPLY in

+                            Q|q) parse_queue $QUEUEDIR/$PKGBUILD.sqf; break ;;

+                            P|p) parse_arguments "$PKGBUILD"; break ;;

+                            A|a) exit 1 ;;

+                            *) unknown_response ;;

+                        esac

+                    done

+                else

+                    if [ "$NAME_TYPE" == "queue" ]; then

+                        parse_queue $QUEUEDIR/$PKGBUILD.sqf

+                    elif [ "$NAME_TYPE" == "pkg" ]; then

+                        parse_arguments "$PKGBUILD"

+                    fi

+                fi

            else

                if [[ -r $QUEUEDIR/$PKGBUILD.sqf ]]; then

                    # Add an entire queue

Richard Cranium

04-03-2020 08:24 PM

Quote:

Originally Posted by FlinchX (Post 6106885)

I want to use sbopkg, as my example shows, perhaps I should've added that explicitly to the title.

Create a virtual machine with a pristine environment. Install slackrepo. Use slackrepo to build. Use slapt-get to install from your slackrepo virtual machine.

I'm doing exactly that with -current for the one laptop I've got running -current. I've got an actual piece of hardware running a stock Slackware64 14.2 image that does the same for my other machines; I've got the extra hardware, which is why I'm using it for 14.2.

If using sbopkg on a machine with other stuff installed floats your boat, I'm not going to try to stop you.

FlinchX

04-07-2020 09:20 AM

Another trouble I'm running into: at first startup, sbopkg asks interactively to create some missing dirs. I can't find any command line argument that will create those missing directories and exit without asking anything.

ponce

04-07-2020 09:40 AM

it's a variable that you can export via the command line or set directly to "NO" in /etc/sbopkg/sbopkg.conf

Code:

MKDIR_PROMPT=${MKDIR_PROMPT:-NO}

willysr

04-07-2020 09:52 AM

Thanks ponce, i set those NO by default now, so directory will be created automatically for future release of sbopkg

drumz

04-07-2020 02:53 PM

Quote:

Originally Posted by FlinchX (Post 6107308)

You can give the name of the queue file (with extension!) and sbopkg will just use the queue file directly, without prompting:

Code:

sbopkg -k -b ffmpeg.sqf

I don't know of a way to prefer the package over the queue file, however.

FlinchX

04-07-2020 03:42 PM

Quote:

Originally Posted by drumz (Post 6108858)

You can give the name of the queue file (with extension!) and sbopkg will just use the queue file directly, without prompting:

Code:

sbopkg -k -b ffmpeg.sqf

Now that you pointed to this, I see it's explicitly mentioned in the manpage. But it's not obvious from the output of sbopkg -h :(

willysr

04-07-2020 07:54 PM

Code:

sbopkg -h

sbopkg 0.38.1

Usage: sbopkg [OPTIONS] <packagename(s)>

Options are:

  -b pkg/queue(s) Build the specified package(s). If one or more queuefiles

                  are specified, build the packages they refer to.

  -B              Bulk process the queue without confirmation.

  -c              Check for updates to installed packages.

  -D localdir    Location of local copy of the repositories.

  -d pkg/queue(s) Like '-b', but only download sources.

  -e error_action Specify what sbopkg is supposed to do on build errors.

                  Valid options are: ask (default), continue, stop.

  -f file        Override default configuration file with specified file.

  -g package(s)  General search for packages matching string.

  -h              Display this help message.

  -i pkg/queue(s) Like '-b', but also install built packages.

  -k              Skip installed packages when building.

  -l              Display the repo's ChangeLog.txt and then quit.

  -o              Display the obsolete source files & prompt for deletion.

  -P              List uninstalled cached package files & prompt for deletion.

  -p              List installed packages from active repo.

  -q              Quiet some of the command-line output.

  -R              When combined with -b or -i, preview the READMEs of the

                  packages to be built/installed before starting the build

                  process. When combined with -p, show the READMEs of all

                  installed packages from the active repo.

  -r              Sync the remote repository with the local mirror and then

                  quit.

  -S              Use sandbox when building packages.

  -s package(s)  Specific search by specific package and, if found,

                  display package information.

  -u              Check for an update to sbopkg.

  -V repo/branch  Set the repository/branch. The repo is optional and, if

                  not given, sbopkg will try to make the best match,

                  starting with the default repo. For a list of valid repos,

                  issue '-V ?'

  -v              Print sbopkg's version on stdout.



Note: multiple arguments to -b, -g, -i, and -s must be quoted ("pkg1 pkg2") or

can be specified multiple times (-i foo -i bar). If using the latter syntax,

build options may also be passed on the command line on a per app basis using

the -b or -i flags in colon-separated groups (where whitespace must also be

quoted). For example, '-i app:opt1="arg1 arg2":opt2=arg1 app2'

FlinchX

04-08-2020 12:32 PM

I gave it some more thought and now I'm not even sure if I should use the patch that I attached a few posts above. Since it's possible to explicitly tell sbopkg to use a queue by using APPNAME.sqf rather than APPNAME, as @drumz pointed above, then I assume it's also possible to tell sbopkg to use the APPNAME as package, by making sure that there's no queue with the same name as well. This can be done by explicitly removing the queue with the same name if it exists, since generating it again later if needed is not a resource hog. Not having a queue also removes the ambiguous scenario when there's both a package and a queue with the same name.

So:

1. If I want to use APPNAME as queue, I should explicitly pass APPNAME.sqf (generate it if needed) to sbopkg

2. If I want to use APPNAME as package, I should remove APPNAME.sqf if it exists, before passing APPNAME to sbopkg

This would allow to use sbopkg as it is now without the patch applied. This would make sure sbookg will never ask me to use the package or the queue, since I'm explicitly telling it to use the queue when I need that. And this would make sure that whenever I need sbopkg to run the package, I'll never run into ambiguity either, since I'm making sure there's no queue with same name.

Did I get it right this time?

willysr

04-08-2020 09:02 PM

So basically sbopkg will find APPNAME according to the repository and queue based on what's inside queue directory. In the latest sbopkg (in git master), it has an ability to generate custom queue and install it from that custom queue.

I just made a custom queue for virt-manager last night and install it as virt-manager_current.sqf in /var/lib/sbopkg/queue and i can install it using sbopkg -i virt-manager_current

All times are GMT -5. The time now is 02:18 PM.