LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (https://www.linuxquestions.org/questions/slackware-14/)
-   -   slackpkg vs. third-party package repository (https://www.linuxquestions.org/questions/slackware-14/slackpkg-vs-third-party-package-repository-4175427364/)

kikinovak 09-15-2012 04:32 AM

slackpkg vs. third-party package repository
 
Hi,

In my office I have a build box for 32-bit as well as 64-bit packages, that I use when performing installations on client machines. What I normally do is burn these to a CD-Rom and also copy them to a USB stick, so once I've installed a vanilla Slackware system, I can easily add these (currently 170) packages.

Now I wonder if there is a way of doing the following:
  1. Put all my packages on my HTTP and/or FTP mirror
  2. Configure slackpkg to use this mirror
  3. Use slackpkg transparently from there on, e. g. 'slackpkg install xscreensaver' would fetch the package from an official Slackware mirror, but 'slackpkg install lame' would use mine.

My packages replace the odd vanilla Slackware package, things like vte or MPlayer. And they are all tagged "_microlinux" so there's no mistake.

Is there a chance for this to succeed, or is this doomed from the start?

Cheers.

Alien Bob 09-15-2012 05:09 AM

Unfortunately that is not going to work. Slackpkg only works with official packages. The program reads the meta-data files in the Slackware tree (ChangeLog.TXT, PACKAGES.TXT, CHECKSUMS.MD5 etc) and from that, it determines the list of packages which make up a release. You can add to the tree what you want, but it won't be picked up by slackpkg.

Eric

kikinovak 09-15-2012 05:28 AM

Quote:

Originally Posted by Alien Bob (Post 4780871)
Unfortunately that is not going to work. Slackpkg only works with official packages. The program reads the meta-data files in the Slackware tree (ChangeLog.TXT, PACKAGES.TXT, CHECKSUMS.MD5 etc) and from that, it determines the list of packages which make up a release. You can add to the tree what you want, but it won't be picked up by slackpkg.

Thanks for the advice, Eric. Well, KISS as usual (CD-Rom, USB disk, manual download) will do very well too.

On a side note: I *think* my Slackware-based Microlinux Enterprise Desktop must be quite a success. When I first installed our school's network back in 2010, using a beefed-up CentOS 5.3, the students (15 to 20 years old "difficult" kids) commented with words like "lame", "boring", etc. Last year I had a mix of Slackware 13.37 and a hand-built KDE 4.6.5 with many tweaks also. The average student worked on it OK, but described the desktop as "pretty lame", "not as good as Windows" (the latter probably because there were still no games). But this year, my personal mix of Slackware 13.37 plus beefed-up Xfce 4.8 with some nice artwork plus many apps seemed to have changed the critic's voice. Since the first week of september, I only get remarks like "Hey, I like this new system" and "This new system is soooo easy to work with" or "Geee it's really fast". I think I'm on the right track here. (Or maybe I'm just more popular at the school, and the system is still crap :D)

Niki

a4z 09-15-2012 09:31 AM

you can have a look at slapt-get , http://software.jaos.org/

with this you can have your own repository and give it a higher priority than the official ones so packages will be updated with your version

zerouno 09-16-2012 01:13 PM

About a year ago, I wrote a small plugin for slackpkg (called slackpkg+sl) that add the support for third-party repositories.

This plugin was studied for the slacky.eu repository. You can find here the discussion for this plugin (in italian language).

This plugin does NOT modify any files of the original slackpkg.

Note that the plugin automatically set CHECKGPG=off so you can find some warning.

Next I started a second version (called slackpkg+ only) that supports other non-slacky repositories, but I interrupted the develop.

damgar 09-16-2012 02:11 PM

Could Eric's repository mirroring script not be modified to use your repo and also perform the upgradepkg portion as well?

zerouno 09-16-2012 05:27 PM

Quote:

Originally Posted by zerouno (Post 4781736)
Next I started a second version (called slackpkg+ only) that supports other non-slacky repositories, but I interrupted the develop.

The second version is out :) at http://files.z01.eu/pkgs/slackpkg+/
Note: it's a beta version.

It presents some problem with:
- repositories that does not contains a MANIFEST.bz2 (you can obtain a warning, non fatal, in slackpkg update)
- repositories that contains multiple packages with the same name (i.e. repository that contains both 32&64 bit packages or 13.1&13.37&current version of the same package); slackpkg use only the first package in list and skip all other.

I've configured only slacky.eu (64bit) and alienbob repository, but you can change and add repositories from slackpkgplus.conf
You can obtains a large repositories list from http://slakfinder.org

Alien Bob 09-16-2012 05:59 PM

I'll certainly have a look zerouno!

Eric

kikinovak 09-17-2012 12:20 AM

Looks like the most viable solution here is the usual combination of vanilla slackpkg and `whoami`.

Alien Bob 03-09-2013 02:54 PM

I finally had time to look at this. I think there is potential here, but some things can be improved if slackpkg itself gets some updates.

In any case, I changed one line in /usr/libexec/slackpkg/functions.d/slackpkgplus.sh and that is the priority of the repositories. In the original file, the non-Slackware repositories have lowest priority, which is not what I want. If I have a package in a non-Slackware repository (say, glibc's multilib version) I want that to have higher priority than the Slackware package so that the Slackware package can never overwrite the multilib/custom package...

Code:

--- /tmp/slackpkgplus.sh        2013-03-09 21:39:19.327016140 +0100
+++ /usr/libexec/slackpkg/functions.d/slackpkgplus.sh  2013-03-09 21:36:25.851581696 +0100
@@ -8,7 +8,8 @@
  CHECKGPG=off
 
  REPOPLUS=${!MIRRORPLUS[*]}
-  PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
+  #PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
+  PRIORITY=( slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') ${PRIORITY[*]} )
 
  function getfile(){
    local URLFILE

With this little change, and this content in /etc/slackpkg/slackpkgplus.conf
Code:

SLACKPKGPLUS=on

#MIRRORPLUS['slacky']=http://darkstar.ist.utl.pt/slackware/addon/slacky/slackware64-13.37/
MIRRORPLUS['alienbob']=http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/
MIRRORPLUS['multilib']=http://taper.alienbase.nl/mirrors/people/alien/multilib/current/

I am now able to manage updates for Slackware packages, my own repository packages and the multilib packages.
The /etc/slackpkg/blacklist file no longer blacklists my packages:
Code:

# You can blacklist using regular expressions.
#
# Don't use *full* regex here, because all of the following
# will be checked for the regex: series, name, version, arch,
# build and fullname.
#
# This one will blacklist all SBo packages:
[0-9]+_SBo
#[0-9]+compat32
#[0-9]+alien

Of course, the slackpkg functionality that still only works with Slackware repositories is slackpkg's "install-new" but I am able to use "upgrade-all", "search", "file-search" and "info" functions with my own repositories. Cool!

Of course I needed to adapt my repository layout for this to work. That is why for my SlackBuilds repository you see a new URL location with "sbrepos/<slackversion>/<architecture>" instead of the old generic "slackbuilds" because I wanted to leave my original repository directory structure intact. Basically I created a "shadow-repository" using hard-links so that it almost does not use any disk space. The multilib repository now has some additional files which will ebable you to add the URL for a specific Slackware version. See the above example.

Eric

GazL 03-09-2013 06:17 PM

Not sure whether this is of interest to anyone but it's a simple tool I call 'slacklist' that I use to keep my system in step with slackware-current. It's not as sophisticated as slackpkg and is designed only to work with local sources of packages, but it does handle combining multiple sources of packages together before comparing them to what is on the existing system.

As an example, if you keep your local copy of slackware64-current in /local/slackware64-current and your locally built packages in /local/packages then to keep things in sync you could use something along the lines of
Code:

PKGPATH="/local/packages:/local/slackware64-current/slackware64"
export PKGPATH

while read package
do
    gpg --verify ${package}.asc $package && upgradepkg --install-new $package
done < <( slacklist install upgrade )

slacklist remove | xargs -r removepkg

I'd always recommend running the slacklist commands separately first though to see what it is going to do before you run something like the above.


Anyway, it was something I wrote to make my life easier, and as it seems somewhat related to the discussion here I thought I'd throw it out there.

zerouno 03-11-2013 11:05 AM

Hi Eric.

The starting idea for slackpkg+ was to add the only slacky repository (where I'm a contributor), and one-only; in effect the original name was slackypkg next renamed slackpkg+sl, so I wrote a VERY DRAFT plugin.

Only after posting in this thread I modified (aka complete rewrite) the plugin to add multiple repositories but I wrote it in a very short time ;), so it is NOT a mature plugin.

In effect a large problem was with your repository becouse it contains multiple version of the same packages. I was looking for a solution. The same problem was for the slakfinder, for your repository and all other listed in 'mixed' section. The problem was solved in slakfinder 2.0 (never released :() with a method "try to guess", not applicable in slackpkg+. So thanks for reorganize your repository (that, in effect, is the only definable 'ufficial'); I will modify the slakfinder in some days.

I paused to devel slackpkg+ for two reasons. The first is the time that I can spent. The second is becouse to improve really the tool and to add very good feature I need to patch some slackpkg file and I don't want to do that.


Returning to the starting idea, the main assumption was that slackpkg+ was to be used to add packages not presents in slackware main repository, so slackpkg+ 0.2 does not allow to overwrite slackware packages with thirdy part packages.
Who use the multilib repository want implicitly override ALL original packages with packages presents in that repository.
But the problem begins with repositories containing both packages already presents in slackware and packages new.
For example, your restricted_repository contains mplayer and vlc. If I add that repository only for vlc, and put that repository as first priority, slackpkg upgrade-all will substitute the mplayer slackware packages with your mplayer, but I want only to add (and update) vlc.
That problem may be partially solved allowing to configure the order in slackpkgplus.conf so the user can choice
( multilib patches %PKGMAIN extra pasture testing restricted sbrepos )
But the problem continue (with no solution without modify the core&structure of slackpkg) when you have two thirdy party repository containing the same package. For example, both restricted_repository and slacky repository contains the openjdk and apache-ant packages. If I want to install your openjdk I must put your repository in priority; next if I want to install slacky apache-ant, I change the order putting slacky repository in priority. Well; now I type slackpkg upgrade-all and slackpkg ask me to substitute your openjdk with slacky openjdk.
IHMO there are only two solution; both require a structural slackpkg change.
1) at moment the only way to know which repository contains the packages that I installed is the suffix (sl,alien,sbo...); if I'd installed openjdk-....-1alien packages, slackpkg upgrade-all should search only for openjdk-*-*-*alien unless the user specify a parameter '--all' or similar. This open many other problems
2) slackpkg should allow to show multiple packages with the same name in list. This, also, open many other problems
but this means to "write a new (another??) remote repository manager"

Other package manager have similar problem.

slackyd (ufficial tool for slacky repository) have in the configurarion file: "upgrade_replacing_official = yes"
slapt-get allow the suffix ":OFFICIAL" ":PREFERRED" ":CUSTOM"

Regarding the link instead download packages for local repositories, the slackpkg+ is native for remote repositories. I don't known how many people download the entire alien/sbo/slacky/... repository on its pc.

For the .asc support, I remove it for semplicity and also becouse not all thirdy party repository support it.


If you know solutions or have some idea... you are welcome :D.

phenixia2003 03-11-2013 11:09 AM

Hello,

Quote:

Originally Posted by Alien Bob (Post 4908161)
Of course, the slackpkg functionality that still only works with Slackware repositories is slackpkg's "install-new" but I am able to use "upgrade-all", "search", "file-search" and "info" functions with my own repositories. Cool!

search and file-search are buggy. I sent a patch to fix it at slackpkg's website.

Quote:

Originally Posted by Alien Bob (Post 4908161)
Of course I needed to adapt my repository layout for this to work. That is why for my SlackBuilds repository you see a new URL location with "sbrepos/<slackversion>/<architecture>" instead of the old generic "slackbuilds" because I wanted to leave my original repository directory structure intact. Basically I created a "shadow-repository" using hard-links so that it almost does not use any disk space. The multilib repository now has some additional files which will ebable you to add the URL for a specific Slackware version. See the above example.

Excuse me but I don't understand why you try to add support for the multilib in a tool which was not designed to manage this. If you think that slackpkg is a better tool to manage the multilib than compat32pkg/multilibpkg (is there something wrong with these tools I should know ?), then I guess it would be better to directly integrate the multilib in Slackware64.


Greetings.

--
SeB

Alien Bob 03-11-2013 04:10 PM

Quote:

Originally Posted by phenixia2003 (Post 4909245)
Excuse me but I don't understand why you try to add support for the multilib in a tool which was not designed to manage this. If you think that slackpkg is a better tool to manage the multilib than compat32pkg/multilibpkg (is there something wrong with these tools I should know ?), then I guess it would be better to directly integrate the multilib in Slackware64.

I am not attacking your ompat32pkg/multilibpkg tool. I am showing how slackpkg can be expanded with custom functionality. It was designed to be able to do this.

In the end, people are free to use whatever tool they feel most comfortable with.

Eric

phenixia2003 03-15-2013 10:23 AM

Hello,

Quote:

Originally Posted by zerouno (Post 4909242)
But the problem continue (with no solution without modify the core&structure of slackpkg) when you have two thirdy party repository containing the same package. For example, both restricted_repository and slacky repository contains the openjdk and apache-ant packages. If I want to install your openjdk I must put your repository in priority; next if I want to install slacky apache-ant, I change the order putting slacky repository in priority. Well; now I type slackpkg upgrade-all and slackpkg ask me to substitute your openjdk with slacky openjdk.


Just an idea, which does not solve the problem related to "upgrade-all", but allows the user to install (or upgrade) a package from a given repository without to have to manually change the variable PRIORITY :

Code:

slackpkg install [repository-name:]package-name

slackpkg upgrade [repository-name:]package-name

With this extension, to install vlc from alien repository, user would have to issue the command :

Code:

$ slackpkg install alienbob:vlc
And, to install openjdk from slacky repository, he will issue that :

Code:

$ slackpkg install slacky:openjdk
To upgrade the installed apache-ant (from slacky) to the latest version, user will issue the command :

Code:

$ slackpkg upgrade slacky:apache-ant
To provide these features, you only need to add the code below after yours in slackpkgplus.sh :

Code:

if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

        NEWINPUTLIST=""

        for pref in $INPUTLIST ; do
                if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                        repository=$(echo "$pref" | cut -f1 -d":")
                        package=$(echo "$pref" | cut -f2- -d":")

                        PRIORITY=( slackpkgplus_${repository} ${PRIORITY[*]} )
                else
                        package=$pref
                fi
               
                NEWINPUTLIST="$NEWINPUTLIST $package"
        done

        INPUTLIST=$NEWINPUTLIST
fi

IMPORTANT NOTES:

1. because the granularity of PRIORITY is too low, this can lead to some trouble when trying to install different packages from different repository at the same time. For instance :

Code:

      (1) $ slackpkg install alienbob:vlc slacky:openjdk


      (2) $ slackpkg install slacky:openjdk alienbob:vlc

When (1) is executed, vlc from alienbob and openjdk from slacky could be installed. But, in the case of (2), vlc and openjdk will be taken from alienbob repository.

As said above, this is because the granularity of PRIORITY is too low, it only allow to give priority to directory, not package. Note that compat32pkg comes with something like "PRIORITY" (called PKGS_PRECEDENCE_ORDER) which allows higher granularity. I guess the function FilterByPrecedenceOrder() from compat32pkg could be adapted to slackpkg+, but I don't really have the time to look at this.

2. The code above accepts only repositories with letters and/or digit (ie line echo "$pref" | grep -q ...). Don't hesitate to improve this.


Hope this help.

--
SeB

phenixia2003 03-15-2013 12:01 PM

Hello,

Quote:

Originally Posted by Alien Bob (Post 4908161)
Of course, the slackpkg functionality that still only works with Slackware repositories is slackpkg's "install-new"

The code below manages the installation of any compat32 package (from multilib repository) which is not currently installed. In other words, this allows the installation of newly added package(s) to the layer-32, or the entire layer-32.

Code:

if [ "$CMD" == "install-new" ] ; then
 
  ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
 
  grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

  NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
 
  if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
    LIST=""
   
    for pkg in $NEWCOMPAT32PKGS ; do
      LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
    done
  fi
fi

Hope this help.

--
SeB

zerouno 03-19-2013 09:29 AM

Thanks for suggests.

Remain to solve the upgrade&upgrade-all problem.


If a package X is present both in slackware and in alienbob repository, and I install it from alienbob, next an upgrade-all should upgrade it only if change in alienbob repository and not in slackware.

I think that the suffix (1mt 1sl 1alien ... ) is sufficient to know the repository

phenixia2003 03-20-2013 06:13 AM

Hello,

Quote:

Originally Posted by zerouno (Post 4914516)
Thanks for suggests.

Remain to solve the upgrade&upgrade-all problem.


If a package X is present both in slackware and in alienbob repository, and I install it from alienbob, next an upgrade-all should upgrade it only if change in alienbob repository and not in slackware.

I think that the suffix (1mt 1sl 1alien ... ) is sufficient to know the repository

Well, I finally updated givepriority() so that, it now supports enhanced priority rules (priority given to package(s) from a given repository). This kind of priority uses the following syntax:

Code:

repository_name:pattern
For instance, to give priority to package "openjdk" from slacky repository, you will use the rule "slackpkgplus_slacky:openjdk".

Code:

  # Found packages in repository.
  # This function selects the package from the higher priority
  # repository directories.
  #
  # This Modified version supports enhanced priority rule (priority
  # given to package(s) from a given repository). This kind of priority
  # uses the following syntax :
  #
  #  repository_name:pattern
  #
  #
function givepriority {
        local DIR
        local ARGUMENT=$1
        local PKGDATA
        local CPRIORITY
        local DIR
        local PKG

        unset NAME
        unset FULLNAME
        unset PKGDATA
       
        for CPRIORITY in ${PRIORITY[@]} ; do
                [ "$PKGDATA" ] && break
               
                if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                  DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
                  PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
               
                  if echo "$ARGUMENT" | grep -q "$PAT" ; then
                    PKGDATA=( $(grep "^${DIR} ${PAT} " ${TMPDIR}/pkglist) )
                  fi
                else       
                  PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                fi
                 
                if [ "$PKGDATA" ]; then
                  NAME=${PKGDATA[1]}
                  FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
                fi
        done
}

So, with this change, the code in my previous post for the extension that I suggested (ie slackpkg install [repository-name:]package-name) must be changed as below :


Code:

if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

        NEWINPUTLIST=""

        for pref in $INPUTLIST ; do
                if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                        repository=$(echo "$pref" | cut -f1 -d":")
                        package=$(echo "$pref" | cut -f2- -d":")

                        PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
                else
                        package=$pref
                fi
               
                NEWINPUTLIST="$NEWINPUTLIST $package"
        done

        INPUTLIST=$NEWINPUTLIST       
fi

Now, about the upgrade/upgrade-all problem. Since givepriority() now supports enhanced priorities, I suggest to allow the user to defines prioritary packages in a variable declared into slackpkgplus.conf. These priorities would be merged with the variable PRIORITY at slackpkg+ init time.

For instance if the user want apache-ant from slacky, and openjdk from alienbob, it will defines this inside slackpkgplus.conf :

Code:

PKGS_PRIORITY=( slacky:apache-ant alienbob:openjdk )
The code to merge these priorities with those defined by PRIORITY will be as below :

Code:

  for pp in ${PKGS_PRIORITY[*]} ; do
    repository=$(echo "$pp" | cut -f1 -d":")
    package=$(echo "$pp" | cut -f2- -d":")
   
    if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
      PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
    fi
  done

With this, when the user will issue the commands :
Code:

$ slackpkg install apache-ant
$ slackpkg install openjdk

slackpkg+ will install the apache-ant from slacky and openjdk from alienbob.

And, when the user will issue the commands :
Code:

$ slackpkg upgrade apache-ant
$ slackpkg upgrade openjdk
$ slackpkg upgrade-all

slackpkg+ will get the latest apache-ant from slacky repository, and the latest openjdk from alienbob repository.

Note, the user can always install/upgrade another version using the extension I suggest. For instance, the command :

Code:

$ slackpkg install slacky:openjdk
$ slackpkg upgrade slacky:openjdk

Will imply install/upgrade of openjdk using the version from slacky repository.

However, since PKGS_PRIORITY includes "alienbob:openjdk", the commands :

Code:

$ slackpkg upgrade openjdk
$ slackpkg upgrade-all

Will imply upgrade of openjdk using the version from alienbob repository.

Well, this starts to be a bit complex, so, to summarize you will find the full code (and a version of slackpkgplus.conf that includes the variable PKGS_PRIORITY) below :

Code:


declare -A MIRRORPLUS
if [ -e /etc/slackpkg/slackpkgplus.conf ];then
  . /etc/slackpkg/slackpkgplus.conf
fi

if [ "$SLACKPKGPLUS" = "on" ];then
  # If CHECKGPG is "on", the system will FAIL the GPG signature of extra repository
  # Use MD5 check instead
  CHECKGPG=off

  REPOPLUS=${!MIRRORPLUS[*]}
  PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
 
    # -- merge priorities from PKGS_PRIORITY with PRIORITY ...
 
  for pp in ${PKGS_PRIORITY[*]} ; do
    repository=$(echo "$pp" | cut -f1 -d":")
    package=$(echo "$pp" | cut -f2- -d":")
   
    if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
      PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
    fi
  done
 
  function getfile(){
    local URLFILE
    URLFILE=$1

    if echo $URLFILE|grep -q /slackpkgplus_;then
      PREPO=$(echo $URLFILE|sed -r 's#^.*/slackpkgplus_([^/]+)/.*$#\1#')
      URLFILE=$(echo $URLFILE|sed "s#^.*/slackpkgplus_$PREPO/#${MIRRORPLUS[$PREPO]}#")
    fi
   
    $DOWNLOADER $2 $URLFILE
    if [ $(basename $1) = "CHECKSUMS.md5" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}CHECKSUMS.md5
        egrep -e ^[a-f0-9]{32} $2-tmp|sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> $2
      done
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}ChangeLog.txt
        echo $PREPO $(md5sum $2-tmp|awk '{print $1}') >>$2
        rm $2-tmp
      done
    fi

  }

    # Found packages in repository.
    # This function selects the package from the higher priority
    # repository directories.
    #
    # This Modified version supports enhanced priority rule (priority
    # given to package(s) from a given repository). This kind of priority
    # uses the following syntax :
    #
    #  repository_name:pattern
    #
    #
  function givepriority {
          local DIR
          local ARGUMENT=$1
          local PKGDATA
          local CPRIORITY
          local DIR
          local PKG

          unset NAME
          unset FULLNAME
          unset PKGDATA
         
          for CPRIORITY in ${PRIORITY[@]} ; do
                  [ "$PKGDATA" ] && break
                 
                  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
                    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
                 
                    if echo "$ARGUMENT" | grep -q "$PAT" ; then
                      PKGDATA=( $(grep "^${DIR} ${PAT} " ${TMPDIR}/pkglist) )
                    fi
                  else       
                    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                  fi
                   
                  if [ "$PKGDATA" ]; then
                    NAME=${PKGDATA[1]}
                    FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
                  fi
          done
  }

  if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

          NEWINPUTLIST=""

          for pref in $INPUTLIST ; do
                  if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                          repository=$(echo "$pref" | cut -f1 -d":")
                          package=$(echo "$pref" | cut -f2- -d":")

                          PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
                  else
                          package=$pref
                  fi
                 
                  NEWINPUTLIST="$NEWINPUTLIST $package"
          done

          INPUTLIST=$NEWINPUTLIST
         
#    echo "PRIORITY  set to ${PRIORITY[*]}"
#    read userkey
  fi

  if [ "$CMD" == "install-new" ] ; then
    ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
   
    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

    NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
   
    if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
      LIST=""
     
      for pkg in $NEWCOMPAT32PKGS ; do
        LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
      done
    fi
  fi
 
fi

Here is the slackpkgplus.conf I used :

Code:

SLACKPKGPLUS=on

#MIRRORPLUS['slacky']=http://darkstar.ist.utl.pt/slackware/addon/slacky/slackware64-14.0/
MIRRORPLUS['slacky']=repository.slacky.eu/slackware64-14.0/
MIRRORPLUS['alienbob']=http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/
MIRRORPLUS['multilib']=http://taper.alienbase.nl/mirrors/people/alien/multilib/14.0/

PKGS_PRIORITY=( slacky:apache-ant alienbob:openjdk )

Hope this help.


--
SeB

phenixia2003 03-22-2013 11:01 AM

Hello,

I slightly modified the function givepriority() so that it is now possible to use regexp in enhanced priority rules. With that, you can give priority for an entire repository through the variable PKGS_PRIORITY. For instance, to give priority to the multilib repository, you simply have to add this in slackpkgplus.conf :

Code:

PKGS_PRIORITY=( multilib:.* )
I also changed the code that merges the content of the variables PKGS_PRIORITY and PRIORITY. This was needed to preserve the priority orders given in PKGS_PRIORITY. Indeed, my previous code was wrong and the priorities were reversed.

Here is the full code (changes in bold) :

Code:


declare -A MIRRORPLUS
if [ -e /etc/slackpkg/slackpkgplus.conf ];then
  . /etc/slackpkg/slackpkgplus.conf
fi

if [ "$SLACKPKGPLUS" = "on" ];then
  # If CHECKGPG is "on", the system will FAIL the GPG signature of extra repository
  # Use MD5 check instead
  CHECKGPG=off

  REPOPLUS=${!MIRRORPLUS[*]}
  PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
 

    # -- merge priorities from PKGS_PRIORITY with PRIORITY, as needed ...
 
  if [ ! -z "$PKGS_PRIORITY" ] ; then
    PREFIX=""
   
    for pp in ${PKGS_PRIORITY[*]} ; do
      repository=$(echo "$pp" | cut -f1 -d":")
      package=$(echo "$pp" | cut -f2- -d":")
   
      if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
        if [ -z "$PREFIX" ] ; then
          PREFIX=( slackpkgplus_${repository}:$package )
        else
          PREFIX=( ${PREFIX[*]} slackpkgplus_${repository}:$package )
        fi
      fi
    done
   
    [ ! -z "$PREFIX" ] && PRIORITY=( ${PREFIX[*]} ${PRIORITY[*]} )
  fi

 
  function getfile(){
    local URLFILE
    URLFILE=$1

    if echo $URLFILE|grep -q /slackpkgplus_;then
      PREPO=$(echo $URLFILE|sed -r 's#^.*/slackpkgplus_([^/]+)/.*$#\1#')
      URLFILE=$(echo $URLFILE|sed "s#^.*/slackpkgplus_$PREPO/#${MIRRORPLUS[$PREPO]}#")
    fi
   
    $DOWNLOADER $2 $URLFILE
    if [ $(basename $1) = "CHECKSUMS.md5" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}CHECKSUMS.md5
        egrep -e ^[a-f0-9]{32} $2-tmp|sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> $2
      done
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}ChangeLog.txt
        echo $PREPO $(md5sum $2-tmp|awk '{print $1}') >>$2
        rm $2-tmp
      done
    fi

  }


    # Found packages in repository.
    # This function selects the package from the higher priority
    # repository directories.
    #
    # This Modified version supports enhanced priority rule (priority
    # given to package(s) from a given repository). This kind of priority
    # uses the following syntax :
    #
    #  repository_name:pattern
    #
    #
  function givepriority {
          local DIR
          local ARGUMENT=$1
          local PKGDATA
          local CPRIORITY
          local DIR
          local PKG

          unset NAME
          unset FULLNAME
          unset PKGDATA
         
          for CPRIORITY in ${PRIORITY[@]} ; do
                  [ "$PKGDATA" ] && break
                 
                  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:]" ; then
                    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
                    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
                 

                    if echo "$ARGUMENT" | grep -q "$PAT" ; then
                      PKGDATA=( $(grep "^${DIR} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                       
                    fi
                  else       
                    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                  fi
                   
                  if [ "$PKGDATA" ]; then
                    NAME=${PKGDATA[1]}
                    FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
                  fi
          done
  }

  if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

          NEWINPUTLIST=""

          for pref in $INPUTLIST ; do
                  if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                          repository=$(echo "$pref" | cut -f1 -d":")
                          package=$(echo "$pref" | cut -f2- -d":")

                          PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
                  else
                          package=$pref
                  fi
                 
                  NEWINPUTLIST="$NEWINPUTLIST $package"
          done

          INPUTLIST=$NEWINPUTLIST
         
  fi

  if [ "$CMD" == "install-new" ] ; then
    ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
   
    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

    NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
   
    if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
      LIST=""
     
      for pkg in $NEWCOMPAT32PKGS ; do
        LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
      done
    fi
  fi
 
fi

Cheers.

--
SeB

zerouno 04-23-2013 06:14 AM

I want to repackage slackpkg+

Eric, what do you think of the phenixia2003 ideas?

Alien Bob 04-23-2013 12:58 PM

I have not implemented phenixia2003's ideas here locally, yet. But his ideas sound good and would give slackpkg+ some needed flexibility.
One thing that I would implement but my ARM port does not leave me any time: to allow more than just the Slackware GPG key. It would be very welcome if slackpg+ would be able to use a separate GPG key for every repository.

Other than that, I have been using slackpkg+ on my desktop machine (with a lot of alien and SBo packages, and multilib) without a single issues (except the failure to use GPG checking).

Eric

zerouno 04-24-2013 09:31 AM

Eric, have you "modularized" the "Restricted-SlackBuilds" repository?

Alien Bob 04-24-2013 10:01 AM

Quote:

Originally Posted by zerouno (Post 4938008)
Eric, have you "modularized" the "Restricted-SlackBuilds" repository?

No I have not yet done so, and I do not have short-term plans in that direction.

Eric

phenixia2003 04-24-2013 10:07 AM

Hello,

Quote:

Originally Posted by Alien Bob (Post 4937394)
It would be very welcome if slackpg+ would be able to use a separate GPG key for every repository.

I had an idea about that. This is not perfect, but I guess that's a good starting point.

First, you have to add the array MIRRORPLUSKEY in slackpkgplus.conf. In this array you must declare the key of each repositories declared into MIRRORPLUS. When a repository does not provide a GPG-KEY, then you must use the keyword NULL. Here is the slackpkgplus.conf I use.

Code:

SLACKPKGPLUS=on

MIRRORPLUS['slacky']=repository.slacky.eu/slackware64-14.0/
MIRRORPLUS['alienbob']=http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/
MIRRORPLUS['multilib']=http://taper.alienbase.nl/mirrors/people/alien/multilib/14.0/

MIRRORPLUSKEY['slacky']="NULL"
MIRRORPLUSKEY['alienbob']="Eric Hameleers <alien@slackware.com>"
MIRRORPLUSKEY['multilib']="Eric Hameleers <alien@slackware.com>"

PKGS_PRIORITY=( multilib:.* slacky:apache-ant alienbob:openjdk )

With this, you must use the new slackpkgplus.sh below (changes since my last post are in bold):

Code:

declare -A MIRRORPLUS
declare -A MIRRORPLUSKEY

if [ -e /etc/slackpkg/slackpkgplus.conf ];then
  . /etc/slackpkg/slackpkgplus.conf
fi

if [ "$SLACKPKGPLUS" = "on" ];then

    # -- In this version CHECKGPG can be set to "on". However, CHECKMD5
    #    must be set to off, because the CHECKSUMS.md5 files on alien
    #    repositories does not includes entries for .asc files.
    #
  CHECKGPG=on
  CHECKMD5=off

 
  REPOPLUS=${!MIRRORPLUS[*]}
  PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
 

    # -- merge priorities from PKGS_PRIORITY with PRIORITY, as needed ...
 
  if [ ! -z "$PKGS_PRIORITY" ] ; then
    PREFIX=""
   
    for pp in ${PKGS_PRIORITY[*]} ; do
      repository=$(echo "$pp" | cut -f1 -d":")
      package=$(echo "$pp" | cut -f2- -d":")
   
      if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
        if [ -z "$PREFIX" ] ; then
          PREFIX=( slackpkgplus_${repository}:$package )
        else
          PREFIX=( ${PREFIX[*]} slackpkgplus_${repository}:$package )
        fi
      fi
    done
   
    [ ! -z "$PREFIX" ] && PRIORITY=( ${PREFIX[*]} ${PRIORITY[*]} )
  fi

 
  function getfile(){
    local URLFILE
    URLFILE=$1

    if echo $URLFILE|grep -q /slackpkgplus_;then
      PREPO=$(echo $URLFILE|sed -r 's#^.*/slackpkgplus_([^/]+)/.*$#\1#')
      URLFILE=$(echo $URLFILE|sed "s#^.*/slackpkgplus_$PREPO/#${MIRRORPLUS[$PREPO]}#")
    fi
   
    $DOWNLOADER $2 $URLFILE
    if [ $(basename $1) = "CHECKSUMS.md5" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}CHECKSUMS.md5
        egrep -e ^[a-f0-9]{32} $2-tmp|sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> $2
      done
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}ChangeLog.txt
        echo $PREPO $(md5sum $2-tmp|awk '{print $1}') >>$2
        rm $2-tmp
      done
    fi

  }


    # Found packages in repository.
    # This function selects the package from the higher priority
    # repository directories.
    #
    # This Modified version supports enhanced priority rule (priority
    # given to package(s) from a given repository). This kind of priority
    # uses the following syntax :
    #
    #  repository_name:pattern
    #
    #
  function givepriority {
          local DIR
          local ARGUMENT=$1
          local PKGDATA
          local CPRIORITY
          local DIR
          local PKG

          unset NAME
          unset FULLNAME
          unset PKGDATA
         
          for CPRIORITY in ${PRIORITY[@]} ; do
                  [ "$PKGDATA" ] && break
                 
                  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:]" ; then
                    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
                    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
                 

                    if echo "$ARGUMENT" | grep -q "$PAT" ; then
                      PKGDATA=( $(grep "^${DIR} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                       
                    fi
                  else       
                    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                  fi
                   
                  if [ "$PKGDATA" ]; then
                    NAME=${PKGDATA[1]}
                    FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
                  fi
          done
  }
 
        # -- Loads the GPG-KEY from the Slackware repository
        #
  function loadSlackwareGpgKey() {
        getfile ${SOURCE}GPG-KEY $TMPDIR/gpgkey
        gpg --yes --batch --delete-key "$SLACKKEY" &>/dev/null
        gpg --import $TMPDIR/gpgkey &>/dev/null && \
        echo -e "\t\t\tSlackware Linux Project's GPG key added"
  }
 
    # -- Loads the GPG-KEY from the external repository ($1)
    #
  function loadRepositoryGpgKey() {
        local l_repo=$1
        local l_url=${MIRRORPLUS[$l_repo]}
        local l_mirrorkey=${MIRRORPLUSKEY[$l_repo]}
       
        if [ "$l_mirrorkey" == "NULL" ] ; then
                echo ""
                echo "(WARNING) ************************************************"
                echo "(WARNING) *    No key for external repository $l_repo    *"
                echo "(WARNING) ************************************************"
                echo ""
                return
        fi
         
        getfile ${l_url}GPG-KEY $TMPDIR/gpgkey
       
        gpg --yes --batch --delete-key "$l_mirrorkey" &>/dev/null
        gpg --import $TMPDIR/gpgkey &>/dev/null && \
        echo -e "\t\t\t$l_repo GPG key added"
  }
 
  # -- This override updatefilelists() from original core-functions.sh
  #
  #
function updatefilelists()
{
        if checkchangelog ; then
                echo -e "\
\n\t\tNo changes in ChangeLog.txt between your last update and now.\n\
\t\tDo you really want to download all other files (y/N)? \c"
                answer
                if [ "$ANSWER" != "Y" ] && [ "$ANSWER" != "y" ]; then
                        cleanup
                fi
        fi
        echo
        cp ${TMPDIR}/ChangeLog.txt ${WORKDIR}/ChangeLog.txt

        #
        # Download MANIFEST, FILELIST.TXT and CHECKSUMS.md5
        #

        # That will be download MANIFEST.bz2 files
        #
        echo -e "\t\tList of all files"
        for i in ${PRIORITY[@]} ; do
       
           
            # -- IMPORTANT:
            #    Enhanced priority rules (ie: repository:name) must be
            #    ignored, otherwise we'll get some nasty error messages.
               
          if ! echo "$i" | grep -q "[a-zA-Z0-9]\+[:]" ; then

                getfile ${SOURCE}${i}/MANIFEST.bz2 $TMPDIR/${i}-MANIFEST.bz2 && \
                DIRS="$DIRS $i"
          fi
        done

        ISOK="1"
        echo -e "\t\tChecksums"
       
          # -- IMPORTANT:
          #      The version of getfile() for slackpkg+ concatenates the files
          #      CHECKSUMS.md5 from the selected repositories. Therefore, the
          #      code to check signature of this file must be removed.
          #
        getfile ${SOURCE}CHECKSUMS.md5 ${TMPDIR}/CHECKSUMS.md5               
       
#        getfile ${SOURCE}CHECKSUMS.md5.asc ${TMPDIR}/CHECKSUMS.md5.asc
        if ! [ -e "${TMPDIR}/CHECKSUMS.md5" ]; then
                echo -e "\
\n\t\tWARNING: Your mirror appears incomplete and is missing the\n\
\t\t        CHECKSUMS.md5 file. We recommend you change your mirror\n\
\t\t        so that package integrity can be verified against \n\
\t\t        CHECKSUMS.md5.\n"
                sleep 10
#        else
#                if [ "$CHECKGPG" = "on" ]; then
#                        ISOK=$(checkgpg ${TMPDIR}/CHECKSUMS.md5)
#                        if [ "$ISOK" = "0" ]; then
#                                read userkey
#                                rm $TMPDIR/CHECKSUMS.md5
#                                rm $TMPDIR/CHECKSUMS.md5.asc
#                                echo -e "\
#\n\t\tERROR: Verification of the  gpg signature on CHECKSUMS.md5\n\
#\t\t      failed! This could mean that the file is out of date\n\
#\t\t      or has been tampered with.\n"
#                                cleanup
#                        fi
#                elif [ "$SLACKKEY" != "" ]; then
#                        echo -e "\
#\n\t\tWARNING: Without CHECKGPG, we can't check if this file is\n\
#\t\t        signed by:\n\
#\n\t\t        $SLACKKEY.\n\
#\n\t\t        Enabling CHECKGPG is highly recommended for best\n\
#\t\t        security.\n"
#                                sleep 10
#                fi

        fi

        ISOK="1"
        echo -e "\t\tPackage List"
        getfile ${SOURCE}FILELIST.TXT ${TMPDIR}/FILELIST.TXT
        if [ "$CHECKMD5" = "on" ]; then
                CHECKSUMSFILE=${TMPDIR}/CHECKSUMS.md5
                ISOK=$(checkmd5 ${TMPDIR}/FILELIST.TXT)
        fi
        if [ "$ISOK" = "1" ]; then
                if ! [ -e $WORKDIR/LASTUPDATE ]; then
                        echo "742868196" > $WORKDIR/LASTUPDATE
                fi
                LASTUPDATE=$(cat $WORKDIR/LASTUPDATE)
                ACTUALDATE=$(date -d "$(head -1 $TMPDIR/FILELIST.TXT)" "+%s")
                if [ $ACTUALDATE -lt $LASTUPDATE ]; then
                        echo -e "\
\n\t\tFILELIST.TXT seems to be older than the last one.\n\
\t\tDo you really want to continue (y/N)? \c"
                        answer
                        if [ "$ANSWER" != "Y" ] && [ "$ANSWER" != "y" ]; then
                                cleanup
                        fi
                        echo
                fi
                echo $ACTUALDATE > $WORKDIR/LASTUPDATE
        else
                rm $TMPDIR/FILELIST.TXT
        fi
       
        if [ -e $TMPDIR/CHECKSUMS.md5 ]; then
                FILELIST="$TMPDIR/CHECKSUMS.md5"
        elif [ -e $TMPDIR/FILELIST.TXT ]; then
                if [ "$ISOK" = "0" ]; then
                        echo -e "\
\n\t\tERROR: CHECKSUMS.md5 signature doesn't match!\n\
\t\t      We strongly recommend that you change your mirror\n\
\t\t      to prevent security problems.\n"
                        cleanup
                fi
                sleep 10
                  FILELIST="$TMPDIR/FILELIST.TXT"
        else
                echo -e "\
\n\t\tERROR: No CHECKSUMS.md5 and no FILELIST.TXT.\n\
\t\t      We strongly recommend that you change your mirror\n\
\t\t      to prevent security problems.\n"
                cleanup
        fi

        # Download all PACKAGES.TXT files
        #
        echo -e "\t\tPackage descriptions"
        for i in $DIRS; do
                getfile ${SOURCE}${i}/PACKAGES.TXT $TMPDIR/${i}-PACKAGES.TXT
        done

        # Format FILELIST.TXT
        #
        echo -e "\tFormatting lists to slackpkg style..."
        echo -e "\t\tPackage List: using $( basename $FILELIST ) as source"
        grep "\.t[blxg]z$" $FILELIST| \
                awk -f /usr/libexec/slackpkg/pkglist.awk |\
                sed -e 's/^M//g' > ${TMPDIR}/pkglist
        cp ${TMPDIR}/pkglist ${WORKDIR}/pkglist               

        # Create the slackware tree under TEMP directory
        for i in $( cut -f7 -d\  ${WORKDIR}/pkglist | sort -u ) ; do
          if ! [ -d ${TEMP}/${i} ]; then
            mkdir -p ${TEMP}/${i}
          fi
        done

        # Format MANIFEST
        #
               
        # bunzip and concatenate all MANIFEST files
        #
        MANFILES=""
        for i in $DIRS; do
                echo "(DEBUG) calling bunzip2 with $TMPDIR/${i}-MANIFEST.bz2"
                read userkey
               
                if [ -s $TMPDIR/${i}-MANIFEST.bz2 ] ; then
                        bunzip2 -c $TMPDIR/${i}-MANIFEST.bz2 | awk -f /usr/libexec/slackpkg/filelist.awk | \
                                gzip > ${TMPDIR}/${i}-filelist.gz
                fi
        done
        cp ${TMPDIR}/*-filelist.gz ${WORKDIR}/

        if [ -r ${WORKDIR}/filelist.gz ]; then
                rm ${WORKDIR}/filelist.gz
                ln -s ${WORKDIR}/${MAIN}-filelist.gz ${WORKDIR}/filelist.gz
        fi

        # Concatenate PACKAGE.TXT files
        #
        echo -e "\t\tPackage descriptions"
        for i in $DIRS; do
                cat $TMPDIR/${i}-PACKAGES.TXT >> $TMPDIR/PACKAGES.TXT
        done
        cp $TMPDIR/PACKAGES.TXT ${WORKDIR}/PACKAGES.TXT

        if [ -e $TMPDIR/CHECKSUMS.md5 ]; then
                cp $TMPDIR/CHECKSUMS.md5 $WORKDIR/CHECKSUMS.md5 2>/dev/null
        fi

        if [ -e $TMPDIR/CHECKSUMS.md5.asc ]; then
                cp $TMPDIR/CHECKSUMS.md5.asc \
                        $WORKDIR/CHECKSUMS.md5.asc 2>/dev/null
        fi
}
 
 
  if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

          NEWINPUTLIST=""

          for pref in $INPUTLIST ; do
                  if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                          repository=$(echo "$pref" | cut -f1 -d":")
                          package=$(echo "$pref" | cut -f2- -d":")

                          PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
                  else
                          package=$pref
                  fi
                 
                  NEWINPUTLIST="$NEWINPUTLIST $package"
          done

          INPUTLIST=$NEWINPUTLIST
         
  fi

  if [ "$CMD" == "install-new" ] ; then
    ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
   
    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

    NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
   
    if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
      LIST=""
     
      for pkg in $NEWCOMPAT32PKGS ; do
        LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
      done
    fi
  fi
 
    # -- New code to handle the command "update"...
   
  if [ "$CMD" == "update" ] ; then
        if [ "$UPARG" = "gpg" ] || [ "$GPGFIRSTTIME" = "0" ]; then
                #
                # Creates .gnupg directory if doesn't exist
                # without this dir, gpg got an error.
                #
                if ! [ -e ~/.gnupg ]; then
                        mkdir ~/.gnupg
                fi
       
                loadSlackwareGpgKey

                for PREPO in $REPOPLUS; do
                        loadRepositoryGpgKey $PREPO
                done
               
                if [ "$UPARG" = "gpg" ]; then
                        cleanup
                fi
        fi
       
        echo "Updating the package lists..."
        updatefilelists
       
        cleanup
        exit
  fi

 
fi

Important:
  • I ran into some troubles because the CHECKSUMS.md5 on repositories "sbrepos" and "multilib" does not includes entries for .asc files. To workaround this, I set "CHECKMD5" to off (but "CHECKGPG" is on).
  • The getfile() in slackpkg+ concatenates the files CHECKSUMS.md5 from the selected repositories. Therefore, the code to check signature - in updatefilelists() - has been removed (colored in red in the code above). This should/could be moved into getfile().
  • I fixed a problem caused by the "enhanced priority rules" in updatefilelists() that leads to some nasty error messages (The correction is colored in green in the code above)
Now, with this new version, if you install apache-ant from alienbob repository you will get this :

Code:

slackpkg -batch=on -default_answer=y install alienbob:apache-ant

Looking for apache-ant in package list. Please wait... DONE

apache-ant-1.8.2-noarch-1alien.tgz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

Package: apache-ant-1.8.2-noarch-1alien.tgz
--2013-04-24 16:39:10--  http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz
Resolving taper.alienbase.nl (taper.alienbase.nl)... 98.158.153.254
Connecting to taper.alienbase.nl (taper.alienbase.nl)|98.158.153.254|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8382749 (8.0M) [application/x-gzip]
Saving to: '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz'

100%[================================================>] 8,382,749    192KB/s  in 55s   

2013-04-24 16:40:11 (149 KB/s) - '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz' saved [8382749/8382749]

--2013-04-24 16:40:11--  http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz.asc
Resolving taper.alienbase.nl (taper.alienbase.nl)... 98.158.153.254
Connecting to taper.alienbase.nl (taper.alienbase.nl)|98.158.153.254|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 198 [text/plain]
Saving to: '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz.asc'

100%[================================================>] 198        --.-K/s  in 0s     

2013-04-24 16:40:11 (615 KB/s) - '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz.asc' saved [198/198]

        Installing apache-ant-1.8.2-noarch-1alien...
Verifying package apache-ant-1.8.2-noarch-1alien.tgz.
Installing package apache-ant-1.8.2-noarch-1alien.tgz:
PACKAGE DESCRIPTION:
# Apache Ant (Java-based build tool analogous to Make)
#
# Ant is a platform-independent build tool for java.
#
# Ant is extended using Java classes.  The configuration files are
# XML- based, calling out a target tree where tasks get executed.
# Each task is run by an object that implements a Task interface.
#
# Homepage: http://ant.apache.org/
#
Executing install script for apache-ant-1.8.2-noarch-1alien.tgz.
Package apache-ant-1.8.2-noarch-1alien.tgz installed.

Hope this helps.


Greetings.

--
SeB

Alien Bob 04-24-2013 03:19 PM

Quote:

Originally Posted by phenixia2003 (Post 4938033)
I ran into some troubles because the CHECKSUMS.md5 on repositories "sbrepos" and "multilib" does not includes entries for .asc files. To workaround this, I set "CHECKMD5" to off (but "CHECKGPG" is on).

I will fix that ASAP. There will be entries in CHECKSUMS.md5 for all meta files, including the .asc file. I am testing an updated gen_repos_files.sh script at the moment.

Eric

Alien Bob 04-24-2013 04:54 PM

Quote:

Originally Posted by Alien Bob (Post 4938027)
No I have not yet done so, and I do not have short-term plans in that direction.

Well I changed my mind, and the restricted_slackbuilds repository now has a 'modularized' version as well. See http://taper.alienbase.nl/mirrors/pe...icted_sbrepos/

Quote:

Originally Posted by Alien Bob (Post 4938183)
I will fix that ASAP. There will be entries in CHECKSUMS.md5 for all meta files, including the .asc file. I am testing an updated gen_repos_files.sh script at the moment.

Fixed. Both the regular and the restricted slackbuilds repositories have been updated with CHECKSUMS.md5 files, that contain md5sums for all package-related files (like the .asc files).

Eric

phenixia2003 04-25-2013 05:21 AM

2 Attachment(s)
Hello,

Quote:

Originally Posted by Alien Bob (Post 4938225)
Fixed. Both the regular and the restricted slackbuilds repositories have been updated with CHECKSUMS.md5 files, that contain md5sums for all package-related files (like the .asc files).

Great, so I removed the line "CHECKMD5=off" in my code and all is ok. However,I got another trouble because there's no file ChangeLog.txt at http://taper.alienbase.nl/mirrors/pe...os/14.0/x86_64.

I also made some changes to get something better.

First, I think that's not a good idea to have one array for the urls of repositories, and another for the keys.

So, I choose to merge these data into the array MIRRORPLUS. Now, when you declare a new repository you will have to supply the key through a field "?key=<key>" right after the url.

For instance, now my slackpkgplus.conf is as below :
Code:

SLACKPKGPLUS=on

  # MIRRORPLUS allow you to declare external repositories. Its syntax is as below:
  #
  #  MIRRORPLUS['<repository-name>']="<repository-url>?key=<repository-key>"
  #
  # When a repository has no GPG-KEY, the field "?key=<repository-key>" can
  # be undefined or must be set to "?key=null"
  # 
MIRRORPLUS['alienbob']="http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/?key=Eric Hameleers <alien@slackware.com>"
MIRRORPLUS['multilib']="http://taper.alienbase.nl/mirrors/people/alien/multilib/14.0/?key=Eric Hameleers <alien@slackware.com>"
MIRRORPLUS['slacky']="repository.slacky.eu/slackware64-14.0/?key=null"
#MIRRORPLUS['robby']="http://rlworkman.net/pkgs/14.0/?key=Robby Workman <rworkman@slackware.com>"

PKGS_PRIORITY=( multilib:.* slacky:apache-ant alienbob:openjdk )

Note that Robby's repository works but only the i486 packages can be installed because the files required by slackpkg (CHECKSUMS.md5,...) are not at architecture level.

At runtime, slackpkgplus.sh parses the array MIRRORPLUS to extract the urls and keys of the defined repositories.

When a repository has no key, and you run "slackpkg update gpg" with CHECKGPG turned on, then a warning is printed on the standard output :
Code:

(WARNING) ************************************************************************************
(WARNING) * There is no GPG-KEY for the external repository <reponame> while CHECKGPG is on. *
(WARNING) * Therefore, any attempt to install/upgrade a package from this repository        *
(WARNING) * will fail.                                                                      *
(WARNING) ************************************************************************************

You will find a copy of slackpkgplus.conf and slackpkgplus.sh in attachment.

Greetings.

--
SeB

TobiSGD 04-25-2013 07:26 AM

phenixia2003: I have copied your script (of course with removing the .txt) to /usr/libexec/slackpkg/functions.d and the config file to /etc/slackpkg/, then I have edited the config file so that it points to my local mirrors and commented out the repos I don't use. Is there anything else I have to do to make that work?
Code:

slackpkg update
is just ignoring the additional repos, no error messages also.

phenixia2003 04-25-2013 08:54 AM

2 Attachment(s)
Hello,

Quote:

Originally Posted by TobiSGD (Post 4938596)
phenixia2003: I have copied your script (of course with removing the .txt) to /usr/libexec/slackpkg/functions.d and the config file to /etc/slackpkg/, then I have edited the config file so that it points to my local mirrors and commented out the repos I don't use. Is there anything else I have to do to make that work?
Code:

slackpkg update
is just ignoring the additional repos, no error messages also.

slackpkg+ was not supporting local mirrors. This is fixed in the new version in attachment (do not forget to remove the .2.txt).

To declare a local mirror In slackpkgplus.conf, you must prepend its url by "file:". For instance, if you have a local mirror in /home/myname/myrepository/14.0", you must declare it as below :

Code:



MIRRORPLUS['localrepo']="file:/home/myname/myrepository/14.0?key=your key"

if your local mirror has no gpg-key, do not put the field "?key=...", or set this to "?key=null". But keep in mind that in this case, you must set CHECKGPG to "off", or run slackpkg with option -checkgpg=off.

I tested this with a local copy of Eric's multilib repository (with -checkmd5=off since there's no md5 for the .asc files on this repository).

If you run into any trouble, let me know.

Greetings.
--
SeB

Alien Bob 04-25-2013 09:20 AM

Quote:

Originally Posted by phenixia2003 (Post 4938658)
slackpkg+ was not supporting local mirrors. This is fixed in the new version in attachment (do not forget to remove the .2.txt).

To declare a local mirror In slackpkgplus.conf, you must prepend its url by "file:". For instance, if you have a local mirror in /home/myname/myrepository/14.0", you must declare it as below :

Code:



MIRRORPLUS['localrepo']="file:/home/myname/myrepository/14.0?key=your key"


Slackpkg uses a "file://" syntax for that, please try to follow the original syntax where possible. Your solution only uses one slash.

Quote:

Eric's multilib repository (with -checkmd5=off since there's no md5 for the .asc files on this repository).
I will re-generate the multilib repository files with my new script, so that the CHECKSUMS.md5 will have entries for the .asc files.

Eric

TobiSGD 04-25-2013 09:44 AM

My local mirror is served via Apache from my home-server to my machines, so the missing support for file://-URLs was not the problem.
I just forgot to make slackpkgplus.sh executable, now it is working as it should. :doh:
Thanks for your help and the great work.

TobiSGD 04-25-2013 09:51 AM

Shouldn't have been so fast, it is not working as it should, I am unable to install any package from the third party repositories:
Code:

root ~ ☺ # slackpkg info vlc

Added external repository alienbob :
                URL=http://192.168.178.201/alien/x86_64/
                KEY=Eric Hameleers <alien@slackware.com>

Added external repository restricted :
                URL=http://192.168.178.201/restricted/x86_64/
                KEY=Eric Hameleers <alien@slackware.com>

PACKAGE NAME:  vlc-2.0.6-x86_64-2alien.txz
PACKAGE MIRROR:  http://www.slackware.com/~alien/slackbuilds/
PACKAGE LOCATION:  ./vlc
PACKAGE SIZE (compressed):  23288 K
PACKAGE SIZE (uncompressed):  91750 K
PACKAGE DESCRIPTION:
vlc: vlc (multimedia player for various audio and video formats)
vlc:
vlc: VLC media player is a highly portable multimedia player for various
vlc: audio and video formats (MPEG-1, MPEG-2, MPEG-4, DivX, mp3, ogg, ...)
vlc: as well as DVDs, VCDs, and various streaming protocols.
vlc: It can also be used as a server to stream in unicast or multicast in
vlc: IPv4 or IPv6 on a high-bandwidth network.
vlc:
vlc:
vlc: vlc home: http://www.videolan.org/vlc/
vlc:

PACKAGE NAME:  vlc-2.0.6-x86_64-2alien.txz
PACKAGE MIRROR:  http://taper.alienbase.nl/mirrors/people/alien/restricted_slackbuilds/
PACKAGE LOCATION:  ./vlc
PACKAGE SIZE (compressed):  24012 K
PACKAGE SIZE (uncompressed):  94120 K
PACKAGE DESCRIPTION:
vlc: vlc (multimedia player for various audio and video formats)
vlc:
vlc: VLC media player is a highly portable multimedia player for various
vlc: audio and video formats (MPEG-1, MPEG-2, MPEG-4, DivX, mp3, ogg, ...)
vlc: as well as DVDs, VCDs, and various streaming protocols.
vlc: It can also be used as a server to stream in unicast or multicast in
vlc: IPv4 or IPv6 on a high-bandwidth network.
vlc:
vlc:
vlc: vlc home: http://www.videolan.org/vlc/
vlc:


root ~ ☺ # slackpkg install restricted:vlc

Added external repository alienbob :
                URL=http://192.168.178.201/alien/x86_64/
                KEY=Eric Hameleers <alien@slackware.com>

Added external repository restricted :
                URL=http://192.168.178.201/restricted/x86_64/
                KEY=Eric Hameleers <alien@slackware.com>

Looking for vlc in package list. Please wait... DONE

No packages match the pattern for install. Try:

        /usr/sbin/slackpkg reinstall|upgrade


root ~ ☺ # slackpkg install alienbob:vlc 

Added external repository alienbob :
                URL=http://192.168.178.201/alien/x86_64/
                KEY=Eric Hameleers <alien@slackware.com>

Added external repository restricted :
                URL=http://192.168.178.201/restricted/x86_64/
                KEY=Eric Hameleers <alien@slackware.com>

Looking for vlc in package list. Please wait... DONE

No packages match the pattern for install. Try:

        /usr/sbin/slackpkg reinstall|upgrade

So slackpkg knows that there are two VLC packages, but it doesn't install any of them. Tried the same with wine, same result.

phenixia2003 04-25-2013 11:15 AM

Hello,

Quote:

Originally Posted by Alien Bob (Post 4938669)
Slackpkg uses a "file://" syntax for that, please try to follow the original syntax where possible. Your solution only uses one slash.

You're right. I will fix this.



Quote:

Originally Posted by Alien Bob (Post 4938669)
I will re-generate the multilib repository files with my new script, so that the CHECKSUMS.md5 will have entries for the .asc files.

Thanks for that.


Quote:

Originally Posted by TobiSGD (Post 4938688)
Shouldn't have been so fast, it is not working as it should, I am unable to install any package from the third party repositories

That's what I get when a package is already installed. Try with "slackpkg upgrade" instead. Anyway, there's a problem when using restricted and alien repositories at the same time: it always selects packages from "alien" repository and never from "restricted". That's strange because I didn't encounter that kind of problem when using slacky and alien repositories at the same time. I will look at this asap.

Greetings.

--
SeB

TobiSGD 04-25-2013 12:45 PM

Quote:

Originally Posted by phenixia2003 (Post 4938749)
That's what I get when a package is already installed. Try with "slackpkg upgrade" instead.

I uninstalled VLC before testing this. I have the same problem when I try to install Wine, which definitely never was installed on this machine.

Quote:

Anyway, there's a problem when using restricted and alien repositories at the same time: it always selects packages from "alien" repository and never from "restricted".
That would be a progress for me, for now it refuses to install any of the packages, even if I disable the restricted repository (of course with running slackpkg update afterwards).

zerouno 04-26-2013 02:40 AM

ops I'd not see that there was other updates from my last visits.
I developed the following to add gpg supports.

Now I see your code.

Code:

# Thanks to AlienBob and phenixia2003 (on LQ) for contributing


declare -A MIRRORPLUS
if [ -e /etc/slackpkg/slackpkgplus.conf ];then
  . /etc/slackpkg/slackpkgplus.conf
fi

if [ "$SLACKPKGPLUS" = "on" ];then
  # If CHECKGPG is "on", the system will FAIL the GPG signature of extra repository
  # Use MD5 check instead
#  CHECKGPG=off

  REPOPLUS=${!MIRRORPLUS[*]}
  PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
 

    # -- merge priorities from PKGS_PRIORITY with PRIORITY, as needed ...
 
  if [ ! -z "$PKGS_PRIORITY" ] ; then
    PREFIX=""
   
    for pp in ${PKGS_PRIORITY[*]} ; do
      repository=$(echo "$pp" | cut -f1 -d":")
      package=$(echo "$pp" | cut -f2- -d":")
   
      if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
        if [ -z "$PREFIX" ] ; then
          PREFIX=( slackpkgplus_${repository}:$package )
        else
          PREFIX=( ${PREFIX[*]} slackpkgplus_${repository}:$package )
        fi
      fi
    done
   
    [ ! -z "$PREFIX" ] && PRIORITY=( ${PREFIX[*]} ${PRIORITY[*]} )
  fi

 
  function checkgpg() {
  set -x
    gpg --verify ${1}.asc ${1} 2>/dev/null && echo "1" || echo "0"
    if [ "$(basename $1)" == "CHECKSUMS.md5" ];then
      for PREPO in $REPOPLUS;do
        egrep -e ^[a-f0-9]{32} ${TMPDIR}/CHECKSUMS.md5-$PREPO|sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> ${TMPDIR}/CHECKSUMS.md5
      done
    fi
    set +x
  }
  function getfile(){
    local URLFILE
    URLFILE=$1

    if echo $URLFILE|grep -q /slackpkgplus_;then
      PREPO=$(echo $URLFILE|sed -r 's#^.*/slackpkgplus_([^/]+)/.*$#\1#')
      URLFILE=$(echo $URLFILE|sed "s#^.*/slackpkgplus_$PREPO/#${MIRRORPLUS[$PREPO]}#")
    fi
   
    $DOWNLOADER $2 $URLFILE
    if [ $(basename $1) = "MANIFEST.bz2" ];then
      if [ ! -s $2 ];then
        echo -n|bzip2 -c >$2
      fi
    fi
    if [ $(basename $1) = "CHECKSUMS.md5" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-$PREPO ${MIRRORPLUS[${PREPO/slackpkgplus_}]}CHECKSUMS.md5
      done
    fi
    if [ $(basename $1) = "CHECKSUMS.md5.asc" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER ${TMPDIR}/CHECKSUMS.md5-$PREPO.asc ${MIRRORPLUS[${PREPO/slackpkgplus_}]}CHECKSUMS.md5.asc
        if [ $? -eq 0 ];then
          if [ $(checkgpg ${TMPDIR}/CHECKSUMS.md5-$PREPO) -ne 1 ];then
            echo
            echo "                        !!! F A T A L !!!"
            echo "    Repository '$PREPO' FAILS to check CHECKSUMS.md5 signature"
            echo "    The file may be corrupted or the gpg key may be not valid."
            echo "    Remember to import keys by launching 'slackpkg update gpg'."
            echo
            sleep 5
            echo > ${TMPDIR}/CHECKSUMS.md5
          fi
        else
          echo
          echo "                  !!! W A R N I N G !!!"
          echo "    Repository '$PREPO' does NOT supports signature checking"
          echo "    You SHOULD to disable GPG check by setting 'CHECKGPG=off'"
          echo "    in /etc/slackpkg/slackpkg.conf"
          echo
          sleep 5
        fi
      done
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}PACKAGES.TXT
        echo $PREPO $(md5sum $2-tmp|awk '{print $1}') >>$2
        rm $2-tmp
      done
    fi
    if [ $(basename $1) = "GPG-KEY" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}GPG-KEY
        if [ $? -eq 0 ];then
          gpg --import $2-tmp
        else
          echo
          echo "                  !!! W A R N I N G !!!"
          echo "    Repository '$PREPO' does NOT contains the GPG-KEY"
          echo "    You SHOULD to disable GPG check by setting 'CHECKGPG=off'"
          echo "    in /etc/slackpkg/slackpkg.conf"
          echo
          sleep 5
        fi
        rm $2-tmp
      done
    fi


  }


    # Found packages in repository.
    # This function selects the package from the higher priority
    # repository directories.
    #
    # This Modified version supports enhanced priority rule (priority
    # given to package(s) from a given repository). This kind of priority
    # uses the following syntax :
    #
    #  repository_name:pattern
    #
    #
  function givepriority {
          local DIR
          local ARGUMENT=$1
          local PKGDATA
          local CPRIORITY
          local DIR
          local PKG

          unset NAME
          unset FULLNAME
          unset PKGDATA
         
          for CPRIORITY in ${PRIORITY[@]} ; do
                  [ "$PKGDATA" ] && break
                 
                  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:]" ; then
                    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
                    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
                 

                    if echo "$ARGUMENT" | grep -q "$PAT" ; then
                      PKGDATA=( $(grep "^${DIR} ${ARGUMENT} " ${TMPDIR}/pkglist) )

                    fi
                  else
                    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                  fi
                   
                  if [ "$PKGDATA" ]; then
                    NAME=${PKGDATA[1]}
                    FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
                  fi
          done
  }

  if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

          NEWINPUTLIST=""

          for pref in $INPUTLIST ; do
                  if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                          repository=$(echo "$pref" | cut -f1 -d":")
                          package=$(echo "$pref" | cut -f2- -d":")

                          PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
                  else
                          package=$pref
                  fi
                 
                  NEWINPUTLIST="$NEWINPUTLIST $package"
          done

          INPUTLIST=$NEWINPUTLIST
         
  fi

  if [ "$CMD" == "install-new" ] ; then
    ls -1 /var/log/packages/*compat32 2>/dev/null | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
   
    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

    NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
   
    if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
      LIST=""
     
      for pkg in $NEWCOMPAT32PKGS ; do
        LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
      done
    fi
  fi
 
fi

Code:

# Enable (on) / Disable (off) slackpkg+
SLACKPKGPLUS=on

# If two or more repositories contains some same packages, you can specify
# from which repository you prefer to search it.
# The syntax is "<repository_name>:<package_name>"
# Accepts regular expressions. To give order to an entire repository use "<repository_name>:.*"

# Examples:
#PKGS_PRIORITY=( slacky:apache-ant alienbob:openjdk )
#
# If you want a multilib system, uncomment the multilib repository and set:
#PKGS_PRIORITY=( multilib:.* )


# Define mirrors (uncomment one or more mirror)
# Note: You MUST disable gpg check if you select the 'salixext' or 'zerouno' !!!
#

# Slackware 14.0 - x86_64
#MIRRORPLUS['multilib']=http://taper.alienbase.nl/mirrors/people/alien/multilib/14.0/
MIRRORPLUS['alienbob']=http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/
MIRRORPLUS['slacky']=http://repository.slacky.eu/slackware64-14.0/
MIRRORPLUS['salixos']=http://download.salixos.org/x86_64/14.0/
MIRRORPLUS['salixext']=http://people.salixos.org/ralvex/packages/x86_64/14.0/

# Slackware 14.0 - i386
#MIRRORPLUS['alienbob']=http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86/
#MIRRORPLUS['slacky']=http://repository.slacky.eu/slackware-14.0/
#MIRRORPLUS['salixos']=http://download.salixos.org/i486/14.0/
#MIRRORPLUS['salixext']=http://people.salixos.org/ralvex/packages/i486/14.0/

# Slackware current - x86_64
#MIRRORPLUS['multilib']=http://taper.alienbase.nl/mirrors/people/alien/multilib/current/
MIRRORPLUS['zerouno']=http://www.z01.eu/repo-slack/slackware64-current/
MIRRORPLUS['slackel']=http://www.slackel.gr/repo/x86_64/current/

# Slackware current - i386
#MIRRORPLUS['slackel']=http://www.slackel.gr/repo/i486/current/

For now I know only alien and slacky repositories that fully supports gpg.

phenixia2003 04-26-2013 02:41 AM

Hello,

Quote:

Originally Posted by TobiSGD (Post 4938783)
I uninstalled VLC before testing this. I have the same problem when I try to install Wine, which definitely never was installed on this machine.

I looked at this and, I think I have found the reason of this issue. Indeed "slackpkg info" gets data about packages from the file PACKAGES.TXT, while "slackpkg search/install/..." uses an internal file built from the file CHECKSUMS.md5.

Is your local mirrors include all these files :
Code:

CHECKSUMS.md5
MANIFEST.bz2
PACKAGES.TXT
FILELIST.TXT

if this is not the case, you should add the missing ones, and run "slackpkg update", then, redo "slackpkg install restricted:vlc". This should give you a list with "vlc". Note that if all is ok, and you install "vlc", you will get the version from the repository "alienbob" and not from "restricted". This is because both of these file have exactly the same version and build number. I have an idea to fix this and I'm going to implement it.

Otherwise, If you have all those files above, or if "slackpkg install restricted:vlc" throws the message "No packages match the pattern", could you post the output of "slackpkg update" ?


Greetings.

--
SeB

zerouno 04-26-2013 03:23 AM

It's a good code, but my original idea was to overwrite less code possible (to not rewrite).

So instead to overwrite the entire updatefilelists(), I solved in getfile() all problems you solved in red and green points

1) repositories that does not contains ChangeLog.txt
2) repositories that does not contains MANIFEST.bz2
3) repositories that does not contains GPG-KEY or CHECKSUMS.md5.asc
4) loading third-party CHECKSUMS.md5.asc


p.s.: now slacky repository fully supports GPG

TobiSGD 04-26-2013 03:41 AM

1 Attachment(s)
Quote:

Originally Posted by phenixia2003 (Post 4939128)
Hello,



I looked at this and, I think I have found the reason of this issue. Indeed "slackpkg info" gets data about packages from the file PACKAGES.TXT, while "slackpkg search/install/..." uses an internal file built from the file CHECKSUMS.md5.

Is your local mirrors include all these files :
Code:

CHECKSUMS.md5
MANIFEST.bz2
PACKAGES.TXT
FILELIST.TXT

if this is not the case, you should add the missing ones, and run "slackpkg update", then, redo "slackpkg install restricted:vlc". This should give you a list with "vlc". Note that if all is ok, and you install "vlc", you will get the version from the repository "alienbob" and not from "restricted". This is because both of these file have exactly the same version and build number. I have an idea to fix this and I'm going to implement it.

Otherwise, If you have all those files above, or if "slackpkg install restricted:vlc" throws the message "No packages match the pattern", could you post the output of "slackpkg update" ?


Greetings.

--
SeB

My repositories are 1:1 copies of AlienBob's, I rsync them once a day, but I have checked and those files exist in both repositories.
Output of
Code:

slackpkg update
is attached to this post.

zerouno 04-26-2013 03:46 AM

This should to solve the Workman repository, but I've not tested it.
Code:

      X86_64=$(ls /var/log/packages/aaa_base*x86_64*|head -1 2>/dev/null)
      for PREPO in $REPOPLUS;do
        if [ -z "$X86_64" ];then
          egrep -e ^[a-f0-9]{32} ${TMPDIR}/CHECKSUMS.md5-$PREPO|egrep -- "-(x86_64|noarch)-" |sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> ${TMPDIR}/CHECKSUMS.md5
        else
          egrep -e ^[a-f0-9]{32} ${TMPDIR}/CHECKSUMS.md5-$PREPO|egrep -v -- "-(x86_64|arm)-" |sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> ${TMPDIR}/CHECKSUMS.md5
        fi
      done


zerouno 04-26-2013 07:38 AM

Found a BUG in slackpkg! (non plus)

The bug i tested in slackpkg search but may be extended in other sections.

This may the reason becouse slackpkg+ does not work with vlc.


The bug:

when you search a package with following requirements, the package is NOT listed.
1) exists a package called 'something-packagesearched' (note the minus sign)
2) the packages 'something-packagesearched' is listed prior than 'packagesearched' in pkglist

A package affected is 'glib':

Code:

# grep -w glib /var/lib/slackpkg/pkglist
slackware64 dbus-glib 0.100.2 x86_64 1 dbus-glib-0.100.2-x86_64-1 ./slackware64/l txz
slackware64 glib 1.2.10 x86_64 3 glib-1.2.10-x86_64-3 ./slackware64/l txz
slackware64 glib-networking 2.34.2 x86_64 1 glib-networking-2.34.2-x86_64-1 ./slackware64/l txz
# slackpkg search glib
The list below shows all packages with name matching "glib".

[ installed ] - glibc-solibs-2.17-x86_64-2
[ installed ] - glibc-zoneinfo-2013b-noarch-2
[ installed ] - dbus-glib-0.100.2-x86_64-1
[ installed ] - glib-networking-2.34.2-x86_64-1
[ installed ] - glib2-2.34.3-x86_64-1
[ installed ] - glibc-2.17-x86_64-2
[ installed ] - glibc-i18n-2.17-x86_64-2
[ installed ] - glibc-profile-2.17-x86_64-2
[ installed ] - taglib-1.8-x86_64-1
[ installed ] - taglib-extras-1.0.1-x86_64-1

The problem is in core-functions.sh near line 700:

Code:

for i in ${PRIORITY[@]}; do
        PKGS=$(grep "^${i}.*${PATTERN}" \
                ${TMPDIR}/pkglist | cut -f6 -d\ )
        for FULLNAME in $PKGS ; do
          echo -e "\n'$FULLNAME - $LIST'"
                NAME=$(cutpkg ${FULLNAME})
                echo $LIST | \
                        grep -qwe "${NAME}-[^-]\+-\(${ARCH}\|fw\|noarch\)-[^-]\+" && \
                        continue
                LIST="$LIST ${FULLNAME}"
        done
done

The 'grep -qwe' is used to avoid duplicates in $LIST, but the grep search for "-w ${NAME}-..." that match in "something-${NAME}-version-..." if it is already in $LIST, and 'dbus-glib' is already in list when 'glib' is checked.
The solution is grep -qwe "^${NAME}-[^-]\+-\(${ARCH}\|fw\|noarch\)-[^-]\+", but I don't known how many occurs there are in all slackpkg functions.

vlc in slackpkg+ is not listed becouse exists at least 'npapi-vlc'.

phenixia2003 04-26-2013 08:28 AM

Hello,

Quote:

Originally Posted by zerouno (Post 4939252)
Found a BUG in slackpkg! (non plus)

vlc in slackpkg+ is not listed becouse exists at least 'npapi-vlc'.

I have sent this patch for that bug last February.

But I think the issue encountered by TobiSGD is elsewhere because, even without this patch, I have no problem with "slackpkg install alienbob:vlc" (or restricted:vlc) on my side.


--
SeB

zerouno 04-26-2013 08:36 AM

slackpkg search is the only affected from that bug?

phenixia2003 04-26-2013 10:14 AM

Quote:

Originally Posted by zerouno (Post 4939289)
slackpkg search is the only affected from that bug?

When I wrote this patch I looked to see if install/upgrade was affected too, but this was not the case. In fact, as opposed to "install|update|reinstall" the list generated by "search|file-search" cannot contain packages with the same basename.

I guess "search|file-search" would have to be overwritten in slackpkg+, otherwise users will get wrong information.

--
SeB

zerouno 04-26-2013 10:21 AM

Quote:

Originally Posted by phenixia2003 (Post 4939340)
I guess "search|file-search" would have to be overwritten in slackpkg+, otherwise users will get wrong information.

we are rewritting the entire slackpkg :D

zerouno 04-26-2013 10:32 AM

Uploaded beta1:
slackpkg+-0.9beta1-noarch-1.txz

Done some merging code.

TobiSGD 04-27-2013 05:17 AM

Quote:

Originally Posted by zerouno (Post 4939352)
Uploaded beta1:
slackpkg+-0.9beta1-noarch-1.txz

Done some merging code.

Whatever caused my problems, it was fixed with installing that package (which by the way is not a Slackware conform package, it simply overwrote my configuration).

Thanks all for your good work.

phenixia2003 04-27-2013 10:51 AM

1 Attachment(s)
Hello,

Quote:

Originally Posted by zerouno (Post 4939352)
Uploaded beta1:
slackpkg+-0.9beta1-noarch-1.txz

Done some merging code.

First, I would say : Nice work :hattip:

I have a fix for the following issue:

In the repositories "alienbob" and "restricted", there are different versions of ffmpeg and vlc but the packages have the same version and build number.

When REPOPLUS=( .... restricted alienbob ... ) and you run "slackpkg -batch=on -default-answer=y install alienbob:vlc-2.0.6", slackpkg installs the "vlc" from "restricted" instead of "alienbob" :
Code:

slackpkg -batch=on -default_answer=y install alienbob:vlc-2.0.6

Looking for vlc-2.0.6 in package list. Please wait... DONE

vlc-2.0.6-x86_64-2alien.txz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

Package: vlc-2.0.6-x86_64-2alien.txz
--2013-04-27 15:53:10--  http://taper.alienbase.nl/mirrors/people/alien/restricted_sbrepos/14.0/x86_64/vlc/vlc-2.0.6-x86_64-2alien.txz

Now when REPOPLUS=( ... alienbob restricted ... ) and you run "slackpkg -batch=on -default_answer=y install restricted:vlc-2.0.6", slackpkg installs "vlc" from "alienbob" instead of "restricted" :

Code:

slackpkg -batch=on -default_answer=y install restricted:vlc-2.0.6

Looking for vlc-2.0.6 in package list. Please wait... DONE

vlc-2.0.6-x86_64-2alien.txz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

Package: vlc-2.0.6-x86_64-2alien.txz
--2013-04-27 15:57:18--  http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/vlc/vlc-2.0.6-x86_64-2alien.txz

This is the consequence of two things :

1) packages data in /var/lib/slackpkg/pkglist are ordered according to REPOPLUS.

IMPORTANT:
When REPOPLUS is changed, the data in /var/lib/slackpkg/pkglist will be re-ordered at next "slackpkg update".
So, when REPOPLUS=( ... alienbob restricted ...), data about alienbob packages are found before the data about restricted packages, and vice-versa when REPOPLUS=( ... restricted alienbob ... )


2) When getpkg() - in /usr/libexec/slackpkg/core-functions.sh) is called, it selects, the 1st line in ${TMPDIR}/pkglist (a copy of /var/lib/slackpkg/pkglist created at runtime) that matches the argument passed to "slackpkg install" (ex vlc):
Code:

PKGNAME=( $(grep -w -m 1 -- "${1/%.t[blxg]z/}" ${TMPDIR}/pkglist) )
This can be solved in two ways :

1) Modify the build number in the repository "alienbob" or "restricted". For instance, build number could start at "1" for "alienbob", and at "100" for "restricted". This is the simplest, but this could be a pain for Eric, I guess.

2) apply the patch in attachment and shown below to slackpkg+slackpkg+-0.9beta1-noarch-1.txz
Code:

--- slackpkgplus.sh        2013-04-27 17:08:00.985845628 +0200
+++ slackpkgplus-new-givepriority.sh        2013-04-27 17:07:00.560165835 +0200
@@ -153,7 +153,10 @@
 
  }
 
-
+    # Global variable required by givepriority()
+    #
+  PRIORITYIDX=1

    # Found packages in repository.
    # This function selects the package from the higher priority
    # repository directories.
@@ -183,11 +186,30 @@
                  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:]" ; then
                    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
                    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
-                 
-
-                    if echo "$ARGUMENT" | grep -q "$PAT" ; then
-                      PKGDATA=( $(grep "^${DIR} ${ARGUMENT} " ${TMPDIR}/pkglist) )
-                       
+                   
+                                # ARGUMENT is always a basename. But PAT can be:
+                                #  1. a regular expression (ie .*)
+                                #  2. a basename (openjdk)
+                                #  3. a partial (or complete) package name (vlc-2.0.6, ).
+                                #
+                                # The current "enhanced priority rule" is applied :
+                                #  + In case (1) and (2) when ARGUMENT contains the pattern PAT
+                                #  + In the case (3) when ARGUMENT starts the pattern PAT.
+                                #
+                    if echo "$ARGUMENT" | grep -q "$PAT" || echo "$PAT" | grep "^$ARGUMENT" ; then
+                          PKGDATA=""
+                          PKGINFOS=$(grep -n -m 1 "^${DIR} ${ARGUMENT} " ${TMPDIR}/pkglist)
+                         
+                          if [ ! -z "$PKGINFOS" ] ; then
+                            LINEIDX=$(echo "$PKGINFOS" | cut -f1 -d":")
+                                PKGDATA=( $(echo "$PKGINFOS" | cut -f2- -d":") )
+                               
+                                        # -- move the line at #LINEIDX to #PRIORITYIDX and
+                                        #    increment PRIORITYIDX
+                                        #
+                                sed -i --expression "${LINEIDX}d" --expression "${PRIORITYIDX}i${PKGDATA[*]}" ${TMPDIR}/pkglist
+                                (( PRIORITYIDX++ ))
+                          fi
                    fi
                  else       
                    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )

Once this patch applied, you will get the expected behavior :

When REPOPLUS ( .... restricted alienbob ... ) and you run "slackpkg -batch=on -default-answer=y install alienbob:vlc-2.0.6", slackpkg installs "vlc" from alienbob :
Code:

slackpkg -batch=on -default_answer=y install alienbob:vlc-2.0.6

Looking for vlc-2.0.6 in package list. Please wait... vlc-2.0.6
DONE

vlc-2.0.6-x86_64-2alien.txz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

Package: vlc-2.0.6-x86_64-2alien.txz
--2013-04-27 17:10:06--  http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/vlc/vlc-2.0.6-x86_64-2alien.txz

When REPOPLUS ( .... alienbob restricted ... ) and you run "slackpkg -batch=on -default-answer=y install restricted:vlc-2.0.6", then slackpkg installs "vlc" from restricted :

Code:

slackpkg -batch=on -default_answer=y install restricted:vlc-2.0.6

Looking for vlc-2.0.6 in package list. Please wait... vlc-2.0.6
DONE

vlc-2.0.6-x86_64-2alien.txz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

Package: vlc-2.0.6-x86_64-2alien.txz
--2013-04-27 17:13:59--  http://taper.alienbase.nl/mirrors/people/alien/restricted_sbrepos/14.0/x86_64/vlc/vlc-2.0.6-x86_64-2alien.txz

Hope this helps.


Greetings
--
SeB

ruario 04-27-2013 02:30 PM

Quote:

Originally Posted by TobiSGD (Post 4939846)
is not a Slackware conform package, it simply overwrote my configuration

@zerouno: Just to clarify in case it is not immediately obvious, open the slackpkg package and have a look at its install/doinst.sh (see in particular the config function and how it is used). You want to do something similar to handle the slackpkgplus.conf, which should be called slackpkgplus.conf.new within the package itself. You should also add a install/slack-desc.

zerouno 04-27-2013 04:02 PM

Sorry. slackpkgplus.conf.new was included in 0.2, but I forgot to copy in 0.9beta1

beta2 is ready.

slackpkg+-0.9beta2-noarch-1mt.txz

It re-add the doinst.sh.
It includes the phenixia2003 patch. Thankyou.

edit: beta2 include slackpkgplus.conf.new, but a removepkg of beta1 _may_ remove old slackpkgplus.conf; backup it before upgrade.

tuxbg 04-28-2013 02:33 AM

Big thanks to your work fellas


All times are GMT -5. The time now is 11:35 AM.