slackpkg vs. third-party package repository

TobiSGD · 04-27-2013, 05:17 AM

Quote:

Originally Posted by zerouno

Uploaded beta1:
slackpkg+-0.9beta1-noarch-1.txz

Done some merging code.

Whatever caused my problems, it was fixed with installing that package (which by the way is not a Slackware conform package, it simply overwrote my configuration).

Thanks all for your good work.

phenixia2003 · 04-27-2013, 10:51 AM

Hello,

Quote:

Originally Posted by zerouno

Uploaded beta1:
slackpkg+-0.9beta1-noarch-1.txz

Done some merging code.

First, I would say : Nice work

I have a fix for the following issue:

In the repositories "alienbob" and "restricted", there are different versions of ffmpeg and vlc but the packages have the same version and build number.

When REPOPLUS=( .... restricted alienbob ... ) and you run "slackpkg -batch=on -default-answer=y install alienbob:vlc-2.0.6", slackpkg installs the "vlc" from "restricted" instead of "alienbob" :
Code:
slackpkg -batch=on -default_answer=y install alienbob:vlc-2.0.6

Looking for vlc-2.0.6 in package list. Please wait... DONE

vlc-2.0.6-x86_64-2alien.txz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

Package: vlc-2.0.6-x86_64-2alien.txz
--2013-04-27 15:53:10--  http://taper.alienbase.nl/mirrors/people/alien/restricted_sbrepos/14.0/x86_64/vlc/vlc-2.0.6-x86_64-2alien.txz
Now when REPOPLUS=( ... alienbob restricted ... ) and you run "slackpkg -batch=on -default_answer=y install restricted:vlc-2.0.6", slackpkg installs "vlc" from "alienbob" instead of "restricted" :
Code:
slackpkg -batch=on -default_answer=y install restricted:vlc-2.0.6

Looking for vlc-2.0.6 in package list. Please wait... DONE

vlc-2.0.6-x86_64-2alien.txz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

Package: vlc-2.0.6-x86_64-2alien.txz
--2013-04-27 15:57:18--  http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/vlc/vlc-2.0.6-x86_64-2alien.txz

This is the consequence of two things :

1) packages data in /var/lib/slackpkg/pkglist are ordered according to REPOPLUS.

IMPORTANT:
When REPOPLUS is changed, the data in /var/lib/slackpkg/pkglist will be re-ordered at next "slackpkg update".

So, when REPOPLUS=( ... alienbob restricted ...), data about alienbob packages are found before the data about restricted packages, and vice-versa when REPOPLUS=( ... restricted alienbob ... )

2) When getpkg() - in /usr/libexec/slackpkg/core-functions.sh) is called, it selects, the 1st line in ${TMPDIR}/pkglist (a copy of /var/lib/slackpkg/pkglist created at runtime) that matches the argument passed to "slackpkg install" (ex vlc):
Code:
PKGNAME=( $(grep -w -m 1 -- "${1/%.t[blxg]z/}" ${TMPDIR}/pkglist) )

This can be solved in two ways :

1) Modify the build number in the repository "alienbob" or "restricted". For instance, build number could start at "1" for "alienbob", and at "100" for "restricted". This is the simplest, but this could be a pain for Eric, I guess.

2) apply the patch in attachment and shown below to slackpkg+slackpkg+-0.9beta1-noarch-1.txz

Code:

--- slackpkgplus.sh	2013-04-27 17:08:00.985845628 +0200
+++ slackpkgplus-new-givepriority.sh	2013-04-27 17:07:00.560165835 +0200
@@ -153,7 +153,10 @@
 
   }
 
-
+    # Global variable required by givepriority() 
+    # 
+  PRIORITYIDX=1
+  
     # Found packages in repository. 
     # This function selects the package from the higher priority
     # repository directories.
@@ -183,11 +186,30 @@
 		  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:]" ; then
 		    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
 		    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
-		  
-
-		    if echo "$ARGUMENT" | grep -q "$PAT" ; then
-		      PKGDATA=( $(grep "^${DIR} ${ARGUMENT} " ${TMPDIR}/pkglist) )
-			
+		    
+				# ARGUMENT is always a basename. But PAT can be: 
+				#   1. a regular expression (ie .*)
+				#   2. a basename (openjdk)
+				#   3. a partial (or complete) package name (vlc-2.0.6, ).
+				#
+				# The current "enhanced priority rule" is applied :
+				#   + In case (1) and (2) when ARGUMENT contains the pattern PAT
+				#   + In the case (3) when ARGUMENT starts the pattern PAT.
+				# 
+		    if echo "$ARGUMENT" | grep -q "$PAT" || echo "$PAT" | grep "^$ARGUMENT" ; then
+			  PKGDATA=""
+			  PKGINFOS=$(grep -n -m 1 "^${DIR} ${ARGUMENT} " ${TMPDIR}/pkglist)
+			  
+			  if [ ! -z "$PKGINFOS" ] ; then
+			    LINEIDX=$(echo "$PKGINFOS" | cut -f1 -d":")
+				PKGDATA=( $(echo "$PKGINFOS" | cut -f2- -d":") )
+				
+					# -- move the line at #LINEIDX to #PRIORITYIDX and
+					#    increment PRIORITYIDX
+					#
+				sed -i --expression "${LINEIDX}d" --expression "${PRIORITYIDX}i${PKGDATA[*]}" ${TMPDIR}/pkglist
+				(( PRIORITYIDX++ ))
+			  fi
 		    fi
 		  else	
 		    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )

Once this patch applied, you will get the expected behavior :

When REPOPLUS ( .... restricted alienbob ... ) and you run "slackpkg -batch=on -default-answer=y install alienbob:vlc-2.0.6", slackpkg installs "vlc" from alienbob :
Code:
slackpkg -batch=on -default_answer=y install alienbob:vlc-2.0.6

Looking for vlc-2.0.6 in package list. Please wait... vlc-2.0.6
DONE

vlc-2.0.6-x86_64-2alien.txz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

Package: vlc-2.0.6-x86_64-2alien.txz
--2013-04-27 17:10:06--  http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/vlc/vlc-2.0.6-x86_64-2alien.txz
When REPOPLUS ( .... alienbob restricted ... ) and you run "slackpkg -batch=on -default-answer=y install restricted:vlc-2.0.6", then slackpkg installs "vlc" from restricted :
Code:
slackpkg -batch=on -default_answer=y install restricted:vlc-2.0.6

Looking for vlc-2.0.6 in package list. Please wait... vlc-2.0.6
DONE

vlc-2.0.6-x86_64-2alien.txz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

Package: vlc-2.0.6-x86_64-2alien.txz
--2013-04-27 17:13:59--  http://taper.alienbase.nl/mirrors/people/alien/restricted_sbrepos/14.0/x86_64/vlc/vlc-2.0.6-x86_64-2alien.txz

Hope this helps.

Greetings
--
SeB

ruario · 04-27-2013, 02:30 PM

Quote:

Originally Posted by TobiSGD

is not a Slackware conform package, it simply overwrote my configuration

@zerouno: Just to clarify in case it is not immediately obvious, open the slackpkg package and have a look at its install/doinst.sh (see in particular the config function and how it is used). You want to do something similar to handle the slackpkgplus.conf, which should be called slackpkgplus.conf.new within the package itself. You should also add a install/slack-desc.

zerouno · 04-27-2013, 04:02 PM

Sorry. slackpkgplus.conf.new was included in 0.2, but I forgot to copy in 0.9beta1

beta2 is ready.

slackpkg+-0.9beta2-noarch-1mt.txz

It re-add the doinst.sh.
It includes the phenixia2003 patch. Thankyou.

edit: beta2 include slackpkgplus.conf.new, but a removepkg of beta1 _may_ remove old slackpkgplus.conf; backup it before upgrade.

tuxbg · 04-28-2013, 02:33 AM

Big thanks to your work fellas

zerouno · 04-29-2013, 06:00 AM

Slackpkg+ is now in rc1.

There are no changes to code, but I added new repositories and documentation (and marked it as release candidate 1

)

slackpkg+-0.9rc1-noarch-3mt.txz

phenixia2003 · 04-29-2013, 11:16 AM

Hello,

Quote:

Originally Posted by zerouno

Slackpkg+ is now in rc1.

There are no changes to code, but I added new repositories and documentation (and marked it as release candidate 1

)

I found a small issue, and I have a new code suggestion.

First with the issue: In slackpkgplus.conf, there's :

Code:

# If you want a multilib system, uncomment the multilib repository and set:
#PKGS_PRIORITY=( multilib:.* )
#
# List repository you want to use (defined below)
REPOPLUS=( alienbob restricted slacky ponce )

This will be ok if PKGS_PRIORITY and PRIORITY are merged when you issue a "slackpkg update" but this is not the case because of this line #17 :

Code:

if [ ! -z "$PKGS_PRIORITY" -a "$CMD" != "update" ] ; then

I guess this was to simplify the change (in green) in this post. I'm ok with that, but in this case, slackpkgplus.conf should be changed as below :

Code:

# List repository you want to use (defined below)
#
#REPOPLUS=( alienbob restricted slacky ponce )
#
# If you want a multilib system, uncomment the multilib repository and
# add its name at start of REPOPLUS like in example below :
#
#REPOPLUS=( multilib alienbob restricted slacky ponce )
#
#
# If two or more repositories contains some same packages, you can specify
# from which repository (declared in REPOPLUS) you prefer to search it.
# The syntax is "<repository_name>:<pattern>". For instance :
#
#PKGS_PRIORITY=( alienbob:openjdk )
#
# pattern can be a package name, or a regular expression, so that you can
# also give priority to all the packages to any repository declared in 
# REPOPLUS. For instance, when REPOPLUS=( alienbob restricted multilib ) 
# and you want to prioritize the multilib, then you will add multilib:.* 
# at start of PKGS_PRIORITY like as below:
#
#PKGS_PRIORITY=( multilib:.* alienbob:openjdk )
#

Another point: When REPOPLUS has been modified, the user must issue a "slackpkg update" for the changes to take effect. This should be mentioned somewhere.

----

The new code suggestion I have is for the search feature. You will find the patch for slackpkg+ rc1 in attachment. Note that "file search" is not implemented.

Here are some examples with REPOPLUS=( multilib alienbob restricted slacky ) done on a Slackware-14.0 VM :

1. Searching for glibc

Code:

 slackpkg search glibc  

DONE

The list below shows all packages with name matching "glibc".

[ Status           ] [ Repository               ] [ Package                                  ]
  upgrade                                           glibc-zoneinfo-2013b_multilib-noarch-1alien --> glibc-zoneinfo-2013b-noarch-1_slack14.0  
  upgrade                                           glibc-solibs-2.15_multilib-x86_64-7alien --> glibc-solibs-2.15-x86_64-7  
  upgrade                                           glibc-2.15_multilib-x86_64-7alien --> glibc-2.15-x86_64-7  
  upgrade                                           glibc-i18n-2.15_multilib-x86_64-7alien --> glibc-i18n-2.15-x86_64-7  
  upgrade                                           glibc-profile-2.15_multilib-x86_64-7alien --> glibc-profile-2.15-x86_64-7  
  uninstalled          multilib                     glibc-debug-2.15_multilib-x86_64-7alien   
   installed           multilib                     glibc-2.15_multilib-x86_64-7alien         
   installed           multilib                     glibc-i18n-2.15_multilib-x86_64-7alien    
   installed           multilib                     glibc-profile-2.15_multilib-x86_64-7alien  
   installed           multilib                     glibc-solibs-2.15_multilib-x86_64-7alien  
   installed           multilib                     glibc-zoneinfo-2013b_multilib-noarch-1alien

2. Searching for vlc

Code:

$ slackpkg search vlc

DONE

The list below shows all packages with name matching "vlc".

[ Status           ] [ Repository               ] [ Package                                  ]
  uninstalled          alienbob                     phonon-vlc-0.6.0-x86_64-1alien            
  uninstalled          alienbob                     npapi-vlc-20130408-x86_64-2alien          
  uninstalled          alienbob                     vlc-2.0.6-x86_64-2alien                   
  uninstalled          restricted                   npapi-vlc-20130408-x86_64-2alien          
  uninstalled          restricted                   vlc-2.0.6-x86_64-2alien

3. Searching for qt

Code:

slackpkg search qt

DONE

The list below shows all packages with name matching "qt".

[ Status           ] [ Repository               ] [ Package                                  ]
   installed                                        kdevelop-pg-qt-1.0.0-x86_64-1             
   installed                                        perlqt-4.8.5-x86_64-1                     
   installed                                        qtruby-4.8.5-x86_64-1                     
   installed                                        smokeqt-4.8.5-x86_64-1                    
   installed                                        libdbusmenu-qt-0.9.2-x86_64-2             
   installed                                        polkit-qt-1-0.103.0-x86_64-1              
   installed                                        qt-4.8.2-x86_64-4                         
   installed                                        qtscriptgenerator-0.2.0-x86_64-1          
   installed           multilib                     qt-compat32-4.8.2-x86_64-4compat32        
  uninstalled          slacky                       razorqt-0.5.1-x86_64-1sl                  
  uninstalled          slacky                       qt3-3.3.8b-x86_64-1sl                     
  uninstalled          slacky                       avidemux-qt4-2.5.6-x86_64-2sl             
  uninstalled          slacky                       bitcoin-qt-0.7.1-x86_64-1sl               
  uninstalled          slacky                       qtransmission-2.75-x86_64-1sl

Edit (Important) :

I found another small issue with multilib repository and install-new. The CHECKSUMS.md5 of this multilib repository contains 2 entries for each compat32 packages :

Code:

cb98e611b682d5607bef143c353e6ea1  ./slackware64-compat32/a-compat32/aaa_elflibs-compat32-14.0-x86_64-4compat32.txz
cb98e611b682d5607bef143c353e6ea1  ./aaa_elflibs-compat32-14.0-x86_64-4compat32.txz
...
d92209cd6ac9bbb65e50e4e905306224  ./slackware64-compat32/a-compat32/bzip2-compat32-1.0.6-x86_64-1compat32.txz
d92209cd6ac9bbb65e50e4e905306224  ./bzip2-compat32-1.0.6-x86_64-1compat32.txz
...
d26a96832f9baa5c1ce3697eaf94eeef  ./slackware64-compat32/a-compat32/cups-compat32-1.5.4-x86_64-2compat32.txz
d26a96832f9baa5c1ce3697eaf94eeef  ./cups-compat32-1.5.4-x86_64-2compat32.txz
...

This leads "slackpkg install-new" to add all the "compat32" packages in the list of new packages. To fix this, apply the patch below to slackpkgplus.sh (0.9rc1)

Code:

--- slackpkgplus.sh	2013-04-30 09:53:45.698893304 +0200
+++ slackpkgplus-install-new-fix.sh	2013-04-30 09:56:27.855502576 +0200
@@ -246,7 +246,7 @@
   if [ "$CMD" == "install-new" ] ; then 
     ls -1 /var/log/packages/*compat32 2>/dev/null | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
     
-    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst
+    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort -u > $TMPDIR/available-compat32-packages.lst
 
     NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)

Greetings.

--
SeB

zerouno · 04-30-2013, 07:57 AM

Quote:

Originally Posted by phenixia2003

First with the issue: In slackpkgplus.conf, there's :

Code:

# If you want a multilib system, uncomment the multilib repository and set:
#PKGS_PRIORITY=( multilib:.* )
#
# List repository you want to use (defined below)
REPOPLUS=( alienbob restricted slacky ponce )

This will be ok if PKGS_PRIORITY and PRIORITY are merged when you issue a "slackpkg update" but this is not the case because of this line #17 :

Code:

if [ ! -z "$PKGS_PRIORITY" -a "$CMD" != "update" ] ; then

I guess this was to simplify the change (in green) in this post. I'm ok with that, but in this case, slackpkgplus.conf should be changed as below :

Code:

# List repository you want to use (defined below)
#
#REPOPLUS=( alienbob restricted slacky ponce )
#
# If you want a multilib system, uncomment the multilib repository and
# add its name at start of REPOPLUS like in example below :
#
#REPOPLUS=( multilib alienbob restricted slacky ponce )
#
#
# If two or more repositories contains some same packages, you can specify
# from which repository (declared in REPOPLUS) you prefer to search it.
# The syntax is "<repository_name>:<pattern>". For instance :
#
#PKGS_PRIORITY=( alienbob:openjdk )
#
# pattern can be a package name, or a regular expression, so that you can
# also give priority to all the packages to any repository declared in 
# REPOPLUS. For instance, when REPOPLUS=( alienbob restricted multilib ) 
# and you want to prioritize the multilib, then you will add multilib:.* 
# at start of PKGS_PRIORITY like as below:
#
#PKGS_PRIORITY=( multilib:.* alienbob:openjdk )
#

Writting documentation (the examples) and putting the workaround "$CMD" != "update", I supposed as implicit that if a user add a repository in PKGS_PRIORITY and/or if he uncomment a MIRRORPLUS[], he will add also the repository in REPOPLUS, but in effect that may not be true.

I think that as workaround this may be sufficient:

Code:

-  REPOPLUS=${REPOPLUS[*]}
+  REPOPLUS=$(echo "${REPOPLUS[*]} ${PKGS_PRIORITY[*]} ${!MIRRORPLUS[*]}"|sed 's/ /\n/g'|sed 's/:.*//'|awk '{if(!a[$1]++)print $1}')

so if the user forgot to add a repository in REPOPLUS (I forgot that few minutes ago

), slackpkg+ automatically add it.

Quote:

The new code suggestion I have is for the search feature. You will find the patch for slackpkg+ rc1 in attachment. Note that "file search" is not implemented.
...
I found another small issue with multilib repository and install-new

Thankyou for fix.

zerouno · 04-30-2013, 08:33 AM

What do you think to add an option in slackpkgplus.conf for multilib?

similar to
[code]# Set this option to make you 64bit system a multilib system
# (install some 32bit libraries so you can launch some 32bit-only
# packages as skype, wine and other)
#COMPAT32=enable[/compat]
if COMPAT32 is set to 'enable', slackpkg+ simply autoadd multilib:.* in PKGS_PRIORITY and in REPOPLUS.
Also if COMPAT32 is set to 'enable', slackpkg+ may allow to use the 32bit-only repository

zerouno · 04-30-2013, 09:11 AM

rc2 is released.
slackpkg+-0.9rc2-noarch-1mt.txz

it include all fix (but comments in slackpkgplus.conf)

phenixia2003 · 04-30-2013, 09:18 AM

Hello,

For me there's nothing to add about the multilib. This is well designed and well documented.

But, it would be a good idea to include something to notify the user to issue a "slackpkg update" when the REPOPLUS has been changed since the last "update".

I said that because I recently got some trouble to understand why slackpkg was unable to see some packages: I had added a repository into REPOPLUS but I had forgotten to issue "slackpkg update". This could be easily implemented using a simple file in /var/lib/slackpkg

--
SeB

zerouno · 04-30-2013, 09:39 AM

Yes, for now I added "# remember to launch 'slackpkg update' if you modify that row." only.
Next I will do some documentation.

zerouno · 04-30-2013, 10:49 AM

Code:

  # Test repositories
  for pp in ${REPOPLUS[*]};do
    echo "${MIRRORPLUS[$pp]}"|grep -q -e ^http:// -e ^https:// -e ^ftp:// -e ^file://
    if [ $? -ne 0 ];then
      echo "Repository '$pp' not configured." >> $TMPDIR/error.log
      echo "Add:" >> $TMPDIR/error.log
      echo "MIRRORPLUS['$pp']=http://repoaddres/..." >> $TMPDIR/error.log
      echo "See documentation in /usr/doc/slackpkg+-* for details" >> $TMPDIR/error.log
      cleanup
    fi
  done
  # post a warning if slackpkgplus.conf is newer than pkglist, but do not block execution.
  if [ /etc/slackpkgplus.conf -nt /var/lib/slackpkg/pkglist -a "$CMD" != "update" ];then
    echo
    echo "NOTICE: remember to re-run 'slackpkg update' after modifing slackpkgplus.conf"
    echo
  fi

phenixia2003 · 04-30-2013, 11:10 AM

Hello,

Quote:

Originally Posted by zerouno

Code:

  # Test repositories
  for pp in ${REPOPLUS[*]};do
    echo "${MIRRORPLUS[$pp]}"|grep -q -e ^http:// -e ^https:// -e ^ftp:// -e ^file://
    if [ $? -ne 0 ];then
      echo "Repository '$pp' not configured." >> $TMPDIR/error.log
      echo "Add:" >> $TMPDIR/error.log
      echo "MIRRORPLUS['$pp']=http://repoaddres/..." >> $TMPDIR/error.log
      echo "See documentation in /usr/doc/slackpkg+-* for details" >> $TMPDIR/error.log
      cleanup
    fi
  done
  # post a warning if slackpkgplus.conf is newer than pkglist, but do not block execution.
  if [ /etc/slackpkgplus.conf -nt /var/lib/slackpkg/pkglist -a "$CMD" != "update" ];then
    echo
    echo "NOTICE: remember to re-run 'slackpkg update' after modifing slackpkgplus.conf"
    echo
  fi

That seems good for me. Just add a sleep (10 sec ?) after the NOTICE, so that the user will have the time to read it.

--
SeB

fl0 · 05-01-2013, 11:48 AM

Hi,

i have installed slackpkg+ and activated only ponce current repo, running slackware64 current, i edit slackpkgplus.conf and run slackpkg update gpg and slackpkg update, but i alwys got these error

Code:

ERROR: Verification of the  gpg signature on CHECKSUMS.md5
                       failed! This could mean that the file is out of date
                       or has been tampered with.

Ideas?

regards fl0