slackpkg vs. third-party package repository

zerouno · 01-15-2016, 05:38 PM

fixed a bug on the patch for $ROOT problem in stable and devel branch.

bamunds · 01-15-2016, 11:44 PM

zerouno and phenexia2003, thank you both for the amazing and improving work on slackpkg+ which you are collaborating on. Please let us know when a beta is available that you want tested and where to get (git) it. I'm willing to help. BrianA_MN

zerouno · 01-16-2016, 04:56 AM

Thankyou.
Currently I need to test the slackpkg+-1.6.1p1-noarch-1mt_pre.txz from http://slakfinder.org/slackpkg+all/

It contains the backported patch

Code:

Version 1.6.1p1 - 16/Jan/2015
  - Bugfix: greylist did not work properly
  - Fixed search that may fails to detect repository in certain conditions.
  - Fixed a bug in slackpkg upgrade when used with $ROOT

zerouno · 01-16-2016, 09:59 AM

The patch slackpkg+-enhanced-pkg-arg-handling.diff.txt introduces a regression when PKGS_PRIORITY is set when applied to slackpkg+ 1.7.a3 (tested before bashism patch to avoid conflicts)

Code:

--- a/src/slackpkgplus.sh
+++ b/src/slackpkgplus.sh
@@ -796,6 +796,16 @@ if [ "$SLACKPKGPLUS" = "on" ];then
           LINEIDX=$(echo "$PKGINFOS" | cut -f1 -d":")
           PKGDATA=( $(echo "$PKGINFOS" | cut -f2- -d":") )
         fi
+      elif echo "$CPRIORITY" | grep -q "^[.][*][:]" ; then
+        PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
+        
+        PKGINFOS=$(grep 2>/dev/null -n -w "${PAT}" ${TMPDIR}/pkglist | grep -m 1 " ${ARGUMENT} ")
+        
+        if [ ! -z "$PKGINFOS" ] ; then
+          LINEIDX=$(echo "$PKGINFOS" | cut -f1 -d":")
+          PKGDATA=( $(echo "$PKGINFOS" | cut -f2- -d":") )
+        fi
+
       else
           # $CPRIORITY is of kind "repository" (ie. slackware, extra, patches,...)
         REPOSITORY="${CPRIORITY}"
@@ -1497,6 +1507,10 @@ if [ "$SLACKPKGPLUS" = "on" ];then
 
         package=$pref
         AUTOPRIORITY=" $AUTOPRIORITY -e $package "
+        if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then
+          PRIORITYLIST=( ${PRIORITYLIST[*]} ".*:${package}" )
+        fi
+
       fi
 
       if [ "$CMD" == "remove" ];then

Code:

PKGS_PRIORITY=( ktown )

Code:

# ls -l /var/log/packages/kde-baseapps*
/var/log/packages/kde-baseapps-15.12.0-i486-2alien

Before patch.

Code:

# slackpkg -batch=on -dialog=off -default_answer=no upgrade kde-baseapps

Checking local integrity... DONE
Looking for kde-baseapps in package list. Please wait... DONE

No packages match the pattern for upgrade. Try:

        /usr/sbin/slackpkg install|reinstall

after patch:

Code:

# slackpkg -batch=on -dialog=off -default_answer=no upgrade kde-baseapps

Checking local integrity... DONE
Looking for kde-baseapps in package list. Please wait... DONE

[ Repository               ] [ Package                                  ]
  slackware                    kde-baseapps-4.14.3-i586-2.txz            

Total package(s): 1

Do you wish to upgrade selected packages (Y/n)? n

slackpkg+ proposes downgrade.

bamunds · 01-16-2016, 11:31 AM

zerouno, I've subscribed to this thread to watch progress you are making. If you move testing to another thread please post note here of the new thread. I'll test new version when you indicate in this thread ""Testers needed" for new slackpkg+, just to make sure there isn't confusion with your statement "I need to test" which to me means that you don't want other testers just yet and you're still working on bugs. Thanks again and I'm able to help when you are ready. BrianA_MN

phenixia2003 · 01-16-2016, 01:09 PM

Hello,

Quote:

Originally Posted by zerouno

The patch slackpkg+-enhanced-pkg-arg-handling.diff.txt introduces a regression when PKGS_PRIORITY is set when applied to slackpkg+ 1.7.a3 (tested before bashism patch to avoid conflicts)

Code:

PKGS_PRIORITY=( ktown )

Code:

# ls -l /var/log/packages/kde-baseapps*
/var/log/packages/kde-baseapps-15.12.0-i486-2alien

Before patch.

Code:

# slackpkg -batch=on -dialog=off -default_answer=no upgrade kde-baseapps

Checking local integrity... DONE
Looking for kde-baseapps in package list. Please wait... DONE

No packages match the pattern for upgrade. Try:

        /usr/sbin/slackpkg install|reinstall

after patch:

Code:

# slackpkg -batch=on -dialog=off -default_answer=no upgrade kde-baseapps

Checking local integrity... DONE
Looking for kde-baseapps in package list. Please wait... DONE

[ Repository               ] [ Package                                  ]
  slackware                    kde-baseapps-4.14.3-i586-2.txz            

Total package(s): 1

Do you wish to upgrade selected packages (Y/n)? n

slackpkg+ proposes downgrade.

The patch slackpkg+-advanced-priority-fix.diff.txt should solve the problem:

Code:

--- slackpkgplus.sh.git.1.7.a5	2016-01-16 20:15:28.603145086 +0100
+++ slackpkgplus.sh	2016-01-16 20:15:28.603145086 +0100
@@ -1363,6 +1363,18 @@
     NEWINPUTLIST=""
     PRIORITYLIST=""
 
+      # The priorities in PRIORITYLIST_SX :
+      #   * are *all* of kind ".*:<pattern>" 
+      #   * are defined to handle cases where a pattern, with version and/or a build number 
+      #     but without any repository, is passed to install|upgrade (ex: install basename-1.0.1)
+      #
+      # Since there's no way to distinguish patterns with version/build number to other, priorities
+      # of kind ".*:<pattern>" are also generated for patterns without version/build number. As a 
+      # consequence, these priorities could interfer with other defined priorities (1). To prevent
+      # this, these priorities are handled after all other priorities.
+      # 
+    PRIORITYLIST_SX=""
+
     for pref in $INPUTLIST ; do
       PRIORITY_FILTER_RULE=""
 
@@ -1506,7 +1518,7 @@
         AUTOPRIORITY=" $AUTOPRIORITY -e $package "
 
         if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then
-          PRIORITYLIST=( ${PRIORITYLIST[*]} ".*:${package}" )
+	  PRIORITYLIST_SX=( ${PRIORITYLIST_SX[*]} ".*:${package}" )
         fi
       fi
 
@@ -1523,8 +1535,11 @@
 
     INPUTLIST=$NEWINPUTLIST
 
-    if [ ! -z "$PRIORITYLIST" ] ; then
-      NEWPRIORITY=( ${PRIORITYLIST[*]} ${PRIORITY[*]} )
+    if [ ! -z "$PRIORITYLIST" ] || [ ! -z "$PRIORITYLIST_SX" ] ; then
+	# PRIORITYLIST_SX includes priority of kind .*:pattern. This kind of priority must be handled
+	# after all others, and are, by consequence, added at the end.
+        #
+      NEWPRIORITY=( ${PRIORITYLIST[*]} ${PRIORITY[*]} ${PRIORITYLIST_SX[*]} )
       unset PRIORITY
 
       # -- This is to avoid duplicated priority rules in the variable

--
SeB

zerouno · 01-16-2016, 04:40 PM

@phenixia2003. thankyou. I will apply it.

There are no other official threads for slackpkg+ (just some for specific bugs reported by users, threads dead after solution; if you refeers to that topic, I saw it just now)

Quote:

Originally Posted by bamunds

just to make sure there isn't confusion with your statement "I need to test" which to me means that you don't want other testers just yet and you're still working on bugs.

No, it means that I have many confusion on my english grammar

.
Should be something as "I need testing (i.e. tester) for...".

To make 1.7 lts I must to find a good way to backport patches from 1.8+ to 1.7; So I'm trying it backporting some patch from 1.7 to 1.6 (even if it won't be an lts). For this goal I made a branch "stable" on github that contains the patches to backport to master. The package 1.6.1p1 is the txz of that branch.

Regards a beta version I have to apply the latest patch from phenixia2003 and the improved search, and stabilizing something. After that should ready for beta.

zerouno · 01-17-2016, 02:29 PM

1.7.a6 released

both fix and improved search are included.

bamunds · 01-17-2016, 02:35 PM

I'm looking forward to the beta for testing.

zerouno · 01-18-2016, 02:31 PM

another givepriority performance improving.

Code:

--- a/src/slackpkgplus.sh
+++ b/src/slackpkgplus.sh
@@ -852,7 +852,8 @@ if [ "$SLACKPKGPLUS" = "on" ];then
           # R1:P is before R2:P in pkglist, and the user issue install|upgrade R2:P, slackpkg
           # will install R1:P instead.
           #
-        sed -i --expression "${LINEIDX}d" --expression "${PRIORITYIDX}i${PKGDATA[*]}" ${TMPDIR}/pkglist
+        mv ${TMPDIR}/pkglist ${TMPDIR}/pkglist.old
+        sed  --expression "${LINEIDX}d" --expression "${PRIORITYIDX}i${PKGDATA[*]}" ${TMPDIR}/pkglist.old > ${TMPDIR}/pkglist
         (( PRIORITYIDX++ ))
       fi
     done

before

Code:

# time slackpkg -batch=on -dialog=off -default_answer=no upgrade ktown

Checking local integrity... DONE
Looking for SLACKPKGPLUS_ktown in package list. Please wait... DONE

No packages match the pattern for upgrade. Try:

        /usr/sbin/slackpkg install|reinstall 



real    0m25.989s
user    0m7.765s
sys     0m15.278s

after

Code:

# time slackpkg -batch=on -dialog=off -default_answer=no upgrade ktown

Checking local integrity... DONE
Looking for SLACKPKGPLUS_ktown in package list. Please wait... DONE

No packages match the pattern for upgrade. Try:

        /usr/sbin/slackpkg install|reinstall 



real    0m13.944s
user    0m6.306s
sys     0m4.397s

zerouno · 01-18-2016, 03:52 PM

The beta is ready.

phenixia2003 · 01-19-2016, 02:26 AM

Hello,

Quote:

Originally Posted by zerouno

another givepriority performance improving.

Code:

--- a/src/slackpkgplus.sh
+++ b/src/slackpkgplus.sh
@@ -852,7 +852,8 @@ if [ "$SLACKPKGPLUS" = "on" ];then
           # R1:P is before R2:P in pkglist, and the user issue install|upgrade R2:P, slackpkg
           # will install R1:P instead.
           #
-        sed -i --expression "${LINEIDX}d" --expression "${PRIORITYIDX}i${PKGDATA[*]}" ${TMPDIR}/pkglist
+        mv ${TMPDIR}/pkglist ${TMPDIR}/pkglist.old
+        sed  --expression "${LINEIDX}d" --expression "${PRIORITYIDX}i${PKGDATA[*]}" ${TMPDIR}/pkglist.old > ${TMPDIR}/pkglist
         (( PRIORITYIDX++ ))
       fi
     done

before

Code:

# time slackpkg -batch=on -dialog=off -default_answer=no upgrade ktown

Checking local integrity... DONE
Looking for SLACKPKGPLUS_ktown in package list. Please wait... DONE

No packages match the pattern for upgrade. Try:

        /usr/sbin/slackpkg install|reinstall 



real    0m25.989s
user    0m7.765s
sys     0m15.278s

after

Code:

# time slackpkg -batch=on -dialog=off -default_answer=no upgrade ktown

Checking local integrity... DONE
Looking for SLACKPKGPLUS_ktown in package list. Please wait... DONE

No packages match the pattern for upgrade. Try:

        /usr/sbin/slackpkg install|reinstall 



real    0m13.944s
user    0m6.306s
sys     0m4.397s

When there are updates, I see no difference.

1. upgrade-all:

Old code:

Code:

$ time slackpkg -dialog=off -batch=on -default_answer=no upgrade-all

Checking local integrity... DONE
Looking for packages to upgrade. Please wait... DONE

[ Repository               ] [ Package                                  ]

<snip>

Total package(s): 312

Do you wish to upgrade selected packages (Y/n)? n

real    0m10.329s
user    0m11.021s
sys     0m2.233s

New code:

Code:

$ time slackpkg -dialog=off -batch=on -default_answer=no upgrade-all

Checking local integrity... DONE
Looking for packages to upgrade. Please wait... DONE

[ Repository               ] [ Package                                  ]

<snip>

Total package(s): 312

Do you wish to upgrade selected packages (Y/n)? n


real    0m10.274s
user    0m10.896s
sys     0m2.241s

2. upgrade ktown :

Old code:

Code:


$ time slackpkg -dialog=off -batch=on -default_answer=no upgrade ktown

Checking local integrity... DONE
Looking for SLACKPKGPLUS_ktown in package list. Please wait... DONE

[ Repository               ] [ Package                                  ]

<snip>

Total package(s): 230

Do you wish to upgrade selected packages (Y/n)? n


real    0m7.683s
user    0m5.952s
sys     0m1.296s

New code:

Code:

$ time slackpkg -dialog=off -batch=on -default_answer=no upgrade ktown

Checking local integrity... DONE
Looking for SLACKPKGPLUS_ktown in package list. Please wait... DONE

[ Repository               ] [ Package                                  ]

<snip>

Total package(s): 230

Do you wish to upgrade selected packages (Y/n)? n

real    0m7.700s
user    0m5.917s
sys     0m1.393s

--
SeB

zerouno · 01-19-2016, 02:53 AM

Should impact repositories in PKGS_PRIORITY when there is an huge pkglist (about 15.000 packages).

The sed -i command write a temporary file then copy it over the original. They are 2 write operation on disk that may take a lot with huge pkglist.
I tried to count the time in givepriority (date +S%N at start and the end of function) and I saw that in upgrade ktown a single cycle on my machine has about 60 millisec with sed -i and 30 millisec without it. When running slackpkg upgrade slackware the cycle has about 20millisec with and without it.
See the 'sys' results in my outputs.
With /tmp in tmpfs should be no impact.

on my host slackware64 with SSD and /tmp in tmpfs and pkglist with 2500 row, there is no difference.

Code:

real    0m9.822s
user    0m6.980s
sys     0m2.209s

zerouno · 01-21-2016, 03:57 PM

here is a pull request to add the spinning to the check-updates.

What do you think?

yars · 01-24-2016, 01:04 AM

I noticed that some slackpkg functions, which use the spinning, is overridden in slackpkg+. So I can rework this pull-request to revert back the lost spinning in these functions. Not sure that will not affect the performance, but spinning is working as a separated process.