slackpkg vs. third-party package repository

zerouno · 04-11-2014, 02:08 AM

Quote:

Originally Posted by samac

Now that you have a notification

slackpkg has always had the notification system.
The phenixia2003 patch just report which repository was updated, not just all. And the 'crontab' system exists from about 30 or more years.

Also, check-updates shown you if a repository was changed, that does NOT meaning that there are new packages for your system.
For example if you had NOT installed 'vlc' from alienbob repository and alienbob upgrade it, check-updates report that the alien repository was updated, but the slackpkg upgrade-all does not report you nothing.

So a real notification system should do that (in background):
slackpkg check-updates
if there are updates on any repository ];then
slackpkg update
fi
slackpkg upgrade-all (report only)
slackpkg install-new (report only)
both that commands should be launched even if check-updates do not reports new package.
This must to be another - different - tool, that is no more slackpkg but another command, with other syntax, etc...

from slackpkg+ site:
"Slackpkg+ is NOT a new tool with new commands. The user can continue to use the standard slackpkg command line:"

Quote:

... runs "slackpkg" update and upgrade, both graphically and automatically

"update" and "graphically" may be. "automatically" must not be.

phenixia2003 · 04-11-2014, 02:53 AM

Hello,

Quote:

Originally Posted by gegechris99

The DE notification is working fine except for the time details in the crontab entry.

Sorry, this is the crontab entry I used to test this feature and I forgot to mention its frequency. I changed the original message and used a better frequency (every 2 hours).

--
SeB

phenixia2003 · 04-11-2014, 03:23 AM

Hello,

Quote:

Originally Posted by zerouno

Also, check-updates shown you if a repository was changed, that does NOT meaning that there are new packages for your system.
For example if you had NOT installed 'vlc' from alienbob repository and alienbob upgrade it, check-updates report that the alien repository was updated, but the slackpkg upgrade-all does not report you nothing.

I think it would be possible, and even easy to identify, for each repository, the number of installed packages for which an update is available. So we could have something like that :

Code:

$ /usr/sbin/slackpkg check-updates

News on ChangeLog.txt

  [ Repository               ] [ Status               ]
    slackware                    Changed (13 updates)
    slackpkgplus                 No changes
    restricted                   No changes
    alienbob                     Changed (4 updates)
    multilib                     No changes

Good or bad idea ?

--
SeB

zerouno · 04-11-2014, 04:08 AM

Do you want to do that slackpkg check-updates check the contents of /var/log/packages and all the pkgorder&blacklist system?

All is possible since slackpkg+ download the entire checksums.md5 (except for the official repository, but we can modify getfile() to download it) and we can simply do a temporary regeneration of pkglist.

Also remember that slackpkg check-updates report the changes from the latest slackpkg update, not from the slackpkg upgrade, and it use the md5sum method for repository diff.
This means that we can obtains

Code:

alienbob       No changes (4 updates)

becouse after the latest slackpkg update the user not launched slackpkg upgrade-all.

Well, all its possible

we can do (for check-updates)
1) download all checkums.md5
2) merge it in a temporary pkglist
3) apply the blacklist system to the $WORKDIR/pkglist and $TMPDIR/pkglist
4) do a diff from both
5) match that with installed packages

Another function that I want to implement is an improved slackpkg upgrade-all that skip all pkgorders and check only in the original repository from which the installed packages was downloaded (and an option to activate/deactivate it), based on the tag of package (1sl vs 1alienbob)
For example, if I installed a package with slackpkg install pkg-1.0-i486-1sl (from slacky repo) even if that package exists in pkg-1.1-i486-1alienbob (that I may have as first in priority) or installed it from installpkg, when I run slackpkg upgrade-all I may want to be sure that slackpkg search for an pkg >= 1.0 from the original (slacky) repository instead any other repository becouse pkg-1.0-i486-1sl may be linked with library form other slacky packages (example: vlc from slacky have 1000+ dependency; vlc from alienbob depends from two or three other packages)

That may be merged in a new check-updates system.

P.S:
slackpkg+ is currently bigger than slackpkg

phenixia2003 · 04-11-2014, 05:08 AM

Hello,

Quote:

Originally Posted by zerouno

Do you want to do that slackpkg check-updates check the contents of /var/log/packages and all the pkgorder&blacklist system?

All is possible since slackpkg+ download the entire checksums.md5 (except for the official repository, but we can modify getfile() to download it) and we can simply do a temporary regeneration of pkglist.

I just looked at this, and finally, I don't think it's a good idea to go further on this.

Quote:

Originally Posted by zerouno

Another function that I want to implement is an improved slackpkg upgrade-all that skip all pkgorders and check only in the original repository from which the installed packages was downloaded (and an option to activate/deactivate it), based on the tag of package (1sl vs 1alienbob)
For example, if I installed a package with slackpkg install pkg-1.0-i486-1sl (from slacky repo) even if that package exists in pkg-1.1-i486-1alienbob (that I may have as first in priority) or installed it from installpkg, when I run slackpkg upgrade-all I may want to be sure that slackpkg search for an pkg >= 1.0 from the original (slacky) repository instead any other repository becouse pkg-1.0-i486-1sl may be linked with library form other slacky packages (example: vlc from slacky have 1000+ dependency; vlc from alienbob depends from two or three other packages)

Why you don't use PKGS_PRIORITY to manage this case ?

Code:

PKGS_PRIORITY=( slacky:vlc alienbob:.* ... )

--
SeB

zerouno · 04-11-2014, 06:22 AM

Quote:

Originally Posted by phenixia2003

I just looked at this, and finally, I don't think it's a good idea to go further on this.

I approve.

Quote:

Why you don't use PKGS_PRIORITY to manage this case ?

Code:

PKGS_PRIORITY=( slacky:vlc alienbob:.* ... )

Its not simple to track (without forgot any) all installed packges, specially for other than me and you (slackpkg+ developers).
The first idea was a tool to help user to create the PKGS_PRIORITY by listing and matching installed packages, or move that list in an external file to avoid a PKGS_PRIORITY line too long, but was not a better idea.

I think that PKGS_PRIORITY should be primarily used for slackpkg install. After installation (where the repository may be selected from cmdline with slackpkg install slacky:vlc instead modifing configuration) I think that the user want to upgrade the package from the original repository. So why I must to do all manually?

phenixia2003 · 04-11-2014, 11:18 AM

Hello,

Quote:

Originally Posted by zerouno

I think that PKGS_PRIORITY should be primarily used for slackpkg install. After installation (where the repository may be selected from cmdline with slackpkg install slacky:vlc instead modifing configuration) I think that the user want to upgrade the package from the original repository. So why I must to do all manually?

Ok. I think the patch below (and in attachment) for slackpkg+1.3.1/1mt is a good starting point to achieve what you want. I did some test and that seems to be ok, but there could have some issues here or there. Furthermore, I have a f*cking migraine today and this does not help ....

Code:

--- slackpkgplus.sh.orig	2014-04-11 11:12:25.029574693 +0200
+++ slackpkgplus.sh	2014-04-11 19:52:50.811005581 +0200
@@ -713,6 +713,29 @@
 	done
 	handle_event "remove"
   }
+  
+    # update, as needed, the priority list ${WORKDIR}/priorities following
+    # the installation or the upgrade of package given by ${1}
+    #
+  function update_priority_list() {
+	local pkg=$(echo "$1" | rev | cut -f2- -d"." | rev)
+	
+	if [ -s ${TMPDIR}/priorities.new ] ; then
+		
+		entry=$(grep ":${pkg}$" ${TMPDIR}/priorities.new)
+
+		if [ ! -z "$entry" ] ; then
+			repo=$(echo $entry | cut -f1 -d":")
+			name=$(echo $entry | cut -f2 -d":")
+			
+			[ ! -e ${WORKDIR}/priorities ] && echo -n "" > ${WORKDIR}/priorities
+			
+			grep -v ":${name}:" ${WORKDIR}/priorities > ${TMPDIR}/priorities.tmp
+			echo "${repo}:${name}:${pkg}" > ${WORKDIR}/priorities
+			cat ${TMPDIR}/priorities.tmp >> ${WORKDIR}/priorities
+		fi
+	fi	  
+  }
 
     # Overrides original upgrade_pkg(). Required by the notification mechanism.
   function upgrade_pkg() {
@@ -736,7 +759,7 @@
 		  FDATE=$(ls -l --full-time /var/log/packages/$(echo $i|sed 's/\.t.z//') |awk '{print $6" "$7}'|sed -r -e 's/\.[0-9]{9}//' -e 's,-,/,' -e 's,-,/,')
 		  echo "$FDATE upgraded:    $i  [$REPOPOS]  (was $PKGFOUND)" >> $WORKDIR/install.log
 		fi
-
+		update_priority_list "$i"
 	done
 	handle_event "upgrade"
   }
@@ -764,7 +787,10 @@
 		  FDATE=$(ls -l --full-time /var/log/packages/$(echo $i|sed 's/\.t.z//') |awk '{print $6" "$7}'|sed -r -e 's/\.[0-9]{9}//' -e 's,-,/,' -e 's,-,/,')
 		  echo "$FDATE $INSTALL_T $i  [$REPOPOS]" >> $WORKDIR/install.log
 		fi
+
+		update_priority_list "$i"
 	done
+	
 	handle_event "install"
   }  
 
@@ -840,6 +866,24 @@
        ARCH="\($ARCH\)\|\([i]*[3456x]86[^_]*\)"
        echo -e "i[3456]86\nx86" > $TMPDIR/greylist.32bit
   fi
+  
+  if [[ "$CMD" == "upgrade" || "$CMD" == "upgrade-all" ]]  && [ -s ${WORKDIR}/priorities ] ; then
+    PREFIX=""
+	for pentry in $(cat ${WORKDIR}/priorities) ; do
+			# data[0] = repository  ; data[1] = basename ; data[2] = "installed package ref"
+		data=( $(echo $pentry | cut -f1,2,3 -d":" --output-delimiter=" ") )
+		
+		if [ -e /var/log/packages/${data[2]} ] ; then
+			if [ -z "$PREFIX" ] ; then
+				PREFIX=( SLACKPKGPLUS_${data[0]}:${data[1]} )
+			else
+				PREFIX=( ${PREFIX[*]} SLACKPKGPLUS_${data[0]}:${data[1]} )
+			fi
+		fi
+	done
+
+    [ ! -z "$PREFIX" ] && PRIORITY=( ${PREFIX[*]} ${PRIORITY[*]} )
+  fi
 
   if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] || [ "$CMD" == "reinstall" ] || [ "$CMD" == "remove" ] ; then
 
@@ -914,10 +958,16 @@
       elif echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
 
         if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then
+        
           repository=$(echo "$pref" | cut -f1 -d":")
           package=$(echo "$pref" | cut -f2- -d":")
+          pkgname=$(grep "^SLACKPKGPLUS_${repository}[ ]${package}[ ]" ${WORKDIR}/pkglist | cut -f6 -d" ")
 
           PRIORITYLIST=( ${PRIORITYLIST[*]} SLACKPKGPLUS_${repository}:$package )
+		  
+          if [ ! -z "$pkgname" ] ; then
+			echo "$repository:$package:$pkgname" >> ${TMPDIR}/priorities.new
+          fi
         fi
 
       # You can specify 'slackpkg install reponame' where reponame is a thirdy part repository

--
SeB

Buumi · 04-12-2014, 02:28 PM

I've been away from Slackware for a while and just installed 14.1 on yesterday. I have followed what's new in the Slackware world and this tool has interested me. Now I have had a chance to test it and so far it works well. Now I don't need to add RSS feed to follow changes in AlienBOB's packages and I can install a few packages from MLED repository to make my XFCE experience better, and keep them updated with slackpkg. Great!

As others said on the previous page, it works great now. I hope it keeps that way and that it doesn't get too complicated to maintain so try to keep it simple and small, please.

Thank you zerouno and phenixia2003 for your work!

phenixia2003 · 04-13-2014, 10:29 AM

Hello,

I reviewed the patch I sent last friday and fixed some issues. The new version (slackpkgplus-1.3.1-1mt_20140413.patch.txt) is for slackpkg+-1.3.1-1mt

This patch ensures that any package installed from a given repository using the command slackpkg install|upgrade <repository>:<package> will be updated only to "newer" version from its original repository when running the command slackpkg upgrade-all or slackpkg upgrade <package>, regardless the priority level of its repository.

Obviously, an upgrade to a version from another repository is always supported using the command :

Code:

$ slackpkg upgrade <repository>:<basename>

Example:

slackpkg+ is configured, with the following priorities :

Code:

  REPOPLUS=( slackpkgplus restricted alienbob slacky )

The package apache-ant is available in repositories alienbob and slacky. With the configure above, running slackpkg install apache-ant, will install the version from alienbob which have priority over slacky. With current slackpkg+, it is possible to install apache-ant from slacky using the command slackpkg install slacky:apache-ant :

Code:

$ slackpkg -checkgpg=off -dialog=off install slacky:apache-ant

Looking for apache-ant in package list. Please wait... --
DONE

apache-ant-1.9.2-x86_64-1sl.txz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

However, running slackpkg upgrade-all, or even slackpkg upgrade apache-ant, will upgrade apache-ant to the version from alienbob repository :

Code:

slackpkg -checkgpg=off -dialog=off upgrade-all

Checking local integrity... DONE
Looking for packages to upgrade. Please wait... -DONE

OpenAL-1.15.1-x86_64-2alien.tgz
apache-ant-1.8.2-noarch-1alien.tgz
...
Do you wish to upgrade selected packages (Y/n)? n

Code:

slackpkg -checkgpg=off -dialog=off upgrade apache-ant

Checking local integrity... DONE
Looking for apache-ant in package list. Please wait... --
DONE

apache-ant-1.8.2-noarch-1alien.tgz

Total package(s): 1

Do you wish to upgrade selected packages (Y/n)? n

To avoid this, it is required to edit slackpkgplus.conf, and update the variable PKGS_PRIORITY to give precedence to apache-ant from slacky over any other version :

Code:

PKGS_PRIORITY=( slacky:apache-ant ... )

With the attached patch, it's not needed anymore. Running the command slackpkg install slacky:apache-ant gives priority to apache-ant from slacky, until it is removed (or superseded by a version from another repository), even when running the commands :

Code:

$ slackpkg upgrade-all
    or
$ slackpkg upgrade apache-ant

Example:

slackpkg+ is configured as below :

Code:

  REPOPLUS=( slackpkgplus restricted alienbob slacky )
  MIRRORPLUS['slacky']=http://repository.slacky.eu/slackware64-14.0/

1. The package apache-ant from slacky is installed as below :

Code:

$ slackpkg -checkgpg=off -dialog=off install slacky:apache-ant

Looking for apache-ant in package list. Please wait... --
DONE

apache-ant-1.9.1-x86_64-1sl.txz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y
...

2. When the command slackpkg upgrade-all (or slackpkg upgrade slacky) is executed, the apache-ant version from alienbob is not shown as an update for the installed version :

Code:

$ slackpkg -checkgpg=off -dialog=off upgrade-all       

Checking local integrity... DONE
Looking for packages to upgrade. Please wait... -||-|\DONE

OpenAL-1.15.1-x86_64-2alien.tgz

Do you wish to upgrade selected packages (Y/n)? n

Code:

$ slackpkg -checkgpg=off -dialog=off upgrade apache-ant       

Checking local integrity... DONE
Looking for apache-ant in package list. Please wait... --
DONE

No packages match the pattern for upgrade. Try:

        /usr/sbin/slackpkg install|reinstall

3. slackpkg+ is reconfigured as below :

Code:

  REPOPLUS=( slackpkgplus restricted alienbob slacky )
  MIRRORPLUS['slacky']=http://repository.slacky.eu/slackware64-14.1/

Here, when the command slackpkg upgrade-all (or slackpkg upgrade slacky) is executed, the apache-ant version from slacky (14.1) is shown as an update for the installed version :

Code:

$ slackpkg -checkgpg=off -dialog=off upgrade-all

Checking local integrity... DONE
Looking for packages to upgrade. Please wait... |/\/-\|/-\||/-\|DONE

OpenAL-1.15.1-x86_64-2alien.tgz
apache-ant-1.9.2-x86_64-1sl.txz
...
Do you wish to upgrade selected packages (Y/n)?

Code:

$ slackpkg -checkgpg=off -dialog=off upgrade apache-ant

Checking local integrity... DONE
Looking for apache-ant in package list. Please wait... --
DONE

apache-ant-1.9.2-x86_64-1sl.txz

Total package(s): 1

Do you wish to upgrade selected packages (Y/n)?

Even if I have (well) tested this patch, there could have some undiscovered issues. Furthermore there's one (important) limitation :

If a package P from a repository R is already installed prior to the installation of this patch and there is another version of package P in a repository R' (which has precedence over R), the execution of slackpkg upgrade P or slackpkg upgrade-all will show P(R') as an update for P. To workaround this, the package P (from R) must be removed and installed again :

Code:

$ slackpkg remove P
$ slackpkg install R:P

IMPORTANT: slackpkg reinstall R:P will not work

Here is an example in the real world :

if apache-ant from slacky is already installed prior to the installation of this patch, the execution of slackpkg upgrade apache-ant or slackpkg upgrade-all will show apache-ant from alienbob as an update. To workaround this, the package must be removed and installed as below :

Code:

$ slackpkg remove apache-ant
$ slackpkg install slacky:apache-ant

IMPORTANT: slackpkg reinstall slacky:apache-ant will not work

I tried to implement a better solution, like creating a priority list according to the installed packages and the list of packages for which there's different version on the configured 3rd party repositories. However this idea can lead to more trouble than expected at first. In fact, there's no way to clearly identify from which repository an installed package comes. The build tag could have been a solution, but this is not the case since there are repositories which use the same tag (alienbob and restricted for instance)

Hope this helps.

--
SeB

gegechris99 · 04-14-2014, 04:39 PM

The slackpkg+ site seems to be down at the moment.

Code:

$slackpkg check-updates

News on ChangeLog.txt

  [ Repository               ] [ Status               ]
    slackware                       Up to date        
    slackpkgplus                 AVAILABLE UPDATES    
    mled                            Up to date        
    restricted                      Up to date        
    alienbob                        Up to date        


==============================================================================
  WARNING! One or more errors occurred while slackpkg was running
------------------------------------------------------------------------------
slackpkgplus: Invalid repository (fails to download CHECKSUMS.md5)

And the DE notification tells me that there is an update available in repository slackpkgplus.

Is there a way to improve the output, and thus the DE notification, to deal with this kind of situation (i.e. one repository is not available for checking)?

Thanks in advance.

zerouno · 04-14-2014, 05:46 PM

Quote:

Originally Posted by gegechris99

The slackpkg+ site seems to be down at the moment.

sorry. Now is up

@phenixia2003:
good. unfortunatly this week I can't test nor apply it.

phenixia2003 · 04-15-2014, 03:36 AM

Hello,

Quote:

Originally Posted by gegechris99

The slackpkg+ site seems to be down at the moment.

And the DE notification tells me that there is an update available in repository slackpkgplus.

Is there a way to improve the output, and thus the DE notification, to deal with this kind of situation (i.e. one repository is not available for checking)?

Thanks in advance.

The attached patch (slackpkgplus-1.3.1-1mt_20140415.patch.txt) (for slackpkg+-1.3.1-1mt) should fix that kind of issue.

Code:

--- slackpkgplus.sh.orig	2014-04-11 11:12:25.029574693 +0200
+++ slackpkgplus.sh	2014-04-15 10:19:16.902409144 +0200
@@ -102,6 +102,7 @@
   # then merge all in a format slackpkg-compatible
   function getfile(){
     local URLFILE
+    local REPO_MD5
     URLFILE=$1
 
     if echo $1|egrep -q '/SLACKPKGPLUS_(file|dir|http|https|ftp)[0-9].*\.asc$';then
@@ -234,6 +235,8 @@
         else
           $DOWNLOADER ${TMPDIR}/CHECKSUMS.md5-$PREPO ${MIRRORPLUS[${PREPO/SLACKPKGPLUS_}]}CHECKSUMS.md5
         fi
+	
+	REPO_MD5=""
 	if [ ! -s ${TMPDIR}/CHECKSUMS.md5-$PREPO ];then
               echo
               echo "                        !!! F A T A L !!!"
@@ -242,10 +245,15 @@
               echo
               sleep 5
 	      echo -e "$PREPO: Invalid repository (fails to download CHECKSUMS.md5)" >> $TMPDIR/error.log
-	fi
 
-        echo "SLACKPKGPLUS_$PREPO[MD5]" $(md5sum ${TMPDIR}/CHECKSUMS.md5-$PREPO|awk '{print $1}') >>$2
+	      if [ -e "${WORKDIR}/ChangeLog.txt" ] ; then
+		PREPO_MD5=$(grep "SLACKPKGPLUS_$PREPO\[MD5\]" ${WORKDIR}/ChangeLog.txt | cut -f2 -d" ")
+	      fi
+	else
+	    PREPO_MD5=$(md5sum ${TMPDIR}/CHECKSUMS.md5-$PREPO|awk '{print $1}')	
+	fi
 
+        echo "SLACKPKGPLUS_$PREPO[MD5]" "$PREPO_MD5" >>$2
       done
     fi
     if [ $(basename $1) = "GPG-KEY" ];then

Hope this helps.

Cheers.

--
SeB

phenixia2003 · 04-15-2014, 03:41 AM

Hello,

Quote:

Originally Posted by zerouno

@phenixia2003:
good. unfortunatly this week I can't test nor apply it.

No problem. Anyway, that kind of change needs to be carefully reviewed before published.

Cheers.

--
SeB

zerouno · 04-15-2014, 10:41 AM

Reopened the development branch
http://slakfinder.org/slackpkg+dev/
https://github.com/zuno/slackpkgplus/tree/devel

currently only I commited the latest fix.

Code:

# slackpkg check-updates

No news is good news


==============================================================================
  WARNING! One or more errors occurred while slackpkg was running
------------------------------------------------------------------------------
slackpkgplus: Invalid repository (fails to download CHECKSUMS.md5)

Code:

# slackpkg check-updates

News on ChangeLog.txt

  [ Repository               ] [ Status               ]
    slackware                       Up to date        
    slackpkgplus                 AVAILABLE UPDATES    
    restricted                      Up to date        
    alienbob                        Up to date        
    slacky                          Up to date

Alien Bob · 04-15-2014, 04:04 PM

I upgraded to the latest slackpkg+ and suddenly the VERBOSE variable is mandatory in slackpkgplus.conf , the script does not set a default value. I kept my existing .conf file because of the customizations, not expecting that new variables are mandatory.
So what I got at "slackpkg update" was this:

Code:

        Formatting lists to slackpkg style...
                Package List: using CHECKSUMS.md5 as source
                Package descriptions
/usr/libexec/slackpkg/functions.d/slackpkgplus.sh: line 60: [: -gt: unary operator expected
Updating install.log



/usr/libexec/slackpkg/functions.d/slackpkgplus.sh: line 71: [: -gt: unary operator expected

/usr/libexec/slackpkg/functions.d/slackpkgplus.sh: line 94: [: -lt: unary operator expected

Please set a default for VERBOSE in slackpkg+

Eric