LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (https://www.linuxquestions.org/questions/slackware-14/)
-   -   slackpkg vs. third-party package repository (https://www.linuxquestions.org/questions/slackware-14/slackpkg-vs-third-party-package-repository-4175427364/)

phenixia2003 03-15-2016 10:23 AM

2 Attachment(s)
Hello,

Quote:

Originally Posted by zerouno (Post 5515337)
I was searching the code for the beta4 ;)

I don't know why, but I had the idea to add the changelog viewer after reading the first post of this thread.

Quote:

Originally Posted by zerouno (Post 5515337)
the dialog function(s) contain a bug regard the size of it.
For the changelog, the dialog is 19x70, but the slackware changelog contains rows up to 79 columns.
For package list, 'package name'+'repository name' may be longer than 70 characters.
This is not a large problem since few packages have long name, and the changelog dialogbox is a non-critical feature, but I think it's the time to solve it.

You can scroll left/right in the changelog viewer with left/right keys.

_____________________

I did some change to the changelog viewer so that it can display changelog entries from 3rd party repositories, too.

To achieve this, I modified getfile() so that, it downloads ChangeLog from 3rd party repositories, merges them with slackware changelog into a file called Unified-ChangeLog.txt, and, generates an index file called Unified-ChangeLog.idx.

Notes:
  1. This does not affect the ChangeLog.txt used by check-updates.

  2. There's no consequence when a repository has no ChangeLog.txt file.

  3. In some case, the ChangeLog.txt of a repository is not directly accessible from the URL specified in slackpkgplus.conf. In that case, getfile() automatically searches in parent URLs. For instance, if the URL specified in slackpkgplus.conf is a/b/c/d and the ChangeLog.txt is in a/, getfile() will try the URLs a/b/c/d/ChangeLog.txt, a/b/c/ChangeLog.txt, a/b/ChangeLog.txt, and a/ChangeLog.txt ...

  4. When the ChangeLog.txt is not found in a repository, a 404 error is reported by the downloader (see below). Maybe it would be better to redirect downloader error in /dev/null when downloading the changelog (lines #574-581 once the patch is applied):
    Code:

    http://taper.alienbase.nl/mirrors/people/alien/multilib/14.1//ChangeLog.txt:
    2016-03-15 16:00:30 ERROR 404: Not Found.

  5. the showChangeLogInfo() function expects the changelog entries follow the "standard" slackware changelog format, which can be described as being :
    Code:

    path/to/package_name.t[blgx]z:[ ]added|moved|rebuilt|upgraded"
    <text|null>
    <SEPARATOR|null>


    SEPARATOR is a string starting with a plus followed by dashed, followed by a plus, and followed by 0 to n spaces.


    Important:
    Some repositories don't follow this format. Therefore, in this case, showChangeLogInfo() will find no changelog entry for packages from these repositories, or could truncate some information from the changelog.

Here is a screenshot of changelog viewer with changelog entries from 3rd party repositories :

As I said earlier, when the repository don't follow the "standard" changelog format, some information can be truncated, which is the case for the entry compat32-tools in the screenshot above.



The patch (Attachment 21154) is for slackpkg+/devel 1.7.b4 :
Code:

--- slackpkgplus.sh.git.1.7.b4        2016-03-15 10:51:40.250215213 +0100
+++ slackpkgplus.sh        2016-03-15 15:46:52.422329359 +0100
@@ -12,6 +12,11 @@
  #
 SLACKDIR_REGEXP="^((slackware)|(slackware64)|(extra)|(pasture)|(patches)|(testing))$"
 
+  # CLOG_PKGREGEX : regular expression used to find package entry in ChangeLog files
+  # CLOG_SEPREGEX : regular expression that match the "standard" entry separator in a ChangeLog file
+CLOG_PKGREGEX="[.]t[blxg]z[:][ ]+(added|moved|rebuilt|upgraded)"
+CLOG_SEPREGEX="^[+][-]+[+][ ]*$"
+
 if [ -e $CONF/slackpkgplus.conf ];then
  # You can override GREYLIST WGETOPTS SLACKPKGPLUS VERBOSE USEBL ALLOW32BIT SENSITIVE_SEARCH from command-line
  EXTGREYLIST=$GREYLIST
@@ -545,6 +550,60 @@
      fi
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
+
+        # ChangeLog.txt from slackware and 3rd party repository are merged
+        # into WORKDIR/Unified-ChangeLog.txt
+        #
+        # At this point, ChangeLog.txt from slackware has been already
+        # download and is stored in WORKDIR/ChangeLog.txt
+
+      cat $WORKDIR/ChangeLog.txt > $WORKDIR/Unified-ChangeLog.txt
+
+      for PREPO in ${REPOPLUS[*]}; do
+        BASEDIR=${MIRRORPLUS[${PREPO/SLACKPKGPLUS_}]%}
+        CLOGNAM=ChangeLog-$PREPO.txt
+
+        BDNAMES=( $(echo $BASEDIR | tr "/" " ") )
+        LIMIT=$(( ${#BDNAMES[@]} - 1 ))
+
+        LEVEL=1
+        while [ ! -s ${TMPDIR}/$CLOGNAM ] && [ $LEVEL -le $LIMIT ] ; do
+
+            URLFILE=$BASEDIR/ChangeLog.txt
+
+            if echo $URLFILE | grep -q "^file://" ; then
+                URLFILE=${URLFILE:6}
+                cp -v $URLFILE ${TMPDIR}/$CLOGNAM
+            elif echo $URLFILE | grep -q "^dir:/" ; then
+                touch ${TMPDIR}/$CLOGNAM
+            else
+                $DOWNLOADER ${TMPDIR}/$CLOGNAM $URLFILE
+            fi
+
+            ((LEVEL++))
+            BASEDIR=$(echo ${BASEDIR%/} |rev|cut -f2- -d/ |rev)
+        done
+
+        if [ -s ${TMPDIR}/$CLOGNAM ] ; then
+            echo -e "[INFO] Merging ChangeLog.txt from repository $PREPO with ${WORKDIR}/Unified-ChangeLog.txt.\n"
+            cat ${TMPDIR}/$CLOGNAM >> ${WORKDIR}/Unified-ChangeLog.txt
+        else
+            echo -e "[INFO] Repository $PREPO has no ChangeLog.txt.\n"
+        fi
+      done
+
+        # Extract each package entry in Unified-ChangeLog.txt and store them in
+        # $WORKDIR/Unified-ChangeLog.idx which is used by showChangeLogInfo()
+        #
+        # The output file is formatted as below :
+        #  <idx>:<pathname>: <status>
+        #
+        # <idx> is the line index of the entry in original Unified-ChangeLog.txt
+        # <pathname> is the full pathname of the package (ie. a/cryptsetup-1.7.1-x86_64-1.txz)
+        # <status> is the package status, which can be added,moved,rebuilt,removed,upgraded)
+        #
+      grep -inE "$CLOG_PKGREGEX" ${WORKDIR}/Unified-ChangeLog.txt > ${WORKDIR}/Unified-ChangeLog.idx
+
      for PREPO in ${REPOPLUS[*]};do
        # Not all repositories have the ChangeLog.txt, so I use md5 of CHECKSUMS.md5 instead
        URLFILE=${MIRRORPLUS[${PREPO/SLACKPKGPLUS_}]}CHECKSUMS.md5
@@ -1082,69 +1141,49 @@
      # Prints, into a dialog box, the changelog entries about the packages listed in file $1
      #
    function showChangeLogInfo() {
-      local PKGREGEX="[.]t[blxg]z[:][ ]+(added|moved|rebuilt|removed|upgraded)"
-      local SEPREGEX="^[+][-]+[+][ ]*$"
      local Cpkg
      local CpkgInfos
-      local Idx
      local CLogStartIdx
-      local CLogEndIdx
      local Pathname
      local Status
      local Cline
 
-          # Extract each package entry in ChangeLog.txt and store them in
-          # $WORKDIR/ChangeLog-packages.idx
-          #
-          # The output file is formatted as below :
-          #  <idx>:<pathname>: <status>
-          #
-          # <idx> is the line index of the entry in original ChangeLog.txt
-          # <pathname> is the full pathname of the package (ie. a/cryptsetup-1.7.1-x86_64-1.txz)
-          # <status> is the package status, which can be added,moved,rebuilt,removed,upgraded)
-          #
-          # [PENDING] this should be done only once when slackpkg update is called
-          #
-      grep -inE "$PKGREGEX" $WORKDIR/ChangeLog.txt > $WORKDIR/ChangeLog-packages.idx
-
      echo -n "" > $TMPDIR/Packages.clog
 
      for Cpkg in $(<$TMPDIR/dialog.out) ; do
 
        #  get infos about the current package from changeLog-packages.idx file, if any. The
        #  variable CpkgInfos is a string formatted as below:
-        #    <idx1>:<clogidx>:<pathname>: <status>
+        #    <clogidx>:<pathname>: <status>
        #
-        #  idx1=index of the line in changelog-packages.idx which match Cpkg
        #  clogidx=line index of the entry in ChangeLog.txt that match Cpkg
        #
-        CpkgInfos=( $(grep -n $Cpkg  $WORKDIR/ChangeLog-packages.idx | tr ":" " ") )
+        CpkgInfos=( $(grep $Cpkg  $WORKDIR/Unified-ChangeLog.idx | tr ":" " ") )
 
        if [ ! -z "$CpkgInfos" ] ; then
-          Idx=${CpkgInfos[0]}
-          CLogStartIdx=${CpkgInfos[1]}
-          Pathname=${CpkgInfos[2]}
-          Status=$(echo ${CpkgInfos[3]} | tr --delete " .")
+          CLogStartIdx=${CpkgInfos[0]}
+          Pathname=${CpkgInfos[1]}
+          Status=$(echo ${CpkgInfos[2]} | tr --delete " .")
 
          echo "$Pathname ($Status)" >> $TMPDIR/Packages.clog
 
          # extra information on package Cpkg can be found in ChangeLog.txt file
          # starting at line CLogStartIdx+1 and ending the line before the first
-          # line matching the regular expression SEPREGEX or PKGREGEX.
+          # line matching the regular expression CLOG_SEPREGEX or CLOG_PKGREGEX.
          #
-          # SEPREGEX match the "standard" changelog separator entry, ie. a string
+          # CLOG_SEPREGEX match the "standard" changelog separator entry, ie. a string
          # which start with a plus followed by dashes and a plus. For instance:
          #  +----------------------+
          #
-          # PKGREGEX match the "standard" changelog package entry, ie. a string
+          # CLOG_PKGREGEX match the "standard" changelog package entry, ie. a string
          # which starts with a package pathname followed by colon, one or more
          # space and the status. For instance:
          #  n/bind-1.2.3-x86_64-1.txz: Upgraded.
 
          ((CLogStartIdx++))
 
-          tail -n "+$CLogStartIdx" $WORKDIR/ChangeLog.txt | while read Cline ; do
-            if ! echo "$Cline" | grep -qiE "($SEPREGEX)|($PKGREGEX)" ; then
+          tail -n "+$CLogStartIdx" $WORKDIR/Unified-ChangeLog.txt | while read Cline ; do
+            if ! echo "$Cline" | grep -qiE "($CLOG_SEPREGEX)|($CLOG_PKGREGEX)" ; then
              echo -e "\t$Cline" >> $TMPDIR/Packages.clog
            else
              break
@@ -1153,7 +1192,11 @@
          echo "" >> $TMPDIR/Packages.clog
        fi
      done
-     
+
+      if [ ! -s $TMPDIR/Packages.clog ] ; then
+        echo "Sorry, no entry in the ChangeLog.txt matching the selected packages." > $TMPDIR/Packages.clog
+      fi
+
      dialog --title "ChangeLog" \
        --backtitle "slackpkg $VERSION" $HINT \
        --textbox $TMPDIR/Packages.clog 19 70

Hope this help.

--
Seb

zerouno 03-21-2016 04:03 PM

I'm sorry for the time, but I'm a lot occupied this period.

Yes, I like the Changelog for 3th party repository, even if just few repositories has useful information (well, when I wrote slackpkg+ there was few repositories with CHECKSUMS.md5 and very few with the GPG-KEY; today all have it) and not all Changelog files have a slackware-compatible format.

I dislike to search Changelog in parent directories becouse that means to go out the path specified from the user. I know the a lot o repositories have Changelog just in parent. So I think that we should add a setting in slackpkgplus.conf where the user explicitly allow to serch changelog in parent.

The 404 problem reported from wget I think that should be managed but it is to review later becouse we have a lot of downloader
1) wget
2) wgetdebug
3) cacheddownloader (that also use curl to retrieve the headers)
4) custom $DOWNLOADCMD

we have the same problem with some not existent MANIFEST.bz2 and other metadata that not all repositories have.

Also I want to reintroduce the function that we removed a lot of time ago, regarding the install-new working for multilib.
Everytime I run the install-new command I have to remember to run slackpkg install multilib; and if I have no a full slackware installation I have to remember which 32bit libraries I have to install. The idea is the same of years ago.. find for 32bit packages that have the corrispondent 64bit installed as setupmultilib.sh does.

phenixia2003 03-22-2016 08:19 AM

Hello,

Quote:

Originally Posted by zerouno (Post 5519351)
I'm sorry for the time, but I'm a lot occupied this period.

I understand and I see no problem with that.


Quote:

Originally Posted by zerouno (Post 5519351)
I dislike to search Changelog in parent directories becouse that means to go out the path specified from the user. I know the a lot o repositories have Changelog just in parent. So I think that we should add a setting in slackpkgplus.conf where the user explicitly allow to serch changelog in parent.

Something like that should do the trick :

1. add the variable SEARCH_CLOG_INPARENT which allows to define whether or not the 3rd party repository changelogs must be searched in parent:
Code:

# Defines if the changelog of any 3rd party repository must be searched in parent URL when not found in base URL.
 #
 # Can be set to "on" or "off" (default)
 #
SEARCH_CLOG_INPARENT=off

2. Once you have applied the patch I sent, replace the two lines below :
Code:

        BDNAMES=( $(echo $BASEDIR | tr "/" " ") )
        LIMIT=$(( ${#BDNAMES[@]} - 1 ))

by :
Code:

      LIMIT=1

      if [ "$SEARCH_CLOG_INPARENT" == "on" ] ; then
            BDNAMES=( $(echo $BASEDIR | tr "/" " ") )
            LIMIT=$(( ${#BDNAMES[@]} - 1 ))
      fi


Quote:

Originally Posted by zerouno (Post 5519351)
The 404 problem reported from wget I think that should be managed but it is to review later becouse we have a lot of downloader
1) wget
2) wgetdebug
3) cacheddownloader (that also use curl to retrieve the headers)
4) custom $DOWNLOADCMD

we have the same problem with some not existent MANIFEST.bz2 and other metadata that not all repositories have.

Ok.

Quote:

Originally Posted by zerouno (Post 5519351)
Also I want to reintroduce the function that we removed a lot of time ago, regarding the install-new working for multilib.
Everytime I run the install-new command I have to remember to run slackpkg install multilib; and if I have no a full slackware installation I have to remember which 32bit libraries I have to install. The idea is the same of years ago.. find for 32bit packages that have the corrispondent 64bit installed as setupmultilib.sh does.

Here is my point of view on this:
  • install-new should be left as is : dedicated to install new packages added to official slackware tree, and only that. Therefore, to install new packages from any other repository, slackpkg install <reponame> must be used instead.
  • Reintroducing install-new only for the multilib repository could be misunderstood by users. They could think that install-new works for all 3rd party repositories, and would not understand why the required package Y added to the repository R is not proposed by slackpkg.
  • As far as i remember, introducing install-new for all repositories is tricky, and anyway, could lead slackpkg to suggest a bunch of unexpected/undesired packages. So, doing that is not an option.


--
SeB

zerouno 04-10-2016 07:43 AM

Sorry for absence..
On my office I've currently double work to do and a lot to do on my house.

However if there are not other think to report, these days I will release beta5 with the only 3th party changelogs, and if it works well I think that we can release an rc release to freeze any other new feature and only fix current.
If someone has something to say, speak now or forever hold your peace :D

I have other ideas but I finished the time to elaborate it; I will add it in a further version.

I whould some feedback from who think to put slackpkg in her own repository and from who asked for a stable release.

My idea is to release slackpkg 1.7 stable before slackware 14.2 (that is currently -rc1) so to test it sufficiently to call it stable when slackware 14.2 will be release.

zerouno 04-11-2016 03:30 PM

I'm seeing the code for changelog and I found something to fix.

(aligned to latest patch excluding SEARCH_CLOG_INPARENT implementation)

* row 1284 and 1343
Code:

if $CLogOpt ; then
...
  cat $TMPDIR/dialog.tmp | sed "s/ on / off /g" > $TMPDIR/dialog.tmp.off
fi

Code:

                sed -i -e "/^$PKGS_REGEX/ s= off = on =" $TMPDIR/dialog.tmp
CLogOpt is defined as CLOGopt
dialog.tmp does not ends with blank space so dialog.tmp.off is always == dialog.tmp


* row 1334
Code:

              if [ -s $TMPDIR/dialog.out ] ; then
                showChangeLogInfo $TMPDIR/dialog.out

if no packages are selected the selection dialogs return without an error, and the user think that it does not work.
I think that should appear a dialog with a similar of "Please, select at least a package"


* row 1187
Code:

              echo -e "\t$Cline" >> $TMPDIR/Packages.clog
changelog should not use tab becouse it has an anomaly when you scroll left<->right with the arrow keys.



Currently I've not tried all, but it is a great work.

Thankyou

phenixia2003 04-12-2016 03:13 AM

Quote:

Originally Posted by zerouno (Post 5529534)
I'm seeing the code for changelog and I found something to fix.

(aligned to latest patch excluding SEARCH_CLOG_INPARENT implementation)

* row 1284 and 1343
Code:

if $CLogOpt ; then
...
  cat $TMPDIR/dialog.tmp | sed "s/ on / off /g" > $TMPDIR/dialog.tmp.off
fi

Code:

                sed -i -e "/^$PKGS_REGEX/ s= off = on =" $TMPDIR/dialog.tmp
CLogOpt is defined as CLOGopt

So the if $CLogOpt statement is always executed. Sorry. Just change it by if $CLOGopt ...

Quote:

Originally Posted by zerouno (Post 5529534)
dialog.tmp does not ends with blank space so dialog.tmp.off is always == dialog.tmp

Sorry, but I don't understand what you mean :
Code:

$ slackpkg upgrade kde-runtime kde-wallpapers kdelibs

Checking local integrity... DONE
Looking for kde-runtime kde-wallpapers kdelibs in package list. Please wait... DONE

(DEBUG) create dialog.tmp.off from dialog.tmp ...
(DEBUG) save /tmp/slackpkg.ewwQMv/{dialog.tmp,dialog.tmp.off} in /tmp


$ diff -u /tmp/dialog.tmp /tmp/dialog.tmp.off
--- /tmp/dialog.tmp    2016-04-12 09:34:20.148145646 +0200
+++ /tmp/dialog.tmp.off 2016-04-12 09:34:20.149145646 +0200
@@ -1,3 +1,3 @@
-kde-runtime-4.14.3-x86_64-2alien.txz "ktown" on "installed: kde-runtime-4.10.5-x86_64-1  -->  available: 4.14.3-2alien(ktown) , 4.10.5-1(slackware64) "
-kde-wallpapers-4.14.3-noarch-1alien.txz "ktown" on "installed: kde-wallpapers-4.10.5-noarch-1  -->  available: 4.14.3-1alien(ktown) , 4.10.5-1(slackware64) "
-kdelibs-4.14.6-x86_64-1alien.txz "ktown" on "installed: kdelibs-4.10.5-x86_64-2  -->  available: 4.14.6-1alien(ktown) , 4.10.5-2(slackware64) "
+kde-runtime-4.14.3-x86_64-2alien.txz "ktown" off "installed: kde-runtime-4.10.5-x86_64-1  -->  available: 4.14.3-2alien(ktown) , 4.10.5-1(slackware64) "
+kde-wallpapers-4.14.3-noarch-1alien.txz "ktown" off "installed: kde-wallpapers-4.10.5-noarch-1  -->  available: 4.14.3-1alien(ktown) , 4.10.5-1(slackware64) "
+kdelibs-4.14.6-x86_64-1alien.txz "ktown" off "installed: kdelibs-4.10.5-x86_64-2  -->  available: 4.14.6-1alien(ktown) , 4.10.5-2(slackware64) "

Quote:

Originally Posted by zerouno (Post 5529534)
* row 1334
Code:

              if [ -s $TMPDIR/dialog.out ] ; then
                showChangeLogInfo $TMPDIR/dialog.out

if no packages are selected the selection dialogs return without an error, and the user think that it does not work.
I think that should appear a dialog with a similar of "Please, select at least a package"

Right. This should do the trick :
Code:

  if [ -s $TMPDIR/dialog.out ] ; then
    showChangeLogInfo $TMPDIR/dialog.out
    ...
  else
    dialog --title "Changelog" --msgbox "Please, select at least one package." 5 40
    ...
  fi

Quote:

Originally Posted by zerouno (Post 5529534)
* row 1187
Code:

              echo -e "\t$Cline" >> $TMPDIR/Packages.clog
changelog should not use tab becouse it has an anomaly when you scroll left<->right with the arrow keys.

Maybe you can supersede it with 4 to 8 spaces.

--
SeB

zerouno 04-12-2016 09:03 AM

Quote:

Sorry, but I don't understand what you mean :
I have tried with slackpkg install, not upgrade.
There may have a different format.
I have not a pc now to test.

phenixia2003 04-12-2016 10:01 AM

Quote:

Originally Posted by zerouno (Post 5529889)
I have tried with slackpkg install, not upgrade.
There may have a different format.
I have not a pc now to test.

Ok, I didn't think about that. sorry.

So to fix the 1st problem, replace :

Code:

cat $TMPDIR/dialog.tmp | sed "s/ on / off /g" > $TMPDIR/dialog.tmp.off
by

Code:

          # in case of install, dialog.tmp is to the format (1), otherwise the format (2)
          # is used :
          #
          # format (1)
          #  <pkg-name> <repository> on|off
          # format (2)
          #  <pkg-name> <repository> on|off <installed-version> --> <available-version>

        if [ "$2" == "install" ] ; then
                cat $TMPDIR/dialog.tmp | sed "s/ on$/ off/g" > $TMPDIR/dialog.tmp.off
        else
                cat $TMPDIR/dialog.tmp | sed "s/ on / off /g" > $TMPDIR/dialog.tmp.off
        fi

and to fix the 2nd problem, replace :
Code:

                sed -i -e "/^$PKGS_REGEX/ s= off = on =" $TMPDIR/dialog.tmp
by
Code:

                  # in case of install, dialog.tmp is to the format (1), otherwise the format (2)
                  # is used :
                  #
                  # format (1)
                  #  <pkg-name> <repository> on|off
                  # format (2)
                  #  <pkg-name> <repository> on|off <installed-version> --> <available-version>

                if [ "$2" == "install" ] ; then
                        sed -i -e "/^$PKGS_REGEX/ s= off$= on=" $TMPDIR/dialog.tmp
                else
                        sed -i -e "/^$PKGS_REGEX/ s= off = on =" $TMPDIR/dialog.tmp
                fi

P.S: I think its better to have comments here to understand why we're doing that. However, you can get rid of one of the two if you want.


--
SeB

zerouno 04-15-2016 01:17 PM

slackpkg+-1.7.b4.1 released

I also removed some unuseful verbosity in cached_downloader for clarity and compact output. If you want some output set VERBOSE=2 or does not use the caching downoader

zerouno 04-20-2016 12:39 PM

May be useful to write - in the Unified-ChangeLog.idx - then repository name:

19127:slackpkgplus: pkg/slackpkg+-1.7.b4.2-noarch-2mt.txz: Rebuilt

zerouno 04-20-2016 03:38 PM

Only preparing the next slackpkg+ changelog I saw what we done in this release.
I needed a separated file to track all changes!!

A very thanks to all contributing with code, bug reporting, idea or comments!!!!

Now begins the bugfixing only.


Code:

== From 1.6 and 1.7 ==

*Configuration file:

- Added SEARCH_CLOG_INPARENT: available 'on'/'off'; default 'off'
  Allow/disallow to search ChangeLog.txt in parent url. May be unsafe.

- Added WW_FILE_SEARCH: available 'on'/'off'; default 'off'
  Allow you to enable or disable the Whole Word search in 'slackpkg file-search'

- Added DETAILED_INFO: available 'none'/'basic'/'filelist'; default 'none'
  Specify the verbosity of output in 'slackpkg info'.

- Added STRICTGPG: available 'on'/'off'; default 'on'
  Allow you to disable the Strict GPG Check. This is NOT SECURE, but some
  repository may need it.


*New features:

- slackpkg+ allow you to see the ChangeLog in dialog box.
    Now 'slackpkg update' download all ChangeLog.txt from all repositories (if
    available). Then when you go to in install/upgrade a package have a chance
    to read it before install the package. Note that the ChangeLog must be in
    the same Slackware changelog format. Some repository does not have the
    ChangeLog.txt in the main url but has it in a parent url. Be sure to set
    SEARCH_CLOG_INPARENT to 'on' to search it in parent. Note that ascend out
    of main url sometime may be unsafe; so if it does not work reset it to 'off'.

- slackpkg info allow you to see more details about the package.
    By default slackpkg info shows you the slackware metadata and package
    description. Set DETAILED_INFO to 'basic' and you can also see the
    repository and url. Set it to 'filelist' and you will see the full file
    listing in the package. Note that packages as kernel-source or other big
    packages give you a loooong list.

- slackpkg+ uses a Strict GPG Check for packages and metadata .asc files.
    This is a security fix.
    slackpkg was bird to install slackware packages from official mirrors; they
    are signed with the official slackware GPG-KEY, so no other packages may be
    added (unless you disable GPG check).
    slackpkg+ introduces the ability to download keys from many repositories; as
    implication, slackpkg+ allow also to copy packages from other repositories and
    the original signature; in this way a repository can contain a mix of packages
    that may confuse the user.
    Strict GPG check disallow users to do that. This improve the security.
    However some repository NEEDs the mixes of packages, so you can disable this
    function by setting STRICTGPG to 'off'

*Improvements

- Cached Download:
    Now it gives you a quick output for a most clear output.
    Now you can use it in slackpkg check-updates.
   
- Performance:
    slackpkg now is faster when search packages or prepare the package list in
    large install/upgrade

- slackpkg search:
    * If a package match in more than one repository, id did show just the package
    that had priority; now it show all matching packages from all repositories.
    It will mark as 'masked' the packages that would not selected in install/upgrade
    mode. Also always show repository source for official packages.
    * It does order the results by status and colors the output
    * It does search in dir:// repositories

   

- Graylist:
    Now it works for slackpkg remove and clean-system too (however it matches
    package name only, not match for repository name or package path)

- slackpkg check-updates:
    * Added spinning
    * Shows downloader output if VERBOSE=3
    * Does not more show notices about remember to run 'slackpkg update'

- slackpkg reinstall:
    Allow to choose the repository as in  install/upgrade; this becouse there are
    more than one repository containing packages with identical name.
 
- dir:// repositories:
    * dir:// repositories and inline repositories has most priority
    * slackpkg search does search in dir:// repositories
    * now dir:// allow subdirectories


*BugFixes

- slackpkg did give 'grep: write error' when running after "sudo su -"

- Fixed a bug on x86 hosts when a repository contains also x86_64 packages.

- Fixed a bug in slackpkg upgrade when used with $ROOT.

- Fixed Download Only feature when upgrade base packages (aaa_base, pkgtools...).

- Fixed a regression in SENSITIVE_SEARCH

- Fixed bad trap CTRL+C when spinning is on.

- Various fixes in TAG_PRIORITY functionality.

- slackpkg could corrupt the slackware database if running when there are
  some manual upgrades in progress.

- slackpkg search did not honour correctly the '+' character



*Various

- Repository name can contains letters, numbers, minus and underscore

- Added and fixed repositories; improved checkrepos.sh

- Code reordering; now slackpkg+ is only slackpkgplus.sh; removed zdialogplus.sh


phenixia2003 04-21-2016 09:07 AM

2 Attachment(s)
Hello,

Quote:

Originally Posted by zerouno (Post 5534076)
May be useful to write - in the Unified-ChangeLog.idx - then repository name:

19127:slackpkgplus: pkg/slackpkg+-1.7.b4.2-noarch-2mt.txz: Rebuilt

I guess you want to print the repository name in front of each entry displayed in the ChangeLog dialog.

I looked at this, but this would be too tricky to do with the current code because of the way files Unified-ChangeLog.txt and Unified-ChangeLog.idx are generated.

So to achieve this, I modified the code, so that, for each repository, there is a couple of files {<reponame>.txt, <reponame>.idx} stored in directory WORKDIR/ChangeLogs, and I updated the functions getfile() and showChangeLogInfo() accordingly.

the patch Attachment 21586 is for slackpkg+/dev 1.7.b4.2 :
Code:

--- slackpkgplus.sh.git.1.7.b4.2        2016-04-21 15:58:55.723360071 +0200
+++ slackpkgplus.sh        2016-04-21 15:58:55.723360071 +0200
@@ -564,18 +564,21 @@
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
 
-      # ChangeLog.txt from slackware and 3rd party repository are merged
-      # into WORKDIR/Unified-ChangeLog.txt
-      #
-      # At this point, ChangeLog.txt from slackware has been already
-      # download and is stored in WORKDIR/ChangeLog.txt
+      # ChangeLog.txt from slackware and 3rd party repository are stored
+      # into WORKDIR/ChangeLogs directory
 
-      cat $WORKDIR/ChangeLog.txt > $WORKDIR/Unified-ChangeLog.txt
+      if [ -e ${WORKDIR}/ChangeLogs ] ; then
+        rm -f ${WORKDIR}/ChangeLogs/{*.txt,*.idx}
+      else
+        mkdir ${WORKDIR}/ChangeLogs
+      fi
+
+      # Copy slackware ChangeLog.txt into directory dedicated to changelogs...
+      cat $WORKDIR/ChangeLog.txt > $WORKDIR/ChangeLogs/slackware.txt
 
      for PREPO in ${REPOPLUS[*]}; do
-        echo "======== Repository: $PREPO ========" >>${WORKDIR}/Unified-ChangeLog.txt
        BASEDIR=${MIRRORPLUS[${PREPO/SLACKPKGPLUS_}]%}
-        CLOGNAM=ChangeLog-$PREPO.txt
+        CLOGNAM=$PREPO.txt
 
        LIMIT=1
 
@@ -604,24 +607,26 @@
        done
 
        if [ -s ${TMPDIR}/$CLOGNAM ] ; then
-          echo -e "                Merging ChangeLog.txt from repository $PREPO with Unified-ChangeLog.txt.\n"
-          cat ${TMPDIR}/$CLOGNAM >> ${WORKDIR}/Unified-ChangeLog.txt
+          echo -e "                Saving ChangeLog.txt from repository $PREPO ...\n"
+          cat ${TMPDIR}/$CLOGNAM >> ${WORKDIR}/ChangeLogs/$CLOGNAM
        else
          echo -e "                Repository $PREPO has no ChangeLog.txt.\n"
        fi
      done
 
-      # Extract each package entry in Unified-ChangeLog.txt and store them in
-      # $WORKDIR/Unified-ChangeLog.idx which is used by showChangeLogInfo()
+      # For each <reponame>.txt file in WORKDIR/ChangeLogs, create a corresponding
+      # <reponame>.idx which is used by showChangeLogInfo()
      #
      # The output file is formatted as below :
      #  <idx>:<pathname>: <status>
      #
-      # <idx> is the line index of the entry in original Unified-ChangeLog.txt
+      # <idx> is the line index of the entry in original changelog <reponame>.txt
      # <pathname> is the full pathname of the package (ie. a/cryptsetup-1.7.1-x86_64-1.txz)
      # <status> is the package status, which can be added,moved,rebuilt,removed,upgraded)
      #
-      grep -inE "$CLOG_PKGREGEX" ${WORKDIR}/Unified-ChangeLog.txt > ${WORKDIR}/Unified-ChangeLog.idx
+      for PREPO in slackware ${REPOPLUS[*]} ; do
+        grep -inE "$CLOG_PKGREGEX" ${WORKDIR}/ChangeLogs/$PREPO.txt > ${WORKDIR}/ChangeLogs/$PREPO.idx
+      done
 
      for PREPO in ${REPOPLUS[*]};do
        # Not all repositories have the ChangeLog.txt, so I use md5 of CHECKSUMS.md5 instead
@@ -1168,33 +1173,42 @@
    function showChangeLogInfo() {
      local Cpkg
      local CpkgInfos
+      local CLogIdxFile
      local CLogStartIdx
      local Pathname
      local Status
      local Cline
+      local Repository
 
      echo -n "" > $TMPDIR/Packages.clog
 
      for Cpkg in $(<$TMPDIR/dialog.out) ; do
 
-        #  get infos about the current package from changeLog-packages.idx file, if any. The
-        #  variable CpkgInfos is a string formatted as below:
-        #    <clogidx>:<pathname>: <status>
+        #  get infos about the current package from *.idx files in WORKDIR/ChangeLogs,
+        #  if any. The  variable CpkgInfos is a string formatted as below:
+        #    path/<reponame>.idx:<clogidx>:<pathname>:<status>
        #
-        #  clogidx=line index of the entry in ChangeLog.txt that match Cpkg
+        #  clogidx=line index of the entry in WORKDIR/ChangeLogs/<reponame>.txt that match Cpkg
        #
-        CpkgInfos=( $(grep $Cpkg  $WORKDIR/Unified-ChangeLog.idx | tr ":" " ") )
+        CpkgInfos=( $(grep -R $Cpkg  $WORKDIR/ChangeLogs/*.idx | tr ":" " ") )
 
        if [ ! -z "$CpkgInfos" ] ; then
-          CLogStartIdx=${CpkgInfos[0]}
-          Pathname=${CpkgInfos[1]}
-          Status=$(echo ${CpkgInfos[2]} | tr --delete " .")
-
-          echo "$Pathname ($Status)" >> $TMPDIR/Packages.clog
-
-          # extra information on package Cpkg can be found in ChangeLog.txt file
-          # starting at line CLogStartIdx+1 and ending the line before the first
-          # line matching the regular expression CLOG_SEPREGEX or CLOG_PKGREGEX.
+          CLogIdxFile=${CpkgInfos[0]}
+          CLogStartIdx=${CpkgInfos[1]}
+          Pathname=${CpkgInfos[2]}
+          Status=$(echo ${CpkgInfos[3]} | tr --delete " .")
+
+          # Get the repository name containing a changelog entry about the current
+          # package (ie Cpkg).
+          #
+          Repository=$(basename $CLogIdxFile .idx)
+
+          echo "$Repository::$Pathname ($Status)" >> $TMPDIR/Packages.clog
+
+          # extra information on package Cpkg can be found in file
+          # WORKDIR/ChangeLogs/${Repository}.txt starting at line
+          # CLogStartIdx+1 and ending the line before the first line matching
+          # the regular expression CLOG_SEPREGEX or CLOG_PKGREGEX.
          #
          # CLOG_SEPREGEX match the "standard" changelog separator entry, ie. a string
          # which start with a plus followed by dashes and a plus. For instance:
@@ -1207,7 +1221,7 @@
 
          ((CLogStartIdx++))
 
-          tail -n "+$CLogStartIdx" $WORKDIR/Unified-ChangeLog.txt | while read Cline ; do
+          tail -n "+$CLogStartIdx" $WORKDIR/ChangeLogs/${Repository}.txt | while read Cline ; do
            if ! echo "$Cline" | grep -qiE "($CLOG_SEPREGEX)|($CLOG_PKGREGEX)" ; then
              echo -e "    $Cline" >> $TMPDIR/Packages.clog
            else

Here is a screenshot of slackpkg with this patch in action :

Cheers.

--
SeB

zerouno 04-28-2016 03:00 AM

The ideas was to put repository name in Unified-ChangeLog.idx so to see it when opening manually that file, but it's a good idea to show it in dialog and put metadata in a dedicaded subdir.

I not jet merging that code, however I found a bug in the old code.

Code:

      #
      # At this point, ChangeLog.txt from slackware has been already
      # download and is stored in WORKDIR/ChangeLog.txt

      cat $WORKDIR/ChangeLog.txt > $WORKDIR/Unified-ChangeLog.txt

this is false.
In effect at this point the $WORKDIR contains the older changelog, so if you run slackpkg update after a slackware update you does not have the new slackware changes in Unified-ChangeLog.
The ChangeLog.txt is downloaded in $TMPDIR
Code:

2016-04-28 09:43:20 URL:http://ftp.osuosl.org/.2/slackware/slackware64-current/ChangeLog.txt [298373/298373] -> "/tmp/slackpkg.9fNs11/ChangeLog.txt" [1]
Also I think that the entire Unified-ChangeLog should be generated in $TMPDIR.
If you abort with ctrl+c the slackpkg update during changelog download, the older Unified-ChangeLog will be removed and the new is partial.

phenixia2003 04-28-2016 09:10 AM

Quote:

Originally Posted by zerouno (Post 5537701)
The ideas was to put repository name in Unified-ChangeLog.idx so to see it when opening manually that file, but it's a good idea to show it in dialog and put metadata in a dedicaded subdir.

I not jet merging that code.

So, do you plan to include it, or do you want to stick with older version with Unified-ChangeLog.{txt,idx} ?

Quote:

Originally Posted by zerouno (Post 5537701)
however I found a bug in the old code.

Oops, sorry.

Quote:

Originally Posted by zerouno (Post 5537701)
Code:

      #
      # At this point, ChangeLog.txt from slackware has been already
      # download and is stored in WORKDIR/ChangeLog.txt

      cat $WORKDIR/ChangeLog.txt > $WORKDIR/Unified-ChangeLog.txt

this is false.
In effect at this point the $WORKDIR contains the older changelog, so if you run slackpkg update after a slackware update you does not have the new slackware changes in Unified-ChangeLog.
The ChangeLog.txt is downloaded in $TMPDIR
Code:

2016-04-28 09:43:20 URL:http://ftp.osuosl.org/.2/slackware/slackware64-current/ChangeLog.txt [298373/298373] -> "/tmp/slackpkg.9fNs11/ChangeLog.txt" [1]

Ah, ok. In fact, on command "update", slackpkg calls updatefilelists() which calls checkchangelog(), so the ChangeLog.txt is downloaded in TMPDIR, then, when checkchangelog() returns, updatefilelists() copies ChangeLog.txt from TMPDIR into WORKDIR.


Quote:

Originally Posted by zerouno (Post 5537701)
Also I think that the entire Unified-ChangeLog should be generated in $TMPDIR.
If you abort with ctrl+c the slackpkg update during changelog download, the older Unified-ChangeLog will be removed and the new is partial.

In this case the Unified-ChangeLog.{txt,idx} (or files in ChangeLogs/) generated in TMPDIR by getfile() will need to be copied into WORKDIR before slackpkg exits. However, since we can't control when getfile() is called to download the ChangeLog file, we have 2 choices here :

1. Duplicate the code which handle the command "update" from original slackpkg, and add the required code to copy the ChangeLog from TMPDIR into WORKDIR :
Code:

    update)
                # If you are using "slackpkg update gpg" OR the system
                # doesn't have Slackware GPG key, download and install
                # the key
                #
                if [ "$UPARG" = "gpg" ] || [ "$GPGFIRSTTIME" = "0" ]; then
                        #
                        # Creates .gnupg directory if doesn't exist
                        # without this dir, gpg got an error.
                        #
                        if ! [ -e ~/.gnupg ]; then
                                mkdir ~/.gnupg
                        fi
                        getfile ${SOURCE}GPG-KEY $TMPDIR/gpgkey
                        gpg --yes --batch --delete-key "$SLACKKEY" &>/dev/null
                        gpg --import $TMPDIR/gpgkey &>/dev/null && \
                        echo -e "\t\t\tSlackware Linux Project's GPG key added"
                        if [ "$UPARG" = "gpg" ]; then
                                cleanup
                        fi
                fi
                echo "Updating the package lists..."
                updatefilelists

                # >> copy the ChangeLog from TMPDIR into WORKDIR
                cp $TMPDIR/Unified-ChangeLog.* $WORKDIR


                # exit to prevent original slackpkg to handle update command
                cleanup

2. copy the Unified-ChangeLog.{txt,idx} from TMPDIR into WORKDIR when the function cleanup() is called, unless this function is called in response to a CTRLC (ie. SIGINT) :
Code:

function cleanup() {
    local lEcode=$?

    if [ $lEcode -ne 130 ] && [ -e $TMPDIR/Unified-ChangeLog.txt ] ; then
        cp $TMPDIR/Unified-ChangeLog.{txt,idx} $WORKDIR
    fi

    ... original cleanup code here ...
}

Which solution do you want to implement ?

Another point which has nothing to do with the last changes.

This morning, when reading this thread, I've tried to run the command slackpkg search mozilla-firefox, but it returned nothing :
Code:

$ slackpkg search mozilla-firefox

Looking for mozilla-firefox in package list. Please wait... DONE

No package name matches the pattern.

However, the commands below show that firefox is installed, and referenced in slackpkg database. Furthermore, firefox is not blacklisted :
Code:

$ ls /var/log/packages/mozilla-firefox-38.6.1esr-x86_64-1_slack14.1
/var/log/packages/mozilla-firefox-38.6.1esr-x86_64-1_slack14.1

$ grep "mozilla-firefox" /var/lib/slackpkg/pkglist
patches mozilla-firefox 38.8.0esr x86_64 1_slack14.1 mozilla-firefox-38.8.0esr-x86_64-1_slack14.1 ./patches/packages txz
slackware64 mozilla-firefox 24.1.0esr x86_64 1 mozilla-firefox-24.1.0esr-x86_64-1 ./slackware64/xap txz
SLACKPKGPLUS_mleddesktop mozilla-firefox-langpack 38.7.1 x86_64 1_microlinux mozilla-firefox-langpack-38.7.1-x86_64-1_microlinux ./SLACKPKGPLUS_mleddesktop/slackware64/locale txz

So, I checked slackpkg+ code on github, and found that "search" works well until this commit. The problem comes from line #920 :

Code:

grep ${GREPOPTS} "^$DIR" $WORKDIR/pkglist|grep ${GREPOPTS} "/SLACKPKGPLUS_$SEARCHSTR/\|/$SEARCHSTR/\|/$SEARCHSTR \| [^ /]*$SEARCHSTR[^ /]* " > $PKGINFOS
This line fails because GREPOPTS include the option --word-regex and there are slashes in the regular-expression. However I don't understand the need of those slashes (in Bold RED). Can you explain this ?

Anyway, without these slash at line #920 :
Code:

grep ${GREPOPTS} "^$DIR" ${WORKDIR}/pkglist ${TMPDIR}/pkglist-pre|grep -E ${GREPOPTS} "SLACKPKGPLUS_$SEARCHSTR|$SEARCHSTR|$SEARCHSTR | [^ ]*$SEARCHSTR[^ ]* " > $PKGINFOS
the command slackpkg search works as expected:
Code:

slackpkg search mozilla-firefox

Looking for mozilla-firefox in package list. Please wait... DONE

The list below shows all packages with name matching "mozilla-firefox".

[ Status          ] [ Repository              ] [ Package                                  ]
  uninstalled              mleddesktop                  mozilla-firefox-langpack-38.7.1-x86_64-1_microlinux 
  upgrade                  patches                      mozilla-firefox-38.6.1esr-x86_64-1_slack14.1 --> mozilla-firefox-38.8.0esr-x86_64-1_slack14.1 
  upgrade                  slackware64                  mozilla-firefox-38.6.1esr-x86_64-1_slack14.1 --> mozilla-firefox-24.1.0esr-x86_64-1 

You can search specific files using "slackpkg file-search file".

Hope this help.

--
SeB

zerouno 04-28-2016 09:33 AM

Now I've few time, so I reply quickly.


Quote:

Originally Posted by phenixia2003 (Post 5537830)
So, do you plan to include it, or do you want to stick with older version with Unified-ChangeLog.{txt,idx} ?

Yes, I will include it (so you can patch it directly)


Quote:

we have 2 choices here :
...
Which solution do you want to implement ?
I think that the second way (cleanup() modify) is the better way. There is already a similar use in that function:
Code:

    if [ "$CMD" == "update" ];then
      if [ "$ANSWER" != "Y" ] && [ "$ANSWER" != "y" ]; then
        touch $WORKDIR/pkglist
      fi
    fi



[quote]
Code:

$ slackpkg search mozilla-firefox

Looking for mozilla-firefox in package list. Please wait... DONE

No package name matches the pattern.

Other users has experienced the same problem with the original slackpkg, but I have no time now to see the problem.



Quote:

Code:

grep ${GREPOPTS} "^$DIR" $WORKDIR/pkglist|grep ${GREPOPTS} "/SLACKPKGPLUS_$SEARCHSTR/\|/$SEARCHSTR/\|/$SEARCHSTR \| [^ /]*$SEARCHSTR[^ /]* " > $PKGINFOS
This line fails because GREPOPTS include the option --word-regex and there are slashes in the regular-expression. However I don't understand the need of those slashes (in Bold RED). Can you explain this ?

Anyway, without these slash at line #920 :
Code:

grep ${GREPOPTS} "^$DIR" ${WORKDIR}/pkglist ${TMPDIR}/pkglist-pre|grep -E ${GREPOPTS} "SLACKPKGPLUS_$SEARCHSTR|$SEARCHSTR|$SEARCHSTR | [^ ]*$SEARCHSTR[^ ]* " > $PKGINFOS

Now I don't remember well, but with the '/' it searches only in url, without it, it search in repository too. I've to search the real reason, but I remember that there was a reason.


All times are GMT -5. The time now is 01:16 PM.