LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (https://www.linuxquestions.org/questions/slackware-14/)
-   -   slackpkg vs. third-party package repository (https://www.linuxquestions.org/questions/slackware-14/slackpkg-vs-third-party-package-repository-4175427364/)

phenixia2003 03-15-2013 12:01 PM

Hello,

Quote:

Originally Posted by Alien Bob (Post 4908161)
Of course, the slackpkg functionality that still only works with Slackware repositories is slackpkg's "install-new"

The code below manages the installation of any compat32 package (from multilib repository) which is not currently installed. In other words, this allows the installation of newly added package(s) to the layer-32, or the entire layer-32.

Code:

if [ "$CMD" == "install-new" ] ; then
 
  ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
 
  grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

  NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
 
  if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
    LIST=""
   
    for pkg in $NEWCOMPAT32PKGS ; do
      LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
    done
  fi
fi

Hope this help.

--
SeB

zerouno 03-19-2013 09:29 AM

Thanks for suggests.

Remain to solve the upgrade&upgrade-all problem.


If a package X is present both in slackware and in alienbob repository, and I install it from alienbob, next an upgrade-all should upgrade it only if change in alienbob repository and not in slackware.

I think that the suffix (1mt 1sl 1alien ... ) is sufficient to know the repository

phenixia2003 03-20-2013 06:13 AM

Hello,

Quote:

Originally Posted by zerouno (Post 4914516)
Thanks for suggests.

Remain to solve the upgrade&upgrade-all problem.


If a package X is present both in slackware and in alienbob repository, and I install it from alienbob, next an upgrade-all should upgrade it only if change in alienbob repository and not in slackware.

I think that the suffix (1mt 1sl 1alien ... ) is sufficient to know the repository

Well, I finally updated givepriority() so that, it now supports enhanced priority rules (priority given to package(s) from a given repository). This kind of priority uses the following syntax:

Code:

repository_name:pattern
For instance, to give priority to package "openjdk" from slacky repository, you will use the rule "slackpkgplus_slacky:openjdk".

Code:

  # Found packages in repository.
  # This function selects the package from the higher priority
  # repository directories.
  #
  # This Modified version supports enhanced priority rule (priority
  # given to package(s) from a given repository). This kind of priority
  # uses the following syntax :
  #
  #  repository_name:pattern
  #
  #
function givepriority {
        local DIR
        local ARGUMENT=$1
        local PKGDATA
        local CPRIORITY
        local DIR
        local PKG

        unset NAME
        unset FULLNAME
        unset PKGDATA
       
        for CPRIORITY in ${PRIORITY[@]} ; do
                [ "$PKGDATA" ] && break
               
                if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                  DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
                  PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
               
                  if echo "$ARGUMENT" | grep -q "$PAT" ; then
                    PKGDATA=( $(grep "^${DIR} ${PAT} " ${TMPDIR}/pkglist) )
                  fi
                else       
                  PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                fi
                 
                if [ "$PKGDATA" ]; then
                  NAME=${PKGDATA[1]}
                  FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
                fi
        done
}

So, with this change, the code in my previous post for the extension that I suggested (ie slackpkg install [repository-name:]package-name) must be changed as below :


Code:

if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

        NEWINPUTLIST=""

        for pref in $INPUTLIST ; do
                if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                        repository=$(echo "$pref" | cut -f1 -d":")
                        package=$(echo "$pref" | cut -f2- -d":")

                        PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
                else
                        package=$pref
                fi
               
                NEWINPUTLIST="$NEWINPUTLIST $package"
        done

        INPUTLIST=$NEWINPUTLIST       
fi

Now, about the upgrade/upgrade-all problem. Since givepriority() now supports enhanced priorities, I suggest to allow the user to defines prioritary packages in a variable declared into slackpkgplus.conf. These priorities would be merged with the variable PRIORITY at slackpkg+ init time.

For instance if the user want apache-ant from slacky, and openjdk from alienbob, it will defines this inside slackpkgplus.conf :

Code:

PKGS_PRIORITY=( slacky:apache-ant alienbob:openjdk )
The code to merge these priorities with those defined by PRIORITY will be as below :

Code:

  for pp in ${PKGS_PRIORITY[*]} ; do
    repository=$(echo "$pp" | cut -f1 -d":")
    package=$(echo "$pp" | cut -f2- -d":")
   
    if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
      PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
    fi
  done

With this, when the user will issue the commands :
Code:

$ slackpkg install apache-ant
$ slackpkg install openjdk

slackpkg+ will install the apache-ant from slacky and openjdk from alienbob.

And, when the user will issue the commands :
Code:

$ slackpkg upgrade apache-ant
$ slackpkg upgrade openjdk
$ slackpkg upgrade-all

slackpkg+ will get the latest apache-ant from slacky repository, and the latest openjdk from alienbob repository.

Note, the user can always install/upgrade another version using the extension I suggest. For instance, the command :

Code:

$ slackpkg install slacky:openjdk
$ slackpkg upgrade slacky:openjdk

Will imply install/upgrade of openjdk using the version from slacky repository.

However, since PKGS_PRIORITY includes "alienbob:openjdk", the commands :

Code:

$ slackpkg upgrade openjdk
$ slackpkg upgrade-all

Will imply upgrade of openjdk using the version from alienbob repository.

Well, this starts to be a bit complex, so, to summarize you will find the full code (and a version of slackpkgplus.conf that includes the variable PKGS_PRIORITY) below :

Code:


declare -A MIRRORPLUS
if [ -e /etc/slackpkg/slackpkgplus.conf ];then
  . /etc/slackpkg/slackpkgplus.conf
fi

if [ "$SLACKPKGPLUS" = "on" ];then
  # If CHECKGPG is "on", the system will FAIL the GPG signature of extra repository
  # Use MD5 check instead
  CHECKGPG=off

  REPOPLUS=${!MIRRORPLUS[*]}
  PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
 
    # -- merge priorities from PKGS_PRIORITY with PRIORITY ...
 
  for pp in ${PKGS_PRIORITY[*]} ; do
    repository=$(echo "$pp" | cut -f1 -d":")
    package=$(echo "$pp" | cut -f2- -d":")
   
    if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
      PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
    fi
  done
 
  function getfile(){
    local URLFILE
    URLFILE=$1

    if echo $URLFILE|grep -q /slackpkgplus_;then
      PREPO=$(echo $URLFILE|sed -r 's#^.*/slackpkgplus_([^/]+)/.*$#\1#')
      URLFILE=$(echo $URLFILE|sed "s#^.*/slackpkgplus_$PREPO/#${MIRRORPLUS[$PREPO]}#")
    fi
   
    $DOWNLOADER $2 $URLFILE
    if [ $(basename $1) = "CHECKSUMS.md5" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}CHECKSUMS.md5
        egrep -e ^[a-f0-9]{32} $2-tmp|sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> $2
      done
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}ChangeLog.txt
        echo $PREPO $(md5sum $2-tmp|awk '{print $1}') >>$2
        rm $2-tmp
      done
    fi

  }

    # Found packages in repository.
    # This function selects the package from the higher priority
    # repository directories.
    #
    # This Modified version supports enhanced priority rule (priority
    # given to package(s) from a given repository). This kind of priority
    # uses the following syntax :
    #
    #  repository_name:pattern
    #
    #
  function givepriority {
          local DIR
          local ARGUMENT=$1
          local PKGDATA
          local CPRIORITY
          local DIR
          local PKG

          unset NAME
          unset FULLNAME
          unset PKGDATA
         
          for CPRIORITY in ${PRIORITY[@]} ; do
                  [ "$PKGDATA" ] && break
                 
                  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
                    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
                 
                    if echo "$ARGUMENT" | grep -q "$PAT" ; then
                      PKGDATA=( $(grep "^${DIR} ${PAT} " ${TMPDIR}/pkglist) )
                    fi
                  else       
                    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                  fi
                   
                  if [ "$PKGDATA" ]; then
                    NAME=${PKGDATA[1]}
                    FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
                  fi
          done
  }

  if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

          NEWINPUTLIST=""

          for pref in $INPUTLIST ; do
                  if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                          repository=$(echo "$pref" | cut -f1 -d":")
                          package=$(echo "$pref" | cut -f2- -d":")

                          PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
                  else
                          package=$pref
                  fi
                 
                  NEWINPUTLIST="$NEWINPUTLIST $package"
          done

          INPUTLIST=$NEWINPUTLIST
         
#    echo "PRIORITY  set to ${PRIORITY[*]}"
#    read userkey
  fi

  if [ "$CMD" == "install-new" ] ; then
    ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
   
    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

    NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
   
    if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
      LIST=""
     
      for pkg in $NEWCOMPAT32PKGS ; do
        LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
      done
    fi
  fi
 
fi

Here is the slackpkgplus.conf I used :

Code:

SLACKPKGPLUS=on

#MIRRORPLUS['slacky']=http://darkstar.ist.utl.pt/slackware/addon/slacky/slackware64-14.0/
MIRRORPLUS['slacky']=repository.slacky.eu/slackware64-14.0/
MIRRORPLUS['alienbob']=http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/
MIRRORPLUS['multilib']=http://taper.alienbase.nl/mirrors/people/alien/multilib/14.0/

PKGS_PRIORITY=( slacky:apache-ant alienbob:openjdk )

Hope this help.


--
SeB

phenixia2003 03-22-2013 11:01 AM

Hello,

I slightly modified the function givepriority() so that it is now possible to use regexp in enhanced priority rules. With that, you can give priority for an entire repository through the variable PKGS_PRIORITY. For instance, to give priority to the multilib repository, you simply have to add this in slackpkgplus.conf :

Code:

PKGS_PRIORITY=( multilib:.* )
I also changed the code that merges the content of the variables PKGS_PRIORITY and PRIORITY. This was needed to preserve the priority orders given in PKGS_PRIORITY. Indeed, my previous code was wrong and the priorities were reversed.

Here is the full code (changes in bold) :

Code:


declare -A MIRRORPLUS
if [ -e /etc/slackpkg/slackpkgplus.conf ];then
  . /etc/slackpkg/slackpkgplus.conf
fi

if [ "$SLACKPKGPLUS" = "on" ];then
  # If CHECKGPG is "on", the system will FAIL the GPG signature of extra repository
  # Use MD5 check instead
  CHECKGPG=off

  REPOPLUS=${!MIRRORPLUS[*]}
  PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
 

    # -- merge priorities from PKGS_PRIORITY with PRIORITY, as needed ...
 
  if [ ! -z "$PKGS_PRIORITY" ] ; then
    PREFIX=""
   
    for pp in ${PKGS_PRIORITY[*]} ; do
      repository=$(echo "$pp" | cut -f1 -d":")
      package=$(echo "$pp" | cut -f2- -d":")
   
      if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
        if [ -z "$PREFIX" ] ; then
          PREFIX=( slackpkgplus_${repository}:$package )
        else
          PREFIX=( ${PREFIX[*]} slackpkgplus_${repository}:$package )
        fi
      fi
    done
   
    [ ! -z "$PREFIX" ] && PRIORITY=( ${PREFIX[*]} ${PRIORITY[*]} )
  fi

 
  function getfile(){
    local URLFILE
    URLFILE=$1

    if echo $URLFILE|grep -q /slackpkgplus_;then
      PREPO=$(echo $URLFILE|sed -r 's#^.*/slackpkgplus_([^/]+)/.*$#\1#')
      URLFILE=$(echo $URLFILE|sed "s#^.*/slackpkgplus_$PREPO/#${MIRRORPLUS[$PREPO]}#")
    fi
   
    $DOWNLOADER $2 $URLFILE
    if [ $(basename $1) = "CHECKSUMS.md5" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}CHECKSUMS.md5
        egrep -e ^[a-f0-9]{32} $2-tmp|sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> $2
      done
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}ChangeLog.txt
        echo $PREPO $(md5sum $2-tmp|awk '{print $1}') >>$2
        rm $2-tmp
      done
    fi

  }


    # Found packages in repository.
    # This function selects the package from the higher priority
    # repository directories.
    #
    # This Modified version supports enhanced priority rule (priority
    # given to package(s) from a given repository). This kind of priority
    # uses the following syntax :
    #
    #  repository_name:pattern
    #
    #
  function givepriority {
          local DIR
          local ARGUMENT=$1
          local PKGDATA
          local CPRIORITY
          local DIR
          local PKG

          unset NAME
          unset FULLNAME
          unset PKGDATA
         
          for CPRIORITY in ${PRIORITY[@]} ; do
                  [ "$PKGDATA" ] && break
                 
                  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:]" ; then
                    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
                    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
                 

                    if echo "$ARGUMENT" | grep -q "$PAT" ; then
                      PKGDATA=( $(grep "^${DIR} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                       
                    fi
                  else       
                    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                  fi
                   
                  if [ "$PKGDATA" ]; then
                    NAME=${PKGDATA[1]}
                    FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
                  fi
          done
  }

  if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

          NEWINPUTLIST=""

          for pref in $INPUTLIST ; do
                  if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                          repository=$(echo "$pref" | cut -f1 -d":")
                          package=$(echo "$pref" | cut -f2- -d":")

                          PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
                  else
                          package=$pref
                  fi
                 
                  NEWINPUTLIST="$NEWINPUTLIST $package"
          done

          INPUTLIST=$NEWINPUTLIST
         
  fi

  if [ "$CMD" == "install-new" ] ; then
    ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
   
    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

    NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
   
    if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
      LIST=""
     
      for pkg in $NEWCOMPAT32PKGS ; do
        LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
      done
    fi
  fi
 
fi

Cheers.

--
SeB

zerouno 04-23-2013 06:14 AM

I want to repackage slackpkg+

Eric, what do you think of the phenixia2003 ideas?

Alien Bob 04-23-2013 12:58 PM

I have not implemented phenixia2003's ideas here locally, yet. But his ideas sound good and would give slackpkg+ some needed flexibility.
One thing that I would implement but my ARM port does not leave me any time: to allow more than just the Slackware GPG key. It would be very welcome if slackpg+ would be able to use a separate GPG key for every repository.

Other than that, I have been using slackpkg+ on my desktop machine (with a lot of alien and SBo packages, and multilib) without a single issues (except the failure to use GPG checking).

Eric

zerouno 04-24-2013 09:31 AM

Eric, have you "modularized" the "Restricted-SlackBuilds" repository?

Alien Bob 04-24-2013 10:01 AM

Quote:

Originally Posted by zerouno (Post 4938008)
Eric, have you "modularized" the "Restricted-SlackBuilds" repository?

No I have not yet done so, and I do not have short-term plans in that direction.

Eric

phenixia2003 04-24-2013 10:07 AM

Hello,

Quote:

Originally Posted by Alien Bob (Post 4937394)
It would be very welcome if slackpg+ would be able to use a separate GPG key for every repository.

I had an idea about that. This is not perfect, but I guess that's a good starting point.

First, you have to add the array MIRRORPLUSKEY in slackpkgplus.conf. In this array you must declare the key of each repositories declared into MIRRORPLUS. When a repository does not provide a GPG-KEY, then you must use the keyword NULL. Here is the slackpkgplus.conf I use.

Code:

SLACKPKGPLUS=on

MIRRORPLUS['slacky']=repository.slacky.eu/slackware64-14.0/
MIRRORPLUS['alienbob']=http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/
MIRRORPLUS['multilib']=http://taper.alienbase.nl/mirrors/people/alien/multilib/14.0/

MIRRORPLUSKEY['slacky']="NULL"
MIRRORPLUSKEY['alienbob']="Eric Hameleers <alien@slackware.com>"
MIRRORPLUSKEY['multilib']="Eric Hameleers <alien@slackware.com>"

PKGS_PRIORITY=( multilib:.* slacky:apache-ant alienbob:openjdk )

With this, you must use the new slackpkgplus.sh below (changes since my last post are in bold):

Code:

declare -A MIRRORPLUS
declare -A MIRRORPLUSKEY

if [ -e /etc/slackpkg/slackpkgplus.conf ];then
  . /etc/slackpkg/slackpkgplus.conf
fi

if [ "$SLACKPKGPLUS" = "on" ];then

    # -- In this version CHECKGPG can be set to "on". However, CHECKMD5
    #    must be set to off, because the CHECKSUMS.md5 files on alien
    #    repositories does not includes entries for .asc files.
    #
  CHECKGPG=on
  CHECKMD5=off

 
  REPOPLUS=${!MIRRORPLUS[*]}
  PRIORITY=( ${PRIORITY[*]} slackpkgplus_$(echo $REPOPLUS|sed 's/ / slackpkgplus_/g') )
 

    # -- merge priorities from PKGS_PRIORITY with PRIORITY, as needed ...
 
  if [ ! -z "$PKGS_PRIORITY" ] ; then
    PREFIX=""
   
    for pp in ${PKGS_PRIORITY[*]} ; do
      repository=$(echo "$pp" | cut -f1 -d":")
      package=$(echo "$pp" | cut -f2- -d":")
   
      if [ ! -z "$repository" ] && [ ! -z "$package" ] ; then
        if [ -z "$PREFIX" ] ; then
          PREFIX=( slackpkgplus_${repository}:$package )
        else
          PREFIX=( ${PREFIX[*]} slackpkgplus_${repository}:$package )
        fi
      fi
    done
   
    [ ! -z "$PREFIX" ] && PRIORITY=( ${PREFIX[*]} ${PRIORITY[*]} )
  fi

 
  function getfile(){
    local URLFILE
    URLFILE=$1

    if echo $URLFILE|grep -q /slackpkgplus_;then
      PREPO=$(echo $URLFILE|sed -r 's#^.*/slackpkgplus_([^/]+)/.*$#\1#')
      URLFILE=$(echo $URLFILE|sed "s#^.*/slackpkgplus_$PREPO/#${MIRRORPLUS[$PREPO]}#")
    fi
   
    $DOWNLOADER $2 $URLFILE
    if [ $(basename $1) = "CHECKSUMS.md5" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}CHECKSUMS.md5
        egrep -e ^[a-f0-9]{32} $2-tmp|sed -r "s# \./# ./slackpkgplus_$PREPO/#" >> $2
      done
    fi
    if [ $(basename $1) = "ChangeLog.txt" ];then
      for PREPO in $REPOPLUS;do
        $DOWNLOADER $2-tmp ${MIRRORPLUS[${PREPO/slackpkgplus_}]}ChangeLog.txt
        echo $PREPO $(md5sum $2-tmp|awk '{print $1}') >>$2
        rm $2-tmp
      done
    fi

  }


    # Found packages in repository.
    # This function selects the package from the higher priority
    # repository directories.
    #
    # This Modified version supports enhanced priority rule (priority
    # given to package(s) from a given repository). This kind of priority
    # uses the following syntax :
    #
    #  repository_name:pattern
    #
    #
  function givepriority {
          local DIR
          local ARGUMENT=$1
          local PKGDATA
          local CPRIORITY
          local DIR
          local PKG

          unset NAME
          unset FULLNAME
          unset PKGDATA
         
          for CPRIORITY in ${PRIORITY[@]} ; do
                  [ "$PKGDATA" ] && break
                 
                  if echo "$CPRIORITY " | grep -q "[a-zA-Z0-9]\+[:]" ; then
                    DIR=$(echo "$CPRIORITY" | cut -f1 -d":")
                    PAT=$(echo "$CPRIORITY" | cut -f2- -d":")
                 

                    if echo "$ARGUMENT" | grep -q "$PAT" ; then
                      PKGDATA=( $(grep "^${DIR} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                       
                    fi
                  else       
                    PKGDATA=( $(grep "^${CPRIORITY} ${ARGUMENT} " ${TMPDIR}/pkglist) )
                  fi
                   
                  if [ "$PKGDATA" ]; then
                    NAME=${PKGDATA[1]}
                    FULLNAME=$(echo "${PKGDATA[5]}.${PKGDATA[7]}")
                  fi
          done
  }
 
        # -- Loads the GPG-KEY from the Slackware repository
        #
  function loadSlackwareGpgKey() {
        getfile ${SOURCE}GPG-KEY $TMPDIR/gpgkey
        gpg --yes --batch --delete-key "$SLACKKEY" &>/dev/null
        gpg --import $TMPDIR/gpgkey &>/dev/null && \
        echo -e "\t\t\tSlackware Linux Project's GPG key added"
  }
 
    # -- Loads the GPG-KEY from the external repository ($1)
    #
  function loadRepositoryGpgKey() {
        local l_repo=$1
        local l_url=${MIRRORPLUS[$l_repo]}
        local l_mirrorkey=${MIRRORPLUSKEY[$l_repo]}
       
        if [ "$l_mirrorkey" == "NULL" ] ; then
                echo ""
                echo "(WARNING) ************************************************"
                echo "(WARNING) *    No key for external repository $l_repo    *"
                echo "(WARNING) ************************************************"
                echo ""
                return
        fi
         
        getfile ${l_url}GPG-KEY $TMPDIR/gpgkey
       
        gpg --yes --batch --delete-key "$l_mirrorkey" &>/dev/null
        gpg --import $TMPDIR/gpgkey &>/dev/null && \
        echo -e "\t\t\t$l_repo GPG key added"
  }
 
  # -- This override updatefilelists() from original core-functions.sh
  #
  #
function updatefilelists()
{
        if checkchangelog ; then
                echo -e "\
\n\t\tNo changes in ChangeLog.txt between your last update and now.\n\
\t\tDo you really want to download all other files (y/N)? \c"
                answer
                if [ "$ANSWER" != "Y" ] && [ "$ANSWER" != "y" ]; then
                        cleanup
                fi
        fi
        echo
        cp ${TMPDIR}/ChangeLog.txt ${WORKDIR}/ChangeLog.txt

        #
        # Download MANIFEST, FILELIST.TXT and CHECKSUMS.md5
        #

        # That will be download MANIFEST.bz2 files
        #
        echo -e "\t\tList of all files"
        for i in ${PRIORITY[@]} ; do
       
           
            # -- IMPORTANT:
            #    Enhanced priority rules (ie: repository:name) must be
            #    ignored, otherwise we'll get some nasty error messages.
               
          if ! echo "$i" | grep -q "[a-zA-Z0-9]\+[:]" ; then

                getfile ${SOURCE}${i}/MANIFEST.bz2 $TMPDIR/${i}-MANIFEST.bz2 && \
                DIRS="$DIRS $i"
          fi
        done

        ISOK="1"
        echo -e "\t\tChecksums"
       
          # -- IMPORTANT:
          #      The version of getfile() for slackpkg+ concatenates the files
          #      CHECKSUMS.md5 from the selected repositories. Therefore, the
          #      code to check signature of this file must be removed.
          #
        getfile ${SOURCE}CHECKSUMS.md5 ${TMPDIR}/CHECKSUMS.md5               
       
#        getfile ${SOURCE}CHECKSUMS.md5.asc ${TMPDIR}/CHECKSUMS.md5.asc
        if ! [ -e "${TMPDIR}/CHECKSUMS.md5" ]; then
                echo -e "\
\n\t\tWARNING: Your mirror appears incomplete and is missing the\n\
\t\t        CHECKSUMS.md5 file. We recommend you change your mirror\n\
\t\t        so that package integrity can be verified against \n\
\t\t        CHECKSUMS.md5.\n"
                sleep 10
#        else
#                if [ "$CHECKGPG" = "on" ]; then
#                        ISOK=$(checkgpg ${TMPDIR}/CHECKSUMS.md5)
#                        if [ "$ISOK" = "0" ]; then
#                                read userkey
#                                rm $TMPDIR/CHECKSUMS.md5
#                                rm $TMPDIR/CHECKSUMS.md5.asc
#                                echo -e "\
#\n\t\tERROR: Verification of the  gpg signature on CHECKSUMS.md5\n\
#\t\t      failed! This could mean that the file is out of date\n\
#\t\t      or has been tampered with.\n"
#                                cleanup
#                        fi
#                elif [ "$SLACKKEY" != "" ]; then
#                        echo -e "\
#\n\t\tWARNING: Without CHECKGPG, we can't check if this file is\n\
#\t\t        signed by:\n\
#\n\t\t        $SLACKKEY.\n\
#\n\t\t        Enabling CHECKGPG is highly recommended for best\n\
#\t\t        security.\n"
#                                sleep 10
#                fi

        fi

        ISOK="1"
        echo -e "\t\tPackage List"
        getfile ${SOURCE}FILELIST.TXT ${TMPDIR}/FILELIST.TXT
        if [ "$CHECKMD5" = "on" ]; then
                CHECKSUMSFILE=${TMPDIR}/CHECKSUMS.md5
                ISOK=$(checkmd5 ${TMPDIR}/FILELIST.TXT)
        fi
        if [ "$ISOK" = "1" ]; then
                if ! [ -e $WORKDIR/LASTUPDATE ]; then
                        echo "742868196" > $WORKDIR/LASTUPDATE
                fi
                LASTUPDATE=$(cat $WORKDIR/LASTUPDATE)
                ACTUALDATE=$(date -d "$(head -1 $TMPDIR/FILELIST.TXT)" "+%s")
                if [ $ACTUALDATE -lt $LASTUPDATE ]; then
                        echo -e "\
\n\t\tFILELIST.TXT seems to be older than the last one.\n\
\t\tDo you really want to continue (y/N)? \c"
                        answer
                        if [ "$ANSWER" != "Y" ] && [ "$ANSWER" != "y" ]; then
                                cleanup
                        fi
                        echo
                fi
                echo $ACTUALDATE > $WORKDIR/LASTUPDATE
        else
                rm $TMPDIR/FILELIST.TXT
        fi
       
        if [ -e $TMPDIR/CHECKSUMS.md5 ]; then
                FILELIST="$TMPDIR/CHECKSUMS.md5"
        elif [ -e $TMPDIR/FILELIST.TXT ]; then
                if [ "$ISOK" = "0" ]; then
                        echo -e "\
\n\t\tERROR: CHECKSUMS.md5 signature doesn't match!\n\
\t\t      We strongly recommend that you change your mirror\n\
\t\t      to prevent security problems.\n"
                        cleanup
                fi
                sleep 10
                  FILELIST="$TMPDIR/FILELIST.TXT"
        else
                echo -e "\
\n\t\tERROR: No CHECKSUMS.md5 and no FILELIST.TXT.\n\
\t\t      We strongly recommend that you change your mirror\n\
\t\t      to prevent security problems.\n"
                cleanup
        fi

        # Download all PACKAGES.TXT files
        #
        echo -e "\t\tPackage descriptions"
        for i in $DIRS; do
                getfile ${SOURCE}${i}/PACKAGES.TXT $TMPDIR/${i}-PACKAGES.TXT
        done

        # Format FILELIST.TXT
        #
        echo -e "\tFormatting lists to slackpkg style..."
        echo -e "\t\tPackage List: using $( basename $FILELIST ) as source"
        grep "\.t[blxg]z$" $FILELIST| \
                awk -f /usr/libexec/slackpkg/pkglist.awk |\
                sed -e 's/^M//g' > ${TMPDIR}/pkglist
        cp ${TMPDIR}/pkglist ${WORKDIR}/pkglist               

        # Create the slackware tree under TEMP directory
        for i in $( cut -f7 -d\  ${WORKDIR}/pkglist | sort -u ) ; do
          if ! [ -d ${TEMP}/${i} ]; then
            mkdir -p ${TEMP}/${i}
          fi
        done

        # Format MANIFEST
        #
               
        # bunzip and concatenate all MANIFEST files
        #
        MANFILES=""
        for i in $DIRS; do
                echo "(DEBUG) calling bunzip2 with $TMPDIR/${i}-MANIFEST.bz2"
                read userkey
               
                if [ -s $TMPDIR/${i}-MANIFEST.bz2 ] ; then
                        bunzip2 -c $TMPDIR/${i}-MANIFEST.bz2 | awk -f /usr/libexec/slackpkg/filelist.awk | \
                                gzip > ${TMPDIR}/${i}-filelist.gz
                fi
        done
        cp ${TMPDIR}/*-filelist.gz ${WORKDIR}/

        if [ -r ${WORKDIR}/filelist.gz ]; then
                rm ${WORKDIR}/filelist.gz
                ln -s ${WORKDIR}/${MAIN}-filelist.gz ${WORKDIR}/filelist.gz
        fi

        # Concatenate PACKAGE.TXT files
        #
        echo -e "\t\tPackage descriptions"
        for i in $DIRS; do
                cat $TMPDIR/${i}-PACKAGES.TXT >> $TMPDIR/PACKAGES.TXT
        done
        cp $TMPDIR/PACKAGES.TXT ${WORKDIR}/PACKAGES.TXT

        if [ -e $TMPDIR/CHECKSUMS.md5 ]; then
                cp $TMPDIR/CHECKSUMS.md5 $WORKDIR/CHECKSUMS.md5 2>/dev/null
        fi

        if [ -e $TMPDIR/CHECKSUMS.md5.asc ]; then
                cp $TMPDIR/CHECKSUMS.md5.asc \
                        $WORKDIR/CHECKSUMS.md5.asc 2>/dev/null
        fi
}
 
 
  if [ "$CMD" == "install" ] || [ "$CMD" == "upgrade" ] ; then

          NEWINPUTLIST=""

          for pref in $INPUTLIST ; do
                  if echo "$pref" | grep -q "[a-zA-Z0-9]\+[:][a-zA-Z0-9]\+" ; then
                          repository=$(echo "$pref" | cut -f1 -d":")
                          package=$(echo "$pref" | cut -f2- -d":")

                          PRIORITY=( slackpkgplus_${repository}:$package ${PRIORITY[*]} )
                  else
                          package=$pref
                  fi
                 
                  NEWINPUTLIST="$NEWINPUTLIST $package"
          done

          INPUTLIST=$NEWINPUTLIST
         
  fi

  if [ "$CMD" == "install-new" ] ; then
    ls -1 /var/log/packages/*compat32 | rev | cut -f1 -d/ | cut -f4- -d- | rev | sort > $TMPDIR/installed-compat32-packages.lst
   
    grep "[[:digit:]]\+compat32[ ]" $WORKDIR/pkglist | cut -f2 -d" " | sort > $TMPDIR/available-compat32-packages.lst

    NEWCOMPAT32PKGS=$(comm -3 $TMPDIR/installed-compat32-packages.lst  $TMPDIR/available-compat32-packages.lst)
   
    if [ ! -z "$NEWCOMPAT32PKGS" ] ; then
      LIST=""
     
      for pkg in $NEWCOMPAT32PKGS ; do
        LIST="$LIST $(grep " ${pkg} " $WORKDIR/pkglist | cut -f6,8 -d" " --output-delimiter=".")"
      done
    fi
  fi
 
    # -- New code to handle the command "update"...
   
  if [ "$CMD" == "update" ] ; then
        if [ "$UPARG" = "gpg" ] || [ "$GPGFIRSTTIME" = "0" ]; then
                #
                # Creates .gnupg directory if doesn't exist
                # without this dir, gpg got an error.
                #
                if ! [ -e ~/.gnupg ]; then
                        mkdir ~/.gnupg
                fi
       
                loadSlackwareGpgKey

                for PREPO in $REPOPLUS; do
                        loadRepositoryGpgKey $PREPO
                done
               
                if [ "$UPARG" = "gpg" ]; then
                        cleanup
                fi
        fi
       
        echo "Updating the package lists..."
        updatefilelists
       
        cleanup
        exit
  fi

 
fi

Important:
  • I ran into some troubles because the CHECKSUMS.md5 on repositories "sbrepos" and "multilib" does not includes entries for .asc files. To workaround this, I set "CHECKMD5" to off (but "CHECKGPG" is on).
  • The getfile() in slackpkg+ concatenates the files CHECKSUMS.md5 from the selected repositories. Therefore, the code to check signature - in updatefilelists() - has been removed (colored in red in the code above). This should/could be moved into getfile().
  • I fixed a problem caused by the "enhanced priority rules" in updatefilelists() that leads to some nasty error messages (The correction is colored in green in the code above)
Now, with this new version, if you install apache-ant from alienbob repository you will get this :

Code:

slackpkg -batch=on -default_answer=y install alienbob:apache-ant

Looking for apache-ant in package list. Please wait... DONE

apache-ant-1.8.2-noarch-1alien.tgz

Total package(s): 1

Do you wish to install selected packages (Y/n)? y

Package: apache-ant-1.8.2-noarch-1alien.tgz
--2013-04-24 16:39:10--  http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz
Resolving taper.alienbase.nl (taper.alienbase.nl)... 98.158.153.254
Connecting to taper.alienbase.nl (taper.alienbase.nl)|98.158.153.254|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8382749 (8.0M) [application/x-gzip]
Saving to: '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz'

100%[================================================>] 8,382,749    192KB/s  in 55s   

2013-04-24 16:40:11 (149 KB/s) - '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz' saved [8382749/8382749]

--2013-04-24 16:40:11--  http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz.asc
Resolving taper.alienbase.nl (taper.alienbase.nl)... 98.158.153.254
Connecting to taper.alienbase.nl (taper.alienbase.nl)|98.158.153.254|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 198 [text/plain]
Saving to: '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz.asc'

100%[================================================>] 198        --.-K/s  in 0s     

2013-04-24 16:40:11 (615 KB/s) - '/var/cache/packages/./slackpkgplus_alienbob/apache-ant/apache-ant-1.8.2-noarch-1alien.tgz.asc' saved [198/198]

        Installing apache-ant-1.8.2-noarch-1alien...
Verifying package apache-ant-1.8.2-noarch-1alien.tgz.
Installing package apache-ant-1.8.2-noarch-1alien.tgz:
PACKAGE DESCRIPTION:
# Apache Ant (Java-based build tool analogous to Make)
#
# Ant is a platform-independent build tool for java.
#
# Ant is extended using Java classes.  The configuration files are
# XML- based, calling out a target tree where tasks get executed.
# Each task is run by an object that implements a Task interface.
#
# Homepage: http://ant.apache.org/
#
Executing install script for apache-ant-1.8.2-noarch-1alien.tgz.
Package apache-ant-1.8.2-noarch-1alien.tgz installed.

Hope this helps.


Greetings.

--
SeB

Alien Bob 04-24-2013 03:19 PM

Quote:

Originally Posted by phenixia2003 (Post 4938033)
I ran into some troubles because the CHECKSUMS.md5 on repositories "sbrepos" and "multilib" does not includes entries for .asc files. To workaround this, I set "CHECKMD5" to off (but "CHECKGPG" is on).

I will fix that ASAP. There will be entries in CHECKSUMS.md5 for all meta files, including the .asc file. I am testing an updated gen_repos_files.sh script at the moment.

Eric

Alien Bob 04-24-2013 04:54 PM

Quote:

Originally Posted by Alien Bob (Post 4938027)
No I have not yet done so, and I do not have short-term plans in that direction.

Well I changed my mind, and the restricted_slackbuilds repository now has a 'modularized' version as well. See http://taper.alienbase.nl/mirrors/pe...icted_sbrepos/

Quote:

Originally Posted by Alien Bob (Post 4938183)
I will fix that ASAP. There will be entries in CHECKSUMS.md5 for all meta files, including the .asc file. I am testing an updated gen_repos_files.sh script at the moment.

Fixed. Both the regular and the restricted slackbuilds repositories have been updated with CHECKSUMS.md5 files, that contain md5sums for all package-related files (like the .asc files).

Eric

phenixia2003 04-25-2013 05:21 AM

2 Attachment(s)
Hello,

Quote:

Originally Posted by Alien Bob (Post 4938225)
Fixed. Both the regular and the restricted slackbuilds repositories have been updated with CHECKSUMS.md5 files, that contain md5sums for all package-related files (like the .asc files).

Great, so I removed the line "CHECKMD5=off" in my code and all is ok. However,I got another trouble because there's no file ChangeLog.txt at http://taper.alienbase.nl/mirrors/pe...os/14.0/x86_64.

I also made some changes to get something better.

First, I think that's not a good idea to have one array for the urls of repositories, and another for the keys.

So, I choose to merge these data into the array MIRRORPLUS. Now, when you declare a new repository you will have to supply the key through a field "?key=<key>" right after the url.

For instance, now my slackpkgplus.conf is as below :
Code:

SLACKPKGPLUS=on

  # MIRRORPLUS allow you to declare external repositories. Its syntax is as below:
  #
  #  MIRRORPLUS['<repository-name>']="<repository-url>?key=<repository-key>"
  #
  # When a repository has no GPG-KEY, the field "?key=<repository-key>" can
  # be undefined or must be set to "?key=null"
  # 
MIRRORPLUS['alienbob']="http://taper.alienbase.nl/mirrors/people/alien/sbrepos/14.0/x86_64/?key=Eric Hameleers <alien@slackware.com>"
MIRRORPLUS['multilib']="http://taper.alienbase.nl/mirrors/people/alien/multilib/14.0/?key=Eric Hameleers <alien@slackware.com>"
MIRRORPLUS['slacky']="repository.slacky.eu/slackware64-14.0/?key=null"
#MIRRORPLUS['robby']="http://rlworkman.net/pkgs/14.0/?key=Robby Workman <rworkman@slackware.com>"

PKGS_PRIORITY=( multilib:.* slacky:apache-ant alienbob:openjdk )

Note that Robby's repository works but only the i486 packages can be installed because the files required by slackpkg (CHECKSUMS.md5,...) are not at architecture level.

At runtime, slackpkgplus.sh parses the array MIRRORPLUS to extract the urls and keys of the defined repositories.

When a repository has no key, and you run "slackpkg update gpg" with CHECKGPG turned on, then a warning is printed on the standard output :
Code:

(WARNING) ************************************************************************************
(WARNING) * There is no GPG-KEY for the external repository <reponame> while CHECKGPG is on. *
(WARNING) * Therefore, any attempt to install/upgrade a package from this repository        *
(WARNING) * will fail.                                                                      *
(WARNING) ************************************************************************************

You will find a copy of slackpkgplus.conf and slackpkgplus.sh in attachment.

Greetings.

--
SeB

TobiSGD 04-25-2013 07:26 AM

phenixia2003: I have copied your script (of course with removing the .txt) to /usr/libexec/slackpkg/functions.d and the config file to /etc/slackpkg/, then I have edited the config file so that it points to my local mirrors and commented out the repos I don't use. Is there anything else I have to do to make that work?
Code:

slackpkg update
is just ignoring the additional repos, no error messages also.

phenixia2003 04-25-2013 08:54 AM

2 Attachment(s)
Hello,

Quote:

Originally Posted by TobiSGD (Post 4938596)
phenixia2003: I have copied your script (of course with removing the .txt) to /usr/libexec/slackpkg/functions.d and the config file to /etc/slackpkg/, then I have edited the config file so that it points to my local mirrors and commented out the repos I don't use. Is there anything else I have to do to make that work?
Code:

slackpkg update
is just ignoring the additional repos, no error messages also.

slackpkg+ was not supporting local mirrors. This is fixed in the new version in attachment (do not forget to remove the .2.txt).

To declare a local mirror In slackpkgplus.conf, you must prepend its url by "file:". For instance, if you have a local mirror in /home/myname/myrepository/14.0", you must declare it as below :

Code:



MIRRORPLUS['localrepo']="file:/home/myname/myrepository/14.0?key=your key"

if your local mirror has no gpg-key, do not put the field "?key=...", or set this to "?key=null". But keep in mind that in this case, you must set CHECKGPG to "off", or run slackpkg with option -checkgpg=off.

I tested this with a local copy of Eric's multilib repository (with -checkmd5=off since there's no md5 for the .asc files on this repository).

If you run into any trouble, let me know.

Greetings.
--
SeB

Alien Bob 04-25-2013 09:20 AM

Quote:

Originally Posted by phenixia2003 (Post 4938658)
slackpkg+ was not supporting local mirrors. This is fixed in the new version in attachment (do not forget to remove the .2.txt).

To declare a local mirror In slackpkgplus.conf, you must prepend its url by "file:". For instance, if you have a local mirror in /home/myname/myrepository/14.0", you must declare it as below :

Code:



MIRRORPLUS['localrepo']="file:/home/myname/myrepository/14.0?key=your key"


Slackpkg uses a "file://" syntax for that, please try to follow the original syntax where possible. Your solution only uses one slash.

Quote:

Eric's multilib repository (with -checkmd5=off since there's no md5 for the .asc files on this repository).
I will re-generate the multilib repository files with my new script, so that the CHECKSUMS.md5 will have entries for the .asc files.

Eric


All times are GMT -5. The time now is 11:25 AM.