slackpkg upgrade-all: comm: file 2 is not in sorted order

hba · 07-11-2015, 09:35 PM

Hi, after executing slackpkg upgrade-all i'm getting:

Code:

Checking local integrity... DONE
Looking for packages to upgrade. Please wait... comm: file 2 is not in sorted order
DONE

No packages match the pattern for upgrade. Try:

	/usr/sbin/slackpkg install|reinstall

I found the message related to the next lines in /usr/libexec/slackpkg/core-functions.sh (while debugging with set -x):

Code:

 643                 upgrade-all)
 644                         listpkgname
 645                         for i in $(comm -1 -2 ${TMPDIR}/lpkg ${TMPDIR}/dpkg | \
 646                                    comm -1 -2 - ${TMPDIR}/spkg) ; do
 647

My first workaround was to use the --nocheck-order option (from comm manpage) which removes the message but it seems that the real problem comes from the function listpkgname:

Code:

 526 # Creates files with mirror package names (spkg), local package
 527 # names (lpkg) and packages unique to one or other file (dpkg)
 528 #
 529 function listpkgname() {
 530         cut -f2 -d\  ${TMPDIR}/pkglist | sort > ${TMPDIR}/spkg
 531         cut -f2 -d\  ${TMPDIR}/tmplist | sort > ${TMPDIR}/lpkg
 532         cat ${TMPDIR}/pkglist ${TMPDIR}/tmplist | \
 533                 cut -f2-6 -d\ |sort | uniq -u | \
 534                 cut -f1 -d\  | uniq > ${TMPDIR}/dpkg
 535 }

The content of pkglist and tmplist (lines #532 to #534) are processed two times, first time is cut-sort-uniq and the second time cut-uniq but never sort, so after the next modification the message also goes away:

Code:

--- core-functions.sh	2015-07-11 20:22:27.273107692 -0500
+++ core-functions.sh.new	2015-07-11 20:07:15.418151457 -0500
@@ -531,7 +531,7 @@ function listpkgname() {
 	cut -f2 -d\  ${TMPDIR}/tmplist | sort > ${TMPDIR}/lpkg
 	cat ${TMPDIR}/pkglist ${TMPDIR}/tmplist | \
 		cut -f2-6 -d\ |sort | uniq -u | \
-		cut -f1 -d\  | uniq > ${TMPDIR}/dpkg
+		cut -f1 -d\  |sort | uniq > ${TMPDIR}/dpkg
 }
 
 function applyblacklist() {

What do you think? Any idea what could be the correct solution?

- Cheers

allend · 07-11-2015, 09:47 PM

Perhaps this is a locale issue.
From /etc/profile.d/lang.sh

Code:

# One side effect of the newer locales is that the sort order
# is no longer according to ASCII values, so the sort order will
# change in many places.  Since this isn't usually expected and
# can break scripts, we'll stick with traditional ASCII sorting.
# If you'd prefer the sort algorithm that goes with your $LANG
# setting, comment this out.
export LC_COLLATE=C

What is the output of

Code:

set | grep LC_COLLATE

?

hba · 07-11-2015, 10:09 PM

Well, i forgot to put mi locales in my post (i thought their could be involved in some way), so here they are:

Code:

LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

And BTW, i'm in -current ^_^

allend · 07-11-2015, 10:13 PM

So, can you replicate the problem when using 'LC_COLLATE=C'?

hba · 07-11-2015, 10:17 PM

It works with LC_COLLATE=C

Didier Spaier · 07-11-2015, 10:41 PM

@All encountering similar issues, please read::

in the the POSIX specification, the chapter 8.2 Internationalization Variables
in the GNU Gettext manual, 2.3 Setting the Locale through Environment Variables aka Setting the Posix Locale.

Some hints or recommendations:

The LANGUAGE variable, that comes handy to set a priority list of languages, is a GNU extension and as such non portable.
Avoid to set LC_ALL system wide, as that overrides all other settings.
For internationalization, preferably only set LANG. That's generally enough to get messages in your preferred language.
Better keep LC_COLLATE set to C as pointed out by allend, when sorting is involved. Additionally this makes sorting faster in scripts probably because then we sort only one byte characters. Only exception: if you need to sort messages written using characters beyond the ASCII set, on a case by case basis.

Additionally, for folks using character classes in grep and sed, 7.3.1 LC_CTYPE is a must read.

Generally, the effect of setting LC_something to a specific locale stated in /usr/share/i18n/locale/<locale name> as a list of Unicode code points (there is also one for the POSIX locale). The Unicode code point of each character can be found e.g. in this handy table.

hba · 07-11-2015, 11:05 PM

A better solution would be to fix slackpkg, so it can work even in an environment where the user wants to use other value for LC_COLLATE ;-)

From /etc/profile.d/lang.sh:

Code:

# If you'd prefer the sort algorithm that goes with your $LANG
# setting, comment this out.
export LC_COLLATE=C

allend · 07-11-2015, 11:39 PM

Your fix is unnecessary in standard Slackware. If you want to use a custom environment, then you will need to maintain that on your setup, which, as you have demonstrated, is easy to do in this instance. Just be aware that your fix will be overwritten if the slackpkg package is upgraded or reinstalled.