[SOLVED] bash sort strange behaviour...

masavini · 06-21-2012, 09:16 AM

hi,
can anyone tell me why this command gives this output?

Code:

$ echo -e "bubu\nZZ\naa\n1" | sort
1
ZZ
aa
bubu

shouldn't it be:

Code:

$ echo -e "bubu\nZZ\naa\n1" | sort
1
aa
bubu
ZZ

?

thanks

druuna · 06-21-2012, 09:34 AM

As mentioned in the sort man page the locale setting effects sort behaviour.

This recreates your original output on my machine:

Code:

$ export LANG=
$ export LC_ALL=POSIX
$ echo -e "bubu\nZZ\naa\n1" | sort
1
ZZ
aa
bubu

This creates the wanted output:

Code:

$ export LANG=en_US
$ export LC_ALL=en_US
$ echo -e "bubu\nZZ\naa\n1" | sort
1
aa
bubu
ZZ

Have a look (in a newly started terminal) what the current settings are by issuing: locale
Installed and usable locales can be found by issuing: locale -a

masavini · 06-21-2012, 10:09 AM

this is my locale:

Code:

$ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=C

my ~/.bashrc contains this line, added to fix this:

Code:

export LC_ALL="C"

... what a mess!

druuna · 06-21-2012, 10:19 AM

Hi,

I would leave the LANG and LC_* settings as set by your distro to make sure that your countries specifics are kept (special language specific characters/time/date/currency symbol/etc). Looking at your first entry you will probably end up with something like this:

Code:

LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

BTW: The above will give the output you expected (not necessarily the correct output, which would be what you originally got).

You can set a specific locale (if needed) at the start of a script or in your terminal. If something is needed regularly you can alias a specific command if needed. I, for example have this alias:
alias ls='LC_COLLATE=POSIX ls --color=auto'
The above sets the sort order of ls to what I prefer.

masavini · 06-23-2012, 05:39 AM

perfect, thank you so much...
solved with:

Code:

LC_ALL=en_US.UTF-8
sort file
LC_ALL=C

Nominal Animal · 06-23-2012, 03:40 PM

Quote:

Originally Posted by masavini

my ~/.bashrc contains this line, added to fix this:

Code:

export LC_ALL="C"

I'd recommend you either replace that line with

Code:

alias grep="LANG=C LC_ALL=C grep"

or remove the line and create file /usr/local/bin/grep containing

Code:

#!/bin/bash
export LANG=C LC_ALL=C
exec /usr/bin/grep "$@"

The alias only works in interactive shells, but the wrapper script works everywhere. You see, because /usr/local/bin is listed before /usr/bin in PATH, the wrapper script gets selected over actual /usr/bin/grep. All it does is set the two environment variables and then replace itself with grep, using the original command-line parameters.

Quote:

Originally Posted by masavini

solved with:

Code:

LC_ALL=en_US.UTF-8
sort file
LC_ALL=C

If you use

Code:

LC_ALL=C sort ...

then LC_ALL gets set only for the sort command and nothing else. Always put it just before the sort and not at the start of the line.

If you run multiple commands you wish to use the C/POSIX locale, you'd better use

Code:

OLD_LC_ALL="$LC_ALL"
export LC_ALL=en_US.UTF-8
commands ... 
export LC_ALL="$OLD_LC_ALL"

instead. The export bit makes sure the value is seen by external commands too, by making sure it is exported to the environment. By saving and restoring the original value your script will behave correctly even if you happen to change your locale later on.