LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (http://www.linuxquestions.org/questions/slackware-14/)
-   -   Unicode Problems? (http://www.linuxquestions.org/questions/slackware-14/unicode-problems-4175467019/)

fl0 06-22-2013 02:41 PM

Unicode Problems?
 
Hi,

i use slackware64-current, and have activated unicode,

i set the parameter in /etc/profile.d/lang.sh and in lilo.conf

Code:

export LANG=en_US.UTF-8
Code:

append=" vt.default_utf8=1 resume=/dev/cryptvg/swap"
if i use vim and try german umlaute (äöü..), it works, but if i try this in the terminal (urxvt), the umlaut is displayed but in the error its not.
Code:

[trav ~]$ ä
bash: $'\303\244': command not found
[trav ~]$

a touch ä is working

ideas? or is this the normal behavior?

STDOUBT 06-22-2013 04:15 PM

I have the same set-up as you (except I don't know what "resume=/dev/cryptvg/swap" is supposed to do, even though I have luks+lvm going...)

I use urxvt, and have the same result with the error output. It occurred to me that this looks like a "command interpreter" issue, so I installed dash from slackbuilds, and "switching" to dash from within urxvt I get:
Code:

$ ä                     
dash: ä: not found
$

so I'm thinking it's a bash limitation. I take it dash is supposed to be fully POSIX, whereas bash may not be?
Here's another fun one:
Code:

$ イロハニホヘト
dash: イロハニホヘト: not found
$

Maybe the POSIX thing is the explanation, but I'm now at the limit of my pay scale.

Didier Spaier 06-22-2013 04:24 PM

Not sure I understand what you mean.

I just built and installed an rxvt-unicode (AKA urxvt) package from slackbuilds.org and when using the basic method for applying ISO 14755 standard (press and hold both Ctrl and Shfi then enter the four hex digits of UTF-8 code), characters are properly displayed.

FYI, to find all hex digits UTF-8 codes of letters with a diaeresis (official name of the umlaute) I type:
Code:

zgrep DIAERESIS /usr/share/i18n/charmaps/UTF-8.gz
EDIT Reading STDOUBT's answer I see now that I didn't understand your question. So yes, this could be a limitation of bash.

Oh, and bash can be put in POSIX mode if invoked with the --posix option. According to 'man bash': "Bash can be configured to be POSIX-conformant by default".

But that doesn't change the output in this case.

mancha 06-22-2013 10:18 PM

I rescued and refactored a pearl from Roman Rakus at Red Hat. It applies cleanly to Bash 4.2
and on my systems modifies the behavior you noted.

Feel free to get bash-4.2-widechars.diff and stress test it (other testers welcome).

If all goes well, Pat might consider its inclusion.

--mancha

Code:

mancha@infinity:~$ müßiggänger
-bash: müßiggänger: command not found


zakame 06-22-2013 10:26 PM

Yep, looks like a bash regression. dash, and zsh displays it fine. ash also fails it, not even accepting characters in the command line:

Code:

zakame@jazz:~% /bin/ash
%n@%m:%~%# miggnger        # should be müßiggänger here
miggnger: not found
%n@%m:%~%#


Martinus2u 06-23-2013 02:45 AM

strange, i'm on -current (almost on the latest commit) and i get

Code:

mar@arnold$ echo "echo Müßiggänger" > ä
mar@arnold$ ll
total 4
drwxr-xr-x  2 mar  users  60 Jun 23 09:43 ./
drwxrwxrwt 20 root root  440 Jun 23 09:42 ../
-rw-r--r--  1 mar  users  20 Jun 23 09:43 ä
mar@arnold$ ä
bash: ./ä: Permission denied
mar@arnold$ chmod 755 ä
mar@arnold$ ä
Müßiggänger

Edit: using Terminal

mancha 06-23-2013 03:28 AM

Quote:

Originally Posted by Martinus2u (Post 4976925)
strange, i'm on -current (almost on the latest commit) and i get

Code:

mar@arnold$ echo "echo Müßiggänger" > ä
mar@arnold$ ll
total 4
drwxr-xr-x  2 mar  users  60 Jun 23 09:43 ./
drwxrwxrwt 20 root root  440 Jun 23 09:42 ../
-rw-r--r--  1 mar  users  20 Jun 23 09:43 ä
mar@arnold$ ä
bash: ./ä: Permission denied
mar@arnold$ chmod 755 ä
mar@arnold$ ä
Müßiggänger

Edit: using Terminal

That isn't what is being reported/corrected in this thread.

fl0 06-23-2013 03:29 AM

Quote:

Originally Posted by mancha+ (Post 4976869)
I rescued and refactored a pearl from Roman Rakus at Red Hat. It applies cleanly to Bash 4.2
and on my systems modifies the behavior you noted.

Feel free to get bash-4.2-widechars.diff and stress test it (other testers welcome).

If all goes well, Pat might consider its inclusion.

--mancha

Code:

mancha@infinity:~$ müßiggänger
-bash: müßiggänger: command not found


Hi @mancha,

thx for the patch, it applies also cleanly and solves the problem, no negative effects for now
Code:

[trav bash]$ möäüß
bash: möäüß: command not found
[trav bash]$


@STDOUBT
Code:

resume=/dev/cryptvg/swap
is needed to hibernate the system, the kernel needs to know the resume location
fl0

STDOUBT 06-23-2013 03:45 AM

Quote:

Originally Posted by fl0 (Post 4976944)
@STDOUBT
Code:

resume=/dev/cryptvg/swap
is needed to hibernate the system, the kernel needs to know the resume location
fl0

Oh, I see. I didn't read that part of README_CRYPT.TXT since I always suspend to RAM.
Thanks for pointing it out!

Didier Spaier 06-23-2013 07:02 AM

I confirm that here (Slackware-14.0), patch linked to by mancha+ in post #4 solves issue raised by fl0, using urxvt provided by slackuilds.org.

I didn't make any other tests so I can't tell if applying this patch has any other effect.

Anyway I will use this new bash version from now and will report anything weird I can notice.

yars 06-23-2013 10:29 AM

I know about these. This is a bug in Bash, because of which the UTF-8 strings are handled incorrectly. I is not a programmer and don't know how to fix it nicely.

Martinus2u 06-23-2013 11:12 AM

Quote:

Originally Posted by mancha+ (Post 4976943)
That isn't what is being reported/corrected in this thread.

ooops, you're right. it was "command not found", not "permission denied". sorry for the mixup.

fl0 06-29-2013 10:25 AM

ok i have used the patched version as my working version , no problems so far.

@Pat
can you include the patch?

regards fl0

yars 06-29-2013 11:48 PM

Quote:

Originally Posted by fl0 (Post 4980784)
ok i have used the patched version as my working version , no problems so far.

Confirmed, I rebuilt bash with these patch, and it works fine.
@mancha:
Thanks for patch!


All times are GMT -5. The time now is 02:15 AM.