LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-04-2011, 10:40 AM   #1
Genisys
LQ Newbie
 
Registered: Mar 2011
Posts: 3

Rep: Reputation: 0
Getting extended ascii (chars > 127) to work in Linux


I have an application where the client is written in Visual Studio (C#), run on PCs, and the server end has traditionally been SCO. We're now migrating to Linux. I can, for example, input "Test This" in a text box on the client, and when the server end is SCO, it is able to 'accept' the character sent to it from the client. When I try this same example on Linux, that character (hex D1) does not 'make it' from the client to the server. (Please forgive my layman's language).

The problem is not on the client, and I have verified that the telnet connection is in fact passing these extended characters, but they are not recognized properly by the Linux server.

In researching this, I've played with setting the LANG environmental variable from LANG=en_US.UTF-8 to several of the other possible values found in /usr/lib/locale, for a european locale (the end user is actually in Spain), and these 'euro' characters are still not handled properly in my application.

Would anyone be able to point me to any specific env variable settings, and/or anything else that would resolve this issue?
 
Old 03-04-2011, 11:20 AM   #2
tronayne
Senior Member
 
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,516

Rep: Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039
That sounds like you don't have a Unicode locale turned on system-wide.

In /etc/login.defs/lang.sh (the name and location may vary depending upon your distribution), there should be a setting to enable en_UTF-8 (you would comment-out the setting for LANG=en_US or, possibly, LANG=en_US.ISO8859-1 and un-comment LANG=en_US.UTF-8).

If you don't have /etc/login.defs/lang.sh, try
Code:
prompt: cd /etc
prompt: find . -type f -print | xargs grep -il "en_UTF-8"
to find the correct file.

You'll probably need to log out and log back again for any change to take effect (the login program has to be executed to make it happen -- if you're running a GUI login-thingy you may want to reboot).

Hope this helps some.

Last edited by tronayne; 03-04-2011 at 11:21 AM.
 
Old 03-04-2011, 11:44 AM   #3
Genisys
LQ Newbie
 
Registered: Mar 2011
Posts: 3

Original Poster
Rep: Reputation: 0
troayne - I do see in /etc/profile.d/lang.sh that the LANG env variable is being set to what you suggest. A "locale" output gives me:

LANG=en_US.UTF-8
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=C

I've also tried "es_ES.utf8" and "es_ES@euro" for LANG, and this doesn't change anything from what I can tell. I've also tried setting the LC_ALL to these two, plus the "en_US.UTF-8", and that still does not make a difference. Here's the output from "set" in case this is useful:

ABTERM=fwycol
BASH=/bin/sh
BASH_ARGC=()
BASH_ARGV=()
BASH_LINENO=()
BASH_SOURCE=()
BASH_VERSINFO=([0]="3" [1]="2" [2]="25" [3]="1" [4]="release" [5]="i686-redhat-linux-gnu")
BASH_VERSION='3.2.25(1)-release'
COLS=80
COLUMNS=78
CVS_RSH=ssh
DIRSTACK=()
EUID=0
EXMENU_PID=15242
EXMENU_SHELL=ABLOG
F10_ABORT=Y
F10_EXIT=Y
FLIP_BREAK='^C'
FLIP_CLIENT=root
FLIP_ESC_DELAY=200
FLIP_SHOWTIME=Y
FLPCTL_2SEC=Y
FLPCTL_ABORT=Y
FLPCTL_CHAN=1
FLPCTL_PID=15238
FLPCTL_TTY=/dev/pts/6
FLPCTL_UNIQUE=N
GINDTA_IDX=0
GROUPS=()
G_BROKEN_FILENAMES=1
HISTSIZE=1000
HOME=/root
HOSTNAME=show.genisys.com
HOSTTYPE=i686
IFS=$' \t\n'
INPUTRC=/etc/inputrc
LANG=en_US.UTF-8
LC_ALL=C
LESSOPEN='|/usr/bin/lesspipe.sh %s'
LINES=24
LINUX_SYSTEM=Y
LOGNAME=root
LS_COLORS=
MACHTYPE=i686-redhat-linux-gnu
MAIL=/var/spool/mail/root
METBIN=/metbin
MET_FAST_RPT_NAMES=N
MET_OPR_COUNT=Y
NOLOKSER=TRUE
OPTERR=1
OPTIND=1
OSTYPE=linux-gnu
PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:.:/metbin:/usr/met:/usr/met/sy:/usr/met/sy/bin:/usr/met/sy/vics_cmd:/etc:/root/bin:/usr/local:/met/sy/bin:/met/sy/cmd
PERL5LIB=/usr/local/lib/perl5
POSIXLY_CORRECT=y
PPID=15242
PS4='+ '
PWD=/etc/profile.d
RUNPATH=./:../:/met/sy:/met/sy/../ss
SCHEMA_SPEED=Y
SHELL=/bin/bash
SHELLOPTS=braceexpand:hashall:interactive-commentsosix
SHLVL=2
SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
SSH_CLIENT='192.168.10.185 50971 22'
SSH_CONNECTION='192.168.10.185 50971 192.168.10.65 22'
SSH_TTY=/dev/pts/6
SY_LOC=/met/sy
TERM=fwycol_f
TY=page
UID=0
USC_STYLE=VICS
USER=root
UV_STYLE=VICS
_=/metbin/ablog

Any other suggestions on what to try? (I greatly appreciate whatever feedback is given here. Thank you all very much!)
 
Old 03-04-2011, 02:32 PM   #4
fcintron
Member
 
Registered: Oct 2010
Location: Mxico
Distribution: Centos
Posts: 72

Rep: Reputation: 4
how is your app sending this data to linux server?

I have an app made in VB .net and my server is Centos 5.5 and I can send without problem


Regards
 
Old 03-04-2011, 02:41 PM   #5
tronayne
Senior Member
 
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,516

Rep: Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039
Did you go all the way out of X to the console, exit and log back in -- you really have to do that (or reboot the system) for the change to be available -- it will show up in set or locale but won't be in effect until you get all the way out to a console, log out and back in (or reboot if you're using a GUI login).

If you go to Dugan Chen's web site (http://www.vcn.bc.ca/~dugan/slackware-fonts/) and download his test file, ucs-fonts.tar.gz, then
  1. create a directory; e.g., unicode
  2. cd unicode
  3. tax xvf ../ucs-fonts.tar.gz
  4. cd examples
  5. cat quickbrown.txt
That will show you that you have, in fact, got your environment settings correctly set.

Hope this helps some.

Last edited by tronayne; 03-04-2011 at 02:43 PM.
 
Old 03-04-2011, 04:52 PM   #6
Genisys
LQ Newbie
 
Registered: Mar 2011
Posts: 3

Original Poster
Rep: Reputation: 0
(I do get all the way out, or reboot the Linux machine when I'm testing changes). I followed the instructions above from tronayne, and when I cat out the file, in the Spanish section I see:

Spanish (es)
------------

El pingC<ino Wenceslao hizo kilC3metros bajo exhaustiva lluvia y
frC-o, aC1oraba a su querido cachorro.

So this is telling me that I don't have something setup correctly, I'm assuming. So thanks for that confirmation. In further testing and analysis, I can now effectively receive these extended characters from the client to the server. So I'm making progress, and it points again to the server. But, I still have issues going from the server to the client. I'm using simple standard output to send text from the server to the client, via telnet. And my guess is that if catting out that quickbrown.txt test file could yield success, it'd probably resolve my issue. The only environmental variables that I can think of to share is:

LANG=en_US.UTF-8
MACHTYPE=i686-redhad-linux-gnu

Apologies to all here for being ignorant, but I simply can't put my finger on this. Anyone care to share their ENV variables (from 'set' and 'locale') in case that'd help? Any other suggestions? Thanks to all who have replied. I really appreciate it.
 
Old 03-04-2011, 05:26 PM   #7
tronayne
Senior Member
 
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,516

Rep: Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039Reputation: 1039
Telnet? Doesn't that strip out the 8th bit or something like that? Is there a setting to make it do binary, or could you use ssh (with PuTTY at http://www.chiark.greenend.org.uk/~sgtatham/putty on the Windows side and SSH on the Linux side).

Just for grins, on my system locale shows the following and the entire quickbrown.txt file displays every language in the file correctly.
Code:
locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE=C
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
Hope this helps some.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
python: how to handle unicode chars in ascii strings? BrianK Programming 5 01-27-2009 05:20 PM
Extended acsii chars linux_vidhyarthi Linux - General 5 04-16-2008 09:16 AM
Making ASCII chars working under X Harp00 Linux - Newbie 0 08-13-2006 07:22 PM
How to get ascii value (decimal ) of chars in linux? dreams Linux - General 8 01-27-2006 08:43 AM
How to enter high-ascii chars in a Linux editor? Kropotkin Linux - Newbie 4 11-27-2004 11:28 AM


All times are GMT -5. The time now is 06:56 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration