LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Closed Thread
 
Search this Thread
Old 01-26-2009, 05:29 PM   #1
BrianK
Senior Member
 
Registered: Mar 2002
Location: Los Angeles, CA
Distribution: Debian, Ubuntu
Posts: 1,334

Rep: Reputation: 51
python: how to handle unicode chars in ascii strings?


So, in python, this assignment is legal, but it breaks conversion:

Code:
In [44]: sys.getdefaultencoding()
Out[44]: 'ascii'

In [45]: a = 'André'

In [46]: a.encode('ascii','replace')
---------------------------------------------------------------------------
exceptions.UnicodeDecodeError                        Traceback (most recent call last)

/hosts/soho/v11/users/briank/<ipython console>

UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 4: ordinal not in range(128)
however, if I specify that the string is unicode, this all works:

Code:
In [47]: a = u'André'

In [48]: a.encode('ascii','replace')
Out[48]: 'Andr??'
The problem I'm having here is that I'm using os.walk to go through a bunch of files... some of those files have paths with unicode chars. I'm unclear on how to get the results of os.walk to be considered unicode such that the encode function works correctly.

At the end of the day, All these paths are going into a database. I just want everything that comes out of that database to be ascii & I'm having a hard time with that when unicode chars make their way into strings that think they are ascii.

Hope that made sense.

So... how can you convert ascii strings with unicode chars into actual ascii strings. Pardon me if my terminology is incorrect - see my first example for what I mean by "unicode chars in ascii strings"
 
Old 01-26-2009, 09:46 PM   #2
crabboy
Moderator
 
Registered: Feb 2001
Location: Atlanta, GA
Distribution: Slackware
Posts: 1,823

Rep: Reputation: 120Reputation: 120
Continue discussion here:
http://www.linuxquestions.org/questi...3/#post3422175
 
  


Closed Thread


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
python: how do you replace unicode chars in large text files? BrianK Programming 1 12-19-2008 12:54 AM
Is it really worth it to use chars, especialy since I can strings? RHLinuxGUY Programming 6 07-23-2006 12:52 PM
cyrillic chars - unicode or koi8_ru ojav Linux - Newbie 1 05-29-2005 02:51 PM
can fedora handle ascii strings for wep keys? ehawk Linux - Wireless Networking 1 10-13-2004 06:34 PM
Fun with strings & chars in C Scrag Programming 4 05-19-2004 03:03 PM


All times are GMT -5. The time now is 02:53 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration