iconv errors

qwertyjjj · 09-04-2011, 09:31 AM

I have a file with French characters in and it needs to be ISO 8859 to display correctly on my webpage.
I keep trying to convert it from UTF8 but it errors at the first slash / apostrophe.

Any ideas why?

Code:

<?php
/*
$Id: $

osCommerce, Open Source E-Commerce Solutions
http://www.oscommerce.com

Copyright © 2007 osCommerce

Released under the GNU General Public License
*/

define('TEXT_MAIN', '
<hr>
<b style="color: rgb(7,75,138);">
Proxy vous offre l\

iconv: illegal input sequence at position 266

Code:

<?php
/*
  $Id: $

  osCommerce, Open Source E-Commerce Solutions
  http://www.oscommerce.com

  Copyright (c) 2007 osCommerce

  Released under the GNU General Public License
*/

define('TEXT_MAIN', '
<hr>
<b style="color: rgb(7,75,138);">
Proxyllllll vous offre l\â€™accÃ¨s Ã* des serveurs proxy et rÃ©seau privÃ© virtuel (VPN) pour une grande variÃ©tÃ© dâ€™utilisations telles que :<br />

David the H. · 09-04-2011, 11:01 AM

Since ISO-8859 is a more limited encoding than unicode, chances are you've simply hit a character that it can't handle.

Not to mention that there are about 15 variations of that encoding, each with a slightly different set of supported characters. Latin1 text is ISO-8859-1 or the "updated" ISO-8859-15, while 2-14,16 are designed with various other, mostly European languages in mind.

http://en.wikipedia.org/wiki/ISO/IEC_8859

Finally, the gcc-based iconv, at least, has two options for handling illegal characters:

Code:

iconv -f UTF-8 -t ISO-8859-1//IGNORE file	#delete any unsupported characters

iconv -f UTF-8 -t ISO-8859-1//TRANSLIT file	#attempt to replace unsupported
						#characters with similar characters
						#from the target encoding

PS: A better, long term solution would probably be to convert your web page to UTF-8. ISO-8859 is old tech. The future is unicode.

qwertyjjj · 09-04-2011, 11:06 AM

Quote:

Originally Posted by David the H.

Since ISO-8859 is a more limited encoding than unicode, chances are you've simply hit a character that it can't handle.

Not to mention that there are about 15 variations of that encoding, each with a slightly different set of supported characters. Latin1 text is ISO-8859-1 or the "updated" ISO-8859-15, while 2-14,16 are designed with various other, mostly European languages in mind.

http://en.wikipedia.org/wiki/ISO/IEC_8859

Finally, the gcc-based iconv, at least, has two options for handling illegal characters:

Code:

iconv -f UTF-8 -t ISO-8859-1//IGNORE file	#delete any unsupported characters

iconv -f UTF-8 -t ISO-8859-1//TRANSLIT file	#attempt to replace unsupported
						#characters with similar characters
						#from the target encoding

PS: A better, long term solution would probably be to convert your web page to UTF-8. ISO-8859 is old tech. The future is unicode.

UTF doesn;t seem to recognise French characters.

qwertyjjj · 09-04-2011, 11:16 AM

Is there a LINUX GUI program that can show you which characters are incorrect?
I keep trying to save my file as ISO with French characters and gedit errors.

It does not like these characters in ISO:
Avec ses prix compétitifs et ses forfaits flexibles, Proxyxxxxxx vous offre l'accès à un serveur PROXY et/ou RÉSEAU PRIVÉ VIRTUEL (VPN) géolocalisé en France, qui vous permettra d’accéder à l’internet sans restrictions de partout dans le monde et ce, pour une grande variété d'utilisat

David the H. · 09-04-2011, 11:59 AM

UTF-8 can handle anything. As I said, it's ISO-8859 that's limited. Did you try the IGNORE or TRANSLIT options I mentioned?

Read the Wikipedia link I gave you, and follow on to the sub-pages as necessary. The link to http://en.wikipedia.org/wiki/ISO/IEC_8859-1 says this about French, for example (slightly abbreviated) :

Code:

Languages commonly supported but with incomplete coverage

Language	Missing characters	Typical workaround	Supported by

French	 	Œ, œ, and Ÿ		digraphs OE, oe, and Y	ISO-8859-15, Windows-1252

qwertyjjj · 09-04-2011, 01:20 PM

Quote:

Originally Posted by David the H.

UTF-8 can handle anything. As I said, it's ISO-8859 that's limited. Did you try the IGNORE or TRANSLIT options I mentioned?

Read the Wikipedia link I gave you, and follow on to the sub-pages as necessary. The link to http://en.wikipedia.org/wiki/ISO/IEC_8859-1 says this about French, for example (slightly abbreviated) :

Code:

Languages commonly supported but with incomplete coverage

Language	Missing characters	Typical workaround	Supported by

French	 	Œ, œ, and Ÿ		digraphs OE, oe, and Y	ISO-8859-15, Windows-1252

When I save the file in UTF8 it displays weird characters instead of the French accents.
I'm using gedit in Linux rather than the terminal...