Quote:
Originally Posted by Hersh R
[...] change the default char set to utf-8 [...]
(I know, this comes 12 years too late, but...)
|
Yes, but this sort of issue comes up
constantly.
If text "comes up gibberish,"
it is always because(!) the client does not know the proper "character encoding." If the client believes that the data is ASCII, it will not look for the multi-byte (probably, but not necessarily, Unicode) characters. It will print everything as ASCII, one byte at a time, equals gibberish. The client does not take it upon itself to "figure out" what the encoding is: it must somehow be
told.
(Note: Some commands, such as
file,
do undertake to "make an intelligent guess" as to how a file might be encoded. But it is infeasible to expect the client to
rely upon such guesses. After all, it might be dead-wrong.)
UTF-8 is a very commonly-used encoding format that is efficient when many of the characters-of-interest
are ASCII. Another variation,
UTF-16, is more memory-efficient when you know that most of the characters of interest
are not going to be ASCII. Both of them fundamentally consist of a way to represent (Unicode) characters using a varying number of bytes each. And yet, while the UTF schemes have gained much traction, there
are other character-sets
(e.g. EBCDIC) and character-encoding schemes in common use.
This StackOverflow post provides almost anything you could want to know about the subject.
