[SOLVED] Wrong charsets on new files on Ubuntu Unity 20.04

Shinare · 03-02-2021, 08:50 AM

Whenever I create a new text file with nano or gedit, (At least with no extension, with .txt and with .php)
the file's charset is us-ascii while it should be utf-8. New files used to always be utf-8 until about one hour ago when I downloaded a text file with us-ascii. The only thing I did with it was copy its contents to clipboard, so it shouldn't cause any system changes, but it's the only related thing that happened.
I've also tried to manually change new files' charsets with

Code:

iconv file -f us-ascii -t utf-8 -o file

but it doesn't do anything.

Code:

$ locale
LANG=en_US.UTF-8
LANGUAGE=en
LC_CTYPE="en_GB.UTF-8"
LC_NUMERIC="en_GB.UTF-8"
LC_TIME="en_GB.UTF-8"
LC_COLLATE="en_GB.UTF-8"
LC_MONETARY="en_GB.UTF-8"
LC_MESSAGES="en_GB.UTF-8"
LC_PAPER="en_GB.UTF-8"
LC_NAME="en_GB.UTF-8"
LC_ADDRESS="en_GB.UTF-8"
LC_TELEPHONE="en_GB.UTF-8"
LC_MEASUREMENT="en_GB.UTF-8"
LC_IDENTIFICATION="en_GB.UTF-8"
LC_ALL=en_GB.UTF-8

This is the only possibly relevant command I could find, and it says UTF-8 everywhere.

smallpond · 03-02-2021, 09:36 AM

My CentOS system calls text files ASCII until I add a character greater than 127, like a euro symbol. Then it calls it UTF-8.

Shinare · 03-02-2021, 09:55 AM

Quote:

Originally Posted by smallpond

My CentOS system calls text files ASCII until I add a character greater than 127, like a euro symbol. Then it calls it UTF-8.

Looks like that might be what's happening to me too, so I guess there isn't any real issue then. Thanks.

BenCollver · 03-02-2021, 04:14 PM

Does it help to add the BOM at the beginning of the file?

Code:

$ printf 'hello world\n' >ascii.txt
$ printf '\xef\xbb\xbfhello world\n' >utf8.txt
$ file ascii.txt 
ascii.txt: ASCII text
$ file utf8.txt 
utf8.txt: UTF-8 Unicode (with BOM) text