LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Slackware (http://www.linuxquestions.org/questions/slackware-14/)
-   -   vsftpd (or mozilla?) and encoding (http://www.linuxquestions.org/questions/slackware-14/vsftpd-or-mozilla-and-encoding-4175458938/)

elesmod 04-20-2013 06:18 PM

vsftpd (or mozilla?) and encoding
 
1 Attachment(s)
I've set up an FTP server on my headless Slackware64 13.37 machine. If I put files with polish diacritical signs (utf-8) in their names and browse them in Seamonkey or Firefox, they appear as gibberish (to me at least). I can switch the encoding in the browsers to uft-8 and then they appear fine, but shouldn't this be done automatically?

So, a file named "ęąśżźćń.txt" appears like shown in the attachment.

However, in Konqueror and in the text based "ftp" client, the encoding is set correctly and the file looks like it should.

From what I've gathered, looking around the Internet, vsftpd sets the encoding according to the system's locale, and mine's:

Code:

$ locale
LANG=pl_PL.utf8
LC_CTYPE="pl_PL.utf8"
LC_NUMERIC="pl_PL.utf8"
LC_TIME="pl_PL.utf8"
LC_COLLATE=C
LC_MONETARY="pl_PL.utf8"
LC_MESSAGES="pl_PL.utf8"
LC_PAPER="pl_PL.utf8"
LC_NAME="pl_PL.utf8"
LC_ADDRESS="pl_PL.utf8"
LC_TELEPHONE="pl_PL.utf8"
LC_MEASUREMENT="pl_PL.utf8"
LC_IDENTIFICATION="pl_PL.utf8"
LC_ALL=

What am I doing wrong? Or maybe it's Firefox's and Seamonkey's fault?

eSelix 04-21-2013 11:36 AM

Ftp server do not send to client which charset it use. You need to configure ftp client application, for example in Firefox, you can set UTF8 in "Edit/Preferences/Content/Fonts & Colors", button "Advanced".

rg3 04-21-2013 11:54 AM

It's probably a Firefox bug, but apparently you can set the language to use during an FTP session (see RFC), so there's a very small chance it could be considered an FTP server bug. You'd need to analyze the network traffic to see if Firefox issues a LANG command or not.

https://tools.ietf.org/html/rfc2640

Didier Spaier 04-21-2013 01:00 PM

The problem is that there is no 100% sure way to detect the encoding of a text string, so the program which read it has to use some kind of heuristic and make assumptions.

This program can rely on a shell variable like LANG or not.

I would type "about:config" in Firefox and try to play with the settings filtered searching for "utf", in the hope one is useful for that purpose.

I don't know if a specification for these settings is available.

elesmod 04-23-2013 07:41 AM

Quote:

Originally Posted by eSelix (Post 4935867)
in Firefox, you can set UTF8 in "Edit/Preferences/Content/Fonts & Colors", button "Advanced".

This helps a little, but other people connecting to my FTP server won't have that utf8 as default setting in Firefox.

Quote:

Originally Posted by rg3 (Post 4935876)
apparently you can set the language to use during an FTP session (see RFC)

(...)

You'd need to analyze the network traffic to see if Firefox issues a LANG command or not.

https://tools.ietf.org/html/rfc2640

I've started reading that memo, but it's too complicated for me. It seems it was written for the people who created the FTP protocol :P
Also, no idea how to analyze the network traffic to see if Firefox issues a LANG command.

Quote:

Originally Posted by Didier Spaier (Post 4935905)
The problem is that there is no 100% sure way to detect the encoding of a text string

(...)

I would type "about:config" in Firefox and try to play with the settings filtered searching for "utf"

The same issue as I've mentioned above, I can't make everyone connecting to my server change their browser settings.

Since it's not something I can set up with "encoding=utf8" in a vsftpd config file, I guess I'll just avoid using non-ASCII letters.

Thanks for your replies, guys.

Didier Spaier 04-23-2013 08:07 AM

Quote:

Originally Posted by elesmod (Post 4937158)
Since it's not something I can set up with "encoding=utf8" in a vsftpd config file, I guess I'll just avoid using non-ASCII letters.

This is kind of a paradox, as ASCII is a subset of UTF-8 ;)

comet.berkeley 04-23-2013 09:10 AM

The Filezilla FTP client site has a good discussion of the issue where ftp servers or clients don't follow RFC 2640 close enough:

https://wiki.filezilla-project.org/Character_Set

elesmod 04-24-2013 08:36 AM

Quote:

Originally Posted by comet.berkeley (Post 4937223)
The Filezilla FTP client site has a good discussion of the issue where ftp servers or clients don't follow RFC 2640 close enough:

https://wiki.filezilla-project.org/Character_Set

Thanks for that link, comet.berkeley. It pushed me to do some more digging.

Here's what I've found:
Quote:

Originally Posted by vsftpd changelog.txt (you can click this link)
v2.0.5 released!
- Pretend we have proper UTF8 support and respond positively to OPTS UTF8 ON.

Pretend?

Quote:

Originally Posted by Red Hat Customer Portal
* The vsftpd server previously did not support the UTF-8 feature. This update
implements the UTF-8 feature for the vsftpd server in accordance with the
Internationalization of the File Transfer Protocol (RFC 2640) standard.

Quote:

Originally Posted by vsftpd-3.0.2 REFS file
(Not implemented in vsftpd):
RFC-2640. Deals with internationalization and the LANG command. I'm not seeing
any vsftpd users with requirements in this area.

Eh? Maybe vsftpd is RFC-2640 compliant, and someone forgot to update that REFS file?

I've tried the following ftp clients:
ftp, ncftp, FileZilla, FireFTP (Firefox addon), Total Commander
and all of them displayed my files with diacritical signs correctly (which means they used utf8).

Moreover, when sending FEAT command in FileZilla, the response I received from my FTP server was:
Quote:

211-Features:
EPRT
EPSV
MDTM
PASV
REST STREAM
SIZE
TVFS
UTF8
211 End
Which means (I think), that most ftp clients are ok and only some web browsers are not compliant with RFC-2640.
I've tested 5 of them (green=ok; red=not_ok):
SeaMonkey 2.17.1
Firefox 20.0
Internet Explorer 9
*gasp*
Konqueror 4.8.5
Google Chrome 26.0.1410.63
(installed specifically to test this)


All times are GMT -5. The time now is 05:07 PM.