Locales problem copying files thru ssh

palaman · 07-23-2009, 04:01 AM

Hi. I'm posting here because a problem I'm having with locales (I think) and I hope anybody could help me out.

I've been using a linux box with Debian Sarge, running a Samba server 3.0 (acting as a PDC to local network) for the last 3 years withtout a problem. Well, now I have a new Dell server where I've installed Ubuntu 9.04 server with Samba. The goal is to replace the old server with the new one. As simple as that.

At this time (almost) everything is working well (domain logons, permissions, users, groups, etc), but the problem comes when I want to transfer all the files to the new server. All files with "non standard" characters (I mean, úáé Ññ, and so on...) are showed with "?" or other characters. But that only happens with the files *transferred* from the old server to the new one. Actually the new files *created* locally into the new server (via samba over the network or from the terminal) are created correctly.

My locales setup are the following:

Old debian server:

Code:

root@tierra:~# locale
LANG=es_ES
LANGUAGE=es_ES
LC_CTYPE="es_ES"
LC_NUMERIC="es_ES"
LC_TIME="es_ES"
LC_COLLATE="es_ES"
LC_MONETARY="es_ES"
LC_MESSAGES="es_ES"
LC_PAPER="es_ES"
LC_NAME="es_ES"
LC_ADDRESS="es_ES"
LC_TELEPHONE="es_ES"
LC_MEASUREMENT="es_ES"
LC_IDENTIFICATION="es_ES"
LC_ALL=

and the New Ubuntu Server

Code:

root@tierra2:~# locale
LANG=es_ES.UTF-8
LC_CTYPE="es_ES.UTF-8"
LC_NUMERIC="es_ES.UTF-8"
LC_TIME="es_ES.UTF-8"
LC_COLLATE="es_ES.UTF-8"
LC_MONETARY="es_ES.UTF-8"
LC_MESSAGES="es_ES.UTF-8"
LC_PAPER="es_ES.UTF-8"
LC_NAME="es_ES.UTF-8"
LC_ADDRESS="es_ES.UTF-8"
LC_TELEPHONE="es_ES.UTF-8"
LC_MEASUREMENT="es_ES.UTF-8"
LC_IDENTIFICATION="es_ES.UTF-8"
LC_ALL=

All I want to do to transfer all the files over ssh, preserving the structure and its attributes (owner, permissions and ACL's) thru tar over the network. To to that I'm running this command:

Code:

root@OldServer:/home/# tar -cf - . | ssh root@NewServer "cd /home/ ; tar -xvf - "

After all the content is copied from the old server to the new one, I proceed to check out in the new server, then I see some files or directories with bad characters on them.

For example, the files in the old server with this names:

Code:

CARPETA DE BEGOÑA
TABLA DE PRÉSTAMOS.XLS
Carta de indemnización.doc

are "converted" to this names once they are in the new server.

Code:

CARPETA DE BEGO?A
TABLA DE PR?STAMOS.XLS
Carta de indemnizaci?n.doc

I've tried to change the locale settings in both, the old server and in the new one to make them use the same settings (es_ES, and es_ES.UTF-8 and viceversa in both servers) with no satisfactory result at all. I Also tried to configure ssh access by modifying .bashrc with some "tricks" I found searching through google, but unfortunaly I got nothing.

Does anybody know some kind of idea why this is happening? or much better, does somebody know how to do what I want?, which is basically to transfer files from one server to another no matter how their locale configurations are. Right now I don't even know if that is possible, though (but i'm starting to doubt it)

Please help...

Thank you in advance.

neonsignal · 08-20-2009, 11:01 PM

You can use the posix format for the tar file ('--format=posix'). That way, the local encoding will be used to convert the filenames to UTF8 inside the tar file.

If you can't redo the transfer, you can repair filenames using the 'convmv' utility.

palaman · 08-21-2009, 05:05 AM

Quote:

Originally Posted by neonsignal

You can use the posix format for the tar file ('--format=posix'). That way, the local encoding will be used to convert the filenames to UTF8 inside the tar file.

If you can't redo the transfer, you can repair filenames using the 'convmv' utility.

Thank you very much, I'll try it as soon as I can, and I'll let you know if that works for me. I didn't know the convmv utility neither, btw. I appreciate your help on this. Thanks again.

Regards.