How get rid of funny characters in filenames?

Bengan · 01-05-2006, 09:52 AM

I am rescuing files from a Linux partition onto an USB drive using an Ubuntu live CD. Some of the files have funny characters in them (due to the fact that they reside on a Swedish machine). Ubuntu gets upset and tells me it cannot create those files on the target drive due to those characters. I can only access the partition where the files are as root. My knowledge of Linux is limited (in particular on the command line), so I would need and appreciate some assistance. I have tried using cp, rename and mv, but all of them complain about the characters. What to do?

Bengt

ssfrstlstnm · 01-05-2006, 12:11 PM

Here is my feable attempt at a solution. My perl is not very good, so this may not work and I can't test it till I get home. Save the following script as replace_funnies

Code:

#/usr/bin/perl

$infile = $ARGV[1];
$outfile = $ARGV[2];

open (INFILE, "<$infile") or die "Can not open $infile!";
open (OUTFILE, ">$outfile") or die "Can not open $outfile for writing!";

while ($line=<INFILE>) {
    $line =~ s/[^\w\s\n\.\?\!]//;
    print OUTFILE $line;
    }

then do

Code:

chmod +x replace_funnies

to make it executable
You run the script like this

Code:

perl replace_funnies infile outfile

Change infile and outfile to match your situation
Right now it is replacing anything that is not A-Z, a-z, 0-9, ., ?, or !. You can add other characters that you don't want to replace by changeing the s/// line. For example to add ( and ) change it to

Code:

$line =~ s/[^\w\s\n\.\?\!\(\)]//;

ssfrstlstnm · 01-05-2006, 05:15 PM

Fixed it up a bit here you go:

Code:

#/usr/bin/perl

$infile = shift @ARGV;
$outfile = shift @ARGV;

if ($#ARGV != 1) {die "usage\: perl replace_funnies infile outfile"}

open (INFILE, "< $infile");
open (OUTFILE, "> $outfile");

while ($line=<INFILE>) {
    $line =~ s/[^\w\s\n\.\?\!\:\;\\,'\"\(\)\$\*\&\#\[\]\{\}\=\+\-\|\<\>]//g;
    print OUTFILE $line;
    }

Bengan · 01-05-2006, 05:24 PM

Thank you. I just had a look around. The Ububtu system does not include Perl. Sorry.

Bengan · 01-05-2006, 05:45 PM

Sorry, I was wrong. I typed your program into a file and made it executable. When I ran it, it gave me the complaints:

"
line 3: =: command not found
line 4: =: command not found
"

Could it be the "shift" statement it doesn't like. What does it do?
I checked the contents of my file against your original and couldn't find any diffs.

ssfrstlstnm · 01-05-2006, 06:27 PM

Probably means you don't have perl installed. Surely ubuntu has perl, its a very common scripting language in linux. Have you looked for it with the synaptic package manager? Do you have your package repositories set up correctly? Post the output of

Code:

cat /etc/apt/sources.list

Bengan · 01-06-2006, 05:58 AM

It looks like there is some kind of Perl installed after all - if I run the script without arguments, it runs the "die" statement so I get the error message correctly. Here is the output from the sources.list:

"
deb http://archive.ubuntu.com/ubuntu hoary main restricted
deb-src http://archive.ubuntu.com/ubuntu hoary main restricted
"

then there is a commented message regarding support and things.
It says that if I want to add software from the "universe" depository, I should uncomment the two commented lines following. Then comes this:

"
# deb http://archive.ubuntu.com/ubuntu hoary universe
# deb-src http://archive.ubuntu.com/ubuntu hoary universe

deb http://security.ubuntu.com/ubuntu hoary-security main restricted
deb-src http://security.ubuntu.com/ubuntu hoary-security main restricted
"

That's it. It doesn't make much sense to me. I took a look at the URLs listed, but found nothing much.

Now, there might be a problem. I'll try to make the story short.
I was running Mandriva 2006 and tried to upgrade KDE to 3.5, and, apparently, deleted too much of the 3.4 I was running. I couldn't because the root partition was full. Threfore I ended up without the graphical interface. I think the only way I can get it up and running again is to re-install the entire system with a different set of partitions, but before that I'd like to salvage the work I have done. Now, some of the files have Swedish characters in their names, and will therefore not be copied onto the USB drive (as you know). This is the reason I run the Ubuntu system - not to disturbe the /home partition, which I might have if I had re-sized the root partition. There is also an empty partition that I would like to include or simply use as the root. It is big enough. That is a different problem, though.

Bengan · 01-06-2006, 06:09 AM

Something just occurred to me - would there be another live CD distro around which could copy files with funny characters? I have a Knoppix CD and a College CD. Would they be live distros?

ssfrstlstnm · 01-06-2006, 09:11 AM

Lets try to configure your locales so that you can see those funny characters. First we need to enable root login cause ubuntu doesn't allow it by default.

Code:

sudo passwd root

Then enter your password and a password for the root account. For security reasons, make the root password something hard to guess. Then you can login as root using su. Now as root try to add the locale for the Swedish character set.

Code:

dpkg-reconfigure locales

Pick the locales that you want to use, and then try to look at those files again.

I couldn't tell from your previous post if your x window system is broken or not. You also mentioned something about KDE, but ubuntu uses gnome.

Bengan · 01-06-2006, 11:03 AM

Yeah, I was using KDE in the original Mandriva installation.

I read the Knoppix homepage and found that they appear to specialise in live CDs only, so I took mine and ran it. It works fine, and has got a KDE desktop. I managed to copy the "funny" files too without problem. Then I tried your dpkg command with no success. Both Ubuntu and Knoppix have a "root terminal" which does not require a password (funny, though not really safe).

The USB drive is automatically mounted and display an icon on the desktop. In Knoppix it is mounted as a read-only device. To enable me to copy the files, I had to open the root terminal, unmount the drive and then mount it again to make it writable. Since neither system allow a non-root user to enter the subdirectories of /home, I had to stay as a root. Then "cp -r *" did the trick. The only problem it had is a directory called "Why?", and I can understand that. It i okay, though, because I have it on a CD and I know it has not been touched since.

Thanks a lot for your assistance. Much appreciated.

ssfrstlstnm · 01-06-2006, 11:49 AM

Ubuntu does not allow direct root login. You have to use sudo, which in my opinion is a silly. That is probably why you could not reconfigure the locales. Not sure what your "root terminal" is, but I doubt if it actually gives you root priveleges. That would be a serious security risk, and would completely defeat the purpose of making you use sudo. Anyway, glad you got it worked out. You should still be able to use the commands above (at least in ubuntu) to get your locales set up once you get to be root.

Bengan · 01-07-2006, 11:38 AM

Well, with the Ubuntu system I was able to copy the files with only the 25 English characters in their names from the "root terminal". I used Knoppix for the rest of the files. I now have all files I wanted on the USB disk. At desktop level I had no access to /home OR the USB-drive, which I had in the "root terminal". Wello, the main thing is that I managed to rescue the files.