Script to determine file encoding

maxmil · 10-18-2007, 02:30 AM

I am looking for a way to determine the encoding of a file.

I need to convert a directory tree of files from ISO-8859-1 to UTF-8. I can do this with iconv.

The problem is that in my directory tree there are both binary files and files that have already been converted to UTF-8. I don't want to touch these files. I need to be able to determine if a file is eligeable for conversion before calling iconv.

Thanks

Tinkster · 10-18-2007, 07:47 PM

`file' does what you're looking for...

It will tell you a files encoding.

Cheers,
Tink

Miseq · 09-13-2008, 02:18 AM

Quote:

Originally Posted by Tinkster

`file' does what you're looking for...

It will tell you a files encoding.

Cheers,
Tink

Your answer just isn't correct. He asked for a tool which will does it automatically and you adwised him to do it himself by hand.

I've just tried to find such tool and I've found auto_ef or libauto_ef. This is tool from Sun Solaris and I'm still trying to find it's source code in OpenSolaris.

Mr. C. · 09-13-2008, 03:26 AM

The missing term was codeset.

U don't know of any utilities, but here's a description of how to determine codeset:

http://codesnipers.com/?q=node/68

It doesn't seem too difficult to write a script to implement this.