LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-23-2011, 02:55 AM   #1
kcynice
Member
 
Registered: Sep 2008
Posts: 104

Rep: Reputation: 16
How to convert a windows like text file into *nix like utf-8 encoding automately?


Mostly, I want to convert many text files(copied from windows workstation) into utf-8 encoding file. Yes, iconv is available for it. However, I have to give source file encoding at the command line parameters! The problem is, at most case, I am not sure the source encoding of it. And, I also want to use a script to convert many files recursively.

What should I do?
 
Old 03-23-2011, 09:08 AM   #2
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 6,680
Blog Entries: 14

Rep: Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022Reputation: 1022
I suspect you're talking about the difference between DOS/Windows text files and UNIX/Linux text files. That is that the former have a carriage return at the end of every line whereas the latter only have a line feed. When you view the file in *nix with vi/vim you see a ^M at the end of every line - that is the carriage return (ctrl-M).

You can convert files with dos2unix or unix2dos commands. Typing "man dos2unix" or "man unix2dos" will give you more details. They're fairly simple to use.
 
Old 03-23-2011, 09:23 PM   #3
kcynice
Member
 
Registered: Sep 2008
Posts: 104

Original Poster
Rep: Reputation: 16
Quote:
Originally Posted by MensaWater View Post
I suspect you're talking about the difference between DOS/Windows text files and UNIX/Linux text files. That is that the former have a carriage return at the end of every line whereas the latter only have a line feed. When you view the file in *nix with vi/vim you see a ^M at the end of every line - that is the carriage return (ctrl-M).
Not exactly, CR/LF is a problem, but not the most. The most issue is, Im a Chinese, most of my files have Chinese words included. When such files are copied from windows to linux, some softwares can't work normally. Yes, i can convert them to utf-8 encoding by hands, but, I can't do so many files one by one, because I can't guarantee all the source file use the same encoding.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Re: Locale problem: Can't convert string from native encoding to 'UTF-8' treotan Ubuntu 1 11-14-2007 09:02 AM
convert text-file from utf-8 to iso-8859-1 [SOLVED] @ngelot Linux - Server 1 06-12-2007 06:47 AM
[Enter] in text documents diffrent on Windows and Linux? UTF-8/UTF-16 problem or? brynjarh Linux - General 1 11-24-2004 06:20 AM
I need perlscript to convert text file in UTF-16 cccc Programming 3 07-04-2004 05:08 AM
how to convert windows formatted text files to *nix formatting? zero79 Linux - General 8 06-11-2004 04:00 AM


All times are GMT -5. The time now is 07:53 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration