LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 08-05-2006, 10:56 AM   #1
hedpe
Member
 
Registered: Jan 2005
Location: Boston, MA
Distribution: Debian
Posts: 380

Rep: Reputation: 30
awk messing up trying to split a unicode line by whitespace


Hey guys,

I am not sure whats causing this, but whenever I need to split a line by whitespace, i've always used this for example:
Code:
awk -F" " '{print $2}'
I've written a script to help me go through greek vocabulary words, it reads a line from a file which is in the format:
Code:
english greek_present greek_past greek_future
In my script, i try to split the line like this:
Code:
    echo $line
    english=$(echo $line | awk -F" " '{print $1}')
    present=$(echo $line | awk -F" " '{print $2}')
    past=$(echo $line | awk -F" " '{print $3}')
    future=$(echo $line | awk -F" " '{print $4}')
    echo  $english $present $past $future
This is where things start getting screwed up. the two echos for testing do not come out the same, here is an example:
Code:
buy αγοράζω αγόρασα αγοράσω
buy αγοράζω αγό αγοράσω
$past only has part of the word, not the full word as "echo $line" shows there should be. I don't know if this is an awk problem with unicode or if its a bash variable problem. My guess is that it is awk, since $line always contains the proper characters, and i've tested that over and over. The problems only begin when it goes through awk.

Also, note that its not always predictable, awk will sometimes split it properly, or will screw up $present, or maybe $future ... it never does the same thing over and over.

Any ideas?

Thanks!
George
 
Old 08-05-2006, 11:10 AM   #2
hedpe
Member
 
Registered: Jan 2005
Location: Boston, MA
Distribution: Debian
Posts: 380

Original Poster
Rep: Reputation: 30
i think quotes around the echo $variables, like echo "$variables" may have fixed it
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Split large file in several files using scripting (awk etc.) chipix Programming 14 10-29-2007 11:16 AM
How to split file , .. awk or split ERBRMN Linux - General 9 08-15-2006 12:02 AM
removing first line with AWK koobi Programming 7 08-03-2006 01:48 AM
split files using awk (or similar) lgualteri Programming 1 06-13-2005 09:17 AM
ignoring the "non-portable whitespace encountered at line " warning Jake13 Linux - Software 3 08-18-2004 12:34 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 08:09 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration