Welcome to the most active Linux Forum on the web.
Go Back > Forums > Non-*NIX Forums > Programming
User Name
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.


  Search this Thread
Old 08-05-2006, 10:56 AM   #1
Registered: Jan 2005
Location: Pittsburgh
Distribution: Ubuntu
Posts: 378

Rep: Reputation: 30
awk messing up trying to split a unicode line by whitespace

Hey guys,

I am not sure whats causing this, but whenever I need to split a line by whitespace, i've always used this for example:
awk -F" " '{print $2}'
I've written a script to help me go through greek vocabulary words, it reads a line from a file which is in the format:
english greek_present greek_past greek_future
In my script, i try to split the line like this:
    echo $line
    english=$(echo $line | awk -F" " '{print $1}')
    present=$(echo $line | awk -F" " '{print $2}')
    past=$(echo $line | awk -F" " '{print $3}')
    future=$(echo $line | awk -F" " '{print $4}')
    echo  $english $present $past $future
This is where things start getting screwed up. the two echos for testing do not come out the same, here is an example:
buy αγοράζω αγόρασα αγοράσω
buy αγοράζω αγό αγοράσω
$past only has part of the word, not the full word as "echo $line" shows there should be. I don't know if this is an awk problem with unicode or if its a bash variable problem. My guess is that it is awk, since $line always contains the proper characters, and i've tested that over and over. The problems only begin when it goes through awk.

Also, note that its not always predictable, awk will sometimes split it properly, or will screw up $present, or maybe $future ... it never does the same thing over and over.

Any ideas?

Old 08-05-2006, 11:10 AM   #2
Registered: Jan 2005
Location: Pittsburgh
Distribution: Ubuntu
Posts: 378

Original Poster
Rep: Reputation: 30
i think quotes around the echo $variables, like echo "$variables" may have fixed it


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Split large file in several files using scripting (awk etc.) chipix Programming 14 10-29-2007 11:16 AM
How to split file , .. awk or split ERBRMN Linux - General 9 08-15-2006 12:02 AM
removing first line with AWK koobi Programming 7 08-03-2006 01:48 AM
split files using awk (or similar) lgualteri Programming 1 06-13-2005 09:17 AM
ignoring the "non-portable whitespace encountered at line " warning Jake13 Linux - Software 3 08-18-2004 12:34 PM > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 05:39 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration