LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-30-2011, 05:24 PM   #1
ivn
LQ Newbie
 
Registered: Sep 2011
Posts: 10

Rep: Reputation: Disabled
Search and Replace from Word List - Scripting


I have a CSV file with some words (mostly colors) in another language. I want to change those words to English.

The words that need to be replaced vary in capitalization; i.e. Nero/NERO/nero ; all should be replaced with standard capitalized English word (Black).

I'm thinking a file containing the translations and somehow search/replace the csv file.

Thoughts?
 
Old 09-30-2011, 06:11 PM   #2
crts
Senior Member
 
Registered: Jan 2010
Posts: 1,608

Rep: Reputation: 449Reputation: 449Reputation: 449Reputation: 449Reputation: 449
Quote:
Originally Posted by ivn View Post
I have a CSV file with some words (mostly colors) in another language. I want to change those words to English.

The words that need to be replaced vary in capitalization; i.e. Nero/NERO/nero ; all should be replaced with standard capitalized English word (Black).

I'm thinking a file containing the translations and somehow search/replace the csv file.

Thoughts?
Hi,

what do you have got so far? How does the CSV file exactly look like? How does the translation file look like? That is, if you have a translation file.

My first thought would be something like:
Code:
sed 's/nero/BLACK/ig' file
Of course, if you have a file with colors and their translation you could read the corresponding values into variables and use a loop to make the changes.

Last edited by crts; 09-30-2011 at 06:59 PM. Reason: corrected flag in sed
 
Old 09-30-2011, 06:42 PM   #3
ivn
LQ Newbie
 
Registered: Sep 2011
Posts: 10

Original Poster
Rep: Reputation: Disabled
I have a script the manipulates the file and does other stuff, and right now I'm trying to tackle the translation.

Sample column of the file:

BIANCO
GRIGIO-MELANGE
BIANCO/NERO
BIANCO/NERO
NERO
NERO
NERO
NERO
NERO
ROSSO
ROSSO
ROSSO
VERDE-MILITARE
VERDE-MILITARE
NAVY
NAVY
NAVY
NAVY
NAVY
NAVY
GRIGIO-MELANGE
GRIGIO-MELANGE
PRINCESS-BLUE
PRINCESS-BLUE
PRINCESS-BLUE

And there are A LOT more color variations. I don't have a translation file, but I was thinking something like:

BIANCO=White
VERDE=Green
...

I saw this somewhere else, but I can't make it work:

Code:
sed -e 's/^/s%/' -e 's/=/%/' -e 's/$/%g/' listfile | sed -f inputfile > outputfile
 
Old 09-30-2011, 07:15 PM   #4
crts
Senior Member
 
Registered: Jan 2010
Posts: 1,608

Rep: Reputation: 449Reputation: 449Reputation: 449Reputation: 449Reputation: 449
Ok,

if you do not have a file yet then why not make it a sed script in the first place? E. g., create a file called 'colors':
Code:
# filename: colors
s/BIANCO/White/ig
s/VERDE/Green/ig
Then invoke sed like this:
Code:
sed -f colors file_to_be_translated
BTW, the sed you mention converts your format
COLOR=TRANSLATION

to the above format. However, you cannot invoke it the way you tried. Try this:
Code:
sed -e "$(sed  's/^/s%/;s/=/%/;s/$/%ig/' colors)" file_to_be_translated
However, I do not recommend the latter. If you want to keep the format
COLOR=TRANSLATION

then create an intermediate file that has the format as suggested above and then use this file as input:
Code:
sed  's/^/s%/;s/=/%/;s/$/%ig/' colors > colors.temp
sed -f colors.temp file_to_be_translated
rm -f colors.temp
The 'i' flag to the 's' command is a GNU extension. It allows for case insensitive RegEx. So you will have to use GNU sed for the above to work.
 
Old 10-01-2011, 02:20 AM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,493

Rep: Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867
If you already had the color file in the format you demonstrated, you could do some thing like:
Code:
#!/bin/bash

for line in $(< colors)
do
    set -- ${line/=/ }

    sed -i "s/$1/$2/g" file_to_be_translated
done
 
Old 10-01-2011, 04:27 PM   #6
ivn
LQ Newbie
 
Registered: Sep 2011
Posts: 10

Original Poster
Rep: Reputation: Disabled
Awesome, thanks for the suggestions. Since I didn't have the word file already created, I went with:

Code:
# filename: colors
s/BIANCO/White/g
s/VERDE/Green/g
I'm on MacOS so couldn't use the "i" option. There wasn't too many varations of upper/lower, so I just created them manually.
 
Old 10-01-2011, 07:28 PM   #7
ivn
LQ Newbie
 
Registered: Sep 2011
Posts: 10

Original Poster
Rep: Reputation: Disabled
Grail, I saw a thread that you posted on:
http://www.linuxquestions.org/questi...column-805271/

I tried:

Code:
awk 'BEGIN{ 
        FS = ","    # input field seperator 
        OFS = ","   # output field seperator
}
     NR > 3 {$2 = tolower($2);
          split($2,arr," ");
          for(x in arr)
              sub(arr[x],toupper(substr(arr[x],1,1))substr(arr[x],2),$2)
         }1' infile.csv
I'm looking for column 3 to be converted only. I don't see any changes when the file prints on the screen though... any thoughts?
 
Old 10-01-2011, 08:16 PM   #8
ivn
LQ Newbie
 
Registered: Sep 2011
Posts: 10

Original Poster
Rep: Reputation: Disabled
Nevermind, got it!

Code:
awk 'BEGIN{ 
        FS = ","    # input field seperator 
        OFS = ","   # output field seperator
}
     NR {$5 = tolower($5);
          split($5,arr," ");
          for(x in arr)
              sub(arr[x],toupper(substr(arr[x],1,1))substr(arr[x],2),$5)
         }1' ../translated.csv
I figured out what the code was doing. And I'm looking for column 4, and it shows column 5, but one of the columns has a comma in there, so awk thinks its 5...
 
Old 10-02-2011, 03:57 AM   #9
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,493

Rep: Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867
NR is not required anymore seeing you removed the test. Also, if you show me an example of the file you are working with I can probably make it simpler
as that example was some time ago
 
Old 10-02-2011, 02:41 PM   #10
ivn
LQ Newbie
 
Registered: Sep 2011
Posts: 10

Original Poster
Rep: Reputation: Disabled
Ok, still having some issues...

I'm using this:

Code:
awk 'BEGIN{ 
        FS = ","    # input field seperator 
        OFS = ","   # output field seperator
}
     NR {$5 = tolower($5);
          split($5,arr," ");
          for(x in arr)
              sub(arr[x],toupper(substr(arr[x],1,1))substr(arr[x],2),$5)
         }1' ./tmp/translated.csv > ./tmp/cased.csv
on this:

Code:
,SKU,"[S]Color=Black/White/Blue,[S]Size=44",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=46",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=48",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=50",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=52",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=54",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=56",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=58",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=60",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=44",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=46",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=48",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=50",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=52",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=54",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=56",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=58",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=60",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=44",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=46",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=48",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=50",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=52",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=54",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=56",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=58",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=60",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=44",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=46",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=48",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=50",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=52",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=54",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=56",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=58",Brand Name T. ASPIDE P. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
and for some reason, I get this (notice it went from: "Brand Name T. ASPIDE P. Product Category Name" - to: "Brand Name T. AsP.de p. Product Category Name"):

Code:
,SKU,"[S]Color=Black/White/Blue,[S]Size=44",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=46",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=48",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=50",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=52",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=54",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=56",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=58",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Blue,[S]Size=60",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=44",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=46",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=48",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=50",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=52",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=54",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=56",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=58",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Green/White,[S]Size=60",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=44",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=46",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=48",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=50",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=52",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=54",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=56",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=58",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/White/Red,[S]Size=60",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=44",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=46",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=48",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=50",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=52",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=54",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=56",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
,SKU,"[S]Color=Black/Anthracite/White,[S]Size=58",Brand Name T. AsP.de p. Product Category Name,1.51337E+11,Brand Name,,,1499.95,Option T. ASPIDE P.,,,,,,,,,,,,,,,,,,Product Category Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"1513374, T. ASPIDE P.,"
Strangely enough, this is the only product where this is the result; every other product gets translated correctly as far as I can tell.

On top of that, I'm then running this on the above result:

Code:
awk 'BEGIN{ 
        FS = ","    # input field seperator 
        OFS = ","   # output field seperator
}
     NR 4 {sub(/-/," "); print

}0' ./tmp/cased.csv > ./tmp/final.csv
And, while using the data subset above does not reproduce the problem, when I use all the data, I get duplicate rows one after the other, i.e.:

row 1
row 1
row 2
row 2
row 3
row 3
so
on
and
so
forth

... send help
 
Old 10-02-2011, 07:05 PM   #11
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,493

Rep: Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867
Right so my first question would be why you have a comma at the start of the file??

Error 1:

Yes it makes perfect sense. If you look at the original line (Just the part that matters):
Code:
Brand Name T. ASPIDE P. Product Category Name
Then consider the line in the script:
Code:
sub(arr[x],toupper(substr(arr[x],1,1))substr(arr[x],2),$5)
Remembering this is part of a loop that goes through each section of the line of input above
and that sub will change the first occurrence of a particular change.

So when it gets to the section, 'P.', the first time it finds 'P' followed by any character, this is due to sub
using the string as a computed regular expression, is in the word 'ASPIDE', hence
these letters are changed accordingly.

Error 2:

Please explain what the following piece of code is supposed to do?
Code:
NR 4 {sub(/-/," "); print

}0
I would also need to see a before and after example of what the new awk is expected to do?

Lastly, I would add that your data is not strictly comma separated as you have sections that are in quotes
which I would assume means these sections should be considered as one. This is more of a warning than anything as
it will not provide the results you require if you start to change elements within the quotes and expect
something different.

This challenge is easily overcome in the latest version of gawk but as you are on a MAC I can almost guarantee you
are no where near the latest version
 
Old 10-02-2011, 08:26 PM   #12
ivn
LQ Newbie
 
Registered: Sep 2011
Posts: 10

Original Poster
Rep: Reputation: Disabled
Thanks for your reply, I really appreciate your help!

First question: I start the data with a comma because I want an empty column before the SKU column.

Error 1:
Yikes. Okay, I understand what is happening now with the first part; I just don't really understand why. As you can probably already tell, all I'm pretty much doing is just scouring the internet to piece this script together, I'm really not familiar with what the heck is going on... (you lost me at: "...sub will change the first occurrence of a particular change.")

All I'm trying to do there in the first part is to convert the first letters of the words in that column to upper case.

Error 2:
What I'm trying to do there is remove the hyphen "-" from some words in the 4th column and replace with a space.

Example Before:
Light-Gray/Black
Dark-Gray/White

Desired Result:
Light Gray/Black
Dark Gray/Black

Regarding data delimitation: One of the columns needs to have a comma in it. Data in the column is structured:

[S]Color=COLOR-GOES-HERE,[S]Size=SIZE-GOES-HERE

Think I should try getting a gawk port for Mac OS X?
 
Old 10-02-2011, 09:21 PM   #13
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,493

Rep: Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867
Quote:
All I'm trying to do there in the first part is to convert the first letters of the words in that column to upper case.
Yes I understand but when what is being searched for in sub contains items that can be computed as a regular expression, ie the dot (.) in a regex means any character, then it will play havoc with what is being searched for. Probably your best solution here is to change the code to create a new field and assign it to the existing one:
Code:
awk 'BEGIN{ 
        FS = ","    # input field seperator 
        OFS = ","   # output field seperator
}
{$5 = tolower($5);
 n = split($5,arr," ");
 for(x=1;x <=n;x++)
     new = ((new)?" ":"")toupper(substr(arr[x],1,1))substr(arr[x],2);
 $5 = new
}1' ./tmp/translated.csv > ./tmp/cased.csv
I haven't tested the above but I hope you can see what it is trying to do?

The 4th column in the example you provided is of the form:
Code:
[S]Size=44
So this would imply you are working on the wrong column?

However, once you find the correct column, the following should work (I will assume 4th still):
Code:
awk 'sub(/-/," ",$4)' FS="," ./tmp/cased.csv > ./tmp/final.csv
Quote:
Think I should try getting a gawk port for Mac OS X?
You could try but I have been told it is not normally the case
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Search word and delete only the word and the line using Sed command kbmukesh Linux - Newbie 4 06-28-2011 06:35 AM
[SOLVED] Scripting Help: Using "File1" as list of objects to search for in "File2" Asharru Programming 10 09-05-2010 06:53 PM
Word list generation (all combinations) from a list of characters llattan Linux - Newbie 1 04-07-2009 07:41 PM
Replace word in VIM ICO Linux - Software 2 03-17-2004 01:28 AM
problem in perl replace command with slash (/) in search/replace string ramesh_ps1 Red Hat 4 09-10-2003 01:04 AM


All times are GMT -5. The time now is 12:13 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration