special charter at bunch of .csv i want to run script with for loop but not removing in single command it is identifying but not removing

dhirendrs · 09-01-2017, 04:15 AM

Dear all,

I am using Centos and with below command i am finding special character in file but i need to go line by line and remove character and then save file and again need to run same command i have arround 100+ files it is very much difficult to do activity for each file one by one so i try to write script with same command but not working .

==================================================================
#!/bin/bash

find ./ -name "*.csv" -o -name "*.html" -o -name "*.css" -o -name "*.js" -type f |
while read file
do
echo " $file"
mv $file $file.icv
iconv -f UTF-8 -t ISO-8859-15 $file.icv > $file
rm -f $file.icv
done

Command :
iconv -f UTF-8 -t ISO-8859-15 test.csv > tempfile
Above file not able to remove bunch of special character from all files

Turbocapitalist · 09-01-2017, 04:30 AM

Are you sure it is finding the files correctly? Boolean logic dictates a specific precedence for logical AND and OR. So grouping is necessary to get the effect it looks like you are aiming for:

Code:

find ./ \( -name "*.csv" -o -name "*.html" -o -name "*.css" -o -name "*.js" \) -type f

About the iconv use, adding -c should remove any characters that cannot be converted.

dhirendrs · 09-01-2017, 04:53 AM

not working

Turbocapitalist · 09-01-2017, 04:56 AM

Then you'll have to define more clearly what you mean by special characters and which ones you want removed. iconv converts and if the characters are still there after the conversion, then they were successfully converted which appears to be something other than what you want.

Have you looked at tr instead? See especially the -c and -d options. I'm not sure how well it handles UTF-8 though.

NevemTeve · 09-01-2017, 06:17 AM

kindly use some punctuation in your typing so that is would look like a piece of text not just a sequence of words also instead of not working please quote the exact error message

Code:

find ./ \( -name "*.csv" -o -name "*.html" -o -name "*.css" -o -name "*.js" \) -type f |
while read file
do
    if iconv -f UTF-8 -t ISO-8859-15 "$file" >"$file.ocv"; then
        printf "Converted '%s'\n" "$file"
        mv "$file.ocv" "$file"
    else
        rm -f "$file.ocv"
        printf "*** Couldn't convert '%s'\n" "$file"
    fi
done

rtmistler · 09-01-2017, 06:30 AM

What I would recommend is finding the files which prove to be the exception to the iconv call and focusing on only those.

astrogeek · 09-01-2017, 02:38 PM

Please place your code snippets inside [CODE]...[/CODE] tags for better readability. You may type those yourself or click the "#" button in the edit controls.

"not working" is not a useful description of the problem. Please explain how it is not working and include any error messages.

If some files seem to produce bad results while others do not, it woul dbe helpful to pick a specific example and try to identify why it fails. If possible reduce the file to a single line or a few lines which cause the failure and post that example here, along with the exact filename and the command you are using.