Greping special character and removing it
 

Hi,

I have a file which has certain records starting characters as ^@.
When viewed it through head command these characters are not visible, however in VI editor I can see them.

I need to find all such records and delete these characters from the begining of the records.

^@|accc|1||zzz|sss|fff|qqqweewrewert|ee|vvtytytyyyy|ss|dd|10.000|Y|10.000||||N||wwwww|||ddddd|zzzz||||

It shall be of a great help kindly assist on this.

Thnx
Vishal

I put that line into a file and then displayed it with head---the first two characters are there. Are you sure that there are not some non-printable characters in the file, and that VI is simply trying to decode them?

Using the line as you posted it, this works:

sed '/^\^@//' filename

Thnx pixellany,

These lines are part of a file generated through some tool. I believe these are non printable lines and shows up in a different color in VI. Now I need to load these records through a tool and it catched these records while loading.

Kindly assist

Thnx
Vishal

First you can try to determine where these hidden characters come from. For example if these files have been edited/created in windows environment, the dos2unix command could help.

In any case you can try to remove them using sed as in:
this removes all control characters from the beginning of each line. The -i.bck option makes sed to edit the file in place but does a backup of the original file, named file.bck.

Another chance is that you determine the octal code of these characters (using the commands od or hexdump) and delete them by means of tr. But this would be a last resort, since it would require a lot of manual work and it would be quicker to remove control characters directly in vi!

Hope this helps.

my colleague was typing while I was attempting to think....;)

Note also that you can use sed to remove special characters by entering the hex code---eg sed 's/\x40//' removes the @ in the data you supplied.

You can use hexdump -C to see what's actually there.