Visit Jeremy's Blog.
Go Back > Forums > Non-*NIX Forums > Programming
User Name
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.


  Search this Thread
Old 06-09-2011, 03:17 PM   #1
LQ Newbie
Registered: Dec 2006
Posts: 6

Rep: Reputation: 0
Replace character with hex value of my choosing from shell cmd line? (Solaris 5, ksh)

Hi everyone. I have a rather odd task I am trying to accomplish.

I am bouncing a file across platforms (windows->solaris->mainframe), and the file is starting out with a "special" character (the registered trademark "circle R") in some of the records. This character is not in the EBCDIC character set on the MF, so it is unrecognizable.

The MF developer I am working with asked if it is possible to replace the character with a specific hex value (AF) before it gets to the MF.

I was putzing around with sed, tr, etc. on the ksh command line, hoping to find an easy way to get one of them to substitute hex instead of ASCII. I have found that the usual shell utilities recognize the trademark character, so homing in on what to replace is solved. But I cannot get anything to actually substitute in the hex sequence I want. E.g. I was thinking something like...

>cat special_file | sed 's/R/\0xAF/g'

But my version of sed does not seem to have hex "editing" capability.

Any thoughts? Thanks in advance.
Old 06-09-2011, 05:19 PM   #2
Registered: May 2002
Location: dracut MA
Distribution: Fedora, RHEL
Posts: 600

Rep: Reputation: 109Reputation: 109
man tr
Old 06-09-2011, 05:22 PM   #3
David the H.
Bash Guru
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958
You might try using iconv instead. This is just the kind of situation it was designed for.
iconv -f UTF-8 -t EBCDIC-US//TRANSLIT file
The TRANSLIT option means it will attempt to replace unsupported characters with supported substitutes. There's also an IGNORE option, which will simply drop them instead.
Old 06-09-2011, 05:36 PM   #4
LQ Guru
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671
The command you posted changes capital R's to 0xAF. Was that just a problem posting it here?

I think you need to use `\xAF' instead of `\0xAF' in sed.

The tr command can also do what you want as a previous poster has stated.

Last edited by jschiwal; 06-09-2011 at 05:48 PM.
Old 06-10-2011, 11:39 AM   #5
LQ Newbie
Registered: Dec 2006
Posts: 6

Original Poster
Rep: Reputation: 0
Thanks to David and jschiwal for the real replies. It appears that my installation of Solaris does not have the EBCDIC character set for iconv. MY /usr/lib/iconv/iconv_data file is as follows:
646 8859 8859.646 8859.646.t
646de 8859 646de.8859 646de.8859.t
646da 8859 646da.8859 646da.8859.t
646en 8859 646en.8859 646en.8859.t
646fr 8859 646fr.8859 646fr.8859.t
646it 8859 646it.8859 646it.8859.t
646es 8859 646es.8859 646es.8859.t
646sv 8859 646sv.8859 646sv.8859.t
8859 646 8859.646 8859.646.t
8859 646de 8859.646de 8859.646de.t
8859 646da 8859.646da 8859.646da.t
8859 646en 8859.646en 8859.646en.t
8859 646fr 8859.646fr 8859.646fr.t
8859 646it 8859.646it 8859.646it.t
8859 646es 8859.646es 8859.646es.t
8859 646sv 8859.646sv 8859.646sv.t

So I could not get iconv to work. And you are correct, sed does not need \0x, it just needs \x (I just typed that command off the top of my head). Unfortunately, it appears that the variant of sed on our system does not have hex substitution in the first place.

I did manage to get tr to work, at least on the Solaris side. After running the file through tr and looking at a hex dump (via od) of the result, it appears that the desired hex value was substituted in (if one uses the correct octal equivalent in the tr command). But when I transfer the file to the mainframe and look at the raw hex there, it seems that the file is being modified by the transfer. I guess if I'm ftp'ing in ASCII mode, there's no reason to expect the non-printable characters to make it through intact.

Anyway, thanks again for the real replies. The experimentation continues...

Last edited by discomurder; 06-10-2011 at 11:40 AM.
Old 06-11-2011, 05:03 PM   #6
LQ Guru
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671
Looking in the sed info file, I see it is a GNU extension.
I also looked in the 1p manpage for the `tr' command. Then I was just lookin at the posix man page. You should be able to translate the character if you use octal codes. 0xAF = \0257.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] sed help - replace line feed with different character bradvan Programming 7 04-22-2012 11:31 PM
how to replace line of file with another line using awk in shell script amit_pansuria Programming 3 03-29-2009 09:43 AM
how to search and replace character from middle of line vishal_titre Linux - General 7 09-30-2008 02:46 AM
How to identify a line and replace another string on that line using Shell script? Sid2007 Programming 10 10-01-2007 08:49 PM
Perl - Tpl file - Need to replace new line character. knnirmal Programming 2 09-07-2004 02:27 PM

All times are GMT -5. The time now is 10:39 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration