LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices



Reply
 
Search this Thread
Old 02-17-2012, 05:10 PM   #1
Heraton
Member
 
Registered: Apr 2011
Location: Germany
Distribution: Mint 10, openSuSE
Posts: 58

Rep: Reputation: 3
Question How to replace 10th and 11th byte in a hex dump


Hello everybody!

I have a feeling that there is a very simple and efficient solution to my task I simply forgot. If I am wrong and you are a mod would you move this to Programming please?


I am working with hex dump looking like that
Code:
01 74 58 d9 00 1e 4c 08 00 21 85 69 57 40 00 50 ea 74 58 d9 00 5e 6d cb 61 74 31 70 ...
I look for a filter statement to replace the n th byte of a long line of data. I can tell n in advance, but would prefer not to hard code it.

My intended usage is
Code:
prog1 | missing_command | prog2
Thanks a lot for reading!



Regards, Heraton
 
Old 02-17-2012, 06:42 PM   #2
Thor_2.0
Senior Member
 
Registered: Nov 2007
Location: Somewhere on my hard drive...
Distribution: Manjaro
Posts: 2,228
Blog Entries: 23

Rep: Reputation: 279Reputation: 279Reputation: 279
You could consider the OR operation....

Quote:
1010101
OR
0001000
=
1011101
Where the selected bit (the one in the middle) was replaced...

Just a flake of a thought...

Thor
 
Old 02-17-2012, 07:05 PM   #3
zer0python
Member
 
Registered: Sep 2003
Posts: 104

Rep: Reputation: 20
Hope this helps

Code:
~[lizard]$ echo $example
01 74 58 d9 00 1e 4c 08 00 21 85 69 57 40 00 50 ea 74 58 d9 00 5e 6d cb 61 74 31 70
~[lizard]$ missing_command() { local n=$1 r=$2; sed -r "s/(.{$((3*$(($n-1))))})..(.*)/\1$r\2/"; }
~[lizard]$ echo $example | missing_command 1 99
99 74 58 d9 00 1e 4c 08 00 21 85 69 57 40 00 50 ea 74 58 d9 00 5e 6d cb 61 74 31 70
~[lizard]$ echo $example | missing_command 2 ff
01 ff 58 d9 00 1e 4c 08 00 21 85 69 57 40 00 50 ea 74 58 d9 00 5e 6d cb 61 74 31 70
~[lizard]$ echo $example | missing_command 10 cc
01 74 58 d9 00 1e 4c 08 00 cc 85 69 57 40 00 50 ea 74 58 d9 00 5e 6d cb 61 74 31 70
 
1 members found this post helpful.
Old 02-17-2012, 07:59 PM   #4
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,154

Rep: Reputation: 333Reputation: 333Reputation: 333Reputation: 333
Here's a simple gawk one-liner to do it:
Code:
$ echo "01 74 58 d9 00 1e 4c 08 00 21 85 69 57 40 00 50 ea 74 58 d9 00 5e 6d cb 61 74 31 70" | gawk -v n=5 -v new=FF '{print $0;$n=new;print $0}' | cat
01 74 58 d9 00 1e 4c 08 00 21 85 69 57 40 00 50 ea 74 58 d9 00 5e 6d cb 61 74 31 70
01 74 58 d9 FF 1e 4c 08 00 21 85 69 57 40 00 50 ea 74 58 d9 00 5e 6d cb 61 74 31 70
Notes:
  1. In that example, "prog1" is echo and "prog2," cat
  2. The two -v settings are where you specify the byte number (n) and the replacement byte (new). If you were to put this in a bash script, you could use a syntax like -v n=${byte_number} -v new=${replacement_byte}
  3. The print $0; is in there just to show the "before" value, a debugging check. Remove it after you test.
 
1 members found this post helpful.
Old 02-18-2012, 08:08 AM   #5
Heraton
Member
 
Registered: Apr 2011
Location: Germany
Distribution: Mint 10, openSuSE
Posts: 58

Original Poster
Rep: Reputation: 3
Thanks a lot!

Thank you very much for your replies!

In fact both the sed as well as the awk example do perform very well, although I have to admit that the sed thing is out of my league. The awk thing rings a bell and looks pretty familiar, so I guess that was, what I had seen before.

Although I already have figured out some ways to automate replacing 4 or 6 byte in a line of hex dump, I assume I am doing things that would let more advanced scripting people run away in sheer terror.


I could pipe things 4 to 6 times, which solves the problem nicely but is anything but high performance...
Code:
cat Data | awk_repl <n> <Val1> | awk_repl <n+1> <Val2> | \
awk_repl <n+2> <Val3> | awk_repl <n+3> <Val14> | cat
I could also use specialised scripts each:
Code:
#!/bin/bash
n2=$(($1+1))
n3=$(($n2+1))
n4=$(($n3+1))

gawk -v p1=$1 -v new1=$2 -v p2=$n2 -v new2=$3 -v p3=$n3 -v new3=$4 -v p4=$n4 -v new4=$5 \
$p1=new1;$p2=new2;$p3=new3;$p4=new4;print $0}'
But those scripts look crude and I would have to write one for each length or go all the way to construct the awk expression depending on a length parameter. Although this sounds mad enough to try once, I don't believe that is the way it is meant to be. So basically what I am still longing for is a way to collapse "-v p1=$1 -v new1=$2 ... p4=$n4 -v new4=$5" to something like "from position n to n+3 replace with something that fits in"

Any suggestions would be highly appreciated.


Regards, Heraton

Last edited by Heraton; 02-18-2012 at 08:10 AM. Reason: added " \ " to the piping code
 
Old 02-18-2012, 02:28 PM   #6
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,154

Rep: Reputation: 333Reputation: 333Reputation: 333Reputation: 333
Well, you could do it in a few different ways. If you only want to cover the case of a set of contiguous lines, this might work:
Code:
gawk -v start=5 -v new=A1:B2:C3 'BEGIN{n=split(new,New,":")}{for(i=1;i<=n;++i)$(i+start-1)=New[i];print $0}
Here's that applied to your sample string:
Code:
$ echo "01 74 58 d9 00 1e 4c 08 00 21 85 69 57 40 00 50 ea 74 58 d9 00 5e 6d cb 61 74 31 70" |gawk -v start=5 -v new=A1:B2:C3 'BEGIN{n=split(new,New,":")}{for(i=1;i<=n;++i)$(i+start-1)=New[i];print $0}'
01 74 58 d9 A1 B2 C3 08 00 21 85 69 57 40 00 50 ea 74 58 d9 00 5e 6d cb 61 74 31 70
(Note that, in this simple case, the redundant pipe to cat has been omitted.)

A more general solution would be to prepare a table of byte numbers and new values, read that in, and then apply it to each line of the input file. (You could even use a three column input table with {byte number, old byte, new byte}, and only replace the bytes when all the old bytes match.) That would be possible as a one-line program, but it would be a long line. Here's such a program written so you could put it in a file, make it executable, and use it as you middle command. (Note that the name of the pattern table is hard codded. It could be passed as a variable, but I was getting lazy.)

N.B.: This is untested code.
Code:
#!/usr/bin/gawk -f
#
# Read in the conversion table
#  byte expected_value new_value
# ...
BEGIN {
  N=0
  while (getline value < "byte_map_table") {
    ++N
    if (n=split(value,Value) != 3) {
      printf("Error: Line $d of the byte map table (%s) contained %d values, not three.\n",N,value,n)  > "/dev/stderr"
      ++error
      continue
    }
    Byte[N]=Value[1]
    Old[N]=Value[2]
    New[N]=Value[3]
  }
  if (error) exit(1)
}

{
  skip=0
  for (i in Byte) {
    if (Byte[i] < 1 || Byte[i] > NF) {
      printf("Warning: Record %d does not contain a byte in column %d. Skipping it.\n", NR, Byte[i]) > /dev/stderr
      skip=1
    }
    if (tolower($i) != tolower(Old[i])) {
      printf("Warning: Record %d: Byte %d is %s, not $s. Skipping it.\n",NR,i,$i,Old[i]) > "/dev/stderr
      skip=1
    }
  }
  if (!skip) {
    for (i in Byte) {
      $i=New[i]
    }
  }
  print $0
}
 
1 members found this post helpful.
Old 02-18-2012, 09:12 PM   #7
Heraton
Member
 
Registered: Apr 2011
Location: Germany
Distribution: Mint 10, openSuSE
Posts: 58

Original Poster
Rep: Reputation: 3
Thumbs up Thanks a lot!

Thank you very much for your answer. Once more your solution is perfectly suitable for my task. For now I'll stick with your first proposal as I need to meet a deadline which expires on Thursday and I am already taking night shifts. But I promise to take a closer look at your second proposal next weekend, as I am sure I can learn quite a lot of that.

Regards, Heraton
 
Old 02-19-2012, 09:03 PM   #8
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.6, Centos 5.10
Posts: 16,324

Rep: Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041Reputation: 2041
Just as an alternative, you might like to know that substr() in Perl can also do assignments as well as extractions; neat huh
http://perldoc.perl.org/functions/substr.html

See also pack & unpack
http://perldoc.perl.org/functions/pack.html
http://perldoc.perl.org/functions/unpack.html
 
1 members found this post helpful.
Old 02-20-2012, 01:11 PM   #9
Heraton
Member
 
Registered: Apr 2011
Location: Germany
Distribution: Mint 10, openSuSE
Posts: 58

Original Poster
Rep: Reputation: 3
Smile Good to know

Thanks for that advice. I guess I will take a look at that soon. Right now I have to meet a deadline and Perl would not be compatible with my intended audience. Anyway, I guess enlightment is waiting down that road too.

Regards, Heraton
 
  


Reply

Tags
manipulation, pipeline, string


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Looking for tcpdump syntax to start hex output at a given byte offset in the packet silentquasar Linux - Networking 1 08-06-2011 07:03 AM
[SOLVED] converting byte in hex in kernel ano Linux - Kernel 3 12-21-2010 11:52 AM
[SOLVED] How do I get a simple hex dump in gdb gung Linux - Software 3 05-03-2010 12:38 AM
LXer: Doing a Reverse Hex Dump LXer Syndicated Linux News 0 06-03-2008 03:50 AM
strange hex dump of floppy Berhanie Linux - General 2 02-07-2004 12:53 PM


All times are GMT -5. The time now is 12:43 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration