LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 02-27-2009, 05:46 PM   #1
SpecB
LQ Newbie
 
Registered: Feb 2009
Posts: 4

Rep: Reputation: 0
How to edit a file hexadecimally using bash?


I have some (256) files that I want to edit, modifying certain bytes of them. One-by-one method is out of the question since it would take hours, but I also can't seem to find a method of editing the file in hex. I know the command hexdump, but it doesn't really help, or at least I didn't find out how yet.
Someone advised me to use vi and its command :%!xxd to edit in hex, but then again this method would require me to modify each and every file one by one. Maybe if I could use vi in a script it would work, but I didn't find any useful information about that either.
 
Old 02-27-2009, 06:11 PM   #2
jlinkels
Senior Member
 
Registered: Oct 2003
Location: Bonaire
Distribution: Debian Lenny/Squeeze/Wheezy/Sid
Posts: 4,067

Rep: Reputation: 491Reputation: 491Reputation: 491Reputation: 491Reputation: 491
Use tr. It is perl script, but included in most distributions. You can specify bytes in octal to search/replace.

For example to replace the sequence 0x49 0x44 0x33 with 0x2d0x2d0x2d you give the command:

Code:
cat oldfile | tr "\111\104\063" "\055\055\055"  > newfile
jlinkels
 
Old 02-27-2009, 07:14 PM   #3
SpecB
LQ Newbie
 
Registered: Feb 2009
Posts: 4

Original Poster
Rep: Reputation: 0
Hmmm, I knew about tr, but didn't know I could use it this way too.
It worked well, thank you!
 
Old 02-27-2009, 07:36 PM   #4
wje_lq
Member
 
Registered: Sep 2007
Location: Mariposa
Distribution: Debian lenny, Slackware 12
Posts: 808

Rep: Reputation: 178Reputation: 178
jlinkels gave a good solution, but in case someone gets the wrong idea:

Quote:
It is perl script
Not so much, at least in some distributions. In stock Slackware 12.1, I get this:
Code:
wally:~$ file -L $(which tr)
/usr/bin/tr: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), stripped
wally:~$
and Xubuntu (Jaunty Jackelope) gives a similar result.

If there are distributions which implement tr as a Perl script (which would be trivial), I'd be curious to know which ones.
 
Old 02-28-2009, 03:29 AM   #5
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269
Interesting, but I'm wondering if you could do this with some set space between offsets, and if it can only change the first instance. I needed to do this at one point and ended up writing a C program to do it. For example say you need to match 0x49 then X bytes later you need to match 0x44 and Y bytes later you need to match 0x33, and if all these 3 match then you change each of them to something else. Can that be done or is it too complicated.

Either way, I was just looking for a simpler solution, but I still have the code I wrote for that program, so if you need it say so.

Last edited by H_TeXMeX_H; 02-28-2009 at 03:32 AM.
 
Old 02-28-2009, 04:27 AM   #6
SpecB
LQ Newbie
 
Registered: Feb 2009
Posts: 4

Original Poster
Rep: Reputation: 0
Well maybe you could "jump through" the file's bytes using two cycles for the rows and columns and maybe "cut" or something with a similar effect, setting X and Y as a parameter, and if the bytes are found in the places needed, it could change them.
I didn't have this problem because the filetype is very simple and can be edited using an external program. I set all of the desired bytes to a basically unused number, then just looked for it and changed all of them using the command aforementioned.
(Although I had some problems with the naming because for some reason the script added a ^M non-printable character to the end of the file extensions, probably due to my lazyness at 2am to write a proper cycle and not just use excel's CONCATENATE to generate the needed commands line by line, heh)
 
Old 02-28-2009, 06:14 AM   #7
jlinkels
Senior Member
 
Registered: Oct 2003
Location: Bonaire
Distribution: Debian Lenny/Squeeze/Wheezy/Sid
Posts: 4,067

Rep: Reputation: 491Reputation: 491Reputation: 491Reputation: 491Reputation: 491
Quote:
Originally Posted by wje_lq View Post
jlinkels If there are distributions which implement tr as a Perl script (which would be trivial), I'd be curious to know which ones.
Arghhh... in Debian Lenny it is an executable, so to see in Etch as well. I am sure I have seen it before in Debian as a Perl script, I looked at the source. For sure.

jlinkels
 
Old 05-03-2009, 06:18 AM   #8
SpecB
LQ Newbie
 
Registered: Feb 2009
Posts: 4

Original Poster
Rep: Reputation: 0
Code:
cat oldfile | tr "\111\104\063" "\055\055\055"  > newfile
While this code works, I've managed to find a weakness of its: in this specific example, it will change every instance of 0x49, 0x44 and 0x33 into 0x2d, whether or not they are together. But what if, lets say, I only want to change the 0x44 bytes IF they are preceeded by 0x49 and followed by 0x33?
The server I'm contacting to be able to use linux lags like hell, so I can't really try out my ideas, but am I right if I think that a regular expression should do the job?

Last edited by SpecB; 05-03-2009 at 06:19 AM.
 
Old 05-03-2009, 06:38 AM   #9
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269
As I said before a while ago I wrote a program to patch a game in this way ... it's kind of a hack:

Code:
// fix game.exe

#include <stdio.h>
#include <stdlib.h>

int main (void) 
{
	// ! init
	FILE * gfile;
	long fsize;
	char * buffer;
	size_t result;
	
	// init
	unsigned int i = 0;

	// open file read binary
	gfile = fopen ( "game.exe" , "rb" );
	if ( gfile == NULL ) { fputs ("ERROR: failed to open file game.exe !",stderr); exit (1); }

	// obtain file size
	fseek (gfile , 0 , SEEK_END);
	fsize = ftell (gfile);
	rewind (gfile);

	// allocate memory to contain file:
	buffer = (char*) malloc (fsize);
	if ( buffer == NULL ) { fputs ("ERROR: not enough memory to contain file game.exe !",stderr); exit (2); }

	// copy file into buffer
	result = fread (buffer,1,fsize,gfile);
	if ( result != fsize ) { fputs ("ERROR: file game.exe cannot be read !",stderr); exit (3); }

	// close file
	fclose (gfile);

	// search
	for ( i=0; i < fsize; i++)
	{
		if ( buffer[i] == '\x62' && buffer[i+1] == '\x02' && buffer[i-4729] == '\x01')
		{
			printf("The offsets are: %x %x %x\n", i-4729, i, i+1);
			printf("Fixing ...\n");
			buffer[i-4729] = '\x00';
			buffer[i] = '\x55';
			buffer[i+1] = '\x07';
			
			// open file write binary
			gfile = fopen ( "game.exe" , "wb" );
			if ( gfile == NULL ) { fputs ("ERROR: failed to open file game.exe !",stderr); exit (1); }
	
			// write file
			result = fwrite (buffer , 1 , fsize , gfile);
			if ( result != fsize ) { fputs ("ERROR: file game.exe cannot be written !",stderr); exit (3); }
	
			// close file
			fclose (gfile);
			
			break;
		}
	}
		
	// end
	free (buffer);
	return 0;
}
The important part is this statement:
Code:
if ( buffer[i] == '\x62' && buffer[i+1] == '\x02' && buffer[i-4729] == '\x01')
it checks the current index for x62, then the next byte for x02, then 4729 bytes back for x01, so change this statement as you see fit. I'm not a professional programmer and wrote this up in about 1 hour some time ago. I also hardcoded the file name as game.exe, this was done for safety reasons, because it is a dangerous program.

EDIT:
Forgot to mention that the bytes are changed here:
Code:
			buffer[i-4729] = '\x00';
			buffer[i] = '\x55';
			buffer[i+1] = '\x07';
And that it breaks out of the loop after it find one match.

Last edited by H_TeXMeX_H; 05-03-2009 at 07:02 AM.
 
  


Reply

Tags
edit, file, script, vi


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
bash to edit one line of a file Frayed Programming 8 04-22-2008 07:13 AM
[bash] edit lines in a file pieperp Programming 4 01-31-2007 08:56 AM
how can i EDIT file by BASH Guru Mind Programming 27 03-28-2006 03:38 AM
Perl or Bash Mass File Edit redneon Programming 1 10-14-2004 11:43 AM
how to edit file from bash commands legend1079 Linux - Newbie 3 03-03-2004 01:59 PM


All times are GMT -5. The time now is 01:46 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration