[SOLVED] Want to remove a few bytes from the header of several binary files

Zero Angel · 09-16-2011, 01:17 AM

I have a large set of binary wav files that I want to batch process. However the first few bytes of those files contains extraneous information that prevents the files from being opened in an audio program. So what I want to do is remove the first 12 bytes from each file, so that the file header becomes a vanilla Wave file header. I can do this in hex editors like Bless, but I don't know of one which can do it as a batch operation.

The exact hex characters I want to remove from the file are these ones:

Code:

66 73 73 6D 01 00 00 00 01 00 00 00

I figure one of you programming types would be able to help me with the problem.

Cheers.

elonica.pl · 09-16-2011, 01:45 AM

Did you try to play with dd ?

http://www.unix.com/shell-programmin...ytes-file.html
http://unix.stackexchange.com/questi...tart-of-a-file

corp769 · 09-16-2011, 01:59 AM

Hello,

You could always use dd to do this:

Code:

dd if=file.wav of=file_new.wav bs=1 skip=12

Of course, this would need to be modified, and you could write a script with this to process multiple files. Let me know if this would work for you.

Cheers,

Josh

Edit - It looks like I got beat!

Zero Angel · 09-16-2011, 02:02 AM

Actually, just as you posted I found a method to find/replace using perl. My test command succeeded:

Code:

perl -pe 's/\x{66}\x{73}\x{73}\x{6D}\x{01}\x{00}\x{00}\x{00}\x{01}\x{00}\x{00}\x{00}//g' < lb_sni_cap_snigen_lt_s_5201820a.smf > lb_sni_cap_snigen_lt_s_5201820a.wav

Now I need to find a way to make into a batch operation for all files in a given directory. Not sure how to do that as my scripting knowledge is fairly basic and i'm not well versed in for-each type commands.

Ramurd · 09-16-2011, 02:04 AM

No idea how this works on binary files, but you can give it a try with tail; something like this:

Code:

tail -c +12 infile > outfile

Zero Angel · 09-16-2011, 02:08 AM

I just tested corp769's solution and it seems to be the best one due to its simplicity. Now I need to find a way to do it in batch for every file in the folder. Can anyone help me with that -- or point me in the direction of a tutorial which will tell me how to batch commands like that?

corp769 · 09-16-2011, 02:08 AM

To save myself a bunch of typing, have a look here - http://www.cyberciti.biz/faq/bash-loop-over-file/
Looping through files in bash is a very simple task, you just need to mess with it so you get the proper output and filenames.

Zero Angel · 09-16-2011, 02:13 AM

This is enough information for me to write a shell script to automate this. Thank you once again corp.

corp769 · 09-16-2011, 02:15 AM

No problem dude. I would gladly help you make the script, but I'm at work, and sadly enough, the government has a contract with microsoft, and I'm stuck using windows for the time being....

Zero Angel · 09-16-2011, 02:34 AM

Code:

#!/bin/bash
for f in *.smf
do
  echo "Processing $f"
  dd if=$f of=`basename $f`.wav bs=1 skip=12
done

corp769 · 09-16-2011, 02:41 AM

Awesome dude, glad you got it!

David the H. · 09-16-2011, 01:40 PM

Good work. I'd only add a few points concerning the solution.

1) Always quote your variable expansions. Otherwise the shell will apply word-splitting to the contents, and your commands will break.

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes

Word splitting is a very important concept in scripting/shell use. Make sure you get it down.

2) Extracting substrings from variables can usually be done with parameter substitutions, making external tools like basename unnecessary most of the time.

http://mywiki.wooledge.org/BashFAQ/073

3) When you do need to use external tools, $(..) is highly recommended over `..`

Code:

#!/bin/bash

for f in *.smf ; do
  echo "Processing $f"
  dd if="$f" of="${f%.*}.wav" bs=1 skip=12
done