[SOLVED] Combine files with offset

AdamD · 02-26-2013, 08:21 AM

Hi, my first post so hello everyone!

I'll get straight to the point - I have four files that I want to combine together but with an offset applied to each file. So fileA starts at 0x00, fileB starts at 0x9C4000, fileC starts at 0xCB2000 and fileD starts at 0xCC0000. What fills the gap between files is not important, what is important is the start byte location of each file. FileA is .SREC file, B is a .xmp file, C is .html file and D is .js file. The output file can output in the form of a text file.

I have tried "cat" but there doesn't seem to be an option to control how the files are concatinated.

Any help will be much appreciated

Regards
AdamD

shivaa · 02-26-2013, 08:47 AM

Could you show us some sample with dummy entries? Many commands and combination of commands can do this, but without visualizing the requirement, it's not appropriate to suggest anything.

AdamD · 02-26-2013, 10:00 AM

Hi Shivaa,

Many thanks for the reply.

The output file combining Files A,B,C,D should be in the form of the following. The gaps between each file start point can be filled with 0's. I hope this explains it a little better.

|0--------------------|10,240,000-----------|13,312,000-----------|13,369,344
[.....FILE A....]000..000[.....FILE B....]000..000[.....FILE C....]000..000[.....FILE D....]

FileA - start byte 0 - Filetype .SREC (this line not needed in output)
S02B0000533A2F50726F6A656374732F495349535F43686F70706572732F4D4B335F4D504455312F4649524DA7
S315000000009421FF609001002490410028BC61002CDF
S315000000107C0802A6900100107C0902A6900100143B
..."continues with many similar lines"
S315FFFFFFEC3C000000600020C47C0803A64E80002066
S309FFFFFFFC4BFFFFF0C4
S705FFFFFFFC01
000000000000000000000 - as many '0' required to fill file to 0x9c4000 or byte number 10,239,999

FileB start byte 10,240,00 - Filetype .xap (this is a bin file and hence characters do not display properly, this line not needed in output)
PK
p~A»{ lv
..."continues with many similar lines unable to display in this box"
@
000000000000000000000 - as many '0' required to fill file to 0xCB2000 or byte number 13,311,999

FileC start byte 13,312,000 - Filetype .html (this line not needed in output)
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" >

<head>
<META HTTP-EQUIV="Pragma" CONTENT="no-cache">
..."continues with many similar lines"
</a>
</object><iframe id="_sl_historyFrame" style="visibility:hidden;height:0px;width:0px;border:0px"></iframe></div>
</form>
</body>
</html>
000000000000000000000 - as many '0' required to fill file to 0xCC0000 or byte number 13,369,343

FileD start byte 13,369,344 - filetype .js (this line not needed in output)
//v2.0.30511.0
if(!window.Silverlight)window.Silverlight={};Silverlight._silverlightCount=0;Silverlight.__onSilverl ightInstalledCalled=false;Silverlight.fwlinkRoot="http
..."continues with many similar lines"
__installationEventFired){a();Silverlight.__installationEventFired=true}}catch(e){}return c}

Guttorm · 02-26-2013, 10:17 AM

Hi

It could be done with cat and truncate, something like this:

Code:

COMBINEDFILE=/tmp/bigfile
cat fileA > $COMBINEDFILE
truncate -s $((0x9C4000)) $COMBINEDFILE
cat fileB >> $COMBINEDFILE
truncate -s $((0xCB2000)) $COMBINEDFILE
cat fileC >> $COMBINEDFILE
truncate -s $((0x9C4000)) $COMBINEDFILE
cat fileC >> $COMBINEDFILE
truncate -s $((0xCC0000)) $COMBINEDFILE

Using truncate has the benefit that the file will become "sparse", so it wont take much disk space.

AdamD · 02-26-2013, 11:00 AM

Hi Guttorm,

Many thanks for the reply. I keep getting "truncate : command not found" when I run this. Has this function / prototype got to be included somehow? I am using RH5 if this helps.. If I "man truncate" it comes up with the doc for it but doesn't execute. I see it's in unistd.h and sys/types.h, any pointers???

Many thanks
AdamD

Guttorm · 02-26-2013, 11:04 AM

Hi

Sorry don't know about RH5. On Debian the truncate command is in a package called coreutils. Maybe use your package manager to search for it?

AdamD · 02-26-2013, 11:24 AM

Hi,

If I "whereis truncate" I get "truncate: /usr/share/man/man3p/truncate.3p.gz /usr/share/man/man2/truncate.2.gz"

So it seems I have the manual for it but it's a function that is not installed on our systems. Nothing I can do about that unfortunately but I'm sure it would have worked all the same. I'll have a chat with our IT guys and see if it can be installed. Also, it might be a permission thing..

Thanks for your reply though
Regards
AdamD

Guttorm · 02-26-2013, 12:00 PM

You could also use dd for this.

Code:

COMBINEDFILE=/tmp/bigfile
dd conv=notrunc bs=1 if=fileA of=$COMBINEDFILE seek=0 
dd conv=notrunc bs=1 if=fileB of=$COMBINEDFILE seek=$((0x9C4000))
dd conv=notrunc bs=1 if=fileC of=$COMBINEDFILE seek=$((0xCB2000))
dd conv=notrunc bs=1 if=fileD of=$COMBINEDFILE seek=$((0xCC0000))

AdamD · 02-27-2013, 02:51 AM

Hi Guttorm,

I think you have cracked it!

I tried it last night and it seems to have worked. I will try to program the file into the flash memory today to see if it locates the files in the correct places and let you know how it goes.

Thank you (Tusen Takk)

AdamD · 02-27-2013, 03:44 AM

Hi Guttorm,

Yes, just tried it and this works - Brilliant!!

Thanks again
AdamD