Software to Open 30GB binary file?

athomas · 08-11-2010, 12:57 PM

Is there such an application that can handle opening a 30GB binary file containing hex data? GHex won't open it.

Les Windoze · 08-11-2010, 01:07 PM

Perl can; what do you need to do w/ it?

athomas · 08-11-2010, 01:10 PM

One of the developers here needs to open it in GHex for some reason, so I'd imagine he wants to just open it like a text file. It's gov. work so I can't get too detailed.

GrapefruiTgirl · 08-11-2010, 01:25 PM

Hard to get good search results that don't say "filesize limit" - most programs seem to have a 2-4Gib limit. However, here's something you might like to look at: http://www.ultraedit.com/downloads/uex.html

That's UltraEdit, which is not a free program, but does claim to offer a "Free 30-day Trial", and claims to be a native Linux application that works with either Gnome or KDE desktop environments. There's also a .tar download which you would build and/or install if yours is not one of the supported distros. After the 30-day trial, you're to purchase a license, and hey, if it's the .gov they can afford it

Note: Normally I wouldn't suggest a non-free application, and there's *probably* a free alternative to this thing, but as I said, I am not seeing much in my search results that will deal with that large a file.

Best of luck!

athomas · 08-11-2010, 01:44 PM

UltraEdit won't open it either. :-/

David1357 · 08-11-2010, 02:27 PM

Quote:

Originally Posted by athomas

Is there such an application that can handle opening a 30GB binary file containing hex data? GHex won't open it.

If you type ":help xxd" in "vim", you will get this:

Code:

Hex editing                                     *hex-editing* *using-xxd*

See section |23.4| of the user manual.

If one has a particular extension that one uses for binary files (such as exe,
bin, etc), you may find it helpful to automate the process with the following
bit of autocmds for your <.vimrc>.  Change that "*.bin" to whatever
comma-separated list of extension(s) you find yourself wanting to edit:

        " vim -b : edit binary using xxd-format!
        augroup Binary
          au!
          au BufReadPre  *.bin let &bin=1
          au BufReadPost *.bin if &bin | %!xxd
          au BufReadPost *.bin set ft=xxd | endif
          au BufWritePre *.bin if &bin | %!xxd -r
          au BufWritePre *.bin endif
          au BufWritePost *.bin if &bin | %!xxd
          au BufWritePost *.bin set nomod | endif
        augroup END

Just change ".bin" to the extension of your file, run vim as "vim -b <filename>", and you will have an hex editor that can handle any file size.

athomas · 08-11-2010, 02:42 PM

Tried that, it just sits there looking at me. It's a VERY powerful machine, so resources aren't an issue at all.

jefro · 08-11-2010, 03:00 PM

Might end up tarring it or some other way to make the file contiguous. Then look at the hard drive sector by sector.

David1357 · 08-11-2010, 03:31 PM

Quote:

Originally Posted by athomas

Tried that, it just sits there looking at me. It's a VERY powerful machine, so resources aren't an issue at all.

It takes a very long time to convert the file to hex. Be prepared to wait several minutes, maybe longer.

Elv13 · 08-12-2010, 01:00 AM

Umm, why not just splitting the file in 200 parts using dd or anything else?

If the file contain a db or a disk image, there is no point to open it all at once anyway. When the job is done, put it back together with cat

Les Windoze · 08-12-2010, 08:47 AM

Quote:

Originally Posted by athomas

...It's gov. work so I can't get too detailed.

I would think if the developer knows what he's looking for, he could write something to seek for it and if found, split the file before and after the location then look at that part in GHex.

Split and cat make sense.

David1357 · 08-12-2010, 03:50 PM

Quote:

Originally Posted by athomas

Tried that, it just sits there looking at me. It's a VERY powerful machine, so resources aren't an issue at all.

After thinking about this problem, I ran aptitude and searched for "hexedit". I installed it and was able to open a 5.3 GB file very quickly.

Here is a link to the hexedit web site.

You might want to send a note to the author about changing his background to something easier on the eyes...