How do you write a magic file test pattern to match the end of a file?

arj · 02-10-2011, 04:26 PM

Hello all,

I am beginning to wonder if this is even possible as multiple searches on SO, Google, Bing and linuxquestions.org have turned up nothing. Basically I am interested in extending the magic patterns located in /usr/share/magic (used by the file(1) utility) to recognize files based on data at or near the end of the file. I have been able to do this for the beginning of a file, as well as for arbitrary offsets into the file from the beginning. The man page does a pretty good job of illustrating some standard usage cases; unfortunately, it does not seem like there is a way to index from the end as opposed to the beginning. The only workaround I could come up with was to adopt a scripted approach using tac and/or lreverse but feel these may be unfriendly to binary data. Also, I wanted to avoid any other scripted processing - I feel like this should be doable with the right file magic. Any ideas?

Thanks in advance,

-Arj

kbp · 02-10-2011, 05:49 PM

I doubt that it's possible, I had a muck around with this stuff for the DFRWS carving challenge. Most files won't have a special "trailer" to define the end, they usually have a header which may or may not define the length of the data/content, and this is what the magic signatures are based on.

hth

xeleema · 02-10-2011, 06:00 PM

Greetingz!

Well, looking at the manpage for "magic" out of the latest version of "file"....

Source: ftp.astron.com/pub/file/file-5.05.tar.gz
Released: 17jan11@19:47)

Code:

./configure --prefix=/home/luser/file-5.05-bin && \
make && \
make install && \
man -M /home/luser/file\-5.05\-bin/share/man magic

Seems like you can't really use regex...

Code:

              regex       A regular expression match in extended POSIX regular expression syntax
                          (like egrep). Regular expressions can take exponential time to process,
                          and their performance is hard to predict, so their use is discouraged.
                          When used in production environments, their performance should be care-
                          fully checked. The type specification can be optionally followed by
                          /[c][s].  The “c” flag makes the match case insensitive, while the “s”
                          flag update the offset to the start offset of the match, rather than the
                          end.  The regular expression is tested against line N + 1 onwards, where
                          N is the given offset.  Line endings are assumed to be in the machine’s
                          native format.  ^ and $ match the beginning and end of individual lines,
                          respectively, not beginning and end of file.

Maybe you could use "search"?

Code:

              search      A literal string search starting at the given offset. The same modifier
                          flags can be used as for string patterns. The modifier flags (if any)
                          must be followed by /number the range, that is, the number of positions
                          at which the match will be attempted, starting from the start offset.
                          This is suitable for searching larger binary expressions with variable
                          offsets, using \ escapes for special characters. The offset works as for
                          regex.

sunnydrake · 02-10-2011, 06:38 PM

just search one string
then > other
this is only a question of performance...
hmmm maybe -1 as offset will do?check source of file which function or code used to read/seek data .. maybe you can trick it.