Download your favorite Linux distribution at LQ ISO.
Go Back > Forums > Linux Forums > Linux - Software
User Name
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.


  Search this Thread
Old 02-10-2011, 03:26 PM   #1
LQ Newbie
Registered: Feb 2011
Posts: 1

Rep: Reputation: 0
How do you write a magic file test pattern to match the end of a file?

Hello all,

I am beginning to wonder if this is even possible as multiple searches on SO, Google, Bing and have turned up nothing. Basically I am interested in extending the magic patterns located in /usr/share/magic (used by the file(1) utility) to recognize files based on data at or near the end of the file. I have been able to do this for the beginning of a file, as well as for arbitrary offsets into the file from the beginning. The man page does a pretty good job of illustrating some standard usage cases; unfortunately, it does not seem like there is a way to index from the end as opposed to the beginning. The only workaround I could come up with was to adopt a scripted approach using tac and/or lreverse but feel these may be unfriendly to binary data. Also, I wanted to avoid any other scripted processing - I feel like this should be doable with the right file magic. Any ideas?

Thanks in advance,

Old 02-10-2011, 04:49 PM   #2
Senior Member
Registered: Aug 2009
Posts: 3,790

Rep: Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653Reputation: 653
I doubt that it's possible, I had a muck around with this stuff for the DFRWS carving challenge. Most files won't have a special "trailer" to define the end, they usually have a header which may or may not define the length of the data/content, and this is what the magic signatures are based on.

Old 02-10-2011, 05:00 PM   #3
Registered: Aug 2005
Location: D.i.t.h.o, Texas
Distribution: Slackware 13.x, rhel3/5, Solaris 8-10(sparc), HP-UX 11.x (pa-risc)
Posts: 988
Blog Entries: 4

Rep: Reputation: 254Reputation: 254Reputation: 254

Well, looking at the manpage for "magic" out of the latest version of "file"....

Released: 17jan11@19:47)

./configure --prefix=/home/luser/file-5.05-bin && \
make && \
make install && \
man -M /home/luser/file\-5.05\-bin/share/man magic
Seems like you can't really use regex...
              regex       A regular expression match in extended POSIX regular expression syntax
                          (like egrep). Regular expressions can take exponential time to process,
                          and their performance is hard to predict, so their use is discouraged.
                          When used in production environments, their performance should be care-
                          fully checked. The type specification can be optionally followed by
                          /[c][s].  The “c” flag makes the match case insensitive, while the “s”
                          flag update the offset to the start offset of the match, rather than the
                          end.  The regular expression is tested against line N + 1 onwards, where
                          N is the given offset.  Line endings are assumed to be in the machine’s
                          native format.  ^ and $ match the beginning and end of individual lines,
                          respectively, not beginning and end of file.
Maybe you could use "search"?
              search      A literal string search starting at the given offset. The same modifier
                          flags can be used as for string patterns. The modifier flags (if any)
                          must be followed by /number the range, that is, the number of positions
                          at which the match will be attempted, starting from the start offset.
                          This is suitable for searching larger binary expressions with variable
                          offsets, using \ escapes for special characters. The offset works as for
Old 02-10-2011, 05:38 PM   #4
Registered: Jul 2009
Location: Kiev,Ukraine
Distribution: Ubuntu,Slax,RedHat
Posts: 289
Blog Entries: 1

Rep: Reputation: 61
just search one string
then > other
this is only a question of performance...
hmmm maybe -1 as offset will do?check source of file which function or code used to read/seek data .. maybe you can trick it.

Last edited by sunnydrake; 02-10-2011 at 05:41 PM.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] file Renaming based on a pattern match Guyverix Programming 6 01-10-2011 04:40 PM
Bash read each file in a directory and match with pattern threeonethree Programming 25 11-29-2010 11:30 AM
Bash test if pattern match of file exists bioinformatics_guy Linux - Newbie 4 04-17-2009 06:42 AM
How to delete the file using pattern match? nishanthhampali Programming 3 04-16-2008 12:50 PM
how to use the sed w option to redirect pattern match to file nickleus Linux - General 11 04-18-2006 08:34 AM > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 09:45 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration