Checking Downloaded Files

Konphine · 02-13-2012, 05:09 PM

Hello,

I've been wanting to check files that I download before I run them on my Windows computer (basically a few *.exe files). I was reading:

http://wiki.linuxquestions.org/wiki/..._into_an_image

If people can hide data this way, couldn't it be a security problem (due to the user's fault)? So my basic question is how would I be able to check files to make sure they weren't concatenated like this or by some other means?

T3RM1NVT0R · 02-13-2012, 05:32 PM

Hi Konphine,

Not sure how you will identify that in Windows but in linux there is a command called "file" by which you can find out the type of that particular file.

For example if I have renamed test.png file to test.txt file and then run the following command:

Code:

file test.txt

It will still show me the file type as PNG image data file :-)

Ofcourse if the file is zipped then you first have to unzip it to find out the file type of the file that was zipped.

TobiSGD · 02-13-2012, 05:39 PM

The file utility will not get the changes made to such altered files (@T3RM1NVT0R: In this case it would have been a good idea to read that article before posting).
I can't think of a method to recognize such files, so I would think the best way to avoid such files is to download files only from trusted sites.

Konphine · 02-13-2012, 06:05 PM

I see. To answer your question T3RM1NVT0R, I download the *.exe's on Linux first before I execute them in Windows.

T3RM1NVT0R · 02-13-2012, 06:15 PM

Hi TobiSGD,

Yes, you are right that it is difficult to identify the type of such files using file command.

I have read that document earlier and tried the steps mentioned in the document (only linux part). If I follow it step by step and at the end when I try to unzip it here is what I get:

Quote:

Archive: output.zip
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of output.zip or
output.zip.zip, and cannot find output.zip.ZIP, period.

This is what I was getting when I tried it. So I created that zip file and put an image file in that. Then run the cat command as mentioned in that document. It worked as mentioned in the document!!!.

Running file on that file keep telling me that it is an image file when it was not. However, when I ran filefrag on that I could find the inconsistency in that file.

So is filefrag good to find out such files? Because if you know that the file has got some problem then it is better not to run them. Avoiding download from untrusted is the best way. Thing that I would like to know is filefrag good utility to find out if something is wrong with the file even if downloaded from trusted source?

Konphine · 02-15-2012, 10:46 PM

I'm bumping this in order to try and see if T3RM1NVT0R's question can get answered:

Quote:

So is filefrag good to find out such files?

Reuti · 02-16-2012, 10:15 AM

How is the fragmentation of a file related to its content? I don’t see that filefrag would help here, or are referring to something different than the e2fs tool?

T3RM1NVT0R · 02-16-2012, 03:03 PM

@ Reuti

I was thinking if file fragmentation is affected if we concatenate the files the way mentioned in OP's link? This is what I get when I run filefrag on .iso file or a zip file or a tar file:

On iso:

Code:

Filesystem type is: ef53
File size of linuxmint-11-lxde-cd-32bit.iso is 704512 (172 blocks, blocksize 4096)
 ext logical physical expected length flags
   0       0   361728             172 eof
linuxmint-11-lxde-cd-32bit.iso: 1 extent found

On tar file:

Code:

Filesystem type is: ef53
File size of MyNetwork.tar is 163840 (40 blocks, blocksize 4096)
 ext logical physical expected length flags
   0       0  4325769              40 eof
MyNetwork.tar: 1 extent found

When I run the same command on the zip file created as per the procedure mentioned in the OP's link I get the following:

Code:

Filesystem type is: ef53
File size of output.zip is 1398231 (342 blocks, blocksize 4096)
 ext logical physical expected length flags
   0       0        0             342 unknown,delalloc,eof
output.zip: 1 extent found

So my question is: Is it possible to determine any incosistency in such file by running filefrag.

Note: I tried this on CentOS system.

Reuti · 02-16-2012, 03:47 PM

Does the output change after some minutes when you repeat the command on the same file?

T3RM1NVT0R · 02-16-2012, 05:23 PM

@ Reuti,

Yes, you are right it changes after sometime. I didn't notice this earlier.

So, we can say that it is always good to download a file from a trusted source. However, if you have downloaded a file from some random site then it you can do the following:

1. Identify the file type by running file command against that file. In this scenario file was concatenated > output.png.
2. Try to open up the file using related applications. For me .png opens with "Eye of Gnome" so if that opens fine then I believe that the file shoul be OK but if not than there is something wrong with the file.

I searched a lot on the internet and did not find any way to identify if a file is corrupt or concatenated is mentioned in OP's post. The best option that I found is the one I mentioned above.

Reuti · 02-16-2012, 05:45 PM

Yep, this I noted too. I couldn’t find any more detailed information about the output of filefrag (even in the source), so: I assume “delalloc” means “delayed allocation”, and after some time it’s written to disk and shows up as physical space.

If you are frightened that a local file was changed, there is an article from IBM to execute only signed files and removing the shell. But the OP is on Windows, so it won’t help there.

For the downloading part, maybe a hash value could be checked too.

T3RM1NVT0R · 02-16-2012, 05:54 PM

Yes that is true that the delalloc settle down after sometime once it is fully written on the disk. OP's main concern is how he can find the file that he is downloading is not concatenated or tampered the way mentioned in that article.

For existing file tampering I read one article which talks about the use of tripwire for monitoring purpose.

If by hash you mean md5sum then it will not be helpful because it will be the same as site owner wants it to be. Suppose he uploaded the file on the website with the md5sum=38a3fee5e70ed1f7d30d32c4d9ec33a5. When you will download the file it will be the same. Because that is what site owner has posted by running md5sum against the file he uploaded :-)

ntubski · 02-16-2012, 08:16 PM

Quote:

Originally Posted by Konphine

If people can hide data this way, couldn't it be a security problem (due to the user's fault)?

How would be this be a security problem? Some extra bytes at the end of a file won't hurt anyone.

Anyway, I wrote a python script that will detect extra bytes at the end of png file. It won't detect hidden bytes in the middle of the file though...

Code:

#!/usr/bin/env python
import sys, os, struct

if len(sys.argv) < 2:
    sys.stderr.write('Usage: %s <filename>\n' % sys.argv[0])
    sys.exit(1)

PNG_HEADER = '\x89PNG\x0D\x0A\x1A\x0A'

filename = sys.argv[1]
fin = open(filename, 'rb')
header = fin.read(len(PNG_HEADER))

if header != PNG_HEADER:
    sys.stderr.write('%s is not a png file\n' % filename)
    sys.stderr.exit(1)

while True:
    chunk_header = fin.read(8)
    if len(chunk_header) < 8:
        break
    chunk_len, chunk_type = struct.unpack('!I4s', chunk_header)
    fin.seek(chunk_len, os.SEEK_CUR)
    chunk_crc = fin.read(4)

    if chunk_type == 'IEND':
        break

if not fin.read():
    sys.stdout.write('This is a png file with no extra data at the end\n')
else:
    sys.stdout.write('This is a png file with EXTRA DATA at the end!\n')