First let me start off by saying I'm not a Linux noob. I've been using it since somewhere around 1997, including setting up servers, routers, and desktops. Up until now my Google-fu has been good enough to solve every problem I've run across with it.
But this one's got me stumped.

I've written a Perl script that, among other things, checks filetypes using the 'file' command, and compares them to the actual extension. This is a security related script, that's basically looking for hidden executables (.exe files that are named .tmp, and the like) so it can flag them as suspicious. It also does a few hash (SHA1, SHA256, MD5) calculations, so I can search for known system files that have unknown hashes, etc.

Well, I've run across this problem on two different machines, with two different versions of Windows (XP and 7), both NTFS filesystems, one running a custom OpenSuSE LiveCD, and the other running a default Knoppix 6.2.1 DVD. Both times it's been a particular pair of files to do with AVG antivirus that gave me problems. The first time, I thought maybe there was some filesystem corruption, and ignored it.

Basically what I've got is a file that I have read permissions for, according to 'ls', but I can't view the file. When the script gets to this file, trying to open it for the MD5 calculation, it dies, because it's not able to read the file, and I never put any error checking in for that, since I figured all files should be readable, without exception.

From the command line, I get this:

You can see that the 'file' command returns that I have no read permission to the dat files, but according to 'ls', I do. I'm also running as root at this point, so I should be able to read everything.

Both machines have returned the I/O error, rather than a permissions error when trying to open the file manually.

It's not a failing hard drive, and from what I can tell, it's not a corrupt filesystem, either. But the fact that it was these same two dat files the first time I ran across this, leads me to think it couldn't be anything breaking, as the chances of it being the same two files in both cases are virtually nil. It's got to be something peculiar about the way AVG saves these files, but I can't imagine what it could be.

Anybody have the slightest clue what's going on here?

Thanks,
Chris

Is it a tape media? If so clean the tape drive and reinsert..or check if the version suits linux

Maybe that is one of the special options of NTFS that Linux is currently not able to handle? I don't have a clue what option it may be, can it be marked as system file or something similar?

For the NTFS filesystem, when mounting on Linux, the permissions of all files and directories are determined by the mount command.
So you won't have the permissions of some files differ from others.

Also, a feature of NTFS is that small files may be contained in the directory, instead of located on the filesystem itself.
This makes me wonder if either the NTFS filesystem is corrupt, or the version of ntfs-3g your version of knoppix uses, doesn't fully support the version of the NTFS file system you are looking at.

Also, read the manpage for ntfs-3g. I'm wondering if the problem may involve Alternate Data Streams. Whether you have access to them, and how you can, depends on how the filesystem was mounted. Another reason for reading up on them, and user_xattrs, is that this feature of NTFS was created to support Mac clients. It was discovered that malware writers were using it to hide code from scanners. Mounting and scanning a Windows filesystem from Linux, make sure that files written as alternate data streams of other files are detected as well.

Could there be an issue with a forbidden character '/,\,\0' being used in the file name? One way to manipulate a file with strange names, or identical names is to list the inode of the file, and access the file using the file command:

Not a tape drive. Otherwise I probably wouldn't have mentioned that it's not a failing hard drive, would I?

No, this is an offline scanner script to do some diagnostics on virus infected machines. So it's only ever run against hard drives in Windows machines.

That's a possibility, I suppose. I've just never run across it before, and I've been using Linux live CDs of various types for years to fix broken Windows machines.

It does seem weird that AVG would mark a file like this, but not a single Windows file is marked with this option, though, doesn't it?





I forgot to mention, also, that the kernel versions on these two machines when I've run across this were different, too. Both 2.6 series, but a few build revisions apart.

So, a couple of decent suggestions, but nothing useful so far for this particular situation. Any kernel and/or other Linux gurus want to take a stab at this?

Chris

I knew that. Which is why this seems so bizarre to me.

Also knew that. Haven't had a problem accessing any other small files before, though, so I don't see what would be different about this.

I thought of filesystem corruption, but due to it being the exact same two files, on two different machines, running two different versions of Windows, and two different live CDs, with two different distributions, leads me to believe that the chances of this being a filesystem corruption problem are essentially zero. The NTFS version idea is also a pretty slim chance. Now the ntfs-3g might not support it, as the two CDs probably have very similar versions on them. I'll have to look that up.

Now that's something that's worth looking at. I'll take a look in the morning, as it's 1 AM here now, and I really should be getting some sleep.



Look at my first post for the directory listing. No forbidden characters. Not even any special characters. Just alphanumeric for these two files.