I'm used to seeing certain change in /dev and /sys mainly as USB devices get plugged and unplugged, but a recent scan showed some very unusual changes I can't account for. For example, /dev/.udev/db/block:ram0 - ram12 and ram15 show md5 changes but NOT ram12, ram13, or ram14. I have not changed any ram modules or settings in the bios.

Another example, /sys/module/8021q/sections/.data and all the other files in the "sections" dir show md5 changes. When I cat these files I see 0xfffff89e4580 and other values that looks like locations in RAM where pointers to call these drivers probably reside. Why would the memory address suddenly change on so many modules? Addresses for 8021q, acpi_cpufreq, arc4, cdrom, cfg80211, cpufreq_ondemand, eeprom_24c32, fat, freq_table, fuse, garp, i2c_i801, iTCO_vendor_support, ip6_tables, ip_tables, iptable_filter, iptable_mangle, iptable_nat, ipv6, kvm, kvm_intel, llc, mac80211, macvlan, macvtap, microcode, mperf, nf_conntrack, nf_defrag_ipv4, nf_defrag_ipv6, nf_nat, rfkill, sd_mod, snd, snd_hda_codec, snd_hwdep, snd_page_alloc, snd_pcm, snd_seq, snd_timer, sr_mod, stp, tun, vfat, vhost_net, xt_state all changed. However other module addresses such as ext4, hid, mousedev, pcmcia, and usbcore have not changed. Why would the memory address for i2c_i801 change and not i2c_core?

I'm trying not to jump to any conclusions, but it looks pretty suspicious. The only change is the system was physically relocated, ie unplugged and replugged. Other than that there have been 0 changes to the OS, no updates, no new software or hardware, nothing changed in /etc, no changes at all. It doesn't seem logical that just moving the hardware would cause all these changes.

Virtual File Systems like /proc and /sys present a dynamic view of kernel internals and /dev is populated dynamically on the fly by scripts, udev and other subsystems, both to the extent that traditional monitoring will be prone to errors and false positives and therefore of limited use. If you want to get into this you best start by posting pre & post event data.

Understood that monitoring /sys and /dev presents some challenges, but most stealth rootkits create parallel filesystem to hide files and the only way to discover them is to locate the altered devices pointing to alt filesystem or other shadow/rooted devices. When you say pre and post data, what do you mean?

That's an understatement ;-p


What do you mean "parallel filesystem"?


You said things changed. So what were the details like before and what after the fact?

When I say parallel filesystem I mean a rootkit will commonly change the partitioning on the hard drive to reserve space for itself on a newly created encrypted partition the OS cannot read where it stores most of its active components. It needs links within the running OS to the hidden filesystem, which show up as drivers or maybe kernel modules. Often it intercepts calls from the regular filesystem to make sure that read/write operations are remapped and do not fall into the reserved space. This is why I call it a shadow filesystem, because of the remapping the OS cannot "see" into that space. No other rootkit files will be found on the regular filesystem other than those drivers for the shadow filesystem. TLD4 uses exactly this type of shadow filesystem for example.

I'm afraid I don't know exactly what changed since tripwire only reports md5 hashes encoded in base64 and apparently truncated. I've been unable to reconstruct the original files based on the tripwire hashes. For the future I will save copies of the originals so a comparison can be made, but based on other research I think this is most likely a false positive. It seems tripwire is showing its age when dealing with modern threats and a lot of additional customization is required to keep pace.