LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 03-08-2019, 11:28 AM   #1
Linux_Kidd
Member
 
Registered: Jan 2006
Location: USA
Posts: 737

Rep: Reputation: 78
dd for srubbing disk for text


so, i ran into a challenge and want to see if using dd is possible.

is it possible to use dd to copy a disk from disk to /dev/null but in the middle try and do pattern matching?

dd --> pattern match --> /dev/null

i am trying to find some data that seems to be hidden from std find or searching.

ideas please. thanks.
 
Old 03-08-2019, 01:08 PM   #2
Linux_Kidd
Member
 
Registered: Jan 2006
Location: USA
Posts: 737

Original Poster
Rep: Reputation: 78
any way better/faster than this. i'll be on a vm (vmdk) under esxi, etc.

dd if=/dev/sda2 bs=16065 | hexdump -C | grep 'some text'
 
Old 03-09-2019, 02:54 AM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,140

Rep: Reputation: 4123Reputation: 4123Reputation: 4123Reputation: 4123Reputation: 4123Reputation: 4123Reputation: 4123Reputation: 4123Reputation: 4123Reputation: 4123Reputation: 4123
It'll work for ASCII text, but I've never done it for an entire disk. Yep, it'll be slow, but everything has a cost.
Be aware that it won't find anything that has been obfuscated - docx or pdf for example.
 
Old 03-09-2019, 03:08 AM   #4
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,930

Rep: Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321
how can be anything "hidden from std find" ?
find / -type f ... (as root)
 
Old 03-09-2019, 04:14 AM   #5
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 21 MATE
Posts: 8,048
Blog Entries: 5

Rep: Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925
Can you please go into more detail about the background behind the "unfindable" data?
 
Old 03-09-2019, 10:48 AM   #6
Linux_Kidd
Member
 
Registered: Jan 2006
Location: USA
Posts: 737

Original Poster
Rep: Reputation: 78
Background.

A replication agent sends files to a replication server whenever a file is created or modified. That data then traverses an IPS before it goes into a VPN to its destination.

The IPS is alerting on a specific signature and we see specific text in the payload. We are now trying to hunt down where that text is located. To keep this short, we know it's on two of many esxi hosts. These two hosts have about a dozen or so live vmdk's. Not all are linux boxes. McAfee took the text and made a dat for it, we ran McAfee w/o exceptions and it found nothing (we did test the dat successfully). We know the data is there somewhere, but need a better way of searching for it.

Using dd seemed to be the best/fastest was to get every disk block scrubbed looking for the match. This would at least tell us which vmdk is the one that holds the text. Once we know what vmdk it is we can then go fishing in deeper waters, etc.
 
Old 03-09-2019, 11:24 AM   #7
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 21 MATE
Posts: 8,048
Blog Entries: 5

Rep: Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925
I don't know about this sort of stuff at all, but does the replication agent not know, perhaps in the packets it receives or in its log, which host and vmdk the data came from?
 
Old 03-09-2019, 01:29 PM   #8
Linux_Kidd
Member
 
Registered: Jan 2006
Location: USA
Posts: 737

Original Poster
Rep: Reputation: 78
Quote:
Originally Posted by hydrurga View Post
I don't know about this sort of stuff at all, but does the replication agent not know, perhaps in the packets it receives or in its log, which host and vmdk the data came from?
Nope, the agent has no idea what the data is in a file, it simply sends it to one of many replication servers locally. There's a 1:1 mapping between replication server and esxi host. The IPS events show only two replication servers as the source IP of the traffic, so with this 1:1 mapping we know it's coming from two esxi hosts, and each host has multiple OS's running, etc.
 
Old 03-10-2019, 05:27 AM   #9
Brains
Senior Member
 
Registered: Apr 2009
Distribution: All OS except Apple
Posts: 1,591

Rep: Reputation: 389Reputation: 389Reputation: 389Reputation: 389
Quote:
Originally Posted by Linux_Kidd View Post
any way better/faster than this. i'll be on a vm (vmdk) under esxi, etc.

dd if=/dev/sda2 bs=16065 | hexdump -C | grep 'some text'
Strings will find data in allocated space, unallocated space, slack space and swap.
I'm guessing your bs=value is block size? kind of an odd number.
Quote:
Originally Posted by Linux_Kidd View Post
to get every disk block scrubbed looking for the match
Reference material
Code:
dd if=/dev/sda2 bs=16065 | strings -t d | grep -iE "some ?text"
 
Old 03-10-2019, 10:40 AM   #10
Linux_Kidd
Member
 
Registered: Jan 2006
Location: USA
Posts: 737

Original Poster
Rep: Reputation: 78
Thanks Brains for the info.

Yeah, that block size was taken from an example I was looking at.

My command is this
Code:
dd if=/dev/sda2 bs=65536 | hexdump -C | egrep 'text1|text2' >> out.txt &
But is using strings any better/faster than using hexdump -C ? I am not after the actual location on disk, but rather just wanting to know what disk on what vmdk has the text.
 
Old 03-10-2019, 01:25 PM   #11
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,930

Rep: Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321Reputation: 7321
since the "slow" part is dd probably hexdump will not be faster/slower than strings. But strings is much safer. grep does not really like binary input.
 
Old 03-10-2019, 02:25 PM   #12
Linux_Kidd
Member
 
Registered: Jan 2006
Location: USA
Posts: 737

Original Poster
Rep: Reputation: 78
Quote:
Originally Posted by pan64 View Post
since the "slow" part is dd probably hexdump will not be faster/slower than strings. But strings is much safer. grep does not really like binary input.
hexdump -C is hex + ascii

How does strings see printable ascii when output of dd is the input to strings?

Last edited by Linux_Kidd; 03-10-2019 at 02:34 PM.
 
Old 03-10-2019, 02:50 PM   #13
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: Rocky Linux
Posts: 4,782

Rep: Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214Reputation: 2214
One problem with using hexdump here is that the text you are seeking may be split over multiple lines in the formatted output.
 
Old 03-10-2019, 04:06 PM   #14
Linux_Kidd
Member
 
Registered: Jan 2006
Location: USA
Posts: 737

Original Poster
Rep: Reputation: 78
Quote:
Originally Posted by rknichols View Post
One problem with using hexdump here is that the text you are seeking may be split over multiple lines in the formatted output.
Good point.

I guess strings is the better way.
 
Old 03-10-2019, 04:48 PM   #15
Brains
Senior Member
 
Registered: Apr 2009
Distribution: All OS except Apple
Posts: 1,591

Rep: Reputation: 389Reputation: 389Reputation: 389Reputation: 389
Quote:
Originally Posted by Linux_Kidd View Post
Good point.

I guess strings is the better way.
Code:
strings /dev/sda2 -n 8 | grep -C10 "some ?text"
Yeah, that's what strings is for, looking for a string. The example above would also show 10 lines before and after the string, the string having a minimum of 8 characters, strings defaults to 4 characters, helps lower false positives.
EDIT: I thought, just in case you would want a copy of the file, I gave you the resources to dig it out with the link I provided earlier. Since you couldn't find the string with other tools could suggest it's not in allocated space where the file system would know where it is.

Last edited by Brains; 03-10-2019 at 04:51 PM.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Bash command to 'cut' text into another text file & modifying text. velgasius Programming 4 10-17-2011 04:55 AM
How to change disk size by editing disk information on the hard disk? xptchina Linux - Virtualization and Cloud 2 12-16-2010 08:12 AM
How to parse text file to a set text column width and output to new text file? jsstevenson Programming 12 04-23-2008 02:36 PM
floppy only reads first disk, if new disk is inserted data from first disk is shown. lakosked Debian 3 02-18-2006 08:44 PM
More text in Text Mode LandRoverMan Linux - Newbie 2 06-10-2003 11:47 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 11:07 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration