LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-30-2009, 10:06 AM   #16
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 899

Original Poster
Rep: Reputation: 70

Quote:
Originally Posted by PTrenholme View Post
Well, yes, I guess you could use dd:
Sorry, but that command line is a bit incomprehensible to me at this stage. I was just saying to Charles that maybe sed might be better. I think that looks 'between the cracks' too.
 
Old 07-30-2009, 10:19 AM   #17
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 899

Original Poster
Rep: Reputation: 70
Okay guys, well thanks for the suggestions. Never mind if nothing comes to mind. There's a very long thread called 'understanding the dd command' by some chap called Awesome Machine where I'm pretty sure it's explained how to do this with sed. I'll check it out before bothering you further.
Thanks again, all. :-)
 
Old 07-30-2009, 10:19 AM   #18
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by Completely Clueless View Post
Shouldn't need forensics for this in Linux.
Where reinventing the wheel equals efficiency and
"using the wrong tool for the right job" not misperception,
there "forensics" is more than just a label.
 
Old 07-30-2009, 10:23 AM   #19
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 899

Original Poster
Rep: Reputation: 70
Sorry! The actual thread is 'learn the dd command' and it starts here:
http://www.linuxquestions.org/questi...ommand-362506/
 
Old 07-30-2009, 10:29 AM   #20
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 899

Original Poster
Rep: Reputation: 70
And with that I think I shall go down the pub. I'll check for any follow-ups later. See y'all. :-)
 
Old 07-30-2009, 02:00 PM   #21
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
OK, have had time to test. The problem was using seek (skip output blocks) instead of skip (skip input blocks).

Thanks to Peter for the smarter grep options and the sanity of hexdumping the output.

Some gotchas ...

What sort of file was the email address in? If it was a wordprocessor file or a double byte character set file or ... anything but a plain text file, then the grep match expression is not going to be easy. Maybe good idea to set up a similar file to test the grep match expression first:
Code:
dd if=testfile | grep -iaC 5 <match expression>| hexdump -C
How big is the partition? A huge amount of output could be produced if the match expression is loose. May be prudent to introduce an outer loop to switch output files or to start with a limited number of blocks first to see how it goes before going for the final run.

dd reports what it did on stderr -- best lost to /dev/null.

It's going to take a long time to run so nice to have an indication of activity. Maybe up the count= so it runs a bit faster.

Which leads to
Code:
echo "" > out
i=0
#while true  # For final run
while [[ $i -lt 300 ]]  # While testing
do
    block="$(dd if=/dev/sda4 skip=$i count=4 2>/dev/null)"
    [[ $? -ne 0 ]] && exit
    echo "$block" | grep -iaC 5 <match expression> | hexdump -C >> out
    echo -n '.'
    let i++ 
done
 
Old 07-31-2009, 06:30 AM   #22
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 899

Original Poster
Rep: Reputation: 70
Quote:
Originally Posted by catkin View Post
OK, have had time to test. The problem was using seek (skip output blocks) instead of skip (skip input blocks).

Thanks to Peter for the smarter grep options and the sanity of hexdumping the output.

Some gotchas ...

What sort of file was the email address in?
Hello again Charles,

Thanks for your follow-up. I shall run your code a little later. I'm just a bit concerned by this statement of yours "what sort of file was the email address is?"_could_ imply that the code only searches within existing files rather than the entire space of the partition in question (50Gb). Just a thought. I'll report back on the results later....

Last edited by Completely Clueless; 07-31-2009 at 06:31 AM.
 
Old 07-31-2009, 06:38 AM   #23
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 899

Original Poster
Rep: Reputation: 70
Oh, I nearly forgot. The address would have been in a temporary Firefox browser cache file. Hence the need to search free space.
 
Old 07-31-2009, 07:52 AM   #24
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
Quote:
Originally Posted by Completely Clueless View Post
Hello again Charles,

Thanks for your follow-up. I shall run your code a little later. I'm just a bit concerned by this statement of yours "what sort of file was the email address is?"_could_ imply that the code only searches within existing files rather than the entire space of the partition in question (50Gb). Just a thought. I'll report back on the results later....
The code searches the whole 50 GB (50 GB! That's going to take a wile to search!). The dd command is very simple minded -- it just ploughs through the whole of its input and copies it to output. By giving /dev/sda4 as input it's reading the raw blocks, not paying any attention to file system structures.

I hope you've umounted /dev/sda4 or later operations may overwrite the block(s) you want.

Aren't Firefox cache files compressed? If so there is no grep match expression that will find email addresses i them.
 
Old 07-31-2009, 10:12 AM   #25
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,187

Rep: Reputation: 354Reputation: 354Reputation: 354Reputation: 354
Depending on how you've set it up, you may have several day's data in you FF cache. Have you tried entering about:cache in your FF address bar and looking at which sites are in your disk cache?

And, if the cache file you need is not there, I suspect that you'll be out of luck since, as catkin noted, the cache files are compressed, so a simple string search will not work.

You might be able to recover some cache file with tools like foremost, but getting them back into your cache is not simple. (You'll probably need to recover the _CACHE_ control file(s) as well. And, since FF will have unlinked the files, they are most probably unrecoverable.

Note that, if you had fully described the problem you were trying to solve in your first post, instead of just asking about one possible solution, you would have probably received the advice about using FF's built-in cache exploration tool (the about:cache address) soon enough to be helpful.

Good luck!
 
Old 07-31-2009, 12:07 PM   #26
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 899

Original Poster
Rep: Reputation: 70
Quote:
Originally Posted by catkin View Post
The code searches the whole 50 GB (50 GB! That's going to take a wile to search!). The dd command is very simple minded -- it just ploughs through the whole of its input and copies it to output. By giving /dev/sda4 as input it's reading the raw blocks, not paying any attention to file system structures.

I hope you've umounted /dev/sda4 or later operations may overwrite the block(s) you want.

Aren't Firefox cache files compressed? If so there is no grep match expression that will find email addresses i them.
Hi Charles,
Well PTrenholme reckons they are. Bummmer. Fortunately, the people I was needing to contact have emailed ME, so the urgency is no longer a problem.

Now I've tried your revised code as it strikes me this would still be a useful tool to have. It now runs, although not for long. It prints a few screenfuls of dots to stdout and then exits (without error!). So something still not right. In your suggested script, you use something like <search expression> which I've edited into 'xyz@yahoo.com' (not the real address published here for obvious reasons, but within single quotes instead of '<>'. I'm just wondering if it's correct to enclose the string in single quotes? Could that be the problem?
 
Old 07-31-2009, 12:49 PM   #27
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
Quote:
Originally Posted by Completely Clueless View Post
It prints a few screenfuls of dots to stdout and then exits (without error!). So something still not right. In your suggested script, you use something like <search expression> which I've edited into 'xyz@yahoo.com' (not the real address published here for obvious reasons, but within single quotes instead of '<>'. I'm just wondering if it's correct to enclose the string in single quotes? Could that be the problem?
Glad you got that important email address!

If you ran my script as posted the behaviour you describe is expected. To scan the whole partition you need to uncomment #while true and comment out while [[ $i -lt 300 ]]. The output is in file "out" which you can watch grow (or not!) using tail -f out (from another terminal unless you bacground the script). The single quotes may not be essentail but they are good practice, in case the match expression matches something in the current directory.
 
Old 07-31-2009, 01:09 PM   #28
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 899

Original Poster
Rep: Reputation: 70
Quote:
Originally Posted by catkin View Post
Glad you got that important email address!

If you ran my script as posted the behaviour you describe is expected. To scan the whole partition you need to uncomment #while true and comment out while [[ $i -lt 300 ]]. The output is in file "out" which you can watch grow (or not!) using tail -f out (from another terminal unless you bacground the script). The single quotes may not be essentail but they are good practice, in case the match expression matches something in the current directory.
Oh right, thanks! I didn't spot the comments aspect and apart from the search term just ran it as it stood. I'll have another try later and report back in due course....
 
Old 08-01-2009, 08:37 AM   #29
Completely Clueless
Member
 
Registered: Mar 2008
Location: Marbella, Spain
Distribution: Many and various...
Posts: 899

Original Poster
Rep: Reputation: 70
Okay. Sorry for the delay. I've changed the code according to your instructions and have arrived at:

Code:
#!/bin/bash

echo "" > out
i=0
while true  # For final run
#while [[ $i -lt 300 ]]
do
    block="$(dd if=/dev/sda4 skip=$i count=4 2>/dev/null)"
    [[ $? -ne 0 ]] && exit
    echo "$block" | grep -iaC 5 'xyz@hotmail.com' | hexdump -C >> out
    echo -n '.'
    let i++ 
done
But it still won't run any sense. It creates a file named 'out' consisting of one byte and then promptly exits! I've not yet read-up on while loops in Bash, but even to me it still doesn't look right. :-/
 
Old 08-01-2009, 08:57 AM   #30
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Mint
Posts: 17,809

Rep: Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743
I have not followed this thread, but some things jump out at me:
Code:
block="$(dd if=/dev/sda4 skip=$i count=4 2>/dev/null)"
Did you mean to assign the literal string to block? If you meant to assign the result of the dd command, then you don't want the outer quotes.
Code:
[[ $? -ne 0 ]] && exit
Do you mean to test the exit code of dd? It seems that this might be testing the exit code of the assignment statement
Code:
echo "$block" | grep -iaC 5 'xyz@hotmail.com' | hexdump -C >> out
This echos the literal string "$block"---I think you meant "echo $block" (without the quotes)
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Searching a string from the file amty Programming 6 11-06-2008 07:04 AM
searching for char string containing a '/' cleopard Programming 3 08-14-2008 04:17 PM
C++: need helping searching for multiple items in a string BrianK Programming 20 02-21-2008 02:59 PM
searching for a string of charcters in some files hhegab Programming 2 04-16-2005 05:07 PM
Searching for a string krazykow Solaris / OpenSolaris 1 03-17-2005 11:55 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 10:17 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration