LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-18-2010, 01:57 PM   #1
vzxen
Member
 
Registered: Jul 2010
Posts: 126

Rep: Reputation: 4
Catch all data being written to Hard Drives


Hi,

I have a Linux CentOS server.
Is there a way to store all changes made to the hard drive by all software on the server ?

Regards,
vzxen
 
Old 07-18-2010, 02:21 PM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by vzxen View Post
Is there a way to store all changes made to the hard drive by all software on the server ?
Any particular reason why you would want this? Details please: be as verbose as possible.
 
Old 07-18-2010, 02:26 PM   #3
vzxen
Member
 
Registered: Jul 2010
Posts: 126

Original Poster
Rep: Reputation: 4
Yes, because I want to keep a track of all files and directories being modified from time to time.
 
Old 07-18-2010, 02:36 PM   #4
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
What is the specific purpose of "keeping track" as you define it? Should it cover the whole file system or just one application or just some users? Can't you use a file system integrity checker like Aide or Samhain? If it must be real time: why? Ever used inotify, Auditd or FUSE LoggedFS?
 
Old 07-18-2010, 06:05 PM   #5
jefro
Moderator
 
Registered: Mar 2008
Posts: 21,981

Rep: Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625
It could be that you could setup a mirrored ramdisk and do it that way?
 
Old 07-18-2010, 11:02 PM   #6
vzxen
Member
 
Registered: Jul 2010
Posts: 126

Original Poster
Rep: Reputation: 4
Quote:
Originally Posted by unSpawn View Post
What is the specific purpose of "keeping track" as you define it? Should it cover the whole file system or just one application or just some users? Can't you use a file system integrity checker like Aide or Samhain? If it must be real time: why? Ever used inotify, Auditd or FUSE LoggedFS?
I want to monitor the entire filesystem.
Who handles the data being written to the HDD in Linux ?
Does it go through the Linux Kernel ?
Is there an equivalent to Microsoft Volume Shadow Copy Service for Windows ?
 
Old 07-19-2010, 01:53 PM   #7
vzxen
Member
 
Registered: Jul 2010
Posts: 126

Original Poster
Rep: Reputation: 4
Please help me with this. I have been banging my head for two days for a possible solution.
 
Old 07-19-2010, 02:01 PM   #8
Bratmon
Member
 
Registered: Jul 2009
Location: 75.126.162.205:80
Distribution: Arch / Mint 17
Posts: 297
Blog Entries: 3

Rep: Reputation: 50
Have a look at lsof.
 
Old 07-19-2010, 03:27 PM   #9
jefro
Moderator
 
Registered: Mar 2008
Posts: 21,981

Rep: Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625Reputation: 3625
Might look at zfs. http://wiki.github.com/behlendorf/zfs/example-zvol
 
Old 07-19-2010, 11:40 PM   #10
vzxen
Member
 
Registered: Jul 2010
Posts: 126

Original Poster
Rep: Reputation: 4
I was reading R1Soft does something in similar fashion.
How does it do it ?
lsof - is only currently opened files.
zfs - is for taking snapshots

Basically I want to keep track of changes to all files on the system.
 
Old 07-20-2010, 09:59 AM   #11
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,399
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
How would you 'keep track of' data being written? By writing to some data file(s)? Then you would have to 'keep track of' that data, and then that would result in more data to keep track of, and so on. And how would you monitor the data? Would you have a reader for all types of data files? Would you or anyone actually inspect that data?
Okay, I'm being cynical. But really, your requirements need to be refined a bit more. What is the objective of keeping track of data being written/modified? Are you concerned about the quantity being stored? The content? The origin? Some per-user or per-group concerns?
You could keep disk usage statistics fairly easily using some basic scripting that runs periodically.
--- rod.
 
Old 07-20-2010, 10:34 AM   #12
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by vzxen View Post
I want to monitor the entire filesystem.
Yes, I know that. You've been repeating that endlessly.

"monitoring the entire filesystem" can be done for many reasons: it can be a hard audit requirement, it can aid troubleshooting or it can be used to say watch application or human user behaviour. (It also is invasive, a performance drain and likely in conflict with Privacy laws.) What I need to know is what you need it for exactly. If you can answer that simple question in detail and without repeating yourself then I'm sure leads or solutions can be offered that suit your case best.
 
Old 07-20-2010, 01:00 PM   #13
vzxen
Member
 
Registered: Jul 2010
Posts: 126

Original Poster
Rep: Reputation: 4
Hi,

I have two reasons:
1) Create a Backup System (I was thinking of doing something similar to R1SOFTs near CDP though its advertised as CDP. I want to learn that and do it for my college)
2) Monitor which user is editing which file. There are many users who access files especially in a college. Hence I want to be able to monitor it as well.

I have read that I can try to do it by making a modified kernel. Or does r1soft do it in another way ?
 
Old 07-20-2010, 02:33 PM   #14
theNbomr
LQ 5k Club
 
Registered: Aug 2005
Distribution: OpenSuse, Fedora, Redhat, Debian
Posts: 5,399
Blog Entries: 2

Rep: Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908Reputation: 908
You can use rsync to make periodic backups that only backup modified files. The utilities 'ps', 'lsof' and 'w' could probably be combined to create some kind of pseudo-realtime monitoring tool for users file activities. You could call it 'BigBrother'.
--- rod.
 
Old 07-21-2010, 12:50 AM   #15
vzxen
Member
 
Registered: Jul 2010
Posts: 126

Original Poster
Rep: Reputation: 4
How does R1Soft do it ?
Are they using a modified kernel ?

Also how can 'ps' be used ?
It shows the current processes right ?

(BTW: I accidentally hit the "Report" button for the above post. Sorry for that!)

Last edited by vzxen; 07-21-2010 at 12:52 AM.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Is my data lost? Installed Linux, no longer can I access my hard drives. Ironkey Linux - Desktop 9 07-12-2009 09:33 AM
compressing data to be later uncompressed and written to a buffer in knoppix dr_zayus69 Linux - Distributions 0 05-26-2005 09:14 AM
Linux partition over written with Fat32 data! hi-clier Linux - Newbie 2 11-21-2004 07:35 PM
Possible to make filesystem on disk with data written to it? redhatman Linux - Newbie 3 08-28-2004 10:49 AM
Exchange data btw. 2 hard drives on same PC zaseny2002 Linux - Newbie 17 04-25-2003 03:46 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 05:11 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration