LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-10-2011, 05:00 AM   #1
NightHorse
Member
 
Registered: Jun 2009
Location: Egypt
Distribution: Ubuntu 10.04
Posts: 117

Rep: Reputation: 16
read, write syncing


Hi all,

I have the following scenario.

1. Script A copies a file to local file system directory local_dir from an SSHFS share.
2. Process B periodically check for files in local_dir, If any files found, reads the data on it and insert it to DB, then deletes the file.

The question is, Am I gonna face synchronization issues? For example: if the ssh connection fails, would the copy fails as a whole or only a portion of the file will be copied? or the read Process reached the end file while the copying process is not done yet.

And is there a way to safely doing the upper scenario without any data loss??
 
Old 06-10-2011, 05:11 AM   #2
Snark1994
Senior Member
 
Registered: Sep 2010
Location: Wales, UK
Distribution: Arch
Posts: 1,632
Blog Entries: 3

Rep: Reputation: 346Reputation: 346Reputation: 346Reputation: 346
Off the top of my head, you could create a hash of the original file on the share, copy the file to local_temp_dir, check the hashes match (ie. that it's copied correctly), and then copy them to local_dir.
 
1 members found this post helpful.
Old 06-10-2011, 05:59 AM   #3
NightHorse
Member
 
Registered: Jun 2009
Location: Egypt
Distribution: Ubuntu 10.04
Posts: 117

Original Poster
Rep: Reputation: 16
Quote:
Originally Posted by Snark1994 View Post
Off the top of my head, you could create a hash of the original file on the share, copy the file to local_temp_dir, check the hashes match (ie. that it's copied correctly), and then copy them to local_dir.
Great, thank a lot Snark, I knew I would get good ideas from here. ;-)

Added to your suggestion:

. I will create another dir called hashes_dir
. On the ssh server create a hash file for each file i.e. filex.hash that contains the hash value
. On the client I copy the hash to the hashes_dir, copy the file to local_dir, when copy is done I check the hash and if it's correct I delete the hash file from hashes_dir.

. Process B checks the local_dir, and when it finds a file on it, it checks for it's hash file in the hashes_dir, if it's not there so I can read it safely.

euhmmm, now time to implement that, cheers. :-D
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Read Write access to a iso9660 filesystem..mount a .iso image as read write ceazar123 Linux - Newbie 16 09-01-2010 10:07 AM
Read Write access to a iso9660 filesystem..mount a .iso image as read write ceazar123 Linux - General 2 08-26-2010 04:32 PM
Reiserfs vs JFS base on Read, Re-read, Write, Re-write Hesi Linux - Desktop 1 03-19-2010 10:45 PM
Open office read only, K-write read/write mode lwtvh Linux - Newbie 1 07-19-2003 12:33 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 03:59 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration