File share between Raspberry Pi and Ubuntu - Running a script

lleb · 06-12-2013, 06:53 PM

here is a check list for you.

On the Laptop:
1. ensure the external HDD never goes to sleep (can wind down in RPMs, but not disconnect from laptop.
2. configure sshd.conf to allow for keys for authentication.
3. Optional: setup and configure NFS to share the external HDD via your LAN.

On the RPi:
1. create rsa key to what ever level of encryption you wish, i personally use 4096.
1a. cd ~/.ssh
1b. ssh-keygen -t rsa -b 4096
1c. follow directions on screen, DO NOT PUT IN A PASSPHRASE as this will be for unattended authentication.
2. create script to rsync XYZ data to your laptop (by doing this there is zero reason to have a mount point or use sshfs)
3. after the rsync is complete, you can rm -rf the data on the local SSD to ensure you do not run out of space.

that is unless you wish to store the data on the external HDD instead of the SSD on the RPi. If that is the case, then NFS is the best way to go and skip sshfs completely. if you are worried about the data being intercepted for security reasons establish the connection via ssh exactly as i described above with the same rsa key we just created and use that to mount the point local. It is what sshfs does so you could use sshfs for that exact situation.

mspear20 · 06-17-2013, 08:53 AM

Hey guys,

I really do appreciate all the help.

Little update:
I currently am using sshfs to have my RPi connect to the other computer. I have forgone the external HD, and am just having it go to the laptops.
I created the script to have it create a new folder and copy the files into the folder that was created every 15 mins using a cron job. It has been working great!
Im not too worried about security, so I basically have set the sshfs connection and left the laptop and RPi sit. Its been going strong for about 6 days now.

Now my next question for you all is this:
When using NFS, If as soon as a file is created on the Pi (new video clip) it will show up on the laptop correct? But if were to shut off the Pi or disconnect, will the files still be on the laptop?
Basically when a file is being created on the Pi, will it also be written directly on the laptop at the same time? Or is it like sshfs where it just shows it in the folder but I still have to have a script to copy the files over?

Sorry, I can't really describe what Im trying to get at. Hopefully you get see where Im going.

Any info helps

Thanks,
Mike

schneidz · 06-17-2013, 09:02 AM

whether you use sshfs/nfs/samba/http/ftp/... if the server is no longer available then the files it is serving will also be unavailable.

mspear20 · 06-17-2013, 09:24 AM

Thanks for the replay,

Ok see this is part of the "hard to explain" part. I totally understand that if the two get disconnected, no files will be avaiable from each other. (In my case from the Pi) Let me try this.

1. Pi and laptop are connected through NFS
2. file is created on Pi
3. Through NFS, file is written to the HD and accessible on the laptop (without the use of any scripts)
4. Take the pi offline
5. On the laptop, Navigate to the NFS folder and see the video file.

Step number 4 is what Im trying to ask. Hope this helps

Thanks as always,
Mike

schneidz · 06-17-2013, 09:33 AM

Quote:

Originally Posted by mspear20

...
1. Pi and laptop are connected through NFS (assuming the laptop is the server and the pi is the client)
2. file is created on Pi
3. Through NFS, file is written to the HD and accessible on the laptop (without the use of any scripts)
4. Take the pi offline
5. On the laptop, Navigate to the NFS folder and see the video file.

Step number 4 is what Im trying to ask. Hope this helps

Thanks as always,
Mike

at this point it wouldnt matter if you no longer have an internet connection since the files now reside on the laptop harddrive (you could even disconnect and take the laptop to alaska and still have access to your files as long as you are no longer trying to download from your pi <@

).

mspear20 · 06-17-2013, 09:43 AM

Awesome!

So let me get this straight:
If I set up the laptop as the NFS server and mount a directory to the Pi, then all I have to do is basically have the video files that are created on the pi, go into the mounted directory. Then this way, it will be written on the latops HD. The files wont even touch the Pis SD card at this point. Correct?

Then a simple script could be written on the laptop to move files around to desired folders.

Yes?

Thanks,
Mike

lleb · 06-17-2013, 11:40 AM

Quote:

Originally Posted by mspear20

Now my next question for you all is this:
When using NFS, If as soon as a file is created on the Pi (new video clip) it will show up on the laptop correct? But if were to shut off the Pi or disconnect, will the files still be on the laptop?
Basically when a file is being created on the Pi, will it also be written directly on the laptop at the same time? Or is it like sshfs where it just shows it in the folder but I still have to have a script to copy the files over?

Sorry, I can't really describe what Im trying to get at. Hopefully you get see where Im going.

Any info helps

Thanks,
Mike

a file share is a file share. if the file has been moved from the RPi to your storage end then it will be on the storage end. if not then it will remain on the RPi.

an other big advantage to using NFS v. sshfs is that you can then configure autofs to automatically mount/umount the share point as you perform the transfer.

if there is zero connection, there is no file transfer. the connection MUST be there in order to share the file across the network. this does not change regardless of the OSs involved.

lleb · 06-17-2013, 11:45 AM

Quote:

Originally Posted by mspear20

Awesome!

So let me get this straight:
If I set up the laptop as the NFS server and mount a directory to the Pi, then all I have to do is basically have the video files that are created on the pi, go into the mounted directory. Then this way, it will be written on the latops HD. The files wont even touch the Pis SD card at this point. Correct?

Then a simple script could be written on the laptop to move files around to desired folders.

Yes?

Thanks,
Mike

yes you could do that, but keep in mind the write speed of your network v. local to the SSD. if the files are large enough you run the risk of corruption due to the MUCH SLOWER connection speed of the network. Sorry there is not a cost effective home use network that is going to be able to read/write as fast as your RPi can read/write locally to your connected SSD card.

can it be done in the commercial world yes. there are fiber solutions that allow for enough bandwidth for that type of application, but they do not come cheap.

your process of writing to the SSD, then using rsync or scp to move the data to some form of external storage is ideal for you. you will find rsync is lighter on the RPi then scp/cp and will not lock the RPi so that it will continue to perform its primary job during the copy.

mspear20 · 06-17-2013, 12:04 PM

Hmmm ok. I was afriad that the speed might be a problem.

Well last week I ordered another RPi that I was going to use for something else, but now I think Ill grab one of my other laptops, throw Ubuntu on it and try NFS with it and see how it works. Thanks for all the advice guys!

I probably will be back with more questions or comments

Thanks again,
Mike

lleb · 06-17-2013, 04:54 PM

you might want to look into CentOS instead of Ubuntu for your NFS server. specifically for privacy. Canonical will take your personal data and sell it to the likes of Facebook, E-Bay, Amazon, etc.... CentOS does not. Also CentOS is a true server class OS while Ubuntu is neither a desktop nor a server OS. it is a wanabe Linux fake environment.

or if you are just not comfortable with the RedHat way of doing things, then use pure Debian as your server. it will be 100x more stable and secure, not to mention zero built in mal/spyware like Ubuntu.

escolom · 06-17-2013, 05:14 PM

I found samba to be flakey between ubunto and windows xp, windows pro. I am not sure why. but since you
are going Linux to Linux.. just share the folder where you have your usb drive mounted. if the rasb pi is connected wirelessly
to the same router it should see the folder. on rasb pi nataulis, add a bookmark for that folder.

you should be okay and then you can write a simple .sh script to copy your files and schedule it with crontab -e
I wish I new the syntax in bash to add a date to folder name. anyone?

schneidz · 06-18-2013, 08:36 AM

^ man date

mspear20 · 06-18-2013, 03:45 PM

Thanks guys,

Like I said I have everything working already with sshfs. Im just looking for a way to make it a little more efficient. Ill work on the suggestions and let you all know if I need anything.

Thanks again,
Mike

mspear20 · 06-28-2013, 08:28 PM

Alright guys, Im back...

Heres whats going on:
I have debian 6.06 installed on my other laptop. For testing I have my RPi connected via ethernet cable to my laptop.
My Laptop, which is the server, has an address of 192.168.1.1/24 and the Pi has 192.168.1.3
I installed nfs on both my Pi and laptop.

On the laptop (/etc/exports)

Code:

/motion_back 192.168.1.1/24(rw,sync,no_root_squash,no_subtree_check)
/motion_counter 192.168.1.1/24(rw, sync,no_root_squash,no_subtree_check)

On the Pi(fstab)

Code:

192.168.1.1:/motion_counter /motion_counter nfs auto,noatime,nolock,bg,nfsvers=3,intr,tcp,actime=1800 0 0

For the sake of testing I do not have the /motion_back in the fstab file yet.

Now after restarting the server/Pi services (exports, nfs server, etc) I go to mount manually on the Pi with the command

Code:

mount 192.168.1.1:/motion_counter /motion_counter

And I get the following errors

Code:

mount.nfs: rcp.statd is not running but is required for remote locking.
mount.nfs: Either use '-o nolock' to keep locks local, or start statd.
mount.nfs: an incorrect mount option was specified

I cannot find this statd anywhere nor do I know how to start it.

Thats what I got at the moment. If you need more info please let me know.

Thanks again,
Mike

mspear20 · 06-29-2013, 09:53 AM

Bit of an update. I was playing around with it some more. The service portmap wasnt started on the Pi. So I started the service rpcbind, which I guess has taken place of portmap on the Pi. Now when I go to mount, I get the server denied access. What do I need to do to allow access?