Data files on NAS
 

Is it straightforward to have a linux-like directory structure on a NAS device?

I want a NAS system on which I can store a large number of pdfs, with a dir structure that I create, and from which I can pull individual files onto my desktop when needed. I'm digitizing thousands of hand-written pages of old scientific working notes which I no longer have the space to store. This wouldn't be a back-up, it'd be the primary copy, with backup somewhere else.

This is probably an idiot question, and I apologize, but all the NAS units I see for sale are geared to media streaming. I actually have a QNAP unit which streams music around the house well, but I haven't been able to create dirs on it or upload dirs from my desktop. Reading about freeNAS and NAS4free, even they seem to be geared to providing GUIs for windows users like the commercial products.

I'm a long-term linux *end-user*, Debian for the past few years, prefer the command line, work with scientific software, but don't have the time to learn arch, for example.

Thanks .....

data. Need more data.
 

Nice questions! The answer depends on a few factors not mentioned, so let me wax general.

NAS devices come in different 'flavors' that determine the exact answer, but in general NAS can refer to anything from a shared drive (ALA NFS or Windows CIFS) to a state-of-the-art SAN array. NAS specifically geared to media streaming may not be optimal for your need, but there are devices that should serve. (And they may be cheaper for the same volume!) Check some of those that mention media streaming. That may be great sales copy, but that device may not be so limited.

You may be looking for something iSCSI capable, but I have made this kind of thing work with a NAS device running (and created for) Windows and a AD network. That would not be my first choice, but the device was small (4 drives only), adaptable (supported NFS, CIFS, iSCSI), and sufficient for my needs. (Also free, it had become redundant at another office.)

The important factor is that it be mounted as if a native local drive (iSCSI supports that, NFS and CIFS do not) and presented as a device. Mounted after the local drives, after networking, but before any applications start, it should appear no different than a native local drive after boot. You can then manage the files and folder structure just as you would any other drive. (EXT4 works well. BTFRS works better in many ways, but is a bit slower. XFS tests out well, but did not perform as well for my database repo. Any of them should work well for a document repo.)

If possible, address your NAS over a network device or channel that does not carry other traffic. There is an excellent reason for most high level SAN setups to use expensive HIB optical interfaces: not only speed, not only multi-channel redundancy, but they are not shared. You want your storage access to be no slower than the speed of media and network. Collisions are not your friend here.

In general most if not all NAS have at least basic windows file sharing I.e.. SMB/CIFS and FTP including your QNAP. You should be able to setup shares, create directories etc and access them via samba on your debian box.

Thanks wpeckham, that's very interesting. I was not aware of SAN but this may be more than I need. I probably need around 12 to 16 TB, but hard to be sure in advance. Speed is not a major requirement.

I'm reading more about this and expect I'll get back to you.

Thanks. I do go into the QNAP via FTP and can upload dirs of music files (fairly large, about 350GB of flacs). The QNAP accepts these but wonn't accept dirs of data files. It looks to me as if QNAP doesn't think the user would want to do this.

Clear
 

I realized I may have been unclear.
What you want is very doable. You do NOT need a SAN. Though a SAN would work well for you, it would cost much more and represent serious overkill for your application.

What is optimal would be a NAS that allows you to mount a file system as if local and manage it using your local OS features. IF you cannot mount that way, you may need to do a little prep work and pre-format space, share and mount. That would be less optimal, but certainly workable.

I have not run into a NAS device that would not support a solution, though I have seen some that were less than optimal. I have not researched recent offerings, so my information on current devices is unlikely to be useful, but I have used this KIND of technology often.

Had I need for such a solution myself, I would try to leverage spare parts and try FreeNAS: a build-your-own solution. If you are comfortable with Linux, network, and storage management it may be an option.

Yes, that's spot on, exactly what I want to be able to do.

I'm experimenting further with QNAP but I can't see any indication that samba is/can be set up on the QNAP. I'll try a couple more things and then see what I can find to set freeNAS up on. I don't want to wipe the QNAP o/s except as a last resort.

I don't have any real experience with networking, so it may take a while to get this going.

Thanks again,

What is the model number of your NAS. samba is the linux implementation of the windows file sharing protocol SMB/CIFS. The manual should provide some help.

How are you trying to copy the directories? I only ask because if it's from a Linux desktop then most file managers will let you type something like "ftp://username@nas-ip/data/music/" in the location bar, ask for the relevant password, then mount the system as if it were local. So if you're doing it graphically that's all you need. For command-line FTP I'm afraid it's bee too long for me to recall how to do recursive copying but I'm sure it is possible.
If you want something a little more secure (i.e. data is sent encrypted not clear text) for whatever reason then you could use SFTP which comes as part of SSH and, I think, can be used on the Qnap:
http://wiki.qnap.com/wiki/SSH
I have to admit that it may be slower due to cryptographic overhead.

It's a TS-419 P II, RAID 5 on 4 disks. Software is 4.1.2 (January 2015).

Manual is not very useful. Mounting from Linux gets half a page. The example given is

mount -t nfs 192.168.1.102:/public /mnt/pub

and this works. But no other dir on QNAP seems to be accessible this way.

I have gone through the networking sections and there is no mention of samba. I guess it could be present but concealed under a different name.

Checked the web site
 

I found several detail, evaluation, review, and (kinda) documentation sites for your device.
While I have never used one, it appears that this device is iSCSI capable.
For fast and easy setup it has CIFS that supports Windows, Linux, and Mac out of the box without going to iSCSI.

The web interface it sports requires less manual than some other devices, and good thing. I found no significant evidence that they have someone who can write an excellent manual. I guess if you have excellent hardware, a better manual would be wasted, but still...

It does not sound like a powerhouse, or a speed daemon, but it does look like a good, solid little NAS device that should serve you well.

Interested that you say CIFS and iSCSI are present. I can't see any indication in the gui, but will persist.

On the general topic, QNAP claims that it will upload dirs from windows, but it turns out that it will only do this through chrome.

I have been looking at the online manual for WD EX 4, for comparison. Again, the focus is on media streaming; nothing about storing files.

Maybe I have a blind spot.

I may go for installing arch on a box of discs. At least learning arch is a worthwhile skill, unlike learning the gui of the moment.

my hokey-pokey d-i-y nas is an old netbook with a cracked screen that i just sshfs into. i had no idea i was doing it wrong.

Point taken, but I do need storage that is secure long-term (say 20 years).

That sounds like a WORN (write once, read never) storage kind of thing ;-)
One thing easy is like schneidz said. If you have a solid hardware base, let's say RAID HDDs and some RAM and proc, you could use anything. A Samba server is easy to set up.
If you want it more advanced, you could try owncloud on your hardware. This gives you full NAS functionality, webdav, samba etc, a gui and all sorts of things. It stores the files 1to1 and you can copy your files and directory structure there using sshfs, scp, ftp... and back them up same way (e.g. rsync) to your preferred backup location.
So even if owncloud wasn#t there in 20 years time, you would still have your directory structure and files to stick them into what will be at hand.

There are many solutions for this kind of storage, I'd make sure to have reliable storage hardware first.