LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 06-14-2018, 04:55 PM   #1
jmgibson1981
Member
 
Registered: Jun 2015
Location: Tucson, AZ USA
Distribution: Ubuntu LTS
Posts: 224

Rep: Reputation: Disabled
Best filesystem choice for media storage?


Just did the math. Getting a new main storage drive and need to decide a filesystem. Thinking about XFS but I don't know where the benefits are.

Currently main data is 1800 video files, average size is 500 megabytes.

EXT4 or XFS?

*EDIT* Forgot to mention, media server.

Last edited by jmgibson1981; 06-14-2018 at 05:15 PM.
 
Old 06-14-2018, 07:05 PM   #2
jefro
Moderator
 
Registered: Mar 2008
Posts: 18,105

Rep: Reputation: 2709Reputation: 2709Reputation: 2709Reputation: 2709Reputation: 2709Reputation: 2709Reputation: 2709Reputation: 2709Reputation: 2709Reputation: 2709Reputation: 2709
Both are current type filesystems. Good support in kernel and distros.

I'd think that there currently more tools for ext4 than exist on XFS. Companies tend to use XFS as they also have backup plans.

There are other filesystems to look at also if you are brave.

There are a few sites that tend to do shoot outs for various filesystems and kernels. Use them for ideas.

https://www.phoronix.com/scan.php?pa...x-416-fs&num=1

https://www.phoronix.com/scan.php?pa...x-415-fs&num=1

Examples. Most of the tests shown have no effect on media server use. Maybe read test would be most important to look at.

Knowing your data speed requirement may also help. 20 users on mpeg2 versus 3 on mpeg4 may be a large different also.

Last edited by jefro; 06-14-2018 at 07:10 PM.
 
Old 06-14-2018, 10:00 PM   #3
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Ubuntu, Devuan, OpenBSD
Posts: 3,126
Blog Entries: 3

Rep: Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365
Quote:
Originally Posted by jefro View Post
There are other filesystems to look at also if you are brave.
Definitely. I'd recommend at least reading about OpenZFS. In particular the checksumming it has is interesting if you are considering long-term storage. You'll need multiple spindles, around 3 to 6, for the full set advantages though.
 
Old 06-14-2018, 10:44 PM   #4
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 16,653

Rep: Reputation: 2453Reputation: 2453Reputation: 2453Reputation: 2453Reputation: 2453Reputation: 2453Reputation: 2453Reputation: 2453Reputation: 2453Reputation: 2453Reputation: 2453
Don't you just love co-incidences ?. I was reading this just this morning. Sorta like he's been reading my notes - similar thoughts drove me to btrfs years ago.

XFS seems the f/s of choice for people using media servers. ext4 will will of course do the job too.
 
Old 06-15-2018, 12:44 AM   #5
AwesomeMachine
LQ Guru
 
Registered: Jan 2005
Location: USA and Italy
Distribution: Debian testing/sid; OpenSuSE; Fedora; Mint
Posts: 5,009

Rep: Reputation: 901Reputation: 901Reputation: 901Reputation: 901Reputation: 901Reputation: 901Reputation: 901Reputation: 901
My opinion is that ext4 is sort of an patchwork extension of ext3 designed to address some of its limitations for 10 years or so. But it is not going to be continued to ext5. It is specifically designed for Linux, and it's what I use on a laptop or desktop PC for the root directory.

ZFS is a great system with many features, but it requires the hardware to go along with it. And, it requires some work to get it set up so it performs as well as XFS right out of the box. XFS is stone-age technology. But it's really really good, well-thought-out and extensively tested stone-age technology.

I have not used ZFS, because it's simply too much trouble for what I tend to do. I use XFS for a partial Debian mirror consisting of roughly 500.000 files in 600GB. I haven't had a problem with any file system in over four years, I think partially because I use double inversion UPS backup power which supplies clean power and prevents improper shutdowns when utility power is periodically lost.

I don't personally think ext4 is much of a special-use file system. So, given the choice of ext4 or XFS for long term storage on a large volume, I would use XFS.
 
Old 06-15-2018, 06:52 AM   #6
jmgibson1981
Member
 
Registered: Jun 2015
Location: Tucson, AZ USA
Distribution: Ubuntu LTS
Posts: 224

Original Poster
Rep: Reputation: Disabled
I'll read the links, thank you. The only reason I haven't thought of zfs is I didn't want to be tied into a specific thing. For now I've been using snapraid and mergerfs with good results. I like the ability to grow the array as needed instead of having to build it to specs on the initial deployment. I'll see if I can get the cash together to go zfs though. If not I will probably end up with xfs. Appreciate the responses.
 
Old 06-15-2018, 09:49 AM   #7
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Ubuntu, Devuan, OpenBSD
Posts: 3,126
Blog Entries: 3

Rep: Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365Reputation: 1365
ZFS does have its shortcomings but it can be tuned quite a bit and so things not needed like de-duplication can be avoided to save resources. Either way, let us know what you chose and how it worked out. It would be very interesting to know how either one work out for your use-case.
 
Old 06-18-2018, 09:47 AM   #8
jmgibson1981
Member
 
Registered: Jun 2015
Location: Tucson, AZ USA
Distribution: Ubuntu LTS
Posts: 224

Original Poster
Rep: Reputation: Disabled
My end result. This drive is more of a stopgap solution to hold me over for a bit as I get cash together to build something more proper for my needs. Ext4, backed up via snapraid. I will be building something around zfs in the near future assuming all goes according to plan.

Again thank you for replies.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Prozgui Error: Not enough storage space or storage media fails to create file debasishk Linux - Software 0 12-31-2013 07:46 PM
filesystem for storage? qrange Linux - Newbie 4 09-15-2011 10:24 PM
Filesystem for 45TB of storage genderbender Linux - General 17 02-05-2011 06:34 PM
choosing a filesystem for storage jimk Linux - General 4 06-13-2005 01:39 AM
XFS vs ext3 vs RieserFS Filesystem choice orbiteleven Linux - General 2 03-24-2004 04:53 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:40 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration