LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware
User Name
Password
Slackware This Forum is for the discussion of Slackware Linux.

Notices


Reply
  Search this Thread
Old 08-23-2016, 08:54 PM   #1
Altiris
Member
 
Registered: Mar 2013
Posts: 556

Rep: Reputation: Disabled
How long does ext4 formatting take on mdadm?


I am installing Slackware 14.2 on mdadm RAID10 with 4 1TB drives. The installer is at the point where it is formatting ext4 on the raid drive (/dev/md0). I understand things may be slow because it is in RAID but can I see a percentage or is there an estimate or how long it should take? I have been waiting about an hour now. When I did a Slackware install on another machine with only 1TB hdd (no raid) it took about 5 seconds to format...so even with 4 HDDs it should not be more than a minute right?
 
Old 08-23-2016, 09:46 PM   #2
TracyTiger
Member
 
Registered: Apr 2011
Location: California, USA
Distribution: Slackware
Posts: 528

Rep: Reputation: 273Reputation: 273Reputation: 273
@Altiris

You may be waiting on the initial building of the arrays. That's usually what takes the most time. Open another terminal screen (Alt F2) and try this command.

Code:
cat /proc/mdstat
or as root

Code:
mdadm --detail /dev/mdx
where "/dev/mdx" is the raid device.

You can see if all the arrays have finished "building" or there are still some in progress.

There may be some better commands but I don't remember any more at the moment.

EDIT: In the past I've run into some problems if I didn't wait for the arrays to build before continuing the installation.

Last edited by TracyTiger; 08-23-2016 at 09:51 PM.
 
2 members found this post helpful.
Old 02-01-2017, 01:35 PM   #3
lazardo
Member
 
Registered: Feb 2010
Location: SD Bay Area
Posts: 270

Rep: Reputation: Disabled
ext4 by default does lazy allocation meaning that after the quick first-stage there's a second stage that runs in the background doing metadata allocation for the entire filesystem.
Code:
iostat -zm 6
mdadm based raid initialization can be optimized two ways.

- The first is
Code:
echo 512000 > /proc/sys/dev/raid/speed_limit_max
, meaning 'use up to 500MB/s of the available I/O subsystem resources for initialization'. This is sane as mdadm has built-in 'ionice'.

- The second is use mdadm's "--layout f2" option for all raid10 including two disk layouts. Originally developed to take advantage of spinning media rotational delay, this will increase large and small block I/O significantly SSD media too.

Mounting the ext4 partition during lazy allocation "-o noatime,nodiratime" will reduce allocation time by nearly half. Not sure why.

If you do the usual, layout the raid array followed by ext4, then ext4 lazy allocation and mdadm initialization are both trying to access the entire array at the same time so the process does take significantly longer than serially, eg, waiting for the mdadm initialization to complete before format.

Cheers,
 
Old 02-01-2017, 01:58 PM   #4
TracyTiger
Member
 
Registered: Apr 2011
Location: California, USA
Distribution: Slackware
Posts: 528

Rep: Reputation: 273Reputation: 273Reputation: 273
What I typically do on a build is to increase the setting "speed_limit_min" to 50,000 or 100,000 and then check that the default "speed_limit_max" is higher than the min setting.

The general idea is to increase the minimum speed from the default of around 1,000.

I don't remember if I actually timed the different settings however.

Last edited by TracyTiger; 02-01-2017 at 02:00 PM. Reason: Fixed tags
 
Old 02-01-2017, 02:58 PM   #5
kikinovak
MLED Founder
 
Registered: Jun 2011
Location: Montpezat (South France)
Distribution: CentOS, OpenSUSE
Posts: 3,453

Rep: Reputation: 2154Reputation: 2154Reputation: 2154Reputation: 2154Reputation: 2154Reputation: 2154Reputation: 2154Reputation: 2154Reputation: 2154Reputation: 2154Reputation: 2154
Quote:
Originally Posted by Altiris View Post
I am installing Slackware 14.2 on mdadm RAID10 with 4 1TB drives. The installer is at the point where it is formatting ext4 on the raid drive (/dev/md0). I understand things may be slow because it is in RAID but can I see a percentage or is there an estimate or how long it should take? I have been waiting about an hour now. When I did a Slackware install on another machine with only 1TB hdd (no raid) it took about 5 seconds to format...so even with 4 HDDs it should not be more than a minute right?
Hi,

Formatting is one thing, but I think you're talking about disk synchronization here. Initial synchronization takes very long. I just finished configuring RAID1 on 2x2 TB, and initial sync took about four hours. I remember a painful installation at a company that had a couple of servers with 4x4 TB hard drives. I installed Slackware using software RAID 5 on these, and the inital sync took more than a day (that is, 24 hours), and this even after increasing the speed_limit_min parameter from 1000 to 50000, otherwise it would have taken roughly a week (if mdstat's ETA is anywhere reliable). The problem here is if you're doing anything else than RAID1, you have to wait until all disks are in sync before the initial reboot.
 
Old 10-24-2018, 11:04 PM   #6
coiks54
Member
 
Registered: Oct 2018
Location: Torino, Italy
Distribution: Always Slackware!
Posts: 31
Blog Entries: 3

Rep: Reputation: Disabled
Also I'm taking more than 24 hours to install the 14.2 on a laptop. If I had known in advance I would have opted for version 13.37, which I already installed on another PC. Now I do not want to interrupt the process since I arrived at the series "n" and for the curiosity to see what has different 14.2 from 13.37, but it is not excluded that after I'll formatting and return on my steps. It's really absurd for a new laptop to stay on for days...
 
Old 10-25-2018, 12:14 AM   #7
Richard Cranium
Senior Member
 
Registered: Apr 2009
Location: McKinney, Texas
Distribution: Slackware64 15.0
Posts: 3,858

Rep: Reputation: 2225Reputation: 2225Reputation: 2225Reputation: 2225Reputation: 2225Reputation: 2225Reputation: 2225Reputation: 2225Reputation: 2225Reputation: 2225Reputation: 2225
The flip side would be to use LVM. Add the raid10 device as a physical volume. Carve out logical volumes as needed out of the PV space. You can always extend a logical volume later, so there's no need to format the entire thing at once.
 
Old 10-25-2018, 12:45 AM   #8
coiks54
Member
 
Registered: Oct 2018
Location: Torino, Italy
Distribution: Always Slackware!
Posts: 31
Blog Entries: 3

Rep: Reputation: Disabled
Thank you Richard, but 1) I do not understand absolutely what you speak and 2) as I said, I almost got to the end (after about 19 hours) and I want to see how it ends.
Hello from Claudio
 
Old 10-25-2018, 05:20 AM   #9
ponce
LQ Guru
 
Registered: Aug 2004
Location: Pisa, Italy
Distribution: Slackware
Posts: 7,097

Rep: Reputation: 4174Reputation: 4174Reputation: 4174Reputation: 4174Reputation: 4174Reputation: 4174Reputation: 4174Reputation: 4174Reputation: 4174Reputation: 4174Reputation: 4174
Hi Claudio,

as I already wrote you in a previous answer please don't revive old threads but open new ones for your problems.

one of the backsides of doing this, for example, is that you confuse people that are trying to help you causing them to waste their time: in specific, you are posting in a thread that is about RAID arrays initialization time and it seems that instead you are speaking about the time needed for installing packages sets
Quote:
Originally Posted by coiks54 View Post
Now I do not want to interrupt the process since I arrived at the series "n"

Last edited by ponce; 10-25-2018 at 05:29 AM.
 
1 members found this post helpful.
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
formatting xp to ext4 pjsr Linux - Newbie 4 10-05-2015 01:52 PM
Problems formatting MDADM array jbruyet Linux - Software 2 11-07-2012 01:19 PM
wrong fs type after formatting ext4 to fat32 imayneed Linux - Newbie 5 08-24-2012 11:36 AM
Formatting external drives in ext4 Cultist Linux - Hardware 1 12-19-2010 07:51 PM
Formatting drive to ext4 simonb1975uk Slackware 2 01-08-2009 11:10 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Slackware

All times are GMT -5. The time now is 01:34 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration