LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-13-2015, 03:03 AM   #1
kaliyugantagonist
LQ Newbie
 
Registered: Feb 2015
Posts: 2

Rep: Reputation: Disabled
Question Allocating space to mounts


My real objective is to clean existing nodes from a Hadoop cluster, claim the space etc. and do a fresh installation of another distro.

Below is the space descr. of one of the master(namenode):

Code:
df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg00-root
                      2.0G  739M  1.2G  39% /
tmpfs                  24G     0   24G   0% /dev/shm
/dev/mapper/mpathap1  194M   65M  120M  36% /boot
/dev/mapper/vg00-home
                      248M   12M  224M   5% /home
/dev/mapper/vg00-nsr  248M   11M  226M   5% /nsr
/dev/mapper/vg00-opt  3.1G   79M  2.8G   3% /opt
/dev/mapper/vg00-itm  434M   11M  402M   3% /opt/IBM/ITM
/dev/mapper/vg00-tmp  2.0G   68M  1.9G   4% /tmp
/dev/mapper/vg00-usr  2.0G  1.6G  305M  85% /usr
/dev/mapper/vg00-usr_local
                      248M   11M  226M   5% /usr/local
/dev/mapper/vg00-var  2.0G  820M  1.1G  43% /var
/dev/mapper/vg00-FSImage
                      917G  3.3G  867G   1% /opt/hadoop-FSImage
/dev/mapper/vg00-Zookeeper
                      917G  200M  870G   1% /opt/hadoop-Zookeeper
And one of the slave(datanodes)(the others too have 4-8 local disks and some mounted NFS drives):

Code:
df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg00-root
                      2.0G  411M  1.5G  22% /
tmpfs                  36G     0   36G   0% /dev/shm
/dev/mapper/mpathap1  194M   67M  118M  37% /boot
/dev/mapper/vg00-home
                      248M   11M  226M   5% /home
/dev/mapper/vg00-nsr  248M   11M  226M   5% /nsr
/dev/mapper/vg00-opt  3.1G   82M  2.8G   3% /opt
/dev/mapper/vg00-itm  434M   11M  402M   3% /opt/IBM/ITM
/dev/mapper/vg00-tmp  2.0G   68M  1.9G   4% /tmp
/dev/mapper/vg00-usr  2.0G  1.2G  690M  64% /usr
/dev/mapper/vg00-usr_local
                      248M   11M  226M   5% /usr/local
/dev/mapper/vg00-var  2.0G  1.5G  392M  80% /var
/dev/mapper/vg00-00   559G   33M  559G   1% /opt/hadoop-00
/dev/mapper/vg00-01   559G   33M  559G   1% /opt/hadoop-01
/dev/mapper/vg00-02   559G   33M  559G   1% /opt/hadoop-02
/dev/mapper/vg00-03   559G   33M  559G   1% /opt/hadoop-03
/dev/mapper/vg00-04   559G   33M  559G   1% /opt/hadoop-04
/dev/mapper/vg00-05   559G   33M  559G   1% /opt/hadoop-05
/dev/mapper/vg00-06   559G   33M  559G   1% /opt/hadoop-06
/dev/mapper/vg00-07   559G   33M  559G   1% /opt/hadoop-07
During the new installation, I have run into space issues for all the nodes/hosts

Code:
Not enough disk space on host (l1032lab.se.com). A minimum of 1GB is required for "/usr" mount. A minimum of 2GB is required for "/" mount.
Not enough disk space on host (l1033lab.se.com). A minimum of 1GB is required for "/usr" mount. A minimum of 2GB is required for "/" mount.
Not enough disk space on host (l1034lab.se.com). A minimum of 1GB is required for "/usr" mount. A minimum of 2GB is required for "/" mount.
Not enough disk space on host (l1035lab.se.com). A minimum of 1GB is required for "/usr" mount. A minimum of 2GB is required for "/" mount.
Now the installation requires a lot of space under /, /usr, /var and /home mounts which on both master and the slaves is less but there is lots of space in master under these two disks:

Code:
/dev/mapper/vg00-FSImage
                      917G  200M  870G   1% /opt/hadoop-FSImage
/dev/mapper/vg00-Zookeeper
                      917G  6.5G  864G   1% /opt/hadoop-Zookeeper
and on slaves under these 8 disks.

Code:
/dev/mapper/vg00-00   559G   33M  559G   1% /opt/hadoop-00
/dev/mapper/vg00-01   559G   33M  559G   1% /opt/hadoop-01
/dev/mapper/vg00-02   559G   33M  559G   1% /opt/hadoop-02
/dev/mapper/vg00-03   559G   33M  559G   1% /opt/hadoop-03
/dev/mapper/vg00-04   559G   33M  559G   1% /opt/hadoop-04
/dev/mapper/vg00-05   559G   33M  559G   1% /opt/hadoop-05
/dev/mapper/vg00-06   559G   33M  559G   1% /opt/hadoop-06
/dev/mapper/vg00-07   559G   33M  559G   1% /opt/hadoop-07
I'm a bit confused as how to proceed - I was wondering if I can partition one or more of these disks and allocate those under /, /usr etc. and proceed with the installation but then will these disk partitioning and mounting/un-mounting corrupt the existing /, /usr etc. mounts ?
 
Old 02-13-2015, 06:27 AM   #2
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,697

Rep: Reputation: 1261Reputation: 1261Reputation: 1261Reputation: 1261Reputation: 1261Reputation: 1261Reputation: 1261Reputation: 1261Reputation: 1261
What distribution are you currently using, and which one are you trying to go to?

A number of distributions have moved to combining root and /usr into one filesystem. In most cases you will need more than 10G for a root+usr filesystem (even 20G can be too small - but that would be a workstation environment, right now I'm using 20G out of 100G for root+usr, but there are a number of desktops installed for testing).

Since your mounts show a minimum usage of tmpfs (/tmp,/run...) I suspect your current distribution is out of date...

If you are distributing data via NFS, then I would also suspect I/O performance problems... (might check into using gluster for network access - it is a distributed file system where performance increases as you add disk servers due to load balancing, and I've been told it has a faster base service than NFS).

You also might consider changing the filesystem -- btrfs should be able to handle the allocations pools itself better than LVM (I believe it is due to less overhead, but also due to better handling in general).

Last edited by jpollard; 02-13-2015 at 06:31 AM.
 
Old 02-13-2015, 07:27 AM   #3
kaliyugantagonist
LQ Newbie
 
Registered: Feb 2015
Posts: 2

Original Poster
Rep: Reputation: Disabled
Question Reallocation failing

Quote:
Originally Posted by jpollard View Post
What distribution are you currently using, and which one are you trying to go to?

A number of distributions have moved to combining root and /usr into one filesystem. In most cases you will need more than 10G for a root+usr filesystem (even 20G can be too small - but that would be a workstation environment, right now I'm using 20G out of 100G for root+usr, but there are a number of desktops installed for testing).

Since your mounts show a minimum usage of tmpfs (/tmp,/run...) I suspect your current distribution is out of date...

If you are distributing data via NFS, then I would also suspect I/O performance problems... (might check into using gluster for network access - it is a distributed file system where performance increases as you add disk servers due to load balancing, and I've been told it has a faster base service than NFS).

You also might consider changing the filesystem -- btrfs should be able to handle the allocations pools itself better than LVM (I believe it is due to less overhead, but also due to better handling in general).
RHEL version

Code:
[root@l1033lab ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.5 (Santiago)
I'm facing the below issue when trying to allocate space using LVM :

Code:
[root@l1033lab ~]# lvextend -L8G /dev/mapper/vg00-root
  Extending logical volume root to 8.00 GiB
  Insufficient free space: 192 extents needed, but only 31 available
Following are the outputs of the different LVM commands which can help in analysing the problem :

Code:
[root@l1033lab ~]# pvs
  PV                   VG   Fmt  Attr PSize   PFree
  /dev/mapper/mpathap2 vg00 lvm2 a--   29.28g 992.00m
  /dev/sda             vg00 lvm2 a--  558.88g      0
  /dev/sdb             vg00 lvm2 a--  558.88g      0
  /dev/sdc             vg00 lvm2 a--  558.88g      0
  /dev/sdd             vg00 lvm2 a--  558.88g      0
  /dev/sde             vg00 lvm2 a--  558.88g      0
  /dev/sdf             vg00 lvm2 a--  558.88g      0
  /dev/sdg             vg00 lvm2 a--  558.88g      0
  /dev/sdh             vg00 lvm2 a--  558.88g      0
[root@l1033lab ~]#
[root@l1033lab ~]# vgs
  VG   #PV #LV #SN Attr   VSize VFree
  vg00   9  18   0 wz--n- 4.39t 992.00m
[root@l1033lab ~]#
[root@l1033lab ~]# lvs
  LV        VG   Attr       LSize   Pool Origin Data%  Move Log Cpy%Sync Convert
  00        vg00 -wi-ao---- 558.88g
  01        vg00 -wi-ao---- 558.88g
  02        vg00 -wi-ao---- 558.88g
  03        vg00 -wi-ao---- 558.88g
  04        vg00 -wi-ao---- 558.88g
  05        vg00 -wi-ao---- 558.88g
  06        vg00 -wi-ao---- 558.88g
  07        vg00 -wi-ao---- 558.88g
  home      vg00 -wi-ao---- 256.00m
  itm       vg00 -wi-ao---- 448.00m
  nsr       vg00 -wi-ao---- 256.00m
  opt       vg00 -wi-ao----   3.12g
  root      vg00 -wi-ao----   2.00g
  swap00    vg00 -wi-ao----  16.00g
  tmp       vg00 -wi-ao----   2.00g
  usr       vg00 -wi-ao----   2.00g
  usr_local vg00 -wi-ao---- 256.00m
  var       vg00 -wi-ao----   2.00g
[root@l1033lab ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg00-root
                      2.0G  411M  1.5G  22% /
tmpfs                  36G     0   36G   0% /dev/shm
/dev/mapper/mpathap1  194M   67M  118M  37% /boot
/dev/mapper/vg00-home
                      248M   11M  226M   5% /home
/dev/mapper/vg00-nsr  248M   11M  226M   5% /nsr
/dev/mapper/vg00-opt  3.1G   82M  2.8G   3% /opt
/dev/mapper/vg00-itm  434M   11M  402M   3% /opt/IBM/ITM
/dev/mapper/vg00-tmp  2.0G   68M  1.9G   4% /tmp
/dev/mapper/vg00-usr  2.0G  1.2G  687M  65% /usr
/dev/mapper/vg00-usr_local
                      248M   11M  226M   5% /usr/local
/dev/mapper/vg00-var  2.0G  1.6G  378M  81% /var
/dev/mapper/vg00-00   559G   33M  559G   1% /opt/hadoop-00
/dev/mapper/vg00-01   559G   33M  559G   1% /opt/hadoop-01
/dev/mapper/vg00-02   559G   33M  559G   1% /opt/hadoop-02
/dev/mapper/vg00-03   559G   33M  559G   1% /opt/hadoop-03
/dev/mapper/vg00-04   559G   33M  559G   1% /opt/hadoop-04
/dev/mapper/vg00-05   559G   33M  559G   1% /opt/hadoop-05
/dev/mapper/vg00-06   559G   33M  559G   1% /opt/hadoop-06
/dev/mapper/vg00-07   559G   33M  559G   1% /opt/hadoop-07
[root@l1033lab ~]# pvdisplay
  --- Physical volume ---
  PV Name               /dev/mapper/mpathap2
  VG Name               vg00
  PV Size               29.30 GiB / not usable 16.00 MiB
  Allocatable           yes
  PE Size               32.00 MiB
  Total PE              937
  Free PE               31
  Allocated PE          906
  PV UUID               rg7did-enjd-rzbK-OyYd-PF2X-NwGg-ObKYk5

  --- Physical volume ---
  PV Name               /dev/sdc
  VG Name               vg00
  PV Size               558.88 GiB / not usable 5.96 MiB
  Allocatable           yes (but full)
  PE Size               32.00 MiB
  Total PE              17884
  Free PE               0
  Allocated PE          17884
  PV UUID               Z1KIM3-BtrD-UllL-Lw2f-NWOP-1IBA-aD36Xb

  --- Physical volume ---
  PV Name               /dev/sdd
  VG Name               vg00
  PV Size               558.88 GiB / not usable 5.96 MiB
  Allocatable           yes (but full)
  PE Size               32.00 MiB
  Total PE              17884
  Free PE               0
  Allocated PE          17884
  PV UUID               rhM16S-XiAK-lNWh-Vy6L-9dco-2vpQ-48WLvw

  --- Physical volume ---
  PV Name               /dev/sde
  VG Name               vg00
  PV Size               558.88 GiB / not usable 5.96 MiB
  Allocatable           yes (but full)
  PE Size               32.00 MiB
  Total PE              17884
  Free PE               0
  Allocated PE          17884
  PV UUID               crLinu-OY1W-qQJM-8FsC-WvJM-ISNN-uqlBFt

  --- Physical volume ---
  PV Name               /dev/sdf
  VG Name               vg00
  PV Size               558.88 GiB / not usable 5.96 MiB
  Allocatable           yes (but full)
  PE Size               32.00 MiB
  Total PE              17884
  Free PE               0
  Allocated PE          17884
  PV UUID               LYcCvY-GfgA-zmrR-ORns-aR50-SZkF-7HLlUj

  --- Physical volume ---
  PV Name               /dev/sdg
  VG Name               vg00
  PV Size               558.88 GiB / not usable 5.96 MiB
  Allocatable           yes (but full)
  PE Size               32.00 MiB
  Total PE              17884
  Free PE               0
  Allocated PE          17884
  PV UUID               yTVSAe-ZFpn-6WpI-qdTm-mUzH-QU3S-r4TqRr

  --- Physical volume ---
  PV Name               /dev/sdh
  VG Name               vg00
  PV Size               558.88 GiB / not usable 5.96 MiB
  Allocatable           yes (but full)
  PE Size               32.00 MiB
  Total PE              17884
  Free PE               0
  Allocated PE          17884
  PV UUID               H0vQMP-LhJx-Ecb9-0pC5-9VuL-0PiU-k3RVfr

  --- Physical volume ---
  PV Name               /dev/sdb
  VG Name               vg00
  PV Size               558.88 GiB / not usable 5.96 MiB
  Allocatable           yes (but full)
  PE Size               32.00 MiB
  Total PE              17884
  Free PE               0
  Allocated PE          17884
  PV UUID               ZP4sQA-65G3-QoJT-hQ5m-UExN-NkbY-PFxa2F

  --- Physical volume ---
  PV Name               /dev/sda
  VG Name               vg00
  PV Size               558.88 GiB / not usable 5.96 MiB
  Allocatable           yes (but full)
  PE Size               32.00 MiB
  Total PE              17884
  Free PE               0
  Allocated PE          17884
  PV UUID               RmHYi9-1UoZ-s8Eo-G0Ds-w3AH-qxzI-nliCYe
The ĎPFreeí is almost NIL on all the slave nodes which is something Iím unable to figure out but I guess thatís why lvextend isnít working.

Last edited by kaliyugantagonist; 02-13-2015 at 07:57 AM. Reason: Added pvdisplay output
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Allocating space to the /tmp directory macanucaire Linux - Newbie 5 06-25-2012 08:17 PM
Allocating hard drive space Armon Linux - General 4 05-05-2005 08:26 PM
Need Help Allocating Space on my HD to install linux! chrishikel Linux - Newbie 6 09-10-2004 02:55 PM
allocating space for new partition ranger12002 Linux - General 12 07-20-2003 03:07 PM
allocating space and a couple other things Cheese Linux - Software 2 12-09-2001 03:55 PM


All times are GMT -5. The time now is 12:31 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration