determine ebs volume less than X GB and then expand the volume

sysmicuser · 09-09-2022, 08:22 PM

I ran the command below to determine ebs volume less than 100 GB

df -Ph Output

Code:

df -Ph
Filesystem                       Size  Used Avail Use% Mounted on
udev                             241G     0  241G   0% /dev
tmpfs                             49G  1.6M   49G   1% /run
/dev/sda1                         29G   13G   17G  43% /
tmpfs                            241G  672K  241G   1% /dev/shm
tmpfs                            5.0M     0  5.0M   0% /run/lock
tmpfs                            241G     0  241G   0% /sys/fs/cgroup
/dev/loop1                        68M   68M     0 100% /snap/lxd/22753
/dev/mapper/vg0-lv_lookups_swap  5.0T  119G  4.9T   3% /var/logt/data
/dev/md1                         4.9T  3.7T  1.3T  75% /var/logt/backup
/dev/mapper/vg0-lv_ssd0          495G  121G  375G  25% /var/logt/ssd0
/dev/mapper/vg0-lv_ssd1          495G  122G  373G  25% /var/logt/ssd1
/dev/mapper/vg0-lv_ssd2          495G  121G  375G  25% /var/logt/ssd2
/dev/mapper/vg0-lv_ssd3          495G  121G  375G  25% /var/logt/ssd3
/dev/mapper/vg0-lv_ssd4          495G  209G  287G  43% /var/logt/ssd4
/dev/mapper/vg0-lv_ssd5          495G  211G  285G  43% /var/logt/ssd5
/dev/mapper/vg0-lv_ssd6          495G  208G  288G  42% /var/logt/ssd6
/dev/mapper/vg0-lv_ssd7          495G  210G  286G  43% /var/logt/ssd7
/dev/sdh                         1.3T  1.1T  261G  80% /var/logt/ebs2/b01
/dev/sdd                         1.3T  1.1T  252G  81% /var/logt/ebs0/b01
/dev/sdg                         1.3T  1.1T  264G  80% /var/logt/ebs2/b00
/dev/sdc                         1.3T  1.1T  246G  82% /var/logt/ebs0/b00
/dev/sdj                         1.3T  1.1T  258G  81% /var/logt/ebs3/b01
/dev/sdf                         1.3T  1.1T  252G  81% /var/logt/ebs1/b01
/dev/sde                         1.3T  1.1T  246G  82% /var/logt/ebs1/b00
/dev/sdi                         1.3T  1.1T  258G  81% /var/logt/ebs3/b00
/dev/sdk                         1.3T  1.1T  269G  80% /var/logt/ebs4/b00
/dev/sdl                         1.3T  1.1T  269G  80% /var/logt/ebs4/b01
/dev/sdp                         1.3T  1.1T  274G  79% /var/logt/ebs6/b00
/dev/sdm                         1.3T  1.1T  271G  80% /var/logt/ebs5/b00
/dev/sdq                         1.3T  1.1T  275G  79% /var/logt/ebs6/b01
/dev/sdo                         1.3T 1021G  279G  79% /var/logt/ebs5/b01
/dev/sds                         1.3T 1023G  278G  79% /var/logt/ebs7/b01
/dev/sdr                         1.3T  1.1T  274G  79% /var/logt/ebs7/b00
/dev/sda15                       105M  5.2M  100M   5% /boot/efi
tmpfs                            241G  900K  241G   1% /tmp
/dev/loop6                        47M   47M     0 100% /snap/snapd/16010
/dev/loop7                        47M   47M     0 100% /snap/snapd/16292
/dev/loop3                        62M   62M     0 100% /snap/core20/1611
/dev/loop4                        64M   64M     0 100% /snap/core20/1623
tmpfs                             49G     0   49G   0% /run/user/4038

Code:

df -Ph|grep ebs
/dev/xvdcj                       4.8T  4.6T  132G  98% /var/logt/ebs4/b00
/dev/xvdcl                       4.8T  4.6T  133G  98% /var/logt/ebs5/b00
/dev/xvdcf                       5.0T  4.8T  218G  96% /var/logt/ebs2/b00
/dev/xvdcb                       4.8T  4.7T   85G  99% /var/logt/ebs0/b00
/dev/xvdci                       5.0T  4.8T  235G  96% /var/logt/ebs3/b01
/dev/xvdce                       4.8T  4.6T  113G  98% /var/logt/ebs1/b01
/dev/xvdcn                       5.9T  4.7T  1.2T  80% /var/logt/ebs6/b00
/dev/xvdcy                       5.9T  4.7T  1.3T  80% /var/logt/ebs7/b00
/dev/xvdcd                       4.8T  4.6T  112G  98% /var/logt/ebs1/b00
/dev/xvdcc                       5.9T  4.9T  989G  84% /var/logt/ebs0/b01
/dev/xvdck                       4.8T  4.6T  133G  98% /var/logt/ebs4/b01
/dev/xvdcx                       5.9T  4.8T  1.2T  81% /var/logt/ebs2/b01
/dev/xvdch                       5.9T  4.7T  1.2T  80% /var/logt/ebs3/b00
/dev/xvdco                       5.9T  4.7T  1.2T  80% /var/logt/ebs6/b01
/dev/xvdcq                       5.9T  4.7T  1.3T  80% /var/logt/ebs7/b01
/dev/xvdcm                       4.8T  4.6T  138G  98% /var/logt/ebs5/b01

I would like to grep for any volume which has an "Available" Column less than 100 GB(Variable).

Later use that to modify ebs volume using aws cli aws modify-volume.

Tried the below, but no charm.

Code:

lsblk -Jb | jq -r '..|select(.type? == "disk" and .size? < 100G).name'
jq: error: syntax error, unexpected IDENT, expecting ';' or ')' (Unix shell quoting issues?) at <top-level>, line 1:
..|select(.type? == "disk" and .size? < 100G).name
jq: 1 compile error

Tried with some space,

lsblk -Jb | jq -r '..|select(.type? == "disk" and .size? < 100G ).name'
jq: error: syntax error, unexpected IDENT, expecting ';' or ')' (Unix shell quoting issues?) at <top-level>, line 1:
..|select(.type? == "disk" and .size? < 100G ).name
jq: 1 compile error

lsblk -nblo NAME,SIZE | awk '$2 < 100*2^30 {print $1}'
xvda
xvda1

Still the above does not give the correct ebs volume and I would like the command to work for multiple instances not just one instance.

sysmicuser · 09-09-2022, 09:04 PM

The closest I have come thus far is

Code:

df -Th | grep -vE "tmpfs|nfs|boot|overlay|vfat|cifs"|grep -i ebs| awk '{print $1,$4,$7}'

Output:

Code:

/dev/sdh 1.1T /var/logt/ebs2/b01
/dev/sdd 1.1T /var/logt/ebs0/b01
/dev/sdg 1.1T /var/logt/ebs2/b00
/dev/sdc 1.1T /var/logt/ebs0/b00
/dev/sdj 1.1T /var/logt/ebs3/b01
/dev/sdf 1.1T /var/logt/ebs1/b01
/dev/sde 1.1T /var/logt/ebs1/b00
/dev/sdi 1.1T /var/logt/ebs3/b00
/dev/sdk 1.1T /var/logt/ebs4/b00
/dev/sdl 1.1T /var/logt/ebs4/b01
/dev/sdp 1.1T /var/logt/ebs6/b00
/dev/sdm 1.1T /var/logt/ebs5/b00
/dev/sdq 1.1T /var/logt/ebs6/b01
/dev/sdo 1021G /var/logt/ebs5/b01
/dev/sds 1023G /var/logt/ebs7/b01
/dev/sdr 1.1T /var/logt/ebs7/b00

sysmicuser · 09-09-2022, 09:59 PM

I feel I am close but not yet there

Code:

df -Th | grep -vE "tmpfs|nfs|boot|overlay|vfat|cifs"|grep -i ebs| awk '{print $4 < 300G}'
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
df -Ph
Filesystem                       Size  Used Avail Use% Mounted on
udev                             241G     0  241G   0% /dev
tmpfs                             49G  1.7M   49G   1% /run
/dev/sda1                         29G   13G   17G  43% /
tmpfs                            241G  672K  241G   1% /dev/shm
tmpfs                            5.0M     0  5.0M   0% /run/lock
tmpfs                            241G     0  241G   0% /sys/fs/cgroup
/dev/loop1                        68M   68M     0 100% /snap/lxd/22753
/dev/mapper/vg0-lv_lookups_swap  5.0T  119G  4.9T   3% /var/logt/data
/dev/md1                         4.9T  3.7T  1.3T  75% /var/logt/backup
/dev/mapper/vg0-lv_ssd0          495G  121G  374G  25% /var/logt/ssd0
/dev/mapper/vg0-lv_ssd1          495G  123G  373G  25% /var/logt/ssd1
/dev/mapper/vg0-lv_ssd2          495G  121G  374G  25% /var/logt/ssd2
/dev/mapper/vg0-lv_ssd3          495G  121G  375G  25% /var/logt/ssd3
/dev/mapper/vg0-lv_ssd4          495G  210G  286G  43% /var/logt/ssd4
/dev/mapper/vg0-lv_ssd5          495G  211G  284G  43% /var/logt/ssd5
/dev/mapper/vg0-lv_ssd6          495G  208G  287G  43% /var/logt/ssd6
/dev/mapper/vg0-lv_ssd7          495G  210G  286G  43% /var/logt/ssd7
/dev/sdh                         1.3T  1.1T  261G  80% /var/logt/ebs2/b01
/dev/sdd                         1.3T  1.1T  252G  81% /var/logt/ebs0/b01
/dev/sdg                         1.3T  1.1T  263G  80% /var/logt/ebs2/b00
/dev/sdc                         1.3T  1.1T  246G  82% /var/logt/ebs0/b00
/dev/sdj                         1.3T  1.1T  258G  81% /var/logt/ebs3/b01
/dev/sdf                         1.3T  1.1T  252G  81% /var/logt/ebs1/b01
/dev/sde                         1.3T  1.1T  246G  82% /var/logt/ebs1/b00
/dev/sdi                         1.3T  1.1T  258G  81% /var/logt/ebs3/b00
/dev/sdk                         1.3T  1.1T  269G  80% /var/logt/ebs4/b00
/dev/sdl                         1.3T  1.1T  269G  80% /var/logt/ebs4/b01
/dev/sdp                         1.3T  1.1T  274G  79% /var/logt/ebs6/b00
/dev/sdm                         1.3T  1.1T  271G  80% /var/logt/ebs5/b00
/dev/sdq                         1.3T  1.1T  275G  79% /var/logt/ebs6/b01
/dev/sdo                         1.3T 1021G  279G  79% /var/logt/ebs5/b01
/dev/sds                         1.3T 1023G  278G  79% /var/logt/ebs7/b01
/dev/sdr                         1.3T  1.1T  274G  79% /var/logt/ebs7/b00
/dev/sda15                       105M  5.2M  100M   5% /boot/efi
tmpfs                            241G  900K  241G   1% /tmp
/dev/loop6                        47M   47M     0 100% /snap/snapd/16010
/dev/loop7                        47M   47M     0 100% /snap/snapd/16292
/dev/loop3                        62M   62M     0 100% /snap/core20/1611
/dev/loop4                        64M   64M     0 100% /snap/core20/1623
tmpfs                             49G     0   49G   0% /run/user/4038
tmpfs                             49G     0   49G   0% /run/user/2500

sysmicuser · 09-09-2022, 10:17 PM

The leading solution could be something like this:
Logic, if the Mounted file system is ebs then check if than 100 GB and then print that line

Code:

df -Ph |awk '$6==/var/logt/ebs*'
awk: cmd. line:2: $6==/var/logt/ebs*
awk: cmd. line:2:                   ^ unexpected newline or end of string


df -Ph |awk '$6==/var/logt/ebs?'
awk: cmd. line:3: (END OF FILE)
awk: cmd. line:3:                  ^ source files / command-line arguments must contain complete functions or rules

Life is hard when you do not know the language but you its possible with the language (awk).

sysmicuser · 09-09-2022, 10:46 PM

Getting there!

Code:

df -Ph | grep ebs|awk 'NR > 1 { gsub("%","",$5); if ($4 > 270) { system("echo " $6 " ebs volumes has less than 100GB disk space") }}'
/var/logt/ebs6/b00 ebs volumes has less than 100GB disk space
/var/logt/ebs6/b01 ebs volumes has less than 100GB disk space
/var/logt/ebs5/b01 ebs volumes has less than 100GB disk space
/var/logt/ebs7/b01 ebs volumes has less than 100GB disk space
/var/logt/ebs7/b00 ebs volumes has less than 100GB disk space

syg00 · 09-10-2022, 12:03 AM

Don't edit out your posts - that awk was pretty close from memory, but now you've deleted it ... :sheesh:.

Change the forward slashes in the regex to pipe symbols (logical or as per your grep). The latter awk (post 5) is way too (unnecessarily) complex. Would have commented earlier but had gardening to do.

sysmicuser · 09-10-2022, 01:08 AM

Apologies for commenting as I was trying a few things. I created a consolidated post and used the last one as it works on the latest Ubuntu (20.04) but not the older version (16.04)

Appreciate any guidance as and when you get time.

https://stackoverflow.com/questions/...ired-df-output