LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (http://www.linuxquestions.org/questions/linux-general-1/)
-   -   How to know which logical volume owns a file? (http://www.linuxquestions.org/questions/linux-general-1/how-to-know-which-logical-volume-owns-a-file-819864/)

xzased 07-14-2010 07:04 PM

How to know which logical volume owns a file?
 
Hi. We have a cluster consisting of 10 logical volumes all part of one filesystem. Is there a way to know which logical volume owns a certain file/inode? I have tried what is suggested at this link, but the output is the filesystem and not a specific logical volume. Thanks in advance.

fulldaykg 07-14-2010 08:21 PM

I think a little more info is necessary. Can you post output from a "df -kh" and/or a "mount". Also, "logical volume" is a retty generic term, and it actually depends. Are you using LVM(2)?

xzased 07-14-2010 09:10 PM

Hi. Thanks for your response, this is the output from df -kh:
Code:

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              39G  6.3G  31G  17% /
/dev/sda5              91G  4.0G  82G  5% /var
/dev/sda1            996M  48M  897M  6% /boot
tmpfs                  32G  132K  32G  1% /dev/shm
fs1                  909G  8.9G  897G  1% /fs1
fs2                    33T  17T  17T  50% /fs2
fs3                    19T  14T  4.3T  77% /fs3

And this is the output from mount:
Code:

/dev/sda2 on / type ext3 (rw)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/sda5 on /var type ext3 (rw)
/dev/sda1 on /boot type ext3 (rw)
tmpfs on /dev/shm type tmpfs (rw)
none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw)
fs1 on /fs1 type ext3 (rw,fsname=fs1,segbits=40,root=100000002)
fs2 on /fs2 type ext3 (rw,fsname=fs2,segbits=40,root=100000002)
fs3 on /fs3 type ext3 (rw,fsname=fs3,segbits=40,root=100000002)

we are using lvm2, this version specifically: lvm2-2.02.40-6.el5
This is on a RedHat 5.3 box. The cluster consists of 20 servers connected to iscsi storage. We know the file is under /fs3, but fs3 consists of different volume groups, say volgrp1, volgrp2, volgrp3, etc...

fulldaykg 07-14-2010 10:36 PM

Gotcha, so I'm going to assume what your asking is more along the lines of, "Can I find the Physical Volume a file is actually on?" Since the Logical Volume is already known to be "fs3"...

A quick breakdown, from most physical to most logical:
- The most basic block here is the Physical Volume, which is the actual partition. the PV is broken up into Physical Extents whose size is defined during VG creation (default PE size is 32MB I believe).
- The Volume Group is created by telling it which PV's it's going to have control over. (i.e. VG1 could contain all the 32MB PE's on the PVs /dev/sdb1 and /dev/sdb2)
- Logical Volumes are kinda like the logical version of PVs. For example, when they're created they are broken up into blocks called Logical Extents (usually an LE is the same size as a PE... makes mapping between them easy).
- Logical Volumes can be defined to use a certain amount of a particular VG, but WILL NOT span multiple VGs.
- However since a VG can contain multiple PV's, the mapping of LE's to PE's is not so easy to determine.

An example,
you partition /dev/sdb1 and /dev/sdb2 as PVs (10GB each).
You then create vg1 which is made up of the two PVs (vg1 has 20GB capacity).
Now you create a LV that uses HALF of vg1 (10GB).
You create a 2GB file... Since the kernel is handling the mapping of LEs to PEs within vg1, your file could actually be spread all over the Physical Extents on /dev/sdb1 AND /dev/sdb2.

So if your question is TRULY "What Logical Volume is my file on?" Then if the file in "/fs3" the LV it is on is fs3 (according to your df/mount output).
But if your question is "What Physical Volume is my file on?" I'm afraid you might be out of luck. I don't believe there's a way to SEE the mapping between LE's and PE's

((I hope that made sense, I'm not the best at explaining things))

xzased 07-15-2010 12:21 AM

Thanks fulldaykg. I was looking something more on "What Physical Volume is my file on?" lol... :( But I found your post helpful.

fulldaykg 07-15-2010 05:38 AM

Yep, sorry :-| ... and the reason it's not easy to find is because abstraction from the physical volume is one of the main points of LVM. It's designed so you can add/remove volumes at will and grow and shrink your logical space.

There are commands to move those PE's to a different PV (in case you needed to replace a hard drive or something: "pvmove"), but on the partition level, you're still just moving extents around and knowing what file is in an extent is not possible (to my knowledge).

rayfordj 07-16-2010 01:46 PM

It can be done, but it is not always an easy feat... especially when dealing with LVM because you will now need to take into account the various method used in allocation (linear, mirror, ...) and if there are multiple segments to the LV knowing when and/or how to jump to the next segment on the PV(s) and if you are spanning multiple disks; what the PE size is; what the filesystem block size is; ... and on an on;

This is the simplest demonstration I can think of from one of my systems so I hope you find it useful. This is a demo from creation of the LV so you can see the entire process. I am sure there are other ways to do it, but this is the one I came up with and it has worked for me so far so I have not looked elsewhere.

Code:

# lvcreate -L 512M -n TEST VG00

# mke2fs -j /dev/VG00/TEST

# mount /dev/VG00/TEST /mnt/TEST

# echo LQ-test data > /mnt/TEST/LQ-test.file

# ls -i /mnt/TEST/LQ-test.file
12 /mnt/TEST/LQ-test.file

# echo stat \<12\> | debugfs /dev/VG00/TEST
debugfs 1.41.9 (22-Aug-2009)
debugfs:  stat <12>
Inode: 12  Type: regular    Mode:  0644  Flags: 0x0
Generation: 3843133402    Version: 0x00000000
User:    0  Group:    0  Size: 13
File ACL: 0    Directory ACL: 0
Links: 1  Blockcount: 8
Fragment:  Address: 0    Number: 0    Size: 0
ctime: 0x4c409eba -- Fri Jul 16 13:02:34 2010
atime: 0x4c409ebf -- Fri Jul 16 13:02:39 2010
mtime: 0x4c409eba -- Fri Jul 16 13:02:34 2010
Size of extra inode fields: 4
Extended attributes stored in inode body:
  selinux = "unconfined_u:object_r:file_t:s0\000" (32)
BLOCKS:
(0):28672
TOTAL: 1

# Note the BLOCKS value(s).  Since my example is small enough, \
it only requires a single filesystem block allocation


# dumpe2fs /dev/VG00/TEST | grep "Block size"
Block size:              4096


# dd if=/dev/VG00/TEST bs=4096 count=1 skip=28672 2>/dev/null| xxd
0000000: 4c51 2d74 6573 7420 6461 7461 0a00 0000  LQ-test data....
<snip>


# Ideally, we would use 512k-byte sectors.  I will be doing so going \
forward.  Need to ensure that we get our conversions correctly too.

# echo $(( 28672*4096/512 ))
229376

# dd if=/dev/VG00/TEST bs=512 count=1 skip=229376 2> /dev/null | xxd
0000000: 4c51 2d74 6573 7420 6461 7461 0a00 0000  LQ-test data....
<snip>

# cat /etc/lvm/backup/VG00
<snip>
        extent_size = 8192              # 4 Megabytes
<snip>

        physical_volumes {

                pv0 {
<snip>
                        device = "/dev/sda2"    # Hint only

<snip>
                        pe_start = 384

                }
        }

        logical_volumes {
<snip>
                TEST {
                        id = "GqmTor-O3dP-ElrP-Y2xZ-Jhzo-Q2do-mb48qI"
                        status = ["READ", "WRITE", "VISIBLE"]
                        flags = []
                        segment_count = 1

                        segment1 {
                                start_extent = 0
                                extent_count = 128      # 512 Megabytes

                                type = "striped"
                                stripe_count = 1        # linear

                                stripes = [
                                        "pv0", 46080
                                ]
                        }
<snip>

We will need to know the extent_size, the pe_start, the LV's segment(s), \
number of extents per segment(s), and what pe_extent each segment starts.


# echo $(( 46080*8192 ))
377487360
The LV's starting PE and the extent_size gives us the start of the LV \
that the file is on in 512k-byte sectors not calculating for pe_start


# echo $(( 377487360+384 ))
377487744
We need to remember the pe_start offset to find the _actual_ start of \
the LV that the file is on in 512k-byte sectors


# echo $(( 377487744+229376 ))
377717120
This is where the file starts on the PV.

# dd if=/dev/sda2 bs=512 count=1 skip=377717120 2>/dev/null | xxd
0000000: 4c51 2d74 6573 7420 6461 7461 0a00 0000  LQ-test data....
<snip>


# umount /mnt/TEST

# lvchange -an /dev/VG00/TEST

# lvremove /dev/VG00/TEST


So, as you can see, with a little bit of effort we can identify that the file LQ-test.file with inode 12 on LV TEST resides on partition 2 of sda. If you have multiple blocks in the debugfs output, you should calculate each to identify all of the PVs that the file may reside on if you have multiple PVs in the VG and/or if the LV has multiple PVs/stripes. There is a little more to it if you want to attempt to extract the data, but since my example had everything it needed to demonstrate the entire contents ("LQ-test data") of the file in a single 512K-byte sector I only used count=1.

Knowing the constants for your specific configuration and what to calculate when, this can be scripted fairly easily.


:study:


All times are GMT -5. The time now is 10:16 AM.