[SOLVED] Remove lvm2 metadat from disk

the_gripmaster · 12-07-2011, 12:36 AM

Just found out that destroying the partition table with

Code:

dd if=/dev/zero of=/dev/sda count=1 bs=1K

does not remove the lvm2 related data from the disk; it looks like it resides deep within the disk which pvscan/vgscan/lvscan can pick up. I tried overwriting upto 50MB of the disk, that even did not help. Is there a way to remove any lvm2 related data from the disk without using pvremove command?

(My Ubuntu automated installation (using preseed) is picking up the leftover lvms and showing an error.)

_bsd · 12-07-2011, 06:44 AM

There may be more than one copy of LVM metadata, here's a good description of metadata storage.

The safest way to clear the LVM metadata is using

Code:

vgchange -an vgname
lvremove lvname(s)
vgremove vgname
pvremove pvname(s) # IFF you want to clear device UUID

What do you have against using lvremove?
You can make a copy of /etc/lvm/backup of any LVM metadata you might wish to restore.

If you read the man page for pvcreate, under the --metadatacopies flag info there is also a (reasonably) good explaination of metadata storage options.

Calling the xxremove commands only clears the appropriate blocks, no more, no less.

the_gripmaster · 12-07-2011, 08:16 AM

Quote:

Originally Posted by _bsd

There may be more than one copy of LVM metadata, here's a good description of metadata storage.

The safest way to clear the LVM metadata is using

Code:

vgchange -an vgname
lvremove lvname(s)
vgremove vgname
pvremove pvname(s) # IFF you want to clear device UUID

What do you have against using lvremove?
You can make a copy of /etc/lvm/backup of any LVM metadata you might wish to restore.

If you read the man page for pvcreate, under the --metadatacopies flag info there is also a (reasonably) good explaination of metadata storage options.

Calling the xxremove commands only clears the appropriate blocks, no more, no less.

I want to do a low-level overwrite of the data. Is anything possible using dd if=/dev/zero...?

Right now I am removing the partitions using

Code:

pvremove -y -ff /dev/sda*

_bsd · 12-07-2011, 09:01 AM

In your first post, the command you posted, was overwriting /dev/sda, now you're overwriting partitions.

What did you initialize as a physical volume /dev/sda or /dev/sdaN ?

If you really don't care about any of the data you can overwrite the entire disk with

Code:

dd if=/dev/zero of=/dev/sda bs=4K

and wait for a while, EVERYTHING will be overwritten.

Otherwise, using the appropriate commands to remove
logical volumes - lvremove
volume groups - vgremove
physical volumes - pvremove

in that order, is the fastest, simplest way to clear LVM data.

the_gripmaster · 12-07-2011, 08:18 PM

Quote:

Originally Posted by _bsd

Code:

dd if=/dev/zero of=/dev/sda bs=4K

.

Thanks, the bs=4K without a count is ok for smaller disks but for larger disks like 1TB, it will take hours. This is what I am doing instead, I don't care about preserving old data, all I want to do is make sure the Linux installer treats it as a disk that needs to be partitioned from scratch

Code:

pvremove -ffy /dev/sda*
dd if=/dev/zero of=/dev/sda bs=512 count=1

syg00 · 12-07-2011, 09:17 PM

Change the count to something like 200. You're not overwritting the partition itself, and when you reallocate a new partition, the old (meta-)data is still there.
Shouldn't matter - but might.

Personally I don't like zapping the entire first sector, just the partition table itself. Used to be fdisk and co. wouldn't open the disk once you did that, but the tools seem a little more robust these days. For a while I ran up a patch for fdisk to get around it.

the_gripmaster · 12-07-2011, 09:41 PM

Quote:

Originally Posted by syg00

Change the count to something like 200. You're not overwritting the partition itself, and when you reallocate a new partition, the old (meta-)data is still there.
Shouldn't matter - but might.

Personally I don't like zapping the entire first sector, just the partition table itself. Used to be fdisk and co. wouldn't open the disk once you did that, but the tools seem a little more robust these days. For a while I ran up a patch for fdisk to get around it.

Is there a way to find out where in the disk is the metadata located? Then I could use dd and skip to overwrite that area of the disk.

syg00 · 12-07-2011, 10:37 PM

No idea - don't know, don't care.
Just went back and re-read your initial post. Hmmm. I'm guessing your preseed sets up the same partitions. Without a (re-)format those areas will still have valid data. If the scans are done before anything else that would explain them "just appearing".
Try dd on each of the partitions - for more than 1 sector. Then delete the partitions, then re-try your install.

Just guessing, but seems logical to me.

the_gripmaster · 12-07-2011, 11:20 PM

Quote:

Originally Posted by syg00

Try dd on each of the partitions - for more than 1 sector. Then delete the partitions, then re-try your install.

Yup, that worked. Thanks.

In case anyone's coming back to this post, this is what I did:

Code:

dd if=/dev/zero of=/dev/sda1 bs=512 count=2; \
dd if=/dev/zero of=/dev/sda2 bs=512 count=2; \
dd if=/dev/zero of=/dev/sda bs=512 count=2

syg00 · 12-07-2011, 11:44 PM

As a side note, this wouldn't happen (so often) if pvcreate checked that it was hitting a partition (only) of type x'8e'. Last time I tested it merrily clobbered anything - except a formatted swap extent.

_bsd · 12-08-2011, 04:47 AM

I just saw this discussion on another thread somewhere;
Someone wanted a way to have rm perform checks so that they couldn't remove important directories, files, etc.
eg: rm -rf /usr
would fail

The analagy was like having a loaded gun pointed at your foot, not fire because it knew it was pointed at your foot.

Like all Unix tools, commands, the entire LVM system is extremely powerful, and actually quite simple to use, once one learns how to do so.

Asking a program to protect you from yourself? Isn't that the Mac philosophy?

syg00 · 12-08-2011, 05:14 AM

Then what's the point of having a partition type specifically for LVM ?. Why not just use the Linux type (x'83') ?.

The whole concept of LVM was flawed from the start IMHO - another block device level driver ?. That relied on yet more software (dm). And (initially) had no means to reduce space allocations ?.
C'mon.

It's better now, but still full of holes. Look at all the trouble people still have when disks fail. Or (physically) moving disks around.

_bsd · 12-08-2011, 08:11 AM

I rely on hardware RAID for dealing with disk failure; at present that's an 8x2TB RAID6 on an LSI controller, and an exact duplicate for backup.

One doesn't always use partitions, often it's the entire block level device.
And, for full system encryption, use an encrypted device, then LVM on top.

Quote:

Then what's the point of having a partition type specifically for LVM ?. Why not just use the Linux type (x'83') ?.

I don't know why they created a specific "type" for LVM partitions but you don't have to set that type to be able to call pvcreate, you can init a regular linux type.
It comes in handy when listing [fg]disk -l /dev/sdN for seeing which partitions are LVM and which are not, but pvscan tells me all I need to know about PVs.

In large disk farms, with many different device types, sizes, etc it's great to be able to create volume groups of all these disparate devices, and then carve up into logical units, growing and shrinking as needed. On my desktop I have /usr /home /opt and /vbox as logical volumes, and I can easily add space as needed.

Backups, snapshots are slamdunk easy.

I don't know of any other "free" software that comes close to LVM featurewise.
Personally I've been using one variant or another since '93 and can't imagine living w/o it.

Again, just my $0.02