* PV on disk without partitions not recognised as LVM2_member
@ 2016-08-18 20:39 Xen
2016-08-19 9:45 ` Xen
2016-08-19 11:14 ` Karel Zak
0 siblings, 2 replies; 9+ messages in thread
From: Xen @ 2016-08-18 20:39 UTC (permalink / raw)
To: util-linux-ng; +Cc: util-linux
I am not getting a majordomo response (as usual, I guess, within a
reasonable amount of time :p) to subscribe to this list, but....
Perchance,
Would someone be will to fix the issue that a Physical Volume from LVM2
(PV) when placed directly on disk (no partitions or partition tables)
will not be recognised as LVM2_member but rather as something else, such
as "dos" (if nothing else is found) or e.g. some RAID device (if a RAID
signature exists at the end of the device.
Ie. I had a disk that had a "promise fasttrack raid member" (from
memory) signature at the end of the disk (last 1MB) and was recognised
as such. When I wiped the signature, it was recognised as "dos":
/dev/sda: PTUUID="ef39c6e5" PTTYPE="dos"
pvck /dev/sda:
Found label on /dev/sda, sector 1, type=LVM2 001
Found text metadata area: offset=4096, size=1044480
The LVM2 udev rules for lvm-metad depend on blkid currently to report
the nature of a block device such that they can know whether to activate
it; as such a PV directly on disk will not get activated by udev rules.
Will also not get scanned, the whole pvscan --cache --activate ay
command will not get executed.
It seems it would be best to solve it at the root of the issue rather
than changing LVM's udev scripts.
Regards.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PV on disk without partitions not recognised as LVM2_member
2016-08-18 20:39 PV on disk without partitions not recognised as LVM2_member Xen
@ 2016-08-19 9:45 ` Xen
2016-08-19 11:14 ` Karel Zak
1 sibling, 0 replies; 9+ messages in thread
From: Xen @ 2016-08-19 9:45 UTC (permalink / raw)
To: Util linux
Xen schreef op 18-08-2016 22:39:
> Ie. I had a disk that had a "promise fasttrack raid member" (from
> memory) signature at the end of the disk (last 1MB) and was recognised
> as such. When I wiped the signature, it was recognised as "dos":
>
> /dev/sda: PTUUID="ef39c6e5" PTTYPE="dos"
Oh, I mean... the PTTYPE would be "dos" but there would not be any
FS_TYPE description (as per the udev name?). In any case.
Both regular "msdos" partition tables and PV signatures get an empty
"TYPE" in blkid.
So I can match those in udev by just matching the empty string. Both are
also given the same PTTYPE of "dos". So I cannot distinguish between
msdos partition tables and PV disks.
Regards.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PV on disk without partitions not recognised as LVM2_member
2016-08-18 20:39 PV on disk without partitions not recognised as LVM2_member Xen
2016-08-19 9:45 ` Xen
@ 2016-08-19 11:14 ` Karel Zak
2016-08-19 15:10 ` Xen
` (3 more replies)
1 sibling, 4 replies; 9+ messages in thread
From: Karel Zak @ 2016-08-19 11:14 UTC (permalink / raw)
To: Xen; +Cc: util-linux-ng, util-linux
On Thu, Aug 18, 2016 at 10:39:30PM +0200, Xen wrote:
> Would someone be will to fix the issue that a Physical Volume from LVM2 (PV)
> when placed directly on disk (no partitions or partition tables) will not be
This is very unusual setup, but according to feedback from LVM guys
it's supported, so I will improve blkid to support it too.
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PV on disk without partitions not recognised as LVM2_member
2016-08-19 11:14 ` Karel Zak
@ 2016-08-19 15:10 ` Xen
2016-08-20 13:30 ` Xen
` (2 subsequent siblings)
3 siblings, 0 replies; 9+ messages in thread
From: Xen @ 2016-08-19 15:10 UTC (permalink / raw)
To: Util linux; +Cc: Karel Zak
Karel Zak schreef op 19-08-2016 13:14:
> On Thu, Aug 18, 2016 at 10:39:30PM +0200, Xen wrote:
>> Would someone be will to fix the issue that a Physical Volume from
>> LVM2 (PV)
>> when placed directly on disk (no partitions or partition tables) will
>> not be
>
> This is very unusual setup, but according to feedback from LVM guys
> it's supported, so I will improve blkid to support it too.
Thank you.
It is just extremely unusable because hardly anyone can do it yet.
The Grub support isn't there to boot from it (although I have a patch
for that) and without being able to boot it no one would ordinarily use
it. So it's just a chicken and the egg problem here.
As soon as they can get activated, people could start using them, but at
this point it's almost impossible (it requires adjustments to systemd or
the initrd that would be manual adjustments at this point).
Thanks :).
Regards.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PV on disk without partitions not recognised as LVM2_member
2016-08-19 11:14 ` Karel Zak
2016-08-19 15:10 ` Xen
@ 2016-08-20 13:30 ` Xen
2016-08-20 14:13 ` Xen
2016-08-30 10:24 ` Karel Zak
3 siblings, 0 replies; 9+ messages in thread
From: Xen @ 2016-08-20 13:30 UTC (permalink / raw)
To: Util linux; +Cc: Karel Zak
Karel Zak schreef op 19-08-2016 13:14:
> On Thu, Aug 18, 2016 at 10:39:30PM +0200, Xen wrote:
>> Would someone be will to fix the issue that a Physical Volume from
>> LVM2 (PV)
>> when placed directly on disk (no partitions or partition tables) will
>> not be
>
> This is very unusual setup, but according to feedback from LVM guys
> it's supported, so I will improve blkid to support it too.
To provide a little feedback.
What is actually the case is that I am installing Grub2 on the PV which
uses the LVM2 "--bootloaderareasize" option to pvcreate.
This creates a bootloader-area of say 1M with an offset like perhaps
2048 sectors, I don't remember how to get that info, after which then
the extents start.
Oh yes, it is this sort of thing:
physical_volumes {
pv0 {
id = "fEGbBn-tbIp-rL7y-m22b-1rQh-r9i5-Qwlqz7"
device = "/dev/sda" # Hint only
status = ["ALLOCATABLE"]
flags = []
dev_size = 1258287104 # 599,998 Gigabytes
pe_start = 4096
pe_count = 153599 # 599,996 Gigabytes
ba_start = 2048
ba_size = 2048 # 1024 Kilobytes
}
}
Grub would naturally create an MBR signature in the first sector, no
partition table, but bootloader code (boot.img).
The result is this:
5182: libblkid: LOWPROBE: [16] LVM2_member:
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: magic sboff=536, kboff=0
5182: libblkid: LOWPROBE: call probefunc()
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: assigning UUID [superblocks]
5182: libblkid: LOWPROBE: wiper set to superblocks::LVM2_member off=0
size=8192
5182: libblkid: LOWPROBE: assigning TYPE [superblocks]
5182: libblkid: LOWPROBE: <-- leaving probing loop (type=LVM2_member)
[SUBLKS idx=16]
5182: libblkid: LOWPROBE: freeing values list
5182: libblkid: LOWPROBE: chain safeprobe topology DISABLED
5182: libblkid: LOWPROBE: chain safeprobe partitions ENABLED
5182: libblkid: LOWPROBE: reseting partitions values
5182: libblkid: LOWPROBE: --> starting probing loop [PARTS idx=-1]
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: magic sboff=510, kboff=0
5182: libblkid: LOWPROBE: dos: ---> call probefunc()
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: magic sboff=0, kboff=0
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x21e3160
5182: libblkid: LOWPROBE: previously wiped area modified -- ignore
previous results
5182: libblkid: LOWPROBE: zeroize wiper
5182: libblkid: LOWPROBE: reseting superblocks values
5182: libblkid: LOWPROBE: free value UUID
5182: libblkid: LOWPROBE: free value TYPE
5182: libblkid: LOWPROBE: assigning PTUUID [partitions]
5182: libblkid: LOWPROBE: dos: <--- (rc = 0)
5182: libblkid: LOWPROBE: assigning PTTYPE [partitions]
5182: libblkid: LOWPROBE: <-- leaving probing loop (type=dos) [PARTS
idx=3]
And ...that's the real truth of it. I'm sorry, I hadn't realized it
would be due to the bootsector.
When I wipe the bootsector actually it does find LVM2_member.
So I am sorry I didn't mention. The culprit is the Grub2 image.
5261: libblkid: LOWPROBE: [16] LVM2_member:
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: magic sboff=536, kboff=0
5261: libblkid: LOWPROBE: call probefunc()
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: assigning UUID [superblocks]
5261: libblkid: LOWPROBE: wiper set to superblocks::LVM2_member off=0
size=8192
5261: libblkid: LOWPROBE: assigning TYPE [superblocks]
5261: libblkid: LOWPROBE: <-- leaving probing loop (type=LVM2_member)
[SUBLKS idx=16]
5261: libblkid: LOWPROBE: freeing values list
5261: libblkid: LOWPROBE: chain safeprobe topology DISABLED
5261: libblkid: LOWPROBE: chain safeprobe partitions ENABLED
5261: libblkid: LOWPROBE: reseting partitions values
5261: libblkid: LOWPROBE: --> starting probing loop [PARTS idx=-1]
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: gpt: ---> call probefunc()
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: gpt: <--- (rc = 1)
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: ultrix: ---> call probefunc()
5261: libblkid: LOWPROBE: buffer read: off=15872 len=512
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: ultrix: <--- (rc = 1)
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: buffer read: off=28672 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: reuse buffer: off=0 len=1024
pr=0x1c4a0e0
5261: libblkid: LOWPROBE: <-- leaving probing loop (failed=1) [PARTS
idx=11]
5261: libblkid: LOWPROBE: parts: start probing for partition entry
5261: libblkid: LOWPROBE: parts: end probing for partition entry
[nothing]
5261: libblkid: LOWPROBE: partitions probe done [rc=1]
5261: libblkid: LOWPROBE: 0x1c4a0e0: end probe
5261: libblkid: LOWPROBE: zeroize wiper
5261: libblkid: LOWPROBE: returning UUID value
5261: libblkid: TAG: found cache tag head UUID
5261: libblkid: LOWPROBE: returning TYPE value
5261: libblkid: TAG: found cache tag head TYPE
5261: libblkid: PROBE: /dev/sda: devno 0x0800, type LVM2_member
Pretty good debug output, by the way.
Anyway. Thanks to Zdenek Kabelac I found this issue. He knew what was
going on.
Regards.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PV on disk without partitions not recognised as LVM2_member
2016-08-19 11:14 ` Karel Zak
2016-08-19 15:10 ` Xen
2016-08-20 13:30 ` Xen
@ 2016-08-20 14:13 ` Xen
2016-08-22 8:40 ` Karel Zak
2016-08-30 10:24 ` Karel Zak
3 siblings, 1 reply; 9+ messages in thread
From: Xen @ 2016-08-20 14:13 UTC (permalink / raw)
To: Util linux; +Cc: Karel Zak
Karel Zak schreef op 19-08-2016 13:14:
> This is very unusual setup, but according to feedback from LVM guys
> it's supported, so I will improve blkid to support it too.
Actually there are more issues. If you have any firmware RAID signature
at the end of your disk, but you are not using it, blkid will also give
that raid signature precendence over LVM2_member, but LVM depends on
that blkid scan to know what to do; and will fail to scan (even though
pvscan could handle it easily) when the required Udev rule doesn't
match.
So when this part is fixed, anyone who has mistakenly perhaps created a
raid array out of their firmware BIOS, will end up with the same
situation because .... how are we going to let LVM2_member take
precedence just because it uses blkid to decide when to call itself
honey?
I mean I can see good reasons why a RAID signature would take
precedence.
In this case it just disrupts the system LVM2 has set up, in which there
is no redundancy and it depends on an external tool to do its scanning
:-/.
If you replaced that entire udev rule file with just a single command
(and a block device filter) it would just always work with no pain, but
because they want to minimize the number of pvscan invocations (even if
it is just 3 extra invocations that don't harm anyone at every boot) the
system now fails to work. In its entirety. Because it depends on blkid
to provide the right signature.
So basically they could simplify the system and do away with udev in
that sense at the cost of a minimal amount of extra pvscan invocations
that don't really bring any cost to the table. But would they do it?
You now have a problem that cannot be solved:
- blkid cannot be asked to give precedence to LMV2_member over raid
signatures
- while lvm2 chooses to depend on blkid, it won't find all PVs. It
limits its search, but you can't really blame blkid for that unless
blkid started giving multiple results for every device. It would need to
become a list, a set. You can't ask that of blkid just for that right.
So now you have an issue that has no solution. Because they depend on
blkid instead of their own internal code, which wasn't really all that
necessary. I mean it is nice to be needed at times.
But the only solution that would really be painless is to just dump that
lvm2 udev file and replace it with something extremely simple. :-/. ;-).
What can I say. It seems obvious.
I mean that doesn't take away from the current issue with blkid not
recognising PV when Grub is present. But it does show the bigger problem
in this... I think.
Anyway, regards, and sorry for the verbosity at times. Regards.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PV on disk without partitions not recognised as LVM2_member
2016-08-20 14:13 ` Xen
@ 2016-08-22 8:40 ` Karel Zak
2016-08-22 12:16 ` Xen
0 siblings, 1 reply; 9+ messages in thread
From: Karel Zak @ 2016-08-22 8:40 UTC (permalink / raw)
To: Xen; +Cc: Util linux
On Sat, Aug 20, 2016 at 04:13:52PM +0200, Xen wrote:
> Karel Zak schreef op 19-08-2016 13:14:
>
> > This is very unusual setup, but according to feedback from LVM guys
> > it's supported, so I will improve blkid to support it too.
>
> Actually there are more issues. If you have any firmware RAID signature at
> the end of your disk, but you are not using it
You have to use "wipefs" to remove unwanted signatures. We do not
support random mess on disks. It's OS installer, mkfs-like, fdisk
tools and users responsibility to keep on-disk stuff in reliable
state.
It's not about RAIDs only, it's possible write many PT, filesystem and
raid signatures to the same disk.
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PV on disk without partitions not recognised as LVM2_member
2016-08-22 8:40 ` Karel Zak
@ 2016-08-22 12:16 ` Xen
0 siblings, 0 replies; 9+ messages in thread
From: Xen @ 2016-08-22 12:16 UTC (permalink / raw)
To: Util linux; +Cc: Karel Zak
Karel Zak schreef op 22-08-2016 10:40:
> On Sat, Aug 20, 2016 at 04:13:52PM +0200, Xen wrote:
>> Karel Zak schreef op 19-08-2016 13:14:
>>
>> > This is very unusual setup, but according to feedback from LVM guys
>> > it's supported, so I will improve blkid to support it too.
>>
>> Actually there are more issues. If you have any firmware RAID
>> signature at
>> the end of your disk, but you are not using it
>
> You have to use "wipefs" to remove unwanted signatures. We do not
> support random mess on disks. It's OS installer, mkfs-like, fdisk
> tools and users responsibility to keep on-disk stuff in reliable
> state.
Right. I didn't know wipefs would do that. I am approaching this just as
a user from this perspective. It's just a practical issue because RAID
(firmware) (what they call fakeraid) is hard to use on Linux, not that
obvious, dm-raid package is required, it is broken on Ubuntu, you are
not likely to use pmc-whatever-undecipherable-string so fast if it is
just JBOD.
JBOD of single disk (for some controllers that is spare disk, kinda) is
probably raw disk + 1 MB.
I know it is crap, some controllers won't allow decent passthough and
require you to configure every disk as a raid disk, creating the
problems here.
I'll take note of wipefs but just the next user will run into it also.
No shortage of things to learn before you can use your system ;-).
My problem is more lack of redundancy; all tools work together, and they
work together perfectly, but if one slightly fails, the whole thing
collapses, because they are all weakest links.
Whole udev system is liability, a single systemd vgscan service (like
real lvm2.service file, no /etc/init.d) would solve every udev problem
there could ever be with loading devices at boot; vgscan -ay will
activate every device in the book that would also ordinarily have been
activated by udev rules regardless.
Dependability on flawed or fallable or fragile system == .....
pvscan/vgscan/vgscan (I mean vgchange service) has the tools to deal
with duplicate devices resulting from raid-based activation and raw
device (same UUID, it handles it) so system is already resilient at the
core but their udev rules make it break.
Single vgchange -ay would solve all issues but it is contrary to the
design philosophy of streamlining everything with events.
I'm in favour of barriers and completion of stages: activate all LVM you
can, only then activate filesystems.
If you have crypttarget based on LVM activation, sure streamlining it
after the device has come online is not bad, or rather, auto-activate
cryptsetup result using udev is not bad (form of hotplug, you might say)
but manual calls work better and are robust and are resilient.
That's just my opinion. Too many things break in the Linux system when
you change the smallest thing and it needs to "stop" as they say ;-).
Just kidding that. People who think they are awesome people sitting in
their chairs say that a lot "this needs to stop!". Sure. I'm sure it
does. Will you do it? Go back to your kitchen then, and make me some
pie.
Claims of impotence, those are.
Anyway.
> It's not about RAIDs only, it's possible write many PT, filesystem and
> raid signatures to the same disk.
I agree but... this just means that person creating firmware raid array
might cause the system to fail booting only because of blkid interlink
and its interdependence. When otherwise it wouldn't. Filesystem / device
is deemed RAID signature but partition table still loads.
So partition table loading not dependent on blkid == resilient.
Microsoft Windows is notorious for failing to boot when you put a disk
on a different controller etc. Millions, millions of boot problems when
you change things. Change system from IDE to RAID, system will not load.
AHCI same. Loads different driver. Doesn't load all drivers by default.
I think Windows 10 has that sorted now. Not want to talk about Windows,
just explanation of where it goes wrong there. Their boot recovery is
abysmal. It just stinks. It stinks worse than Linux ever was, but Linux
is now also not so resilient anymore.
Don't want to crack down on Linux, I am a Linux user. Still, just
saying.
I would have a lot less headaches (or footaches) if the system was more
robust. Just saying.
Just add an invalid line for /var to /etc/fstab and you will have a
failing boot system.
Users are always asked to edit that file.
nofail and noauto not heeded. Just saying. SystemD pulls it in, you
can't mask it either.
Crypt device that you don't need but it is in /etc/cryttab will require
to be unlocked at boot, there is no intelligence to determine whether it
would actually halt or obstruct the system, also no way to unlock it
after boot, there is no software for that (no LUKS software for that)
that is moderately user friendly.
Post-boot systems perhaps possible with VeraCrypt these days, nothing
else.
VeraCrypt designers cause too many iteration cycles over TrueCrypt, now
unusable system. Not user friendly anymore. Takes too long to unlock
something. Not usable anymore. But they know better than actual users.
Always know better than users.
Sorry for the rant, was not my intent ;-).
Regards, and thanks for your help.
Kudos.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: PV on disk without partitions not recognised as LVM2_member
2016-08-19 11:14 ` Karel Zak
` (2 preceding siblings ...)
2016-08-20 14:13 ` Xen
@ 2016-08-30 10:24 ` Karel Zak
3 siblings, 0 replies; 9+ messages in thread
From: Karel Zak @ 2016-08-30 10:24 UTC (permalink / raw)
To: Xen; +Cc: util-linux
On Fri, Aug 19, 2016 at 01:14:29PM +0200, Karel Zak wrote:
> On Thu, Aug 18, 2016 at 10:39:30PM +0200, Xen wrote:
> > Would someone be will to fix the issue that a Physical Volume from LVM2 (PV)
> > when placed directly on disk (no partitions or partition tables) will not be
>
> This is very unusual setup, but according to feedback from LVM guys
> it's supported, so I will improve blkid to support it too.
Fixed in the git tree (for the next v2.29). Thanks.
Karel
--
Karel Zak <kzak@redhat.com>
http://karelzak.blogspot.com
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2016-08-30 10:24 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-08-18 20:39 PV on disk without partitions not recognised as LVM2_member Xen
2016-08-19 9:45 ` Xen
2016-08-19 11:14 ` Karel Zak
2016-08-19 15:10 ` Xen
2016-08-20 13:30 ` Xen
2016-08-20 14:13 ` Xen
2016-08-22 8:40 ` Karel Zak
2016-08-22 12:16 ` Xen
2016-08-30 10:24 ` Karel Zak
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.