grubenv on md, Btrfs, LUKS, etc

grubenv on md, Btrfs, LUKS, etc

[Chris Murphy]

Hi,

GRUB code can certainly read files that are on Btrfs, md devices,
LUKS, LVM, and so on. But GRUB code can also write to the physical
block for grubenv - but are there safe guards that prevent it from
doing so if grubenv is on something like Btrfs, mdadm raid5, LUKS?

And also what about XFS? This used to be safe, but now with reflink
support, grubenv could be reflink copied, meaning any overwrite is
disallowed and must be COW'd. How is that handled?

I'm pretty sure on Btrfs GRUB knows is can't write to grubenv, I'm
just curious about the other cases.

-- 
Chris Murphy

Re: grubenv on md, Btrfs, LUKS, etc

[Chris Murphy]

On Mon, Sep 17, 2018 at 10:59 PM, Chris Murphy <lists@colorremedies.com> wrote:
> Hi,
>
> GRUB code can certainly read files that are on Btrfs, md devices,
> LUKS, LVM, and so on. But GRUB code can also write to the physical
> block for grubenv - but are there safe guards that prevent it from
> doing so if grubenv is on something like Btrfs, mdadm raid5, LUKS?
>
> And also what about XFS? This used to be safe, but now with reflink
> support, grubenv could be reflink copied, meaning any overwrite is
> disallowed and must be COW'd. How is that handled?
>
> I'm pretty sure on Btrfs GRUB knows is can't write to grubenv, I'm
> just curious about the other cases.

OK so it allows me to create a grubenv on Btrfs without any complaint.
Will the bootloader actually try to write to this if grub.cfg contains
save_env?

$ sudo grub2-editenv --verbose grubenv create
[sudo] password for chris:
[chris@f29h ~]$ ll
-rw-r--r--. 1 root  root     1024 Sep 18 13:37 grubenv
[chris@f29h ~]$ stat -f grubenv
  File: "grubenv"
    ID: ac9ba8ecdce5b017 Namelen: 255     Type: btrfs
Block size: 4096       Fundamental block size: 4096
Blocks: Total: 46661632   Free: 37479747   Available: 37422535
Inodes: Total: 0          Free: 0
[chris@f29h ~]$


-- 
Chris Murphy

Re: grubenv on md, Btrfs, LUKS, etc

[Daniel Kiper]

On Tue, Sep 18, 2018 at 01:40:06PM -0600, Chris Murphy wrote:
> On Mon, Sep 17, 2018 at 10:59 PM, Chris Murphy <lists@colorremedies.com> wrote:
> > Hi,
> >
> > GRUB code can certainly read files that are on Btrfs, md devices,
> > LUKS, LVM, and so on. But GRUB code can also write to the physical
> > block for grubenv - but are there safe guards that prevent it from
> > doing so if grubenv is on something like Btrfs, mdadm raid5, LUKS?
> >
> > And also what about XFS? This used to be safe, but now with reflink
> > support, grubenv could be reflink copied, meaning any overwrite is
> > disallowed and must be COW'd. How is that handled?
> >
> > I'm pretty sure on Btrfs GRUB knows is can't write to grubenv, I'm
> > just curious about the other cases.
>
> OK so it allows me to create a grubenv on Btrfs without any complaint.
> Will the bootloader actually try to write to this if grub.cfg contains
> save_env?
>
> $ sudo grub2-editenv --verbose grubenv create
> [sudo] password for chris:
> [chris@f29h ~]$ ll
> -rw-r--r--. 1 root  root     1024 Sep 18 13:37 grubenv
> [chris@f29h ~]$ stat -f grubenv
>   File: "grubenv"
>     ID: ac9ba8ecdce5b017 Namelen: 255     Type: btrfs
> Block size: 4096       Fundamental block size: 4096
> Blocks: Total: 46661632   Free: 37479747   Available: 37422535
> Inodes: Total: 0          Free: 0
> [chris@f29h ~]$

There two things here. grub2-editenv should create grubenv properly
and double check that space on disk is reserved. If it is not then
it should complain. GRUB2 during boot should check was grubenv file
properly created. If it was not it should not update grubenv and
complain.

I am not sure is anything like that implemented in GRUB2. If does
not I am happy to see and review the patches.

Daniel

Re: grubenv on md, Btrfs, LUKS, etc

[Chris Murphy]

On Fri, Sep 21, 2018 at 12:35 PM, Daniel Kiper <dkiper@net-space.pl> wrote:
> On Tue, Sep 18, 2018 at 01:40:06PM -0600, Chris Murphy wrote:
>> On Mon, Sep 17, 2018 at 10:59 PM, Chris Murphy <lists@colorremedies.com> wrote:
>> > Hi,
>> >
>> > GRUB code can certainly read files that are on Btrfs, md devices,
>> > LUKS, LVM, and so on. But GRUB code can also write to the physical
>> > block for grubenv - but are there safe guards that prevent it from
>> > doing so if grubenv is on something like Btrfs, mdadm raid5, LUKS?
>> >
>> > And also what about XFS? This used to be safe, but now with reflink
>> > support, grubenv could be reflink copied, meaning any overwrite is
>> > disallowed and must be COW'd. How is that handled?
>> >
>> > I'm pretty sure on Btrfs GRUB knows is can't write to grubenv, I'm
>> > just curious about the other cases.
>>
>> OK so it allows me to create a grubenv on Btrfs without any complaint.
>> Will the bootloader actually try to write to this if grub.cfg contains
>> save_env?
>>
>> $ sudo grub2-editenv --verbose grubenv create
>> [sudo] password for chris:
>> [chris@f29h ~]$ ll
>> -rw-r--r--. 1 root  root     1024 Sep 18 13:37 grubenv
>> [chris@f29h ~]$ stat -f grubenv
>>   File: "grubenv"
>>     ID: ac9ba8ecdce5b017 Namelen: 255     Type: btrfs
>> Block size: 4096       Fundamental block size: 4096
>> Blocks: Total: 46661632   Free: 37479747   Available: 37422535
>> Inodes: Total: 0          Free: 0
>> [chris@f29h ~]$
>
> There two things here. grub2-editenv should create grubenv properly
> and double check that space on disk is reserved. If it is not then
> it should complain. GRUB2 during boot should check was grubenv file
> properly created. If it was not it should not update grubenv and
> complain.
>
> I am not sure is anything like that implemented in GRUB2. If does
> not I am happy to see and review the patches.

When I create a grubenv on Btrfs is does succeed and it's an inline
extent, so no mattter what it's checksummed. There is a message on the
next boot:

error: ../../grub-core/commands/loadenv.c:215:sparse file not allowed.

And the grubenv is not overwritten even though the grub.cfg is using
save_env and when this same grub.cfg is used on ext4 it does overwrite
grubenv. So... It appears loadenv.c must have some inhibitor for
writing to Btrfs, which is a good thing.

I'm uncertain whether GRUB avoids writing to any other case (LUKS, md
RAID, LVM). In particular, XFS, because XFS now supports reflinks, so
strictly speaking even if overwriting 2 sectors does not cause
corruption today (no inline extent support yet), it probably should
refuse to write to XFS as well.

Anyway, I've got a couple of different opinions from XFS devel@ and
ext4 devel@ about this. My understanding is each file system (ext4,
XFS, Btrfs at least) have reserve areas that can reliably be written
to by the bootloader (pre-boot), and it seems like those need to be
used instead of depending on grubenv.

https://www.spinics.net/lists/linux-ext4/msg62364.html

https://www.spinics.net/lists/linux-xfs/msg21902.html


Thanks,

-- 
Chris Murphy

Re: grubenv on md, Btrfs, LUKS, etc

[Daniel Kiper]

On Wed, Sep 26, 2018 at 04:45:51PM -0600, Chris Murphy wrote:
> On Fri, Sep 21, 2018 at 12:35 PM, Daniel Kiper <dkiper@net-space.pl> wrote:
> > On Tue, Sep 18, 2018 at 01:40:06PM -0600, Chris Murphy wrote:
> >> On Mon, Sep 17, 2018 at 10:59 PM, Chris Murphy <lists@colorremedies.com> wrote:
> >> > Hi,
> >> >
> >> > GRUB code can certainly read files that are on Btrfs, md devices,
> >> > LUKS, LVM, and so on. But GRUB code can also write to the physical
> >> > block for grubenv - but are there safe guards that prevent it from
> >> > doing so if grubenv is on something like Btrfs, mdadm raid5, LUKS?
> >> >
> >> > And also what about XFS? This used to be safe, but now with reflink
> >> > support, grubenv could be reflink copied, meaning any overwrite is
> >> > disallowed and must be COW'd. How is that handled?
> >> >
> >> > I'm pretty sure on Btrfs GRUB knows is can't write to grubenv, I'm
> >> > just curious about the other cases.
> >>
> >> OK so it allows me to create a grubenv on Btrfs without any complaint.
> >> Will the bootloader actually try to write to this if grub.cfg contains
> >> save_env?
> >>
> >> $ sudo grub2-editenv --verbose grubenv create
> >> [sudo] password for chris:
> >> [chris@f29h ~]$ ll
> >> -rw-r--r--. 1 root  root     1024 Sep 18 13:37 grubenv
> >> [chris@f29h ~]$ stat -f grubenv
> >>   File: "grubenv"
> >>     ID: ac9ba8ecdce5b017 Namelen: 255     Type: btrfs
> >> Block size: 4096       Fundamental block size: 4096
> >> Blocks: Total: 46661632   Free: 37479747   Available: 37422535
> >> Inodes: Total: 0          Free: 0
> >> [chris@f29h ~]$
> >
> > There two things here. grub2-editenv should create grubenv properly
> > and double check that space on disk is reserved. If it is not then
> > it should complain. GRUB2 during boot should check was grubenv file
> > properly created. If it was not it should not update grubenv and
> > complain.
> >
> > I am not sure is anything like that implemented in GRUB2. If does
> > not I am happy to see and review the patches.
>
> When I create a grubenv on Btrfs is does succeed and it's an inline
> extent, so no mattter what it's checksummed. There is a message on the
> next boot:
>
> error: ../../grub-core/commands/loadenv.c:215:sparse file not allowed.
>
> And the grubenv is not overwritten even though the grub.cfg is using
> save_env and when this same grub.cfg is used on ext4 it does overwrite
> grubenv. So... It appears loadenv.c must have some inhibitor for
> writing to Btrfs, which is a good thing.

Great! That is in line with what I said earlier.

> I'm uncertain whether GRUB avoids writing to any other case (LUKS, md
> RAID, LVM). In particular, XFS, because XFS now supports reflinks, so
> strictly speaking even if overwriting 2 sectors does not cause
> corruption today (no inline extent support yet), it probably should
> refuse to write to XFS as well.

Yep!

> Anyway, I've got a couple of different opinions from XFS devel@ and
> ext4 devel@ about this. My understanding is each file system (ext4,
> XFS, Btrfs at least) have reserve areas that can reliably be written
> to by the bootloader (pre-boot), and it seems like those need to be
> used instead of depending on grubenv.
>
> https://www.spinics.net/lists/linux-ext4/msg62364.html
>
> https://www.spinics.net/lists/linux-xfs/msg21902.html

If something like that exits I am happy to use it. However, I would not
change user interface in any way. Everything should happen auto-magically
from user POV.

Daniel

Re: grubenv on md, Btrfs, LUKS, etc

[Chris Murphy]

On Thu, Sep 27, 2018 at 10:38 AM, Daniel Kiper <dkiper@net-space.pl> wrote:
> On Wed, Sep 26, 2018 at 04:45:51PM -0600, Chris Murphy wrote:
>> On Fri, Sep 21, 2018 at 12:35 PM, Daniel Kiper <dkiper@net-space.pl> wrote:
>> > On Tue, Sep 18, 2018 at 01:40:06PM -0600, Chris Murphy wrote:
>> >> On Mon, Sep 17, 2018 at 10:59 PM, Chris Murphy <lists@colorremedies.com> wrote:
>> >> > Hi,
>> >> >
>> >> > GRUB code can certainly read files that are on Btrfs, md devices,
>> >> > LUKS, LVM, and so on. But GRUB code can also write to the physical
>> >> > block for grubenv - but are there safe guards that prevent it from
>> >> > doing so if grubenv is on something like Btrfs, mdadm raid5, LUKS?
>> >> >
>> >> > And also what about XFS? This used to be safe, but now with reflink
>> >> > support, grubenv could be reflink copied, meaning any overwrite is
>> >> > disallowed and must be COW'd. How is that handled?
>> >> >
>> >> > I'm pretty sure on Btrfs GRUB knows is can't write to grubenv, I'm
>> >> > just curious about the other cases.
>> >>
>> >> OK so it allows me to create a grubenv on Btrfs without any complaint.
>> >> Will the bootloader actually try to write to this if grub.cfg contains
>> >> save_env?
>> >>
>> >> $ sudo grub2-editenv --verbose grubenv create
>> >> [sudo] password for chris:
>> >> [chris@f29h ~]$ ll
>> >> -rw-r--r--. 1 root  root     1024 Sep 18 13:37 grubenv
>> >> [chris@f29h ~]$ stat -f grubenv
>> >>   File: "grubenv"
>> >>     ID: ac9ba8ecdce5b017 Namelen: 255     Type: btrfs
>> >> Block size: 4096       Fundamental block size: 4096
>> >> Blocks: Total: 46661632   Free: 37479747   Available: 37422535
>> >> Inodes: Total: 0          Free: 0
>> >> [chris@f29h ~]$
>> >
>> > There two things here. grub2-editenv should create grubenv properly
>> > and double check that space on disk is reserved. If it is not then
>> > it should complain. GRUB2 during boot should check was grubenv file
>> > properly created. If it was not it should not update grubenv and
>> > complain.
>> >
>> > I am not sure is anything like that implemented in GRUB2. If does
>> > not I am happy to see and review the patches.
>>
>> When I create a grubenv on Btrfs is does succeed and it's an inline
>> extent, so no mattter what it's checksummed. There is a message on the
>> next boot:
>>
>> error: ../../grub-core/commands/loadenv.c:215:sparse file not allowed.
>>
>> And the grubenv is not overwritten even though the grub.cfg is using
>> save_env and when this same grub.cfg is used on ext4 it does overwrite
>> grubenv. So... It appears loadenv.c must have some inhibitor for
>> writing to Btrfs, which is a good thing.
>
> Great! That is in line with what I said earlier.
>
>> I'm uncertain whether GRUB avoids writing to any other case (LUKS, md
>> RAID, LVM). In particular, XFS, because XFS now supports reflinks, so
>> strictly speaking even if overwriting 2 sectors does not cause
>> corruption today (no inline extent support yet), it probably should
>> refuse to write to XFS as well.
>
> Yep!
>
>> Anyway, I've got a couple of different opinions from XFS devel@ and
>> ext4 devel@ about this. My understanding is each file system (ext4,
>> XFS, Btrfs at least) have reserve areas that can reliably be written
>> to by the bootloader (pre-boot), and it seems like those need to be
>> used instead of depending on grubenv.
>>
>> https://www.spinics.net/lists/linux-ext4/msg62364.html
>>
>> https://www.spinics.net/lists/linux-xfs/msg21902.html
>
> If something like that exits I am happy to use it. However, I would not
> change user interface in any way. Everything should happen auto-magically
> from user POV.
>

The user space interface could remain unchange, i.e. grub-editenv
behavior. However, the grubenv file itself is also user facing, isn't
it? The grubenv file probably can't be entirely deprecated because
some file systems don't have reserve areas for bootloaders. Whereas on
a file system like Btrfs, there's at least two reserve areas, and
they're fairly large compared to other file systems (big enough to
embed a GRUB core.img) - so in that case would grubenv exist?

grub-editenv alone can properly negotiate this.


-- 
Chris Murphy

Re: grubenv on md, Btrfs, LUKS, etc

[Daniel Kiper]

On Thu, Sep 27, 2018 at 05:00:31PM -0600, Chris Murphy wrote:
> On Thu, Sep 27, 2018 at 10:38 AM, Daniel Kiper <dkiper@net-space.pl> wrote:
> > On Wed, Sep 26, 2018 at 04:45:51PM -0600, Chris Murphy wrote:
> >> On Fri, Sep 21, 2018 at 12:35 PM, Daniel Kiper <dkiper@net-space.pl> wrote:
> >> > On Tue, Sep 18, 2018 at 01:40:06PM -0600, Chris Murphy wrote:
> >> >> On Mon, Sep 17, 2018 at 10:59 PM, Chris Murphy <lists@colorremedies.com> wrote:
> >> >> > Hi,
> >> >> >
> >> >> > GRUB code can certainly read files that are on Btrfs, md devices,
> >> >> > LUKS, LVM, and so on. But GRUB code can also write to the physical
> >> >> > block for grubenv - but are there safe guards that prevent it from
> >> >> > doing so if grubenv is on something like Btrfs, mdadm raid5, LUKS?
> >> >> >
> >> >> > And also what about XFS? This used to be safe, but now with reflink
> >> >> > support, grubenv could be reflink copied, meaning any overwrite is
> >> >> > disallowed and must be COW'd. How is that handled?
> >> >> >
> >> >> > I'm pretty sure on Btrfs GRUB knows is can't write to grubenv, I'm
> >> >> > just curious about the other cases.
> >> >>
> >> >> OK so it allows me to create a grubenv on Btrfs without any complaint.
> >> >> Will the bootloader actually try to write to this if grub.cfg contains
> >> >> save_env?
> >> >>
> >> >> $ sudo grub2-editenv --verbose grubenv create
> >> >> [sudo] password for chris:
> >> >> [chris@f29h ~]$ ll
> >> >> -rw-r--r--. 1 root  root     1024 Sep 18 13:37 grubenv
> >> >> [chris@f29h ~]$ stat -f grubenv
> >> >>   File: "grubenv"
> >> >>     ID: ac9ba8ecdce5b017 Namelen: 255     Type: btrfs
> >> >> Block size: 4096       Fundamental block size: 4096
> >> >> Blocks: Total: 46661632   Free: 37479747   Available: 37422535
> >> >> Inodes: Total: 0          Free: 0
> >> >> [chris@f29h ~]$
> >> >
> >> > There two things here. grub2-editenv should create grubenv properly
> >> > and double check that space on disk is reserved. If it is not then
> >> > it should complain. GRUB2 during boot should check was grubenv file
> >> > properly created. If it was not it should not update grubenv and
> >> > complain.
> >> >
> >> > I am not sure is anything like that implemented in GRUB2. If does
> >> > not I am happy to see and review the patches.
> >>
> >> When I create a grubenv on Btrfs is does succeed and it's an inline
> >> extent, so no mattter what it's checksummed. There is a message on the
> >> next boot:
> >>
> >> error: ../../grub-core/commands/loadenv.c:215:sparse file not allowed.
> >>
> >> And the grubenv is not overwritten even though the grub.cfg is using
> >> save_env and when this same grub.cfg is used on ext4 it does overwrite
> >> grubenv. So... It appears loadenv.c must have some inhibitor for
> >> writing to Btrfs, which is a good thing.
> >
> > Great! That is in line with what I said earlier.
> >
> >> I'm uncertain whether GRUB avoids writing to any other case (LUKS, md
> >> RAID, LVM). In particular, XFS, because XFS now supports reflinks, so
> >> strictly speaking even if overwriting 2 sectors does not cause
> >> corruption today (no inline extent support yet), it probably should
> >> refuse to write to XFS as well.
> >
> > Yep!
> >
> >> Anyway, I've got a couple of different opinions from XFS devel@ and
> >> ext4 devel@ about this. My understanding is each file system (ext4,
> >> XFS, Btrfs at least) have reserve areas that can reliably be written
> >> to by the bootloader (pre-boot), and it seems like those need to be
> >> used instead of depending on grubenv.
> >>
> >> https://www.spinics.net/lists/linux-ext4/msg62364.html
> >>
> >> https://www.spinics.net/lists/linux-xfs/msg21902.html
> >
> > If something like that exits I am happy to use it. However, I would not
> > change user interface in any way. Everything should happen auto-magically
> > from user POV.
> >
>
> The user space interface could remain unchange, i.e. grub-editenv
> behavior. However, the grubenv file itself is also user facing, isn't

Great!

> it? The grubenv file probably can't be entirely deprecated because
> some file systems don't have reserve areas for bootloaders. Whereas on

I am OK with that.

> a file system like Btrfs, there's at least two reserve areas, and
> they're fairly large compared to other file systems (big enough to
> embed a GRUB core.img) - so in that case would grubenv exist?

I do not think that it is needed. Even it may make more confusion
in such cases. So, probably no for btrfs and other filesystems
which have reserved areas for bootloader. However, I think that
docs also should be updated properly.

> grub-editenv alone can properly negotiate this.

Perfect!

Daniel