From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9AC6BC433ED for ; Thu, 1 Apr 2021 01:08:17 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1107660C3D for ; Thu, 1 Apr 2021 01:08:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1107660C3D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=zededa.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.104091.198680 (Exim 4.92) (envelope-from ) id 1lRlnE-0006bo-MZ; Thu, 01 Apr 2021 01:06:40 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 104091.198680; Thu, 01 Apr 2021 01:06:40 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lRlnE-0006bh-JU; Thu, 01 Apr 2021 01:06:40 +0000 Received: by outflank-mailman (input) for mailman id 104091; Thu, 01 Apr 2021 01:06:39 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lRlnD-0006bc-ON for xen-devel@lists.xenproject.org; Thu, 01 Apr 2021 01:06:39 +0000 Received: from mail-qt1-x82d.google.com (unknown [2607:f8b0:4864:20::82d]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 57233a77-0717-43da-8c2d-705c60856b16; Thu, 01 Apr 2021 01:06:37 +0000 (UTC) Received: by mail-qt1-x82d.google.com with SMTP id g24so393038qts.6 for ; Wed, 31 Mar 2021 18:06:37 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 57233a77-0717-43da-8c2d-705c60856b16 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zededa.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=I4XnTBqWaNHfJM0uifctrfC1GtW1564zJxX4MuGrMVI=; b=j4JCaWgTgHeE9hNCyf3zLgLl3NZvmOoLiw4Ep/EC6ix7jX0/TAzMjB+byX+ZGjleg1 tlNWElFHsnba0WSpmSvZVrmBUBLfaeYKGv3Mc8ilJN6xtyu/5Jt9XvWQbvt0PXJpvx95 jYOogg0fpdsx6ZmBvcrEa70SERHP4L6GMG4AQZm3ejWOFjLPGEy9gTUj76prrx6RccTT a14OEn6o3qJwbyRWZT5BJw5TUimbm3lXCtkhpPKCKJeEGZCB9w/T8Qp+ybP9yht/6RBD B4DlodB28DAGjJVdFlWDy4P57Yb3RaWiE+gTVk08iOh92euQ/XYTWh8wKOGJHUsDQkId rBlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=I4XnTBqWaNHfJM0uifctrfC1GtW1564zJxX4MuGrMVI=; b=FQJKh2Jo0ko1zJ1u6uFXxvNEzUni9WXk/R8JfpQOrzFcNu715evzZZGE6ZnPRHuuUB uBbQf/kHnmggYuhCc7ZETLYxrQrb54XtKYWHsDD0UW4sohuzNxAKC94bu7zjC7oFpk1f VFbdslzAot2qQmdnBflRNYPD8HRAp+WzRQrOKzOaHj2jbMzUEstPaaPBDfjDv78zhYCX xYA9yoSBMME2yhCcHkCPIesDEjlJDM4bo3BdVAk8OydmhfaLJ59Z4KiykwPYMljYhhlM LAnp0iAgl1K92eUUhaG6mBzhzOmZEhYhiBMO3+QfxaO6DaJqYkLnDtm9rZZ6YJ/4MwXN BJpA== X-Gm-Message-State: AOAM532RuWtGjI4URlBunrU3fRDkwVG8NVMTQwT/M1vvGgoZdLo7/Trb eGCSHjVTlICmvhKbHCh+Sfp28hadt1WS4KrxWjJQhA== X-Google-Smtp-Source: ABdhPJwGa4irE89lKL5j7+qT7fRWuH62QG8dMMrr/BtnompBjfBQ5fQ3cVmy8GlUz5hnj8n23BMFDOCaDyC0diXaLjY= X-Received: by 2002:ac8:1093:: with SMTP id a19mr5064689qtj.365.1617239197376; Wed, 31 Mar 2021 18:06:37 -0700 (PDT) MIME-Version: 1.0 References: <23baec07-a3ae-949a-a1fb-09100efd53e5@citrix.com> In-Reply-To: <23baec07-a3ae-949a-a1fb-09100efd53e5@citrix.com> From: Roman Shaposhnik Date: Wed, 31 Mar 2021 18:06:31 -0700 Message-ID: Subject: Re: multiboot2 and module2 boot issues via GRUB2 To: Andrew Cooper Cc: Xen-devel , grub-devel@gnu.org Content-Type: multipart/alternative; boundary="000000000000e0809d05beded750" --000000000000e0809d05beded750 Content-Type: text/plain; charset="UTF-8" Hi Andrew! first of all -- thanks for pointing me in the right direction. So after reading relevant sources: comments inline. On Tue, Mar 30, 2021 at 12:08 PM Andrew Cooper wrote: > On 30/03/2021 19:28, Roman Shaposhnik wrote: > > Hi! > > > > seems like I've run into an issue with multiboot2 and module2 > > commands that I can't quite explain. Since it may be something > > super simply and silly -- I wanted to reach out here before I do > > a GRUB/Xen/LK source deepdive. > > > > So here's the deal: whenever I boot straight up Linux kernel > > I can do the following sequence of commands: > > linux /kernel > > initrd foo.cpio.gz bar.cpio.gz > > and have linux kernel effectively stack content of bar.cpio.gz > > on top of foo.cpio.gz and present a unified initramfs that way. > > > > I'm trying to replicate it with Xen, but: > > multiboot2 /boot/xen.gz > > module2 /kernel > > module2 foo.cpio.gz > > module2 bar.cpio.gz > > only seems to be picking up foo.cpio.gz > > > > Has anyone run into this issue before? > > I can explain why that happens. Not sure if it counts as a feature, bug > or mis-expectation, but CC'ing grub-devel for their input. > > The initrd command is presumably concatenating those two files together > in memory, and presenting Linux a single initrd pointer. > It is indeed what seems to be happening. > For the module2 example, you're putting 3 distinct files in memory, and > giving Xen a list 3 modules. > And that is also correct -- nothing like that is possible with modules. Kernel actually needs to be aware of them. So the question then becomes... > Xen is capable of taking various things via modules, such as an > XSM/Flask policy, or microcode, so has logic to identify these if > present and separate them from "other stuff". However, there is a > hardcoded expectation that the first module is the dom0 kernel, and the > next unrecognised module, if present, is *the* initrd. I expect that Xen isn't handing bar.cpio.gz on to dom0, but I'm not sure > whether passing two distinct initrd-like-things to Linux is even possible. > > What you presumably want is some `initrd` side effect in Grub so you can > write `module2 foo.cpio.gz bar.cpio.gz` and have it concatenate things > together in memory and present one MB2 module, but I suspect that exact > syntax might be ambiguous with command line handling. I have no idea > whether such a command currently exists. > ...I guess there's no mechanism out-of-the box to achieve what I want? And the obvious next question: is my EVE usecase esoteric enough that I should just go ahead and do a custom GRUB patch or is there a more general interest in this? Thanks, Roman. --000000000000e0809d05beded750 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi Andrew!

first of all -- thanks for pointing me in the right direction. So aft= er reading relevant sources: comments inline.

On Tue, Mar 30, 2021 at = 12:08 PM Andrew Cooper <and= rew.cooper3@citrix.com> wrote:
On 30/03/2021 19:28, Roman Shaposhnik wrote:
> Hi!
>
> seems like I've run into an issue with multiboot2=C2=A0and module2=
> commands that I can't quite explain. Since it may be something
> super simply and silly -- I wanted to reach out here before I do
> a GRUB/Xen/LK source deepdive.
>
> So here's the deal: whenever I boot straight up Linux kernel
> I can do the following sequence of commands:
> =C2=A0 =C2=A0linux /kernel
> =C2=A0 =C2=A0initrd foo.cpio.gz bar.cpio.gz
> and have linux kernel effectively stack content of bar.cpio.gz
> on top of foo.cpio.gz and present a unified initramfs that way.
>
> I'm trying to replicate it with Xen, but:
> =C2=A0 =C2=A0 =C2=A0multiboot2 /boot/xen.gz
> =C2=A0 =C2=A0 =C2=A0module2 /kernel
> =C2=A0 =C2=A0 =C2=A0module2 foo.cpio.gz
> =C2=A0 =C2=A0 =C2=A0module2 bar.cpio.gz
> only seems to be picking up foo.cpio.gz
>
> Has anyone run into this=C2=A0issue before?

I can explain why that happens.=C2=A0 Not sure if it counts as a feature, b= ug
or mis-expectation, but CC'ing grub-devel for their input.

The initrd command is presumably concatenating those two files together
in memory, and presenting Linux a single initrd pointer.

It is indeed what seems to be happening.
=C2=A0=
For the module2 example, you're putting 3 distinct files in memory, and=
giving Xen a list 3 modules.

And that i= s also correct -- nothing like that is possible with modules.
Ker= nel actually needs to be aware of them. So the question then
beco= mes...
=C2=A0
Xen is capable of taking various things via modules, such as an
XSM/Flask policy, or microcode, so has logic to identify these if
present and separate them from "other stuff".=C2=A0 However, ther= e is a
hardcoded expectation that the first module is the dom0 kernel, and the
next unrecognised module, if present, is *the* initrd.=C2=A0
I expect that Xen isn't= handing bar.cpio.gz on to dom0, but I'm not sure
whether passing two distinct initrd-like-things to Linux is even possible.<= br>
What you presumably want is some `initrd` side effect in Grub so you can write `module2 foo.cpio.gz bar.cpio.gz` and have it concatenate things
together in memory and present one MB2 module, but I suspect that exact
syntax might be ambiguous with command line handling.=C2=A0 I have no idea<= br> whether such a command currently exists.

...I guess there's no mechanism out-of-the box to achieve=C2=A0w= hat I want?
=C2=A0
And the obvious nex= t question: is my EVE usecase esoteric enough that
I should just = go ahead and do a custom=C2=A0GRUB patch or is there a more
gener= al interest in this?

Thanks,
Roman.
=
--000000000000e0809d05beded750--