Is: Fix for 4MB BIOS payload in hvmloader. Was:Re: [edk2] [PATCH 0/5] OvmfPkg: complete the 4MB flash image support ("-bios" / emulated variables)

Is: Fix for 4MB BIOS payload in hvmloader. Was:Re: [edk2] [PATCH 0/5] OvmfPkg: complete the 4MB flash image support ("-bios" / emulated variables)

[Konrad Rzeszutek Wilk]

Adding Jan (autor of patch) and Julien (Xen release manager);

Pls see below.

On Thu, May 18, 2017 at 02:36:33PM +0200, Laszlo Ersek wrote:
> On 05/16/17 06:20, Gary Lin wrote:
> > On Mon, May 15, 2017 at 05:40:59PM -0700, Jordan Justen wrote:
> >> On 2017-05-12 01:40:34, Laszlo Ersek wrote:
> >>> On 05/12/17 04:02, Gary Lin wrote:
> >>>> On Mon, May 08, 2017 at 12:27:59PM +0800, Gary Lin wrote:
> >>>>> On Sat, May 06, 2017 at 09:30:18PM +0200, Laszlo Ersek wrote:
> >>>>>> (All hail Saturday!)
> >>>>>>
> >>>>>> Gary, can you please fetch this from my repo (URL & branch name below)
> >>>>>> and test it with Xen? Please test both the 4MB and the 2MB build. (I
> >>>>>> also tested both, with qemu + "-bios".)
> >>>>> Hi Laszlo,
> >>>>>
> >>>>> I have done some simples test with xen, and the 2MB build seems fine.
> >>>>> It booted into grub2 menu successfully. However, the 4MB build never boots.
> >>>>> The QEMU window showed less than 1 sec and then disappeared.
> >>>>>
> >>>>> Here is the snippet from 'xl dmesg'
> >>>>>
> >>>>> (d15)  - CPU0 ... 39-bit phys ... fixed MTRRs ... var MTRRs [1/8] ... done.
> >>>>> (d15) Testing HVM environment:
> >>>>> (d15)  - REP INSB across page boundaries ... passed
> >>>>> (d15)  - GS base MSRs and SWAPGS ... passed
> >>>>> (d15) Passed 2 of 2 tests
> >>>>> (d15) Writing SMBIOS tables ...
> >>>>> (d15) Loading OVMF ...
> >>>>> (d15) no BIOS ROM image found
> >>>>> (d15) *** HVMLoader bug at hvmloader.c:381
> >>>>> (d15) *** HVMLoader crashed.
> >>>>>
> >>>>> I'm pretty sure that the ovmf path is right, so it seems Xen just rejected
> >>>>> the 4MB build :-\
> >>>>>
> >>>>> I'll try to dig more information.
> >>>>>
> >>>> There is a function in the xen hvmloader clearing the memory from
> >>>> 0x400000 to 0x800000. Unfortunately, the hvm_start_info struct of the
> >>>> 4MB OVMF was loaded to 0x588000, so the struct was cleared mistakenly
> >>>> and hvmloader cannot find the firmware. Xen is not ready for the 4MB
> >>>> build yet :-\
> >>>>
> >>>> The discussion in xen-devel:
> >>>> https://lists.xen.org/archives/html/xen-devel/2017-05/msg01053.html
> >>>
> >>> Thank you for the feedback!
> >>>
> >>> In this case, I think we should drop the last patch from this series.
> >>
> >> Can we come up with a plan for trying to fix this? Gary, would it be
> >> okay if we opened a bug and assigned it to you? Or, do you have
> >> another suggestion for a possible Xen owner?
> >>
> > Jan Beulich (also a SUSE employee) is working on the patch(*), and it
> > works for me.
> 
> We should distinguish a TianoCore BZ entry for this, from a Xen bug
> report for this. The former would depend on the latter.
> 
> The TianoCore BZ assignee's job would be to monitor the upstream Xen
> fix, and to submit the last patch of this series -- separated out -- to
> edk2-devel once upstream Xen commits the fix.
> 
> Upstream Xen does not have a bug tracker that is widely used in their
> community:
> 
> https://wiki.xen.org/wiki/Reporting_Bugs_against_Xen_Project
> 
> > The primary location for reporting bugs against the hypervisor and
> > associated bundled tools [...] is by posting to the xen-devel mailing
> > list (list info). Please tag your subject line with a '[BUG]' prefix.
> > Note that you do not need to be subscribed to the list to post
> > (although non-subscribers are moderated this usually happens pretty
> > quickly) and that list policy is to CC people so you shouldn't miss
> > any replies.
> >
> > [...]
> >
> > Although a bugzilla instance does exist it is not well maintained or
> > widely used by developers. If you really want to file a bug in
> > bugzilla you are strongly recommended to also post to the mailing
> > list.
> 
> This does not make things easier for us, because rather than recurrently
> check a simple bug status field in the Xen tracker, we'd have to be
> tapped into Xen development, and follow the email thread & any relevant
> commits closely. In reality I'm not even subscribed to xen-devel.
> 
> The situation is further hampered by the fact that Xen is (apparently)
> right at 4.9.0-rc5, so they likely won't commit Jan's hvmloader patch
> until Xen 4.9 is out. This is a problem for a potential TianoCore-side
> BZ because the delay will make us forget about the issue.
> 
> Thanks,
> Laszlo
> 
> > 
> > Cheers,
> > 
> > Gary Lin
> > 
> > (*) https://lists.xen.org/archives/html/xen-devel/2017-05/msg01242.html
> >> Thanks,
> >>
> >> -Jordan
> >>
> >>>
> >>> However, your test results also confirm that the 2MB build continues to
> >>> work with Xen, which means that the reworking of the
> >>> EmuVariableFvbRuntimeDxe driver in this series, and the underlying
> >>> tweaks+cleanups series, cause no regression.
> >>>
> >>> Can you please respond, with your "Regression-tested-by", to:
> >>>
> >>> (1) the full series
> >>>
> >>>   [edk2] [PATCH 0/7] OvmfPkg: small cleanups and tweaks
> >>>
> >>> (2) and patches 1 through 3 in this series? (Patch #4 is just
> >>> documentation, for which Tested-by would be strange.)
> >>>
> >>> Thank you!
> >>> Laszlo
> >>>
> >>>
> >>>>>> Note: this series depends on:
> >>>>>>
> >>>>>>   [edk2] [PATCH 0/7] OvmfPkg: small cleanups and tweaks
> >>>>>>   https://lists.01.org/pipermail/edk2-devel/2017-May/010527.html
> >>>>>>
> >>>>>> and it has been pushed to my github repo as such.
> >>>>>>
> >>>>>> Repo:   https://github.com/lersek/edk2.git
> >>>>>> Branch: emu4k
> >>>>>>
> >>>>>> Cc: Gary Ching-Pang Lin <glin@suse.com>
> >>>>>> Cc: Jordan Justen <jordan.l.justen@intel.com>
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Laszlo
> >>>>>>
> >>>>>> Laszlo Ersek (5):
> >>>>>>   OvmfPkg/EmuVariableFvbRuntimeDxe: strip trailing whitespace
> >>>>>>   OvmfPkg/EmuVariableFvbRuntimeDxe: change block size to 4KB
> >>>>>>   OvmfPkg/PlatformPei: align EmuVariableNvStore at any page boundary
> >>>>>>   OvmfPkg/README: document 4MB flash layout
> >>>>>>   OvmfPkg: make the 4MB flash size the default (again)
> >>>>>>
> >>>>>>  OvmfPkg/OvmfPkgIa32.dsc                |   2 +-
> >>>>>>  OvmfPkg/OvmfPkgIa32X64.dsc             |   2 +-
> >>>>>>  OvmfPkg/OvmfPkgX64.dsc                 |   2 +-
> >>>>>>  OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb.h |  26 ++-
> >>>>>>  OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb.c | 191 +++++++++-----------
> >>>>>>  OvmfPkg/PlatformPei/Platform.c         |  20 +-
> >>>>>>  OvmfPkg/README                         |  39 +++-
> >>>>>>  7 files changed, 143 insertions(+), 139 deletions(-)
> >>>>>>
> >>>>>> -- 
> >>>>>> 2.9.3
> >>>>>>
> >>>>>>
> >>>>> _______________________________________________
> >>>>> edk2-devel mailing list
> >>>>> edk2-devel@lists.01.org
> >>>>> https://lists.01.org/mailman/listinfo/edk2-devel
> >>>>>
> >>>
> >>> _______________________________________________
> >>> edk2-devel mailing list
> >>> edk2-devel@lists.01.org
> >>> https://lists.01.org/mailman/listinfo/edk2-devel
> >>
> 
> _______________________________________________
> edk2-devel mailing list
> edk2-devel@lists.01.org
> https://lists.01.org/mailman/listinfo/edk2-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Is: Fix for 4MB BIOS payload in hvmloader. Was:Re: [edk2] [PATCH 0/5] OvmfPkg: complete the 4MB flash image support ("-bios" / emulated variables)

[Jan Beulich]

>>> On 23.05.17 at 16:12, <konrad@kernel.org> wrote:
> On Thu, May 18, 2017 at 02:36:33PM +0200, Laszlo Ersek wrote:
>> The situation is further hampered by the fact that Xen is (apparently)
>> right at 4.9.0-rc5, so they likely won't commit Jan's hvmloader patch
>> until Xen 4.9 is out. This is a problem for a potential TianoCore-side
>> BZ because the delay will make us forget about the issue.

The patch went in in time for rc6.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Is: Fix for 4MB BIOS payload in hvmloader. Was:Re: [edk2] [PATCH 0/5] OvmfPkg: complete the 4MB flash image support ("-bios" / emulated variables)

[Julien Grall]

Hi,

On 23/05/17 15:12, Konrad Rzeszutek Wilk wrote:
>>> The primary location for reporting bugs against the hypervisor and
>>> associated bundled tools [...] is by posting to the xen-devel mailing
>>> list (list info). Please tag your subject line with a '[BUG]' prefix.
>>> Note that you do not need to be subscribed to the list to post
>>> (although non-subscribers are moderated this usually happens pretty
>>> quickly) and that list policy is to CC people so you shouldn't miss
>>> any replies.
>>>
>>> [...]
>>>
>>> Although a bugzilla instance does exist it is not well maintained or
>>> widely used by developers. If you really want to file a bug in
>>> bugzilla you are strongly recommended to also post to the mailing
>>> list.
>>
>> This does not make things easier for us, because rather than recurrently
>> check a simple bug status field in the Xen tracker, we'd have to be
>> tapped into Xen development, and follow the email thread & any relevant
>> commits closely. In reality I'm not even subscribed to xen-devel.

We recently introduced a tracker ([1]) to help us following the state of 
bugs/features. It is managed by maintainers and read-only for the others.

The usual process is to report the bug on the mailing-list and a 
maintainer will create a bug on the tracker if we have no immediate 
solution.

In this case, the patch made rc6. So I believe the problem is resolved.

Cheers,

[1] https://xenproject.atlassian.net/projects/XEN/issues

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Is: Fix for 4MB BIOS payload in hvmloader. Was:Re: [edk2] [PATCH 0/5] OvmfPkg: complete the 4MB flash image support ("-bios" / emulated variables)

[Laszlo Ersek]

On 05/23/17 17:01, Jan Beulich wrote:
>>>> On 23.05.17 at 16:12, <konrad@kernel.org> wrote:
>> On Thu, May 18, 2017 at 02:36:33PM +0200, Laszlo Ersek wrote:
>>> The situation is further hampered by the fact that Xen is (apparently)
>>> right at 4.9.0-rc5, so they likely won't commit Jan's hvmloader patch
>>> until Xen 4.9 is out. This is a problem for a potential TianoCore-side
>>> BZ because the delay will make us forget about the issue.
> 
> The patch went in in time for rc6.

Thank you Jan for the speedy fix!

While reviewing version 2 of this patch set, Jordan wrote in
<https://lists.01.org/pipermail/edk2-devel/2017-May/010776.html>:

> With the understanding that we're holding off on the final patch for
> now to coordinate with Xen:
>
> Series Reviewed-by: Jordan Justen <jordan.l.justen at intel.com>

"the final patch" referenced there was:

  [edk2] [PATCH v2 5/5] OvmfPkg: make the 4MB flash size the default
                        (again)
  https://lists.01.org/pipermail/edk2-devel/2017-May/010763.html

So I have now modified the commit message on that patch, adding the
following paragraph:

> Xen gained support for the 4MB flash image in Xen commit 0d6968635ce5
> ("hvmloader: avoid tests when they would clobber used memory",
> 2017-05-19), which is part of Xen 4.9.0-rc6.

Seeing Gary's Tested-by on the Xen commit, I've also pushed the reworded
edk2 patch, as commit 1c47fcd465a4.

Thanks!
Laszlo

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Is: Fix for 4MB BIOS payload in hvmloader. Was:Re: [edk2] [PATCH 0/5] OvmfPkg: complete the 4MB flash image support ("-bios" / emulated variables)

[Laszlo Ersek]

On 05/23/17 17:02, Julien Grall wrote:
> Hi,
> 
> On 23/05/17 15:12, Konrad Rzeszutek Wilk wrote:
>>>> The primary location for reporting bugs against the hypervisor and
>>>> associated bundled tools [...] is by posting to the xen-devel mailing
>>>> list (list info). Please tag your subject line with a '[BUG]' prefix.
>>>> Note that you do not need to be subscribed to the list to post
>>>> (although non-subscribers are moderated this usually happens pretty
>>>> quickly) and that list policy is to CC people so you shouldn't miss
>>>> any replies.
>>>>
>>>> [...]
>>>>
>>>> Although a bugzilla instance does exist it is not well maintained or
>>>> widely used by developers. If you really want to file a bug in
>>>> bugzilla you are strongly recommended to also post to the mailing
>>>> list.
>>>
>>> This does not make things easier for us, because rather than recurrently
>>> check a simple bug status field in the Xen tracker, we'd have to be
>>> tapped into Xen development, and follow the email thread & any relevant
>>> commits closely. In reality I'm not even subscribed to xen-devel.
> 
> We recently introduced a tracker ([1]) to help us following the state of
> bugs/features. It is managed by maintainers and read-only for the others.
> 
> The usual process is to report the bug on the mailing-list and a
> maintainer will create a bug on the tracker if we have no immediate
> solution.

That's perfect; we'll be able to reference Xen issue reports with URLs
like (just an example) <https://xenproject.atlassian.net/browse/XEN-86>,
from TianoCore BZs. That should enable us to quickly re-check the state
of the Xen report whenever we revisit the dependent TianoCore BZ.

... Can you please add the information about "xenproject.atlassian.net"
to the Xen Wiki article at
<https://wiki.xen.org/wiki/Reporting_Bugs_against_Xen_Project>?

> In this case, the patch made rc6. So I believe the problem is resolved.

Yes, it is. Thanks!
Laszlo

> Cheers,
> 
> [1] https://xenproject.atlassian.net/projects/XEN/issues
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel