Re: [Qemu-devel] [v10][PATCH 03/10] piix: create host bridge to passthrough

[Alex Williamson <alex.williamson@redhat.com>]

On Tue, 9 Feb 2016 19:47:49 +0000
"Kay, Allen M" <allen.m.kay@intel.com> wrote:

> > -----Original Message-----
> > From: Alex Williamson [mailto:alex.williamson@redhat.com]
> > Sent: Tuesday, February 09, 2016 9:44 AM
> > Cc: ehabkost@redhat.com; mst@redhat.com;
> > stefano.stabellini@eu.citrix.com; qemu-devel@nongnu.org;
> > pbonzini@redhat.com; rth@twiddle.net; Kay, Allen M
> > <allen.m.kay@intel.com>
> > Subject: Re: [Qemu-devel] [v10][PATCH 03/10] piix: create host bridge to
> > passthrough
> > 
> > 
> > Tiejun's email bounced, so adding Allen for comment.
> >  
> 
> Tiejun has left Intel.  Shuai Ruan is his replacement (cc'ed).
> 
> >  Host bridge registers
> > 52h, a4h, and a8h seem to be for versions of IGD prior to SandyBridge or just
> > wrong. Even the GMCH (50h) doesn't seem to be necessary for the VM host
> > bridge, so I'm thinking about dropping all of these for vfio-based IGD
> > assignment.  
> 
> I think dropping those register access in GMCH should be fine.  52h/a4h/a8h were for Ironlake IGD.  50h is now mirrored in IGD.  They can be removed as BDW/SKL driver no longer access them.
> 
> >  Do we know which guest drivers have specific dependencies on
> > which host bridge and LPC bridge registers?  It would even be nice to
> > document the standard registers for future maintainability.  Thanks,
> >   
> 
> BDW/SKL drivers no longer need to access those registers in GMCH, especially in Universal Passthrough Mode.  From my perspective,  I'm OK with limiting KVM IGD passthrough to SKL+ platforms so that we don't need to add those hacks for KVM IGD passthrough.
> 
> For older HW, there weren't anything documented.  We added those register accesses in Xen/QEMU as we brought up Ironlake/SNB platforms on Xen.

Hi Allen,

Thanks for the reply.  I'm totally onboard with you for BDW/SKL for
supported platforms, but I'd like to understand what the incremental
investment is for each back step within reason for older GPUs, at least
for best-effort, community support.  If we want to assume BDW/SKL and
Universal Passthrough Mode, then we could abandon the host bridge and
ISA bridge modifications altogether, right?  On the other hand, it's
not clear to me that UPT drivers are that pervasive and if we need to
enable some degree of host bridge/ISA bridge poke thrus, why not enable
a fair chunk of users, including me.

My IVB desktop seems to be working well (win10 + linux) with only
poking through the standard host bridge and ISA bridge
identifications.  Yes, I need to know about the different GMCH layout
of IVB vs BDW in the IGD device, but I've already taken care of that.
It seems like SNB is pretty similar to IVB (they run on the same
chipsets), but I haven't yet figured out the magic to make an SNB
laptop light up with IGD assignment, which would be useful to show that
OpRegion passthrough is actually doing something.

If we ignore IronLake and older GPUs, how many VM chipset hacks do we
need?  What combinations would require GMCH mirrored in the host
bridge, and couldn't we mirror it from the IGD device itself since it's
present in the same location all the way back through SNB.

> > On Mon, 8 Feb 2016 20:32:35 -0700
> > Alex Williamson <alex.williamson@redhat.com> wrote:
> >   
> > > On Wed, 15 Jul 2015 13:37:43 +0800
> > > Tiejun Chen <tiejun.chen@intel.com> wrote:
> > > > +/* Here we just expose minimal host bridge offset subset. */ static
> > > > +const IGDHostInfo igd_host_bridge_infos[] = {
> > > > +    {0x08, 2},  /* revision id */
> > > > +    {0x2c, 2},  /* sybsystem vendor id */
> > > > +    {0x2e, 2},  /* sybsystem id */
> > > > +    {0x50, 2},  /* SNB: processor graphics control register */
> > > > +    {0x52, 2},  /* processor graphics control register */
> > > > +    {0xa4, 4},  /* SNB: graphics base of stolen memory */
> > > > +    {0xa8, 4},  /* SNB: base of GTT stolen memory */ };  
> > >
> > > Hi,
> > >
> > > I'm confused by these last 4 registers, could you please help me
> > > understand them?
> > >
> > > Offset 0x50 is the GMCH register for SandyBridge and newer processors,
> > > that makes sense, but I suspect we really want to mask out the GMS
> > > field to indicate the size of stolen memory is zero?  Is Xen providing
> > > the VM with direct access to host stolen memory or does it have support
> > > in the VM BIOS for matching the host address?
> > >  
> 
> Xen does provide 1:1 stolen memory mapping in the guest.  However, I do agree with you we should disable stolen memory in the guest by mask out GMS field.  This will avoid complications involving stolen memory/RMRR support.  The only features uses stolen memory are Frame Buffer Compression and PAVP content protection.  PAVP won't work in the guest anyways as it requires access to ME/HECI device.

We certainly don't want to get into the nastiness of RMRRs in a VM, but
do the stolen memory use cases you've outlined explain the DMAR faults
to that region that I was seeing just booting a VM with a Linux live
CD?  Again, it seems just as easy, if not easier to copy GMCH from the
IGD itself into the host bridge.
 
> > > 0x52 is unknown to me, it's listed as reserved for anything since
> > > SandyBridge, does this date back to chipset-based graphics versus the
> > > current processor-based graphics?
> > >  
> 
> I believe this is same as 0x50 but on old Ironlake platform.  Xen started IGD passthrough support in Ironlake.

Ok, unless anyone shouts really loudly and it doesn't affect anything
newer than IronLake, I'm happy to let those fall off the plate.  I don't
have any older systems that I care to make work with this.

> > > The comment on 0xa4 suggests it should be the BDSM regiser, but that's
> > > at offset 0xb0 for everything since SandyBridge.  0xa4 is the second
> > > DWORD of the Top Of Memory (TOM) register.
> > >
> > > I'm guessing by the description of 0xa8 that it might be the BGSM
> > > register, but that's at 0xb4.  0xa8 is the first DWORD of the Top Of
> > > Upper Usable DRAM (TOUUD).  Neither TOM or TOUUD seem to make any  
> > sense  
> > > to expose to the VM, which really makes me wonder if they're actually
> > > used.
> > >  
> 
> TOM (0xa0/0xa4) and TOUUD (0xa8) were used in Ironlake generation of Windows driver.  They are no longer needed in BDW/SKL drivers.

Ok, so the code comment is pretty misleading for these.  Would anything
since SNB need these?  What about the BDSM register in the host
bridge?  Is the Xen experience of not needing to copy it relevant since
stolen memory is identity mapped into the VM anyway?  Maybe Xen
achieves the same effect by not copying it and exposing it as zero.
Thanks,

Alex