Re: PCI pass-through vs PoD

From: Jan Beulich <jbeulich@suse.com>
To: "Andrew Cooper" <amc96@srcf.net>,
	"Roger Pau Monné" <roger.pau@citrix.com>
Cc: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	Anthony Perard <anthony.perard@citrix.com>,
	Ian Jackson <iwj@xenproject.org>, Paul Durrant <paul@xen.org>,
	George Dunlap <george.dunlap@citrix.com>, Wei Liu <wl@xen.org>
Subject: Re: PCI pass-through vs PoD
Date: Wed, 17 Nov 2021 12:23:14 +0100	[thread overview]
Message-ID: <3d7d5069-591d-4535-c13a-5976e1172a68@suse.com> (raw)
In-Reply-To: <d2650a7e-f681-301c-6959-bc84a502255a@srcf.net>

On 17.11.2021 12:09, Andrew Cooper wrote:
> On 17/11/2021 10:13, Jan Beulich wrote:
>> On 17.11.2021 09:55, Roger Pau Monné wrote:
>>> On Wed, Nov 17, 2021 at 09:39:17AM +0100, Jan Beulich wrote:
>>>> On 13.09.2021 11:02, Jan Beulich wrote:
>>>>> libxl__domain_config_setdefault() checks whether PoD is going to be
>>>>> enabled and fails domain creation if at the same time devices would get
>>>>> assigned. Nevertheless setting up of IOMMU page tables is allowed.
>>> I'm unsure whether allowing enabling the IOMMU with PoD is the right
>>> thing to do, at least for our toolstack.
>> May I ask about the reasons of you being unsure?
> 
> PoD and passthrough is a total nonsense.  You cannot have IOMMU mappings 
> to bits of the guest physical address space which don't exist.
> 
> It is now the case that IOMMU (or not) must be specified at domain 
> creation time, which is ahead of creating PoD pages.  Certainly as far 
> as Xen is concerned, the logic probably wants reversing to have 
> add_to_physmap&friends reject PoD if an IOMMU was configured.
> 
> A toolstack could, in principle, defer the decision to first device 
> assignment.

Right, which is what I consider the preferred approach.

> However, this is terrible behaviour all around, because one way or 
> another we've got to force-populate all PoD pages (which is potentially 
> minutes worth of work to do),

Sure.

> and liable to suffer -ENOMEM,

Not if (as suggested) we first check that the PoD cache is large enough
to cover all PoD entries.

> or we have 
> to reject a control operation with -EBUSY for a task which is dependent 
> on the guest kernel actions in a known-buggy area.

Why reject anything?

> There is no point trying to make this work.  If a user wants a device, 
> they don't get to have PoD.  Anything else is a waste of time and effort 
> on our behalf for a usecase that doesn't exist in practice.

Not sure where you take the latter from. I suppose I'll submit the patch
as I have it now (once I have properly resolved dependencies on other
patches I have queued and/or pending), and if that's not deemed acceptable
plus if at the same time I don't really agree with proposed alternatives,
I'll leave fixing the bug to someone else. Of course the expectation then
is that such a bug fix come forward within a reasonable time frame ...

Jan