All of
 help / color / mirror / Atom feed
From: Roman Skakun <>
To: Julien Grall <>
Cc: "" <>,
	Bertrand Marquis <>,
	Andrii Anisov <>,
	Roman Skakun <>,
	Oleksandr Tyshchenko <>,
	Stefano Stabellini <>
Subject: Re: Disable IOMMU in Dom0
Date: Thu, 9 Sep 2021 14:25:29 +0000	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

[-- Attachment #1: Type: text/plain, Size: 4041 bytes --]

Hi Julien,

Thanks for the clarification!

I aim towards to prepare implementation for upstream to disable SWIOTLB for IOMMU-protected devices in Dom0.
To provide this functionality need to add additional binding for each protected device in device-tree.
After this step, I will also prepare the patch to make ensure that ballooning code prepares all allocations below 4GB.

We are going to prepare this functionality only for device-tree based system configurations.
We don't have resources to support ACPI configuration.

Would you be ok with upstreaming only device-tree configuration?

From: Julien Grall <>
Sent: Wednesday, September 1, 2021 1:22 PM
To: Roman Skakun <>; Stefano Stabellini <>
Cc: <>; Bertrand Marquis <>; Andrii Anisov <>; Roman Skakun <>; Oleksandr Tyshchenko <>
Subject: Re: Disable IOMMU in Dom0

Hi Roman

On 01/09/2021 10:59, Roman Skakun wrote:
>> If you have a setup  where Dom0 is not 1:1 mapped (which is not currently
>> possible with upstream  Xen but it is possible with cache coloring) and
>> uses the IOMMU to make  device DMA work like regular DomUs, then passing
>> XENFEAT_not_direct_mapped  to Linux would make it work. Without
>> XENFEAT_not_direct_mapped,  Linux would try to use swiotlb-xen which has
>> code that relies on  Linux being 1:1 mapped to work properly.
> I'm using 1:1 Dom0.
> According to your patch series, xen-swiotlb fops will be applied for all
> devices
> because XENFEAT_direct_mapped is active, as shown here:
>*L56__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4ibLDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_Iv_mvn3O$ [elixir[.]bootlin[.]com]
> <*L56__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4ibLDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_Iv_mvn3O$ [elixir[.]bootlin[.]com]>
> I agreed, that xen-swiotlb should work correctly, but in my case, I
> retrieved MFN here:
>*L366__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4ibLDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_IgZgXPjC$ [elixir[.]bootlin[.]com]
> <*L366__;Iw!!GF_29dbcQIUBPA!i7I0DxCbP4ibLDwzRkeFkgRQbKh-fVD9McLqabG1TzZs9smOVBeowPS_IgZgXPjC$ [elixir[.]bootlin[.]com]>
> which is greater than 32bit and xen-swiotlb tries to use bounce buffer
> as expected.
> It can lead to decrease a performance because I have a long buffer ~4MB.
> I thought, that we can disable swiotlb fops for devices which are
> controlled by IOMMU.

Yes you can disable swiotlb for devices which are controlled by the
IOMMU. But this will not make your problem disappear, it simply hides
until it bites you in a more subttle way.

 From what you wrote, you have a 32-bit DMA capable. So you always need
to have an address below 4GB. For foreign mapping, there is no guarantee
the Guest Physical Address will actually be below 4GB.

Today, the ballooning code only ask Linux to steal *a* RAM page for
mapping the foreign page. This may or may not be below 4GB depending on
how you assigned the RAM to dom0 (IIRC you had some RAM above 4GB).

But that's the current behavior. One of your work colleague is looking
at avoid to steal RAM page to avoid exhausting the memory. So foreign
mapping may end up to be a lot higher in memory.

IOW, you will need to be able to bounce the DMA buffer for your device.
If you want to avoid bouncing, the proper way would be to rework the
ballonning code so pages are allocated below 4GB.


Julien Grall

[-- Attachment #2: Type: text/html, Size: 6017 bytes --]

  reply	other threads:[~2021-09-09 14:26 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-31 19:54 Disable IOMMU in Dom0 Roman Skakun
2021-08-31 21:50 ` Stefano Stabellini
2021-09-01  9:59   ` Roman Skakun
2021-09-01 10:22     ` Julien Grall
2021-09-09 14:25       ` Roman Skakun [this message]
2021-09-09 19:53         ` Stefano Stabellini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.