* [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems @ 2019-11-29 14:21 Kai-Heng Feng 2019-12-01 12:42 ` Lucas Stach 2019-12-02 17:00 ` Christoph Hellwig 0 siblings, 2 replies; 15+ messages in thread From: Kai-Heng Feng @ 2019-11-29 14:21 UTC (permalink / raw) To: joro; +Cc: Alex Deucher, iommu, Kai-Heng Feng, linux-kernel Serious screen flickering when Stoney Ridge outputs to a 4K monitor. According to Alex Deucher, IOMMU isn't enabled on Windows, so let's do the same here to avoid screen flickering on 4K monitor. Cc: Alex Deucher <alexander.deucher@amd.com> Bug: https://gitlab.freedesktop.org/drm/amd/issues/961 Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com> --- v2: - Find Stoney graphics instead of host bridge. drivers/iommu/amd_iommu_init.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c index 568c52317757..139aa6fdadda 100644 --- a/drivers/iommu/amd_iommu_init.c +++ b/drivers/iommu/amd_iommu_init.c @@ -2516,6 +2516,7 @@ static int __init early_amd_iommu_init(void) struct acpi_table_header *ivrs_base; acpi_status status; int i, remap_cache_sz, ret = 0; + u32 pci_id; if (!amd_iommu_detected) return -ENODEV; @@ -2603,6 +2604,16 @@ static int __init early_amd_iommu_init(void) if (ret) goto out; + /* Disable IOMMU if there's Stoney Ridge graphics */ + for (i = 0; i < 32; i++) { + pci_id = read_pci_config(0, i, 0, 0); + if ((pci_id & 0xffff) == 0x1002 && (pci_id >> 16) == 0x98e4) { + pr_info("Disable IOMMU on Stoney Ridge\n"); + amd_iommu_disabled = true; + break; + } + } + /* Disable any previously enabled IOMMUs */ if (!is_kdump_kernel() || amd_iommu_disabled) disable_iommus(); @@ -2711,7 +2722,7 @@ static int __init state_next(void) ret = early_amd_iommu_init(); init_state = ret ? IOMMU_INIT_ERROR : IOMMU_ACPI_FINISHED; if (init_state == IOMMU_ACPI_FINISHED && amd_iommu_disabled) { - pr_info("AMD IOMMU disabled on kernel command-line\n"); + pr_info("AMD IOMMU disabled\n"); init_state = IOMMU_CMDLINE_DISABLED; ret = -EINVAL; } -- 2.17.1 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-11-29 14:21 [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems Kai-Heng Feng @ 2019-12-01 12:42 ` Lucas Stach 2019-12-02 16:36 ` Deucher, Alexander 2019-12-02 17:00 ` Christoph Hellwig 1 sibling, 1 reply; 15+ messages in thread From: Lucas Stach @ 2019-12-01 12:42 UTC (permalink / raw) To: Kai-Heng Feng, joro; +Cc: Alex Deucher, iommu, linux-kernel Am Freitag, den 29.11.2019, 22:21 +0800 schrieb Kai-Heng Feng: > Serious screen flickering when Stoney Ridge outputs to a 4K monitor. > > According to Alex Deucher, IOMMU isn't enabled on Windows, so let's do > the same here to avoid screen flickering on 4K monitor. This doesn't seem like a good solution, especially if there isn't a method for the user to opt-out. Some users might prefer having the IOMMU support to 4K display output. But before using the big hammer of disabling or breaking one of those features, we should take a look at what's the issue here. Screen flickering caused by the IOMMU being active hints to the IOMMU not being able to sustain the translation bandwidth required by the high- bandwidth isochronous transfers caused by 4K scanout, most likely due to insufficient TLB space. As far as I know the framebuffer memory for the display buffers is located in stolen RAM, and thus contigous in memory. I don't know the details of the GPU integration on those APUs, but maybe there even is a way to bypass the IOMMU for the stolen VRAM regions? If there isn't and all GPU traffic passes through the IOMMU when active, we should check if the stolen RAM is mapped with hugepages on the IOMMU side. All the stolen RAM can most likely be mapped with a few hugepage mappings, which should reduce IOMMU TLB demand by a large margin. Regards, Lucas > Cc: Alex Deucher <alexander.deucher@amd.com> > Bug: https://gitlab.freedesktop.org/drm/amd/issues/961 > Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com> > --- > v2: > - Find Stoney graphics instead of host bridge. > > drivers/iommu/amd_iommu_init.c | 13 ++++++++++++- > 1 file changed, 12 insertions(+), 1 deletion(-) > > diff --git a/drivers/iommu/amd_iommu_init.c b/drivers/iommu/amd_iommu_init.c > index 568c52317757..139aa6fdadda 100644 > --- a/drivers/iommu/amd_iommu_init.c > +++ b/drivers/iommu/amd_iommu_init.c > @@ -2516,6 +2516,7 @@ static int __init early_amd_iommu_init(void) > struct acpi_table_header *ivrs_base; > acpi_status status; > int i, remap_cache_sz, ret = 0; > + u32 pci_id; > > if (!amd_iommu_detected) > return -ENODEV; > @@ -2603,6 +2604,16 @@ static int __init early_amd_iommu_init(void) > if (ret) > goto out; > > + /* Disable IOMMU if there's Stoney Ridge graphics */ > + for (i = 0; i < 32; i++) { > + pci_id = read_pci_config(0, i, 0, 0); > + if ((pci_id & 0xffff) == 0x1002 && (pci_id >> 16) == 0x98e4) { > + pr_info("Disable IOMMU on Stoney Ridge\n"); > + amd_iommu_disabled = true; > + break; > + } > + } > + > /* Disable any previously enabled IOMMUs */ > if (!is_kdump_kernel() || amd_iommu_disabled) > disable_iommus(); > @@ -2711,7 +2722,7 @@ static int __init state_next(void) > ret = early_amd_iommu_init(); > init_state = ret ? IOMMU_INIT_ERROR : IOMMU_ACPI_FINISHED; > if (init_state == IOMMU_ACPI_FINISHED && amd_iommu_disabled) { > - pr_info("AMD IOMMU disabled on kernel command-line\n"); > + pr_info("AMD IOMMU disabled\n"); > init_state = IOMMU_CMDLINE_DISABLED; > ret = -EINVAL; > } _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* RE: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-12-01 12:42 ` Lucas Stach @ 2019-12-02 16:36 ` Deucher, Alexander 2019-12-04 16:08 ` Deucher, Alexander 0 siblings, 1 reply; 15+ messages in thread From: Deucher, Alexander @ 2019-12-02 16:36 UTC (permalink / raw) To: Lucas Stach, Kai-Heng Feng, joro, Koenig, Christian; +Cc: iommu, linux-kernel > -----Original Message----- > From: Lucas Stach <dev@lynxeye.de> > Sent: Sunday, December 1, 2019 7:43 AM > To: Kai-Heng Feng <kai.heng.feng@canonical.com>; joro@8bytes.org > Cc: Deucher, Alexander <Alexander.Deucher@amd.com>; > iommu@lists.linux-foundation.org; linux-kernel@vger.kernel.org > Subject: Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge > systems > > Am Freitag, den 29.11.2019, 22:21 +0800 schrieb Kai-Heng Feng: > > Serious screen flickering when Stoney Ridge outputs to a 4K monitor. > > > > According to Alex Deucher, IOMMU isn't enabled on Windows, so let's do > > the same here to avoid screen flickering on 4K monitor. > > This doesn't seem like a good solution, especially if there isn't a method for > the user to opt-out. Some users might prefer having the IOMMU support to > 4K display output. > > But before using the big hammer of disabling or breaking one of those > features, we should take a look at what's the issue here. Screen flickering > caused by the IOMMU being active hints to the IOMMU not being able to > sustain the translation bandwidth required by the high- bandwidth > isochronous transfers caused by 4K scanout, most likely due to insufficient > TLB space. > > As far as I know the framebuffer memory for the display buffers is located in > stolen RAM, and thus contigous in memory. I don't know the details of the > GPU integration on those APUs, but maybe there even is a way to bypass the > IOMMU for the stolen VRAM regions? > > If there isn't and all GPU traffic passes through the IOMMU when active, we > should check if the stolen RAM is mapped with hugepages on the IOMMU > side. All the stolen RAM can most likely be mapped with a few hugepage > mappings, which should reduce IOMMU TLB demand by a large margin. The is no issue when we scan out of the carve out region. The issue occurs when we scan out of regular system memory (scatter/gather). Many newer laptops have very small carve out regions (e.g., 32 MB), so we have to use regular system pages to support multiple high resolution displays. The problem is, the latency gets too high at some point when the IOMMU is involved. Huge pages would probably help in this case, but I'm not sure if there is any way to guarantee that we get huge pages for system memory. I guess we could use CMA or something like that. Alex > > Regards, > Lucas > > > Cc: Alex Deucher <alexander.deucher@amd.com> > > Bug: > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitl > > > ab.freedesktop.org%2Fdrm%2Famd%2Fissues%2F961&data=02%7C01% > 7Calexa > > > nder.deucher%40amd.com%7C30540b2bf2be417c4d9508d7765bf07f%7C3dd > 8961fe4 > > > 884e608e11a82d994e183d%7C0%7C0%7C637108010075463266&sdata=1 > ZIZUWos > > cPiB4auOY10jlGzoFeWszYMDBQG0CtrrOO8%3D&reserved=0 > > Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com> > > --- > > v2: > > - Find Stoney graphics instead of host bridge. > > > > drivers/iommu/amd_iommu_init.c | 13 ++++++++++++- > > 1 file changed, 12 insertions(+), 1 deletion(-) > > > > diff --git a/drivers/iommu/amd_iommu_init.c > > b/drivers/iommu/amd_iommu_init.c index 568c52317757..139aa6fdadda > > 100644 > > --- a/drivers/iommu/amd_iommu_init.c > > +++ b/drivers/iommu/amd_iommu_init.c > > @@ -2516,6 +2516,7 @@ static int __init early_amd_iommu_init(void) > > struct acpi_table_header *ivrs_base; > > acpi_status status; > > int i, remap_cache_sz, ret = 0; > > + u32 pci_id; > > > > if (!amd_iommu_detected) > > return -ENODEV; > > @@ -2603,6 +2604,16 @@ static int __init early_amd_iommu_init(void) > > if (ret) > > goto out; > > > > + /* Disable IOMMU if there's Stoney Ridge graphics */ > > + for (i = 0; i < 32; i++) { > > + pci_id = read_pci_config(0, i, 0, 0); > > + if ((pci_id & 0xffff) == 0x1002 && (pci_id >> 16) == 0x98e4) { > > + pr_info("Disable IOMMU on Stoney Ridge\n"); > > + amd_iommu_disabled = true; > > + break; > > + } > > + } > > + > > /* Disable any previously enabled IOMMUs */ > > if (!is_kdump_kernel() || amd_iommu_disabled) > > disable_iommus(); > > @@ -2711,7 +2722,7 @@ static int __init state_next(void) > > ret = early_amd_iommu_init(); > > init_state = ret ? IOMMU_INIT_ERROR : > IOMMU_ACPI_FINISHED; > > if (init_state == IOMMU_ACPI_FINISHED && > amd_iommu_disabled) { > > - pr_info("AMD IOMMU disabled on kernel command- > line\n"); > > + pr_info("AMD IOMMU disabled\n"); > > init_state = IOMMU_CMDLINE_DISABLED; > > ret = -EINVAL; > > } _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* RE: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-12-02 16:36 ` Deucher, Alexander @ 2019-12-04 16:08 ` Deucher, Alexander 2019-12-06 5:59 ` Kai-Heng Feng 2019-12-06 8:57 ` Christian König 0 siblings, 2 replies; 15+ messages in thread From: Deucher, Alexander @ 2019-12-04 16:08 UTC (permalink / raw) To: Lucas Stach, Kai-Heng Feng, joro, Koenig, Christian; +Cc: iommu, linux-kernel > -----Original Message----- > From: Deucher, Alexander > Sent: Monday, December 2, 2019 11:37 AM > To: Lucas Stach <dev@lynxeye.de>; Kai-Heng Feng > <kai.heng.feng@canonical.com>; joro@8bytes.org; Koenig, Christian > (Christian.Koenig@amd.com) <Christian.Koenig@amd.com> > Cc: iommu@lists.linux-foundation.org; linux-kernel@vger.kernel.org > Subject: RE: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge > systems > > > -----Original Message----- > > From: Lucas Stach <dev@lynxeye.de> > > Sent: Sunday, December 1, 2019 7:43 AM > > To: Kai-Heng Feng <kai.heng.feng@canonical.com>; joro@8bytes.org > > Cc: Deucher, Alexander <Alexander.Deucher@amd.com>; > > iommu@lists.linux-foundation.org; linux-kernel@vger.kernel.org > > Subject: Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge > > systems > > > > Am Freitag, den 29.11.2019, 22:21 +0800 schrieb Kai-Heng Feng: > > > Serious screen flickering when Stoney Ridge outputs to a 4K monitor. > > > > > > According to Alex Deucher, IOMMU isn't enabled on Windows, so let's > > > do the same here to avoid screen flickering on 4K monitor. > > > > This doesn't seem like a good solution, especially if there isn't a > > method for the user to opt-out. Some users might prefer having the > > IOMMU support to 4K display output. > > > > But before using the big hammer of disabling or breaking one of those > > features, we should take a look at what's the issue here. Screen > > flickering caused by the IOMMU being active hints to the IOMMU not > > being able to sustain the translation bandwidth required by the high- > > bandwidth isochronous transfers caused by 4K scanout, most likely due > > to insufficient TLB space. > > > > As far as I know the framebuffer memory for the display buffers is > > located in stolen RAM, and thus contigous in memory. I don't know the > > details of the GPU integration on those APUs, but maybe there even is > > a way to bypass the IOMMU for the stolen VRAM regions? > > > > If there isn't and all GPU traffic passes through the IOMMU when > > active, we should check if the stolen RAM is mapped with hugepages on > > the IOMMU side. All the stolen RAM can most likely be mapped with a > > few hugepage mappings, which should reduce IOMMU TLB demand by a > large margin. > > The is no issue when we scan out of the carve out region. The issue occurs > when we scan out of regular system memory (scatter/gather). Many newer > laptops have very small carve out regions (e.g., 32 MB), so we have to use > regular system pages to support multiple high resolution displays. The > problem is, the latency gets too high at some point when the IOMMU is > involved. Huge pages would probably help in this case, but I'm not sure if > there is any way to guarantee that we get huge pages for system memory. I > guess we could use CMA or something like that. Thomas recently sent out a patch set to add huge page support to ttm: https://patchwork.freedesktop.org/series/70090/ We'd still need a way to guarantee huge pages for the display buffer. Alex > > Alex > > > > > Regards, > > Lucas > > > > > Cc: Alex Deucher <alexander.deucher@amd.com> > > > Bug: > > > > https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi > > > tl > > > > > > ab.freedesktop.org%2Fdrm%2Famd%2Fissues%2F961&data=02%7C01% > > 7Calexa > > > > > > nder.deucher%40amd.com%7C30540b2bf2be417c4d9508d7765bf07f%7C3dd > > 8961fe4 > > > > > > 884e608e11a82d994e183d%7C0%7C0%7C637108010075463266&sdata=1 > > ZIZUWos > > > cPiB4auOY10jlGzoFeWszYMDBQG0CtrrOO8%3D&reserved=0 > > > Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com> > > > --- > > > v2: > > > - Find Stoney graphics instead of host bridge. > > > > > > drivers/iommu/amd_iommu_init.c | 13 ++++++++++++- > > > 1 file changed, 12 insertions(+), 1 deletion(-) > > > > > > diff --git a/drivers/iommu/amd_iommu_init.c > > > b/drivers/iommu/amd_iommu_init.c index 568c52317757..139aa6fdadda > > > 100644 > > > --- a/drivers/iommu/amd_iommu_init.c > > > +++ b/drivers/iommu/amd_iommu_init.c > > > @@ -2516,6 +2516,7 @@ static int __init early_amd_iommu_init(void) > > > struct acpi_table_header *ivrs_base; > > > acpi_status status; > > > int i, remap_cache_sz, ret = 0; > > > + u32 pci_id; > > > > > > if (!amd_iommu_detected) > > > return -ENODEV; > > > @@ -2603,6 +2604,16 @@ static int __init early_amd_iommu_init(void) > > > if (ret) > > > goto out; > > > > > > + /* Disable IOMMU if there's Stoney Ridge graphics */ > > > + for (i = 0; i < 32; i++) { > > > + pci_id = read_pci_config(0, i, 0, 0); > > > + if ((pci_id & 0xffff) == 0x1002 && (pci_id >> 16) == 0x98e4) { > > > + pr_info("Disable IOMMU on Stoney Ridge\n"); > > > + amd_iommu_disabled = true; > > > + break; > > > + } > > > + } > > > + > > > /* Disable any previously enabled IOMMUs */ > > > if (!is_kdump_kernel() || amd_iommu_disabled) > > > disable_iommus(); > > > @@ -2711,7 +2722,7 @@ static int __init state_next(void) > > > ret = early_amd_iommu_init(); > > > init_state = ret ? IOMMU_INIT_ERROR : > > IOMMU_ACPI_FINISHED; > > > if (init_state == IOMMU_ACPI_FINISHED && > > amd_iommu_disabled) { > > > - pr_info("AMD IOMMU disabled on kernel command- > > line\n"); > > > + pr_info("AMD IOMMU disabled\n"); > > > init_state = IOMMU_CMDLINE_DISABLED; > > > ret = -EINVAL; > > > } _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-12-04 16:08 ` Deucher, Alexander @ 2019-12-06 5:59 ` Kai-Heng Feng 2019-12-06 8:57 ` Christian König 1 sibling, 0 replies; 15+ messages in thread From: Kai-Heng Feng @ 2019-12-06 5:59 UTC (permalink / raw) To: Deucher, Alexander; +Cc: Koenig, Christian, iommu, linux-kernel > On Dec 5, 2019, at 00:08, Deucher, Alexander <alexander.deucher@amd.com> wrote: > >> -----Original Message----- >> From: Deucher, Alexander >> Sent: Monday, December 2, 2019 11:37 AM >> To: Lucas Stach <dev@lynxeye.de>; Kai-Heng Feng >> <kai.heng.feng@canonical.com>; joro@8bytes.org; Koenig, Christian >> (Christian.Koenig@amd.com) <Christian.Koenig@amd.com> >> Cc: iommu@lists.linux-foundation.org; linux-kernel@vger.kernel.org >> Subject: RE: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge >> systems >> >>> -----Original Message----- >>> From: Lucas Stach <dev@lynxeye.de> >>> Sent: Sunday, December 1, 2019 7:43 AM >>> To: Kai-Heng Feng <kai.heng.feng@canonical.com>; joro@8bytes.org >>> Cc: Deucher, Alexander <Alexander.Deucher@amd.com>; >>> iommu@lists.linux-foundation.org; linux-kernel@vger.kernel.org >>> Subject: Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge >>> systems >>> >>> Am Freitag, den 29.11.2019, 22:21 +0800 schrieb Kai-Heng Feng: >>>> Serious screen flickering when Stoney Ridge outputs to a 4K monitor. >>>> >>>> According to Alex Deucher, IOMMU isn't enabled on Windows, so let's >>>> do the same here to avoid screen flickering on 4K monitor. >>> >>> This doesn't seem like a good solution, especially if there isn't a >>> method for the user to opt-out. Some users might prefer having the >>> IOMMU support to 4K display output. >>> >>> But before using the big hammer of disabling or breaking one of those >>> features, we should take a look at what's the issue here. Screen >>> flickering caused by the IOMMU being active hints to the IOMMU not >>> being able to sustain the translation bandwidth required by the high- >>> bandwidth isochronous transfers caused by 4K scanout, most likely due >>> to insufficient TLB space. >>> >>> As far as I know the framebuffer memory for the display buffers is >>> located in stolen RAM, and thus contigous in memory. I don't know the >>> details of the GPU integration on those APUs, but maybe there even is >>> a way to bypass the IOMMU for the stolen VRAM regions? >>> >>> If there isn't and all GPU traffic passes through the IOMMU when >>> active, we should check if the stolen RAM is mapped with hugepages on >>> the IOMMU side. All the stolen RAM can most likely be mapped with a >>> few hugepage mappings, which should reduce IOMMU TLB demand by a >> large margin. >> >> The is no issue when we scan out of the carve out region. The issue occurs >> when we scan out of regular system memory (scatter/gather). Many newer >> laptops have very small carve out regions (e.g., 32 MB), so we have to use >> regular system pages to support multiple high resolution displays. The >> problem is, the latency gets too high at some point when the IOMMU is >> involved. Huge pages would probably help in this case, but I'm not sure if >> there is any way to guarantee that we get huge pages for system memory. I >> guess we could use CMA or something like that. > > Thomas recently sent out a patch set to add huge page support to ttm: > https://patchwork.freedesktop.org/series/70090/ > We'd still need a way to guarantee huge pages for the display buffer. Is there an amdgpu counterpart to let me test out? Kai-Heng > > Alex > >> >> Alex >> >>> >>> Regards, >>> Lucas >>> >>>> Cc: Alex Deucher <alexander.deucher@amd.com> >>>> Bug: >>>> >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi >>>> tl >>>> >>> >> ab.freedesktop.org%2Fdrm%2Famd%2Fissues%2F961&data=02%7C01% >>> 7Calexa >>>> >>> >> nder.deucher%40amd.com%7C30540b2bf2be417c4d9508d7765bf07f%7C3dd >>> 8961fe4 >>>> >>> >> 884e608e11a82d994e183d%7C0%7C0%7C637108010075463266&sdata=1 >>> ZIZUWos >>>> cPiB4auOY10jlGzoFeWszYMDBQG0CtrrOO8%3D&reserved=0 >>>> Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com> >>>> --- >>>> v2: >>>> - Find Stoney graphics instead of host bridge. >>>> >>>> drivers/iommu/amd_iommu_init.c | 13 ++++++++++++- >>>> 1 file changed, 12 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/drivers/iommu/amd_iommu_init.c >>>> b/drivers/iommu/amd_iommu_init.c index 568c52317757..139aa6fdadda >>>> 100644 >>>> --- a/drivers/iommu/amd_iommu_init.c >>>> +++ b/drivers/iommu/amd_iommu_init.c >>>> @@ -2516,6 +2516,7 @@ static int __init early_amd_iommu_init(void) >>>> struct acpi_table_header *ivrs_base; >>>> acpi_status status; >>>> int i, remap_cache_sz, ret = 0; >>>> + u32 pci_id; >>>> >>>> if (!amd_iommu_detected) >>>> return -ENODEV; >>>> @@ -2603,6 +2604,16 @@ static int __init early_amd_iommu_init(void) >>>> if (ret) >>>> goto out; >>>> >>>> + /* Disable IOMMU if there's Stoney Ridge graphics */ >>>> + for (i = 0; i < 32; i++) { >>>> + pci_id = read_pci_config(0, i, 0, 0); >>>> + if ((pci_id & 0xffff) == 0x1002 && (pci_id >> 16) == 0x98e4) { >>>> + pr_info("Disable IOMMU on Stoney Ridge\n"); >>>> + amd_iommu_disabled = true; >>>> + break; >>>> + } >>>> + } >>>> + >>>> /* Disable any previously enabled IOMMUs */ >>>> if (!is_kdump_kernel() || amd_iommu_disabled) >>>> disable_iommus(); >>>> @@ -2711,7 +2722,7 @@ static int __init state_next(void) >>>> ret = early_amd_iommu_init(); >>>> init_state = ret ? IOMMU_INIT_ERROR : >>> IOMMU_ACPI_FINISHED; >>>> if (init_state == IOMMU_ACPI_FINISHED && >>> amd_iommu_disabled) { >>>> - pr_info("AMD IOMMU disabled on kernel command- >>> line\n"); >>>> + pr_info("AMD IOMMU disabled\n"); >>>> init_state = IOMMU_CMDLINE_DISABLED; >>>> ret = -EINVAL; >>>> } _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-12-04 16:08 ` Deucher, Alexander 2019-12-06 5:59 ` Kai-Heng Feng @ 2019-12-06 8:57 ` Christian König 1 sibling, 0 replies; 15+ messages in thread From: Christian König @ 2019-12-06 8:57 UTC (permalink / raw) To: Deucher, Alexander, Lucas Stach, Kai-Heng Feng, joro; +Cc: iommu, linux-kernel Am 04.12.19 um 17:08 schrieb Deucher, Alexander: >> -----Original Message----- >> From: Deucher, Alexander >> Sent: Monday, December 2, 2019 11:37 AM >> To: Lucas Stach <dev@lynxeye.de>; Kai-Heng Feng >> <kai.heng.feng@canonical.com>; joro@8bytes.org; Koenig, Christian >> (Christian.Koenig@amd.com) <Christian.Koenig@amd.com> >> Cc: iommu@lists.linux-foundation.org; linux-kernel@vger.kernel.org >> Subject: RE: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge >> systems >> >>> -----Original Message----- >>> From: Lucas Stach <dev@lynxeye.de> >>> Sent: Sunday, December 1, 2019 7:43 AM >>> To: Kai-Heng Feng <kai.heng.feng@canonical.com>; joro@8bytes.org >>> Cc: Deucher, Alexander <Alexander.Deucher@amd.com>; >>> iommu@lists.linux-foundation.org; linux-kernel@vger.kernel.org >>> Subject: Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge >>> systems >>> >>> Am Freitag, den 29.11.2019, 22:21 +0800 schrieb Kai-Heng Feng: >>>> Serious screen flickering when Stoney Ridge outputs to a 4K monitor. >>>> >>>> According to Alex Deucher, IOMMU isn't enabled on Windows, so let's >>>> do the same here to avoid screen flickering on 4K monitor. >>> This doesn't seem like a good solution, especially if there isn't a >>> method for the user to opt-out. Some users might prefer having the >>> IOMMU support to 4K display output. >>> >>> But before using the big hammer of disabling or breaking one of those >>> features, we should take a look at what's the issue here. Screen >>> flickering caused by the IOMMU being active hints to the IOMMU not >>> being able to sustain the translation bandwidth required by the high- >>> bandwidth isochronous transfers caused by 4K scanout, most likely due >>> to insufficient TLB space. >>> >>> As far as I know the framebuffer memory for the display buffers is >>> located in stolen RAM, and thus contigous in memory. I don't know the >>> details of the GPU integration on those APUs, but maybe there even is >>> a way to bypass the IOMMU for the stolen VRAM regions? >>> >>> If there isn't and all GPU traffic passes through the IOMMU when >>> active, we should check if the stolen RAM is mapped with hugepages on >>> the IOMMU side. All the stolen RAM can most likely be mapped with a >>> few hugepage mappings, which should reduce IOMMU TLB demand by a >> large margin. >> >> The is no issue when we scan out of the carve out region. The issue occurs >> when we scan out of regular system memory (scatter/gather). Many newer >> laptops have very small carve out regions (e.g., 32 MB), so we have to use >> regular system pages to support multiple high resolution displays. The >> problem is, the latency gets too high at some point when the IOMMU is >> involved. Huge pages would probably help in this case, but I'm not sure if >> there is any way to guarantee that we get huge pages for system memory. I >> guess we could use CMA or something like that. > Thomas recently sent out a patch set to add huge page support to ttm: > https://patchwork.freedesktop.org/series/70090/ > We'd still need a way to guarantee huge pages for the display buffer. That unfortunately won't help in this case since the TTM work Thomas is doing only affects the CPU page tables. Additional to that we already allocate huge pages for the display buffer in a best effort manner and it doesn't seem to help. If I understood the hardware guys correctly even transparent mode adds to much latency so that the display block might run into an underflow. The only solution documented to work is to either disabling the IOMMU or not using scan-out from system memory. Alex, we should probably kick of another internal discussion with the hardware guys about that. Christian. > > Alex > >> Alex >> >>> Regards, >>> Lucas >>> >>>> Cc: Alex Deucher <alexander.deucher@amd.com> >>>> Bug: >>>> >> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgi >>>> tl >>>> >> ab.freedesktop.org%2Fdrm%2Famd%2Fissues%2F961&data=02%7C01% >>> 7Calexa >> nder.deucher%40amd.com%7C30540b2bf2be417c4d9508d7765bf07f%7C3dd >>> 8961fe4 >> 884e608e11a82d994e183d%7C0%7C0%7C637108010075463266&sdata=1 >>> ZIZUWos >>>> cPiB4auOY10jlGzoFeWszYMDBQG0CtrrOO8%3D&reserved=0 >>>> Signed-off-by: Kai-Heng Feng <kai.heng.feng@canonical.com> >>>> --- >>>> v2: >>>> - Find Stoney graphics instead of host bridge. >>>> >>>> drivers/iommu/amd_iommu_init.c | 13 ++++++++++++- >>>> 1 file changed, 12 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/drivers/iommu/amd_iommu_init.c >>>> b/drivers/iommu/amd_iommu_init.c index 568c52317757..139aa6fdadda >>>> 100644 >>>> --- a/drivers/iommu/amd_iommu_init.c >>>> +++ b/drivers/iommu/amd_iommu_init.c >>>> @@ -2516,6 +2516,7 @@ static int __init early_amd_iommu_init(void) >>>> struct acpi_table_header *ivrs_base; >>>> acpi_status status; >>>> int i, remap_cache_sz, ret = 0; >>>> + u32 pci_id; >>>> >>>> if (!amd_iommu_detected) >>>> return -ENODEV; >>>> @@ -2603,6 +2604,16 @@ static int __init early_amd_iommu_init(void) >>>> if (ret) >>>> goto out; >>>> >>>> + /* Disable IOMMU if there's Stoney Ridge graphics */ >>>> + for (i = 0; i < 32; i++) { >>>> + pci_id = read_pci_config(0, i, 0, 0); >>>> + if ((pci_id & 0xffff) == 0x1002 && (pci_id >> 16) == 0x98e4) { >>>> + pr_info("Disable IOMMU on Stoney Ridge\n"); >>>> + amd_iommu_disabled = true; >>>> + break; >>>> + } >>>> + } >>>> + >>>> /* Disable any previously enabled IOMMUs */ >>>> if (!is_kdump_kernel() || amd_iommu_disabled) >>>> disable_iommus(); >>>> @@ -2711,7 +2722,7 @@ static int __init state_next(void) >>>> ret = early_amd_iommu_init(); >>>> init_state = ret ? IOMMU_INIT_ERROR : >>> IOMMU_ACPI_FINISHED; >>>> if (init_state == IOMMU_ACPI_FINISHED && >>> amd_iommu_disabled) { >>>> - pr_info("AMD IOMMU disabled on kernel command- >>> line\n"); >>>> + pr_info("AMD IOMMU disabled\n"); >>>> init_state = IOMMU_CMDLINE_DISABLED; >>>> ret = -EINVAL; >>>> } _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-11-29 14:21 [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems Kai-Heng Feng 2019-12-01 12:42 ` Lucas Stach @ 2019-12-02 17:00 ` Christoph Hellwig 2019-12-06 5:57 ` Kai-Heng Feng 1 sibling, 1 reply; 15+ messages in thread From: Christoph Hellwig @ 2019-12-02 17:00 UTC (permalink / raw) To: Kai-Heng Feng; +Cc: Alex Deucher, iommu, linux-kernel On Fri, Nov 29, 2019 at 10:21:54PM +0800, Kai-Heng Feng wrote: > Serious screen flickering when Stoney Ridge outputs to a 4K monitor. > > According to Alex Deucher, IOMMU isn't enabled on Windows, so let's do > the same here to avoid screen flickering on 4K monitor. Disabling the IOMMU entirely seem pretty severe. Isn't it enough to identity map the GPU device? _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-12-02 17:00 ` Christoph Hellwig @ 2019-12-06 5:57 ` Kai-Heng Feng 2019-12-17 9:53 ` Joerg Roedel 0 siblings, 1 reply; 15+ messages in thread From: Kai-Heng Feng @ 2019-12-06 5:57 UTC (permalink / raw) To: Joerg Roedel; +Cc: Alex Deucher, iommu, Kernel development list Hi Joerg, > On Dec 3, 2019, at 01:00, Christoph Hellwig <hch@infradead.org> wrote: > > On Fri, Nov 29, 2019 at 10:21:54PM +0800, Kai-Heng Feng wrote: >> Serious screen flickering when Stoney Ridge outputs to a 4K monitor. >> >> According to Alex Deucher, IOMMU isn't enabled on Windows, so let's do >> the same here to avoid screen flickering on 4K monitor. > > Disabling the IOMMU entirely seem pretty severe. Isn't it enough to > identity map the GPU device? Ok, there's set_device_exclusion_range() to exclude the device from IOMMU. However I don't know how to generate range_start and range_length, which are read from ACPI. Can you please give me some advice here? Kai-Henge _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-12-06 5:57 ` Kai-Heng Feng @ 2019-12-17 9:53 ` Joerg Roedel 2019-12-18 17:45 ` Kai-Heng Feng 0 siblings, 1 reply; 15+ messages in thread From: Joerg Roedel @ 2019-12-17 9:53 UTC (permalink / raw) To: Kai-Heng Feng; +Cc: Alex Deucher, iommu, Kernel development list On Fri, Dec 06, 2019 at 01:57:41PM +0800, Kai-Heng Feng wrote: > Hi Joerg, > > > On Dec 3, 2019, at 01:00, Christoph Hellwig <hch@infradead.org> wrote: > > > > On Fri, Nov 29, 2019 at 10:21:54PM +0800, Kai-Heng Feng wrote: > >> Serious screen flickering when Stoney Ridge outputs to a 4K monitor. > >> > >> According to Alex Deucher, IOMMU isn't enabled on Windows, so let's do > >> the same here to avoid screen flickering on 4K monitor. > > > > Disabling the IOMMU entirely seem pretty severe. Isn't it enough to > > identity map the GPU device? > > Ok, there's set_device_exclusion_range() to exclude the device from IOMMU. > However I don't know how to generate range_start and range_length, which are read from ACPI. set_device_exclusion_range() is not the solution here. The best is if the GPU device is put into a passthrough domain at boot, in which it will be identity mapped. DMA still goes through the IOMMU in this case, but it only needs to lookup the device-table, page-table walks will not be done anymore. The best way to implement this is to put it into the amd_iommu_add_device() in drivers/iommu/amd_iommu.c. There is this check: if (dev_data->iommu_v2) iommu_request_dm_for_dev(dev); The iommu_request_dm_for_dev() function causes the device to be identity mapped. The check can be extended to also check for a device white-list for devices that need identity mapping. HTH, Joerg _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-12-17 9:53 ` Joerg Roedel @ 2019-12-18 17:45 ` Kai-Heng Feng 2019-12-19 19:15 ` Deucher, Alexander 0 siblings, 1 reply; 15+ messages in thread From: Kai-Heng Feng @ 2019-12-18 17:45 UTC (permalink / raw) To: Joerg Roedel; +Cc: Alex Deucher, iommu, Kernel development list > On Dec 17, 2019, at 17:53, Joerg Roedel <joro@8bytes.org> wrote: > > On Fri, Dec 06, 2019 at 01:57:41PM +0800, Kai-Heng Feng wrote: >> Hi Joerg, >> >>> On Dec 3, 2019, at 01:00, Christoph Hellwig <hch@infradead.org> wrote: >>> >>> On Fri, Nov 29, 2019 at 10:21:54PM +0800, Kai-Heng Feng wrote: >>>> Serious screen flickering when Stoney Ridge outputs to a 4K monitor. >>>> >>>> According to Alex Deucher, IOMMU isn't enabled on Windows, so let's do >>>> the same here to avoid screen flickering on 4K monitor. >>> >>> Disabling the IOMMU entirely seem pretty severe. Isn't it enough to >>> identity map the GPU device? >> >> Ok, there's set_device_exclusion_range() to exclude the device from IOMMU. >> However I don't know how to generate range_start and range_length, which are read from ACPI. > > set_device_exclusion_range() is not the solution here. The best is if > the GPU device is put into a passthrough domain at boot, in which it > will be identity mapped. DMA still goes through the IOMMU in this case, > but it only needs to lookup the device-table, page-table walks will not > be done anymore. > > The best way to implement this is to put it into the > amd_iommu_add_device() in drivers/iommu/amd_iommu.c. There is this > check: > > if (dev_data->iommu_v2) > iommu_request_dm_for_dev(dev); > > The iommu_request_dm_for_dev() function causes the device to be identity > mapped. The check can be extended to also check for a device white-list > for devices that need identity mapping. My patch looks like this but the original behavior (4K screen flickering) is still the same: diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index bd25674ee4db..f913a25c9e92 100644 --- a/drivers/iommu/amd_iommu.c +++ b/drivers/iommu/amd_iommu.c @@ -42,6 +42,7 @@ #include <asm/iommu.h> #include <asm/gart.h> #include <asm/dma.h> +#include <asm/pci-direct.h> #include "amd_iommu_proto.h" #include "amd_iommu_types.h" @@ -2159,6 +2160,8 @@ static int amd_iommu_add_device(struct device *dev) struct iommu_domain *domain; struct amd_iommu *iommu; int ret, devid; + bool need_identity_mapping = false; + u32 header; if (!check_device(dev) || get_dev_data(dev)) return 0; @@ -2184,7 +2187,11 @@ static int amd_iommu_add_device(struct device *dev) BUG_ON(!dev_data); - if (dev_data->iommu_v2) + header = read_pci_config(0, PCI_BUS_NUM(devid), PCI_SLOT(devid), PCI_FUNC(devid)); + if ((header & 0xffff) == 0x1002 && (header >> 16) == 0x98e4) + need_identity_mapping = true; + + if (dev_data->iommu_v2 || need_identity_mapping) iommu_request_dm_for_dev(dev); /* Domains are initialized for this device - have a look what we ended up with */ $ dmesg | grep -i direct [ 0.011446] Using GB pages for direct mapping [ 0.703369] pci 0000:00:01.0: Using iommu direct mapping [ 0.703830] pci 0000:00:08.0: Using iommu direct mapping So the graphics device (pci 0000:00:01.0:) is using direct mapping after the change. Kai-Heng > > HTH, > > Joerg _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply related [flat|nested] 15+ messages in thread
* RE: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-12-18 17:45 ` Kai-Heng Feng @ 2019-12-19 19:15 ` Deucher, Alexander 2019-12-20 2:13 ` Kai-Heng Feng 0 siblings, 1 reply; 15+ messages in thread From: Deucher, Alexander @ 2019-12-19 19:15 UTC (permalink / raw) To: Kai-Heng Feng, Joerg Roedel; +Cc: iommu, Kernel development list > -----Original Message----- > From: Kai-Heng Feng <kai.heng.feng@canonical.com> > Sent: Wednesday, December 18, 2019 12:45 PM > To: Joerg Roedel <joro@8bytes.org> > Cc: Christoph Hellwig <hch@infradead.org>; Deucher, Alexander > <Alexander.Deucher@amd.com>; iommu@lists.linux-foundation.org; Kernel > development list <linux-kernel@vger.kernel.org> > Subject: Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge > systems > > > > > On Dec 17, 2019, at 17:53, Joerg Roedel <joro@8bytes.org> wrote: > > > > On Fri, Dec 06, 2019 at 01:57:41PM +0800, Kai-Heng Feng wrote: > >> Hi Joerg, > >> > >>> On Dec 3, 2019, at 01:00, Christoph Hellwig <hch@infradead.org> wrote: > >>> > >>> On Fri, Nov 29, 2019 at 10:21:54PM +0800, Kai-Heng Feng wrote: > >>>> Serious screen flickering when Stoney Ridge outputs to a 4K monitor. > >>>> > >>>> According to Alex Deucher, IOMMU isn't enabled on Windows, so let's > >>>> do the same here to avoid screen flickering on 4K monitor. > >>> > >>> Disabling the IOMMU entirely seem pretty severe. Isn't it enough to > >>> identity map the GPU device? > >> > >> Ok, there's set_device_exclusion_range() to exclude the device from > IOMMU. > >> However I don't know how to generate range_start and range_length, > which are read from ACPI. > > > > set_device_exclusion_range() is not the solution here. The best is if > > the GPU device is put into a passthrough domain at boot, in which it > > will be identity mapped. DMA still goes through the IOMMU in this > > case, but it only needs to lookup the device-table, page-table walks > > will not be done anymore. > > > > The best way to implement this is to put it into the > > amd_iommu_add_device() in drivers/iommu/amd_iommu.c. There is this > > check: > > > > if (dev_data->iommu_v2) > > iommu_request_dm_for_dev(dev); > > > > The iommu_request_dm_for_dev() function causes the device to be > > identity mapped. The check can be extended to also check for a device > > white-list for devices that need identity mapping. > > My patch looks like this but the original behavior (4K screen flickering) is still > the same: Does reverting the patch to disable ATS along with this patch help? Alex > > diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c > index bd25674ee4db..f913a25c9e92 100644 > --- a/drivers/iommu/amd_iommu.c > +++ b/drivers/iommu/amd_iommu.c > @@ -42,6 +42,7 @@ > #include <asm/iommu.h> > #include <asm/gart.h> > #include <asm/dma.h> > +#include <asm/pci-direct.h> > > #include "amd_iommu_proto.h" > #include "amd_iommu_types.h" > @@ -2159,6 +2160,8 @@ static int amd_iommu_add_device(struct device > *dev) > struct iommu_domain *domain; > struct amd_iommu *iommu; > int ret, devid; > + bool need_identity_mapping = false; > + u32 header; > > if (!check_device(dev) || get_dev_data(dev)) > return 0; > @@ -2184,7 +2187,11 @@ static int amd_iommu_add_device(struct device > *dev) > > BUG_ON(!dev_data); > > - if (dev_data->iommu_v2) > + header = read_pci_config(0, PCI_BUS_NUM(devid), PCI_SLOT(devid), > PCI_FUNC(devid)); > + if ((header & 0xffff) == 0x1002 && (header >> 16) == 0x98e4) > + need_identity_mapping = true; > + > + if (dev_data->iommu_v2 || need_identity_mapping) > iommu_request_dm_for_dev(dev); > > /* Domains are initialized for this device - have a look what we ended up > with */ > > > $ dmesg | grep -i direct > [ 0.011446] Using GB pages for direct mapping > [ 0.703369] pci 0000:00:01.0: Using iommu direct mapping > [ 0.703830] pci 0000:00:08.0: Using iommu direct mapping > > So the graphics device (pci 0000:00:01.0:) is using direct mapping after the > change. > > Kai-Heng > > > > > HTH, > > > > Joerg _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-12-19 19:15 ` Deucher, Alexander @ 2019-12-20 2:13 ` Kai-Heng Feng 2020-01-06 8:37 ` Kai-Heng Feng 0 siblings, 1 reply; 15+ messages in thread From: Kai-Heng Feng @ 2019-12-20 2:13 UTC (permalink / raw) To: Deucher, Alexander; +Cc: iommu, Kernel development list > On Dec 20, 2019, at 03:15, Deucher, Alexander <Alexander.Deucher@amd.com> wrote: > >> -----Original Message----- >> From: Kai-Heng Feng <kai.heng.feng@canonical.com> >> Sent: Wednesday, December 18, 2019 12:45 PM >> To: Joerg Roedel <joro@8bytes.org> >> Cc: Christoph Hellwig <hch@infradead.org>; Deucher, Alexander >> <Alexander.Deucher@amd.com>; iommu@lists.linux-foundation.org; Kernel >> development list <linux-kernel@vger.kernel.org> >> Subject: Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge >> systems >> >> >> >>> On Dec 17, 2019, at 17:53, Joerg Roedel <joro@8bytes.org> wrote: >>> >>> On Fri, Dec 06, 2019 at 01:57:41PM +0800, Kai-Heng Feng wrote: >>>> Hi Joerg, >>>> >>>>> On Dec 3, 2019, at 01:00, Christoph Hellwig <hch@infradead.org> wrote: >>>>> >>>>> On Fri, Nov 29, 2019 at 10:21:54PM +0800, Kai-Heng Feng wrote: >>>>>> Serious screen flickering when Stoney Ridge outputs to a 4K monitor. >>>>>> >>>>>> According to Alex Deucher, IOMMU isn't enabled on Windows, so let's >>>>>> do the same here to avoid screen flickering on 4K monitor. >>>>> >>>>> Disabling the IOMMU entirely seem pretty severe. Isn't it enough to >>>>> identity map the GPU device? >>>> >>>> Ok, there's set_device_exclusion_range() to exclude the device from >> IOMMU. >>>> However I don't know how to generate range_start and range_length, >> which are read from ACPI. >>> >>> set_device_exclusion_range() is not the solution here. The best is if >>> the GPU device is put into a passthrough domain at boot, in which it >>> will be identity mapped. DMA still goes through the IOMMU in this >>> case, but it only needs to lookup the device-table, page-table walks >>> will not be done anymore. >>> >>> The best way to implement this is to put it into the >>> amd_iommu_add_device() in drivers/iommu/amd_iommu.c. There is this >>> check: >>> >>> if (dev_data->iommu_v2) >>> iommu_request_dm_for_dev(dev); >>> >>> The iommu_request_dm_for_dev() function causes the device to be >>> identity mapped. The check can be extended to also check for a device >>> white-list for devices that need identity mapping. >> >> My patch looks like this but the original behavior (4K screen flickering) is still >> the same: > > Does reverting the patch to disable ATS along with this patch help? Unfortunately it doesn't help. Kai-Heng > > Alex > >> >> diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c >> index bd25674ee4db..f913a25c9e92 100644 >> --- a/drivers/iommu/amd_iommu.c >> +++ b/drivers/iommu/amd_iommu.c >> @@ -42,6 +42,7 @@ >> #include <asm/iommu.h> >> #include <asm/gart.h> >> #include <asm/dma.h> >> +#include <asm/pci-direct.h> >> >> #include "amd_iommu_proto.h" >> #include "amd_iommu_types.h" >> @@ -2159,6 +2160,8 @@ static int amd_iommu_add_device(struct device >> *dev) >> struct iommu_domain *domain; >> struct amd_iommu *iommu; >> int ret, devid; >> + bool need_identity_mapping = false; >> + u32 header; >> >> if (!check_device(dev) || get_dev_data(dev)) >> return 0; >> @@ -2184,7 +2187,11 @@ static int amd_iommu_add_device(struct device >> *dev) >> >> BUG_ON(!dev_data); >> >> - if (dev_data->iommu_v2) >> + header = read_pci_config(0, PCI_BUS_NUM(devid), PCI_SLOT(devid), >> PCI_FUNC(devid)); >> + if ((header & 0xffff) == 0x1002 && (header >> 16) == 0x98e4) >> + need_identity_mapping = true; >> + >> + if (dev_data->iommu_v2 || need_identity_mapping) >> iommu_request_dm_for_dev(dev); >> >> /* Domains are initialized for this device - have a look what we ended up >> with */ >> >> >> $ dmesg | grep -i direct >> [ 0.011446] Using GB pages for direct mapping >> [ 0.703369] pci 0000:00:01.0: Using iommu direct mapping >> [ 0.703830] pci 0000:00:08.0: Using iommu direct mapping >> >> So the graphics device (pci 0000:00:01.0:) is using direct mapping after the >> change. >> >> Kai-Heng >> >>> >>> HTH, >>> >>> Joerg _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2019-12-20 2:13 ` Kai-Heng Feng @ 2020-01-06 8:37 ` Kai-Heng Feng 2020-02-05 10:10 ` Kai-Heng Feng 0 siblings, 1 reply; 15+ messages in thread From: Kai-Heng Feng @ 2020-01-06 8:37 UTC (permalink / raw) To: Deucher, Alexander; +Cc: iommu, Kernel development list > On Dec 20, 2019, at 10:13, Kai-Heng Feng <kai.heng.feng@canonical.com> wrote: > > > >> On Dec 20, 2019, at 03:15, Deucher, Alexander <Alexander.Deucher@amd.com> wrote: >> >>> -----Original Message----- >>> From: Kai-Heng Feng <kai.heng.feng@canonical.com> >>> Sent: Wednesday, December 18, 2019 12:45 PM >>> To: Joerg Roedel <joro@8bytes.org> >>> Cc: Christoph Hellwig <hch@infradead.org>; Deucher, Alexander >>> <Alexander.Deucher@amd.com>; iommu@lists.linux-foundation.org; Kernel >>> development list <linux-kernel@vger.kernel.org> >>> Subject: Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge >>> systems >>> >>> >>> >>>> On Dec 17, 2019, at 17:53, Joerg Roedel <joro@8bytes.org> wrote: >>>> >>>> On Fri, Dec 06, 2019 at 01:57:41PM +0800, Kai-Heng Feng wrote: >>>>> Hi Joerg, >>>>> >>>>>> On Dec 3, 2019, at 01:00, Christoph Hellwig <hch@infradead.org> wrote: >>>>>> >>>>>> On Fri, Nov 29, 2019 at 10:21:54PM +0800, Kai-Heng Feng wrote: >>>>>>> Serious screen flickering when Stoney Ridge outputs to a 4K monitor. >>>>>>> >>>>>>> According to Alex Deucher, IOMMU isn't enabled on Windows, so let's >>>>>>> do the same here to avoid screen flickering on 4K monitor. >>>>>> >>>>>> Disabling the IOMMU entirely seem pretty severe. Isn't it enough to >>>>>> identity map the GPU device? >>>>> >>>>> Ok, there's set_device_exclusion_range() to exclude the device from >>> IOMMU. >>>>> However I don't know how to generate range_start and range_length, >>> which are read from ACPI. >>>> >>>> set_device_exclusion_range() is not the solution here. The best is if >>>> the GPU device is put into a passthrough domain at boot, in which it >>>> will be identity mapped. DMA still goes through the IOMMU in this >>>> case, but it only needs to lookup the device-table, page-table walks >>>> will not be done anymore. >>>> >>>> The best way to implement this is to put it into the >>>> amd_iommu_add_device() in drivers/iommu/amd_iommu.c. There is this >>>> check: >>>> >>>> if (dev_data->iommu_v2) >>>> iommu_request_dm_for_dev(dev); >>>> >>>> The iommu_request_dm_for_dev() function causes the device to be >>>> identity mapped. The check can be extended to also check for a device >>>> white-list for devices that need identity mapping. >>> >>> My patch looks like this but the original behavior (4K screen flickering) is still >>> the same: >> >> Does reverting the patch to disable ATS along with this patch help? > > Unfortunately it doesn't help. Any further suggestion to let me try? Kai-Heng > > Kai-Heng > >> >> Alex >> >>> >>> diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c >>> index bd25674ee4db..f913a25c9e92 100644 >>> --- a/drivers/iommu/amd_iommu.c >>> +++ b/drivers/iommu/amd_iommu.c >>> @@ -42,6 +42,7 @@ >>> #include <asm/iommu.h> >>> #include <asm/gart.h> >>> #include <asm/dma.h> >>> +#include <asm/pci-direct.h> >>> >>> #include "amd_iommu_proto.h" >>> #include "amd_iommu_types.h" >>> @@ -2159,6 +2160,8 @@ static int amd_iommu_add_device(struct device >>> *dev) >>> struct iommu_domain *domain; >>> struct amd_iommu *iommu; >>> int ret, devid; >>> + bool need_identity_mapping = false; >>> + u32 header; >>> >>> if (!check_device(dev) || get_dev_data(dev)) >>> return 0; >>> @@ -2184,7 +2187,11 @@ static int amd_iommu_add_device(struct device >>> *dev) >>> >>> BUG_ON(!dev_data); >>> >>> - if (dev_data->iommu_v2) >>> + header = read_pci_config(0, PCI_BUS_NUM(devid), PCI_SLOT(devid), >>> PCI_FUNC(devid)); >>> + if ((header & 0xffff) == 0x1002 && (header >> 16) == 0x98e4) >>> + need_identity_mapping = true; >>> + >>> + if (dev_data->iommu_v2 || need_identity_mapping) >>> iommu_request_dm_for_dev(dev); >>> >>> /* Domains are initialized for this device - have a look what we ended up >>> with */ >>> >>> >>> $ dmesg | grep -i direct >>> [ 0.011446] Using GB pages for direct mapping >>> [ 0.703369] pci 0000:00:01.0: Using iommu direct mapping >>> [ 0.703830] pci 0000:00:08.0: Using iommu direct mapping >>> >>> So the graphics device (pci 0000:00:01.0:) is using direct mapping after the >>> change. >>> >>> Kai-Heng >>> >>>> >>>> HTH, >>>> >>>> Joerg _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2020-01-06 8:37 ` Kai-Heng Feng @ 2020-02-05 10:10 ` Kai-Heng Feng 2020-02-05 13:24 ` Joerg Roedel 0 siblings, 1 reply; 15+ messages in thread From: Kai-Heng Feng @ 2020-02-05 10:10 UTC (permalink / raw) To: Joerg Roedel, Deucher, Alexander; +Cc: iommu, Kernel development list Hi Joerg, > On Jan 6, 2020, at 16:37, Kai-Heng Feng <kai.heng.feng@canonical.com> wrote: > > > >> On Dec 20, 2019, at 10:13, Kai-Heng Feng <kai.heng.feng@canonical.com> wrote: >> >> >> >>> On Dec 20, 2019, at 03:15, Deucher, Alexander <Alexander.Deucher@amd.com> wrote: >>> >>>> -----Original Message----- >>>> From: Kai-Heng Feng <kai.heng.feng@canonical.com> >>>> Sent: Wednesday, December 18, 2019 12:45 PM >>>> To: Joerg Roedel <joro@8bytes.org> >>>> Cc: Christoph Hellwig <hch@infradead.org>; Deucher, Alexander >>>> <Alexander.Deucher@amd.com>; iommu@lists.linux-foundation.org; Kernel >>>> development list <linux-kernel@vger.kernel.org> >>>> Subject: Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge >>>> systems >>>> >>>> >>>> >>>>> On Dec 17, 2019, at 17:53, Joerg Roedel <joro@8bytes.org> wrote: >>>>> >>>>> On Fri, Dec 06, 2019 at 01:57:41PM +0800, Kai-Heng Feng wrote: >>>>>> Hi Joerg, >>>>>> >>>>>>> On Dec 3, 2019, at 01:00, Christoph Hellwig <hch@infradead.org> wrote: >>>>>>> >>>>>>> On Fri, Nov 29, 2019 at 10:21:54PM +0800, Kai-Heng Feng wrote: >>>>>>>> Serious screen flickering when Stoney Ridge outputs to a 4K monitor. >>>>>>>> >>>>>>>> According to Alex Deucher, IOMMU isn't enabled on Windows, so let's >>>>>>>> do the same here to avoid screen flickering on 4K monitor. >>>>>>> >>>>>>> Disabling the IOMMU entirely seem pretty severe. Isn't it enough to >>>>>>> identity map the GPU device? >>>>>> >>>>>> Ok, there's set_device_exclusion_range() to exclude the device from >>>> IOMMU. >>>>>> However I don't know how to generate range_start and range_length, >>>> which are read from ACPI. >>>>> >>>>> set_device_exclusion_range() is not the solution here. The best is if >>>>> the GPU device is put into a passthrough domain at boot, in which it >>>>> will be identity mapped. DMA still goes through the IOMMU in this >>>>> case, but it only needs to lookup the device-table, page-table walks >>>>> will not be done anymore. >>>>> >>>>> The best way to implement this is to put it into the >>>>> amd_iommu_add_device() in drivers/iommu/amd_iommu.c. There is this >>>>> check: >>>>> >>>>> if (dev_data->iommu_v2) >>>>> iommu_request_dm_for_dev(dev); >>>>> >>>>> The iommu_request_dm_for_dev() function causes the device to be >>>>> identity mapped. The check can be extended to also check for a device >>>>> white-list for devices that need identity mapping. >>>> >>>> My patch looks like this but the original behavior (4K screen flickering) is still >>>> the same: >>> >>> Does reverting the patch to disable ATS along with this patch help? >> >> Unfortunately it doesn't help. > > Any further suggestion to let me try? Since using identity mapping with ATS doesn't help, Is it possible to merge this patch as is? Kai-Heng > > Kai-Heng > >> >> Kai-Heng >> >>> >>> Alex >>> >>>> >>>> diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c >>>> index bd25674ee4db..f913a25c9e92 100644 >>>> --- a/drivers/iommu/amd_iommu.c >>>> +++ b/drivers/iommu/amd_iommu.c >>>> @@ -42,6 +42,7 @@ >>>> #include <asm/iommu.h> >>>> #include <asm/gart.h> >>>> #include <asm/dma.h> >>>> +#include <asm/pci-direct.h> >>>> >>>> #include "amd_iommu_proto.h" >>>> #include "amd_iommu_types.h" >>>> @@ -2159,6 +2160,8 @@ static int amd_iommu_add_device(struct device >>>> *dev) >>>> struct iommu_domain *domain; >>>> struct amd_iommu *iommu; >>>> int ret, devid; >>>> + bool need_identity_mapping = false; >>>> + u32 header; >>>> >>>> if (!check_device(dev) || get_dev_data(dev)) >>>> return 0; >>>> @@ -2184,7 +2187,11 @@ static int amd_iommu_add_device(struct device >>>> *dev) >>>> >>>> BUG_ON(!dev_data); >>>> >>>> - if (dev_data->iommu_v2) >>>> + header = read_pci_config(0, PCI_BUS_NUM(devid), PCI_SLOT(devid), >>>> PCI_FUNC(devid)); >>>> + if ((header & 0xffff) == 0x1002 && (header >> 16) == 0x98e4) >>>> + need_identity_mapping = true; >>>> + >>>> + if (dev_data->iommu_v2 || need_identity_mapping) >>>> iommu_request_dm_for_dev(dev); >>>> >>>> /* Domains are initialized for this device - have a look what we ended up >>>> with */ >>>> >>>> >>>> $ dmesg | grep -i direct >>>> [ 0.011446] Using GB pages for direct mapping >>>> [ 0.703369] pci 0000:00:01.0: Using iommu direct mapping >>>> [ 0.703830] pci 0000:00:08.0: Using iommu direct mapping >>>> >>>> So the graphics device (pci 0000:00:01.0:) is using direct mapping after the >>>> change. >>>> >>>> Kai-Heng >>>> >>>>> >>>>> HTH, >>>>> >>>>> Joerg > _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems 2020-02-05 10:10 ` Kai-Heng Feng @ 2020-02-05 13:24 ` Joerg Roedel 0 siblings, 0 replies; 15+ messages in thread From: Joerg Roedel @ 2020-02-05 13:24 UTC (permalink / raw) To: Kai-Heng Feng; +Cc: Deucher, Alexander, iommu, Kernel development list On Wed, Feb 05, 2020 at 06:10:11PM +0800, Kai-Heng Feng wrote: > Since using identity mapping with ATS doesn't help, > Is it possible to merge this patch as is? Can you please re-send the patch to me after 5.6-rc1 is out? Thanks, Joerg _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2020-02-05 13:24 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-11-29 14:21 [PATCH v2] iommu/amd: Disable IOMMU on Stoney Ridge systems Kai-Heng Feng 2019-12-01 12:42 ` Lucas Stach 2019-12-02 16:36 ` Deucher, Alexander 2019-12-04 16:08 ` Deucher, Alexander 2019-12-06 5:59 ` Kai-Heng Feng 2019-12-06 8:57 ` Christian König 2019-12-02 17:00 ` Christoph Hellwig 2019-12-06 5:57 ` Kai-Heng Feng 2019-12-17 9:53 ` Joerg Roedel 2019-12-18 17:45 ` Kai-Heng Feng 2019-12-19 19:15 ` Deucher, Alexander 2019-12-20 2:13 ` Kai-Heng Feng 2020-01-06 8:37 ` Kai-Heng Feng 2020-02-05 10:10 ` Kai-Heng Feng 2020-02-05 13:24 ` Joerg Roedel
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).