* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 9:49 ` Miles Chen
0 siblings, 0 replies; 28+ messages in thread
From: Miles Chen @ 2020-06-04 9:49 UTC (permalink / raw)
To: David Hildenbrand
Cc: wsd_upstream, Joerg Roedel, linux-kernel, Chao Hao, iommu,
linux-mediatek, Yong Wu, Matthias Brugger, yingjoe.chen,
linux-arm-kernel
On Thu, 2020-06-04 at 10:25 +0200, David Hildenbrand wrote:
> On 04.06.20 10:01, Miles Chen wrote:
> > To build this driver as a kernel module, we cannot use
> > the unexported symbol "max_pfn" to setup enable_4GB.
> >
> > Use totalram_pages() instead to setup enable_4GB.
> >
> > Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
> > Signed-off-by: Miles Chen <miles.chen@mediatek.com>
> > Cc: David Hildenbrand <david@redhat.com>
> > Cc: Yong Wu <yong.wu@mediatek.com>
> > Cc: Chao Hao <chao.hao@mediatek.com>
> > ---
> > drivers/iommu/mtk_iommu.c | 5 ++---
> > 1 file changed, 2 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c
> > index 5f4d6df59cf6..c2798a6e0e38 100644
> > --- a/drivers/iommu/mtk_iommu.c
> > +++ b/drivers/iommu/mtk_iommu.c
> > @@ -3,7 +3,6 @@
> > * Copyright (c) 2015-2016 MediaTek Inc.
> > * Author: Yong Wu <yong.wu@mediatek.com>
> > */
> > -#include <linux/memblock.h>
> > #include <linux/bug.h>
> > #include <linux/clk.h>
> > #include <linux/component.h>
> > @@ -626,8 +625,8 @@ static int mtk_iommu_probe(struct platform_device *pdev)
> > return -ENOMEM;
> > data->protect_base = ALIGN(virt_to_phys(protect), MTK_PROTECT_PA_ALIGN);
> >
> > - /* Whether the current dram is over 4GB */
> > - data->enable_4GB = !!(max_pfn > (BIT_ULL(32) >> PAGE_SHIFT));
> > + /* Whether the current dram is over 4GB, note: DRAM start at 1GB */
> > + data->enable_4GB = !!(totalram_pages() > ((SZ_2G + SZ_1G) >> PAGE_SHIFT));
>
> A similar thing seems to be done by
> drivers/media/platform/mtk-vpu/mtk_vpu.c:
> vpu->enable_4GB = !!(totalram_pages() > (SZ_2G >> PAGE_SHIFT));
>
> I do wonder if some weird memory hotplug setups might give you false
> negatives.
>
> E.g., start a VM with 1GB and hotplug 1GB - it will be hotplugged on
> x86-64 above 4GB, turning max_pfn into 5GB. totalram_pages() should
> return something < 2GB.
>
> Same can happen when you have a VM and use ballooning to fake-unplug
> memory, making totalram_pages() return something < 4GB, but leaving
> usable pfns >= 4GB
Yes. Yingjoe also told me that this patch is not correct.
Thanks for pointing this out. totalram_pages() does not work
for some cases:
e.g., DRAM start @0x4000_0000 and DRAM size is 0x1_0000_0000 but we
reserve large amount of memory, which makes totalram_pages() < 3GB but
it is possible to allocate a pfn >= 4GB.
I will discuss this internally.
Miles
> .
>
> but
> ... I don't know if I understood what "enable_4GB" needs/implies
> ... I don't know if this is applicable to VMs
> at all (on real HW such
> memory hotplug setups should not exist)
> ... I don't know how this code would react to memory hotplug, so if the
> condition changes after the driver loaded and enable_4GB would
> suddenly apply. Again, most probably not relevant on real HW, only
> for VMs.
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 9:49 ` Miles Chen
0 siblings, 0 replies; 28+ messages in thread
From: Miles Chen @ 2020-06-04 9:49 UTC (permalink / raw)
To: David Hildenbrand
Cc: wsd_upstream, Joerg Roedel, linux-kernel, Chao Hao, iommu,
linux-mediatek, Yong Wu, Matthias Brugger, yingjoe.chen,
linux-arm-kernel
On Thu, 2020-06-04 at 10:25 +0200, David Hildenbrand wrote:
> On 04.06.20 10:01, Miles Chen wrote:
> > To build this driver as a kernel module, we cannot use
> > the unexported symbol "max_pfn" to setup enable_4GB.
> >
> > Use totalram_pages() instead to setup enable_4GB.
> >
> > Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
> > Signed-off-by: Miles Chen <miles.chen@mediatek.com>
> > Cc: David Hildenbrand <david@redhat.com>
> > Cc: Yong Wu <yong.wu@mediatek.com>
> > Cc: Chao Hao <chao.hao@mediatek.com>
> > ---
> > drivers/iommu/mtk_iommu.c | 5 ++---
> > 1 file changed, 2 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c
> > index 5f4d6df59cf6..c2798a6e0e38 100644
> > --- a/drivers/iommu/mtk_iommu.c
> > +++ b/drivers/iommu/mtk_iommu.c
> > @@ -3,7 +3,6 @@
> > * Copyright (c) 2015-2016 MediaTek Inc.
> > * Author: Yong Wu <yong.wu@mediatek.com>
> > */
> > -#include <linux/memblock.h>
> > #include <linux/bug.h>
> > #include <linux/clk.h>
> > #include <linux/component.h>
> > @@ -626,8 +625,8 @@ static int mtk_iommu_probe(struct platform_device *pdev)
> > return -ENOMEM;
> > data->protect_base = ALIGN(virt_to_phys(protect), MTK_PROTECT_PA_ALIGN);
> >
> > - /* Whether the current dram is over 4GB */
> > - data->enable_4GB = !!(max_pfn > (BIT_ULL(32) >> PAGE_SHIFT));
> > + /* Whether the current dram is over 4GB, note: DRAM start at 1GB */
> > + data->enable_4GB = !!(totalram_pages() > ((SZ_2G + SZ_1G) >> PAGE_SHIFT));
>
> A similar thing seems to be done by
> drivers/media/platform/mtk-vpu/mtk_vpu.c:
> vpu->enable_4GB = !!(totalram_pages() > (SZ_2G >> PAGE_SHIFT));
>
> I do wonder if some weird memory hotplug setups might give you false
> negatives.
>
> E.g., start a VM with 1GB and hotplug 1GB - it will be hotplugged on
> x86-64 above 4GB, turning max_pfn into 5GB. totalram_pages() should
> return something < 2GB.
>
> Same can happen when you have a VM and use ballooning to fake-unplug
> memory, making totalram_pages() return something < 4GB, but leaving
> usable pfns >= 4GB
Yes. Yingjoe also told me that this patch is not correct.
Thanks for pointing this out. totalram_pages() does not work
for some cases:
e.g., DRAM start @0x4000_0000 and DRAM size is 0x1_0000_0000 but we
reserve large amount of memory, which makes totalram_pages() < 3GB but
it is possible to allocate a pfn >= 4GB.
I will discuss this internally.
Miles
> .
>
> but
> ... I don't know if I understood what "enable_4GB" needs/implies
> ... I don't know if this is applicable to VMs
> at all (on real HW such
> memory hotplug setups should not exist)
> ... I don't know how this code would react to memory hotplug, so if the
> condition changes after the driver loaded and enable_4GB would
> suddenly apply. Again, most probably not relevant on real HW, only
> for VMs.
>
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 9:49 ` Miles Chen
0 siblings, 0 replies; 28+ messages in thread
From: Miles Chen @ 2020-06-04 9:49 UTC (permalink / raw)
To: David Hildenbrand
Cc: wsd_upstream, linux-kernel, Chao Hao, iommu, linux-mediatek,
Matthias Brugger, yingjoe.chen, linux-arm-kernel
On Thu, 2020-06-04 at 10:25 +0200, David Hildenbrand wrote:
> On 04.06.20 10:01, Miles Chen wrote:
> > To build this driver as a kernel module, we cannot use
> > the unexported symbol "max_pfn" to setup enable_4GB.
> >
> > Use totalram_pages() instead to setup enable_4GB.
> >
> > Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
> > Signed-off-by: Miles Chen <miles.chen@mediatek.com>
> > Cc: David Hildenbrand <david@redhat.com>
> > Cc: Yong Wu <yong.wu@mediatek.com>
> > Cc: Chao Hao <chao.hao@mediatek.com>
> > ---
> > drivers/iommu/mtk_iommu.c | 5 ++---
> > 1 file changed, 2 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c
> > index 5f4d6df59cf6..c2798a6e0e38 100644
> > --- a/drivers/iommu/mtk_iommu.c
> > +++ b/drivers/iommu/mtk_iommu.c
> > @@ -3,7 +3,6 @@
> > * Copyright (c) 2015-2016 MediaTek Inc.
> > * Author: Yong Wu <yong.wu@mediatek.com>
> > */
> > -#include <linux/memblock.h>
> > #include <linux/bug.h>
> > #include <linux/clk.h>
> > #include <linux/component.h>
> > @@ -626,8 +625,8 @@ static int mtk_iommu_probe(struct platform_device *pdev)
> > return -ENOMEM;
> > data->protect_base = ALIGN(virt_to_phys(protect), MTK_PROTECT_PA_ALIGN);
> >
> > - /* Whether the current dram is over 4GB */
> > - data->enable_4GB = !!(max_pfn > (BIT_ULL(32) >> PAGE_SHIFT));
> > + /* Whether the current dram is over 4GB, note: DRAM start at 1GB */
> > + data->enable_4GB = !!(totalram_pages() > ((SZ_2G + SZ_1G) >> PAGE_SHIFT));
>
> A similar thing seems to be done by
> drivers/media/platform/mtk-vpu/mtk_vpu.c:
> vpu->enable_4GB = !!(totalram_pages() > (SZ_2G >> PAGE_SHIFT));
>
> I do wonder if some weird memory hotplug setups might give you false
> negatives.
>
> E.g., start a VM with 1GB and hotplug 1GB - it will be hotplugged on
> x86-64 above 4GB, turning max_pfn into 5GB. totalram_pages() should
> return something < 2GB.
>
> Same can happen when you have a VM and use ballooning to fake-unplug
> memory, making totalram_pages() return something < 4GB, but leaving
> usable pfns >= 4GB
Yes. Yingjoe also told me that this patch is not correct.
Thanks for pointing this out. totalram_pages() does not work
for some cases:
e.g., DRAM start @0x4000_0000 and DRAM size is 0x1_0000_0000 but we
reserve large amount of memory, which makes totalram_pages() < 3GB but
it is possible to allocate a pfn >= 4GB.
I will discuss this internally.
Miles
> .
>
> but
> ... I don't know if I understood what "enable_4GB" needs/implies
> ... I don't know if this is applicable to VMs
> at all (on real HW such
> memory hotplug setups should not exist)
> ... I don't know how this code would react to memory hotplug, so if the
> condition changes after the driver loaded and enable_4GB would
> suddenly apply. Again, most probably not relevant on real HW, only
> for VMs.
>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
2020-06-04 9:49 ` Miles Chen
(?)
(?)
@ 2020-06-04 11:32 ` David Hildenbrand
-1 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-04 11:32 UTC (permalink / raw)
To: Miles Chen
Cc: Joerg Roedel, Matthias Brugger, iommu, linux-arm-kernel,
linux-mediatek, linux-kernel, wsd_upstream, Yong Wu, Chao Hao,
yingjoe.chen
On 04.06.20 11:49, Miles Chen wrote:
> On Thu, 2020-06-04 at 10:25 +0200, David Hildenbrand wrote:
>> On 04.06.20 10:01, Miles Chen wrote:
>>> To build this driver as a kernel module, we cannot use
>>> the unexported symbol "max_pfn" to setup enable_4GB.
>>>
>>> Use totalram_pages() instead to setup enable_4GB.
>>>
>>> Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
>>> Signed-off-by: Miles Chen <miles.chen@mediatek.com>
>>> Cc: David Hildenbrand <david@redhat.com>
>>> Cc: Yong Wu <yong.wu@mediatek.com>
>>> Cc: Chao Hao <chao.hao@mediatek.com>
>>> ---
>>> drivers/iommu/mtk_iommu.c | 5 ++---
>>> 1 file changed, 2 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c
>>> index 5f4d6df59cf6..c2798a6e0e38 100644
>>> --- a/drivers/iommu/mtk_iommu.c
>>> +++ b/drivers/iommu/mtk_iommu.c
>>> @@ -3,7 +3,6 @@
>>> * Copyright (c) 2015-2016 MediaTek Inc.
>>> * Author: Yong Wu <yong.wu@mediatek.com>
>>> */
>>> -#include <linux/memblock.h>
>>> #include <linux/bug.h>
>>> #include <linux/clk.h>
>>> #include <linux/component.h>
>>> @@ -626,8 +625,8 @@ static int mtk_iommu_probe(struct platform_device *pdev)
>>> return -ENOMEM;
>>> data->protect_base = ALIGN(virt_to_phys(protect), MTK_PROTECT_PA_ALIGN);
>>>
>>> - /* Whether the current dram is over 4GB */
>>> - data->enable_4GB = !!(max_pfn > (BIT_ULL(32) >> PAGE_SHIFT));
>>> + /* Whether the current dram is over 4GB, note: DRAM start at 1GB */
>>> + data->enable_4GB = !!(totalram_pages() > ((SZ_2G + SZ_1G) >> PAGE_SHIFT));
>>
>> A similar thing seems to be done by
>> drivers/media/platform/mtk-vpu/mtk_vpu.c:
>> vpu->enable_4GB = !!(totalram_pages() > (SZ_2G >> PAGE_SHIFT));
>>
>> I do wonder if some weird memory hotplug setups might give you false
>> negatives.
>>
>> E.g., start a VM with 1GB and hotplug 1GB - it will be hotplugged on
>> x86-64 above 4GB, turning max_pfn into 5GB. totalram_pages() should
>> return something < 2GB.
>>
>> Same can happen when you have a VM and use ballooning to fake-unplug
>> memory, making totalram_pages() return something < 4GB, but leaving
>> usable pfns >= 4GB
>
> Yes. Yingjoe also told me that this patch is not correct.
>
> Thanks for pointing this out. totalram_pages() does not work
> for some cases:
>
Just a thought: If memory hotplug is applicable as well, you might
either want to always assume data->enable_4GB, or handle memory hotplug
events from the memory notifier, when new memory gets onlined (not sure
how tricky that is).
--
Thanks,
David / dhildenb
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 11:32 ` David Hildenbrand
0 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-04 11:32 UTC (permalink / raw)
To: Miles Chen
Cc: wsd_upstream, Joerg Roedel, linux-kernel, Chao Hao, iommu,
linux-mediatek, Yong Wu, Matthias Brugger, yingjoe.chen,
linux-arm-kernel
On 04.06.20 11:49, Miles Chen wrote:
> On Thu, 2020-06-04 at 10:25 +0200, David Hildenbrand wrote:
>> On 04.06.20 10:01, Miles Chen wrote:
>>> To build this driver as a kernel module, we cannot use
>>> the unexported symbol "max_pfn" to setup enable_4GB.
>>>
>>> Use totalram_pages() instead to setup enable_4GB.
>>>
>>> Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
>>> Signed-off-by: Miles Chen <miles.chen@mediatek.com>
>>> Cc: David Hildenbrand <david@redhat.com>
>>> Cc: Yong Wu <yong.wu@mediatek.com>
>>> Cc: Chao Hao <chao.hao@mediatek.com>
>>> ---
>>> drivers/iommu/mtk_iommu.c | 5 ++---
>>> 1 file changed, 2 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c
>>> index 5f4d6df59cf6..c2798a6e0e38 100644
>>> --- a/drivers/iommu/mtk_iommu.c
>>> +++ b/drivers/iommu/mtk_iommu.c
>>> @@ -3,7 +3,6 @@
>>> * Copyright (c) 2015-2016 MediaTek Inc.
>>> * Author: Yong Wu <yong.wu@mediatek.com>
>>> */
>>> -#include <linux/memblock.h>
>>> #include <linux/bug.h>
>>> #include <linux/clk.h>
>>> #include <linux/component.h>
>>> @@ -626,8 +625,8 @@ static int mtk_iommu_probe(struct platform_device *pdev)
>>> return -ENOMEM;
>>> data->protect_base = ALIGN(virt_to_phys(protect), MTK_PROTECT_PA_ALIGN);
>>>
>>> - /* Whether the current dram is over 4GB */
>>> - data->enable_4GB = !!(max_pfn > (BIT_ULL(32) >> PAGE_SHIFT));
>>> + /* Whether the current dram is over 4GB, note: DRAM start at 1GB */
>>> + data->enable_4GB = !!(totalram_pages() > ((SZ_2G + SZ_1G) >> PAGE_SHIFT));
>>
>> A similar thing seems to be done by
>> drivers/media/platform/mtk-vpu/mtk_vpu.c:
>> vpu->enable_4GB = !!(totalram_pages() > (SZ_2G >> PAGE_SHIFT));
>>
>> I do wonder if some weird memory hotplug setups might give you false
>> negatives.
>>
>> E.g., start a VM with 1GB and hotplug 1GB - it will be hotplugged on
>> x86-64 above 4GB, turning max_pfn into 5GB. totalram_pages() should
>> return something < 2GB.
>>
>> Same can happen when you have a VM and use ballooning to fake-unplug
>> memory, making totalram_pages() return something < 4GB, but leaving
>> usable pfns >= 4GB
>
> Yes. Yingjoe also told me that this patch is not correct.
>
> Thanks for pointing this out. totalram_pages() does not work
> for some cases:
>
Just a thought: If memory hotplug is applicable as well, you might
either want to always assume data->enable_4GB, or handle memory hotplug
events from the memory notifier, when new memory gets onlined (not sure
how tricky that is).
--
Thanks,
David / dhildenb
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 11:32 ` David Hildenbrand
0 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-04 11:32 UTC (permalink / raw)
To: Miles Chen
Cc: wsd_upstream, Joerg Roedel, linux-kernel, Chao Hao, iommu,
linux-mediatek, Yong Wu, Matthias Brugger, yingjoe.chen,
linux-arm-kernel
On 04.06.20 11:49, Miles Chen wrote:
> On Thu, 2020-06-04 at 10:25 +0200, David Hildenbrand wrote:
>> On 04.06.20 10:01, Miles Chen wrote:
>>> To build this driver as a kernel module, we cannot use
>>> the unexported symbol "max_pfn" to setup enable_4GB.
>>>
>>> Use totalram_pages() instead to setup enable_4GB.
>>>
>>> Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
>>> Signed-off-by: Miles Chen <miles.chen@mediatek.com>
>>> Cc: David Hildenbrand <david@redhat.com>
>>> Cc: Yong Wu <yong.wu@mediatek.com>
>>> Cc: Chao Hao <chao.hao@mediatek.com>
>>> ---
>>> drivers/iommu/mtk_iommu.c | 5 ++---
>>> 1 file changed, 2 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c
>>> index 5f4d6df59cf6..c2798a6e0e38 100644
>>> --- a/drivers/iommu/mtk_iommu.c
>>> +++ b/drivers/iommu/mtk_iommu.c
>>> @@ -3,7 +3,6 @@
>>> * Copyright (c) 2015-2016 MediaTek Inc.
>>> * Author: Yong Wu <yong.wu@mediatek.com>
>>> */
>>> -#include <linux/memblock.h>
>>> #include <linux/bug.h>
>>> #include <linux/clk.h>
>>> #include <linux/component.h>
>>> @@ -626,8 +625,8 @@ static int mtk_iommu_probe(struct platform_device *pdev)
>>> return -ENOMEM;
>>> data->protect_base = ALIGN(virt_to_phys(protect), MTK_PROTECT_PA_ALIGN);
>>>
>>> - /* Whether the current dram is over 4GB */
>>> - data->enable_4GB = !!(max_pfn > (BIT_ULL(32) >> PAGE_SHIFT));
>>> + /* Whether the current dram is over 4GB, note: DRAM start at 1GB */
>>> + data->enable_4GB = !!(totalram_pages() > ((SZ_2G + SZ_1G) >> PAGE_SHIFT));
>>
>> A similar thing seems to be done by
>> drivers/media/platform/mtk-vpu/mtk_vpu.c:
>> vpu->enable_4GB = !!(totalram_pages() > (SZ_2G >> PAGE_SHIFT));
>>
>> I do wonder if some weird memory hotplug setups might give you false
>> negatives.
>>
>> E.g., start a VM with 1GB and hotplug 1GB - it will be hotplugged on
>> x86-64 above 4GB, turning max_pfn into 5GB. totalram_pages() should
>> return something < 2GB.
>>
>> Same can happen when you have a VM and use ballooning to fake-unplug
>> memory, making totalram_pages() return something < 4GB, but leaving
>> usable pfns >= 4GB
>
> Yes. Yingjoe also told me that this patch is not correct.
>
> Thanks for pointing this out. totalram_pages() does not work
> for some cases:
>
Just a thought: If memory hotplug is applicable as well, you might
either want to always assume data->enable_4GB, or handle memory hotplug
events from the memory notifier, when new memory gets onlined (not sure
how tricky that is).
--
Thanks,
David / dhildenb
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 11:32 ` David Hildenbrand
0 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-04 11:32 UTC (permalink / raw)
To: Miles Chen
Cc: wsd_upstream, linux-kernel, Chao Hao, iommu, linux-mediatek,
Matthias Brugger, yingjoe.chen, linux-arm-kernel
On 04.06.20 11:49, Miles Chen wrote:
> On Thu, 2020-06-04 at 10:25 +0200, David Hildenbrand wrote:
>> On 04.06.20 10:01, Miles Chen wrote:
>>> To build this driver as a kernel module, we cannot use
>>> the unexported symbol "max_pfn" to setup enable_4GB.
>>>
>>> Use totalram_pages() instead to setup enable_4GB.
>>>
>>> Suggested-by: Mike Rapoport <rppt@linux.ibm.com>
>>> Signed-off-by: Miles Chen <miles.chen@mediatek.com>
>>> Cc: David Hildenbrand <david@redhat.com>
>>> Cc: Yong Wu <yong.wu@mediatek.com>
>>> Cc: Chao Hao <chao.hao@mediatek.com>
>>> ---
>>> drivers/iommu/mtk_iommu.c | 5 ++---
>>> 1 file changed, 2 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/drivers/iommu/mtk_iommu.c b/drivers/iommu/mtk_iommu.c
>>> index 5f4d6df59cf6..c2798a6e0e38 100644
>>> --- a/drivers/iommu/mtk_iommu.c
>>> +++ b/drivers/iommu/mtk_iommu.c
>>> @@ -3,7 +3,6 @@
>>> * Copyright (c) 2015-2016 MediaTek Inc.
>>> * Author: Yong Wu <yong.wu@mediatek.com>
>>> */
>>> -#include <linux/memblock.h>
>>> #include <linux/bug.h>
>>> #include <linux/clk.h>
>>> #include <linux/component.h>
>>> @@ -626,8 +625,8 @@ static int mtk_iommu_probe(struct platform_device *pdev)
>>> return -ENOMEM;
>>> data->protect_base = ALIGN(virt_to_phys(protect), MTK_PROTECT_PA_ALIGN);
>>>
>>> - /* Whether the current dram is over 4GB */
>>> - data->enable_4GB = !!(max_pfn > (BIT_ULL(32) >> PAGE_SHIFT));
>>> + /* Whether the current dram is over 4GB, note: DRAM start at 1GB */
>>> + data->enable_4GB = !!(totalram_pages() > ((SZ_2G + SZ_1G) >> PAGE_SHIFT));
>>
>> A similar thing seems to be done by
>> drivers/media/platform/mtk-vpu/mtk_vpu.c:
>> vpu->enable_4GB = !!(totalram_pages() > (SZ_2G >> PAGE_SHIFT));
>>
>> I do wonder if some weird memory hotplug setups might give you false
>> negatives.
>>
>> E.g., start a VM with 1GB and hotplug 1GB - it will be hotplugged on
>> x86-64 above 4GB, turning max_pfn into 5GB. totalram_pages() should
>> return something < 2GB.
>>
>> Same can happen when you have a VM and use ballooning to fake-unplug
>> memory, making totalram_pages() return something < 4GB, but leaving
>> usable pfns >= 4GB
>
> Yes. Yingjoe also told me that this patch is not correct.
>
> Thanks for pointing this out. totalram_pages() does not work
> for some cases:
>
Just a thought: If memory hotplug is applicable as well, you might
either want to always assume data->enable_4GB, or handle memory hotplug
events from the memory notifier, when new memory gets onlined (not sure
how tricky that is).
--
Thanks,
David / dhildenb
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
2020-06-04 11:32 ` David Hildenbrand
(?)
(?)
@ 2020-06-04 15:06 ` Christoph Hellwig
-1 siblings, 0 replies; 28+ messages in thread
From: Christoph Hellwig @ 2020-06-04 15:06 UTC (permalink / raw)
To: David Hildenbrand
Cc: Miles Chen, wsd_upstream, Joerg Roedel, linux-kernel, Chao Hao,
iommu, linux-mediatek, Yong Wu, Matthias Brugger, yingjoe.chen,
linux-arm-kernel
On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
> Just a thought: If memory hotplug is applicable as well, you might
> either want to always assume data->enable_4GB, or handle memory hotplug
> events from the memory notifier, when new memory gets onlined (not sure
> how tricky that is).
We probably want a highest_pfn_possible() or similar API instead of
having drivers poking into random VM internals.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 15:06 ` Christoph Hellwig
0 siblings, 0 replies; 28+ messages in thread
From: Christoph Hellwig @ 2020-06-04 15:06 UTC (permalink / raw)
To: David Hildenbrand
Cc: wsd_upstream, Joerg Roedel, iommu, linux-kernel, Chao Hao,
Miles Chen, linux-mediatek, linux-arm-kernel, Matthias Brugger,
yingjoe.chen, Yong Wu
On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
> Just a thought: If memory hotplug is applicable as well, you might
> either want to always assume data->enable_4GB, or handle memory hotplug
> events from the memory notifier, when new memory gets onlined (not sure
> how tricky that is).
We probably want a highest_pfn_possible() or similar API instead of
having drivers poking into random VM internals.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 15:06 ` Christoph Hellwig
0 siblings, 0 replies; 28+ messages in thread
From: Christoph Hellwig @ 2020-06-04 15:06 UTC (permalink / raw)
To: David Hildenbrand
Cc: wsd_upstream, Joerg Roedel, iommu, linux-kernel, Chao Hao,
Miles Chen, linux-mediatek, linux-arm-kernel, Matthias Brugger,
yingjoe.chen, Yong Wu
On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
> Just a thought: If memory hotplug is applicable as well, you might
> either want to always assume data->enable_4GB, or handle memory hotplug
> events from the memory notifier, when new memory gets onlined (not sure
> how tricky that is).
We probably want a highest_pfn_possible() or similar API instead of
having drivers poking into random VM internals.
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 15:06 ` Christoph Hellwig
0 siblings, 0 replies; 28+ messages in thread
From: Christoph Hellwig @ 2020-06-04 15:06 UTC (permalink / raw)
To: David Hildenbrand
Cc: wsd_upstream, iommu, linux-kernel, Chao Hao, Miles Chen,
linux-mediatek, linux-arm-kernel, Matthias Brugger, yingjoe.chen
On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
> Just a thought: If memory hotplug is applicable as well, you might
> either want to always assume data->enable_4GB, or handle memory hotplug
> events from the memory notifier, when new memory gets onlined (not sure
> how tricky that is).
We probably want a highest_pfn_possible() or similar API instead of
having drivers poking into random VM internals.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
2020-06-04 15:06 ` Christoph Hellwig
(?)
(?)
@ 2020-06-04 15:27 ` David Hildenbrand
-1 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-04 15:27 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Miles Chen, wsd_upstream, Joerg Roedel, linux-kernel, Chao Hao,
iommu, linux-mediatek, Yong Wu, Matthias Brugger, yingjoe.chen,
linux-arm-kernel
On 04.06.20 17:06, Christoph Hellwig wrote:
> On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
>> Just a thought: If memory hotplug is applicable as well, you might
>> either want to always assume data->enable_4GB, or handle memory hotplug
>> events from the memory notifier, when new memory gets onlined (not sure
>> how tricky that is).
>
> We probably want a highest_pfn_possible() or similar API instead of
> having drivers poking into random VM internals.
Well, memory notifiers are a reasonable api used accross the kernel to
get notified when new memory is onlined to the buddy that could be used
for allocations.
highest_pfn_possible() would have to default to something linked to
MAX_PHYSMEM_BITS whenever memory hotplug is configured, I am not sure
how helpful that is (IOW, you can just default to enable_4GB=true in
that case instead in most cases).
--
Thanks,
David / dhildenb
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 15:27 ` David Hildenbrand
0 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-04 15:27 UTC (permalink / raw)
To: Christoph Hellwig
Cc: wsd_upstream, Joerg Roedel, iommu, linux-kernel, Chao Hao,
Miles Chen, linux-mediatek, linux-arm-kernel, Matthias Brugger,
yingjoe.chen, Yong Wu
On 04.06.20 17:06, Christoph Hellwig wrote:
> On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
>> Just a thought: If memory hotplug is applicable as well, you might
>> either want to always assume data->enable_4GB, or handle memory hotplug
>> events from the memory notifier, when new memory gets onlined (not sure
>> how tricky that is).
>
> We probably want a highest_pfn_possible() or similar API instead of
> having drivers poking into random VM internals.
Well, memory notifiers are a reasonable api used accross the kernel to
get notified when new memory is onlined to the buddy that could be used
for allocations.
highest_pfn_possible() would have to default to something linked to
MAX_PHYSMEM_BITS whenever memory hotplug is configured, I am not sure
how helpful that is (IOW, you can just default to enable_4GB=true in
that case instead in most cases).
--
Thanks,
David / dhildenb
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 15:27 ` David Hildenbrand
0 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-04 15:27 UTC (permalink / raw)
To: Christoph Hellwig
Cc: wsd_upstream, Joerg Roedel, iommu, linux-kernel, Chao Hao,
Miles Chen, linux-mediatek, linux-arm-kernel, Matthias Brugger,
yingjoe.chen, Yong Wu
On 04.06.20 17:06, Christoph Hellwig wrote:
> On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
>> Just a thought: If memory hotplug is applicable as well, you might
>> either want to always assume data->enable_4GB, or handle memory hotplug
>> events from the memory notifier, when new memory gets onlined (not sure
>> how tricky that is).
>
> We probably want a highest_pfn_possible() or similar API instead of
> having drivers poking into random VM internals.
Well, memory notifiers are a reasonable api used accross the kernel to
get notified when new memory is onlined to the buddy that could be used
for allocations.
highest_pfn_possible() would have to default to something linked to
MAX_PHYSMEM_BITS whenever memory hotplug is configured, I am not sure
how helpful that is (IOW, you can just default to enable_4GB=true in
that case instead in most cases).
--
Thanks,
David / dhildenb
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-04 15:27 ` David Hildenbrand
0 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-04 15:27 UTC (permalink / raw)
To: Christoph Hellwig
Cc: wsd_upstream, iommu, linux-kernel, Chao Hao, Miles Chen,
linux-mediatek, linux-arm-kernel, Matthias Brugger, yingjoe.chen
On 04.06.20 17:06, Christoph Hellwig wrote:
> On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
>> Just a thought: If memory hotplug is applicable as well, you might
>> either want to always assume data->enable_4GB, or handle memory hotplug
>> events from the memory notifier, when new memory gets onlined (not sure
>> how tricky that is).
>
> We probably want a highest_pfn_possible() or similar API instead of
> having drivers poking into random VM internals.
Well, memory notifiers are a reasonable api used accross the kernel to
get notified when new memory is onlined to the buddy that could be used
for allocations.
highest_pfn_possible() would have to default to something linked to
MAX_PHYSMEM_BITS whenever memory hotplug is configured, I am not sure
how helpful that is (IOW, you can just default to enable_4GB=true in
that case instead in most cases).
--
Thanks,
David / dhildenb
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
2020-06-04 15:27 ` David Hildenbrand
(?)
(?)
@ 2020-06-05 12:52 ` David Hildenbrand
-1 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-05 12:52 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Miles Chen, wsd_upstream, Joerg Roedel, linux-kernel, Chao Hao,
iommu, linux-mediatek, Yong Wu, Matthias Brugger, yingjoe.chen,
linux-arm-kernel
On 04.06.20 17:27, David Hildenbrand wrote:
> On 04.06.20 17:06, Christoph Hellwig wrote:
>> On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
>>> Just a thought: If memory hotplug is applicable as well, you might
>>> either want to always assume data->enable_4GB, or handle memory hotplug
>>> events from the memory notifier, when new memory gets onlined (not sure
>>> how tricky that is).
>>
>> We probably want a highest_pfn_possible() or similar API instead of
>> having drivers poking into random VM internals.
>
> Well, memory notifiers are a reasonable api used accross the kernel to
> get notified when new memory is onlined to the buddy that could be used
> for allocations.
>
> highest_pfn_possible() would have to default to something linked to
> MAX_PHYSMEM_BITS whenever memory hotplug is configured, I am not sure
> how helpful that is (IOW, you can just default to enable_4GB=true in
> that case instead in most cases).
Correction: At least on x86-64 we have max_possible_pfn, which will
consult the ACPI SRAT table to figure out the maximum possible PFN.
(Without SRAT, max_possible_pfn will point at the end of initial boot
memory and not consider hotplug memory - something that e.g., newer QEMU
versions work around by creating SRAT tables if memory hotplug might be
possible, even if there is no actual NUMA configuration).
pci-swiotlb.c similarly relies on that to figure out if there are any
!DMA addresses to handle.
--
Thanks,
David / dhildenb
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-05 12:52 ` David Hildenbrand
0 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-05 12:52 UTC (permalink / raw)
To: Christoph Hellwig
Cc: wsd_upstream, Joerg Roedel, iommu, linux-kernel, Chao Hao,
Miles Chen, linux-mediatek, linux-arm-kernel, Matthias Brugger,
yingjoe.chen, Yong Wu
On 04.06.20 17:27, David Hildenbrand wrote:
> On 04.06.20 17:06, Christoph Hellwig wrote:
>> On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
>>> Just a thought: If memory hotplug is applicable as well, you might
>>> either want to always assume data->enable_4GB, or handle memory hotplug
>>> events from the memory notifier, when new memory gets onlined (not sure
>>> how tricky that is).
>>
>> We probably want a highest_pfn_possible() or similar API instead of
>> having drivers poking into random VM internals.
>
> Well, memory notifiers are a reasonable api used accross the kernel to
> get notified when new memory is onlined to the buddy that could be used
> for allocations.
>
> highest_pfn_possible() would have to default to something linked to
> MAX_PHYSMEM_BITS whenever memory hotplug is configured, I am not sure
> how helpful that is (IOW, you can just default to enable_4GB=true in
> that case instead in most cases).
Correction: At least on x86-64 we have max_possible_pfn, which will
consult the ACPI SRAT table to figure out the maximum possible PFN.
(Without SRAT, max_possible_pfn will point at the end of initial boot
memory and not consider hotplug memory - something that e.g., newer QEMU
versions work around by creating SRAT tables if memory hotplug might be
possible, even if there is no actual NUMA configuration).
pci-swiotlb.c similarly relies on that to figure out if there are any
!DMA addresses to handle.
--
Thanks,
David / dhildenb
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-05 12:52 ` David Hildenbrand
0 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-05 12:52 UTC (permalink / raw)
To: Christoph Hellwig
Cc: wsd_upstream, Joerg Roedel, iommu, linux-kernel, Chao Hao,
Miles Chen, linux-mediatek, linux-arm-kernel, Matthias Brugger,
yingjoe.chen, Yong Wu
On 04.06.20 17:27, David Hildenbrand wrote:
> On 04.06.20 17:06, Christoph Hellwig wrote:
>> On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
>>> Just a thought: If memory hotplug is applicable as well, you might
>>> either want to always assume data->enable_4GB, or handle memory hotplug
>>> events from the memory notifier, when new memory gets onlined (not sure
>>> how tricky that is).
>>
>> We probably want a highest_pfn_possible() or similar API instead of
>> having drivers poking into random VM internals.
>
> Well, memory notifiers are a reasonable api used accross the kernel to
> get notified when new memory is onlined to the buddy that could be used
> for allocations.
>
> highest_pfn_possible() would have to default to something linked to
> MAX_PHYSMEM_BITS whenever memory hotplug is configured, I am not sure
> how helpful that is (IOW, you can just default to enable_4GB=true in
> that case instead in most cases).
Correction: At least on x86-64 we have max_possible_pfn, which will
consult the ACPI SRAT table to figure out the maximum possible PFN.
(Without SRAT, max_possible_pfn will point at the end of initial boot
memory and not consider hotplug memory - something that e.g., newer QEMU
versions work around by creating SRAT tables if memory hotplug might be
possible, even if there is no actual NUMA configuration).
pci-swiotlb.c similarly relies on that to figure out if there are any
!DMA addresses to handle.
--
Thanks,
David / dhildenb
_______________________________________________
Linux-mediatek mailing list
Linux-mediatek@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-mediatek
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: [PATCH] iommu/mediatek: Use totalram_pages to setup enable_4GB
@ 2020-06-05 12:52 ` David Hildenbrand
0 siblings, 0 replies; 28+ messages in thread
From: David Hildenbrand @ 2020-06-05 12:52 UTC (permalink / raw)
To: Christoph Hellwig
Cc: wsd_upstream, iommu, linux-kernel, Chao Hao, Miles Chen,
linux-mediatek, linux-arm-kernel, Matthias Brugger, yingjoe.chen
On 04.06.20 17:27, David Hildenbrand wrote:
> On 04.06.20 17:06, Christoph Hellwig wrote:
>> On Thu, Jun 04, 2020 at 01:32:40PM +0200, David Hildenbrand wrote:
>>> Just a thought: If memory hotplug is applicable as well, you might
>>> either want to always assume data->enable_4GB, or handle memory hotplug
>>> events from the memory notifier, when new memory gets onlined (not sure
>>> how tricky that is).
>>
>> We probably want a highest_pfn_possible() or similar API instead of
>> having drivers poking into random VM internals.
>
> Well, memory notifiers are a reasonable api used accross the kernel to
> get notified when new memory is onlined to the buddy that could be used
> for allocations.
>
> highest_pfn_possible() would have to default to something linked to
> MAX_PHYSMEM_BITS whenever memory hotplug is configured, I am not sure
> how helpful that is (IOW, you can just default to enable_4GB=true in
> that case instead in most cases).
Correction: At least on x86-64 we have max_possible_pfn, which will
consult the ACPI SRAT table to figure out the maximum possible PFN.
(Without SRAT, max_possible_pfn will point at the end of initial boot
memory and not consider hotplug memory - something that e.g., newer QEMU
versions work around by creating SRAT tables if memory hotplug might be
possible, even if there is no actual NUMA configuration).
pci-swiotlb.c similarly relies on that to figure out if there are any
!DMA addresses to handle.
--
Thanks,
David / dhildenb
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 28+ messages in thread