* [PATCH v2 1/5] mm: page_alloc: remain memblock_next_valid_pfn() when CONFIG_HAVE_ARCH_PFN_VALID is enable
2018-03-24 12:24 [PATCH v2 0/5] optimize memblock_next_valid_pfn() and early_pfn_valid() Jia He
@ 2018-03-24 12:24 ` Jia He
2018-03-27 16:52 ` Daniel Vacek
2018-03-28 9:13 ` Wei Yang
2018-03-24 12:24 ` [PATCH v2 2/5] mm: page_alloc: reduce unnecessary binary search in memblock_next_valid_pfn() Jia He
` (3 subsequent siblings)
4 siblings, 2 replies; 16+ messages in thread
From: Jia He @ 2018-03-24 12:24 UTC (permalink / raw)
To: Andrew Morton, Michal Hocko, Catalin Marinas, Mel Gorman,
Will Deacon, Mark Rutland, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin
Cc: Pavel Tatashin, Daniel Jordan, AKASHI Takahiro, Gioh Kim,
Steven Sistare, Daniel Vacek, Eugeniu Rosca, Vlastimil Babka,
linux-kernel, linux-mm, James Morse, Ard Biesheuvel,
Steve Capper, x86, Greg Kroah-Hartman, Kate Stewart,
Philippe Ombredanne, Johannes Weiner, Kemi Wang, Petr Tesarik,
YASUAKI ISHIMATSU, Andrey Ryabinin, Nikolay Borisov, Jia He,
Jia He
Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
where possible") optimized the loop in memmap_init_zone(). But it causes
possible panic bug. So Daniel Vacek reverted it later.
But memblock_next_valid_pfn is valid when CONFIG_HAVE_ARCH_PFN_VALID is
enabled. And as verified by Eugeniu Rosca, arm can benifit from this
commit. So remain the memblock_next_valid_pfn.
Signed-off-by: Jia He <jia.he@hxt-semitech.com>
---
include/linux/memblock.h | 4 ++++
mm/memblock.c | 29 +++++++++++++++++++++++++++++
mm/page_alloc.c | 11 ++++++++++-
3 files changed, 43 insertions(+), 1 deletion(-)
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index 0257aee..efbbe4b 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -203,6 +203,10 @@ void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,
i >= 0; __next_mem_pfn_range(&i, nid, p_start, p_end, p_nid))
#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
+#ifdef CONFIG_HAVE_ARCH_PFN_VALID
+unsigned long memblock_next_valid_pfn(unsigned long pfn);
+#endif
+
/**
* for_each_free_mem_range - iterate through free memblock areas
* @i: u64 used as loop variable
diff --git a/mm/memblock.c b/mm/memblock.c
index ba7c878..bea5a9c 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -1102,6 +1102,35 @@ void __init_memblock __next_mem_pfn_range(int *idx, int nid,
*out_nid = r->nid;
}
+#ifdef CONFIG_HAVE_ARCH_PFN_VALID
+unsigned long __init_memblock memblock_next_valid_pfn(unsigned long pfn)
+{
+ struct memblock_type *type = &memblock.memory;
+ unsigned int right = type->cnt;
+ unsigned int mid, left = 0;
+ phys_addr_t addr = PFN_PHYS(++pfn);
+
+ do {
+ mid = (right + left) / 2;
+
+ if (addr < type->regions[mid].base)
+ right = mid;
+ else if (addr >= (type->regions[mid].base +
+ type->regions[mid].size))
+ left = mid + 1;
+ else {
+ /* addr is within the region, so pfn is valid */
+ return pfn;
+ }
+ } while (left < right);
+
+ if (right == type->cnt)
+ return -1UL;
+ else
+ return PHYS_PFN(type->regions[right].base);
+}
+#endif /*CONFIG_HAVE_ARCH_PFN_VALID*/
+
/**
* memblock_set_node - set node ID on memblock regions
* @base: base of area to set node ID for
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index c19f5ac..2a967f7 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5483,8 +5483,17 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
if (context != MEMMAP_EARLY)
goto not_early;
- if (!early_pfn_valid(pfn))
+ if (!early_pfn_valid(pfn)) {
+#if (defined CONFIG_HAVE_MEMBLOCK) && (defined CONFIG_HAVE_ARCH_PFN_VALID)
+ /*
+ * Skip to the pfn preceding the next valid one (or
+ * end_pfn), such that we hit a valid pfn (or end_pfn)
+ * on our next iteration of the loop.
+ */
+ pfn = memblock_next_valid_pfn(pfn) - 1;
+#endif
continue;
+ }
if (!early_pfn_in_nid(pfn, nid))
continue;
if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised))
--
2.7.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH v2 1/5] mm: page_alloc: remain memblock_next_valid_pfn() when CONFIG_HAVE_ARCH_PFN_VALID is enable
2018-03-24 12:24 ` [PATCH v2 1/5] mm: page_alloc: remain memblock_next_valid_pfn() when CONFIG_HAVE_ARCH_PFN_VALID is enable Jia He
@ 2018-03-27 16:52 ` Daniel Vacek
2018-03-28 1:49 ` Jia He
2018-03-28 9:26 ` Jia He
2018-03-28 9:13 ` Wei Yang
1 sibling, 2 replies; 16+ messages in thread
From: Daniel Vacek @ 2018-03-27 16:52 UTC (permalink / raw)
To: Jia He
Cc: Andrew Morton, Michal Hocko, Catalin Marinas, Mel Gorman,
Will Deacon, Mark Rutland, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin, Pavel Tatashin, Daniel Jordan, AKASHI Takahiro,
Gioh Kim, Steven Sistare, Eugeniu Rosca, Vlastimil Babka,
open list, linux-mm, James Morse, Ard Biesheuvel, Steve Capper,
x86, Greg Kroah-Hartman, Kate Stewart, Philippe Ombredanne,
Johannes Weiner, Kemi Wang, Petr Tesarik, YASUAKI ISHIMATSU,
Andrey Ryabinin, Nikolay Borisov, Jia He
On Sat, Mar 24, 2018 at 1:24 PM, Jia He <hejianet@gmail.com> wrote:
> Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
> where possible") optimized the loop in memmap_init_zone(). But it causes
> possible panic bug. So Daniel Vacek reverted it later.
>
> But memblock_next_valid_pfn is valid when CONFIG_HAVE_ARCH_PFN_VALID is
> enabled. And as verified by Eugeniu Rosca, arm can benifit from this
> commit. So remain the memblock_next_valid_pfn.
It is not dependent on CONFIG_HAVE_ARCH_PFN_VALID option but on
arm(64) implementation of pfn_valid() function, IIUC. So it should
really be moved from generic source file to arm specific location. I'd
say somewhere close to the pfn_valid() implementation. Such as to
arch/arm{,64}/mm/ init.c-ish?
--nX
> Signed-off-by: Jia He <jia.he@hxt-semitech.com>
> ---
> include/linux/memblock.h | 4 ++++
> mm/memblock.c | 29 +++++++++++++++++++++++++++++
> mm/page_alloc.c | 11 ++++++++++-
> 3 files changed, 43 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/memblock.h b/include/linux/memblock.h
> index 0257aee..efbbe4b 100644
> --- a/include/linux/memblock.h
> +++ b/include/linux/memblock.h
> @@ -203,6 +203,10 @@ void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,
> i >= 0; __next_mem_pfn_range(&i, nid, p_start, p_end, p_nid))
> #endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>
> +#ifdef CONFIG_HAVE_ARCH_PFN_VALID
> +unsigned long memblock_next_valid_pfn(unsigned long pfn);
> +#endif
> +
> /**
> * for_each_free_mem_range - iterate through free memblock areas
> * @i: u64 used as loop variable
> diff --git a/mm/memblock.c b/mm/memblock.c
> index ba7c878..bea5a9c 100644
> --- a/mm/memblock.c
> +++ b/mm/memblock.c
> @@ -1102,6 +1102,35 @@ void __init_memblock __next_mem_pfn_range(int *idx, int nid,
> *out_nid = r->nid;
> }
>
> +#ifdef CONFIG_HAVE_ARCH_PFN_VALID
> +unsigned long __init_memblock memblock_next_valid_pfn(unsigned long pfn)
> +{
> + struct memblock_type *type = &memblock.memory;
> + unsigned int right = type->cnt;
> + unsigned int mid, left = 0;
> + phys_addr_t addr = PFN_PHYS(++pfn);
> +
> + do {
> + mid = (right + left) / 2;
> +
> + if (addr < type->regions[mid].base)
> + right = mid;
> + else if (addr >= (type->regions[mid].base +
> + type->regions[mid].size))
> + left = mid + 1;
> + else {
> + /* addr is within the region, so pfn is valid */
> + return pfn;
> + }
> + } while (left < right);
> +
> + if (right == type->cnt)
> + return -1UL;
> + else
> + return PHYS_PFN(type->regions[right].base);
> +}
> +#endif /*CONFIG_HAVE_ARCH_PFN_VALID*/
> +
> /**
> * memblock_set_node - set node ID on memblock regions
> * @base: base of area to set node ID for
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index c19f5ac..2a967f7 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -5483,8 +5483,17 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
> if (context != MEMMAP_EARLY)
> goto not_early;
>
> - if (!early_pfn_valid(pfn))
> + if (!early_pfn_valid(pfn)) {
> +#if (defined CONFIG_HAVE_MEMBLOCK) && (defined CONFIG_HAVE_ARCH_PFN_VALID)
> + /*
> + * Skip to the pfn preceding the next valid one (or
> + * end_pfn), such that we hit a valid pfn (or end_pfn)
> + * on our next iteration of the loop.
> + */
> + pfn = memblock_next_valid_pfn(pfn) - 1;
> +#endif
> continue;
> + }
> if (!early_pfn_in_nid(pfn, nid))
> continue;
> if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised))
> --
> 2.7.4
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v2 1/5] mm: page_alloc: remain memblock_next_valid_pfn() when CONFIG_HAVE_ARCH_PFN_VALID is enable
2018-03-27 16:52 ` Daniel Vacek
@ 2018-03-28 1:49 ` Jia He
2018-03-28 9:26 ` Jia He
1 sibling, 0 replies; 16+ messages in thread
From: Jia He @ 2018-03-28 1:49 UTC (permalink / raw)
To: Daniel Vacek
Cc: Andrew Morton, Michal Hocko, Catalin Marinas, Mel Gorman,
Will Deacon, Mark Rutland, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin, Pavel Tatashin, Daniel Jordan, AKASHI Takahiro,
Gioh Kim, Steven Sistare, Eugeniu Rosca, Vlastimil Babka,
open list, linux-mm, James Morse, Ard Biesheuvel, Steve Capper,
x86, Greg Kroah-Hartman, Kate Stewart, Philippe Ombredanne,
Johannes Weiner, Kemi Wang, Petr Tesarik, YASUAKI ISHIMATSU,
Andrey Ryabinin, Nikolay Borisov, Jia He
On 3/28/2018 12:52 AM, Daniel Vacek Wrote:
> On Sat, Mar 24, 2018 at 1:24 PM, Jia He <hejianet@gmail.com> wrote:
>> Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
>> where possible") optimized the loop in memmap_init_zone(). But it causes
>> possible panic bug. So Daniel Vacek reverted it later.
>>
>> But memblock_next_valid_pfn is valid when CONFIG_HAVE_ARCH_PFN_VALID is
>> enabled. And as verified by Eugeniu Rosca, arm can benifit from this
>> commit. So remain the memblock_next_valid_pfn.
> It is not dependent on CONFIG_HAVE_ARCH_PFN_VALID option but on
> arm(64) implementation of pfn_valid() function, IIUC. So it should
> really be moved from generic source file to arm specific location. I'd
> say somewhere close to the pfn_valid() implementation. Such as to
> arch/arm{,64}/mm/ init.c-ish?
>
> --nX
Ok, thanks for your suggestions.
I will try to move the related codes to arm arch directory.
Cheer,
Jia
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v2 1/5] mm: page_alloc: remain memblock_next_valid_pfn() when CONFIG_HAVE_ARCH_PFN_VALID is enable
@ 2018-03-28 1:49 ` Jia He
0 siblings, 0 replies; 16+ messages in thread
From: Jia He @ 2018-03-28 1:49 UTC (permalink / raw)
To: Daniel Vacek
Cc: Andrew Morton, Michal Hocko, Catalin Marinas, Mel Gorman,
Will Deacon, Mark Rutland, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin, Pavel Tatashin, Daniel Jordan, AKASHI Takahiro,
Gioh Kim, Steven Sistare, Eugeniu Rosca, Vlastimil Babka,
open list, linux-mm, James Morse, Ard Biesheuvel, Steve Capper,
x86, Greg Kroah-Hartman, Kate Stewart, Philippe Ombredanne,
Johannes Weiner, Kemi Wang, Petr Tesarik, YASUAKI ISHIMATSU,
Andrey Ryabinin, Nikolay Borisov, Jia He
On 3/28/2018 12:52 AM, Daniel Vacek Wrote:
> On Sat, Mar 24, 2018 at 1:24 PM, Jia He <hejianet@gmail.com> wrote:
>> Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
>> where possible") optimized the loop in memmap_init_zone(). But it causes
>> possible panic bug. So Daniel Vacek reverted it later.
>>
>> But memblock_next_valid_pfn is valid when CONFIG_HAVE_ARCH_PFN_VALID is
>> enabled. And as verified by Eugeniu Rosca, arm can benifit from this
>> commit. So remain the memblock_next_valid_pfn.
> It is not dependent on CONFIG_HAVE_ARCH_PFN_VALID option but on
> arm(64) implementation of pfn_valid() function, IIUC. So it should
> really be moved from generic source file to arm specific location. I'd
> say somewhere close to the pfn_valid() implementation. Such as to
> arch/arm{,64}/mm/ init.c-ish?
>
> --nX
Ok, thanks for your suggestions.
I will try to move theA related codes to arm arch directory.
Cheer,
Jia
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v2 1/5] mm: page_alloc: remain memblock_next_valid_pfn() when CONFIG_HAVE_ARCH_PFN_VALID is enable
2018-03-27 16:52 ` Daniel Vacek
2018-03-28 1:49 ` Jia He
@ 2018-03-28 9:26 ` Jia He
2018-03-28 10:42 ` Daniel Vacek
1 sibling, 1 reply; 16+ messages in thread
From: Jia He @ 2018-03-28 9:26 UTC (permalink / raw)
To: Daniel Vacek
Cc: Andrew Morton, Michal Hocko, Catalin Marinas, Mel Gorman,
Will Deacon, Mark Rutland, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin, Pavel Tatashin, Daniel Jordan, AKASHI Takahiro,
Gioh Kim, Steven Sistare, Eugeniu Rosca, Vlastimil Babka,
open list, linux-mm, James Morse, Ard Biesheuvel, Steve Capper,
x86, Greg Kroah-Hartman, Kate Stewart, Philippe Ombredanne,
Johannes Weiner, Kemi Wang, Petr Tesarik, YASUAKI ISHIMATSU,
Andrey Ryabinin, Nikolay Borisov, Jia He
On 3/28/2018 12:52 AM, Daniel Vacek Wrote:
> On Sat, Mar 24, 2018 at 1:24 PM, Jia He <hejianet@gmail.com> wrote:
>> Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
>> where possible") optimized the loop in memmap_init_zone(). But it causes
>> possible panic bug. So Daniel Vacek reverted it later.
>>
>> But memblock_next_valid_pfn is valid when CONFIG_HAVE_ARCH_PFN_VALID is
>> enabled. And as verified by Eugeniu Rosca, arm can benifit from this
>> commit. So remain the memblock_next_valid_pfn.
> It is not dependent on CONFIG_HAVE_ARCH_PFN_VALID option but on
> arm(64) implementation of pfn_valid() function, IIUC. So it should
> really be moved from generic source file to arm specific location. I'd
> say somewhere close to the pfn_valid() implementation. Such as to
> arch/arm{,64}/mm/ init.c-ish?
>
> --nX
Hi Daniel
I didn't catch the reason why "It is not dependent on
CONFIG_HAVE_ARCH_PFN_VALID option but
on arm(64) implementation of pfn_valid() function"? Can you explain more
about it? Thanks
What's your thought if I changed the codes as followed?
in include/linux/memblock.h
#ifdef CONFIG_HAVE_ARCH_PFN_VALID
extern unsigned long memblock_next_valid_pfn(unsigned long pfn);
#else
#define memblock_next_valid_pfn(pfn) (pfn + 1)
#endif
Cheers,
Jia
>
>> Signed-off-by: Jia He <jia.he@hxt-semitech.com>
>> ---
>> include/linux/memblock.h | 4 ++++
>> mm/memblock.c | 29 +++++++++++++++++++++++++++++
>> mm/page_alloc.c | 11 ++++++++++-
>> 3 files changed, 43 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/linux/memblock.h b/include/linux/memblock.h
>> index 0257aee..efbbe4b 100644
>> --- a/include/linux/memblock.h
>> +++ b/include/linux/memblock.h
>> @@ -203,6 +203,10 @@ void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,
>> i >= 0; __next_mem_pfn_range(&i, nid, p_start, p_end, p_nid))
>> #endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>>
>> +#ifdef CONFIG_HAVE_ARCH_PFN_VALID
>> +unsigned long memblock_next_valid_pfn(unsigned long pfn);
>> +#endif
>> +
>> /**
>> * for_each_free_mem_range - iterate through free memblock areas
>> * @i: u64 used as loop variable
>> diff --git a/mm/memblock.c b/mm/memblock.c
>> index ba7c878..bea5a9c 100644
>> --- a/mm/memblock.c
>> +++ b/mm/memblock.c
>> @@ -1102,6 +1102,35 @@ void __init_memblock __next_mem_pfn_range(int *idx, int nid,
>> *out_nid = r->nid;
>> }
>>
>> +#ifdef CONFIG_HAVE_ARCH_PFN_VALID
>> +unsigned long __init_memblock memblock_next_valid_pfn(unsigned long pfn)
>> +{
>> + struct memblock_type *type = &memblock.memory;
>> + unsigned int right = type->cnt;
>> + unsigned int mid, left = 0;
>> + phys_addr_t addr = PFN_PHYS(++pfn);
>> +
>> + do {
>> + mid = (right + left) / 2;
>> +
>> + if (addr < type->regions[mid].base)
>> + right = mid;
>> + else if (addr >= (type->regions[mid].base +
>> + type->regions[mid].size))
>> + left = mid + 1;
>> + else {
>> + /* addr is within the region, so pfn is valid */
>> + return pfn;
>> + }
>> + } while (left < right);
>> +
>> + if (right == type->cnt)
>> + return -1UL;
>> + else
>> + return PHYS_PFN(type->regions[right].base);
>> +}
>> +#endif /*CONFIG_HAVE_ARCH_PFN_VALID*/
>> +
>> /**
>> * memblock_set_node - set node ID on memblock regions
>> * @base: base of area to set node ID for
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index c19f5ac..2a967f7 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -5483,8 +5483,17 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
>> if (context != MEMMAP_EARLY)
>> goto not_early;
>>
>> - if (!early_pfn_valid(pfn))
>> + if (!early_pfn_valid(pfn)) {
>> +#if (defined CONFIG_HAVE_MEMBLOCK) && (defined CONFIG_HAVE_ARCH_PFN_VALID)
>> + /*
>> + * Skip to the pfn preceding the next valid one (or
>> + * end_pfn), such that we hit a valid pfn (or end_pfn)
>> + * on our next iteration of the loop.
>> + */
>> + pfn = memblock_next_valid_pfn(pfn) - 1;
>> +#endif
>> continue;
>> + }
>> if (!early_pfn_in_nid(pfn, nid))
>> continue;
>> if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised))
>> --
>> 2.7.4
>>
--
Cheers,
Jia
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v2 1/5] mm: page_alloc: remain memblock_next_valid_pfn() when CONFIG_HAVE_ARCH_PFN_VALID is enable
2018-03-28 9:26 ` Jia He
@ 2018-03-28 10:42 ` Daniel Vacek
0 siblings, 0 replies; 16+ messages in thread
From: Daniel Vacek @ 2018-03-28 10:42 UTC (permalink / raw)
To: Jia He
Cc: Andrew Morton, Michal Hocko, Catalin Marinas, Mel Gorman,
Will Deacon, Mark Rutland, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin, Pavel Tatashin, Daniel Jordan, AKASHI Takahiro,
Gioh Kim, Steven Sistare, Eugeniu Rosca, Vlastimil Babka,
open list, linux-mm, James Morse, Ard Biesheuvel, Steve Capper,
x86, Greg Kroah-Hartman, Kate Stewart, Philippe Ombredanne,
Johannes Weiner, Kemi Wang, Petr Tesarik, YASUAKI ISHIMATSU,
Andrey Ryabinin, Nikolay Borisov, Jia He
On Wed, Mar 28, 2018 at 11:26 AM, Jia He <hejianet@gmail.com> wrote:
>
>
> On 3/28/2018 12:52 AM, Daniel Vacek Wrote:
>>
>> On Sat, Mar 24, 2018 at 1:24 PM, Jia He <hejianet@gmail.com> wrote:
>>>
>>> Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
>>> where possible") optimized the loop in memmap_init_zone(). But it causes
>>> possible panic bug. So Daniel Vacek reverted it later.
>>>
>>> But memblock_next_valid_pfn is valid when CONFIG_HAVE_ARCH_PFN_VALID is
>>> enabled. And as verified by Eugeniu Rosca, arm can benifit from this
>>> commit. So remain the memblock_next_valid_pfn.
>>
>> It is not dependent on CONFIG_HAVE_ARCH_PFN_VALID option but on
>> arm(64) implementation of pfn_valid() function, IIUC. So it should
>> really be moved from generic source file to arm specific location. I'd
>> say somewhere close to the pfn_valid() implementation. Such as to
>> arch/arm{,64}/mm/ init.c-ish?
>>
>> --nX
>
> Hi Daniel
> I didn't catch the reason why "It is not dependent on
> CONFIG_HAVE_ARCH_PFN_VALID option but
> on arm(64) implementation of pfn_valid() function"? Can you explain more
> about it? Thanks
Arm implementation of pfn_valid() is actually based on memblock as
HAVE_MEMBLOCK is mandatory for arm so memblock is guaranteed to always
be available, IIUC. Correct me if I am wrong here.
With that said, you are fine with using memblock to skip gaps and
finding next valid frame.
Though the generic version of pfn_valid() is based on mem sections and
memblock_next_valid_pfn() does not always return the next valid one
but skips more resulting in some valid frames to be skipped (as if
they were invalid). And that's why kernel was eventually crashing on
some !arm machines.
Now, if any other architecture defines CONFIG_HAVE_ARCH_PFN_VALID and
implements it's own version of pfn_valid(), there is no guarantee that
it will be based on memblock data or somehow equivalent to the arm
implementation, right?
At this moment only arm implements CONFIG_HAVE_ARCH_PFN_VALID. Maybe
it could be generalized to something like CONFIG_MEMBLOCK_PFN_VALID
and moved to generic code. And then you can base your changes on that.
But I am not sure if that is possible.
> What's your thought if I changed the codes as followed?
> in include/linux/memblock.h
> #ifdef CONFIG_HAVE_ARCH_PFN_VALID
> extern unsigned long memblock_next_valid_pfn(unsigned long pfn);
> #else
> #define memblock_next_valid_pfn(pfn) (pfn + 1)
> #endif
I think I'd rather do something like this:
if (!early_pfn_valid(pfn)) {
pfn = skip_to_last_invalid_pfn(pfn);
continue;
}
And than for arm define:
#if (defined CONFIG_HAVE_MEMBLOCK) && (defined CONFIG_HAVE_ARCH_PFN_VALID)
#define skip_to_last_invalid_pfn(pfn) (memblock_next_valid_pfn(pfn,
&early_idx) - 1)
#endif
And for the generic fallback:
#ifndef skip_to_last_invalid_pfn
#define skip_to_last_invalid_pfn(pfn) (pfn)
#endif
--nX
> Cheers,
> Jia
>
>>
>>> Signed-off-by: Jia He <jia.he@hxt-semitech.com>
>>> ---
>>> include/linux/memblock.h | 4 ++++
>>> mm/memblock.c | 29 +++++++++++++++++++++++++++++
>>> mm/page_alloc.c | 11 ++++++++++-
>>> 3 files changed, 43 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/include/linux/memblock.h b/include/linux/memblock.h
>>> index 0257aee..efbbe4b 100644
>>> --- a/include/linux/memblock.h
>>> +++ b/include/linux/memblock.h
>>> @@ -203,6 +203,10 @@ void __next_mem_pfn_range(int *idx, int nid,
>>> unsigned long *out_start_pfn,
>>> i >= 0; __next_mem_pfn_range(&i, nid, p_start, p_end,
>>> p_nid))
>>> #endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>>>
>>> +#ifdef CONFIG_HAVE_ARCH_PFN_VALID
>>> +unsigned long memblock_next_valid_pfn(unsigned long pfn);
>>> +#endif
>>> +
>>> /**
>>> * for_each_free_mem_range - iterate through free memblock areas
>>> * @i: u64 used as loop variable
>>> diff --git a/mm/memblock.c b/mm/memblock.c
>>> index ba7c878..bea5a9c 100644
>>> --- a/mm/memblock.c
>>> +++ b/mm/memblock.c
>>> @@ -1102,6 +1102,35 @@ void __init_memblock __next_mem_pfn_range(int
>>> *idx, int nid,
>>> *out_nid = r->nid;
>>> }
>>>
>>> +#ifdef CONFIG_HAVE_ARCH_PFN_VALID
>>> +unsigned long __init_memblock memblock_next_valid_pfn(unsigned long pfn)
>>> +{
>>> + struct memblock_type *type = &memblock.memory;
>>> + unsigned int right = type->cnt;
>>> + unsigned int mid, left = 0;
>>> + phys_addr_t addr = PFN_PHYS(++pfn);
>>> +
>>> + do {
>>> + mid = (right + left) / 2;
>>> +
>>> + if (addr < type->regions[mid].base)
>>> + right = mid;
>>> + else if (addr >= (type->regions[mid].base +
>>> + type->regions[mid].size))
>>> + left = mid + 1;
>>> + else {
>>> + /* addr is within the region, so pfn is valid */
>>> + return pfn;
>>> + }
>>> + } while (left < right);
>>> +
>>> + if (right == type->cnt)
>>> + return -1UL;
>>> + else
>>> + return PHYS_PFN(type->regions[right].base);
>>> +}
>>> +#endif /*CONFIG_HAVE_ARCH_PFN_VALID*/
>>> +
>>> /**
>>> * memblock_set_node - set node ID on memblock regions
>>> * @base: base of area to set node ID for
>>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>>> index c19f5ac..2a967f7 100644
>>> --- a/mm/page_alloc.c
>>> +++ b/mm/page_alloc.c
>>> @@ -5483,8 +5483,17 @@ void __meminit memmap_init_zone(unsigned long
>>> size, int nid, unsigned long zone,
>>> if (context != MEMMAP_EARLY)
>>> goto not_early;
>>>
>>> - if (!early_pfn_valid(pfn))
>>> + if (!early_pfn_valid(pfn)) {
>>> +#if (defined CONFIG_HAVE_MEMBLOCK) && (defined
>>> CONFIG_HAVE_ARCH_PFN_VALID)
>>> + /*
>>> + * Skip to the pfn preceding the next valid one
>>> (or
>>> + * end_pfn), such that we hit a valid pfn (or
>>> end_pfn)
>>> + * on our next iteration of the loop.
>>> + */
>>> + pfn = memblock_next_valid_pfn(pfn) - 1;
>>> +#endif
>>> continue;
>>> + }
>>> if (!early_pfn_in_nid(pfn, nid))
>>> continue;
>>> if (!update_defer_init(pgdat, pfn, end_pfn,
>>> &nr_initialised))
>>> --
>>> 2.7.4
>>>
>
> --
> Cheers,
> Jia
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v2 1/5] mm: page_alloc: remain memblock_next_valid_pfn() when CONFIG_HAVE_ARCH_PFN_VALID is enable
2018-03-24 12:24 ` [PATCH v2 1/5] mm: page_alloc: remain memblock_next_valid_pfn() when CONFIG_HAVE_ARCH_PFN_VALID is enable Jia He
2018-03-27 16:52 ` Daniel Vacek
@ 2018-03-28 9:13 ` Wei Yang
1 sibling, 0 replies; 16+ messages in thread
From: Wei Yang @ 2018-03-28 9:13 UTC (permalink / raw)
To: Jia He
Cc: Andrew Morton, Michal Hocko, Catalin Marinas, Mel Gorman,
Will Deacon, Mark Rutland, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin, Pavel Tatashin, Daniel Jordan, AKASHI Takahiro,
Gioh Kim, Steven Sistare, Daniel Vacek, Eugeniu Rosca,
Vlastimil Babka, linux-kernel, linux-mm, James Morse,
Ard Biesheuvel, Steve Capper, x86, Greg Kroah-Hartman,
Kate Stewart, Philippe Ombredanne, Johannes Weiner, Kemi Wang,
Petr Tesarik, YASUAKI ISHIMATSU, Andrey Ryabinin,
Nikolay Borisov, Jia He
On Sat, Mar 24, 2018 at 05:24:38AM -0700, Jia He wrote:
>Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
>where possible") optimized the loop in memmap_init_zone(). But it causes
>possible panic bug. So Daniel Vacek reverted it later.
>
Why this has a bug? Do you have some link about it?
If the audience could know the potential risk, it would be helpful to review
the code and decide whether to take it back.
>But memblock_next_valid_pfn is valid when CONFIG_HAVE_ARCH_PFN_VALID is
>enabled. And as verified by Eugeniu Rosca, arm can benifit from this
>commit. So remain the memblock_next_valid_pfn.
>
>Signed-off-by: Jia He <jia.he@hxt-semitech.com>
>---
> include/linux/memblock.h | 4 ++++
> mm/memblock.c | 29 +++++++++++++++++++++++++++++
> mm/page_alloc.c | 11 ++++++++++-
> 3 files changed, 43 insertions(+), 1 deletion(-)
>
>diff --git a/include/linux/memblock.h b/include/linux/memblock.h
>index 0257aee..efbbe4b 100644
>--- a/include/linux/memblock.h
>+++ b/include/linux/memblock.h
>@@ -203,6 +203,10 @@ void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,
> i >= 0; __next_mem_pfn_range(&i, nid, p_start, p_end, p_nid))
> #endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
>
>+#ifdef CONFIG_HAVE_ARCH_PFN_VALID
>+unsigned long memblock_next_valid_pfn(unsigned long pfn);
>+#endif
>+
> /**
> * for_each_free_mem_range - iterate through free memblock areas
> * @i: u64 used as loop variable
>diff --git a/mm/memblock.c b/mm/memblock.c
>index ba7c878..bea5a9c 100644
>--- a/mm/memblock.c
>+++ b/mm/memblock.c
>@@ -1102,6 +1102,35 @@ void __init_memblock __next_mem_pfn_range(int *idx, int nid,
> *out_nid = r->nid;
> }
>
>+#ifdef CONFIG_HAVE_ARCH_PFN_VALID
>+unsigned long __init_memblock memblock_next_valid_pfn(unsigned long pfn)
>+{
>+ struct memblock_type *type = &memblock.memory;
>+ unsigned int right = type->cnt;
>+ unsigned int mid, left = 0;
>+ phys_addr_t addr = PFN_PHYS(++pfn);
>+
>+ do {
>+ mid = (right + left) / 2;
>+
>+ if (addr < type->regions[mid].base)
>+ right = mid;
>+ else if (addr >= (type->regions[mid].base +
>+ type->regions[mid].size))
>+ left = mid + 1;
>+ else {
>+ /* addr is within the region, so pfn is valid */
>+ return pfn;
>+ }
>+ } while (left < right);
>+
>+ if (right == type->cnt)
>+ return -1UL;
>+ else
>+ return PHYS_PFN(type->regions[right].base);
>+}
>+#endif /*CONFIG_HAVE_ARCH_PFN_VALID*/
>+
> /**
> * memblock_set_node - set node ID on memblock regions
> * @base: base of area to set node ID for
>diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>index c19f5ac..2a967f7 100644
>--- a/mm/page_alloc.c
>+++ b/mm/page_alloc.c
>@@ -5483,8 +5483,17 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
> if (context != MEMMAP_EARLY)
> goto not_early;
>
>- if (!early_pfn_valid(pfn))
>+ if (!early_pfn_valid(pfn)) {
>+#if (defined CONFIG_HAVE_MEMBLOCK) && (defined CONFIG_HAVE_ARCH_PFN_VALID)
In commit b92df1de5d28, it use CONFIG_HAVE_MEMBLOCK_NODE_MAP.
Not get the point of your change.
>+ /*
>+ * Skip to the pfn preceding the next valid one (or
>+ * end_pfn), such that we hit a valid pfn (or end_pfn)
>+ * on our next iteration of the loop.
>+ */
>+ pfn = memblock_next_valid_pfn(pfn) - 1;
>+#endif
> continue;
>+ }
> if (!early_pfn_in_nid(pfn, nid))
> continue;
> if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised))
>--
>2.7.4
--
Wei Yang
Help you, Help me
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v2 2/5] mm: page_alloc: reduce unnecessary binary search in memblock_next_valid_pfn()
2018-03-24 12:24 [PATCH v2 0/5] optimize memblock_next_valid_pfn() and early_pfn_valid() Jia He
2018-03-24 12:24 ` [PATCH v2 1/5] mm: page_alloc: remain memblock_next_valid_pfn() when CONFIG_HAVE_ARCH_PFN_VALID is enable Jia He
@ 2018-03-24 12:24 ` Jia He
2018-03-24 12:24 ` [PATCH v2 3/5] mm/memblock: introduce memblock_search_pfn_regions() Jia He
` (2 subsequent siblings)
4 siblings, 0 replies; 16+ messages in thread
From: Jia He @ 2018-03-24 12:24 UTC (permalink / raw)
To: Andrew Morton, Michal Hocko, Catalin Marinas, Mel Gorman,
Will Deacon, Mark Rutland, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin
Cc: Pavel Tatashin, Daniel Jordan, AKASHI Takahiro, Gioh Kim,
Steven Sistare, Daniel Vacek, Eugeniu Rosca, Vlastimil Babka,
linux-kernel, linux-mm, James Morse, Ard Biesheuvel,
Steve Capper, x86, Greg Kroah-Hartman, Kate Stewart,
Philippe Ombredanne, Johannes Weiner, Kemi Wang, Petr Tesarik,
YASUAKI ISHIMATSU, Andrey Ryabinin, Nikolay Borisov, Jia He,
Jia He
Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
where possible") optimized the loop in memmap_init_zone(). But there is
still some room for improvement. E.g. if pfn and pfn+1 are in the same
memblock region, we can simply pfn++ instead of doing the binary search
in memblock_next_valid_pfn.
Signed-off-by: Jia He <jia.he@hxt-semitech.com>
---
include/linux/memblock.h | 2 +-
mm/memblock.c | 73 +++++++++++++++++++++++++++++-------------------
mm/page_alloc.c | 3 +-
3 files changed, 47 insertions(+), 31 deletions(-)
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index efbbe4b..a8fb2ab 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -204,7 +204,7 @@ void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,
#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
#ifdef CONFIG_HAVE_ARCH_PFN_VALID
-unsigned long memblock_next_valid_pfn(unsigned long pfn);
+unsigned long memblock_next_valid_pfn(unsigned long pfn, int *idx);
#endif
/**
diff --git a/mm/memblock.c b/mm/memblock.c
index bea5a9c..06c1a08 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -1102,35 +1102,6 @@ void __init_memblock __next_mem_pfn_range(int *idx, int nid,
*out_nid = r->nid;
}
-#ifdef CONFIG_HAVE_ARCH_PFN_VALID
-unsigned long __init_memblock memblock_next_valid_pfn(unsigned long pfn)
-{
- struct memblock_type *type = &memblock.memory;
- unsigned int right = type->cnt;
- unsigned int mid, left = 0;
- phys_addr_t addr = PFN_PHYS(++pfn);
-
- do {
- mid = (right + left) / 2;
-
- if (addr < type->regions[mid].base)
- right = mid;
- else if (addr >= (type->regions[mid].base +
- type->regions[mid].size))
- left = mid + 1;
- else {
- /* addr is within the region, so pfn is valid */
- return pfn;
- }
- } while (left < right);
-
- if (right == type->cnt)
- return -1UL;
- else
- return PHYS_PFN(type->regions[right].base);
-}
-#endif /*CONFIG_HAVE_ARCH_PFN_VALID*/
-
/**
* memblock_set_node - set node ID on memblock regions
* @base: base of area to set node ID for
@@ -1162,6 +1133,50 @@ int __init_memblock memblock_set_node(phys_addr_t base, phys_addr_t size,
}
#endif /* CONFIG_HAVE_MEMBLOCK_NODE_MAP */
+#ifdef CONFIG_HAVE_ARCH_PFN_VALID
+unsigned long __init_memblock memblock_next_valid_pfn(unsigned long pfn,
+ int *last_idx)
+{
+ struct memblock_type *type = &memblock.memory;
+ unsigned int right = type->cnt;
+ unsigned int mid, left = 0;
+ unsigned long start_pfn, end_pfn;
+ phys_addr_t addr = PFN_PHYS(++pfn);
+
+ /* fast path, return pfh+1 if next pfn is in the same region */
+ if (*last_idx != -1) {
+ start_pfn = PFN_DOWN(type->regions[*last_idx].base);
+ end_pfn = PFN_DOWN(type->regions[*last_idx].base +
+ type->regions[*last_idx].size);
+
+ if (pfn < end_pfn && pfn > start_pfn)
+ return pfn;
+ }
+
+ /* slow path, do the binary searching */
+ do {
+ mid = (right + left) / 2;
+
+ if (addr < type->regions[mid].base)
+ right = mid;
+ else if (addr >= (type->regions[mid].base +
+ type->regions[mid].size))
+ left = mid + 1;
+ else {
+ *last_idx = mid;
+ return pfn;
+ }
+ } while (left < right);
+
+ if (right == type->cnt)
+ return -1UL;
+
+ *last_idx = right;
+
+ return PHYS_PFN(type->regions[*last_idx].base);
+}
+#endif /*CONFIG_HAVE_ARCH_PFN_VALID*/
+
static phys_addr_t __init memblock_alloc_range_nid(phys_addr_t size,
phys_addr_t align, phys_addr_t start,
phys_addr_t end, int nid, ulong flags)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 2a967f7..0bb0274 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5459,6 +5459,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
unsigned long end_pfn = start_pfn + size;
pg_data_t *pgdat = NODE_DATA(nid);
unsigned long pfn;
+ int idx = -1;
unsigned long nr_initialised = 0;
struct page *page;
#ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
@@ -5490,7 +5491,7 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
* end_pfn), such that we hit a valid pfn (or end_pfn)
* on our next iteration of the loop.
*/
- pfn = memblock_next_valid_pfn(pfn) - 1;
+ pfn = memblock_next_valid_pfn(pfn, &idx) - 1;
#endif
continue;
}
--
2.7.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v2 3/5] mm/memblock: introduce memblock_search_pfn_regions()
2018-03-24 12:24 [PATCH v2 0/5] optimize memblock_next_valid_pfn() and early_pfn_valid() Jia He
2018-03-24 12:24 ` [PATCH v2 1/5] mm: page_alloc: remain memblock_next_valid_pfn() when CONFIG_HAVE_ARCH_PFN_VALID is enable Jia He
2018-03-24 12:24 ` [PATCH v2 2/5] mm: page_alloc: reduce unnecessary binary search in memblock_next_valid_pfn() Jia He
@ 2018-03-24 12:24 ` Jia He
2018-03-24 12:24 ` [PATCH v2 4/5] arm64: introduce pfn_valid_region() Jia He
2018-03-24 12:24 ` [PATCH v2 5/5] mm: page_alloc: reduce unnecessary binary search in early_pfn_valid() Jia He
4 siblings, 0 replies; 16+ messages in thread
From: Jia He @ 2018-03-24 12:24 UTC (permalink / raw)
To: Andrew Morton, Michal Hocko, Catalin Marinas, Mel Gorman,
Will Deacon, Mark Rutland, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin
Cc: Pavel Tatashin, Daniel Jordan, AKASHI Takahiro, Gioh Kim,
Steven Sistare, Daniel Vacek, Eugeniu Rosca, Vlastimil Babka,
linux-kernel, linux-mm, James Morse, Ard Biesheuvel,
Steve Capper, x86, Greg Kroah-Hartman, Kate Stewart,
Philippe Ombredanne, Johannes Weiner, Kemi Wang, Petr Tesarik,
YASUAKI ISHIMATSU, Andrey Ryabinin, Nikolay Borisov, Jia He,
Jia He
This api is the preparation for further optimizing early_pfn_valid
Signed-off-by: Jia He <jia.he@hxt-semitech.com>
---
include/linux/memblock.h | 2 ++
mm/memblock.c | 9 +++++++++
2 files changed, 11 insertions(+)
diff --git a/include/linux/memblock.h b/include/linux/memblock.h
index a8fb2ab..104bca6 100644
--- a/include/linux/memblock.h
+++ b/include/linux/memblock.h
@@ -207,6 +207,8 @@ void __next_mem_pfn_range(int *idx, int nid, unsigned long *out_start_pfn,
unsigned long memblock_next_valid_pfn(unsigned long pfn, int *idx);
#endif
+int memblock_search_pfn_regions(unsigned long pfn);
+
/**
* for_each_free_mem_range - iterate through free memblock areas
* @i: u64 used as loop variable
diff --git a/mm/memblock.c b/mm/memblock.c
index 06c1a08..15fcde2 100644
--- a/mm/memblock.c
+++ b/mm/memblock.c
@@ -1661,6 +1661,15 @@ static int __init_memblock memblock_search(struct memblock_type *type, phys_addr
return -1;
}
+/* search memblock with the input pfn, return the region idx */
+int __init_memblock memblock_search_pfn_regions(unsigned long pfn)
+{
+ struct memblock_type *type = &memblock.memory;
+ int mid = memblock_search(type, PFN_PHYS(pfn));
+
+ return mid;
+}
+
bool __init memblock_is_reserved(phys_addr_t addr)
{
return memblock_search(&memblock.reserved, addr) != -1;
--
2.7.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH v2 4/5] arm64: introduce pfn_valid_region()
2018-03-24 12:24 [PATCH v2 0/5] optimize memblock_next_valid_pfn() and early_pfn_valid() Jia He
` (2 preceding siblings ...)
2018-03-24 12:24 ` [PATCH v2 3/5] mm/memblock: introduce memblock_search_pfn_regions() Jia He
@ 2018-03-24 12:24 ` Jia He
2018-03-25 14:16 ` kbuild test robot
2018-03-24 12:24 ` [PATCH v2 5/5] mm: page_alloc: reduce unnecessary binary search in early_pfn_valid() Jia He
4 siblings, 1 reply; 16+ messages in thread
From: Jia He @ 2018-03-24 12:24 UTC (permalink / raw)
To: Andrew Morton, Michal Hocko, Catalin Marinas, Mel Gorman,
Will Deacon, Mark Rutland, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin
Cc: Pavel Tatashin, Daniel Jordan, AKASHI Takahiro, Gioh Kim,
Steven Sistare, Daniel Vacek, Eugeniu Rosca, Vlastimil Babka,
linux-kernel, linux-mm, James Morse, Ard Biesheuvel,
Steve Capper, x86, Greg Kroah-Hartman, Kate Stewart,
Philippe Ombredanne, Johannes Weiner, Kemi Wang, Petr Tesarik,
YASUAKI ISHIMATSU, Andrey Ryabinin, Nikolay Borisov, Jia He,
Jia He
This is the preparation for further optimizing in early_pfn_valid
on arm64.
Signed-off-by: Jia He <jia.he@hxt-semitech.com>
---
arch/arm64/include/asm/page.h | 3 ++-
arch/arm64/mm/init.c | 25 ++++++++++++++++++++++++-
2 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/page.h b/arch/arm64/include/asm/page.h
index 60d02c8..da2cba3 100644
--- a/arch/arm64/include/asm/page.h
+++ b/arch/arm64/include/asm/page.h
@@ -38,7 +38,8 @@ extern void clear_page(void *to);
typedef struct page *pgtable_t;
#ifdef CONFIG_HAVE_ARCH_PFN_VALID
-extern int pfn_valid(unsigned long);
+extern int pfn_valid(unsigned long pfn);
+extern int pfn_valid_region(unsigned long pfn, int *last_idx);
#endif
#include <asm/memory.h>
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 00e7b90..9122102 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -290,7 +290,30 @@ int pfn_valid(unsigned long pfn)
return memblock_is_map_memory(pfn << PAGE_SHIFT);
}
EXPORT_SYMBOL(pfn_valid);
-#endif
+
+int pfn_valid_region(unsigned long pfn, int *last_idx)
+{
+ unsigned long start_pfn, end_pfn;
+ struct memblock_type *type = &memblock.memory;
+
+ if (*last_idx != -1) {
+ start_pfn = PFN_DOWN(type->regions[*last_idx].base);
+ end_pfn= PFN_DOWN(type->regions[*last_idx].base +
+ type->regions[*last_idx].size);
+
+ if (pfn >= start_pfn && end_pfn < end_pfn)
+ return !memblock_is_nomap(
+ &memblock.memory.regions[*last_idx]);
+ }
+
+ *last_idx = memblock_search_pfn_regions(pfn);
+ if (*last_idx == -1)
+ return false;
+
+ return !memblock_is_nomap(&memblock.memory.regions[*last_idx]);
+}
+EXPORT_SYMBOL(pfn_valid_region);
+#endif /*CONFIG_HAVE_ARCH_PFN_VALID*/
#ifndef CONFIG_SPARSEMEM
static void __init arm64_memory_present(void)
--
2.7.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH v2 4/5] arm64: introduce pfn_valid_region()
2018-03-24 12:24 ` [PATCH v2 4/5] arm64: introduce pfn_valid_region() Jia He
@ 2018-03-25 14:16 ` kbuild test robot
0 siblings, 0 replies; 16+ messages in thread
From: kbuild test robot @ 2018-03-25 14:16 UTC (permalink / raw)
To: Jia He
Cc: kbuild-all, Andrew Morton, Michal Hocko, Catalin Marinas,
Mel Gorman, Will Deacon, Mark Rutland, Thomas Gleixner,
Ingo Molnar, H. Peter Anvin, Pavel Tatashin, Daniel Jordan,
AKASHI Takahiro, Gioh Kim, Steven Sistare, Daniel Vacek,
Eugeniu Rosca, Vlastimil Babka, linux-kernel, linux-mm,
James Morse, Ard Biesheuvel, Steve Capper, x86,
Greg Kroah-Hartman, Kate Stewart, Philippe Ombredanne,
Johannes Weiner, Kemi Wang, Petr Tesarik, YASUAKI ISHIMATSU,
Andrey Ryabinin, Nikolay Borisov, Jia He, Jia He
[-- Attachment #1: Type: text/plain, Size: 2074 bytes --]
Hi Jia,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on mmotm/master]
[cannot apply to v4.16-rc6]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Jia-He/optimize-memblock_next_valid_pfn-and-early_pfn_valid/20180325-175026
base: git://git.cmpxchg.org/linux-mmotm.git master
config: arm64-defconfig (attached as .config)
compiler: aarch64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=arm64
All warnings (new ones prefixed by >>):
arch/arm64/mm/init.c: In function 'pfn_valid_region':
>> arch/arm64/mm/init.c:304:35: warning: self-comparison always evaluates to false [-Wtautological-compare]
if (pfn >= start_pfn && end_pfn < end_pfn)
^
vim +304 arch/arm64/mm/init.c
293
294 int pfn_valid_region(unsigned long pfn, int *last_idx)
295 {
296 unsigned long start_pfn, end_pfn;
297 struct memblock_type *type = &memblock.memory;
298
299 if (*last_idx != -1) {
300 start_pfn = PFN_DOWN(type->regions[*last_idx].base);
301 end_pfn= PFN_DOWN(type->regions[*last_idx].base +
302 type->regions[*last_idx].size);
303
> 304 if (pfn >= start_pfn && end_pfn < end_pfn)
305 return !memblock_is_nomap(
306 &memblock.memory.regions[*last_idx]);
307 }
308
309 *last_idx = memblock_search_pfn_regions(pfn);
310 if (*last_idx == -1)
311 return false;
312
313 return !memblock_is_nomap(&memblock.memory.regions[*last_idx]);
314 }
315 EXPORT_SYMBOL(pfn_valid_region);
316 #endif /*CONFIG_HAVE_ARCH_PFN_VALID*/
317
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 37385 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v2 4/5] arm64: introduce pfn_valid_region()
@ 2018-03-25 14:16 ` kbuild test robot
0 siblings, 0 replies; 16+ messages in thread
From: kbuild test robot @ 2018-03-25 14:16 UTC (permalink / raw)
To: Jia He
Cc: kbuild-all, Andrew Morton, Michal Hocko, Catalin Marinas,
Mel Gorman, Will Deacon, Mark Rutland, Thomas Gleixner,
Ingo Molnar, H. Peter Anvin, Pavel Tatashin, Daniel Jordan,
AKASHI Takahiro, Gioh Kim, Steven Sistare, Daniel Vacek,
Eugeniu Rosca, Vlastimil Babka, linux-kernel, linux-mm,
James Morse, Ard Biesheuvel, Steve Capper, x86,
Greg Kroah-Hartman, Kate Stewart, Philippe Ombredanne,
Johannes Weiner, Kemi Wang, Petr Tesarik, YASUAKI ISHIMATSU,
Andrey Ryabinin, Nikolay Borisov, Jia He
[-- Attachment #1: Type: text/plain, Size: 2074 bytes --]
Hi Jia,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on mmotm/master]
[cannot apply to v4.16-rc6]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Jia-He/optimize-memblock_next_valid_pfn-and-early_pfn_valid/20180325-175026
base: git://git.cmpxchg.org/linux-mmotm.git master
config: arm64-defconfig (attached as .config)
compiler: aarch64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0
reproduce:
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=arm64
All warnings (new ones prefixed by >>):
arch/arm64/mm/init.c: In function 'pfn_valid_region':
>> arch/arm64/mm/init.c:304:35: warning: self-comparison always evaluates to false [-Wtautological-compare]
if (pfn >= start_pfn && end_pfn < end_pfn)
^
vim +304 arch/arm64/mm/init.c
293
294 int pfn_valid_region(unsigned long pfn, int *last_idx)
295 {
296 unsigned long start_pfn, end_pfn;
297 struct memblock_type *type = &memblock.memory;
298
299 if (*last_idx != -1) {
300 start_pfn = PFN_DOWN(type->regions[*last_idx].base);
301 end_pfn= PFN_DOWN(type->regions[*last_idx].base +
302 type->regions[*last_idx].size);
303
> 304 if (pfn >= start_pfn && end_pfn < end_pfn)
305 return !memblock_is_nomap(
306 &memblock.memory.regions[*last_idx]);
307 }
308
309 *last_idx = memblock_search_pfn_regions(pfn);
310 if (*last_idx == -1)
311 return false;
312
313 return !memblock_is_nomap(&memblock.memory.regions[*last_idx]);
314 }
315 EXPORT_SYMBOL(pfn_valid_region);
316 #endif /*CONFIG_HAVE_ARCH_PFN_VALID*/
317
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 37385 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PATCH v2 5/5] mm: page_alloc: reduce unnecessary binary search in early_pfn_valid()
2018-03-24 12:24 [PATCH v2 0/5] optimize memblock_next_valid_pfn() and early_pfn_valid() Jia He
` (3 preceding siblings ...)
2018-03-24 12:24 ` [PATCH v2 4/5] arm64: introduce pfn_valid_region() Jia He
@ 2018-03-24 12:24 ` Jia He
2018-03-25 10:48 ` kbuild test robot
4 siblings, 1 reply; 16+ messages in thread
From: Jia He @ 2018-03-24 12:24 UTC (permalink / raw)
To: Andrew Morton, Michal Hocko, Catalin Marinas, Mel Gorman,
Will Deacon, Mark Rutland, Thomas Gleixner, Ingo Molnar,
H. Peter Anvin
Cc: Pavel Tatashin, Daniel Jordan, AKASHI Takahiro, Gioh Kim,
Steven Sistare, Daniel Vacek, Eugeniu Rosca, Vlastimil Babka,
linux-kernel, linux-mm, James Morse, Ard Biesheuvel,
Steve Capper, x86, Greg Kroah-Hartman, Kate Stewart,
Philippe Ombredanne, Johannes Weiner, Kemi Wang, Petr Tesarik,
YASUAKI ISHIMATSU, Andrey Ryabinin, Nikolay Borisov, Jia He,
Jia He
Commit b92df1de5d28 ("mm: page_alloc: skip over regions of invalid pfns
where possible") optimized the loop in memmap_init_zone(). But there is
still some room for improvement. E.g. in early_pfn_valid(), if pfn and
pfn+1 are in the same memblock region, we can record the last returned
memblock region index and check if pfn++ is still in the same region.
Currently it only improve the performance on arm64 and will have no
impact on other arches.
Signed-off-by: Jia He <jia.he@hxt-semitech.com>
---
arch/x86/include/asm/mmzone_32.h | 2 +-
include/linux/mmzone.h | 12 +++++++++---
mm/page_alloc.c | 2 +-
3 files changed, 11 insertions(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/mmzone_32.h b/arch/x86/include/asm/mmzone_32.h
index 73d8dd1..329d3ba 100644
--- a/arch/x86/include/asm/mmzone_32.h
+++ b/arch/x86/include/asm/mmzone_32.h
@@ -49,7 +49,7 @@ static inline int pfn_valid(int pfn)
return 0;
}
-#define early_pfn_valid(pfn) pfn_valid((pfn))
+#define early_pfn_valid(pfn, last_region_idx) pfn_valid((pfn))
#endif /* CONFIG_DISCONTIGMEM */
diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h
index d797716..3a686af 100644
--- a/include/linux/mmzone.h
+++ b/include/linux/mmzone.h
@@ -1267,9 +1267,15 @@ static inline int pfn_present(unsigned long pfn)
})
#else
#define pfn_to_nid(pfn) (0)
-#endif
+#endif /*CONFIG_NUMA*/
+
+#ifdef CONFIG_HAVE_ARCH_PFN_VALID
+#define early_pfn_valid(pfn, last_region_idx) \
+ pfn_valid_region(pfn, last_region_idx)
+#else
+#define early_pfn_valid(pfn, last_region_idx) pfn_valid(pfn)
+#endif /*CONFIG_HAVE_ARCH_PFN_VALID*/
-#define early_pfn_valid(pfn) pfn_valid(pfn)
void sparse_init(void);
#else
#define sparse_init() do {} while (0)
@@ -1288,7 +1294,7 @@ struct mminit_pfnnid_cache {
};
#ifndef early_pfn_valid
-#define early_pfn_valid(pfn) (1)
+#define early_pfn_valid(pfn, last_region_idx) (1)
#endif
void memory_present(int nid, unsigned long start, unsigned long end);
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 0bb0274..68aef71 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5484,8 +5484,8 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
if (context != MEMMAP_EARLY)
goto not_early;
- if (!early_pfn_valid(pfn)) {
#if (defined CONFIG_HAVE_MEMBLOCK) && (defined CONFIG_HAVE_ARCH_PFN_VALID)
+ if (!early_pfn_valid(pfn, &idx)) {
/*
* Skip to the pfn preceding the next valid one (or
* end_pfn), such that we hit a valid pfn (or end_pfn)
--
2.7.4
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PATCH v2 5/5] mm: page_alloc: reduce unnecessary binary search in early_pfn_valid()
2018-03-24 12:24 ` [PATCH v2 5/5] mm: page_alloc: reduce unnecessary binary search in early_pfn_valid() Jia He
@ 2018-03-25 10:48 ` kbuild test robot
0 siblings, 0 replies; 16+ messages in thread
From: kbuild test robot @ 2018-03-25 10:48 UTC (permalink / raw)
To: Jia He
Cc: kbuild-all, Andrew Morton, Michal Hocko, Catalin Marinas,
Mel Gorman, Will Deacon, Mark Rutland, Thomas Gleixner,
Ingo Molnar, H. Peter Anvin, Pavel Tatashin, Daniel Jordan,
AKASHI Takahiro, Gioh Kim, Steven Sistare, Daniel Vacek,
Eugeniu Rosca, Vlastimil Babka, linux-kernel, linux-mm,
James Morse, Ard Biesheuvel, Steve Capper, x86,
Greg Kroah-Hartman, Kate Stewart, Philippe Ombredanne,
Johannes Weiner, Kemi Wang, Petr Tesarik, YASUAKI ISHIMATSU,
Andrey Ryabinin, Nikolay Borisov, Jia He, Jia He
[-- Attachment #1: Type: text/plain, Size: 7866 bytes --]
Hi Jia,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on mmotm/master]
[also build test ERROR on next-20180323]
[cannot apply to v4.16-rc6]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Jia-He/optimize-memblock_next_valid_pfn-and-early_pfn_valid/20180325-175026
base: git://git.cmpxchg.org/linux-mmotm.git master
config: i386-randconfig-x013-201812 (attached as .config)
compiler: gcc-7 (Debian 7.3.0-1) 7.3.0
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
All errors (new ones prefixed by >>):
mm/page_alloc.c: In function 'memmap_init_zone':
>> mm/page_alloc.c:5499:4: error: continue statement not within a loop
continue;
^~~~~~~~
>> mm/page_alloc.c:5501:4: error: break statement not within loop or switch
break;
^~~~~
mm/page_alloc.c:5520:5: error: continue statement not within a loop
continue;
^~~~~~~~
mm/page_alloc.c:5462:6: warning: unused variable 'idx' [-Wunused-variable]
int idx = -1;
^~~
mm/page_alloc.c: At top level:
>> mm/page_alloc.c:5551:1: error: expected identifier or '(' before '}' token
}
^
vim +5499 mm/page_alloc.c
^1da177e4 Linus Torvalds 2005-04-16 5468
22b31eec6 Hugh Dickins 2009-01-06 5469 if (highest_memmap_pfn < end_pfn - 1)
22b31eec6 Hugh Dickins 2009-01-06 5470 highest_memmap_pfn = end_pfn - 1;
22b31eec6 Hugh Dickins 2009-01-06 5471
4b94ffdc4 Dan Williams 2016-01-15 5472 /*
4b94ffdc4 Dan Williams 2016-01-15 5473 * Honor reservation requested by the driver for this ZONE_DEVICE
4b94ffdc4 Dan Williams 2016-01-15 5474 * memory
4b94ffdc4 Dan Williams 2016-01-15 5475 */
4b94ffdc4 Dan Williams 2016-01-15 5476 if (altmap && start_pfn == altmap->base_pfn)
4b94ffdc4 Dan Williams 2016-01-15 5477 start_pfn += altmap->reserve;
4b94ffdc4 Dan Williams 2016-01-15 5478
cbe8dd4af Greg Ungerer 2006-01-12 5479 for (pfn = start_pfn; pfn < end_pfn; pfn++) {
a2f3aa025 Dave Hansen 2007-01-10 5480 /*
b72d0ffb5 Andrew Morton 2016-03-15 5481 * There can be holes in boot-time mem_map[]s handed to this
b72d0ffb5 Andrew Morton 2016-03-15 5482 * function. They do not exist on hotplugged memory.
a2f3aa025 Dave Hansen 2007-01-10 5483 */
b72d0ffb5 Andrew Morton 2016-03-15 5484 if (context != MEMMAP_EARLY)
b72d0ffb5 Andrew Morton 2016-03-15 5485 goto not_early;
b72d0ffb5 Andrew Morton 2016-03-15 5486
c0b211780 Jia He 2018-03-24 5487 #if (defined CONFIG_HAVE_MEMBLOCK) && (defined CONFIG_HAVE_ARCH_PFN_VALID)
94200be7f Jia He 2018-03-24 5488 if (!early_pfn_valid(pfn, &idx)) {
c0b211780 Jia He 2018-03-24 5489 /*
c0b211780 Jia He 2018-03-24 5490 * Skip to the pfn preceding the next valid one (or
c0b211780 Jia He 2018-03-24 5491 * end_pfn), such that we hit a valid pfn (or end_pfn)
c0b211780 Jia He 2018-03-24 5492 * on our next iteration of the loop.
c0b211780 Jia He 2018-03-24 5493 */
5ce6c7e68 Jia He 2018-03-24 5494 pfn = memblock_next_valid_pfn(pfn, &idx) - 1;
c0b211780 Jia He 2018-03-24 5495 #endif
d41dee369 Andy Whitcroft 2005-06-23 5496 continue;
c0b211780 Jia He 2018-03-24 5497 }
751679573 Andy Whitcroft 2006-10-21 5498 if (!early_pfn_in_nid(pfn, nid))
751679573 Andy Whitcroft 2006-10-21 @5499 continue;
b72d0ffb5 Andrew Morton 2016-03-15 5500 if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised))
3a80a7fa7 Mel Gorman 2015-06-30 @5501 break;
342332e6a Taku Izumi 2016-03-15 5502
342332e6a Taku Izumi 2016-03-15 5503 #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
342332e6a Taku Izumi 2016-03-15 5504 /*
b72d0ffb5 Andrew Morton 2016-03-15 5505 * Check given memblock attribute by firmware which can affect
b72d0ffb5 Andrew Morton 2016-03-15 5506 * kernel memory layout. If zone==ZONE_MOVABLE but memory is
b72d0ffb5 Andrew Morton 2016-03-15 5507 * mirrored, it's an overlapped memmap init. skip it.
342332e6a Taku Izumi 2016-03-15 5508 */
342332e6a Taku Izumi 2016-03-15 5509 if (mirrored_kernelcore && zone == ZONE_MOVABLE) {
b72d0ffb5 Andrew Morton 2016-03-15 5510 if (!r || pfn >= memblock_region_memory_end_pfn(r)) {
342332e6a Taku Izumi 2016-03-15 5511 for_each_memblock(memory, tmp)
342332e6a Taku Izumi 2016-03-15 5512 if (pfn < memblock_region_memory_end_pfn(tmp))
342332e6a Taku Izumi 2016-03-15 5513 break;
342332e6a Taku Izumi 2016-03-15 5514 r = tmp;
342332e6a Taku Izumi 2016-03-15 5515 }
342332e6a Taku Izumi 2016-03-15 5516 if (pfn >= memblock_region_memory_base_pfn(r) &&
342332e6a Taku Izumi 2016-03-15 5517 memblock_is_mirror(r)) {
342332e6a Taku Izumi 2016-03-15 5518 /* already initialized as NORMAL */
342332e6a Taku Izumi 2016-03-15 5519 pfn = memblock_region_memory_end_pfn(r);
342332e6a Taku Izumi 2016-03-15 @5520 continue;
342332e6a Taku Izumi 2016-03-15 5521 }
342332e6a Taku Izumi 2016-03-15 5522 }
342332e6a Taku Izumi 2016-03-15 5523 #endif
ac5d2539b Mel Gorman 2015-06-30 5524
b72d0ffb5 Andrew Morton 2016-03-15 5525 not_early:
d08f92e7d Pavel Tatashin 2018-03-23 5526 page = pfn_to_page(pfn);
d08f92e7d Pavel Tatashin 2018-03-23 5527 __init_single_page(page, pfn, zone, nid);
d08f92e7d Pavel Tatashin 2018-03-23 5528 if (context == MEMMAP_HOTPLUG)
d08f92e7d Pavel Tatashin 2018-03-23 5529 SetPageReserved(page);
d08f92e7d Pavel Tatashin 2018-03-23 5530
ac5d2539b Mel Gorman 2015-06-30 5531 /*
ac5d2539b Mel Gorman 2015-06-30 5532 * Mark the block movable so that blocks are reserved for
ac5d2539b Mel Gorman 2015-06-30 5533 * movable at startup. This will force kernel allocations
ac5d2539b Mel Gorman 2015-06-30 5534 * to reserve their blocks rather than leaking throughout
ac5d2539b Mel Gorman 2015-06-30 5535 * the address space during boot when many long-lived
974a786e6 Mel Gorman 2015-11-06 5536 * kernel allocations are made.
ac5d2539b Mel Gorman 2015-06-30 5537 *
ac5d2539b Mel Gorman 2015-06-30 5538 * bitmap is created for zone's valid pfn range. but memmap
ac5d2539b Mel Gorman 2015-06-30 5539 * can be created for invalid pages (for alignment)
ac5d2539b Mel Gorman 2015-06-30 5540 * check here not to call set_pageblock_migratetype() against
ac5d2539b Mel Gorman 2015-06-30 5541 * pfn out of zone.
9bb5a391f Michal Hocko 2018-01-31 5542 *
9bb5a391f Michal Hocko 2018-01-31 5543 * Please note that MEMMAP_HOTPLUG path doesn't clear memmap
9bb5a391f Michal Hocko 2018-01-31 5544 * because this is done early in sparse_add_one_section
ac5d2539b Mel Gorman 2015-06-30 5545 */
ac5d2539b Mel Gorman 2015-06-30 5546 if (!(pfn & (pageblock_nr_pages - 1))) {
ac5d2539b Mel Gorman 2015-06-30 5547 set_pageblock_migratetype(page, MIGRATE_MOVABLE);
9b6e63cbf Michal Hocko 2017-10-03 5548 cond_resched();
^1da177e4 Linus Torvalds 2005-04-16 5549 }
^1da177e4 Linus Torvalds 2005-04-16 5550 }
ac5d2539b Mel Gorman 2015-06-30 @5551 }
^1da177e4 Linus Torvalds 2005-04-16 5552
:::::: The code at line 5499 was first introduced by commit
:::::: 7516795739bd53175629b90fab0ad488d7a6a9f7 [PATCH] Reintroduce NODES_SPAN_OTHER_NODES for powerpc
:::::: TO: Andy Whitcroft <apw@shadowen.org>
:::::: CC: Linus Torvalds <torvalds@g5.osdl.org>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 25403 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH v2 5/5] mm: page_alloc: reduce unnecessary binary search in early_pfn_valid()
@ 2018-03-25 10:48 ` kbuild test robot
0 siblings, 0 replies; 16+ messages in thread
From: kbuild test robot @ 2018-03-25 10:48 UTC (permalink / raw)
To: Jia He
Cc: kbuild-all, Andrew Morton, Michal Hocko, Catalin Marinas,
Mel Gorman, Will Deacon, Mark Rutland, Thomas Gleixner,
Ingo Molnar, H. Peter Anvin, Pavel Tatashin, Daniel Jordan,
AKASHI Takahiro, Gioh Kim, Steven Sistare, Daniel Vacek,
Eugeniu Rosca, Vlastimil Babka, linux-kernel, linux-mm,
James Morse, Ard Biesheuvel, Steve Capper, x86,
Greg Kroah-Hartman, Kate Stewart, Philippe Ombredanne,
Johannes Weiner, Kemi Wang, Petr Tesarik, YASUAKI ISHIMATSU,
Andrey Ryabinin, Nikolay Borisov, Jia He
[-- Attachment #1: Type: text/plain, Size: 7866 bytes --]
Hi Jia,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on mmotm/master]
[also build test ERROR on next-20180323]
[cannot apply to v4.16-rc6]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Jia-He/optimize-memblock_next_valid_pfn-and-early_pfn_valid/20180325-175026
base: git://git.cmpxchg.org/linux-mmotm.git master
config: i386-randconfig-x013-201812 (attached as .config)
compiler: gcc-7 (Debian 7.3.0-1) 7.3.0
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
All errors (new ones prefixed by >>):
mm/page_alloc.c: In function 'memmap_init_zone':
>> mm/page_alloc.c:5499:4: error: continue statement not within a loop
continue;
^~~~~~~~
>> mm/page_alloc.c:5501:4: error: break statement not within loop or switch
break;
^~~~~
mm/page_alloc.c:5520:5: error: continue statement not within a loop
continue;
^~~~~~~~
mm/page_alloc.c:5462:6: warning: unused variable 'idx' [-Wunused-variable]
int idx = -1;
^~~
mm/page_alloc.c: At top level:
>> mm/page_alloc.c:5551:1: error: expected identifier or '(' before '}' token
}
^
vim +5499 mm/page_alloc.c
^1da177e4 Linus Torvalds 2005-04-16 5468
22b31eec6 Hugh Dickins 2009-01-06 5469 if (highest_memmap_pfn < end_pfn - 1)
22b31eec6 Hugh Dickins 2009-01-06 5470 highest_memmap_pfn = end_pfn - 1;
22b31eec6 Hugh Dickins 2009-01-06 5471
4b94ffdc4 Dan Williams 2016-01-15 5472 /*
4b94ffdc4 Dan Williams 2016-01-15 5473 * Honor reservation requested by the driver for this ZONE_DEVICE
4b94ffdc4 Dan Williams 2016-01-15 5474 * memory
4b94ffdc4 Dan Williams 2016-01-15 5475 */
4b94ffdc4 Dan Williams 2016-01-15 5476 if (altmap && start_pfn == altmap->base_pfn)
4b94ffdc4 Dan Williams 2016-01-15 5477 start_pfn += altmap->reserve;
4b94ffdc4 Dan Williams 2016-01-15 5478
cbe8dd4af Greg Ungerer 2006-01-12 5479 for (pfn = start_pfn; pfn < end_pfn; pfn++) {
a2f3aa025 Dave Hansen 2007-01-10 5480 /*
b72d0ffb5 Andrew Morton 2016-03-15 5481 * There can be holes in boot-time mem_map[]s handed to this
b72d0ffb5 Andrew Morton 2016-03-15 5482 * function. They do not exist on hotplugged memory.
a2f3aa025 Dave Hansen 2007-01-10 5483 */
b72d0ffb5 Andrew Morton 2016-03-15 5484 if (context != MEMMAP_EARLY)
b72d0ffb5 Andrew Morton 2016-03-15 5485 goto not_early;
b72d0ffb5 Andrew Morton 2016-03-15 5486
c0b211780 Jia He 2018-03-24 5487 #if (defined CONFIG_HAVE_MEMBLOCK) && (defined CONFIG_HAVE_ARCH_PFN_VALID)
94200be7f Jia He 2018-03-24 5488 if (!early_pfn_valid(pfn, &idx)) {
c0b211780 Jia He 2018-03-24 5489 /*
c0b211780 Jia He 2018-03-24 5490 * Skip to the pfn preceding the next valid one (or
c0b211780 Jia He 2018-03-24 5491 * end_pfn), such that we hit a valid pfn (or end_pfn)
c0b211780 Jia He 2018-03-24 5492 * on our next iteration of the loop.
c0b211780 Jia He 2018-03-24 5493 */
5ce6c7e68 Jia He 2018-03-24 5494 pfn = memblock_next_valid_pfn(pfn, &idx) - 1;
c0b211780 Jia He 2018-03-24 5495 #endif
d41dee369 Andy Whitcroft 2005-06-23 5496 continue;
c0b211780 Jia He 2018-03-24 5497 }
751679573 Andy Whitcroft 2006-10-21 5498 if (!early_pfn_in_nid(pfn, nid))
751679573 Andy Whitcroft 2006-10-21 @5499 continue;
b72d0ffb5 Andrew Morton 2016-03-15 5500 if (!update_defer_init(pgdat, pfn, end_pfn, &nr_initialised))
3a80a7fa7 Mel Gorman 2015-06-30 @5501 break;
342332e6a Taku Izumi 2016-03-15 5502
342332e6a Taku Izumi 2016-03-15 5503 #ifdef CONFIG_HAVE_MEMBLOCK_NODE_MAP
342332e6a Taku Izumi 2016-03-15 5504 /*
b72d0ffb5 Andrew Morton 2016-03-15 5505 * Check given memblock attribute by firmware which can affect
b72d0ffb5 Andrew Morton 2016-03-15 5506 * kernel memory layout. If zone==ZONE_MOVABLE but memory is
b72d0ffb5 Andrew Morton 2016-03-15 5507 * mirrored, it's an overlapped memmap init. skip it.
342332e6a Taku Izumi 2016-03-15 5508 */
342332e6a Taku Izumi 2016-03-15 5509 if (mirrored_kernelcore && zone == ZONE_MOVABLE) {
b72d0ffb5 Andrew Morton 2016-03-15 5510 if (!r || pfn >= memblock_region_memory_end_pfn(r)) {
342332e6a Taku Izumi 2016-03-15 5511 for_each_memblock(memory, tmp)
342332e6a Taku Izumi 2016-03-15 5512 if (pfn < memblock_region_memory_end_pfn(tmp))
342332e6a Taku Izumi 2016-03-15 5513 break;
342332e6a Taku Izumi 2016-03-15 5514 r = tmp;
342332e6a Taku Izumi 2016-03-15 5515 }
342332e6a Taku Izumi 2016-03-15 5516 if (pfn >= memblock_region_memory_base_pfn(r) &&
342332e6a Taku Izumi 2016-03-15 5517 memblock_is_mirror(r)) {
342332e6a Taku Izumi 2016-03-15 5518 /* already initialized as NORMAL */
342332e6a Taku Izumi 2016-03-15 5519 pfn = memblock_region_memory_end_pfn(r);
342332e6a Taku Izumi 2016-03-15 @5520 continue;
342332e6a Taku Izumi 2016-03-15 5521 }
342332e6a Taku Izumi 2016-03-15 5522 }
342332e6a Taku Izumi 2016-03-15 5523 #endif
ac5d2539b Mel Gorman 2015-06-30 5524
b72d0ffb5 Andrew Morton 2016-03-15 5525 not_early:
d08f92e7d Pavel Tatashin 2018-03-23 5526 page = pfn_to_page(pfn);
d08f92e7d Pavel Tatashin 2018-03-23 5527 __init_single_page(page, pfn, zone, nid);
d08f92e7d Pavel Tatashin 2018-03-23 5528 if (context == MEMMAP_HOTPLUG)
d08f92e7d Pavel Tatashin 2018-03-23 5529 SetPageReserved(page);
d08f92e7d Pavel Tatashin 2018-03-23 5530
ac5d2539b Mel Gorman 2015-06-30 5531 /*
ac5d2539b Mel Gorman 2015-06-30 5532 * Mark the block movable so that blocks are reserved for
ac5d2539b Mel Gorman 2015-06-30 5533 * movable at startup. This will force kernel allocations
ac5d2539b Mel Gorman 2015-06-30 5534 * to reserve their blocks rather than leaking throughout
ac5d2539b Mel Gorman 2015-06-30 5535 * the address space during boot when many long-lived
974a786e6 Mel Gorman 2015-11-06 5536 * kernel allocations are made.
ac5d2539b Mel Gorman 2015-06-30 5537 *
ac5d2539b Mel Gorman 2015-06-30 5538 * bitmap is created for zone's valid pfn range. but memmap
ac5d2539b Mel Gorman 2015-06-30 5539 * can be created for invalid pages (for alignment)
ac5d2539b Mel Gorman 2015-06-30 5540 * check here not to call set_pageblock_migratetype() against
ac5d2539b Mel Gorman 2015-06-30 5541 * pfn out of zone.
9bb5a391f Michal Hocko 2018-01-31 5542 *
9bb5a391f Michal Hocko 2018-01-31 5543 * Please note that MEMMAP_HOTPLUG path doesn't clear memmap
9bb5a391f Michal Hocko 2018-01-31 5544 * because this is done early in sparse_add_one_section
ac5d2539b Mel Gorman 2015-06-30 5545 */
ac5d2539b Mel Gorman 2015-06-30 5546 if (!(pfn & (pageblock_nr_pages - 1))) {
ac5d2539b Mel Gorman 2015-06-30 5547 set_pageblock_migratetype(page, MIGRATE_MOVABLE);
9b6e63cbf Michal Hocko 2017-10-03 5548 cond_resched();
^1da177e4 Linus Torvalds 2005-04-16 5549 }
^1da177e4 Linus Torvalds 2005-04-16 5550 }
ac5d2539b Mel Gorman 2015-06-30 @5551 }
^1da177e4 Linus Torvalds 2005-04-16 5552
:::::: The code at line 5499 was first introduced by commit
:::::: 7516795739bd53175629b90fab0ad488d7a6a9f7 [PATCH] Reintroduce NODES_SPAN_OTHER_NODES for powerpc
:::::: TO: Andy Whitcroft <apw@shadowen.org>
:::::: CC: Linus Torvalds <torvalds@g5.osdl.org>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 25403 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread