* Re: [PATCH v5] pseries/drmem: update LMBs after LPM
2021-05-17 9:06 [PATCH v5] pseries/drmem: update LMBs after LPM Laurent Dufour
@ 2021-07-01 8:36 ` Laurent Dufour
2021-07-19 9:10 ` Laurent Dufour
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Laurent Dufour @ 2021-07-01 8:36 UTC (permalink / raw)
To: mpe, benh, paulus
Cc: Nathan Lynch, Aneesh Kumar K . V, linuxppc-dev, linux-kernel,
Tyrel Datwyler
Hi Michael,
Do you mind taking this patch of 5.14?
Thanks,
Laurent.
Le 17/05/2021 à 11:06, Laurent Dufour a écrit :
> After a LPM, the device tree node ibm,dynamic-reconfiguration-memory may be
> updated by the hypervisor in the case the NUMA topology of the LPAR's
> memory is updated.
>
> This is handled by the kernel, but the memory's node is not updated because
> there is no way to move a memory block between nodes from the Linux kernel
> point of view.
>
> If later a memory block is added or removed, drmem_update_dt() is called
> and it is overwriting the DT node ibm,dynamic-reconfiguration-memory to
> match the added or removed LMB. But the LMB's associativity node has not
> been updated after the DT node update and thus the node is overwritten by
> the Linux's topology instead of the hypervisor one.
>
> Introduce a hook called when the ibm,dynamic-reconfiguration-memory node is
> updated to force an update of the LMB's associativity. However, ignore the
> call to that hook when the update has been triggered by drmem_update_dt().
> Because, in that case, the LMB tree has been used to set the DT property
> and thus it doesn't need to be updated back. Since drmem_update_dt() is
> called under the protection of the device_hotplug_lock and the hook is
> called in the same context, use a simple boolean variable to detect that
> call.
>
> Cc: Nathan Lynch <nathanl@linux.ibm.com>
> Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Cc: Tyrel Datwyler <tyreld@linux.ibm.com>
> Signed-off-by: Laurent Dufour <ldufour@linux.ibm.com>
> ---
>
> V5:
> - Reword the commit's description to address Nathan's comments.
> V4:
> - Prevent the LMB to be updated back in the case the request came from the
> LMB tree's update.
> V3:
> - Check rd->dn->name instead of rd->dn->full_name
> V2:
> - Take Tyrel's idea to rely on OF_RECONFIG_UPDATE_PROPERTY instead of
> introducing a new hook mechanism.
> ---
> arch/powerpc/include/asm/drmem.h | 1 +
> arch/powerpc/mm/drmem.c | 46 +++++++++++++++++++
> .../platforms/pseries/hotplug-memory.c | 4 ++
> 3 files changed, 51 insertions(+)
>
> diff --git a/arch/powerpc/include/asm/drmem.h b/arch/powerpc/include/asm/drmem.h
> index bf2402fed3e0..4265d5e95c2c 100644
> --- a/arch/powerpc/include/asm/drmem.h
> +++ b/arch/powerpc/include/asm/drmem.h
> @@ -111,6 +111,7 @@ int drmem_update_dt(void);
> int __init
> walk_drmem_lmbs_early(unsigned long node, void *data,
> int (*func)(struct drmem_lmb *, const __be32 **, void *));
> +void drmem_update_lmbs(struct property *prop);
> #endif
>
> static inline void invalidate_lmb_associativity_index(struct drmem_lmb *lmb)
> diff --git a/arch/powerpc/mm/drmem.c b/arch/powerpc/mm/drmem.c
> index 9af3832c9d8d..22197b18d85e 100644
> --- a/arch/powerpc/mm/drmem.c
> +++ b/arch/powerpc/mm/drmem.c
> @@ -18,6 +18,7 @@ static int n_root_addr_cells, n_root_size_cells;
>
> static struct drmem_lmb_info __drmem_info;
> struct drmem_lmb_info *drmem_info = &__drmem_info;
> +static bool in_drmem_update;
>
> u64 drmem_lmb_memory_max(void)
> {
> @@ -178,6 +179,11 @@ int drmem_update_dt(void)
> if (!memory)
> return -1;
>
> + /*
> + * Set in_drmem_update to prevent the notifier callback to process the
> + * DT property back since the change is coming from the LMB tree.
> + */
> + in_drmem_update = true;
> prop = of_find_property(memory, "ibm,dynamic-memory", NULL);
> if (prop) {
> rc = drmem_update_dt_v1(memory, prop);
> @@ -186,6 +192,7 @@ int drmem_update_dt(void)
> if (prop)
> rc = drmem_update_dt_v2(memory, prop);
> }
> + in_drmem_update = false;
>
> of_node_put(memory);
> return rc;
> @@ -307,6 +314,45 @@ int __init walk_drmem_lmbs_early(unsigned long node, void *data,
> return ret;
> }
>
> +/*
> + * Update the LMB associativity index.
> + */
> +static int update_lmb(struct drmem_lmb *updated_lmb,
> + __maybe_unused const __be32 **usm,
> + __maybe_unused void *data)
> +{
> + struct drmem_lmb *lmb;
> +
> + for_each_drmem_lmb(lmb) {
> + if (lmb->drc_index != updated_lmb->drc_index)
> + continue;
> +
> + lmb->aa_index = updated_lmb->aa_index;
> + break;
> + }
> + return 0;
> +}
> +
> +/*
> + * Update the LMB associativity index.
> + *
> + * This needs to be called when the hypervisor is updating the
> + * dynamic-reconfiguration-memory node property.
> + */
> +void drmem_update_lmbs(struct property *prop)
> +{
> + /*
> + * Don't update the LMBs if triggered by the update done in
> + * drmem_update_dt(), the LMB values have been used to the update the DT
> + * property in that case.
> + */
> + if (in_drmem_update)
> + return;
> + if (!strcmp(prop->name, "ibm,dynamic-memory"))
> + __walk_drmem_v1_lmbs(prop->value, NULL, NULL, update_lmb);
> + else if (!strcmp(prop->name, "ibm,dynamic-memory-v2"))
> + __walk_drmem_v2_lmbs(prop->value, NULL, NULL, update_lmb);
> +}
> #endif
>
> static int init_drmem_lmb_size(struct device_node *dn)
> diff --git a/arch/powerpc/platforms/pseries/hotplug-memory.c b/arch/powerpc/platforms/pseries/hotplug-memory.c
> index 8377f1f7c78e..672ffbee2e78 100644
> --- a/arch/powerpc/platforms/pseries/hotplug-memory.c
> +++ b/arch/powerpc/platforms/pseries/hotplug-memory.c
> @@ -949,6 +949,10 @@ static int pseries_memory_notifier(struct notifier_block *nb,
> case OF_RECONFIG_DETACH_NODE:
> err = pseries_remove_mem_node(rd->dn);
> break;
> + case OF_RECONFIG_UPDATE_PROPERTY:
> + if (!strcmp(rd->dn->name,
> + "ibm,dynamic-reconfiguration-memory"))
> + drmem_update_lmbs(rd->prop);
> }
> return notifier_from_errno(err);
> }
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v5] pseries/drmem: update LMBs after LPM
2021-05-17 9:06 [PATCH v5] pseries/drmem: update LMBs after LPM Laurent Dufour
2021-07-01 8:36 ` Laurent Dufour
@ 2021-07-19 9:10 ` Laurent Dufour
2021-08-03 17:32 ` Nathan Lynch
2021-08-18 13:38 ` Michael Ellerman
3 siblings, 0 replies; 6+ messages in thread
From: Laurent Dufour @ 2021-07-19 9:10 UTC (permalink / raw)
To: mpe, benh, paulus
Cc: Nathan Lynch, Aneesh Kumar K . V, linuxppc-dev, linux-kernel,
Tyrel Datwyler
Hi Michael,
Is there a way to get that patch in 5.14?
Thanks,
Laurent.
Le 17/05/2021 à 11:06, Laurent Dufour a écrit :
> After a LPM, the device tree node ibm,dynamic-reconfiguration-memory may be
> updated by the hypervisor in the case the NUMA topology of the LPAR's
> memory is updated.
>
> This is handled by the kernel, but the memory's node is not updated because
> there is no way to move a memory block between nodes from the Linux kernel
> point of view.
>
> If later a memory block is added or removed, drmem_update_dt() is called
> and it is overwriting the DT node ibm,dynamic-reconfiguration-memory to
> match the added or removed LMB. But the LMB's associativity node has not
> been updated after the DT node update and thus the node is overwritten by
> the Linux's topology instead of the hypervisor one.
>
> Introduce a hook called when the ibm,dynamic-reconfiguration-memory node is
> updated to force an update of the LMB's associativity. However, ignore the
> call to that hook when the update has been triggered by drmem_update_dt().
> Because, in that case, the LMB tree has been used to set the DT property
> and thus it doesn't need to be updated back. Since drmem_update_dt() is
> called under the protection of the device_hotplug_lock and the hook is
> called in the same context, use a simple boolean variable to detect that
> call.
>
> Cc: Nathan Lynch <nathanl@linux.ibm.com>
> Cc: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Cc: Tyrel Datwyler <tyreld@linux.ibm.com>
> Signed-off-by: Laurent Dufour <ldufour@linux.ibm.com>
> ---
>
> V5:
> - Reword the commit's description to address Nathan's comments.
> V4:
> - Prevent the LMB to be updated back in the case the request came from the
> LMB tree's update.
> V3:
> - Check rd->dn->name instead of rd->dn->full_name
> V2:
> - Take Tyrel's idea to rely on OF_RECONFIG_UPDATE_PROPERTY instead of
> introducing a new hook mechanism.
> ---
> arch/powerpc/include/asm/drmem.h | 1 +
> arch/powerpc/mm/drmem.c | 46 +++++++++++++++++++
> .../platforms/pseries/hotplug-memory.c | 4 ++
> 3 files changed, 51 insertions(+)
>
> diff --git a/arch/powerpc/include/asm/drmem.h b/arch/powerpc/include/asm/drmem.h
> index bf2402fed3e0..4265d5e95c2c 100644
> --- a/arch/powerpc/include/asm/drmem.h
> +++ b/arch/powerpc/include/asm/drmem.h
> @@ -111,6 +111,7 @@ int drmem_update_dt(void);
> int __init
> walk_drmem_lmbs_early(unsigned long node, void *data,
> int (*func)(struct drmem_lmb *, const __be32 **, void *));
> +void drmem_update_lmbs(struct property *prop);
> #endif
>
> static inline void invalidate_lmb_associativity_index(struct drmem_lmb *lmb)
> diff --git a/arch/powerpc/mm/drmem.c b/arch/powerpc/mm/drmem.c
> index 9af3832c9d8d..22197b18d85e 100644
> --- a/arch/powerpc/mm/drmem.c
> +++ b/arch/powerpc/mm/drmem.c
> @@ -18,6 +18,7 @@ static int n_root_addr_cells, n_root_size_cells;
>
> static struct drmem_lmb_info __drmem_info;
> struct drmem_lmb_info *drmem_info = &__drmem_info;
> +static bool in_drmem_update;
>
> u64 drmem_lmb_memory_max(void)
> {
> @@ -178,6 +179,11 @@ int drmem_update_dt(void)
> if (!memory)
> return -1;
>
> + /*
> + * Set in_drmem_update to prevent the notifier callback to process the
> + * DT property back since the change is coming from the LMB tree.
> + */
> + in_drmem_update = true;
> prop = of_find_property(memory, "ibm,dynamic-memory", NULL);
> if (prop) {
> rc = drmem_update_dt_v1(memory, prop);
> @@ -186,6 +192,7 @@ int drmem_update_dt(void)
> if (prop)
> rc = drmem_update_dt_v2(memory, prop);
> }
> + in_drmem_update = false;
>
> of_node_put(memory);
> return rc;
> @@ -307,6 +314,45 @@ int __init walk_drmem_lmbs_early(unsigned long node, void *data,
> return ret;
> }
>
> +/*
> + * Update the LMB associativity index.
> + */
> +static int update_lmb(struct drmem_lmb *updated_lmb,
> + __maybe_unused const __be32 **usm,
> + __maybe_unused void *data)
> +{
> + struct drmem_lmb *lmb;
> +
> + for_each_drmem_lmb(lmb) {
> + if (lmb->drc_index != updated_lmb->drc_index)
> + continue;
> +
> + lmb->aa_index = updated_lmb->aa_index;
> + break;
> + }
> + return 0;
> +}
> +
> +/*
> + * Update the LMB associativity index.
> + *
> + * This needs to be called when the hypervisor is updating the
> + * dynamic-reconfiguration-memory node property.
> + */
> +void drmem_update_lmbs(struct property *prop)
> +{
> + /*
> + * Don't update the LMBs if triggered by the update done in
> + * drmem_update_dt(), the LMB values have been used to the update the DT
> + * property in that case.
> + */
> + if (in_drmem_update)
> + return;
> + if (!strcmp(prop->name, "ibm,dynamic-memory"))
> + __walk_drmem_v1_lmbs(prop->value, NULL, NULL, update_lmb);
> + else if (!strcmp(prop->name, "ibm,dynamic-memory-v2"))
> + __walk_drmem_v2_lmbs(prop->value, NULL, NULL, update_lmb);
> +}
> #endif
>
> static int init_drmem_lmb_size(struct device_node *dn)
> diff --git a/arch/powerpc/platforms/pseries/hotplug-memory.c b/arch/powerpc/platforms/pseries/hotplug-memory.c
> index 8377f1f7c78e..672ffbee2e78 100644
> --- a/arch/powerpc/platforms/pseries/hotplug-memory.c
> +++ b/arch/powerpc/platforms/pseries/hotplug-memory.c
> @@ -949,6 +949,10 @@ static int pseries_memory_notifier(struct notifier_block *nb,
> case OF_RECONFIG_DETACH_NODE:
> err = pseries_remove_mem_node(rd->dn);
> break;
> + case OF_RECONFIG_UPDATE_PROPERTY:
> + if (!strcmp(rd->dn->name,
> + "ibm,dynamic-reconfiguration-memory"))
> + drmem_update_lmbs(rd->prop);
> }
> return notifier_from_errno(err);
> }
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v5] pseries/drmem: update LMBs after LPM
2021-05-17 9:06 [PATCH v5] pseries/drmem: update LMBs after LPM Laurent Dufour
2021-07-01 8:36 ` Laurent Dufour
2021-07-19 9:10 ` Laurent Dufour
@ 2021-08-03 17:32 ` Nathan Lynch
2021-08-03 17:34 ` Laurent Dufour
2021-08-18 13:38 ` Michael Ellerman
3 siblings, 1 reply; 6+ messages in thread
From: Nathan Lynch @ 2021-08-03 17:32 UTC (permalink / raw)
To: Laurent Dufour, mpe, benh, paulus
Cc: linuxppc-dev, linux-kernel, Aneesh Kumar K . V, Tyrel Datwyler
Laurent Dufour <ldufour@linux.ibm.com> writes:
> V5:
> - Reword the commit's description to address Nathan's comments.
Thanks. Still don't like the global variable usage but:
Reviewed-by: Nathan Lynch <nathanl@linux.ibm.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v5] pseries/drmem: update LMBs after LPM
2021-08-03 17:32 ` Nathan Lynch
@ 2021-08-03 17:34 ` Laurent Dufour
0 siblings, 0 replies; 6+ messages in thread
From: Laurent Dufour @ 2021-08-03 17:34 UTC (permalink / raw)
To: Nathan Lynch, mpe, benh, paulus
Cc: linuxppc-dev, linux-kernel, Aneesh Kumar K . V, Tyrel Datwyler
Le 03/08/2021 à 19:32, Nathan Lynch a écrit :
> Laurent Dufour <ldufour@linux.ibm.com> writes:
>> V5:
>> - Reword the commit's description to address Nathan's comments.
>
> Thanks. Still don't like the global variable usage but:
>
> Reviewed-by: Nathan Lynch <nathanl@linux.ibm.com>
>
Thanks Nathan,
I don't like either the global variable usage but I can't see any smarter way to
achieve that.
Laurent.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH v5] pseries/drmem: update LMBs after LPM
2021-05-17 9:06 [PATCH v5] pseries/drmem: update LMBs after LPM Laurent Dufour
` (2 preceding siblings ...)
2021-08-03 17:32 ` Nathan Lynch
@ 2021-08-18 13:38 ` Michael Ellerman
3 siblings, 0 replies; 6+ messages in thread
From: Michael Ellerman @ 2021-08-18 13:38 UTC (permalink / raw)
To: paulus, benh, mpe, Laurent Dufour
Cc: Tyrel Datwyler, Aneesh Kumar K . V, linux-kernel, Nathan Lynch,
linuxppc-dev
On Mon, 17 May 2021 11:06:06 +0200, Laurent Dufour wrote:
> After a LPM, the device tree node ibm,dynamic-reconfiguration-memory may be
> updated by the hypervisor in the case the NUMA topology of the LPAR's
> memory is updated.
>
> This is handled by the kernel, but the memory's node is not updated because
> there is no way to move a memory block between nodes from the Linux kernel
> point of view.
>
> [...]
Applied to powerpc/next.
[1/1] pseries/drmem: update LMBs after LPM
https://git.kernel.org/powerpc/c/d144f4d5a8a804133d20ff311d7be70bcdbfaac2
cheers
^ permalink raw reply [flat|nested] 6+ messages in thread