On Thu, Nov 16, 2017 at 9:31 PM, Nathan Fontenot <nfont@linux.vnet.ibm.com>
wrote:

>
>
> On 11/15/2017 11:37 PM, Bharata B Rao wrote:
> > On Fri, Oct 20, 2017 at 6:51 PM, Nathan Fontenot <
> nfont@linux.vnet.ibm.com <mailto:nfont@linux.vnet.ibm.com>> wrote:
> >
> >     This patch set provides a set of updates to de-couple the LMB
> information
> >     provided in the ibm,dynamic-memory device tree property from the
> device
> >     tree property format. A part of this patch series introduces a new
> >     device tree property format for dynamic memory, ibm-dynamic-meory-v2.
> >     By separating the device tree format from the information provided by
> >     the device tree property consumers of this information need not know
> >     what format is currently being used and provide multiple parsing
> routines
> >     for the information.
> >
> >     The first two patches update the of_get_assoc_arrays() and
> >     of_get_usable_memory() routines to look up the device node for the
> >     properties they parse. This is needed because the calling routines
> for
> >     these two functions will not have the device node to pass in in
> >     subsequent patches.
> >
> >     The third patch adds a new kernel structure, struct drmem_lmb, that
> >     is used to represent each of the possible LMBs specified in the
> >     ibm,dynamic-memory* device tree properties. The patch adds code
> >     to parse the property and build the LMB array data, and updates
> prom.c
> >     to use this new data structure instead of parsing the device tree
> directly.
> >
> >     The fourth and fifth patches update the numa and pseries hotplug code
> >     respectively to use the new LMB array data instead of parsing the
> >     device tree directly.
> >
> >     The sixth patch moves the of_drconf_cell struct to drmem.h where it
> >     fits better than prom.h
> >
> >     The seventh patch introduces support for the ibm,dynamic-memory-v2
> >     property format by updating the new drmem.c code to be able to parse
> >     and create this new device tree format.
> >
> >     The last patch in the series updates the architecture vector to
> indicate
> >     support for ibm,dynamic-memory-v2.
> >
> >
> > Here we are consolidating LMBs into LMB sets but still end up working
> with individual LMBs during hotplug. Can we instead start working with LMB
> sets together during hotplug ? In other words
>
> In a sense we do do this when handling memory DLPAR indexed-count
> requests. This takes a starting
> drc index for a LMB and adds/removes the following <count> contiguous
> LMBs. This operation is
> all-or-nothing, if any LMB fails to add/remove we revert back to the
> original state.
>

I am aware of count-indexed and we do use it for memory hotplug/unplug for
KVM on Power. However the RTAS and configure-connector calls there are
still per-LMB.


> Thi isn't exactly what you're asking for but...
> >
> > - The RTAS calls involved during DRC acquire stage can be done only once
> per LMB set.
> > - One configure-connector call for the entire LMB set.
>
> these two interfaces work on a single drc index, not a set of drc indexes.
> Working on a set
> of LMBs would require extending the current rtas calls or creating new
> ones.
>

Yes.


>
> One thing we can look into doing for indexed-count requests is to perform
> each of the
> steps for all LMBs in the set at once, i.e. make the acquire call for
> LMBs, then make the
> configure-connector calls for all the LMBs...
>

That is what I am hinting at to check the feasibility of such a mechanism.
Given that all the LMBs of the set are supposed to have similar attributes
(like node associativity etc), it makes sense to have a single DRC acquire
call and single configure-connector call for the entire set.


> The only drawback is this approach would make handling failures and
> backing out of the
> updates a bit messier, but I've never really thought that optimizing for
> the failure
> case to be as important.
>

Yes, error recovery can be messy given that we have multiple calls under
DRC acquire call (get-sensor-state and set-indicator).

BTW I thought this reorganization involving ibm,drc-info and
ibm,dynamic-memory-v2 was for representing and hotplugging huge amounts of
memory efficiently and quickly. So you have not yet found the per-LMB calls
to be hurting when huge amount of memory is involved ?

Regards,
Bharata.