* [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support
@ 2016-10-26 21:59 Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 1/6] spapr: update spapr hotplug documentation Michael Roth
` (6 more replies)
0 siblings, 7 replies; 8+ messages in thread
From: Michael Roth @ 2016-10-26 21:59 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-ppc, david, bharata, nfont, jallen
This series is based on David's ppc-for-2.8 branch, and is also available from:
https://github.com/mdroth/qemu/commits/spapr-hotplug-event-update
Changes since v1:
* Rebased on latest ppc-for-2.8
* Dropped patches relating to option vector manipulation (now merged)
* Moved patch introducing modern-hotplug-events option to come after
that actual implementation of modern-hotplug-events support. (David)
* Removed uneeded source name string from sPAPREventSource in favor of
manual lookup of the name when creating DT. (David)
* Don't split function return signatures across lines (David)
* Add comment to clarify hotplug events being enabled regardless
of CAS-negotiation.
* Avoid unecessary initialization of hotplug_ctrl in
spapr_lmb_release. (David)
Changes since RFC:
* Rebased on latest ppc-for-2.8 (with DT refactoring)
* Submitting as v1 now that PAPR Hotplug ACR is committed
* address Patchew warnings
* add comments to clarify spapr->ov5/ov5_cas usage. (David)
* revise comment to clarify intent when setting spapr->ov5
OV5_HP_EVT bit. (Bharata)
* drop internal usage of spapr_ovec_from_bitmap() in favor of
directly assigning bitmap to sPAPROptionVector instances. (David)
* standardize meaning of 'vector_len' variable through spapr_ovec_*
functions to be the byte-wise length of option vectors entries,
and not including the preceeding length byte itself. (David)
* fix spapr_ovec_populate_dt() to parse up to OV_MAXBITS bits
rather than OV_MAXBITS - 1. (David)
* fix spapr_ovec_populate_dt() encode the minimum of 1 option
vector byte instead of the max of OV_MAXBYTES in cases where
no option bits are set. (David)
* add some comments to spapr_ovec_populate_dt() to clarify what
is being encoded into length byte of ibm,architecture-vec-5
* switch 'legacy-hotplug-events' option to
'modern-hotplug-events' (David)
* modify rtas_event_log_to_source() to check for OV5_HP_EVT
option rather than relying on whether the hotplug source is
specifically enabled. Assert the latter in cases where
OV5_HP_EVT is set. (Bharata)
* drop global EventSource list in favor of an sPAPREventSource
list field within sPAPRMachineState (David)
* add CPU unplug hook in mc->unplug_request (Bharata)
Patches 1-4 add support for an updated event format for hotplug events,
which includes a new way to specify a range of DRCs/LMBs to hotplug/unplug
using a starting position and count, which is necessary for memory unplug.
Patches 5-6 add support for memory unplug using the new event format.
In addition to kernel 4.8 or later, there are a number of patches required
to enable support on the guest kernel side. I've including the minimum set
of patches in my branch here:
https://github.com/mdroth/linux/commits/spapr-hotplug-event-update
*powerpc/pseries: advertise Hot Plug Event support to firmware
powerpc/pseries: Implement indexed-count hotplug memory remove
powerpc/pseries: Implement indexed-count hotplug memory add
Note that there is currently an issue that arises when attempting to
offline an LMB that was onlined using a guest kernel's auto-onlining
mechanism, which can prevent full completion of memory unplug requests.
This is being investigated, but for the purposes of testing this can
be worked around currently by disabling auto-onlining in guests via:
"echo offline >/sys/devices/system/memory/auto_online_blocks"
and instead onlining the blocks manually or via udev.
docs/specs/ppc-spapr-hotplug.txt | 55 +++++++++++++++++++++++++++------
hw/ppc/spapr.c | 178 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
hw/ppc/spapr_drc.c | 17 +++++++++++
hw/ppc/spapr_events.c | 278 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------
include/hw/ppc/spapr.h | 10 ++++--
include/hw/ppc/spapr_ovec.h | 1 +
6 files changed, 469 insertions(+), 70 deletions(-)
^ permalink raw reply [flat|nested] 8+ messages in thread
* [Qemu-devel] [PATCH v2 1/6] spapr: update spapr hotplug documentation
2016-10-26 21:59 [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support Michael Roth
@ 2016-10-26 21:59 ` Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 2/6] spapr_events: add support for dedicated hotplug event source Michael Roth
` (5 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Michael Roth @ 2016-10-26 21:59 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-ppc, david, bharata, nfont, jallen
This updates the existing documentation to reflect recent updates to
the hotplug event structure, which are in draft form but slated
for inclusion in PAPR/LoPAPR.
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
---
docs/specs/ppc-spapr-hotplug.txt | 55 +++++++++++++++++++++++++++++++++-------
1 file changed, 46 insertions(+), 9 deletions(-)
diff --git a/docs/specs/ppc-spapr-hotplug.txt b/docs/specs/ppc-spapr-hotplug.txt
index 631b0ca..f57e2a0 100644
--- a/docs/specs/ppc-spapr-hotplug.txt
+++ b/docs/specs/ppc-spapr-hotplug.txt
@@ -233,12 +233,27 @@ tools by host-level management such as an HMC. This level of management is not
applicable to PowerKVM, hence the reason for extending the notification
framework to support hotplug events.
-Note that these events are not yet formally part of the PAPR+ specification,
-but support for this format has already been implemented in DR-related
-guest tools such as powerpc-utils/librtas, as well as kernel patches that have
-been submitted to handle in-kernel processing of memory/cpu-related hotplug
-events[1], and is planned for formal inclusion is PAPR+ specification. The
-hotplug-specific payload is QEMU implemented as follows (with all values
+The format for these EPOW-signalled events is described below under
+"hotplug/unplug event structure". Note that these events are not
+formally part of the PAPR+ specification, and have been superseded by a
+newer format, also described below under "hotplug/unplug event structure",
+and so are now deemed a "legacy" format. The formats are similar, but the
+"modern" format contains additional fields/flags, which are denoted for the
+purposes of this documentation with "#ifdef GUEST_SUPPORTS_MODERN" guards.
+
+QEMU should assume support only for "legacy" fields/flags unless the guest
+advertises support for the "modern" format via ibm,client-architecture-support
+hcall by setting byte 5, bit 6 of it's ibm,architecture-vec-5 option vector
+structure (as described by LoPAPR v11, B.6.2.3). As with "legacy" format events,
+"modern" format events are surfaced to the guest via check-exception RTAS calls,
+but use a dedicated event source to signal the guest. This event source is
+advertised to the guest by the addition of a "hot-plug-events" node under
+"/event-sources" node of the guest's device tree using the standard format
+described in LoPAPR v11, B.6.12.1.
+
+== hotplug/unplug event structure ==
+
+The hotplug-specific payload in QEMU is implemented as follows (with all values
encoded in big-endian format):
struct rtas_event_log_v6_hp {
@@ -263,14 +278,23 @@ struct rtas_event_log_v6_hp {
#define RTAS_LOG_V6_HP_ACTION_ADD 1
#define RTAS_LOG_V6_HP_ACTION_REMOVE 2
uint8_t hotplug_action; /* action (add/remove) */
-#define RTAS_LOG_V6_HP_ID_DRC_NAME 1
-#define RTAS_LOG_V6_HP_ID_DRC_INDEX 2
-#define RTAS_LOG_V6_HP_ID_DRC_COUNT 3
+#define RTAS_LOG_V6_HP_ID_DRC_NAME 1
+#define RTAS_LOG_V6_HP_ID_DRC_INDEX 2
+#define RTAS_LOG_V6_HP_ID_DRC_COUNT 3
+#ifdef GUEST_SUPPORTS_MODERN
+#define RTAS_LOG_V6_HP_ID_DRC_COUNT_INDEXED 4
+#endif
uint8_t hotplug_identifier; /* type of the resource identifier,
* which serves as the discriminator
* for the 'drc' union field below
*/
+#ifdef GUEST_SUPPORTS_MODERN
+ uint8_t capabilities; /* capability flags, currently unused
+ * by QEMU
+ */
+#else
uint8_t reserved;
+#endif
union {
uint32_t index; /* DRC index of resource to take action
* on
@@ -278,6 +302,19 @@ struct rtas_event_log_v6_hp {
uint32_t count; /* number of DR resources to take
* action on (guest chooses which)
*/
+#ifdef GUEST_SUPPORTS_MODERN
+ struct {
+ uint32_t count; /* number of DR resources to take
+ * action on
+ */
+ uint32_t index; /* DRC index of first resource to take
+ * action on. guest will take action
+ * on DRC index <index> through
+ * DRC index <index + count - 1> in
+ * sequential order
+ */
+ } count_indexed;
+#endif
char name[1]; /* string representing the name of the
* DRC to take action on
*/
--
1.9.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Qemu-devel] [PATCH v2 2/6] spapr_events: add support for dedicated hotplug event source
2016-10-26 21:59 [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 1/6] spapr: update spapr hotplug documentation Michael Roth
@ 2016-10-26 21:59 ` Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 3/6] spapr: add hotplug interrupt machine options Michael Roth
` (4 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Michael Roth @ 2016-10-26 21:59 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-ppc, david, bharata, nfont, jallen
Hotplug events were previously delivered using an EPOW interrupt
and were queued by linux guests into a circular buffer. For traditional
EPOW events like shutdown/resets, this isn't an issue, but for hotplug
events there are cases where this buffer can be exhausted, resulting
in the loss of hotplug events, resets, etc.
Newer-style hotplug event are delivered using a dedicated event source.
We enable this in supported guests by adding standard an additional
event source in the guest device-tree via /event-sources, and, if
the guest advertises support for the newer-style hotplug events,
using the corresponding interrupt to signal the available of
hotplug/unplug events.
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
hw/ppc/spapr.c | 9 +-
hw/ppc/spapr_events.c | 202 ++++++++++++++++++++++++++++++++++++--------
include/hw/ppc/spapr.h | 6 +-
include/hw/ppc/spapr_ovec.h | 1 +
4 files changed, 177 insertions(+), 41 deletions(-)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 0b3820b..9ddf2ff 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -973,7 +973,7 @@ static void *spapr_build_fdt(sPAPRMachineState *spapr,
}
/* /event-sources */
- spapr_dt_events(fdt, spapr->check_exception_irq);
+ spapr_dt_events(spapr, fdt);
/* /rtas */
spapr_dt_rtas(spapr, fdt);
@@ -1789,6 +1789,11 @@ static void ppc_spapr_init(MachineState *machine)
spapr_ovec_set(spapr->ov5, OV5_FORM1_AFFINITY);
+ /* advertise support for dedicated HP event source to guests */
+ if (spapr->use_hotplug_event_source) {
+ spapr_ovec_set(spapr->ov5, OV5_HP_EVT);
+ }
+
/* init CPUs */
if (machine->cpu_model == NULL) {
machine->cpu_model = kvm_enabled() ? "host" : smc->tcg_default_cpu;
@@ -1912,7 +1917,7 @@ static void ppc_spapr_init(MachineState *machine)
}
g_free(filename);
- /* Set up EPOW events infrastructure */
+ /* Set up RTAS event infrastructure */
spapr_events_init(spapr);
/* Set up the RTC RTAS interfaces */
diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
index 89aa5a7..9b0bd41 100644
--- a/hw/ppc/spapr_events.c
+++ b/hw/ppc/spapr_events.c
@@ -40,6 +40,7 @@
#include "hw/ppc/spapr_drc.h"
#include "qemu/help_option.h"
#include "qemu/bcd.h"
+#include "hw/ppc/spapr_ovec.h"
#include <libfdt.h>
struct rtas_error_log {
@@ -206,27 +207,132 @@ struct hp_log_full {
struct rtas_event_log_v6_hp hp;
} QEMU_PACKED;
-#define EVENT_MASK_INTERNAL_ERRORS 0x80000000
-#define EVENT_MASK_EPOW 0x40000000
-#define EVENT_MASK_HOTPLUG 0x10000000
-#define EVENT_MASK_IO 0x08000000
+typedef enum EventClass {
+ EVENT_CLASS_INTERNAL_ERRORS = 0,
+ EVENT_CLASS_EPOW = 1,
+ EVENT_CLASS_RESERVED = 2,
+ EVENT_CLASS_HOT_PLUG = 3,
+ EVENT_CLASS_IO = 4,
+ EVENT_CLASS_MAX
+} EventClassIndex;
+#define EVENT_CLASS_MASK(index) (1 << (31 - index))
+
+static const char * const event_names[EVENT_CLASS_MAX] = {
+ [EVENT_CLASS_INTERNAL_ERRORS] = "internal-errors",
+ [EVENT_CLASS_EPOW] = "epow-events",
+ [EVENT_CLASS_HOT_PLUG] = "hot-plug-events",
+ [EVENT_CLASS_IO] = "ibm,io-events",
+};
+
+struct sPAPREventSource {
+ int irq;
+ uint32_t mask;
+ bool enabled;
+};
+
+static sPAPREventSource *spapr_event_sources_new(void)
+{
+ return g_new0(sPAPREventSource, EVENT_CLASS_MAX);
+}
+
+static void spapr_event_sources_register(sPAPREventSource *event_sources,
+ EventClassIndex index, int irq)
+{
+ /* we only support 1 irq per event class at the moment */
+ g_assert(event_sources);
+ g_assert(!event_sources[index].enabled);
+ event_sources[index].irq = irq;
+ event_sources[index].mask = EVENT_CLASS_MASK(index);
+ event_sources[index].enabled = true;
+}
+
+static const sPAPREventSource *
+spapr_event_sources_get_source(sPAPREventSource *event_sources,
+ EventClassIndex index)
+{
+ g_assert(index < EVENT_CLASS_MAX);
+ g_assert(event_sources);
+
+ return &event_sources[index];
+}
-void spapr_dt_events(void *fdt, uint32_t check_exception_irq)
+void spapr_dt_events(sPAPRMachineState *spapr, void *fdt)
{
- int event_sources, epow_events;
- uint32_t irq_ranges[] = {cpu_to_be32(check_exception_irq), cpu_to_be32(1)};
- uint32_t interrupts[] = {cpu_to_be32(check_exception_irq), 0};
+ uint32_t irq_ranges[EVENT_CLASS_MAX * 2];
+ int i, count = 0, event_sources;
+ sPAPREventSource *events = spapr->event_sources;
+
+ g_assert(events);
_FDT(event_sources = fdt_add_subnode(fdt, 0, "event-sources"));
- _FDT(fdt_setprop(fdt, event_sources, "interrupt-controller", NULL, 0));
- _FDT(fdt_setprop_cell(fdt, event_sources, "#interrupt-cells", 2));
- _FDT(fdt_setprop(fdt, event_sources, "interrupt-ranges",
- irq_ranges, sizeof(irq_ranges)));
+ for (i = 0, count = 0; i < EVENT_CLASS_MAX; i++) {
+ int node_offset;
+ uint32_t interrupts[2];
+ const sPAPREventSource *source =
+ spapr_event_sources_get_source(events, i);
+ const char *source_name = event_names[i];
+
+ if (!source->enabled) {
+ continue;
+ }
+
+ interrupts[0] = cpu_to_be32(source->irq);
+ interrupts[1] = 0;
+
+ _FDT(node_offset = fdt_add_subnode(fdt, event_sources, source_name));
+ _FDT(fdt_setprop(fdt, node_offset, "interrupts", interrupts,
+ sizeof(interrupts)));
+
+ irq_ranges[count++] = interrupts[0];
+ irq_ranges[count++] = cpu_to_be32(1);
+ }
+
+ irq_ranges[count] = cpu_to_be32(count);
+ count++;
+
+ _FDT((fdt_setprop(fdt, event_sources, "interrupt-controller", NULL, 0)));
+ _FDT((fdt_setprop_cell(fdt, event_sources, "#interrupt-cells", 2)));
+ _FDT((fdt_setprop(fdt, event_sources, "interrupt-ranges",
+ irq_ranges, count * sizeof(uint32_t))));
+}
+
+static const sPAPREventSource *
+rtas_event_log_to_source(sPAPRMachineState *spapr, int log_type)
+{
+ const sPAPREventSource *source;
+
+ g_assert(spapr->event_sources);
+
+ switch (log_type) {
+ case RTAS_LOG_TYPE_HOTPLUG:
+ source = spapr_event_sources_get_source(spapr->event_sources,
+ EVENT_CLASS_HOT_PLUG);
+ if (spapr_ovec_test(spapr->ov5_cas, OV5_HP_EVT)) {
+ g_assert(source->enabled);
+ break;
+ }
+ /* fall back to epow for legacy hotplug interrupt source */
+ case RTAS_LOG_TYPE_EPOW:
+ source = spapr_event_sources_get_source(spapr->event_sources,
+ EVENT_CLASS_EPOW);
+ break;
+ default:
+ source = NULL;
+ }
+
+ return source;
+}
+
+static int rtas_event_log_to_irq(sPAPRMachineState *spapr, int log_type)
+{
+ const sPAPREventSource *source;
- _FDT(epow_events = fdt_add_subnode(fdt, event_sources, "epow-events"));
- _FDT(fdt_setprop(fdt, epow_events, "interrupts",
- interrupts, sizeof(interrupts)));
+ source = rtas_event_log_to_source(spapr, log_type);
+ g_assert(source);
+ g_assert(source->enabled);
+
+ return source->irq;
}
static void rtas_event_log_queue(int log_type, void *data, bool exception)
@@ -247,19 +353,15 @@ static sPAPREventLogEntry *rtas_event_log_dequeue(uint32_t event_mask,
sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
sPAPREventLogEntry *entry = NULL;
- /* we only queue EPOW events atm. */
- if ((event_mask & EVENT_MASK_EPOW) == 0) {
- return NULL;
- }
-
QTAILQ_FOREACH(entry, &spapr->pending_events, next) {
+ const sPAPREventSource *source =
+ rtas_event_log_to_source(spapr, entry->log_type);
+
if (entry->exception != exception) {
continue;
}
- /* EPOW and hotplug events are surfaced in the same manner */
- if (entry->log_type == RTAS_LOG_TYPE_EPOW ||
- entry->log_type == RTAS_LOG_TYPE_HOTPLUG) {
+ if (source->mask & event_mask) {
break;
}
}
@@ -276,19 +378,15 @@ static bool rtas_event_log_contains(uint32_t event_mask, bool exception)
sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
sPAPREventLogEntry *entry = NULL;
- /* we only queue EPOW events atm. */
- if ((event_mask & EVENT_MASK_EPOW) == 0) {
- return false;
- }
-
QTAILQ_FOREACH(entry, &spapr->pending_events, next) {
+ const sPAPREventSource *source =
+ rtas_event_log_to_source(spapr, entry->log_type);
+
if (entry->exception != exception) {
continue;
}
- /* EPOW and hotplug events are surfaced in the same manner */
- if (entry->log_type == RTAS_LOG_TYPE_EPOW ||
- entry->log_type == RTAS_LOG_TYPE_HOTPLUG) {
+ if (source->mask & event_mask) {
return true;
}
}
@@ -376,7 +474,9 @@ static void spapr_powerdown_req(Notifier *n, void *opaque)
rtas_event_log_queue(RTAS_LOG_TYPE_EPOW, new_epow, true);
- qemu_irq_pulse(xics_get_qirq(spapr->xics, spapr->check_exception_irq));
+ qemu_irq_pulse(xics_get_qirq(spapr->xics,
+ rtas_event_log_to_irq(spapr,
+ RTAS_LOG_TYPE_EPOW)));
}
static void spapr_hotplug_set_signalled(uint32_t drc_index)
@@ -458,7 +558,9 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
rtas_event_log_queue(RTAS_LOG_TYPE_HOTPLUG, new_hp, true);
- qemu_irq_pulse(xics_get_qirq(spapr->xics, spapr->check_exception_irq));
+ qemu_irq_pulse(xics_get_qirq(spapr->xics,
+ rtas_event_log_to_irq(spapr,
+ RTAS_LOG_TYPE_HOTPLUG)));
}
void spapr_hotplug_req_add_by_index(sPAPRDRConnector *drc)
@@ -504,6 +606,7 @@ static void check_exception(PowerPCCPU *cpu, sPAPRMachineState *spapr,
uint64_t xinfo;
sPAPREventLogEntry *event;
struct rtas_error_log *hdr;
+ int i;
if ((nargs < 6) || (nargs > 7) || nret != 1) {
rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
@@ -540,8 +643,14 @@ static void check_exception(PowerPCCPU *cpu, sPAPRMachineState *spapr,
* do the latter here, since our code relies on edge-triggered
* interrupts.
*/
- if (rtas_event_log_contains(mask, true)) {
- qemu_irq_pulse(xics_get_qirq(spapr->xics, spapr->check_exception_irq));
+ for (i = 0; i < EVENT_CLASS_MAX; i++) {
+ if (rtas_event_log_contains(EVENT_CLASS_MASK(i), true)) {
+ const sPAPREventSource *source =
+ spapr_event_sources_get_source(spapr->event_sources, i);
+
+ g_assert(source->enabled);
+ qemu_irq_pulse(xics_get_qirq(spapr->xics, source->irq));
+ }
}
return;
@@ -593,8 +702,27 @@ out_no_events:
void spapr_events_init(sPAPRMachineState *spapr)
{
QTAILQ_INIT(&spapr->pending_events);
- spapr->check_exception_irq = xics_spapr_alloc(spapr->xics, 0, false,
- &error_fatal);
+
+ spapr->event_sources = spapr_event_sources_new();
+
+ spapr_event_sources_register(spapr->event_sources, EVENT_CLASS_EPOW,
+ xics_spapr_alloc(spapr->xics, 0, false,
+ &error_fatal));
+
+ /* NOTE: if machine supports modern/dedicated hotplug event source,
+ * we add it to the device-tree unconditionally. This means we may
+ * have cases where the source is enabled in QEMU, but unused by the
+ * guest because it does not support modern hotplug events, so we
+ * take care to rely on checking for negotiation of OV5_HP_EVT option
+ * before attempting to use it to signal events, rather than simply
+ * checking that it's enabled.
+ */
+ if (spapr->use_hotplug_event_source) {
+ spapr_event_sources_register(spapr->event_sources, EVENT_CLASS_HOT_PLUG,
+ xics_spapr_alloc(spapr->xics, 0, false,
+ &error_fatal));
+ }
+
spapr->epow_notifier.notify = spapr_powerdown_req;
qemu_register_powerdown_notifier(&spapr->epow_notifier);
spapr_rtas_register(RTAS_CHECK_EXCEPTION, "check-exception",
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index b6f9f1b..21af971 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -13,6 +13,7 @@ struct sPAPRPHBState;
struct sPAPRNVRAM;
typedef struct sPAPRConfigureConnectorState sPAPRConfigureConnectorState;
typedef struct sPAPREventLogEntry sPAPREventLogEntry;
+typedef struct sPAPREventSource sPAPREventSource;
#define HPTE64_V_HPTE_DIRTY 0x0000000000000040ULL
#define SPAPR_ENTRY_POINT 0x100
@@ -77,9 +78,10 @@ struct sPAPRMachineState {
sPAPROptionVector *ov5_cas; /* negotiated (via CAS) option vectors */
bool cas_reboot;
- uint32_t check_exception_irq;
Notifier epow_notifier;
QTAILQ_HEAD(, sPAPREventLogEntry) pending_events;
+ bool use_hotplug_event_source;
+ sPAPREventSource *event_sources;
/* Migration state */
int htab_save_index;
@@ -584,7 +586,7 @@ struct sPAPREventLogEntry {
};
void spapr_events_init(sPAPRMachineState *sm);
-void spapr_dt_events(void *fdt, uint32_t check_exception_irq);
+void spapr_dt_events(sPAPRMachineState *sm, void *fdt);
int spapr_h_cas_compose_response(sPAPRMachineState *sm,
target_ulong addr, target_ulong size,
bool cpu_update,
diff --git a/include/hw/ppc/spapr_ovec.h b/include/hw/ppc/spapr_ovec.h
index 81cdd14..6a06da3 100644
--- a/include/hw/ppc/spapr_ovec.h
+++ b/include/hw/ppc/spapr_ovec.h
@@ -45,6 +45,7 @@ typedef struct sPAPROptionVector sPAPROptionVector;
/* option vector 5 */
#define OV5_DRCONF_MEMORY OV_BIT(2, 2)
#define OV5_FORM1_AFFINITY OV_BIT(5, 0)
+#define OV5_HP_EVT OV_BIT(6, 5)
/* interfaces */
sPAPROptionVector *spapr_ovec_new(void);
--
1.9.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Qemu-devel] [PATCH v2 3/6] spapr: add hotplug interrupt machine options
2016-10-26 21:59 [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 1/6] spapr: update spapr hotplug documentation Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 2/6] spapr_events: add support for dedicated hotplug event source Michael Roth
@ 2016-10-26 21:59 ` Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 4/6] spapr: Add DRC count indexed hotplug identifier type Michael Roth
` (3 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Michael Roth @ 2016-10-26 21:59 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-ppc, david, bharata, nfont, jallen
This adds machine options of the form:
-machine pseries,modern-hotplug-events=true
-machine pseries,modern-hotplug-events=false
If false, QEMU will force the use of "legacy" style hotplug events,
which are surfaced through EPOW events instead of a dedicated
hot plug event source, and lack certain features necessary, mainly,
for memory unplug support.
If true, QEMU will enable support for "modern" dedicated hot plug
event source. Note that we will still default to "legacy" style unless
the guest advertises support for the "modern" hotplug events via
ibm,client-architecture-support hcall during early boot.
For pseries-2.7 and earlier we default to false, for newer machine
types we default to true.
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
hw/ppc/spapr.c | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 9ddf2ff..fe91883 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2143,16 +2143,41 @@ static void spapr_set_kvm_type(Object *obj, const char *value, Error **errp)
spapr->kvm_type = g_strdup(value);
}
+static bool spapr_get_modern_hotplug_events(Object *obj, Error **errp)
+{
+ sPAPRMachineState *spapr = SPAPR_MACHINE(obj);
+
+ return spapr->use_hotplug_event_source;
+}
+
+static void spapr_set_modern_hotplug_events(Object *obj, bool value,
+ Error **errp)
+{
+ sPAPRMachineState *spapr = SPAPR_MACHINE(obj);
+
+ spapr->use_hotplug_event_source = value;
+}
+
static void spapr_machine_initfn(Object *obj)
{
sPAPRMachineState *spapr = SPAPR_MACHINE(obj);
spapr->htab_fd = -1;
+ spapr->use_hotplug_event_source = true;
object_property_add_str(obj, "kvm-type",
spapr_get_kvm_type, spapr_set_kvm_type, NULL);
object_property_set_description(obj, "kvm-type",
"Specifies the KVM virtualization mode (HV, PR)",
NULL);
+ object_property_add_bool(obj, "modern-hotplug-events",
+ spapr_get_modern_hotplug_events,
+ spapr_set_modern_hotplug_events,
+ NULL);
+ object_property_set_description(obj, "modern-hotplug-events",
+ "Use dedicated hotplug event mechanism in"
+ " place of standard EPOW events when possible"
+ " (required for memory hot-unplug support)",
+ NULL);
}
static void spapr_machine_finalizefn(Object *obj)
@@ -2599,7 +2624,10 @@ static void phb_placement_2_7(sPAPRMachineState *spapr, uint32_t index,
static void spapr_machine_2_7_instance_options(MachineState *machine)
{
+ sPAPRMachineState *spapr = SPAPR_MACHINE(machine);
+
spapr_machine_2_8_instance_options(machine);
+ spapr->use_hotplug_event_source = false;
}
static void spapr_machine_2_7_class_options(MachineClass *mc)
--
1.9.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Qemu-devel] [PATCH v2 4/6] spapr: Add DRC count indexed hotplug identifier type
2016-10-26 21:59 [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support Michael Roth
` (2 preceding siblings ...)
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 3/6] spapr: add hotplug interrupt machine options Michael Roth
@ 2016-10-26 21:59 ` Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 5/6] spapr: use count+index for memory hotplug Michael Roth
` (2 subsequent siblings)
6 siblings, 0 replies; 8+ messages in thread
From: Michael Roth @ 2016-10-26 21:59 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-ppc, david, bharata, nfont, jallen
From: Bharata B Rao <bharata@linux.vnet.ibm.com>
Add support for DRC count indexed hotplug ID type which is primarily
needed for memory hot unplug. This type allows for specifying the
number of DRs that should be plugged/unplugged starting from a given
DRC index.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
* updated rtas_event_log_v6_hp to reflect count/index field ordering
used in PAPR hotplug ACR
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
hw/ppc/spapr_events.c | 76 ++++++++++++++++++++++++++++++++++++++++----------
include/hw/ppc/spapr.h | 4 +++
2 files changed, 65 insertions(+), 15 deletions(-)
diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
index 9b0bd41..f85a9c3 100644
--- a/hw/ppc/spapr_events.c
+++ b/hw/ppc/spapr_events.c
@@ -175,6 +175,16 @@ struct epow_log_full {
struct rtas_event_log_v6_epow epow;
} QEMU_PACKED;
+union drc_identifier {
+ uint32_t index;
+ uint32_t count;
+ struct {
+ uint32_t count;
+ uint32_t index;
+ } count_indexed;
+ char name[1];
+} QEMU_PACKED;
+
struct rtas_event_log_v6_hp {
#define RTAS_LOG_V6_SECTION_ID_HOTPLUG 0x4850 /* HP */
struct rtas_event_log_v6_section_header hdr;
@@ -191,12 +201,9 @@ struct rtas_event_log_v6_hp {
#define RTAS_LOG_V6_HP_ID_DRC_NAME 1
#define RTAS_LOG_V6_HP_ID_DRC_INDEX 2
#define RTAS_LOG_V6_HP_ID_DRC_COUNT 3
+#define RTAS_LOG_V6_HP_ID_DRC_COUNT_INDEXED 4
uint8_t reserved;
- union {
- uint32_t index;
- uint32_t count;
- char name[1];
- } drc;
+ union drc_identifier drc_id;
} QEMU_PACKED;
struct hp_log_full {
@@ -488,7 +495,7 @@ static void spapr_hotplug_set_signalled(uint32_t drc_index)
static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
sPAPRDRConnectorType drc_type,
- uint32_t drc)
+ union drc_identifier *drc_id)
{
sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
struct hp_log_full *new_hp;
@@ -533,7 +540,7 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
case SPAPR_DR_CONNECTOR_TYPE_PCI:
hp->hotplug_type = RTAS_LOG_V6_HP_TYPE_PCI;
if (hp->hotplug_action == RTAS_LOG_V6_HP_ACTION_ADD) {
- spapr_hotplug_set_signalled(drc);
+ spapr_hotplug_set_signalled(drc_id->index);
}
break;
case SPAPR_DR_CONNECTOR_TYPE_LMB:
@@ -551,9 +558,18 @@ static void spapr_hotplug_req_event(uint8_t hp_id, uint8_t hp_action,
}
if (hp_id == RTAS_LOG_V6_HP_ID_DRC_COUNT) {
- hp->drc.count = cpu_to_be32(drc);
+ hp->drc_id.count = cpu_to_be32(drc_id->count);
} else if (hp_id == RTAS_LOG_V6_HP_ID_DRC_INDEX) {
- hp->drc.index = cpu_to_be32(drc);
+ hp->drc_id.index = cpu_to_be32(drc_id->index);
+ } else if (hp_id == RTAS_LOG_V6_HP_ID_DRC_COUNT_INDEXED) {
+ /* we should not be using count_indexed value unless the guest
+ * supports dedicated hotplug event source
+ */
+ g_assert(spapr_ovec_test(spapr->ov5_cas, OV5_HP_EVT));
+ hp->drc_id.count_indexed.count =
+ cpu_to_be32(drc_id->count_indexed.count);
+ hp->drc_id.count_indexed.index =
+ cpu_to_be32(drc_id->count_indexed.index);
}
rtas_event_log_queue(RTAS_LOG_TYPE_HOTPLUG, new_hp, true);
@@ -567,34 +583,64 @@ void spapr_hotplug_req_add_by_index(sPAPRDRConnector *drc)
{
sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
sPAPRDRConnectorType drc_type = drck->get_type(drc);
- uint32_t index = drck->get_index(drc);
+ union drc_identifier drc_id;
+ drc_id.index = drck->get_index(drc);
spapr_hotplug_req_event(RTAS_LOG_V6_HP_ID_DRC_INDEX,
- RTAS_LOG_V6_HP_ACTION_ADD, drc_type, index);
+ RTAS_LOG_V6_HP_ACTION_ADD, drc_type, &drc_id);
}
void spapr_hotplug_req_remove_by_index(sPAPRDRConnector *drc)
{
sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
sPAPRDRConnectorType drc_type = drck->get_type(drc);
- uint32_t index = drck->get_index(drc);
+ union drc_identifier drc_id;
+ drc_id.index = drck->get_index(drc);
spapr_hotplug_req_event(RTAS_LOG_V6_HP_ID_DRC_INDEX,
- RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, index);
+ RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);
}
void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
uint32_t count)
{
+ union drc_identifier drc_id;
+
+ drc_id.count = count;
spapr_hotplug_req_event(RTAS_LOG_V6_HP_ID_DRC_COUNT,
- RTAS_LOG_V6_HP_ACTION_ADD, drc_type, count);
+ RTAS_LOG_V6_HP_ACTION_ADD, drc_type, &drc_id);
}
void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
uint32_t count)
{
+ union drc_identifier drc_id;
+
+ drc_id.count = count;
spapr_hotplug_req_event(RTAS_LOG_V6_HP_ID_DRC_COUNT,
- RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, count);
+ RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);
+}
+
+void spapr_hotplug_req_add_by_count_indexed(sPAPRDRConnectorType drc_type,
+ uint32_t count, uint32_t index)
+{
+ union drc_identifier drc_id;
+
+ drc_id.count_indexed.count = count;
+ drc_id.count_indexed.index = index;
+ spapr_hotplug_req_event(RTAS_LOG_V6_HP_ID_DRC_COUNT_INDEXED,
+ RTAS_LOG_V6_HP_ACTION_ADD, drc_type, &drc_id);
+}
+
+void spapr_hotplug_req_remove_by_count_indexed(sPAPRDRConnectorType drc_type,
+ uint32_t count, uint32_t index)
+{
+ union drc_identifier drc_id;
+
+ drc_id.count_indexed.count = count;
+ drc_id.count_indexed.index = index;
+ spapr_hotplug_req_event(RTAS_LOG_V6_HP_ID_DRC_COUNT_INDEXED,
+ RTAS_LOG_V6_HP_ACTION_REMOVE, drc_type, &drc_id);
}
static void check_exception(PowerPCCPU *cpu, sPAPRMachineState *spapr,
diff --git a/include/hw/ppc/spapr.h b/include/hw/ppc/spapr.h
index 21af971..bd5bcf7 100644
--- a/include/hw/ppc/spapr.h
+++ b/include/hw/ppc/spapr.h
@@ -610,6 +610,10 @@ void spapr_hotplug_req_add_by_count(sPAPRDRConnectorType drc_type,
uint32_t count);
void spapr_hotplug_req_remove_by_count(sPAPRDRConnectorType drc_type,
uint32_t count);
+void spapr_hotplug_req_add_by_count_indexed(sPAPRDRConnectorType drc_type,
+ uint32_t count, uint32_t index);
+void spapr_hotplug_req_remove_by_count_indexed(sPAPRDRConnectorType drc_type,
+ uint32_t count, uint32_t index);
void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp);
void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
sPAPRMachineState *spapr);
--
1.9.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Qemu-devel] [PATCH v2 5/6] spapr: use count+index for memory hotplug
2016-10-26 21:59 [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support Michael Roth
` (3 preceding siblings ...)
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 4/6] spapr: Add DRC count indexed hotplug identifier type Michael Roth
@ 2016-10-26 21:59 ` Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 6/6] spapr: Memory hot-unplug support Michael Roth
2016-10-27 2:38 ` [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support David Gibson
6 siblings, 0 replies; 8+ messages in thread
From: Michael Roth @ 2016-10-26 21:59 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-ppc, david, bharata, nfont, jallen
Commit 0a417869:
spapr: Move memory hotplug to RTAS_LOG_V6_HP_ID_DRC_COUNT type
dropped per-DRC/per-LMB hotplugs event in favor of a bulk add via a
single LMB count value. This was to avoid overrunning the guest EPOW
event queue with hotplug events. This works fine, but relies on the
guest exhaustively scanning for pluggable LMBs to satisfy the
requested count by issuing rtas-get-sensor(DR_ENTITY_SENSE, ...) calls
until all the LMBs associated with the DIMM are identified.
With newer support for dedicated hotplug event source, this queue
exhaustion is no longer as much of an issue due to implementation
details on the guest side, but we still try to avoid excessive hotplug
events by now supporting both a count and a starting index to avoid
unecessary work. This patch makes use of that approach when the
capability is available.
Cc: bharata@linux.vnet.ibm.com
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
hw/ppc/spapr.c | 22 ++++++++++++++++++----
1 file changed, 18 insertions(+), 4 deletions(-)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index fe91883..531dfeb 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2202,14 +2202,16 @@ static void spapr_nmi(NMIState *n, int cpu_index, Error **errp)
}
}
-static void spapr_add_lmbs(DeviceState *dev, uint64_t addr, uint64_t size,
- uint32_t node, Error **errp)
+static void spapr_add_lmbs(DeviceState *dev, uint64_t addr_start, uint64_t size,
+ uint32_t node, bool dedicated_hp_event_source,
+ Error **errp)
{
sPAPRDRConnector *drc;
sPAPRDRConnectorClass *drck;
uint32_t nr_lmbs = size/SPAPR_MEMORY_BLOCK_SIZE;
int i, fdt_offset, fdt_size;
void *fdt;
+ uint64_t addr = addr_start;
for (i = 0; i < nr_lmbs; i++) {
drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB,
@@ -2228,7 +2230,17 @@ static void spapr_add_lmbs(DeviceState *dev, uint64_t addr, uint64_t size,
* guest only in case of hotplugged memory
*/
if (dev->hotplugged) {
- spapr_hotplug_req_add_by_count(SPAPR_DR_CONNECTOR_TYPE_LMB, nr_lmbs);
+ if (dedicated_hp_event_source) {
+ drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB,
+ addr_start / SPAPR_MEMORY_BLOCK_SIZE);
+ drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+ spapr_hotplug_req_add_by_count_indexed(SPAPR_DR_CONNECTOR_TYPE_LMB,
+ nr_lmbs,
+ drck->get_index(drc));
+ } else {
+ spapr_hotplug_req_add_by_count(SPAPR_DR_CONNECTOR_TYPE_LMB,
+ nr_lmbs);
+ }
}
}
@@ -2261,7 +2273,9 @@ static void spapr_memory_plug(HotplugHandler *hotplug_dev, DeviceState *dev,
goto out;
}
- spapr_add_lmbs(dev, addr, size, node, &error_abort);
+ spapr_add_lmbs(dev, addr, size, node,
+ spapr_ovec_test(ms->ov5_cas, OV5_HP_EVT),
+ &error_abort);
out:
error_propagate(errp, local_err);
--
1.9.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [Qemu-devel] [PATCH v2 6/6] spapr: Memory hot-unplug support
2016-10-26 21:59 [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support Michael Roth
` (4 preceding siblings ...)
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 5/6] spapr: use count+index for memory hotplug Michael Roth
@ 2016-10-26 21:59 ` Michael Roth
2016-10-27 2:38 ` [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support David Gibson
6 siblings, 0 replies; 8+ messages in thread
From: Michael Roth @ 2016-10-26 21:59 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-ppc, david, bharata, nfont, jallen
From: Bharata B Rao <bharata@linux.vnet.ibm.com>
Add support to hot remove pc-dimm memory devices.
Since we're introducing a machine-level unplug_request hook, we also
had handling for CPU unplug there as well to ensure CPU unplug
continues to work as it did before.
Signed-off-by: Bharata B Rao <bharata@linux.vnet.ibm.com>
* add hooks to CAS/cmdline enablement of hotplug ACR support
* add hook for CPU unplug
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
---
hw/ppc/spapr.c | 119 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
hw/ppc/spapr_drc.c | 17 ++++++++
2 files changed, 135 insertions(+), 1 deletion(-)
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 531dfeb..c8e2921 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2281,6 +2281,90 @@ out:
error_propagate(errp, local_err);
}
+typedef struct sPAPRDIMMState {
+ uint32_t nr_lmbs;
+} sPAPRDIMMState;
+
+static void spapr_lmb_release(DeviceState *dev, void *opaque)
+{
+ sPAPRDIMMState *ds = (sPAPRDIMMState *)opaque;
+ HotplugHandler *hotplug_ctrl;
+
+ if (--ds->nr_lmbs) {
+ return;
+ }
+
+ g_free(ds);
+
+ /*
+ * Now that all the LMBs have been removed by the guest, call the
+ * pc-dimm unplug handler to cleanup up the pc-dimm device.
+ */
+ hotplug_ctrl = qdev_get_hotplug_handler(dev);
+ hotplug_handler_unplug(hotplug_ctrl, dev, &error_abort);
+}
+
+static void spapr_del_lmbs(DeviceState *dev, uint64_t addr_start, uint64_t size,
+ Error **errp)
+{
+ sPAPRDRConnector *drc;
+ sPAPRDRConnectorClass *drck;
+ uint32_t nr_lmbs = size / SPAPR_MEMORY_BLOCK_SIZE;
+ int i;
+ sPAPRDIMMState *ds = g_malloc0(sizeof(sPAPRDIMMState));
+ uint64_t addr = addr_start;
+
+ ds->nr_lmbs = nr_lmbs;
+ for (i = 0; i < nr_lmbs; i++) {
+ drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB,
+ addr / SPAPR_MEMORY_BLOCK_SIZE);
+ g_assert(drc);
+
+ drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+ drck->detach(drc, dev, spapr_lmb_release, ds, errp);
+ addr += SPAPR_MEMORY_BLOCK_SIZE;
+ }
+
+ drc = spapr_dr_connector_by_id(SPAPR_DR_CONNECTOR_TYPE_LMB,
+ addr_start / SPAPR_MEMORY_BLOCK_SIZE);
+ drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+ spapr_hotplug_req_remove_by_count_indexed(SPAPR_DR_CONNECTOR_TYPE_LMB,
+ nr_lmbs,
+ drck->get_index(drc));
+}
+
+static void spapr_memory_unplug(HotplugHandler *hotplug_dev, DeviceState *dev,
+ Error **errp)
+{
+ sPAPRMachineState *ms = SPAPR_MACHINE(hotplug_dev);
+ PCDIMMDevice *dimm = PC_DIMM(dev);
+ PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm);
+ MemoryRegion *mr = ddc->get_memory_region(dimm);
+
+ pc_dimm_memory_unplug(dev, &ms->hotplug_memory, mr);
+ object_unparent(OBJECT(dev));
+}
+
+static void spapr_memory_unplug_request(HotplugHandler *hotplug_dev,
+ DeviceState *dev, Error **errp)
+{
+ Error *local_err = NULL;
+ PCDIMMDevice *dimm = PC_DIMM(dev);
+ PCDIMMDeviceClass *ddc = PC_DIMM_GET_CLASS(dimm);
+ MemoryRegion *mr = ddc->get_memory_region(dimm);
+ uint64_t size = memory_region_size(mr);
+ uint64_t addr;
+
+ addr = object_property_get_int(OBJECT(dimm), PC_DIMM_ADDR_PROP, &local_err);
+ if (local_err) {
+ goto out;
+ }
+
+ spapr_del_lmbs(dev, addr, size, &error_abort);
+out:
+ error_propagate(errp, local_err);
+}
+
void *spapr_populate_hotplug_cpu_dt(CPUState *cs, int *fdt_offset,
sPAPRMachineState *spapr)
{
@@ -2354,10 +2438,42 @@ static void spapr_machine_device_plug(HotplugHandler *hotplug_dev,
static void spapr_machine_device_unplug(HotplugHandler *hotplug_dev,
DeviceState *dev, Error **errp)
{
+ sPAPRMachineState *sms = SPAPR_MACHINE(qdev_get_machine());
MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
- error_setg(errp, "Memory hot unplug not supported by sPAPR");
+ if (spapr_ovec_test(sms->ov5_cas, OV5_HP_EVT)) {
+ spapr_memory_unplug(hotplug_dev, dev, errp);
+ } else {
+ error_setg(errp, "Memory hot unplug not supported for this guest");
+ }
+ } else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
+ if (!mc->query_hotpluggable_cpus) {
+ error_setg(errp, "CPU hot unplug not supported on this machine");
+ return;
+ }
+ spapr_core_unplug(hotplug_dev, dev, errp);
+ }
+}
+
+static void spapr_machine_device_unplug_request(HotplugHandler *hotplug_dev,
+ DeviceState *dev, Error **errp)
+{
+ sPAPRMachineState *sms = SPAPR_MACHINE(qdev_get_machine());
+ MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
+
+ if (object_dynamic_cast(OBJECT(dev), TYPE_PC_DIMM)) {
+ if (spapr_ovec_test(sms->ov5_cas, OV5_HP_EVT)) {
+ spapr_memory_unplug_request(hotplug_dev, dev, errp);
+ } else {
+ /* NOTE: this means there is a window after guest reset, prior to
+ * CAS negotiation, where unplug requests will fail due to the
+ * capability not being detected yet. This is a bit different than
+ * the case with PCI unplug, where the events will be queued and
+ * eventually handled by the guest after boot
+ */
+ error_setg(errp, "Memory hot unplug not supported for this guest");
+ }
} else if (object_dynamic_cast(OBJECT(dev), TYPE_SPAPR_CPU_CORE)) {
if (!mc->query_hotpluggable_cpus) {
error_setg(errp, "CPU hot unplug not supported on this machine");
@@ -2503,6 +2619,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
hc->plug = spapr_machine_device_plug;
hc->unplug = spapr_machine_device_unplug;
mc->cpu_index_to_socket_id = spapr_cpu_index_to_socket_id;
+ hc->unplug_request = spapr_machine_device_unplug_request;
smc->dr_lmb_enabled = true;
smc->tcg_default_cpu = "POWER8";
diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 6e54fd4..a0c44ee 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -68,6 +68,23 @@ static uint32_t set_isolation_state(sPAPRDRConnector *drc,
}
}
+ /*
+ * Fail any requests to ISOLATE the LMB DRC if this LMB doesn't
+ * belong to a DIMM device that is marked for removal.
+ *
+ * Currently the guest userspace tool drmgr that drives the memory
+ * hotplug/unplug will just try to remove a set of 'removable' LMBs
+ * in response to a hot unplug request that is based on drc-count.
+ * If the LMB being removed doesn't belong to a DIMM device that is
+ * actually being unplugged, fail the isolation request here.
+ */
+ if (drc->type == SPAPR_DR_CONNECTOR_TYPE_LMB) {
+ if ((state == SPAPR_DR_ISOLATION_STATE_ISOLATED) &&
+ !drc->awaiting_release) {
+ return RTAS_OUT_HW_ERROR;
+ }
+ }
+
drc->isolation_state = state;
if (drc->isolation_state == SPAPR_DR_ISOLATION_STATE_ISOLATED) {
--
1.9.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support
2016-10-26 21:59 [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support Michael Roth
` (5 preceding siblings ...)
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 6/6] spapr: Memory hot-unplug support Michael Roth
@ 2016-10-27 2:38 ` David Gibson
6 siblings, 0 replies; 8+ messages in thread
From: David Gibson @ 2016-10-27 2:38 UTC (permalink / raw)
To: Michael Roth; +Cc: qemu-devel, qemu-ppc, bharata, nfont, jallen
[-- Attachment #1: Type: text/plain, Size: 4677 bytes --]
On Wed, Oct 26, 2016 at 04:59:19PM -0500, Michael Roth wrote:
> This series is based on David's ppc-for-2.8 branch, and is also available from:
>
> https://github.com/mdroth/qemu/commits/spapr-hotplug-event-update
Applied to ppc-for-2.8.
>
> Changes since v1:
> * Rebased on latest ppc-for-2.8
> * Dropped patches relating to option vector manipulation (now merged)
> * Moved patch introducing modern-hotplug-events option to come after
> that actual implementation of modern-hotplug-events support. (David)
> * Removed uneeded source name string from sPAPREventSource in favor of
> manual lookup of the name when creating DT. (David)
> * Don't split function return signatures across lines (David)
> * Add comment to clarify hotplug events being enabled regardless
> of CAS-negotiation.
> * Avoid unecessary initialization of hotplug_ctrl in
> spapr_lmb_release. (David)
>
> Changes since RFC:
> * Rebased on latest ppc-for-2.8 (with DT refactoring)
> * Submitting as v1 now that PAPR Hotplug ACR is committed
> * address Patchew warnings
> * add comments to clarify spapr->ov5/ov5_cas usage. (David)
> * revise comment to clarify intent when setting spapr->ov5
> OV5_HP_EVT bit. (Bharata)
> * drop internal usage of spapr_ovec_from_bitmap() in favor of
> directly assigning bitmap to sPAPROptionVector instances. (David)
> * standardize meaning of 'vector_len' variable through spapr_ovec_*
> functions to be the byte-wise length of option vectors entries,
> and not including the preceeding length byte itself. (David)
> * fix spapr_ovec_populate_dt() to parse up to OV_MAXBITS bits
> rather than OV_MAXBITS - 1. (David)
> * fix spapr_ovec_populate_dt() encode the minimum of 1 option
> vector byte instead of the max of OV_MAXBYTES in cases where
> no option bits are set. (David)
> * add some comments to spapr_ovec_populate_dt() to clarify what
> is being encoded into length byte of ibm,architecture-vec-5
> * switch 'legacy-hotplug-events' option to
> 'modern-hotplug-events' (David)
> * modify rtas_event_log_to_source() to check for OV5_HP_EVT
> option rather than relying on whether the hotplug source is
> specifically enabled. Assert the latter in cases where
> OV5_HP_EVT is set. (Bharata)
> * drop global EventSource list in favor of an sPAPREventSource
> list field within sPAPRMachineState (David)
> * add CPU unplug hook in mc->unplug_request (Bharata)
>
>
> Patches 1-4 add support for an updated event format for hotplug events,
> which includes a new way to specify a range of DRCs/LMBs to hotplug/unplug
> using a starting position and count, which is necessary for memory unplug.
>
> Patches 5-6 add support for memory unplug using the new event format.
>
> In addition to kernel 4.8 or later, there are a number of patches required
> to enable support on the guest kernel side. I've including the minimum set
> of patches in my branch here:
>
> https://github.com/mdroth/linux/commits/spapr-hotplug-event-update
>
> *powerpc/pseries: advertise Hot Plug Event support to firmware
> powerpc/pseries: Implement indexed-count hotplug memory remove
> powerpc/pseries: Implement indexed-count hotplug memory add
>
> Note that there is currently an issue that arises when attempting to
> offline an LMB that was onlined using a guest kernel's auto-onlining
> mechanism, which can prevent full completion of memory unplug requests.
> This is being investigated, but for the purposes of testing this can
> be worked around currently by disabling auto-onlining in guests via:
>
> "echo offline >/sys/devices/system/memory/auto_online_blocks"
>
> and instead onlining the blocks manually or via udev.
>
> docs/specs/ppc-spapr-hotplug.txt | 55 +++++++++++++++++++++++++++------
> hw/ppc/spapr.c | 178 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-----
> hw/ppc/spapr_drc.c | 17 +++++++++++
> hw/ppc/spapr_events.c | 278 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-------------------------------
> include/hw/ppc/spapr.h | 10 ++++--
> include/hw/ppc/spapr_ovec.h | 1 +
> 6 files changed, 469 insertions(+), 70 deletions(-)
>
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2016-10-27 4:24 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-10-26 21:59 [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 1/6] spapr: update spapr hotplug documentation Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 2/6] spapr_events: add support for dedicated hotplug event source Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 3/6] spapr: add hotplug interrupt machine options Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 4/6] spapr: Add DRC count indexed hotplug identifier type Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 5/6] spapr: use count+index for memory hotplug Michael Roth
2016-10-26 21:59 ` [Qemu-devel] [PATCH v2 6/6] spapr: Memory hot-unplug support Michael Roth
2016-10-27 2:38 ` [Qemu-devel] [PATCH v2 0/6] spapr: dedicated hotplug events and memory unplug support David Gibson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.