* [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability
@ 2022-09-17 2:51 Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 01/11] drivers/char: allow using both dbgp=xhci and dbgp=ehci Marek Marczykowski-Górecki
` (11 more replies)
0 siblings, 12 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel
Cc: Marek Marczykowski-Górecki, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Stefano Stabellini, Wei Liu,
Roger Pau Monné,
Paul Durrant, Kevin Tian, Connor Davis
This is integration of https://github.com/connojd/xue into mainline Xen.
This patch series includes several patches that I made in the process, some are
very loosely related.
The driver developed by Connor supports console via USB3 debug capability. The
capability is designed to operate mostly independently of normal XHCI driver,
so this patch series allows dom0 to drive standard USB3 controller part, while
Xen uses DbC for console output.
Changes since RFC:
- move the driver to xue.c, remove non-Xen parts, remove now unneeded abstraction
- adjust for Xen code style
- build for x86 only
- drop patch hidding the device from dom0
Changes since v1:
- drop ehci patch - already applied
- adjust for review comments from Jan (see changelogs in individual patches)
Changes since v2:
- add runtime option to share (or not) the controller with dom0 or other domains
- add RX support
- several smaller changes according to review comments
Changes since v3:
- put controller sharing behind experimental kconfig option
- several other changes according to review comments
Changes since v4:
- drop first 4 patches - already applied to staging
- split dbgp=xhci into dbc=xhci
Changes since v5:
- roll dbc=xhci back into dbgp=xhci, but make it work together with dbgp=ehci
Changes since v6:
- reorder patches - put acked ones early (I've put acked IOMMU ones early too,
even without VT-d, because they do make it work on AMD, and it's kind
of required to get the console work with IOMMU enabled)
- drop barriers patch (at least for now)
- new patches for suspend support and console=ehci alias
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <george.dunlap@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien@xen.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Wei Liu <wl@xen.org>
Cc: "Roger Pau Monné" <roger.pau@citrix.com>
Cc: Paul Durrant <paul@xen.org>
Cc: Kevin Tian <kevin.tian@intel.com>
Cc: Connor Davis <connojdavis@gmail.com>
Marek Marczykowski-Górecki (11):
drivers/char: allow using both dbgp=xhci and dbgp=ehci
IOMMU: add common API for device reserved memory
IOMMU/AMD: wire common device reserved memory API
drivers/char: mark DMA buffers as reserved for the XHCI
drivers/char: add RX support to the XHCI driver
drivers/char: fix handling cable re-plug in XHCI console driver
drivers/char: allow driving the rest of XHCI by a domain while Xen uses DbC
IOMMU/VT-d: wire common device reserved memory API
console: support multiple serial console simultaneously
drivers/char: suspend handling in XHCI console driver
drivers/char: add console=ehci as an alias for console=dbgp
docs/misc/xen-command-line.pandoc | 30 +-
xen/drivers/char/Kconfig | 11 +-
xen/drivers/char/console.c | 98 ++++--
xen/drivers/char/ehci-dbgp.c | 14 +-
xen/drivers/char/serial.c | 12 +-
xen/drivers/char/xhci-dbc.c | 392 +++++++++++++++++++++---
xen/drivers/passthrough/amd/iommu_acpi.c | 21 +-
xen/drivers/passthrough/iommu.c | 46 +++-
xen/drivers/passthrough/vtd/dmar.c | 201 +++++++-----
xen/include/xen/iommu.h | 14 +-
xen/include/xen/serial.h | 4 +-
11 files changed, 693 insertions(+), 150 deletions(-)
base-commit: 3007efadf74d6146a1c0ff1c2fbbae6b53ce7898
--
git-series 0.9.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* [PATCH v7 01/11] drivers/char: allow using both dbgp=xhci and dbgp=ehci
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
@ 2022-09-17 2:51 ` Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 02/11] IOMMU: add common API for device reserved memory Marek Marczykowski-Górecki
` (10 subsequent siblings)
11 siblings, 0 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel
Cc: Marek Marczykowski-Górecki, Jan Beulich, Andrew Cooper,
George Dunlap, Julien Grall, Stefano Stabellini, Wei Liu
This allows configuring EHCI and XHCI consoles separately,
simultaneously.
This changes string_param() to custom_param() in both ehci and xhci
drivers. Both drivers parse only values applicable to them.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Changes in v7:
- restore memset in xhci_parse_dbgp()
Changes in v6:
- keep dbgp=xhci, but use custom_param() to parse multiple values
separately
- modify ehci-dbgp to use custom_param() too
- change console=dbc to console=xhci, as 'dbc' doesn't appear in any
other option anymore
- update comment in serial.h
new in v5
---
docs/misc/xen-command-line.pandoc | 6 ++++--
xen/drivers/char/ehci-dbgp.c | 14 ++++++++++++--
xen/drivers/char/serial.c | 6 ++++++
xen/drivers/char/xhci-dbc.c | 27 +++++++++++++++++++--------
xen/include/xen/serial.h | 3 ++-
5 files changed, 43 insertions(+), 13 deletions(-)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 9a79385a3712..c8b07042f58e 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -409,7 +409,7 @@ The following are examples of correct specifications:
Specify the size of the console ring buffer.
### console
-> `= List of [ vga | com1[H,L] | com2[H,L] | pv | dbgp | none ]`
+> `= List of [ vga | com1[H,L] | com2[H,L] | pv | dbgp | xhci | none ]`
> Default: `console=com1,vga`
@@ -428,7 +428,9 @@ cleared. This allows a single port to be shared by two subsystems
`pv` indicates that Xen should use Xen's PV console. This option is
only available when used together with `pv-in-pvh`.
-`dbgp` indicates that Xen should use a USB debug port.
+`dbgp` indicates that Xen should use a USB2 debug port.
+
+`xhci` indicates that Xen should use a USB3 debug port.
`none` indicates that Xen should not use a console. This option only
makes sense on its own.
diff --git a/xen/drivers/char/ehci-dbgp.c b/xen/drivers/char/ehci-dbgp.c
index 92c588ec0aa3..8a0b95850609 100644
--- a/xen/drivers/char/ehci-dbgp.c
+++ b/xen/drivers/char/ehci-dbgp.c
@@ -1464,7 +1464,17 @@ static struct uart_driver __read_mostly ehci_dbgp_driver = {
static struct ehci_dbgp ehci_dbgp = { .state = dbgp_unsafe, .phys_port = 1 };
static char __initdata opt_dbgp[30];
-string_param("dbgp", opt_dbgp);
+
+static int __init parse_ehci_dbgp(const char *opt)
+{
+ if ( strncmp(opt, "ehci", 4) )
+ return 0;
+
+ strlcpy(opt_dbgp, opt, sizeof(opt_dbgp));
+
+ return 0;
+}
+custom_param("dbgp", parse_ehci_dbgp);
void __init ehci_dbgp_init(void)
{
@@ -1472,7 +1482,7 @@ void __init ehci_dbgp_init(void)
u32 debug_port, offset, bar_val;
const char *e;
- if ( strncmp(opt_dbgp, "ehci", 4) )
+ if ( !opt_dbgp[0] )
return;
if ( isdigit(opt_dbgp[4]) || !opt_dbgp[4] )
diff --git a/xen/drivers/char/serial.c b/xen/drivers/char/serial.c
index 47899222cef8..9d9445039232 100644
--- a/xen/drivers/char/serial.c
+++ b/xen/drivers/char/serial.c
@@ -311,6 +311,12 @@ int __init serial_parse_handle(const char *conf)
goto common;
}
+ if ( !strncmp(conf, "xhci", 4) && (!conf[4] || conf[4] == ',') )
+ {
+ handle = SERHND_XHCI;
+ goto common;
+ }
+
if ( !strncmp(conf, "dtuart", 6) )
{
handle = SERHND_DTUART;
diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c
index ca7d4a62139e..4712faaabef7 100644
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -245,6 +245,7 @@ struct dbc {
uint64_t xhc_dbc_offset;
void __iomem *xhc_mmio;
+ bool enable; /* whether dbgp=xhci was set at all */
bool open;
unsigned int xhc_num; /* look for n-th xhc */
};
@@ -1058,18 +1059,14 @@ static struct xhci_dbc_ctx ctx __aligned(16);
static uint8_t out_wrk_buf[DBC_WORK_RING_CAP];
static struct xhci_string_descriptor str_buf[DBC_STRINGS_COUNT];
-static char __initdata opt_dbgp[30];
-
-string_param("dbgp", opt_dbgp);
-
-void __init xhci_dbc_uart_init(void)
+static int __init xhci_parse_dbgp(const char *opt_dbgp)
{
struct dbc_uart *uart = &dbc_uart;
struct dbc *dbc = &uart->dbc;
const char *e;
if ( strncmp(opt_dbgp, "xhci", 4) )
- return;
+ return 0;
memset(dbc, 0, sizeof(*dbc));
@@ -1087,12 +1084,26 @@ void __init xhci_dbc_uart_init(void)
printk(XENLOG_ERR
"Invalid dbgp= PCI device spec: '%s'\n",
opt_dbgp + 8);
- return;
+ return -EINVAL;
}
dbc->sbdf = PCI_SBDF(0, bus, slot, func);
}
+ dbc->enable = true;
+
+ return 0;
+}
+custom_param("dbgp", xhci_parse_dbgp);
+
+void __init xhci_dbc_uart_init(void)
+{
+ struct dbc_uart *uart = &dbc_uart;
+ struct dbc *dbc = &uart->dbc;
+
+ if ( !dbc->enable )
+ return;
+
dbc->dbc_ctx = &ctx;
dbc->dbc_erst = &erst;
dbc->dbc_ering.trb = evt_trb;
@@ -1102,7 +1113,7 @@ void __init xhci_dbc_uart_init(void)
dbc->dbc_str = str_buf;
if ( dbc_open(dbc) )
- serial_register_uart(SERHND_DBGP, &dbc_uart_driver, &dbc_uart);
+ serial_register_uart(SERHND_XHCI, &dbc_uart_driver, &dbc_uart);
}
#ifdef DBC_DEBUG
diff --git a/xen/include/xen/serial.h b/xen/include/xen/serial.h
index 4cd4ae5e6f1c..f0aff7ea7661 100644
--- a/xen/include/xen/serial.h
+++ b/xen/include/xen/serial.h
@@ -91,10 +91,11 @@ struct uart_driver {
};
/* 'Serial handles' are composed from the following fields. */
-#define SERHND_IDX (3<<0) /* COM1, COM2, DBGP, DTUART? */
+#define SERHND_IDX (3<<0) /* COM1, COM2, DBGP, XHCI, DTUART? */
# define SERHND_COM1 (0<<0)
# define SERHND_COM2 (1<<0)
# define SERHND_DBGP (2<<0)
+# define SERHND_XHCI (3<<0)
# define SERHND_DTUART (0<<0) /* Steal SERHND_COM1 value */
#define SERHND_HI (1<<2) /* Mux/demux each transferred char by MSB. */
#define SERHND_LO (1<<3) /* Ditto, except that the MSB is cleared. */
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v7 02/11] IOMMU: add common API for device reserved memory
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 01/11] drivers/char: allow using both dbgp=xhci and dbgp=ehci Marek Marczykowski-Górecki
@ 2022-09-17 2:51 ` Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 03/11] IOMMU/AMD: wire common device reserved memory API Marek Marczykowski-Górecki
` (9 subsequent siblings)
11 siblings, 0 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel
Cc: Marek Marczykowski-Górecki, Jan Beulich, Paul Durrant,
Roger Pau Monné
Add API similar to rmrr= and ivmd= arguments, but in a common code. This
will allow drivers to register reserved memory regardless of the IOMMU
vendor.
The direct reason for this API is xhci-dbc console driver (aka xue),
that needs to use DMA. But future change may unify command line
arguments for user-supplied reserved memory, and it may be useful for
other drivers in the future too.
This commit just introduces an API, subsequent patches will plug it in
appropriate places. The reserved memory ranges needs to be saved
locally, because at the point when they are collected, Xen doesn't know
yet which IOMMU driver will be used.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in v5:
- fix indentation, keep full "reserved_device_memory" for consistency
with iommu_get_reserved_device_memory
Changes in v4:
- mark functions as __init
- use pci_sbdf_t type
Changes in v3:
- adjust code style
---
xen/drivers/passthrough/iommu.c | 46 ++++++++++++++++++++++++++++++++++-
xen/include/xen/iommu.h | 14 ++++++++++-
2 files changed, 60 insertions(+)
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index 134cdb47e0dc..5e2a720d29b9 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -669,6 +669,52 @@ bool_t iommu_has_feature(struct domain *d, enum iommu_feature feature)
return is_iommu_enabled(d) && test_bit(feature, dom_iommu(d)->features);
}
+#define MAX_EXTRA_RESERVED_RANGES 20
+struct extra_reserved_range {
+ unsigned long start;
+ unsigned long nr;
+ pci_sbdf_t sbdf;
+};
+static unsigned int __initdata nr_extra_reserved_ranges;
+static struct extra_reserved_range __initdata
+ extra_reserved_ranges[MAX_EXTRA_RESERVED_RANGES];
+
+int __init iommu_add_extra_reserved_device_memory(unsigned long start,
+ unsigned long nr,
+ pci_sbdf_t sbdf)
+{
+ unsigned int idx;
+
+ if ( nr_extra_reserved_ranges >= MAX_EXTRA_RESERVED_RANGES )
+ return -ENOMEM;
+
+ idx = nr_extra_reserved_ranges++;
+ extra_reserved_ranges[idx].start = start;
+ extra_reserved_ranges[idx].nr = nr;
+ extra_reserved_ranges[idx].sbdf = sbdf;
+
+ return 0;
+}
+
+int __init iommu_get_extra_reserved_device_memory(iommu_grdm_t *func,
+ void *ctxt)
+{
+ unsigned int idx;
+ int ret;
+
+ for ( idx = 0; idx < nr_extra_reserved_ranges; idx++ )
+ {
+ ret = func(extra_reserved_ranges[idx].start,
+ extra_reserved_ranges[idx].nr,
+ extra_reserved_ranges[idx].sbdf.sbdf,
+ ctxt);
+ if ( ret < 0 )
+ return ret;
+ }
+
+ return 0;
+}
+
/*
* Local variables:
* mode: C
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 1240d7762d99..4f22fc1bed55 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -304,6 +304,20 @@ struct iommu_ops {
#endif
};
+/*
+ * To be called by Xen internally, to register extra RMRR/IVMD ranges.
+ * Needs to be called before IOMMU initialization.
+ */
+extern int iommu_add_extra_reserved_device_memory(unsigned long start,
+ unsigned long nr,
+ pci_sbdf_t sbdf);
+/*
+ * To be called by specific IOMMU driver during initialization,
+ * to fetch ranges registered with iommu_add_extra_reserved_device_memory().
+ */
+extern int iommu_get_extra_reserved_device_memory(iommu_grdm_t *func,
+ void *ctxt);
+
#include <asm/iommu.h>
#ifndef iommu_call
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v7 03/11] IOMMU/AMD: wire common device reserved memory API
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 01/11] drivers/char: allow using both dbgp=xhci and dbgp=ehci Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 02/11] IOMMU: add common API for device reserved memory Marek Marczykowski-Górecki
@ 2022-09-17 2:51 ` Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 04/11] drivers/char: mark DMA buffers as reserved for the XHCI Marek Marczykowski-Górecki
` (8 subsequent siblings)
11 siblings, 0 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel; +Cc: Marek Marczykowski-Górecki, Jan Beulich, Andrew Cooper
Register common device reserved memory similar to how ivmd= parameter is
handled.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in v3:
- use variable initializer
- use pfn_to_paddr()
---
xen/drivers/passthrough/amd/iommu_acpi.c | 21 +++++++++++++++++++++
1 file changed, 21 insertions(+)
diff --git a/xen/drivers/passthrough/amd/iommu_acpi.c b/xen/drivers/passthrough/amd/iommu_acpi.c
index ac6835225bae..3b577c9b390c 100644
--- a/xen/drivers/passthrough/amd/iommu_acpi.c
+++ b/xen/drivers/passthrough/amd/iommu_acpi.c
@@ -1078,6 +1078,25 @@ static inline bool_t is_ivmd_block(u8 type)
type == ACPI_IVRS_TYPE_MEMORY_IOMMU);
}
+static int __init cf_check add_one_extra_ivmd(unsigned long start,
+ unsigned long nr,
+ uint32_t id, void *ctxt)
+{
+ struct acpi_ivrs_memory ivmd = {
+ .header = {
+ .length = sizeof(ivmd),
+ .flags = ACPI_IVMD_UNITY | ACPI_IVMD_READ | ACPI_IVMD_WRITE,
+ .device_id = id,
+ .type = ACPI_IVRS_TYPE_MEMORY_ONE,
+ },
+ };
+
+ ivmd.start_address = pfn_to_paddr(start);
+ ivmd.memory_length = pfn_to_paddr(nr);
+
+ return parse_ivmd_block(&ivmd);
+}
+
static int __init cf_check parse_ivrs_table(struct acpi_table_header *table)
{
const struct acpi_ivrs_header *ivrs_block;
@@ -1121,6 +1140,8 @@ static int __init cf_check parse_ivrs_table(struct acpi_table_header *table)
AMD_IOMMU_DEBUG("IVMD: %u command line provided entries\n", nr_ivmd);
for ( i = 0; !error && i < nr_ivmd; ++i )
error = parse_ivmd_block(user_ivmds + i);
+ if ( !error )
+ error = iommu_get_extra_reserved_device_memory(add_one_extra_ivmd, NULL);
/* Each IO-APIC must have been mentioned in the table. */
for ( apic = 0; !error && iommu_intremap && apic < nr_ioapics; ++apic )
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v7 04/11] drivers/char: mark DMA buffers as reserved for the XHCI
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
` (2 preceding siblings ...)
2022-09-17 2:51 ` [PATCH v7 03/11] IOMMU/AMD: wire common device reserved memory API Marek Marczykowski-Górecki
@ 2022-09-17 2:51 ` Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 05/11] drivers/char: add RX support to the XHCI driver Marek Marczykowski-Górecki
` (7 subsequent siblings)
11 siblings, 0 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel
Cc: Marek Marczykowski-Górecki, Jan Beulich, Andrew Cooper,
George Dunlap, Julien Grall, Stefano Stabellini, Wei Liu
The important part is to include those buffers in IOMMU page table
relevant for the USB controller. Otherwise, DbC will stop working as
soon as IOMMU is enabled, regardless of to which domain device assigned
(be it xen or dom0).
If the device is passed through to dom0 or other domain (see later
patches), that domain will effectively have access to those buffers too.
It does give such domain yet another way to DoS the system (as is the
case when having PCI device assigned already), but also possibly steal
the console ring content. Thus, such domain should be a trusted one.
In any case, prevent anything else being placed on those pages by adding
artificial padding.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in v5:
- add missing alignment
Changes in v3:
- adjust for xhci-dbc rename
- do not raise MAX_USER_RMRR_PAGES
- adjust alignment of DMA buffers
---
xen/drivers/char/xhci-dbc.c | 43 +++++++++++++++++++++++++-------------
1 file changed, 29 insertions(+), 14 deletions(-)
diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c
index 4712faaabef7..f55f73e382fc 100644
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -20,6 +20,7 @@
*/
#include <xen/delay.h>
+#include <xen/iommu.h>
#include <xen/mm.h>
#include <xen/param.h>
#include <xen/serial.h>
@@ -1051,13 +1052,21 @@ static struct uart_driver dbc_uart_driver = {
};
/* Those are accessed via DMA. */
-static struct xhci_trb evt_trb[DBC_TRB_RING_CAP];
-static struct xhci_trb out_trb[DBC_TRB_RING_CAP];
-static struct xhci_trb in_trb[DBC_TRB_RING_CAP];
-static struct xhci_erst_segment erst __aligned(16);
-static struct xhci_dbc_ctx ctx __aligned(16);
-static uint8_t out_wrk_buf[DBC_WORK_RING_CAP];
-static struct xhci_string_descriptor str_buf[DBC_STRINGS_COUNT];
+struct dbc_dma_bufs {
+ struct xhci_trb evt_trb[DBC_TRB_RING_CAP];
+ struct xhci_trb out_trb[DBC_TRB_RING_CAP];
+ struct xhci_trb in_trb[DBC_TRB_RING_CAP];
+ uint8_t out_wrk_buf[DBC_WORK_RING_CAP];
+ struct xhci_erst_segment erst __aligned(16);
+ struct xhci_dbc_ctx ctx __aligned(16);
+ struct xhci_string_descriptor str_buf[DBC_STRINGS_COUNT];
+ /*
+ * Don't place anything else on this page - it will be
+ * DMA-reachable by the USB controller.
+ */
+};
+static struct dbc_dma_bufs __section(".bss.page_aligned") __aligned(PAGE_SIZE)
+ dbc_dma_bufs;
static int __init xhci_parse_dbgp(const char *opt_dbgp)
{
@@ -1104,16 +1113,22 @@ void __init xhci_dbc_uart_init(void)
if ( !dbc->enable )
return;
- dbc->dbc_ctx = &ctx;
- dbc->dbc_erst = &erst;
- dbc->dbc_ering.trb = evt_trb;
- dbc->dbc_oring.trb = out_trb;
- dbc->dbc_iring.trb = in_trb;
- dbc->dbc_owork.buf = out_wrk_buf;
- dbc->dbc_str = str_buf;
+ dbc->dbc_ctx = &dbc_dma_bufs.ctx;
+ dbc->dbc_erst = &dbc_dma_bufs.erst;
+ dbc->dbc_ering.trb = dbc_dma_bufs.evt_trb;
+ dbc->dbc_oring.trb = dbc_dma_bufs.out_trb;
+ dbc->dbc_iring.trb = dbc_dma_bufs.in_trb;
+ dbc->dbc_owork.buf = dbc_dma_bufs.out_wrk_buf;
+ dbc->dbc_str = dbc_dma_bufs.str_buf;
if ( dbc_open(dbc) )
+ {
+ iommu_add_extra_reserved_device_memory(
+ PFN_DOWN(virt_to_maddr(&dbc_dma_bufs)),
+ PFN_UP(sizeof(dbc_dma_bufs)),
+ uart->dbc.sbdf);
serial_register_uart(SERHND_XHCI, &dbc_uart_driver, &dbc_uart);
+ }
}
#ifdef DBC_DEBUG
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v7 05/11] drivers/char: add RX support to the XHCI driver
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
` (3 preceding siblings ...)
2022-09-17 2:51 ` [PATCH v7 04/11] drivers/char: mark DMA buffers as reserved for the XHCI Marek Marczykowski-Górecki
@ 2022-09-17 2:51 ` Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 06/11] drivers/char: fix handling cable re-plug in XHCI console driver Marek Marczykowski-Górecki
` (6 subsequent siblings)
11 siblings, 0 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel
Cc: Marek Marczykowski-Górecki, Jan Beulich, Andrew Cooper,
George Dunlap, Julien Grall, Stefano Stabellini, Wei Liu
Add another work ring buffer for received data, and point IN TRB at it.
Ensure there is always at least one pending IN TRB, so the controller
has a way to send incoming data to the driver.
Note that both "success" and "short packet" completion codes are okay -
in fact it will be "short packet" most of the time, as the TRB length is
about maximum size, not required size.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in v4:
- adjust return types
- add some const
New patch in v3
---
docs/misc/xen-command-line.pandoc | 6 +-
xen/drivers/char/xhci-dbc.c | 129 +++++++++++++++++++++++++++++++-
2 files changed, 132 insertions(+), 3 deletions(-)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index c8b07042f58e..92d668afa06c 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -728,9 +728,9 @@ Available alternatives, with their meaning, are:
Specify the USB controller to use, either by instance number (when going
over the PCI busses sequentially) or by PCI device (must be on segment 0).
-Use `ehci` for EHCI debug port, use `xhci` for XHCI debug capability (output
-only). XHCI driver will wait indefinitely for the debug host to connect - make
-sure the cable is connected.
+Use `ehci` for EHCI debug port, use `xhci` for XHCI debug capability.
+XHCI driver will wait indefinitely for the debug host to connect - make sure
+the cable is connected.
### debug_stack_lines
> `= <integer>`
diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c
index f55f73e382fc..00ab4ae4a27e 100644
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -111,6 +111,7 @@ enum {
enum {
XHCI_TRB_CC_SUCCESS = 1,
XHCI_TRB_CC_TRB_ERR = 5,
+ XHCI_TRB_CC_SHORT_PACKET = 13,
};
/* DbC endpoint types */
@@ -239,6 +240,7 @@ struct dbc {
struct xhci_trb_ring dbc_oring;
struct xhci_trb_ring dbc_iring;
struct dbc_work_ring dbc_owork;
+ struct dbc_work_ring dbc_iwork;
struct xhci_string_descriptor *dbc_str;
pci_sbdf_t sbdf;
@@ -444,6 +446,16 @@ static void xhci_trb_norm_set_ioc(struct xhci_trb *trb)
trb->ctrl |= 0x20;
}
+static uint64_t xhci_trb_norm_buf(const struct xhci_trb *trb)
+{
+ return trb->params;
+}
+
+static uint32_t xhci_trb_norm_len(const struct xhci_trb *trb)
+{
+ return trb->status & 0x1FFFF;
+}
+
/**
* Fields for Transfer Event TRBs (see section 6.4.2.1). Note that event
* TRBs are read-only from software
@@ -453,6 +465,17 @@ static uint64_t xhci_trb_tfre_ptr(const struct xhci_trb *trb)
return trb->params;
}
+static uint32_t xhci_trb_tfre_cc(const struct xhci_trb *trb)
+{
+ return trb->status >> 24;
+}
+
+/* Amount of data _not_ transferred */
+static uint32_t xhci_trb_tfre_len(const struct xhci_trb *trb)
+{
+ return trb->status & 0x1FFFF;
+}
+
/* Fields for link TRBs (section 6.4.4.1) */
static void xhci_trb_link_set_rsp(struct xhci_trb *trb, uint64_t rsp)
{
@@ -494,6 +517,14 @@ static bool xhci_trb_ring_full(const struct xhci_trb_ring *ring)
return ((ring->enq + 1) & (DBC_TRB_RING_CAP - 1)) == ring->deq;
}
+static unsigned int xhci_trb_ring_size(const struct xhci_trb_ring *ring)
+{
+ if ( ring->enq >= ring->deq )
+ return ring->enq - ring->deq;
+
+ return DBC_TRB_RING_CAP - ring->deq + ring->enq;
+}
+
static bool dbc_work_ring_full(const struct dbc_work_ring *ring)
{
return ((ring->enq + 1) & (DBC_WORK_RING_CAP - 1)) == ring->deq;
@@ -507,6 +538,14 @@ static unsigned int dbc_work_ring_size(const struct dbc_work_ring *ring)
return DBC_WORK_RING_CAP - ring->deq + ring->enq;
}
+static unsigned int dbc_work_ring_space_to_end(const struct dbc_work_ring *ring)
+{
+ if ( ring->enq >= ring->deq )
+ return DBC_WORK_RING_CAP - ring->enq;
+
+ return ring->deq - ring->enq;
+}
+
static void dbc_push_trb(struct dbc *dbc, struct xhci_trb_ring *ring,
uint64_t dma, uint64_t len)
{
@@ -567,6 +606,31 @@ static unsigned int dbc_push_work(struct dbc *dbc, struct dbc_work_ring *ring,
return i;
}
+static void dbc_rx_trb(struct dbc *dbc, struct xhci_trb *trb,
+ uint64_t not_transferred)
+{
+ struct dbc_work_ring *ring = &dbc->dbc_iwork;
+ unsigned int rx_len;
+ unsigned int end, start = ring->enq;
+
+ if ( xhci_trb_type(trb) != XHCI_TRB_NORM )
+ /* Can be Link TRB for example. */
+ return;
+
+ ASSERT(xhci_trb_norm_buf(trb) == ring->dma + ring->enq);
+ ASSERT(xhci_trb_norm_len(trb) >= not_transferred);
+ rx_len = xhci_trb_norm_len(trb) - not_transferred;
+
+ /* It can hit the ring end, but should not wrap around. */
+ ASSERT(ring->enq + rx_len <= DBC_WORK_RING_CAP);
+ ring->enq = (ring->enq + rx_len) & (DBC_WORK_RING_CAP - 1);
+
+ end = ring->enq;
+
+ if ( end > start )
+ cache_flush(&ring->buf[start], end - start);
+}
+
/*
* Note that if IN transfer support is added, then this
* will need to be changed; it assumes an OUT transfer ring only
@@ -576,6 +640,7 @@ static void dbc_pop_events(struct dbc *dbc)
struct dbc_reg *reg = dbc->dbc_reg;
struct xhci_trb_ring *er = &dbc->dbc_ering;
struct xhci_trb_ring *tr = &dbc->dbc_oring;
+ struct xhci_trb_ring *ir = &dbc->dbc_iring;
struct xhci_trb *event = &er->trb[er->deq];
uint64_t erdp = readq(®->erdp);
uint32_t portsc;
@@ -601,6 +666,14 @@ static void dbc_pop_events(struct dbc *dbc)
trb_idx = (event_ptr - tr->dma) >> XHCI_TRB_SHIFT;
tr->deq = (trb_idx + 1) & (DBC_TRB_RING_CAP - 1);
}
+ else if ( event_ptr - ir->dma < DBC_TRB_RING_BYTES )
+ {
+ trb_idx = (event_ptr - ir->dma) >> XHCI_TRB_SHIFT;
+ if ( xhci_trb_tfre_cc(event) == XHCI_TRB_CC_SUCCESS ||
+ xhci_trb_tfre_cc(event) == XHCI_TRB_CC_SHORT_PACKET )
+ dbc_rx_trb(dbc, &ir->trb[trb_idx], xhci_trb_tfre_len(event));
+ ir->deq = (trb_idx + 1) & (DBC_TRB_RING_CAP - 1);
+ }
else
dbc_alert("event: TRB 0x%lx not found in any ring\n",
event_ptr);
@@ -871,6 +944,7 @@ static bool __init dbc_open(struct dbc *dbc)
return false;
dbc_init_work_ring(dbc, &dbc->dbc_owork);
+ dbc_init_work_ring(dbc, &dbc->dbc_iwork);
dbc_enable_dbc(dbc);
dbc->open = true;
@@ -947,6 +1021,33 @@ static void dbc_flush(struct dbc *dbc, struct xhci_trb_ring *trb,
}
/**
+ * Ensure DbC has a pending transfer TRB to receive data into.
+ *
+ * @param dbc the dbc to flush
+ * @param trb the ring for the TRBs to transfer
+ * @param wrk the work ring to receive data into
+ */
+static void dbc_enqueue_in(struct dbc *dbc, struct xhci_trb_ring *trb,
+ struct dbc_work_ring *wrk)
+{
+ struct dbc_reg *reg = dbc->dbc_reg;
+ uint32_t db = (readl(®->db) & 0xFFFF00FF) | (trb->db << 8);
+
+ /* Check if there is already queued TRB */
+ if ( xhci_trb_ring_size(trb) >= 1 )
+ return;
+
+ if ( dbc_work_ring_full(wrk) )
+ return;
+
+ dbc_push_trb(dbc, trb, wrk->dma + wrk->enq,
+ dbc_work_ring_space_to_end(wrk));
+
+ wmb();
+ writel(db, ®->db);
+}
+
+/**
* Queue a single character to the DbC. A transfer TRB will be created
* if the character is a newline and the DbC will be notified that data is
* available for writing to the debug host.
@@ -969,6 +1070,19 @@ static int64_t dbc_putc(struct dbc *dbc, char c)
return 1;
}
+static int dbc_getc(struct dbc *dbc, char *c)
+{
+ struct dbc_work_ring *wrk = &dbc->dbc_iwork;
+
+ if ( dbc_work_ring_size(wrk) == 0 )
+ return 0;
+
+ *c = wrk->buf[wrk->deq];
+ wrk->deq = (wrk->deq + 1) & (DBC_WORK_RING_CAP - 1);
+
+ return 1;
+}
+
struct dbc_uart {
struct dbc dbc;
struct timer timer;
@@ -987,10 +1101,16 @@ static void cf_check dbc_uart_poll(void *data)
if ( spin_trylock_irqsave(&port->tx_lock, flags) )
{
if ( dbc_ensure_running(dbc) )
+ {
dbc_flush(dbc, &dbc->dbc_oring, &dbc->dbc_owork);
+ dbc_enqueue_in(dbc, &dbc->dbc_iring, &dbc->dbc_iwork);
+ }
spin_unlock_irqrestore(&port->tx_lock, flags);
}
+ while ( dbc_work_ring_size(&dbc->dbc_iwork) )
+ serial_rx_interrupt(port, guest_cpu_user_regs());
+
serial_tx_interrupt(port, guest_cpu_user_regs());
set_timer(&uart->timer, NOW() + MICROSECS(DBC_POLL_INTERVAL));
}
@@ -1029,6 +1149,12 @@ static void cf_check dbc_uart_putc(struct serial_port *port, char c)
dbc_putc(&uart->dbc, c);
}
+static int cf_check dbc_uart_getc(struct serial_port *port, char *c)
+{
+ struct dbc_uart *uart = port->uart;
+ return dbc_getc(&uart->dbc, c);
+}
+
static void cf_check dbc_uart_flush(struct serial_port *port)
{
s_time_t goal;
@@ -1048,6 +1174,7 @@ static struct uart_driver dbc_uart_driver = {
.init_postirq = dbc_uart_init_postirq,
.tx_ready = dbc_uart_tx_ready,
.putc = dbc_uart_putc,
+ .getc = dbc_uart_getc,
.flush = dbc_uart_flush,
};
@@ -1057,6 +1184,7 @@ struct dbc_dma_bufs {
struct xhci_trb out_trb[DBC_TRB_RING_CAP];
struct xhci_trb in_trb[DBC_TRB_RING_CAP];
uint8_t out_wrk_buf[DBC_WORK_RING_CAP];
+ uint8_t in_wrk_buf[DBC_WORK_RING_CAP];
struct xhci_erst_segment erst __aligned(16);
struct xhci_dbc_ctx ctx __aligned(16);
struct xhci_string_descriptor str_buf[DBC_STRINGS_COUNT];
@@ -1119,6 +1247,7 @@ void __init xhci_dbc_uart_init(void)
dbc->dbc_oring.trb = dbc_dma_bufs.out_trb;
dbc->dbc_iring.trb = dbc_dma_bufs.in_trb;
dbc->dbc_owork.buf = dbc_dma_bufs.out_wrk_buf;
+ dbc->dbc_iwork.buf = dbc_dma_bufs.in_wrk_buf;
dbc->dbc_str = dbc_dma_bufs.str_buf;
if ( dbc_open(dbc) )
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v7 06/11] drivers/char: fix handling cable re-plug in XHCI console driver
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
` (4 preceding siblings ...)
2022-09-17 2:51 ` [PATCH v7 05/11] drivers/char: add RX support to the XHCI driver Marek Marczykowski-Górecki
@ 2022-09-17 2:51 ` Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 07/11] drivers/char: allow driving the rest of XHCI by a domain while Xen uses DbC Marek Marczykowski-Górecki
` (5 subsequent siblings)
11 siblings, 0 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel
Cc: Marek Marczykowski-Górecki, Jan Beulich, Andrew Cooper,
George Dunlap, Julien Grall, Stefano Stabellini, Wei Liu
When cable is unplugged, dbc_ensure_running() correctly detects this
situation (DBC_CTRL_DCR flag is clear), and prevent sending data
immediately to the device. It gets only queued in work ring buffers.
When cable is plugged in again, subsequent dbc_flush() will send the
buffered data.
But there is a corner case, where no subsequent data was buffered in the
work buffer, but a TRB was still pending. Ring the doorbell to let the
controller re-send them. For console output it is rare corner case (TRB
is pending for a very short time), but for console input it is very
normal case (there is always one pending TRB for input).
Extract doorbell ringing into separate function to avoid duplication.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
---
Changes in v6:
- keep barriers consistent
---
xen/drivers/char/xhci-dbc.c | 17 ++++++++++++-----
1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c
index 00ab4ae4a27e..5f47733c1801 100644
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -546,6 +546,15 @@ static unsigned int dbc_work_ring_space_to_end(const struct dbc_work_ring *ring)
return ring->deq - ring->enq;
}
+static void dbc_ring_doorbell(struct dbc *dbc, int doorbell)
+{
+ uint32_t __iomem *db_reg = &dbc->dbc_reg->db;
+ uint32_t db = (readl(db_reg) & ~DBC_DOORBELL_TARGET_MASK) |
+ (doorbell << DBC_DOORBELL_TARGET_SHIFT);
+
+ writel(db, db_reg);
+}
+
static void dbc_push_trb(struct dbc *dbc, struct xhci_trb_ring *ring,
uint64_t dma, uint64_t len)
{
@@ -973,6 +982,8 @@ static bool dbc_ensure_running(struct dbc *dbc)
writel(ctrl | (1U << DBC_CTRL_DRC), ®->ctrl);
writel(readl(®->portsc) | (1U << DBC_PSC_PED), ®->portsc);
wmb();
+ dbc_ring_doorbell(dbc, dbc->dbc_iring.db);
+ dbc_ring_doorbell(dbc, dbc->dbc_oring.db);
}
return true;
@@ -990,10 +1001,6 @@ static bool dbc_ensure_running(struct dbc *dbc)
static void dbc_flush(struct dbc *dbc, struct xhci_trb_ring *trb,
struct dbc_work_ring *wrk)
{
- struct dbc_reg *reg = dbc->dbc_reg;
- uint32_t db = (readl(®->db) & ~DBC_DOORBELL_TARGET_MASK) |
- (trb->db << DBC_DOORBELL_TARGET_SHIFT);
-
if ( xhci_trb_ring_full(trb) )
return;
@@ -1017,7 +1024,7 @@ static void dbc_flush(struct dbc *dbc, struct xhci_trb_ring *trb,
}
wmb();
- writel(db, ®->db);
+ dbc_ring_doorbell(dbc, trb->db);
}
/**
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v7 07/11] drivers/char: allow driving the rest of XHCI by a domain while Xen uses DbC
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
` (5 preceding siblings ...)
2022-09-17 2:51 ` [PATCH v7 06/11] drivers/char: fix handling cable re-plug in XHCI console driver Marek Marczykowski-Górecki
@ 2022-09-17 2:51 ` Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 08/11] IOMMU/VT-d: wire common device reserved memory API Marek Marczykowski-Górecki
` (4 subsequent siblings)
11 siblings, 0 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel
Cc: Marek Marczykowski-Górecki, Jan Beulich, Andrew Cooper,
George Dunlap, Julien Grall, Stefano Stabellini, Wei Liu
That's possible, because the capability was designed specifically to
allow separate driver handle it, in parallel to unmodified xhci driver
(separate set of registers, pretending the port is "disconnected" for
the main xhci driver etc). It works with Linux dom0, although requires
an awful hack - re-enabling bus mastering behind dom0's backs.
Linux driver does similar thing - see
drivers/usb/early/xhci-dbc.c:xdbc_handle_events().
When controller sharing is enabled in kconfig (option marked as
experimental), dom0 is allowed to use the controller even if Xen uses it
for debug console. Additionally, option `dbgp=xhci,share=` is available
to either prevent even dom0 from using it (`no` value), or allow any
domain using it (`any` value).
In any case, to avoid Linux messing with the DbC, mark this MMIO area as
read-only. This might cause issues for Linux's driver (if it tries to
write something on the same page - like anoter xcap), but makes Xen's
use safe. In practice, as of Linux 5.18, it seems to work without
issues.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes in v5:
- drop CONFIG_XHCI_SHARE
- make XHCI_SHARE_HWDOM = 0
- use parse_boolean
- add comment about mmio_ro_ranges
- fix doc
Changes in v4:
- minor fix for cmdline parsing
- make sharing opt-in build time, with option marked as EXPERIMENTAL
- change cmdline syntax to share=<bool>|hwdom
- make share=hwdom default (if enabled build-time)
Changes in v3:
- adjust for xhci-dbc rename
- adjust for dbc_ensure_running() split
- wrap long lines
- add runtime option for sharing USB controller
---
docs/misc/xen-command-line.pandoc | 14 ++-
xen/drivers/char/xhci-dbc.c | 129 +++++++++++++++++++++++++++++--
2 files changed, 134 insertions(+), 9 deletions(-)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 92d668afa06c..bbb4652bbfcd 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -723,7 +723,7 @@ Available alternatives, with their meaning, are:
### dbgp
> `= ehci[ <integer> | @pci<bus>:<slot>.<func> ]`
-> `= xhci[ <integer> | @pci<bus>:<slot>.<func> ]`
+> `= xhci[ <integer> | @pci<bus>:<slot>.<func> ][,share=<bool>|hwdom]`
Specify the USB controller to use, either by instance number (when going
over the PCI busses sequentially) or by PCI device (must be on segment 0).
@@ -731,6 +731,18 @@ over the PCI busses sequentially) or by PCI device (must be on segment 0).
Use `ehci` for EHCI debug port, use `xhci` for XHCI debug capability.
XHCI driver will wait indefinitely for the debug host to connect - make sure
the cable is connected.
+The `share` option for xhci controls who else can use the controller:
+* `no`: use the controller exclusively for console, even hardware domain
+ (dom0) cannot use it
+* `hwdom`: hardware domain may use the controller too, ports not used for debug
+ console will be available for normal devices; this is the default
+* `yes`: the controller can be assigned to any domain; it is not safe to assign
+ the controller to untrusted domain
+
+Choosing `share=hwdom` (the default) or `share=yes` allows a domain to reset the
+controller, which may cause small portion of the console output to be lost.
+
+The `share=yes` configuration is not security supported.
### debug_stack_lines
> `= <integer>`
diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c
index 5f47733c1801..5f92234a9594 100644
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -23,6 +23,7 @@
#include <xen/iommu.h>
#include <xen/mm.h>
#include <xen/param.h>
+#include <xen/rangeset.h>
#include <xen/serial.h>
#include <xen/timer.h>
#include <xen/types.h>
@@ -232,6 +233,12 @@ struct dbc_work_ring {
uint64_t dma;
};
+enum xhci_share {
+ XHCI_SHARE_HWDOM = 0,
+ XHCI_SHARE_NONE,
+ XHCI_SHARE_ANY
+};
+
struct dbc {
struct dbc_reg __iomem *dbc_reg;
struct xhci_dbc_ctx *dbc_ctx;
@@ -250,6 +257,7 @@ struct dbc {
bool enable; /* whether dbgp=xhci was set at all */
bool open;
+ enum xhci_share share;
unsigned int xhc_num; /* look for n-th xhc */
};
@@ -961,13 +969,56 @@ static bool __init dbc_open(struct dbc *dbc)
}
/*
- * Ensure DbC is still running, handle events, and possibly re-enable if cable
- * was re-plugged. Returns true if DbC is operational.
+ * Ensure DbC is still running, handle events, and possibly
+ * re-enable/re-configure if cable was re-plugged or controller was reset.
+ * Returns true if DbC is operational.
*/
static bool dbc_ensure_running(struct dbc *dbc)
{
struct dbc_reg *reg = dbc->dbc_reg;
uint32_t ctrl;
+ uint16_t cmd;
+
+ if ( dbc->share != XHCI_SHARE_NONE )
+ {
+ /*
+ * Re-enable memory decoding and later bus mastering, if dom0 (or
+ * other) disabled it in the meantime.
+ */
+ cmd = pci_conf_read16(dbc->sbdf, PCI_COMMAND);
+ if ( !(cmd & PCI_COMMAND_MEMORY) )
+ {
+ cmd |= PCI_COMMAND_MEMORY;
+ pci_conf_write16(dbc->sbdf, PCI_COMMAND, cmd);
+ }
+
+ /*
+ * FIXME: Make Linux coordinate XHCI reset, so the DbC driver can
+ * prepare for it properly, instead of only detecting it after the
+ * fact. See EHCI driver for similar handling.
+ */
+ if ( dbc->open && !(readl(®->ctrl) & (1U << DBC_CTRL_DCE)) )
+ {
+ if ( !dbc_init_dbc(dbc) )
+ return false;
+
+ dbc_init_work_ring(dbc, &dbc->dbc_owork);
+ dbc_enable_dbc(dbc);
+ }
+ else
+ {
+ /*
+ * dbc_init_dbc() takes care about it, so check only if it wasn't
+ * called.
+ */
+ cmd = pci_conf_read16(dbc->sbdf, PCI_COMMAND);
+ if ( !(cmd & PCI_COMMAND_MASTER) )
+ {
+ cmd |= PCI_COMMAND_MASTER;
+ pci_conf_write16(dbc->sbdf, PCI_COMMAND, cmd);
+ }
+ }
+ }
dbc_pop_events(dbc);
@@ -1136,10 +1187,38 @@ static void __init cf_check dbc_uart_init_postirq(struct serial_port *port)
init_timer(&uart->timer, dbc_uart_poll, port, 0);
set_timer(&uart->timer, NOW() + MILLISECS(1));
- if ( pci_ro_device(0, uart->dbc.sbdf.bus, uart->dbc.sbdf.devfn) )
- printk(XENLOG_WARNING
- "Failed to mark read-only %pp used for XHCI console\n",
- &uart->dbc.sbdf);
+ switch ( uart->dbc.share )
+ {
+ case XHCI_SHARE_NONE:
+ if ( pci_ro_device(0, uart->dbc.sbdf.bus, uart->dbc.sbdf.devfn) )
+ printk(XENLOG_WARNING
+ "Failed to mark read-only %pp used for XHCI console\n",
+ &uart->dbc.sbdf);
+ break;
+ case XHCI_SHARE_HWDOM:
+ if ( pci_hide_device(0, uart->dbc.sbdf.bus, uart->dbc.sbdf.devfn) )
+ printk(XENLOG_WARNING
+ "Failed to hide %pp used for XHCI console\n",
+ &uart->dbc.sbdf);
+ break;
+ case XHCI_SHARE_ANY:
+ /* Do not hide. */
+ break;
+ }
+#ifdef CONFIG_X86
+ /*
+ * This marks the whole page as R/O, which may include other registers
+ * unrelated to DbC. Xen needs only DbC area protected, but it seems
+ * Linux's XHCI driver (as of 5.18) works without writting to the whole
+ * page, so keep it simple.
+ */
+ if ( rangeset_add_range(mmio_ro_ranges,
+ PFN_DOWN(uart->dbc.xhc_mmio_phys + uart->dbc.xhc_dbc_offset),
+ PFN_UP(uart->dbc.xhc_mmio_phys + uart->dbc.xhc_dbc_offset +
+ sizeof(*uart->dbc.dbc_reg)) - 1) )
+ printk(XENLOG_INFO
+ "Error while adding MMIO range of device to mmio_ro_ranges\n");
+#endif
}
static int cf_check dbc_uart_tx_ready(struct serial_port *port)
@@ -1207,13 +1286,15 @@ static int __init xhci_parse_dbgp(const char *opt_dbgp)
{
struct dbc_uart *uart = &dbc_uart;
struct dbc *dbc = &uart->dbc;
- const char *e;
+ const char *e, *opt;
+ int val;
if ( strncmp(opt_dbgp, "xhci", 4) )
return 0;
memset(dbc, 0, sizeof(*dbc));
+ e = &opt_dbgp[4];
if ( isdigit(opt_dbgp[4]) )
{
dbc->xhc_num = simple_strtoul(opt_dbgp + 4, &e, 10);
@@ -1223,7 +1304,7 @@ static int __init xhci_parse_dbgp(const char *opt_dbgp)
unsigned int bus, slot, func;
e = parse_pci(opt_dbgp + 8, NULL, &bus, &slot, &func);
- if ( !e || *e )
+ if ( !e || (*e && *e != ',') )
{
printk(XENLOG_ERR
"Invalid dbgp= PCI device spec: '%s'\n",
@@ -1233,6 +1314,38 @@ static int __init xhci_parse_dbgp(const char *opt_dbgp)
dbc->sbdf = PCI_SBDF(0, bus, slot, func);
}
+ opt = e;
+
+ /* other options */
+ while ( *opt == ',' )
+ {
+ opt++;
+ e = strchr(opt, ',');
+ if ( !e )
+ e = strchr(opt, '\0');
+
+ if ( (val = parse_boolean("share", opt, e)) != -1 )
+ {
+ if ( val == -2 && !cmdline_strcmp(opt + 6, "hwdom") )
+ dbc->share = XHCI_SHARE_HWDOM;
+ else if ( val == 0 )
+ dbc->share = XHCI_SHARE_NONE;
+ else if ( val == 1 )
+ dbc->share = XHCI_SHARE_ANY;
+ else
+ break;
+ }
+ else
+ break;
+
+ opt = e;
+ }
+
+ if ( *opt )
+ {
+ printk(XENLOG_ERR "Invalid dbgp= parameters: '%s'\n", opt);
+ return -EINVAL;
+ }
dbc->enable = true;
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v7 08/11] IOMMU/VT-d: wire common device reserved memory API
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
` (6 preceding siblings ...)
2022-09-17 2:51 ` [PATCH v7 07/11] drivers/char: allow driving the rest of XHCI by a domain while Xen uses DbC Marek Marczykowski-Górecki
@ 2022-09-17 2:51 ` Marek Marczykowski-Górecki
2022-09-22 9:29 ` Marek Marczykowski-Górecki
2022-09-23 7:21 ` Tian, Kevin
2022-09-17 2:51 ` [PATCH v7 09/11] console: support multiple serial console simultaneously Marek Marczykowski-Górecki
` (3 subsequent siblings)
11 siblings, 2 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel; +Cc: Marek Marczykowski-Górecki, Kevin Tian
Re-use rmrr= parameter handling code to handle common device reserved
memory.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Changes in v3:
- make MAX_USER_RMRR_PAGES applicable only to user-configured RMRR
---
xen/drivers/passthrough/vtd/dmar.c | 201 +++++++++++++++++-------------
1 file changed, 119 insertions(+), 82 deletions(-)
diff --git a/xen/drivers/passthrough/vtd/dmar.c b/xen/drivers/passthrough/vtd/dmar.c
index 367304c8739c..3df5f6b69719 100644
--- a/xen/drivers/passthrough/vtd/dmar.c
+++ b/xen/drivers/passthrough/vtd/dmar.c
@@ -861,111 +861,139 @@ static struct user_rmrr __initdata user_rmrrs[MAX_USER_RMRR];
/* Macro for RMRR inclusive range formatting. */
#define ERMRRU_FMT "[%lx-%lx]"
-#define ERMRRU_ARG(eru) eru.base_pfn, eru.end_pfn
+#define ERMRRU_ARG base_pfn, end_pfn
+
+static int __init add_one_user_rmrr(unsigned long base_pfn,
+ unsigned long end_pfn,
+ unsigned int dev_count,
+ uint32_t *sbdf);
static int __init add_user_rmrr(void)
{
+ unsigned int i;
+ int ret;
+
+ for ( i = 0; i < nr_rmrr; i++ )
+ {
+ ret = add_one_user_rmrr(user_rmrrs[i].base_pfn,
+ user_rmrrs[i].end_pfn,
+ user_rmrrs[i].dev_count,
+ user_rmrrs[i].sbdf);
+ if ( ret < 0 )
+ return ret;
+ }
+ return 0;
+}
+
+/* Returns 1 on success, 0 when ignoring and < 0 on error. */
+static int __init add_one_user_rmrr(unsigned long base_pfn,
+ unsigned long end_pfn,
+ unsigned int dev_count,
+ uint32_t *sbdf)
+{
struct acpi_rmrr_unit *rmrr, *rmrru;
- unsigned int idx, seg, i;
- unsigned long base, end;
+ unsigned int idx, seg;
+ unsigned long base_iter;
bool overlap;
- for ( i = 0; i < nr_rmrr; i++ )
+ if ( iommu_verbose )
+ printk(XENLOG_DEBUG VTDPREFIX
+ "Adding RMRR for %d device ([0]: %#x) range "ERMRRU_FMT"\n",
+ dev_count, sbdf[0], ERMRRU_ARG);
+
+ if ( base_pfn > end_pfn )
{
- base = user_rmrrs[i].base_pfn;
- end = user_rmrrs[i].end_pfn;
+ printk(XENLOG_ERR VTDPREFIX
+ "Invalid RMRR Range "ERMRRU_FMT"\n",
+ ERMRRU_ARG);
+ return 0;
+ }
- if ( base > end )
+ overlap = false;
+ list_for_each_entry(rmrru, &acpi_rmrr_units, list)
+ {
+ if ( pfn_to_paddr(base_pfn) <= rmrru->end_address &&
+ rmrru->base_address <= pfn_to_paddr(end_pfn) )
{
printk(XENLOG_ERR VTDPREFIX
- "Invalid RMRR Range "ERMRRU_FMT"\n",
- ERMRRU_ARG(user_rmrrs[i]));
- continue;
+ "Overlapping RMRRs: "ERMRRU_FMT" and [%lx-%lx]\n",
+ ERMRRU_ARG,
+ paddr_to_pfn(rmrru->base_address),
+ paddr_to_pfn(rmrru->end_address));
+ overlap = true;
+ break;
}
+ }
+ /* Don't add overlapping RMRR. */
+ if ( overlap )
+ return 0;
- if ( (end - base) >= MAX_USER_RMRR_PAGES )
+ base_iter = base_pfn;
+ do
+ {
+ if ( !mfn_valid(_mfn(base_iter)) )
{
printk(XENLOG_ERR VTDPREFIX
- "RMRR range "ERMRRU_FMT" exceeds "\
- __stringify(MAX_USER_RMRR_PAGES)" pages\n",
- ERMRRU_ARG(user_rmrrs[i]));
- continue;
+ "Invalid pfn in RMRR range "ERMRRU_FMT"\n",
+ ERMRRU_ARG);
+ break;
}
+ } while ( base_iter++ < end_pfn );
- overlap = false;
- list_for_each_entry(rmrru, &acpi_rmrr_units, list)
- {
- if ( pfn_to_paddr(base) <= rmrru->end_address &&
- rmrru->base_address <= pfn_to_paddr(end) )
- {
- printk(XENLOG_ERR VTDPREFIX
- "Overlapping RMRRs: "ERMRRU_FMT" and [%lx-%lx]\n",
- ERMRRU_ARG(user_rmrrs[i]),
- paddr_to_pfn(rmrru->base_address),
- paddr_to_pfn(rmrru->end_address));
- overlap = true;
- break;
- }
- }
- /* Don't add overlapping RMRR. */
- if ( overlap )
- continue;
+ /* Invalid pfn in range as the loop ended before end_pfn was reached. */
+ if ( base_iter <= end_pfn )
+ return 0;
- do
- {
- if ( !mfn_valid(_mfn(base)) )
- {
- printk(XENLOG_ERR VTDPREFIX
- "Invalid pfn in RMRR range "ERMRRU_FMT"\n",
- ERMRRU_ARG(user_rmrrs[i]));
- break;
- }
- } while ( base++ < end );
+ rmrr = xzalloc(struct acpi_rmrr_unit);
+ if ( !rmrr )
+ return -ENOMEM;
- /* Invalid pfn in range as the loop ended before end_pfn was reached. */
- if ( base <= end )
- continue;
+ rmrr->scope.devices = xmalloc_array(u16, dev_count);
+ if ( !rmrr->scope.devices )
+ {
+ xfree(rmrr);
+ return -ENOMEM;
+ }
- rmrr = xzalloc(struct acpi_rmrr_unit);
- if ( !rmrr )
- return -ENOMEM;
+ seg = 0;
+ for ( idx = 0; idx < dev_count; idx++ )
+ {
+ rmrr->scope.devices[idx] = sbdf[idx];
+ seg |= PCI_SEG(sbdf[idx]);
+ }
+ if ( seg != PCI_SEG(sbdf[0]) )
+ {
+ printk(XENLOG_ERR VTDPREFIX
+ "Segments are not equal for RMRR range "ERMRRU_FMT"\n",
+ ERMRRU_ARG);
+ scope_devices_free(&rmrr->scope);
+ xfree(rmrr);
+ return 0;
+ }
- rmrr->scope.devices = xmalloc_array(u16, user_rmrrs[i].dev_count);
- if ( !rmrr->scope.devices )
- {
- xfree(rmrr);
- return -ENOMEM;
- }
+ rmrr->segment = seg;
+ rmrr->base_address = pfn_to_paddr(base_pfn);
+ /* Align the end_address to the end of the page */
+ rmrr->end_address = pfn_to_paddr(end_pfn) | ~PAGE_MASK;
+ rmrr->scope.devices_cnt = dev_count;
- seg = 0;
- for ( idx = 0; idx < user_rmrrs[i].dev_count; idx++ )
- {
- rmrr->scope.devices[idx] = user_rmrrs[i].sbdf[idx];
- seg |= PCI_SEG(user_rmrrs[i].sbdf[idx]);
- }
- if ( seg != PCI_SEG(user_rmrrs[i].sbdf[0]) )
- {
- printk(XENLOG_ERR VTDPREFIX
- "Segments are not equal for RMRR range "ERMRRU_FMT"\n",
- ERMRRU_ARG(user_rmrrs[i]));
- scope_devices_free(&rmrr->scope);
- xfree(rmrr);
- continue;
- }
+ if ( register_one_rmrr(rmrr) )
+ printk(XENLOG_ERR VTDPREFIX
+ "Could not register RMMR range "ERMRRU_FMT"\n",
+ ERMRRU_ARG);
- rmrr->segment = seg;
- rmrr->base_address = pfn_to_paddr(user_rmrrs[i].base_pfn);
- /* Align the end_address to the end of the page */
- rmrr->end_address = pfn_to_paddr(user_rmrrs[i].end_pfn) | ~PAGE_MASK;
- rmrr->scope.devices_cnt = user_rmrrs[i].dev_count;
+ return 1;
+}
- if ( register_one_rmrr(rmrr) )
- printk(XENLOG_ERR VTDPREFIX
- "Could not register RMMR range "ERMRRU_FMT"\n",
- ERMRRU_ARG(user_rmrrs[i]));
- }
+static int __init cf_check add_one_extra_rmrr(xen_pfn_t start, xen_ulong_t nr, u32 id, void *ctxt)
+{
+ u32 sbdf_array[] = { id };
+ return add_one_user_rmrr(start, start+nr, 1, sbdf_array);
+}
- return 0;
+static int __init add_extra_rmrr(void)
+{
+ return iommu_get_extra_reserved_device_memory(add_one_extra_rmrr, NULL);
}
#include <asm/tboot.h>
@@ -1010,7 +1038,7 @@ int __init acpi_dmar_init(void)
{
iommu_init_ops = &intel_iommu_init_ops;
- return add_user_rmrr();
+ return add_user_rmrr() || add_extra_rmrr();
}
return ret;
@@ -1108,6 +1136,15 @@ static int __init cf_check parse_rmrr_param(const char *str)
else
end = start;
+ if ( (end - start) >= MAX_USER_RMRR_PAGES )
+ {
+ printk(XENLOG_ERR VTDPREFIX
+ "RMRR range "ERMRRU_FMT" exceeds "\
+ __stringify(MAX_USER_RMRR_PAGES)" pages\n",
+ start, end);
+ return -E2BIG;
+ }
+
user_rmrrs[nr_rmrr].base_pfn = start;
user_rmrrs[nr_rmrr].end_pfn = end;
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v7 09/11] console: support multiple serial console simultaneously
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
` (7 preceding siblings ...)
2022-09-17 2:51 ` [PATCH v7 08/11] IOMMU/VT-d: wire common device reserved memory API Marek Marczykowski-Górecki
@ 2022-09-17 2:51 ` Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 10/11] drivers/char: suspend handling in XHCI console driver Marek Marczykowski-Górecki
` (2 subsequent siblings)
11 siblings, 0 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel
Cc: Marek Marczykowski-Górecki, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Stefano Stabellini, Wei Liu
Previously only one serial console was supported at the same time. Using
console=com1,dbgp,vga silently ignored all but last serial console (in
this case: only dbgp and vga were active).
Fix this by storing not a single sercon_handle, but an array of them, up
to MAX_SERCONS entries. The value of MAX_SERCONS can be chosen in
kconfig, the default (4) is arbitrary, inspired by the number of
SERHND_IDX values.
Make console_steal() aware of multiple consoles too. It can now either
steal output from specific console (for gdbstub), or from all of them at
once (for console suspend).
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
Changes in v4:
- use unsigned int for loop counters
- other minor changes
Changes in v3:
- adjust console_steal() for multiple consoles too
- add MAX_SERCONS to kconfig
- add warning about sync_console impact
- add warning if too many consoles are configured
- log issue with PCI spec parsing
---
docs/misc/xen-command-line.pandoc | 4 +-
xen/drivers/char/Kconfig | 11 ++++-
xen/drivers/char/console.c | 98 ++++++++++++++++++++++++--------
xen/include/xen/serial.h | 1 +-
4 files changed, 92 insertions(+), 22 deletions(-)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index bbb4652bbfcd..1c755563c40d 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -435,6 +435,9 @@ only available when used together with `pv-in-pvh`.
`none` indicates that Xen should not use a console. This option only
makes sense on its own.
+Specifying more than one serial console will increase console latency,
+especially when `sync_console` option is used.
+
### console_timestamps
> `= none | date | datems | boot | raw`
@@ -2417,6 +2420,7 @@ vulnerabilities.
Flag to force synchronous console output. Useful for debugging, but
not suitable for production environments due to incurred overhead.
+If multiple consoles are configured, the incurred overhead is even bigger.
### tboot (x86)
> `= 0x<phys_addr>`
diff --git a/xen/drivers/char/Kconfig b/xen/drivers/char/Kconfig
index 06350c387371..7b5ff0c414ec 100644
--- a/xen/drivers/char/Kconfig
+++ b/xen/drivers/char/Kconfig
@@ -85,6 +85,17 @@ config SERIAL_TX_BUFSIZE
Default value is 16384 (16kiB).
+config MAX_SERCONS
+ int "Maximum number of serial consoles active at once"
+ default 4
+ help
+ Controls how many serial consoles can be active at once. Configuring more
+ using `console=` parameter will be ignored.
+ When multiple consoles are configured, overhead of `sync_console` option
+ is even bigger.
+
+ Default value is 4.
+
config XHCI
bool "XHCI DbC UART driver"
depends on X86
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index e8468c121ad0..60d42284f606 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -113,7 +113,9 @@ static char *__read_mostly conring = _conring;
static uint32_t __read_mostly conring_size = _CONRING_SIZE;
static uint32_t conringc, conringp;
-static int __read_mostly sercon_handle = -1;
+#define MAX_SERCONS CONFIG_MAX_SERCONS
+static int __read_mostly sercon_handle[MAX_SERCONS];
+static unsigned int __read_mostly nr_sercon_handle = 0;
#ifdef CONFIG_X86
/* Tristate: 0 disabled, 1 user enabled, -1 default enabled */
@@ -393,32 +395,61 @@ long read_console_ring(struct xen_sysctl_readconsole *op)
static char serial_rx_ring[SERIAL_RX_SIZE];
static unsigned int serial_rx_cons, serial_rx_prod;
-static void (*serial_steal_fn)(const char *, size_t nr) = early_puts;
+/* The last entry means "steal from all consoles" */
+static void (*serial_steal_fn[])(const char *, size_t nr) = {
+ [MAX_SERCONS] = early_puts,
+};
+/*
+ * Redirect console *handle* output to *fn*. Use SERHND_STEAL_ALL as *handle* to
+ * redirect all the consoles.
+ */
int console_steal(int handle, void (*fn)(const char *, size_t nr))
{
- if ( (handle == -1) || (handle != sercon_handle) )
- return 0;
+ unsigned int i;
+
+ if ( handle == -1 )
+ return -ENOENT;
+ if ( serial_steal_fn[MAX_SERCONS] != NULL )
+ return -EBUSY;
+ if ( handle == SERHND_STEAL_ALL )
+ {
+ serial_steal_fn[MAX_SERCONS] = fn;
+ return MAX_SERCONS;
+ }
+ for ( i = 0; i < nr_sercon_handle; i++ )
+ if ( handle == sercon_handle[i] )
+ break;
+ if ( i == nr_sercon_handle )
+ return -ENOENT;
- if ( serial_steal_fn != NULL )
+ if ( serial_steal_fn[i] != NULL )
return -EBUSY;
- serial_steal_fn = fn;
- return 1;
+ serial_steal_fn[i] = fn;
+ return i;
}
void console_giveback(int id)
{
- if ( id == 1 )
- serial_steal_fn = NULL;
+ if ( id >= 0 && id <= MAX_SERCONS )
+ serial_steal_fn[id] = NULL;
}
void console_serial_puts(const char *s, size_t nr)
{
- if ( serial_steal_fn != NULL )
- serial_steal_fn(s, nr);
+ unsigned int i;
+
+ if ( serial_steal_fn[MAX_SERCONS] != NULL )
+ serial_steal_fn[MAX_SERCONS](s, nr);
else
- serial_puts(sercon_handle, s, nr);
+ for ( i = 0; i < nr_sercon_handle; i++ )
+ {
+ if ( serial_steal_fn[i] != NULL )
+ serial_steal_fn[i](s, nr);
+ else
+ serial_puts(sercon_handle[i], s, nr);
+ }
/* Copy all serial output into PV console */
pv_console_puts(s, nr);
@@ -957,6 +988,7 @@ void __init console_init_preirq(void)
{
char *p;
int sh;
+ unsigned int i;
serial_init_preirq();
@@ -977,8 +1009,12 @@ void __init console_init_preirq(void)
continue;
else if ( (sh = serial_parse_handle(p)) >= 0 )
{
- sercon_handle = sh;
- serial_steal_fn = NULL;
+ if ( nr_sercon_handle < MAX_SERCONS )
+ sercon_handle[nr_sercon_handle++] = sh;
+ else
+ printk("Too many consoles (max %d), ignoring '%s'\n",
+ MAX_SERCONS, p);
+ serial_steal_fn[MAX_SERCONS] = NULL;
}
else
{
@@ -996,7 +1032,8 @@ void __init console_init_preirq(void)
opt_console_xen = 0;
#endif
- serial_set_rx_handler(sercon_handle, serial_rx);
+ for ( i = 0; i < nr_sercon_handle; i++ )
+ serial_set_rx_handler(sercon_handle[i], serial_rx);
pv_console_set_rx_handler(serial_rx);
/* HELLO WORLD --- start-of-day banner text. */
@@ -1014,7 +1051,8 @@ void __init console_init_preirq(void)
if ( opt_sync_console )
{
- serial_start_sync(sercon_handle);
+ for ( i = 0; i < nr_sercon_handle; i++ )
+ serial_start_sync(sercon_handle[i]);
add_taint(TAINT_SYNC_CONSOLE);
printk("Console output is synchronous.\n");
warning_add(warning_sync_console);
@@ -1121,13 +1159,19 @@ int __init console_has(const char *device)
void console_start_log_everything(void)
{
- serial_start_log_everything(sercon_handle);
+ unsigned int i;
+
+ for ( i = 0; i < nr_sercon_handle; i++ )
+ serial_start_log_everything(sercon_handle[i]);
atomic_inc(&print_everything);
}
void console_end_log_everything(void)
{
- serial_end_log_everything(sercon_handle);
+ unsigned int i;
+
+ for ( i = 0; i < nr_sercon_handle; i++ )
+ serial_end_log_everything(sercon_handle[i]);
atomic_dec(&print_everything);
}
@@ -1149,23 +1193,32 @@ void console_unlock_recursive_irqrestore(unsigned long flags)
void console_force_unlock(void)
{
+ unsigned int i;
+
watchdog_disable();
spin_debug_disable();
spin_lock_init(&console_lock);
- serial_force_unlock(sercon_handle);
+ for ( i = 0 ; i < nr_sercon_handle ; i++ )
+ serial_force_unlock(sercon_handle[i]);
console_locks_busted = 1;
console_start_sync();
}
void console_start_sync(void)
{
+ unsigned int i;
+
atomic_inc(&print_everything);
- serial_start_sync(sercon_handle);
+ for ( i = 0 ; i < nr_sercon_handle ; i++ )
+ serial_start_sync(sercon_handle[i]);
}
void console_end_sync(void)
{
- serial_end_sync(sercon_handle);
+ unsigned int i;
+
+ for ( i = 0; i < nr_sercon_handle; i++ )
+ serial_end_sync(sercon_handle[i]);
atomic_dec(&print_everything);
}
@@ -1291,7 +1344,8 @@ static int suspend_steal_id;
int console_suspend(void)
{
- suspend_steal_id = console_steal(sercon_handle, suspend_steal_fn);
+ if ( nr_sercon_handle )
+ suspend_steal_id = console_steal(SERHND_STEAL_ALL, suspend_steal_fn);
serial_suspend();
return 0;
}
diff --git a/xen/include/xen/serial.h b/xen/include/xen/serial.h
index f0aff7ea7661..226139841e71 100644
--- a/xen/include/xen/serial.h
+++ b/xen/include/xen/serial.h
@@ -100,6 +100,7 @@ struct uart_driver {
#define SERHND_HI (1<<2) /* Mux/demux each transferred char by MSB. */
#define SERHND_LO (1<<3) /* Ditto, except that the MSB is cleared. */
#define SERHND_COOKED (1<<4) /* Newline/carriage-return translation? */
+#define SERHND_STEAL_ALL 0xff /* Synthetic handle used in console_steal() */
/* Three-stage initialisation (before/during/after IRQ-subsystem setup). */
void serial_init_preirq(void);
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v7 10/11] drivers/char: suspend handling in XHCI console driver
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
` (8 preceding siblings ...)
2022-09-17 2:51 ` [PATCH v7 09/11] console: support multiple serial console simultaneously Marek Marczykowski-Górecki
@ 2022-09-17 2:51 ` Marek Marczykowski-Górecki
2022-09-20 15:07 ` Jan Beulich
2022-09-17 2:51 ` [PATCH v7 11/11] drivers/char: add console=ehci as an alias for console=dbgp Marek Marczykowski-Górecki
2022-09-19 13:33 ` [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Jan Beulich
11 siblings, 1 reply; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel
Cc: Marek Marczykowski-Górecki, Andrew Cooper, George Dunlap,
Jan Beulich, Julien Grall, Stefano Stabellini, Wei Liu
Similar to the EHCI driver - save/restore relevant BAR and command
register, re-configure DbC on resume and stop/start timer.
On resume trigger sending anything that was queued in the meantime.
Save full BAR value, instead of just the address part, to ease restoring
on resume.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
New in v7
Without this patch, the console is broken after S3, and in some cases
the suspend doesn't succeed at all (when xhci console is enabled).
Very similar (if not the same) functions might be used for coordinated
reset handling. I tried to include it in this patch too, but it's a bit
more involved, mostly due to share=yes case (PHYSDEVOP_dbgp_op can be
called by the hardware domain only).
---
xen/drivers/char/xhci-dbc.c | 55 +++++++++++++++++++++++++++++++++-----
1 file changed, 49 insertions(+), 6 deletions(-)
diff --git a/xen/drivers/char/xhci-dbc.c b/xen/drivers/char/xhci-dbc.c
index 5f92234a9594..81a4fd5b12c3 100644
--- a/xen/drivers/char/xhci-dbc.c
+++ b/xen/drivers/char/xhci-dbc.c
@@ -251,7 +251,7 @@ struct dbc {
struct xhci_string_descriptor *dbc_str;
pci_sbdf_t sbdf;
- uint64_t xhc_mmio_phys;
+ uint64_t bar_val;
uint64_t xhc_dbc_offset;
void __iomem *xhc_mmio;
@@ -259,6 +259,9 @@ struct dbc {
bool open;
enum xhci_share share;
unsigned int xhc_num; /* look for n-th xhc */
+ /* state saved across suspend */
+ bool suspended;
+ uint16_t pci_cr;
};
static void *dbc_sys_map_xhc(uint64_t phys, size_t size)
@@ -358,8 +361,9 @@ static bool __init dbc_init_xhc(struct dbc *dbc)
pci_conf_write16(dbc->sbdf, PCI_COMMAND, cmd);
- dbc->xhc_mmio_phys = (bar0 & PCI_BASE_ADDRESS_MEM_MASK) | (bar1 << 32);
- dbc->xhc_mmio = dbc_sys_map_xhc(dbc->xhc_mmio_phys, xhc_mmio_size);
+ dbc->bar_val = bar0 | (bar1 << 32);
+ dbc->xhc_mmio = dbc_sys_map_xhc(dbc->bar_val & PCI_BASE_ADDRESS_MEM_MASK,
+ xhc_mmio_size);
if ( dbc->xhc_mmio == NULL )
return false;
@@ -979,6 +983,9 @@ static bool dbc_ensure_running(struct dbc *dbc)
uint32_t ctrl;
uint16_t cmd;
+ if ( dbc->suspended )
+ return false;
+
if ( dbc->share != XHCI_SHARE_NONE )
{
/*
@@ -1213,9 +1220,11 @@ static void __init cf_check dbc_uart_init_postirq(struct serial_port *port)
* page, so keep it simple.
*/
if ( rangeset_add_range(mmio_ro_ranges,
- PFN_DOWN(uart->dbc.xhc_mmio_phys + uart->dbc.xhc_dbc_offset),
- PFN_UP(uart->dbc.xhc_mmio_phys + uart->dbc.xhc_dbc_offset +
- sizeof(*uart->dbc.dbc_reg)) - 1) )
+ PFN_DOWN((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
+ uart->dbc.xhc_dbc_offset),
+ PFN_UP((uart->dbc.bar_val & PCI_BASE_ADDRESS_MEM_MASK) +
+ uart->dbc.xhc_dbc_offset +
+ sizeof(*uart->dbc.dbc_reg)) - 1) )
printk(XENLOG_INFO
"Error while adding MMIO range of device to mmio_ro_ranges\n");
#endif
@@ -1255,6 +1264,38 @@ static void cf_check dbc_uart_flush(struct serial_port *port)
set_timer(&uart->timer, goal);
}
+static void cf_check dbc_uart_suspend(struct serial_port *port)
+{
+ struct dbc_uart *uart = port->uart;
+ struct dbc *dbc = &uart->dbc;
+
+ dbc_pop_events(dbc);
+ stop_timer(&uart->timer);
+ dbc->pci_cr = pci_conf_read16(dbc->sbdf, PCI_COMMAND);
+ dbc->suspended = true;
+}
+
+static void cf_check dbc_uart_resume(struct serial_port *port)
+{
+ struct dbc_uart *uart = port->uart;
+ struct dbc *dbc = &uart->dbc;
+
+ pci_conf_write32(dbc->sbdf, PCI_BASE_ADDRESS_0, dbc->bar_val & 0xFFFFFFFF);
+ pci_conf_write32(dbc->sbdf, PCI_BASE_ADDRESS_1, dbc->bar_val >> 32);
+ pci_conf_write16(dbc->sbdf, PCI_COMMAND, dbc->pci_cr);
+
+ if ( !dbc_init_dbc(dbc) )
+ {
+ dbc_error("resume failed\n");
+ return;
+ }
+
+ dbc_enable_dbc(dbc);
+ dbc->suspended = false;
+ dbc_flush(dbc, &dbc->dbc_oring, &dbc->dbc_owork);
+ set_timer(&uart->timer, NOW() + MICROSECS(DBC_POLL_INTERVAL));
+}
+
static struct uart_driver dbc_uart_driver = {
.init_preirq = dbc_uart_init_preirq,
.init_postirq = dbc_uart_init_postirq,
@@ -1262,6 +1303,8 @@ static struct uart_driver dbc_uart_driver = {
.putc = dbc_uart_putc,
.getc = dbc_uart_getc,
.flush = dbc_uart_flush,
+ .suspend = dbc_uart_suspend,
+ .resume = dbc_uart_resume,
};
/* Those are accessed via DMA. */
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* [PATCH v7 11/11] drivers/char: add console=ehci as an alias for console=dbgp
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
` (9 preceding siblings ...)
2022-09-17 2:51 ` [PATCH v7 10/11] drivers/char: suspend handling in XHCI console driver Marek Marczykowski-Górecki
@ 2022-09-17 2:51 ` Marek Marczykowski-Górecki
2022-09-19 8:49 ` Jan Beulich
2022-09-19 13:33 ` [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Jan Beulich
11 siblings, 1 reply; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-17 2:51 UTC (permalink / raw)
To: xen-devel
Cc: Marek Marczykowski-Górecki, Jan Beulich, Andrew Cooper,
George Dunlap, Julien Grall, Stefano Stabellini, Wei Liu
Make it consistent with console=xhci.
Suggested-by: Jan Beulich <jbeulich@suse.com>
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
docs/misc/xen-command-line.pandoc | 4 ++--
xen/drivers/char/serial.c | 6 ++++++
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line.pandoc
index 1c755563c40d..74b519f0c5bd 100644
--- a/docs/misc/xen-command-line.pandoc
+++ b/docs/misc/xen-command-line.pandoc
@@ -409,7 +409,7 @@ The following are examples of correct specifications:
Specify the size of the console ring buffer.
### console
-> `= List of [ vga | com1[H,L] | com2[H,L] | pv | dbgp | xhci | none ]`
+> `= List of [ vga | com1[H,L] | com2[H,L] | pv | dbgp | ehci | xhci | none ]`
> Default: `console=com1,vga`
@@ -428,7 +428,7 @@ cleared. This allows a single port to be shared by two subsystems
`pv` indicates that Xen should use Xen's PV console. This option is
only available when used together with `pv-in-pvh`.
-`dbgp` indicates that Xen should use a USB2 debug port.
+`dbgp` or `ehci` indicates that Xen should use a USB2 debug port.
`xhci` indicates that Xen should use a USB3 debug port.
diff --git a/xen/drivers/char/serial.c b/xen/drivers/char/serial.c
index 9d9445039232..00efe69574f3 100644
--- a/xen/drivers/char/serial.c
+++ b/xen/drivers/char/serial.c
@@ -311,6 +311,12 @@ int __init serial_parse_handle(const char *conf)
goto common;
}
+ if ( !strncmp(conf, "ehci", 4) && (!conf[4] || conf[4] == ',') )
+ {
+ handle = SERHND_DBGP;
+ goto common;
+ }
+
if ( !strncmp(conf, "xhci", 4) && (!conf[4] || conf[4] == ',') )
{
handle = SERHND_XHCI;
--
git-series 0.9.1
^ permalink raw reply related [flat|nested] 21+ messages in thread
* Re: [PATCH v7 11/11] drivers/char: add console=ehci as an alias for console=dbgp
2022-09-17 2:51 ` [PATCH v7 11/11] drivers/char: add console=ehci as an alias for console=dbgp Marek Marczykowski-Górecki
@ 2022-09-19 8:49 ` Jan Beulich
0 siblings, 0 replies; 21+ messages in thread
From: Jan Beulich @ 2022-09-19 8:49 UTC (permalink / raw)
To: Marek Marczykowski-Górecki
Cc: Andrew Cooper, George Dunlap, Julien Grall, Stefano Stabellini,
Wei Liu, xen-devel
On 17.09.2022 04:51, Marek Marczykowski-Górecki wrote:
> Make it consistent with console=xhci.
>
> Suggested-by: Jan Beulich <jbeulich@suse.com>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Acked-by: Jan Beulich <jbeulich@suse.com>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
` (10 preceding siblings ...)
2022-09-17 2:51 ` [PATCH v7 11/11] drivers/char: add console=ehci as an alias for console=dbgp Marek Marczykowski-Górecki
@ 2022-09-19 13:33 ` Jan Beulich
2022-09-19 13:49 ` Marek Marczykowski-Górecki
2022-09-20 1:38 ` Henry Wang
11 siblings, 2 replies; 21+ messages in thread
From: Jan Beulich @ 2022-09-19 13:33 UTC (permalink / raw)
To: Henry Wang
Cc: Andrew Cooper, George Dunlap, Julien Grall, Stefano Stabellini,
Wei Liu, Roger Pau Monné,
Paul Durrant, Kevin Tian, Connor Davis,
Marek Marczykowski-Górecki, xen-devel
On 17.09.2022 04:51, Marek Marczykowski-Górecki wrote:
> This is integration of https://github.com/connojd/xue into mainline Xen.
> This patch series includes several patches that I made in the process, some are
> very loosely related.
>
> The driver developed by Connor supports console via USB3 debug capability. The
> capability is designed to operate mostly independently of normal XHCI driver,
> so this patch series allows dom0 to drive standard USB3 controller part, while
> Xen uses DbC for console output.
>[...]
> Marek Marczykowski-Górecki (11):
> drivers/char: allow using both dbgp=xhci and dbgp=ehci
> IOMMU: add common API for device reserved memory
> IOMMU/AMD: wire common device reserved memory API
> drivers/char: mark DMA buffers as reserved for the XHCI
> drivers/char: add RX support to the XHCI driver
> drivers/char: fix handling cable re-plug in XHCI console driver
> drivers/char: allow driving the rest of XHCI by a domain while Xen uses DbC
> IOMMU/VT-d: wire common device reserved memory API
> console: support multiple serial console simultaneously
> drivers/char: suspend handling in XHCI console driver
> drivers/char: add console=ehci as an alias for console=dbgp
Henry,
this series is kind of on the edge between a feature submission and
corrections to existing code, as the base patch introducing the new
driver was merged only recently, and at least some of the things here
aren't clearly "bug" fixes. Additionally it's on the side of larger
ones considering the point in time.
To summarize state: Patches 2-7 are ready to be committed, and Marek
tells me that they're independent of patch 1 (except for a context
conflict). Patch 11 probably also falls in this category. Patch 10,
otoh, is pretty likely to be viewed as a new feature, and hence
likely wants postponing. In any event - if I was to commit any of
these, this couldn't happen earlier than next Monday, as the laptop
I'm currently working with is not (yet) set up to do commits from.
Do you have any particular opinion on the disposition of this series?
Thanks, Jan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability
2022-09-19 13:33 ` [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Jan Beulich
@ 2022-09-19 13:49 ` Marek Marczykowski-Górecki
2022-09-20 1:38 ` Henry Wang
1 sibling, 0 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-19 13:49 UTC (permalink / raw)
To: Henry Wang
Cc: Jan Beulich, Andrew Cooper, George Dunlap, Julien Grall,
Stefano Stabellini, Wei Liu, Roger Pau Monné,
Paul Durrant, Kevin Tian, Connor Davis, xen-devel
[-- Attachment #1: Type: text/plain, Size: 2849 bytes --]
On Mon, Sep 19, 2022 at 03:33:55PM +0200, Jan Beulich wrote:
> On 17.09.2022 04:51, Marek Marczykowski-Górecki wrote:
> > This is integration of https://github.com/connojd/xue into mainline Xen.
> > This patch series includes several patches that I made in the process, some are
> > very loosely related.
> >
> > The driver developed by Connor supports console via USB3 debug capability. The
> > capability is designed to operate mostly independently of normal XHCI driver,
> > so this patch series allows dom0 to drive standard USB3 controller part, while
> > Xen uses DbC for console output.
> >[...]
> > Marek Marczykowski-Górecki (11):
> > drivers/char: allow using both dbgp=xhci and dbgp=ehci
> > IOMMU: add common API for device reserved memory
> > IOMMU/AMD: wire common device reserved memory API
> > drivers/char: mark DMA buffers as reserved for the XHCI
> > drivers/char: add RX support to the XHCI driver
> > drivers/char: fix handling cable re-plug in XHCI console driver
> > drivers/char: allow driving the rest of XHCI by a domain while Xen uses DbC
> > IOMMU/VT-d: wire common device reserved memory API
> > console: support multiple serial console simultaneously
> > drivers/char: suspend handling in XHCI console driver
> > drivers/char: add console=ehci as an alias for console=dbgp
>
> Henry,
>
> this series is kind of on the edge between a feature submission and
> corrections to existing code, as the base patch introducing the new
> driver was merged only recently, and at least some of the things here
> aren't clearly "bug" fixes. Additionally it's on the side of larger
> ones considering the point in time.
>
> To summarize state: Patches 2-7 are ready to be committed, and Marek
> tells me that they're independent of patch 1 (except for a context
> conflict). Patch 11 probably also falls in this category. Patch 10,
> otoh, is pretty likely to be viewed as a new feature, and hence
> likely wants postponing. In any event - if I was to commit any of
> these, this couldn't happen earlier than next Monday, as the laptop
> I'm currently working with is not (yet) set up to do commits from.
I'd like to add for the risk analysis, that most of this code get
exercised only when explicit dbgp=xhci parameter is given (which is a
new feature introduced with a patch already committed for 4.17). Exception
to this are patches:
- patch 1 touches also dbgp=ehci case
- patches 2, 3, and 8 (the last one missing an ack) touch IOMMU code in
general, but changes are rather not invasive
- patch 9 touches generic console code, but can be safely dropped (is
completely independent from the others)
> Do you have any particular opinion on the disposition of this series?
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* RE: [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability
2022-09-19 13:33 ` [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Jan Beulich
2022-09-19 13:49 ` Marek Marczykowski-Górecki
@ 2022-09-20 1:38 ` Henry Wang
1 sibling, 0 replies; 21+ messages in thread
From: Henry Wang @ 2022-09-20 1:38 UTC (permalink / raw)
To: Jan Beulich, Marek Marczykowski-Górecki
Cc: Andrew Cooper, George Dunlap, Julien Grall, Stefano Stabellini,
Wei Liu, Roger Pau Monné,
Paul Durrant, Kevin Tian, Connor Davis, xen-devel
Hi Jan,
> -----Original Message-----
> From: Jan Beulich <jbeulich@suse.com>
> > Marek Marczykowski-Górecki (11):
> > drivers/char: allow using both dbgp=xhci and dbgp=ehci
> > IOMMU: add common API for device reserved memory
> > IOMMU/AMD: wire common device reserved memory API
> > drivers/char: mark DMA buffers as reserved for the XHCI
> > drivers/char: add RX support to the XHCI driver
> > drivers/char: fix handling cable re-plug in XHCI console driver
> > drivers/char: allow driving the rest of XHCI by a domain while Xen uses
> DbC
> > IOMMU/VT-d: wire common device reserved memory API
> > console: support multiple serial console simultaneously
> > drivers/char: suspend handling in XHCI console driver
> > drivers/char: add console=ehci as an alias for console=dbgp
>
> Henry,
>
> this series is kind of on the edge between a feature submission and
> corrections to existing code, as the base patch introducing the new
> driver was merged only recently, and at least some of the things here
> aren't clearly "bug" fixes. Additionally it's on the side of larger
> ones considering the point in time.
>
> To summarize state: Patches 2-7 are ready to be committed, and Marek
> tells me that they're independent of patch 1 (except for a context
> conflict). Patch 11 probably also falls in this category. Patch 10,
> otoh, is pretty likely to be viewed as a new feature, and hence
> likely wants postponing. In any event - if I was to commit any of
> these, this couldn't happen earlier than next Monday, as the laptop
> I'm currently working with is not (yet) set up to do commits from.
>
> Do you have any particular opinion on the disposition of this series?
Thank you for the information and the detailed summary!
From the cover letter and the risk analysis from Marek, I am ok to
commit patch #2-#7 and patch#11, as long as they are independent
patches and will not break current driver. Thank you for taking care
of the committing :))
Since from the discussion earlier, the release-ack tag is a little bit too
early in this stage, it is up to you to add the release-ack tag when you
do the committing.
Kind regards,
Henry
>
> Thanks, Jan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v7 10/11] drivers/char: suspend handling in XHCI console driver
2022-09-17 2:51 ` [PATCH v7 10/11] drivers/char: suspend handling in XHCI console driver Marek Marczykowski-Górecki
@ 2022-09-20 15:07 ` Jan Beulich
0 siblings, 0 replies; 21+ messages in thread
From: Jan Beulich @ 2022-09-20 15:07 UTC (permalink / raw)
To: Marek Marczykowski-Górecki
Cc: Andrew Cooper, George Dunlap, Julien Grall, Stefano Stabellini,
Wei Liu, xen-devel
On 17.09.2022 04:51, Marek Marczykowski-Górecki wrote:
> @@ -259,6 +259,9 @@ struct dbc {
> bool open;
> enum xhci_share share;
> unsigned int xhc_num; /* look for n-th xhc */
> + /* state saved across suspend */
> + bool suspended;
> + uint16_t pci_cr;
> };
While perhaps of limited relevance right here, may I nevertheless
suggest to move the new boolean next to the other one in context.
That's both to avoid setting a bad precedent (of not using available
padding slots for data like this) and because imo the comment also
isn't really applicable to this field (but just the other one you
add). Preferably with that adjustment
Acked-by: Jan Beulich <jbeulich@suse.com>
Jan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v7 08/11] IOMMU/VT-d: wire common device reserved memory API
2022-09-17 2:51 ` [PATCH v7 08/11] IOMMU/VT-d: wire common device reserved memory API Marek Marczykowski-Górecki
@ 2022-09-22 9:29 ` Marek Marczykowski-Górecki
2022-09-23 7:21 ` Tian, Kevin
1 sibling, 0 replies; 21+ messages in thread
From: Marek Marczykowski-Górecki @ 2022-09-22 9:29 UTC (permalink / raw)
To: Kevin Tian; +Cc: xen-devel
[-- Attachment #1: Type: text/plain, Size: 9724 bytes --]
On Sat, Sep 17, 2022 at 04:51:27AM +0200, Marek Marczykowski-Górecki wrote:
> Re-use rmrr= parameter handling code to handle common device reserved
> memory.
>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Kevin, can you please review this patch? It's unchanged since v3, and
pending review for some moths already.
> ---
> Changes in v3:
> - make MAX_USER_RMRR_PAGES applicable only to user-configured RMRR
> ---
> xen/drivers/passthrough/vtd/dmar.c | 201 +++++++++++++++++-------------
> 1 file changed, 119 insertions(+), 82 deletions(-)
>
> diff --git a/xen/drivers/passthrough/vtd/dmar.c b/xen/drivers/passthrough/vtd/dmar.c
> index 367304c8739c..3df5f6b69719 100644
> --- a/xen/drivers/passthrough/vtd/dmar.c
> +++ b/xen/drivers/passthrough/vtd/dmar.c
> @@ -861,111 +861,139 @@ static struct user_rmrr __initdata user_rmrrs[MAX_USER_RMRR];
>
> /* Macro for RMRR inclusive range formatting. */
> #define ERMRRU_FMT "[%lx-%lx]"
> -#define ERMRRU_ARG(eru) eru.base_pfn, eru.end_pfn
> +#define ERMRRU_ARG base_pfn, end_pfn
> +
> +static int __init add_one_user_rmrr(unsigned long base_pfn,
> + unsigned long end_pfn,
> + unsigned int dev_count,
> + uint32_t *sbdf);
>
> static int __init add_user_rmrr(void)
> {
> + unsigned int i;
> + int ret;
> +
> + for ( i = 0; i < nr_rmrr; i++ )
> + {
> + ret = add_one_user_rmrr(user_rmrrs[i].base_pfn,
> + user_rmrrs[i].end_pfn,
> + user_rmrrs[i].dev_count,
> + user_rmrrs[i].sbdf);
> + if ( ret < 0 )
> + return ret;
> + }
> + return 0;
> +}
> +
> +/* Returns 1 on success, 0 when ignoring and < 0 on error. */
> +static int __init add_one_user_rmrr(unsigned long base_pfn,
> + unsigned long end_pfn,
> + unsigned int dev_count,
> + uint32_t *sbdf)
> +{
> struct acpi_rmrr_unit *rmrr, *rmrru;
> - unsigned int idx, seg, i;
> - unsigned long base, end;
> + unsigned int idx, seg;
> + unsigned long base_iter;
> bool overlap;
>
> - for ( i = 0; i < nr_rmrr; i++ )
> + if ( iommu_verbose )
> + printk(XENLOG_DEBUG VTDPREFIX
> + "Adding RMRR for %d device ([0]: %#x) range "ERMRRU_FMT"\n",
> + dev_count, sbdf[0], ERMRRU_ARG);
> +
> + if ( base_pfn > end_pfn )
> {
> - base = user_rmrrs[i].base_pfn;
> - end = user_rmrrs[i].end_pfn;
> + printk(XENLOG_ERR VTDPREFIX
> + "Invalid RMRR Range "ERMRRU_FMT"\n",
> + ERMRRU_ARG);
> + return 0;
> + }
>
> - if ( base > end )
> + overlap = false;
> + list_for_each_entry(rmrru, &acpi_rmrr_units, list)
> + {
> + if ( pfn_to_paddr(base_pfn) <= rmrru->end_address &&
> + rmrru->base_address <= pfn_to_paddr(end_pfn) )
> {
> printk(XENLOG_ERR VTDPREFIX
> - "Invalid RMRR Range "ERMRRU_FMT"\n",
> - ERMRRU_ARG(user_rmrrs[i]));
> - continue;
> + "Overlapping RMRRs: "ERMRRU_FMT" and [%lx-%lx]\n",
> + ERMRRU_ARG,
> + paddr_to_pfn(rmrru->base_address),
> + paddr_to_pfn(rmrru->end_address));
> + overlap = true;
> + break;
> }
> + }
> + /* Don't add overlapping RMRR. */
> + if ( overlap )
> + return 0;
>
> - if ( (end - base) >= MAX_USER_RMRR_PAGES )
> + base_iter = base_pfn;
> + do
> + {
> + if ( !mfn_valid(_mfn(base_iter)) )
> {
> printk(XENLOG_ERR VTDPREFIX
> - "RMRR range "ERMRRU_FMT" exceeds "\
> - __stringify(MAX_USER_RMRR_PAGES)" pages\n",
> - ERMRRU_ARG(user_rmrrs[i]));
> - continue;
> + "Invalid pfn in RMRR range "ERMRRU_FMT"\n",
> + ERMRRU_ARG);
> + break;
> }
> + } while ( base_iter++ < end_pfn );
>
> - overlap = false;
> - list_for_each_entry(rmrru, &acpi_rmrr_units, list)
> - {
> - if ( pfn_to_paddr(base) <= rmrru->end_address &&
> - rmrru->base_address <= pfn_to_paddr(end) )
> - {
> - printk(XENLOG_ERR VTDPREFIX
> - "Overlapping RMRRs: "ERMRRU_FMT" and [%lx-%lx]\n",
> - ERMRRU_ARG(user_rmrrs[i]),
> - paddr_to_pfn(rmrru->base_address),
> - paddr_to_pfn(rmrru->end_address));
> - overlap = true;
> - break;
> - }
> - }
> - /* Don't add overlapping RMRR. */
> - if ( overlap )
> - continue;
> + /* Invalid pfn in range as the loop ended before end_pfn was reached. */
> + if ( base_iter <= end_pfn )
> + return 0;
>
> - do
> - {
> - if ( !mfn_valid(_mfn(base)) )
> - {
> - printk(XENLOG_ERR VTDPREFIX
> - "Invalid pfn in RMRR range "ERMRRU_FMT"\n",
> - ERMRRU_ARG(user_rmrrs[i]));
> - break;
> - }
> - } while ( base++ < end );
> + rmrr = xzalloc(struct acpi_rmrr_unit);
> + if ( !rmrr )
> + return -ENOMEM;
>
> - /* Invalid pfn in range as the loop ended before end_pfn was reached. */
> - if ( base <= end )
> - continue;
> + rmrr->scope.devices = xmalloc_array(u16, dev_count);
> + if ( !rmrr->scope.devices )
> + {
> + xfree(rmrr);
> + return -ENOMEM;
> + }
>
> - rmrr = xzalloc(struct acpi_rmrr_unit);
> - if ( !rmrr )
> - return -ENOMEM;
> + seg = 0;
> + for ( idx = 0; idx < dev_count; idx++ )
> + {
> + rmrr->scope.devices[idx] = sbdf[idx];
> + seg |= PCI_SEG(sbdf[idx]);
> + }
> + if ( seg != PCI_SEG(sbdf[0]) )
> + {
> + printk(XENLOG_ERR VTDPREFIX
> + "Segments are not equal for RMRR range "ERMRRU_FMT"\n",
> + ERMRRU_ARG);
> + scope_devices_free(&rmrr->scope);
> + xfree(rmrr);
> + return 0;
> + }
>
> - rmrr->scope.devices = xmalloc_array(u16, user_rmrrs[i].dev_count);
> - if ( !rmrr->scope.devices )
> - {
> - xfree(rmrr);
> - return -ENOMEM;
> - }
> + rmrr->segment = seg;
> + rmrr->base_address = pfn_to_paddr(base_pfn);
> + /* Align the end_address to the end of the page */
> + rmrr->end_address = pfn_to_paddr(end_pfn) | ~PAGE_MASK;
> + rmrr->scope.devices_cnt = dev_count;
>
> - seg = 0;
> - for ( idx = 0; idx < user_rmrrs[i].dev_count; idx++ )
> - {
> - rmrr->scope.devices[idx] = user_rmrrs[i].sbdf[idx];
> - seg |= PCI_SEG(user_rmrrs[i].sbdf[idx]);
> - }
> - if ( seg != PCI_SEG(user_rmrrs[i].sbdf[0]) )
> - {
> - printk(XENLOG_ERR VTDPREFIX
> - "Segments are not equal for RMRR range "ERMRRU_FMT"\n",
> - ERMRRU_ARG(user_rmrrs[i]));
> - scope_devices_free(&rmrr->scope);
> - xfree(rmrr);
> - continue;
> - }
> + if ( register_one_rmrr(rmrr) )
> + printk(XENLOG_ERR VTDPREFIX
> + "Could not register RMMR range "ERMRRU_FMT"\n",
> + ERMRRU_ARG);
>
> - rmrr->segment = seg;
> - rmrr->base_address = pfn_to_paddr(user_rmrrs[i].base_pfn);
> - /* Align the end_address to the end of the page */
> - rmrr->end_address = pfn_to_paddr(user_rmrrs[i].end_pfn) | ~PAGE_MASK;
> - rmrr->scope.devices_cnt = user_rmrrs[i].dev_count;
> + return 1;
> +}
>
> - if ( register_one_rmrr(rmrr) )
> - printk(XENLOG_ERR VTDPREFIX
> - "Could not register RMMR range "ERMRRU_FMT"\n",
> - ERMRRU_ARG(user_rmrrs[i]));
> - }
> +static int __init cf_check add_one_extra_rmrr(xen_pfn_t start, xen_ulong_t nr, u32 id, void *ctxt)
> +{
> + u32 sbdf_array[] = { id };
> + return add_one_user_rmrr(start, start+nr, 1, sbdf_array);
> +}
>
> - return 0;
> +static int __init add_extra_rmrr(void)
> +{
> + return iommu_get_extra_reserved_device_memory(add_one_extra_rmrr, NULL);
> }
>
> #include <asm/tboot.h>
> @@ -1010,7 +1038,7 @@ int __init acpi_dmar_init(void)
> {
> iommu_init_ops = &intel_iommu_init_ops;
>
> - return add_user_rmrr();
> + return add_user_rmrr() || add_extra_rmrr();
> }
>
> return ret;
> @@ -1108,6 +1136,15 @@ static int __init cf_check parse_rmrr_param(const char *str)
> else
> end = start;
>
> + if ( (end - start) >= MAX_USER_RMRR_PAGES )
> + {
> + printk(XENLOG_ERR VTDPREFIX
> + "RMRR range "ERMRRU_FMT" exceeds "\
> + __stringify(MAX_USER_RMRR_PAGES)" pages\n",
> + start, end);
> + return -E2BIG;
> + }
> +
> user_rmrrs[nr_rmrr].base_pfn = start;
> user_rmrrs[nr_rmrr].end_pfn = end;
>
> --
> git-series 0.9.1
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* RE: [PATCH v7 08/11] IOMMU/VT-d: wire common device reserved memory API
2022-09-17 2:51 ` [PATCH v7 08/11] IOMMU/VT-d: wire common device reserved memory API Marek Marczykowski-Górecki
2022-09-22 9:29 ` Marek Marczykowski-Górecki
@ 2022-09-23 7:21 ` Tian, Kevin
2022-09-26 23:54 ` Marczykowski, Marek
1 sibling, 1 reply; 21+ messages in thread
From: Tian, Kevin @ 2022-09-23 7:21 UTC (permalink / raw)
To: Marczykowski, Marek, xen-devel; +Cc: Marczykowski, Marek
> From: Marek Marczykowski-Górecki
> Sent: Saturday, September 17, 2022 10:51 AM
>
> Re-use rmrr= parameter handling code to handle common device reserved
> memory.
>
> Signed-off-by: Marek Marczykowski-Górecki
> <marmarek@invisiblethingslab.com>
> ---
> Changes in v3:
> - make MAX_USER_RMRR_PAGES applicable only to user-configured RMRR
> ---
> xen/drivers/passthrough/vtd/dmar.c | 201 +++++++++++++++++-------------
> 1 file changed, 119 insertions(+), 82 deletions(-)
>
> diff --git a/xen/drivers/passthrough/vtd/dmar.c
> b/xen/drivers/passthrough/vtd/dmar.c
> index 367304c8739c..3df5f6b69719 100644
> --- a/xen/drivers/passthrough/vtd/dmar.c
> +++ b/xen/drivers/passthrough/vtd/dmar.c
> @@ -861,111 +861,139 @@ static struct user_rmrr __initdata
> user_rmrrs[MAX_USER_RMRR];
>
> /* Macro for RMRR inclusive range formatting. */
> #define ERMRRU_FMT "[%lx-%lx]"
> -#define ERMRRU_ARG(eru) eru.base_pfn, eru.end_pfn
> +#define ERMRRU_ARG base_pfn, end_pfn
> +
> +static int __init add_one_user_rmrr(unsigned long base_pfn,
> + unsigned long end_pfn,
> + unsigned int dev_count,
> + uint32_t *sbdf);
Just move the function here then no need of a declaration.
>
> static int __init add_user_rmrr(void)
> {
> + unsigned int i;
> + int ret;
> +
> + for ( i = 0; i < nr_rmrr; i++ )
> + {
> + ret = add_one_user_rmrr(user_rmrrs[i].base_pfn,
> + user_rmrrs[i].end_pfn,
> + user_rmrrs[i].dev_count,
> + user_rmrrs[i].sbdf);
> + if ( ret < 0 )
> + return ret;
> + }
> + return 0;
> +}
> +
> +/* Returns 1 on success, 0 when ignoring and < 0 on error. */
I'm not sure the point of 'ignoring'. If user specifies a wrong RMRR
range (overlap, mfn invalid, etc.) just return an error similar to
-ENOMEM. Ignoring a user-specified range implies that something
would potentially get broken hence better fail it early.
> +static int __init add_one_user_rmrr(unsigned long base_pfn,
> + unsigned long end_pfn,
> + unsigned int dev_count,
> + uint32_t *sbdf)
> +{
> struct acpi_rmrr_unit *rmrr, *rmrru;
> - unsigned int idx, seg, i;
> - unsigned long base, end;
> + unsigned int idx, seg;
> + unsigned long base_iter;
> bool overlap;
>
> - for ( i = 0; i < nr_rmrr; i++ )
> + if ( iommu_verbose )
> + printk(XENLOG_DEBUG VTDPREFIX
> + "Adding RMRR for %d device ([0]: %#x) range "ERMRRU_FMT"\n",
> + dev_count, sbdf[0], ERMRRU_ARG);
> +
> + if ( base_pfn > end_pfn )
> {
> - base = user_rmrrs[i].base_pfn;
> - end = user_rmrrs[i].end_pfn;
> + printk(XENLOG_ERR VTDPREFIX
> + "Invalid RMRR Range "ERMRRU_FMT"\n",
> + ERMRRU_ARG);
> + return 0;
> + }
>
> - if ( base > end )
> + overlap = false;
> + list_for_each_entry(rmrru, &acpi_rmrr_units, list)
> + {
> + if ( pfn_to_paddr(base_pfn) <= rmrru->end_address &&
> + rmrru->base_address <= pfn_to_paddr(end_pfn) )
> {
> printk(XENLOG_ERR VTDPREFIX
> - "Invalid RMRR Range "ERMRRU_FMT"\n",
> - ERMRRU_ARG(user_rmrrs[i]));
> - continue;
> + "Overlapping RMRRs: "ERMRRU_FMT" and [%lx-%lx]\n",
> + ERMRRU_ARG,
> + paddr_to_pfn(rmrru->base_address),
> + paddr_to_pfn(rmrru->end_address));
> + overlap = true;
> + break;
> }
> + }
> + /* Don't add overlapping RMRR. */
> + if ( overlap )
> + return 0;
>
> - if ( (end - base) >= MAX_USER_RMRR_PAGES )
> + base_iter = base_pfn;
> + do
> + {
> + if ( !mfn_valid(_mfn(base_iter)) )
> {
> printk(XENLOG_ERR VTDPREFIX
> - "RMRR range "ERMRRU_FMT" exceeds "\
> - __stringify(MAX_USER_RMRR_PAGES)" pages\n",
> - ERMRRU_ARG(user_rmrrs[i]));
> - continue;
> + "Invalid pfn in RMRR range "ERMRRU_FMT"\n",
> + ERMRRU_ARG);
> + break;
> }
> + } while ( base_iter++ < end_pfn );
>
> - overlap = false;
> - list_for_each_entry(rmrru, &acpi_rmrr_units, list)
> - {
> - if ( pfn_to_paddr(base) <= rmrru->end_address &&
> - rmrru->base_address <= pfn_to_paddr(end) )
> - {
> - printk(XENLOG_ERR VTDPREFIX
> - "Overlapping RMRRs: "ERMRRU_FMT" and [%lx-%lx]\n",
> - ERMRRU_ARG(user_rmrrs[i]),
> - paddr_to_pfn(rmrru->base_address),
> - paddr_to_pfn(rmrru->end_address));
> - overlap = true;
> - break;
> - }
> - }
> - /* Don't add overlapping RMRR. */
> - if ( overlap )
> - continue;
> + /* Invalid pfn in range as the loop ended before end_pfn was reached. */
> + if ( base_iter <= end_pfn )
> + return 0;
>
> - do
> - {
> - if ( !mfn_valid(_mfn(base)) )
> - {
> - printk(XENLOG_ERR VTDPREFIX
> - "Invalid pfn in RMRR range "ERMRRU_FMT"\n",
> - ERMRRU_ARG(user_rmrrs[i]));
> - break;
> - }
> - } while ( base++ < end );
> + rmrr = xzalloc(struct acpi_rmrr_unit);
> + if ( !rmrr )
> + return -ENOMEM;
>
> - /* Invalid pfn in range as the loop ended before end_pfn was reached.
> */
> - if ( base <= end )
> - continue;
> + rmrr->scope.devices = xmalloc_array(u16, dev_count);
> + if ( !rmrr->scope.devices )
> + {
> + xfree(rmrr);
> + return -ENOMEM;
> + }
>
> - rmrr = xzalloc(struct acpi_rmrr_unit);
> - if ( !rmrr )
> - return -ENOMEM;
> + seg = 0;
> + for ( idx = 0; idx < dev_count; idx++ )
> + {
> + rmrr->scope.devices[idx] = sbdf[idx];
> + seg |= PCI_SEG(sbdf[idx]);
> + }
> + if ( seg != PCI_SEG(sbdf[0]) )
> + {
> + printk(XENLOG_ERR VTDPREFIX
> + "Segments are not equal for RMRR range "ERMRRU_FMT"\n",
> + ERMRRU_ARG);
> + scope_devices_free(&rmrr->scope);
> + xfree(rmrr);
> + return 0;
> + }
>
> - rmrr->scope.devices = xmalloc_array(u16, user_rmrrs[i].dev_count);
> - if ( !rmrr->scope.devices )
> - {
> - xfree(rmrr);
> - return -ENOMEM;
> - }
> + rmrr->segment = seg;
> + rmrr->base_address = pfn_to_paddr(base_pfn);
> + /* Align the end_address to the end of the page */
> + rmrr->end_address = pfn_to_paddr(end_pfn) | ~PAGE_MASK;
> + rmrr->scope.devices_cnt = dev_count;
>
> - seg = 0;
> - for ( idx = 0; idx < user_rmrrs[i].dev_count; idx++ )
> - {
> - rmrr->scope.devices[idx] = user_rmrrs[i].sbdf[idx];
> - seg |= PCI_SEG(user_rmrrs[i].sbdf[idx]);
> - }
> - if ( seg != PCI_SEG(user_rmrrs[i].sbdf[0]) )
> - {
> - printk(XENLOG_ERR VTDPREFIX
> - "Segments are not equal for RMRR range "ERMRRU_FMT"\n",
> - ERMRRU_ARG(user_rmrrs[i]));
> - scope_devices_free(&rmrr->scope);
> - xfree(rmrr);
> - continue;
> - }
> + if ( register_one_rmrr(rmrr) )
> + printk(XENLOG_ERR VTDPREFIX
> + "Could not register RMMR range "ERMRRU_FMT"\n",
> + ERMRRU_ARG);
>
> - rmrr->segment = seg;
> - rmrr->base_address = pfn_to_paddr(user_rmrrs[i].base_pfn);
> - /* Align the end_address to the end of the page */
> - rmrr->end_address = pfn_to_paddr(user_rmrrs[i].end_pfn) |
> ~PAGE_MASK;
> - rmrr->scope.devices_cnt = user_rmrrs[i].dev_count;
> + return 1;
> +}
>
> - if ( register_one_rmrr(rmrr) )
> - printk(XENLOG_ERR VTDPREFIX
> - "Could not register RMMR range "ERMRRU_FMT"\n",
> - ERMRRU_ARG(user_rmrrs[i]));
> - }
> +static int __init cf_check add_one_extra_rmrr(xen_pfn_t start, xen_ulong_t
> nr, u32 id, void *ctxt)
> +{
> + u32 sbdf_array[] = { id };
> + return add_one_user_rmrr(start, start+nr, 1, sbdf_array);
> +}
>
> - return 0;
> +static int __init add_extra_rmrr(void)
> +{
> + return iommu_get_extra_reserved_device_memory(add_one_extra_rmrr,
> NULL);
> }
>
> #include <asm/tboot.h>
> @@ -1010,7 +1038,7 @@ int __init acpi_dmar_init(void)
> {
> iommu_init_ops = &intel_iommu_init_ops;
>
> - return add_user_rmrr();
> + return add_user_rmrr() || add_extra_rmrr();
> }
>
> return ret;
> @@ -1108,6 +1136,15 @@ static int __init cf_check parse_rmrr_param(const
> char *str)
> else
> end = start;
>
> + if ( (end - start) >= MAX_USER_RMRR_PAGES )
> + {
> + printk(XENLOG_ERR VTDPREFIX
> + "RMRR range "ERMRRU_FMT" exceeds "\
> + __stringify(MAX_USER_RMRR_PAGES)" pages\n",
> + start, end);
> + return -E2BIG;
> + }
> +
why moving this error check out of add_one_user_rmrr()? I didn't
get why it's special from other checks there, e.g. having base>end...
> user_rmrrs[nr_rmrr].base_pfn = start;
> user_rmrrs[nr_rmrr].end_pfn = end;
>
> --
> git-series 0.9.1
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [PATCH v7 08/11] IOMMU/VT-d: wire common device reserved memory API
2022-09-23 7:21 ` Tian, Kevin
@ 2022-09-26 23:54 ` Marczykowski, Marek
2022-09-28 5:15 ` Tian, Kevin
0 siblings, 1 reply; 21+ messages in thread
From: Marczykowski, Marek @ 2022-09-26 23:54 UTC (permalink / raw)
To: Tian, Kevin; +Cc: xen-devel
[-- Attachment #1: Type: text/plain, Size: 3419 bytes --]
On Fri, Sep 23, 2022 at 07:21:04AM +0000, Tian, Kevin wrote:
> > From: Marek Marczykowski-Górecki
> > Sent: Saturday, September 17, 2022 10:51 AM
> >
> > Re-use rmrr= parameter handling code to handle common device reserved
> > memory.
> >
> > Signed-off-by: Marek Marczykowski-Górecki
> > <marmarek@invisiblethingslab.com>
> > ---
> > Changes in v3:
> > - make MAX_USER_RMRR_PAGES applicable only to user-configured RMRR
> > ---
> > xen/drivers/passthrough/vtd/dmar.c | 201 +++++++++++++++++-------------
> > 1 file changed, 119 insertions(+), 82 deletions(-)
> >
> > diff --git a/xen/drivers/passthrough/vtd/dmar.c
> > b/xen/drivers/passthrough/vtd/dmar.c
> > index 367304c8739c..3df5f6b69719 100644
> > --- a/xen/drivers/passthrough/vtd/dmar.c
> > +++ b/xen/drivers/passthrough/vtd/dmar.c
> > @@ -861,111 +861,139 @@ static struct user_rmrr __initdata
> > user_rmrrs[MAX_USER_RMRR];
> >
> > /* Macro for RMRR inclusive range formatting. */
> > #define ERMRRU_FMT "[%lx-%lx]"
> > -#define ERMRRU_ARG(eru) eru.base_pfn, eru.end_pfn
> > +#define ERMRRU_ARG base_pfn, end_pfn
> > +
> > +static int __init add_one_user_rmrr(unsigned long base_pfn,
> > + unsigned long end_pfn,
> > + unsigned int dev_count,
> > + uint32_t *sbdf);
>
> Just move the function here then no need of a declaration.
Ok.
> >
> > static int __init add_user_rmrr(void)
> > {
> > + unsigned int i;
> > + int ret;
> > +
> > + for ( i = 0; i < nr_rmrr; i++ )
> > + {
> > + ret = add_one_user_rmrr(user_rmrrs[i].base_pfn,
> > + user_rmrrs[i].end_pfn,
> > + user_rmrrs[i].dev_count,
> > + user_rmrrs[i].sbdf);
> > + if ( ret < 0 )
> > + return ret;
> > + }
> > + return 0;
> > +}
> > +
> > +/* Returns 1 on success, 0 when ignoring and < 0 on error. */
>
> I'm not sure the point of 'ignoring'. If user specifies a wrong RMRR
> range (overlap, mfn invalid, etc.) just return an error similar to
> -ENOMEM. Ignoring a user-specified range implies that something
> would potentially get broken hence better fail it early.
That's the behaviour that was here before, I simply added a comment
about it explicitly (previously it used 'continue' heavily, now it's a
separate function so it's a return value).
While I agree in principle, I don't think such change should be part of
this patch.
(...)
> > @@ -1108,6 +1136,15 @@ static int __init cf_check parse_rmrr_param(const
> > char *str)
> > else
> > end = start;
> >
> > + if ( (end - start) >= MAX_USER_RMRR_PAGES )
> > + {
> > + printk(XENLOG_ERR VTDPREFIX
> > + "RMRR range "ERMRRU_FMT" exceeds "\
> > + __stringify(MAX_USER_RMRR_PAGES)" pages\n",
> > + start, end);
> > + return -E2BIG;
> > + }
> > +
>
> why moving this error check out of add_one_user_rmrr()? I didn't
> get why it's special from other checks there, e.g. having base>end...
To allow internal callers bypass MAX_USER_RMRR_PAGES, and make it apply
really only to user-provided ranges.
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 21+ messages in thread
* RE: [PATCH v7 08/11] IOMMU/VT-d: wire common device reserved memory API
2022-09-26 23:54 ` Marczykowski, Marek
@ 2022-09-28 5:15 ` Tian, Kevin
0 siblings, 0 replies; 21+ messages in thread
From: Tian, Kevin @ 2022-09-28 5:15 UTC (permalink / raw)
To: Marczykowski, Marek; +Cc: xen-devel
> From: Marczykowski, Marek <marmarek@invisiblethingslab.com>
> Sent: Tuesday, September 27, 2022 7:54 AM
>
> On Fri, Sep 23, 2022 at 07:21:04AM +0000, Tian, Kevin wrote:
> > > From: Marek Marczykowski-Górecki
> > > Sent: Saturday, September 17, 2022 10:51 AM
> > >
> > > Re-use rmrr= parameter handling code to handle common device
> reserved
> > > memory.
> > >
> > > Signed-off-by: Marek Marczykowski-Górecki
> > > <marmarek@invisiblethingslab.com>
> > > ---
> > > Changes in v3:
> > > - make MAX_USER_RMRR_PAGES applicable only to user-configured
> RMRR
> > > ---
> > > xen/drivers/passthrough/vtd/dmar.c | 201 +++++++++++++++++------------
> -
> > > 1 file changed, 119 insertions(+), 82 deletions(-)
> > >
> > > diff --git a/xen/drivers/passthrough/vtd/dmar.c
> > > b/xen/drivers/passthrough/vtd/dmar.c
> > > index 367304c8739c..3df5f6b69719 100644
> > > --- a/xen/drivers/passthrough/vtd/dmar.c
> > > +++ b/xen/drivers/passthrough/vtd/dmar.c
> > > @@ -861,111 +861,139 @@ static struct user_rmrr __initdata
> > > user_rmrrs[MAX_USER_RMRR];
> > >
> > > /* Macro for RMRR inclusive range formatting. */
> > > #define ERMRRU_FMT "[%lx-%lx]"
> > > -#define ERMRRU_ARG(eru) eru.base_pfn, eru.end_pfn
> > > +#define ERMRRU_ARG base_pfn, end_pfn
> > > +
> > > +static int __init add_one_user_rmrr(unsigned long base_pfn,
> > > + unsigned long end_pfn,
> > > + unsigned int dev_count,
> > > + uint32_t *sbdf);
> >
> > Just move the function here then no need of a declaration.
>
> Ok.
>
> > >
> > > static int __init add_user_rmrr(void)
> > > {
> > > + unsigned int i;
> > > + int ret;
> > > +
> > > + for ( i = 0; i < nr_rmrr; i++ )
> > > + {
> > > + ret = add_one_user_rmrr(user_rmrrs[i].base_pfn,
> > > + user_rmrrs[i].end_pfn,
> > > + user_rmrrs[i].dev_count,
> > > + user_rmrrs[i].sbdf);
> > > + if ( ret < 0 )
> > > + return ret;
> > > + }
> > > + return 0;
> > > +}
> > > +
> > > +/* Returns 1 on success, 0 when ignoring and < 0 on error. */
> >
> > I'm not sure the point of 'ignoring'. If user specifies a wrong RMRR
> > range (overlap, mfn invalid, etc.) just return an error similar to
> > -ENOMEM. Ignoring a user-specified range implies that something
> > would potentially get broken hence better fail it early.
>
> That's the behaviour that was here before, I simply added a comment
> about it explicitly (previously it used 'continue' heavily, now it's a
> separate function so it's a return value).
> While I agree in principle, I don't think such change should be part of
> this patch.
>
> (...)
>
> > > @@ -1108,6 +1136,15 @@ static int __init cf_check
> parse_rmrr_param(const
> > > char *str)
> > > else
> > > end = start;
> > >
> > > + if ( (end - start) >= MAX_USER_RMRR_PAGES )
> > > + {
> > > + printk(XENLOG_ERR VTDPREFIX
> > > + "RMRR range "ERMRRU_FMT" exceeds "\
> > > + __stringify(MAX_USER_RMRR_PAGES)" pages\n",
> > > + start, end);
> > > + return -E2BIG;
> > > + }
> > > +
> >
> > why moving this error check out of add_one_user_rmrr()? I didn't
> > get why it's special from other checks there, e.g. having base>end...
>
> To allow internal callers bypass MAX_USER_RMRR_PAGES, and make it apply
> really only to user-provided ranges.
>
With above clarification and order adjustment,
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2022-09-28 5:15 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-09-17 2:51 [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 01/11] drivers/char: allow using both dbgp=xhci and dbgp=ehci Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 02/11] IOMMU: add common API for device reserved memory Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 03/11] IOMMU/AMD: wire common device reserved memory API Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 04/11] drivers/char: mark DMA buffers as reserved for the XHCI Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 05/11] drivers/char: add RX support to the XHCI driver Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 06/11] drivers/char: fix handling cable re-plug in XHCI console driver Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 07/11] drivers/char: allow driving the rest of XHCI by a domain while Xen uses DbC Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 08/11] IOMMU/VT-d: wire common device reserved memory API Marek Marczykowski-Górecki
2022-09-22 9:29 ` Marek Marczykowski-Górecki
2022-09-23 7:21 ` Tian, Kevin
2022-09-26 23:54 ` Marczykowski, Marek
2022-09-28 5:15 ` Tian, Kevin
2022-09-17 2:51 ` [PATCH v7 09/11] console: support multiple serial console simultaneously Marek Marczykowski-Górecki
2022-09-17 2:51 ` [PATCH v7 10/11] drivers/char: suspend handling in XHCI console driver Marek Marczykowski-Górecki
2022-09-20 15:07 ` Jan Beulich
2022-09-17 2:51 ` [PATCH v7 11/11] drivers/char: add console=ehci as an alias for console=dbgp Marek Marczykowski-Górecki
2022-09-19 8:49 ` Jan Beulich
2022-09-19 13:33 ` [PATCH v7 00/11] Add Xue - console over USB 3 Debug Capability Jan Beulich
2022-09-19 13:49 ` Marek Marczykowski-Górecki
2022-09-20 1:38 ` Henry Wang
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.