[PATCH v7 00/10] usb: early: add support for early printk through USB3 debug port

[PATCH v7 00/10] usb: early: add support for early printk through USB3 debug port

[Lu Baolu]

Hi,

This patch series adds support for early printk through USB3 debug port.
USB3 debug port is described in xHCI specification as an optional extended
capability.

Patch 1 to 9 add the driver for xHCI debug capability. It interfaces with
the register set and provides the required ops (read/write/control) to upper
layers. Early printk is one consumer of these ops. The hooks for early printk
are introduced in patch 8. This design is similar to what we have done in
drivers/usb/early/ehci-dbgp.c.

Patch 10 is the design document and user guide.

Change log:
v1->v2:
(1) Patch 1 re-implemented. "debugfs" has been replaced with sysfs.
    The scope reduced from all extended capabilities to debug port
    specific.
(2) Patch 11 changed. Removed unnecessary .bulk_out_size setting.

v2->v3:
(1) Patch 11 got acked by Johan Hovold.

v3->v4:
(1) Patch 1 code refactored by using xhci_find_ext_cap_by_id() helper.
(2) Patch 3 "bus hung state" changed to "port hung state" in commit message.
(3) Patch 12 added verified platform information.

v4->v5:
(1) Patch 1 & 3 code refactored by using xhci_find_next_ext_cap() helper.
(2) Patch 12 updated with more user guide information.
(3) All patches rebased to the new usb-next branch which contains commit
    d5ddcdf(xhci: rework xhci extended capability list parsing functions).

v5->v6:
(1) refresh the patches against 4.5-rc1.
(2) change "usb: xhci: dbc" to "usb: dbc" in the patch titles as they
    are not part of xhci driver.

v6->v7:
(1) two patches removed from the patch series. One belongs to xHCI host
    controller driver, and another belongs to usb/serial.
(2) add fix for 0-DAY kernel test warning:
    drivers/usb/early/xhci-dbc.c:917:6-10: WARNING:
    Unsigned expression compared with zero: size > 0
(3) add fix for 0-DAY kernel test error:
    arch/x86/built-in.o: In function `setup_early_printk':
    early_printk.c:(.init.text+0xceba): undefined reference to `early_xdbc_init'
    early_printk.c:(.init.text+0xcec8): undefined reference to `early_xdbc_console'

Lu Baolu (10):
  x86: fixmap: add permanent fixmap for xhci debug port
  usb: dbc: probe and setup xhci debug capability
  usb: dbc: add support for Intel xHCI dbc quirk
  usb: dbc: add debug buffer
  usb: dbc: add bulk out and bulk in interfaces
  usb: dbc: handle dbc-configured exit
  usb: dbc: handle endpoint stall
  x86: early_printk: add USB3 debug port earlyprintk support
  usb: dbc: add handshake between debug target and host
  usb: doc: add document for xHCI DbC driver

 Documentation/kernel-parameters.txt |    1 +
 Documentation/usb/xhci-dbc.txt      |  350 +++++++++
 MAINTAINERS                         |    8 +
 arch/x86/Kconfig.debug              |   14 +
 arch/x86/include/asm/fixmap.h       |    4 +
 arch/x86/kernel/early_printk.c      |    5 +
 drivers/usb/Kconfig                 |    3 +
 drivers/usb/Makefile                |    2 +-
 drivers/usb/early/Makefile          |    1 +
 drivers/usb/early/xhci-dbc.c        | 1394 +++++++++++++++++++++++++++++++++++
 include/linux/usb/xhci-dbc.h        |  224 ++++++
 11 files changed, 2005 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/usb/xhci-dbc.txt
 create mode 100644 drivers/usb/early/xhci-dbc.c
 create mode 100644 include/linux/usb/xhci-dbc.h

-- 
2.1.4

[PATCH v7 01/10] x86: fixmap: add permanent fixmap for xhci debug port

[Lu Baolu]

xHCI compatible USB3 host controller may provide debug capability
which enables low-level system debug over USB. In order to probing
this debug capability, Linux kernel needs to map and access the
mmio of the host controller during early boot.

This patch adds permenent fixmap pages in fixed_addresses table for
xHCI mmio access.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 arch/x86/include/asm/fixmap.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
index 6d7d0e5..84298cb 100644
--- a/arch/x86/include/asm/fixmap.h
+++ b/arch/x86/include/asm/fixmap.h
@@ -77,6 +77,10 @@ enum fixed_addresses {
 #ifdef CONFIG_PROVIDE_OHCI1394_DMA_INIT
 	FIX_OHCI1394_BASE,
 #endif
+#ifdef CONFIG_EARLY_PRINTK_XDBC
+	FIX_XDBC_BASE,
+	FIX_XDBC_END = FIX_XDBC_BASE + 15,
+#endif
 #ifdef CONFIG_X86_LOCAL_APIC
 	FIX_APIC_BASE,	/* local (CPU) APIC) -- required for SMP or not */
 #endif
-- 
2.1.4

[PATCH v7 02/10] usb: dbc: probe and setup xhci debug capability

[Lu Baolu]

xHCI debug capability (DbC) is an optional functionality provided
by an xHCI host controller. Software learns this capability by
walking through the extended capability list in mmio of the host.

This patch introduces the code to probe and initialize the debug
capability hardware during early boot. With hardware initialization
done, the debug target (system under debug which has DbC enabled)
will present a debug device through the debug port. The debug device
is fully compliant with the USB framework and provides the equivalent
of a very high performance (USB3) full-duplex serial link between the
debug host and target.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 MAINTAINERS                  |   7 +
 arch/x86/Kconfig.debug       |  12 +
 drivers/usb/early/Makefile   |   1 +
 drivers/usb/early/xhci-dbc.c | 774 +++++++++++++++++++++++++++++++++++++++++++
 include/linux/usb/xhci-dbc.h | 187 +++++++++++
 5 files changed, 981 insertions(+)
 create mode 100644 drivers/usb/early/xhci-dbc.c
 create mode 100644 include/linux/usb/xhci-dbc.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 30aca4a..e6d7076 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -11505,6 +11505,13 @@ S:	Supported
 F:	drivers/usb/host/xhci*
 F:	drivers/usb/host/pci-quirks*
 
+USB XHCI DEBUG PORT
+M:	Lu Baolu <baolu.lu@linux.intel.com>
+L:	linux-usb@vger.kernel.org
+S:	Supported
+F:	drivers/usb/early/xhci-dbc.c
+F:	include/linux/usb/xhci-dbc.h
+
 USB ZD1201 DRIVER
 L:	linux-wireless@vger.kernel.org
 W:	http://linux-lc100020.sourceforge.net
diff --git a/arch/x86/Kconfig.debug b/arch/x86/Kconfig.debug
index 9b18ed9..ba60cb1 100644
--- a/arch/x86/Kconfig.debug
+++ b/arch/x86/Kconfig.debug
@@ -48,6 +48,18 @@ config EARLY_PRINTK_EFI
 	  This is useful for kernel debugging when your machine crashes very
 	  early before the console code is initialized.
 
+config EARLY_PRINTK_XDBC
+	bool "Early printk via xHCI debug port"
+	depends on EARLY_PRINTK && PCI
+	---help---
+	  Write kernel log output directly into the xHCI debug port.
+
+	  This is useful for kernel debugging when your machine crashes very
+	  early before the console code is initialized. For normal operation
+	  it is not recommended because it looks ugly and doesn't cooperate
+	  with klogd/syslogd or the X server. You should normally N here,
+	  unless you want to debug such a crash.
+
 config X86_PTDUMP_CORE
 	def_bool n
 
diff --git a/drivers/usb/early/Makefile b/drivers/usb/early/Makefile
index 24bbe51..2db5906 100644
--- a/drivers/usb/early/Makefile
+++ b/drivers/usb/early/Makefile
@@ -3,3 +3,4 @@
 #
 
 obj-$(CONFIG_EARLY_PRINTK_DBGP) += ehci-dbgp.o
+obj-$(CONFIG_EARLY_PRINTK_XDBC) += xhci-dbc.o
diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
new file mode 100644
index 0000000..254a0a8
--- /dev/null
+++ b/drivers/usb/early/xhci-dbc.c
@@ -0,0 +1,774 @@
+/**
+ * xhci-dbc.c - xHCI debug capability driver
+ *
+ * Copyright (C) 2015 Intel Corporation
+ *
+ * Author: Lu Baolu <baolu.lu@linux.intel.com>
+ * Some code shared with EHCI debug port and xHCI driver.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+#include <linux/pci_regs.h>
+#include <linux/pci_ids.h>
+#include <linux/bootmem.h>
+#include <linux/io.h>
+#include <asm/pci-direct.h>
+#include <asm/fixmap.h>
+#include <linux/bcd.h>
+#include <linux/export.h>
+#include <linux/version.h>
+#include <linux/usb/xhci-dbc.h>
+
+#include "../host/xhci.h"
+
+#define	XDBC_PROTOCOL		1	/* GNU Remote Debug Command Set */
+#define	XDBC_VENDOR_ID		0x1d6b	/* Linux Foundation 0x1d6b */
+#define	XDBC_PRODUCT_ID		0x0004	/* __le16 idProduct; device 0004 */
+#define	XDBC_DEVICE_REV		0x0010	/* 0.10 */
+
+static struct xdbc_state xdbc_stat;
+static struct xdbc_state *xdbcp = &xdbc_stat;
+
+#ifdef DBC_DEBUG
+/* place holder */
+#define	xdbc_trace	printk
+static void xdbc_dbg_dump_regs(char *str)
+{
+	if (!xdbcp->xdbc_reg) {
+		xdbc_trace("register not mapped\n");
+		return;
+	}
+
+	xdbc_trace("XDBC registers: %s\n", str);
+	xdbc_trace("  Capability: %08x\n",
+			readl(&xdbcp->xdbc_reg->capability));
+	xdbc_trace("  Door bell: %08x\n",
+			readl(&xdbcp->xdbc_reg->doorbell));
+	xdbc_trace("  Event Ring Segment Table Size: %08x\n",
+			readl(&xdbcp->xdbc_reg->ersts));
+	xdbc_trace("  Event Ring Segment Table Base Address: %16llx\n",
+			xdbc_read64(&xdbcp->xdbc_reg->erstba));
+	xdbc_trace("  Event Ring Dequeue Pointer: %16llx\n",
+			xdbc_read64(&xdbcp->xdbc_reg->erdp));
+	xdbc_trace("  Port status and control: %08x\n",
+			readl(&xdbcp->xdbc_reg->portsc));
+	xdbc_trace("  Debug Capability Context Pointer: %16llx\n",
+			xdbc_read64(&xdbcp->xdbc_reg->dccp));
+	xdbc_trace("  Device Descriptor Info Register 1: %08x\n",
+			readl(&xdbcp->xdbc_reg->devinfo1));
+	xdbc_trace("  Device Descriptor Info Register 2: %08x\n",
+			readl(&xdbcp->xdbc_reg->devinfo2));
+}
+
+static void xdbc_dbg_dump_info_context(char *str)
+{
+	int i;
+	u64 addr;
+	u32 *field;
+
+	if (!xdbcp->dbcc_base)
+		return;
+
+	xdbc_trace("%s:\n", str);
+
+	field = (u32 *)xdbcp->dbcc_base;
+	addr = xdbcp->dbcc_dma;
+	for (i = 0; i < XDBC_INFO_CONTEXT_SIZE;) {
+		xdbc_trace("@%016llx %08x %08x %08x %08x\n",
+			addr,
+			le32_to_cpu(field[i]),
+			le32_to_cpu(field[i + 1]),
+			le32_to_cpu(field[i + 2]),
+			le32_to_cpu(field[i + 3]));
+		addr += 16;
+		i += 4;
+	}
+}
+
+static void xdbc_dbg_dump_erst(char *str)
+{
+	int i;
+	u64 addr = xdbcp->erst_dma;
+	struct xdbc_erst_entry *entry;
+
+	if (!xdbcp->erst_base)
+		return;
+
+	xdbc_trace("%s\n", str);
+
+	for (i = 0; i < xdbcp->erst_size / sizeof(*entry); i++) {
+		entry = (struct xdbc_erst_entry *)xdbcp->erst_base + i;
+		xdbc_trace("@%016llx %08x %08x %08x %08x\n",
+			addr,
+			lower_32_bits(le64_to_cpu(entry->seg_addr)),
+			upper_32_bits(le64_to_cpu(entry->seg_addr)),
+			le32_to_cpu(entry->seg_size),
+			le32_to_cpu(entry->rsvd));
+		addr += sizeof(*entry);
+	}
+}
+
+static void xdbc_dbg_dump_segment(struct xdbc_segment *seg, char *str)
+{
+	int i;
+	u64 addr = seg->dma;
+	struct xdbc_trb *trb;
+
+	if (!seg->trbs)
+		return;
+
+	xdbc_trace("%s\n", str);
+
+	for (i = 0; i < XDBC_TRBS_PER_SEGMENT; i++) {
+		trb = &seg->trbs[i];
+		xdbc_trace("@%016llx %08x %08x %08x %08x\n", addr,
+			le32_to_cpu(trb->field[0]),
+			le32_to_cpu(trb->field[1]),
+			le32_to_cpu(trb->field[2]),
+			le32_to_cpu(trb->field[3]));
+		addr += sizeof(*trb);
+	}
+}
+
+static void xdbc_dbg_dump_string(char *str)
+{
+	u32 *string = (u32 *)xdbcp->string_base;
+	int i, max;
+
+	max = xdbcp->string_size / sizeof(u32);
+
+	xdbc_trace("%s\n", str);
+
+	for (i = 0; i < max; i += 4) {
+		xdbc_trace("@%016llx %08x %08x %08x %08x\n",
+			xdbcp->string_dma + i * 16,
+			le32_to_cpu(string[0]),
+			le32_to_cpu(string[1]),
+			le32_to_cpu(string[2]),
+			le32_to_cpu(string[3]));
+		string += 4;
+	}
+}
+
+static void xdbc_dbg_dump_data(char *str)
+{
+	xdbc_trace("XDBC data structure: %s\n", str);
+	xdbc_dbg_dump_erst("ERST:");
+	xdbc_dbg_dump_segment(&xdbcp->evt_seg, "Event Ring Segment:");
+	xdbc_dbg_dump_segment(&xdbcp->out_seg, "TXout Ring Segment:");
+	xdbc_dbg_dump_segment(&xdbcp->in_seg, "TXin Ring Segment:");
+	xdbc_dbg_dump_info_context("DBCC:");
+	xdbc_dbg_dump_string("String Descriptor:");
+}
+
+#else
+static inline void xdbc_trace(const char *fmt, ...) { }
+static inline void xdbc_dbg_dump_regs(char *str) { }
+static inline void xdbc_dbg_dump_data(char *str) { }
+#endif	/* DBC_DEBUG */
+
+/*
+ * FIXME: kernel provided delay interfaces, like usleep, isn't ready yet
+ *        at the time DbC gets initialized. Below implementation is only
+ *        for x86 platform. Need to reconsider this when porting it onto
+ *        other architectures.
+ */
+static inline void xdbc_udelay(int us)
+{
+	while (us-- > 0)
+		outb(0x1, 0x80);
+}
+
+static void __iomem *xdbc_map_pci_mmio(u32 bus,
+		u32 dev, u32 func, u8 bar, size_t *length)
+{
+	u32 val, sz;
+	u64 val64, sz64, mask64;
+	u8 byte;
+	unsigned long idx, max_idx;
+	void __iomem *base;
+
+	val = read_pci_config(bus, dev, func, bar);
+	write_pci_config(bus, dev, func, bar, ~0);
+	sz = read_pci_config(bus, dev, func, bar);
+	write_pci_config(bus, dev, func, bar, val);
+	if (val == 0xffffffff || sz == 0xffffffff) {
+		xdbc_trace("invalid mmio bar\n");
+		return NULL;
+	}
+
+	val64 = val & PCI_BASE_ADDRESS_MEM_MASK;
+	sz64 = sz & PCI_BASE_ADDRESS_MEM_MASK;
+	mask64 = (u32)PCI_BASE_ADDRESS_MEM_MASK;
+
+	if ((val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
+			PCI_BASE_ADDRESS_MEM_TYPE_64) {
+		val = read_pci_config(bus, dev, func, bar + 4);
+		write_pci_config(bus, dev, func, bar + 4, ~0);
+		sz = read_pci_config(bus, dev, func, bar + 4);
+		write_pci_config(bus, dev, func, bar + 4, val);
+
+		val64 |= ((u64)val << 32);
+		sz64 |= ((u64)sz << 32);
+		mask64 |= ((u64)~0 << 32);
+	}
+
+	sz64 &= mask64;
+
+	if (sizeof(dma_addr_t) < 8 || !sz64) {
+		xdbc_trace("can't handle 64bit BAR\n");
+		return NULL;
+	}
+
+	sz64 = 1ULL << __ffs64(sz64);
+
+	if (sz64 > (FIX_XDBC_END - FIX_XDBC_BASE + 1) * PAGE_SIZE) {
+		xdbc_trace("mmio size beyond 64k not supported\n");
+		return NULL;
+	}
+
+	xdbc_trace("bar: base 0x%llx size 0x%llx offset %03x\n",
+			val64, sz64, bar);
+
+	/* check if the mem space is enabled */
+	byte = read_pci_config_byte(bus, dev, func, PCI_COMMAND);
+	if (!(byte & PCI_COMMAND_MEMORY)) {
+		byte  |= PCI_COMMAND_MEMORY;
+		write_pci_config_byte(bus, dev, func, PCI_COMMAND, byte);
+		xdbc_trace("mmio for xhci enabled\n");
+	}
+
+	/* 64k mmio will be fix-mapped */
+	max_idx = FIX_XDBC_END - FIX_XDBC_BASE;
+	for (idx = 0; idx <= max_idx; idx++)
+		set_fixmap_nocache(FIX_XDBC_BASE + idx,
+			(val64 & PAGE_MASK) + (max_idx - idx) * PAGE_SIZE);
+	base = (void __iomem *)__fix_to_virt(FIX_XDBC_END);
+	base += val64 & ~PAGE_MASK;
+
+	/* save in the state block */
+	xdbcp->bus = bus;
+	xdbcp->dev = dev;
+	xdbcp->func = func;
+	xdbcp->bar = bar;
+	xdbcp->xhci_base = base;
+	xdbcp->xhci_length = sz64;
+
+	if (length)
+		*length = sz64;
+
+	return base;
+}
+
+/*
+ * FIXME: The bootmem allocator isn't ready at the time when DbC gets
+ *        initialized. Below implementation reserves DMA memory blocks
+ *        in the kernel static data segment.
+ */
+static void *xdbc_get_page(dma_addr_t *dma_addr,
+		enum xdbc_page_type type)
+{
+	void *virt;
+	static char event_page[PAGE_SIZE] __aligned(PAGE_SIZE);
+	static char in_ring_page[PAGE_SIZE] __aligned(PAGE_SIZE);
+	static char out_ring_page[PAGE_SIZE] __aligned(PAGE_SIZE);
+	static char table_page[PAGE_SIZE] __aligned(PAGE_SIZE);
+
+	switch (type) {
+	case XDBC_PAGE_EVENT:
+		virt = (void *)event_page;
+		break;
+	case XDBC_PAGE_TXIN:
+		virt = (void *)in_ring_page;
+		break;
+	case XDBC_PAGE_TXOUT:
+		virt = (void *)out_ring_page;
+		break;
+	case XDBC_PAGE_TABLE:
+		virt = (void *)table_page;
+		break;
+	default:
+		return NULL;
+	}
+
+	memset(virt, 0, PAGE_SIZE);
+
+	if (dma_addr)
+		*dma_addr = (dma_addr_t)__pa(virt);
+
+	return virt;
+}
+
+typedef void (*xdbc_walk_excap_cb)(int cap_offset, void *data);
+
+/*
+ * xdbc_walk_excap:
+ *
+ * xHCI extended capability list walker.
+ *
+ * @bus - xHC PCI bus#
+ * @dev - xHC PCI dev#
+ * @func - xHC PCI function#
+ * @cap - capability ID
+ * @oneshot - return immediately once hit match
+ * @cb - call back
+ * @data - callback private data
+ *
+ * Return the last cap offset, otherwize 0.
+ */
+static u32 xdbc_walk_excap(u32 bus, u32 dev, u32 func, int cap,
+		bool oneshot, xdbc_walk_excap_cb cb, void *data)
+{
+	void __iomem *base;
+	int offset = 0;
+	size_t len = 0;
+
+	if (xdbcp->xhci_base && xdbcp->xhci_length) {
+		if (xdbcp->bus != bus ||
+				xdbcp->dev != dev ||
+				xdbcp->func != func) {
+			xdbc_trace("only one DbC can be used\n");
+			return 0;
+		}
+
+		len = xdbcp->xhci_length;
+		base = xdbcp->xhci_base;
+	} else {
+		base = xdbc_map_pci_mmio(bus, dev, func,
+				PCI_BASE_ADDRESS_0, &len);
+		if (!base)
+			return 0;
+	}
+
+	do {
+		offset = xhci_find_next_ext_cap(base, offset, cap);
+		if (!offset)
+			break;
+
+		if (cb)
+			cb(offset, data);
+		if (oneshot)
+			break;
+	} while (1);
+
+	return offset;
+}
+
+static u32 __init xdbc_find_dbgp(int xdbc_num,
+		u32 *rbus, u32 *rdev, u32 *rfunc)
+{
+	u32 bus, dev, func, class;
+	unsigned cap;
+
+	for (bus = 0; bus < XDBC_PCI_MAX_BUSES; bus++) {
+		for (dev = 0; dev < XDBC_PCI_MAX_DEVICES; dev++) {
+			for (func = 0; func < XDBC_PCI_MAX_FUNCTION; func++) {
+				class = read_pci_config(bus, dev, func,
+						PCI_CLASS_REVISION);
+				if ((class >> 8) != PCI_CLASS_SERIAL_USB_XHCI)
+					continue;
+
+				if (xdbc_num-- != 0)
+					continue;
+
+				cap = xdbc_walk_excap(bus, dev, func,
+						XHCI_EXT_CAPS_DEBUG,
+						true, NULL, NULL);
+				*rbus = bus;
+				*rdev = dev;
+				*rfunc = func;
+				return cap;
+			}
+		}
+	}
+
+	return 0;
+}
+
+static int handshake(void __iomem *ptr, u32 mask, u32 done,
+		int wait_usec, int delay_usec)
+{
+	u32	result;
+
+	do {
+		result = readl(ptr);
+		result &= mask;
+		if (result == done)
+			return 0;
+		xdbc_udelay(delay_usec);
+		wait_usec -= delay_usec;
+	} while (wait_usec > 0);
+
+	return -ETIMEDOUT;
+}
+
+static void __init xdbc_bios_handoff(void)
+{
+	int ext_cap_offset;
+	int timeout;
+	u32 val;
+
+	ext_cap_offset = xdbc_walk_excap(xdbcp->bus,
+					xdbcp->dev,
+					xdbcp->func,
+					XHCI_EXT_CAPS_LEGACY,
+					true, NULL, NULL);
+	val = readl(xdbcp->xhci_base + ext_cap_offset);
+
+	/* If the BIOS owns the HC, signal that the OS wants it, and wait */
+	if (val & XHCI_HC_BIOS_OWNED) {
+		writel(val | XHCI_HC_OS_OWNED,
+				xdbcp->xhci_base + ext_cap_offset);
+		timeout = handshake(xdbcp->xhci_base + ext_cap_offset,
+				XHCI_HC_BIOS_OWNED, 0, 5000, 10);
+
+		/* Assume a buggy BIOS and take HC ownership anyway */
+		if (timeout) {
+			xdbc_trace("xHCI BIOS handoff failed (BIOS bug ?)\n");
+			writel(val & ~XHCI_HC_BIOS_OWNED,
+					xdbcp->xhci_base + ext_cap_offset);
+		}
+	}
+
+	/* Disable any BIOS SMIs and clear all SMI events*/
+	val = readl(xdbcp->xhci_base + ext_cap_offset +
+			XHCI_LEGACY_CONTROL_OFFSET);
+	val &= XHCI_LEGACY_DISABLE_SMI;
+	val |= XHCI_LEGACY_SMI_EVENTS;
+	writel(val, xdbcp->xhci_base + ext_cap_offset +
+			XHCI_LEGACY_CONTROL_OFFSET);
+}
+
+/*
+ * xdbc_alloc_ring: allocate physical memory for a ring
+ */
+static int xdbc_alloc_ring(struct xdbc_segment *seg,
+		struct xdbc_ring *ring,
+		enum xdbc_page_type type)
+{
+	struct xdbc_trb *link_trb;
+
+	seg->trbs = xdbc_get_page(&seg->dma, type);
+	if (!seg->trbs)
+		return -ENOMEM;
+
+	ring->segment = seg;
+	ring->enqueue = seg->trbs;
+	ring->dequeue = seg->trbs;
+	ring->cycle_state = 1;
+
+	if (type == XDBC_PAGE_TXIN || type == XDBC_PAGE_TXOUT) {
+		link_trb = &seg->trbs[XDBC_TRBS_PER_SEGMENT - 1];
+		link_trb->field[0] = cpu_to_le32(lower_32_bits(seg->dma));
+		link_trb->field[1] = cpu_to_le32(upper_32_bits(seg->dma));
+		link_trb->field[3] = cpu_to_le32(TRB_TYPE(TRB_LINK)) |
+				cpu_to_le32(LINK_TOGGLE);
+	}
+
+	return 0;
+}
+
+static inline void xdbc_put_utf16(u16 *s, const char *c, size_t size)
+{
+	int i;
+
+	for (i = 0; i < size; i++)
+		s[i] = cpu_to_le16(c[i]);
+}
+
+static int xdbc_mem_init(void)
+{
+	struct xdbc_erst_entry *entry;
+	struct xdbc_strings *strings;
+	struct xdbc_context *context;
+	struct xdbc_ep_context *ep_in, *ep_out;
+	struct usb_string_descriptor *s_desc;
+	unsigned int max_burst;
+	u32 string_length;
+	int ret, index = 0;
+	u32 dev_info;
+
+	/* allocate table page */
+	xdbcp->table_base = xdbc_get_page(&xdbcp->table_dma,
+			XDBC_PAGE_TABLE);
+	if (!xdbcp->table_base) {
+		xdbc_trace("falied to alloc table page\n");
+		return -ENOMEM;
+	}
+
+	/* allocate and initialize event ring */
+	ret = xdbc_alloc_ring(&xdbcp->evt_seg, &xdbcp->evt_ring,
+			XDBC_PAGE_EVENT);
+	if (ret < 0) {
+		xdbc_trace("failed to alloc event ring\n");
+		return ret;
+	}
+
+	/* allocate event ring segment table */
+	xdbcp->erst_size = 16;
+	xdbcp->erst_base = xdbcp->table_base +
+			index * XDBC_TABLE_ENTRY_SIZE;
+	xdbcp->erst_dma = xdbcp->table_dma +
+			index * XDBC_TABLE_ENTRY_SIZE;
+	index += XDBC_ERST_ENTRY_NUM;
+
+	/* Initialize Event Ring Segment Table */
+	entry = (struct xdbc_erst_entry *)xdbcp->erst_base;
+	entry->seg_addr = cpu_to_le64(xdbcp->evt_seg.dma);
+	entry->seg_size = cpu_to_le32(XDBC_TRBS_PER_SEGMENT);
+	entry->rsvd = 0;
+
+	/* Initialize ERST registers */
+	writel(1, &xdbcp->xdbc_reg->ersts);
+	xdbc_write64(xdbcp->erst_dma, &xdbcp->xdbc_reg->erstba);
+	xdbc_write64(xdbcp->evt_seg.dma, &xdbcp->xdbc_reg->erdp);
+
+	/* debug capability contexts */
+	BUILD_BUG_ON(sizeof(struct xdbc_info_context) != 64);
+	BUILD_BUG_ON(sizeof(struct xdbc_ep_context) != 64);
+	BUILD_BUG_ON(sizeof(struct xdbc_context) != 64 * 3);
+
+	xdbcp->dbcc_size = 64 * 3;
+	xdbcp->dbcc_base = xdbcp->table_base +
+			index * XDBC_TABLE_ENTRY_SIZE;
+	xdbcp->dbcc_dma = xdbcp->table_dma +
+			index * XDBC_TABLE_ENTRY_SIZE;
+	index += XDBC_DBCC_ENTRY_NUM;
+
+	/* IN/OUT endpoint transfer ring */
+	ret = xdbc_alloc_ring(&xdbcp->in_seg, &xdbcp->in_ring,
+			XDBC_PAGE_TXIN);
+	if (ret < 0) {
+		xdbc_trace("failed to alloc IN transfer ring\n");
+		return ret;
+	}
+
+	ret = xdbc_alloc_ring(&xdbcp->out_seg, &xdbcp->out_ring,
+			XDBC_PAGE_TXOUT);
+	if (ret < 0) {
+		xdbc_trace("failed to alloc OUT transfer ring\n");
+		return ret;
+	}
+
+	/* strings */
+	xdbcp->string_size = sizeof(struct xdbc_strings);
+	xdbcp->string_base = xdbcp->table_base +
+			index * XDBC_TABLE_ENTRY_SIZE;
+	xdbcp->string_dma = xdbcp->table_dma +
+			index * XDBC_TABLE_ENTRY_SIZE;
+	index += XDBC_STRING_ENTRY_NUM;
+
+	strings = (struct xdbc_strings *)xdbcp->string_base;
+
+	/* serial string */
+	s_desc = (struct usb_string_descriptor *)strings->serial;
+	s_desc->bLength = (strlen(XDBC_STRING_SERIAL) + 1) * 2;
+	s_desc->bDescriptorType = USB_DT_STRING;
+	xdbc_put_utf16(s_desc->wData, XDBC_STRING_SERIAL,
+			strlen(XDBC_STRING_SERIAL));
+
+	string_length = s_desc->bLength;
+	string_length <<= 8;
+
+	/* product string */
+	s_desc = (struct usb_string_descriptor *)strings->product;
+	s_desc->bLength = (strlen(XDBC_STRING_PRODUCT) + 1) * 2;
+	s_desc->bDescriptorType = USB_DT_STRING;
+	xdbc_put_utf16(s_desc->wData, XDBC_STRING_PRODUCT,
+			strlen(XDBC_STRING_PRODUCT));
+
+	string_length += s_desc->bLength;
+	string_length <<= 8;
+
+	/* manufacture string */
+	s_desc = (struct usb_string_descriptor *)strings->manufacture;
+	s_desc->bLength = (strlen(XDBC_STRING_MANUFACTURE) + 1) * 2;
+	s_desc->bDescriptorType = USB_DT_STRING;
+	xdbc_put_utf16(s_desc->wData, XDBC_STRING_MANUFACTURE,
+			strlen(XDBC_STRING_MANUFACTURE));
+
+	string_length += s_desc->bLength;
+	string_length <<= 8;
+
+	/* string 0 */
+	strings->string0[0] = 4;
+	strings->string0[1] = USB_DT_STRING;
+	strings->string0[2] = 0x09;
+	strings->string0[3] = 0x04;
+
+	string_length += 4;
+
+	/* populate the contexts */
+	context = (struct xdbc_context *)xdbcp->dbcc_base;
+	context->info.string0 = cpu_to_le64(xdbcp->string_dma);
+	context->info.manufacture = cpu_to_le64(xdbcp->string_dma +
+			XDBC_MAX_STRING_LENGTH);
+	context->info.product = cpu_to_le64(xdbcp->string_dma +
+			XDBC_MAX_STRING_LENGTH * 2);
+	context->info.serial = cpu_to_le64(xdbcp->string_dma +
+			XDBC_MAX_STRING_LENGTH * 3);
+	context->info.length = cpu_to_le32(string_length);
+
+	max_burst = DEBUG_MAX_BURST(readl(&xdbcp->xdbc_reg->control));
+	ep_out = (struct xdbc_ep_context *)&context->out;
+	ep_out->ep_info1 = 0;
+	ep_out->ep_info2 = cpu_to_le32(EP_TYPE(BULK_OUT_EP) |
+			MAX_PACKET(1024) | MAX_BURST(max_burst));
+	ep_out->deq = cpu_to_le64(xdbcp->out_seg.dma |
+			xdbcp->out_ring.cycle_state);
+
+	ep_in = (struct xdbc_ep_context *)&context->in;
+	ep_in->ep_info1 = 0;
+	ep_in->ep_info2 = cpu_to_le32(EP_TYPE(BULK_OUT_EP) |
+			MAX_PACKET(1024) | MAX_BURST(max_burst));
+	ep_in->deq = cpu_to_le64(xdbcp->in_seg.dma |
+			xdbcp->in_ring.cycle_state);
+
+	/* write DbC context pointer register */
+	xdbc_write64(xdbcp->dbcc_dma, &xdbcp->xdbc_reg->dccp);
+
+	/* device descriptor info registers */
+	dev_info = cpu_to_le32((XDBC_VENDOR_ID << 16) | XDBC_PROTOCOL);
+	writel(dev_info, &xdbcp->xdbc_reg->devinfo1);
+	dev_info = cpu_to_le32((XDBC_DEVICE_REV << 16) | XDBC_PRODUCT_ID);
+	writel(dev_info, &xdbcp->xdbc_reg->devinfo2);
+
+	return 0;
+}
+
+/*
+ * xdbc_start: start DbC
+ *
+ * Set DbC enable bit and wait until DbC run bit being set or timed out.
+ */
+static int xdbc_start(void)
+{
+	u32 ctrl, status;
+
+	ctrl = readl(&xdbcp->xdbc_reg->control);
+	writel(ctrl | CTRL_DCE | CTRL_LSE, &xdbcp->xdbc_reg->control);
+
+	if (handshake(&xdbcp->xdbc_reg->control, CTRL_DCE,
+			CTRL_DCE, 100000, 100) < 0) {
+		xdbc_trace("falied to initialize hardware\n");
+		return -ENODEV;
+	}
+
+	/* wait for port connection */
+	if (handshake(&xdbcp->xdbc_reg->portsc, PORTSC_CCS,
+			PORTSC_CCS, 5000000, 100) < 0) {
+		xdbc_trace("waiting for connection timed out\n");
+		return -ETIMEDOUT;
+	}
+	xdbc_trace("port connection detected\n");
+
+	/* wait for debug device to be configured */
+	if (handshake(&xdbcp->xdbc_reg->control, CTRL_DCR,
+			CTRL_DCR, 5000000, 100) < 0) {
+		xdbc_trace("waiting for device configuration timed out\n");
+		return -ETIMEDOUT;
+	}
+
+	/* port should have a valid port# */
+	status = readl(&xdbcp->xdbc_reg->status);
+	if (!DCST_DPN(status)) {
+		xdbc_trace("invalid root hub port number\n");
+		return -ENODEV;
+	}
+
+	xdbc_trace("root hub port number %d\n", DCST_DPN(status));
+
+	xdbc_trace("DbC is running now, control 0x%08x\n",
+			readl(&xdbcp->xdbc_reg->control));
+
+	return 0;
+}
+
+static int xdbc_setup(void)
+{
+	int ret;
+
+	writel(0, &xdbcp->xdbc_reg->control);
+	if (handshake(&xdbcp->xdbc_reg->control, CTRL_DCE,
+			0, 100000, 100) < 0) {
+		xdbc_trace("falied to initialize hardware\n");
+		return -ETIMEDOUT;
+	}
+
+	/* allocate and initialize all memory data structures */
+	ret = xdbc_mem_init();
+	if (ret < 0) {
+		xdbc_trace("failed to initialize memory\n");
+		return ret;
+	}
+
+	/*
+	 * Memory barrier to ensure hardware sees the bits
+	 * setting above.
+	 */
+	mmiowb();
+
+	/* dump registers and data structures */
+	xdbc_dbg_dump_regs("hardware setup completed");
+	xdbc_dbg_dump_data("hardware setup completed");
+
+	ret = xdbc_start();
+	if (ret < 0) {
+		xdbc_trace("failed to start DbC, cable connected?\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+int __init early_xdbc_init(char *s)
+{
+	u32 bus = 0, dev = 0, func = 0;
+	unsigned long dbgp_num = 0;
+	u32 offset;
+	int ret;
+
+	if (!early_pci_allowed())
+		return -EPERM;
+
+	/* FIXME: early printk "keep" option will be supported later */
+	if (strstr(s, "keep"))
+		return -EPERM;
+
+	if (xdbcp->xdbc_reg)
+		return 0;
+
+	if (*s && kstrtoul(s, 0, &dbgp_num))
+		dbgp_num = 0;
+
+	xdbc_trace("dbgp_num: %lu\n", dbgp_num);
+
+	offset = xdbc_find_dbgp(dbgp_num, &bus, &dev, &func);
+	if (!offset)
+		return -ENODEV;
+
+	xdbc_trace("Found xHCI debug capability on %02x:%02x.%1x\n",
+			bus, dev, func);
+
+	if (!xdbcp->xhci_base)
+		return -EINVAL;
+
+	xdbcp->xdbc_reg = (struct xdbc_regs __iomem *)
+			(xdbcp->xhci_base + offset);
+	xdbc_dbg_dump_regs("debug capability located");
+
+	/* hand over the owner of host from BIOS */
+	xdbc_bios_handoff();
+
+	ret = xdbc_setup();
+	if (ret < 0) {
+		pr_notice("failed to setup xHCI DbC connection\n");
+		xdbcp->xhci_base = NULL;
+		xdbcp->xdbc_reg = NULL;
+		return ret;
+	}
+
+	return 0;
+}
diff --git a/include/linux/usb/xhci-dbc.h b/include/linux/usb/xhci-dbc.h
new file mode 100644
index 0000000..153fb87
--- /dev/null
+++ b/include/linux/usb/xhci-dbc.h
@@ -0,0 +1,187 @@
+/*
+ * xHCI debug capability driver
+ *
+ * Copyright (C) 2015 Intel Corporation
+ *
+ * Author: Lu Baolu <baolu.lu@linux.intel.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __LINUX_XHCI_DBC_H
+#define __LINUX_XHCI_DBC_H
+
+#include <linux/types.h>
+#include <linux/usb/ch9.h>
+
+/**
+ * struct xdbc_regs - xHCI Debug Capability Register interface.
+ */
+struct xdbc_regs {
+	__le32	capability;
+	__le32	doorbell;
+	__le32	ersts;		/* Event Ring Segment Table Size*/
+	__le32	rvd0;		/* 0c~0f reserved bits */
+	__le64	erstba;		/* Event Ring Segment Table Base Address */
+	__le64	erdp;		/* Event Ring Dequeue Pointer */
+	__le32	control;
+#define	DEBUG_MAX_BURST(p)	(((p) >> 16) & 0xff)
+#define	CTRL_DCR		(1 << 0)	/* DbC Run */
+#define	CTRL_PED		(1 << 1)	/* Port Enable/Disable */
+#define	CTRL_HOT		(1 << 2)	/* Halt Out TR */
+#define	CTRL_HIT		(1 << 3)	/* Halt In TR */
+#define	CTRL_DRC		(1 << 4)	/* DbC run change */
+#define	CTRL_DCE		(1 << 31)	/* DbC enable */
+#define	CTRL_LSE		(1 << 1)
+	__le32	status;
+#define	DCST_DPN(p)		(((p) >> 24) & 0xff)
+	__le32	portsc;		/* Port status and control */
+#define	PORTSC_CCS		(1 << 0)
+#define	PORTSC_CSC		(1 << 17)
+#define	PORTSC_PRC		(1 << 21)
+#define	PORTSC_PLC		(1 << 22)
+#define	PORTSC_CEC		(1 << 23)
+	__le32	rvd1;		/* 2b~28 reserved bits */
+	__le64	dccp;		/* Debug Capability Context Pointer */
+	__le32	devinfo1;	/* Device Descriptor Info Register 1 */
+	__le32	devinfo2;	/* Device Descriptor Info Register 2 */
+};
+
+/*
+ * xHCI Debug Capability data structures
+ */
+struct xdbc_trb {
+	__le32 field[4];
+};
+
+struct xdbc_erst_entry {
+	__le64	seg_addr;
+	__le32	seg_size;
+	__le32	rsvd;
+};
+
+struct xdbc_info_context {
+	__le64	string0;
+	__le64	manufacture;
+	__le64	product;
+	__le64	serial;
+	__le32	length;
+	__le32	rsvdz[7];
+};
+
+struct xdbc_ep_context {
+	__le32	ep_info1;
+	__le32	ep_info2;
+	__le64	deq;
+	__le32	tx_info;
+	__le32	rsvd0[11];
+};
+
+struct xdbc_context {
+	struct xdbc_info_context	info;
+	struct xdbc_ep_context		out;
+	struct xdbc_ep_context		in;
+};
+
+#define	XDBC_INFO_CONTEXT_SIZE		48
+
+#define	XDBC_MAX_STRING_LENGTH		64
+#define	XDBC_STRING_MANUFACTURE		"Linux"
+#define	XDBC_STRING_PRODUCT		"Remote GDB"
+#define	XDBC_STRING_SERIAL		"0001"
+struct xdbc_strings {
+	char	string0[XDBC_MAX_STRING_LENGTH];
+	char	manufacture[XDBC_MAX_STRING_LENGTH];
+	char	product[XDBC_MAX_STRING_LENGTH];
+	char	serial[XDBC_MAX_STRING_LENGTH];
+};
+
+/*
+ * software state structure
+ */
+struct xdbc_segment {
+	struct xdbc_trb		*trbs;
+	dma_addr_t		dma;
+};
+
+#define	XDBC_TRBS_PER_SEGMENT	256
+
+struct xdbc_ring {
+	struct xdbc_segment	*segment;
+	struct xdbc_trb		*enqueue;
+	struct xdbc_trb		*dequeue;
+	u32			cycle_state;
+};
+
+enum xdbc_page_type {
+	XDBC_PAGE_EVENT,
+	XDBC_PAGE_TXIN,
+	XDBC_PAGE_TXOUT,
+	XDBC_PAGE_TABLE,
+};
+
+struct xdbc_state {
+	/* pci device info*/
+	u32		bus;
+	u32		dev;
+	u32		func;
+	u8		bar;
+	void __iomem	*xhci_base;
+	size_t		xhci_length;
+#define	XDBC_PCI_MAX_BUSES		256
+#define	XDBC_PCI_MAX_DEVICES		32
+#define	XDBC_PCI_MAX_FUNCTION		8
+
+	/* DbC register base */
+	struct		xdbc_regs __iomem *xdbc_reg;
+
+	/* DbC table page */
+	dma_addr_t	table_dma;
+	void		*table_base;
+
+#define	XDBC_TABLE_ENTRY_SIZE		64
+#define	XDBC_ERST_ENTRY_NUM		1
+#define	XDBC_DBCC_ENTRY_NUM		3
+#define	XDBC_STRING_ENTRY_NUM		4
+
+	/* event ring segment table */
+	dma_addr_t	erst_dma;
+	size_t		erst_size;
+	void		*erst_base;
+
+	/* event ring segments */
+	struct xdbc_ring	evt_ring;
+	struct xdbc_segment	evt_seg;
+
+	/* debug capability contexts */
+	dma_addr_t	dbcc_dma;
+	size_t		dbcc_size;
+	void		*dbcc_base;
+
+	/* descriptor strings */
+	dma_addr_t	string_dma;
+	size_t		string_size;
+	void		*string_base;
+
+	/* bulk OUT endpoint */
+	struct xdbc_ring	out_ring;
+	struct xdbc_segment	out_seg;
+
+	/* bulk IN endpoint */
+	struct xdbc_ring	in_ring;
+	struct xdbc_segment	in_seg;
+};
+
+#define	XDBC_MAX_PACKET		1024
+
+/* door bell target */
+#define	OUT_EP_DOORBELL		0
+#define	IN_EP_DOORBELL		1
+#define	DOOR_BELL_TARGET(p)	(((p) & 0xff) << 8)
+
+#define	xdbc_read64(regs)	xhci_read_64(NULL, (regs))
+#define	xdbc_write64(val, regs)	xhci_write_64(NULL, (val), (regs))
+
+#endif /* __LINUX_XHCI_DBC_H */
-- 
2.1.4

[PATCH v7 03/10] usb: dbc: add support for Intel xHCI dbc quirk

[Lu Baolu]

On Intel platforms, if the debug target is connected with debug
host, enabling DCE bit in command register leads to a port hung
state. In the hung state, the host system will not see a port
connected status bit set. Hence debug target fails to be probed.

The state could be resolved by performing a port reset to the
debug port from the host xHCI. This patch introduces this work
around.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/usb/early/xhci-dbc.c | 52 ++++++++++++++++++++++++++++++++++++++++++++
 include/linux/usb/xhci-dbc.h |  2 ++
 2 files changed, 54 insertions(+)

diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
index 254a0a8..41ce116 100644
--- a/drivers/usb/early/xhci-dbc.c
+++ b/drivers/usb/early/xhci-dbc.c
@@ -255,6 +255,8 @@ static void __iomem *xdbc_map_pci_mmio(u32 bus,
 	xdbcp->bar = bar;
 	xdbcp->xhci_base = base;
 	xdbcp->xhci_length = sz64;
+	xdbcp->vendor = read_pci_config_16(bus, dev, func, PCI_VENDOR_ID);
+	xdbcp->device = read_pci_config_16(bus, dev, func, PCI_DEVICE_ID);
 
 	if (length)
 		*length = sz64;
@@ -638,6 +640,52 @@ static int xdbc_mem_init(void)
 	return 0;
 }
 
+static void xdbc_reset_debug_port_callback(int cap_offset, void *data)
+{
+	u8 major;
+	u32 val, port_offset, port_count;
+	u32 cap_length;
+	void __iomem *ops_reg;
+	void __iomem *portsc;
+	int i;
+
+	val = readl(xdbcp->xhci_base + cap_offset);
+	major = (u8) XHCI_EXT_PORT_MAJOR(val);
+
+	/* only reset super-speed port */
+	if (major != 0x3)
+		return;
+
+	val = readl(xdbcp->xhci_base + cap_offset + 8);
+	port_offset = XHCI_EXT_PORT_OFF(val);
+	port_count = XHCI_EXT_PORT_COUNT(val);
+	xdbc_trace("Extcap Port offset %d count %d\n",
+			port_offset, port_count);
+
+	cap_length = readl(xdbcp->xhci_base) & 0xff;
+	ops_reg = xdbcp->xhci_base + cap_length;
+
+	port_offset--;
+	for (i = port_offset; i < (port_offset + port_count); i++) {
+		portsc = ops_reg + 0x400 + i * 0x10;
+		val = readl(portsc);
+		/* reset the port if CCS bit is cleared */
+		if (!(val & 0x1))
+			writel(val | (1 << 4), portsc);
+	}
+}
+
+static void xdbc_reset_debug_port(void)
+{
+	xdbc_walk_excap(xdbcp->bus,
+			xdbcp->dev,
+			xdbcp->func,
+			XHCI_EXT_CAPS_PROTOCOL,
+			false,
+			xdbc_reset_debug_port_callback,
+			NULL);
+}
+
 /*
  * xdbc_start: start DbC
  *
@@ -656,6 +704,10 @@ static int xdbc_start(void)
 		return -ENODEV;
 	}
 
+	/* reset port to avoid bus hang */
+	if (xdbcp->vendor == PCI_VENDOR_ID_INTEL)
+		xdbc_reset_debug_port();
+
 	/* wait for port connection */
 	if (handshake(&xdbcp->xdbc_reg->portsc, PORTSC_CCS,
 			PORTSC_CCS, 5000000, 100) < 0) {
diff --git a/include/linux/usb/xhci-dbc.h b/include/linux/usb/xhci-dbc.h
index 153fb87..fc0ef9a 100644
--- a/include/linux/usb/xhci-dbc.h
+++ b/include/linux/usb/xhci-dbc.h
@@ -128,6 +128,8 @@ struct xdbc_state {
 	u32		dev;
 	u32		func;
 	u8		bar;
+	u16		vendor;
+	u16		device;
 	void __iomem	*xhci_base;
 	size_t		xhci_length;
 #define	XDBC_PCI_MAX_BUSES		256
-- 
2.1.4

[PATCH v7 04/10] usb: dbc: add debug buffer

[Lu Baolu]

"printk" is not suitable for dbc debugging especially when console
is in usage. This patch adds a debug buffer in dbc driver and puts
the debug messages in this local buffer. The debug buffer could be
dumped whenever the console is not in use. This part of code will
not be visible unless DBC_DEBUG is defined.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/usb/early/xhci-dbc.c | 62 ++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 60 insertions(+), 2 deletions(-)

diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
index 41ce116..6855048 100644
--- a/drivers/usb/early/xhci-dbc.c
+++ b/drivers/usb/early/xhci-dbc.c
@@ -32,8 +32,64 @@ static struct xdbc_state xdbc_stat;
 static struct xdbc_state *xdbcp = &xdbc_stat;
 
 #ifdef DBC_DEBUG
-/* place holder */
-#define	xdbc_trace	printk
+#define	XDBC_DEBUG_BUF_SIZE	(PAGE_SIZE * 32)
+#define	MSG_MAX_LINE		128
+static char xdbc_debug_buf[XDBC_DEBUG_BUF_SIZE];
+static void xdbc_trace(const char *fmt, ...)
+{
+	int i, size;
+	va_list args;
+	static int pos;
+	char temp_buf[MSG_MAX_LINE];
+
+	if (pos >= XDBC_DEBUG_BUF_SIZE - 1)
+		return;
+
+	memset(temp_buf, 0, MSG_MAX_LINE);
+	va_start(args, fmt);
+	vsnprintf(temp_buf, MSG_MAX_LINE - 1, fmt, args);
+	va_end(args);
+
+	i = 0;
+	size = strlen(temp_buf);
+	while (i < size) {
+		xdbc_debug_buf[pos] = temp_buf[i];
+		pos++;
+		i++;
+
+		if (pos >= XDBC_DEBUG_BUF_SIZE - 1)
+			break;
+	}
+}
+
+static void xdbc_dump_debug_buffer(void)
+{
+	int index = 0;
+	int count = 0;
+	char dump_buf[MSG_MAX_LINE];
+
+	xdbc_trace("The end of DbC trace buffer\n");
+	pr_notice("DBC debug buffer:\n");
+	memset(dump_buf, 0, MSG_MAX_LINE);
+
+	while (index < XDBC_DEBUG_BUF_SIZE) {
+		if (!xdbc_debug_buf[index])
+			break;
+
+		if (xdbc_debug_buf[index] == '\n' ||
+				count >= MSG_MAX_LINE - 1) {
+			pr_notice("DBC: @%08x %s\n", index, dump_buf);
+			memset(dump_buf, 0, MSG_MAX_LINE);
+			count = 0;
+		} else {
+			dump_buf[count] = xdbc_debug_buf[index];
+			count++;
+		}
+
+		index++;
+	}
+}
+
 static void xdbc_dbg_dump_regs(char *str)
 {
 	if (!xdbcp->xdbc_reg) {
@@ -165,6 +221,7 @@ static void xdbc_dbg_dump_data(char *str)
 
 #else
 static inline void xdbc_trace(const char *fmt, ...) { }
+static inline void xdbc_dump_debug_buffer(void) { }
 static inline void xdbc_dbg_dump_regs(char *str) { }
 static inline void xdbc_dbg_dump_data(char *str) { }
 #endif	/* DBC_DEBUG */
@@ -819,6 +876,7 @@ int __init early_xdbc_init(char *s)
 		pr_notice("failed to setup xHCI DbC connection\n");
 		xdbcp->xhci_base = NULL;
 		xdbcp->xdbc_reg = NULL;
+		xdbc_dump_debug_buffer();
 		return ret;
 	}
 
-- 
2.1.4

[PATCH v7 05/10] usb: dbc: add bulk out and bulk in interfaces

[Lu Baolu]

This patch adds interfaces for bulk out and bulk in ops. These
interfaces could be used to implement early printk bootconsole
or hook to various system debuggers.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/usb/early/xhci-dbc.c | 373 +++++++++++++++++++++++++++++++++++++++++++
 include/linux/usb/xhci-dbc.h |  30 ++++
 2 files changed, 403 insertions(+)

diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
index 6855048..f59c80ef 100644
--- a/drivers/usb/early/xhci-dbc.c
+++ b/drivers/usb/early/xhci-dbc.c
@@ -219,11 +219,21 @@ static void xdbc_dbg_dump_data(char *str)
 	xdbc_dbg_dump_string("String Descriptor:");
 }
 
+static void xdbc_dbg_dump_trb(struct xdbc_trb *trb, char *str)
+{
+	xdbc_trace("DBC trb: %s\n", str);
+	xdbc_trace("@%016llx %08x %08x %08x %08x\n", (u64)__pa(trb),
+				le32_to_cpu(trb->field[0]),
+				le32_to_cpu(trb->field[1]),
+				le32_to_cpu(trb->field[2]),
+				le32_to_cpu(trb->field[3]));
+}
 #else
 static inline void xdbc_trace(const char *fmt, ...) { }
 static inline void xdbc_dump_debug_buffer(void) { }
 static inline void xdbc_dbg_dump_regs(char *str) { }
 static inline void xdbc_dbg_dump_data(char *str) { }
+static inline void xdbc_dbg_dump_trb(struct xdbc_trb *trb, char *str) { }
 #endif	/* DBC_DEBUG */
 
 /*
@@ -334,6 +344,7 @@ static void *xdbc_get_page(dma_addr_t *dma_addr,
 	static char in_ring_page[PAGE_SIZE] __aligned(PAGE_SIZE);
 	static char out_ring_page[PAGE_SIZE] __aligned(PAGE_SIZE);
 	static char table_page[PAGE_SIZE] __aligned(PAGE_SIZE);
+	static char bulk_buf_page[PAGE_SIZE] __aligned(PAGE_SIZE);
 
 	switch (type) {
 	case XDBC_PAGE_EVENT:
@@ -348,6 +359,9 @@ static void *xdbc_get_page(dma_addr_t *dma_addr,
 	case XDBC_PAGE_TABLE:
 		virt = (void *)table_page;
 		break;
+	case XDBC_PAGE_BUFFER:
+		virt = (void *)bulk_buf_page;
+		break;
 	default:
 		return NULL;
 	}
@@ -694,6 +708,12 @@ static int xdbc_mem_init(void)
 	dev_info = cpu_to_le32((XDBC_DEVICE_REV << 16) | XDBC_PRODUCT_ID);
 	writel(dev_info, &xdbcp->xdbc_reg->devinfo2);
 
+	/* get and store the transfer buffer */
+	xdbcp->out_buf = xdbc_get_page(&xdbcp->out_dma,
+			XDBC_PAGE_BUFFER);
+	xdbcp->in_buf = xdbcp->out_buf + XDBC_MAX_PACKET;
+	xdbcp->in_dma = xdbcp->out_dma + XDBC_MAX_PACKET;
+
 	return 0;
 }
 
@@ -789,6 +809,9 @@ static int xdbc_start(void)
 
 	xdbc_trace("root hub port number %d\n", DCST_DPN(status));
 
+	xdbcp->in_ep_state = EP_RUNNING;
+	xdbcp->out_ep_state = EP_RUNNING;
+
 	xdbc_trace("DbC is running now, control 0x%08x\n",
 			readl(&xdbcp->xdbc_reg->control));
 
@@ -882,3 +905,353 @@ int __init early_xdbc_init(char *s)
 
 	return 0;
 }
+
+static void xdbc_queue_trb(struct xdbc_ring *ring,
+		u32 field1, u32 field2, u32 field3, u32 field4)
+{
+	struct xdbc_trb *trb, *link_trb;
+
+	trb = ring->enqueue;
+	trb->field[0] = cpu_to_le32(field1);
+	trb->field[1] = cpu_to_le32(field2);
+	trb->field[2] = cpu_to_le32(field3);
+	trb->field[3] = cpu_to_le32(field4);
+
+	xdbc_dbg_dump_trb(trb, "enqueue trb");
+
+	++(ring->enqueue);
+	if (ring->enqueue >= &ring->segment->trbs[TRBS_PER_SEGMENT - 1]) {
+		link_trb = ring->enqueue;
+		if (ring->cycle_state)
+			link_trb->field[3] |= cpu_to_le32(TRB_CYCLE);
+		else
+			link_trb->field[3] &= cpu_to_le32(~TRB_CYCLE);
+
+		ring->enqueue = ring->segment->trbs;
+		ring->cycle_state ^= 1;
+	}
+}
+
+static void xdbc_ring_doorbell(int target)
+{
+	writel(DOOR_BELL_TARGET(target), &xdbcp->xdbc_reg->doorbell);
+}
+
+static void xdbc_handle_port_status(struct xdbc_trb *evt_trb)
+{
+	u32 port_reg;
+
+	port_reg = readl(&xdbcp->xdbc_reg->portsc);
+
+	if (port_reg & PORTSC_CSC) {
+		xdbc_trace("%s: connect status change event\n", __func__);
+		writel(port_reg | PORTSC_CSC, &xdbcp->xdbc_reg->portsc);
+		port_reg = readl(&xdbcp->xdbc_reg->portsc);
+	}
+
+	if (port_reg & PORTSC_PRC) {
+		xdbc_trace("%s: port reset change event\n", __func__);
+		writel(port_reg | PORTSC_PRC, &xdbcp->xdbc_reg->portsc);
+		port_reg = readl(&xdbcp->xdbc_reg->portsc);
+	}
+
+	if (port_reg & PORTSC_PLC) {
+		xdbc_trace("%s: port link status change event\n", __func__);
+		writel(port_reg | PORTSC_PLC, &xdbcp->xdbc_reg->portsc);
+		port_reg = readl(&xdbcp->xdbc_reg->portsc);
+	}
+
+	if (port_reg & PORTSC_CEC) {
+		xdbc_trace("%s: config error change\n", __func__);
+		writel(port_reg | PORTSC_CEC, &xdbcp->xdbc_reg->portsc);
+		port_reg = readl(&xdbcp->xdbc_reg->portsc);
+	}
+}
+
+static void xdbc_handle_tx_event(struct xdbc_trb *evt_trb)
+{
+	u32 comp_code;
+	u32 tx_dma_high, tx_dma_low;
+	u64 in_dma, out_dma;
+	size_t remain_length;
+	int ep_id;
+
+	tx_dma_low = le32_to_cpu(evt_trb->field[0]);
+	tx_dma_high = le32_to_cpu(evt_trb->field[1]);
+	comp_code = GET_COMP_CODE(le32_to_cpu(evt_trb->field[2]));
+	remain_length = EVENT_TRB_LEN(le32_to_cpu(evt_trb->field[2]));
+	ep_id = TRB_TO_EP_ID(le32_to_cpu(evt_trb->field[3]));
+	in_dma = __pa(xdbcp->in_pending);
+	out_dma = __pa(xdbcp->out_pending);
+
+	/*
+	 * Possible Completion Codes for DbC Transfer Event are Success,
+	 * Stall Error, USB Transaction Error, Babble Detected Error,
+	 * TRB Error, Short Packet, Undefined Error, Event Ring Full Error,
+	 * and Vendor Defined Error. TRB error, undefined error and vendor
+	 * defined error will result in HOT/HIT set and be handled the same
+	 * way as Stall error.
+	 */
+	switch (comp_code) {
+	case COMP_SUCCESS:
+		remain_length = 0;
+	case COMP_SHORT_TX:
+		xdbc_trace("%s: endpoint %d remains %d bytes\n", __func__,
+			ep_id, remain_length);
+		break;
+	case COMP_TRB_ERR:
+	case COMP_BABBLE:
+	case COMP_TX_ERR:
+	case COMP_STALL:
+	default:
+		xdbc_trace("%s: endpoint %d halted\n", __func__, ep_id);
+		if (ep_id == XDBC_EPID_OUT)
+			xdbcp->out_ep_state = EP_HALTED;
+		if (ep_id == XDBC_EPID_IN)
+			xdbcp->in_ep_state = EP_HALTED;
+
+		break;
+	}
+
+	if (lower_32_bits(in_dma) == tx_dma_low &&
+			upper_32_bits(in_dma) == tx_dma_high) {
+		xdbcp->in_complete = comp_code;
+		xdbcp->in_complete_length =
+				(remain_length > xdbcp->in_length) ?
+				0 : xdbcp->in_length - remain_length;
+	}
+
+	if (lower_32_bits(out_dma) == tx_dma_low &&
+			upper_32_bits(out_dma) == tx_dma_high) {
+		xdbcp->out_complete = comp_code;
+		xdbcp->out_complete_length =
+				(remain_length > xdbcp->out_length) ?
+				0 : xdbcp->out_length - remain_length;
+	}
+}
+
+static void xdbc_handle_events(void)
+{
+	struct xdbc_trb *evt_trb;
+	bool update_erdp = false;
+
+	evt_trb = xdbcp->evt_ring.dequeue;
+	while ((le32_to_cpu(evt_trb->field[3]) & TRB_CYCLE) ==
+			xdbcp->evt_ring.cycle_state) {
+		/*
+		 * Memory barrier to ensure software sees the trbs
+		 * enqueued by hardware.
+		 */
+		rmb();
+
+		xdbc_dbg_dump_trb(evt_trb, "event trb");
+
+		/* FIXME: Handle more event types. */
+		switch ((le32_to_cpu(evt_trb->field[3]) & TRB_TYPE_BITMASK)) {
+		case TRB_TYPE(TRB_PORT_STATUS):
+			xdbc_handle_port_status(evt_trb);
+			break;
+		case TRB_TYPE(TRB_TRANSFER):
+			xdbc_handle_tx_event(evt_trb);
+			break;
+		default:
+			break;
+		}
+
+		/* advance to the next trb */
+		++(xdbcp->evt_ring.dequeue);
+		if (xdbcp->evt_ring.dequeue ==
+				&xdbcp->evt_seg.trbs[TRBS_PER_SEGMENT]) {
+			xdbcp->evt_ring.dequeue = xdbcp->evt_seg.trbs;
+			xdbcp->evt_ring.cycle_state ^= 1;
+		}
+
+		evt_trb = xdbcp->evt_ring.dequeue;
+		update_erdp = true;
+	}
+
+	/* update event ring dequeue pointer */
+	if (update_erdp)
+		xdbc_write64(__pa(xdbcp->evt_ring.dequeue),
+				&xdbcp->xdbc_reg->erdp);
+}
+
+/*
+ * Check and dispatch events in event ring. It also checks status
+ * of hardware. This function will be called from multiple threads.
+ * An atomic lock is applied to protect the access of event ring.
+ */
+static int xdbc_check_event(void)
+{
+	/* event ring is under checking by other thread? */
+	if (!test_bit(XDBC_ATOMIC_EVENT, &xdbcp->atomic_flags) &&
+			!test_and_set_bit(XDBC_ATOMIC_EVENT,
+			&xdbcp->atomic_flags))
+		return 0;
+
+	xdbc_handle_events();
+
+	test_and_clear_bit(XDBC_ATOMIC_EVENT, &xdbcp->atomic_flags);
+
+	return 0;
+}
+
+#define	BULK_IN_COMPLETED(p)	((xdbcp->in_pending == (p)) && \
+				 xdbcp->in_complete)
+#define	BULK_OUT_COMPLETED(p)	((xdbcp->out_pending == (p)) && \
+				 xdbcp->out_complete)
+
+/*
+ * Wait for a bulk-in or bulk-out transfer completion or timed out.
+ * Return count of the actually transferred bytes or error.
+ */
+static int xdbc_wait_until_bulk_done(struct xdbc_trb *trb, int loops)
+{
+	int timeout = 0;
+	bool read;
+
+	if (trb != xdbcp->in_pending &&
+			trb != xdbcp->out_pending)
+		return -EINVAL;
+
+	read = (trb == xdbcp->in_pending);
+
+	do {
+		if (xdbc_check_event() < 0)
+			break;
+
+		if (read && BULK_IN_COMPLETED(trb)) {
+			if (xdbcp->in_ep_state == EP_HALTED)
+				return -EAGAIN;
+			else
+				return xdbcp->in_complete_length;
+		}
+
+		if (!read && BULK_OUT_COMPLETED(trb)) {
+			if (xdbcp->out_ep_state == EP_HALTED)
+				return -EAGAIN;
+			else
+				return xdbcp->out_complete_length;
+		}
+
+		xdbc_udelay(10);
+	} while ((timeout++ < loops) || !loops);
+
+	return -EIO;
+}
+
+static int xdbc_bulk_transfer(void *data, int size, int loops, bool read)
+{
+	u64 addr;
+	u32 length, control;
+	struct xdbc_trb *trb;
+	struct xdbc_ring *ring;
+	u32 cycle;
+	int ret;
+
+	if (size > XDBC_MAX_PACKET) {
+		xdbc_trace("%s: bad parameter, size %d", __func__, size);
+		return -EINVAL;
+	}
+
+	ring = (read ? &xdbcp->in_ring : &xdbcp->out_ring);
+	trb = ring->enqueue;
+	cycle = ring->cycle_state;
+
+	length = TRB_LEN(size);
+	control = TRB_TYPE(TRB_NORMAL) | TRB_IOC;
+
+	if (cycle)
+		control &= cpu_to_le32(~TRB_CYCLE);
+	else
+		control |= cpu_to_le32(TRB_CYCLE);
+
+	if (read) {
+		memset(xdbcp->in_buf, 0, XDBC_MAX_PACKET);
+		addr = xdbcp->in_dma;
+
+		xdbcp->in_pending = trb;
+		xdbcp->in_length = size;
+		xdbcp->in_complete = 0;
+		xdbcp->in_complete_length = 0;
+	} else {
+		memcpy(xdbcp->out_buf, data, size);
+		addr = xdbcp->out_dma;
+
+		xdbcp->out_pending = trb;
+		xdbcp->out_length = size;
+		xdbcp->out_complete = 0;
+		xdbcp->out_complete_length = 0;
+	}
+
+	xdbc_queue_trb(ring, lower_32_bits(addr),
+			upper_32_bits(addr),
+			length, control);
+
+	/*
+	 * Memory barrier to ensure hardware sees the trbs
+	 * enqueued above.
+	 */
+	wmb();
+	if (cycle)
+		trb->field[3] |= cpu_to_le32(cycle);
+	else
+		trb->field[3] &= cpu_to_le32(~TRB_CYCLE);
+
+	xdbc_ring_doorbell(read ? IN_EP_DOORBELL : OUT_EP_DOORBELL);
+
+	ret = xdbc_wait_until_bulk_done(trb, loops);
+
+	if (read)
+		xdbcp->in_pending = NULL;
+	else
+		xdbcp->out_pending = NULL;
+
+	if (ret > 0) {
+		if (read)
+			memcpy(data, xdbcp->in_buf, size);
+		else
+			memset(xdbcp->out_buf, 0, XDBC_MAX_PACKET);
+	} else {
+		xdbc_trace("%s: bulk %s transfer results in error %d\n",
+				__func__, read ? "in" : "out", ret);
+	}
+
+	return ret;
+}
+
+int xdbc_bulk_read(void *data, int size, int loops)
+{
+	int ret;
+
+	do {
+		if (!test_bit(XDBC_ATOMIC_BULKIN, &xdbcp->atomic_flags) &&
+				!test_and_set_bit(XDBC_ATOMIC_BULKIN,
+				&xdbcp->atomic_flags))
+			break;
+	} while (1);
+
+	ret = xdbc_bulk_transfer(data, size, loops, true);
+
+	test_and_clear_bit(XDBC_ATOMIC_BULKIN, &xdbcp->atomic_flags);
+
+	return ret;
+}
+
+int xdbc_bulk_write(const char *bytes, int size)
+{
+	int ret;
+
+	do {
+		if (!test_bit(XDBC_ATOMIC_BULKOUT, &xdbcp->atomic_flags) &&
+				!test_and_set_bit(XDBC_ATOMIC_BULKOUT,
+				&xdbcp->atomic_flags))
+			break;
+	} while (1);
+
+	ret = xdbc_bulk_transfer((void *)bytes, size, XDBC_LOOPS, false);
+
+	test_and_clear_bit(XDBC_ATOMIC_BULKOUT, &xdbcp->atomic_flags);
+
+	return ret;
+}
diff --git a/include/linux/usb/xhci-dbc.h b/include/linux/usb/xhci-dbc.h
index fc0ef9a..289ba58 100644
--- a/include/linux/usb/xhci-dbc.h
+++ b/include/linux/usb/xhci-dbc.h
@@ -120,8 +120,17 @@ enum xdbc_page_type {
 	XDBC_PAGE_TXIN,
 	XDBC_PAGE_TXOUT,
 	XDBC_PAGE_TABLE,
+	XDBC_PAGE_BUFFER,
 };
 
+enum xdbc_ep_state {
+	EP_DISABLED,
+	EP_RUNNING,
+	EP_HALTED,
+};
+#define	XDBC_EPID_OUT	2
+#define	XDBC_EPID_IN	1
+
 struct xdbc_state {
 	/* pci device info*/
 	u32		bus;
@@ -170,13 +179,34 @@ struct xdbc_state {
 	/* bulk OUT endpoint */
 	struct xdbc_ring	out_ring;
 	struct xdbc_segment	out_seg;
+	void			*out_buf;
+	dma_addr_t		out_dma;
+	struct xdbc_trb		*out_pending;		/* IN */
+	size_t			out_length;		/* IN */
+	u32			out_complete;		/* OUT */
+	size_t			out_complete_length;	/* OUT */
+	enum xdbc_ep_state	out_ep_state;
 
 	/* bulk IN endpoint */
 	struct xdbc_ring	in_ring;
 	struct xdbc_segment	in_seg;
+	void			*in_buf;
+	dma_addr_t		in_dma;
+	struct xdbc_trb		*in_pending;		/* IN */
+	size_t			in_length;		/* IN */
+	u32			in_complete;		/* OUT */
+	size_t			in_complete_length;	/* OUT */
+	enum xdbc_ep_state	in_ep_state;
+
+	/* atomic flags */
+	unsigned long		atomic_flags;
+#define	XDBC_ATOMIC_BULKOUT	0
+#define	XDBC_ATOMIC_BULKIN	1
+#define	XDBC_ATOMIC_EVENT	2
 };
 
 #define	XDBC_MAX_PACKET		1024
+#define	XDBC_LOOPS		1000
 
 /* door bell target */
 #define	OUT_EP_DOORBELL		0
-- 
2.1.4

[PATCH v7 06/10] usb: dbc: handle dbc-configured exit

[Lu Baolu]

DbC might exit configured state in some cases (refer to 7.6.4.4 in
xHCI spec 1.1). Software needs detect and clear this situation by
clearing DCCTRL.DCR and wait until the DbC configured before read
or write oprations.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/usb/early/xhci-dbc.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)

diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
index f59c80ef..c81df40 100644
--- a/drivers/usb/early/xhci-dbc.c
+++ b/drivers/usb/early/xhci-dbc.c
@@ -1140,6 +1140,29 @@ static int xdbc_wait_until_bulk_done(struct xdbc_trb *trb, int loops)
 	return -EIO;
 }
 
+static int xdbc_wait_until_dbc_configured(void)
+{
+	int timeout = 0;
+	u32 reg;
+
+	/* Port exits configured state */
+	reg = readl(&xdbcp->xdbc_reg->control);
+	if (!(reg & CTRL_DRC))
+		return 0;
+
+	/* clear run change bit (RW1C) */
+	writel(reg | CTRL_DRC, &xdbcp->xdbc_reg->control);
+
+	do {
+		if (readl(&xdbcp->xdbc_reg->control) & CTRL_DCR)
+			return 0;
+
+		xdbc_udelay(10);
+	} while (timeout++ < XDBC_LOOPS);
+
+	return -ETIMEDOUT;
+}
+
 static int xdbc_bulk_transfer(void *data, int size, int loops, bool read)
 {
 	u64 addr;
@@ -1154,6 +1177,11 @@ static int xdbc_bulk_transfer(void *data, int size, int loops, bool read)
 		return -EINVAL;
 	}
 
+	if (xdbc_wait_until_dbc_configured()) {
+		xdbc_trace("%s: hardware not ready\n", __func__);
+		return -EPERM;
+	}
+
 	ring = (read ? &xdbcp->in_ring : &xdbcp->out_ring);
 	trb = ring->enqueue;
 	cycle = ring->cycle_state;
-- 
2.1.4

[PATCH v7 07/10] usb: dbc: handle endpoint stall

[Lu Baolu]

In case of endpoint stall, software is able to detect the situation
by reading DCCTRL.HIT or DCCTRL.HOT bits. DbC follows the normal USB
framework to handle endpoint stall. When software detects endpoint
stall situation, it should wait until endpoint is recovered before
read or write oprations.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/usb/early/xhci-dbc.c | 36 ++++++++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)

diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
index c81df40..344d93e 100644
--- a/drivers/usb/early/xhci-dbc.c
+++ b/drivers/usb/early/xhci-dbc.c
@@ -1163,6 +1163,37 @@ static int xdbc_wait_until_dbc_configured(void)
 	return -ETIMEDOUT;
 }
 
+static int xdbc_wait_until_epstall_cleared(bool read)
+{
+	int timeout = 0;
+
+	if (read) {
+		do {
+			if (!(readl(&xdbcp->xdbc_reg->control) & CTRL_HIT)) {
+				xdbcp->in_ep_state = EP_RUNNING;
+
+				return 0;
+			}
+
+			xdbcp->in_ep_state = EP_HALTED;
+			xdbc_udelay(10);
+		} while (timeout++ < XDBC_LOOPS);
+	} else {
+		do {
+			if (!(readl(&xdbcp->xdbc_reg->control) & CTRL_HOT)) {
+				xdbcp->out_ep_state = EP_RUNNING;
+
+				return 0;
+			}
+
+			xdbcp->out_ep_state = EP_HALTED;
+			xdbc_udelay(10);
+		} while (timeout++ < XDBC_LOOPS);
+	}
+
+	return -ETIMEDOUT;
+}
+
 static int xdbc_bulk_transfer(void *data, int size, int loops, bool read)
 {
 	u64 addr;
@@ -1182,6 +1213,11 @@ static int xdbc_bulk_transfer(void *data, int size, int loops, bool read)
 		return -EPERM;
 	}
 
+	if (xdbc_wait_until_epstall_cleared(read)) {
+		xdbc_trace("%s: endpoint not ready\n", __func__);
+		return -EPERM;
+	}
+
 	ring = (read ? &xdbcp->in_ring : &xdbcp->out_ring);
 	trb = ring->enqueue;
 	cycle = ring->cycle_state;
-- 
2.1.4

[PATCH v7 08/10] x86: early_printk: add USB3 debug port earlyprintk support

[Lu Baolu]

Add support for early printk by writing debug messages to the USB3
debug port. Users can use this type of early printk by specifying
kernel parameter of "earlyprintk=xdbc". This gives users a chance
of providing debug output.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 Documentation/kernel-parameters.txt |  1 +
 arch/x86/Kconfig.debug              |  2 ++
 arch/x86/kernel/early_printk.c      |  5 +++++
 drivers/usb/Kconfig                 |  3 +++
 drivers/usb/Makefile                |  2 +-
 drivers/usb/early/xhci-dbc.c        | 43 +++++++++++++++++++++++++++++++++++++
 include/linux/usb/xhci-dbc.h        |  5 +++++
 7 files changed, 60 insertions(+), 1 deletion(-)

diff --git a/Documentation/kernel-parameters.txt b/Documentation/kernel-parameters.txt
index 87d40a7..460dc73 100644
--- a/Documentation/kernel-parameters.txt
+++ b/Documentation/kernel-parameters.txt
@@ -1067,6 +1067,7 @@ bytes respectively. Such letter suffixes can also be entirely omitted.
 			earlyprintk=ttySn[,baudrate]
 			earlyprintk=dbgp[debugController#]
 			earlyprintk=pciserial,bus:device.function[,baudrate]
+			earlyprintk=xdbc[xhciController#]
 
 			earlyprintk is useful when the kernel crashes before
 			the normal console is initialized. It is not enabled by
diff --git a/arch/x86/Kconfig.debug b/arch/x86/Kconfig.debug
index ba60cb1..d470071 100644
--- a/arch/x86/Kconfig.debug
+++ b/arch/x86/Kconfig.debug
@@ -29,6 +29,7 @@ config EARLY_PRINTK
 config EARLY_PRINTK_DBGP
 	bool "Early printk via EHCI debug port"
 	depends on EARLY_PRINTK && PCI
+	select USB_EARLY_PRINTK
 	---help---
 	  Write kernel log output directly into the EHCI debug port.
 
@@ -51,6 +52,7 @@ config EARLY_PRINTK_EFI
 config EARLY_PRINTK_XDBC
 	bool "Early printk via xHCI debug port"
 	depends on EARLY_PRINTK && PCI
+	select USB_EARLY_PRINTK
 	---help---
 	  Write kernel log output directly into the xHCI debug port.
 
diff --git a/arch/x86/kernel/early_printk.c b/arch/x86/kernel/early_printk.c
index 21bf924..ba4c471 100644
--- a/arch/x86/kernel/early_printk.c
+++ b/arch/x86/kernel/early_printk.c
@@ -17,6 +17,7 @@
 #include <asm/intel-mid.h>
 #include <asm/pgtable.h>
 #include <linux/usb/ehci_def.h>
+#include <linux/usb/xhci-dbc.h>
 #include <linux/efi.h>
 #include <asm/efi.h>
 #include <asm/pci_x86.h>
@@ -373,6 +374,10 @@ static int __init setup_early_printk(char *buf)
 		if (!strncmp(buf, "dbgp", 4) && !early_dbgp_init(buf + 4))
 			early_console_register(&early_dbgp_console, keep);
 #endif
+#ifdef CONFIG_EARLY_PRINTK_XDBC
+		if (!strncmp(buf, "xdbc", 4) && !early_xdbc_init(buf + 4))
+			early_console_register(&early_xdbc_console, keep);
+#endif
 #ifdef CONFIG_HVC_XEN
 		if (!strncmp(buf, "xen", 3))
 			early_console_register(&xenboot_console, keep);
diff --git a/drivers/usb/Kconfig b/drivers/usb/Kconfig
index 8ed451d..1c4dc23 100644
--- a/drivers/usb/Kconfig
+++ b/drivers/usb/Kconfig
@@ -19,6 +19,9 @@ config USB_EHCI_BIG_ENDIAN_MMIO
 config USB_EHCI_BIG_ENDIAN_DESC
 	bool
 
+config USB_EARLY_PRINTK
+	bool
+
 menuconfig USB_SUPPORT
 	bool "USB support"
 	depends on HAS_IOMEM
diff --git a/drivers/usb/Makefile b/drivers/usb/Makefile
index d5c57f1..6ff98ae 100644
--- a/drivers/usb/Makefile
+++ b/drivers/usb/Makefile
@@ -48,7 +48,7 @@ obj-$(CONFIG_USB_MICROTEK)	+= image/
 obj-$(CONFIG_USB_SERIAL)	+= serial/
 
 obj-$(CONFIG_USB)		+= misc/
-obj-$(CONFIG_EARLY_PRINTK_DBGP)	+= early/
+obj-$(CONFIG_USB_EARLY_PRINTK)	+= early/
 
 obj-$(CONFIG_USB_ATM)		+= atm/
 obj-$(CONFIG_USB_SPEEDTOUCH)	+= atm/
diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
index 344d93e..6c24ba0 100644
--- a/drivers/usb/early/xhci-dbc.c
+++ b/drivers/usb/early/xhci-dbc.c
@@ -10,6 +10,7 @@
  * it under the terms of the GNU General Public License version 2 as
  * published by the Free Software Foundation.
  */
+#include <linux/console.h>
 #include <linux/pci_regs.h>
 #include <linux/pci_ids.h>
 #include <linux/bootmem.h>
@@ -1319,3 +1320,45 @@ int xdbc_bulk_write(const char *bytes, int size)
 
 	return ret;
 }
+
+/*
+ * Start a bulk-in or bulk-out transfer, wait until transfer completion
+ * or error. Return the count of actually transferred bytes or error.
+ */
+static void early_xdbc_write(struct console *con, const char *str, u32 n)
+{
+	int chunk, ret;
+	static char buf[XDBC_MAX_PACKET];
+	int use_cr = 0;
+
+	if (!xdbcp->xdbc_reg)
+		return;
+	memset(buf, 0, XDBC_MAX_PACKET);
+	while (n > 0) {
+		for (chunk = 0; chunk < XDBC_MAX_PACKET && n > 0;
+		     str++, chunk++, n--) {
+			if (!use_cr && *str == '\n') {
+				use_cr = 1;
+				buf[chunk] = '\r';
+				str--;
+				n++;
+				continue;
+			}
+			if (use_cr)
+				use_cr = 0;
+			buf[chunk] = *str;
+		}
+		if (chunk > 0) {
+			ret = xdbc_bulk_write(buf, chunk);
+			if (ret < 0)
+				break;
+		}
+	}
+}
+
+struct console early_xdbc_console = {
+	.name =		"earlyxdbc",
+	.write =	early_xdbc_write,
+	.flags =	CON_PRINTBUFFER,
+	.index =	-1,
+};
diff --git a/include/linux/usb/xhci-dbc.h b/include/linux/usb/xhci-dbc.h
index 289ba58..a556eb8 100644
--- a/include/linux/usb/xhci-dbc.h
+++ b/include/linux/usb/xhci-dbc.h
@@ -216,4 +216,9 @@ struct xdbc_state {
 #define	xdbc_read64(regs)	xhci_read_64(NULL, (regs))
 #define	xdbc_write64(val, regs)	xhci_write_64(NULL, (val), (regs))
 
+#ifdef CONFIG_EARLY_PRINTK_XDBC
+extern int early_xdbc_init(char *s);
+extern struct console early_xdbc_console;
+#endif /* CONFIG_EARLY_PRINTK_XDBC */
+
 #endif /* __LINUX_XHCI_DBC_H */
-- 
2.1.4

[PATCH v7 09/10] usb: dbc: add handshake between debug target and host

[Lu Baolu]

After DbC setup, debug target needs to wait until tty driver and
application (e.g. mincom) on debug taget start.  Otherwise, out
messages might be ignored.

This patch adds a ping/pong mechanism between debug target and
host. Debug target will be waiting there until user presses 'Y'
or 'y' in the tty application.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 drivers/usb/early/xhci-dbc.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
index 6c24ba0..1806ea1 100644
--- a/drivers/usb/early/xhci-dbc.c
+++ b/drivers/usb/early/xhci-dbc.c
@@ -32,6 +32,9 @@
 static struct xdbc_state xdbc_stat;
 static struct xdbc_state *xdbcp = &xdbc_stat;
 
+static int early_xdbc_read(struct console *con, char *str, unsigned n);
+static void early_xdbc_write(struct console *con, const char *str, u32 n);
+
 #ifdef DBC_DEBUG
 #define	XDBC_DEBUG_BUF_SIZE	(PAGE_SIZE * 32)
 #define	MSG_MAX_LINE		128
@@ -860,8 +863,12 @@ int __init early_xdbc_init(char *s)
 {
 	u32 bus = 0, dev = 0, func = 0;
 	unsigned long dbgp_num = 0;
+	char *ping = "Press Y to continue...\n";
+	char pong[64];
+	int size;
 	u32 offset;
 	int ret;
+	int retry = 20;
 
 	if (!early_pci_allowed())
 		return -EPERM;
@@ -904,6 +911,21 @@ int __init early_xdbc_init(char *s)
 		return ret;
 	}
 
+	while (retry > 0) {
+		early_xdbc_write(NULL, ping, strlen(ping));
+		size = early_xdbc_read(NULL, pong, 64);
+		if (size > 0) {
+			xdbc_trace("%s: pong message: %s\n", __func__, pong);
+			if (pong[0] == 'Y' || pong[0] == 'y')
+				break;
+		} else {
+			xdbc_trace("%s: pong message error %d\n",
+				__func__, size);
+		}
+
+		retry--;
+	}
+
 	return 0;
 }
 
@@ -1325,6 +1347,11 @@ int xdbc_bulk_write(const char *bytes, int size)
  * Start a bulk-in or bulk-out transfer, wait until transfer completion
  * or error. Return the count of actually transferred bytes or error.
  */
+static int early_xdbc_read(struct console *con, char *str, unsigned n)
+{
+	return xdbc_bulk_read(str, n, 0);
+}
+
 static void early_xdbc_write(struct console *con, const char *str, u32 n)
 {
 	int chunk, ret;
-- 
2.1.4

[PATCH v7 10/10] usb: doc: add document for xHCI DbC driver

[Lu Baolu]

Add Documentation/usb/xhci-dbc.txt. This document includes
development status and user guide for USB3 debug port.

Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
---
 Documentation/usb/xhci-dbc.txt | 350 +++++++++++++++++++++++++++++++++++++++++
 MAINTAINERS                    |   1 +
 drivers/usb/early/xhci-dbc.c   |   3 +
 3 files changed, 354 insertions(+)
 create mode 100644 Documentation/usb/xhci-dbc.txt

diff --git a/Documentation/usb/xhci-dbc.txt b/Documentation/usb/xhci-dbc.txt
new file mode 100644
index 0000000..564fd8f
--- /dev/null
+++ b/Documentation/usb/xhci-dbc.txt
@@ -0,0 +1,350 @@
+                    xHCI debug capability driver
+
+                 Lu Baolu <baolu.lu@linux.intel.com>
+
+                    Last-updated: September 2015
+
+
+	Contents:
+	---------
+	* What is xHCI DbC?
+	* Debug topologies
+	* Debug stacks
+	* Port Multiplexing
+	* Hardware initialization
+	* External reset
+	* Port reset
+	* Interrupt/DMA/Memory during early boot
+	* Endpoint STALL
+	* Debug device information
+	* How to use DbC early printk?
+	* Limitations
+
+	What is xHCI DbC?
+	-----------------
+
+The xHCI Debugging Capability defined in section 7.6 of xHCI spec 1.1
+provides an optional functionality that enables low-level system debug
+over USB. It provides a means of connecting two systems where one system
+is a Debug Host and the other a Debug Target (System Under Test). The
+Debug Capability provides an interface that is completely independent
+of the xHCI interface. A Debug Target enumerates as a USB debug device
+to the Debug Host, allowing a Debug Host to access a Debug Target through
+the standard USB software stack.
+
+	Debug topologies
+	----------------
+
+Multiple Debug Targets may be attached to a single Debug Host. Debug
+Targets may be connected to any downstream facing port below a Debug
+Host (i.e. anywhere in the fabric, root port or external hub puts).
+A Debug Target may only connect to a Debug Host through a Root Hub port
+of the target. That means connection of a Debug Target to a Debug Host
+through the ports of an external hub is not supported.
+
+Below is a typical connection between Debug Host and Debug target. Two
+Debug targets are connected to a single Debug host.
+
+
+         ________________                ________________
+        |   Debug Host   |              |  Debug Target  |
+        |________________|              |________________|
+        |xHC without DbC |              |  xHC with DbC  |
+        |or DbC disabled |              |     enabled    |
+        |________________|              |________________|
+            |P1|  |p2|                      |P1|  |p2|
+            |__|  |__|                      |__|  |__|
+              |    |                         |
+              |    |_________________________|
+              |_
+                |
+         _______|________                ________________
+        |       HUB      |              |  Debug Target  |
+        |________________|              |________________|
+        | Superspeed hub |              |  xHC with DbC  |
+        |                |              |     enabled    |
+        |________________|              |________________|
+            |P1|  |p2|                      |P1|  |p2|
+            |__|  |__|                      |__|  |__|
+                   |                         |
+                   |_________________________|
+
+	Debug stacks
+	------------
+
+Below is a software stack diagram of both Debug Host and Debug Target.
+
+         ________________                ________________
+        |   Debug Host   |              |  Debug Target  |
+        |________________|              |________________|
+        |   debug App    |              |                |
+        |                |              | system debug   |
+        |   usb_debug    |              |     hooks      |
+        |                |              |                |
+        |    usbcore     |              |                |
+        |                |              |debug capability|
+        |    xhci_hcd    |              |     driver     |
+        |________________|              |________________|
+        |xHC without DbC |              |  xHC with DbC  |
+        |or DbC disabled |              |     enabled    |
+        |________________|              |________________|
+            |P1|  |p2|                      |P1|  |p2|
+            |__|  |__|                      |__|  |__|
+                   |                         |
+                   |_________________________|
+
+
+	Port Multiplexing
+	-----------------
+
+A debug port is always multiplexed with the first xHCI root hub port.
+Whenever debug capability is supported and enabled, and the first root
+hub port is detected to be connected to a downstream super-speed port
+of a Debug Host, the root hub port is assigned to the debug capability
+and operating in an upstream facing mode. Otherwise, all root hub ports
+act as normal downstream facing ports. When the root port is assigned
+to debug capability, it appears through the xHCI as a fully functional
+root hub port that never sees a device attach.
+
+	Hardware initialization
+	-----------------------
+
+xHCI debug capability is initialized during early boot. "early_param"
+micro provides one option. It also provides a global function so that
+early code can call it to initialize the hardware.
+
+int __init early_xdbc_init(char *s)
+
+early_param("string", early_xdbc_init)
+
+Keep in mind that things such as interrupt, system memory, DMA memory,
+PCI configure space access, etc., are all different from a normal device
+driver.
+
+Software probes the debug capability and all its memory and register
+interfaces by walking through the xHCI extended capability List. Debug
+capability is implemented with the capability ID of 10.
+
+After a xHCI debug capability interface was detected, software could
+initialize the hardware so that debug host can enumerate it as a debug
+device. The USB debug device provided by xHC debug capacity experiences
+three modes: disabled mode, enumeration mode and run mode. In each mode,
+software and xHC itself should complete required steps to move it to the
+next mode. Below diagram describes the evolution of each mode.
+
+      __________             ___________             ___________
+     |          |           |           |           |           |
+     |          |           |           |           |           |
+     | Disabled | DbC enable|Enumeration| DbC run   |    Run    |
+     |   mode   |---------->|    mode   |---------->|    mode   |
+     |          |flag 0 to 1|           |flag 0 to 1|           |
+     |          |           |           |           |           |
+     |          |           |           |           |           |
+     |__________|           |___________|           |___________|
+
+To change DbC mode from disabled to enumeration, software should
+1) allocate and initialize all DbC memory data structures. All data
+structures required by a DbC are defined in 7.6 of the spec 1.1;
+2) initialize the registers. All registers required to be initialized
+in disabled mode are defined in 7.6.4.1 of the spec 1.1. With data
+structure and registers initialized, software can Set the debug
+capability enable (DCE) bit to 1 in the Debug Capability Control
+Register (DCCTRL) and DbC enters enumeration mode as soon as this
+bit is set.
+
+DbC hardware is responsible to change DbC mode from enumerate to run.
+As soon as DbC enters enumeration mode, DbC appears as a normal USB
+device which can be enumerated by debug host. Hence, DbC hardware
+should support a default control endpoint, which responds to standard
+USB requests, e.g. SET_ADDRESS, GET_DESCRIPTOR, GET_CONFIGURATION, etc.
+
+After DbC has been configure by debug host, it enters run mode, it's
+the working mode.
+
+	External reset
+	--------------
+
+External reset means DbC reset caused by something outside of DbC hardware
+and software. The external reset sources depends on the DbC System Bus
+Reset(SBR) bit in status register.
+
+If SBR reads 1, the reset source includes,
+
+1) Assertion of chip hardware reset;
+2) System bus reset (e.g. the assertion of PCI RST#);
+3) Transition from the PCI PM D3hot to D0.
+
+If SBR reads 0, the reset source includes,
+
+1) Assertion of chip hardware reset;
+2) Assertion of host controller reset;
+3) Light host controller Reset.
+
+Resetting the DbC shall clear Debug Capability Enable(DCE) bit to 0.
+Software can determing the reset event during runtime by checking the
+DCE bit. The debug device enters disabled mode after reset. DbC driver
+needs to follow above steps to re-initialize the hardware and bring
+the debug device to run state.
+
+	Port reset
+	----------
+
+Debug port resets itself when it detects reset signal from the debug
+host. Software can determine the port reset event by reading the DbC
+port control and staus registers.
+
+The debug device enters enumeration state as soon as the reset signal
+completes. DbC driver should follow above steps to bring the debug
+device into running mode.
+
+	Interrupt/DMA/Memory during early boot
+	--------------------------------------
+
+The driver code needs to take special care during early boot, especially
+when it comes to memory allocation, interrupt, DMA, device MMIO and PCI
+configuration space. This section defines the interfaces used in DbC driver
+for the prior aspects.
+
+All events generated by DbC are put in the event ring, software will
+periodically poll the Event Ring Not Empty bit in the Debug Capability
+Status Register (DCST) to check pending events. To do this, DbC driver
+should 1)poll the event ring after a transfer trb queued and wait until
+transfer completes, or 2) start a thread to do the periodically poll.
+
+A segment of fixed virtual address is reserved for MMIO access purpose.
+Debug capability driver will map the MMIO physical address (exposed in
+PCI BAR) with this fixed virtual address segment.
+
+Debug Capability needs contiguous memory for DMA purpose. The driver
+reserves DMA memory by stating arrays of PAGE_SIZE and ask the compiler
+to align the arrays to PAGE_SIZE.
+
+Base on the following reasons, the driver uses an outb to port 0x80 as
+an I/O delay: timer subsystem might not be ready yet when DbC starts
+to initialize; DbC driver is independent of OS as possible as it can.
+
+	Endpoint STALL
+	--------------
+
+Endpoint STALL happens when data buffer error, parameter Error,
+TRB error, vendor defined error, or undefined error is detected.
+A transfer event will be generated to notify software.
+
+To clear the STALL situation, debug host will send a ClearFeature
+(ENDPOINT_HALT) request. DbC will clear the halt transfer ring
+flag, clear any internal endpoint state, and move the TR dequeue
+pointer to the next TRB in transfer ring.
+
+DbC does not support Soft Retry. Driver must check and determine
+whether to retry the failed transfer.
+
+	Debug device information
+	------------------------
+
+When debug target boots with xHCI debug capability enabled, it appears
+to debug host as a debug device. The debug device is built using one
+interface which declares two bulk endpoints: an IN and an OUT.
+
+class code:	0xdc (diagnostic device, assigned by USB-IF)
+subclass code:	0x02 (debug device, assigned by USB-IF)
+Manufacturer:	"Linux"
+Product:	"Remote GDB"
+Serial:		"0001"
+
+The USB device ID:
+idVendor:	0x1d6b (Linux Foundation)
+idProduct:	0x0004
+
+	How to use DbC early printk?
+	----------------------------
+
+Before using any kernel functionalities based on DbC, users need to check
+whether debug port is supported by the xHCI host in the system.
+
+On a machine which supports USB3 debug port, a file named "debug_port_state"
+will be created under /sys/bus/pci/drivers/xhci_hcd/<pci_bus_name>/. Reading
+this file will show the state (disabled, enabled or configured) of the debug
+port. On a machine that doesn't support USB3 debug port, this file doesn't
+exist.
+
+On debug target system, user needs to enable the following kernel
+config option:
+
+    CONFIG_PCI
+    CONFIG_EARLY_PRINTK
+    CONFIG_EARLY_PRINTK_XDBC
+
+Users also need to add below kernel parameter:
+
+    "earlyprintk=xdbc"
+
+If there are multiple xHCI controllers in the system, user can append
+a host contoller index to this kernel parameter. The index is started
+from 0.
+
+On debug host side, user needs to make sure usb_debug module is included.
+
+On some platforms, such as Intel, you need to disable auto-pm of usb
+subsystem on debug host when you are debugging with DbC.
+
+#echo on | tee /sys/bus/usb/devices/*/power/control
+#echo on > /sys/bus/pci/devices/<xhci_pci_bus_name>/power/control
+
+Before starting the debug target, user could connect the debug port
+on debug target with a root port or port of external hub on the debug
+host. The cable used to connect these two ports should be a USB 3.0
+super-speed A-to-A debugging cable.
+
+During early boot of debug target, DbC hardware gets initialized. Debug
+host should be able to enuerate debug target as a debug device. Debug
+host will bind the debug device with usb_debug driver module and create
+a tty file for serial communication application.
+
+On Intel platform, if the debug target is connected with debug host,
+enabling DCE bit in command register might lead to a hung port state.
+In the hung state, the host system will not see a port connected status
+bit set. Hence debug target fails to be probed. Current DbC driver has
+a quirk for this. But this quirk has been only verified to work on some
+devices. In case that user finds this quirk doesn't work with his/her
+device, he/she can plug out and then plug in the debug cable for work
+around.
+
+If things go smoothly, user should be able to see below kernel message
+on debug host.
+
+# tail -f /var/log/kern.log
+
+[ 1815.983374] usb 4-3: new SuperSpeed USB device number 4 using xhci_hcd
+[ 1815.999595] usb 4-3: LPM exit latency is zeroed, disabling LPM.
+[ 1815.999899] usb 4-3: New USB device found, idVendor=1d6b, idProduct=0004
+[ 1815.999902] usb 4-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
+[ 1815.999903] usb 4-3: Product: Remote GDB
+[ 1815.999904] usb 4-3: Manufacturer: Linux
+[ 1815.999905] usb 4-3: SerialNumber: 0001
+[ 1816.000240] usb_debug 4-3:1.0: xhci_dbc converter detected
+[ 1816.000360] usb 4-3: xhci_dbc converter now attached to ttyUSB0
+
+After tty file being created, user needs to open serial communication
+application, such as minicom. After configuring minicom to open the
+tty file created above, user should be able to see below welcome message
+in minicom:
+
+Press Y to continue...
+
+After 'Y' key is pressed, debug target will go ahead with booting and
+all early printk messages should be routed to the minicom on debug
+host.
+
+	Limitations
+	-----------
+
+Early printk through DbC has been verified to work with Intel Sunrise Point
+chip with below known issues:
+
+1. DbC debug device doesn't support suspend/resume. Users need to disable
+auto-pm of the host controller on debug host.
+
+2. Early prink "keep" option doesn't support by current phase.
+
+3. After several restarts of debug target, debug host might fail to read
+the device descriptor of debug device. Users need to restart the debug host.
diff --git a/MAINTAINERS b/MAINTAINERS
index e6d7076..81ba241 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -11511,6 +11511,7 @@ L:	linux-usb@vger.kernel.org
 S:	Supported
 F:	drivers/usb/early/xhci-dbc.c
 F:	include/linux/usb/xhci-dbc.h
+F:	Documentation/usb/xhci-dbc.txt
 
 USB ZD1201 DRIVER
 L:	linux-wireless@vger.kernel.org
diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
index 1806ea1..8a14a23 100644
--- a/drivers/usb/early/xhci-dbc.c
+++ b/drivers/usb/early/xhci-dbc.c
@@ -6,6 +6,9 @@
  * Author: Lu Baolu <baolu.lu@linux.intel.com>
  * Some code shared with EHCI debug port and xHCI driver.
  *
+ * Please read Documentation/usb/xhci-dbc.txt before you start to develop
+ * or use code in this file.
+ *
  * This program is free software; you can redistribute it and/or modify
  * it under the terms of the GNU General Public License version 2 as
  * published by the Free Software Foundation.
-- 
2.1.4

Re: [PATCH v7 00/10] usb: early: add support for early printk through USB3 debug port

[Lu Baolu]

On 01/26/2016 08:58 PM, Lu Baolu wrote:
> Hi,
>
> This patch series adds support for early printk through USB3 debug port.
> USB3 debug port is described in xHCI specification as an optional extended
> capability.
>
> Patch 1 to 9 add the driver for xHCI debug capability. It interfaces with
> the register set and provides the required ops (read/write/control) to upper
> layers. Early printk is one consumer of these ops. The hooks for early printk
> are introduced in patch 8. This design is similar to what we have done in
> drivers/usb/early/ehci-dbgp.c.
>
> Patch 10 is the design document and user guide.
>
> Change log:
> v1->v2:
> (1) Patch 1 re-implemented. "debugfs" has been replaced with sysfs.
>     The scope reduced from all extended capabilities to debug port
>     specific.
> (2) Patch 11 changed. Removed unnecessary .bulk_out_size setting.
>
> v2->v3:
> (1) Patch 11 got acked by Johan Hovold.
>
> v3->v4:
> (1) Patch 1 code refactored by using xhci_find_ext_cap_by_id() helper.
> (2) Patch 3 "bus hung state" changed to "port hung state" in commit message.
> (3) Patch 12 added verified platform information.
>
> v4->v5:
> (1) Patch 1 & 3 code refactored by using xhci_find_next_ext_cap() helper.
> (2) Patch 12 updated with more user guide information.
> (3) All patches rebased to the new usb-next branch which contains commit
>     d5ddcdf(xhci: rework xhci extended capability list parsing functions).
>
> v5->v6:
> (1) refresh the patches against 4.5-rc1.
> (2) change "usb: xhci: dbc" to "usb: dbc" in the patch titles as they
>     are not part of xhci driver.
>
> v6->v7:
> (1) two patches removed from the patch series. One belongs to xHCI host
>     controller driver, and another belongs to usb/serial.
> (2) add fix for 0-DAY kernel test warning:
>     drivers/usb/early/xhci-dbc.c:917:6-10: WARNING:
>     Unsigned expression compared with zero: size > 0
> (3) add fix for 0-DAY kernel test error:
>     arch/x86/built-in.o: In function `setup_early_printk':
>     early_printk.c:(.init.text+0xceba): undefined reference to `early_xdbc_init'
>     early_printk.c:(.init.text+0xcec8): undefined reference to `early_xdbc_console'
>
> Lu Baolu (10):
>   x86: fixmap: add permanent fixmap for xhci debug port
>   usb: dbc: probe and setup xhci debug capability
>   usb: dbc: add support for Intel xHCI dbc quirk
>   usb: dbc: add debug buffer
>   usb: dbc: add bulk out and bulk in interfaces
>   usb: dbc: handle dbc-configured exit
>   usb: dbc: handle endpoint stall
>   x86: early_printk: add USB3 debug port earlyprintk support
>   usb: dbc: add handshake between debug target and host
>   usb: doc: add document for xHCI DbC driver

Hi Greg,

I removed two patches from this series which belong to usb/xhci and usb/serial.
The remaining patches are all for usb3 debug port driver. Will you queue this
for usb-next? Or, do you want me to do anything else?

Thanks,
Baolu

>
>  Documentation/kernel-parameters.txt |    1 +
>  Documentation/usb/xhci-dbc.txt      |  350 +++++++++
>  MAINTAINERS                         |    8 +
>  arch/x86/Kconfig.debug              |   14 +
>  arch/x86/include/asm/fixmap.h       |    4 +
>  arch/x86/kernel/early_printk.c      |    5 +
>  drivers/usb/Kconfig                 |    3 +
>  drivers/usb/Makefile                |    2 +-
>  drivers/usb/early/Makefile          |    1 +
>  drivers/usb/early/xhci-dbc.c        | 1394 +++++++++++++++++++++++++++++++++++
>  include/linux/usb/xhci-dbc.h        |  224 ++++++
>  11 files changed, 2005 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/usb/xhci-dbc.txt
>  create mode 100644 drivers/usb/early/xhci-dbc.c
>  create mode 100644 include/linux/usb/xhci-dbc.h
>

Re: [PATCH v7 00/10] usb: early: add support for early printk through USB3 debug port

[Greg Kroah-Hartman]

On Tue, Feb 02, 2016 at 10:34:34PM +0800, Lu Baolu wrote:
> 
> 
> On 01/26/2016 08:58 PM, Lu Baolu wrote:
> > Hi,
> >
> > This patch series adds support for early printk through USB3 debug port.
> > USB3 debug port is described in xHCI specification as an optional extended
> > capability.
> >
> > Patch 1 to 9 add the driver for xHCI debug capability. It interfaces with
> > the register set and provides the required ops (read/write/control) to upper
> > layers. Early printk is one consumer of these ops. The hooks for early printk
> > are introduced in patch 8. This design is similar to what we have done in
> > drivers/usb/early/ehci-dbgp.c.
> >
> > Patch 10 is the design document and user guide.
> >
> > Change log:
> > v1->v2:
> > (1) Patch 1 re-implemented. "debugfs" has been replaced with sysfs.
> >     The scope reduced from all extended capabilities to debug port
> >     specific.
> > (2) Patch 11 changed. Removed unnecessary .bulk_out_size setting.
> >
> > v2->v3:
> > (1) Patch 11 got acked by Johan Hovold.
> >
> > v3->v4:
> > (1) Patch 1 code refactored by using xhci_find_ext_cap_by_id() helper.
> > (2) Patch 3 "bus hung state" changed to "port hung state" in commit message.
> > (3) Patch 12 added verified platform information.
> >
> > v4->v5:
> > (1) Patch 1 & 3 code refactored by using xhci_find_next_ext_cap() helper.
> > (2) Patch 12 updated with more user guide information.
> > (3) All patches rebased to the new usb-next branch which contains commit
> >     d5ddcdf(xhci: rework xhci extended capability list parsing functions).
> >
> > v5->v6:
> > (1) refresh the patches against 4.5-rc1.
> > (2) change "usb: xhci: dbc" to "usb: dbc" in the patch titles as they
> >     are not part of xhci driver.
> >
> > v6->v7:
> > (1) two patches removed from the patch series. One belongs to xHCI host
> >     controller driver, and another belongs to usb/serial.
> > (2) add fix for 0-DAY kernel test warning:
> >     drivers/usb/early/xhci-dbc.c:917:6-10: WARNING:
> >     Unsigned expression compared with zero: size > 0
> > (3) add fix for 0-DAY kernel test error:
> >     arch/x86/built-in.o: In function `setup_early_printk':
> >     early_printk.c:(.init.text+0xceba): undefined reference to `early_xdbc_init'
> >     early_printk.c:(.init.text+0xcec8): undefined reference to `early_xdbc_console'
> >
> > Lu Baolu (10):
> >   x86: fixmap: add permanent fixmap for xhci debug port
> >   usb: dbc: probe and setup xhci debug capability
> >   usb: dbc: add support for Intel xHCI dbc quirk
> >   usb: dbc: add debug buffer
> >   usb: dbc: add bulk out and bulk in interfaces
> >   usb: dbc: handle dbc-configured exit
> >   usb: dbc: handle endpoint stall
> >   x86: early_printk: add USB3 debug port earlyprintk support
> >   usb: dbc: add handshake between debug target and host
> >   usb: doc: add document for xHCI DbC driver
> 
> Hi Greg,
> 
> I removed two patches from this series which belong to usb/xhci and usb/serial.
> The remaining patches are all for usb3 debug port driver. Will you queue this
> for usb-next? Or, do you want me to do anything else?

Given that these are touching xhci code, in a way, I need an
ack from the XHCI maintainer before I can do anything with them, as I
can't test them myself.

thanks,

greg k-h

Re: [PATCH v7 00/10] usb: early: add support for early printk through USB3 debug port

[Lu Baolu]

On 02/04/2016 05:43 AM, Greg Kroah-Hartman wrote:
> On Tue, Feb 02, 2016 at 10:34:34PM +0800, Lu Baolu wrote:
>>
>> On 01/26/2016 08:58 PM, Lu Baolu wrote:
>>> Hi,
>>>
>>> This patch series adds support for early printk through USB3 debug port.
>>> USB3 debug port is described in xHCI specification as an optional extended
>>> capability.
>>>
>>> Patch 1 to 9 add the driver for xHCI debug capability. It interfaces with
>>> the register set and provides the required ops (read/write/control) to upper
>>> layers. Early printk is one consumer of these ops. The hooks for early printk
>>> are introduced in patch 8. This design is similar to what we have done in
>>> drivers/usb/early/ehci-dbgp.c.
>>>
>>> Patch 10 is the design document and user guide.
>>>
>>> Change log:
>>> v1->v2:
>>> (1) Patch 1 re-implemented. "debugfs" has been replaced with sysfs.
>>>     The scope reduced from all extended capabilities to debug port
>>>     specific.
>>> (2) Patch 11 changed. Removed unnecessary .bulk_out_size setting.
>>>
>>> v2->v3:
>>> (1) Patch 11 got acked by Johan Hovold.
>>>
>>> v3->v4:
>>> (1) Patch 1 code refactored by using xhci_find_ext_cap_by_id() helper.
>>> (2) Patch 3 "bus hung state" changed to "port hung state" in commit message.
>>> (3) Patch 12 added verified platform information.
>>>
>>> v4->v5:
>>> (1) Patch 1 & 3 code refactored by using xhci_find_next_ext_cap() helper.
>>> (2) Patch 12 updated with more user guide information.
>>> (3) All patches rebased to the new usb-next branch which contains commit
>>>     d5ddcdf(xhci: rework xhci extended capability list parsing functions).
>>>
>>> v5->v6:
>>> (1) refresh the patches against 4.5-rc1.
>>> (2) change "usb: xhci: dbc" to "usb: dbc" in the patch titles as they
>>>     are not part of xhci driver.
>>>
>>> v6->v7:
>>> (1) two patches removed from the patch series. One belongs to xHCI host
>>>     controller driver, and another belongs to usb/serial.
>>> (2) add fix for 0-DAY kernel test warning:
>>>     drivers/usb/early/xhci-dbc.c:917:6-10: WARNING:
>>>     Unsigned expression compared with zero: size > 0
>>> (3) add fix for 0-DAY kernel test error:
>>>     arch/x86/built-in.o: In function `setup_early_printk':
>>>     early_printk.c:(.init.text+0xceba): undefined reference to `early_xdbc_init'
>>>     early_printk.c:(.init.text+0xcec8): undefined reference to `early_xdbc_console'
>>>
>>> Lu Baolu (10):
>>>   x86: fixmap: add permanent fixmap for xhci debug port
>>>   usb: dbc: probe and setup xhci debug capability
>>>   usb: dbc: add support for Intel xHCI dbc quirk
>>>   usb: dbc: add debug buffer
>>>   usb: dbc: add bulk out and bulk in interfaces
>>>   usb: dbc: handle dbc-configured exit
>>>   usb: dbc: handle endpoint stall
>>>   x86: early_printk: add USB3 debug port earlyprintk support
>>>   usb: dbc: add handshake between debug target and host
>>>   usb: doc: add document for xHCI DbC driver
>> Hi Greg,
>>
>> I removed two patches from this series which belong to usb/xhci and usb/serial.
>> The remaining patches are all for usb3 debug port driver. Will you queue this
>> for usb-next? Or, do you want me to do anything else?
> Given that these are touching xhci code, in a way, I need an
> ack from the XHCI maintainer before I can do anything with them, as I
> can't test them myself.

Okay, I will check with the xHCI maintainer.

Thanks,
Baolu

>

Re: [PATCH v7 02/10] usb: dbc: probe and setup xhci debug capability

[Mathias Nyman]

Hi

Most of my concerns are due to the early printk restrictions.
As you say in the documentation:
"Keep in mind that things such as interrupt, system memory, DMA memory,
PCI configure space access, etc., are all different from a normal device
driver."

I don't know the details of these subsystems, or if what is done here is
done the "right" way.

DMA capable memory is allocated with static char xpage[PAGE_SIZE] __aligned,
and DMA address is found using  __pa()

xdbc_map_pci_mmio() does some pci magic and fixmapping that I don't know the
details of.

writing to ioport 0x80 outb(1, 0x80) is used as 1us delay, sometimes up to
5 million times to get a 5 second timeout

Adding Bjorn Helgaas and Dan Williams in case they have an opinoin about
the DMA and PCI cases.

more comments inline

On 26.01.2016 14:58, Lu Baolu wrote:
> xHCI debug capability (DbC) is an optional functionality provided
> by an xHCI host controller. Software learns this capability by
> walking through the extended capability list in mmio of the host.
>
> This patch introduces the code to probe and initialize the debug
> capability hardware during early boot. With hardware initialization
> done, the debug target (system under debug which has DbC enabled)
> will present a debug device through the debug port. The debug device
> is fully compliant with the USB framework and provides the equivalent
> of a very high performance (USB3) full-duplex serial link between the
> debug host and target.
>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> ---
>   MAINTAINERS                  |   7 +
>   arch/x86/Kconfig.debug       |  12 +
>   drivers/usb/early/Makefile   |   1 +
>   drivers/usb/early/xhci-dbc.c | 774 +++++++++++++++++++++++++++++++++++++++++++
>   include/linux/usb/xhci-dbc.h | 187 +++++++++++
>   5 files changed, 981 insertions(+)
>   create mode 100644 drivers/usb/early/xhci-dbc.c
>   create mode 100644 include/linux/usb/xhci-dbc.h
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 30aca4a..e6d7076 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -11505,6 +11505,13 @@ S:	Supported
>   F:	drivers/usb/host/xhci*
>   F:	drivers/usb/host/pci-quirks*
>
> +USB XHCI DEBUG PORT
> +M:	Lu Baolu <baolu.lu@linux.intel.com>
> +L:	linux-usb@vger.kernel.org
> +S:	Supported
> +F:	drivers/usb/early/xhci-dbc.c
> +F:	include/linux/usb/xhci-dbc.h
> +
>   USB ZD1201 DRIVER
>   L:	linux-wireless@vger.kernel.org
>   W:	http://linux-lc100020.sourceforge.net
> diff --git a/arch/x86/Kconfig.debug b/arch/x86/Kconfig.debug
> index 9b18ed9..ba60cb1 100644
> --- a/arch/x86/Kconfig.debug
> +++ b/arch/x86/Kconfig.debug
> @@ -48,6 +48,18 @@ config EARLY_PRINTK_EFI
>   	  This is useful for kernel debugging when your machine crashes very
>   	  early before the console code is initialized.
>
> +config EARLY_PRINTK_XDBC
> +	bool "Early printk via xHCI debug port"
> +	depends on EARLY_PRINTK && PCI
> +	---help---
> +	  Write kernel log output directly into the xHCI debug port.
> +
> +	  This is useful for kernel debugging when your machine crashes very
> +	  early before the console code is initialized. For normal operation
> +	  it is not recommended because it looks ugly and doesn't cooperate
> +	  with klogd/syslogd or the X server. You should normally N here,
> +	  unless you want to debug such a crash.
> +
>   config X86_PTDUMP_CORE
>   	def_bool n
>
> diff --git a/drivers/usb/early/Makefile b/drivers/usb/early/Makefile
> index 24bbe51..2db5906 100644
> --- a/drivers/usb/early/Makefile
> +++ b/drivers/usb/early/Makefile
> @@ -3,3 +3,4 @@
>   #
>
>   obj-$(CONFIG_EARLY_PRINTK_DBGP) += ehci-dbgp.o
> +obj-$(CONFIG_EARLY_PRINTK_XDBC) += xhci-dbc.o
> diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
> new file mode 100644
> index 0000000..254a0a8
> --- /dev/null
> +++ b/drivers/usb/early/xhci-dbc.c
> @@ -0,0 +1,774 @@
> +/**
> + * xhci-dbc.c - xHCI debug capability driver
> + *
> + * Copyright (C) 2015 Intel Corporation
> + *
> + * Author: Lu Baolu <baolu.lu@linux.intel.com>
> + * Some code shared with EHCI debug port and xHCI driver.
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + */
> +#include <linux/pci_regs.h>
> +#include <linux/pci_ids.h>
> +#include <linux/bootmem.h>
> +#include <linux/io.h>
> +#include <asm/pci-direct.h>
> +#include <asm/fixmap.h>
> +#include <linux/bcd.h>
> +#include <linux/export.h>
> +#include <linux/version.h>
> +#include <linux/usb/xhci-dbc.h>
> +
> +#include "../host/xhci.h"
> +
> +#define	XDBC_PROTOCOL		1	/* GNU Remote Debug Command Set */
> +#define	XDBC_VENDOR_ID		0x1d6b	/* Linux Foundation 0x1d6b */
> +#define	XDBC_PRODUCT_ID		0x0004	/* __le16 idProduct; device 0004 */
> +#define	XDBC_DEVICE_REV		0x0010	/* 0.10 */
> +
> +static struct xdbc_state xdbc_stat;
> +static struct xdbc_state *xdbcp = &xdbc_stat;
> +
>

...

> +
> +#else
> +static inline void xdbc_trace(const char *fmt, ...) { }
> +static inline void xdbc_dbg_dump_regs(char *str) { }
> +static inline void xdbc_dbg_dump_data(char *str) { }
> +#endif	/* DBC_DEBUG */
> +
> +/*
> + * FIXME: kernel provided delay interfaces, like usleep, isn't ready yet
> + *        at the time DbC gets initialized. Below implementation is only
> + *        for x86 platform. Need to reconsider this when porting it onto
> + *        other architectures.
> + */
> +static inline void xdbc_udelay(int us)
> +{
> +	while (us-- > 0)
> +		outb(0x1, 0x80);
> +}

This is one of the concerns, the FIXME text describes the problem but still wondering if
there is any other solution. In many cases later in this file we will wait up to 5 seconds
for something, ending up writing 5 million times to ioport 0x80.

> +
> +static void __iomem *xdbc_map_pci_mmio(u32 bus,
> +		u32 dev, u32 func, u8 bar, size_t *length)
> +{
> +	u32 val, sz;
> +	u64 val64, sz64, mask64;
> +	u8 byte;
> +	unsigned long idx, max_idx;
> +	void __iomem *base;
> +
> +	val = read_pci_config(bus, dev, func, bar);
> +	write_pci_config(bus, dev, func, bar, ~0);
> +	sz = read_pci_config(bus, dev, func, bar);
> +	write_pci_config(bus, dev, func, bar, val);
> +	if (val == 0xffffffff || sz == 0xffffffff) {
> +		xdbc_trace("invalid mmio bar\n");
> +		return NULL;
> +	}
> +
> +	val64 = val & PCI_BASE_ADDRESS_MEM_MASK;
> +	sz64 = sz & PCI_BASE_ADDRESS_MEM_MASK;
> +	mask64 = (u32)PCI_BASE_ADDRESS_MEM_MASK;
> +
> +	if ((val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
> +			PCI_BASE_ADDRESS_MEM_TYPE_64) {
> +		val = read_pci_config(bus, dev, func, bar + 4);
> +		write_pci_config(bus, dev, func, bar + 4, ~0);
> +		sz = read_pci_config(bus, dev, func, bar + 4);
> +		write_pci_config(bus, dev, func, bar + 4, val);
> +
> +		val64 |= ((u64)val << 32);
> +		sz64 |= ((u64)sz << 32);
> +		mask64 |= ((u64)~0 << 32);
> +	}
> +
> +	sz64 &= mask64;
> +
> +	if (sizeof(dma_addr_t) < 8 || !sz64) {
> +		xdbc_trace("can't handle 64bit BAR\n");
> +		return NULL;
> +	}
> +
> +	sz64 = 1ULL << __ffs64(sz64);
> +
> +	if (sz64 > (FIX_XDBC_END - FIX_XDBC_BASE + 1) * PAGE_SIZE) {
> +		xdbc_trace("mmio size beyond 64k not supported\n");
> +		return NULL;
> +	}
> +
> +	xdbc_trace("bar: base 0x%llx size 0x%llx offset %03x\n",
> +			val64, sz64, bar);
> +
> +	/* check if the mem space is enabled */
> +	byte = read_pci_config_byte(bus, dev, func, PCI_COMMAND);
> +	if (!(byte & PCI_COMMAND_MEMORY)) {
> +		byte  |= PCI_COMMAND_MEMORY;
> +		write_pci_config_byte(bus, dev, func, PCI_COMMAND, byte);
> +		xdbc_trace("mmio for xhci enabled\n");
> +	}
> +
> +	/* 64k mmio will be fix-mapped */
> +	max_idx = FIX_XDBC_END - FIX_XDBC_BASE;
> +	for (idx = 0; idx <= max_idx; idx++)
> +		set_fixmap_nocache(FIX_XDBC_BASE + idx,
> +			(val64 & PAGE_MASK) + (max_idx - idx) * PAGE_SIZE);
> +	base = (void __iomem *)__fix_to_virt(FIX_XDBC_END);
> +	base += val64 & ~PAGE_MASK;
> +
> +	/* save in the state block */
> +	xdbcp->bus = bus;
> +	xdbcp->dev = dev;
> +	xdbcp->func = func;
> +	xdbcp->bar = bar;
> +	xdbcp->xhci_base = base;
> +	xdbcp->xhci_length = sz64;
> +
> +	if (length)
> +		*length = sz64;
> +
> +	return base;
> +}
> +
> +/*
> + * FIXME: The bootmem allocator isn't ready at the time when DbC gets
> + *        initialized. Below implementation reserves DMA memory blocks
> + *        in the kernel static data segment.
> + */
> +static void *xdbc_get_page(dma_addr_t *dma_addr,
> +		enum xdbc_page_type type)
> +{
> +	void *virt;
> +	static char event_page[PAGE_SIZE] __aligned(PAGE_SIZE);
> +	static char in_ring_page[PAGE_SIZE] __aligned(PAGE_SIZE);
> +	static char out_ring_page[PAGE_SIZE] __aligned(PAGE_SIZE);
> +	static char table_page[PAGE_SIZE] __aligned(PAGE_SIZE);
> +
> +	switch (type) {
> +	case XDBC_PAGE_EVENT:
> +		virt = (void *)event_page;
> +		break;
> +	case XDBC_PAGE_TXIN:
> +		virt = (void *)in_ring_page;
> +		break;
> +	case XDBC_PAGE_TXOUT:
> +		virt = (void *)out_ring_page;
> +		break;
> +	case XDBC_PAGE_TABLE:
> +		virt = (void *)table_page;
> +		break;
> +	default:
> +		return NULL;
> +	}
> +
> +	memset(virt, 0, PAGE_SIZE);
> +
> +	if (dma_addr)
> +		*dma_addr = (dma_addr_t)__pa(virt);
> +
> +	return virt;

another concern, I can't say if this is ok:

void *virt
static char event_page[PAGE_SIZE] __aligened(PAGE_SIZE)
virt = (void *)event_page
*dma_addr = (dma_addr_t)__pa(virt)

> +}
> +
> +typedef void (*xdbc_walk_excap_cb)(int cap_offset, void *data);
> +
> +/*
> + * xdbc_walk_excap:
> + *
> + * xHCI extended capability list walker.
> + *
> + * @bus - xHC PCI bus#
> + * @dev - xHC PCI dev#
> + * @func - xHC PCI function#
> + * @cap - capability ID
> + * @oneshot - return immediately once hit match
> + * @cb - call back
> + * @data - callback private data
> + *
> + * Return the last cap offset, otherwize 0.
> + */
> +static u32 xdbc_walk_excap(u32 bus, u32 dev, u32 func, int cap,
> +		bool oneshot, xdbc_walk_excap_cb cb, void *data)
> +{
> +	void __iomem *base;
> +	int offset = 0;
> +	size_t len = 0;
> +
> +	if (xdbcp->xhci_base && xdbcp->xhci_length) {
> +		if (xdbcp->bus != bus ||
> +				xdbcp->dev != dev ||
> +				xdbcp->func != func) {
> +			xdbc_trace("only one DbC can be used\n");
> +			return 0;
> +		}
> +
> +		len = xdbcp->xhci_length;
> +		base = xdbcp->xhci_base;
> +	} else {
> +		base = xdbc_map_pci_mmio(bus, dev, func,
> +				PCI_BASE_ADDRESS_0, &len);
> +		if (!base)
> +			return 0;
> +	}
> +
> +	do {
> +		offset = xhci_find_next_ext_cap(base, offset, cap);
> +		if (!offset)
> +			break;
> +
> +		if (cb)
> +			cb(offset, data);
> +		if (oneshot)
> +			break;
> +	} while (1);
> +
> +	return offset;
> +}
> +
> +static u32 __init xdbc_find_dbgp(int xdbc_num,
> +		u32 *rbus, u32 *rdev, u32 *rfunc)
> +{
> +	u32 bus, dev, func, class;
> +	unsigned cap;
> +
> +	for (bus = 0; bus < XDBC_PCI_MAX_BUSES; bus++) {
> +		for (dev = 0; dev < XDBC_PCI_MAX_DEVICES; dev++) {
> +			for (func = 0; func < XDBC_PCI_MAX_FUNCTION; func++) {
> +				class = read_pci_config(bus, dev, func,
> +						PCI_CLASS_REVISION);
> +				if ((class >> 8) != PCI_CLASS_SERIAL_USB_XHCI)
> +					continue;
> +

Can "for_each_pci_dev()" or similar be used here?  Is it too early for that as well?

> +				if (xdbc_num-- != 0)
> +					continue;
> +
> +				cap = xdbc_walk_excap(bus, dev, func,
> +						XHCI_EXT_CAPS_DEBUG,
> +						true, NULL, NULL);
> +				*rbus = bus;
> +				*rdev = dev;
> +				*rfunc = func;
> +				return cap;
> +			}
> +		}
> +	}
> +
> +	return 0;
> +}
> +
> +static int handshake(void __iomem *ptr, u32 mask, u32 done,
> +		int wait_usec, int delay_usec)
> +{
> +	u32	result;
> +
> +	do {
> +		result = readl(ptr);
> +		result &= mask;
> +		if (result == done)
> +			return 0;
> +		xdbc_udelay(delay_usec);
> +		wait_usec -= delay_usec;
> +	} while (wait_usec > 0);
> +
> +	return -ETIMEDOUT;
> +}
> +

...

> +/*
> + * xdbc_start: start DbC
> + *
> + * Set DbC enable bit and wait until DbC run bit being set or timed out.
> + */
> +static int xdbc_start(void)
> +{
> +	u32 ctrl, status;
> +
> +	ctrl = readl(&xdbcp->xdbc_reg->control);
> +	writel(ctrl | CTRL_DCE | CTRL_LSE, &xdbcp->xdbc_reg->control);
> +
> +	if (handshake(&xdbcp->xdbc_reg->control, CTRL_DCE,
> +			CTRL_DCE, 100000, 100) < 0) {
> +		xdbc_trace("falied to initialize hardware\n");
> +		return -ENODEV;
> +	}
> +
> +	/* wait for port connection */
> +	if (handshake(&xdbcp->xdbc_reg->portsc, PORTSC_CCS,
> +			PORTSC_CCS, 5000000, 100) < 0) {
> +		xdbc_trace("waiting for connection timed out\n");
> +		return -ETIMEDOUT;
> +	}
> +	xdbc_trace("port connection detected\n");
> +
> +	/* wait for debug device to be configured */
> +	if (handshake(&xdbcp->xdbc_reg->control, CTRL_DCR,
> +			CTRL_DCR, 5000000, 100) < 0) {
> +		xdbc_trace("waiting for device configuration timed out\n");
> +		return -ETIMEDOUT;
> +	}
> +

If I follow this correcly the initialization goes like:

early_xdbc_init()
   xdbc_setup()
     write 0 to ctrl reg   // clears (disables/stops) the debug capability.
     xdbc_start()
       writel DCE to ctrl reg // this will steal the first port from xhci host driver to debug use.
       wait max 5 sec for connection (PORTSC_CCS)
       wait max 5 second for connected host machine to configure this "device" (CTRL_DCR)

Is this debug capability limited to work only when the debug device is rebooted and a separate
debug host is connected?
If a host connection is not discoved in first 5 seconds, and configured in another 5 seconds, it
gives up alltogether.

Does this work together with xhci_hcd driver? I think xhci_hcd does a HCRST when driver loads, leading to
host driver taking back the first port (DCE = 0). Looks like loading xhci_hcd will kill the xdbc early printk.

xdbc should probably give back the port if connect / configure fails (set back DCE=0)

-Mathias

Leaving some of the initialization code below for reference


> +	/* port should have a valid port# */
> +	status = readl(&xdbcp->xdbc_reg->status);
> +	if (!DCST_DPN(status)) {
> +		xdbc_trace("invalid root hub port number\n");
> +		return -ENODEV;
> +	}
> +
> +	xdbc_trace("root hub port number %d\n", DCST_DPN(status));
> +
> +	xdbc_trace("DbC is running now, control 0x%08x\n",
> +			readl(&xdbcp->xdbc_reg->control));
> +
> +	return 0;
> +}
> +
> +static int xdbc_setup(void)
> +{
> +	int ret;
> +
> +	writel(0, &xdbcp->xdbc_reg->control);
> +	if (handshake(&xdbcp->xdbc_reg->control, CTRL_DCE,
> +			0, 100000, 100) < 0) {
> +		xdbc_trace("falied to initialize hardware\n");
> +		return -ETIMEDOUT;
> +	}
> +
> +	/* allocate and initialize all memory data structures */
> +	ret = xdbc_mem_init();
> +	if (ret < 0) {
> +		xdbc_trace("failed to initialize memory\n");
> +		return ret;
> +	}
> +
> +	/*
> +	 * Memory barrier to ensure hardware sees the bits
> +	 * setting above.
> +	 */
> +	mmiowb();
> +
> +	/* dump registers and data structures */
> +	xdbc_dbg_dump_regs("hardware setup completed");
> +	xdbc_dbg_dump_data("hardware setup completed");
> +
> +	ret = xdbc_start();
> +	if (ret < 0) {
> +		xdbc_trace("failed to start DbC, cable connected?\n");
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> +
> +int __init early_xdbc_init(char *s)
> +{
> +	u32 bus = 0, dev = 0, func = 0;
> +	unsigned long dbgp_num = 0;
> +	u32 offset;
> +	int ret;
> +
> +	if (!early_pci_allowed())
> +		return -EPERM;
> +
> +	/* FIXME: early printk "keep" option will be supported later */
> +	if (strstr(s, "keep"))
> +		return -EPERM;
> +
> +	if (xdbcp->xdbc_reg)
> +		return 0;
> +
> +	if (*s && kstrtoul(s, 0, &dbgp_num))
> +		dbgp_num = 0;
> +
> +	xdbc_trace("dbgp_num: %lu\n", dbgp_num);
> +
> +	offset = xdbc_find_dbgp(dbgp_num, &bus, &dev, &func);
> +	if (!offset)
> +		return -ENODEV;
> +
> +	xdbc_trace("Found xHCI debug capability on %02x:%02x.%1x\n",
> +			bus, dev, func);
> +
> +	if (!xdbcp->xhci_base)
> +		return -EINVAL;
> +
> +	xdbcp->xdbc_reg = (struct xdbc_regs __iomem *)
> +			(xdbcp->xhci_base + offset);
> +	xdbc_dbg_dump_regs("debug capability located");
> +
> +	/* hand over the owner of host from BIOS */
> +	xdbc_bios_handoff();
> +
> +	ret = xdbc_setup();
> +	if (ret < 0) {
> +		pr_notice("failed to setup xHCI DbC connection\n");
> +		xdbcp->xhci_base = NULL;
> +		xdbcp->xdbc_reg = NULL;
> +		return ret;
> +	}
> +
> +	return 0;
> +}
> diff --git a/include/linux/usb/xhci-dbc.h b/include/linux/usb/xhci-dbc.h
> new file mode 100644
> index 0000000..153fb87
> --- /dev/null
> +++ b/include/linux/usb/xhci-dbc.h
> @@ -0,0 +1,187 @@
> +/*
> + * xHCI debug capability driver
> + *

Re: [PATCH v7 02/10] usb: dbc: probe and setup xhci debug capability

[Lu Baolu]

Hi Mathias,

Thanks for your time.

On 02/16/2016 10:19 PM, Mathias Nyman wrote:
> Hi
>
> Most of my concerns are due to the early printk restrictions.
> As you say in the documentation:
> "Keep in mind that things such as interrupt, system memory, DMA memory,
> PCI configure space access, etc., are all different from a normal device
> driver."
>
> I don't know the details of these subsystems, or if what is done here is
> done the "right" way.
>
> DMA capable memory is allocated with static char xpage[PAGE_SIZE] __aligned,
> and DMA address is found using  __pa()
>
> xdbc_map_pci_mmio() does some pci magic and fixmapping that I don't know the
> details of.
>
> writing to ioport 0x80 outb(1, 0x80) is used as 1us delay, sometimes up to
> 5 million times to get a 5 second timeout
>
> Adding Bjorn Helgaas and Dan Williams in case they have an opinoin about
> the DMA and PCI cases.

All the quirks in this file have been documented and verified to work.
Your concerns are right, I'll be glad to see other opinions.

>
>
> more comments inline
>
> On 26.01.2016 14:58, Lu Baolu wrote:
>> xHCI debug capability (DbC) is an optional functionality provided
>> by an xHCI host controller. Software learns this capability by
>> walking through the extended capability list in mmio of the host.
>>
>> This patch introduces the code to probe and initialize the debug
>> capability hardware during early boot. With hardware initialization
>> done, the debug target (system under debug which has DbC enabled)
>> will present a debug device through the debug port. The debug device
>> is fully compliant with the USB framework and provides the equivalent
>> of a very high performance (USB3) full-duplex serial link between the
>> debug host and target.
>>
>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>> ---
>>   MAINTAINERS                  |   7 +
>>   arch/x86/Kconfig.debug       |  12 +
>>   drivers/usb/early/Makefile   |   1 +
>>   drivers/usb/early/xhci-dbc.c | 774 +++++++++++++++++++++++++++++++++++++++++++
>>   include/linux/usb/xhci-dbc.h | 187 +++++++++++
>>   5 files changed, 981 insertions(+)
>>   create mode 100644 drivers/usb/early/xhci-dbc.c
>>   create mode 100644 include/linux/usb/xhci-dbc.h
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 30aca4a..e6d7076 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -11505,6 +11505,13 @@ S:    Supported
>>   F:    drivers/usb/host/xhci*
>>   F:    drivers/usb/host/pci-quirks*
>>
>> +USB XHCI DEBUG PORT
>> +M:    Lu Baolu <baolu.lu@linux.intel.com>
>> +L:    linux-usb@vger.kernel.org
>> +S:    Supported
>> +F:    drivers/usb/early/xhci-dbc.c
>> +F:    include/linux/usb/xhci-dbc.h
>> +
>>   USB ZD1201 DRIVER
>>   L:    linux-wireless@vger.kernel.org
>>   W:    http://linux-lc100020.sourceforge.net
>> diff --git a/arch/x86/Kconfig.debug b/arch/x86/Kconfig.debug
>> index 9b18ed9..ba60cb1 100644
>> --- a/arch/x86/Kconfig.debug
>> +++ b/arch/x86/Kconfig.debug
>> @@ -48,6 +48,18 @@ config EARLY_PRINTK_EFI
>>         This is useful for kernel debugging when your machine crashes very
>>         early before the console code is initialized.
>>
>> +config EARLY_PRINTK_XDBC
>> +    bool "Early printk via xHCI debug port"
>> +    depends on EARLY_PRINTK && PCI
>> +    ---help---
>> +      Write kernel log output directly into the xHCI debug port.
>> +
>> +      This is useful for kernel debugging when your machine crashes very
>> +      early before the console code is initialized. For normal operation
>> +      it is not recommended because it looks ugly and doesn't cooperate
>> +      with klogd/syslogd or the X server. You should normally N here,
>> +      unless you want to debug such a crash.
>> +
>>   config X86_PTDUMP_CORE
>>       def_bool n
>>
>> diff --git a/drivers/usb/early/Makefile b/drivers/usb/early/Makefile
>> index 24bbe51..2db5906 100644
>> --- a/drivers/usb/early/Makefile
>> +++ b/drivers/usb/early/Makefile
>> @@ -3,3 +3,4 @@
>>   #
>>
>>   obj-$(CONFIG_EARLY_PRINTK_DBGP) += ehci-dbgp.o
>> +obj-$(CONFIG_EARLY_PRINTK_XDBC) += xhci-dbc.o
>> diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
>> new file mode 100644
>> index 0000000..254a0a8
>> --- /dev/null
>> +++ b/drivers/usb/early/xhci-dbc.c
>> @@ -0,0 +1,774 @@
>> +/**
>> + * xhci-dbc.c - xHCI debug capability driver
>> + *
>> + * Copyright (C) 2015 Intel Corporation
>> + *
>> + * Author: Lu Baolu <baolu.lu@linux.intel.com>
>> + * Some code shared with EHCI debug port and xHCI driver.
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + */
>> +#include <linux/pci_regs.h>
>> +#include <linux/pci_ids.h>
>> +#include <linux/bootmem.h>
>> +#include <linux/io.h>
>> +#include <asm/pci-direct.h>
>> +#include <asm/fixmap.h>
>> +#include <linux/bcd.h>
>> +#include <linux/export.h>
>> +#include <linux/version.h>
>> +#include <linux/usb/xhci-dbc.h>
>> +
>> +#include "../host/xhci.h"
>> +
>> +#define    XDBC_PROTOCOL        1    /* GNU Remote Debug Command Set */
>> +#define    XDBC_VENDOR_ID        0x1d6b    /* Linux Foundation 0x1d6b */
>> +#define    XDBC_PRODUCT_ID        0x0004    /* __le16 idProduct; device 0004 */
>> +#define    XDBC_DEVICE_REV        0x0010    /* 0.10 */
>> +
>> +static struct xdbc_state xdbc_stat;
>> +static struct xdbc_state *xdbcp = &xdbc_stat;
>> +
>>
>
> ...
>
>> +
>> +#else
>> +static inline void xdbc_trace(const char *fmt, ...) { }
>> +static inline void xdbc_dbg_dump_regs(char *str) { }
>> +static inline void xdbc_dbg_dump_data(char *str) { }
>> +#endif    /* DBC_DEBUG */
>> +
>> +/*
>> + * FIXME: kernel provided delay interfaces, like usleep, isn't ready yet
>> + *        at the time DbC gets initialized. Below implementation is only
>> + *        for x86 platform. Need to reconsider this when porting it onto
>> + *        other architectures.
>> + */
>> +static inline void xdbc_udelay(int us)
>> +{
>> +    while (us-- > 0)
>> +        outb(0x1, 0x80);
>> +}
>
> This is one of the concerns, the FIXME text describes the problem but still wondering if
> there is any other solution. In many cases later in this file we will wait up to 5 seconds
> for something, ending up writing 5 million times to ioport 0x80.
>
>> +
>> +static void __iomem *xdbc_map_pci_mmio(u32 bus,
>> +        u32 dev, u32 func, u8 bar, size_t *length)
>> +{
>> +    u32 val, sz;
>> +    u64 val64, sz64, mask64;
>> +    u8 byte;
>> +    unsigned long idx, max_idx;
>> +    void __iomem *base;
>> +
>> +    val = read_pci_config(bus, dev, func, bar);
>> +    write_pci_config(bus, dev, func, bar, ~0);
>> +    sz = read_pci_config(bus, dev, func, bar);
>> +    write_pci_config(bus, dev, func, bar, val);
>> +    if (val == 0xffffffff || sz == 0xffffffff) {
>> +        xdbc_trace("invalid mmio bar\n");
>> +        return NULL;
>> +    }
>> +
>> +    val64 = val & PCI_BASE_ADDRESS_MEM_MASK;
>> +    sz64 = sz & PCI_BASE_ADDRESS_MEM_MASK;
>> +    mask64 = (u32)PCI_BASE_ADDRESS_MEM_MASK;
>> +
>> +    if ((val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
>> +            PCI_BASE_ADDRESS_MEM_TYPE_64) {
>> +        val = read_pci_config(bus, dev, func, bar + 4);
>> +        write_pci_config(bus, dev, func, bar + 4, ~0);
>> +        sz = read_pci_config(bus, dev, func, bar + 4);
>> +        write_pci_config(bus, dev, func, bar + 4, val);
>> +
>> +        val64 |= ((u64)val << 32);
>> +        sz64 |= ((u64)sz << 32);
>> +        mask64 |= ((u64)~0 << 32);
>> +    }
>> +
>> +    sz64 &= mask64;
>> +
>> +    if (sizeof(dma_addr_t) < 8 || !sz64) {
>> +        xdbc_trace("can't handle 64bit BAR\n");
>> +        return NULL;
>> +    }
>> +
>> +    sz64 = 1ULL << __ffs64(sz64);
>> +
>> +    if (sz64 > (FIX_XDBC_END - FIX_XDBC_BASE + 1) * PAGE_SIZE) {
>> +        xdbc_trace("mmio size beyond 64k not supported\n");
>> +        return NULL;
>> +    }
>> +
>> +    xdbc_trace("bar: base 0x%llx size 0x%llx offset %03x\n",
>> +            val64, sz64, bar);
>> +
>> +    /* check if the mem space is enabled */
>> +    byte = read_pci_config_byte(bus, dev, func, PCI_COMMAND);
>> +    if (!(byte & PCI_COMMAND_MEMORY)) {
>> +        byte  |= PCI_COMMAND_MEMORY;
>> +        write_pci_config_byte(bus, dev, func, PCI_COMMAND, byte);
>> +        xdbc_trace("mmio for xhci enabled\n");
>> +    }
>> +
>> +    /* 64k mmio will be fix-mapped */
>> +    max_idx = FIX_XDBC_END - FIX_XDBC_BASE;
>> +    for (idx = 0; idx <= max_idx; idx++)
>> +        set_fixmap_nocache(FIX_XDBC_BASE + idx,
>> +            (val64 & PAGE_MASK) + (max_idx - idx) * PAGE_SIZE);
>> +    base = (void __iomem *)__fix_to_virt(FIX_XDBC_END);
>> +    base += val64 & ~PAGE_MASK;
>> +
>> +    /* save in the state block */
>> +    xdbcp->bus = bus;
>> +    xdbcp->dev = dev;
>> +    xdbcp->func = func;
>> +    xdbcp->bar = bar;
>> +    xdbcp->xhci_base = base;
>> +    xdbcp->xhci_length = sz64;
>> +
>> +    if (length)
>> +        *length = sz64;
>> +
>> +    return base;
>> +}
>> +
>> +/*
>> + * FIXME: The bootmem allocator isn't ready at the time when DbC gets
>> + *        initialized. Below implementation reserves DMA memory blocks
>> + *        in the kernel static data segment.
>> + */
>> +static void *xdbc_get_page(dma_addr_t *dma_addr,
>> +        enum xdbc_page_type type)
>> +{
>> +    void *virt;
>> +    static char event_page[PAGE_SIZE] __aligned(PAGE_SIZE);
>> +    static char in_ring_page[PAGE_SIZE] __aligned(PAGE_SIZE);
>> +    static char out_ring_page[PAGE_SIZE] __aligned(PAGE_SIZE);
>> +    static char table_page[PAGE_SIZE] __aligned(PAGE_SIZE);
>> +
>> +    switch (type) {
>> +    case XDBC_PAGE_EVENT:
>> +        virt = (void *)event_page;
>> +        break;
>> +    case XDBC_PAGE_TXIN:
>> +        virt = (void *)in_ring_page;
>> +        break;
>> +    case XDBC_PAGE_TXOUT:
>> +        virt = (void *)out_ring_page;
>> +        break;
>> +    case XDBC_PAGE_TABLE:
>> +        virt = (void *)table_page;
>> +        break;
>> +    default:
>> +        return NULL;
>> +    }
>> +
>> +    memset(virt, 0, PAGE_SIZE);
>> +
>> +    if (dma_addr)
>> +        *dma_addr = (dma_addr_t)__pa(virt);
>> +
>> +    return virt;
>
> another concern, I can't say if this is ok:
>
> void *virt
> static char event_page[PAGE_SIZE] __aligened(PAGE_SIZE)
> virt = (void *)event_page
> *dma_addr = (dma_addr_t)__pa(virt)
>
>> +}
>> +
>> +typedef void (*xdbc_walk_excap_cb)(int cap_offset, void *data);
>> +
>> +/*
>> + * xdbc_walk_excap:
>> + *
>> + * xHCI extended capability list walker.
>> + *
>> + * @bus - xHC PCI bus#
>> + * @dev - xHC PCI dev#
>> + * @func - xHC PCI function#
>> + * @cap - capability ID
>> + * @oneshot - return immediately once hit match
>> + * @cb - call back
>> + * @data - callback private data
>> + *
>> + * Return the last cap offset, otherwize 0.
>> + */
>> +static u32 xdbc_walk_excap(u32 bus, u32 dev, u32 func, int cap,
>> +        bool oneshot, xdbc_walk_excap_cb cb, void *data)
>> +{
>> +    void __iomem *base;
>> +    int offset = 0;
>> +    size_t len = 0;
>> +
>> +    if (xdbcp->xhci_base && xdbcp->xhci_length) {
>> +        if (xdbcp->bus != bus ||
>> +                xdbcp->dev != dev ||
>> +                xdbcp->func != func) {
>> +            xdbc_trace("only one DbC can be used\n");
>> +            return 0;
>> +        }
>> +
>> +        len = xdbcp->xhci_length;
>> +        base = xdbcp->xhci_base;
>> +    } else {
>> +        base = xdbc_map_pci_mmio(bus, dev, func,
>> +                PCI_BASE_ADDRESS_0, &len);
>> +        if (!base)
>> +            return 0;
>> +    }
>> +
>> +    do {
>> +        offset = xhci_find_next_ext_cap(base, offset, cap);
>> +        if (!offset)
>> +            break;
>> +
>> +        if (cb)
>> +            cb(offset, data);
>> +        if (oneshot)
>> +            break;
>> +    } while (1);
>> +
>> +    return offset;
>> +}
>> +
>> +static u32 __init xdbc_find_dbgp(int xdbc_num,
>> +        u32 *rbus, u32 *rdev, u32 *rfunc)
>> +{
>> +    u32 bus, dev, func, class;
>> +    unsigned cap;
>> +
>> +    for (bus = 0; bus < XDBC_PCI_MAX_BUSES; bus++) {
>> +        for (dev = 0; dev < XDBC_PCI_MAX_DEVICES; dev++) {
>> +            for (func = 0; func < XDBC_PCI_MAX_FUNCTION; func++) {
>> +                class = read_pci_config(bus, dev, func,
>> +                        PCI_CLASS_REVISION);
>> +                if ((class >> 8) != PCI_CLASS_SERIAL_USB_XHCI)
>> +                    continue;
>> +
>
> Can "for_each_pci_dev()" or similar be used here?  Is it too early for that as well?

Yes. It's too early for "for_each_pci_dev()".

>
>> +                if (xdbc_num-- != 0)
>> +                    continue;
>> +
>> +                cap = xdbc_walk_excap(bus, dev, func,
>> +                        XHCI_EXT_CAPS_DEBUG,
>> +                        true, NULL, NULL);
>> +                *rbus = bus;
>> +                *rdev = dev;
>> +                *rfunc = func;
>> +                return cap;
>> +            }
>> +        }
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +static int handshake(void __iomem *ptr, u32 mask, u32 done,
>> +        int wait_usec, int delay_usec)
>> +{
>> +    u32    result;
>> +
>> +    do {
>> +        result = readl(ptr);
>> +        result &= mask;
>> +        if (result == done)
>> +            return 0;
>> +        xdbc_udelay(delay_usec);
>> +        wait_usec -= delay_usec;
>> +    } while (wait_usec > 0);
>> +
>> +    return -ETIMEDOUT;
>> +}
>> +
>
> ...
>
>> +/*
>> + * xdbc_start: start DbC
>> + *
>> + * Set DbC enable bit and wait until DbC run bit being set or timed out.
>> + */
>> +static int xdbc_start(void)
>> +{
>> +    u32 ctrl, status;
>> +
>> +    ctrl = readl(&xdbcp->xdbc_reg->control);
>> +    writel(ctrl | CTRL_DCE | CTRL_LSE, &xdbcp->xdbc_reg->control);
>> +
>> +    if (handshake(&xdbcp->xdbc_reg->control, CTRL_DCE,
>> +            CTRL_DCE, 100000, 100) < 0) {
>> +        xdbc_trace("falied to initialize hardware\n");
>> +        return -ENODEV;
>> +    }
>> +
>> +    /* wait for port connection */
>> +    if (handshake(&xdbcp->xdbc_reg->portsc, PORTSC_CCS,
>> +            PORTSC_CCS, 5000000, 100) < 0) {
>> +        xdbc_trace("waiting for connection timed out\n");
>> +        return -ETIMEDOUT;
>> +    }
>> +    xdbc_trace("port connection detected\n");
>> +
>> +    /* wait for debug device to be configured */
>> +    if (handshake(&xdbcp->xdbc_reg->control, CTRL_DCR,
>> +            CTRL_DCR, 5000000, 100) < 0) {
>> +        xdbc_trace("waiting for device configuration timed out\n");
>> +        return -ETIMEDOUT;
>> +    }
>> +
>
> If I follow this correcly the initialization goes like:
>
> early_xdbc_init()
>   xdbc_setup()
>     write 0 to ctrl reg   // clears (disables/stops) the debug capability.
>     xdbc_start()
>       writel DCE to ctrl reg // this will steal the first port from xhci host driver to debug use.
>       wait max 5 sec for connection (PORTSC_CCS)
>       wait max 5 second for connected host machine to configure this "device" (CTRL_DCR)
>
> Is this debug capability limited to work only when the debug device is rebooted and a separate
> debug host is connected?

Yes. The usage case is very limited. It's used for getting kernel messages at early boot stage.

> If a host connection is not discoved in first 5 seconds, and configured in another 5 seconds, it
> gives up alltogether.
>
> Does this work together with xhci_hcd driver? I think xhci_hcd does a HCRST when driver loads, leading to
> host driver taking back the first port (DCE = 0). Looks like loading xhci_hcd will kill the xdbc early printk.

By default,  all early printk consoles end when a normal console got created. xhci_hcd gets loaded after
that, and HCRST will take back the port. Hence, xdbc early printk doesn't need to work with xhci_hcd.

Early printk has a "keep" option. When "keep" is specified. The early printk boot console will keep alive
in the whole system life. To achieve that with xdbc, there are more work to make it work with xhci_hcd.
That's my next work. Current xdbc early printk will just ignore "keep" option.

My final goal is to make it a reasonable alternative for serial port given the fact that USB 3.0
is more and more popular.

>
> xdbc should probably give back the port if connect / configure fails (set back DCE=0)

Yes. That sounds more reasonable.

>
> -Mathias

Thanks,
Baolu

>
> Leaving some of the initialization code below for reference
>
>
>> +    /* port should have a valid port# */
>> +    status = readl(&xdbcp->xdbc_reg->status);
>> +    if (!DCST_DPN(status)) {
>> +        xdbc_trace("invalid root hub port number\n");
>> +        return -ENODEV;
>> +    }
>> +
>> +    xdbc_trace("root hub port number %d\n", DCST_DPN(status));
>> +
>> +    xdbc_trace("DbC is running now, control 0x%08x\n",
>> +            readl(&xdbcp->xdbc_reg->control));
>> +
>> +    return 0;
>> +}
>> +
>> +static int xdbc_setup(void)
>> +{
>> +    int ret;
>> +
>> +    writel(0, &xdbcp->xdbc_reg->control);
>> +    if (handshake(&xdbcp->xdbc_reg->control, CTRL_DCE,
>> +            0, 100000, 100) < 0) {
>> +        xdbc_trace("falied to initialize hardware\n");
>> +        return -ETIMEDOUT;
>> +    }
>> +
>> +    /* allocate and initialize all memory data structures */
>> +    ret = xdbc_mem_init();
>> +    if (ret < 0) {
>> +        xdbc_trace("failed to initialize memory\n");
>> +        return ret;
>> +    }
>> +
>> +    /*
>> +     * Memory barrier to ensure hardware sees the bits
>> +     * setting above.
>> +     */
>> +    mmiowb();
>> +
>> +    /* dump registers and data structures */
>> +    xdbc_dbg_dump_regs("hardware setup completed");
>> +    xdbc_dbg_dump_data("hardware setup completed");
>> +
>> +    ret = xdbc_start();
>> +    if (ret < 0) {
>> +        xdbc_trace("failed to start DbC, cable connected?\n");
>> +        return ret;
>> +    }
>> +
>> +    return 0;
>> +}
>> +
>> +int __init early_xdbc_init(char *s)
>> +{
>> +    u32 bus = 0, dev = 0, func = 0;
>> +    unsigned long dbgp_num = 0;
>> +    u32 offset;
>> +    int ret;
>> +
>> +    if (!early_pci_allowed())
>> +        return -EPERM;
>> +
>> +    /* FIXME: early printk "keep" option will be supported later */
>> +    if (strstr(s, "keep"))
>> +        return -EPERM;
>> +
>> +    if (xdbcp->xdbc_reg)
>> +        return 0;
>> +
>> +    if (*s && kstrtoul(s, 0, &dbgp_num))
>> +        dbgp_num = 0;
>> +
>> +    xdbc_trace("dbgp_num: %lu\n", dbgp_num);
>> +
>> +    offset = xdbc_find_dbgp(dbgp_num, &bus, &dev, &func);
>> +    if (!offset)
>> +        return -ENODEV;
>> +
>> +    xdbc_trace("Found xHCI debug capability on %02x:%02x.%1x\n",
>> +            bus, dev, func);
>> +
>> +    if (!xdbcp->xhci_base)
>> +        return -EINVAL;
>> +
>> +    xdbcp->xdbc_reg = (struct xdbc_regs __iomem *)
>> +            (xdbcp->xhci_base + offset);
>> +    xdbc_dbg_dump_regs("debug capability located");
>> +
>> +    /* hand over the owner of host from BIOS */
>> +    xdbc_bios_handoff();
>> +
>> +    ret = xdbc_setup();
>> +    if (ret < 0) {
>> +        pr_notice("failed to setup xHCI DbC connection\n");
>> +        xdbcp->xhci_base = NULL;
>> +        xdbcp->xdbc_reg = NULL;
>> +        return ret;
>> +    }
>> +
>> +    return 0;
>> +}
>> diff --git a/include/linux/usb/xhci-dbc.h b/include/linux/usb/xhci-dbc.h
>> new file mode 100644
>> index 0000000..153fb87
>> --- /dev/null
>> +++ b/include/linux/usb/xhci-dbc.h
>> @@ -0,0 +1,187 @@
>> +/*
>> + * xHCI debug capability driver
>> + *
>
>

Re: [PATCH v7 04/10] usb: dbc: add debug buffer

[Mathias Nyman]

On 26.01.2016 14:58, Lu Baolu wrote:
> "printk" is not suitable for dbc debugging especially when console
> is in usage. This patch adds a debug buffer in dbc driver and puts
> the debug messages in this local buffer. The debug buffer could be
> dumped whenever the console is not in use. This part of code will
> not be visible unless DBC_DEBUG is defined.
>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> ---
>   drivers/usb/early/xhci-dbc.c | 62 ++++++++++++++++++++++++++++++++++++++++++--
>   1 file changed, 60 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
> index 41ce116..6855048 100644
> --- a/drivers/usb/early/xhci-dbc.c
> +++ b/drivers/usb/early/xhci-dbc.c
> @@ -32,8 +32,64 @@ static struct xdbc_state xdbc_stat;
>   static struct xdbc_state *xdbcp = &xdbc_stat;
>
>   #ifdef DBC_DEBUG
> -/* place holder */
> -#define	xdbc_trace	printk
> +#define	XDBC_DEBUG_BUF_SIZE	(PAGE_SIZE * 32)

Does it really need to be this huge? minimum 4096 * 32 ~ 128k
The kernel ring  buffer is about the same size (16k - 256k)

> +#define	MSG_MAX_LINE		128

with 128 characters per line this would fit ~1000 lines

> +static char xdbc_debug_buf[XDBC_DEBUG_BUF_SIZE];
> +static void xdbc_trace(const char *fmt, ...)
> +{
> +	int i, size;
> +	va_list args;
> +	static int pos;
> +	char temp_buf[MSG_MAX_LINE];
> +
> +	if (pos >= XDBC_DEBUG_BUF_SIZE - 1)
> +		return;
> +
> +	memset(temp_buf, 0, MSG_MAX_LINE);
> +	va_start(args, fmt);
> +	vsnprintf(temp_buf, MSG_MAX_LINE - 1, fmt, args);
> +	va_end(args);
> +
> +	i = 0;
> +	size = strlen(temp_buf);
> +	while (i < size) {
> +		xdbc_debug_buf[pos] = temp_buf[i];
> +		pos++;
> +		i++;
> +
> +		if (pos >= XDBC_DEBUG_BUF_SIZE - 1)
> +			break;
> +	}

how about something like:

size = min(XDBC_DEBUG_BUF_SIZE - pos, size)
memcpy(xdbc_debug_buf + pos, temp_buf, size)
pos += size;

(might need some "-1" and off by one checking..)

> +}
> +
> +static void xdbc_dump_debug_buffer(void)
> +{
> +	int index = 0;
> +	int count = 0;
> +	char dump_buf[MSG_MAX_LINE];
> +
> +	xdbc_trace("The end of DbC trace buffer\n");
> +	pr_notice("DBC debug buffer:\n");
> +	memset(dump_buf, 0, MSG_MAX_LINE);
> +
> +	while (index < XDBC_DEBUG_BUF_SIZE) {
> +		if (!xdbc_debug_buf[index])
> +			break;
> +
> +		if (xdbc_debug_buf[index] == '\n' ||
> +				count >= MSG_MAX_LINE - 1) {
> +			pr_notice("DBC: @%08x %s\n", index, dump_buf);

Is showing the he index (position in debug buffer) useful here?

> +			memset(dump_buf, 0, MSG_MAX_LINE);
> +			count = 0;
> +		} else {
> +			dump_buf[count] = xdbc_debug_buf[index];
> +			count++;
> +		}
> +
> +		index++;
> +	}

So we have one huge buffer that xdbc keeps on filling as the initialization progresses.
It is never emptied, or overwritten (circular).
When dumped it always dumps the whole thing, copying one character
at a time.

As this is only used for debugging during xdbc development/debugging, and never enabled
even if xdbc early printk is used, I don't think optimization really matters.

Perhaps take a look if we really need PAGE_SIZE * 32 bytes, is xdbc driver even nearly
writing that much debug data.
  
-Mathias

Re: [PATCH v7 05/10] usb: dbc: add bulk out and bulk in interfaces

[Mathias Nyman]

On 26.01.2016 14:58, Lu Baolu wrote:
> This patch adds interfaces for bulk out and bulk in ops. These
> interfaces could be used to implement early printk bootconsole
> or hook to various system debuggers.
>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> ---
>   drivers/usb/early/xhci-dbc.c | 373 +++++++++++++++++++++++++++++++++++++++++++
>   include/linux/usb/xhci-dbc.h |  30 ++++
>   2 files changed, 403 insertions(+)
>

...

> +
> +/*
> + * Check and dispatch events in event ring. It also checks status
> + * of hardware. This function will be called from multiple threads.
> + * An atomic lock is applied to protect the access of event ring.
> + */
> +static int xdbc_check_event(void)
> +{
> +	/* event ring is under checking by other thread? */
> +	if (!test_bit(XDBC_ATOMIC_EVENT, &xdbcp->atomic_flags) &&
> +			!test_and_set_bit(XDBC_ATOMIC_EVENT,
> +			&xdbcp->atomic_flags))
> +		return 0;

homemade trylock, can't the real ones be used?

> +
> +	xdbc_handle_events();
> +
> +	test_and_clear_bit(XDBC_ATOMIC_EVENT, &xdbcp->atomic_flags);
> +
> +	return 0;
>  +}
> +
> +#define	BULK_IN_COMPLETED(p)	((xdbcp->in_pending == (p)) && \
> +				 xdbcp->in_complete)
> +#define	BULK_OUT_COMPLETED(p)	((xdbcp->out_pending == (p)) && \
> +				 xdbcp->out_complete)
> +

...

> +}
> +
> +int xdbc_bulk_read(void *data, int size, int loops)
> +{
> +	int ret;
> +
> +	do {
> +		if (!test_bit(XDBC_ATOMIC_BULKIN, &xdbcp->atomic_flags) &&
> +				!test_and_set_bit(XDBC_ATOMIC_BULKIN,
> +				&xdbcp->atomic_flags))
> +			break;
> +	} while (1);

homemeade spin_lock, can't the real one be used?

If the xdbc_bulk_write() can be accessed from interrupt context (handler, soft, timer) it
may deadlock

> +
> +	ret = xdbc_bulk_transfer(data, size, loops, true);
> +
> +	test_and_clear_bit(XDBC_ATOMIC_BULKIN, &xdbcp->atomic_flags);
> +
> +	return ret;
> +}
> +
> +int xdbc_bulk_write(const char *bytes, int size)
> +{
> +	int ret;
> +
> +	do {
> +		if (!test_bit(XDBC_ATOMIC_BULKOUT, &xdbcp->atomic_flags) &&
> +				!test_and_set_bit(XDBC_ATOMIC_BULKOUT,
> +				&xdbcp->atomic_flags))
> +			break;
> +	} while (1);

Another homemeade spin_lock, can't the real one be used?

same issue here, deadlock if accessible from interrupt context.


Would it make sense to have only one spinlock, and start one separate thread for
reading the event ring. The thread would,  lock, handle pending events, unlock,
then call shedule, in a loop. ehci early debug code has some variant of this.

So the lock would be taken while events are being handled.

The same lock would be used for bulk_read and bulk_write. Yes this would prevent read and
write at the same time, and the read and writes need to be modified to not block until
the reansfer is finished, just to write the TRBs on the ring, update ring pointers,
and ring the doorbell.

Or is all this impossibe due to the earlyness of the code?

-Mathias

Re: [PATCH v7 04/10] usb: dbc: add debug buffer

[Lu Baolu]

On 02/18/2016 07:43 PM, Mathias Nyman wrote:
> On 26.01.2016 14:58, Lu Baolu wrote:
>> "printk" is not suitable for dbc debugging especially when console
>> is in usage. This patch adds a debug buffer in dbc driver and puts
>> the debug messages in this local buffer. The debug buffer could be
>> dumped whenever the console is not in use. This part of code will
>> not be visible unless DBC_DEBUG is defined.
>>
>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>> ---
>>   drivers/usb/early/xhci-dbc.c | 62 ++++++++++++++++++++++++++++++++++++++++++--
>>   1 file changed, 60 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
>> index 41ce116..6855048 100644
>> --- a/drivers/usb/early/xhci-dbc.c
>> +++ b/drivers/usb/early/xhci-dbc.c
>> @@ -32,8 +32,64 @@ static struct xdbc_state xdbc_stat;
>>   static struct xdbc_state *xdbcp = &xdbc_stat;
>>
>>   #ifdef DBC_DEBUG
>> -/* place holder */
>> -#define    xdbc_trace    printk
>> +#define    XDBC_DEBUG_BUF_SIZE    (PAGE_SIZE * 32)
>
> Does it really need to be this huge? minimum 4096 * 32 ~ 128k
> The kernel ring  buffer is about the same size (16k - 256k)

This debug buffer is only used when DBC_DEBUG is defined.

128k is a little large. I will change it to 16k.

>
>> +#define    MSG_MAX_LINE        128
>
> with 128 characters per line this would fit ~1000 lines
>
>> +static char xdbc_debug_buf[XDBC_DEBUG_BUF_SIZE];
>> +static void xdbc_trace(const char *fmt, ...)
>> +{
>> +    int i, size;
>> +    va_list args;
>> +    static int pos;
>> +    char temp_buf[MSG_MAX_LINE];
>> +
>> +    if (pos >= XDBC_DEBUG_BUF_SIZE - 1)
>> +        return;
>> +
>> +    memset(temp_buf, 0, MSG_MAX_LINE);
>> +    va_start(args, fmt);
>> +    vsnprintf(temp_buf, MSG_MAX_LINE - 1, fmt, args);
>> +    va_end(args);
>> +
>> +    i = 0;
>> +    size = strlen(temp_buf);
>> +    while (i < size) {
>> +        xdbc_debug_buf[pos] = temp_buf[i];
>> +        pos++;
>> +        i++;
>> +
>> +        if (pos >= XDBC_DEBUG_BUF_SIZE - 1)
>> +            break;
>> +    }
>
> how about something like:
>
> size = min(XDBC_DEBUG_BUF_SIZE - pos, size)
> memcpy(xdbc_debug_buf + pos, temp_buf, size)
> pos += size;
>
> (might need some "-1" and off by one checking..)

Yes. This looks better.

>
>> +}
>> +
>> +static void xdbc_dump_debug_buffer(void)
>> +{
>> +    int index = 0;
>> +    int count = 0;
>> +    char dump_buf[MSG_MAX_LINE];
>> +
>> +    xdbc_trace("The end of DbC trace buffer\n");
>> +    pr_notice("DBC debug buffer:\n");
>> +    memset(dump_buf, 0, MSG_MAX_LINE);
>> +
>> +    while (index < XDBC_DEBUG_BUF_SIZE) {
>> +        if (!xdbc_debug_buf[index])
>> +            break;
>> +
>> +        if (xdbc_debug_buf[index] == '\n' ||
>> +                count >= MSG_MAX_LINE - 1) {
>> +            pr_notice("DBC: @%08x %s\n", index, dump_buf);
>
> Is showing the he index (position in debug buffer) useful here?

It helps to check the debug log especially when it shows the hardware
registers or memory block.

>
>
>> +            memset(dump_buf, 0, MSG_MAX_LINE);
>> +            count = 0;
>> +        } else {
>> +            dump_buf[count] = xdbc_debug_buf[index];
>> +            count++;
>> +        }
>> +
>> +        index++;
>> +    }
>
> So we have one huge buffer that xdbc keeps on filling as the initialization progresses.
> It is never emptied, or overwritten (circular).
> When dumped it always dumps the whole thing, copying one character
> at a time.
>
> As this is only used for debugging during xdbc development/debugging, and never enabled
> even if xdbc early printk is used, I don't think optimization really matters.

Yes, it's only a helper for the persons who develop and debug this driver.

>
> Perhaps take a look if we really need PAGE_SIZE * 32 bytes, is xdbc driver even nearly
> writing that much debug data.

I will reduce the debug buffer size.

>  
> -Mathias
>
>

Thanks for your time.

Regards,
-Baolu

Re: [PATCH v7 05/10] usb: dbc: add bulk out and bulk in interfaces

[Lu Baolu]

On 02/18/2016 09:32 PM, Mathias Nyman wrote:
> On 26.01.2016 14:58, Lu Baolu wrote:
>> This patch adds interfaces for bulk out and bulk in ops. These
>> interfaces could be used to implement early printk bootconsole
>> or hook to various system debuggers.
>>
>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>> ---
>>   drivers/usb/early/xhci-dbc.c | 373 +++++++++++++++++++++++++++++++++++++++++++
>>   include/linux/usb/xhci-dbc.h |  30 ++++
>>   2 files changed, 403 insertions(+)
>>
>
> ...
>
>> +
>> +/*
>> + * Check and dispatch events in event ring. It also checks status
>> + * of hardware. This function will be called from multiple threads.
>> + * An atomic lock is applied to protect the access of event ring.
>> + */
>> +static int xdbc_check_event(void)
>> +{
>> +    /* event ring is under checking by other thread? */
>> +    if (!test_bit(XDBC_ATOMIC_EVENT, &xdbcp->atomic_flags) &&
>> +            !test_and_set_bit(XDBC_ATOMIC_EVENT,
>> +            &xdbcp->atomic_flags))
>> +        return 0;
>
> homemade trylock, can't the real ones be used?
>
>> +
>> +    xdbc_handle_events();
>> +
>> +    test_and_clear_bit(XDBC_ATOMIC_EVENT, &xdbcp->atomic_flags);
>> +
>> +    return 0;
>>  +}
>> +
>> +#define    BULK_IN_COMPLETED(p)    ((xdbcp->in_pending == (p)) && \
>> +                 xdbcp->in_complete)
>> +#define    BULK_OUT_COMPLETED(p)    ((xdbcp->out_pending == (p)) && \
>> +                 xdbcp->out_complete)
>> +
>
> ...
>
>> +}
>> +
>> +int xdbc_bulk_read(void *data, int size, int loops)
>> +{
>> +    int ret;
>> +
>> +    do {
>> +        if (!test_bit(XDBC_ATOMIC_BULKIN, &xdbcp->atomic_flags) &&
>> +                !test_and_set_bit(XDBC_ATOMIC_BULKIN,
>> +                &xdbcp->atomic_flags))
>> +            break;
>> +    } while (1);
>
> homemeade spin_lock, can't the real one be used?
>
> If the xdbc_bulk_write() can be accessed from interrupt context (handler, soft, timer) it
> may deadlock
>
>> +
>> +    ret = xdbc_bulk_transfer(data, size, loops, true);
>> +
>> +    test_and_clear_bit(XDBC_ATOMIC_BULKIN, &xdbcp->atomic_flags);
>> +
>> +    return ret;
>> +}
>> +
>> +int xdbc_bulk_write(const char *bytes, int size)
>> +{
>> +    int ret;
>> +
>> +    do {
>> +        if (!test_bit(XDBC_ATOMIC_BULKOUT, &xdbcp->atomic_flags) &&
>> +                !test_and_set_bit(XDBC_ATOMIC_BULKOUT,
>> +                &xdbcp->atomic_flags))
>> +            break;
>> +    } while (1);
>
> Another homemeade spin_lock, can't the real one be used?
>
> same issue here, deadlock if accessible from interrupt context.

I will try to rework this spin_lock with the real one and keep avoiding deadlock in mind.

>
>
> Would it make sense to have only one spinlock, and start one separate thread for
> reading the event ring. The thread would,  lock, handle pending events, unlock,
> then call shedule, in a loop. ehci early debug code has some variant of this.

Let me try to find this part of code.

>
> So the lock would be taken while events are being handled.
>
> The same lock would be used for bulk_read and bulk_write. Yes this would prevent read and
> write at the same time, and the read and writes need to be modified to not block until
> the reansfer is finished, just to write the TRBs on the ring, update ring pointers,
> and ring the doorbell.
>
> Or is all this impossibe due to the earlyness of the code?

It's not only due to earlyness of the code. But also, these read/write ops were designed to
be used by a debugger (for example kgdb) as well. Using the kernel provided interface
might make things simple, but what should happen when the debugger is used
to debug the kernel subsystem itself?

So, it seems that I should implement read/write ops depends on the use case. For this
time being, let's focus on the boot console case.

Some transfers take place when the thread/lock subsystem is not initialized yet.
But after thread/lock subsystem is able to be used, we are able to use the real one.

Let me wrapper them in functions. For the transfers taken place before the subsystem
initialization (that's single thread context, no worry about deadlock), it will use the
current methods (it might be possible to drop lock due the single thread context),
and after the subsystem being initialized, it will use those provided by the kernel.

>
> -Mathias
>

Very appreciated for your time.

Regards,
-Baolu

Re: [PATCH v7 07/10] usb: dbc: handle endpoint stall

[Mathias Nyman]

On 26.01.2016 14:58, Lu Baolu wrote:
> In case of endpoint stall, software is able to detect the situation
> by reading DCCTRL.HIT or DCCTRL.HOT bits. DbC follows the normal USB
> framework to handle endpoint stall. When software detects endpoint
> stall situation, it should wait until endpoint is recovered before
> read or write oprations.
>
> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
> ---
>   drivers/usb/early/xhci-dbc.c | 36 ++++++++++++++++++++++++++++++++++++
>   1 file changed, 36 insertions(+)
>
> diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
> index c81df40..344d93e 100644
> --- a/drivers/usb/early/xhci-dbc.c
> +++ b/drivers/usb/early/xhci-dbc.c
> @@ -1163,6 +1163,37 @@ static int xdbc_wait_until_dbc_configured(void)
>   	return -ETIMEDOUT;
>   }
>
> +static int xdbc_wait_until_epstall_cleared(bool read)
> +{
> +	int timeout = 0;
> +
> +	if (read) {
> +		do {
> +			if (!(readl(&xdbcp->xdbc_reg->control) & CTRL_HIT)) {
> +				xdbcp->in_ep_state = EP_RUNNING;
> +
> +				return 0;
> +			}
> +
> +			xdbcp->in_ep_state = EP_HALTED;
> +			xdbc_udelay(10);
> +		} while (timeout++ < XDBC_LOOPS);
> +	} else {
> +		do {
> +			if (!(readl(&xdbcp->xdbc_reg->control) & CTRL_HOT)) {
> +				xdbcp->out_ep_state = EP_RUNNING;
> +
> +				return 0;
> +			}
> +
> +			xdbcp->out_ep_state = EP_HALTED;
> +			xdbc_udelay(10);
> +		} while (timeout++ < XDBC_LOOPS);
> +	}
> +
> +	return -ETIMEDOUT;
> +}
>

how about something like:

enum xdbc_ep_state *ep_state;
u32 halt_bit;

if (read) {
	ep_state = &xdbcp->in_ep_state
	halt_bit = CTRL_HIT
} else {
  	ep_state = &xdbcp->out_ep_state
	halt_bit = CTRL_HOT
}
do {
	if (!(readl(..) & halt_bit)) {
		*ep_state = EP_RUNNING;
		return 0;
	}
	*ep_state = EP_HALTED;
	delay()
} while (..)


+
>   static int xdbc_bulk_transfer(void *data, int size, int loops, bool read)
>   {
>   	u64 addr;
> @@ -1182,6 +1213,11 @@ static int xdbc_bulk_transfer(void *data, int size, int loops, bool read)
>   		return -EPERM;
>   	}
>
> +	if (xdbc_wait_until_epstall_cleared(read)) {
> +		xdbc_trace("%s: endpoint not ready\n", __func__);
> +		return -EPERM;

Is -EPERM appropriate here?
Not sure about what error codes the device side is using, but usually HALT is set due to some
Data buffer/transmission/TRB error.
In this case the failure is that debug host failed to send a clearFeature(EP_HALT) request in time.

-Mathias

Re: [PATCH v7 07/10] usb: dbc: handle endpoint stall

[Felipe Balbi]

[-- Attachment #1: Type: text/plain, Size: 3057 bytes --]


Hi,

Mathias Nyman <mathias.nyman@linux.intel.com> writes:
> [ text/plain ]
> On 26.01.2016 14:58, Lu Baolu wrote:
>> In case of endpoint stall, software is able to detect the situation
>> by reading DCCTRL.HIT or DCCTRL.HOT bits. DbC follows the normal USB
>> framework to handle endpoint stall. When software detects endpoint
>> stall situation, it should wait until endpoint is recovered before
>> read or write oprations.
>>
>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>> ---
>>   drivers/usb/early/xhci-dbc.c | 36 ++++++++++++++++++++++++++++++++++++
>>   1 file changed, 36 insertions(+)
>>
>> diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
>> index c81df40..344d93e 100644
>> --- a/drivers/usb/early/xhci-dbc.c
>> +++ b/drivers/usb/early/xhci-dbc.c
>> @@ -1163,6 +1163,37 @@ static int xdbc_wait_until_dbc_configured(void)
>>   	return -ETIMEDOUT;
>>   }
>>
>> +static int xdbc_wait_until_epstall_cleared(bool read)
>> +{
>> +	int timeout = 0;
>> +
>> +	if (read) {
>> +		do {
>> +			if (!(readl(&xdbcp->xdbc_reg->control) & CTRL_HIT)) {
>> +				xdbcp->in_ep_state = EP_RUNNING;
>> +
>> +				return 0;
>> +			}
>> +
>> +			xdbcp->in_ep_state = EP_HALTED;
>> +			xdbc_udelay(10);
>> +		} while (timeout++ < XDBC_LOOPS);
>> +	} else {
>> +		do {
>> +			if (!(readl(&xdbcp->xdbc_reg->control) & CTRL_HOT)) {
>> +				xdbcp->out_ep_state = EP_RUNNING;
>> +
>> +				return 0;
>> +			}
>> +
>> +			xdbcp->out_ep_state = EP_HALTED;
>> +			xdbc_udelay(10);
>> +		} while (timeout++ < XDBC_LOOPS);
>> +	}
>> +
>> +	return -ETIMEDOUT;
>> +}
>>
>
> how about something like:
>
> enum xdbc_ep_state *ep_state;
> u32 halt_bit;
>
> if (read) {
> 	ep_state = &xdbcp->in_ep_state
> 	halt_bit = CTRL_HIT
> } else {
>   	ep_state = &xdbcp->out_ep_state
> 	halt_bit = CTRL_HOT
> }
> do {
> 	if (!(readl(..) & halt_bit)) {
> 		*ep_state = EP_RUNNING;
> 		return 0;
> 	}
> 	*ep_state = EP_HALTED;
> 	delay()
> } while (..)

I'll agree, this looks better. Might also want to refactor the handshake
loop to its own function:

static int xdbg_ep_state_handshake(enum xdbc_ep_state *ep_state, u32 halt_bit)
{
        do {
		if (!(readl(...) & halt_bit)) {
                	...
                }
                *ep_state = EP_HALTED;
                delay(...);
        } while (...)

>> @@ -1182,6 +1213,11 @@ static int xdbc_bulk_transfer(void *data, int size, int loops, bool read)
>>   		return -EPERM;
>>   	}
>>
>> +	if (xdbc_wait_until_epstall_cleared(read)) {
>> +		xdbc_trace("%s: endpoint not ready\n", __func__);
>> +		return -EPERM;
>
> Is -EPERM appropriate here?
>
> Not sure about what error codes the device side is using, but usually
> HALT is set due to some Data buffer/transmission/TRB error.

EIO perhaps ?

> In this case the failure is that debug host failed to send a
> clearFeature(EP_HALT) request in time.

I haven't read the spec, but does it define a maximum time for this to
happen ?

-- 
balbi

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 818 bytes --]

Re: [PATCH v7 07/10] usb: dbc: handle endpoint stall

[Lu Baolu]

On 03/02/2016 08:58 PM, Mathias Nyman wrote:
> On 26.01.2016 14:58, Lu Baolu wrote:
>> In case of endpoint stall, software is able to detect the situation
>> by reading DCCTRL.HIT or DCCTRL.HOT bits. DbC follows the normal USB
>> framework to handle endpoint stall. When software detects endpoint
>> stall situation, it should wait until endpoint is recovered before
>> read or write oprations.
>>
>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>> ---
>>   drivers/usb/early/xhci-dbc.c | 36 ++++++++++++++++++++++++++++++++++++
>>   1 file changed, 36 insertions(+)
>>
>> diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
>> index c81df40..344d93e 100644
>> --- a/drivers/usb/early/xhci-dbc.c
>> +++ b/drivers/usb/early/xhci-dbc.c
>> @@ -1163,6 +1163,37 @@ static int xdbc_wait_until_dbc_configured(void)
>>       return -ETIMEDOUT;
>>   }
>>
>> +static int xdbc_wait_until_epstall_cleared(bool read)
>> +{
>> +    int timeout = 0;
>> +
>> +    if (read) {
>> +        do {
>> +            if (!(readl(&xdbcp->xdbc_reg->control) & CTRL_HIT)) {
>> +                xdbcp->in_ep_state = EP_RUNNING;
>> +
>> +                return 0;
>> +            }
>> +
>> +            xdbcp->in_ep_state = EP_HALTED;
>> +            xdbc_udelay(10);
>> +        } while (timeout++ < XDBC_LOOPS);
>> +    } else {
>> +        do {
>> +            if (!(readl(&xdbcp->xdbc_reg->control) & CTRL_HOT)) {
>> +                xdbcp->out_ep_state = EP_RUNNING;
>> +
>> +                return 0;
>> +            }
>> +
>> +            xdbcp->out_ep_state = EP_HALTED;
>> +            xdbc_udelay(10);
>> +        } while (timeout++ < XDBC_LOOPS);
>> +    }
>> +
>> +    return -ETIMEDOUT;
>> +}
>>
>
> how about something like:
>
> enum xdbc_ep_state *ep_state;
> u32 halt_bit;
>
> if (read) {
>     ep_state = &xdbcp->in_ep_state
>     halt_bit = CTRL_HIT
> } else {
>      ep_state = &xdbcp->out_ep_state
>     halt_bit = CTRL_HOT
> }
> do {
>     if (!(readl(..) & halt_bit)) {
>         *ep_state = EP_RUNNING;
>         return 0;
>     }
>     *ep_state = EP_HALTED;
>     delay()
> } while (..)

Sure. I will refine this part of code.

>
>
> +
>>   static int xdbc_bulk_transfer(void *data, int size, int loops, bool read)
>>   {
>>       u64 addr;
>> @@ -1182,6 +1213,11 @@ static int xdbc_bulk_transfer(void *data, int size, int loops, bool read)
>>           return -EPERM;
>>       }
>>
>> +    if (xdbc_wait_until_epstall_cleared(read)) {
>> +        xdbc_trace("%s: endpoint not ready\n", __func__);
>> +        return -EPERM;
>
> Is -EPERM appropriate here?
> Not sure about what error codes the device side is using, but usually HALT is set due to some
> Data buffer/transmission/TRB error.
> In this case the failure is that debug host failed to send a clearFeature(EP_HALT) request in time.

Yes. This is done by debug host firmware. OS needs to check and wait.

This return value is used to tell the up layer what happened. -EPERM is definitely not appropriate here.
I will use EIO as suggested by Felipe.

>
> -Mathias
>

Thanks,
-Baolu

Re: [PATCH v7 07/10] usb: dbc: handle endpoint stall

[Lu Baolu]

On 03/02/2016 08:58 PM, Felipe Balbi wrote:
> Hi,
>
> Mathias Nyman <mathias.nyman@linux.intel.com> writes:
>> [ text/plain ]
>> On 26.01.2016 14:58, Lu Baolu wrote:
>>> In case of endpoint stall, software is able to detect the situation
>>> by reading DCCTRL.HIT or DCCTRL.HOT bits. DbC follows the normal USB
>>> framework to handle endpoint stall. When software detects endpoint
>>> stall situation, it should wait until endpoint is recovered before
>>> read or write oprations.
>>>
>>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>>> ---
>>>   drivers/usb/early/xhci-dbc.c | 36 ++++++++++++++++++++++++++++++++++++
>>>   1 file changed, 36 insertions(+)
>>>
>>> diff --git a/drivers/usb/early/xhci-dbc.c b/drivers/usb/early/xhci-dbc.c
>>> index c81df40..344d93e 100644
>>> --- a/drivers/usb/early/xhci-dbc.c
>>> +++ b/drivers/usb/early/xhci-dbc.c
>>> @@ -1163,6 +1163,37 @@ static int xdbc_wait_until_dbc_configured(void)
>>>   	return -ETIMEDOUT;
>>>   }
>>>
>>> +static int xdbc_wait_until_epstall_cleared(bool read)
>>> +{
>>> +	int timeout = 0;
>>> +
>>> +	if (read) {
>>> +		do {
>>> +			if (!(readl(&xdbcp->xdbc_reg->control) & CTRL_HIT)) {
>>> +				xdbcp->in_ep_state = EP_RUNNING;
>>> +
>>> +				return 0;
>>> +			}
>>> +
>>> +			xdbcp->in_ep_state = EP_HALTED;
>>> +			xdbc_udelay(10);
>>> +		} while (timeout++ < XDBC_LOOPS);
>>> +	} else {
>>> +		do {
>>> +			if (!(readl(&xdbcp->xdbc_reg->control) & CTRL_HOT)) {
>>> +				xdbcp->out_ep_state = EP_RUNNING;
>>> +
>>> +				return 0;
>>> +			}
>>> +
>>> +			xdbcp->out_ep_state = EP_HALTED;
>>> +			xdbc_udelay(10);
>>> +		} while (timeout++ < XDBC_LOOPS);
>>> +	}
>>> +
>>> +	return -ETIMEDOUT;
>>> +}
>>>
>> how about something like:
>>
>> enum xdbc_ep_state *ep_state;
>> u32 halt_bit;
>>
>> if (read) {
>> 	ep_state = &xdbcp->in_ep_state
>> 	halt_bit = CTRL_HIT
>> } else {
>>   	ep_state = &xdbcp->out_ep_state
>> 	halt_bit = CTRL_HOT
>> }
>> do {
>> 	if (!(readl(..) & halt_bit)) {
>> 		*ep_state = EP_RUNNING;
>> 		return 0;
>> 	}
>> 	*ep_state = EP_HALTED;
>> 	delay()
>> } while (..)
> I'll agree, this looks better. Might also want to refactor the handshake
> loop to its own function:
>
> static int xdbg_ep_state_handshake(enum xdbc_ep_state *ep_state, u32 halt_bit)
> {
>         do {
> 		if (!(readl(...) & halt_bit)) {
>                 	...
>                 }
>                 *ep_state = EP_HALTED;
>                 delay(...);
>         } while (...)

Sure. Thanks.

>
>>> @@ -1182,6 +1213,11 @@ static int xdbc_bulk_transfer(void *data, int size, int loops, bool read)
>>>   		return -EPERM;
>>>   	}
>>>
>>> +	if (xdbc_wait_until_epstall_cleared(read)) {
>>> +		xdbc_trace("%s: endpoint not ready\n", __func__);
>>> +		return -EPERM;
>> Is -EPERM appropriate here?
>>
>> Not sure about what error codes the device side is using, but usually
>> HALT is set due to some Data buffer/transmission/TRB error.
> EIO perhaps ?

Sure. I will replace it with -EIO.

>
>> In this case the failure is that debug host failed to send a
>> clearFeature(EP_HALT) request in time.
> I haven't read the spec, but does it define a maximum time for this to
> happen ?

I took a quick look at chapter 9.4 USB3 spec. It only requires that
"ClearFeature (ENDPOINT_HALT) request results in the endpoint
no longer returning a STALL Transaction Packet". xHCI spec doesn't
define a maximum time for this either.

Thanks,
-Baolu

>