linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
@ 2016-11-09 14:20 Amir Levy
  2016-11-09 14:20 ` [PATCH v9 1/8] thunderbolt: Macro rename Amir Levy
                   ` (9 more replies)
  0 siblings, 10 replies; 28+ messages in thread
From: Amir Levy @ 2016-11-09 14:20 UTC (permalink / raw)
  To: gregkh
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang, Amir Levy

This driver enables Thunderbolt Networking on non-Apple platforms
running Linux.

Thunderbolt Networking provides peer-to-peer connections to transfer
files between computers, perform PC migrations, and/or set up small
workgroups with shared storage.

This is a virtual connection that emulates an Ethernet adapter that
enables Ethernet networking with the benefit of Thunderbolt superfast
medium capability.

Thunderbolt Networking enables two hosts and several devices that
have a Thunderbolt controller to be connected together in a linear
(Daisy chain) series from a single port.

Thunderbolt Networking for Linux is compatible with Thunderbolt
Networking on systems running macOS or Windows and also supports
Thunderbolt generation 2 and 3 controllers.

Note that all pre-existing Thunderbolt generation 3 features, such as
USB, Display and other Thunderbolt device connectivity will continue
to function exactly as they did prior to enabling Thunderbolt Networking.

Code and Software Specifications:
This kernel code creates a virtual ethernet device for computer to
computer communication over a Thunderbolt cable.
The new driver is a separate driver to the existing Thunderbolt driver.
It is designed to work on systems running Linux that
interface with Intel Connection Manager (ICM) firmware based
Thunderbolt controllers that support Thunderbolt Networking.
The kernel code operates in coordination with the Thunderbolt user-
space daemon to implement full Thunderbolt networking functionality.

Hardware Specifications:
Thunderbolt Hardware specs have not yet been published but are used
where necessary for register definitions. 

Acked-by: Andreas Noever <andreas.noever@gmail.com>
Tested-by: Mario Limonciello <mario.limonciello@dell.com>

Changes since v8:
 - Added support for more Thunderbolt device IDs

These patches were pushed to GitHub where they can be reviewed more
comfortably with green/red highlighting:
	https://github.com/01org/thunderbolt-software-kernel-tree

Daemon code:
	https://github.com/01org/thunderbolt-software-daemon

For reference, here's a link to version 8:
[v8]:	https://lkml.org/lkml/2016/9/28/378

Amir Levy (8):
  thunderbolt: Macro rename
  thunderbolt: Updating the register definitions
  thunderbolt: Communication with the ICM (firmware)
  thunderbolt: Networking state machine
  thunderbolt: Networking transmit and receive
  thunderbolt: Kconfig for Thunderbolt Networking
  thunderbolt: Networking doc
  thunderbolt: Adding maintainer entry

 Documentation/00-INDEX                   |    2 +
 Documentation/thunderbolt/networking.txt |  132 ++
 MAINTAINERS                              |    8 +-
 drivers/thunderbolt/Kconfig              |   27 +-
 drivers/thunderbolt/Makefile             |    3 +-
 drivers/thunderbolt/icm/Makefile         |    2 +
 drivers/thunderbolt/icm/icm_nhi.c        | 1520 ++++++++++++++++++++
 drivers/thunderbolt/icm/icm_nhi.h        |   85 ++
 drivers/thunderbolt/icm/net.c            | 2254 ++++++++++++++++++++++++++++++
 drivers/thunderbolt/icm/net.h            |  287 ++++
 drivers/thunderbolt/nhi_regs.h           |  115 +-
 11 files changed, 4426 insertions(+), 9 deletions(-)
 create mode 100644 Documentation/thunderbolt/networking.txt
 create mode 100644 drivers/thunderbolt/icm/Makefile
 create mode 100644 drivers/thunderbolt/icm/icm_nhi.c
 create mode 100644 drivers/thunderbolt/icm/icm_nhi.h
 create mode 100644 drivers/thunderbolt/icm/net.c
 create mode 100644 drivers/thunderbolt/icm/net.h

-- 
2.7.4

^ permalink raw reply	[flat|nested] 28+ messages in thread

* [PATCH v9 1/8] thunderbolt: Macro rename
  2016-11-09 14:20 [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Amir Levy
@ 2016-11-09 14:20 ` Amir Levy
  2016-11-09 14:20 ` [PATCH v9 2/8] thunderbolt: Updating the register definitions Amir Levy
                   ` (8 subsequent siblings)
  9 siblings, 0 replies; 28+ messages in thread
From: Amir Levy @ 2016-11-09 14:20 UTC (permalink / raw)
  To: gregkh
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang, Amir Levy

This first patch updates the NHI Thunderbolt controller registers file to
reflect that it is not only for Cactus Ridge.
No functional change intended.

Signed-off-by: Amir Levy <amir.jer.levy@intel.com>
Signed-off-by: Andreas Noever <andreas.noever@gmail.com>
---
 drivers/thunderbolt/nhi_regs.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/thunderbolt/nhi_regs.h b/drivers/thunderbolt/nhi_regs.h
index 86b996c..75cf069 100644
--- a/drivers/thunderbolt/nhi_regs.h
+++ b/drivers/thunderbolt/nhi_regs.h
@@ -1,11 +1,11 @@
 /*
- * Thunderbolt Cactus Ridge driver - NHI registers
+ * Thunderbolt driver - NHI registers
  *
  * Copyright (c) 2014 Andreas Noever <andreas.noever@gmail.com>
  */
 
-#ifndef DSL3510_REGS_H_
-#define DSL3510_REGS_H_
+#ifndef NHI_REGS_H_
+#define NHI_REGS_H_
 
 #include <linux/types.h>
 
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v9 2/8] thunderbolt: Updating the register definitions
  2016-11-09 14:20 [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Amir Levy
  2016-11-09 14:20 ` [PATCH v9 1/8] thunderbolt: Macro rename Amir Levy
@ 2016-11-09 14:20 ` Amir Levy
  2016-11-09 14:20 ` [PATCH v9 3/8] thunderbolt: Communication with the ICM (firmware) Amir Levy
                   ` (7 subsequent siblings)
  9 siblings, 0 replies; 28+ messages in thread
From: Amir Levy @ 2016-11-09 14:20 UTC (permalink / raw)
  To: gregkh
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang, Amir Levy

Adding more Thunderbolt(TM) register definitions
and some helper macros.

Signed-off-by: Amir Levy <amir.jer.levy@intel.com>
---
 drivers/thunderbolt/nhi_regs.h | 109 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 109 insertions(+)

diff --git a/drivers/thunderbolt/nhi_regs.h b/drivers/thunderbolt/nhi_regs.h
index 75cf069..b8e961f 100644
--- a/drivers/thunderbolt/nhi_regs.h
+++ b/drivers/thunderbolt/nhi_regs.h
@@ -9,6 +9,11 @@
 
 #include <linux/types.h>
 
+#define NHI_MMIO_BAR 0
+
+#define TBT_RING_MIN_NUM_BUFFERS	2
+#define TBT_RING_MAX_FRAME_SIZE		(4 * 1024)
+
 enum ring_flags {
 	RING_FLAG_ISOCH_ENABLE = 1 << 27, /* TX only? */
 	RING_FLAG_E2E_FLOW_CONTROL = 1 << 28,
@@ -39,6 +44,33 @@ struct ring_desc {
 	u32 time; /* write zero */
 } __packed;
 
+/**
+ * struct tbt_buf_desc - TX/RX ring buffer descriptor.
+ * This is same as struct ring_desc, but without the use of bitfields and
+ * with explicit endianity.
+ */
+struct tbt_buf_desc {
+	__le64 phys;
+	__le32 attributes;
+	__le32 time;
+};
+
+#define DESC_ATTR_LEN_SHIFT		0
+#define DESC_ATTR_LEN_MASK		GENMASK(11, DESC_ATTR_LEN_SHIFT)
+#define DESC_ATTR_EOF_SHIFT		12
+#define DESC_ATTR_EOF_MASK		GENMASK(15, DESC_ATTR_EOF_SHIFT)
+#define DESC_ATTR_SOF_SHIFT		16
+#define DESC_ATTR_SOF_MASK		GENMASK(19, DESC_ATTR_SOF_SHIFT)
+#define DESC_ATTR_TX_ISOCH_DMA_EN	BIT(20)	/* TX */
+#define DESC_ATTR_RX_CRC_ERR		BIT(20)	/* RX after use */
+#define DESC_ATTR_DESC_DONE		BIT(21)
+#define DESC_ATTR_REQ_STS		BIT(22)	/* TX and RX before use */
+#define DESC_ATTR_RX_BUF_OVRN_ERR	BIT(22)	/* RX after use */
+#define DESC_ATTR_INT_EN		BIT(23)
+#define DESC_ATTR_OFFSET_SHIFT		24
+#define DESC_ATTR_OFFSET_MASK		GENMASK(31, DESC_ATTR_OFFSET_SHIFT)
+
+
 /* NHI registers in bar 0 */
 
 /*
@@ -60,6 +92,30 @@ struct ring_desc {
  */
 #define REG_RX_RING_BASE	0x08000
 
+#define REG_RING_STEP			16
+#define REG_RING_PHYS_LO_OFFSET		0
+#define REG_RING_PHYS_HI_OFFSET		4
+#define REG_RING_CONS_PROD_OFFSET	8	/* cons - RO, prod - RW */
+#define REG_RING_CONS_SHIFT		0
+#define REG_RING_CONS_MASK		GENMASK(15, REG_RING_CONS_SHIFT)
+#define REG_RING_PROD_SHIFT		16
+#define REG_RING_PROD_MASK		GENMASK(31, REG_RING_PROD_SHIFT)
+#define REG_RING_SIZE_OFFSET		12
+#define REG_RING_SIZE_SHIFT		0
+#define REG_RING_SIZE_MASK		GENMASK(15, REG_RING_SIZE_SHIFT)
+#define REG_RING_BUF_SIZE_SHIFT		16
+#define REG_RING_BUF_SIZE_MASK		GENMASK(27, REG_RING_BUF_SIZE_SHIFT)
+
+#define TBT_RING_CONS_PROD_REG(iobase, ringbase, ringnumber) \
+			      ((iobase) + (ringbase) + \
+			      ((ringnumber) * REG_RING_STEP) + \
+			      REG_RING_CONS_PROD_OFFSET)
+
+#define TBT_REG_RING_PROD_EXTRACT(val) (((val) & REG_RING_PROD_MASK) >> \
+				       REG_RING_PROD_SHIFT)
+
+#define TBT_REG_RING_CONS_EXTRACT(val) (((val) & REG_RING_CONS_MASK) >> \
+				       REG_RING_CONS_SHIFT)
 /*
  * 32 bytes per entry, one entry for every hop (REG_HOP_COUNT)
  * 00: enum_ring_flags
@@ -77,6 +133,19 @@ struct ring_desc {
  * ..: unknown
  */
 #define REG_RX_OPTIONS_BASE	0x29800
+#define REG_RX_OPTS_TX_E2E_HOP_ID_SHIFT	12
+#define REG_RX_OPTS_TX_E2E_HOP_ID_MASK	\
+				GENMASK(22, REG_RX_OPTS_TX_E2E_HOP_ID_SHIFT)
+#define REG_RX_OPTS_MASK_OFFSET		4
+#define REG_RX_OPTS_MASK_EOF_SHIFT	0
+#define REG_RX_OPTS_MASK_EOF_MASK	GENMASK(15, REG_RX_OPTS_MASK_EOF_SHIFT)
+#define REG_RX_OPTS_MASK_SOF_SHIFT	16
+#define REG_RX_OPTS_MASK_SOF_MASK	GENMASK(31, REG_RX_OPTS_MASK_SOF_SHIFT)
+
+#define REG_OPTS_STEP			32
+#define REG_OPTS_E2E_EN			BIT(28)
+#define REG_OPTS_RAW			BIT(30)
+#define REG_OPTS_VALID			BIT(31)
 
 /*
  * three bitfields: tx, rx, rx overflow
@@ -86,6 +155,7 @@ struct ring_desc {
  */
 #define REG_RING_NOTIFY_BASE	0x37800
 #define RING_NOTIFY_REG_COUNT(nhi) ((31 + 3 * nhi->hop_count) / 32)
+#define REG_RING_NOTIFY_STEP	4
 
 /*
  * two bitfields: rx, tx
@@ -94,8 +164,47 @@ struct ring_desc {
  */
 #define REG_RING_INTERRUPT_BASE	0x38200
 #define RING_INTERRUPT_REG_COUNT(nhi) ((31 + 2 * nhi->hop_count) / 32)
+#define REG_RING_INT_TX_PROCESSED(ring_num)		BIT(ring_num)
+#define REG_RING_INT_RX_PROCESSED(ring_num, num_paths)	BIT((ring_num) + \
+							    (num_paths))
+#define RING_INT_DISABLE(base, val) iowrite32( \
+			ioread32((base) + REG_RING_INTERRUPT_BASE) & ~(val), \
+			(base) + REG_RING_INTERRUPT_BASE)
+#define RING_INT_ENABLE(base, val) iowrite32( \
+			ioread32((base) + REG_RING_INTERRUPT_BASE) | (val), \
+			(base) + REG_RING_INTERRUPT_BASE)
+#define RING_INT_DISABLE_TX(base, ring_num) \
+	RING_INT_DISABLE(base, REG_RING_INT_TX_PROCESSED(ring_num))
+#define RING_INT_DISABLE_RX(base, ring_num, num_paths) \
+	RING_INT_DISABLE(base, REG_RING_INT_RX_PROCESSED(ring_num, num_paths))
+#define RING_INT_ENABLE_TX(base, ring_num) \
+	RING_INT_ENABLE(base, REG_RING_INT_TX_PROCESSED(ring_num))
+#define RING_INT_ENABLE_RX(base, ring_num, num_paths) \
+	RING_INT_ENABLE(base, REG_RING_INT_RX_PROCESSED(ring_num, num_paths))
+#define RING_INT_DISABLE_TX_RX(base, ring_num, num_paths) \
+	RING_INT_DISABLE(base, REG_RING_INT_TX_PROCESSED(ring_num) | \
+			       REG_RING_INT_RX_PROCESSED(ring_num, num_paths))
+
+#define REG_RING_INTERRUPT_STEP	4
+
+#define REG_INT_THROTTLING_RATE	0x38c00
+#define REG_INT_THROTTLING_RATE_STEP	4
+#define NUM_INT_VECTORS			16
+
+#define REG_INT_VEC_ALLOC_BASE	0x38c40
+#define REG_INT_VEC_ALLOC_STEP		4
+#define REG_INT_VEC_ALLOC_FIELD_BITS	4
+#define REG_INT_VEC_ALLOC_FIELD_MASK	(BIT(REG_INT_VEC_ALLOC_FIELD_BITS) - 1)
+#define REG_INT_VEC_ALLOC_PER_REG	((BITS_PER_BYTE * sizeof(u32)) / \
+					 REG_INT_VEC_ALLOC_FIELD_BITS)
 
 /* The last 11 bits contain the number of hops supported by the NHI port. */
 #define REG_HOP_COUNT		0x39640
+#define REG_HOP_COUNT_TOTAL_PATHS_MASK	GENMASK(10, 0)
+
+#define REG_HOST_INTERFACE_RST	0x39858
+
+#define REG_DMA_MISC		0x39864
+#define REG_DMA_MISC_INT_AUTO_CLEAR	BIT(2)
 
 #endif
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v9 3/8] thunderbolt: Communication with the ICM (firmware)
  2016-11-09 14:20 [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Amir Levy
  2016-11-09 14:20 ` [PATCH v9 1/8] thunderbolt: Macro rename Amir Levy
  2016-11-09 14:20 ` [PATCH v9 2/8] thunderbolt: Updating the register definitions Amir Levy
@ 2016-11-09 14:20 ` Amir Levy
  2016-11-09 14:20 ` [PATCH v9 4/8] thunderbolt: Networking state machine Amir Levy
                   ` (6 subsequent siblings)
  9 siblings, 0 replies; 28+ messages in thread
From: Amir Levy @ 2016-11-09 14:20 UTC (permalink / raw)
  To: gregkh
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang, Amir Levy

This patch provides the communication protocol between the
Intel Connection Manager(ICM) firmware that is operational in the
Thunderbolt controller in non-Apple hardware.
The ICM firmware-based controller is used for establishing and maintaining
the Thunderbolt Networking connection - we need to be able to communicate
with it.

Signed-off-by: Amir Levy <amir.jer.levy@intel.com>
---
 drivers/thunderbolt/icm/Makefile  |    2 +
 drivers/thunderbolt/icm/icm_nhi.c | 1257 +++++++++++++++++++++++++++++++++++++
 drivers/thunderbolt/icm/icm_nhi.h |   85 +++
 drivers/thunderbolt/icm/net.h     |  217 +++++++
 4 files changed, 1561 insertions(+)
 create mode 100644 drivers/thunderbolt/icm/Makefile
 create mode 100644 drivers/thunderbolt/icm/icm_nhi.c
 create mode 100644 drivers/thunderbolt/icm/icm_nhi.h
 create mode 100644 drivers/thunderbolt/icm/net.h

diff --git a/drivers/thunderbolt/icm/Makefile b/drivers/thunderbolt/icm/Makefile
new file mode 100644
index 0000000..f0d0fbb
--- /dev/null
+++ b/drivers/thunderbolt/icm/Makefile
@@ -0,0 +1,2 @@
+obj-${CONFIG_THUNDERBOLT_ICM} += thunderbolt-icm.o
+thunderbolt-icm-objs := icm_nhi.o
diff --git a/drivers/thunderbolt/icm/icm_nhi.c b/drivers/thunderbolt/icm/icm_nhi.c
new file mode 100644
index 0000000..c843ce8
--- /dev/null
+++ b/drivers/thunderbolt/icm/icm_nhi.c
@@ -0,0 +1,1257 @@
+/*******************************************************************************
+ *
+ * Intel Thunderbolt(TM) driver
+ * Copyright(c) 2014 - 2016 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ ******************************************************************************/
+
+#include <linux/printk.h>
+#include <linux/crc32.h>
+#include <linux/delay.h>
+#include <linux/dmi.h>
+#include "icm_nhi.h"
+#include "net.h"
+
+#define NHI_GENL_VERSION 1
+#define NHI_GENL_NAME "thunderbolt"
+
+#define DEVICE_DATA(num_ports, dma_port, nvm_ver_offset, nvm_auth_on_boot,\
+		    support_full_e2e) \
+	((num_ports) | ((dma_port) << 4) | ((nvm_ver_offset) << 10) | \
+	 ((nvm_auth_on_boot) << 22) | ((support_full_e2e) << 23))
+#define DEVICE_DATA_NUM_PORTS(device_data) ((device_data) & 0xf)
+#define DEVICE_DATA_DMA_PORT(device_data) (((device_data) >> 4) & 0x3f)
+#define DEVICE_DATA_NVM_VER_OFFSET(device_data) (((device_data) >> 10) & 0xfff)
+#define DEVICE_DATA_NVM_AUTH_ON_BOOT(device_data) (((device_data) >> 22) & 0x1)
+#define DEVICE_DATA_SUPPORT_FULL_E2E(device_data) (((device_data) >> 23) & 0x1)
+
+#define USEC_TO_256_NSECS(usec) DIV_ROUND_UP((usec) * NSEC_PER_USEC, 256)
+
+/* NHI genetlink commands */
+enum {
+	NHI_CMD_UNSPEC,
+	NHI_CMD_SUBSCRIBE,
+	NHI_CMD_UNSUBSCRIBE,
+	NHI_CMD_QUERY_INFORMATION,
+	NHI_CMD_MSG_TO_ICM,
+	NHI_CMD_MSG_FROM_ICM,
+	NHI_CMD_MAILBOX,
+	NHI_CMD_APPROVE_TBT_NETWORKING,
+	NHI_CMD_ICM_IN_SAFE_MODE,
+	__NHI_CMD_MAX,
+};
+#define NHI_CMD_MAX (__NHI_CMD_MAX - 1)
+
+/* NHI genetlink policy */
+static const struct nla_policy nhi_genl_policy[NHI_ATTR_MAX + 1] = {
+	[NHI_ATTR_DRV_VERSION]		= { .type = NLA_NUL_STRING, },
+	[NHI_ATTR_NVM_VER_OFFSET]	= { .type = NLA_U16, },
+	[NHI_ATTR_NUM_PORTS]		= { .type = NLA_U8, },
+	[NHI_ATTR_DMA_PORT]		= { .type = NLA_U8, },
+	[NHI_ATTR_SUPPORT_FULL_E2E]	= { .type = NLA_FLAG, },
+	[NHI_ATTR_MAILBOX_CMD]		= { .type = NLA_U32, },
+	[NHI_ATTR_PDF]			= { .type = NLA_U32, },
+	[NHI_ATTR_MSG_TO_ICM]		= { .type = NLA_BINARY,
+					.len = TBT_ICM_RING_MAX_FRAME_SIZE },
+	[NHI_ATTR_MSG_FROM_ICM]		= { .type = NLA_BINARY,
+					.len = TBT_ICM_RING_MAX_FRAME_SIZE },
+};
+
+/* NHI genetlink family */
+static struct genl_family nhi_genl_family = {
+	.id		= GENL_ID_GENERATE,
+	.hdrsize	= FIELD_SIZEOF(struct tbt_nhi_ctxt, id),
+	.name		= NHI_GENL_NAME,
+	.version	= NHI_GENL_VERSION,
+	.maxattr	= NHI_ATTR_MAX,
+};
+
+static LIST_HEAD(controllers_list);
+static DEFINE_MUTEX(controllers_list_mutex);
+static atomic_t subscribers = ATOMIC_INIT(0);
+/*
+ * Some of the received generic netlink messages are replied in a different
+ * context. The reply has to include the netlink portid of sender, therefore
+ * saving it in global variable (current assuption is one sender).
+ */
+static u32 portid;
+
+static bool nhi_nvm_authenticated(struct tbt_nhi_ctxt *nhi_ctxt)
+{
+	enum icm_operation_mode op_mode;
+	u32 *msg_head, port_id, reg;
+	struct sk_buff *skb;
+	int i;
+
+	if (!nhi_ctxt->nvm_auth_on_boot)
+		return true;
+
+	/*
+	 * The check for NVM authentication can take time for iCM,
+	 * especially in low power configuration.
+	 */
+	for (i = 0; i < 5; i++) {
+		u32 status = ioread32(nhi_ctxt->iobase + REG_FW_STS);
+
+		if (status & REG_FW_STS_NVM_AUTH_DONE)
+			break;
+
+		msleep(30);
+	}
+	/*
+	 * The check for authentication is done after checking if iCM
+	 * is present so it shouldn't reach the max tries (=5).
+	 * Anyway, the check for full functionality below covers the error case.
+	 */
+	reg = ioread32(nhi_ctxt->iobase + REG_OUTMAIL_CMD);
+	op_mode = (reg & REG_OUTMAIL_CMD_OP_MODE_MASK) >>
+		  REG_OUTMAIL_CMD_OP_MODE_SHIFT;
+	if (op_mode == FULL_FUNCTIONALITY)
+		return true;
+
+	dev_warn(&nhi_ctxt->pdev->dev, "controller id %#x is in operation mode %#x status %#lx, NVM image update might be required\n",
+		 nhi_ctxt->id, op_mode,
+		 (reg & REG_OUTMAIL_CMD_STS_MASK)>>REG_OUTMAIL_CMD_STS_SHIFT);
+
+	skb = genlmsg_new(NLMSG_ALIGN(nhi_genl_family.hdrsize), GFP_KERNEL);
+	if (!skb) {
+		dev_err(&nhi_ctxt->pdev->dev, "genlmsg_new failed: not enough memory to send controller operational mode\n");
+		return false;
+	}
+
+	/* keeping port_id into a local variable for next use */
+	port_id = portid;
+	msg_head = genlmsg_put(skb, port_id, 0, &nhi_genl_family, 0,
+			       NHI_CMD_ICM_IN_SAFE_MODE);
+	if (!msg_head) {
+		nlmsg_free(skb);
+		dev_err(&nhi_ctxt->pdev->dev, "genlmsg_put failed: not enough memory to send controller operational mode\n");
+		return false;
+	}
+
+	*msg_head = nhi_ctxt->id;
+
+	genlmsg_end(skb, msg_head);
+
+	genlmsg_unicast(&init_net, skb, port_id);
+
+	return false;
+}
+
+int nhi_send_message(struct tbt_nhi_ctxt *nhi_ctxt, enum pdf_value pdf,
+		     u32 msg_len, const void *msg, bool ignore_icm_resp)
+{
+	u32 prod_cons, prod, cons, attr;
+	struct tbt_icm_ring_shared_memory *shared_mem;
+	void __iomem *reg = TBT_RING_CONS_PROD_REG(nhi_ctxt->iobase,
+						   REG_TX_RING_BASE,
+						   TBT_ICM_RING_NUM);
+
+	if (nhi_ctxt->d0_exit)
+		return -ENODEV;
+
+	prod_cons = ioread32(reg);
+	prod = TBT_REG_RING_PROD_EXTRACT(prod_cons);
+	cons = TBT_REG_RING_CONS_EXTRACT(prod_cons);
+	if (prod >= TBT_ICM_RING_NUM_TX_BUFS) {
+		dev_warn(&nhi_ctxt->pdev->dev,
+			 "controller id %#x is not functional, producer %u out of range\n",
+			 nhi_ctxt->id, prod);
+		return -ENODEV;
+	}
+	if (TBT_TX_RING_FULL(prod, cons, TBT_ICM_RING_NUM_TX_BUFS)) {
+		dev_err(&nhi_ctxt->pdev->dev,
+			"controller id %#x is not functional, TX ring full\n",
+			nhi_ctxt->id);
+		return -ENOSPC;
+	}
+
+	attr = (msg_len << DESC_ATTR_LEN_SHIFT) & DESC_ATTR_LEN_MASK;
+	attr |= (pdf << DESC_ATTR_EOF_SHIFT) & DESC_ATTR_EOF_MASK;
+
+	shared_mem = nhi_ctxt->icm_ring_shared_mem;
+	shared_mem->tx_buf_desc[prod].attributes = cpu_to_le32(attr);
+
+	memcpy(shared_mem->tx_buf[prod], msg, msg_len);
+
+	prod_cons &= ~REG_RING_PROD_MASK;
+	prod_cons |= (((prod + 1) % TBT_ICM_RING_NUM_TX_BUFS) <<
+		      REG_RING_PROD_SHIFT) & REG_RING_PROD_MASK;
+
+	nhi_ctxt->wait_for_icm_resp = true;
+	nhi_ctxt->ignore_icm_resp = ignore_icm_resp;
+
+	iowrite32(prod_cons, reg);
+
+	return 0;
+}
+
+static int nhi_send_driver_ready_command(struct tbt_nhi_ctxt *nhi_ctxt)
+{
+	struct driver_ready_command {
+		__be32 req_code;
+		__be32 crc;
+	} drv_rdy_cmd = {
+		.req_code = cpu_to_be32(CC_DRV_READY),
+	};
+	u32 crc32;
+
+	crc32 = __crc32c_le(~0, (unsigned char const *)&drv_rdy_cmd,
+			    offsetof(struct driver_ready_command, crc));
+
+	drv_rdy_cmd.crc = cpu_to_be32(~crc32);
+
+	return nhi_send_message(nhi_ctxt, PDF_SW_TO_FW_COMMAND,
+				sizeof(drv_rdy_cmd), &drv_rdy_cmd, false);
+}
+
+/**
+ * nhi_search_ctxt - search by id the controllers_list.
+ * Should be called under controllers_list_mutex.
+ *
+ * @id: id of the controller
+ *
+ * Return: driver context if found, NULL otherwise.
+ */
+static struct tbt_nhi_ctxt *nhi_search_ctxt(u32 id)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt;
+
+	list_for_each_entry(nhi_ctxt, &controllers_list, node)
+		if (nhi_ctxt->id == id)
+			return nhi_ctxt;
+
+	return NULL;
+}
+
+static int nhi_genl_subscribe(__always_unused struct sk_buff *u_skb,
+			      struct genl_info *info)
+			      __acquires(&nhi_ctxt->send_sem)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt;
+
+	/*
+	 * To send driver ready command to iCM, need at least one subscriber
+	 * that will handle the response.
+	 * Currently the assumption is one user mode daemon as subscriber
+	 * so one portid global variable (without locking).
+	 */
+	if (atomic_inc_return(&subscribers) >= 1) {
+		portid = info->snd_portid;
+		if (mutex_lock_interruptible(&controllers_list_mutex)) {
+			atomic_dec_if_positive(&subscribers);
+			return -ERESTART;
+		}
+		list_for_each_entry(nhi_ctxt, &controllers_list, node) {
+			int res;
+
+			if (nhi_ctxt->d0_exit ||
+			    !nhi_nvm_authenticated(nhi_ctxt))
+				continue;
+
+			res = down_timeout(&nhi_ctxt->send_sem,
+					   msecs_to_jiffies(10*MSEC_PER_SEC));
+			if (res) {
+				dev_err(&nhi_ctxt->pdev->dev,
+					"%s: controller id %#x is not functional, timeout on waiting for FW response to previous message\n",
+					__func__, nhi_ctxt->id);
+				continue;
+			}
+
+			if (!mutex_trylock(&nhi_ctxt->d0_exit_send_mutex)) {
+				up(&nhi_ctxt->send_sem);
+				continue;
+			}
+
+			res = nhi_send_driver_ready_command(nhi_ctxt);
+
+			mutex_unlock(&nhi_ctxt->d0_exit_send_mutex);
+			if (res)
+				up(&nhi_ctxt->send_sem);
+		}
+		mutex_unlock(&controllers_list_mutex);
+	}
+
+	return 0;
+}
+
+static int nhi_genl_unsubscribe(__always_unused struct sk_buff *u_skb,
+				__always_unused struct genl_info *info)
+{
+	atomic_dec_if_positive(&subscribers);
+
+	return 0;
+}
+
+static int nhi_genl_query_information(__always_unused struct sk_buff *u_skb,
+				      struct genl_info *info)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt;
+	struct sk_buff *skb;
+	bool msg_too_long;
+	int res = -ENODEV;
+	u32 *msg_head;
+
+	if (!info || !info->userhdr)
+		return -EINVAL;
+
+	skb = genlmsg_new(NLMSG_ALIGN(nhi_genl_family.hdrsize) +
+			  nla_total_size(sizeof(DRV_VERSION)) +
+			  nla_total_size(sizeof(nhi_ctxt->nvm_ver_offset)) +
+			  nla_total_size(sizeof(nhi_ctxt->num_ports)) +
+			  nla_total_size(sizeof(nhi_ctxt->dma_port)) +
+			  nla_total_size(0),	/* nhi_ctxt->support_full_e2e */
+			  GFP_KERNEL);
+	if (!skb)
+		return -ENOMEM;
+
+	msg_head = genlmsg_put_reply(skb, info, &nhi_genl_family, 0,
+				     NHI_CMD_QUERY_INFORMATION);
+	if (!msg_head) {
+		res = -ENOMEM;
+		goto genl_put_reply_failure;
+	}
+
+	if (mutex_lock_interruptible(&controllers_list_mutex)) {
+		res = -ERESTART;
+		goto genl_put_reply_failure;
+	}
+
+	nhi_ctxt = nhi_search_ctxt(*(u32 *)info->userhdr);
+	if (nhi_ctxt && !nhi_ctxt->d0_exit) {
+		*msg_head = nhi_ctxt->id;
+
+		msg_too_long = !!nla_put_string(skb, NHI_ATTR_DRV_VERSION,
+						DRV_VERSION);
+
+		msg_too_long = msg_too_long ||
+			       nla_put_u16(skb, NHI_ATTR_NVM_VER_OFFSET,
+					   nhi_ctxt->nvm_ver_offset);
+
+		msg_too_long = msg_too_long ||
+			       nla_put_u8(skb, NHI_ATTR_NUM_PORTS,
+					  nhi_ctxt->num_ports);
+
+		msg_too_long = msg_too_long ||
+			       nla_put_u8(skb, NHI_ATTR_DMA_PORT,
+					  nhi_ctxt->dma_port);
+
+		if (msg_too_long) {
+			res = -EMSGSIZE;
+			goto release_ctl_list_lock;
+		}
+
+		if (nhi_ctxt->support_full_e2e &&
+		    nla_put_flag(skb, NHI_ATTR_SUPPORT_FULL_E2E)) {
+			res = -EMSGSIZE;
+			goto release_ctl_list_lock;
+		}
+		mutex_unlock(&controllers_list_mutex);
+
+		genlmsg_end(skb, msg_head);
+
+		return genlmsg_reply(skb, info);
+	}
+
+release_ctl_list_lock:
+	mutex_unlock(&controllers_list_mutex);
+	genlmsg_cancel(skb, msg_head);
+
+genl_put_reply_failure:
+	nlmsg_free(skb);
+
+	return res;
+}
+
+static int nhi_genl_msg_to_icm(__always_unused struct sk_buff *u_skb,
+			       struct genl_info *info)
+			       __acquires(&nhi_ctxt->send_sem)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt;
+	int res = -ENODEV;
+	int msg_len;
+	void *msg;
+
+	if (!info || !info->userhdr || !info->attrs ||
+	    !info->attrs[NHI_ATTR_PDF] || !info->attrs[NHI_ATTR_MSG_TO_ICM])
+		return -EINVAL;
+
+	msg_len = nla_len(info->attrs[NHI_ATTR_MSG_TO_ICM]);
+	if (msg_len > TBT_ICM_RING_MAX_FRAME_SIZE)
+		return -ENOBUFS;
+
+	msg = nla_data(info->attrs[NHI_ATTR_MSG_TO_ICM]);
+
+	if (mutex_lock_interruptible(&controllers_list_mutex))
+		return -ERESTART;
+
+	nhi_ctxt = nhi_search_ctxt(*(u32 *)info->userhdr);
+	if (nhi_ctxt && !nhi_ctxt->d0_exit) {
+		/*
+		 * waiting 10 seconds to receive a FW response
+		 * if not, just give up and pop up an error
+		 */
+		res = down_timeout(&nhi_ctxt->send_sem,
+				   msecs_to_jiffies(10 * MSEC_PER_SEC));
+		if (res) {
+			void __iomem *rx_prod_cons = TBT_RING_CONS_PROD_REG(
+							nhi_ctxt->iobase,
+							REG_RX_RING_BASE,
+							TBT_ICM_RING_NUM);
+			void __iomem *tx_prod_cons = TBT_RING_CONS_PROD_REG(
+							nhi_ctxt->iobase,
+							REG_TX_RING_BASE,
+							TBT_ICM_RING_NUM);
+			dev_err(&nhi_ctxt->pdev->dev,
+				"%s: controller id %#x is not functional, timeout on waiting for FW response to previous message, tx prod&cons=%#x, rx prod&cons=%#x\n",
+				__func__, nhi_ctxt->id, ioread32(tx_prod_cons),
+				ioread32(rx_prod_cons));
+			goto release_ctl_list_lock;
+		}
+
+		if (!mutex_trylock(&nhi_ctxt->d0_exit_send_mutex)) {
+			up(&nhi_ctxt->send_sem);
+			goto release_ctl_list_lock;
+		}
+
+		mutex_unlock(&controllers_list_mutex);
+
+		res = nhi_send_message(nhi_ctxt,
+				       nla_get_u32(info->attrs[NHI_ATTR_PDF]),
+				       msg_len, msg, false);
+
+		mutex_unlock(&nhi_ctxt->d0_exit_send_mutex);
+		if (res)
+			up(&nhi_ctxt->send_sem);
+
+		return res;
+	}
+
+release_ctl_list_lock:
+	mutex_unlock(&controllers_list_mutex);
+	return res;
+}
+
+int nhi_mailbox(struct tbt_nhi_ctxt *nhi_ctxt, u32 cmd, u32 data, bool deinit)
+{
+	u32 delay = deinit ? U32_C(20) : U32_C(100);
+	int i;
+
+	iowrite32(data, nhi_ctxt->iobase + REG_INMAIL_DATA);
+	iowrite32(cmd, nhi_ctxt->iobase + REG_INMAIL_CMD);
+
+#define NHI_INMAIL_CMD_RETRIES 50
+	/*
+	 * READ_ONCE fetches the value of nhi_ctxt->d0_exit every time
+	 * and avoid optimization.
+	 * deinit = true to continue the loop even if D3 process has been
+	 * carried out.
+	 */
+	for (i = 0; (i < NHI_INMAIL_CMD_RETRIES) &&
+		    (deinit || !READ_ONCE(nhi_ctxt->d0_exit)); i++) {
+		cmd = ioread32(nhi_ctxt->iobase + REG_INMAIL_CMD);
+
+		if (cmd & REG_INMAIL_CMD_ERROR)
+			return -EIO;
+
+		if (!(cmd & REG_INMAIL_CMD_REQUEST))
+			break;
+
+		msleep(delay);
+	}
+
+	if (i == NHI_INMAIL_CMD_RETRIES) {
+		if (!deinit)
+			dev_err(&nhi_ctxt->pdev->dev,
+				"controller id %#x is not functional, inmail timeout\n",
+				nhi_ctxt->id);
+		return -ETIMEDOUT;
+	}
+
+	return 0;
+}
+
+static int nhi_mailbox_generic(struct tbt_nhi_ctxt *nhi_ctxt, u32 mb_cmd)
+	__releases(&controllers_list_mutex)
+{
+	int res = -ENODEV;
+
+	if (mutex_lock_interruptible(&nhi_ctxt->mailbox_mutex)) {
+		res = -ERESTART;
+		goto release_ctl_list_lock;
+	}
+
+	if (!mutex_trylock(&nhi_ctxt->d0_exit_mailbox_mutex)) {
+		mutex_unlock(&nhi_ctxt->mailbox_mutex);
+		goto release_ctl_list_lock;
+	}
+
+	mutex_unlock(&controllers_list_mutex);
+
+	res = nhi_mailbox(nhi_ctxt, mb_cmd, 0, false);
+	mutex_unlock(&nhi_ctxt->d0_exit_mailbox_mutex);
+	mutex_unlock(&nhi_ctxt->mailbox_mutex);
+
+	return res;
+
+release_ctl_list_lock:
+	mutex_unlock(&controllers_list_mutex);
+	return res;
+}
+
+static int nhi_genl_mailbox(__always_unused struct sk_buff *u_skb,
+			    struct genl_info *info)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt;
+	u32 cmd, mb_cmd;
+
+	if (!info || !info->userhdr || !info->attrs ||
+	    !info->attrs[NHI_ATTR_MAILBOX_CMD])
+		return -EINVAL;
+
+	cmd = nla_get_u32(info->attrs[NHI_ATTR_MAILBOX_CMD]);
+	mb_cmd = ((cmd << REG_INMAIL_CMD_CMD_SHIFT) &
+		  REG_INMAIL_CMD_CMD_MASK) | REG_INMAIL_CMD_REQUEST;
+
+	if (mutex_lock_interruptible(&controllers_list_mutex))
+		return -ERESTART;
+
+	nhi_ctxt = nhi_search_ctxt(*(u32 *)info->userhdr);
+	if (nhi_ctxt && !nhi_ctxt->d0_exit)
+		return nhi_mailbox_generic(nhi_ctxt, mb_cmd);
+
+	mutex_unlock(&controllers_list_mutex);
+	return -ENODEV;
+}
+
+
+static int nhi_genl_send_msg(struct tbt_nhi_ctxt *nhi_ctxt, enum pdf_value pdf,
+			     const u8 *msg, u32 msg_len)
+{
+	u32 *msg_head, port_id;
+	struct sk_buff *skb;
+	int res;
+
+	if (atomic_read(&subscribers) < 1)
+		return -ENOTCONN;
+
+	skb = genlmsg_new(NLMSG_ALIGN(nhi_genl_family.hdrsize) +
+			  nla_total_size(msg_len) +
+			  nla_total_size(sizeof(pdf)),
+			  GFP_KERNEL);
+	if (!skb)
+		return -ENOMEM;
+
+	port_id = portid;
+	msg_head = genlmsg_put(skb, port_id, 0, &nhi_genl_family, 0,
+			       NHI_CMD_MSG_FROM_ICM);
+	if (!msg_head) {
+		res = -ENOMEM;
+		goto genl_put_reply_failure;
+	}
+
+	*msg_head = nhi_ctxt->id;
+
+	if (nla_put_u32(skb, NHI_ATTR_PDF, pdf) ||
+	    nla_put(skb, NHI_ATTR_MSG_FROM_ICM, msg_len, msg)) {
+		res = -EMSGSIZE;
+		goto nla_put_failure;
+	}
+
+	genlmsg_end(skb, msg_head);
+
+	return genlmsg_unicast(&init_net, skb, port_id);
+
+nla_put_failure:
+	genlmsg_cancel(skb, msg_head);
+genl_put_reply_failure:
+	nlmsg_free(skb);
+
+	return res;
+}
+
+static bool nhi_msg_from_icm_analysis(struct tbt_nhi_ctxt *nhi_ctxt,
+					enum pdf_value pdf,
+					const u8 *msg, u32 msg_len)
+{
+	/*
+	 * preparation for messages that won't be sent,
+	 * currently unused in this patch.
+	 */
+	bool send_event = true;
+
+	switch (pdf) {
+	case PDF_ERROR_NOTIFICATION:
+		/* fallthrough */
+	case PDF_WRITE_CONFIGURATION_REGISTERS:
+		/* fallthrough */
+	case PDF_READ_CONFIGURATION_REGISTERS:
+		if (nhi_ctxt->wait_for_icm_resp) {
+			nhi_ctxt->wait_for_icm_resp = false;
+			up(&nhi_ctxt->send_sem);
+		}
+		/* fallthrough */
+	default:
+		break;
+	}
+
+	return send_event;
+}
+
+static void nhi_msgs_from_icm(struct work_struct *work)
+			      __releases(&nhi_ctxt->send_sem)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt = container_of(work, typeof(*nhi_ctxt),
+						     icm_msgs_work);
+	void __iomem *reg = TBT_RING_CONS_PROD_REG(nhi_ctxt->iobase,
+						   REG_RX_RING_BASE,
+						   TBT_ICM_RING_NUM);
+	u32 prod_cons, prod, cons;
+
+	prod_cons = ioread32(reg);
+	prod = TBT_REG_RING_PROD_EXTRACT(prod_cons);
+	cons = TBT_REG_RING_CONS_EXTRACT(prod_cons);
+	if (prod >= TBT_ICM_RING_NUM_RX_BUFS) {
+		dev_warn(&nhi_ctxt->pdev->dev,
+			 "controller id %#x is not functional, producer %u out of range\n",
+			 nhi_ctxt->id, prod);
+		return;
+	}
+	if (cons >= TBT_ICM_RING_NUM_RX_BUFS) {
+		dev_warn(&nhi_ctxt->pdev->dev,
+			 "controller id %#x is not functional, consumer %u out of range\n",
+			 nhi_ctxt->id, cons);
+		return;
+	}
+
+	while (!TBT_RX_RING_EMPTY(prod, cons, TBT_ICM_RING_NUM_RX_BUFS) &&
+	       !nhi_ctxt->d0_exit) {
+		struct tbt_buf_desc *rx_desc;
+		u8 *msg;
+		u32 msg_len;
+		enum pdf_value pdf;
+		bool send_event;
+
+		cons = (cons + 1) % TBT_ICM_RING_NUM_RX_BUFS;
+		rx_desc = &(nhi_ctxt->icm_ring_shared_mem->rx_buf_desc[cons]);
+		if (!(le32_to_cpu(rx_desc->attributes) & DESC_ATTR_DESC_DONE))
+			usleep_range(10, 20);
+
+		rmb(); /* read the descriptor and the buffer after DD check */
+		pdf = (le32_to_cpu(rx_desc->attributes) & DESC_ATTR_EOF_MASK)
+		      >> DESC_ATTR_EOF_SHIFT;
+		msg = nhi_ctxt->icm_ring_shared_mem->rx_buf[cons];
+		msg_len = (le32_to_cpu(rx_desc->attributes)&DESC_ATTR_LEN_MASK)
+			  >> DESC_ATTR_LEN_SHIFT;
+
+		send_event = nhi_msg_from_icm_analysis(nhi_ctxt, pdf, msg,
+						       msg_len);
+
+		if (send_event)
+			nhi_genl_send_msg(nhi_ctxt, pdf, msg, msg_len);
+
+		/* set the descriptor for another receive */
+		rx_desc->attributes = cpu_to_le32(DESC_ATTR_REQ_STS |
+						  DESC_ATTR_INT_EN);
+		rx_desc->time = 0;
+	}
+
+	/* free the descriptors for more receive */
+	prod_cons &= ~REG_RING_CONS_MASK;
+	prod_cons |= (cons << REG_RING_CONS_SHIFT) & REG_RING_CONS_MASK;
+	iowrite32(prod_cons, reg);
+
+	if (!nhi_ctxt->d0_exit) {
+		unsigned long flags;
+
+		spin_lock_irqsave(&nhi_ctxt->lock, flags);
+		/* enable RX interrupt */
+		RING_INT_ENABLE_RX(nhi_ctxt->iobase, TBT_ICM_RING_NUM,
+				   nhi_ctxt->num_paths);
+
+		spin_unlock_irqrestore(&nhi_ctxt->lock, flags);
+	}
+}
+
+static irqreturn_t nhi_icm_ring_rx_msix(int __always_unused irq, void *data)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt = data;
+
+	spin_lock(&nhi_ctxt->lock);
+	/*
+	 * disable RX interrupt
+	 * We like to allow interrupt mitigation until the work item
+	 * will be completed.
+	 */
+	RING_INT_DISABLE_RX(nhi_ctxt->iobase, TBT_ICM_RING_NUM,
+			    nhi_ctxt->num_paths);
+
+	spin_unlock(&nhi_ctxt->lock);
+
+	schedule_work(&nhi_ctxt->icm_msgs_work);
+
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t nhi_msi(int __always_unused irq, void *data)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt = data;
+	u32 isr0, isr1, imr0, imr1;
+
+	/* clear on read */
+	isr0 = ioread32(nhi_ctxt->iobase + REG_RING_NOTIFY_BASE);
+	isr1 = ioread32(nhi_ctxt->iobase + REG_RING_NOTIFY_BASE +
+							REG_RING_NOTIFY_STEP);
+	if (unlikely(!isr0 && !isr1))
+		return IRQ_NONE;
+
+	spin_lock(&nhi_ctxt->lock);
+
+	imr0 = ioread32(nhi_ctxt->iobase + REG_RING_INTERRUPT_BASE);
+	imr1 = ioread32(nhi_ctxt->iobase + REG_RING_INTERRUPT_BASE +
+			REG_RING_INTERRUPT_STEP);
+	/* disable the arrived interrupts */
+	iowrite32(imr0 & ~isr0,
+		  nhi_ctxt->iobase + REG_RING_INTERRUPT_BASE);
+	iowrite32(imr1 & ~isr1,
+		  nhi_ctxt->iobase + REG_RING_INTERRUPT_BASE +
+		  REG_RING_INTERRUPT_STEP);
+
+	spin_unlock(&nhi_ctxt->lock);
+
+	if (isr0 & REG_RING_INT_RX_PROCESSED(TBT_ICM_RING_NUM,
+					     nhi_ctxt->num_paths))
+		schedule_work(&nhi_ctxt->icm_msgs_work);
+
+	return IRQ_HANDLED;
+}
+
+/**
+ * nhi_set_int_vec - Mapping of the MSIX vector entry to the ring
+ * @nhi_ctxt: contains data on NHI controller
+ * @path: ring to be mapped
+ * @msix_msg_id: msix entry to be mapped
+ */
+static inline void nhi_set_int_vec(struct tbt_nhi_ctxt *nhi_ctxt, u32 path,
+				   u8 msix_msg_id)
+{
+	void __iomem *reg;
+	u32 step, shift, ivr;
+
+	if (msix_msg_id % 2)
+		path += nhi_ctxt->num_paths;
+
+	step = path / REG_INT_VEC_ALLOC_PER_REG;
+	shift = (path % REG_INT_VEC_ALLOC_PER_REG) *
+		REG_INT_VEC_ALLOC_FIELD_BITS;
+	reg = nhi_ctxt->iobase + REG_INT_VEC_ALLOC_BASE +
+					(step * REG_INT_VEC_ALLOC_STEP);
+	ivr = ioread32(reg) & ~(REG_INT_VEC_ALLOC_FIELD_MASK << shift);
+	iowrite32(ivr | (msix_msg_id << shift), reg);
+}
+
+/* NHI genetlink operations array */
+static const struct genl_ops nhi_ops[] = {
+	{
+		.cmd = NHI_CMD_SUBSCRIBE,
+		.policy = nhi_genl_policy,
+		.doit = nhi_genl_subscribe,
+	},
+	{
+		.cmd = NHI_CMD_UNSUBSCRIBE,
+		.policy = nhi_genl_policy,
+		.doit = nhi_genl_unsubscribe,
+	},
+	{
+		.cmd = NHI_CMD_QUERY_INFORMATION,
+		.policy = nhi_genl_policy,
+		.doit = nhi_genl_query_information,
+	},
+	{
+		.cmd = NHI_CMD_MSG_TO_ICM,
+		.policy = nhi_genl_policy,
+		.doit = nhi_genl_msg_to_icm,
+		.flags = GENL_ADMIN_PERM,
+	},
+	{
+		.cmd = NHI_CMD_MAILBOX,
+		.policy = nhi_genl_policy,
+		.doit = nhi_genl_mailbox,
+		.flags = GENL_ADMIN_PERM,
+	},
+};
+
+static int nhi_suspend(struct device *dev) __releases(&nhi_ctxt->send_sem)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt = pci_get_drvdata(to_pci_dev(dev));
+	void __iomem *rx_reg, *tx_reg;
+	u32 rx_reg_val, tx_reg_val;
+
+	/* must be after negotiation_events, since messages might be sent */
+	nhi_ctxt->d0_exit = true;
+
+	rx_reg = nhi_ctxt->iobase + REG_RX_OPTIONS_BASE +
+		 (TBT_ICM_RING_NUM * REG_OPTS_STEP);
+	rx_reg_val = ioread32(rx_reg) & ~REG_OPTS_E2E_EN;
+	tx_reg = nhi_ctxt->iobase + REG_TX_OPTIONS_BASE +
+		 (TBT_ICM_RING_NUM * REG_OPTS_STEP);
+	tx_reg_val = ioread32(tx_reg) & ~REG_OPTS_E2E_EN;
+	/* disable RX flow control  */
+	iowrite32(rx_reg_val, rx_reg);
+	/* disable TX flow control  */
+	iowrite32(tx_reg_val, tx_reg);
+	/* disable RX ring  */
+	iowrite32(rx_reg_val & ~REG_OPTS_VALID, rx_reg);
+
+	mutex_lock(&nhi_ctxt->d0_exit_mailbox_mutex);
+	mutex_lock(&nhi_ctxt->d0_exit_send_mutex);
+
+	cancel_work_sync(&nhi_ctxt->icm_msgs_work);
+
+	if (nhi_ctxt->wait_for_icm_resp) {
+		nhi_ctxt->wait_for_icm_resp = false;
+		nhi_ctxt->ignore_icm_resp = false;
+		/*
+		 * if there is response, it is lost, so unlock the send
+		 * for the next resume.
+		 */
+		up(&nhi_ctxt->send_sem);
+	}
+
+	mutex_unlock(&nhi_ctxt->d0_exit_send_mutex);
+	mutex_unlock(&nhi_ctxt->d0_exit_mailbox_mutex);
+
+	/* wait for all TX to finish  */
+	usleep_range(5 * USEC_PER_MSEC, 7 * USEC_PER_MSEC);
+
+	/* disable all interrupts */
+	iowrite32(0, nhi_ctxt->iobase + REG_RING_INTERRUPT_BASE);
+	/* disable TX ring  */
+	iowrite32(tx_reg_val & ~REG_OPTS_VALID, tx_reg);
+
+	return 0;
+}
+
+static int nhi_resume(struct device *dev) __acquires(&nhi_ctxt->send_sem)
+{
+	dma_addr_t phys;
+	struct tbt_nhi_ctxt *nhi_ctxt = pci_get_drvdata(to_pci_dev(dev));
+	struct tbt_buf_desc *desc;
+	void __iomem *iobase = nhi_ctxt->iobase;
+	void __iomem *reg;
+	int i;
+
+	if (nhi_ctxt->msix_entries) {
+		iowrite32(ioread32(iobase + REG_DMA_MISC) |
+						REG_DMA_MISC_INT_AUTO_CLEAR,
+			  iobase + REG_DMA_MISC);
+		/*
+		 * Vector #0, which is TX complete to ICM,
+		 * isn't been used currently.
+		 */
+		nhi_set_int_vec(nhi_ctxt, 0, 1);
+
+		for (i = 2; i < nhi_ctxt->num_vectors; i++)
+			nhi_set_int_vec(nhi_ctxt, nhi_ctxt->num_paths - (i/2),
+					i);
+	}
+
+	/* configure TX descriptors */
+	for (i = 0, phys = nhi_ctxt->icm_ring_shared_mem_dma_addr;
+	     i < TBT_ICM_RING_NUM_TX_BUFS;
+	     i++, phys += TBT_ICM_RING_MAX_FRAME_SIZE) {
+		desc = &nhi_ctxt->icm_ring_shared_mem->tx_buf_desc[i];
+		desc->phys = cpu_to_le64(phys);
+		desc->attributes = cpu_to_le32(DESC_ATTR_REQ_STS);
+	}
+	/* configure RX descriptors */
+	for (i = 0;
+	     i < TBT_ICM_RING_NUM_RX_BUFS;
+	     i++, phys += TBT_ICM_RING_MAX_FRAME_SIZE) {
+		desc = &nhi_ctxt->icm_ring_shared_mem->rx_buf_desc[i];
+		desc->phys = cpu_to_le64(phys);
+		desc->attributes = cpu_to_le32(DESC_ATTR_REQ_STS |
+					       DESC_ATTR_INT_EN);
+	}
+
+	/* configure throttling rate for interrupts */
+	for (i = 0, reg = iobase + REG_INT_THROTTLING_RATE;
+	     i < NUM_INT_VECTORS;
+	     i++, reg += REG_INT_THROTTLING_RATE_STEP) {
+		iowrite32(USEC_TO_256_NSECS(128), reg);
+	}
+
+	/* configure TX for ICM ring */
+	reg = iobase + REG_TX_RING_BASE + (TBT_ICM_RING_NUM * REG_RING_STEP);
+	phys = nhi_ctxt->icm_ring_shared_mem_dma_addr +
+		offsetof(struct tbt_icm_ring_shared_memory, tx_buf_desc);
+	iowrite32(lower_32_bits(phys), reg + REG_RING_PHYS_LO_OFFSET);
+	iowrite32(upper_32_bits(phys), reg + REG_RING_PHYS_HI_OFFSET);
+	iowrite32((TBT_ICM_RING_NUM_TX_BUFS << REG_RING_SIZE_SHIFT) &
+			REG_RING_SIZE_MASK,
+		  reg + REG_RING_SIZE_OFFSET);
+
+	reg = iobase + REG_TX_OPTIONS_BASE + (TBT_ICM_RING_NUM*REG_OPTS_STEP);
+	iowrite32(REG_OPTS_RAW | REG_OPTS_VALID, reg);
+
+	/* configure RX for ICM ring */
+	reg = iobase + REG_RX_RING_BASE + (TBT_ICM_RING_NUM * REG_RING_STEP);
+	phys = nhi_ctxt->icm_ring_shared_mem_dma_addr +
+		offsetof(struct tbt_icm_ring_shared_memory, rx_buf_desc);
+	iowrite32(lower_32_bits(phys), reg + REG_RING_PHYS_LO_OFFSET);
+	iowrite32(upper_32_bits(phys), reg + REG_RING_PHYS_HI_OFFSET);
+	iowrite32(((TBT_ICM_RING_NUM_RX_BUFS << REG_RING_SIZE_SHIFT) &
+			REG_RING_SIZE_MASK) |
+		  ((TBT_ICM_RING_MAX_FRAME_SIZE << REG_RING_BUF_SIZE_SHIFT) &
+			REG_RING_BUF_SIZE_MASK),
+		  reg + REG_RING_SIZE_OFFSET);
+	iowrite32(((TBT_ICM_RING_NUM_RX_BUFS - 1) << REG_RING_CONS_SHIFT) &
+			REG_RING_CONS_MASK,
+		  reg + REG_RING_CONS_PROD_OFFSET);
+
+	reg = iobase + REG_RX_OPTIONS_BASE + (TBT_ICM_RING_NUM*REG_OPTS_STEP);
+	iowrite32(REG_OPTS_RAW | REG_OPTS_VALID, reg);
+
+	/* enable RX interrupt */
+	RING_INT_ENABLE_RX(iobase, TBT_ICM_RING_NUM, nhi_ctxt->num_paths);
+
+	if (likely((atomic_read(&subscribers) > 0) &&
+		   nhi_nvm_authenticated(nhi_ctxt))) {
+		down(&nhi_ctxt->send_sem);
+		nhi_ctxt->d0_exit = false;
+		mutex_lock(&nhi_ctxt->d0_exit_send_mutex);
+		/*
+		 * interrupts are enabled here before send due to
+		 * implicit barrier in mutex
+		 */
+		nhi_send_driver_ready_command(nhi_ctxt);
+		mutex_unlock(&nhi_ctxt->d0_exit_send_mutex);
+	} else {
+		nhi_ctxt->d0_exit = false;
+	}
+
+	return 0;
+}
+
+static void icm_nhi_shutdown(struct pci_dev *pdev)
+{
+	nhi_suspend(&pdev->dev);
+}
+
+static void icm_nhi_remove(struct pci_dev *pdev)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt = pci_get_drvdata(pdev);
+	int i;
+
+	nhi_suspend(&pdev->dev);
+
+	if (nhi_ctxt->net_workqueue)
+		destroy_workqueue(nhi_ctxt->net_workqueue);
+
+	/*
+	 * disable irq for msix or msi
+	 */
+	if (likely(nhi_ctxt->msix_entries)) {
+		/* Vector #0 isn't been used currently */
+		devm_free_irq(&pdev->dev, nhi_ctxt->msix_entries[1].vector,
+			      nhi_ctxt);
+		pci_disable_msix(pdev);
+	} else {
+		devm_free_irq(&pdev->dev, pdev->irq, nhi_ctxt);
+		pci_disable_msi(pdev);
+	}
+
+	/*
+	 * remove controller from the controllers list
+	 */
+	mutex_lock(&controllers_list_mutex);
+	list_del(&nhi_ctxt->node);
+	mutex_unlock(&controllers_list_mutex);
+
+	nhi_mailbox(
+		nhi_ctxt,
+		((CC_DRV_UNLOADS_AND_DISCONNECT_INTER_DOMAIN_PATHS
+		  << REG_INMAIL_CMD_CMD_SHIFT) &
+		 REG_INMAIL_CMD_CMD_MASK) |
+		REG_INMAIL_CMD_REQUEST,
+		0, true);
+
+	usleep_range(1 * USEC_PER_MSEC, 5 * USEC_PER_MSEC);
+	iowrite32(1, nhi_ctxt->iobase + REG_HOST_INTERFACE_RST);
+
+	mutex_destroy(&nhi_ctxt->d0_exit_send_mutex);
+	mutex_destroy(&nhi_ctxt->d0_exit_mailbox_mutex);
+	mutex_destroy(&nhi_ctxt->mailbox_mutex);
+	for (i = 0; i < nhi_ctxt->num_ports; i++)
+		mutex_destroy(&(nhi_ctxt->net_devices[i].state_mutex));
+}
+
+static int icm_nhi_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt;
+	void __iomem *iobase;
+	int i, res;
+	bool enable_msi = false;
+
+	res = pcim_enable_device(pdev);
+	if (res) {
+		dev_err(&pdev->dev, "cannot enable PCI device, aborting\n");
+		return res;
+	}
+
+	res = pcim_iomap_regions(pdev, 1 << NHI_MMIO_BAR, pci_name(pdev));
+	if (res) {
+		dev_err(&pdev->dev, "cannot obtain PCI resources, aborting\n");
+		return res;
+	}
+
+	/* cannot fail - table is allocated in pcim_iomap_regions */
+	iobase = pcim_iomap_table(pdev)[NHI_MMIO_BAR];
+
+	/* check if ICM is running */
+	if (!(ioread32(iobase + REG_FW_STS) & REG_FW_STS_ICM_EN)) {
+		dev_err(&pdev->dev, "ICM isn't present, aborting\n");
+		return -ENODEV;
+	}
+
+	nhi_ctxt = devm_kzalloc(&pdev->dev, sizeof(*nhi_ctxt), GFP_KERNEL);
+	if (!nhi_ctxt)
+		return -ENOMEM;
+
+	nhi_ctxt->pdev = pdev;
+	nhi_ctxt->iobase = iobase;
+	nhi_ctxt->id = (PCI_DEVID(pdev->bus->number, pdev->devfn) << 16) |
+								id->device;
+	/*
+	 * Number of paths represents the number of rings available for
+	 * the controller.
+	 */
+	nhi_ctxt->num_paths = ioread32(iobase + REG_HOP_COUNT) &
+						REG_HOP_COUNT_TOTAL_PATHS_MASK;
+
+	nhi_ctxt->nvm_auth_on_boot = DEVICE_DATA_NVM_AUTH_ON_BOOT(
+							id->driver_data);
+	nhi_ctxt->support_full_e2e = DEVICE_DATA_SUPPORT_FULL_E2E(
+							id->driver_data);
+
+	nhi_ctxt->dma_port = DEVICE_DATA_DMA_PORT(id->driver_data);
+	/*
+	 * Number of ports in the controller
+	 */
+	nhi_ctxt->num_ports = DEVICE_DATA_NUM_PORTS(id->driver_data);
+	nhi_ctxt->nvm_ver_offset = DEVICE_DATA_NVM_VER_OFFSET(id->driver_data);
+
+	mutex_init(&nhi_ctxt->d0_exit_send_mutex);
+	mutex_init(&nhi_ctxt->d0_exit_mailbox_mutex);
+	mutex_init(&nhi_ctxt->mailbox_mutex);
+
+	sema_init(&nhi_ctxt->send_sem, 1);
+
+	INIT_WORK(&nhi_ctxt->icm_msgs_work, nhi_msgs_from_icm);
+
+	spin_lock_init(&nhi_ctxt->lock);
+
+	nhi_ctxt->net_devices = devm_kcalloc(&pdev->dev,
+					     nhi_ctxt->num_ports,
+					     sizeof(struct port_net_dev),
+					     GFP_KERNEL);
+	if (!nhi_ctxt->net_devices)
+		return -ENOMEM;
+
+	for (i = 0; i < nhi_ctxt->num_ports; i++)
+		mutex_init(&(nhi_ctxt->net_devices[i].state_mutex));
+
+	/*
+	 * allocating RX and TX vectors for ICM and per port
+	 * for thunderbolt networking.
+	 * The mapping of the vector is carried out by
+	 * nhi_set_int_vec and looks like:
+	 * 0=tx icm, 1=rx icm, 2=tx data port 0,
+	 * 3=rx data port 0...
+	 */
+	nhi_ctxt->num_vectors = (1 + nhi_ctxt->num_ports) * 2;
+	nhi_ctxt->msix_entries = devm_kcalloc(&pdev->dev,
+					      nhi_ctxt->num_vectors,
+					      sizeof(struct msix_entry),
+					      GFP_KERNEL);
+	if (likely(nhi_ctxt->msix_entries)) {
+		for (i = 0; i < nhi_ctxt->num_vectors; i++)
+			nhi_ctxt->msix_entries[i].entry = i;
+		res = pci_enable_msix_exact(pdev,
+					    nhi_ctxt->msix_entries,
+					    nhi_ctxt->num_vectors);
+
+		if (res ||
+		    /*
+		     * Allocating ICM RX only.
+		     * vector #0, which is TX complete to ICM,
+		     * isn't been used currently
+		     */
+		    devm_request_irq(&pdev->dev,
+				     nhi_ctxt->msix_entries[1].vector,
+				     nhi_icm_ring_rx_msix, 0, pci_name(pdev),
+				     nhi_ctxt)) {
+			devm_kfree(&pdev->dev, nhi_ctxt->msix_entries);
+			nhi_ctxt->msix_entries = NULL;
+			enable_msi = true;
+		}
+	} else {
+		enable_msi = true;
+	}
+	/*
+	 * In case allocation didn't succeed, use msi instead of msix
+	 */
+	if (enable_msi) {
+		res = pci_enable_msi(pdev);
+		if (res) {
+			dev_err(&pdev->dev, "cannot enable MSI, aborting\n");
+			return res;
+		}
+		res = devm_request_irq(&pdev->dev, pdev->irq, nhi_msi, 0,
+				       pci_name(pdev), nhi_ctxt);
+		if (res) {
+			dev_err(&pdev->dev,
+				"request_irq failed %d, aborting\n", res);
+			return res;
+		}
+	}
+	/*
+	 * try to work with address space of 64 bits.
+	 * In case this doesn't work, work with 32 bits.
+	 */
+	if (!dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(64))) {
+		nhi_ctxt->pci_using_dac = true;
+	} else {
+		res = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+		if (res) {
+			dev_err(&pdev->dev,
+				"No suitable DMA available, aborting\n");
+			return res;
+		}
+	}
+
+	BUILD_BUG_ON(sizeof(struct tbt_buf_desc) != 16);
+	BUILD_BUG_ON(sizeof(struct tbt_icm_ring_shared_memory) > PAGE_SIZE);
+	nhi_ctxt->icm_ring_shared_mem = dmam_alloc_coherent(
+			&pdev->dev, sizeof(*nhi_ctxt->icm_ring_shared_mem),
+			&nhi_ctxt->icm_ring_shared_mem_dma_addr,
+			GFP_KERNEL | __GFP_ZERO);
+	if (nhi_ctxt->icm_ring_shared_mem == NULL) {
+		dev_err(&pdev->dev, "dmam_alloc_coherent failed, aborting\n");
+		return -ENOMEM;
+	}
+
+	nhi_ctxt->net_workqueue = create_singlethread_workqueue("thunderbolt");
+	if (!nhi_ctxt->net_workqueue) {
+		dev_err(&pdev->dev, "create_singlethread_workqueue failed, aborting\n");
+		return -ENOMEM;
+	}
+
+	pci_set_master(pdev);
+	pci_set_drvdata(pdev, nhi_ctxt);
+
+	nhi_resume(&pdev->dev);
+	/*
+	 * Add the new controller at the end of the list
+	 */
+	mutex_lock(&controllers_list_mutex);
+	list_add_tail(&nhi_ctxt->node, &controllers_list);
+	mutex_unlock(&controllers_list_mutex);
+
+	return res;
+}
+
+/*
+ * The tunneled pci bridges are siblings of us. Use resume_noirq to reenable
+ * the tunnels asap. A corresponding pci quirk blocks the downstream bridges
+ * resume_noirq until we are done.
+ */
+static const struct dev_pm_ops icm_nhi_pm_ops = {
+	SET_SYSTEM_SLEEP_PM_OPS(nhi_suspend, nhi_resume)
+};
+
+static const struct pci_device_id nhi_pci_device_ids[] = {
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_REDWOOD_RIDGE_2C_NHI),
+					DEVICE_DATA(1, 5, 0xa, false, false) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_REDWOOD_RIDGE_4C_NHI),
+					DEVICE_DATA(2, 5, 0xa, false, false) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_FALCON_RIDGE_2C_NHI),
+					DEVICE_DATA(1, 5, 0xa, false, false) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_FALCON_RIDGE_4C_NHI),
+					DEVICE_DATA(2, 5, 0xa, false, false) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_WIN_RIDGE_2C_NHI),
+					DEVICE_DATA(1, 3, 0xa, false, false) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_2C_NHI),
+					DEVICE_DATA(1, 5, 0xa, true, true) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_4C_NHI),
+					DEVICE_DATA(2, 5, 0xa, true, true) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_USBONLY_NHI),
+					DEVICE_DATA(1, 5, 0xa, true, true) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_LP_NHI),
+					DEVICE_DATA(1, 3, 0xa, true, true) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_LP_USBONLY_NHI),
+					DEVICE_DATA(1, 3, 0xa, true, true) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_C_2C_NHI),
+					DEVICE_DATA(1, 5, 0xa, true, true) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_C_4C_NHI),
+					DEVICE_DATA(2, 5, 0xa, true, true) },
+	{ PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_C_USBONLY_NHI),
+					DEVICE_DATA(1, 5, 0xa, true, true) },
+	{ 0, }
+};
+
+MODULE_DEVICE_TABLE(pci, nhi_pci_device_ids);
+MODULE_LICENSE("GPL");
+MODULE_VERSION(DRV_VERSION);
+
+static struct pci_driver icm_nhi_driver = {
+	.name = "thunderbolt",
+	.id_table = nhi_pci_device_ids,
+	.probe = icm_nhi_probe,
+	.remove = icm_nhi_remove,
+	.shutdown = icm_nhi_shutdown,
+	.driver.pm = &icm_nhi_pm_ops,
+};
+
+static int __init icm_nhi_init(void)
+{
+	int rc;
+
+	if (dmi_match(DMI_BOARD_VENDOR, "Apple Inc."))
+		return -ENODEV;
+
+	rc = genl_register_family_with_ops(&nhi_genl_family, nhi_ops);
+	if (rc)
+		goto failure;
+
+	rc = pci_register_driver(&icm_nhi_driver);
+	if (rc)
+		goto failure_genl;
+
+	return 0;
+
+failure_genl:
+	genl_unregister_family(&nhi_genl_family);
+
+failure:
+	pr_debug("nhi: error %d occurred in %s\n", rc, __func__);
+	return rc;
+}
+
+static void __exit icm_nhi_unload(void)
+{
+	genl_unregister_family(&nhi_genl_family);
+	pci_unregister_driver(&icm_nhi_driver);
+}
+
+module_init(icm_nhi_init);
+module_exit(icm_nhi_unload);
diff --git a/drivers/thunderbolt/icm/icm_nhi.h b/drivers/thunderbolt/icm/icm_nhi.h
new file mode 100644
index 0000000..1db37e5
--- /dev/null
+++ b/drivers/thunderbolt/icm/icm_nhi.h
@@ -0,0 +1,85 @@
+/*******************************************************************************
+ *
+ * Intel Thunderbolt(TM) driver
+ * Copyright(c) 2014 - 2016 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ ******************************************************************************/
+
+#ifndef ICM_NHI_H_
+#define ICM_NHI_H_
+
+#include <linux/pci.h>
+#include "../nhi_regs.h"
+
+#define DRV_VERSION "16.1.55.1"
+
+#define PCI_DEVICE_ID_INTEL_WIN_RIDGE_2C_NHI		0x157d /*Tbt 2 Low Pwr*/
+#define PCI_DEVICE_ID_INTEL_WIN_RIDGE_2C_BRIDGE		0x157e
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_LP_NHI		0x15bf /*Tbt 3 Low Pwr*/
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_LP_BRIDGE	0x15c0
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_C_4C_NHI	0x15d2 /*Thunderbolt 3*/
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_C_4C_BRIDGE	0x15d3
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_C_2C_NHI	0x15d9
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_C_2C_BRIDGE	0x15da
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_LP_USBONLY_NHI	0x15dc
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_USBONLY_NHI	0x15dd
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_C_USBONLY_NHI	0x15de
+
+#define TBT_ICM_RING_MAX_FRAME_SIZE	256
+#define TBT_ICM_RING_NUM		0
+#define TBT_RING_MAX_FRM_DATA_SZ	(TBT_RING_MAX_FRAME_SIZE - \
+					 sizeof(struct tbt_frame_header))
+
+enum icm_operation_mode {
+	SAFE_MODE,
+	AUTHENTICATION_MODE_FUNCTIONALITY,
+	ENDPOINT_OPERATION_MODE,
+	FULL_FUNCTIONALITY,
+};
+
+#define TBT_ICM_RING_NUM_TX_BUFS TBT_RING_MIN_NUM_BUFFERS
+#define TBT_ICM_RING_NUM_RX_BUFS ((PAGE_SIZE - (TBT_ICM_RING_NUM_TX_BUFS * \
+	(sizeof(struct tbt_buf_desc) + TBT_ICM_RING_MAX_FRAME_SIZE))) / \
+	(sizeof(struct tbt_buf_desc) + TBT_ICM_RING_MAX_FRAME_SIZE))
+
+/* struct tbt_icm_ring_shared_memory - memory area for DMA */
+struct tbt_icm_ring_shared_memory {
+	u8 tx_buf[TBT_ICM_RING_NUM_TX_BUFS][TBT_ICM_RING_MAX_FRAME_SIZE];
+	u8 rx_buf[TBT_ICM_RING_NUM_RX_BUFS][TBT_ICM_RING_MAX_FRAME_SIZE];
+	struct tbt_buf_desc tx_buf_desc[TBT_ICM_RING_NUM_TX_BUFS];
+	struct tbt_buf_desc rx_buf_desc[TBT_ICM_RING_NUM_RX_BUFS];
+} __aligned(TBT_ICM_RING_MAX_FRAME_SIZE);
+
+/* mailbox data from SW */
+#define REG_INMAIL_DATA		0x39900
+
+/* mailbox command from SW */
+#define REG_INMAIL_CMD		0x39904
+#define REG_INMAIL_CMD_CMD_SHIFT	0
+#define REG_INMAIL_CMD_CMD_MASK		GENMASK(7, REG_INMAIL_CMD_CMD_SHIFT)
+#define REG_INMAIL_CMD_ERROR		BIT(30)
+#define REG_INMAIL_CMD_REQUEST		BIT(31)
+
+/* mailbox command from FW */
+#define REG_OUTMAIL_CMD		0x3990C
+#define REG_OUTMAIL_CMD_STS_SHIFT	0
+#define REG_OUTMAIL_CMD_STS_MASK	GENMASK(7, REG_OUTMAIL_CMD_STS_SHIFT)
+#define REG_OUTMAIL_CMD_OP_MODE_SHIFT	8
+#define REG_OUTMAIL_CMD_OP_MODE_MASK	\
+				GENMASK(11, REG_OUTMAIL_CMD_OP_MODE_SHIFT)
+#define REG_OUTMAIL_CMD_REQUEST		BIT(31)
+
+#define REG_FW_STS		0x39944
+#define REG_FW_STS_ICM_EN		GENMASK(1, 0)
+#define REG_FW_STS_NVM_AUTH_DONE	BIT(31)
+
+#endif
diff --git a/drivers/thunderbolt/icm/net.h b/drivers/thunderbolt/icm/net.h
new file mode 100644
index 0000000..0281201
--- /dev/null
+++ b/drivers/thunderbolt/icm/net.h
@@ -0,0 +1,217 @@
+/*******************************************************************************
+ *
+ * Intel Thunderbolt(TM) driver
+ * Copyright(c) 2014 - 2016 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ ******************************************************************************/
+
+#ifndef NET_H_
+#define NET_H_
+
+#include <linux/pci.h>
+#include <linux/netdevice.h>
+#include <linux/mutex.h>
+#include <linux/semaphore.h>
+#include <net/genetlink.h>
+
+/*
+ * Each physical port contains 2 channels.
+ * Devices are exposed to user based on physical ports.
+ */
+#define CHANNELS_PER_PORT_NUM 2
+/*
+ * Calculate host physical port number (Zero-based numbering) from
+ * host channel/link which starts from 1.
+ */
+#define PORT_NUM_FROM_LINK(link) (((link) - 1) / CHANNELS_PER_PORT_NUM)
+
+#define TBT_TX_RING_FULL(prod, cons, size) ((((prod) + 1) % (size)) == (cons))
+#define TBT_TX_RING_EMPTY(prod, cons) ((prod) == (cons))
+#define TBT_RX_RING_FULL(prod, cons) ((prod) == (cons))
+#define TBT_RX_RING_EMPTY(prod, cons, size) ((((cons) + 1) % (size)) == (prod))
+
+#define PATH_FROM_PORT(num_paths, port_num) (((num_paths) - 1) - (port_num))
+
+/* Protocol Defined Field values for SW<->FW communication in raw mode */
+enum pdf_value {
+	PDF_READ_CONFIGURATION_REGISTERS = 1,
+	PDF_WRITE_CONFIGURATION_REGISTERS,
+	PDF_ERROR_NOTIFICATION,
+	PDF_ERROR_ACKNOWLEDGMENT,
+	PDF_PLUG_EVENT_NOTIFICATION,
+	PDF_INTER_DOMAIN_REQUEST,
+	PDF_INTER_DOMAIN_RESPONSE,
+	PDF_CM_OVERRIDE,
+	PDF_RESET_CIO_SWITCH,
+	PDF_FW_TO_SW_NOTIFICATION,
+	PDF_SW_TO_FW_COMMAND,
+	PDF_FW_TO_SW_RESPONSE
+};
+
+/*
+ * SW->FW commands
+ * CC = Command Code
+ */
+enum {
+	CC_GET_THUNDERBOLT_TOPOLOGY = 1,
+	CC_GET_VIDEO_RESOURCES_DATA,
+	CC_DRV_READY,
+	CC_APPROVE_PCI_CONNECTION,
+	CC_CHALLENGE_PCI_CONNECTION,
+	CC_ADD_DEVICE_AND_KEY,
+	CC_APPROVE_INTER_DOMAIN_CONNECTION = 0x10
+};
+
+/*
+ * FW->SW responses
+ * RC = response code
+ */
+enum {
+	RC_GET_TBT_TOPOLOGY = 1,
+	RC_GET_VIDEO_RESOURCES_DATA,
+	RC_DRV_READY,
+	RC_APPROVE_PCI_CONNECTION,
+	RC_CHALLENGE_PCI_CONNECTION,
+	RC_ADD_DEVICE_AND_KEY,
+	RC_INTER_DOMAIN_PKT_SENT = 8,
+	RC_APPROVE_INTER_DOMAIN_CONNECTION = 0x10
+};
+
+/*
+ * FW->SW notifications
+ * NC = notification code
+ */
+enum {
+	NC_DEVICE_CONNECTED = 3,
+	NC_DEVICE_DISCONNECTED,
+	NC_DP_DEVICE_CONNECTED_NOT_TUNNELED,
+	NC_INTER_DOMAIN_CONNECTED,
+	NC_INTER_DOMAIN_DISCONNECTED
+};
+
+/*
+ * SW -> FW mailbox commands
+ * CC = Command Code
+ */
+enum {
+	CC_STOP_CM_ACTIVITY,
+	CC_ENTER_PASS_THROUGH_MODE,
+	CC_ENTER_CM_OWNERSHIP_MODE,
+	CC_DRV_LOADED,
+	CC_DRV_UNLOADED,
+	CC_SAVE_CURRENT_CONNECTED_DEVICES,
+	CC_DISCONNECT_PCIE_PATHS,
+	CC_DRV_UNLOADS_AND_DISCONNECT_INTER_DOMAIN_PATHS,
+	DISCONNECT_PORT_A_INTER_DOMAIN_PATH = 0x10,
+	DISCONNECT_PORT_B_INTER_DOMAIN_PATH,
+	DP_TUNNEL_MODE_IN_ORDER_PER_CAPABILITIES = 0x1E,
+	DP_TUNNEL_MODE_MAXIMIZE_SNK_SRC_TUNNELS,
+	CC_SET_FW_MODE_FD1_D1_CERT = 0x20,
+	CC_SET_FW_MODE_FD1_D1_ALL,
+	CC_SET_FW_MODE_FD1_DA_CERT,
+	CC_SET_FW_MODE_FD1_DA_ALL,
+	CC_SET_FW_MODE_FDA_D1_CERT,
+	CC_SET_FW_MODE_FDA_D1_ALL,
+	CC_SET_FW_MODE_FDA_DA_CERT,
+	CC_SET_FW_MODE_FDA_DA_ALL
+};
+
+
+/* NHI genetlink attributes */
+enum {
+	NHI_ATTR_UNSPEC,
+	NHI_ATTR_DRV_VERSION,
+	NHI_ATTR_NVM_VER_OFFSET,
+	NHI_ATTR_NUM_PORTS,
+	NHI_ATTR_DMA_PORT,
+	NHI_ATTR_SUPPORT_FULL_E2E,
+	NHI_ATTR_MAILBOX_CMD,
+	NHI_ATTR_PDF,
+	NHI_ATTR_MSG_TO_ICM,
+	NHI_ATTR_MSG_FROM_ICM,
+	__NHI_ATTR_MAX,
+};
+#define NHI_ATTR_MAX (__NHI_ATTR_MAX - 1)
+
+struct port_net_dev {
+	struct net_device *net_dev;
+	struct mutex state_mutex;
+};
+
+/**
+ *  struct tbt_nhi_ctxt - thunderbolt native host interface context
+ *  @node:				node in the controllers list.
+ *  @pdev:				pci device information.
+ *  @iobase:				address of I/O.
+ *  @msix_entries:			MSI-X vectors.
+ *  @icm_ring_shared_mem:		virtual address of iCM ring.
+ *  @icm_ring_shared_mem_dma_addr:	DMA addr of iCM ring.
+ *  @send_sem:				semaphore for sending messages to iCM
+ *					one at a time.
+ *  @mailbox_mutex:			mutex for sending mailbox commands to
+ *					iCM one at a time.
+ *  @d0_exit_send_mutex:		synchronizing the d0 exit with messages.
+ *  @d0_exit_mailbox_mutex:		synchronizing the d0 exit with mailbox.
+ *  @lock:				synchronizing the interrupt registers
+ *					access.
+ *  @icm_msgs_work:			work queue for handling messages
+ *					from iCM.
+ *  @net_devices:			net devices per port.
+ *  @net_workqueue:			work queue to send net messages.
+ *  @id:				id of the controller.
+ *  @num_paths:				number of paths supported by controller.
+ *  @nvm_ver_offset:			offset of NVM version in NVM.
+ *  @num_vectors:			number of MSI-X vectors.
+ *  @num_ports:				number of ports in the controller.
+ *  @dma_port:				DMA port.
+ *  @d0_exit:				whether controller exit D0 state.
+ *  @nvm_auth_on_boot:			whether iCM authenticates the NVM
+ *					during boot.
+ *  @wait_for_icm_resp:			whether to wait for iCM response.
+ *  @ignore_icm_resp:			whether to ignore iCM response.
+ *  @pci_using_dac:			whether using DAC.
+ *  @support_full_e2e:			whether controller support full E2E.
+ */
+struct tbt_nhi_ctxt {
+	struct list_head node;
+	struct pci_dev *pdev;
+	void __iomem *iobase;
+	struct msix_entry *msix_entries;
+	struct tbt_icm_ring_shared_memory *icm_ring_shared_mem;
+	dma_addr_t icm_ring_shared_mem_dma_addr;
+	struct semaphore send_sem;
+	struct mutex mailbox_mutex;
+	struct mutex d0_exit_send_mutex;
+	struct mutex d0_exit_mailbox_mutex;
+	spinlock_t lock;
+	struct work_struct icm_msgs_work;
+	struct port_net_dev *net_devices;
+	struct workqueue_struct *net_workqueue;
+	u32 id;
+	u32 num_paths;
+	u16 nvm_ver_offset;
+	u8 num_vectors;
+	u8 num_ports;
+	u8 dma_port;
+	bool d0_exit;
+	bool nvm_auth_on_boot : 1;
+	bool wait_for_icm_resp : 1;
+	bool ignore_icm_resp : 1;
+	bool pci_using_dac : 1;
+	bool support_full_e2e : 1;
+};
+
+int nhi_send_message(struct tbt_nhi_ctxt *nhi_ctxt, enum pdf_value pdf,
+		      u32 msg_len, const void *msg, bool ignore_icm_resp);
+int nhi_mailbox(struct tbt_nhi_ctxt *nhi_ctxt, u32 cmd, u32 data, bool deinit);
+
+#endif
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v9 4/8] thunderbolt: Networking state machine
  2016-11-09 14:20 [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Amir Levy
                   ` (2 preceding siblings ...)
  2016-11-09 14:20 ` [PATCH v9 3/8] thunderbolt: Communication with the ICM (firmware) Amir Levy
@ 2016-11-09 14:20 ` Amir Levy
  2016-11-09 14:20 ` [PATCH v9 5/8] thunderbolt: Networking transmit and receive Amir Levy
                   ` (5 subsequent siblings)
  9 siblings, 0 replies; 28+ messages in thread
From: Amir Levy @ 2016-11-09 14:20 UTC (permalink / raw)
  To: gregkh
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang, Amir Levy

This patch builds the peer to peer communication path.
Communication is established by a negotiation process whereby messages are
sent back and forth between the peers until a connection is established.
This includes the Thunderbolt Network driver communication with the second
peer via Intel Connection Manager(ICM) firmware.
  +--------------------+            +--------------------+
  |Host 1              |            |Host 2              |
  |                    |            |                    |
  |     +-----------+  |            |     +-----------+  |
  |     |Thunderbolt|  |            |     |Thunderbolt|  |
  |     |Networking |  |            |     |Networking |  |
  |     |Driver     |  |            |     |Driver     |  |
  |     +-----------+  |            |     +-----------+  |
  |              ^     |            |              ^     |
  |              |     |            |              |     |
  | +------------+---+ |            | +------------+---+ |
  | |Thunderbolt |   | |            | |Thunderbolt |   | |
  | |Controller  v   | |            | |Controller  v   | |
  | |         +---+  | |            | |         +---+  | |
  | |         |ICM|<-+-+------------+-+-------->|ICM|  | |
  | |         +---+  | |            | |         +---+  | |
  | +----------------+ |            | +----------------+ |
  +--------------------+            +--------------------+
Note that this patch only establishes the link between the two hosts and
not Network Packet handling - this is dealt with in the next patch.

Signed-off-by: Amir Levy <amir.jer.levy@intel.com>
---
 drivers/thunderbolt/icm/Makefile  |   2 +-
 drivers/thunderbolt/icm/icm_nhi.c | 262 ++++++++++++-
 drivers/thunderbolt/icm/net.c     | 783 ++++++++++++++++++++++++++++++++++++++
 drivers/thunderbolt/icm/net.h     |  70 ++++
 4 files changed, 1109 insertions(+), 8 deletions(-)
 create mode 100644 drivers/thunderbolt/icm/net.c

diff --git a/drivers/thunderbolt/icm/Makefile b/drivers/thunderbolt/icm/Makefile
index f0d0fbb..94a2797 100644
--- a/drivers/thunderbolt/icm/Makefile
+++ b/drivers/thunderbolt/icm/Makefile
@@ -1,2 +1,2 @@
 obj-${CONFIG_THUNDERBOLT_ICM} += thunderbolt-icm.o
-thunderbolt-icm-objs := icm_nhi.o
+thunderbolt-icm-objs := icm_nhi.o net.o
diff --git a/drivers/thunderbolt/icm/icm_nhi.c b/drivers/thunderbolt/icm/icm_nhi.c
index c843ce8..edc910b 100644
--- a/drivers/thunderbolt/icm/icm_nhi.c
+++ b/drivers/thunderbolt/icm/icm_nhi.c
@@ -64,6 +64,13 @@ static const struct nla_policy nhi_genl_policy[NHI_ATTR_MAX + 1] = {
 					.len = TBT_ICM_RING_MAX_FRAME_SIZE },
 	[NHI_ATTR_MSG_FROM_ICM]		= { .type = NLA_BINARY,
 					.len = TBT_ICM_RING_MAX_FRAME_SIZE },
+	[NHI_ATTR_LOCAL_ROUTE_STRING]	= {
+					.len = sizeof(struct route_string) },
+	[NHI_ATTR_LOCAL_UUID]		= { .len = sizeof(uuid_be) },
+	[NHI_ATTR_REMOTE_UUID]		= { .len = sizeof(uuid_be) },
+	[NHI_ATTR_LOCAL_DEPTH]		= { .type = NLA_U8, },
+	[NHI_ATTR_ENABLE_FULL_E2E]	= { .type = NLA_FLAG, },
+	[NHI_ATTR_MATCH_FRAME_ID]	= { .type = NLA_FLAG, },
 };
 
 /* NHI genetlink family */
@@ -480,6 +487,29 @@ int nhi_mailbox(struct tbt_nhi_ctxt *nhi_ctxt, u32 cmd, u32 data, bool deinit)
 	return 0;
 }
 
+static inline bool nhi_is_path_disconnected(u32 cmd, u8 num_ports)
+{
+	return (cmd >= DISCONNECT_PORT_A_INTER_DOMAIN_PATH &&
+		cmd < (DISCONNECT_PORT_A_INTER_DOMAIN_PATH + num_ports));
+}
+
+static int nhi_mailbox_disconn_path(struct tbt_nhi_ctxt *nhi_ctxt, u32 cmd)
+	__releases(&controllers_list_mutex)
+{
+	struct port_net_dev *port;
+	u32 port_num = cmd - DISCONNECT_PORT_A_INTER_DOMAIN_PATH;
+
+	port = &(nhi_ctxt->net_devices[port_num]);
+	mutex_lock(&port->state_mutex);
+
+	mutex_unlock(&controllers_list_mutex);
+	port->medium_sts = MEDIUM_READY_FOR_APPROVAL;
+	if (port->net_dev)
+		negotiation_events(port->net_dev, MEDIUM_DISCONNECTED);
+	mutex_unlock(&port->state_mutex);
+	return  0;
+}
+
 static int nhi_mailbox_generic(struct tbt_nhi_ctxt *nhi_ctxt, u32 mb_cmd)
 	__releases(&controllers_list_mutex)
 {
@@ -526,13 +556,90 @@ static int nhi_genl_mailbox(__always_unused struct sk_buff *u_skb,
 		return -ERESTART;
 
 	nhi_ctxt = nhi_search_ctxt(*(u32 *)info->userhdr);
-	if (nhi_ctxt && !nhi_ctxt->d0_exit)
-		return nhi_mailbox_generic(nhi_ctxt, mb_cmd);
+	if (nhi_ctxt && !nhi_ctxt->d0_exit) {
+
+		/* rwsem is released later by the below functions */
+		if (nhi_is_path_disconnected(cmd, nhi_ctxt->num_ports))
+			return nhi_mailbox_disconn_path(nhi_ctxt, cmd);
+		else
+			return nhi_mailbox_generic(nhi_ctxt, mb_cmd);
+
+	}
 
 	mutex_unlock(&controllers_list_mutex);
 	return -ENODEV;
 }
 
+static int nhi_genl_approve_networking(__always_unused struct sk_buff *u_skb,
+				       struct genl_info *info)
+{
+	struct tbt_nhi_ctxt *nhi_ctxt;
+	struct route_string *route_str;
+	int res = -ENODEV;
+	u8 port_num;
+
+	if (!info || !info->userhdr || !info->attrs ||
+	    !info->attrs[NHI_ATTR_LOCAL_ROUTE_STRING] ||
+	    !info->attrs[NHI_ATTR_LOCAL_UUID] ||
+	    !info->attrs[NHI_ATTR_REMOTE_UUID] ||
+	    !info->attrs[NHI_ATTR_LOCAL_DEPTH])
+		return -EINVAL;
+
+	/*
+	 * route_str is an unique topological address
+	 * used for approving remote controller
+	 */
+	route_str = nla_data(info->attrs[NHI_ATTR_LOCAL_ROUTE_STRING]);
+	/* extracts the port we're connected to */
+	port_num = PORT_NUM_FROM_LINK(L0_PORT_NUM(route_str->lo));
+
+	if (mutex_lock_interruptible(&controllers_list_mutex))
+		return -ERESTART;
+
+	nhi_ctxt = nhi_search_ctxt(*(u32 *)info->userhdr);
+	if (nhi_ctxt && !nhi_ctxt->d0_exit) {
+		struct port_net_dev *port;
+
+		if (port_num >= nhi_ctxt->num_ports) {
+			res = -EINVAL;
+			goto free_ctl_list;
+		}
+
+		port = &(nhi_ctxt->net_devices[port_num]);
+
+		mutex_lock(&port->state_mutex);
+		mutex_unlock(&controllers_list_mutex);
+
+		if (port->medium_sts != MEDIUM_READY_FOR_APPROVAL)
+			goto unlock;
+
+		port->medium_sts = MEDIUM_READY_FOR_CONNECTION;
+
+		if (!port->net_dev) {
+			port->net_dev = nhi_alloc_etherdev(nhi_ctxt, port_num,
+							   info);
+			if (!port->net_dev) {
+				mutex_unlock(&port->state_mutex);
+				return -ENOMEM;
+			}
+		} else {
+			nhi_update_etherdev(nhi_ctxt, port->net_dev, info);
+
+			negotiation_events(port->net_dev,
+					   MEDIUM_READY_FOR_CONNECTION);
+		}
+
+unlock:
+		mutex_unlock(&port->state_mutex);
+
+		return 0;
+	}
+
+free_ctl_list:
+	mutex_unlock(&controllers_list_mutex);
+
+	return res;
+}
 
 static int nhi_genl_send_msg(struct tbt_nhi_ctxt *nhi_ctxt, enum pdf_value pdf,
 			     const u8 *msg, u32 msg_len)
@@ -579,17 +686,127 @@ static int nhi_genl_send_msg(struct tbt_nhi_ctxt *nhi_ctxt, enum pdf_value pdf,
 	return res;
 }
 
+static bool nhi_handle_inter_domain_msg(struct tbt_nhi_ctxt *nhi_ctxt,
+					struct thunderbolt_ip_header *hdr)
+{
+	struct port_net_dev *port;
+	u8 port_num;
+
+	const uuid_be proto_uuid = APPLE_THUNDERBOLT_IP_PROTOCOL_UUID;
+
+	if (uuid_be_cmp(proto_uuid, hdr->apple_tbt_ip_proto_uuid) != 0)
+		return true;
+
+	port_num = PORT_NUM_FROM_LINK(
+				L0_PORT_NUM(be32_to_cpu(hdr->route_str.lo)));
+
+	if (unlikely(port_num >= nhi_ctxt->num_ports))
+		return false;
+
+	port = &(nhi_ctxt->net_devices[port_num]);
+	mutex_lock(&port->state_mutex);
+	if (port->net_dev != NULL)
+		negotiation_messages(port->net_dev, hdr);
+	mutex_unlock(&port->state_mutex);
+
+	return false;
+}
+
+static void nhi_handle_notification_msg(struct tbt_nhi_ctxt *nhi_ctxt,
+					const u8 *msg)
+{
+	struct port_net_dev *port;
+	u8 port_num;
+
+#define INTER_DOMAIN_LINK_SHIFT 0
+#define INTER_DOMAIN_LINK_MASK	GENMASK(2, INTER_DOMAIN_LINK_SHIFT)
+	switch (msg[3]) {
+
+	case NC_INTER_DOMAIN_CONNECTED:
+		port_num = PORT_NUM_FROM_MSG(msg[5]);
+#define INTER_DOMAIN_APPROVED BIT(3)
+		if (port_num < nhi_ctxt->num_ports &&
+		    !(msg[5] & INTER_DOMAIN_APPROVED))
+			nhi_ctxt->net_devices[port_num].medium_sts =
+						MEDIUM_READY_FOR_APPROVAL;
+		break;
+
+	case NC_INTER_DOMAIN_DISCONNECTED:
+		port_num = PORT_NUM_FROM_MSG(msg[5]);
+
+		if (unlikely(port_num >= nhi_ctxt->num_ports))
+			break;
+
+		port = &(nhi_ctxt->net_devices[port_num]);
+		mutex_lock(&port->state_mutex);
+		port->medium_sts = MEDIUM_DISCONNECTED;
+
+		if (port->net_dev != NULL)
+			negotiation_events(port->net_dev,
+					   MEDIUM_DISCONNECTED);
+		mutex_unlock(&port->state_mutex);
+		break;
+	}
+}
+
+static bool nhi_handle_icm_response_msg(struct tbt_nhi_ctxt *nhi_ctxt,
+					const u8 *msg)
+{
+	struct port_net_dev *port;
+	bool send_event = true;
+	u8 port_num;
+
+	if (nhi_ctxt->ignore_icm_resp &&
+	    msg[3] == RC_INTER_DOMAIN_PKT_SENT) {
+		nhi_ctxt->ignore_icm_resp = false;
+		send_event = false;
+	}
+	if (nhi_ctxt->wait_for_icm_resp) {
+		nhi_ctxt->wait_for_icm_resp = false;
+		up(&nhi_ctxt->send_sem);
+	}
+
+	if (msg[3] == RC_APPROVE_INTER_DOMAIN_CONNECTION) {
+#define APPROVE_INTER_DOMAIN_ERROR BIT(0)
+		if (unlikely(msg[2] & APPROVE_INTER_DOMAIN_ERROR))
+			return send_event;
+
+		port_num = PORT_NUM_FROM_LINK((msg[5]&INTER_DOMAIN_LINK_MASK)>>
+					       INTER_DOMAIN_LINK_SHIFT);
+
+		if (unlikely(port_num >= nhi_ctxt->num_ports))
+			return send_event;
+
+		port = &(nhi_ctxt->net_devices[port_num]);
+		mutex_lock(&port->state_mutex);
+		port->medium_sts = MEDIUM_CONNECTED;
+
+		if (port->net_dev != NULL)
+			negotiation_events(port->net_dev, MEDIUM_CONNECTED);
+		mutex_unlock(&port->state_mutex);
+	}
+
+	return send_event;
+}
+
 static bool nhi_msg_from_icm_analysis(struct tbt_nhi_ctxt *nhi_ctxt,
 					enum pdf_value pdf,
 					const u8 *msg, u32 msg_len)
 {
-	/*
-	 * preparation for messages that won't be sent,
-	 * currently unused in this patch.
-	 */
 	bool send_event = true;
 
 	switch (pdf) {
+	case PDF_INTER_DOMAIN_REQUEST:
+	case PDF_INTER_DOMAIN_RESPONSE:
+		send_event = nhi_handle_inter_domain_msg(
+					nhi_ctxt,
+					(struct thunderbolt_ip_header *)msg);
+		break;
+
+	case PDF_FW_TO_SW_NOTIFICATION:
+		nhi_handle_notification_msg(nhi_ctxt, msg);
+		break;
+
 	case PDF_ERROR_NOTIFICATION:
 		/* fallthrough */
 	case PDF_WRITE_CONFIGURATION_REGISTERS:
@@ -599,7 +816,12 @@ static bool nhi_msg_from_icm_analysis(struct tbt_nhi_ctxt *nhi_ctxt,
 			nhi_ctxt->wait_for_icm_resp = false;
 			up(&nhi_ctxt->send_sem);
 		}
-		/* fallthrough */
+		break;
+
+	case PDF_FW_TO_SW_RESPONSE:
+		send_event = nhi_handle_icm_response_msg(nhi_ctxt, msg);
+		break;
+
 	default:
 		break;
 	}
@@ -788,6 +1010,12 @@ static const struct genl_ops nhi_ops[] = {
 		.doit = nhi_genl_mailbox,
 		.flags = GENL_ADMIN_PERM,
 	},
+	{
+		.cmd = NHI_CMD_APPROVE_TBT_NETWORKING,
+		.policy = nhi_genl_policy,
+		.doit = nhi_genl_approve_networking,
+		.flags = GENL_ADMIN_PERM,
+	},
 };
 
 static int nhi_suspend(struct device *dev) __releases(&nhi_ctxt->send_sem)
@@ -795,6 +1023,17 @@ static int nhi_suspend(struct device *dev) __releases(&nhi_ctxt->send_sem)
 	struct tbt_nhi_ctxt *nhi_ctxt = pci_get_drvdata(to_pci_dev(dev));
 	void __iomem *rx_reg, *tx_reg;
 	u32 rx_reg_val, tx_reg_val;
+	int i;
+
+	for (i = 0; i < nhi_ctxt->num_ports; i++) {
+		struct port_net_dev *port = &nhi_ctxt->net_devices[i];
+
+		mutex_lock(&port->state_mutex);
+		port->medium_sts = MEDIUM_DISCONNECTED;
+		if (port->net_dev)
+			negotiation_events(port->net_dev, MEDIUM_DISCONNECTED);
+		mutex_unlock(&port->state_mutex);
+	}
 
 	/* must be after negotiation_events, since messages might be sent */
 	nhi_ctxt->d0_exit = true;
@@ -954,6 +1193,15 @@ static void icm_nhi_remove(struct pci_dev *pdev)
 
 	nhi_suspend(&pdev->dev);
 
+	for (i = 0; i < nhi_ctxt->num_ports; i++) {
+		mutex_lock(&nhi_ctxt->net_devices[i].state_mutex);
+		if (nhi_ctxt->net_devices[i].net_dev) {
+			nhi_dealloc_etherdev(nhi_ctxt->net_devices[i].net_dev);
+			nhi_ctxt->net_devices[i].net_dev = NULL;
+		}
+		mutex_unlock(&nhi_ctxt->net_devices[i].state_mutex);
+	}
+
 	if (nhi_ctxt->net_workqueue)
 		destroy_workqueue(nhi_ctxt->net_workqueue);
 
diff --git a/drivers/thunderbolt/icm/net.c b/drivers/thunderbolt/icm/net.c
new file mode 100644
index 0000000..beeafb3
--- /dev/null
+++ b/drivers/thunderbolt/icm/net.c
@@ -0,0 +1,783 @@
+/*******************************************************************************
+ *
+ * Intel Thunderbolt(TM) driver
+ * Copyright(c) 2014 - 2016 Intel Corporation.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ ******************************************************************************/
+
+#include <linux/etherdevice.h>
+#include <linux/crc32.h>
+#include <linux/prefetch.h>
+#include <linux/highmem.h>
+#include <linux/if_vlan.h>
+#include <linux/jhash.h>
+#include <linux/vmalloc.h>
+#include <net/ip6_checksum.h>
+#include "icm_nhi.h"
+#include "net.h"
+
+#define DEFAULT_MSG_ENABLE (NETIF_MSG_PROBE | NETIF_MSG_LINK | NETIF_MSG_IFUP)
+static int debug = -1;
+module_param(debug, int, 0000);
+MODULE_PARM_DESC(debug, "Debug level (0=none,...,16=all)");
+
+#define TBT_NET_RX_HDR_SIZE 256
+
+#define NUM_TX_LOGIN_RETRIES 60
+
+#define APPLE_THUNDERBOLT_IP_PROTOCOL_REVISION 1
+
+#define LOGIN_TX_PATH 0xf
+
+#define TBT_NET_MTU (64 * 1024)
+
+/* Number of Rx buffers we bundle into one write to the hardware */
+#define TBT_NET_RX_BUFFER_WRITE	16
+
+#define TBT_NET_MULTICAST_HASH_TABLE_SIZE 1024
+#define TBT_NET_ETHER_ADDR_HASH(addr) (((addr[4] >> 4) | (addr[5] << 4)) % \
+				       TBT_NET_MULTICAST_HASH_TABLE_SIZE)
+
+#define BITS_PER_U32 (sizeof(u32) * BITS_PER_BYTE)
+
+#define TBT_NET_NUM_TX_BUFS 256
+#define TBT_NET_NUM_RX_BUFS 256
+#define TBT_NET_SIZE_TOTAL_DESCS ((TBT_NET_NUM_TX_BUFS + TBT_NET_NUM_RX_BUFS) \
+				  * sizeof(struct tbt_buf_desc))
+
+
+#define TBT_NUM_FRAMES_PER_PAGE (PAGE_SIZE / TBT_RING_MAX_FRAME_SIZE)
+
+#define TBT_NUM_BUFS_BETWEEN(idx1, idx2, num_bufs) \
+	(((num_bufs) - 1) - \
+	 ((((idx1) - (idx2)) + (num_bufs)) & ((num_bufs) - 1)))
+
+#define TX_WAKE_THRESHOLD (2 * DIV_ROUND_UP(TBT_NET_MTU, \
+			   TBT_RING_MAX_FRM_DATA_SZ))
+
+#define TBT_NET_DESC_ATTR_SOF_EOF (((PDF_TBT_NET_START_OF_FRAME << \
+				     DESC_ATTR_SOF_SHIFT) & \
+				    DESC_ATTR_SOF_MASK) | \
+				   ((PDF_TBT_NET_END_OF_FRAME << \
+				     DESC_ATTR_EOF_SHIFT) & \
+				    DESC_ATTR_EOF_MASK))
+
+/* E2E workaround */
+#define TBT_EXIST_BUT_UNUSED_HOPID 2
+
+enum tbt_net_frame_pdf {
+	PDF_TBT_NET_MIDDLE_FRAME,
+	PDF_TBT_NET_START_OF_FRAME,
+	PDF_TBT_NET_END_OF_FRAME,
+};
+
+struct thunderbolt_ip_login {
+	struct thunderbolt_ip_header header;
+	__be32 protocol_revision;
+	__be32 transmit_path;
+	__be32 reserved[4];
+	__be32 crc;
+};
+
+struct thunderbolt_ip_login_response {
+	struct thunderbolt_ip_header header;
+	__be32 status;
+	__be32 receiver_mac_address[2];
+	__be32 receiver_mac_address_length;
+	__be32 reserved[4];
+	__be32 crc;
+};
+
+struct thunderbolt_ip_logout {
+	struct thunderbolt_ip_header header;
+	__be32 crc;
+};
+
+struct thunderbolt_ip_status {
+	struct thunderbolt_ip_header header;
+	__be32 status;
+	__be32 crc;
+};
+
+struct approve_inter_domain_connection_cmd {
+	__be32 req_code;
+	__be32 attributes;
+#define AIDC_ATTR_LINK_SHIFT	16
+#define AIDC_ATTR_LINK_MASK	GENMASK(18, AIDC_ATTR_LINK_SHIFT)
+#define AIDC_ATTR_DEPTH_SHIFT	20
+#define AIDC_ATTR_DEPTH_MASK	GENMASK(23, AIDC_ATTR_DEPTH_SHIFT)
+	uuid_be remote_uuid;
+	__be16 transmit_ring_number;
+	__be16 transmit_path;
+	__be16 receive_ring_number;
+	__be16 receive_path;
+	__be32 crc;
+
+};
+
+enum neg_event {
+	RECEIVE_LOGOUT = NUM_MEDIUM_STATUSES,
+	RECEIVE_LOGIN_RESPONSE,
+	RECEIVE_LOGIN,
+	NUM_NEG_EVENTS
+};
+
+enum disconnect_path_stage {
+	STAGE_1 = BIT(0),
+	STAGE_2 = BIT(1)
+};
+
+/**
+ *  struct tbt_port - the basic tbt_port structure
+ *  @tbt_nhi_ctxt:		context of the nhi controller.
+ *  @net_dev:			networking device object.
+ *  @login_retry_work:		work queue for sending login requests.
+ *  @login_response_work:	work queue for sending login responses.
+ *  @work_struct logout_work:	work queue for sending logout requests.
+ *  @status_reply_work:		work queue for sending logout replies.
+ *  @approve_inter_domain_work:	work queue for sending interdomain to icm.
+ *  @route_str:			allows to route the messages to destination.
+ *  @interdomain_local_uuid:	allows to route the messages from local source.
+ *  @interdomain_remote_uuid:	allows to route the messages to destination.
+ *  @command_id			a number that identifies the command.
+ *  @negotiation_status:	holds the network negotiation state.
+ *  @msg_enable:		used for debugging filters.
+ *  @seq_num:			a number that identifies the session.
+ *  @login_retry_count:		counts number of login retries sent.
+ *  @local_depth:		depth of the remote peer in the chain.
+ *  @transmit_path:		routing parameter for the icm.
+ *  @frame_id:			counting ID of frames.
+ *  @num:			port number.
+ *  @local_path:		routing parameter for the icm.
+ *  @enable_full_e2e:		whether to enable full E2E.
+ *  @match_frame_id:		whether to match frame id on incoming packets.
+ */
+struct tbt_port {
+	struct tbt_nhi_ctxt *nhi_ctxt;
+	struct net_device *net_dev;
+	struct delayed_work login_retry_work;
+	struct work_struct login_response_work;
+	struct work_struct logout_work;
+	struct work_struct status_reply_work;
+	struct work_struct approve_inter_domain_work;
+	struct route_string route_str;
+	uuid_be interdomain_local_uuid;
+	uuid_be interdomain_remote_uuid;
+	u32 command_id;
+	u16 negotiation_status;
+	u16 msg_enable;
+	u8 seq_num;
+	u8 login_retry_count;
+	u8 local_depth;
+	u8 transmit_path;
+	u16 frame_id;
+	u8 num;
+	u8 local_path;
+	bool enable_full_e2e : 1;
+	bool match_frame_id : 1;
+};
+
+static void disconnect_path(struct tbt_port *port,
+			    enum disconnect_path_stage stage)
+{
+	u32 cmd = (DISCONNECT_PORT_A_INTER_DOMAIN_PATH + port->num);
+
+	cmd <<= REG_INMAIL_CMD_CMD_SHIFT;
+	cmd &= REG_INMAIL_CMD_CMD_MASK;
+	cmd |= REG_INMAIL_CMD_REQUEST;
+
+	mutex_lock(&port->nhi_ctxt->mailbox_mutex);
+	if (!mutex_trylock(&port->nhi_ctxt->d0_exit_mailbox_mutex)) {
+		netif_notice(port, link, port->net_dev, "controller id %#x is existing D0\n",
+			     port->nhi_ctxt->id);
+	} else {
+		nhi_mailbox(port->nhi_ctxt, cmd, stage, false);
+
+		port->nhi_ctxt->net_devices[port->num].medium_sts =
+					MEDIUM_READY_FOR_CONNECTION;
+
+		mutex_unlock(&port->nhi_ctxt->d0_exit_mailbox_mutex);
+	}
+	mutex_unlock(&port->nhi_ctxt->mailbox_mutex);
+}
+
+static void tbt_net_tear_down(struct net_device *net_dev, bool send_logout)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+	void __iomem *iobase = port->nhi_ctxt->iobase;
+	void __iomem *tx_reg = NULL;
+	u32 tx_reg_val = 0;
+
+	netif_carrier_off(net_dev);
+	netif_stop_queue(net_dev);
+
+	if (port->negotiation_status & BIT(MEDIUM_CONNECTED)) {
+		void __iomem *rx_reg = iobase + REG_RX_OPTIONS_BASE +
+		      (port->local_path * REG_OPTS_STEP);
+		u32 rx_reg_val = ioread32(rx_reg) & ~REG_OPTS_E2E_EN;
+
+		tx_reg = iobase + REG_TX_OPTIONS_BASE +
+			 (port->local_path * REG_OPTS_STEP);
+		tx_reg_val = ioread32(tx_reg) & ~REG_OPTS_E2E_EN;
+
+		disconnect_path(port, STAGE_1);
+
+		/* disable RX flow control  */
+		iowrite32(rx_reg_val, rx_reg);
+		/* disable TX flow control  */
+		iowrite32(tx_reg_val, tx_reg);
+		/* disable RX ring  */
+		iowrite32(rx_reg_val & ~REG_OPTS_VALID, rx_reg);
+
+		rx_reg = iobase + REG_RX_RING_BASE +
+			 (port->local_path * REG_RING_STEP);
+		iowrite32(0, rx_reg + REG_RING_PHYS_LO_OFFSET);
+		iowrite32(0, rx_reg + REG_RING_PHYS_HI_OFFSET);
+	}
+
+	/* Stop login messages */
+	cancel_delayed_work_sync(&port->login_retry_work);
+
+	if (send_logout)
+		queue_work(port->nhi_ctxt->net_workqueue, &port->logout_work);
+
+	if (port->negotiation_status & BIT(MEDIUM_CONNECTED)) {
+		unsigned long flags;
+
+		/* wait for TX to finish */
+		usleep_range(5 * USEC_PER_MSEC, 7 * USEC_PER_MSEC);
+		/* disable TX ring  */
+		iowrite32(tx_reg_val & ~REG_OPTS_VALID, tx_reg);
+
+		disconnect_path(port, STAGE_2);
+
+		spin_lock_irqsave(&port->nhi_ctxt->lock, flags);
+		/* disable RX and TX interrupts */
+		RING_INT_DISABLE_TX_RX(iobase, port->local_path,
+				       port->nhi_ctxt->num_paths);
+		spin_unlock_irqrestore(&port->nhi_ctxt->lock, flags);
+	}
+}
+
+static inline int send_message(struct tbt_port *port, const char *func,
+				enum pdf_value pdf, u32 msg_len,
+				const void *msg)
+{
+	u32 crc_offset = msg_len - sizeof(__be32);
+	__be32 *crc = (__be32 *)((u8 *)msg + crc_offset);
+	bool is_intdom = (pdf == PDF_INTER_DOMAIN_RESPONSE);
+	int res;
+
+	*crc = cpu_to_be32(~__crc32c_le(~0, msg, crc_offset));
+	res = down_timeout(&port->nhi_ctxt->send_sem,
+			   msecs_to_jiffies(3 * MSEC_PER_SEC));
+	if (res) {
+		netif_err(port, link, port->net_dev, "%s: controller id %#x timeout on send semaphore\n",
+			  func, port->nhi_ctxt->id);
+		return res;
+	}
+
+	if (!mutex_trylock(&port->nhi_ctxt->d0_exit_send_mutex)) {
+		up(&port->nhi_ctxt->send_sem);
+		netif_notice(port, link, port->net_dev, "%s: controller id %#x is existing D0\n",
+			     func, port->nhi_ctxt->id);
+		return -ENODEV;
+	}
+
+	res = nhi_send_message(port->nhi_ctxt, pdf, msg_len, msg, is_intdom);
+
+	mutex_unlock(&port->nhi_ctxt->d0_exit_send_mutex);
+	if (res)
+		up(&port->nhi_ctxt->send_sem);
+
+	return res;
+}
+
+static void approve_inter_domain(struct work_struct *work)
+{
+	struct tbt_port *port = container_of(work, typeof(*port),
+					     approve_inter_domain_work);
+	struct approve_inter_domain_connection_cmd approve_msg = {
+		.req_code = cpu_to_be32(CC_APPROVE_INTER_DOMAIN_CONNECTION),
+		.transmit_path = cpu_to_be16(LOGIN_TX_PATH),
+	};
+	u32 aidc = (L0_PORT_NUM(port->route_str.lo) << AIDC_ATTR_LINK_SHIFT) &
+		    AIDC_ATTR_LINK_MASK;
+
+	aidc |= (port->local_depth << AIDC_ATTR_DEPTH_SHIFT) &
+		 AIDC_ATTR_DEPTH_MASK;
+
+	approve_msg.attributes = cpu_to_be32(aidc);
+
+	memcpy(&approve_msg.remote_uuid, &port->interdomain_remote_uuid,
+	       sizeof(approve_msg.remote_uuid));
+	approve_msg.transmit_ring_number = cpu_to_be16(port->local_path);
+	approve_msg.receive_ring_number = cpu_to_be16(port->local_path);
+	approve_msg.receive_path = cpu_to_be16(port->transmit_path);
+
+	send_message(port, __func__, PDF_SW_TO_FW_COMMAND, sizeof(approve_msg),
+		     &approve_msg);
+}
+
+static inline void prepare_header(struct thunderbolt_ip_header *header,
+				  struct tbt_port *port,
+				  enum thunderbolt_ip_packet_type packet_type,
+				  u8 len_dwords)
+{
+	const uuid_be proto_uuid = APPLE_THUNDERBOLT_IP_PROTOCOL_UUID;
+
+	header->packet_type = cpu_to_be32(packet_type);
+	header->route_str.hi = cpu_to_be32(port->route_str.hi);
+	header->route_str.lo = cpu_to_be32(port->route_str.lo);
+	header->attributes = cpu_to_be32(
+		((port->seq_num << HDR_ATTR_SEQ_NUM_SHIFT) &
+		 HDR_ATTR_SEQ_NUM_MASK) |
+		((len_dwords << HDR_ATTR_LEN_SHIFT) & HDR_ATTR_LEN_MASK));
+	memcpy(&header->apple_tbt_ip_proto_uuid, &proto_uuid,
+	       sizeof(header->apple_tbt_ip_proto_uuid));
+	memcpy(&header->initiator_uuid, &port->interdomain_local_uuid,
+	       sizeof(header->initiator_uuid));
+	memcpy(&header->target_uuid, &port->interdomain_remote_uuid,
+	       sizeof(header->target_uuid));
+	header->command_id = cpu_to_be32(port->command_id);
+
+	port->command_id++;
+}
+
+static void status_reply(struct work_struct *work)
+{
+	struct tbt_port *port = container_of(work, typeof(*port),
+					     status_reply_work);
+	struct thunderbolt_ip_status status_msg = {
+		.status = 0,
+	};
+
+	prepare_header(&status_msg.header, port,
+		       THUNDERBOLT_IP_STATUS_TYPE,
+		       (offsetof(struct thunderbolt_ip_status, crc) -
+			offsetof(struct thunderbolt_ip_status,
+				 header.apple_tbt_ip_proto_uuid)) /
+		       sizeof(u32));
+
+	send_message(port, __func__, PDF_INTER_DOMAIN_RESPONSE,
+		     sizeof(status_msg), &status_msg);
+
+}
+
+static void logout(struct work_struct *work)
+{
+	struct tbt_port *port = container_of(work, typeof(*port),
+					     logout_work);
+	struct thunderbolt_ip_logout logout_msg;
+
+	prepare_header(&logout_msg.header, port,
+		       THUNDERBOLT_IP_LOGOUT_TYPE,
+		       (offsetof(struct thunderbolt_ip_logout, crc) -
+			offsetof(struct thunderbolt_ip_logout,
+			       header.apple_tbt_ip_proto_uuid)) / sizeof(u32));
+
+	send_message(port, __func__, PDF_INTER_DOMAIN_RESPONSE,
+		     sizeof(logout_msg), &logout_msg);
+
+}
+
+static void login_response(struct work_struct *work)
+{
+	struct tbt_port *port = container_of(work, typeof(*port),
+					     login_response_work);
+	struct thunderbolt_ip_login_response login_res_msg = {
+		.receiver_mac_address_length = cpu_to_be32(ETH_ALEN),
+	};
+
+	prepare_header(&login_res_msg.header, port,
+		       THUNDERBOLT_IP_LOGIN_RESPONSE_TYPE,
+		       (offsetof(struct thunderbolt_ip_login_response, crc) -
+			offsetof(struct thunderbolt_ip_login_response,
+			       header.apple_tbt_ip_proto_uuid)) / sizeof(u32));
+
+	ether_addr_copy((u8 *)login_res_msg.receiver_mac_address,
+			port->net_dev->dev_addr);
+
+	send_message(port, __func__, PDF_INTER_DOMAIN_RESPONSE,
+		     sizeof(login_res_msg), &login_res_msg);
+
+}
+
+static void login_retry(struct work_struct *work)
+{
+	struct tbt_port *port = container_of(work, typeof(*port),
+					     login_retry_work.work);
+	struct thunderbolt_ip_login login_msg = {
+		.protocol_revision = cpu_to_be32(
+				APPLE_THUNDERBOLT_IP_PROTOCOL_REVISION),
+		.transmit_path = cpu_to_be32(LOGIN_TX_PATH),
+	};
+
+
+	if (port->nhi_ctxt->d0_exit)
+		return;
+
+	port->login_retry_count++;
+
+	prepare_header(&login_msg.header, port,
+		       THUNDERBOLT_IP_LOGIN_TYPE,
+		       (offsetof(struct thunderbolt_ip_login, crc) -
+		       offsetof(struct thunderbolt_ip_login,
+		       header.apple_tbt_ip_proto_uuid)) / sizeof(u32));
+
+	if (send_message(port, __func__, PDF_INTER_DOMAIN_RESPONSE,
+			 sizeof(login_msg), &login_msg) == -ENODEV)
+		return;
+
+	if (likely(port->login_retry_count < NUM_TX_LOGIN_RETRIES))
+		queue_delayed_work(port->nhi_ctxt->net_workqueue,
+				   &port->login_retry_work,
+				   msecs_to_jiffies(5 * MSEC_PER_SEC));
+	else
+		netif_notice(port, link, port->net_dev, "port %u (%#x) login timeout after %u retries\n",
+			     port->num, port->negotiation_status,
+			     port->login_retry_count);
+}
+
+void negotiation_events(struct net_device *net_dev,
+			enum medium_status medium_sts)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+	void __iomem *iobase = port->nhi_ctxt->iobase;
+	u32 sof_eof_en, tx_ring_conf, rx_ring_conf, e2e_en;
+	void __iomem *reg;
+	unsigned long flags;
+	u16 hop_id;
+	bool send_logout;
+
+	if (!netif_running(net_dev)) {
+		netif_dbg(port, link, net_dev, "port %u (%#x) is down\n",
+			  port->num, port->negotiation_status);
+		return;
+	}
+
+	netif_dbg(port, link, net_dev, "port %u (%#x) receive event %u\n",
+		  port->num, port->negotiation_status, medium_sts);
+
+	switch (medium_sts) {
+	case MEDIUM_DISCONNECTED:
+		send_logout = (port->negotiation_status
+				& (BIT(MEDIUM_CONNECTED)
+				   |  BIT(MEDIUM_READY_FOR_CONNECTION)));
+		send_logout = send_logout && !(port->negotiation_status &
+					       BIT(RECEIVE_LOGOUT));
+
+		tbt_net_tear_down(net_dev, send_logout);
+		port->negotiation_status = BIT(MEDIUM_DISCONNECTED);
+		break;
+
+	case MEDIUM_CONNECTED:
+		/*
+		 * check if meanwhile other side sent logout
+		 * if yes, just don't allow connection to take place
+		 * and disconnect path
+		 */
+		if (port->negotiation_status & BIT(RECEIVE_LOGOUT)) {
+			disconnect_path(port, STAGE_1 | STAGE_2);
+			break;
+		}
+
+		port->negotiation_status = BIT(MEDIUM_CONNECTED);
+
+		/* configure TX ring */
+		reg = iobase + REG_TX_RING_BASE +
+		      (port->local_path * REG_RING_STEP);
+
+		tx_ring_conf = (TBT_NET_NUM_TX_BUFS << REG_RING_SIZE_SHIFT) &
+				REG_RING_SIZE_MASK;
+
+		iowrite32(tx_ring_conf, reg + REG_RING_SIZE_OFFSET);
+
+		/* enable the rings */
+		reg = iobase + REG_TX_OPTIONS_BASE +
+		      (port->local_path * REG_OPTS_STEP);
+		if (port->enable_full_e2e) {
+			iowrite32(REG_OPTS_VALID | REG_OPTS_E2E_EN, reg);
+			hop_id = port->local_path;
+		} else {
+			iowrite32(REG_OPTS_VALID, reg);
+			hop_id = TBT_EXIST_BUT_UNUSED_HOPID;
+		}
+
+		reg = iobase + REG_RX_OPTIONS_BASE +
+		      (port->local_path * REG_OPTS_STEP);
+
+		sof_eof_en = (BIT(PDF_TBT_NET_START_OF_FRAME) <<
+			      REG_RX_OPTS_MASK_SOF_SHIFT) &
+			     REG_RX_OPTS_MASK_SOF_MASK;
+
+		sof_eof_en |= (BIT(PDF_TBT_NET_END_OF_FRAME) <<
+			       REG_RX_OPTS_MASK_EOF_SHIFT) &
+			      REG_RX_OPTS_MASK_EOF_MASK;
+
+		iowrite32(sof_eof_en, reg + REG_RX_OPTS_MASK_OFFSET);
+
+		e2e_en = REG_OPTS_VALID | REG_OPTS_E2E_EN;
+		e2e_en |= (hop_id << REG_RX_OPTS_TX_E2E_HOP_ID_SHIFT) &
+			  REG_RX_OPTS_TX_E2E_HOP_ID_MASK;
+
+		iowrite32(e2e_en, reg);
+
+		/*
+		 * Configure RX ring
+		 * must be after enable ring for E2E to work
+		 */
+		reg = iobase + REG_RX_RING_BASE +
+		      (port->local_path * REG_RING_STEP);
+
+		rx_ring_conf = (TBT_NET_NUM_RX_BUFS << REG_RING_SIZE_SHIFT) &
+				REG_RING_SIZE_MASK;
+
+		rx_ring_conf |= (TBT_RING_MAX_FRAME_SIZE <<
+				 REG_RING_BUF_SIZE_SHIFT) &
+				REG_RING_BUF_SIZE_MASK;
+
+		iowrite32(rx_ring_conf, reg + REG_RING_SIZE_OFFSET);
+
+		spin_lock_irqsave(&port->nhi_ctxt->lock, flags);
+		/* enable RX interrupt */
+		iowrite32(ioread32(iobase + REG_RING_INTERRUPT_BASE) |
+			  REG_RING_INT_RX_PROCESSED(port->local_path,
+						    port->nhi_ctxt->num_paths),
+			  iobase + REG_RING_INTERRUPT_BASE);
+		spin_unlock_irqrestore(&port->nhi_ctxt->lock, flags);
+
+		netif_info(port, link, net_dev, "Thunderbolt(TM) Networking port %u - ready\n",
+			   port->num);
+
+		netif_carrier_on(net_dev);
+		netif_start_queue(net_dev);
+		break;
+
+	case MEDIUM_READY_FOR_CONNECTION:
+		/*
+		 * If medium is connected, no reason to go back,
+		 * keep it 'connected'.
+		 * If received login response, don't need to trigger login
+		 * retries again.
+		 */
+		if (unlikely(port->negotiation_status &
+			     (BIT(MEDIUM_CONNECTED) |
+			      BIT(RECEIVE_LOGIN_RESPONSE))))
+			break;
+
+		port->negotiation_status = BIT(MEDIUM_READY_FOR_CONNECTION);
+		port->login_retry_count = 0;
+		queue_delayed_work(port->nhi_ctxt->net_workqueue,
+				   &port->login_retry_work, 0);
+		break;
+
+	default:
+		break;
+	}
+}
+
+void negotiation_messages(struct net_device *net_dev,
+			  struct thunderbolt_ip_header *hdr)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+	__be32 status;
+
+	if (!netif_running(net_dev)) {
+		netif_dbg(port, link, net_dev, "port %u (%#x) is down\n",
+			  port->num, port->negotiation_status);
+		return;
+	}
+
+	switch (hdr->packet_type) {
+	case cpu_to_be32(THUNDERBOLT_IP_LOGIN_TYPE):
+		port->transmit_path = be32_to_cpu(
+			((struct thunderbolt_ip_login *)hdr)->transmit_path);
+		netif_dbg(port, link, net_dev, "port %u (%#x) receive ThunderboltIP login message with transmit path %u\n",
+			  port->num, port->negotiation_status,
+			  port->transmit_path);
+
+		if (unlikely(port->negotiation_status &
+			     BIT(MEDIUM_DISCONNECTED)))
+			break;
+
+		queue_work(port->nhi_ctxt->net_workqueue,
+			   &port->login_response_work);
+
+		if (unlikely(port->negotiation_status & BIT(MEDIUM_CONNECTED)))
+			break;
+
+		/*
+		 *  In case a login response received from other peer
+		 * on my login and acked their login for the first time,
+		 * so just approve the inter-domain now
+		 */
+		if (port->negotiation_status & BIT(RECEIVE_LOGIN_RESPONSE)) {
+			if (!(port->negotiation_status & BIT(RECEIVE_LOGIN)))
+				queue_work(port->nhi_ctxt->net_workqueue,
+					   &port->approve_inter_domain_work);
+		/*
+		 * if we reached the number of max retries or previous
+		 * logout, schedule another round of login retries
+		 */
+		} else if ((port->login_retry_count >= NUM_TX_LOGIN_RETRIES) ||
+			   (port->negotiation_status & BIT(RECEIVE_LOGOUT))) {
+			port->negotiation_status &= ~(BIT(RECEIVE_LOGOUT));
+			port->login_retry_count = 0;
+			queue_delayed_work(port->nhi_ctxt->net_workqueue,
+					   &port->login_retry_work, 0);
+		}
+
+		port->negotiation_status |= BIT(RECEIVE_LOGIN);
+
+		break;
+
+	case cpu_to_be32(THUNDERBOLT_IP_LOGIN_RESPONSE_TYPE):
+		status = ((struct thunderbolt_ip_login_response *)hdr)->status;
+		if (likely(status == 0)) {
+			netif_dbg(port, link, net_dev, "port %u (%#x) receive ThunderboltIP login response message\n",
+				  port->num,
+				  port->negotiation_status);
+
+			if (unlikely(port->negotiation_status &
+				     (BIT(MEDIUM_DISCONNECTED) |
+				      BIT(MEDIUM_CONNECTED) |
+				      BIT(RECEIVE_LOGIN_RESPONSE))))
+				break;
+
+			port->negotiation_status |=
+						BIT(RECEIVE_LOGIN_RESPONSE);
+			cancel_delayed_work_sync(&port->login_retry_work);
+			/*
+			 * login was received from other peer and now response
+			 * on our login so approve the inter-domain
+			 */
+			if (port->negotiation_status & BIT(RECEIVE_LOGIN))
+				queue_work(port->nhi_ctxt->net_workqueue,
+					   &port->approve_inter_domain_work);
+			else
+				port->negotiation_status &=
+							~BIT(RECEIVE_LOGOUT);
+		} else {
+			netif_notice(port, link, net_dev, "port %u (%#x) receive ThunderboltIP login response message with status %u\n",
+				     port->num,
+				     port->negotiation_status,
+				     be32_to_cpu(status));
+		}
+		break;
+
+	case cpu_to_be32(THUNDERBOLT_IP_LOGOUT_TYPE):
+		netif_dbg(port, link, net_dev, "port %u (%#x) receive ThunderboltIP logout message\n",
+			  port->num, port->negotiation_status);
+
+		queue_work(port->nhi_ctxt->net_workqueue,
+			   &port->status_reply_work);
+		port->negotiation_status &= ~(BIT(RECEIVE_LOGIN) |
+					      BIT(RECEIVE_LOGIN_RESPONSE));
+		port->negotiation_status |= BIT(RECEIVE_LOGOUT);
+
+		if (!(port->negotiation_status & BIT(MEDIUM_CONNECTED))) {
+			tbt_net_tear_down(net_dev, false);
+			break;
+		}
+
+		tbt_net_tear_down(net_dev, true);
+
+		port->negotiation_status |= BIT(MEDIUM_READY_FOR_CONNECTION);
+		port->negotiation_status &= ~(BIT(MEDIUM_CONNECTED));
+		break;
+
+	case cpu_to_be32(THUNDERBOLT_IP_STATUS_TYPE):
+		netif_dbg(port, link, net_dev, "port %u (%#x) receive ThunderboltIP status message with status %u\n",
+			  port->num, port->negotiation_status,
+			  be32_to_cpu(
+			  ((struct thunderbolt_ip_status *)hdr)->status));
+		break;
+	}
+}
+
+void nhi_dealloc_etherdev(struct net_device *net_dev)
+{
+	unregister_netdev(net_dev);
+	free_netdev(net_dev);
+}
+
+void nhi_update_etherdev(struct tbt_nhi_ctxt *nhi_ctxt,
+			 struct net_device *net_dev, struct genl_info *info)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+
+	nla_memcpy(&(port->route_str),
+		   info->attrs[NHI_ATTR_LOCAL_ROUTE_STRING],
+		   sizeof(port->route_str));
+	nla_memcpy(&port->interdomain_remote_uuid,
+		   info->attrs[NHI_ATTR_REMOTE_UUID],
+		   sizeof(port->interdomain_remote_uuid));
+	port->local_depth = nla_get_u8(info->attrs[NHI_ATTR_LOCAL_DEPTH]);
+	port->enable_full_e2e = nhi_ctxt->support_full_e2e ?
+		nla_get_flag(info->attrs[NHI_ATTR_ENABLE_FULL_E2E]) : false;
+	port->match_frame_id =
+		nla_get_flag(info->attrs[NHI_ATTR_MATCH_FRAME_ID]);
+	port->frame_id = 0;
+}
+
+struct net_device *nhi_alloc_etherdev(struct tbt_nhi_ctxt *nhi_ctxt,
+				      u8 port_num, struct genl_info *info)
+{
+	struct tbt_port *port;
+	struct net_device *net_dev = alloc_etherdev(sizeof(struct tbt_port));
+	u32 hash;
+
+	if (!net_dev)
+		return NULL;
+
+	SET_NETDEV_DEV(net_dev, &nhi_ctxt->pdev->dev);
+
+	port = netdev_priv(net_dev);
+	port->nhi_ctxt = nhi_ctxt;
+	port->net_dev = net_dev;
+	nla_memcpy(&port->interdomain_local_uuid,
+		   info->attrs[NHI_ATTR_LOCAL_UUID],
+		   sizeof(port->interdomain_local_uuid));
+	nhi_update_etherdev(nhi_ctxt, net_dev, info);
+	port->num = port_num;
+	port->local_path = PATH_FROM_PORT(nhi_ctxt->num_paths, port_num);
+
+	port->msg_enable = netif_msg_init(debug, DEFAULT_MSG_ENABLE);
+
+	net_dev->addr_assign_type = NET_ADDR_PERM;
+	/* unicast and locally administred MAC */
+	net_dev->dev_addr[0] = (port_num << 4) | 0x02;
+	hash = jhash2((u32 *)&port->interdomain_local_uuid,
+		      sizeof(port->interdomain_local_uuid)/sizeof(u32), 0);
+
+	memcpy(net_dev->dev_addr + 1, &hash, sizeof(hash));
+	hash = jhash2((u32 *)&port->interdomain_local_uuid,
+		      sizeof(port->interdomain_local_uuid)/sizeof(u32), hash);
+
+	net_dev->dev_addr[5] = hash & 0xff;
+
+	scnprintf(net_dev->name, sizeof(net_dev->name), "tbtnet%%dp%hhu",
+		  port_num);
+
+	INIT_DELAYED_WORK(&port->login_retry_work, login_retry);
+	INIT_WORK(&port->login_response_work, login_response);
+	INIT_WORK(&port->logout_work, logout);
+	INIT_WORK(&port->status_reply_work, status_reply);
+	INIT_WORK(&port->approve_inter_domain_work, approve_inter_domain);
+
+	netif_info(port, probe, net_dev,
+		   "Thunderbolt(TM) Networking port %u - MAC Address: %pM\n",
+		   port_num, net_dev->dev_addr);
+
+	return net_dev;
+}
diff --git a/drivers/thunderbolt/icm/net.h b/drivers/thunderbolt/icm/net.h
index 0281201..1cb6701 100644
--- a/drivers/thunderbolt/icm/net.h
+++ b/drivers/thunderbolt/icm/net.h
@@ -23,6 +23,10 @@
 #include <linux/semaphore.h>
 #include <net/genetlink.h>
 
+#define APPLE_THUNDERBOLT_IP_PROTOCOL_UUID	\
+	UUID_BE(0x9E588F79, 0x478A, 0x1636,		\
+		0x64, 0x56, 0xC6, 0x97, 0xDD, 0xC8, 0x20, 0xA9)
+
 /*
  * Each physical port contains 2 channels.
  * Devices are exposed to user based on physical ports.
@@ -33,6 +37,9 @@
  * host channel/link which starts from 1.
  */
 #define PORT_NUM_FROM_LINK(link) (((link) - 1) / CHANNELS_PER_PORT_NUM)
+#define PORT_NUM_FROM_MSG(msg) PORT_NUM_FROM_LINK(((msg) & \
+			       INTER_DOMAIN_LINK_MASK) >> \
+			       INTER_DOMAIN_LINK_SHIFT)
 
 #define TBT_TX_RING_FULL(prod, cons, size) ((((prod) + 1) % (size)) == (cons))
 #define TBT_TX_RING_EMPTY(prod, cons) ((prod) == (cons))
@@ -125,6 +132,17 @@ enum {
 	CC_SET_FW_MODE_FDA_DA_ALL
 };
 
+struct route_string {
+	u32 hi;
+	u32 lo;
+};
+
+struct route_string_be {
+	__be32 hi;
+	__be32 lo;
+};
+
+#define L0_PORT_NUM(cpu_route_str_lo) ((cpu_route_str_lo) & GENMASK(5, 0))
 
 /* NHI genetlink attributes */
 enum {
@@ -138,12 +156,53 @@ enum {
 	NHI_ATTR_PDF,
 	NHI_ATTR_MSG_TO_ICM,
 	NHI_ATTR_MSG_FROM_ICM,
+	NHI_ATTR_LOCAL_ROUTE_STRING,
+	NHI_ATTR_LOCAL_UUID,
+	NHI_ATTR_REMOTE_UUID,
+	NHI_ATTR_LOCAL_DEPTH,
+	NHI_ATTR_ENABLE_FULL_E2E,
+	NHI_ATTR_MATCH_FRAME_ID,
 	__NHI_ATTR_MAX,
 };
 #define NHI_ATTR_MAX (__NHI_ATTR_MAX - 1)
 
+/* ThunderboltIP Packet Types */
+enum thunderbolt_ip_packet_type {
+	THUNDERBOLT_IP_LOGIN_TYPE,
+	THUNDERBOLT_IP_LOGIN_RESPONSE_TYPE,
+	THUNDERBOLT_IP_LOGOUT_TYPE,
+	THUNDERBOLT_IP_STATUS_TYPE
+};
+
+struct thunderbolt_ip_header {
+	struct route_string_be route_str;
+	__be32 attributes;
+#define HDR_ATTR_LEN_SHIFT	0
+#define HDR_ATTR_LEN_MASK	GENMASK(5, HDR_ATTR_LEN_SHIFT)
+#define HDR_ATTR_SEQ_NUM_SHIFT	27
+#define HDR_ATTR_SEQ_NUM_MASK	GENMASK(28, HDR_ATTR_SEQ_NUM_SHIFT)
+	uuid_be apple_tbt_ip_proto_uuid;
+	uuid_be initiator_uuid;
+	uuid_be target_uuid;
+	__be32 packet_type;
+	__be32 command_id;
+};
+
+enum medium_status {
+	/* Handle cable disconnection or peer down */
+	MEDIUM_DISCONNECTED,
+	/* Connection is fully established */
+	MEDIUM_CONNECTED,
+	/*  Awaiting for being approved by user-space module */
+	MEDIUM_READY_FOR_APPROVAL,
+	/* Approved by user-space, awaiting for establishment flow to finish */
+	MEDIUM_READY_FOR_CONNECTION,
+	NUM_MEDIUM_STATUSES
+};
+
 struct port_net_dev {
 	struct net_device *net_dev;
+	enum medium_status medium_sts;
 	struct mutex state_mutex;
 };
 
@@ -213,5 +272,16 @@ struct tbt_nhi_ctxt {
 int nhi_send_message(struct tbt_nhi_ctxt *nhi_ctxt, enum pdf_value pdf,
 		      u32 msg_len, const void *msg, bool ignore_icm_resp);
 int nhi_mailbox(struct tbt_nhi_ctxt *nhi_ctxt, u32 cmd, u32 data, bool deinit);
+struct net_device *nhi_alloc_etherdev(struct tbt_nhi_ctxt *nhi_ctxt,
+				      u8 port_num, struct genl_info *info);
+void nhi_update_etherdev(struct tbt_nhi_ctxt *nhi_ctxt,
+			 struct net_device *net_dev, struct genl_info *info);
+void nhi_dealloc_etherdev(struct net_device *net_dev);
+void negotiation_events(struct net_device *net_dev,
+			enum medium_status medium_sts);
+void negotiation_messages(struct net_device *net_dev,
+			  struct thunderbolt_ip_header *hdr);
+void tbt_net_rx_msi(struct net_device *net_dev);
+void tbt_net_tx_msi(struct net_device *net_dev);
 
 #endif
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v9 5/8] thunderbolt: Networking transmit and receive
  2016-11-09 14:20 [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Amir Levy
                   ` (3 preceding siblings ...)
  2016-11-09 14:20 ` [PATCH v9 4/8] thunderbolt: Networking state machine Amir Levy
@ 2016-11-09 14:20 ` Amir Levy
  2016-11-09 14:20 ` [PATCH v9 6/8] thunderbolt: Kconfig for Thunderbolt Networking Amir Levy
                   ` (4 subsequent siblings)
  9 siblings, 0 replies; 28+ messages in thread
From: Amir Levy @ 2016-11-09 14:20 UTC (permalink / raw)
  To: gregkh
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang, Amir Levy

This patch provides the handling interface for sending and receiving
network packets between the hosts over the full communication route
(using the communication path established in the previous patch).

The Thunderbolt Network driver interfaces the Linux network stack
and the hardware controller configuration to handle packet transmissions:
  +----------------+            +----------------+
  |Host 1          |            |Host 2          |
  |                |            |                |
  |   +-------+    |            |    +-------+   |
  |   |Network|    |            |    |Network|   |
  |   |Stack  |    |            |    |Stack  |   |
  |   +-------+    |            |    +-------+   |
  |       ^        |            |        ^       |
  |       |        |            |        |       |
  |       v        |            |        v       |
  | +-----------+  |            |  +-----------+ |
  | |Thunderbolt|  |            |  |Thunderbolt| |
  | |Networking |  |            |  |Networking | |
  | |Driver     |  |            |  |Driver     | |
  | +-----------+  |            |  +-----------+ |
  |       ^        |            |        ^       |
  |       |        |            |        |       |
  |       v        |            |        v       |
  | +-----------+  |            |  +-----------+ |
  | |Thunderbolt|  |            |  |Thunderbolt| |
  | |Controller |<-+------------+->|Controller | |
  | +-----------+  |            |  +-----------+ |
  +----------------+            +----------------+

Signed-off-by: Amir Levy <amir.jer.levy@intel.com>
---
 drivers/thunderbolt/icm/icm_nhi.c |   15 +
 drivers/thunderbolt/icm/net.c     | 1471 +++++++++++++++++++++++++++++++++++++
 2 files changed, 1486 insertions(+)

diff --git a/drivers/thunderbolt/icm/icm_nhi.c b/drivers/thunderbolt/icm/icm_nhi.c
index edc910b..b1cc347 100644
--- a/drivers/thunderbolt/icm/icm_nhi.c
+++ b/drivers/thunderbolt/icm/icm_nhi.c
@@ -928,6 +928,7 @@ static irqreturn_t nhi_msi(int __always_unused irq, void *data)
 {
 	struct tbt_nhi_ctxt *nhi_ctxt = data;
 	u32 isr0, isr1, imr0, imr1;
+	int i;
 
 	/* clear on read */
 	isr0 = ioread32(nhi_ctxt->iobase + REG_RING_NOTIFY_BASE);
@@ -950,6 +951,20 @@ static irqreturn_t nhi_msi(int __always_unused irq, void *data)
 
 	spin_unlock(&nhi_ctxt->lock);
 
+	for (i = 0; i < nhi_ctxt->num_ports; ++i) {
+		struct net_device *net_dev =
+				nhi_ctxt->net_devices[i].net_dev;
+		if (net_dev) {
+			u8 path = PATH_FROM_PORT(nhi_ctxt->num_paths, i);
+
+			if (isr0 & REG_RING_INT_RX_PROCESSED(
+					path, nhi_ctxt->num_paths))
+				tbt_net_rx_msi(net_dev);
+			if (isr0 & REG_RING_INT_TX_PROCESSED(path))
+				tbt_net_tx_msi(net_dev);
+		}
+	}
+
 	if (isr0 & REG_RING_INT_RX_PROCESSED(TBT_ICM_RING_NUM,
 					     nhi_ctxt->num_paths))
 		schedule_work(&nhi_ctxt->icm_msgs_work);
diff --git a/drivers/thunderbolt/icm/net.c b/drivers/thunderbolt/icm/net.c
index beeafb3..cf985dd 100644
--- a/drivers/thunderbolt/icm/net.c
+++ b/drivers/thunderbolt/icm/net.c
@@ -124,6 +124,17 @@ struct approve_inter_domain_connection_cmd {
 
 };
 
+struct tbt_frame_header {
+	/* size of the data with the frame */
+	__le32 frame_size;
+	/* running index on the frames */
+	__le16 frame_index;
+	/* ID of the frame to match frames to specific packet */
+	__le16 frame_id;
+	/* how many frames assembles a full packet */
+	__le32 frame_count;
+};
+
 enum neg_event {
 	RECEIVE_LOGOUT = NUM_MEDIUM_STATUSES,
 	RECEIVE_LOGIN_RESPONSE,
@@ -131,15 +142,81 @@ enum neg_event {
 	NUM_NEG_EVENTS
 };
 
+enum frame_status {
+	GOOD_FRAME,
+	GOOD_AS_FIRST_FRAME,
+	GOOD_AS_FIRST_MULTICAST_FRAME,
+	FRAME_NOT_READY,
+	FRAME_ERROR,
+};
+
+enum packet_filter {
+	/* all multicast MAC addresses */
+	PACKET_TYPE_ALL_MULTICAST,
+	/* all types of MAC addresses: multicast, unicast and broadcast */
+	PACKET_TYPE_PROMISCUOUS,
+	/* all unicast MAC addresses */
+	PACKET_TYPE_UNICAST_PROMISCUOUS,
+};
+
 enum disconnect_path_stage {
 	STAGE_1 = BIT(0),
 	STAGE_2 = BIT(1)
 };
 
+struct tbt_net_stats {
+	u64 tx_packets;
+	u64 tx_bytes;
+	u64 tx_errors;
+	u64 rx_packets;
+	u64 rx_bytes;
+	u64 rx_length_errors;
+	u64 rx_over_errors;
+	u64 rx_crc_errors;
+	u64 rx_missed_errors;
+	u64 multicast;
+};
+
+static const char tbt_net_gstrings_stats[][ETH_GSTRING_LEN] = {
+	"tx_packets",
+	"tx_bytes",
+	"tx_errors",
+	"rx_packets",
+	"rx_bytes",
+	"rx_length_errors",
+	"rx_over_errors",
+	"rx_crc_errors",
+	"rx_missed_errors",
+	"multicast",
+};
+
+struct tbt_buffer {
+	dma_addr_t dma;
+	union {
+		struct tbt_frame_header *hdr;
+		struct page *page;
+	};
+	u32 page_offset;
+};
+
+struct tbt_desc_ring {
+	/* pointer to the descriptor ring memory */
+	struct tbt_buf_desc *desc;
+	/* physical address of the descriptor ring */
+	dma_addr_t dma;
+	/* array of buffer structs */
+	struct tbt_buffer *buffers;
+	/* last descriptor that was associated with a buffer */
+	u16 last_allocated;
+	/* next descriptor to check for DD status bit */
+	u16 next_to_clean;
+};
+
 /**
  *  struct tbt_port - the basic tbt_port structure
  *  @tbt_nhi_ctxt:		context of the nhi controller.
  *  @net_dev:			networking device object.
+ *  @napi:			network API
  *  @login_retry_work:		work queue for sending login requests.
  *  @login_response_work:	work queue for sending login responses.
  *  @work_struct logout_work:	work queue for sending logout requests.
@@ -155,6 +232,11 @@ enum disconnect_path_stage {
  *  @login_retry_count:		counts number of login retries sent.
  *  @local_depth:		depth of the remote peer in the chain.
  *  @transmit_path:		routing parameter for the icm.
+ *  @tx_ring:			transmit ring from where the packets are sent.
+ *  @rx_ring:			receive ring  where the packets are received.
+ *  @stats:			network statistics of the rx/tx packets.
+ *  @packet_filters:		defines filters for the received packets.
+ *  @multicast_hash_table:	hash table of multicast addresses.
  *  @frame_id:			counting ID of frames.
  *  @num:			port number.
  *  @local_path:		routing parameter for the icm.
@@ -164,6 +246,7 @@ enum disconnect_path_stage {
 struct tbt_port {
 	struct tbt_nhi_ctxt *nhi_ctxt;
 	struct net_device *net_dev;
+	struct napi_struct napi;
 	struct delayed_work login_retry_work;
 	struct work_struct login_response_work;
 	struct work_struct logout_work;
@@ -179,6 +262,17 @@ struct tbt_port {
 	u8 login_retry_count;
 	u8 local_depth;
 	u8 transmit_path;
+	struct tbt_desc_ring tx_ring ____cacheline_aligned_in_smp;
+	struct tbt_desc_ring rx_ring;
+	struct tbt_net_stats stats;
+	u32 packet_filters;
+	/*
+	 * hash table of 1024 boolean entries with hashing of
+	 * the multicast address
+	 */
+	u32 multicast_hash_table[DIV_ROUND_UP(
+					TBT_NET_MULTICAST_HASH_TABLE_SIZE,
+					BITS_PER_U32)];
 	u16 frame_id;
 	u8 num;
 	u8 local_path;
@@ -225,6 +319,8 @@ static void tbt_net_tear_down(struct net_device *net_dev, bool send_logout)
 		      (port->local_path * REG_OPTS_STEP);
 		u32 rx_reg_val = ioread32(rx_reg) & ~REG_OPTS_E2E_EN;
 
+		napi_disable(&port->napi);
+
 		tx_reg = iobase + REG_TX_OPTIONS_BASE +
 			 (port->local_path * REG_OPTS_STEP);
 		tx_reg_val = ioread32(tx_reg) & ~REG_OPTS_E2E_EN;
@@ -266,8 +362,1336 @@ static void tbt_net_tear_down(struct net_device *net_dev, bool send_logout)
 				       port->nhi_ctxt->num_paths);
 		spin_unlock_irqrestore(&port->nhi_ctxt->lock, flags);
 	}
+
+	port->rx_ring.next_to_clean = 0;
+	port->rx_ring.last_allocated = TBT_NET_NUM_RX_BUFS - 1;
+
+}
+
+void tbt_net_tx_msi(struct net_device *net_dev)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+	void __iomem *iobase = port->nhi_ctxt->iobase;
+	u32 prod_cons, prod, cons;
+
+	prod_cons = ioread32(TBT_RING_CONS_PROD_REG(iobase, REG_TX_RING_BASE,
+						    port->local_path));
+	prod = TBT_REG_RING_PROD_EXTRACT(prod_cons);
+	cons = TBT_REG_RING_CONS_EXTRACT(prod_cons);
+	if (prod >= TBT_NET_NUM_TX_BUFS || cons >= TBT_NET_NUM_TX_BUFS)
+		return;
+
+	if (TBT_NUM_BUFS_BETWEEN(prod, cons, TBT_NET_NUM_TX_BUFS) >=
+							TX_WAKE_THRESHOLD) {
+		netif_wake_queue(port->net_dev);
+	} else {
+		spin_lock(&port->nhi_ctxt->lock);
+		/* enable TX interrupt */
+		RING_INT_ENABLE_TX(iobase, port->local_path);
+		spin_unlock(&port->nhi_ctxt->lock);
+	}
+}
+
+static irqreturn_t tbt_net_tx_msix(int __always_unused irq, void *data)
+{
+	struct tbt_port *port = data;
+	void __iomem *iobase = port->nhi_ctxt->iobase;
+	u32 prod_cons, prod, cons;
+
+	prod_cons = ioread32(TBT_RING_CONS_PROD_REG(iobase,
+						    REG_TX_RING_BASE,
+						    port->local_path));
+	prod = TBT_REG_RING_PROD_EXTRACT(prod_cons);
+	cons = TBT_REG_RING_CONS_EXTRACT(prod_cons);
+	if (prod < TBT_NET_NUM_TX_BUFS && cons < TBT_NET_NUM_TX_BUFS &&
+	    TBT_NUM_BUFS_BETWEEN(prod, cons, TBT_NET_NUM_TX_BUFS) >=
+							TX_WAKE_THRESHOLD) {
+		spin_lock(&port->nhi_ctxt->lock);
+		/* disable TX interrupt */
+		RING_INT_DISABLE_TX(iobase, port->local_path);
+		spin_unlock(&port->nhi_ctxt->lock);
+
+		netif_wake_queue(port->net_dev);
+	}
+
+	return IRQ_HANDLED;
+}
+
+void tbt_net_rx_msi(struct net_device *net_dev)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+
+	napi_schedule_irqoff(&port->napi);
+}
+
+static irqreturn_t tbt_net_rx_msix(int __always_unused irq, void *data)
+{
+	struct tbt_port *port = data;
+
+	if (likely(napi_schedule_prep(&port->napi))) {
+		struct tbt_nhi_ctxt *nhi_ctx = port->nhi_ctxt;
+
+		spin_lock(&nhi_ctx->lock);
+		/* disable RX interrupt */
+		RING_INT_DISABLE_RX(nhi_ctx->iobase, port->local_path,
+				    nhi_ctx->num_paths);
+		spin_unlock(&nhi_ctx->lock);
+
+		__napi_schedule_irqoff(&port->napi);
+	}
+
+	return IRQ_HANDLED;
+}
+
+static void tbt_net_pull_tail(struct sk_buff *skb)
+{
+	skb_frag_t *frag = &skb_shinfo(skb)->frags[0];
+	unsigned int pull_len;
+	unsigned char *va;
+
+	/*
+	 * it is valid to use page_address instead of kmap since we are
+	 * working with pages allocated out of the lomem pool
+	 */
+	va = skb_frag_address(frag);
+
+	pull_len = eth_get_headlen(va, TBT_NET_RX_HDR_SIZE);
+
+	/* align pull length to size of long to optimize memcpy performance */
+	skb_copy_to_linear_data(skb, va, ALIGN(pull_len, sizeof(long)));
+
+	/* update all of the pointers */
+	skb_frag_size_sub(frag, pull_len);
+	frag->page_offset += pull_len;
+	skb->data_len -= pull_len;
+	skb->tail += pull_len;
+}
+
+static inline bool tbt_net_alloc_mapped_page(struct device *dev,
+					     struct tbt_buffer *buf, gfp_t gfp)
+{
+	if (!buf->page) {
+		buf->page = alloc_page(gfp | __GFP_COLD);
+		if (unlikely(!buf->page))
+			return false;
+
+		buf->dma = dma_map_page(dev, buf->page, 0, PAGE_SIZE,
+					DMA_FROM_DEVICE);
+		if (dma_mapping_error(dev, buf->dma)) {
+			__free_page(buf->page);
+			buf->page = NULL;
+			return false;
+		}
+		buf->page_offset = 0;
+	}
+	return true;
+}
+
+static bool tbt_net_alloc_rx_buffers(struct device *dev,
+				     struct tbt_desc_ring *rx_ring,
+				     u16 cleaned_count, void __iomem *reg,
+				     gfp_t gfp)
+{
+	u16 i = (rx_ring->last_allocated + 1) & (TBT_NET_NUM_RX_BUFS - 1);
+	bool res = false;
+
+	while (cleaned_count--) {
+		struct tbt_buf_desc *desc = &rx_ring->desc[i];
+		struct tbt_buffer *buf = &rx_ring->buffers[i];
+
+		/* making sure next_to_clean won't get old buffer */
+		desc->attributes = cpu_to_le32(DESC_ATTR_REQ_STS |
+					       DESC_ATTR_INT_EN);
+		if (tbt_net_alloc_mapped_page(dev, buf, gfp)) {
+			res = true;
+			rx_ring->last_allocated = i;
+			i = (i + 1) & (TBT_NET_NUM_RX_BUFS - 1);
+			desc->phys = cpu_to_le64(buf->dma + buf->page_offset);
+		} else {
+			break;
+		}
+	}
+
+	if (res) {
+		iowrite32((rx_ring->last_allocated << REG_RING_CONS_SHIFT) &
+			  REG_RING_CONS_MASK, reg);
+	}
+
+	return res;
+}
+
+static inline bool tbt_net_multicast_mac_set(const u32 *multicast_hash_table,
+					     const u8 *ether_addr)
+{
+	u16 hash_val = TBT_NET_ETHER_ADDR_HASH(ether_addr);
+
+	return !!(multicast_hash_table[hash_val / BITS_PER_U32] &
+		  BIT(hash_val % BITS_PER_U32));
+}
+
+static enum frame_status tbt_net_check_frame(struct tbt_port *port,
+					     u16 frame_num, u32 *count,
+					     u16 index, u16 *id, u32 *size)
+{
+	struct tbt_desc_ring *rx_ring = &port->rx_ring;
+	__le32 desc_attr = rx_ring->desc[frame_num].attributes;
+	enum frame_status res = GOOD_AS_FIRST_FRAME;
+	u32 len, frame_count, frame_size;
+	struct tbt_frame_header *hdr;
+
+	if (!(desc_attr & cpu_to_le32(DESC_ATTR_DESC_DONE)))
+		return FRAME_NOT_READY;
+
+	rmb(); /* read other fields from desc after checking DD */
+
+	if (unlikely(desc_attr & cpu_to_le32(DESC_ATTR_RX_CRC_ERR))) {
+		++port->stats.rx_crc_errors;
+		goto err;
+	} else if (unlikely(desc_attr &
+				cpu_to_le32(DESC_ATTR_RX_BUF_OVRN_ERR))) {
+		++port->stats.rx_over_errors;
+		goto err;
+	}
+
+	len = (le32_to_cpu(desc_attr) & DESC_ATTR_LEN_MASK)
+	      >> DESC_ATTR_LEN_SHIFT;
+	if (len == 0)
+		len = TBT_RING_MAX_FRAME_SIZE;
+	/* should be greater than just header i.e. contains data */
+	if (unlikely(len <= sizeof(struct tbt_frame_header))) {
+		++port->stats.rx_length_errors;
+		goto err;
+	}
+
+	prefetchw(rx_ring->buffers[frame_num].page);
+	hdr = page_address(rx_ring->buffers[frame_num].page) +
+				rx_ring->buffers[frame_num].page_offset;
+	/* prefetch first cache line of first page */
+	prefetch(hdr);
+
+	/* we are reusing so sync this buffer for CPU use */
+	dma_sync_single_range_for_cpu(&port->nhi_ctxt->pdev->dev,
+				      rx_ring->buffers[frame_num].dma,
+				      rx_ring->buffers[frame_num].page_offset,
+				      TBT_RING_MAX_FRAME_SIZE,
+				      DMA_FROM_DEVICE);
+
+	frame_count = le32_to_cpu(hdr->frame_count);
+	frame_size = le32_to_cpu(hdr->frame_size);
+
+	if (unlikely((frame_size > len - sizeof(struct tbt_frame_header)) ||
+		     (frame_size == 0))) {
+		++port->stats.rx_length_errors;
+		goto err;
+	}
+	/*
+	 * In case we're in the middle of packet, validate the frame header
+	 * based on first fragment of the packet
+	 */
+	if (*count) {
+		/* check the frame count fits the count field */
+		if (frame_count != *count) {
+			++port->stats.rx_length_errors;
+			goto check_as_first;
+		}
+
+		/*
+		 * check the frame identifiers are incremented correctly,
+		 * and id is matching
+		 */
+		if ((le16_to_cpu(hdr->frame_index) != index) ||
+		    (le16_to_cpu(hdr->frame_id) != *id)) {
+			++port->stats.rx_missed_errors;
+			goto check_as_first;
+		}
+
+		*size += frame_size;
+		if (*size > TBT_NET_MTU) {
+			++port->stats.rx_length_errors;
+			goto err;
+		}
+		res = GOOD_FRAME;
+	} else { /* start of packet, validate the frame header */
+		const u8 *addr;
+
+check_as_first:
+		rx_ring->next_to_clean = frame_num;
+
+		/* validate the first packet has a valid frame count */
+		if (unlikely(frame_count == 0 ||
+			     frame_count > (TBT_NET_NUM_RX_BUFS / 4))) {
+			++port->stats.rx_length_errors;
+			goto err;
+		}
+
+		/* validate the first packet has a valid frame index */
+		if (hdr->frame_index != 0) {
+			++port->stats.rx_missed_errors;
+			goto err;
+		}
+
+		BUILD_BUG_ON(TBT_NET_RX_HDR_SIZE > TBT_RING_MAX_FRM_DATA_SZ);
+		if ((frame_count > 1) && (frame_size < TBT_NET_RX_HDR_SIZE)) {
+			++port->stats.rx_length_errors;
+			goto err;
+		}
+
+		addr = (u8 *)(hdr + 1);
+
+		/* check the packet can go through the filter */
+		if (is_multicast_ether_addr(addr)) {
+			if (!is_broadcast_ether_addr(addr)) {
+				if ((port->packet_filters &
+				     (BIT(PACKET_TYPE_PROMISCUOUS) |
+				      BIT(PACKET_TYPE_ALL_MULTICAST))) ||
+				    tbt_net_multicast_mac_set(
+					port->multicast_hash_table, addr))
+					res = GOOD_AS_FIRST_MULTICAST_FRAME;
+				else
+					goto err;
+			}
+		} else if (!(port->packet_filters &
+			     (BIT(PACKET_TYPE_PROMISCUOUS) |
+			      BIT(PACKET_TYPE_UNICAST_PROMISCUOUS))) &&
+			   !ether_addr_equal(port->net_dev->dev_addr, addr)) {
+			goto err;
+		}
+
+		*size = frame_size;
+		*count = frame_count;
+		*id = le16_to_cpu(hdr->frame_id);
+	}
+
+#if (PREFETCH_STRIDE < 128)
+	prefetch((u8 *)hdr + PREFETCH_STRIDE);
+#endif
+
+	return res;
+
+err:
+	rx_ring->next_to_clean = (frame_num + 1) & (TBT_NET_NUM_RX_BUFS - 1);
+	return FRAME_ERROR;
+}
+
+static inline unsigned int tbt_net_max_frm_data_size(
+						__maybe_unused u32 frame_size)
+{
+#if (TBT_NUM_FRAMES_PER_PAGE > 1)
+	return ALIGN(frame_size + sizeof(struct tbt_frame_header),
+		     L1_CACHE_BYTES) -
+	       sizeof(struct tbt_frame_header);
+#else
+	return TBT_RING_MAX_FRM_DATA_SZ;
+#endif
+}
+
+static int tbt_net_poll(struct napi_struct *napi, int budget)
+{
+	struct tbt_port *port = container_of(napi, struct tbt_port, napi);
+	void __iomem *reg = TBT_RING_CONS_PROD_REG(port->nhi_ctxt->iobase,
+						   REG_RX_RING_BASE,
+						   port->local_path);
+	struct tbt_desc_ring *rx_ring = &port->rx_ring;
+	u16 cleaned_count = TBT_NUM_BUFS_BETWEEN(rx_ring->last_allocated,
+						 rx_ring->next_to_clean,
+						 TBT_NET_NUM_RX_BUFS);
+	unsigned long flags;
+	int rx_packets = 0;
+
+loop:
+	while (likely(rx_packets < budget)) {
+		struct sk_buff *skb;
+		enum frame_status status;
+		bool multicast = false;
+		u32 frame_count = 0, size;
+		u16 j, frame_id;
+		int i;
+
+		/*
+		 * return some buffers to hardware, one at a time is too slow
+		 * so allocate  TBT_NET_RX_BUFFER_WRITE buffers at the same time
+		 */
+		if (cleaned_count >= TBT_NET_RX_BUFFER_WRITE) {
+			tbt_net_alloc_rx_buffers(&port->nhi_ctxt->pdev->dev,
+						 rx_ring, cleaned_count, reg,
+						 GFP_ATOMIC);
+			cleaned_count = 0;
+		}
+
+		status = tbt_net_check_frame(port, rx_ring->next_to_clean,
+					     &frame_count, 0, &frame_id,
+					     &size);
+		if (status == FRAME_NOT_READY)
+			break;
+
+		if (status == FRAME_ERROR) {
+			++cleaned_count;
+			continue;
+		}
+
+		multicast = (status == GOOD_AS_FIRST_MULTICAST_FRAME);
+
+		/*
+		 *  i is incremented up to the frame_count frames received,
+		 *  j cyclicly goes over the location from the next frame
+		 *  to clean in the ring
+		 */
+		j = (rx_ring->next_to_clean + 1);
+		j &= (TBT_NET_NUM_RX_BUFS - 1);
+		for (i = 1; i < frame_count; ++i) {
+			status = tbt_net_check_frame(port, j, &frame_count, i,
+						     &frame_id, &size);
+			if (status == FRAME_NOT_READY)
+				goto out;
+
+			j = (j + 1) & (TBT_NET_NUM_RX_BUFS - 1);
+
+			/* if a new frame is found, start over */
+			if (status == GOOD_AS_FIRST_FRAME ||
+			    status == GOOD_AS_FIRST_MULTICAST_FRAME) {
+				multicast = (status ==
+					     GOOD_AS_FIRST_MULTICAST_FRAME);
+				cleaned_count += i;
+				i = 0;
+				continue;
+			}
+
+			if (status == FRAME_ERROR) {
+				cleaned_count += (i + 1);
+				goto loop;
+			}
+		}
+
+		/* allocate a skb to store the frags */
+		skb = netdev_alloc_skb_ip_align(port->net_dev,
+						TBT_NET_RX_HDR_SIZE);
+		if (unlikely(!skb))
+			break;
+
+		/*
+		 * we will be copying header into skb->data in
+		 * tbt_net_pull_tail so it is in our interest to prefetch
+		 * it now to avoid a possible cache miss
+		 */
+		prefetchw(skb->data);
+
+		/*
+		 * if overall size of packet smaller than TBT_NET_RX_HDR_SIZE
+		 * which is a small buffer size we decided to allocate
+		 * as the base to RX
+		 */
+		if (size <= TBT_NET_RX_HDR_SIZE) {
+			struct tbt_buffer *buf =
+				&(rx_ring->buffers[rx_ring->next_to_clean]);
+			u8 *va = page_address(buf->page) + buf->page_offset +
+				 sizeof(struct tbt_frame_header);
+
+			memcpy(__skb_put(skb, size), va,
+			       ALIGN(size, sizeof(long)));
+
+			/*
+			 * Reuse buffer as-is,
+			 * just make sure it is local
+			 * Access to local memory is faster than non-local
+			 * memory so let's reuse.
+			 * If not local, let's free it and reallocate later.
+			 */
+			if (likely(page_to_nid(buf->page) == numa_node_id()))
+				/* sync the buffer for use by the device */
+				dma_sync_single_range_for_device(
+						&port->nhi_ctxt->pdev->dev,
+						buf->dma, buf->page_offset,
+						TBT_RING_MAX_FRAME_SIZE,
+						DMA_FROM_DEVICE);
+			else {
+				/* this page cannot be reused so discard it */
+				put_page(buf->page);
+				buf->page = NULL;
+				dma_unmap_page(&port->nhi_ctxt->pdev->dev,
+					       buf->dma, PAGE_SIZE,
+					       DMA_FROM_DEVICE);
+			}
+			rx_ring->next_to_clean = (rx_ring->next_to_clean + 1) &
+						 (TBT_NET_NUM_RX_BUFS - 1);
+		} else {
+			for (i = 0; i < frame_count;  ++i) {
+				struct tbt_buffer *buf = &(rx_ring->buffers[
+						rx_ring->next_to_clean]);
+				struct tbt_frame_header *hdr =
+						page_address(buf->page) +
+						buf->page_offset;
+				u32 frm_size = le32_to_cpu(hdr->frame_size);
+
+				unsigned int truesize =
+					tbt_net_max_frm_data_size(frm_size);
+
+				/* add frame to skb struct */
+				skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
+						buf->page,
+						sizeof(struct tbt_frame_header)
+							+ buf->page_offset,
+						frm_size, truesize);
+
+#if (TBT_NUM_FRAMES_PER_PAGE > 1)
+				/* move offset up to the next cache line */
+				buf->page_offset += (truesize +
+					sizeof(struct tbt_frame_header));
+
+				/*
+				 * we can reuse buffer if there is space
+				 * available and it is local
+				 */
+				if (page_to_nid(buf->page) == numa_node_id()
+				    && buf->page_offset <=
+					PAGE_SIZE - TBT_RING_MAX_FRAME_SIZE) {
+					/*
+					 * bump ref count on page before
+					 * it is given to the stack
+					 */
+					get_page(buf->page);
+					/*
+					 * sync the buffer for use by the
+					 * device
+					 */
+					dma_sync_single_range_for_device(
+						&port->nhi_ctxt->pdev->dev,
+						buf->dma, buf->page_offset,
+						TBT_RING_MAX_FRAME_SIZE,
+						DMA_FROM_DEVICE);
+				} else
+#endif
+				{
+					buf->page = NULL;
+					dma_unmap_page(
+						&port->nhi_ctxt->pdev->dev,
+						buf->dma, PAGE_SIZE,
+						DMA_FROM_DEVICE);
+				}
+
+				rx_ring->next_to_clean =
+						(rx_ring->next_to_clean + 1) &
+						(TBT_NET_NUM_RX_BUFS - 1);
+			}
+			/*
+			 * place header from the first
+			 * fragment in linear portion of buffer
+			 */
+			tbt_net_pull_tail(skb);
+		}
+
+		/*
+		 * The Thunderbolt medium doesn't have any restriction on
+		 * minimum frame size, thus doesn't need any padding in
+		 * transmit.
+		 * The network stack accepts Runt Ethernet frames,
+		 * therefor there is neither padding in receive.
+		 */
+
+		skb->protocol = eth_type_trans(skb, port->net_dev);
+		napi_gro_receive(&port->napi, skb);
+
+		++rx_packets;
+		port->stats.rx_bytes += size;
+		if (multicast)
+			++port->stats.multicast;
+		cleaned_count += frame_count;
+	}
+
+out:
+	port->stats.rx_packets += rx_packets;
+
+	if (cleaned_count)
+		tbt_net_alloc_rx_buffers(&port->nhi_ctxt->pdev->dev,
+					 rx_ring, cleaned_count, reg,
+					 GFP_ATOMIC);
+
+	/* If all work not completed, return budget and keep polling */
+	if (rx_packets >= budget)
+		return budget;
+
+	/* Work is done so exit the polling mode and re-enable the interrupt */
+	napi_complete(napi);
+
+	spin_lock_irqsave(&port->nhi_ctxt->lock, flags);
+	/* enable RX interrupt */
+	RING_INT_ENABLE_RX(port->nhi_ctxt->iobase, port->local_path,
+			   port->nhi_ctxt->num_paths);
+
+	spin_unlock_irqrestore(&port->nhi_ctxt->lock, flags);
+
+	return 0;
+}
+
+static int tbt_net_open(struct net_device *net_dev)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+	int res = 0;
+	int i, j;
+
+	/* change link state to off until path establishment finishes */
+	netif_carrier_off(net_dev);
+
+	/*
+	 * if we previously succeeded to allocate msix entries,
+	 * now request IRQ for them:
+	 *  2=tx data port 0,
+	 *  3=rx data port 0,
+	 *  4=tx data port 1,
+	 *  5=rx data port 1,
+	 *  ...
+	 *  if not, if msi is used, nhi_msi will handle icm & data paths
+	 */
+	if (port->nhi_ctxt->msix_entries) {
+		char name[] = "tbt-net-xx-xx";
+
+		scnprintf(name, sizeof(name), "tbt-net-rx-%02u", port->num);
+		res = devm_request_irq(&port->nhi_ctxt->pdev->dev,
+			port->nhi_ctxt->msix_entries[3+(port->num*2)].vector,
+			tbt_net_rx_msix, 0, name, port);
+		if (res) {
+			netif_err(port, ifup, net_dev, "request_irq %s failed %d\n",
+				  name, res);
+			goto out;
+		}
+		name[8] = 't';
+		res = devm_request_irq(&port->nhi_ctxt->pdev->dev,
+			port->nhi_ctxt->msix_entries[2+(port->num*2)].vector,
+			tbt_net_tx_msix, 0, name, port);
+		if (res) {
+			netif_err(port, ifup, net_dev, "request_irq %s failed %d\n",
+				  name, res);
+			goto request_irq_failure;
+		}
+	}
+	/*
+	 * Verifying that all buffer sizes are well defined.
+	 * Starting with frame(s) will not tip over the
+	 * page boundary
+	 */
+	BUILD_BUG_ON(TBT_NUM_FRAMES_PER_PAGE < 1);
+	/*
+	 * Just to make sure we have enough place for containing
+	 * 3 max MTU packets for TX
+	 */
+	BUILD_BUG_ON((TBT_NET_NUM_TX_BUFS * TBT_RING_MAX_FRAME_SIZE) <
+		     (TBT_NET_MTU * 3));
+	/* make sure the number of TX Buffers is power of 2 */
+	BUILD_BUG_ON_NOT_POWER_OF_2(TBT_NET_NUM_TX_BUFS);
+	/*
+	 * Just to make sure we have enough place for containing
+	 * 3 max MTU packets for RX
+	 */
+	BUILD_BUG_ON((TBT_NET_NUM_RX_BUFS * TBT_RING_MAX_FRAME_SIZE) <
+		     (TBT_NET_MTU * 3));
+	/* make sure the number of RX Buffers is power of 2 */
+	BUILD_BUG_ON_NOT_POWER_OF_2(TBT_NET_NUM_RX_BUFS);
+
+	port->rx_ring.last_allocated = TBT_NET_NUM_RX_BUFS - 1;
+
+	port->tx_ring.buffers = vzalloc(TBT_NET_NUM_TX_BUFS *
+					sizeof(struct tbt_buffer));
+	if (!port->tx_ring.buffers)
+		goto ring_alloc_failure;
+	port->rx_ring.buffers = vzalloc(TBT_NET_NUM_RX_BUFS *
+					sizeof(struct tbt_buffer));
+	if (!port->rx_ring.buffers)
+		goto ring_alloc_failure;
+
+	/*
+	 * Allocate TX and RX descriptors
+	 * if the total size is less than a page, do a central allocation
+	 * Otherwise, split TX and RX
+	 */
+	if (TBT_NET_SIZE_TOTAL_DESCS <= PAGE_SIZE) {
+		port->tx_ring.desc = dmam_alloc_coherent(
+				&port->nhi_ctxt->pdev->dev,
+				TBT_NET_SIZE_TOTAL_DESCS,
+				&port->tx_ring.dma,
+				GFP_KERNEL | __GFP_ZERO);
+		if (!port->tx_ring.desc)
+			goto ring_alloc_failure;
+		/* RX starts where TX finishes */
+		port->rx_ring.desc = &port->tx_ring.desc[TBT_NET_NUM_TX_BUFS];
+		port->rx_ring.dma = port->tx_ring.dma +
+			(TBT_NET_NUM_TX_BUFS * sizeof(struct tbt_buf_desc));
+	} else {
+		port->tx_ring.desc = dmam_alloc_coherent(
+				&port->nhi_ctxt->pdev->dev,
+				TBT_NET_NUM_TX_BUFS *
+						sizeof(struct tbt_buf_desc),
+				&port->tx_ring.dma,
+				GFP_KERNEL | __GFP_ZERO);
+		if (!port->tx_ring.desc)
+			goto ring_alloc_failure;
+		port->rx_ring.desc = dmam_alloc_coherent(
+				&port->nhi_ctxt->pdev->dev,
+				TBT_NET_NUM_RX_BUFS *
+						sizeof(struct tbt_buf_desc),
+				&port->rx_ring.dma,
+				GFP_KERNEL | __GFP_ZERO);
+		if (!port->rx_ring.desc)
+			goto rx_desc_alloc_failure;
+	}
+
+	/* allocate TX buffers and configure the descriptors */
+	for (i = 0; i < TBT_NET_NUM_TX_BUFS; i++) {
+		port->tx_ring.buffers[i].hdr = dma_alloc_coherent(
+			&port->nhi_ctxt->pdev->dev,
+			TBT_NUM_FRAMES_PER_PAGE * TBT_RING_MAX_FRAME_SIZE,
+			&port->tx_ring.buffers[i].dma,
+			GFP_KERNEL);
+		if (!port->tx_ring.buffers[i].hdr)
+			goto buffers_alloc_failure;
+
+		port->tx_ring.desc[i].phys =
+				cpu_to_le64(port->tx_ring.buffers[i].dma);
+		port->tx_ring.desc[i].attributes =
+				cpu_to_le32(DESC_ATTR_REQ_STS |
+					    TBT_NET_DESC_ATTR_SOF_EOF);
+
+		/*
+		 * In case the page is bigger than the frame size,
+		 * make the next buffer descriptor points
+		 * on the next frame memory address within the page
+		 */
+		for (i++, j = 1; (i < TBT_NET_NUM_TX_BUFS) &&
+				 (j < TBT_NUM_FRAMES_PER_PAGE); i++, j++) {
+			port->tx_ring.buffers[i].dma =
+				port->tx_ring.buffers[i - 1].dma +
+				TBT_RING_MAX_FRAME_SIZE;
+			port->tx_ring.buffers[i].hdr =
+				(void *)(port->tx_ring.buffers[i - 1].hdr) +
+				TBT_RING_MAX_FRAME_SIZE;
+			/* move the next offset i.e. TBT_RING_MAX_FRAME_SIZE */
+			port->tx_ring.buffers[i].page_offset =
+				port->tx_ring.buffers[i - 1].page_offset +
+				TBT_RING_MAX_FRAME_SIZE;
+			port->tx_ring.desc[i].phys =
+				cpu_to_le64(port->tx_ring.buffers[i].dma);
+			port->tx_ring.desc[i].attributes =
+				cpu_to_le32(DESC_ATTR_REQ_STS |
+					    TBT_NET_DESC_ATTR_SOF_EOF);
+		}
+		i--;
+	}
+
+	port->negotiation_status =
+			BIT(port->nhi_ctxt->net_devices[port->num].medium_sts);
+	if (port->negotiation_status == BIT(MEDIUM_READY_FOR_CONNECTION)) {
+		port->login_retry_count = 0;
+		queue_delayed_work(port->nhi_ctxt->net_workqueue,
+				   &port->login_retry_work, 0);
+	}
+
+	netif_info(port, ifup, net_dev, "Thunderbolt(TM) Networking port %u - ready for ThunderboltIP negotiation\n",
+		   port->num);
+	return 0;
+
+buffers_alloc_failure:
+	/*
+	 * Rollback the Tx buffers that were already allocated
+	 * until the failure
+	 */
+	for (i--; i >= 0; i--) {
+		/* free only for first buffer allocation */
+		if (port->tx_ring.buffers[i].page_offset == 0)
+			dma_free_coherent(&port->nhi_ctxt->pdev->dev,
+					  TBT_NUM_FRAMES_PER_PAGE *
+						TBT_RING_MAX_FRAME_SIZE,
+					  port->tx_ring.buffers[i].hdr,
+					  port->tx_ring.buffers[i].dma);
+		port->tx_ring.buffers[i].hdr = NULL;
+	}
+	/*
+	 * For central allocation, free all
+	 * otherwise free RX and then TX separately
+	 */
+	if (TBT_NET_SIZE_TOTAL_DESCS <= PAGE_SIZE) {
+		dmam_free_coherent(&port->nhi_ctxt->pdev->dev,
+				   TBT_NET_SIZE_TOTAL_DESCS,
+				   port->tx_ring.desc,
+				   port->tx_ring.dma);
+		port->rx_ring.desc = NULL;
+	} else {
+		dmam_free_coherent(&port->nhi_ctxt->pdev->dev,
+				   TBT_NET_NUM_RX_BUFS *
+						sizeof(struct tbt_buf_desc),
+				   port->rx_ring.desc,
+				   port->rx_ring.dma);
+		port->rx_ring.desc = NULL;
+rx_desc_alloc_failure:
+		dmam_free_coherent(&port->nhi_ctxt->pdev->dev,
+				   TBT_NET_NUM_TX_BUFS *
+						sizeof(struct tbt_buf_desc),
+				   port->tx_ring.desc,
+				   port->tx_ring.dma);
+	}
+	port->tx_ring.desc = NULL;
+ring_alloc_failure:
+	vfree(port->tx_ring.buffers);
+	port->tx_ring.buffers = NULL;
+	vfree(port->rx_ring.buffers);
+	port->rx_ring.buffers = NULL;
+	res = -ENOMEM;
+	netif_err(port, ifup, net_dev, "Thunderbolt(TM) Networking port %u - unable to allocate memory\n",
+		  port->num);
+
+	if (!port->nhi_ctxt->msix_entries)
+		goto out;
+
+	devm_free_irq(&port->nhi_ctxt->pdev->dev,
+		      port->nhi_ctxt->msix_entries[2 + (port->num * 2)].vector,
+		      port);
+request_irq_failure:
+	devm_free_irq(&port->nhi_ctxt->pdev->dev,
+		      port->nhi_ctxt->msix_entries[3 + (port->num * 2)].vector,
+		      port);
+out:
+	return res;
+}
+
+static int tbt_net_close(struct net_device *net_dev)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+	int i;
+
+	/*
+	 * Close connection, disable rings, flow controls
+	 * and interrupts
+	 */
+	tbt_net_tear_down(net_dev, !(port->negotiation_status &
+				     BIT(RECEIVE_LOGOUT)));
+
+	cancel_work_sync(&port->login_response_work);
+	cancel_work_sync(&port->logout_work);
+	cancel_work_sync(&port->status_reply_work);
+	cancel_work_sync(&port->approve_inter_domain_work);
+
+	/* Rollback the Tx buffers that were allocated */
+	for (i = 0; i < TBT_NET_NUM_TX_BUFS; i++) {
+		if (port->tx_ring.buffers[i].page_offset == 0)
+			dma_free_coherent(&port->nhi_ctxt->pdev->dev,
+					  TBT_NUM_FRAMES_PER_PAGE *
+						TBT_RING_MAX_FRAME_SIZE,
+					  port->tx_ring.buffers[i].hdr,
+					  port->tx_ring.buffers[i].dma);
+		port->tx_ring.buffers[i].hdr = NULL;
+	}
+	/* Unmap the Rx buffers that were allocated */
+	for (i = 0; i < TBT_NET_NUM_RX_BUFS; i++)
+		if (port->rx_ring.buffers[i].page) {
+			put_page(port->rx_ring.buffers[i].page);
+			port->rx_ring.buffers[i].page = NULL;
+			dma_unmap_page(&port->nhi_ctxt->pdev->dev,
+				       port->rx_ring.buffers[i].dma, PAGE_SIZE,
+				       DMA_FROM_DEVICE);
+		}
+
+	/*
+	 * For central allocation, free all
+	 * otherwise free RX and then TX separately
+	 */
+	if (TBT_NET_SIZE_TOTAL_DESCS <= PAGE_SIZE) {
+		dmam_free_coherent(&port->nhi_ctxt->pdev->dev,
+				   TBT_NET_SIZE_TOTAL_DESCS,
+				   port->tx_ring.desc,
+				   port->tx_ring.dma);
+		port->rx_ring.desc = NULL;
+	} else {
+		dmam_free_coherent(&port->nhi_ctxt->pdev->dev,
+				   TBT_NET_NUM_RX_BUFS *
+						sizeof(struct tbt_buf_desc),
+				   port->rx_ring.desc,
+				   port->rx_ring.dma);
+		port->rx_ring.desc = NULL;
+		dmam_free_coherent(&port->nhi_ctxt->pdev->dev,
+				   TBT_NET_NUM_TX_BUFS *
+						sizeof(struct tbt_buf_desc),
+				   port->tx_ring.desc,
+				   port->tx_ring.dma);
+	}
+	port->tx_ring.desc = NULL;
+
+	vfree(port->tx_ring.buffers);
+	port->tx_ring.buffers = NULL;
+	vfree(port->rx_ring.buffers);
+	port->rx_ring.buffers = NULL;
+
+	devm_free_irq(&port->nhi_ctxt->pdev->dev,
+		      port->nhi_ctxt->msix_entries[3 + (port->num * 2)].vector,
+		      port);
+	devm_free_irq(&port->nhi_ctxt->pdev->dev,
+		      port->nhi_ctxt->msix_entries[2 + (port->num * 2)].vector,
+		      port);
+
+	netif_info(port, ifdown, net_dev, "Thunderbolt(TM) Networking port %u - is down\n",
+		   port->num);
+
+	return 0;
+}
+
+static bool tbt_net_xmit_csum(struct sk_buff *skb,
+			      struct tbt_desc_ring *tx_ring, u32 first,
+			      u32 last, u32 frame_count)
+{
+
+	struct tbt_frame_header *hdr = tx_ring->buffers[first].hdr;
+	__wsum wsum = (__force __wsum)htonl(skb->len -
+					    skb_transport_offset(skb));
+	int offset = skb_transport_offset(skb);
+	__sum16 *tucso;  /* TCP UDP Checksum Segment Offset */
+	__be16 protocol = skb->protocol;
+	u8 *dest = (u8 *)(hdr + 1);
+	int len;
+
+	if (skb->ip_summed != CHECKSUM_PARTIAL) {
+		for (; first != last;
+			first = (first + 1) & (TBT_NET_NUM_TX_BUFS - 1)) {
+			hdr = tx_ring->buffers[first].hdr;
+			hdr->frame_count = cpu_to_le32(frame_count);
+		}
+		return true;
+	}
+
+	if (protocol == htons(ETH_P_8021Q)) {
+		struct vlan_hdr *vhdr, vh;
+
+		vhdr = skb_header_pointer(skb, ETH_HLEN, sizeof(vh), &vh);
+		if (!vhdr)
+			return false;
+
+		protocol = vhdr->h_vlan_encapsulated_proto;
+	}
+
+	/*
+	 * Data points on the beginning of packet.
+	 * Check is the checksum absolute place in the
+	 * packet.
+	 * ipcso will update IP checksum.
+	 * tucso will update TCP/UPD checksum.
+	 */
+	if (protocol == htons(ETH_P_IP)) {
+		__sum16 *ipcso = (__sum16 *)(dest +
+			((u8 *)&(ip_hdr(skb)->check) - skb->data));
+
+		*ipcso = 0;
+		*ipcso = ip_fast_csum(dest + skb_network_offset(skb),
+				      ip_hdr(skb)->ihl);
+		if (ip_hdr(skb)->protocol == IPPROTO_TCP)
+			tucso = (__sum16 *)(dest +
+				((u8 *)&(tcp_hdr(skb)->check) - skb->data));
+		else if (ip_hdr(skb)->protocol == IPPROTO_UDP)
+			tucso = (__sum16 *)(dest +
+				((u8 *)&(udp_hdr(skb)->check) - skb->data));
+		else
+			return false;
+
+		*tucso = ~csum_tcpudp_magic(ip_hdr(skb)->saddr,
+					    ip_hdr(skb)->daddr, 0,
+					    ip_hdr(skb)->protocol, 0);
+	} else if (skb_is_gso(skb)) {
+		if (skb_is_gso_v6(skb)) {
+			tucso = (__sum16 *)(dest +
+				((u8 *)&(tcp_hdr(skb)->check) - skb->data));
+			*tucso = ~csum_ipv6_magic(&ipv6_hdr(skb)->saddr,
+						  &ipv6_hdr(skb)->daddr,
+						  0, IPPROTO_TCP, 0);
+		} else if ((protocol == htons(ETH_P_IPV6)) &&
+			   (skb_shinfo(skb)->gso_type & SKB_GSO_UDP)) {
+			tucso = (__sum16 *)(dest +
+				((u8 *)&(udp_hdr(skb)->check) - skb->data));
+			*tucso = ~csum_ipv6_magic(&ipv6_hdr(skb)->saddr,
+						  &ipv6_hdr(skb)->daddr,
+						  0, IPPROTO_UDP, 0);
+		} else {
+			return false;
+		}
+	} else if (protocol == htons(ETH_P_IPV6)) {
+		tucso = (__sum16 *)(dest + skb_checksum_start_offset(skb) +
+				    skb->csum_offset);
+		*tucso = ~csum_ipv6_magic(&ipv6_hdr(skb)->saddr,
+					  &ipv6_hdr(skb)->daddr,
+					  0, ipv6_hdr(skb)->nexthdr, 0);
+	} else {
+		return false;
+	}
+
+	/* First frame was headers, rest of the frames is data */
+	for (; first != last; first = (first + 1) & (TBT_NET_NUM_TX_BUFS - 1),
+								offset = 0) {
+		hdr = tx_ring->buffers[first].hdr;
+		dest = (u8 *)(hdr + 1) + offset;
+		len = le32_to_cpu(hdr->frame_size) - offset;
+		wsum = csum_partial(dest, len, wsum);
+		hdr->frame_count = cpu_to_le32(frame_count);
+	}
+	*tucso = csum_fold(wsum);
+
+	return true;
+}
+
+static netdev_tx_t tbt_net_xmit_frame(struct sk_buff *skb,
+				      struct net_device *net_dev)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+	void __iomem *iobase = port->nhi_ctxt->iobase;
+	void __iomem *reg = TBT_RING_CONS_PROD_REG(iobase,
+						   REG_TX_RING_BASE,
+						   port->local_path);
+	struct tbt_desc_ring *tx_ring = &port->tx_ring;
+	struct tbt_frame_header *hdr;
+	u32 prod_cons, prod, cons, first;
+	/* len equivalent to the fragment length */
+	unsigned int len = skb_headlen(skb);
+	/* data_len is overall packet length */
+	unsigned int data_len = skb->len;
+	u32 frm_idx, frag_num = 0;
+	const u8 *src = skb->data;
+	bool unmap = false;
+	__le32 *attr;
+	u8 *dest;
+
+	if (unlikely(data_len == 0 || data_len > TBT_NET_MTU))
+		goto invalid_packet;
+
+	prod_cons = ioread32(reg);
+	prod = TBT_REG_RING_PROD_EXTRACT(prod_cons);
+	cons = TBT_REG_RING_CONS_EXTRACT(prod_cons);
+	if (prod >= TBT_NET_NUM_TX_BUFS || cons >= TBT_NET_NUM_TX_BUFS)
+		goto tx_error;
+
+	if (data_len > (TBT_NUM_BUFS_BETWEEN(prod, cons, TBT_NET_NUM_TX_BUFS) *
+			TBT_RING_MAX_FRM_DATA_SZ)) {
+		unsigned long flags;
+
+		netif_stop_queue(net_dev);
+
+		spin_lock_irqsave(&port->nhi_ctxt->lock, flags);
+		/*
+		 * Enable TX interrupt to be notified about available buffers
+		 * and restart transmission upon this.
+		 */
+		RING_INT_ENABLE_TX(iobase, port->local_path);
+		spin_unlock_irqrestore(&port->nhi_ctxt->lock, flags);
+
+		return NETDEV_TX_BUSY;
+	}
+
+	first = prod;
+	attr = &tx_ring->desc[prod].attributes;
+	hdr = tx_ring->buffers[prod].hdr;
+	dest = (u8 *)(hdr + 1);
+	/* if overall packet is bigger than the frame data size */
+	for (frm_idx = 0; data_len > TBT_RING_MAX_FRM_DATA_SZ; ++frm_idx) {
+		u32 size_left = TBT_RING_MAX_FRM_DATA_SZ;
+
+		*attr &= cpu_to_le32(~(DESC_ATTR_LEN_MASK |
+				      DESC_ATTR_INT_EN |
+				      DESC_ATTR_DESC_DONE));
+		hdr->frame_size = cpu_to_le32(TBT_RING_MAX_FRM_DATA_SZ);
+		hdr->frame_index = cpu_to_le16(frm_idx);
+		hdr->frame_id = cpu_to_le16(port->frame_id);
+
+		do {
+			if (len > size_left) {
+				/*
+				 * Copy data onto tx buffer data with full
+				 * frame size then break
+				 * and go to next frame
+				 */
+				memcpy(dest, src, size_left);
+				len -= size_left;
+				dest += size_left;
+				src += size_left;
+				break;
+			}
+
+			memcpy(dest, src, len);
+			size_left -= len;
+			dest += len;
+
+			if (unmap) {
+				kunmap_atomic((void *)src);
+				unmap = false;
+			}
+			/*
+			 * Ensure all fragments have been processed
+			 */
+			if (frag_num < skb_shinfo(skb)->nr_frags) {
+				const skb_frag_t *frag =
+					&(skb_shinfo(skb)->frags[frag_num]);
+				len = skb_frag_size(frag);
+				/* map and then unmap quickly */
+				src = kmap_atomic(skb_frag_page(frag)) +
+							frag->page_offset;
+				unmap = true;
+				++frag_num;
+			} else if (unlikely(size_left > 0)) {
+				goto invalid_packet;
+			}
+		} while (size_left > 0);
+
+		data_len -= TBT_RING_MAX_FRM_DATA_SZ;
+		prod = (prod + 1) & (TBT_NET_NUM_TX_BUFS - 1);
+		attr = &tx_ring->desc[prod].attributes;
+		hdr = tx_ring->buffers[prod].hdr;
+		dest = (u8 *)(hdr + 1);
+	}
+
+	*attr &= cpu_to_le32(~(DESC_ATTR_LEN_MASK | DESC_ATTR_DESC_DONE));
+	/* Enable the interrupts, for resuming from stop queue later (if so) */
+	*attr |= cpu_to_le32(DESC_ATTR_INT_EN |
+		(((sizeof(struct tbt_frame_header) + data_len) <<
+		  DESC_ATTR_LEN_SHIFT) & DESC_ATTR_LEN_MASK));
+	hdr->frame_size = cpu_to_le32(data_len);
+	hdr->frame_index = cpu_to_le16(frm_idx);
+	hdr->frame_id = cpu_to_le16(port->frame_id);
+
+	/* In case  the remaining data_len is smaller than a frame */
+	while (len < data_len) {
+		memcpy(dest, src, len);
+		data_len -= len;
+		dest += len;
+
+		if (unmap) {
+			kunmap_atomic((void *)src);
+			unmap = false;
+		}
+
+		if (frag_num < skb_shinfo(skb)->nr_frags) {
+			const skb_frag_t *frag =
+					&(skb_shinfo(skb)->frags[frag_num]);
+			len = skb_frag_size(frag);
+			src = kmap_atomic(skb_frag_page(frag)) +
+							frag->page_offset;
+			unmap = true;
+			++frag_num;
+		} else if (unlikely(data_len > 0)) {
+			goto invalid_packet;
+		}
+	}
+	memcpy(dest, src, data_len);
+	if (unmap) {
+		kunmap_atomic((void *)src);
+		unmap = false;
+	}
+
+	++frm_idx;
+	prod = (prod + 1) & (TBT_NET_NUM_TX_BUFS - 1);
+
+	if (!tbt_net_xmit_csum(skb, tx_ring, first, prod, frm_idx))
+		goto invalid_packet;
+
+	if (port->match_frame_id)
+		++port->frame_id;
+
+	prod_cons &= ~REG_RING_PROD_MASK;
+	prod_cons |= (prod << REG_RING_PROD_SHIFT) & REG_RING_PROD_MASK;
+	wmb(); /* make sure producer update is done after buffers are ready */
+	iowrite32(prod_cons, reg);
+
+	++port->stats.tx_packets;
+	port->stats.tx_bytes += skb->len;
+
+	dev_consume_skb_any(skb);
+	return NETDEV_TX_OK;
+
+invalid_packet:
+	netif_err(port, tx_err, net_dev, "port %u invalid transmit packet\n",
+		  port->num);
+tx_error:
+	++port->stats.tx_errors;
+	dev_kfree_skb_any(skb);
+	return NETDEV_TX_OK;
 }
 
+static void tbt_net_set_rx_mode(struct net_device *net_dev)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+	struct netdev_hw_addr *ha;
+
+	if (net_dev->flags & IFF_PROMISC)
+		port->packet_filters |= BIT(PACKET_TYPE_PROMISCUOUS);
+	else
+		port->packet_filters &= ~BIT(PACKET_TYPE_PROMISCUOUS);
+	if (net_dev->flags & IFF_ALLMULTI)
+		port->packet_filters |= BIT(PACKET_TYPE_ALL_MULTICAST);
+	else
+		port->packet_filters &= ~BIT(PACKET_TYPE_ALL_MULTICAST);
+
+	/* if you have more than a single MAC address */
+	if (netdev_uc_count(net_dev) > 1)
+		port->packet_filters |= BIT(PACKET_TYPE_UNICAST_PROMISCUOUS);
+	/* if have a single MAC address */
+	else if (netdev_uc_count(net_dev) == 1) {
+		netdev_for_each_uc_addr(ha, net_dev)
+			/* checks whether the MAC is what we set */
+			if (ether_addr_equal(ha->addr, net_dev->dev_addr))
+				port->packet_filters &=
+					~BIT(PACKET_TYPE_UNICAST_PROMISCUOUS);
+			else
+				port->packet_filters |=
+					BIT(PACKET_TYPE_UNICAST_PROMISCUOUS);
+	} else {
+		port->packet_filters &= ~BIT(PACKET_TYPE_UNICAST_PROMISCUOUS);
+	}
+
+	/* Populate the multicast hash table with received MAC addresses */
+	memset(port->multicast_hash_table, 0,
+	       sizeof(port->multicast_hash_table));
+	netdev_for_each_mc_addr(ha, net_dev) {
+		u16 hash_val = TBT_NET_ETHER_ADDR_HASH(ha->addr);
+
+		port->multicast_hash_table[hash_val / BITS_PER_U32] |=
+						BIT(hash_val % BITS_PER_U32);
+	}
+
+}
+
+static struct rtnl_link_stats64 *tbt_net_get_stats64(
+					struct net_device *net_dev,
+					struct rtnl_link_stats64 *stats)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+
+	memset(stats, 0, sizeof(*stats));
+	stats->tx_packets = port->stats.tx_packets;
+	stats->tx_bytes = port->stats.tx_bytes;
+	stats->tx_errors = port->stats.tx_errors;
+	stats->rx_packets = port->stats.rx_packets;
+	stats->rx_bytes = port->stats.rx_bytes;
+	stats->rx_length_errors = port->stats.rx_length_errors;
+	stats->rx_over_errors = port->stats.rx_over_errors;
+	stats->rx_crc_errors = port->stats.rx_crc_errors;
+	stats->rx_missed_errors = port->stats.rx_missed_errors;
+	stats->rx_errors = stats->rx_length_errors + stats->rx_over_errors +
+			   stats->rx_crc_errors + stats->rx_missed_errors;
+	stats->multicast = port->stats.multicast;
+	return stats;
+}
+
+static int tbt_net_set_mac_address(struct net_device *net_dev, void *addr)
+{
+	struct sockaddr *saddr = addr;
+
+	if (!is_valid_ether_addr(saddr->sa_data))
+		return -EADDRNOTAVAIL;
+
+	memcpy(net_dev->dev_addr, saddr->sa_data, net_dev->addr_len);
+
+	return 0;
+}
+
+static int tbt_net_change_mtu(struct net_device *net_dev, int new_mtu)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+
+	/* MTU < 68 is an error and causes problems on some kernels */
+	if (new_mtu < 68 || new_mtu > (TBT_NET_MTU - ETH_HLEN))
+		return -EINVAL;
+
+	netif_info(port, probe, net_dev, "Thunderbolt(TM) Networking port %u - changing MTU from %u to %d\n",
+		   port->num, net_dev->mtu, new_mtu);
+
+	net_dev->mtu = new_mtu;
+
+	return 0;
+}
+
+static const struct net_device_ops tbt_netdev_ops = {
+	/* called when the network is up'ed */
+	.ndo_open		= tbt_net_open,
+	/* called when the network is down'ed */
+	.ndo_stop		= tbt_net_close,
+	.ndo_start_xmit		= tbt_net_xmit_frame,
+	.ndo_set_rx_mode	= tbt_net_set_rx_mode,
+	.ndo_get_stats64	= tbt_net_get_stats64,
+	.ndo_set_mac_address	= tbt_net_set_mac_address,
+	.ndo_change_mtu		= tbt_net_change_mtu,
+	.ndo_validate_addr	= eth_validate_addr,
+};
+
+static int tbt_net_get_settings(__maybe_unused struct net_device *net_dev,
+				struct ethtool_cmd *ecmd)
+{
+	ecmd->supported |= SUPPORTED_20000baseKR2_Full;
+	ecmd->advertising |= ADVERTISED_20000baseKR2_Full;
+	ecmd->autoneg = AUTONEG_DISABLE;
+	ecmd->transceiver = XCVR_INTERNAL;
+	ecmd->supported |= SUPPORTED_FIBRE;
+	ecmd->advertising |= ADVERTISED_FIBRE;
+	ecmd->port = PORT_FIBRE;
+	ethtool_cmd_speed_set(ecmd, SPEED_20000);
+	ecmd->duplex = DUPLEX_FULL;
+
+	return 0;
+}
+
+
+static u32 tbt_net_get_msglevel(struct net_device *net_dev)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+
+	return port->msg_enable;
+}
+
+static void tbt_net_set_msglevel(struct net_device *net_dev, u32 data)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+
+	port->msg_enable = data;
+}
+
+static void tbt_net_get_strings(__maybe_unused struct net_device *net_dev,
+				u32 stringset, u8 *data)
+{
+	if (stringset == ETH_SS_STATS)
+		memcpy(data, tbt_net_gstrings_stats,
+		       sizeof(tbt_net_gstrings_stats));
+}
+
+static void tbt_net_get_ethtool_stats(struct net_device *net_dev,
+				      __maybe_unused struct ethtool_stats *sts,
+				      u64 *data)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+
+	memcpy(data, &port->stats, sizeof(port->stats));
+}
+
+static int tbt_net_get_sset_count(__maybe_unused struct net_device *net_dev,
+				  int sset)
+{
+	if (sset == ETH_SS_STATS)
+		return sizeof(tbt_net_gstrings_stats) / ETH_GSTRING_LEN;
+	return -EOPNOTSUPP;
+}
+
+static void tbt_net_get_drvinfo(struct net_device *net_dev,
+				struct ethtool_drvinfo *drvinfo)
+{
+	struct tbt_port *port = netdev_priv(net_dev);
+
+	strlcpy(drvinfo->driver, "Thunderbolt(TM) Networking",
+		sizeof(drvinfo->driver));
+	strlcpy(drvinfo->version, DRV_VERSION, sizeof(drvinfo->version));
+
+	strlcpy(drvinfo->bus_info, pci_name(port->nhi_ctxt->pdev),
+		sizeof(drvinfo->bus_info));
+	drvinfo->n_stats = tbt_net_get_sset_count(net_dev, ETH_SS_STATS);
+}
+
+static const struct ethtool_ops tbt_net_ethtool_ops = {
+	.get_settings		= tbt_net_get_settings,
+	.get_drvinfo		= tbt_net_get_drvinfo,
+	.get_link		= ethtool_op_get_link,
+	.get_msglevel		= tbt_net_get_msglevel,
+	.set_msglevel		= tbt_net_set_msglevel,
+	.get_strings		= tbt_net_get_strings,
+	.get_ethtool_stats	= tbt_net_get_ethtool_stats,
+	.get_sset_count		= tbt_net_get_sset_count,
+};
+
 static inline int send_message(struct tbt_port *port, const char *func,
 				enum pdf_value pdf, u32 msg_len,
 				const void *msg)
@@ -496,6 +1920,10 @@ void negotiation_events(struct net_device *net_dev,
 		/* configure TX ring */
 		reg = iobase + REG_TX_RING_BASE +
 		      (port->local_path * REG_RING_STEP);
+		iowrite32(lower_32_bits(port->tx_ring.dma),
+			  reg + REG_RING_PHYS_LO_OFFSET);
+		iowrite32(upper_32_bits(port->tx_ring.dma),
+			  reg + REG_RING_PHYS_HI_OFFSET);
 
 		tx_ring_conf = (TBT_NET_NUM_TX_BUFS << REG_RING_SIZE_SHIFT) &
 				REG_RING_SIZE_MASK;
@@ -538,6 +1966,10 @@ void negotiation_events(struct net_device *net_dev,
 		 */
 		reg = iobase + REG_RX_RING_BASE +
 		      (port->local_path * REG_RING_STEP);
+		iowrite32(lower_32_bits(port->rx_ring.dma),
+			  reg + REG_RING_PHYS_LO_OFFSET);
+		iowrite32(upper_32_bits(port->rx_ring.dma),
+			  reg + REG_RING_PHYS_HI_OFFSET);
 
 		rx_ring_conf = (TBT_NET_NUM_RX_BUFS << REG_RING_SIZE_SHIFT) &
 				REG_RING_SIZE_MASK;
@@ -547,6 +1979,17 @@ void negotiation_events(struct net_device *net_dev,
 				REG_RING_BUF_SIZE_MASK;
 
 		iowrite32(rx_ring_conf, reg + REG_RING_SIZE_OFFSET);
+		/* allocate RX buffers and configure the descriptors */
+		if (!tbt_net_alloc_rx_buffers(&port->nhi_ctxt->pdev->dev,
+					      &port->rx_ring,
+					      TBT_NET_NUM_RX_BUFS,
+					      reg + REG_RING_CONS_PROD_OFFSET,
+					      GFP_KERNEL)) {
+			netif_err(port, link, net_dev, "Thunderbolt(TM) Networking port %u - no memory for receive buffers\n",
+				  port->num);
+			tbt_net_tear_down(net_dev, true);
+			break;
+		}
 
 		spin_lock_irqsave(&port->nhi_ctxt->lock, flags);
 		/* enable RX interrupt */
@@ -559,6 +2002,7 @@ void negotiation_events(struct net_device *net_dev,
 		netif_info(port, link, net_dev, "Thunderbolt(TM) Networking port %u - ready\n",
 			   port->num);
 
+		napi_enable(&port->napi);
 		netif_carrier_on(net_dev);
 		netif_start_queue(net_dev);
 		break;
@@ -769,15 +2213,42 @@ struct net_device *nhi_alloc_etherdev(struct tbt_nhi_ctxt *nhi_ctxt,
 	scnprintf(net_dev->name, sizeof(net_dev->name), "tbtnet%%dp%hhu",
 		  port_num);
 
+	net_dev->netdev_ops = &tbt_netdev_ops;
+
+	netif_napi_add(net_dev, &port->napi, tbt_net_poll, NAPI_POLL_WEIGHT);
+
+	net_dev->hw_features = NETIF_F_SG |
+			       NETIF_F_ALL_TSO |
+			       NETIF_F_UFO |
+			       NETIF_F_GRO |
+			       NETIF_F_IP_CSUM |
+			       NETIF_F_IPV6_CSUM;
+	net_dev->features = net_dev->hw_features;
+	if (nhi_ctxt->pci_using_dac)
+		net_dev->features |= NETIF_F_HIGHDMA;
+
 	INIT_DELAYED_WORK(&port->login_retry_work, login_retry);
 	INIT_WORK(&port->login_response_work, login_response);
 	INIT_WORK(&port->logout_work, logout);
 	INIT_WORK(&port->status_reply_work, status_reply);
 	INIT_WORK(&port->approve_inter_domain_work, approve_inter_domain);
 
+	net_dev->ethtool_ops = &tbt_net_ethtool_ops;
+
+	tbt_net_change_mtu(net_dev, TBT_NET_MTU - ETH_HLEN);
+
+	if (register_netdev(net_dev))
+		goto err_register;
+
+	netif_carrier_off(net_dev);
+
 	netif_info(port, probe, net_dev,
 		   "Thunderbolt(TM) Networking port %u - MAC Address: %pM\n",
 		   port_num, net_dev->dev_addr);
 
 	return net_dev;
+
+err_register:
+	free_netdev(net_dev);
+	return NULL;
 }
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v9 6/8] thunderbolt: Kconfig for Thunderbolt Networking
  2016-11-09 14:20 [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Amir Levy
                   ` (4 preceding siblings ...)
  2016-11-09 14:20 ` [PATCH v9 5/8] thunderbolt: Networking transmit and receive Amir Levy
@ 2016-11-09 14:20 ` Amir Levy
  2016-11-09 14:20 ` [PATCH v9 7/8] thunderbolt: Networking doc Amir Levy
                   ` (3 subsequent siblings)
  9 siblings, 0 replies; 28+ messages in thread
From: Amir Levy @ 2016-11-09 14:20 UTC (permalink / raw)
  To: gregkh
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang, Amir Levy

Update to the Kconfig Thunderbolt description to add
Thunderbolt networking as an option.
The menu item "Thunderbolt support" now offers:
"Apple Hardware Support" (existing)
        and/or
"Thunderbolt Networking" (new)

You can choose the driver for your platform or build both drivers -
each driver will detect if it can run on the specific platform.
If the Thunderbolt Networking option is chosen, Thunderbolt Networking
will be enabled between Linux non-Apple systems, macOS and
Windows based systems.
Thunderbolt Networking will not affect any other Thunderbolt feature that
was previous available to Linux users on either Apple or
non-Apple platforms.

Signed-off-by: Amir Levy <amir.jer.levy@intel.com>
---
 drivers/thunderbolt/Kconfig  | 27 +++++++++++++++++++++++----
 drivers/thunderbolt/Makefile |  3 ++-
 2 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/drivers/thunderbolt/Kconfig b/drivers/thunderbolt/Kconfig
index c121acc..376e5bb 100644
--- a/drivers/thunderbolt/Kconfig
+++ b/drivers/thunderbolt/Kconfig
@@ -1,13 +1,32 @@
-menuconfig THUNDERBOLT
-	tristate "Thunderbolt support for Apple devices"
+config THUNDERBOLT
+	tristate "Thunderbolt support"
 	depends on PCI
 	select CRC32
 	help
-	  Cactus Ridge Thunderbolt Controller driver
+	  Thunderbolt Controller driver
+
+if THUNDERBOLT
+
+config THUNDERBOLT_APPLE
+	tristate "Apple hardware support"
+	help
 	  This driver is required if you want to hotplug Thunderbolt devices on
 	  Apple hardware.
 
 	  Device chaining is currently not supported.
 
-	  To compile this driver a module, choose M here. The module will be
+	  To compile this driver as a module, choose M here. The module will be
 	  called thunderbolt.
+
+config THUNDERBOLT_ICM
+	tristate "Thunderbolt Networking"
+	help
+	  This driver is required if you want Thunderbolt Networking on
+	  non-Apple hardware.
+	  It creates a virtual Ethernet device that enables computer to
+	  computer communication over a Thunderbolt cable.
+
+	  To compile this driver as a module, choose M here. The module will be
+	  called thunderbolt_icm.
+
+endif
diff --git a/drivers/thunderbolt/Makefile b/drivers/thunderbolt/Makefile
index 5d1053c..b6aa6a3 100644
--- a/drivers/thunderbolt/Makefile
+++ b/drivers/thunderbolt/Makefile
@@ -1,3 +1,4 @@
-obj-${CONFIG_THUNDERBOLT} := thunderbolt.o
+obj-${CONFIG_THUNDERBOLT_APPLE} := thunderbolt.o
 thunderbolt-objs := nhi.o ctl.o tb.o switch.o cap.o path.o tunnel_pci.o eeprom.o
 
+obj-${CONFIG_THUNDERBOLT_ICM} += icm/
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v9 7/8] thunderbolt: Networking doc
  2016-11-09 14:20 [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Amir Levy
                   ` (5 preceding siblings ...)
  2016-11-09 14:20 ` [PATCH v9 6/8] thunderbolt: Kconfig for Thunderbolt Networking Amir Levy
@ 2016-11-09 14:20 ` Amir Levy
  2016-11-09 16:00   ` Greg KH
  2016-11-09 14:20 ` [PATCH v9 8/8] thunderbolt: Adding maintainer entry Amir Levy
                   ` (2 subsequent siblings)
  9 siblings, 1 reply; 28+ messages in thread
From: Amir Levy @ 2016-11-09 14:20 UTC (permalink / raw)
  To: gregkh
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang, Amir Levy

Adding Thunderbolt(TM) networking documentation.

Signed-off-by: Amir Levy <amir.jer.levy@intel.com>
---
 Documentation/00-INDEX                   |   2 +
 Documentation/thunderbolt/networking.txt | 132 +++++++++++++++++++++++++++++++
 2 files changed, 134 insertions(+)
 create mode 100644 Documentation/thunderbolt/networking.txt

diff --git a/Documentation/00-INDEX b/Documentation/00-INDEX
index 3acc4f1..0239e68 100644
--- a/Documentation/00-INDEX
+++ b/Documentation/00-INDEX
@@ -440,6 +440,8 @@ this_cpu_ops.txt
 	- List rationale behind and the way to use this_cpu operations.
 thermal/
 	- directory with information on managing thermal issues (CPU/temp)
+thunderbolt/
+	- directory with info regarding Thunderbolt.
 trace/
 	- directory with info on tracing technologies within linux
 unaligned-memory-access.txt
diff --git a/Documentation/thunderbolt/networking.txt b/Documentation/thunderbolt/networking.txt
new file mode 100644
index 0000000..88d1c12
--- /dev/null
+++ b/Documentation/thunderbolt/networking.txt
@@ -0,0 +1,132 @@
+Intel Thunderbolt(TM) Networking driver
+=======================================
+
+Copyright(c) 2013 - 2016 Intel Corporation.
+
+Contact Information:
+Intel Thunderbolt mailing list <thunderbolt-software@lists.01.org>
+Edited by Amir Levy <amir.jer.levy@intel.com>
+
+Overview
+========
+
+* The Thunderbolt Networking driver enables peer to peer networking on non-Apple
+  platforms running Linux.
+
+* The driver creates a virtual Ethernet device that enables computer to computer
+  communication over the Thunderbolt cable.
+
+* Using Thunderbolt Networking you can perform high speed file transfers between
+  computers, perform PC migrations and/or set up small workgroups with shared
+  storage without compromising any other Thunderbolt functionality.
+
+* The driver is located in drivers/thunderbolt/icm.
+
+* This driver will function only on non-Apple platforms with firmware based
+  Thunderbolt controllers that support Thunderbolt Networking.
+
+  +----------------+            +----------------+
+  |Host 1          |            |Host 2          |
+  |                |            |                |
+  |   +-------+    |            |    +-------+   |
+  |   |Network|    |            |    |Network|   |
+  |   |Stack  |    |            |    |Stack  |   |
+  |   +-------+    |            |    +-------+   |
+  |       ^        |            |        ^       |
+  |       |        |            |        |       |
+  |       v        |            |        v       |
+  | +-----------+  |            |  +-----------+ |
+  | |Thunderbolt|  |            |  |Thunderbolt| |
+  | |Networking |  |            |  |Networking | |
+  | |Driver     |  |            |  |Driver     | |
+  | +-----------+  |            |  +-----------+ |
+  |       ^        |            |        ^       |
+  |       |        |            |        |       |
+  |       v        |            |        v       |
+  | +-----------+  |            |  +-----------+ |
+  | |Thunderbolt|  |            |  |Thunderbolt| |
+  | |Controller |<-+------------+->|Controller | |
+  | |with ICM   |  |            |  |with ICM   | |
+  | |enabled    |  |            |  |enabled    | |
+  | +-----------+  |            |  +-----------+ |
+  +----------------+            +----------------+
+
+Files
+=====
+
+The following files are located in the drivers/thunderbolt/icm directory:
+
+- icm_nhi.c/h:	These files allow communication with the firmware (Intel
+  Connection Manager) based controller. They also create an interface for
+  netlink communication with a user space daemon.
+
+- net.c/net.h:	These files implement the 'eth' interface for the
+  Thunderbolt(TM) Networking.
+
+Interface to User Space
+=======================
+
+The interface to the user space module is implemented through a Generic Netlink.
+This is the communications protocol between the Thunderbolt driver and the user
+space application.
+
+Note that this interface mediates user space communication with ICM.
+(Existing Linux tools can be used to configure the network interface.)
+
+The Thunderbolt Daemon utilizes this interface to communicate with the driver.
+To be accessed by the user space module, both kernel and user space modules
+have to register with the same GENL_NAME.
+For the purpose of the Thunderbolt Network driver, "thunderbolt" is used.
+The registration is done at driver initialization time for all instances
+of the Thunderbolt controllers. The communication is carried through pre-defined
+Thunderbolt messages. Each specific message has a callback function that is
+called when the related message is received.
+
+Message Definitions:
+* NHI_CMD_UNSPEC: Not used.
+* NHI_CMD_SUBSCRIBE: Subscription request from daemon to driver to open the
+  communication channel.
+* NHI_CMD_UNSUBSCRIBE: Request from daemon to driver to unsubscribe and
+  to close communication channel.
+* NHI_CMD_QUERY_INFORMATION: Request information from the driver such as
+  driver version, FW version offset, number of ports in the controller
+  and DMA port.
+* NHI_CMD_MSG_TO_ICM: Message from user space module to FW.
+* NHI_CMD_MSG_FROM_ICM: Response from FW to user space module.
+* NHI_CMD_MAILBOX: Message that uses mailbox mechanism such as FW policy
+  changes or disconnect path.
+* NHI_CMD_APPROVE_TBT_NETWORKING: Request from user space module to FW to
+  establish path.
+* NHI_CMD_ICM_IN_SAFE_MODE: Indication that the FW has entered safe mode.
+
+Communication with Intel Connection Manager(ICM) Firmware
+=========================================================
+
+There are several circular buffers in Thunderbolt each using Direct Memory
+Access (DMA).
+
+Communication with ICM utilizes circular buffer ring #0. (The other rings are
+used for peer to peer communication, packet transmission and receiving).
+
+The driver allocates a shared memory that is physically mapped onto the DMA
+physical space at ring #0.
+For the software to communicate with the firmware, the driver sends a command
+in ring #0. The command contains a pre-defined field (PDF) value notifying the
+firmware that the driver is ready. To proceed, the driver must receive the
+appropriate PDF value in response from the firmware.
+
+Once the exchange is completed, messages can be sent to the firmware through
+the driver. Similarly, the firmware can now send notifications about hardware
+and firmware events.
+
+Information
+===========
+
+Mailing list:
+	thunderbolt-software@lists.01.org
+	Register at: https://lists.01.org/mailman/listinfo/thunderbolt-software
+	Archives at: https://lists.01.org/pipermail/thunderbolt-software/
+
+For additional information about Thunderbolt technology visit:
+	https://01.org/thunderbolt-sw
+	https://thunderbolttechnology.net/
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* [PATCH v9 8/8] thunderbolt: Adding maintainer entry
  2016-11-09 14:20 [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Amir Levy
                   ` (6 preceding siblings ...)
  2016-11-09 14:20 ` [PATCH v9 7/8] thunderbolt: Networking doc Amir Levy
@ 2016-11-09 14:20 ` Amir Levy
  2016-11-09 14:36 ` [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Simon Guinot
  2016-11-09 16:02 ` Greg KH
  9 siblings, 0 replies; 28+ messages in thread
From: Amir Levy @ 2016-11-09 14:20 UTC (permalink / raw)
  To: gregkh
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang, Amir Levy

Add Amir Levy as maintainer for Thunderbolt(TM) ICM driver

Signed-off-by: Amir Levy <amir.jer.levy@intel.com>
---
 MAINTAINERS | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/MAINTAINERS b/MAINTAINERS
index 411e3b8..87763c44 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -10652,7 +10652,13 @@ F:	include/uapi/linux/stm.h
 THUNDERBOLT DRIVER
 M:	Andreas Noever <andreas.noever@gmail.com>
 S:	Maintained
-F:	drivers/thunderbolt/
+F:	drivers/thunderbolt/*
+
+THUNDERBOLT ICM DRIVER
+M:	Amir Levy <amir.jer.levy@intel.com>
+S:	Maintained
+F:	drivers/thunderbolt/icm/
+F:	Documentation/thunderbolt/networking.txt
 
 TI BQ27XXX POWER SUPPLY DRIVER
 R:	Andrew F. Davis <afd@ti.com>
-- 
2.7.4

^ permalink raw reply related	[flat|nested] 28+ messages in thread

* Re: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-09 14:20 [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Amir Levy
                   ` (7 preceding siblings ...)
  2016-11-09 14:20 ` [PATCH v9 8/8] thunderbolt: Adding maintainer entry Amir Levy
@ 2016-11-09 14:36 ` Simon Guinot
  2016-11-09 15:42   ` Levy, Amir (Jer)
  2016-11-09 16:02 ` Greg KH
  9 siblings, 1 reply; 28+ messages in thread
From: Simon Guinot @ 2016-11-09 14:36 UTC (permalink / raw)
  To: Amir Levy
  Cc: gregkh, andreas.noever, bhelgaas, corbet, linux-kernel,
	linux-pci, netdev, linux-doc, mario_limonciello,
	thunderbolt-linux, mika.westerberg, tomas.winkler, xiong.y.zhang,
	Michael Jamet


[-- Attachment #1.1: Type: text/plain, Size: 2919 bytes --]

On Wed, Nov 09, 2016 at 04:20:00PM +0200, Amir Levy wrote:
> This driver enables Thunderbolt Networking on non-Apple platforms
> running Linux.
> 
> Thunderbolt Networking provides peer-to-peer connections to transfer
> files between computers, perform PC migrations, and/or set up small
> workgroups with shared storage.
> 
> This is a virtual connection that emulates an Ethernet adapter that
> enables Ethernet networking with the benefit of Thunderbolt superfast
> medium capability.
> 
> Thunderbolt Networking enables two hosts and several devices that
> have a Thunderbolt controller to be connected together in a linear
> (Daisy chain) series from a single port.
> 
> Thunderbolt Networking for Linux is compatible with Thunderbolt
> Networking on systems running macOS or Windows and also supports
> Thunderbolt generation 2 and 3 controllers.
> 
> Note that all pre-existing Thunderbolt generation 3 features, such as
> USB, Display and other Thunderbolt device connectivity will continue
> to function exactly as they did prior to enabling Thunderbolt Networking.
> 
> Code and Software Specifications:
> This kernel code creates a virtual ethernet device for computer to
> computer communication over a Thunderbolt cable.
> The new driver is a separate driver to the existing Thunderbolt driver.
> It is designed to work on systems running Linux that
> interface with Intel Connection Manager (ICM) firmware based
> Thunderbolt controllers that support Thunderbolt Networking.
> The kernel code operates in coordination with the Thunderbolt user-
> space daemon to implement full Thunderbolt networking functionality.
> 
> Hardware Specifications:
> Thunderbolt Hardware specs have not yet been published but are used
> where necessary for register definitions. 

Hi Amir,

I have an ASUS "All Series/Z87-DELUXE/QUAD" motherboard with a
Thunderbolt 2 "Falcon Ridge" chipset (device ID 156d).

Is the thunderbolt-icm driver supposed to work with this chipset ?

I have installed both a 4.8.6 Linux kernel (patched with your v9
series) and the thunderbolt-software-daemon (27 october release)
inside a Debian system (Jessie).

If I connect the ASUS motherboard with a MacBook Pro (Thunderbolt 2,
device ID 156c), I can see that the thunderbolt-icm driver is loaded and
that the thunderbolt-software-daemon is well started. But the Ethernet
interface is not created.

I have attached to this email the syslog file. There is the logs from
both the kernel and the daemon inside. Note that the daemon logs are
everything but clear about what could be the issue. Maybe I missed some
kind of configuration ? But I failed to find any valuable information
about configuring the driver and/or the daemon in the various
documentation files.

Please, can you provide some guidance ? I'd really like to test your
patch series.

Thanks in advance.

Simon

[-- Attachment #1.2: syslog --]
[-- Type: text/plain, Size: 154325 bytes --]

Nov  9 16:30:30 debian8 systemd-modules-load[218]: Inserted module 'lp'
Nov  9 16:30:30 debian8 systemd-modules-load[218]: Inserted module 'ppdev'
Nov  9 16:30:30 debian8 systemd-modules-load[218]: Inserted module 'parport_pc'
Nov  9 16:30:30 debian8 systemd-modules-load[218]: Inserted module 'fuse'
Nov  9 16:30:30 debian8 systemd[1]: Started Create Static Device Nodes in /dev.
Nov  9 16:30:30 debian8 systemd[1]: Starting udev Kernel Device Manager...
Nov  9 16:30:30 debian8 systemd[1]: Started Load Kernel Modules.
Nov  9 16:30:30 debian8 systemd[1]: Mounting FUSE Control File System...
Nov  9 16:30:30 debian8 systemd[1]: Starting Apply Kernel Variables...
Nov  9 16:30:30 debian8 systemd[1]: Mounted Configuration File System.
Nov  9 16:30:30 debian8 systemd[1]: Mounted FUSE Control File System.
Nov  9 16:30:30 debian8 systemd[1]: Started udev Kernel Device Manager.
Nov  9 16:30:30 debian8 systemd[1]: Starting Copy rules generated while the root was ro...
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: MD array assembly...
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: Set preliminary keymap...
Nov  9 16:30:30 debian8 systemd[1]: Started Apply Kernel Variables.
Nov  9 16:30:30 debian8 systemd[1]: Started Copy rules generated while the root was ro.
Nov  9 16:30:30 debian8 mdadm-raid[247]: Generating udev events for MD arrays...done.
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: MD array assembly.
Nov  9 16:30:30 debian8 keyboard-setup[248]: Setting preliminary keymap...done.
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: Set preliminary keymap.
Nov  9 16:30:30 debian8 systemd[1]: Starting Remount Root and Kernel File Systems...
Nov  9 16:30:30 debian8 systemd[1]: Started Remount Root and Kernel File Systems.
Nov  9 16:30:30 debian8 systemd[1]: Started Various fixups to make systemd work better on Debian.
Nov  9 16:30:30 debian8 systemd[1]: Starting Load/Save Random Seed...
Nov  9 16:30:30 debian8 systemd[1]: Starting Local File Systems (Pre).
Nov  9 16:30:30 debian8 systemd[1]: Reached target Local File Systems (Pre).
Nov  9 16:30:30 debian8 systemd[1]: Started Load/Save Random Seed.
Nov  9 16:30:30 debian8 mtp-probe: checking bus 3, device 3: "/sys/devices/pci0000:00/0000:00:14.0/usb3/3-4"
Nov  9 16:30:30 debian8 mtp-probe: checking bus 3, device 4: "/sys/devices/pci0000:00/0000:00:14.0/usb3/3-5"
Nov  9 16:30:30 debian8 mtp-probe: checking bus 3, device 5: "/sys/devices/pci0000:00/0000:00:14.0/usb3/3-6"
Nov  9 16:30:30 debian8 kernel: [    0.000000] Linux version 4.8.6+ (sguinot@debian8) (gcc version 4.9.2 (Debian 4.9.2-10) ) #2 SMP Mon Nov 7 18:52:35 CET 2016
Nov  9 16:30:30 debian8 kernel: [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.8.6+ root=UUID=6e1b0503-9be7-40ef-8da7-ef7e35297f58 ro console=ttyS1,115200n8
Nov  9 16:30:30 debian8 mtp-probe: bus: 3, device: 4 was not an MTP device
Nov  9 16:30:30 debian8 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
Nov  9 16:30:30 debian8 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
Nov  9 16:30:30 debian8 kernel: [    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
Nov  9 16:30:30 debian8 kernel: [    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
Nov  9 16:30:30 debian8 mtp-probe: bus: 3, device: 5 was not an MTP device
Nov  9 16:30:30 debian8 kernel: [    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
Nov  9 16:30:30 debian8 kernel: [    0.000000] x86/fpu: Using 'eager' FPU context switches.
Nov  9 16:30:30 debian8 kernel: [    0.000000] e820: BIOS-provided physical RAM map:
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x0000000000000000-0x0000000000057fff] usable
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x0000000000058000-0x0000000000058fff] reserved
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x0000000000059000-0x000000000009efff] usable
Nov  9 16:30:30 debian8 mtp-probe: bus: 3, device: 3 was not an MTP device
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x000000000009f000-0x000000000009ffff] reserved
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000003508dfff] usable
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x000000003508e000-0x0000000035094fff] ACPI NVS
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x0000000035095000-0x00000000354ecfff] usable
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x00000000354ed000-0x0000000035960fff] reserved
Nov  9 16:30:30 debian8 systemd[1]: Found device ST1000DM003-9YN162 1.
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x0000000035961000-0x00000000486c4fff] usable
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x00000000486c5000-0x00000000488cdfff] reserved
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x00000000488ce000-0x00000000488e6fff] ACPI data
Nov  9 16:30:30 debian8 systemd[1]: Starting File System Check on /dev/disk/by-uuid/057B-B280...
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x00000000488e7000-0x0000000048e2cfff] ACPI NVS
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x0000000048e2d000-0x0000000049f48fff] reserved
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x0000000049f49000-0x0000000049ffefff] type 20
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x0000000049fff000-0x0000000049ffffff] usable
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x000000004b000000-0x000000004f1fffff] reserved
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x00000000e0000000-0x00000000efffffff] reserved
Nov  9 16:30:30 debian8 systemd[1]: Found device ST1000DM003-9YN162 4.
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x00000000fed00000-0x00000000fed03fff] reserved
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
Nov  9 16:30:30 debian8 systemd[1]: Starting File System Check on /dev/disk/by-uuid/39f58bf3-20a1-4c5d-8c38-7804598f1b07...
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
Nov  9 16:30:30 debian8 kernel: [    0.000000] BIOS-e820: [mem 0x0000000100000000-0x00000001afdfffff] usable
Nov  9 16:30:30 debian8 systemd[1]: Found device ST1000DM003-9YN162 3.
Nov  9 16:30:30 debian8 systemd[1]: Found device ST1000DM003-9YN162 3.
Nov  9 16:30:30 debian8 systemd[1]: Activating swap Swap Partition...
Nov  9 16:30:30 debian8 systemd[1]: Activating swap /dev/disk/by-uuid/e37efc6f-badb-45c0-8854-f115567dc8fc...
Nov  9 16:30:30 debian8 systemd[1]: Activated swap Swap Partition.
Nov  9 16:30:30 debian8 systemd[1]: Activated swap /dev/disk/by-uuid/e37efc6f-badb-45c0-8854-f115567dc8fc.
Nov  9 16:30:30 debian8 kernel: [    0.000000] NX (Execute Disable) protection: active
Nov  9 16:30:30 debian8 systemd[1]: Starting Swap.
Nov  9 16:30:30 debian8 kernel: [    0.000000] efi: EFI v2.31 by American Megatrends
Nov  9 16:30:30 debian8 kernel: [    0.000000] efi:  ESRT=0x49f47818  ACPI=0x488d3000  ACPI 2.0=0x488d3000  SMBIOS=0xf04c0  MPS=0xfd4f0 
Nov  9 16:30:30 debian8 kernel: [    0.000000] esrt: Reserving ESRT space from 0x0000000049f47818 to 0x0000000049f47850.
Nov  9 16:30:30 debian8 kernel: [    0.000000] SMBIOS 2.7 present.
Nov  9 16:30:30 debian8 kernel: [    0.000000] DMI: ASUS All Series/Z87-DELUXE/QUAD, BIOS 2103 08/18/2014
Nov  9 16:30:30 debian8 kernel: [    0.000000] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
Nov  9 16:30:30 debian8 kernel: [    0.000000] e820: remove [mem 0x000a0000-0x000fffff] usable
Nov  9 16:30:30 debian8 kernel: [    0.000000] e820: last_pfn = 0x1afe00 max_arch_pfn = 0x400000000
Nov  9 16:30:30 debian8 kernel: [    0.000000] MTRR default type: uncachable
Nov  9 16:30:30 debian8 kernel: [    0.000000] MTRR fixed ranges enabled:
Nov  9 16:30:30 debian8 kernel: [    0.000000]   00000-9FFFF write-back
Nov  9 16:30:30 debian8 kernel: [    0.000000]   A0000-BFFFF uncachable
Nov  9 16:30:30 debian8 kernel: [    0.000000]   C0000-CFFFF write-protect
Nov  9 16:30:30 debian8 systemd[1]: Reached target Swap.
Nov  9 16:30:30 debian8 kernel: [    0.000000]   D0000-DFFFF uncachable
Nov  9 16:30:30 debian8 kernel: [    0.000000]   E0000-FFFFF write-protect
Nov  9 16:30:30 debian8 kernel: [    0.000000] MTRR variable ranges enabled:
Nov  9 16:30:30 debian8 kernel: [    0.000000]   0 base 0000000000 mask 7F00000000 write-back
Nov  9 16:30:30 debian8 kernel: [    0.000000]   1 base 0100000000 mask 7F80000000 write-back
Nov  9 16:30:30 debian8 kernel: [    0.000000]   2 base 0180000000 mask 7FC0000000 write-back
Nov  9 16:30:30 debian8 systemd-fsck[388]: /dev/sda4: clean, 119435/60145664 files, 7633198/240576768 blocks
Nov  9 16:30:30 debian8 systemd[1]: Started File System Check on /dev/disk/by-uuid/39f58bf3-20a1-4c5d-8c38-7804598f1b07.
Nov  9 16:30:30 debian8 kernel: [    0.000000]   3 base 0080000000 mask 7F80000000 uncachable
Nov  9 16:30:30 debian8 kernel: [    0.000000]   4 base 0060000000 mask 7FE0000000 uncachable
Nov  9 16:30:30 debian8 kernel: [    0.000000]   5 base 0050000000 mask 7FF0000000 uncachable
Nov  9 16:30:30 debian8 kernel: [    0.000000]   6 base 004C000000 mask 7FFC000000 uncachable
Nov  9 16:30:30 debian8 systemd[1]: Mounting /home...
Nov  9 16:30:30 debian8 kernel: [    0.000000]   7 base 004B000000 mask 7FFF000000 uncachable
Nov  9 16:30:30 debian8 kernel: [    0.000000]   8 base 01B0000000 mask 7FF0000000 uncachable
Nov  9 16:30:30 debian8 kernel: [    0.000000]   9 base 01AFE00000 mask 7FFFE00000 uncachable
Nov  9 16:30:30 debian8 systemd[1]: Mounted /home.
Nov  9 16:30:30 debian8 kernel: [    0.000000] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WC  UC- WT  
Nov  9 16:30:30 debian8 kernel: [    0.000000] e820: update [mem 0x4b000000-0xffffffff] usable ==> reserved
Nov  9 16:30:30 debian8 kernel: [    0.000000] e820: last_pfn = 0x4a000 max_arch_pfn = 0x400000000
Nov  9 16:30:30 debian8 kernel: [    0.000000] found SMP MP-table at [mem 0x000fd800-0x000fd80f] mapped at [ffff9882400fd800]
Nov  9 16:30:30 debian8 systemd[1]: Starting system-systemd\x2drfkill.slice.
Nov  9 16:30:30 debian8 kernel: [    0.000000] Base memory trampoline at [ffff988240097000] 97000 size 24576
Nov  9 16:30:30 debian8 kernel: [    0.000000] Using GB pages for direct mapping
Nov  9 16:30:30 debian8 kernel: [    0.000000] BRK [0x181960000, 0x181960fff] PGTABLE
Nov  9 16:30:30 debian8 systemd[1]: Created slice system-systemd\x2drfkill.slice.
Nov  9 16:30:30 debian8 kernel: [    0.000000] BRK [0x181961000, 0x181961fff] PGTABLE
Nov  9 16:30:30 debian8 kernel: [    0.000000] BRK [0x181962000, 0x181962fff] PGTABLE
Nov  9 16:30:30 debian8 kernel: [    0.000000] BRK [0x181963000, 0x181963fff] PGTABLE
Nov  9 16:30:30 debian8 kernel: [    0.000000] BRK [0x181964000, 0x181964fff] PGTABLE
Nov  9 16:30:30 debian8 kernel: [    0.000000] BRK [0x181965000, 0x181965fff] PGTABLE
Nov  9 16:30:30 debian8 kernel: [    0.000000] BRK [0x181966000, 0x181966fff] PGTABLE
Nov  9 16:30:30 debian8 kernel: [    0.000000] BRK [0x181967000, 0x181967fff] PGTABLE
Nov  9 16:30:30 debian8 systemd[1]: Starting Load/Save RF Kill Switch Status of rfkill0...
Nov  9 16:30:30 debian8 kernel: [    0.000000] BRK [0x181968000, 0x181968fff] PGTABLE
Nov  9 16:30:30 debian8 kernel: [    0.000000] BRK [0x181969000, 0x181969fff] PGTABLE
Nov  9 16:30:30 debian8 kernel: [    0.000000] BRK [0x18196a000, 0x18196afff] PGTABLE
Nov  9 16:30:30 debian8 kernel: [    0.000000] RAMDISK: [mem 0x17560000-0x22fcffff]
Nov  9 16:30:30 debian8 systemd[1]: Mounted Configuration File System.
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: Early table checksum verification disabled
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: RSDP 0x00000000488D3000 000024 (v02 ALASKA)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: XSDT 0x00000000488D3080 00007C (v01 ALASKA A M I    01072009 AMI  00010013)
Nov  9 16:30:30 debian8 systemd[1]: Started Set Up Additional Binary Formats.
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: FACP 0x00000000488E1878 00010C (v05 ALASKA A M I    01072009 AMI  00010013)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: DSDT 0x00000000488D3198 00E6DC (v02 ALASKA A M I    00000031 INTL 20091112)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: FACS 0x0000000048E2B080 000040
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: APIC 0x00000000488E1988 000092 (v03 ALASKA A M I    01072009 AMI  00010013)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: FPDT 0x00000000488E1A20 000044 (v01 ALASKA A M I    01072009 AMI  00010013)
Nov  9 16:30:30 debian8 systemd[1]: Started Various fixups to make systemd work better on Debian.
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: LPIT 0x00000000488E1A68 00005C (v01 ALASKA A M I    00000000 AMI. 00000005)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: SSDT 0x00000000488E1AC8 000539 (v01 PmRef  Cpu0Ist  00003000 INTL 20091112)
Nov  9 16:30:30 debian8 systemd[1]: Started File System Check on Root Device.
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: SSDT 0x00000000488E2008 000AD8 (v01 PmRef  CpuPm    00003000 INTL 20091112)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: MCFG 0x00000000488E2AE0 00003C (v01 ALASKA A M I    01072009 MSFT 00000097)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: HPET 0x00000000488E2B20 000038 (v01 ALASKA A M I    01072009 AMI. 00000005)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: SSDT 0x00000000488E2B58 00036D (v01 SataRe SataTabl 00001000 INTL 20091112)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: SSDT 0x00000000488E2EC8 0034E1 (v01 SaSsdt SaSsdt   00003000 INTL 20091112)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: BGRT 0x00000000488E6408 000038 (v00 ALASKA A M I    01072009 AMI  00010013)
Nov  9 16:30:30 debian8 systemd-fsck[374]: fsck.fat 3.0.27 (2014-11-12)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: Local APIC address 0xfee00000
Nov  9 16:30:30 debian8 kernel: [    0.000000] No NUMA configuration found
Nov  9 16:30:30 debian8 kernel: [    0.000000] Faking a node at [mem 0x0000000000000000-0x00000001afdfffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] NODE_DATA(0) allocated [mem 0x1afdfb000-0x1afdfffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] Zone ranges:
Nov  9 16:30:30 debian8 kernel: [    0.000000]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
Nov  9 16:30:30 debian8 systemd-fsck[374]: /dev/sda1: 9 files, 69/130812 clusters
Nov  9 16:30:30 debian8 kernel: [    0.000000]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000]   Normal   [mem 0x0000000100000000-0x00000001afdfffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000]   Device   empty
Nov  9 16:30:30 debian8 systemd[1]: Started File System Check on /dev/disk/by-uuid/057B-B280.
Nov  9 16:30:30 debian8 kernel: [    0.000000] Movable zone start for each node
Nov  9 16:30:30 debian8 kernel: [    0.000000] Early memory node ranges
Nov  9 16:30:30 debian8 kernel: [    0.000000]   node   0: [mem 0x0000000000001000-0x0000000000057fff]
Nov  9 16:30:30 debian8 kernel: [    0.000000]   node   0: [mem 0x0000000000059000-0x000000000009efff]
Nov  9 16:30:30 debian8 systemd[1]: Mounting /boot/efi...
Nov  9 16:30:30 debian8 kernel: [    0.000000]   node   0: [mem 0x0000000000100000-0x000000003508dfff]
Nov  9 16:30:30 debian8 kernel: [    0.000000]   node   0: [mem 0x0000000035095000-0x00000000354ecfff]
Nov  9 16:30:30 debian8 kernel: [    0.000000]   node   0: [mem 0x0000000035961000-0x00000000486c4fff]
Nov  9 16:30:30 debian8 systemd[1]: Started Load/Save RF Kill Switch Status of rfkill0.
Nov  9 16:30:30 debian8 kernel: [    0.000000]   node   0: [mem 0x0000000049fff000-0x0000000049ffffff]
Nov  9 16:30:30 debian8 systemd[1]: Starting Sound Card.
Nov  9 16:30:30 debian8 systemd[1]: Reached target Sound Card.
Nov  9 16:30:30 debian8 systemd[1]: Mounted /boot/efi.
Nov  9 16:30:30 debian8 systemd[1]: Starting Local File Systems.
Nov  9 16:30:30 debian8 systemd[1]: Reached target Local File Systems.
Nov  9 16:30:30 debian8 systemd[1]: Starting Create Volatile Files and Directories...
Nov  9 16:30:30 debian8 systemd[1]: Starting Remote File Systems.
Nov  9 16:30:30 debian8 systemd[1]: Reached target Remote File Systems.
Nov  9 16:30:30 debian8 systemd[1]: Starting Trigger Flushing of Journal to Persistent Storage...
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: Prepare console...
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: Raise network interfaces....
Nov  9 16:30:30 debian8 systemd[1]: Started Trigger Flushing of Journal to Persistent Storage.
Nov  9 16:30:30 debian8 systemd[1]: Started Create Volatile Files and Directories.
Nov  9 16:30:30 debian8 systemd[1]: Starting Update UTMP about System Boot/Shutdown...
Nov  9 16:30:30 debian8 kbd[428]: Setting console screen modes.
Nov  9 16:30:30 debian8 networking[429]: Configuring network interfaces.../etc/network/interfaces:11: too few parameters for iface line
Nov  9 16:30:30 debian8 networking[429]: ifquery: couldn't read interfaces file "/etc/network/interfaces"
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: Raise network interfaces..
Nov  9 16:30:30 debian8 networking[429]: /etc/network/interfaces:11: too few parameters for iface line
Nov  9 16:30:30 debian8 kernel: [    0.000000]   node   0: [mem 0x0000000100000000-0x00000001afdfffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x00000001afdfffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] On node 0 totalpages: 1015784
Nov  9 16:30:30 debian8 networking[429]: ifquery: couldn't read interfaces file "/etc/network/interfaces"
Nov  9 16:30:30 debian8 kernel: [    0.000000]   DMA zone: 64 pages used for memmap
Nov  9 16:30:30 debian8 kernel: [    0.000000]   DMA zone: 26 pages reserved
Nov  9 16:30:30 debian8 networking[429]: /etc/network/interfaces:11: too few parameters for iface line
Nov  9 16:30:30 debian8 kernel: [    0.000000]   DMA zone: 3997 pages, LIFO batch:0
Nov  9 16:30:30 debian8 kernel: [    0.000000]   DMA32 zone: 4554 pages used for memmap
Nov  9 16:30:30 debian8 kernel: [    0.000000]   DMA32 zone: 291403 pages, LIFO batch:31
Nov  9 16:30:30 debian8 kernel: [    0.000000]   Normal zone: 11256 pages used for memmap
Nov  9 16:30:30 debian8 kernel: [    0.000000]   Normal zone: 720384 pages, LIFO batch:31
Nov  9 16:30:30 debian8 networking[429]: ifup: couldn't read interfaces file "/etc/network/interfaces"
Nov  9 16:30:30 debian8 kernel: [    0.000000] Reserving Intel graphics memory at 0x000000004b200000-0x000000004f1fffff
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: PM-Timer IO Port: 0x1808
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: Local APIC address 0xfee00000
Nov  9 16:30:30 debian8 networking[429]: failed.
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
Nov  9 16:30:30 debian8 kernel: [    0.000000] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: IRQ0 used by override.
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: IRQ9 used by override.
Nov  9 16:30:30 debian8 systemd[1]: Starting Network.
Nov  9 16:30:30 debian8 kernel: [    0.000000] Using ACPI (MADT) for SMP configuration information
Nov  9 16:30:30 debian8 kernel: [    0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
Nov  9 16:30:30 debian8 kernel: [    0.000000] smpboot: Allowing 8 CPUs, 0 hotplug CPUs
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
Nov  9 16:30:30 debian8 systemd[1]: Reached target Network.
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x00058000-0x00058fff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x3508e000-0x35094fff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x354ed000-0x35960fff]
Nov  9 16:30:30 debian8 systemd[1]: Starting Network is Online.
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x486c5000-0x488cdfff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x488ce000-0x488e6fff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x488e7000-0x48e2cfff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x48e2d000-0x49f48fff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x49f49000-0x49ffefff]
Nov  9 16:30:30 debian8 systemd[1]: Reached target Network is Online.
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x4a000000-0x4affffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x4b000000-0x4f1fffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0x4f200000-0xdfffffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xe0000000-0xefffffff]
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: RPC portmapper replacement...
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xf0000000-0xfebfffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfec00000-0xfec00fff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfec01000-0xfecfffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfed00000-0xfed03fff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfed04000-0xfed1bfff]
Nov  9 16:30:30 debian8 systemd[1]: Started Update UTMP about System Boot/Shutdown.
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfed1c000-0xfed1ffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfed20000-0xfedfffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfee00000-0xfee00fff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xfee01000-0xfeffffff]
Nov  9 16:30:30 debian8 kernel: [    0.000000] PM: Registered nosave memory: [mem 0xff000000-0xffffffff]
Nov  9 16:30:30 debian8 kbd[428]: setterm: $TERM is not defined.
Nov  9 16:30:30 debian8 kernel: [    0.000000] e820: [mem 0x4f200000-0xdfffffff] available for PCI devices
Nov  9 16:30:30 debian8 kernel: [    0.000000] Booting paravirtualized kernel on bare hardware
Nov  9 16:30:30 debian8 kernel: [    0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: Prepare console.
Nov  9 16:30:30 debian8 kernel: [    0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:8 nr_node_ids:1
Nov  9 16:30:30 debian8 kernel: [    0.000000] percpu: Embedded 35 pages/cpu @ffff9883efa00000 s104792 r8192 d30376 u262144
Nov  9 16:30:30 debian8 kernel: [    0.000000] pcpu-alloc: s104792 r8192 d30376 u262144 alloc=1*2097152
Nov  9 16:30:30 debian8 kernel: [    0.000000] pcpu-alloc: [0] 0 1 2 3 4 5 6 7 
Nov  9 16:30:30 debian8 kernel: [    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 999884
Nov  9 16:30:30 debian8 kernel: [    0.000000] Policy zone: Normal
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: Set console font and keymap...
Nov  9 16:30:30 debian8 kernel: [    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.8.6+ root=UUID=6e1b0503-9be7-40ef-8da7-ef7e35297f58 ro console=ttyS1,115200n8
Nov  9 16:30:30 debian8 kernel: [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
Nov  9 16:30:30 debian8 kernel: [    0.000000] Calgary: detecting Calgary via BIOS EBDA area
Nov  9 16:30:30 debian8 kernel: [    0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
Nov  9 16:30:30 debian8 rpcbind[467]: Starting rpcbind daemon....
Nov  9 16:30:30 debian8 kernel: [    0.000000] Memory: 3499716K/4063136K available (6083K kernel code, 1232K rwdata, 2776K rodata, 1376K init, 808K bss, 563420K reserved, 0K cma-reserved)
Nov  9 16:30:30 debian8 kernel: [    0.000000] Hierarchical RCU implementation.
Nov  9 16:30:30 debian8 kernel: [    0.000000] 	Build-time adjustment of leaf fanout to 64.
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: RPC portmapper replacement.
Nov  9 16:30:30 debian8 kernel: [    0.000000] 	RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=8.
Nov  9 16:30:30 debian8 kernel: [    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=8
Nov  9 16:30:30 debian8 kernel: [    0.000000] NR_IRQS:33024 nr_irqs:488 16
Nov  9 16:30:30 debian8 kernel: [    0.000000] Console: colour dummy device 80x25
Nov  9 16:30:30 debian8 systemd[1]: Starting RPC Port Mapper.
Nov  9 16:30:30 debian8 kernel: [    0.000000] console [ttyS1] enabled
Nov  9 16:30:30 debian8 kernel: [    0.000000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484882848 ns
Nov  9 16:30:30 debian8 kernel: [    0.000000] hpet clockevent registered
Nov  9 16:30:30 debian8 systemd[1]: Reached target RPC Port Mapper.
Nov  9 16:30:30 debian8 kernel: [    0.000000] tsc: Fast TSC calibration using PIT
Nov  9 16:30:30 debian8 kernel: [    0.000000] tsc: Detected 3398.200 MHz processor
Nov  9 16:30:30 debian8 kernel: [    0.000017] Calibrating delay loop (skipped), value calculated using timer frequency.. 6796.40 BogoMIPS (lpj=13592800)
Nov  9 16:30:30 debian8 kernel: [    0.000404] pid_max: default: 32768 minimum: 301
Nov  9 16:30:30 debian8 kernel: [    0.000582] ACPI: Core revision 20160422
Nov  9 16:30:30 debian8 kernel: [    0.008302] ACPI: 5 ACPI AML tables successfully acquired and loaded
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: NFS support files common to client and server...
Nov  9 16:30:30 debian8 kernel: [    0.008537] 
Nov  9 16:30:30 debian8 kernel: [    0.009185] Security Framework initialized
Nov  9 16:30:30 debian8 kernel: [    0.009335] Yama: becoming mindful.
Nov  9 16:30:30 debian8 kernel: [    0.009467] AppArmor: AppArmor disabled by boot time parameter
Nov  9 16:30:30 debian8 console-setup[472]: Setting up console font and keymap...done.
Nov  9 16:30:30 debian8 kernel: [    0.009813] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
Nov  9 16:30:30 debian8 kernel: [    0.010979] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
Nov  9 16:30:30 debian8 kernel: [    0.011653] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes)
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: Set console font and keymap.
Nov  9 16:30:30 debian8 kernel: [    0.011894] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes)
Nov  9 16:30:30 debian8 kernel: [    0.012346] CPU: Physical Processor ID: 0
Nov  9 16:30:30 debian8 kernel: [    0.012492] CPU: Processor Core ID: 0
Nov  9 16:30:30 debian8 kernel: [    0.012629] ENERGY_PERF_BIAS: Set to 'normal', was 'performance'
Nov  9 16:30:30 debian8 kernel: [    0.012844] ENERGY_PERF_BIAS: View and update with x86_energy_perf_policy(8)
Nov  9 16:30:30 debian8 rpc.statd[541]: Version 1.2.8 starting
Nov  9 16:30:30 debian8 kernel: [    0.013098] mce: CPU supports 9 MCE banks
Nov  9 16:30:30 debian8 kernel: [    0.013250] CPU0: Thermal monitoring enabled (TM1)
Nov  9 16:30:30 debian8 kernel: [    0.013434] process: using mwait in idle threads
Nov  9 16:30:30 debian8 sm-notify[542]: Version 1.2.8 starting
Nov  9 16:30:30 debian8 kernel: [    0.013603] Last level iTLB entries: 4KB 1024, 2MB 1024, 4MB 1024
Nov  9 16:30:30 debian8 kernel: [    0.013822] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 1024, 1GB 4
Nov  9 16:30:30 debian8 kernel: [    0.014300] Freeing SMP alternatives memory: 24K (ffffffffba28e000 - ffffffffba294000)
Nov  9 16:30:30 debian8 kernel: [    0.022238] ftrace: allocating 24996 entries in 98 pages
Nov  9 16:30:30 debian8 kernel: [    0.028985] smpboot: APIC(0) Converting physical 0 to logical package 0
Nov  9 16:30:30 debian8 nfs-common[535]: Starting NFS common utilities: statd idmapd.
Nov  9 16:30:30 debian8 kernel: [    0.029223] smpboot: Max logical packages: 2
Nov  9 16:30:30 debian8 kernel: [    0.029443] x2apic: IRQ remapping doesn't support X2APIC mode
Nov  9 16:30:30 debian8 kernel: [    0.030047] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: NFS support files common to client and server.
Nov  9 16:30:30 debian8 kernel: [    0.069946] TSC deadline timer enabled
Nov  9 16:30:30 debian8 kernel: [    0.069948] smpboot: CPU0: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (family: 0x6, model: 0x3c, stepping: 0x3)
Nov  9 16:30:30 debian8 kernel: [    0.070317] Performance Events: PEBS fmt2+, Haswell events, 16-deep LBR, full-width counters, Intel PMU driver.
Nov  9 16:30:30 debian8 kernel: [    0.070721] ... version:                3
Nov  9 16:30:30 debian8 kernel: [    0.070867] ... bit width:              48
Nov  9 16:30:30 debian8 kernel: [    0.071016] ... generic registers:      4
Nov  9 16:30:30 debian8 kernel: [    0.071161] ... value mask:             0000ffffffffffff
Nov  9 16:30:30 debian8 kernel: [    0.071352] ... max period:             0000ffffffffffff
Nov  9 16:30:30 debian8 kernel: [    0.071543] ... fixed-purpose events:   3
Nov  9 16:30:30 debian8 kernel: [    0.071689] ... event mask:             000000070000000f
Nov  9 16:30:30 debian8 kernel: [    0.072295] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
Nov  9 16:30:30 debian8 kernel: [    0.072649] x86: Booting SMP configuration:
Nov  9 16:30:30 debian8 kernel: [    0.072801] .... node  #0, CPUs:      #1 #2 #3 #4 #5 #6 #7
Nov  9 16:30:30 debian8 systemd[1]: Starting System Initialization.
Nov  9 16:30:30 debian8 kernel: [    0.632101] x86: Booted up 1 node, 8 CPUs
Nov  9 16:30:30 debian8 kernel: [    0.632262] smpboot: Total of 8 processors activated (54378.36 BogoMIPS)
Nov  9 16:30:30 debian8 kernel: [    0.654844] devtmpfs: initialized
Nov  9 16:30:30 debian8 kernel: [    0.655011] x86/mm: Memory block size: 128MB
Nov  9 16:30:30 debian8 systemd[1]: Reached target System Initialization.
Nov  9 16:30:30 debian8 systemd[1]: Starting Avahi mDNS/DNS-SD Stack Activation Socket.
Nov  9 16:30:30 debian8 systemd[1]: Listening on Avahi mDNS/DNS-SD Stack Activation Socket.
Nov  9 16:30:30 debian8 systemd[1]: Starting D-Bus System Message Bus Socket.
Nov  9 16:30:30 debian8 systemd[1]: Listening on D-Bus System Message Bus Socket.
Nov  9 16:30:30 debian8 systemd[1]: Starting ACPID Listen Socket.
Nov  9 16:30:30 debian8 systemd[1]: Listening on ACPID Listen Socket.
Nov  9 16:30:30 debian8 systemd[1]: Starting ACPI Events Check.
Nov  9 16:30:30 debian8 systemd[1]: Started ACPI Events Check.
Nov  9 16:30:30 debian8 systemd[1]: Starting CUPS Printing Service Sockets.
Nov  9 16:30:30 debian8 systemd[1]: Listening on CUPS Printing Service Sockets.
Nov  9 16:30:30 debian8 systemd[1]: Starting Sockets.
Nov  9 16:30:30 debian8 systemd[1]: Reached target Sockets.
Nov  9 16:30:30 debian8 systemd[1]: Starting Daily Cleanup of Temporary Directories.
Nov  9 16:30:30 debian8 systemd[1]: Started Daily Cleanup of Temporary Directories.
Nov  9 16:30:30 debian8 systemd[1]: Starting Timers.
Nov  9 16:30:30 debian8 systemd[1]: Reached target Timers.
Nov  9 16:30:30 debian8 systemd[1]: Started Manage Sound Card State (restore and store).
Nov  9 16:30:30 debian8 systemd[1]: Starting Restore Sound Card State...
Nov  9 16:30:30 debian8 kernel: [    0.670815] PM: Registering ACPI NVS region [mem 0x3508e000-0x35094fff] (28672 bytes)
Nov  9 16:30:30 debian8 systemd[1]: Starting CUPS Printer Service Spool.
Nov  9 16:30:30 debian8 kernel: [    0.671095] PM: Registering ACPI NVS region [mem 0x488e7000-0x48e2cfff] (5529600 bytes)
Nov  9 16:30:30 debian8 kernel: [    0.671493] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
Nov  9 16:30:30 debian8 kernel: [    0.686807] pinctrl core: initialized pinctrl subsystem
Nov  9 16:30:30 debian8 kernel: [    0.702866] NET: Registered protocol family 16
Nov  9 16:30:30 debian8 systemd[1]: Started CUPS Printer Service Spool.
Nov  9 16:30:30 debian8 kernel: [    0.718816] cpuidle: using governor ladder
Nov  9 16:30:30 debian8 kernel: [    0.734824] cpuidle: using governor menu
Nov  9 16:30:30 debian8 systemd[1]: Starting Paths.
Nov  9 16:30:30 debian8 kernel: [    0.734981] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
Nov  9 16:30:30 debian8 kernel: [    0.735251] ACPI: bus type PCI registered
Nov  9 16:30:30 debian8 kernel: [    0.735397] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
Nov  9 16:30:30 debian8 kernel: [    0.766874] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
Nov  9 16:30:30 debian8 systemd[1]: Reached target Paths.
Nov  9 16:30:30 debian8 kernel: [    0.767205] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in E820
Nov  9 16:30:30 debian8 kernel: [    0.767452] pmd_set_huge: Cannot satisfy [mem 0xe0000000-0xe0200000] with a huge-page mapping due to MTRR override.
Nov  9 16:30:30 debian8 systemd[1]: Starting Basic System.
Nov  9 16:30:30 debian8 kernel: [    0.768004] PCI: Using configuration type 1 for base access
Nov  9 16:30:30 debian8 kernel: [    0.768239] core: PMU erratum BJ122, BV98, HSD29 worked around, HT is on
Nov  9 16:30:30 debian8 kernel: [    0.768645] mtrr: your CPUs had inconsistent fixed MTRR settings
Nov  9 16:30:30 debian8 kernel: [    0.768861] mtrr: probably your BIOS does not setup all CPUs.
Nov  9 16:30:30 debian8 kernel: [    0.769066] mtrr: corrected configuration.
Nov  9 16:30:30 debian8 kernel: [    0.782910] HugeTLB registered 1 GB page size, pre-allocated 0 pages
Nov  9 16:30:30 debian8 systemd[1]: Reached target Basic System.
Nov  9 16:30:30 debian8 kernel: [    0.783138] HugeTLB registered 2 MB page size, pre-allocated 0 pages
Nov  9 16:30:30 debian8 kernel: [    0.830910] ACPI: Added _OSI(Module Device)
Nov  9 16:30:30 debian8 systemd[1]: Starting Bluetooth service...
Nov  9 16:30:30 debian8 kernel: [    0.831063] ACPI: Added _OSI(Processor Device)
Nov  9 16:30:30 debian8 kernel: [    0.831224] ACPI: Added _OSI(3.0 _SCP Extensions)
Nov  9 16:30:30 debian8 kernel: [    0.831395] ACPI: Added _OSI(Processor Aggregator Device)
Nov  9 16:30:30 debian8 kernel: [    0.831715] ACPI: Executed 1 blocks of module-level executable AML code
Nov  9 16:30:30 debian8 kernel: [    0.834727] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored
Nov  9 16:30:30 debian8 systemd[1]: Starting Accounts Service...
Nov  9 16:30:30 debian8 kernel: [    0.835611] ACPI: Dynamic OEM Table Load:
Nov  9 16:30:30 debian8 kernel: [    0.835773] ACPI: SSDT 0xFFFF9883EA24F800 0003D3 (v01 PmRef  Cpu0Cst  00003001 INTL 20091112)
Nov  9 16:30:30 debian8 kernel: [    0.836498] ACPI: Dynamic OEM Table Load:
Nov  9 16:30:30 debian8 kernel: [    0.836659] ACPI: SSDT 0xFFFF9883EB0B2800 0005AA (v01 PmRef  ApIst    00003000 INTL 20091112)
Nov  9 16:30:30 debian8 systemd[1]: Starting Deferred execution scheduler...
Nov  9 16:30:30 debian8 kernel: [    0.837422] ACPI: Dynamic OEM Table Load:
Nov  9 16:30:30 debian8 kernel: [    0.837582] ACPI: SSDT 0xFFFF9883EA69AA00 000119 (v01 PmRef  ApCst    00003000 INTL 20091112)
Nov  9 16:30:30 debian8 kernel: [    0.839158] ACPI: Interpreter enabled
Nov  9 16:30:30 debian8 systemd[1]: Started Deferred execution scheduler.
Nov  9 16:30:30 debian8 systemd[1]: Starting OpenBSD Secure Shell server...
Nov  9 16:30:30 debian8 kernel: [    0.839308] ACPI: (supports S0 S3 S4 S5)
Nov  9 16:30:30 debian8 kernel: [    0.839451] ACPI: Using IOAPIC for interrupt routing
Nov  9 16:30:30 debian8 kernel: [    0.839647] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
Nov  9 16:30:30 debian8 kernel: [    0.845546] ACPI: Power Resource [FN00] (off)
Nov  9 16:30:30 debian8 kernel: [    0.845756] ACPI: Power Resource [FN01] (off)
Nov  9 16:30:30 debian8 systemd[1]: Started OpenBSD Secure Shell server.
Nov  9 16:30:30 debian8 kernel: [    0.845963] ACPI: Power Resource [FN02] (off)
Nov  9 16:30:30 debian8 kernel: [    0.846169] ACPI: Power Resource [FN03] (off)
Nov  9 16:30:30 debian8 systemd[1]: Starting Modem Manager...
Nov  9 16:30:30 debian8 kernel: [    0.846375] ACPI: Power Resource [FN04] (off)
Nov  9 16:30:30 debian8 kernel: [    0.847093] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-fe])
Nov  9 16:30:30 debian8 kernel: [    0.847318] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
Nov  9 16:30:30 debian8 kernel: [    0.847762] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug PME]
Nov  9 16:30:30 debian8 kernel: [    0.848108] acpi PNP0A08:00: _OSC: OS now controls [AER PCIeCapability]
Nov  9 16:30:30 debian8 kernel: [    0.848345] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using BIOS configuration
Nov  9 16:30:30 debian8 systemd[1]: Starting Restore /etc/resolv.conf if the system crashed before the ppp link was shut down....
Nov  9 16:30:30 debian8 kernel: [    0.848900] PCI host bridge to bus 0000:00
Nov  9 16:30:30 debian8 kernel: [    0.849051] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7 window]
Nov  9 16:30:30 debian8 kernel: [    0.849294] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
Nov  9 16:30:30 debian8 systemd[1]: Starting Run anacron jobs...
Nov  9 16:30:30 debian8 kernel: [    0.849537] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
Nov  9 16:30:30 debian8 kernel: [    0.849804] pci_bus 0000:00: root bus resource [mem 0x000d0000-0x000d3fff window]
Nov  9 16:30:30 debian8 kernel: [    0.850071] pci_bus 0000:00: root bus resource [mem 0x000d4000-0x000d7fff window]
Nov  9 16:30:30 debian8 kernel: [    0.850339] pci_bus 0000:00: root bus resource [mem 0x000d8000-0x000dbfff window]
Nov  9 16:30:30 debian8 systemd[1]: Started Run anacron jobs.
Nov  9 16:30:30 debian8 kernel: [    0.850606] pci_bus 0000:00: root bus resource [mem 0x000dc000-0x000dffff window]
Nov  9 16:30:30 debian8 kernel: [    0.850873] pci_bus 0000:00: root bus resource [mem 0x4f200000-0xfeafffff window]
Nov  9 16:30:30 debian8 kernel: [    0.851143] pci_bus 0000:00: root bus resource [bus 00-fe]
Nov  9 16:30:30 debian8 kernel: [    0.851344] pci 0000:00:00.0: [8086:0c00] type 00 class 0x060000
Nov  9 16:30:30 debian8 kernel: [    0.851405] pci 0000:00:01.0: [8086:0c01] type 01 class 0x060400
Nov  9 16:30:30 debian8 kernel: [    0.851428] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 systemd[1]: Starting Regular background program processing daemon...
Nov  9 16:30:30 debian8 kernel: [    0.851487] pci 0000:00:01.0: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 kernel: [    0.851718] pci 0000:00:02.0: [8086:0412] type 00 class 0x030000
Nov  9 16:30:30 debian8 kernel: [    0.851724] pci 0000:00:02.0: reg 0x10: [mem 0xde400000-0xde7fffff 64bit]
Nov  9 16:30:30 debian8 anacron[565]: Anacron 2.3 started on 2016-11-09
Nov  9 16:30:30 debian8 kernel: [    0.851728] pci 0000:00:02.0: reg 0x18: [mem 0x50000000-0x5fffffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.851731] pci 0000:00:02.0: reg 0x20: [io  0xf000-0xf03f]
Nov  9 16:30:30 debian8 kernel: [    0.851784] pci 0000:00:03.0: [8086:0c0c] type 00 class 0x040300
Nov  9 16:30:30 debian8 kernel: [    0.851789] pci 0000:00:03.0: reg 0x10: [mem 0xdf134000-0xdf137fff 64bit]
Nov  9 16:30:30 debian8 anacron[565]: Normal exit (0 jobs run)
Nov  9 16:30:30 debian8 kernel: [    0.851861] pci 0000:00:14.0: [8086:8c31] type 00 class 0x0c0330
Nov  9 16:30:30 debian8 kernel: [    0.851874] pci 0000:00:14.0: reg 0x10: [mem 0xdf120000-0xdf12ffff 64bit]
Nov  9 16:30:30 debian8 kernel: [    0.851922] pci 0000:00:14.0: PME# supported from D3hot D3cold
Nov  9 16:30:30 debian8 systemd[1]: Started Regular background program processing daemon.
Nov  9 16:30:30 debian8 kernel: [    0.851949] pci 0000:00:14.0: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 kernel: [    0.852176] pci 0000:00:16.0: [8086:8c3a] type 00 class 0x078000
Nov  9 16:30:30 debian8 kernel: [    0.852189] pci 0000:00:16.0: reg 0x10: [mem 0xdf13e000-0xdf13e00f 64bit]
Nov  9 16:30:30 debian8 kernel: [    0.852241] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.852293] pci 0000:00:19.0: [8086:153b] type 00 class 0x020000
Nov  9 16:30:30 debian8 kernel: [    0.852304] pci 0000:00:19.0: reg 0x10: [mem 0xdf100000-0xdf11ffff]
Nov  9 16:30:30 debian8 kernel: [    0.852310] pci 0000:00:19.0: reg 0x14: [mem 0xdf13c000-0xdf13cfff]
Nov  9 16:30:30 debian8 systemd[1]: Starting Network Manager...
Nov  9 16:30:30 debian8 kernel: [    0.852316] pci 0000:00:19.0: reg 0x18: [io  0xf060-0xf07f]
Nov  9 16:30:30 debian8 kernel: [    0.852363] pci 0000:00:19.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.852389] pci 0000:00:19.0: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 kernel: [    0.852617] pci 0000:00:1a.0: [8086:8c2d] type 00 class 0x0c0320
Nov  9 16:30:30 debian8 systemd[1]: Starting /etc/rc.local Compatibility...
Nov  9 16:30:30 debian8 kernel: [    0.852631] pci 0000:00:1a.0: reg 0x10: [mem 0xdf13b000-0xdf13b3ff]
Nov  9 16:30:30 debian8 kernel: [    0.852701] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.852737] pci 0000:00:1a.0: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 systemd[1]: Started getty on tty2-tty6 if dbus and logind are not available.
Nov  9 16:30:30 debian8 kernel: [    0.852965] pci 0000:00:1b.0: [8086:8c20] type 00 class 0x040300
Nov  9 16:30:30 debian8 kernel: [    0.852977] pci 0000:00:1b.0: reg 0x10: [mem 0xdf130000-0xdf133fff 64bit]
Nov  9 16:30:30 debian8 kernel: [    0.853032] pci 0000:00:1b.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.853061] pci 0000:00:1b.0: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 systemd[1]: Starting Login Service...
Nov  9 16:30:30 debian8 kernel: [    0.853287] pci 0000:00:1c.0: [8086:8c10] type 01 class 0x060400
Nov  9 16:30:30 debian8 kernel: [    0.853337] pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.853388] pci 0000:00:1c.0: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: start Samba NetBIOS nameserver (nmbd)...
Nov  9 16:30:30 debian8 kernel: [    0.853615] pci 0000:00:1c.1: [8086:8c12] type 01 class 0x060400
Nov  9 16:30:30 debian8 kernel: [    0.853666] pci 0000:00:1c.1: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.853715] pci 0000:00:1c.1: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 kernel: [    0.853942] pci 0000:00:1c.3: [8086:8c16] type 01 class 0x060400
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: exim Mail Transport Agent...
Nov  9 16:30:30 debian8 kernel: [    0.853993] pci 0000:00:1c.3: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.854041] pci 0000:00:1c.3: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 kernel: [    0.854267] pci 0000:00:1c.4: [8086:8c18] type 01 class 0x060400
Nov  9 16:30:30 debian8 kernel: [    0.854319] pci 0000:00:1c.4: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.854367] pci 0000:00:1c.4: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: daemon to balance interrupts for SMP systems...
Nov  9 16:30:30 debian8 kernel: [    0.854599] pci 0000:00:1d.0: [8086:8c26] type 00 class 0x0c0320
Nov  9 16:30:30 debian8 kernel: [    0.854613] pci 0000:00:1d.0: reg 0x10: [mem 0xdf13a000-0xdf13a3ff]
Nov  9 16:30:30 debian8 kernel: [    0.854682] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.854718] pci 0000:00:1d.0: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 kernel: [    0.854947] pci 0000:00:1f.0: [8086:8c44] type 00 class 0x060100
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: git-daemon service...
Nov  9 16:30:30 debian8 kernel: [    0.855076] pci 0000:00:1f.3: [8086:8c22] type 00 class 0x0c0500
Nov  9 16:30:30 debian8 kernel: [    0.855087] pci 0000:00:1f.3: reg 0x10: [mem 0xdf139000-0xdf1390ff 64bit]
Nov  9 16:30:30 debian8 kernel: [    0.855103] pci 0000:00:1f.3: reg 0x20: [io  0xf040-0xf05f]
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: keep memory of all UPnP devices that announced themselves...
Nov  9 16:30:30 debian8 kernel: [    0.855191] pci 0000:01:00.0: [1b21:1242] type 00 class 0x0c0330
Nov  9 16:30:30 debian8 kernel: [    0.855205] pci 0000:01:00.0: reg 0x10: [mem 0xdf000000-0xdf007fff 64bit]
Nov  9 16:30:30 debian8 kernel: [    0.855275] pci 0000:01:00.0: PME# supported from D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.855296] pci 0000:01:00.0: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: start Samba daemons for the AD DC...
Nov  9 16:30:30 debian8 kernel: [    0.867182] pci 0000:00:01.0: PCI bridge to [bus 01]
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: Speech Dispatcher...
Nov  9 16:30:30 debian8 systemd[1]: Starting LSB: Start the GNUstep distributed object mapper...
Nov  9 16:30:30 debian8 systemd[1]: Starting Avahi mDNS/DNS-SD Stack...
Nov  9 16:30:30 debian8 systemd[1]: Starting D-Bus System Message Bus...
Nov  9 16:30:30 debian8 systemd[1]: Started D-Bus System Message Bus.
Nov  9 16:30:30 debian8 bluetoothd[559]: Bluetooth daemon 5.23
Nov  9 16:30:30 debian8 ModemManager[563]: <info>  ModemManager (version 1.4.0) starting in system bus...
Nov  9 16:30:30 debian8 cron[569]: (CRON) INFO (pidfile fd = 3)
Nov  9 16:30:30 debian8 cron[569]: (CRON) INFO (Running @reboot jobs)
Nov  9 16:30:30 debian8 speech-dispatcher[580]: speech-dispatcher disabled; edit /etc/default/speech-dispatcher.
Nov  9 16:30:30 debian8 irqbalance[575]: Starting SMP IRQ Balancer: irqbalance.
Nov  9 16:30:30 debian8 gdomap[581]: GNUstep distributed object mapper disabled, see /etc/default/gdomap.
Nov  9 16:30:30 debian8 minissdpd[630]: setsockopt(udp, IP_ADD_MEMBERSHIP)(0.0.0.0): No such device
Nov  9 16:30:30 debian8 minissdpd[630]: Failed to add IPv4 multicast membership for interface 0.0.0.0.
Nov  9 16:30:30 debian8 kernel: [    0.867362] pci 0000:00:01.0:   bridge window [mem 0xdf000000-0xdf0fffff]
Nov  9 16:30:30 debian8 minissdpd[578]: Starting UPnP devices daemon: MiniSSDPd.
Nov  9 16:30:30 debian8 kernel: [    0.867408] acpiphp: Slot [1] registered
Nov  9 16:30:30 debian8 git-daemon[651]: Ready to rumble
Nov  9 16:30:30 debian8 kernel: [    0.867554] pci 0000:00:1c.0: PCI bridge to [bus 02]
Nov  9 16:30:30 debian8 kernel: [    0.867787] pci 0000:03:00.0: [1b21:0612] type 00 class 0x010601
Nov  9 16:30:30 debian8 kernel: [    0.867803] pci 0000:03:00.0: reg 0x10: [io  0xe050-0xe057]
Nov  9 16:30:30 debian8 kernel: [    0.867812] pci 0000:03:00.0: reg 0x14: [io  0xe040-0xe043]
Nov  9 16:30:30 debian8 kernel: [    0.867822] pci 0000:03:00.0: reg 0x18: [io  0xe030-0xe037]
Nov  9 16:30:30 debian8 avahi-daemon[582]: Found user 'avahi' (UID 106) and group 'avahi' (GID 114).
Nov  9 16:30:30 debian8 kernel: [    0.867831] pci 0000:03:00.0: reg 0x1c: [io  0xe020-0xe023]
Nov  9 16:30:30 debian8 kernel: [    0.867841] pci 0000:03:00.0: reg 0x20: [io  0xe000-0xe01f]
Nov  9 16:30:30 debian8 avahi-daemon[582]: Successfully dropped root privileges.
Nov  9 16:30:30 debian8 kernel: [    0.867850] pci 0000:03:00.0: reg 0x24: [mem 0xdef00000-0xdef001ff]
Nov  9 16:30:30 debian8 kernel: [    0.867927] pci 0000:03:00.0: System wakeup disabled by ACPI
Nov  9 16:30:30 debian8 kernel: [    0.879191] pci 0000:00:1c.1: PCI bridge to [bus 03]
Nov  9 16:30:30 debian8 kernel: [    0.879372] pci 0000:00:1c.1:   bridge window [io  0xe000-0xefff]
Nov  9 16:30:30 debian8 kernel: [    0.879374] pci 0000:00:1c.1:   bridge window [mem 0xdef00000-0xdeffffff]
Nov  9 16:30:30 debian8 avahi-daemon[582]: avahi-daemon 0.6.31 starting up.
Nov  9 16:30:30 debian8 kernel: [    0.879422] pci 0000:04:00.0: [10b5:8608] type 01 class 0x060400
Nov  9 16:30:30 debian8 kernel: [    0.879437] pci 0000:04:00.0: reg 0x10: [mem 0xdee00000-0xdee1ffff]
Nov  9 16:30:30 debian8 kernel: [    0.879531] pci 0000:04:00.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.891197] pci 0000:00:1c.3: PCI bridge to [bus 04-0c]
Nov  9 16:30:30 debian8 avahi-daemon[582]: Successfully called chroot().
Nov  9 16:30:30 debian8 kernel: [    0.891387] pci 0000:00:1c.3:   bridge window [io  0xc000-0xdfff]
Nov  9 16:30:30 debian8 kernel: [    0.891389] pci 0000:00:1c.3:   bridge window [mem 0xde800000-0xdeefffff]
Nov  9 16:30:30 debian8 kernel: [    0.891392] pci 0000:00:1c.3:   bridge window [mem 0xaa100000-0xaa1fffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.891475] pci 0000:05:01.0: [10b5:8608] type 01 class 0x060400
Nov  9 16:30:30 debian8 avahi-daemon[582]: Successfully dropped remaining capabilities.
Nov  9 16:30:30 debian8 kernel: [    0.891581] pci 0000:05:01.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.891646] pci 0000:05:04.0: [10b5:8608] type 01 class 0x060400
Nov  9 16:30:30 debian8 kernel: [    0.891751] pci 0000:05:04.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.891813] pci 0000:05:05.0: [10b5:8608] type 01 class 0x060400
Nov  9 16:30:30 debian8 kernel: [    0.891918] pci 0000:05:05.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.891981] pci 0000:05:06.0: [10b5:8608] type 01 class 0x060400
Nov  9 16:30:30 debian8 avahi-daemon[582]: No service file found in /etc/avahi/services.
Nov  9 16:30:30 debian8 kernel: [    0.892086] pci 0000:05:06.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.892150] pci 0000:05:07.0: [10b5:8608] type 01 class 0x060400
Nov  9 16:30:30 debian8 kernel: [    0.892255] pci 0000:05:07.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 systemd[1]: Started Avahi mDNS/DNS-SD Stack.
Nov  9 16:30:30 debian8 kernel: [    0.892317] pci 0000:05:08.0: [10b5:8608] type 01 class 0x060400
Nov  9 16:30:30 debian8 kernel: [    0.892422] pci 0000:05:08.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.892484] pci 0000:05:09.0: [10b5:8608] type 01 class 0x060400
Nov  9 16:30:30 debian8 kernel: [    0.892588] pci 0000:05:09.0: PME# supported from D0 D3hot D3cold
Nov  9 16:30:30 debian8 systemd[1]: Started Bluetooth service.
Nov  9 16:30:30 debian8 kernel: [    0.892664] pci 0000:04:00.0: PCI bridge to [bus 05-0c]
Nov  9 16:30:30 debian8 kernel: [    0.892858] pci 0000:04:00.0:   bridge window [io  0xc000-0xdfff]
Nov  9 16:30:30 debian8 kernel: [    0.892861] pci 0000:04:00.0:   bridge window [mem 0xde800000-0xdedfffff]
Nov  9 16:30:30 debian8 kernel: [    0.892866] pci 0000:04:00.0:   bridge window [mem 0xaa100000-0xaa1fffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.892904] pci 0000:05:01.0: PCI bridge to [bus 06]
Nov  9 16:30:30 debian8 systemd[1]: Starting Bluetooth.
Nov  9 16:30:30 debian8 kernel: [    0.893133] pci 0000:05:04.0: PCI bridge to [bus 07]
Nov  9 16:30:30 debian8 kernel: [    0.893379] pci 0000:08:00.0: [10ec:8168] type 00 class 0x020000
Nov  9 16:30:30 debian8 systemd[1]: Reached target Bluetooth.
Nov  9 16:30:30 debian8 kernel: [    0.893401] pci 0000:08:00.0: reg 0x10: [io  0xd000-0xd0ff]
Nov  9 16:30:30 debian8 kernel: [    0.893433] pci 0000:08:00.0: reg 0x18: [mem 0xded00000-0xded00fff 64bit]
Nov  9 16:30:30 debian8 kernel: [    0.893453] pci 0000:08:00.0: reg 0x20: [mem 0xaa100000-0xaa103fff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.893563] pci 0000:08:00.0: supports D1 D2
Nov  9 16:30:30 debian8 kernel: [    0.893564] pci 0000:08:00.0: PME# supported from D0 D1 D2 D3hot D3cold
Nov  9 16:30:30 debian8 systemd[1]: Starting System Logging Service...
Nov  9 16:30:30 debian8 kernel: [    0.903211] pci 0000:05:05.0: PCI bridge to [bus 08]
Nov  9 16:30:30 debian8 kernel: [    0.903395] pci 0000:05:05.0:   bridge window [io  0xd000-0xdfff]
Nov  9 16:30:30 debian8 systemd[1]: Starting Permit User Sessions...
Nov  9 16:30:30 debian8 kernel: [    0.903398] pci 0000:05:05.0:   bridge window [mem 0xded00000-0xdedfffff]
Nov  9 16:30:30 debian8 kernel: [    0.903403] pci 0000:05:05.0:   bridge window [mem 0xaa100000-0xaa1fffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.903443] pci 0000:05:06.0: PCI bridge to [bus 09]
Nov  9 16:30:30 debian8 kernel: [    0.903691] pci 0000:0a:00.0: [14e4:43b1] type 00 class 0x028000
Nov  9 16:30:30 debian8 kernel: [    0.903719] pci 0000:0a:00.0: reg 0x10: [mem 0xdea00000-0xdea07fff 64bit]
Nov  9 16:30:30 debian8 kernel: [    0.903739] pci 0000:0a:00.0: reg 0x18: [mem 0xde800000-0xde9fffff 64bit]
Nov  9 16:30:30 debian8 systemd[1]: Starting ACPI event daemon...
Nov  9 16:30:30 debian8 kernel: [    0.903879] pci 0000:0a:00.0: supports D1 D2
Nov  9 16:30:30 debian8 kernel: [    0.903880] pci 0000:0a:00.0: PME# supported from D0 D1 D2 D3hot D3cold
Nov  9 16:30:30 debian8 kernel: [    0.915217] pci 0000:05:07.0: PCI bridge to [bus 0a]
Nov  9 16:30:30 debian8 kernel: [    0.915404] pci 0000:05:07.0:   bridge window [mem 0xde800000-0xdeafffff]
Nov  9 16:30:30 debian8 avahi-daemon[582]: Network interface enumeration completed.
Nov  9 16:30:30 debian8 kernel: [    0.915447] pci 0000:05:08.0: PCI bridge to [bus 0b]
Nov  9 16:30:30 debian8 kernel: [    0.915692] pci 0000:0c:00.0: [1b21:0612] type 00 class 0x010601
Nov  9 16:30:30 debian8 kernel: [    0.915712] pci 0000:0c:00.0: reg 0x10: [io  0xc050-0xc057]
Nov  9 16:30:30 debian8 avahi-daemon[582]: Registering HINFO record with values 'X86_64'/'LINUX'.
Nov  9 16:30:30 debian8 kernel: [    0.915725] pci 0000:0c:00.0: reg 0x14: [io  0xc040-0xc043]
Nov  9 16:30:30 debian8 kernel: [    0.915737] pci 0000:0c:00.0: reg 0x18: [io  0xc030-0xc037]
Nov  9 16:30:30 debian8 kernel: [    0.915750] pci 0000:0c:00.0: reg 0x1c: [io  0xc020-0xc023]
Nov  9 16:30:30 debian8 kernel: [    0.915762] pci 0000:0c:00.0: reg 0x20: [io  0xc000-0xc01f]
Nov  9 16:30:30 debian8 kernel: [    0.915774] pci 0000:0c:00.0: reg 0x24: [mem 0xdec00000-0xdec001ff]
Nov  9 16:30:30 debian8 avahi-daemon[582]: Server startup complete. Host name is debian8.local. Local service cookie is 2384464375.
Nov  9 16:30:30 debian8 kernel: [    0.927222] pci 0000:05:09.0: PCI bridge to [bus 0c]
Nov  9 16:30:30 debian8 kernel: [    0.927406] pci 0000:05:09.0:   bridge window [io  0xc000-0xcfff]
Nov  9 16:30:30 debian8 bluetoothd[559]: Starting SDP server
Nov  9 16:30:30 debian8 kernel: [    0.927409] pci 0000:05:09.0:   bridge window [mem 0xdec00000-0xdecfffff]
Nov  9 16:30:30 debian8 kernel: [    0.927499] acpiphp: Slot [1-1] registered
Nov  9 16:30:30 debian8 kernel: [    0.927651] pci 0000:00:1c.4: PCI bridge to [bus 0d-77]
Nov  9 16:30:30 debian8 kernel: [    0.927842] pci 0000:00:1c.4:   bridge window [mem 0xb0000000-0xde0fffff]
Nov  9 16:30:30 debian8 kernel: [    0.927845] pci 0000:00:1c.4:   bridge window [mem 0x60000000-0xa9ffffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.928305] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15)
Nov  9 16:30:30 debian8 systemd[1]: Started ACPI event daemon.
Nov  9 16:30:30 debian8 kernel: [    0.928663] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 *4 5 6 7 10 11 12 14 15)
Nov  9 16:30:30 debian8 kernel: [    0.929019] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 10 11 12 *14 15)
Nov  9 16:30:30 debian8 kernel: [    0.929374] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 10 11 12 14 *15)
Nov  9 16:30:30 debian8 kernel: [    0.929729] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 *5 6 7 10 11 12 14 15)
Nov  9 16:30:30 debian8 systemd[1]: Starting CUPS Printing Service...
Nov  9 16:30:30 debian8 kernel: [    0.930085] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 *10 11 12 14 15)
Nov  9 16:30:30 debian8 kernel: [    0.930439] ACPI: PCI Interrupt Link [LNKG] (IRQs *3 4 5 6 7 10 11 12 14 15)
Nov  9 16:30:30 debian8 kernel: [    0.930794] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 *6 7 10 11 12 14 15)
Nov  9 16:30:30 debian8 systemd[1]: Started CUPS Printing Service.
Nov  9 16:30:30 debian8 kernel: [    0.931338] ACPI: Enabled 6 GPEs in block 00 to 3F
Nov  9 16:30:30 debian8 kernel: [    0.931577] vgaarb: setting as boot device: PCI:0000:00:02.0
Nov  9 16:30:30 debian8 kernel: [    0.931781] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
Nov  9 16:30:30 debian8 kernel: [    0.932071] vgaarb: loaded
Nov  9 16:30:30 debian8 kernel: [    0.932172] vgaarb: bridge control possible 0000:00:02.0
Nov  9 16:30:30 debian8 systemd[1]: Starting Make remote CUPS printers available locally...
Nov  9 16:30:30 debian8 kernel: [    0.935148] PCI: Using ACPI for IRQ routing
Nov  9 16:30:30 debian8 kernel: [    0.940190] PCI: pci_cache_line_size set to 64 bytes
Nov  9 16:30:30 debian8 systemd[1]: Started Make remote CUPS printers available locally.
Nov  9 16:30:30 debian8 kernel: [    0.940251] e820: reserve RAM buffer [mem 0x00058000-0x0005ffff]
Nov  9 16:30:30 debian8 kernel: [    0.940252] e820: reserve RAM buffer [mem 0x0009f000-0x0009ffff]
Nov  9 16:30:30 debian8 kernel: [    0.940253] e820: reserve RAM buffer [mem 0x3508e000-0x37ffffff]
Nov  9 16:30:30 debian8 kernel: [    0.940253] e820: reserve RAM buffer [mem 0x354ed000-0x37ffffff]
Nov  9 16:30:30 debian8 kernel: [    0.940254] e820: reserve RAM buffer [mem 0x486c5000-0x4bffffff]
Nov  9 16:30:30 debian8 systemd[1]: Started Restore Sound Card State.
Nov  9 16:30:30 debian8 kernel: [    0.940254] e820: reserve RAM buffer [mem 0x4a000000-0x4bffffff]
Nov  9 16:30:30 debian8 kernel: [    0.940255] e820: reserve RAM buffer [mem 0x1afe00000-0x1afffffff]
Nov  9 16:30:30 debian8 systemd[1]: Started Restore /etc/resolv.conf if the system crashed before the ppp link was shut down..
Nov  9 16:30:30 debian8 kernel: [    0.940344] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
Nov  9 16:30:30 debian8 kernel: [    0.940623] hpet0: 8 comparators, 64-bit 14.318180 MHz counter
Nov  9 16:30:30 debian8 kernel: [    0.942846] clocksource: Switched to clocksource hpet
Nov  9 16:30:30 debian8 kernel: [    0.947004] VFS: Disk quotas dquot_6.6.0
Nov  9 16:30:30 debian8 kernel: [    0.962865] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Nov  9 16:30:30 debian8 kernel: [    0.978911] pnp: PnP ACPI init
Nov  9 16:30:30 debian8 kernel: [    0.979074] system 00:00: [mem 0xfed40000-0xfed44fff] has been reserved
Nov  9 16:30:30 debian8 systemd[1]: Started /etc/rc.local Compatibility.
Nov  9 16:30:30 debian8 kernel: [    0.979312] system 00:00: Plug and Play ACPI device, IDs PNP0c01 (active)
Nov  9 16:30:30 debian8 kernel: [    0.979425] system 00:01: [io  0x0680-0x069f] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.979638] system 00:01: [io  0xffff] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.979830] system 00:01: [io  0xffff] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.980022] system 00:01: [io  0xffff] has been reserved
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: daemon to balance interrupts for SMP systems.
Nov  9 16:30:30 debian8 kernel: [    0.980214] system 00:01: [io  0x1c00-0x1cfe] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.980427] system 00:01: [io  0x1d00-0x1dfe] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.980640] system 00:01: [io  0x1e00-0x1efe] has been reserved
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: git-daemon service.
Nov  9 16:30:30 debian8 kernel: [    0.980853] system 00:01: [io  0x1f00-0x1ffe] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.981066] system 00:01: [io  0x1800-0x18fe] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.981279] system 00:01: [io  0x164e-0x164f] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.981492] system 00:01: Plug and Play ACPI device, IDs PNP0c02 (active)
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: keep memory of all UPnP devices that announced themselves.
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: Speech Dispatcher.
Nov  9 16:30:30 debian8 systemd[1]: Started LSB: Start the GNUstep distributed object mapper.
Nov  9 16:30:30 debian8 dbus[586]: [system] Activating via systemd: service name='org.freedesktop.PolicyKit1' unit='polkitd.service'
Nov  9 16:30:30 debian8 dbus[586]: [system] Activating via systemd: service name='org.freedesktop.hostname1' unit='dbus-org.freedesktop.hostname1.service'
Nov  9 16:30:30 debian8 bluetoothd[559]: Bluetooth management interface 1.13 initialized
Nov  9 16:30:30 debian8 systemd[1]: Started Permit User Sessions.
Nov  9 16:30:30 debian8 systemd[1]: Started Login Service.
Nov  9 16:30:30 debian8 systemd[1]: Starting Hostname Service...
Nov  9 16:30:30 debian8 systemd[1]: Starting Light Display Manager...
Nov  9 16:30:30 debian8 systemd[1]: Starting Authenticate and Authorize Users to Run Privileged Tasks...
Nov  9 16:30:30 debian8 systemd[1]: Starting Getty on tty1...
Nov  9 16:30:30 debian8 systemd[1]: Started Getty on tty1.
Nov  9 16:30:30 debian8 kernel: [    0.981508] pnp 00:02: Plug and Play ACPI device, IDs PNP0b00 (active)
Nov  9 16:30:30 debian8 systemd[1]: Starting Login Prompts.
Nov  9 16:30:30 debian8 kernel: [    0.981539] system 00:03: [io  0x1854-0x1857] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.981752] system 00:03: Plug and Play ACPI device, IDs INT3f0d PNP0c02 (active)
Nov  9 16:30:30 debian8 kernel: [    0.981807] system 00:04: [io  0x0290-0x029f] has been reserved
Nov  9 16:30:30 debian8 systemd[1]: Reached target Login Prompts.
Nov  9 16:30:30 debian8 kernel: [    0.982020] system 00:04: Plug and Play ACPI device, IDs PNP0c02 (active)
Nov  9 16:30:30 debian8 kernel: [    0.982054] system 00:05: [io  0x04d0-0x04d1] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.982267] system 00:05: Plug and Play ACPI device, IDs PNP0c02 (active)
Nov  9 16:30:30 debian8 kernel: [    0.982545] system 00:06: [mem 0xfed1c000-0xfed1ffff] has been reserved
Nov  9 16:30:30 debian8 bluetoothd[559]: Sap driver initialization failed.
Nov  9 16:30:30 debian8 bluetoothd[559]: sap-server: Operation not permitted (1)
Nov  9 16:30:30 debian8 kernel: [    0.982782] system 00:06: [mem 0xfed10000-0xfed17fff] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.983025] system 00:06: [mem 0xfed18000-0xfed18fff] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.983264] system 00:06: [mem 0xfed19000-0xfed19fff] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.983501] system 00:06: [mem 0xe0000000-0xefffffff] has been reserved
Nov  9 16:30:30 debian8 acpid: starting up with netlink and the input layer
Nov  9 16:30:30 debian8 kernel: [    0.983739] system 00:06: [mem 0xfed20000-0xfed3ffff] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.983976] system 00:06: [mem 0xfed90000-0xfed93fff] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.984213] system 00:06: [mem 0xfed45000-0xfed8ffff] has been reserved
Nov  9 16:30:30 debian8 dbus[586]: [system] Successfully activated service 'org.freedesktop.hostname1'
Nov  9 16:30:30 debian8 kernel: [    0.984450] system 00:06: [mem 0xff000000-0xffffffff] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.984687] system 00:06: [mem 0xfee00000-0xfeefffff] could not be reserved
Nov  9 16:30:30 debian8 kernel: [    0.984937] system 00:06: [mem 0xdffdf000-0xdffdffff] has been reserved
Nov  9 16:30:30 debian8 kernel: [    0.985174] system 00:06: [mem 0xdffe0000-0xdffeffff] has been reserved
Nov  9 16:30:30 debian8 systemd[1]: Started Hostname Service.
Nov  9 16:30:30 debian8 kernel: [    0.985412] system 00:06: Plug and Play ACPI device, IDs PNP0c02 (active)
Nov  9 16:30:30 debian8 kernel: [    0.985554] pnp: PnP ACPI: found 7 devices
Nov  9 16:30:30 debian8 kernel: [    0.991383] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
Nov  9 16:30:30 debian8 kernel: [    0.991780] pci 0000:00:1c.4: bridge window [io  0x1000-0x0fff] to [bus 0d-77] add_size 1000
Nov  9 16:30:30 debian8 kernel: [    0.991782] pci 0000:00:1c.4: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:30:30 debian8 kernel: [    0.991783] pci 0000:00:1c.4: res[13]=[io  0x1000-0x1fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:30:30 debian8 kernel: [    0.991785] pci 0000:00:1c.4: BAR 13: assigned [io  0x2000-0x2fff]
Nov  9 16:30:30 debian8 acpid: 1 rule loaded
Nov  9 16:30:30 debian8 kernel: [    0.992008] pci 0000:00:01.0: PCI bridge to [bus 01]
Nov  9 16:30:30 debian8 kernel: [    0.992189] pci 0000:00:01.0:   bridge window [mem 0xdf000000-0xdf0fffff]
Nov  9 16:30:30 debian8 kernel: [    0.992434] pci 0000:00:1c.0: PCI bridge to [bus 02]
Nov  9 16:30:30 debian8 kernel: [    0.992620] pci 0000:00:1c.1: PCI bridge to [bus 03]
Nov  9 16:30:30 debian8 kernel: [    0.992800] pci 0000:00:1c.1:   bridge window [io  0xe000-0xefff]
Nov  9 16:30:30 debian8 kernel: [    0.993021] pci 0000:00:1c.1:   bridge window [mem 0xdef00000-0xdeffffff]
Nov  9 16:30:30 debian8 acpid: waiting for events: event logging is off
Nov  9 16:30:30 debian8 kernel: [    0.993269] pci 0000:05:01.0: PCI bridge to [bus 06]
Nov  9 16:30:30 debian8 kernel: [    0.993459] pci 0000:05:04.0: PCI bridge to [bus 07]
Nov  9 16:30:30 debian8 systemd[1]: Started System Logging Service.
Nov  9 16:30:30 debian8 kernel: [    0.993650] pci 0000:05:05.0: PCI bridge to [bus 08]
Nov  9 16:30:30 debian8 kernel: [    0.993831] pci 0000:05:05.0:   bridge window [io  0xd000-0xdfff]
Nov  9 16:30:30 debian8 kernel: [    0.994054] pci 0000:05:05.0:   bridge window [mem 0xded00000-0xdedfffff]
Nov  9 16:30:30 debian8 kernel: [    0.994299] pci 0000:05:05.0:   bridge window [mem 0xaa100000-0xaa1fffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.994580] pci 0000:05:06.0: PCI bridge to [bus 09]
Nov  9 16:30:30 debian8 kernel: [    0.994771] pci 0000:05:07.0: PCI bridge to [bus 0a]
Nov  9 16:30:30 debian8 kernel: [    0.994957] pci 0000:05:07.0:   bridge window [mem 0xde800000-0xdeafffff]
Nov  9 16:30:30 debian8 kernel: [    0.995208] pci 0000:05:08.0: PCI bridge to [bus 0b]
Nov  9 16:30:30 debian8 kernel: [    0.995398] pci 0000:05:09.0: PCI bridge to [bus 0c]
Nov  9 16:30:30 debian8 kernel: [    0.995579] pci 0000:05:09.0:   bridge window [io  0xc000-0xcfff]
Nov  9 16:30:30 debian8 kernel: [    0.995801] pci 0000:05:09.0:   bridge window [mem 0xdec00000-0xdecfffff]
Nov  9 16:30:30 debian8 kernel: [    0.996051] pci 0000:04:00.0: PCI bridge to [bus 05-0c]
Nov  9 16:30:30 debian8 kernel: [    0.996241] pci 0000:04:00.0:   bridge window [io  0xc000-0xdfff]
Nov  9 16:30:30 debian8 kernel: [    0.996463] pci 0000:04:00.0:   bridge window [mem 0xde800000-0xdedfffff]
Nov  9 16:30:30 debian8 kernel: [    0.996708] pci 0000:04:00.0:   bridge window [mem 0xaa100000-0xaa1fffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.996990] pci 0000:00:1c.3: PCI bridge to [bus 04-0c]
Nov  9 16:30:30 debian8 kernel: [    0.997179] pci 0000:00:1c.3:   bridge window [io  0xc000-0xdfff]
Nov  9 16:30:30 debian8 kernel: [    0.997400] pci 0000:00:1c.3:   bridge window [mem 0xde800000-0xdeefffff]
Nov  9 16:30:30 debian8 kernel: [    0.997644] pci 0000:00:1c.3:   bridge window [mem 0xaa100000-0xaa1fffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.997923] pci 0000:00:1c.4: PCI bridge to [bus 0d-77]
Nov  9 16:30:30 debian8 kernel: [    0.998113] pci 0000:00:1c.4:   bridge window [io  0x2000-0x2fff]
Nov  9 16:30:30 debian8 kernel: [    0.998333] pci 0000:00:1c.4:   bridge window [mem 0xb0000000-0xde0fffff]
Nov  9 16:30:30 debian8 kernel: [    0.998578] pci 0000:00:1c.4:   bridge window [mem 0x60000000-0xa9ffffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.998860] pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7 window]
Nov  9 16:30:30 debian8 kernel: [    0.998861] pci_bus 0000:00: resource 5 [io  0x0d00-0xffff window]
Nov  9 16:30:30 debian8 kernel: [    0.998862] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]
Nov  9 16:30:30 debian8 kernel: [    0.998862] pci_bus 0000:00: resource 7 [mem 0x000d0000-0x000d3fff window]
Nov  9 16:30:30 debian8 kernel: [    0.998863] pci_bus 0000:00: resource 8 [mem 0x000d4000-0x000d7fff window]
Nov  9 16:30:30 debian8 kernel: [    0.998863] pci_bus 0000:00: resource 9 [mem 0x000d8000-0x000dbfff window]
Nov  9 16:30:30 debian8 kernel: [    0.998864] pci_bus 0000:00: resource 10 [mem 0x000dc000-0x000dffff window]
Nov  9 16:30:30 debian8 kernel: [    0.998865] pci_bus 0000:00: resource 11 [mem 0x4f200000-0xfeafffff window]
Nov  9 16:30:30 debian8 kernel: [    0.998865] pci_bus 0000:01: resource 1 [mem 0xdf000000-0xdf0fffff]
Nov  9 16:30:30 debian8 kernel: [    0.998866] pci_bus 0000:03: resource 0 [io  0xe000-0xefff]
Nov  9 16:30:30 debian8 kernel: [    0.998867] pci_bus 0000:03: resource 1 [mem 0xdef00000-0xdeffffff]
Nov  9 16:30:30 debian8 kernel: [    0.998867] pci_bus 0000:04: resource 0 [io  0xc000-0xdfff]
Nov  9 16:30:30 debian8 kernel: [    0.998868] pci_bus 0000:04: resource 1 [mem 0xde800000-0xdeefffff]
Nov  9 16:30:30 debian8 kernel: [    0.998869] pci_bus 0000:04: resource 2 [mem 0xaa100000-0xaa1fffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.998869] pci_bus 0000:05: resource 0 [io  0xc000-0xdfff]
Nov  9 16:30:30 debian8 kernel: [    0.998870] pci_bus 0000:05: resource 1 [mem 0xde800000-0xdedfffff]
Nov  9 16:30:30 debian8 kernel: [    0.998871] pci_bus 0000:05: resource 2 [mem 0xaa100000-0xaa1fffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.998871] pci_bus 0000:08: resource 0 [io  0xd000-0xdfff]
Nov  9 16:30:30 debian8 kernel: [    0.998872] pci_bus 0000:08: resource 1 [mem 0xded00000-0xdedfffff]
Nov  9 16:30:30 debian8 kernel: [    0.998873] pci_bus 0000:08: resource 2 [mem 0xaa100000-0xaa1fffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    0.998874] pci_bus 0000:0a: resource 1 [mem 0xde800000-0xdeafffff]
Nov  9 16:30:30 debian8 kernel: [    0.998874] pci_bus 0000:0c: resource 0 [io  0xc000-0xcfff]
Nov  9 16:30:30 debian8 kernel: [    0.998875] pci_bus 0000:0c: resource 1 [mem 0xdec00000-0xdecfffff]
Nov  9 16:30:30 debian8 kernel: [    0.998876] pci_bus 0000:0d: resource 0 [io  0x2000-0x2fff]
Nov  9 16:30:30 debian8 kernel: [    0.998876] pci_bus 0000:0d: resource 1 [mem 0xb0000000-0xde0fffff]
Nov  9 16:30:30 debian8 kernel: [    0.998877] pci_bus 0000:0d: resource 2 [mem 0x60000000-0xa9ffffff 64bit pref]
Nov  9 16:30:30 debian8 kernel: [    1.078856] NET: Registered protocol family 2
Nov  9 16:30:30 debian8 kernel: [    1.094888] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
Nov  9 16:30:30 debian8 kernel: [    1.095209] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
Nov  9 16:30:30 debian8 kernel: [    1.095545] TCP: Hash tables configured (established 32768 bind 32768)
Nov  9 16:30:30 debian8 kernel: [    1.095795] UDP hash table entries: 2048 (order: 4, 65536 bytes)
Nov  9 16:30:30 debian8 kernel: [    1.096026] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
Nov  9 16:30:30 debian8 kernel: [    1.096300] NET: Registered protocol family 1
Nov  9 16:30:30 debian8 kernel: [    1.096467] pci 0000:00:02.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
Nov  9 16:30:30 debian8 kernel: [    1.118905] PCI: CLS mismatch (64 != 128), using 64 bytes
Nov  9 16:30:30 debian8 kernel: [    1.139004] Unpacking initramfs...
Nov  9 16:30:30 debian8 kernel: [    3.234304] Freeing initrd memory: 190912K (ffff988257560000 - ffff988262fd0000)
Nov  9 16:30:30 debian8 kernel: [    3.234572] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Nov  9 16:30:30 debian8 kernel: [    3.234813] software IO TLB [mem 0x3636d000-0x3a36d000] (64MB) mapped at [ffff98827636d000-ffff98827a36cfff]
Nov  9 16:30:30 debian8 kernel: [    3.235255] RAPL PMU: API unit is 2^-32 Joules, 4 fixed counters, 655360 ms ovfl timer
Nov  9 16:30:30 debian8 kernel: [    3.235538] RAPL PMU: hw unit of domain pp0-core 2^-14 Joules
Nov  9 16:30:30 debian8 kernel: [    3.235744] RAPL PMU: hw unit of domain package 2^-14 Joules
Nov  9 16:30:30 debian8 kernel: [    3.235948] RAPL PMU: hw unit of domain dram 2^-14 Joules
Nov  9 16:30:30 debian8 kernel: [    3.236142] RAPL PMU: hw unit of domain pp1-gpu 2^-14 Joules
Nov  9 16:30:30 debian8 kernel: [    3.236600] simple-framebuffer simple-framebuffer.0: framebuffer at 0x50000000, 0x300000 bytes, mapped to 0xffffb6fe80800000
Nov  9 16:30:30 debian8 kernel: [    3.236999] simple-framebuffer simple-framebuffer.0: format=a8r8g8b8, mode=1024x768x32, linelength=4096
Nov  9 16:30:30 debian8 kernel: [    3.238424] Console: switching to colour frame buffer device 128x48
Nov  9 16:30:30 debian8 kernel: [    3.239672] simple-framebuffer simple-framebuffer.0: fb0: simplefb registered!
Nov  9 16:30:30 debian8 kernel: [    3.258871] futex hash table entries: 2048 (order: 5, 131072 bytes)
Nov  9 16:30:30 debian8 kernel: [    3.274750] audit: initializing netlink subsys (disabled)
Nov  9 16:30:30 debian8 kernel: [    3.274959] audit: type=2000 audit(1478705418.264:1): initialized
Nov  9 16:30:30 debian8 kernel: [    3.275579] workingset: timestamp_bits=40 max_order=20 bucket_order=0
Nov  9 16:30:30 debian8 kernel: [    3.275818] zbud: loaded
Nov  9 16:30:30 debian8 kernel: [    3.513227] Key type asymmetric registered
Nov  9 16:30:30 debian8 kernel: [    3.513377] Asymmetric key parser 'x509' registered
Nov  9 16:30:30 debian8 kernel: [    3.534741] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
Nov  9 16:30:30 debian8 kernel: [    3.535034] io scheduler noop registered
Nov  9 16:30:30 debian8 kernel: [    3.535177] io scheduler deadline registered
Nov  9 16:30:30 debian8 kernel: [    3.535335] io scheduler cfq registered (default)
Nov  9 16:30:30 debian8 kernel: [    3.536951] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
Nov  9 16:30:30 debian8 kernel: [    3.537154] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
Nov  9 16:30:30 debian8 kernel: [    3.537399] intel_idle: MWAIT substates: 0x42120
Nov  9 16:30:30 debian8 kernel: [    3.537399] intel_idle: v0.4.1 model 0x3C
Nov  9 16:30:30 debian8 kernel: [    3.537589] intel_idle: lapic_timer_reliable_states 0xffffffff
Nov  9 16:30:30 debian8 kernel: [    3.537720] GHES: HEST is not enabled!
Nov  9 16:30:30 debian8 kernel: [    3.537887] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
Nov  9 16:30:30 debian8 kernel: [    3.538380] Linux agpgart interface v0.103
Nov  9 16:30:30 debian8 kernel: [    3.538559] AMD IOMMUv2 driver by Joerg Roedel <jroedel@suse.de>
Nov  9 16:30:30 debian8 kernel: [    3.538785] AMD IOMMUv2 functionality not available on this system
Nov  9 16:30:30 debian8 kernel: [    3.539366] i8042: PNP: No PS/2 controller found. Probing ports directly.
Nov  9 16:30:30 debian8 kernel: [    3.541968] serio: i8042 KBD port at 0x60,0x64 irq 1
Nov  9 16:30:30 debian8 kernel: [    3.542150] serio: i8042 AUX port at 0x60,0x64 irq 12
Nov  9 16:30:30 debian8 kernel: [    3.542430] mousedev: PS/2 mouse device common for all mice
Nov  9 16:30:30 debian8 kernel: [    3.542657] rtc_cmos 00:02: RTC can wake from S4
Nov  9 16:30:30 debian8 kernel: [    3.542934] rtc_cmos 00:02: rtc core: registered rtc_cmos as rtc0
Nov  9 16:30:30 debian8 kernel: [    3.543172] rtc_cmos 00:02: alarms up to one month, y3k, 242 bytes nvram, hpet irqs
Nov  9 16:30:30 debian8 kernel: [    3.543450] intel_pstate: Intel P-state driver initializing
Nov  9 16:30:30 debian8 kernel: [    3.543953] ledtrig-cpu: registered to indicate activity on CPUs
Nov  9 16:30:30 debian8 kernel: [    3.606773] NET: Registered protocol family 10
Nov  9 16:30:30 debian8 kernel: [    3.626798] mip6: Mobile IPv6
Nov  9 16:30:30 debian8 kernel: [    3.626924] NET: Registered protocol family 17
Nov  9 16:30:30 debian8 kernel: [    3.627105] mpls_gso: MPLS GSO support
Nov  9 16:30:30 debian8 kernel: [    3.627858] microcode: sig=0x306c3, pf=0x2, revision=0x19
Nov  9 16:30:30 debian8 kernel: [    3.628350] microcode: Microcode Update Driver: v2.01 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
Nov  9 16:30:30 debian8 kernel: [    3.629196] registered taskstats version 1
Nov  9 16:30:30 debian8 kernel: [    3.629393] zswap: loaded using pool lzo/zbud
Nov  9 16:30:30 debian8 kernel: [    3.630938] rtc_cmos 00:02: setting system clock to 2016-11-09 15:30:18 UTC (1478705418)
Nov  9 16:30:30 debian8 kernel: [    3.631359] PM: Hibernation image not present or could not be loaded.
Nov  9 16:30:30 debian8 kernel: [    3.632576] Freeing unused kernel memory: 1376K (ffffffffba136000 - ffffffffba28e000)
Nov  9 16:30:30 debian8 kernel: [    3.632874] Write protecting the kernel read-only data: 10240k
Nov  9 16:30:30 debian8 kernel: [    3.633337] Freeing unused kernel memory: 44K (ffff9883c11f5000 - ffff9883c1200000)
Nov  9 16:30:30 debian8 kernel: [    3.637353] Freeing unused kernel memory: 1320K (ffff9883c14b6000 - ffff9883c1600000)
Nov  9 16:30:30 debian8 kernel: [    3.645454] x86/mm: Checked W+X mappings: passed, no W+X pages found.
Nov  9 16:30:30 debian8 kernel: [    3.653777] random: systemd-udevd: uninitialized urandom read (16 bytes read)
Nov  9 16:30:30 debian8 kernel: [    3.659586] FUJITSU Extended Socket Network Device Driver - version 1.1 - Copyright (c) 2015 FUJITSU LIMITED
Nov  9 16:30:30 debian8 kernel: [    3.665227] thermal LNXTHERM:00: registered as thermal_zone0
Nov  9 16:30:30 debian8 kernel: [    3.665454] ACPI: Thermal Zone [TZ00] (28 C)
Nov  9 16:30:30 debian8 kernel: [    3.665871] pps_core: LinuxPPS API ver. 1 registered
Nov  9 16:30:30 debian8 kernel: [    3.665988] thermal LNXTHERM:01: registered as thermal_zone1
Nov  9 16:30:30 debian8 kernel: [    3.665988] ACPI: Thermal Zone [TZ01] (30 C)
Nov  9 16:30:30 debian8 kernel: [    3.666456] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
Nov  9 16:30:30 debian8 kernel: [    3.668256] PTP clock support registered
Nov  9 16:30:30 debian8 kernel: [    3.671405] r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
Nov  9 16:30:30 debian8 kernel: [    3.672237] r8169 0000:08:00.0: can't disable ASPM; OS doesn't have ASPM control
Nov  9 16:30:30 debian8 kernel: [    3.674776] SCSI subsystem initialized
Nov  9 16:30:30 debian8 kernel: [    3.675759] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
Nov  9 16:30:30 debian8 kernel: [    3.675818] ACPI: bus type USB registered
Nov  9 16:30:30 debian8 kernel: [    3.675863] usbcore: registered new interface driver usbfs
Nov  9 16:30:30 debian8 kernel: [    3.675878] usbcore: registered new interface driver hub
Nov  9 16:30:30 debian8 kernel: [    3.675934] usbcore: registered new device driver usb
Nov  9 16:30:30 debian8 kernel: [    3.676797] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
Nov  9 16:30:30 debian8 kernel: [    3.677493] e1000e 0000:00:19.0: Interrupt Throttling Rate (ints/sec) set to dynamic conservative mode
Nov  9 16:30:30 debian8 kernel: [    3.678883] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
Nov  9 16:30:30 debian8 kernel: [    3.680305] libata version 3.00 loaded.
Nov  9 16:30:30 debian8 kernel: [    3.680782] ehci-pci: EHCI PCI platform driver
Nov  9 16:30:30 debian8 kernel: [    3.681403] ahci 0000:03:00.0: version 3.0
Nov  9 16:30:30 debian8 kernel: [    3.681531] ahci 0000:03:00.0: SSS flag set, parallel bus scan disabled
Nov  9 16:30:30 debian8 kernel: [    3.681830] ahci 0000:03:00.0: AHCI 0001.0200 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
Nov  9 16:30:30 debian8 kernel: [    3.682156] ahci 0000:03:00.0: flags: 64bit ncq sntf stag led clo pmp pio slum part ccc 
Nov  9 16:30:30 debian8 kernel: [    3.683055] scsi host0: ahci
Nov  9 16:30:30 debian8 kernel: [    3.683267] scsi host1: ahci
Nov  9 16:30:30 debian8 kernel: [    3.683437] ata1: SATA max UDMA/133 abar m512@0xdef00000 port 0xdef00100 irq 34
Nov  9 16:30:30 debian8 kernel: [    3.683732] ata2: SATA max UDMA/133 abar m512@0xdef00000 port 0xdef00180 irq 34
Nov  9 16:30:30 debian8 kernel: [    3.684142] ahci 0000:0c:00.0: SSS flag set, parallel bus scan disabled
Nov  9 16:30:30 debian8 kernel: [    3.684457] ahci 0000:0c:00.0: AHCI 0001.0200 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
Nov  9 16:30:30 debian8 kernel: [    3.684789] ahci 0000:0c:00.0: flags: 64bit ncq sntf stag led clo pmp pio slum part ccc 
Nov  9 16:30:30 debian8 kernel: [    3.685412] scsi host2: ahci
Nov  9 16:30:30 debian8 kernel: [    3.685610] scsi host3: ahci
Nov  9 16:30:30 debian8 kernel: [    3.685763] ata3: SATA max UDMA/133 abar m512@0xdec00000 port 0xdec00100 irq 35
Nov  9 16:30:30 debian8 kernel: [    3.686026] ata4: SATA max UDMA/133 abar m512@0xdec00000 port 0xdec00180 irq 35
Nov  9 16:30:30 debian8 kernel: [    3.691605] r8169 0000:08:00.0 eth0: RTL8168g/8111g at 0xffffb6fe8074e000, 00:e0:4c:68:00:3d, XID 0c000800 IRQ 36
Nov  9 16:30:30 debian8 kernel: [    3.691980] r8169 0000:08:00.0 eth0: jumbo features [frames: 9200 bytes, tx checksumming: ko]
Nov  9 16:30:30 debian8 kernel: [    3.770626] e1000e 0000:00:19.0 0000:00:19.0 (uninitialized): registered PHC clock
Nov  9 16:30:30 debian8 kernel: [    3.859033] e1000e 0000:00:19.0 eth1: (PCI Express:2.5GT/s:Width x1) 88:88:88:88:87:88
Nov  9 16:30:30 debian8 kernel: [    3.859327] e1000e 0000:00:19.0 eth1: Intel(R) PRO/1000 Network Connection
Nov  9 16:30:30 debian8 kernel: [    3.859617] e1000e 0000:00:19.0 eth1: MAC: 11, PHY: 12, PBA No: FFFFFF-0FF
Nov  9 16:30:30 debian8 kernel: [    3.860020] ehci-pci 0000:00:1a.0: EHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.860235] ehci-pci 0000:00:1a.0: new USB bus registered, assigned bus number 1
Nov  9 16:30:30 debian8 kernel: [    3.860516] ehci-pci 0000:00:1a.0: debug port 2
Nov  9 16:30:30 debian8 kernel: [    3.864561] ehci-pci 0000:00:1a.0: cache line size of 64 is not supported
Nov  9 16:30:30 debian8 kernel: [    3.864573] ehci-pci 0000:00:1a.0: irq 16, io mem 0xdf13b000
Nov  9 16:30:30 debian8 kernel: [    3.878733] ehci-pci 0000:00:1a.0: USB 2.0 started, EHCI 1.00
Nov  9 16:30:30 debian8 kernel: [    3.878969] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
Nov  9 16:30:30 debian8 kernel: [    3.879213] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Nov  9 16:30:30 debian8 kernel: [    3.879472] usb usb1: Product: EHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.879649] usb usb1: Manufacturer: Linux 4.8.6+ ehci_hcd
Nov  9 16:30:30 debian8 kernel: [    3.879844] usb usb1: SerialNumber: 0000:00:1a.0
Nov  9 16:30:30 debian8 kernel: [    3.880120] hub 1-0:1.0: USB hub found
Nov  9 16:30:30 debian8 kernel: [    3.880264] hub 1-0:1.0: 2 ports detected
Nov  9 16:30:30 debian8 kernel: [    3.880550] ehci-pci 0000:00:1d.0: EHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.880749] ehci-pci 0000:00:1d.0: new USB bus registered, assigned bus number 2
Nov  9 16:30:30 debian8 kernel: [    3.881019] ehci-pci 0000:00:1d.0: debug port 2
Nov  9 16:30:30 debian8 kernel: [    3.885068] ehci-pci 0000:00:1d.0: cache line size of 64 is not supported
Nov  9 16:30:30 debian8 kernel: [    3.885073] ehci-pci 0000:00:1d.0: irq 23, io mem 0xdf13a000
Nov  9 16:30:30 debian8 kernel: [    3.898738] ehci-pci 0000:00:1d.0: USB 2.0 started, EHCI 1.00
Nov  9 16:30:30 debian8 kernel: [    3.898989] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002
Nov  9 16:30:30 debian8 kernel: [    3.899241] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Nov  9 16:30:30 debian8 kernel: [    3.899508] usb usb2: Product: EHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.899691] usb usb2: Manufacturer: Linux 4.8.6+ ehci_hcd
Nov  9 16:30:30 debian8 kernel: [    3.899896] usb usb2: SerialNumber: 0000:00:1d.0
Nov  9 16:30:30 debian8 kernel: [    3.900229] hub 2-0:1.0: USB hub found
Nov  9 16:30:30 debian8 kernel: [    3.900380] hub 2-0:1.0: 2 ports detected
Nov  9 16:30:30 debian8 kernel: [    3.900807] xhci_hcd 0000:00:14.0: xHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.901018] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 3
Nov  9 16:30:30 debian8 kernel: [    3.902370] xhci_hcd 0000:00:14.0: hcc params 0x200077c1 hci version 0x100 quirks 0x00009810
Nov  9 16:30:30 debian8 kernel: [    3.902685] xhci_hcd 0000:00:14.0: cache line size of 64 is not supported
Nov  9 16:30:30 debian8 kernel: [    3.902787] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
Nov  9 16:30:30 debian8 kernel: [    3.903029] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Nov  9 16:30:30 debian8 kernel: [    3.903287] usb usb3: Product: xHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.903462] usb usb3: Manufacturer: Linux 4.8.6+ xhci-hcd
Nov  9 16:30:30 debian8 kernel: [    3.903656] usb usb3: SerialNumber: 0000:00:14.0
Nov  9 16:30:30 debian8 kernel: [    3.903899] hub 3-0:1.0: USB hub found
Nov  9 16:30:30 debian8 kernel: [    3.904053] hub 3-0:1.0: 14 ports detected
Nov  9 16:30:30 debian8 kernel: [    3.906330] xhci_hcd 0000:00:14.0: xHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.906519] xhci_hcd 0000:00:14.0: new USB bus registered, assigned bus number 4
Nov  9 16:30:30 debian8 kernel: [    3.906852] usb usb4: New USB device found, idVendor=1d6b, idProduct=0003
Nov  9 16:30:30 debian8 kernel: [    3.907094] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Nov  9 16:30:30 debian8 kernel: [    3.907350] usb usb4: Product: xHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.907526] usb usb4: Manufacturer: Linux 4.8.6+ xhci-hcd
Nov  9 16:30:30 debian8 kernel: [    3.907719] usb usb4: SerialNumber: 0000:00:14.0
Nov  9 16:30:30 debian8 kernel: [    3.907978] hub 4-0:1.0: USB hub found
Nov  9 16:30:30 debian8 kernel: [    3.908127] hub 4-0:1.0: 6 ports detected
Nov  9 16:30:30 debian8 kernel: [    3.909002] xhci_hcd 0000:01:00.0: xHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.909191] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 5
Nov  9 16:30:30 debian8 kernel: [    3.968189] xhci_hcd 0000:01:00.0: hcc params 0x0200eec1 hci version 0x110 quirks 0x00000010
Nov  9 16:30:30 debian8 kernel: [    3.968570] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002
Nov  9 16:30:30 debian8 kernel: [    3.968812] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Nov  9 16:30:30 debian8 kernel: [    3.969068] usb usb5: Product: xHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.969244] usb usb5: Manufacturer: Linux 4.8.6+ xhci-hcd
Nov  9 16:30:30 debian8 kernel: [    3.969437] usb usb5: SerialNumber: 0000:01:00.0
Nov  9 16:30:30 debian8 kernel: [    3.969671] hub 5-0:1.0: USB hub found
Nov  9 16:30:30 debian8 kernel: [    3.969814] hub 5-0:1.0: 2 ports detected
Nov  9 16:30:30 debian8 kernel: [    3.970001] xhci_hcd 0000:01:00.0: xHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.970189] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 6
Nov  9 16:30:30 debian8 kernel: [    3.970471] usb usb6: We don't know the algorithms for LPM for this host, disabling LPM.
Nov  9 16:30:30 debian8 kernel: [    3.970821] usb usb6: New USB device found, idVendor=1d6b, idProduct=0003
Nov  9 16:30:30 debian8 kernel: [    3.971064] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Nov  9 16:30:30 debian8 kernel: [    3.971322] usb usb6: Product: xHCI Host Controller
Nov  9 16:30:30 debian8 kernel: [    3.971498] usb usb6: Manufacturer: Linux 4.8.6+ xhci-hcd
Nov  9 16:30:30 debian8 kernel: [    3.971692] usb usb6: SerialNumber: 0000:01:00.0
Nov  9 16:30:30 debian8 kernel: [    3.971935] hub 6-0:1.0: USB hub found
Nov  9 16:30:30 debian8 kernel: [    3.972078] hub 6-0:1.0: 2 ports detected
Nov  9 16:30:30 debian8 kernel: [    3.997177] ata1: SATA link down (SStatus 0 SControl 300)
Nov  9 16:30:30 debian8 kernel: [    4.001038] ata3: SATA link down (SStatus 0 SControl 300)
Nov  9 16:30:30 debian8 kernel: [    4.206721] usb 1-1: new high-speed USB device number 2 using ehci-pci
Nov  9 16:30:30 debian8 kernel: [    4.226721] usb 2-1: new high-speed USB device number 2 using ehci-pci
Nov  9 16:30:30 debian8 kernel: [    4.230713] usb 3-3: new high-speed USB device number 2 using xhci_hcd
Nov  9 16:30:30 debian8 kernel: [    4.258706] tsc: Refined TSC clocksource calibration: 3398.030 MHz
Nov  9 16:30:30 debian8 kernel: [    4.258936] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x30fb0a05005, max_idle_ns: 440795375639 ns
Nov  9 16:30:30 debian8 kernel: [    4.355123] usb 1-1: New USB device found, idVendor=8087, idProduct=8008
Nov  9 16:30:30 debian8 kernel: [    4.355401] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
Nov  9 16:30:30 debian8 kernel: [    4.356187] hub 1-1:1.0: USB hub found
Nov  9 16:30:30 debian8 kernel: [    4.356437] hub 1-1:1.0: 6 ports detected
Nov  9 16:30:30 debian8 kernel: [    4.375105] usb 2-1: New USB device found, idVendor=8087, idProduct=8000
Nov  9 16:30:30 debian8 kernel: [    4.375370] usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
Nov  9 16:30:30 debian8 kernel: [    4.375986] hub 2-1:1.0: USB hub found
Nov  9 16:30:30 debian8 kernel: [    4.376176] hub 2-1:1.0: 8 ports detected
Nov  9 16:30:30 debian8 kernel: [    4.396467] usb 3-3: New USB device found, idVendor=174c, idProduct=2074
Nov  9 16:30:30 debian8 kernel: [    4.396733] usb 3-3: New USB device strings: Mfr=2, Product=3, SerialNumber=0
Nov  9 16:30:30 debian8 kernel: [    4.397012] usb 3-3: Product: ASM107x
Nov  9 16:30:30 debian8 kernel: [    4.397158] usb 3-3: Manufacturer: ASUS Tek.
Nov  9 16:30:30 debian8 kernel: [    4.397808] hub 3-3:1.0: USB hub found
Nov  9 16:30:30 debian8 kernel: [    4.398131] hub 3-3:1.0: 4 ports detected
Nov  9 16:30:30 debian8 kernel: [    4.470758] ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Nov  9 16:30:30 debian8 kernel: [    4.471614] ata2.00: ATA-8: ST1000DM003-9YN162, CC4B, max UDMA/133
Nov  9 16:30:30 debian8 kernel: [    4.471841] ata2.00: 1953525168 sectors, multi 16: LBA48 NCQ (depth 31/32), AA
Nov  9 16:30:30 debian8 kernel: [    4.472663] ata2.00: configured for UDMA/133
Nov  9 16:30:30 debian8 kernel: [    4.473007] scsi 1:0:0:0: Direct-Access     ATA      ST1000DM003-9YN1 CC4B PQ: 0 ANSI: 5
Nov  9 16:30:30 debian8 kernel: [    4.511825] sd 1:0:0:0: [sda] 1953525168 512-byte logical blocks: (1.00 TB/932 GiB)
Nov  9 16:30:30 debian8 kernel: [    4.512112] sd 1:0:0:0: [sda] 4096-byte physical blocks
Nov  9 16:30:30 debian8 kernel: [    4.512349] sd 1:0:0:0: [sda] Write Protect is off
Nov  9 16:30:30 debian8 kernel: [    4.512524] sd 1:0:0:0: [sda] Mode Sense: 00 3a 00 00
Nov  9 16:30:30 debian8 kernel: [    4.512539] sd 1:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Nov  9 16:30:30 debian8 kernel: [    4.526503] usb 4-3: new SuperSpeed USB device number 2 using xhci_hcd
Nov  9 16:30:30 debian8 kernel: [    4.551263]  sda: sda1 sda2 sda3 sda4
Nov  9 16:30:30 debian8 kernel: [    4.551900] sd 1:0:0:0: [sda] Attached SCSI disk
Nov  9 16:30:30 debian8 kernel: [    4.654059] usb 4-3: New USB device found, idVendor=174c, idProduct=3074
Nov  9 16:30:30 debian8 kernel: [    4.654302] usb 4-3: New USB device strings: Mfr=2, Product=3, SerialNumber=0
Nov  9 16:30:30 debian8 kernel: [    4.654558] usb 4-3: Product: ASM107x
Nov  9 16:30:30 debian8 kernel: [    4.654698] usb 4-3: Manufacturer: ASUS Tek.
Nov  9 16:30:30 debian8 kernel: [    4.710725] usb 3-4: new full-speed USB device number 3 using xhci_hcd
Nov  9 16:30:30 debian8 kernel: [    4.714554] hub 4-3:1.0: USB hub found
Nov  9 16:30:30 debian8 kernel: [    4.726679] hub 4-3:1.0: 4 ports detected
Nov  9 16:30:30 debian8 kernel: [    4.825816] ata4: SATA link down (SStatus 0 SControl 300)
Nov  9 16:30:30 debian8 kernel: [    4.827420] sd 1:0:0:0: Attached scsi generic sg0 type 0
Nov  9 16:30:30 debian8 kernel: [    4.857032] usb 3-4: New USB device found, idVendor=0b05, idProduct=17cf
Nov  9 16:30:30 debian8 kernel: [    4.857283] usb 3-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
Nov  9 16:30:30 debian8 kernel: [    4.857549] usb 3-4: Product: BCM20702A0
Nov  9 16:30:30 debian8 kernel: [    4.857698] usb 3-4: Manufacturer: Broadcom Corp
Nov  9 16:30:30 debian8 kernel: [    4.857872] usb 3-4: SerialNumber: 6C71D9497B00
Nov  9 16:30:30 debian8 kernel: [    4.978662] usb 3-5: new low-speed USB device number 4 using xhci_hcd
Nov  9 16:30:30 debian8 kernel: [    4.992660] random: fast init done
Nov  9 16:30:30 debian8 kernel: [    5.124224] usb 3-5: New USB device found, idVendor=04b3, idProduct=3025
Nov  9 16:30:30 debian8 kernel: [    5.124467] usb 3-5: New USB device strings: Mfr=1, Product=2, SerialNumber=0
Nov  9 16:30:30 debian8 kernel: [    5.124723] usb 3-5: Product: USB NetVista Full Width Keyboard
Nov  9 16:30:30 debian8 kernel: [    5.124934] usb 3-5: Manufacturer: CHICONY
Nov  9 16:30:30 debian8 kernel: [    5.126552] hidraw: raw HID events driver (C) Jiri Kosina
Nov  9 16:30:30 debian8 kernel: [    5.129361] usbcore: registered new interface driver usbhid
Nov  9 16:30:30 debian8 kernel: [    5.129569] usbhid: USB HID core driver
Nov  9 16:30:30 debian8 kernel: [    5.130092] input: CHICONY USB NetVista Full Width Keyboard as /devices/pci0000:00/0000:00:14.0/usb3/3-5/3-5:1.0/0003:04B3:3025.0001/input/input3
Nov  9 16:30:30 debian8 kernel: [    5.186794] hid-generic 0003:04B3:3025.0001: input,hidraw0: USB HID v1.10 Keyboard [CHICONY USB NetVista Full Width Keyboard] on usb-0000:00:14.0-5/input0
Nov  9 16:30:30 debian8 kernel: [    5.195652] PM: Starting manual resume from disk
Nov  9 16:30:30 debian8 kernel: [    5.195825] PM: Hibernation image partition 8:3 present
Nov  9 16:30:30 debian8 kernel: [    5.195825] PM: Looking for hibernation image.
Nov  9 16:30:30 debian8 kernel: [    5.196068] PM: Image not found (code -22)
Nov  9 16:30:30 debian8 kernel: [    5.196069] PM: Hibernation image not present or could not be loaded.
Nov  9 16:30:30 debian8 kernel: [    5.242671] usb 3-6: new low-speed USB device number 5 using xhci_hcd
Nov  9 16:30:30 debian8 kernel: [    5.282874] clocksource: Switched to clocksource tsc
Nov  9 16:30:30 debian8 kernel: [    5.386118] usb 3-6: New USB device found, idVendor=17ef, idProduct=6019
Nov  9 16:30:30 debian8 kernel: [    5.386387] usb 3-6: New USB device strings: Mfr=0, Product=2, SerialNumber=0
Nov  9 16:30:30 debian8 kernel: [    5.386701] usb 3-6: Product: Lenovo Optical USB Mouse
Nov  9 16:30:30 debian8 kernel: [    5.390458] input: Lenovo Optical USB Mouse as /devices/pci0000:00/0000:00:14.0/usb3/3-6/3-6:1.0/0003:17EF:6019.0002/input/input4
Nov  9 16:30:30 debian8 kernel: [    5.391071] hid-generic 0003:17EF:6019.0002: input,hidraw1: USB HID v1.11 Mouse [Lenovo Optical USB Mouse] on usb-0000:00:14.0-6/input0
Nov  9 16:30:30 debian8 kernel: [    5.538686] EXT4-fs (sda2): mounted filesystem with ordered data mode. Opts: (null)
Nov  9 16:30:30 debian8 kernel: [    7.085908] lp: driver loaded but no devices found
Nov  9 16:30:30 debian8 kernel: [    7.175486] ppdev: user-space parallel port driver
Nov  9 16:30:30 debian8 kernel: [    7.253918] fuse init (API version 7.25)
Nov  9 16:30:30 debian8 kernel: [    7.799453] random: crng init done
Nov  9 16:30:30 debian8 kernel: [    8.205477] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input5
Nov  9 16:30:30 debian8 kernel: [    8.205785] ACPI: Power Button [PWRB]
Nov  9 16:30:30 debian8 kernel: [    8.205966] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input6
Nov  9 16:30:30 debian8 kernel: [    8.206235] ACPI: Power Button [PWRF]
Nov  9 16:30:30 debian8 kernel: [    8.276995] wmi: Mapper loaded
Nov  9 16:30:30 debian8 kernel: [    8.299164] EFI Variables Facility v0.08 2004-May-17
Nov  9 16:30:30 debian8 kernel: [    8.303729] pstore: using zlib compression
Nov  9 16:30:30 debian8 kernel: [    8.303882] pstore: Registered efi as persistent store backend
Nov  9 16:30:30 debian8 kernel: [    8.539759] EXT4-fs (sda2): re-mounted. Opts: errors=remount-ro
Nov  9 16:30:30 debian8 kernel: [    8.552815] e1000e 0000:00:19.0 eth4: renamed from eth1
Nov  9 16:30:30 debian8 kernel: [    8.619147] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt
Nov  9 16:30:30 debian8 kernel: [    8.798750] input: PC Speaker as /devices/platform/pcspkr/input/input7
Nov  9 16:30:30 debian8 kernel: [    8.994456] [drm] Initialized drm 1.1.0 20060810
Nov  9 16:30:30 debian8 kernel: [    9.114052] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
Nov  9 16:30:30 debian8 kernel: [    9.205423] asus_wmi: ASUS WMI generic driver loaded
Nov  9 16:30:30 debian8 kernel: [    9.303106] iTCO_vendor_support: vendor-support=0
Nov  9 16:30:30 debian8 kernel: [    9.426361] asus_wmi: Initialization: 0x0
Nov  9 16:30:30 debian8 kernel: [    9.426532] asus_wmi: BIOS WMI version: 0.9
Nov  9 16:30:30 debian8 kernel: [    9.426707] asus_wmi: SFUN value: 0x0
Nov  9 16:30:30 debian8 kernel: [    9.427063] input: Eee PC WMI hotkeys as /devices/platform/eeepc-wmi/input/input8
Nov  9 16:30:30 debian8 kernel: [    9.427396] asus_wmi: Number of fans: 1
Nov  9 16:30:30 debian8 kernel: [    9.597077] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11
Nov  9 16:30:30 debian8 kernel: [    9.597311] iTCO_wdt: Found a Lynx Point TCO device (Version=2, TCOBASE=0x1860)
Nov  9 16:30:30 debian8 kernel: [    9.597635] iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
Nov  9 16:30:30 debian8 kernel: [    9.648510] r8169 0000:08:00.0 eth5: renamed from eth0
Nov  9 16:30:30 debian8 kernel: [    9.651479] AVX2 version of gcm_enc/dec engaged.
Nov  9 16:30:30 debian8 kernel: [    9.651648] AES CTR mode by8 optimization enabled
Nov  9 16:30:30 debian8 kernel: [    9.886501] [drm] Memory usable by graphics device = 2048M
Nov  9 16:30:30 debian8 kernel: [    9.886705] checking generic (50000000 300000) vs hw (50000000 10000000)
Nov  9 16:30:30 debian8 kernel: [    9.886706] fb: switching to inteldrmfb from simple
Nov  9 16:30:30 debian8 kernel: [    9.886907] Console: switching to colour dummy device 80x25
Nov  9 16:30:30 debian8 kernel: [    9.887168] [drm] Replacing VGA console driver
Nov  9 16:30:30 debian8 kernel: [    9.893574] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
Nov  9 16:30:30 debian8 kernel: [    9.893815] [drm] Driver supports precise vblank timestamp query.
Nov  9 16:30:30 debian8 kernel: [    9.896222] vgaarb: device changed decodes: PCI:0000:00:02.0,olddecodes=io+mem,decodes=io+mem:owns=io+mem
Nov  9 16:30:30 debian8 kernel: [    9.915390] ACPI: Video Device [GFX0] (multi-head: yes  rom: no  post: no)
Nov  9 16:30:30 debian8 kernel: [    9.915719] input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/LNXVIDEO:00/input/input9
Nov  9 16:30:30 debian8 kernel: [    9.916158] [drm] Initialized i915 1.6.0 20160711 for 0000:00:02.0 on minor 0
Nov  9 16:30:30 debian8 kernel: [    9.931670] fbcon: inteldrmfb (fb0) is primary device
Nov  9 16:30:30 debian8 kernel: [    9.971882] Console: switching to colour frame buffer device 240x75
Nov  9 16:30:30 debian8 kernel: [    9.985025] bcma: bus0: Found chip with id 0x4352, rev 0x03 and package 0x00
Nov  9 16:30:30 debian8 kernel: [    9.985055] bcma: bus0: Core 0 found: ChipCommon (manuf 0x4BF, id 0x800, rev 0x2B, class 0x0)
Nov  9 16:30:30 debian8 kernel: [    9.985071] bcma: bus0: Core 1 found: IEEE 802.11 (manuf 0x4BF, id 0x812, rev 0x2A, class 0x0)
Nov  9 16:30:30 debian8 kernel: [    9.985102] bcma: bus0: Core 2 found: ARM CR4 (manuf 0x4BF, id 0x83E, rev 0x02, class 0x0)
Nov  9 16:30:30 debian8 kernel: [    9.985135] bcma: bus0: Core 3 found: PCIe Gen2 (manuf 0x4BF, id 0x83C, rev 0x01, class 0x0)
Nov  9 16:30:30 debian8 kernel: [    9.985151] bcma: bus0: Core 4 found: USB 2.0 Device (manuf 0x4BF, id 0x81A, rev 0x11, class 0x0)
Nov  9 16:30:30 debian8 kernel: [   10.018208] bcma: Unsupported SPROM revision: 11
Nov  9 16:30:30 debian8 kernel: [   10.018209] bcma: bus0: Invalid SPROM read from the PCIe card, trying to use fallback SPROM
Nov  9 16:30:30 debian8 kernel: [   10.018209] bcma: bus0: Using fallback SPROM failed (err -2)
Nov  9 16:30:30 debian8 kernel: [   10.018210] bcma: bus0: No SPROM available
Nov  9 16:30:30 debian8 kernel: [   10.018728] Error: Driver 'pcspkr' is already registered, aborting...
Nov  9 16:30:30 debian8 kernel: [   10.020262] bcma: bus0: Bus registered
Nov  9 16:30:30 debian8 kernel: [   10.020773] i915 0000:00:02.0: fb0: inteldrmfb frame buffer device
Nov  9 16:30:30 debian8 kernel: [   10.344412] Bluetooth: Core ver 2.21
Nov  9 16:30:30 debian8 kernel: [   10.344555] NET: Registered protocol family 31
Nov  9 16:30:30 debian8 kernel: [   10.344716] Bluetooth: HCI device and connection manager initialized
Nov  9 16:30:30 debian8 kernel: [   10.344945] Bluetooth: HCI socket layer initialized
Nov  9 16:30:30 debian8 kernel: [   10.345123] Bluetooth: L2CAP socket layer initialized
Nov  9 16:30:30 debian8 kernel: [   10.345309] Bluetooth: SCO socket layer initialized
Nov  9 16:30:30 debian8 kernel: [   10.463781] Adding 4163580k swap on /dev/sda3.  Priority:-1 extents:1 across:4163580k FS
Nov  9 16:30:30 debian8 kernel: [   10.622841] usbcore: registered new interface driver btusb
Nov  9 16:30:30 debian8 kernel: [   10.627247] Bluetooth: hci0: BCM: chip id 63
Nov  9 16:30:30 debian8 kernel: [   10.643256] Bluetooth: hci0: debian8
Nov  9 16:30:30 debian8 kernel: [   10.644254] Bluetooth: hci0: BCM20702A1 (001.002.014) build 0000
Nov  9 16:30:30 debian8 kernel: [   10.653809] bluetooth hci0: Direct firmware load for brcm/BCM20702A1-0b05-17cf.hcd failed with error -2
Nov  9 16:30:30 debian8 kernel: [   10.654144] Bluetooth: hci0: BCM: Patch brcm/BCM20702A1-0b05-17cf.hcd not found
Nov  9 16:30:30 debian8 kernel: [   11.086033] snd_hda_intel 0000:00:03.0: bound 0000:00:02.0 (ops i915_audio_component_bind_ops [i915])
Nov  9 16:30:30 debian8 kernel: [   11.113762] EXT4-fs (sda4): mounted filesystem with ordered data mode. Opts: (null)
Nov  9 16:30:30 debian8 kernel: [   11.309420] input: HDA Intel HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:03.0/sound/card0/input10
Nov  9 16:30:30 debian8 kernel: [   11.309824] input: HDA Intel HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:03.0/sound/card0/input11
Nov  9 16:30:30 debian8 kernel: [   11.310198] input: HDA Intel HDMI HDMI/DP,pcm=8 as /devices/pci0000:00/0000:00:03.0/sound/card0/input12
Nov  9 16:30:30 debian8 kernel: [   11.418077] snd_hda_codec_realtek hdaudioC1D0: ALC1150: SKU not ready 0x00000000
Nov  9 16:30:30 debian8 kernel: [   11.418907] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC1150: line_outs=4 (0x14/0x15/0x16/0x17/0x0) type:line
Nov  9 16:30:30 debian8 kernel: [   11.419285] snd_hda_codec_realtek hdaudioC1D0:    speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
Nov  9 16:30:30 debian8 kernel: [   11.419566] snd_hda_codec_realtek hdaudioC1D0:    hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
Nov  9 16:30:30 debian8 kernel: [   11.419844] snd_hda_codec_realtek hdaudioC1D0:    mono: mono_out=0x0
Nov  9 16:30:30 debian8 kernel: [   11.420071] snd_hda_codec_realtek hdaudioC1D0:    dig-out=0x11/0x1e
Nov  9 16:30:30 debian8 kernel: [   11.420295] snd_hda_codec_realtek hdaudioC1D0:    inputs:
Nov  9 16:30:30 debian8 kernel: [   11.420490] snd_hda_codec_realtek hdaudioC1D0:      Front Mic=0x19
Nov  9 16:30:30 debian8 kernel: [   11.420711] snd_hda_codec_realtek hdaudioC1D0:      Rear Mic=0x18
Nov  9 16:30:30 debian8 kernel: [   11.420938] snd_hda_codec_realtek hdaudioC1D0:      Line=0x1a
Nov  9 16:30:30 debian8 kernel: [   11.432840] input: HDA Digital PCBeep as /devices/pci0000:00/0000:00:1b.0/sound/card1/input13
Nov  9 16:30:30 debian8 kernel: [   11.433740] input: HDA Intel PCH Front Mic as /devices/pci0000:00/0000:00:1b.0/sound/card1/input14
Nov  9 16:30:30 debian8 kernel: [   11.434096] input: HDA Intel PCH Rear Mic as /devices/pci0000:00/0000:00:1b.0/sound/card1/input15
Nov  9 16:30:30 debian8 kernel: [   11.434485] input: HDA Intel PCH Line as /devices/pci0000:00/0000:00:1b.0/sound/card1/input16
Nov  9 16:30:30 debian8 kernel: [   11.434870] input: HDA Intel PCH Line Out Front as /devices/pci0000:00/0000:00:1b.0/sound/card1/input17
Nov  9 16:30:30 debian8 kernel: [   11.435283] input: HDA Intel PCH Line Out Surround as /devices/pci0000:00/0000:00:1b.0/sound/card1/input18
Nov  9 16:30:30 debian8 kernel: [   11.435701] input: HDA Intel PCH Line Out CLFE as /devices/pci0000:00/0000:00:1b.0/sound/card1/input19
Nov  9 16:30:30 debian8 kernel: [   11.436112] input: HDA Intel PCH Line Out Side as /devices/pci0000:00/0000:00:1b.0/sound/card1/input20
Nov  9 16:30:30 debian8 kernel: [   11.436507] input: HDA Intel PCH Front Headphone as /devices/pci0000:00/0000:00:1b.0/sound/card1/input21
Nov  9 16:30:30 debian8 kernel: [   12.270467] intel_rapl: Found RAPL domain package
Nov  9 16:30:30 debian8 kernel: [   12.270649] intel_rapl: Found RAPL domain core
Nov  9 16:30:30 debian8 kernel: [   12.270813] intel_rapl: Found RAPL domain uncore
Nov  9 16:30:30 debian8 kernel: [   12.270982] intel_rapl: Found RAPL domain dram
Nov  9 16:30:30 debian8 kernel: [   12.385687] b43-phy0: Broadcom 4352 WLAN found (core revision 42)
Nov  9 16:30:30 debian8 kernel: [   12.386248] b43-phy0 ERROR: FOUND UNSUPPORTED PHY (Analog 12, Type 11 (AC), Revision 1)
Nov  9 16:30:30 debian8 kernel: [   12.386542] b43: probe of bcma0:1 failed with error -95
Nov  9 16:30:30 debian8 kernel: [   12.386742] Broadcom 43xx driver loaded [ Features: PNLS ]
Nov  9 16:30:30 debian8 kernel: [   12.886267] RPC: Registered named UNIX socket transport module.
Nov  9 16:30:30 debian8 kernel: [   12.886483] RPC: Registered udp transport module.
Nov  9 16:30:30 debian8 kernel: [   12.886654] RPC: Registered tcp transport module.
Nov  9 16:30:30 debian8 kernel: [   12.886825] RPC: Registered tcp NFSv4.1 backchannel transport module.
Nov  9 16:30:30 debian8 kernel: [   12.930288] FS-Cache: Loaded
Nov  9 16:30:30 debian8 kernel: [   13.037241] FS-Cache: Netfs 'nfs' registered for caching
Nov  9 16:30:30 debian8 kernel: [   13.161992] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
Nov  9 16:30:30 debian8 kernel: [   15.001250] Bluetooth: BNEP (Ethernet Emulation) ver 1.3
Nov  9 16:30:30 debian8 kernel: [   15.001446] Bluetooth: BNEP filters: protocol multicast
Nov  9 16:30:30 debian8 kernel: [   15.001637] Bluetooth: BNEP socket layer initialized
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> NetworkManager (version 0.9.10.0) is starting...
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> Read config: /etc/NetworkManager/NetworkManager.conf
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> WEXT support is enabled
Nov  9 16:30:30 debian8 polkitd[868]: started daemon version 0.105 using authority implementation `local' version `0.105'
Nov  9 16:30:30 debian8 dbus[586]: [system] Successfully activated service 'org.freedesktop.PolicyKit1'
Nov  9 16:30:30 debian8 systemd[1]: Started Authenticate and Authorize Users to Run Privileged Tasks.
Nov  9 16:30:30 debian8 accounts-daemon[560]: started daemon version 0.6.37
Nov  9 16:30:30 debian8 systemd[1]: Started Accounts Service.
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> init!
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> update_system_hostname
Nov  9 16:30:30 debian8 NetworkManager[570]: <info>       interface-parser: parsing file /etc/network/interfaces
Nov  9 16:30:30 debian8 NetworkManager[570]: <info>       interface-parser: source line includes interfaces file(s) /etc/network/interfaces.d/*
Nov  9 16:30:30 debian8 NetworkManager[570]: <warn> interfaces file /etc/network/interfaces.d/* doesn't exist
Nov  9 16:30:30 debian8 NetworkManager[570]: <warn> Can't parse iface line 'iface eth0 dhcp'
Nov  9 16:30:30 debian8 NetworkManager[570]: <info>       interface-parser: finished parsing file /etc/network/interfaces
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> management mode: unmanaged
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> devices added (path: /sys/devices/pci0000:00/0000:00:19.0/net/eth4, iface: eth4)
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> device added (path: /sys/devices/pci0000:00/0000:00:19.0/net/eth4, iface: eth4): no ifupdown configuration found.
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> devices added (path: /sys/devices/pci0000:00/0000:00:1c.3/0000:04:00.0/0000:05:05.0/0000:08:00.0/net/eth5, iface: eth5)
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> device added (path: /sys/devices/pci0000:00/0000:00:1c.3/0000:04:00.0/0000:05:05.0/0000:08:00.0/net/eth5, iface: eth5): no ifupdown configuration found.
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> devices added (path: /sys/devices/virtual/net/lo, iface: lo)
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> device added (path: /sys/devices/virtual/net/lo, iface: lo): no ifupdown configuration found.
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> end _init.
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> Loaded plugin ifupdown: (C) 2008 Canonical Ltd.  To report bugs please use the NetworkManager mailing list.
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> Loaded plugin keyfile: (c) 2007 - 2013 Red Hat, Inc.  To report bugs please use the NetworkManager mailing list.
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (17085600) ... get_connections.
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (17085600) ... get_connections (managed=false): return empty list.
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> new connection /etc/NetworkManager/system-connections/Wired connection 1
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> get unmanaged devices count: 0
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> monitoring kernel firmware directory '/lib/firmware'.
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> monitoring ifupdown state file '/run/network/ifstate'.
Nov  9 16:30:30 debian8 systemd[1]: Started Light Display Manager.
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> WiFi hardware radio set enabled
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> WWAN hardware radio set enabled
Nov  9 16:30:30 debian8 rsyslogd-2007: action 'action 17' suspended, next retry is Wed Nov  9 16:31:00 2016 [try http://www.rsyslog.com/e/2007 ]
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> Loaded device plugin: /usr/lib/x86_64-linux-gnu/NetworkManager/libnm-device-plugin-bluetooth.so
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> Loaded device plugin: /usr/lib/x86_64-linux-gnu/NetworkManager/libnm-device-plugin-wifi.so
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> Loaded device plugin: /usr/lib/x86_64-linux-gnu/NetworkManager/libnm-device-plugin-adsl.so
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> Loaded device plugin: /usr/lib/x86_64-linux-gnu/NetworkManager/libnm-device-plugin-wwan.so
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> WiFi enabled by radio killswitch; enabled by state file
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> WWAN enabled by radio killswitch; enabled by state file
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> WiMAX enabled by radio killswitch; enabled by state file
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> Networking is enabled by state file
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (lo): link connected
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (lo): carrier is ON
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (lo): new Generic device (driver: 'unknown' ifindex: 1)
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (lo): exported as /org/freedesktop/NetworkManager/Devices/0
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (eth5): carrier is OFF
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (eth5): new Ethernet device (driver: 'r8169' ifindex: 2)
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (eth5): exported as /org/freedesktop/NetworkManager/Devices/1
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (eth5): device state change: unmanaged -> unavailable (reason 'managed') [10 20 2]
Nov  9 16:30:30 debian8 systemd[1]: Started Network Manager.
Nov  9 16:30:30 debian8 kernel: [   15.936554] r8169 0000:08:00.0: Direct firmware load for rtl_nic/rtl8168g-2.fw failed with error -2
Nov  9 16:30:30 debian8 kernel: [   15.936878] r8169 0000:08:00.0 eth5: unable to load firmware patch rtl_nic/rtl8168g-2.fw (-2)
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (eth5): preparing device
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (eth4): carrier is OFF
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (eth4): new Ethernet device (driver: 'e1000e' ifindex: 3)
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (eth4): exported as /org/freedesktop/NetworkManager/Devices/2
Nov  9 16:30:30 debian8 NetworkManager[570]: <info> (eth4): device state change: unmanaged -> unavailable (reason 'managed') [10 20 2]
Nov  9 16:30:30 debian8 kernel: [   15.952676] r8169 0000:08:00.0 eth5: link down
Nov  9 16:30:30 debian8 kernel: [   15.952679] r8169 0000:08:00.0 eth5: link down
Nov  9 16:30:30 debian8 systemd[1]: Started Modem Manager.
Nov  9 16:30:31 debian8 NetworkManager[570]: <info> (eth4): preparing device
Nov  9 16:30:31 debian8 NetworkManager[570]: <info> use BlueZ version 5
Nov  9 16:30:31 debian8 NetworkManager[570]: <info> ModemManager available in the bus
Nov  9 16:30:31 debian8 exim4[574]: Starting MTA: exim4.
Nov  9 16:30:31 debian8 systemd[1]: Started LSB: exim Mail Transport Agent.
Nov  9 16:30:32 debian8 systemd[1]: Starting user-117.slice.
Nov  9 16:30:32 debian8 systemd[1]: Created slice user-117.slice.
Nov  9 16:30:32 debian8 ModemManager[563]: <warn>  Couldn't find support for device at '/sys/devices/pci0000:00/0000:00:19.0': not supported by any plugin
Nov  9 16:30:32 debian8 ModemManager[563]: <warn>  Couldn't find support for device at '/sys/devices/pci0000:00/0000:00:1c.3/0000:04:00.0/0000:05:05.0/0000:08:00.0': not supported by any plugin
Nov  9 16:30:32 debian8 systemd[1]: Starting User Manager for UID 117...
Nov  9 16:30:32 debian8 systemd[1]: Starting Session c1 of user lightdm.
Nov  9 16:30:32 debian8 systemd[1]: Started Session c1 of user lightdm.
Nov  9 16:30:33 debian8 systemd[1]: Started LSB: start Samba daemons for the AD DC.
Nov  9 16:30:33 debian8 nmbd[573]: Starting NetBIOS name server: nmbd.
Nov  9 16:30:33 debian8 systemd[1]: Started LSB: start Samba NetBIOS nameserver (nmbd).
Nov  9 16:30:33 debian8 systemd[916]: Starting Paths.
Nov  9 16:30:33 debian8 systemd[916]: Reached target Paths.
Nov  9 16:30:33 debian8 systemd[916]: Starting Timers.
Nov  9 16:30:33 debian8 systemd[916]: Reached target Timers.
Nov  9 16:30:33 debian8 systemd[916]: Starting Sockets.
Nov  9 16:30:33 debian8 systemd[916]: Reached target Sockets.
Nov  9 16:30:33 debian8 systemd[916]: Starting Basic System.
Nov  9 16:30:33 debian8 systemd[916]: Reached target Basic System.
Nov  9 16:30:33 debian8 systemd[1]: Starting LSB: start Samba SMB/CIFS daemon (smbd)...
Nov  9 16:30:33 debian8 systemd[916]: Starting Default.
Nov  9 16:30:33 debian8 systemd[916]: Reached target Default.
Nov  9 16:30:33 debian8 systemd[916]: Startup finished in 90ms.
Nov  9 16:30:33 debian8 systemd[1]: Started User Manager for UID 117.
Nov  9 16:30:33 debian8 lightdm[873]: ** (lightdm:873): WARNING **: Error using VT_ACTIVATE 7 on /dev/console: Input/output error
Nov  9 16:30:33 debian8 lightdm[873]: ** (lightdm:873): WARNING **: Error using VT_WAITACTIVE 7 on /dev/console: Input/output error
Nov  9 16:30:33 debian8 smbd[925]: Starting SMB/CIFS daemon: smbd.
Nov  9 16:30:33 debian8 systemd[1]: Started LSB: start Samba SMB/CIFS daemon (smbd).
Nov  9 16:30:33 debian8 systemd[1]: Starting Multi-User System.
Nov  9 16:30:33 debian8 systemd[1]: Reached target Multi-User System.
Nov  9 16:30:33 debian8 systemd[1]: Starting Graphical Interface.
Nov  9 16:30:33 debian8 systemd[1]: Reached target Graphical Interface.
Nov  9 16:30:33 debian8 systemd[1]: Starting Update UTMP about System Runlevel Changes...
Nov  9 16:30:33 debian8 systemd[1]: Started Update UTMP about System Runlevel Changes.
Nov  9 16:30:33 debian8 systemd[1]: Startup finished in 5.805s (kernel) + 12.824s (userspace) = 18.629s.
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> (eth5): link connected
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> (eth5): device state change: unavailable -> disconnected (reason 'carrier-changed') [20 30 40]
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Auto-activating connection 'Wired connection 1'.
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) starting connection 'Wired connection 1'
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 1 of 5 (Device Prepare) scheduled...
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 1 of 5 (Device Prepare) started...
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> (eth5): device state change: disconnected -> prepare (reason 'none') [30 40 0]
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> NetworkManager state is now CONNECTING
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 2 of 5 (Device Configure) scheduled...
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 1 of 5 (Device Prepare) complete.
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 2 of 5 (Device Configure) starting...
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> (eth5): device state change: prepare -> config (reason 'none') [40 50 0]
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 2 of 5 (Device Configure) successful.
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 3 of 5 (IP Configure Start) scheduled.
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 2 of 5 (Device Configure) complete.
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 3 of 5 (IP Configure Start) started...
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> (eth5): device state change: config -> ip-config (reason 'none') [50 70 0]
Nov  9 16:30:33 debian8 kernel: [   18.931173] r8169 0000:08:00.0 eth5: link up
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) Beginning DHCPv4 transaction (timeout in 45 seconds)
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> dhclient started with pid 944
Nov  9 16:30:33 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 3 of 5 (IP Configure Start) complete.
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> (eth5): DHCPv4 state changed nbi -> preinit
Nov  9 16:30:34 debian8 dhclient: DHCPREQUEST on eth5 to 255.255.255.255 port 67
Nov  9 16:30:34 debian8 dhclient: DHCPACK from 10.21.59.1
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> (eth5): DHCPv4 state changed preinit -> reboot
Nov  9 16:30:34 debian8 NetworkManager[570]: <info>   address 10.21.59.167
Nov  9 16:30:34 debian8 NetworkManager[570]: <info>   plen 24 (255.255.255.0)
Nov  9 16:30:34 debian8 NetworkManager[570]: <info>   gateway 10.21.59.1
Nov  9 16:30:34 debian8 NetworkManager[570]: <info>   server identifier 10.21.62.9
Nov  9 16:30:34 debian8 NetworkManager[570]: <info>   lease time 14400
Nov  9 16:30:34 debian8 NetworkManager[570]: <info>   hostname 'debian8'
Nov  9 16:30:34 debian8 NetworkManager[570]: <info>   nameserver '10.21.62.9'
Nov  9 16:30:34 debian8 NetworkManager[570]: <info>   nameserver '10.21.62.10'
Nov  9 16:30:34 debian8 NetworkManager[570]: <info>   domain name 'lacie.com'
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 5 of 5 (IPv4 Configure Commit) scheduled...
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 5 of 5 (IPv4 Commit) started...
Nov  9 16:30:34 debian8 avahi-daemon[582]: Joining mDNS multicast group on interface eth5.IPv4 with address 10.21.59.167.
Nov  9 16:30:34 debian8 avahi-daemon[582]: New relevant interface eth5.IPv4 for mDNS.
Nov  9 16:30:34 debian8 avahi-daemon[582]: Registering new address record for 10.21.59.167 on eth5.IPv4.
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> (eth5): device state change: ip-config -> ip-check (reason 'none') [70 80 0]
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> Activation (eth5) Stage 5 of 5 (IPv4 Commit) complete.
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> (eth5): device state change: ip-check -> secondaries (reason 'none') [80 90 0]
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> (eth5): device state change: secondaries -> activated (reason 'none') [90 100 0]
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> NetworkManager state is now CONNECTED_LOCAL
Nov  9 16:30:34 debian8 dhclient: bound to 10.21.59.167 -- renewal in 7049 seconds.
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> NetworkManager state is now CONNECTED_GLOBAL
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> Policy set 'Wired connection 1' (eth5) as default for IPv4 routing and DNS.
Nov  9 16:30:34 debian8 NetworkManager[570]: <info> Activation (eth5) successful, device activated.
Nov  9 16:30:34 debian8 dbus[586]: [system] Activating via systemd: service name='org.freedesktop.nm_dispatcher' unit='dbus-org.freedesktop.nm-dispatcher.service'
Nov  9 16:30:34 debian8 systemd[1]: Starting Network Manager Script Dispatcher Service...
Nov  9 16:30:34 debian8 dbus[586]: [system] Successfully activated service 'org.freedesktop.nm_dispatcher'
Nov  9 16:30:34 debian8 systemd[1]: Started Network Manager Script Dispatcher Service.
Nov  9 16:30:34 debian8 nm-dispatcher: Dispatching action 'up' for eth5
Nov  9 16:30:34 debian8 org.a11y.Bus[953]: Activating service name='org.a11y.atspi.Registry'
Nov  9 16:30:34 debian8 org.a11y.Bus[953]: Successfully activated service 'org.a11y.atspi.Registry'
Nov  9 16:30:34 debian8 org.a11y.atspi.Registry[973]: SpiRegistry daemon is running with well-known name - org.a11y.atspi.Registry
Nov  9 16:30:34 debian8 systemd[1]: Reloading OpenBSD Secure Shell server.
Nov  9 16:30:35 debian8 systemd[1]: Reloaded OpenBSD Secure Shell server.
Nov  9 16:30:35 debian8 avahi-daemon[582]: Joining mDNS multicast group on interface eth5.IPv6 with address fe80::2e0:4cff:fe68:3d.
Nov  9 16:30:35 debian8 avahi-daemon[582]: New relevant interface eth5.IPv6 for mDNS.
Nov  9 16:30:35 debian8 avahi-daemon[582]: Registering new address record for fe80::2e0:4cff:fe68:3d on eth5.*.
Nov  9 16:30:38 debian8 NetworkManager[570]: <info> startup complete
Nov  9 16:31:43 debian8 systemd[1]: Starting user-1001.slice.
Nov  9 16:31:43 debian8 systemd[1]: Created slice user-1001.slice.
Nov  9 16:31:43 debian8 systemd[1]: Starting User Manager for UID 1001...
Nov  9 16:31:43 debian8 systemd[1]: Starting Session 1 of user sguinot.
Nov  9 16:31:43 debian8 systemd[1]: Started Session 1 of user sguinot.
Nov  9 16:31:43 debian8 systemd[1035]: Starting Paths.
Nov  9 16:31:43 debian8 systemd[1035]: Reached target Paths.
Nov  9 16:31:43 debian8 systemd[1035]: Starting Timers.
Nov  9 16:31:43 debian8 systemd[1035]: Reached target Timers.
Nov  9 16:31:43 debian8 systemd[1035]: Starting Sockets.
Nov  9 16:31:43 debian8 systemd[1035]: Reached target Sockets.
Nov  9 16:31:43 debian8 systemd[1035]: Starting Basic System.
Nov  9 16:31:43 debian8 systemd[1035]: Reached target Basic System.
Nov  9 16:31:43 debian8 systemd[1035]: Starting Default.
Nov  9 16:31:43 debian8 systemd[1035]: Reached target Default.
Nov  9 16:31:43 debian8 systemd[1035]: Startup finished in 17ms.
Nov  9 16:31:43 debian8 systemd[1]: Started User Manager for UID 1001.
Nov  9 16:32:37 debian8 kernel: [  142.351927] pci 0000:0d:00.0: [8086:156d] type 01 class 0x060400
Nov  9 16:32:37 debian8 kernel: [  142.352079] pci 0000:0d:00.0: supports D1 D2
Nov  9 16:32:37 debian8 kernel: [  142.352082] pci 0000:0d:00.0: PME# supported from D0 D1 D2 D3hot D3cold
Nov  9 16:32:37 debian8 kernel: [  142.352154] pci 0000:0d:00.0: System wakeup disabled by ACPI
Nov  9 16:32:37 debian8 kernel: [  142.363957] pci 0000:0e:00.0: [8086:156d] type 01 class 0x060400
Nov  9 16:32:37 debian8 kernel: [  142.364106] pci 0000:0e:00.0: supports D1 D2
Nov  9 16:32:37 debian8 kernel: [  142.364109] pci 0000:0e:00.0: PME# supported from D0 D1 D2 D3hot D3cold
Nov  9 16:32:37 debian8 kernel: [  142.364231] pci 0000:0e:03.0: [8086:156d] type 01 class 0x060400
Nov  9 16:32:37 debian8 kernel: [  142.364372] pci 0000:0e:03.0: supports D1 D2
Nov  9 16:32:37 debian8 kernel: [  142.364374] pci 0000:0e:03.0: PME# supported from D0 D1 D2 D3hot D3cold
Nov  9 16:32:37 debian8 kernel: [  142.364482] pci 0000:0e:04.0: [8086:156d] type 01 class 0x060400
Nov  9 16:32:37 debian8 kernel: [  142.364621] pci 0000:0e:04.0: supports D1 D2
Nov  9 16:32:37 debian8 kernel: [  142.364623] pci 0000:0e:04.0: PME# supported from D0 D1 D2 D3hot D3cold
Nov  9 16:32:37 debian8 kernel: [  142.364725] pci 0000:0e:05.0: [8086:156d] type 01 class 0x060400
Nov  9 16:32:37 debian8 kernel: [  142.364865] pci 0000:0e:05.0: supports D1 D2
Nov  9 16:32:37 debian8 kernel: [  142.364866] pci 0000:0e:05.0: PME# supported from D0 D1 D2 D3hot D3cold
Nov  9 16:32:37 debian8 kernel: [  142.364968] pci 0000:0e:06.0: [8086:156d] type 01 class 0x060400
Nov  9 16:32:37 debian8 kernel: [  142.365107] pci 0000:0e:06.0: supports D1 D2
Nov  9 16:32:37 debian8 kernel: [  142.365109] pci 0000:0e:06.0: PME# supported from D0 D1 D2 D3hot D3cold
Nov  9 16:32:37 debian8 kernel: [  142.365233] pci 0000:0d:00.0: PCI bridge to [bus 0e-77]
Nov  9 16:32:37 debian8 kernel: [  142.365450] pci 0000:0d:00.0:   bridge window [mem 0xb0000000-0xde0fffff]
Nov  9 16:32:37 debian8 kernel: [  142.365458] pci 0000:0d:00.0:   bridge window [mem 0x60000000-0xa9ffffff 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.365545] pci 0000:0f:00.0: [8086:156c] type 00 class 0x088000
Nov  9 16:32:37 debian8 kernel: [  142.365571] pci 0000:0f:00.0: reg 0x10: [mem 0xde000000-0xde03ffff]
Nov  9 16:32:37 debian8 kernel: [  142.365586] pci 0000:0f:00.0: reg 0x14: [mem 0xde040000-0xde040fff]
Nov  9 16:32:37 debian8 kernel: [  142.365767] pci 0000:0f:00.0: supports D1 D2
Nov  9 16:32:37 debian8 kernel: [  142.365769] pci 0000:0f:00.0: PME# supported from D0 D1 D2 D3hot D3cold
Nov  9 16:32:37 debian8 kernel: [  142.365894] pci 0000:0e:00.0: PCI bridge to [bus 0f]
Nov  9 16:32:37 debian8 kernel: [  142.366106] pci 0000:0e:00.0:   bridge window [mem 0xde000000-0xde0fffff]
Nov  9 16:32:37 debian8 kernel: [  142.366167] pci 0000:0e:03.0: PCI bridge to [bus 10-3a]
Nov  9 16:32:37 debian8 kernel: [  142.366371] pci 0000:0e:03.0:   bridge window [mem 0xb0000000-0xbfffffff]
Nov  9 16:32:37 debian8 kernel: [  142.366377] pci 0000:0e:03.0:   bridge window [mem 0x60000000-0x7fffffff 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.366430] pci 0000:0e:04.0: PCI bridge to [bus 3b-43]
Nov  9 16:32:37 debian8 kernel: [  142.366634] pci 0000:0e:04.0:   bridge window [mem 0xc0000000-0xc7ffffff]
Nov  9 16:32:37 debian8 kernel: [  142.366641] pci 0000:0e:04.0:   bridge window [mem 0x80000000-0x87ffffff 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.366694] pci 0000:0e:05.0: PCI bridge to [bus 44-6e]
Nov  9 16:32:37 debian8 kernel: [  142.366898] pci 0000:0e:05.0:   bridge window [mem 0xc8000000-0xd5ffffff]
Nov  9 16:32:37 debian8 kernel: [  142.366905] pci 0000:0e:05.0:   bridge window [mem 0x90000000-0xa9ffffff 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.366957] pci 0000:0e:06.0: PCI bridge to [bus 6f-77]
Nov  9 16:32:37 debian8 kernel: [  142.367161] pci 0000:0e:06.0:   bridge window [mem 0xd6000000-0xddffffff]
Nov  9 16:32:37 debian8 kernel: [  142.367168] pci 0000:0e:06.0:   bridge window [mem 0x88000000-0x8fffffff 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.367206] pci_bus 0000:0e: Allocating resources
Nov  9 16:32:37 debian8 kernel: [  142.367238] pci 0000:0e:00.0: bridge window [io  0x1000-0x0fff] to [bus 0f] add_size 1000
Nov  9 16:32:37 debian8 kernel: [  142.367240] pci 0000:0e:00.0: bridge window [mem 0x00100000-0x000fffff 64bit pref] to [bus 0f] add_size 200000 add_align 100000
Nov  9 16:32:37 debian8 kernel: [  142.367251] pci 0000:0e:03.0: bridge window [io  0x1000-0x0fff] to [bus 10-3a] add_size 1000
Nov  9 16:32:37 debian8 kernel: [  142.367261] pci 0000:0e:04.0: bridge window [io  0x1000-0x0fff] to [bus 3b-43] add_size 1000
Nov  9 16:32:37 debian8 kernel: [  142.367271] pci 0000:0e:05.0: bridge window [io  0x1000-0x0fff] to [bus 44-6e] add_size 1000
Nov  9 16:32:37 debian8 kernel: [  142.367281] pci 0000:0e:06.0: bridge window [io  0x1000-0x0fff] to [bus 6f-77] add_size 1000
Nov  9 16:32:37 debian8 kernel: [  142.367291] pci 0000:0e:00.0: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.367292] pci 0000:0e:03.0: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.367294] pci 0000:0e:04.0: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.367295] pci 0000:0e:05.0: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.367296] pci 0000:0e:06.0: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.367298] pci 0000:0d:00.0: bridge window [io  0x1000-0x0fff] to [bus 0e-77] add_size 5000
Nov  9 16:32:37 debian8 kernel: [  142.367300] pci 0000:0d:00.0: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 5000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.367301] pci 0000:0d:00.0: res[13]=[io  0x1000-0x5fff] res_to_dev_res add_size 5000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.367304] pci 0000:0d:00.0: BAR 13: no space for [io  size 0x5000]
Nov  9 16:32:37 debian8 kernel: [  142.367540] pci 0000:0d:00.0: BAR 13: failed to assign [io  size 0x5000]
Nov  9 16:32:37 debian8 kernel: [  142.367821] pci 0000:0d:00.0: BAR 13: no space for [io  size 0x5000]
Nov  9 16:32:37 debian8 kernel: [  142.368054] pci 0000:0d:00.0: BAR 13: failed to assign [io  size 0x5000]
Nov  9 16:32:37 debian8 kernel: [  142.368307] pci 0000:0e:00.0: res[15]=[mem 0x00100000-0x000fffff 64bit pref] res_to_dev_res add_size 200000 min_align 100000
Nov  9 16:32:37 debian8 kernel: [  142.368308] pci 0000:0e:00.0: res[15]=[mem 0x00100000-0x002fffff 64bit pref] res_to_dev_res add_size 200000 min_align 100000
Nov  9 16:32:37 debian8 kernel: [  142.368309] pci 0000:0e:00.0: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.368310] pci 0000:0e:00.0: res[13]=[io  0x1000-0x1fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.368311] pci 0000:0e:03.0: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.368311] pci 0000:0e:03.0: res[13]=[io  0x1000-0x1fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.368312] pci 0000:0e:04.0: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.368313] pci 0000:0e:04.0: res[13]=[io  0x1000-0x1fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.368314] pci 0000:0e:05.0: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.368315] pci 0000:0e:05.0: res[13]=[io  0x1000-0x1fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.368317] pci 0000:0e:06.0: res[13]=[io  0x1000-0x0fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.368318] pci 0000:0e:06.0: res[13]=[io  0x1000-0x1fff] res_to_dev_res add_size 1000 min_align 1000
Nov  9 16:32:37 debian8 kernel: [  142.368321] pci 0000:0e:00.0: BAR 15: no space for [mem size 0x00200000 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.368606] pci 0000:0e:00.0: BAR 15: failed to assign [mem size 0x00200000 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.368897] pci 0000:0e:00.0: BAR 13: no space for [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.369128] pci 0000:0e:00.0: BAR 13: failed to assign [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.369373] pci 0000:0e:03.0: BAR 13: no space for [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.369604] pci 0000:0e:03.0: BAR 13: failed to assign [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.369848] pci 0000:0e:04.0: BAR 13: no space for [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.370080] pci 0000:0e:04.0: BAR 13: failed to assign [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.370324] pci 0000:0e:05.0: BAR 13: no space for [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.370555] pci 0000:0e:05.0: BAR 13: failed to assign [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.370800] pci 0000:0e:06.0: BAR 13: no space for [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.371031] pci 0000:0e:06.0: BAR 13: failed to assign [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.371275] pci 0000:0e:06.0: BAR 13: no space for [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.371507] pci 0000:0e:06.0: BAR 13: failed to assign [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.371751] pci 0000:0e:05.0: BAR 13: no space for [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.371987] pci 0000:0e:05.0: BAR 13: failed to assign [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.372231] pci 0000:0e:04.0: BAR 13: no space for [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.372462] pci 0000:0e:04.0: BAR 13: failed to assign [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.372706] pci 0000:0e:03.0: BAR 13: no space for [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.372938] pci 0000:0e:03.0: BAR 13: failed to assign [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.373182] pci 0000:0e:00.0: BAR 15: no space for [mem size 0x00200000 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.373460] pci 0000:0e:00.0: BAR 15: failed to assign [mem size 0x00200000 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.373750] pci 0000:0e:00.0: BAR 13: no space for [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.373981] pci 0000:0e:00.0: BAR 13: failed to assign [io  size 0x1000]
Nov  9 16:32:37 debian8 kernel: [  142.374225] pci 0000:0e:00.0: PCI bridge to [bus 0f]
Nov  9 16:32:37 debian8 kernel: [  142.374412] pci 0000:0e:00.0:   bridge window [mem 0xde000000-0xde0fffff]
Nov  9 16:32:37 debian8 kernel: [  142.374667] pci 0000:0e:03.0: PCI bridge to [bus 10-3a]
Nov  9 16:32:37 debian8 kernel: [  142.374863] pci 0000:0e:03.0:   bridge window [mem 0xb0000000-0xbfffffff]
Nov  9 16:32:37 debian8 kernel: [  142.375112] pci 0000:0e:03.0:   bridge window [mem 0x60000000-0x7fffffff 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.375398] pci 0000:0e:04.0: PCI bridge to [bus 3b-43]
Nov  9 16:32:37 debian8 kernel: [  142.375593] pci 0000:0e:04.0:   bridge window [mem 0xc0000000-0xc7ffffff]
Nov  9 16:32:37 debian8 kernel: [  142.375858] pci 0000:0e:04.0:   bridge window [mem 0x80000000-0x87ffffff 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.376140] pci 0000:0e:05.0: PCI bridge to [bus 44-6e]
Nov  9 16:32:37 debian8 kernel: [  142.376333] pci 0000:0e:05.0:   bridge window [mem 0xc8000000-0xd5ffffff]
Nov  9 16:32:37 debian8 kernel: [  142.376578] pci 0000:0e:05.0:   bridge window [mem 0x90000000-0xa9ffffff 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.376860] pci 0000:0e:06.0: PCI bridge to [bus 6f-77]
Nov  9 16:32:37 debian8 kernel: [  142.377052] pci 0000:0e:06.0:   bridge window [mem 0xd6000000-0xddffffff]
Nov  9 16:32:37 debian8 kernel: [  142.377298] pci 0000:0e:06.0:   bridge window [mem 0x88000000-0x8fffffff 64bit pref]
Nov  9 16:32:37 debian8 kernel: [  142.377580] pci 0000:0d:00.0: PCI bridge to [bus 0e-77]
Nov  9 16:32:37 debian8 kernel: [  142.377772] pci 0000:0d:00.0:   bridge window [mem 0xb0000000-0xde0fffff]
Nov  9 16:32:37 debian8 kernel: [  142.378018] pci 0000:0d:00.0:   bridge window [mem 0x60000000-0xa9ffffff 64bit pref]
Nov  9 16:32:37 debian8 systemd[1]: Expecting device sys-module-thunderbolt.device...
Nov  9 16:32:37 debian8 systemd[1]: Starting Thunderbolt Manager...
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: Thunderbolt daemon version: 16.1.54.1-source      Thread: 140713088374592, func main line 72 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/thunderbolt.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: Thunderbolt module version: 16.1.55.1      Thread: 140713088374592, func main line 73 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/thunderbolt.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: Starting initialization      Thread: 140713088374592, func main line 74 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/thunderbolt.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: subscribing driver      Thread: 140713088374592, func driver_subscribe line 145 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::send_message_sync entry, sending netlink message: command only      Thread: 140713088374592, func send_message_sync line 456 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: netlink message was allocated      Thread: 140713088374592, func alloc_message line 134 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::send_message_sync(struct nl_msg* msg) entry      Thread: 140713088374592, func send_message_sync line 475 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Send Message:
Nov  9 16:32:37 debian8 thunderboltd[1081]: --------------------------   BEGIN NETLINK MESSAGE ---------------------------
Nov  9 16:32:37 debian8 thunderboltd[1081]: [NETLINK HEADER] 16 octets
Nov  9 16:32:37 debian8 thunderboltd[1081]: .nlmsg_len = 24
Nov  9 16:32:37 debian8 thunderboltd[1081]: .type = 24 <0x18>
Nov  9 16:32:37 debian8 thunderboltd[1081]: .flags = 0 <>
Nov  9 16:32:37 debian8 thunderboltd[1081]: .seq = 0
Nov  9 16:32:37 debian8 thunderboltd[1081]: .port = 0
Nov  9 16:32:37 debian8 thunderboltd[1081]: [PAYLOAD] 8 octets
Nov  9 16:32:37 debian8 thunderboltd[1081]: 01 01 00 00 00 00 00 00                         ........
Nov  9 16:32:37 debian8 thunderboltd[1081]: ---------------------------  END NETLINK MESSAGE   ---------------------------
Nov  9 16:32:37 debian8 systemd[1]: Started Thunderbolt Manager.
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: Netlink ACK received       Thread: 140713062659840, func cb_ack line 260 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::send_message_sync(struct nl_msg* msg) exit      Thread: 140713088374592, func send_message_sync line 495 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: netlink message was free      Thread: 140713088374592, func operator() line 117 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::send_message_sync exit, sending netlink message: command only      Thread: 140713088374592, func send_message_sync line 465 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::register_events_callback entry      Thread: 140713088374592, func register_events_callback line 237 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::register_events_callback exit      Thread: 140713088374592, func register_events_callback line 239 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::cb_valid entry, message arrived      Thread: 140713062659840, func cb_valid line 270 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: --------------------------   BEGIN NETLINK MESSAGE ---------------------------
Nov  9 16:32:37 debian8 thunderboltd[1081]: [NETLINK HEADER] 16 octets
Nov  9 16:32:37 debian8 thunderboltd[1081]: .nlmsg_len = 48
Nov  9 16:32:37 debian8 thunderboltd[1081]: .type = 24 <0x18>
Nov  9 16:32:37 debian8 thunderboltd[1081]: .flags = 0 <>
Nov  9 16:32:37 debian8 thunderboltd[1081]: .seq = 0
Nov  9 16:32:37 debian8 thunderboltd[1081]: .port = 1081
Nov  9 16:32:37 debian8 thunderboltd[1081]: [GENERIC NETLINK HEADER] 4 octets
Nov  9 16:32:37 debian8 thunderboltd[1081]: .cmd = 5
Nov  9 16:32:37 debian8 thunderboltd[1081]: .version = 1
Nov  9 16:32:37 debian8 thunderboltd[1081]: .unused = 0
Nov  9 16:32:37 debian8 thunderboltd[1081]: [PAYLOAD] 28 octets
Nov  9 16:32:37 debian8 thunderboltd[1081]: 6c 15 00 0f 08 00 07 00 0c 00 00 00 10 00 09 00 l...............
Nov  9 16:32:37 debian8 thunderboltd[1081]: 01 00 0a 03 00 00 0a 50 3e fa ad aa             .......P>...
Nov  9 16:32:37 debian8 thunderboltd[1081]: ---------------------------  END NETLINK MESSAGE   ---------------------------
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: validate netlink message: PASS      Thread: 140713062659840, func cb_valid line 291 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: LinuxControllerCommandSender::OnEvent entry, PDF: 12      Thread: 140713062659840, func OnEvent line 63 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/LinuxControllerCommandSender.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: Message from FW: PDF_FW_TO_SW_RESPONSE, Controller: f00156c      Thread: 140713062659840, func OnEvent line 79 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/LinuxControllerCommandSender.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: FW response: DRIVER_READY_RESPONSE, Controller: 0f00156c      Thread: 140713062659840, func HandleFwToSwResponse line 42 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/MessageFromFwEvent.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: New controller detected: 0f00156c       Thread: 140713062659840, func OnDriverReadyResponse line 103 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Shared/src/ConnectionManagerBase.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: Create new controller class      Thread: 140713062659840, func OnDriverReadyResponse line 107 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Shared/src/ConnectionManagerBase.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GetGenerationFromControllerID devID=156c      Thread: 140713062659840, func GetGenerationFromControllerID line 136 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Shared/src/Utils.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GetNomOfPortsFromControllerID devID=156c      Thread: 140713062659840, func GetNomOfPortsFromControllerID line 113 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Shared/src/Utils.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: Controller device ID: 156C, TBT generation: 2, port count: 2, security level: 0      Thread: 140713062659840, func OnDriverReadyResponse line 121 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Shared/src/ConnectionManagerBase.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: Controller settings: Certified only: False, Override first depth: True, Allow dock at any depth: False      Thread: 140713062659840, func OnDriverReadyResponse line 124 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Shared/src/ConnectionManagerBase.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: Netlink message received      Thread: 140713062659840, func OnEvent line 116 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/LinuxControllerCommandSender.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: LinuxControllerCommandSender::OnEvent exit, PDF: 12      Thread: 140713062659840, func OnEvent line 121 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/LinuxControllerCommandSender.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::cb_valid exit, message arrived      Thread: 140713062659840, func cb_valid line 362 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::send_message_sync entry, sending netlink message: command only      Thread: 140713054267136, func send_message_sync line 456 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: netlink message was allocated      Thread: 140713054267136, func alloc_message line 134 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::send_message_sync(struct nl_msg* msg) entry      Thread: 140713054267136, func send_message_sync line 475 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Send Message:
Nov  9 16:32:37 debian8 thunderboltd[1081]: --------------------------   BEGIN NETLINK MESSAGE ---------------------------
Nov  9 16:32:37 debian8 thunderboltd[1081]: [NETLINK HEADER] 16 octets
Nov  9 16:32:37 debian8 thunderboltd[1081]: .nlmsg_len = 24
Nov  9 16:32:37 debian8 thunderboltd[1081]: .type = 24 <0x18>
Nov  9 16:32:37 debian8 thunderboltd[1081]: .flags = 0 <>
Nov  9 16:32:37 debian8 thunderboltd[1081]: .seq = 0
Nov  9 16:32:37 debian8 thunderboltd[1081]: .port = 0
Nov  9 16:32:37 debian8 thunderboltd[1081]: [PAYLOAD] 8 octets
Nov  9 16:32:37 debian8 thunderboltd[1081]: 03 01 00 00 6c 15 00 0f                         ....l...
Nov  9 16:32:37 debian8 thunderboltd[1081]: ---------------------------  END NETLINK MESSAGE   ---------------------------
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::cb_valid entry, message arrived      Thread: 140713062659840, func cb_valid line 270 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: --------------------------   BEGIN NETLINK MESSAGE ---------------------------
Nov  9 16:32:37 debian8 thunderboltd[1081]: [NETLINK HEADER] 16 octets
Nov  9 16:32:37 debian8 thunderboltd[1081]: .nlmsg_len = 64
Nov  9 16:32:37 debian8 thunderboltd[1081]: .type = 24 <0x18>
Nov  9 16:32:37 debian8 thunderboltd[1081]: .flags = 0 <>
Nov  9 16:32:37 debian8 thunderboltd[1081]: .seq = 1478705559
Nov  9 16:32:37 debian8 thunderboltd[1081]: .port = 1081
Nov  9 16:32:37 debian8 thunderboltd[1081]: [GENERIC NETLINK HEADER] 4 octets
Nov  9 16:32:37 debian8 thunderboltd[1081]: .cmd = 3
Nov  9 16:32:37 debian8 thunderboltd[1081]: .version = 1
Nov  9 16:32:37 debian8 thunderboltd[1081]: .unused = 0
Nov  9 16:32:37 debian8 thunderboltd[1081]: [PAYLOAD] 44 octets
Nov  9 16:32:37 debian8 thunderboltd[1081]: 6c 15 00 0f 0e 00 01 00 31 36 2e 31 2e 35 35 2e l.......16.1.55.
Nov  9 16:32:37 debian8 thunderboltd[1081]: 31 00 00 00 06 00 02 00 0a 00 00 00 05 00 03 00 1...............
Nov  9 16:32:37 debian8 thunderboltd[1081]: 02 00 00 00 05 00 04 00 05 00 00 00             ............
Nov  9 16:32:37 debian8 thunderboltd[1081]: ---------------------------  END NETLINK MESSAGE   ---------------------------
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: validate netlink message: PASS      Thread: 140713062659840, func cb_valid line 291 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: LinuxControllerCommandSender::OnEvent entry, PDF: -1      Thread: 140713062659840, func OnEvent line 63 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/LinuxControllerCommandSender.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: Netlink message received from Driver      Thread: 140713062659840, func OnEvent line 99 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/LinuxControllerCommandSender.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GetGenerationFromControllerID devID=156c      Thread: 140713062659840, func GetGenerationFromControllerID line 136 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Shared/src/Utils.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GetNomOfPortsFromControllerID devID=156c      Thread: 140713062659840, func GetNomOfPortsFromControllerID line 113 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Shared/src/Utils.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: Netlink message received      Thread: 140713062659840, func OnEvent line 116 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/LinuxControllerCommandSender.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: LinuxControllerCommandSender::OnEvent exit, PDF: -1      Thread: 140713062659840, func OnEvent line 121 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/LinuxControllerCommandSender.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::cb_valid exit, message arrived      Thread: 140713062659840, func cb_valid line 362 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: Netlink ACK received       Thread: 140713062659840, func cb_ack line 260 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::send_message_sync(struct nl_msg* msg) exit      Thread: 140713054267136, func send_message_sync line 495 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Debug: netlink message was free      Thread: 140713054267136, func operator() line 117 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: GenlWrapper::send_message_sync exit, sending netlink message: command only      Thread: 140713054267136, func send_message_sync line 465 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/src/GenlWrapper.cpp)
Nov  9 16:32:37 debian8 thunderboltd[1081]: Info: Daemon is running...      Thread: 140713088374592, func main line 89 (/home/sguinot/thunderbolt-software-daemon.git/daemon/ThunderboltService/Linux/thunderbolt.cpp)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

^ permalink raw reply	[flat|nested] 28+ messages in thread

* RE: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-09 14:36 ` [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Simon Guinot
@ 2016-11-09 15:42   ` Levy, Amir (Jer)
  2016-11-15 10:59     ` Simon Guinot
  0 siblings, 1 reply; 28+ messages in thread
From: Levy, Amir (Jer) @ 2016-11-09 15:42 UTC (permalink / raw)
  To: Simon Guinot
  Cc: gregkh, andreas.noever, bhelgaas, corbet, linux-kernel,
	linux-pci, netdev, linux-doc, mario_limonciello,
	thunderbolt-linux, Westerberg, Mika, Winkler, Tomas, Zhang,
	Xiong Y, Jamet, Michael

On Wed, Nov 9 2016, 04:36 PM, Simon Guinot wrote:
> On Wed, Nov 09, 2016 at 04:20:00PM +0200, Amir Levy wrote:
> > This driver enables Thunderbolt Networking on non-Apple platforms 
> > running Linux.
> >
> > Thunderbolt Networking provides peer-to-peer connections to transfer 
> > files between computers, perform PC migrations, and/or set up small 
> > workgroups with shared storage.
> >
> > This is a virtual connection that emulates an Ethernet adapter that 
> > enables Ethernet networking with the benefit of Thunderbolt 
> > superfast medium capability.
> >
> > Thunderbolt Networking enables two hosts and several devices that 
> > have a Thunderbolt controller to be connected together in a linear 
> > (Daisy
> > chain) series from a single port.
> >
> > Thunderbolt Networking for Linux is compatible with Thunderbolt 
> > Networking on systems running macOS or Windows and also supports 
> > Thunderbolt generation 2 and 3 controllers.
> >
> > Note that all pre-existing Thunderbolt generation 3 features, such 
> > as USB, Display and other Thunderbolt device connectivity will 
> > continue to function exactly as they did prior to enabling Thunderbolt Networking.
> >
> > Code and Software Specifications:
> > This kernel code creates a virtual ethernet device for computer to 
> > computer communication over a Thunderbolt cable.
> > The new driver is a separate driver to the existing Thunderbolt driver.
> > It is designed to work on systems running Linux that interface with 
> > Intel Connection Manager (ICM) firmware based Thunderbolt 
> > controllers that support Thunderbolt Networking.
> > The kernel code operates in coordination with the Thunderbolt user- 
> > space daemon to implement full Thunderbolt networking functionality.
> >
> > Hardware Specifications:
> > Thunderbolt Hardware specs have not yet been published but are used 
> > where necessary for register definitions.
> 
> Hi Amir,
> 
> I have an ASUS "All Series/Z87-DELUXE/QUAD" motherboard with a 
> Thunderbolt 2 "Falcon Ridge" chipset (device ID 156d).
> 
> Is the thunderbolt-icm driver supposed to work with this chipset ?
> 

Yes, the thunderbolt-icm supports Falcon Ridge, device ID 156c.
156d is the bridge - http://lxr.free-electrons.com/source/include/linux/pci_ids.h#L2619

> I have installed both a 4.8.6 Linux kernel (patched with your v9
> series) and the thunderbolt-software-daemon (27 october release) 
> inside a Debian system (Jessie).
> 
> If I connect the ASUS motherboard with a MacBook Pro (Thunderbolt 2, 
> device ID 156c), I can see that the thunderbolt-icm driver is loaded 
> and that the thunderbolt-software-daemon is well started. But the 
> Ethernet interface is not created.
> 
> I have attached to this email the syslog file. There is the logs from 
> both the kernel and the daemon inside. Note that the daemon logs are 
> everything but clear about what could be the issue. Maybe I missed 
> some kind of configuration ? But I failed to find any valuable 
> information about configuring the driver and/or the daemon in the various documentation files.
> 
> Please, can you provide some guidance ? I'd really like to test your 
> patch series.

First, thank you very much for willing to test it.
Thunderbolt Networking support was added during Falcon Ridge, in the latest FR images.
Do you know which Thunderbolt image version you have on your system?
Currently I submitted only Thunderbolt Networking feature in Linux, and we plan to add
more features like reading the image version and updating the image.
If you don't know the image version, the only thing I can suggest is to load windows, install thunderbolt SW
and check in the Thunderbolt application the image version.
To know if image update is needed, you can check - https://thunderbolttechnology.net/updates

> 
> Thanks in advance.
> 
> Simon

Thanks,
Amir

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v9 7/8] thunderbolt: Networking doc
  2016-11-09 14:20 ` [PATCH v9 7/8] thunderbolt: Networking doc Amir Levy
@ 2016-11-09 16:00   ` Greg KH
  2016-11-10 11:47     ` Levy, Amir (Jer)
  2016-11-10 14:24     ` Jonathan Corbet
  0 siblings, 2 replies; 28+ messages in thread
From: Greg KH @ 2016-11-09 16:00 UTC (permalink / raw)
  To: Amir Levy
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang

On Wed, Nov 09, 2016 at 04:20:07PM +0200, Amir Levy wrote:
> Adding Thunderbolt(TM) networking documentation.
> 
> Signed-off-by: Amir Levy <amir.jer.levy@intel.com>
> ---
>  Documentation/00-INDEX                   |   2 +
>  Documentation/thunderbolt/networking.txt | 132 +++++++++++++++++++++++++++++++

Note, new files should be in .rst format, and live in the new
subdirectory for them (somewhere in Documentation, don't know off the
top of my head...)

>  2 files changed, 134 insertions(+)
>  create mode 100644 Documentation/thunderbolt/networking.txt
> 
> diff --git a/Documentation/00-INDEX b/Documentation/00-INDEX
> index 3acc4f1..0239e68 100644
> --- a/Documentation/00-INDEX
> +++ b/Documentation/00-INDEX
> @@ -440,6 +440,8 @@ this_cpu_ops.txt
>  	- List rationale behind and the way to use this_cpu operations.
>  thermal/
>  	- directory with information on managing thermal issues (CPU/temp)
> +thunderbolt/
> +	- directory with info regarding Thunderbolt.
>  trace/
>  	- directory with info on tracing technologies within linux
>  unaligned-memory-access.txt

This change is probably not needed.

> diff --git a/Documentation/thunderbolt/networking.txt b/Documentation/thunderbolt/networking.txt
> new file mode 100644
> index 0000000..88d1c12
> --- /dev/null
> +++ b/Documentation/thunderbolt/networking.txt
> @@ -0,0 +1,132 @@
> +Intel Thunderbolt(TM) Networking driver
> +=======================================
> +
> +Copyright(c) 2013 - 2016 Intel Corporation.
> +
> +Contact Information:
> +Intel Thunderbolt mailing list <thunderbolt-software@lists.01.org>
> +Edited by Amir Levy <amir.jer.levy@intel.com>
> +
> +Overview
> +========
> +
> +* The Thunderbolt Networking driver enables peer to peer networking on non-Apple
> +  platforms running Linux.
> +
> +* The driver creates a virtual Ethernet device that enables computer to computer
> +  communication over the Thunderbolt cable.
> +
> +* Using Thunderbolt Networking you can perform high speed file transfers between
> +  computers, perform PC migrations and/or set up small workgroups with shared
> +  storage without compromising any other Thunderbolt functionality.
> +
> +* The driver is located in drivers/thunderbolt/icm.
> +
> +* This driver will function only on non-Apple platforms with firmware based
> +  Thunderbolt controllers that support Thunderbolt Networking.
> +
> +  +----------------+            +----------------+
> +  |Host 1          |            |Host 2          |
> +  |                |            |                |
> +  |   +-------+    |            |    +-------+   |
> +  |   |Network|    |            |    |Network|   |
> +  |   |Stack  |    |            |    |Stack  |   |
> +  |   +-------+    |            |    +-------+   |
> +  |       ^        |            |        ^       |
> +  |       |        |            |        |       |
> +  |       v        |            |        v       |
> +  | +-----------+  |            |  +-----------+ |
> +  | |Thunderbolt|  |            |  |Thunderbolt| |
> +  | |Networking |  |            |  |Networking | |
> +  | |Driver     |  |            |  |Driver     | |
> +  | +-----------+  |            |  +-----------+ |
> +  |       ^        |            |        ^       |
> +  |       |        |            |        |       |
> +  |       v        |            |        v       |
> +  | +-----------+  |            |  +-----------+ |
> +  | |Thunderbolt|  |            |  |Thunderbolt| |
> +  | |Controller |<-+------------+->|Controller | |
> +  | |with ICM   |  |            |  |with ICM   | |
> +  | |enabled    |  |            |  |enabled    | |
> +  | +-----------+  |            |  +-----------+ |
> +  +----------------+            +----------------+
> +
> +Files
> +=====
> +
> +The following files are located in the drivers/thunderbolt/icm directory:
> +
> +- icm_nhi.c/h:	These files allow communication with the firmware (Intel
> +  Connection Manager) based controller. They also create an interface for
> +  netlink communication with a user space daemon.
> +
> +- net.c/net.h:	These files implement the 'eth' interface for the
> +  Thunderbolt(TM) Networking.

You don't have to list the files, right?

> +
> +Interface to User Space
> +=======================
> +
> +The interface to the user space module is implemented through a Generic Netlink.

Huh?  Not a normal network device?

> +This is the communications protocol between the Thunderbolt driver and the user
> +space application.

What userspace application?

> +Note that this interface mediates user space communication with ICM.

What is ICM?

> +(Existing Linux tools can be used to configure the network interface.)

Why aren't they sufficient for everything here?

> +
> +The Thunderbolt Daemon utilizes this interface to communicate with the driver.

What "Thunderbolt Daemon"?  What is "this"?

> +To be accessed by the user space module, both kernel and user space modules
> +have to register with the same GENL_NAME.

What userspace modules?

How do you set GENL_NAME?  What is that?

> +For the purpose of the Thunderbolt Network driver, "thunderbolt" is used.

What do you mean?

> +The registration is done at driver initialization time for all instances
> +of the Thunderbolt controllers.

Who does this?

> The communication is carried through pre-defined
> +Thunderbolt messages. Each specific message has a callback function that is
> +called when the related message is received.

Is this the internal structure of the driver?  If so, why not just put
it all in the code itself and build the documentation from it?

> +
> +Message Definitions:
> +* NHI_CMD_UNSPEC: Not used.
> +* NHI_CMD_SUBSCRIBE: Subscription request from daemon to driver to open the
> +  communication channel.
> +* NHI_CMD_UNSUBSCRIBE: Request from daemon to driver to unsubscribe and
> +  to close communication channel.
> +* NHI_CMD_QUERY_INFORMATION: Request information from the driver such as
> +  driver version, FW version offset, number of ports in the controller
> +  and DMA port.
> +* NHI_CMD_MSG_TO_ICM: Message from user space module to FW.
> +* NHI_CMD_MSG_FROM_ICM: Response from FW to user space module.
> +* NHI_CMD_MAILBOX: Message that uses mailbox mechanism such as FW policy
> +  changes or disconnect path.
> +* NHI_CMD_APPROVE_TBT_NETWORKING: Request from user space module to FW to
> +  establish path.
> +* NHI_CMD_ICM_IN_SAFE_MODE: Indication that the FW has entered safe mode.
> +
> +Communication with Intel Connection Manager(ICM) Firmware
> +=========================================================
> +
> +There are several circular buffers in Thunderbolt each using Direct Memory
> +Access (DMA).

Again, internal documentation?

Isn't this just a "normal" network device?  Why document any of this in
a separate way?  What can a user do with this?  Who is the audience of
this file?

still confused,

greg k-h

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-09 14:20 [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Amir Levy
                   ` (8 preceding siblings ...)
  2016-11-09 14:36 ` [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Simon Guinot
@ 2016-11-09 16:02 ` Greg KH
  2016-11-10 11:39   ` Levy, Amir (Jer)
  9 siblings, 1 reply; 28+ messages in thread
From: Greg KH @ 2016-11-09 16:02 UTC (permalink / raw)
  To: Amir Levy
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang

On Wed, Nov 09, 2016 at 04:20:00PM +0200, Amir Levy wrote:
> This driver enables Thunderbolt Networking on non-Apple platforms
> running Linux.
> 
> Thunderbolt Networking provides peer-to-peer connections to transfer
> files between computers, perform PC migrations, and/or set up small
> workgroups with shared storage.
> 
> This is a virtual connection that emulates an Ethernet adapter that
> enables Ethernet networking with the benefit of Thunderbolt superfast
> medium capability.
> 
> Thunderbolt Networking enables two hosts and several devices that
> have a Thunderbolt controller to be connected together in a linear
> (Daisy chain) series from a single port.
> 
> Thunderbolt Networking for Linux is compatible with Thunderbolt
> Networking on systems running macOS or Windows and also supports
> Thunderbolt generation 2 and 3 controllers.
> 
> Note that all pre-existing Thunderbolt generation 3 features, such as
> USB, Display and other Thunderbolt device connectivity will continue
> to function exactly as they did prior to enabling Thunderbolt Networking.
> 
> Code and Software Specifications:
> This kernel code creates a virtual ethernet device for computer to
> computer communication over a Thunderbolt cable.
> The new driver is a separate driver to the existing Thunderbolt driver.
> It is designed to work on systems running Linux that
> interface with Intel Connection Manager (ICM) firmware based
> Thunderbolt controllers that support Thunderbolt Networking.
> The kernel code operates in coordination with the Thunderbolt user-
> space daemon to implement full Thunderbolt networking functionality.
> 
> Hardware Specifications:
> Thunderbolt Hardware specs have not yet been published but are used
> where necessary for register definitions. 
> 
> Acked-by: Andreas Noever <andreas.noever@gmail.com>
> Tested-by: Mario Limonciello <mario.limonciello@dell.com>

This whole series is acked and tested by these people?  If so, why did
you not include that in each patch?

And how about getting some internal-Intel kernel developers to review
and sign-off on this code?  Don't make the community do the review when
you have access to resources like this.  You have an internal mailing
list for this very purpose, use it!

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 28+ messages in thread

* RE: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-09 16:02 ` Greg KH
@ 2016-11-10 11:39   ` Levy, Amir (Jer)
  2016-11-10 11:44     ` Greg KH
  0 siblings, 1 reply; 28+ messages in thread
From: Levy, Amir (Jer) @ 2016-11-10 11:39 UTC (permalink / raw)
  To: Greg KH
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	Westerberg, Mika, Winkler, Tomas, Zhang, Xiong Y

On Wed, Nov 9 2016, 06:02 PM, Greg KH wrote:
> On Wed, Nov 09, 2016 at 04:20:00PM +0200, Amir Levy wrote:
> > This driver enables Thunderbolt Networking on non-Apple platforms
> > running Linux.
> >
> > Thunderbolt Networking provides peer-to-peer connections to transfer
> > files between computers, perform PC migrations, and/or set up small
> > workgroups with shared storage.
> >
> > This is a virtual connection that emulates an Ethernet adapter that
> > enables Ethernet networking with the benefit of Thunderbolt superfast
> > medium capability.
> >
> > Thunderbolt Networking enables two hosts and several devices that have
> > a Thunderbolt controller to be connected together in a linear (Daisy
> > chain) series from a single port.
> >
> > Thunderbolt Networking for Linux is compatible with Thunderbolt
> > Networking on systems running macOS or Windows and also supports
> > Thunderbolt generation 2 and 3 controllers.
> >
> > Note that all pre-existing Thunderbolt generation 3 features, such as
> > USB, Display and other Thunderbolt device connectivity will continue
> > to function exactly as they did prior to enabling Thunderbolt Networking.
> >
> > Code and Software Specifications:
> > This kernel code creates a virtual ethernet device for computer to
> > computer communication over a Thunderbolt cable.
> > The new driver is a separate driver to the existing Thunderbolt driver.
> > It is designed to work on systems running Linux that interface with
> > Intel Connection Manager (ICM) firmware based Thunderbolt controllers
> > that support Thunderbolt Networking.
> > The kernel code operates in coordination with the Thunderbolt user-
> > space daemon to implement full Thunderbolt networking functionality.
> >
> > Hardware Specifications:
> > Thunderbolt Hardware specs have not yet been published but are used
> > where necessary for register definitions.
> >
> > Acked-by: Andreas Noever <andreas.noever@gmail.com>
> > Tested-by: Mario Limonciello <mario.limonciello@dell.com>
> 
> This whole series is acked and tested by these people?  If so, why did you not
> include that in each patch?
> 

Will add in next patch set.

> And how about getting some internal-Intel kernel developers to review and
> sign-off on this code?  Don't make the community do the review when you
> have access to resources like this.  You have an internal mailing list for this
> very purpose, use it!
> 

The review with the internal-Intel kernel developers was done before submitting
the first patch set.
Version 9 that we have here isn't so different from Version 0.

> thanks,
> 
> greg k-h

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-10 11:39   ` Levy, Amir (Jer)
@ 2016-11-10 11:44     ` Greg KH
  2016-11-10 11:48       ` Levy, Amir (Jer)
  0 siblings, 1 reply; 28+ messages in thread
From: Greg KH @ 2016-11-10 11:44 UTC (permalink / raw)
  To: Levy, Amir (Jer)
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	Westerberg, Mika, Winkler, Tomas, Zhang, Xiong Y

On Thu, Nov 10, 2016 at 11:39:19AM +0000, Levy, Amir (Jer) wrote:
> > And how about getting some internal-Intel kernel developers to review and
> > sign-off on this code?  Don't make the community do the review when you
> > have access to resources like this.  You have an internal mailing list for this
> > very purpose, use it!
> > 
> 
> The review with the internal-Intel kernel developers was done before submitting
> the first patch set.

Then why is their signed-off-by:s not on the patchset showing that they
"bless" this series?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 28+ messages in thread

* RE: [PATCH v9 7/8] thunderbolt: Networking doc
  2016-11-09 16:00   ` Greg KH
@ 2016-11-10 11:47     ` Levy, Amir (Jer)
  2016-11-10 11:53       ` Greg KH
  2016-11-10 14:24     ` Jonathan Corbet
  1 sibling, 1 reply; 28+ messages in thread
From: Levy, Amir (Jer) @ 2016-11-10 11:47 UTC (permalink / raw)
  To: Greg KH
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	Westerberg, Mika, Winkler, Tomas, Zhang, Xiong Y

On Wed, Nov 9 2016, 06:00 PM, Greg KH wrote:
> On Wed, Nov 09, 2016 at 04:20:07PM +0200, Amir Levy wrote:
> > Adding Thunderbolt(TM) networking documentation.
> >
> > Signed-off-by: Amir Levy <amir.jer.levy@intel.com>
> > ---
> >  Documentation/00-INDEX                   |   2 +
> >  Documentation/thunderbolt/networking.txt | 132
> > +++++++++++++++++++++++++++++++
> 
> Note, new files should be in .rst format, and live in the new subdirectory for
> them (somewhere in Documentation, don't know off the top of my head...)
> 
> >  2 files changed, 134 insertions(+)
> >  create mode 100644 Documentation/thunderbolt/networking.txt
> >
> > diff --git a/Documentation/00-INDEX b/Documentation/00-INDEX index
> > 3acc4f1..0239e68 100644
> > --- a/Documentation/00-INDEX
> > +++ b/Documentation/00-INDEX
> > @@ -440,6 +440,8 @@ this_cpu_ops.txt
> >  	- List rationale behind and the way to use this_cpu operations.
> >  thermal/
> >  	- directory with information on managing thermal issues (CPU/temp)
> > +thunderbolt/
> > +	- directory with info regarding Thunderbolt.
> >  trace/
> >  	- directory with info on tracing technologies within linux
> > unaligned-memory-access.txt
> 
> This change is probably not needed.
> 
> > diff --git a/Documentation/thunderbolt/networking.txt
> > b/Documentation/thunderbolt/networking.txt
> > new file mode 100644
> > index 0000000..88d1c12
> > --- /dev/null
> > +++ b/Documentation/thunderbolt/networking.txt
> > @@ -0,0 +1,132 @@
> > +Intel Thunderbolt(TM) Networking driver
> > +=======================================
> > +
> > +Copyright(c) 2013 - 2016 Intel Corporation.
> > +
> > +Contact Information:
> > +Intel Thunderbolt mailing list <thunderbolt-software@lists.01.org>
> > +Edited by Amir Levy <amir.jer.levy@intel.com>
> > +
> > +Overview
> > +========
> > +
> > +* The Thunderbolt Networking driver enables peer to peer networking
> > +on non-Apple
> > +  platforms running Linux.
> > +
> > +* The driver creates a virtual Ethernet device that enables computer
> > +to computer
> > +  communication over the Thunderbolt cable.
> > +
> > +* Using Thunderbolt Networking you can perform high speed file
> > +transfers between
> > +  computers, perform PC migrations and/or set up small workgroups
> > +with shared
> > +  storage without compromising any other Thunderbolt functionality.
> > +
> > +* The driver is located in drivers/thunderbolt/icm.
> > +
> > +* This driver will function only on non-Apple platforms with firmware
> > +based
> > +  Thunderbolt controllers that support Thunderbolt Networking.
> > +
> > +  +----------------+            +----------------+
> > +  |Host 1          |            |Host 2          |
> > +  |                |            |                |
> > +  |   +-------+    |            |    +-------+   |
> > +  |   |Network|    |            |    |Network|   |
> > +  |   |Stack  |    |            |    |Stack  |   |
> > +  |   +-------+    |            |    +-------+   |
> > +  |       ^        |            |        ^       |
> > +  |       |        |            |        |       |
> > +  |       v        |            |        v       |
> > +  | +-----------+  |            |  +-----------+ |
> > +  | |Thunderbolt|  |            |  |Thunderbolt| |
> > +  | |Networking |  |            |  |Networking | |
> > +  | |Driver     |  |            |  |Driver     | |
> > +  | +-----------+  |            |  +-----------+ |
> > +  |       ^        |            |        ^       |
> > +  |       |        |            |        |       |
> > +  |       v        |            |        v       |
> > +  | +-----------+  |            |  +-----------+ |
> > +  | |Thunderbolt|  |            |  |Thunderbolt| |
> > +  | |Controller |<-+------------+->|Controller | |
> > +  | |with ICM   |  |            |  |with ICM   | |
> > +  | |enabled    |  |            |  |enabled    | |
> > +  | +-----------+  |            |  +-----------+ |
> > +  +----------------+            +----------------+
> > +
> > +Files
> > +=====
> > +
> > +The following files are located in the drivers/thunderbolt/icm directory:
> > +
> > +- icm_nhi.c/h:	These files allow communication with the firmware
> (Intel
> > +  Connection Manager) based controller. They also create an interface
> > +for
> > +  netlink communication with a user space daemon.
> > +
> > +- net.c/net.h:	These files implement the 'eth' interface for the
> > +  Thunderbolt(TM) Networking.
> 
> You don't have to list the files, right?
> 
> > +
> > +Interface to User Space
> > +=======================
> > +
> > +The interface to the user space module is implemented through a Generic
> Netlink.
> 
> Huh?  Not a normal network device?
> 
> > +This is the communications protocol between the Thunderbolt driver
> > +and the user space application.
> 
> What userspace application?
> 
> > +Note that this interface mediates user space communication with ICM.
> 
> What is ICM?
> 
> > +(Existing Linux tools can be used to configure the network
> > +interface.)
> 
> Why aren't they sufficient for everything here?
> 
> > +
> > +The Thunderbolt Daemon utilizes this interface to communicate with the
> driver.
> 
> What "Thunderbolt Daemon"?  What is "this"?
> 
> > +To be accessed by the user space module, both kernel and user space
> > +modules have to register with the same GENL_NAME.
> 
> What userspace modules?
> 
> How do you set GENL_NAME?  What is that?
> 
> > +For the purpose of the Thunderbolt Network driver, "thunderbolt" is
> used.
> 
> What do you mean?
> 
> > +The registration is done at driver initialization time for all
> > +instances of the Thunderbolt controllers.
> 
> Who does this?
> 
> > The communication is carried through pre-defined
> > +Thunderbolt messages. Each specific message has a callback function
> > +that is called when the related message is received.
> 
> Is this the internal structure of the driver?  If so, why not just put it all in the
> code itself and build the documentation from it?
> 
> > +
> > +Message Definitions:
> > +* NHI_CMD_UNSPEC: Not used.
> > +* NHI_CMD_SUBSCRIBE: Subscription request from daemon to driver to
> > +open the
> > +  communication channel.
> > +* NHI_CMD_UNSUBSCRIBE: Request from daemon to driver to
> unsubscribe
> > +and
> > +  to close communication channel.
> > +* NHI_CMD_QUERY_INFORMATION: Request information from the driver
> such
> > +as
> > +  driver version, FW version offset, number of ports in the
> > +controller
> > +  and DMA port.
> > +* NHI_CMD_MSG_TO_ICM: Message from user space module to FW.
> > +* NHI_CMD_MSG_FROM_ICM: Response from FW to user space module.
> > +* NHI_CMD_MAILBOX: Message that uses mailbox mechanism such as
> FW
> > +policy
> > +  changes or disconnect path.
> > +* NHI_CMD_APPROVE_TBT_NETWORKING: Request from user space
> module to
> > +FW to
> > +  establish path.
> > +* NHI_CMD_ICM_IN_SAFE_MODE: Indication that the FW has entered
> safe mode.
> > +
> > +Communication with Intel Connection Manager(ICM) Firmware
> >
> +=========================================================
> > +
> > +There are several circular buffers in Thunderbolt each using Direct
> > +Memory Access (DMA).
> 
> Again, internal documentation?
> 
> Isn't this just a "normal" network device?  Why document any of this in a
> separate way?  What can a user do with this?  Who is the audience of this
> file?
> 
> still confused,
> 
> greg k-h

This document was one of the suggestions we got from internal-Intel kernel developers.
If you think it isn't needed, I'll remove it from next patch set.

^ permalink raw reply	[flat|nested] 28+ messages in thread

* RE: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-10 11:44     ` Greg KH
@ 2016-11-10 11:48       ` Levy, Amir (Jer)
  0 siblings, 0 replies; 28+ messages in thread
From: Levy, Amir (Jer) @ 2016-11-10 11:48 UTC (permalink / raw)
  To: Greg KH
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	Westerberg, Mika, Winkler, Tomas, Zhang, Xiong Y

On Thu, Nov 10 2016, 01:44 PM, Greg KH wrote:
> On Thu, Nov 10, 2016 at 11:39:19AM +0000, Levy, Amir (Jer) wrote:
> > > And how about getting some internal-Intel kernel developers to
> > > review and sign-off on this code?  Don't make the community do the
> > > review when you have access to resources like this.  You have an
> > > internal mailing list for this very purpose, use it!
> > >
> >
> > The review with the internal-Intel kernel developers was done before
> > submitting the first patch set.
> 
> Then why is their signed-off-by:s not on the patchset showing that they
> "bless" this series?
> 

Working on it.

> thanks,
> 
> greg k-h

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v9 7/8] thunderbolt: Networking doc
  2016-11-10 11:47     ` Levy, Amir (Jer)
@ 2016-11-10 11:53       ` Greg KH
  0 siblings, 0 replies; 28+ messages in thread
From: Greg KH @ 2016-11-10 11:53 UTC (permalink / raw)
  To: Levy, Amir (Jer)
  Cc: andreas.noever, bhelgaas, corbet, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	Westerberg, Mika, Winkler, Tomas, Zhang, Xiong Y

On Thu, Nov 10, 2016 at 11:47:57AM +0000, Levy, Amir (Jer) wrote:
> On Wed, Nov 9 2016, 06:00 PM, Greg KH wrote:
> > On Wed, Nov 09, 2016 at 04:20:07PM +0200, Amir Levy wrote:
> > > Adding Thunderbolt(TM) networking documentation.
> > >
> > > Signed-off-by: Amir Levy <amir.jer.levy@intel.com>
> > > ---
> > >  Documentation/00-INDEX                   |   2 +
> > >  Documentation/thunderbolt/networking.txt | 132
> > > +++++++++++++++++++++++++++++++
> > 
> > Note, new files should be in .rst format, and live in the new subdirectory for
> > them (somewhere in Documentation, don't know off the top of my head...)
> > 
> > >  2 files changed, 134 insertions(+)
> > >  create mode 100644 Documentation/thunderbolt/networking.txt
> > >
> > > diff --git a/Documentation/00-INDEX b/Documentation/00-INDEX index
> > > 3acc4f1..0239e68 100644
> > > --- a/Documentation/00-INDEX
> > > +++ b/Documentation/00-INDEX
> > > @@ -440,6 +440,8 @@ this_cpu_ops.txt
> > >  	- List rationale behind and the way to use this_cpu operations.
> > >  thermal/
> > >  	- directory with information on managing thermal issues (CPU/temp)
> > > +thunderbolt/
> > > +	- directory with info regarding Thunderbolt.
> > >  trace/
> > >  	- directory with info on tracing technologies within linux
> > > unaligned-memory-access.txt
> > 
> > This change is probably not needed.
> > 
> > > diff --git a/Documentation/thunderbolt/networking.txt
> > > b/Documentation/thunderbolt/networking.txt
> > > new file mode 100644
> > > index 0000000..88d1c12
> > > --- /dev/null
> > > +++ b/Documentation/thunderbolt/networking.txt
> > > @@ -0,0 +1,132 @@
> > > +Intel Thunderbolt(TM) Networking driver
> > > +=======================================
> > > +
> > > +Copyright(c) 2013 - 2016 Intel Corporation.
> > > +
> > > +Contact Information:
> > > +Intel Thunderbolt mailing list <thunderbolt-software@lists.01.org>
> > > +Edited by Amir Levy <amir.jer.levy@intel.com>
> > > +
> > > +Overview
> > > +========
> > > +
> > > +* The Thunderbolt Networking driver enables peer to peer networking
> > > +on non-Apple
> > > +  platforms running Linux.
> > > +
> > > +* The driver creates a virtual Ethernet device that enables computer
> > > +to computer
> > > +  communication over the Thunderbolt cable.
> > > +
> > > +* Using Thunderbolt Networking you can perform high speed file
> > > +transfers between
> > > +  computers, perform PC migrations and/or set up small workgroups
> > > +with shared
> > > +  storage without compromising any other Thunderbolt functionality.
> > > +
> > > +* The driver is located in drivers/thunderbolt/icm.
> > > +
> > > +* This driver will function only on non-Apple platforms with firmware
> > > +based
> > > +  Thunderbolt controllers that support Thunderbolt Networking.
> > > +
> > > +  +----------------+            +----------------+
> > > +  |Host 1          |            |Host 2          |
> > > +  |                |            |                |
> > > +  |   +-------+    |            |    +-------+   |
> > > +  |   |Network|    |            |    |Network|   |
> > > +  |   |Stack  |    |            |    |Stack  |   |
> > > +  |   +-------+    |            |    +-------+   |
> > > +  |       ^        |            |        ^       |
> > > +  |       |        |            |        |       |
> > > +  |       v        |            |        v       |
> > > +  | +-----------+  |            |  +-----------+ |
> > > +  | |Thunderbolt|  |            |  |Thunderbolt| |
> > > +  | |Networking |  |            |  |Networking | |
> > > +  | |Driver     |  |            |  |Driver     | |
> > > +  | +-----------+  |            |  +-----------+ |
> > > +  |       ^        |            |        ^       |
> > > +  |       |        |            |        |       |
> > > +  |       v        |            |        v       |
> > > +  | +-----------+  |            |  +-----------+ |
> > > +  | |Thunderbolt|  |            |  |Thunderbolt| |
> > > +  | |Controller |<-+------------+->|Controller | |
> > > +  | |with ICM   |  |            |  |with ICM   | |
> > > +  | |enabled    |  |            |  |enabled    | |
> > > +  | +-----------+  |            |  +-----------+ |
> > > +  +----------------+            +----------------+
> > > +
> > > +Files
> > > +=====
> > > +
> > > +The following files are located in the drivers/thunderbolt/icm directory:
> > > +
> > > +- icm_nhi.c/h:	These files allow communication with the firmware
> > (Intel
> > > +  Connection Manager) based controller. They also create an interface
> > > +for
> > > +  netlink communication with a user space daemon.
> > > +
> > > +- net.c/net.h:	These files implement the 'eth' interface for the
> > > +  Thunderbolt(TM) Networking.
> > 
> > You don't have to list the files, right?
> > 
> > > +
> > > +Interface to User Space
> > > +=======================
> > > +
> > > +The interface to the user space module is implemented through a Generic
> > Netlink.
> > 
> > Huh?  Not a normal network device?
> > 
> > > +This is the communications protocol between the Thunderbolt driver
> > > +and the user space application.
> > 
> > What userspace application?
> > 
> > > +Note that this interface mediates user space communication with ICM.
> > 
> > What is ICM?
> > 
> > > +(Existing Linux tools can be used to configure the network
> > > +interface.)
> > 
> > Why aren't they sufficient for everything here?
> > 
> > > +
> > > +The Thunderbolt Daemon utilizes this interface to communicate with the
> > driver.
> > 
> > What "Thunderbolt Daemon"?  What is "this"?
> > 
> > > +To be accessed by the user space module, both kernel and user space
> > > +modules have to register with the same GENL_NAME.
> > 
> > What userspace modules?
> > 
> > How do you set GENL_NAME?  What is that?
> > 
> > > +For the purpose of the Thunderbolt Network driver, "thunderbolt" is
> > used.
> > 
> > What do you mean?
> > 
> > > +The registration is done at driver initialization time for all
> > > +instances of the Thunderbolt controllers.
> > 
> > Who does this?
> > 
> > > The communication is carried through pre-defined
> > > +Thunderbolt messages. Each specific message has a callback function
> > > +that is called when the related message is received.
> > 
> > Is this the internal structure of the driver?  If so, why not just put it all in the
> > code itself and build the documentation from it?
> > 
> > > +
> > > +Message Definitions:
> > > +* NHI_CMD_UNSPEC: Not used.
> > > +* NHI_CMD_SUBSCRIBE: Subscription request from daemon to driver to
> > > +open the
> > > +  communication channel.
> > > +* NHI_CMD_UNSUBSCRIBE: Request from daemon to driver to
> > unsubscribe
> > > +and
> > > +  to close communication channel.
> > > +* NHI_CMD_QUERY_INFORMATION: Request information from the driver
> > such
> > > +as
> > > +  driver version, FW version offset, number of ports in the
> > > +controller
> > > +  and DMA port.
> > > +* NHI_CMD_MSG_TO_ICM: Message from user space module to FW.
> > > +* NHI_CMD_MSG_FROM_ICM: Response from FW to user space module.
> > > +* NHI_CMD_MAILBOX: Message that uses mailbox mechanism such as
> > FW
> > > +policy
> > > +  changes or disconnect path.
> > > +* NHI_CMD_APPROVE_TBT_NETWORKING: Request from user space
> > module to
> > > +FW to
> > > +  establish path.
> > > +* NHI_CMD_ICM_IN_SAFE_MODE: Indication that the FW has entered
> > safe mode.
> > > +
> > > +Communication with Intel Connection Manager(ICM) Firmware
> > >
> > +=========================================================
> > > +
> > > +There are several circular buffers in Thunderbolt each using Direct
> > > +Memory Access (DMA).
> > 
> > Again, internal documentation?
> > 
> > Isn't this just a "normal" network device?  Why document any of this in a
> > separate way?  What can a user do with this?  Who is the audience of this
> > file?
> > 
> > still confused,
> > 
> > greg k-h
> 
> This document was one of the suggestions we got from internal-Intel kernel developers.
> If you think it isn't needed, I'll remove it from next patch set.

I'm not saying it is not needed, I'm saying that if you want it, it
needs to actually be useful :)

As it is, look at the questions I asked above, I really don't understand
this document.  And if I'm not the target audience for this, then great,
who is?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v9 7/8] thunderbolt: Networking doc
  2016-11-09 16:00   ` Greg KH
  2016-11-10 11:47     ` Levy, Amir (Jer)
@ 2016-11-10 14:24     ` Jonathan Corbet
  1 sibling, 0 replies; 28+ messages in thread
From: Jonathan Corbet @ 2016-11-10 14:24 UTC (permalink / raw)
  To: Greg KH
  Cc: Amir Levy, andreas.noever, bhelgaas, linux-kernel, linux-pci,
	netdev, linux-doc, mario_limonciello, thunderbolt-linux,
	mika.westerberg, tomas.winkler, xiong.y.zhang

On Wed, 9 Nov 2016 17:00:02 +0100
Greg KH <gregkh@linuxfoundation.org> wrote:

> >  Documentation/00-INDEX                   |   2 +
> >  Documentation/thunderbolt/networking.txt | 132 +++++++++++++++++++++++++++++++  
> 
> Note, new files should be in .rst format, and live in the new
> subdirectory for them (somewhere in Documentation, don't know off the
> top of my head...)

This one's almost in RST already, happily.

We haven't really figured out a hierarchy for device-specific docs like
this yet; I should get on that, I guess.

I do believe that we should separate documents for end users from those
aimed at kernel developers; those are two very different audiences
looking for different kinds of information.

Thanks,

jon

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-09 15:42   ` Levy, Amir (Jer)
@ 2016-11-15 10:59     ` Simon Guinot
  2016-11-18  8:48       ` Levy, Amir (Jer)
  0 siblings, 1 reply; 28+ messages in thread
From: Simon Guinot @ 2016-11-15 10:59 UTC (permalink / raw)
  To: Levy, Amir (Jer)
  Cc: gregkh, andreas.noever, bhelgaas, corbet, linux-kernel,
	linux-pci, netdev, linux-doc, mario_limonciello,
	thunderbolt-linux, Westerberg, Mika, Winkler, Tomas, Zhang,
	Xiong Y, Jamet, Michael, remi.rerolle

[-- Attachment #1: Type: text/plain, Size: 4442 bytes --]

On Wed, Nov 09, 2016 at 03:42:53PM +0000, Levy, Amir (Jer) wrote:
> On Wed, Nov 9 2016, 04:36 PM, Simon Guinot wrote:
> > On Wed, Nov 09, 2016 at 04:20:00PM +0200, Amir Levy wrote:
> > > This driver enables Thunderbolt Networking on non-Apple platforms 
> > > running Linux.
> > >
> > > Thunderbolt Networking provides peer-to-peer connections to transfer 
> > > files between computers, perform PC migrations, and/or set up small 
> > > workgroups with shared storage.
> > >
> > > This is a virtual connection that emulates an Ethernet adapter that 
> > > enables Ethernet networking with the benefit of Thunderbolt 
> > > superfast medium capability.
> > >
> > > Thunderbolt Networking enables two hosts and several devices that 
> > > have a Thunderbolt controller to be connected together in a linear 
> > > (Daisy
> > > chain) series from a single port.
> > >
> > > Thunderbolt Networking for Linux is compatible with Thunderbolt 
> > > Networking on systems running macOS or Windows and also supports 
> > > Thunderbolt generation 2 and 3 controllers.
> > >
> > > Note that all pre-existing Thunderbolt generation 3 features, such 
> > > as USB, Display and other Thunderbolt device connectivity will 
> > > continue to function exactly as they did prior to enabling Thunderbolt Networking.
> > >
> > > Code and Software Specifications:
> > > This kernel code creates a virtual ethernet device for computer to 
> > > computer communication over a Thunderbolt cable.
> > > The new driver is a separate driver to the existing Thunderbolt driver.
> > > It is designed to work on systems running Linux that interface with 
> > > Intel Connection Manager (ICM) firmware based Thunderbolt 
> > > controllers that support Thunderbolt Networking.
> > > The kernel code operates in coordination with the Thunderbolt user- 
> > > space daemon to implement full Thunderbolt networking functionality.
> > >
> > > Hardware Specifications:
> > > Thunderbolt Hardware specs have not yet been published but are used 
> > > where necessary for register definitions.
> > 
> > Hi Amir,
> > 
> > I have an ASUS "All Series/Z87-DELUXE/QUAD" motherboard with a 
> > Thunderbolt 2 "Falcon Ridge" chipset (device ID 156d).
> > 
> > Is the thunderbolt-icm driver supposed to work with this chipset ?
> > 
> 
> Yes, the thunderbolt-icm supports Falcon Ridge, device ID 156c.
> 156d is the bridge - http://lxr.free-electrons.com/source/include/linux/pci_ids.h#L2619
> 
> > I have installed both a 4.8.6 Linux kernel (patched with your v9
> > series) and the thunderbolt-software-daemon (27 october release) 
> > inside a Debian system (Jessie).
> > 
> > If I connect the ASUS motherboard with a MacBook Pro (Thunderbolt 2, 
> > device ID 156c), I can see that the thunderbolt-icm driver is loaded 
> > and that the thunderbolt-software-daemon is well started. But the 
> > Ethernet interface is not created.
> > 
> > I have attached to this email the syslog file. There is the logs from 
> > both the kernel and the daemon inside. Note that the daemon logs are 
> > everything but clear about what could be the issue. Maybe I missed 
> > some kind of configuration ? But I failed to find any valuable 
> > information about configuring the driver and/or the daemon in the various documentation files.
> > 
> > Please, can you provide some guidance ? I'd really like to test your 
> > patch series.
> 
> First, thank you very much for willing to test it.
> Thunderbolt Networking support was added during Falcon Ridge, in the latest FR images.
> Do you know which Thunderbolt image version you have on your system?
> Currently I submitted only Thunderbolt Networking feature in Linux, and we plan to add
> more features like reading the image version and updating the image.
> If you don't know the image version, the only thing I can suggest is to load windows, install thunderbolt SW
> and check in the Thunderbolt application the image version.
> To know if image update is needed, you can check - https://thunderbolttechnology.net/updates

Hi Amir,

From the Windows Thunderbolt software, I can read 13.00 for the firmware
version. And from https://thunderbolttechnology.net/updates, I can see
that there is no update available for my ASUS motherboard.

Am I good to go ?

BTW, it is quite a shame that the Thunderbolt firmware version can't be
read from Linux.

Simon

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

^ permalink raw reply	[flat|nested] 28+ messages in thread

* RE: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-15 10:59     ` Simon Guinot
@ 2016-11-18  8:48       ` Levy, Amir (Jer)
  2016-11-18 10:07         ` gregkh
  2016-11-18 11:20         ` Simon Guinot
  0 siblings, 2 replies; 28+ messages in thread
From: Levy, Amir (Jer) @ 2016-11-18  8:48 UTC (permalink / raw)
  To: Simon Guinot
  Cc: gregkh, andreas.noever, bhelgaas, corbet, linux-kernel,
	linux-pci, netdev, linux-doc, mario_limonciello,
	thunderbolt-linux, Westerberg, Mika, Winkler, Tomas, Zhang,
	Xiong Y, Jamet, Michael, remi.rerolle

On Tue, Nov 15 2016, 12:59 PM, Simon Guinot wrote:
> On Wed, Nov 09, 2016 at 03:42:53PM +0000, Levy, Amir (Jer) wrote:
> > On Wed, Nov 9 2016, 04:36 PM, Simon Guinot wrote:
> > > Hi Amir,
> > >
> > > I have an ASUS "All Series/Z87-DELUXE/QUAD" motherboard with a 
> > > Thunderbolt 2 "Falcon Ridge" chipset (device ID 156d).
> > >
> > > Is the thunderbolt-icm driver supposed to work with this chipset ?
> > >
> >
> > Yes, the thunderbolt-icm supports Falcon Ridge, device ID 156c.
> > 156d is the bridge -
> > http://lxr.free-electrons.com/source/include/linux/pci_ids.h#L2619
> >
> > > I have installed both a 4.8.6 Linux kernel (patched with your v9
> > > series) and the thunderbolt-software-daemon (27 october release) 
> > > inside a Debian system (Jessie).
> > >
> > > If I connect the ASUS motherboard with a MacBook Pro (Thunderbolt 
> > > 2, device ID 156c), I can see that the thunderbolt-icm driver is 
> > > loaded and that the thunderbolt-software-daemon is well started. 
> > > But the Ethernet interface is not created.
> > >
> > > I have attached to this email the syslog file. There is the logs 
> > > from both the kernel and the daemon inside. Note that the daemon 
> > > logs are everything but clear about what could be the issue. Maybe 
> > > I missed some kind of configuration ? But I failed to find any 
> > > valuable information about configuring the driver and/or the 
> > > daemon in
> the various documentation files.
> > >
> > > Please, can you provide some guidance ? I'd really like to test 
> > > your patch series.
> >
> > First, thank you very much for willing to test it.
> > Thunderbolt Networking support was added during Falcon Ridge, in the
> latest FR images.
> > Do you know which Thunderbolt image version you have on your system?
> > Currently I submitted only Thunderbolt Networking feature in Linux, 
> > and we plan to add more features like reading the image version and
> updating the image.
> > If you don't know the image version, the only thing I can suggest is 
> > to load windows, install thunderbolt SW and check in the Thunderbolt
> application the image version.
> > To know if image update is needed, you can check - 
> > https://thunderbolttechnology.net/updates
> 
> Hi Amir,
> 
> From the Windows Thunderbolt software, I can read 13.00 for the 
> firmware version. And from https://thunderbolttechnology.net/updates, 
> I can see that there is no update available for my ASUS motherboard.
> 
> Am I good to go ?
> 

Thunderbolt Networking is supported on both Thunderbolt(tm) 2 and Thunderbolt(tm) 3 systems.  
Thunderbolt 2 systems must have updated NVM (version 25 or later) in order for the functionality to work properly.  
If the system does not have the update, please contact the OEM directly for an updated NVM.  
For best functionality and support, Intel recommends using Thunderbolt 3 systems for all validation and testing.

> BTW, it is quite a shame that the Thunderbolt firmware version can't 
> be read from Linux.
> 

This is WIP, once this patch will be upstream, we will be able to focus more
on aligning Linux with the Thunderbolt features that we have for windows.

Regards,
Amir

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-18  8:48       ` Levy, Amir (Jer)
@ 2016-11-18 10:07         ` gregkh
  2016-11-20  6:30           ` Levy, Amir (Jer)
  2016-11-18 11:20         ` Simon Guinot
  1 sibling, 1 reply; 28+ messages in thread
From: gregkh @ 2016-11-18 10:07 UTC (permalink / raw)
  To: Levy, Amir (Jer)
  Cc: Simon Guinot, andreas.noever, bhelgaas, corbet, linux-kernel,
	linux-pci, netdev, linux-doc, mario_limonciello,
	thunderbolt-linux, Westerberg, Mika, Winkler, Tomas, Zhang,
	Xiong Y, Jamet, Michael, remi.rerolle

On Fri, Nov 18, 2016 at 08:48:36AM +0000, Levy, Amir (Jer) wrote:
> > BTW, it is quite a shame that the Thunderbolt firmware version can't 
> > be read from Linux.
> > 
> 
> This is WIP, once this patch will be upstream, we will be able to focus more
> on aligning Linux with the Thunderbolt features that we have for windows.

Why is this patch somehow holding that work back?  You aren't just
sitting around waiting for people to review this and not doing anything
else, right?  Is there some basic building block in these patches that
your firmware download code is going to rely on?

confused,

greg k-h

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-18  8:48       ` Levy, Amir (Jer)
  2016-11-18 10:07         ` gregkh
@ 2016-11-18 11:20         ` Simon Guinot
  2016-11-22 17:28           ` Simon Guinot
  1 sibling, 1 reply; 28+ messages in thread
From: Simon Guinot @ 2016-11-18 11:20 UTC (permalink / raw)
  To: Levy, Amir (Jer)
  Cc: gregkh, andreas.noever, bhelgaas, corbet, linux-kernel,
	linux-pci, netdev, linux-doc, mario_limonciello,
	thunderbolt-linux, Westerberg, Mika, Winkler, Tomas, Zhang,
	Xiong Y, Jamet, Michael, remi.rerolle

[-- Attachment #1: Type: text/plain, Size: 4034 bytes --]

On Fri, Nov 18, 2016 at 08:48:36AM +0000, Levy, Amir (Jer) wrote:
> On Tue, Nov 15 2016, 12:59 PM, Simon Guinot wrote:
> > On Wed, Nov 09, 2016 at 03:42:53PM +0000, Levy, Amir (Jer) wrote:
> > > On Wed, Nov 9 2016, 04:36 PM, Simon Guinot wrote:
> > > > Hi Amir,
> > > >
> > > > I have an ASUS "All Series/Z87-DELUXE/QUAD" motherboard with a 
> > > > Thunderbolt 2 "Falcon Ridge" chipset (device ID 156d).
> > > >
> > > > Is the thunderbolt-icm driver supposed to work with this chipset ?
> > > >
> > >
> > > Yes, the thunderbolt-icm supports Falcon Ridge, device ID 156c.
> > > 156d is the bridge -
> > > http://lxr.free-electrons.com/source/include/linux/pci_ids.h#L2619
> > >
> > > > I have installed both a 4.8.6 Linux kernel (patched with your v9
> > > > series) and the thunderbolt-software-daemon (27 october release) 
> > > > inside a Debian system (Jessie).
> > > >
> > > > If I connect the ASUS motherboard with a MacBook Pro (Thunderbolt 
> > > > 2, device ID 156c), I can see that the thunderbolt-icm driver is 
> > > > loaded and that the thunderbolt-software-daemon is well started. 
> > > > But the Ethernet interface is not created.
> > > >
> > > > I have attached to this email the syslog file. There is the logs 
> > > > from both the kernel and the daemon inside. Note that the daemon 
> > > > logs are everything but clear about what could be the issue. Maybe 
> > > > I missed some kind of configuration ? But I failed to find any 
> > > > valuable information about configuring the driver and/or the 
> > > > daemon in
> > the various documentation files.
> > > >
> > > > Please, can you provide some guidance ? I'd really like to test 
> > > > your patch series.
> > >
> > > First, thank you very much for willing to test it.
> > > Thunderbolt Networking support was added during Falcon Ridge, in the
> > latest FR images.
> > > Do you know which Thunderbolt image version you have on your system?
> > > Currently I submitted only Thunderbolt Networking feature in Linux, 
> > > and we plan to add more features like reading the image version and
> > updating the image.
> > > If you don't know the image version, the only thing I can suggest is 
> > > to load windows, install thunderbolt SW and check in the Thunderbolt
> > application the image version.
> > > To know if image update is needed, you can check - 
> > > https://thunderbolttechnology.net/updates
> > 
> > Hi Amir,
> > 
> > From the Windows Thunderbolt software, I can read 13.00 for the 
> > firmware version. And from https://thunderbolttechnology.net/updates, 
> > I can see that there is no update available for my ASUS motherboard.
> > 
> > Am I good to go ?
> > 
> 
> Thunderbolt Networking is supported on both Thunderbolt(tm) 2 and Thunderbolt(tm) 3 systems.  
> Thunderbolt 2 systems must have updated NVM (version 25 or later) in order for the functionality to work properly.  
> If the system does not have the update, please contact the OEM directly for an updated NVM.  
> For best functionality and support, Intel recommends using Thunderbolt 3 systems for all validation and testing.

Maybe it is worth mentioning in the documentation and/or in the Kconfig
help message that a minimal firmware version is needed for Thunderbolt 2
controllers.

It would have saved some time for me :)

> 
> > BTW, it is quite a shame that the Thunderbolt firmware version can't 
> > be read from Linux.
> > 
> 
> This is WIP, once this patch will be upstream, we will be able to focus more
> on aligning Linux with the Thunderbolt features that we have for windows.

Well, I rather see the firmware identification and update as basic
features on the top of which ones you can build a driver. For example in
this case this would allow the ICM driver and/or the userland daemon to
exit with a useful error message rather than just not working without any
explanation.

Next week I'll try the driver with a Thunderbolt 3 controller.

Thanks for your help!

Simon

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

^ permalink raw reply	[flat|nested] 28+ messages in thread

* RE: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-18 10:07         ` gregkh
@ 2016-11-20  6:30           ` Levy, Amir (Jer)
  2016-11-21 12:22             ` gregkh
  0 siblings, 1 reply; 28+ messages in thread
From: Levy, Amir (Jer) @ 2016-11-20  6:30 UTC (permalink / raw)
  To: gregkh
  Cc: Simon Guinot, andreas.noever, bhelgaas, corbet, linux-kernel,
	linux-pci, netdev, linux-doc, mario_limonciello,
	thunderbolt-linux, Westerberg, Mika, Winkler, Tomas, Zhang,
	Xiong Y, Jamet, Michael, remi.rerolle

On Fri, Nov 18 2016, 12:07 PM, gregkh@linuxfoundation.org wrote:
> On Fri, Nov 18, 2016 at 08:48:36AM +0000, Levy, Amir (Jer) wrote:
> > > BTW, it is quite a shame that the Thunderbolt firmware version can't
> > > be read from Linux.
> > >
> >
> > This is WIP, once this patch will be upstream, we will be able to
> > focus more on aligning Linux with the Thunderbolt features that we have
> for windows.
> 
> Why is this patch somehow holding that work back?  You aren't just sitting
> around waiting for people to review this and not doing anything else, right?
> Is there some basic building block in these patches that your firmware
> download code is going to rely on?
> 
> confused,
> 
> greg k-h

All the Thunderbolt SW features (including networking and FW update) depend 
on the communication with FW, which is patch 3/8 in the series.
The patch also sets up a generic netlink for user space communication.
So yes, the communication with Thunderbolt FW is a basic building block.

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-20  6:30           ` Levy, Amir (Jer)
@ 2016-11-21 12:22             ` gregkh
  0 siblings, 0 replies; 28+ messages in thread
From: gregkh @ 2016-11-21 12:22 UTC (permalink / raw)
  To: Levy, Amir (Jer)
  Cc: Simon Guinot, andreas.noever, bhelgaas, corbet, linux-kernel,
	linux-pci, netdev, linux-doc, mario_limonciello,
	thunderbolt-linux, Westerberg, Mika, Winkler, Tomas, Zhang,
	Xiong Y, Jamet, Michael, remi.rerolle

On Sun, Nov 20, 2016 at 06:30:19AM +0000, Levy, Amir (Jer) wrote:
> On Fri, Nov 18 2016, 12:07 PM, gregkh@linuxfoundation.org wrote:
> > On Fri, Nov 18, 2016 at 08:48:36AM +0000, Levy, Amir (Jer) wrote:
> > > > BTW, it is quite a shame that the Thunderbolt firmware version can't
> > > > be read from Linux.
> > > >
> > >
> > > This is WIP, once this patch will be upstream, we will be able to
> > > focus more on aligning Linux with the Thunderbolt features that we have
> > for windows.
> > 
> > Why is this patch somehow holding that work back?  You aren't just sitting
> > around waiting for people to review this and not doing anything else, right?
> > Is there some basic building block in these patches that your firmware
> > download code is going to rely on?
> > 
> > confused,
> > 
> > greg k-h
> 
> All the Thunderbolt SW features (including networking and FW update) depend 
> on the communication with FW, which is patch 3/8 in the series.
> The patch also sets up a generic netlink for user space communication.

It's that "generic netlink" connection that I really want a whole lot of
revewers to read over as it's very unusual and "different" from all
other driver subsystems.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 28+ messages in thread

* Re: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-18 11:20         ` Simon Guinot
@ 2016-11-22 17:28           ` Simon Guinot
  2016-11-22 17:36             ` Mario.Limonciello
  2016-11-24 20:35             ` Levy, Amir (Jer)
  0 siblings, 2 replies; 28+ messages in thread
From: Simon Guinot @ 2016-11-22 17:28 UTC (permalink / raw)
  To: Levy, Amir (Jer)
  Cc: gregkh, andreas.noever, bhelgaas, corbet, linux-kernel,
	linux-pci, netdev, linux-doc, mario_limonciello,
	thunderbolt-linux, Westerberg, Mika, Winkler, Tomas, Zhang,
	Xiong Y, Jamet, Michael, remi.rerolle

[-- Attachment #1: Type: text/plain, Size: 5819 bytes --]

On Fri, Nov 18, 2016 at 12:20:07PM +0100, Simon Guinot wrote:
> On Fri, Nov 18, 2016 at 08:48:36AM +0000, Levy, Amir (Jer) wrote:
> > On Tue, Nov 15 2016, 12:59 PM, Simon Guinot wrote:
> > > On Wed, Nov 09, 2016 at 03:42:53PM +0000, Levy, Amir (Jer) wrote:
> > > > On Wed, Nov 9 2016, 04:36 PM, Simon Guinot wrote:
> > > > > Hi Amir,
> > > > >
> > > > > I have an ASUS "All Series/Z87-DELUXE/QUAD" motherboard with a 
> > > > > Thunderbolt 2 "Falcon Ridge" chipset (device ID 156d).
> > > > >
> > > > > Is the thunderbolt-icm driver supposed to work with this chipset ?
> > > > >
> > > >
> > > > Yes, the thunderbolt-icm supports Falcon Ridge, device ID 156c.
> > > > 156d is the bridge -
> > > > http://lxr.free-electrons.com/source/include/linux/pci_ids.h#L2619
> > > >
> > > > > I have installed both a 4.8.6 Linux kernel (patched with your v9
> > > > > series) and the thunderbolt-software-daemon (27 october release) 
> > > > > inside a Debian system (Jessie).
> > > > >
> > > > > If I connect the ASUS motherboard with a MacBook Pro (Thunderbolt 
> > > > > 2, device ID 156c), I can see that the thunderbolt-icm driver is 
> > > > > loaded and that the thunderbolt-software-daemon is well started. 
> > > > > But the Ethernet interface is not created.
> > > > >
> > > > > I have attached to this email the syslog file. There is the logs 
> > > > > from both the kernel and the daemon inside. Note that the daemon 
> > > > > logs are everything but clear about what could be the issue. Maybe 
> > > > > I missed some kind of configuration ? But I failed to find any 
> > > > > valuable information about configuring the driver and/or the 
> > > > > daemon in
> > > the various documentation files.
> > > > >
> > > > > Please, can you provide some guidance ? I'd really like to test 
> > > > > your patch series.
> > > >
> > > > First, thank you very much for willing to test it.
> > > > Thunderbolt Networking support was added during Falcon Ridge, in the
> > > latest FR images.
> > > > Do you know which Thunderbolt image version you have on your system?
> > > > Currently I submitted only Thunderbolt Networking feature in Linux, 
> > > > and we plan to add more features like reading the image version and
> > > updating the image.
> > > > If you don't know the image version, the only thing I can suggest is 
> > > > to load windows, install thunderbolt SW and check in the Thunderbolt
> > > application the image version.
> > > > To know if image update is needed, you can check - 
> > > > https://thunderbolttechnology.net/updates
> > > 
> > > Hi Amir,
> > > 
> > > From the Windows Thunderbolt software, I can read 13.00 for the 
> > > firmware version. And from https://thunderbolttechnology.net/updates, 
> > > I can see that there is no update available for my ASUS motherboard.
> > > 
> > > Am I good to go ?
> > > 
> > 
> > Thunderbolt Networking is supported on both Thunderbolt(tm) 2 and Thunderbolt(tm) 3 systems.  
> > Thunderbolt 2 systems must have updated NVM (version 25 or later) in order for the functionality to work properly.  
> > If the system does not have the update, please contact the OEM directly for an updated NVM.  
> > For best functionality and support, Intel recommends using Thunderbolt 3 systems for all validation and testing.
> 
> Maybe it is worth mentioning in the documentation and/or in the Kconfig
> help message that a minimal firmware version is needed for Thunderbolt 2
> controllers.
> 
> It would have saved some time for me :)
> 
> > 
> > > BTW, it is quite a shame that the Thunderbolt firmware version can't 
> > > be read from Linux.
> > > 
> > 
> > This is WIP, once this patch will be upstream, we will be able to focus more
> > on aligning Linux with the Thunderbolt features that we have for windows.
> 
> Well, I rather see the firmware identification and update as basic
> features on the top of which ones you can build a driver. For example in
> this case this would allow the ICM driver and/or the userland daemon to
> exit with a useful error message rather than just not working without any
> explanation.
> 
> Next week I'll try the driver with a Thunderbolt 3 controller.

Hi Amir,

I tested the thunderbolt-icm driver (v9 series) on an Gigabyte
motherboard (Z170X-UD5 TH-CF) with a Thunderbolt 3 controller (Alpine
Ridge 4C).

I can see that the network interface is well created when the
motherboard is connected to a MacBook Pro (Thunderbolt 2 or 3).

And here are the TCP bandwidths measured using the iperf3 benchmark:

- MacBook Pro Thunderbolt 2: 8.46Gbits/sec
- MacBook Pro Thunderbolt 3: 11.8Gbits/sec

Are this results consistent with your expectations ?

From the MacOS system interface on the MacBook Pro Thunderbolt 3,
I noticed that the interface appears as dual lane (2x 20Gb/sec). But
when two MacBook Pro are connected together, the interface appears as
single lane (1x 40Gb/sec). Is some lane bonding support missing in the
Linux implementation ?

Here are a couple of additional questions:

- When the network interface is created, there is no IP address
  assigned (or negotiated ?) on the Linux side. But it is done on the
  MacOS side. And in the Linux kernel logs I can also read the message:
  "ready for ThunderboltIP negotiation". Is there something missing or
  not working on the Linux side ? What is the correct way to configure
  or negotiate the IP address. For my tests I did it manually...

- When the Linux machine is started with the Thunderbolt wire already
  connected to a MacBook Pro, sometimes (but not every time) the
  network interface is not created. The Thunderbolt wire needs to be
  replugged.

FWIW you get my

Tested-by: Simon Guinot <simon.guinot@sequanux.org>

Simon

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

^ permalink raw reply	[flat|nested] 28+ messages in thread

* RE: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-22 17:28           ` Simon Guinot
@ 2016-11-22 17:36             ` Mario.Limonciello
  2016-11-24 20:35             ` Levy, Amir (Jer)
  1 sibling, 0 replies; 28+ messages in thread
From: Mario.Limonciello @ 2016-11-22 17:36 UTC (permalink / raw)
  To: simon.guinot, amir.jer.levy
  Cc: gregkh, andreas.noever, bhelgaas, corbet, linux-kernel,
	linux-pci, netdev, linux-doc, thunderbolt-linux, mika.westerberg,
	tomas.winkler, xiong.y.zhang, michael.jamet, remi.rerolle

> Here are a couple of additional questions:
> 
> - When the network interface is created, there is no IP address
>   assigned (or negotiated ?) on the Linux side. But it is done on the
>   MacOS side. And in the Linux kernel logs I can also read the message:
>   "ready for ThunderboltIP negotiation". Is there something missing or
>   not working on the Linux side ? What is the correct way to configure
>   or negotiate the IP address. For my tests I did it manually...
> 
> - When the Linux machine is started with the Thunderbolt wire already
>   connected to a MacBook Pro, sometimes (but not every time) the
>   network interface is not created. The Thunderbolt wire needs to be
>   replugged.
> 
> FWIW you get my
> 
> Tested-by: Simon Guinot <simon.guinot@sequanux.org>
> 
> Simon

Simon,

Since I also performed testing on the previous patchset, I'll share what I did.

I configured Network Manager to use the TBT interface to share an internet
connection to another box.  This configures a static IP address on the local
Linux side and sets up routing.

Network manager remembers setup this in a configuration database.  
When the interface goes up it will then set up a DHCP server to hand
out an IP address to the other side.

^ permalink raw reply	[flat|nested] 28+ messages in thread

* RE: [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking
  2016-11-22 17:28           ` Simon Guinot
  2016-11-22 17:36             ` Mario.Limonciello
@ 2016-11-24 20:35             ` Levy, Amir (Jer)
  1 sibling, 0 replies; 28+ messages in thread
From: Levy, Amir (Jer) @ 2016-11-24 20:35 UTC (permalink / raw)
  To: Simon Guinot
  Cc: gregkh, andreas.noever, bhelgaas, corbet, linux-kernel,
	linux-pci, netdev, linux-doc, mario_limonciello,
	thunderbolt-linux, Westerberg, Mika, Winkler, Tomas, Zhang,
	Xiong Y, remi.rerolle

On Tue, Nov 22 2016, 07:28 PM, Simon Guinot wrote:
> Hi Amir,
> 
> I tested the thunderbolt-icm driver (v9 series) on an Gigabyte 
> motherboard
> (Z170X-UD5 TH-CF) with a Thunderbolt 3 controller (Alpine Ridge 4C).
> 
> I can see that the network interface is well created when the 
> motherboard is connected to a MacBook Pro (Thunderbolt 2 or 3).
> 
> And here are the TCP bandwidths measured using the iperf3 benchmark:

AFAIK, in UD5, Thunderbolt 3 is installed on 4 lanes of PCI Express Gen 3,
which is good for performance.

> 
> - MacBook Pro Thunderbolt 2: 8.46Gbits/sec
> - MacBook Pro Thunderbolt 3: 11.8Gbits/sec
> 
> Are this results consistent with your expectations ?

Thunderbolt 2 - yes,
Thunderbolt 3 - we didn't check the bandwidth with MacBook Pro Thunderbolt 3 yet.
The bandwidth in Linux<->Linux and Linux<->Windows setups with Thunderbolt 3
is around 15G (one direction).

> 
> From the MacOS system interface on the MacBook Pro Thunderbolt 3, I 
> noticed that the interface appears as dual lane (2x 20Gb/sec). But 
> when two MacBook Pro are connected together, the interface appears as 
> single lane (1x 40Gb/sec). Is some lane bonding support missing in the 
> Linux implementation ?

At the moment the iCM doesn't lane bond the inter domain link.
This will be added in a future drop of the iCM once the networking BW will be such that requires the bonding.

> 
> Here are a couple of additional questions:
> 
> - When the network interface is created, there is no IP address
>   assigned (or negotiated ?) on the Linux side. But it is done on the
>   MacOS side. And in the Linux kernel logs I can also read the message:
>   "ready for ThunderboltIP negotiation". Is there something missing or
>   not working on the Linux side ? What is the correct way to configure
>   or negotiate the IP address. For my tests I did it manually...

This is not related to Thunderbolt, but to OS administration.
I usually use ZeroConf in any OS that support Thunderbolt Networking, but static IP also works.
"ready for ThunderboltIP negotiation" is an important part of the path establishment, but isn't related to IP address.

Thanks,
Amir

^ permalink raw reply	[flat|nested] 28+ messages in thread

end of thread, other threads:[~2016-11-24 20:35 UTC | newest]

Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-11-09 14:20 [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Amir Levy
2016-11-09 14:20 ` [PATCH v9 1/8] thunderbolt: Macro rename Amir Levy
2016-11-09 14:20 ` [PATCH v9 2/8] thunderbolt: Updating the register definitions Amir Levy
2016-11-09 14:20 ` [PATCH v9 3/8] thunderbolt: Communication with the ICM (firmware) Amir Levy
2016-11-09 14:20 ` [PATCH v9 4/8] thunderbolt: Networking state machine Amir Levy
2016-11-09 14:20 ` [PATCH v9 5/8] thunderbolt: Networking transmit and receive Amir Levy
2016-11-09 14:20 ` [PATCH v9 6/8] thunderbolt: Kconfig for Thunderbolt Networking Amir Levy
2016-11-09 14:20 ` [PATCH v9 7/8] thunderbolt: Networking doc Amir Levy
2016-11-09 16:00   ` Greg KH
2016-11-10 11:47     ` Levy, Amir (Jer)
2016-11-10 11:53       ` Greg KH
2016-11-10 14:24     ` Jonathan Corbet
2016-11-09 14:20 ` [PATCH v9 8/8] thunderbolt: Adding maintainer entry Amir Levy
2016-11-09 14:36 ` [PATCH v9 0/8] thunderbolt: Introducing Thunderbolt(TM) Networking Simon Guinot
2016-11-09 15:42   ` Levy, Amir (Jer)
2016-11-15 10:59     ` Simon Guinot
2016-11-18  8:48       ` Levy, Amir (Jer)
2016-11-18 10:07         ` gregkh
2016-11-20  6:30           ` Levy, Amir (Jer)
2016-11-21 12:22             ` gregkh
2016-11-18 11:20         ` Simon Guinot
2016-11-22 17:28           ` Simon Guinot
2016-11-22 17:36             ` Mario.Limonciello
2016-11-24 20:35             ` Levy, Amir (Jer)
2016-11-09 16:02 ` Greg KH
2016-11-10 11:39   ` Levy, Amir (Jer)
2016-11-10 11:44     ` Greg KH
2016-11-10 11:48       ` Levy, Amir (Jer)

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).