[net-next 0/6 v2][pull request] Intel Wired LAN Driver Updates

[net-next 0/6 v2][pull request] Intel Wired LAN Driver Updates

[Jeff Kirsher]

The following series contains updates to ixgbe, igbvf and igb.
This version of the series contains the following changes:

- igb fix/add check if subordinate VFs are assigned to VM's
- igbvf fix for trunk VLAN
- ixgbe 2 fixes for ethtool and 1 endianess fix

-v2 update the igb patch to resolve a variable initialization warning

The following are changes since commit fd38f734cb8200529e281338514945fcbff2364b:
  igbvf: convert to ndo_fix_features
and are available in the git repository at
  git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/net-next.git

Emil Tantilov (3):
  ixgbe: fix endianess when writing driver version to firmware
  ixgbe: allow eeprom writes via ethtool
  ixgbe: change the eeprom version reported by ethtool

Greg Rose (2):
  igbvf: Fix trunk vlan
  igb: Check if subordinate VFs are assigned to virtual machines

Jacob Keller (1):
  ixgbe: add hardware timestamping support

 drivers/net/ethernet/intel/igb/igb.h             |    3 +
 drivers/net/ethernet/intel/igb/igb_main.c        |  177 +++++++--
 drivers/net/ethernet/intel/igbvf/netdev.c        |    4 +-
 drivers/net/ethernet/intel/ixgbe/ixgbe.h         |   24 ++-
 drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c   |    2 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_common.c  |   16 +-
 drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c |   84 ++++-
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c    |  452 +++++++++++++++++++++-
 drivers/net/ethernet/intel/ixgbe/ixgbe_type.h    |   32 ++
 9 files changed, 743 insertions(+), 51 deletions(-)

-- 
1.7.6.4

[net-next 1/6] igbvf: Fix trunk vlan

[Jeff Kirsher]

From: Greg Rose <gregory.v.rose@intel.com>

Changes to clean up the VLAN Rx path by Jiri Pirko broke trunk VLAN.
Trunk VLANs in a VF driver are those set using

"ip link set <pfdev> vf <n> <vlanid>"

Signed-off-by: Greg Rose <gregory.v.rose@intel.com>
CC: Jiri Pirko <jpirko@redhat.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/igbvf/netdev.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/intel/igbvf/netdev.c b/drivers/net/ethernet/intel/igbvf/netdev.c
index 32b3044..23cc40f 100644
--- a/drivers/net/ethernet/intel/igbvf/netdev.c
+++ b/drivers/net/ethernet/intel/igbvf/netdev.c
@@ -102,8 +102,8 @@ static void igbvf_receive_skb(struct igbvf_adapter *adapter,
 {
 	if (status & E1000_RXD_STAT_VP) {
 		u16 vid = le16_to_cpu(vlan) & E1000_RXD_SPC_VLAN_MASK;
-
-		__vlan_hwaccel_put_tag(skb, vid);
+		if (test_bit(vid, adapter->active_vlans))
+			__vlan_hwaccel_put_tag(skb, vid);
 	}
 	netif_receive_skb(skb);
 }
-- 
1.7.6.4

[net-next 2/6 v2] igb: Check if subordinate VFs are assigned to virtual machines

[Jeff Kirsher]

From: Greg Rose <gregory.v.rose@intel.com>

Kvm and the Xen pci-back driver will set a flag in the virtual function
pci device dev_flags when the VF is assigned to a guest VM.  Before
destroying subordinate VFs check to see if the flag is set and if so
skip the call to pci_disable_sriov() to avoid system crashes.

Copy the maintainer for the Xen pci-back driver.  Also CC'ing
maintainers of all drivers found to call pci_disable_sriov().

V2 - Fix  uninitialized variable warning

Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Christian Benvenuti <benve@cisco.com>
Cc: Sathya Perla <sathya.perla@emulex.com>
Cc: Dimitris Michailidis <dm@chelsio.com>
Cc: Jon Mason <jdmason@kudzu.us>
Cc: James Smart <james.smart@emulex.com>
Signed-off-by: Greg Rose <gregory.v.rose@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/igb/igb.h      |    3 +
 drivers/net/ethernet/intel/igb/igb_main.c |  177 ++++++++++++++++++++++++-----
 2 files changed, 150 insertions(+), 30 deletions(-)

diff --git a/drivers/net/ethernet/intel/igb/igb.h b/drivers/net/ethernet/intel/igb/igb.h
index 4c500a7..5594430 100644
--- a/drivers/net/ethernet/intel/igb/igb.h
+++ b/drivers/net/ethernet/intel/igb/igb.h
@@ -72,6 +72,8 @@ struct igb_adapter;
 #define IGB_MAX_VF_MC_ENTRIES              30
 #define IGB_MAX_VF_FUNCTIONS               8
 #define IGB_MAX_VFTA_ENTRIES               128
+#define IGB_82576_VF_DEV_ID                0x10CA
+#define IGB_I350_VF_DEV_ID                 0x1520
 
 struct vf_data_storage {
 	unsigned char vf_mac_addresses[ETH_ALEN];
@@ -83,6 +85,7 @@ struct vf_data_storage {
 	u16 pf_vlan; /* When set, guest VLAN config not allowed. */
 	u16 pf_qos;
 	u16 tx_rate;
+	struct pci_dev *vfdev;
 };
 
 #define IGB_VF_FLAG_CTS            0x00000001 /* VF is clear to send data */
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index c10cc71..837adbb 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -162,6 +162,9 @@ static int igb_ndo_set_vf_bw(struct net_device *netdev, int vf, int tx_rate);
 static int igb_ndo_get_vf_config(struct net_device *netdev, int vf,
 				 struct ifla_vf_info *ivi);
 static void igb_check_vf_rate_limit(struct igb_adapter *);
+static int igb_vf_configure(struct igb_adapter *adapter, int vf);
+static int igb_find_enabled_vfs(struct igb_adapter *adapter);
+static int igb_check_vf_assignment(struct igb_adapter *adapter);
 
 #ifdef CONFIG_PM
 static int igb_suspend(struct pci_dev *, pm_message_t);
@@ -2232,8 +2235,12 @@ static void __devexit igb_remove(struct pci_dev *pdev)
 	/* reclaim resources allocated to VFs */
 	if (adapter->vf_data) {
 		/* disable iov and allow time for transactions to clear */
-		pci_disable_sriov(pdev);
-		msleep(500);
+		if (!igb_check_vf_assignment(adapter)) {
+			pci_disable_sriov(pdev);
+			msleep(500);
+		} else {
+			dev_info(&pdev->dev, "VF(s) assigned to guests!\n");
+		}
 
 		kfree(adapter->vf_data);
 		adapter->vf_data = NULL;
@@ -2270,42 +2277,49 @@ static void __devinit igb_probe_vfs(struct igb_adapter * adapter)
 {
 #ifdef CONFIG_PCI_IOV
 	struct pci_dev *pdev = adapter->pdev;
+	int old_vfs = igb_find_enabled_vfs(adapter);
+	int i;
 
-	if (adapter->vfs_allocated_count) {
-		adapter->vf_data = kcalloc(adapter->vfs_allocated_count,
-		                           sizeof(struct vf_data_storage),
-		                           GFP_KERNEL);
-		/* if allocation failed then we do not support SR-IOV */
-		if (!adapter->vf_data) {
-			adapter->vfs_allocated_count = 0;
-			dev_err(&pdev->dev, "Unable to allocate memory for VF "
-			        "Data Storage\n");
-		}
+	if (old_vfs) {
+		dev_info(&pdev->dev, "%d pre-allocated VFs found - override "
+			 "max_vfs setting of %d\n", old_vfs, max_vfs);
+		adapter->vfs_allocated_count = old_vfs;
 	}
 
-	if (pci_enable_sriov(pdev, adapter->vfs_allocated_count)) {
-		kfree(adapter->vf_data);
-		adapter->vf_data = NULL;
-#endif /* CONFIG_PCI_IOV */
+	if (!adapter->vfs_allocated_count)
+		return;
+
+	adapter->vf_data = kcalloc(adapter->vfs_allocated_count,
+				sizeof(struct vf_data_storage), GFP_KERNEL);
+	/* if allocation failed then we do not support SR-IOV */
+	if (!adapter->vf_data) {
 		adapter->vfs_allocated_count = 0;
-#ifdef CONFIG_PCI_IOV
-	} else {
-		unsigned char mac_addr[ETH_ALEN];
-		int i;
-		dev_info(&pdev->dev, "%d vfs allocated\n",
-		         adapter->vfs_allocated_count);
-		for (i = 0; i < adapter->vfs_allocated_count; i++) {
-			random_ether_addr(mac_addr);
-			igb_set_vf_mac(adapter, i, mac_addr);
-		}
-		/* DMA Coalescing is not supported in IOV mode. */
-		if (adapter->flags & IGB_FLAG_DMAC)
-			adapter->flags &= ~IGB_FLAG_DMAC;
+		dev_err(&pdev->dev, "Unable to allocate memory for VF "
+			"Data Storage\n");
+		goto out;
 	}
+
+	if (!old_vfs) {
+		if (pci_enable_sriov(pdev, adapter->vfs_allocated_count))
+			goto err_out;
+	}
+	dev_info(&pdev->dev, "%d VFs allocated\n",
+		 adapter->vfs_allocated_count);
+	for (i = 0; i < adapter->vfs_allocated_count; i++)
+		igb_vf_configure(adapter, i);
+
+	/* DMA Coalescing is not supported in IOV mode. */
+	adapter->flags &= ~IGB_FLAG_DMAC;
+	goto out;
+err_out:
+	kfree(adapter->vf_data);
+	adapter->vf_data = NULL;
+	adapter->vfs_allocated_count = 0;
+out:
+	return;
 #endif /* CONFIG_PCI_IOV */
 }
 
-
 /**
  * igb_init_hw_timer - Initialize hardware timer used with IEEE 1588 timestamp
  * @adapter: board private structure to initialize
@@ -4917,6 +4931,109 @@ static int igb_notify_dca(struct notifier_block *nb, unsigned long event,
 }
 #endif /* CONFIG_IGB_DCA */
 
+#ifdef CONFIG_PCI_IOV
+static int igb_vf_configure(struct igb_adapter *adapter, int vf)
+{
+	unsigned char mac_addr[ETH_ALEN];
+	struct pci_dev *pdev = adapter->pdev;
+	struct e1000_hw *hw = &adapter->hw;
+	struct pci_dev *pvfdev;
+	unsigned int device_id;
+	u16 thisvf_devfn;
+
+	random_ether_addr(mac_addr);
+	igb_set_vf_mac(adapter, vf, mac_addr);
+
+	switch (adapter->hw.mac.type) {
+	case e1000_82576:
+		device_id = IGB_82576_VF_DEV_ID;
+		/* VF Stride for 82576 is 2 */
+		thisvf_devfn = (pdev->devfn + 0x80 + (vf << 1)) |
+			(pdev->devfn & 1);
+		break;
+	case e1000_i350:
+		device_id = IGB_I350_VF_DEV_ID;
+		/* VF Stride for I350 is 4 */
+		thisvf_devfn = (pdev->devfn + 0x80 + (vf << 2)) |
+				(pdev->devfn & 3);
+		break;
+	default:
+		device_id = 0;
+		thisvf_devfn = 0;
+		break;
+	}
+
+	pvfdev = pci_get_device(hw->vendor_id, device_id, NULL);
+	while (pvfdev) {
+		if (pvfdev->devfn == thisvf_devfn)
+			break;
+		pvfdev = pci_get_device(hw->vendor_id,
+					device_id, pvfdev);
+	}
+
+	if (pvfdev)
+		adapter->vf_data[vf].vfdev = pvfdev;
+	else
+		dev_err(&pdev->dev,
+			"Couldn't find pci dev ptr for VF %4.4x\n",
+			thisvf_devfn);
+	return pvfdev != NULL;
+}
+
+static int igb_find_enabled_vfs(struct igb_adapter *adapter)
+{
+	struct e1000_hw *hw = &adapter->hw;
+	struct pci_dev *pdev = adapter->pdev;
+	struct pci_dev *pvfdev;
+	u16 vf_devfn = 0;
+	u16 vf_stride;
+	unsigned int device_id;
+	int vfs_found = 0;
+
+	switch (adapter->hw.mac.type) {
+	case e1000_82576:
+		device_id = IGB_82576_VF_DEV_ID;
+		/* VF Stride for 82576 is 2 */
+		vf_stride = 2;
+		break;
+	case e1000_i350:
+		device_id = IGB_I350_VF_DEV_ID;
+		/* VF Stride for I350 is 4 */
+		vf_stride = 4;
+		break;
+	default:
+		device_id = 0;
+		vf_stride = 0;
+		break;
+	}
+
+	vf_devfn = pdev->devfn + 0x80;
+	pvfdev = pci_get_device(hw->vendor_id, device_id, NULL);
+	while (pvfdev) {
+		if (pvfdev->devfn == vf_devfn)
+			vfs_found++;
+		vf_devfn += vf_stride;
+		pvfdev = pci_get_device(hw->vendor_id,
+					device_id, pvfdev);
+	}
+
+	return vfs_found;
+}
+
+static int igb_check_vf_assignment(struct igb_adapter *adapter)
+{
+	int i;
+	for (i = 0; i < adapter->vfs_allocated_count; i++) {
+		if (adapter->vf_data[i].vfdev) {
+			if (adapter->vf_data[i].vfdev->dev_flags &
+			    PCI_DEV_FLAGS_ASSIGNED)
+				return true;
+		}
+	}
+	return false;
+}
+
+#endif
 static void igb_ping_all_vfs(struct igb_adapter *adapter)
 {
 	struct e1000_hw *hw = &adapter->hw;
-- 
1.7.6.4

[net-next 3/6] ixgbe: fix endianess when writing driver version to firmware

[Jeff Kirsher]

From: Emil Tantilov <emil.s.tantilov@intel.com>

This patch makes sure that register writes are in little endian and
also converts the reads back to big-endian.

Signed-off-by: Emil Tantilov <emil.s.tantilov@intel.com>
Tested-by: Stephen Ko <stephen.s.ko@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_common.c |   16 ++++++++--------
 1 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
index 35fa444..834f044 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_common.c
@@ -3341,7 +3341,7 @@ static u8 ixgbe_calculate_checksum(u8 *buffer, u32 length)
  *  Communicates with the manageability block.  On success return 0
  *  else return IXGBE_ERR_HOST_INTERFACE_COMMAND.
  **/
-static s32 ixgbe_host_interface_command(struct ixgbe_hw *hw, u8 *buffer,
+static s32 ixgbe_host_interface_command(struct ixgbe_hw *hw, u32 *buffer,
 					u32 length)
 {
 	u32 hicr, i;
@@ -3374,7 +3374,7 @@ static s32 ixgbe_host_interface_command(struct ixgbe_hw *hw, u8 *buffer,
 	 */
 	for (i = 0; i < dword_len; i++)
 		IXGBE_WRITE_REG_ARRAY(hw, IXGBE_FLEX_MNG,
-				      i, *((u32 *)buffer + i));
+				      i, cpu_to_le32(buffer[i]));
 
 	/* Setting this bit tells the ARC that a new command is pending. */
 	IXGBE_WRITE_REG(hw, IXGBE_HICR, hicr | IXGBE_HICR_C);
@@ -3398,9 +3398,10 @@ static s32 ixgbe_host_interface_command(struct ixgbe_hw *hw, u8 *buffer,
 	dword_len = hdr_size >> 2;
 
 	/* first pull in the header so we know the buffer length */
-	for (i = 0; i < dword_len; i++)
-		*((u32 *)buffer + i) =
-			IXGBE_READ_REG_ARRAY(hw, IXGBE_FLEX_MNG, i);
+	for (i = 0; i < dword_len; i++) {
+		buffer[i] = IXGBE_READ_REG_ARRAY(hw, IXGBE_FLEX_MNG, i);
+		le32_to_cpus(&buffer[i]);
+	}
 
 	/* If there is any thing in data position pull it in */
 	buf_len = ((struct ixgbe_hic_hdr *)buffer)->buf_len;
@@ -3418,8 +3419,7 @@ static s32 ixgbe_host_interface_command(struct ixgbe_hw *hw, u8 *buffer,
 
 	/* Pull in the rest of the buffer (i is where we left off)*/
 	for (; i < buf_len; i++)
-		*((u32 *)buffer + i) =
-			IXGBE_READ_REG_ARRAY(hw, IXGBE_FLEX_MNG, i);
+		buffer[i] = IXGBE_READ_REG_ARRAY(hw, IXGBE_FLEX_MNG, i);
 
 out:
 	return ret_val;
@@ -3465,7 +3465,7 @@ s32 ixgbe_set_fw_drv_ver_generic(struct ixgbe_hw *hw, u8 maj, u8 min,
 	fw_cmd.pad2 = 0;
 
 	for (i = 0; i <= FW_CEM_MAX_RETRIES; i++) {
-		ret_val = ixgbe_host_interface_command(hw, (u8 *)&fw_cmd,
+		ret_val = ixgbe_host_interface_command(hw, (u32 *)&fw_cmd,
 						       sizeof(fw_cmd));
 		if (ret_val != 0)
 			continue;
-- 
1.7.6.4

[net-next 4/6] ixgbe: allow eeprom writes via ethtool

[Jeff Kirsher]

From: Emil Tantilov <emil.s.tantilov@intel.com>

Implement support for ethtool -E

Signed-off-by: Emil Tantilov <emil.s.tantilov@intel.com>
Tested-by: Phil Schmitt <phillip.j.schmitt@intel.com>
Tested-by: Stephen Ko <stephen.s.ko@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c   |    2 +
 drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c |   71 ++++++++++++++++++++++
 2 files changed, 73 insertions(+), 0 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c
index e02e911..ef2afef 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_82598.c
@@ -1305,6 +1305,8 @@ static struct ixgbe_mac_operations mac_ops_82598 = {
 static struct ixgbe_eeprom_operations eeprom_ops_82598 = {
 	.init_params		= &ixgbe_init_eeprom_params_generic,
 	.read			= &ixgbe_read_eerd_generic,
+	.write			= &ixgbe_write_eeprom_generic,
+	.write_buffer		= &ixgbe_write_eeprom_buffer_bit_bang_generic,
 	.read_buffer		= &ixgbe_read_eerd_buffer_generic,
 	.calc_checksum          = &ixgbe_calc_eeprom_checksum_generic,
 	.validate_checksum	= &ixgbe_validate_eeprom_checksum_generic,
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
index e102ff6..7acfce3 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
@@ -814,6 +814,76 @@ static int ixgbe_get_eeprom(struct net_device *netdev,
 	return ret_val;
 }
 
+static int ixgbe_set_eeprom(struct net_device *netdev,
+			    struct ethtool_eeprom *eeprom, u8 *bytes)
+{
+	struct ixgbe_adapter *adapter = netdev_priv(netdev);
+	struct ixgbe_hw *hw = &adapter->hw;
+	u16 *eeprom_buff;
+	void *ptr;
+	int max_len, first_word, last_word, ret_val = 0;
+	u16 i;
+
+	if (eeprom->len == 0)
+		return -EINVAL;
+
+	if (eeprom->magic != (hw->vendor_id | (hw->device_id << 16)))
+		return -EINVAL;
+
+	max_len = hw->eeprom.word_size * 2;
+
+	first_word = eeprom->offset >> 1;
+	last_word = (eeprom->offset + eeprom->len - 1) >> 1;
+	eeprom_buff = kmalloc(max_len, GFP_KERNEL);
+	if (!eeprom_buff)
+		return -ENOMEM;
+
+	ptr = eeprom_buff;
+
+	if (eeprom->offset & 1) {
+		/*
+		 * need read/modify/write of first changed EEPROM word
+		 * only the second byte of the word is being modified
+		 */
+		ret_val = hw->eeprom.ops.read(hw, first_word, &eeprom_buff[0]);
+		if (ret_val)
+			goto err;
+
+		ptr++;
+	}
+	if ((eeprom->offset + eeprom->len) & 1) {
+		/*
+		 * need read/modify/write of last changed EEPROM word
+		 * only the first byte of the word is being modified
+		 */
+		ret_val = hw->eeprom.ops.read(hw, last_word,
+					  &eeprom_buff[last_word - first_word]);
+		if (ret_val)
+			goto err;
+	}
+
+	/* Device's eeprom is always little-endian, word addressable */
+	for (i = 0; i < last_word - first_word + 1; i++)
+		le16_to_cpus(&eeprom_buff[i]);
+
+	memcpy(ptr, bytes, eeprom->len);
+
+	for (i = 0; i < last_word - first_word + 1; i++)
+		cpu_to_le16s(&eeprom_buff[i]);
+
+	ret_val = hw->eeprom.ops.write_buffer(hw, first_word,
+					      last_word - first_word + 1,
+					      eeprom_buff);
+
+	/* Update the checksum */
+	if (ret_val == 0)
+		hw->eeprom.ops.update_checksum(hw);
+
+err:
+	kfree(eeprom_buff);
+	return ret_val;
+}
+
 static void ixgbe_get_drvinfo(struct net_device *netdev,
                               struct ethtool_drvinfo *drvinfo)
 {
@@ -2524,6 +2594,7 @@ static const struct ethtool_ops ixgbe_ethtool_ops = {
 	.get_link               = ethtool_op_get_link,
 	.get_eeprom_len         = ixgbe_get_eeprom_len,
 	.get_eeprom             = ixgbe_get_eeprom,
+	.set_eeprom             = ixgbe_set_eeprom,
 	.get_ringparam          = ixgbe_get_ringparam,
 	.set_ringparam          = ixgbe_set_ringparam,
 	.get_pauseparam         = ixgbe_get_pauseparam,
-- 
1.7.6.4

[net-next 5/6] ixgbe: add hardware timestamping support

[Jeff Kirsher]

From: Jacob Keller <jacob.e.keller@intel.com>

Enable hardware offloading of timestamps via the SO_TIMESTAMPING
functionality. Adds support for enabling the hardware cycle counter,
and enabling the SO_TIMESTAMPING method for extracting the hardware
timestamps through the skb.

In the initial version, the cyclecounter structure was not properly
converting the cycles into nanoseconds, due to an incorrectly
calculated incvalue. v3 fixed this issue by recalculating the math and
giving a proper right-shift value to the cycle counter
structure. However, the DMA clock which is used to generate the
cyclecounter register changes frequency based on the link speed. This
version recalculates the incvalue based on the link speed every time
the device changes link speed.

The cyclecounter has the potential to miss a wrap-around of the
systim register (this should occur no more often than every 35
seconds) unless some activity regarding the cycle counter occurs at
least once within this time. This version adds a cycle counter read
every time the watchdog task is run, which should occur at least once
within this timeframe. Any packets being timestamped will also count
as a read due to the call to timecompare_update.

This version fixes an issue regarding timecompare not updating
detected skew after the clock offset is changed due to ptpd or outside
influence from the OS. Now the skew detection is forced just before we
hand a timestamp up to the kernel stack

Signed-off-by: Jacob E Keller <jacob.e.keller@intel.com>
Tested-by: Stephen Ko <stephen.s.ko@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe.h      |   21 ++
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c |  445 ++++++++++++++++++++++++-
 drivers/net/ethernet/intel/ixgbe/ixgbe_type.h |   32 ++
 3 files changed, 497 insertions(+), 1 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe.h b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
index 6c4d693..9e6635e 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
@@ -36,6 +36,10 @@
 #include <linux/aer.h>
 #include <linux/if_vlan.h>
 
+#include <linux/clocksource.h>
+#include <linux/timecompare.h>
+#include <linux/net_tstamp.h>
+
 #include "ixgbe_type.h"
 #include "ixgbe_common.h"
 #include "ixgbe_dcb.h"
@@ -103,6 +107,7 @@
 #define IXGBE_TX_FLAGS_FSO		(u32)(1 << 6)
 #define IXGBE_TX_FLAGS_TXSW		(u32)(1 << 7)
 #define IXGBE_TX_FLAGS_MAPPED_AS_PAGE	(u32)(1 << 8)
+#define IXGBE_TX_FLAGS_TSTAMP		(u32)(1 << 9)
 #define IXGBE_TX_FLAGS_VLAN_MASK	0xffff0000
 #define IXGBE_TX_FLAGS_VLAN_PRIO_MASK	0xe0000000
 #define IXGBE_TX_FLAGS_VLAN_PRIO_SHIFT  29
@@ -319,6 +324,16 @@ struct ixgbe_q_vector {
 	char name[IFNAMSIZ + 9];
 };
 
+#define IXGBE_INCVAL_BASE_10GB   32
+#define IXGBE_INCVAL_BASE_1GB    320
+#define IXGBE_INCVAL_BASE_100    3200
+#define IXGBE_INCVAL_SHIFT_10GB  28
+#define IXGBE_INCVAL_SHIFT_1GB   24
+#define IXGBE_INCVAL_SHIFT_100   21
+#define IXGBE_INCVAL_DIVISOR     5
+#define IXGBE_INCVAL_SHIFT_82599 7
+#define IXGBE_INCPER_SHIFT_82599 24
+
 /*
  * microsecond values for various ITR rates shifted by 2 to fit itr register
  * with the first 3 bits reserved 0
@@ -409,6 +424,7 @@ struct ixgbe_adapter {
 #define IXGBE_FLAG2_SFP_NEEDS_RESET             (u32)(1 << 5)
 #define IXGBE_FLAG2_RESET_REQUESTED             (u32)(1 << 6)
 #define IXGBE_FLAG2_FDIR_REQUIRES_REINIT        (u32)(1 << 7)
+#define IXGBE_FLAG2_CYCLECOUNTER_RUNNING        (u32)(1 << 8)
 
 	unsigned long active_vlans[BITS_TO_LONGS(VLAN_N_VID)];
 	u16 bd_number;
@@ -464,6 +480,11 @@ struct ixgbe_adapter {
 	struct net_device *netdev;
 	struct pci_dev *pdev;
 
+	struct cyclecounter cycles;
+	struct timecounter clock;
+	struct timecompare compare;
+	struct hwtstamp_config hwtstamp_config;
+
 	u32 test_icr;
 	struct ixgbe_ring test_tx_ring;
 	struct ixgbe_ring test_rx_ring;
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index fb7d884..c92c3a7 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -135,6 +135,220 @@ MODULE_VERSION(DRV_VERSION);
 
 #define DEFAULT_DEBUG_LEVEL_SHIFT 3
 
+static cycle_t ixgbe_read_clock(const struct cyclecounter *tc)
+{
+	struct ixgbe_adapter *adapter =
+		container_of(tc, struct ixgbe_adapter, cycles);
+	struct ixgbe_hw *hw = &adapter->hw;
+	u64 stamp = 0;
+
+
+	stamp |= (u64)IXGBE_READ_REG(hw, IXGBE_SYSTIML);
+	stamp |= (u64)IXGBE_READ_REG(hw, IXGBE_SYSTIMH) << 32;
+
+	return stamp;
+}
+
+/**
+ * ixgbe_init_cyclecounter - initialize the systim cyclecounter
+ * @adapter: pointer to private adapter structure
+ *
+ * We need to call this function every time that the link goes up
+ * because the frequency of the DMA clock changes when the link
+ * speed changes.
+ */
+static void ixgbe_init_cyclecounter(struct ixgbe_adapter *adapter)
+{
+	struct ixgbe_hw *hw = &adapter->hw;
+	u64 incval = 0;
+
+	/*
+	 * (re)initialize hardware cycle counter
+	 * we keep this running just in case someone wants it
+	 * enabled in the future. However, we disable the flag
+	 * just in case so that no one tries to read during
+	 * the process of initializing.
+	 */
+	adapter->flags2 &= ~IXGBE_FLAG2_CYCLECOUNTER_RUNNING;
+
+	/*
+	 * 82598EB hardware does not support timestamping,
+	 * so we return after making sure the disable
+	 * the cyclecounter.
+	 */
+	if (hw->mac.type == ixgbe_mac_82598EB)
+		return;
+
+
+	memset(&adapter->cycles, 0, sizeof(adapter->cycles));
+	adapter->cycles.read = ixgbe_read_clock;
+	adapter->cycles.mask = CLOCKSOURCE_MASK(64);
+	adapter->cycles.mult = 1;
+
+	/**
+	 * Scale the NIC cycle counter by a large factor so that
+	 * relatively small corrections to the frequency can be added
+	 * or subtracted. The drawbacks of a large factor include
+	 * (a) the clock register overflows more quickly, (b) the cycle
+	 * counter structure must be able to convert the systim value
+	 * to nanoseconds using only a multiplier and a right-shift,
+	 * and (c) the value must fit within the timinca register space
+	 * => math based on internal DMA clock rate and available bits
+	 */
+	switch (adapter->link_speed) {
+	case IXGBE_LINK_SPEED_100_FULL:
+		incval = IXGBE_INCVAL_BASE_100;
+		incval <<= IXGBE_INCVAL_SHIFT_100;
+		incval /= IXGBE_INCVAL_DIVISOR;
+
+		adapter->cycles.shift = IXGBE_INCVAL_SHIFT_100;
+		break;
+	case IXGBE_LINK_SPEED_1GB_FULL:
+		incval = IXGBE_INCVAL_BASE_1GB;
+		incval <<= IXGBE_INCVAL_SHIFT_1GB;
+		incval /= IXGBE_INCVAL_DIVISOR;
+
+		adapter->cycles.shift = IXGBE_INCVAL_SHIFT_1GB;
+		break;
+	case IXGBE_LINK_SPEED_10GB_FULL:
+	default:
+		incval = IXGBE_INCVAL_BASE_10GB;
+		incval <<= IXGBE_INCVAL_SHIFT_10GB;
+		incval /= IXGBE_INCVAL_DIVISOR;
+
+		adapter->cycles.shift = IXGBE_INCVAL_SHIFT_10GB;
+		break;
+	}
+
+	switch (hw->mac.type) {
+	case ixgbe_mac_X540:
+		IXGBE_WRITE_REG(hw, IXGBE_TIMINCA, (u32)incval);
+		break;
+	case ixgbe_mac_82599EB:
+		incval >>= IXGBE_INCVAL_SHIFT_82599;
+		adapter->cycles.shift -= IXGBE_INCVAL_SHIFT_82599;
+		IXGBE_WRITE_REG(hw, IXGBE_TIMINCA,
+			    (1 << IXGBE_INCPER_SHIFT_82599) |
+				(u32)incval);
+		break;
+	default:
+		/* other devices aren't supported */
+		return;
+	}
+
+	/* reset the system time registers */
+	IXGBE_WRITE_REG(hw, IXGBE_SYSTIML, 0x00000000);
+	IXGBE_WRITE_REG(hw, IXGBE_SYSTIMH, 0x00000000);
+	IXGBE_WRITE_FLUSH(hw);
+
+	/* Enable the watchdog task to decect cycle counter overflow */
+	adapter->flags2 |= IXGBE_FLAG2_CYCLECOUNTER_RUNNING;
+
+	/* reset the ns time counter */
+	timecounter_init(&adapter->clock,
+			 &adapter->cycles,
+			 ktime_to_ns(ktime_get_real()));
+}
+
+/**
+ * ixgbe_systim_to_hwtstamp - convert system time value to hw timestamp
+ * @adapter: board private structure
+ * @shhwtstamps: timestamp structure to update
+ * @regval: unsigned 64bit system time value.
+ *
+ * We need to convert the system time value stored in the RX/TXSTMP registers
+ * into a hwtstamp which can be used by the upper level timestamping functions
+ */
+static void ixgbe_systim_to_hwtstamp(struct ixgbe_adapter *adapter,
+				     struct skb_shared_hwtstamps *shhwtstamps,
+				     u64 regval)
+{
+	u64 ns;
+
+	/* timestamps aren't valid if the cyclecounter isn't initialized */
+	if (!(adapter->flags2 & IXGBE_FLAG2_CYCLECOUNTER_RUNNING))
+		return;
+
+	ns = timecounter_cyc2time(&adapter->clock, regval);
+
+	/*
+	 * force the timecompare structure to detect skew here in
+	 * order to prevent the case where the wall clock has been
+	 * adjusted by a factor which makes the previous skew
+	 * invalid. this prevents us from giving invalid timestamps to
+	 * the kernel stack.
+	 */
+	timecompare_update(&adapter->compare, 0);
+	memset(shhwtstamps, 0, sizeof(struct skb_shared_hwtstamps));
+	shhwtstamps->hwtstamp = ns_to_ktime(ns);
+	shhwtstamps->syststamp = timecompare_transform(&adapter->compare, ns);
+}
+
+/**
+ * ixgbe_tx_hwtstamp - utility function which checks for TX time stamp
+ * @q_vector: pointer to q_vector containing needed info
+ * @buffer: pointer to ixgbe_tx_buffer structure
+ *
+ * If we were asked to do hardware stamping and such a time stamp is
+ * available, then it must have been for this skb here because we only
+ * allow only one such packet into the queue.
+ */
+static void ixgbe_tx_hwtstamp(struct ixgbe_q_vector *q_vector,
+			      struct ixgbe_tx_buffer *buffer_info)
+{
+	struct ixgbe_adapter *adapter = q_vector->adapter;
+	struct ixgbe_hw *hw = &adapter->hw;
+	struct skb_shared_hwtstamps shhwtstamps;
+	u64 regval = 0;
+
+	/* if skb does not support hw timestamp or TX stamp not valid exit */
+	if (likely(!(buffer_info->tx_flags & IXGBE_TX_FLAGS_TSTAMP)) ||
+	    !(IXGBE_READ_REG(hw, IXGBE_TSYNCTXCTL) & IXGBE_TSYNCTXCTL_VALID))
+		return;
+
+	regval |= (u64)IXGBE_READ_REG(hw, IXGBE_TXSTMPL);
+	regval |= (u64)IXGBE_READ_REG(hw, IXGBE_TXSTMPH) << 32;
+
+	ixgbe_systim_to_hwtstamp(adapter, &shhwtstamps, regval);
+	skb_tstamp_tx(buffer_info->skb, &shhwtstamps);
+}
+
+/**
+ * ixgbe_rx_hwtstamp - utility function which checks for RX time stamp
+ * @q_vector: pointer to q_vector containing needed info
+ * @skb: pointer to the skb
+ *
+ * If we were asked to do hardware stamping and such a time stamp is
+ * available, then it must have been for this skb here because we only
+ * allow only one such packet into the queue.
+ */
+static void ixgbe_rx_hwtstamp(struct ixgbe_q_vector *q_vector,
+			      struct sk_buff *skb)
+{
+	struct ixgbe_adapter *adapter = q_vector->adapter;
+	struct ixgbe_hw *hw = &adapter->hw;
+	u64 regval = 0;
+
+	/*
+	 * If this bit is set, then the RX registers contain the time stamp. No
+	 * other packet will be time stamped until we read these registers, so
+	 * read the registers to make them available again. Because only one
+	 * packet can be time stamped at a time, we know that the register
+	 * values must belong to this one here and therefore we don't need to
+	 * compare any of the additional attributes stored for it.
+	 *
+	 * If nothing went wrong, then it should have a skb_shared_tx that we
+	 * can turn into a skb_shared_hwtstamps.
+	 */
+	if (!(IXGBE_READ_REG(hw, IXGBE_TSYNCRXCTL) & IXGBE_TSYNCRXCTL_VALID))
+		return;
+
+	regval |= (u64)IXGBE_READ_REG(hw, IXGBE_RXSTMPL);
+	regval |= (u64)IXGBE_READ_REG(hw, IXGBE_RXSTMPH) << 32;
+
+	ixgbe_systim_to_hwtstamp(adapter, skb_hwtstamps(skb), regval);
+}
+
 static void ixgbe_service_event_schedule(struct ixgbe_adapter *adapter)
 {
 	if (!test_bit(__IXGBE_DOWN, &adapter->state) &&
@@ -773,6 +987,9 @@ static bool ixgbe_clean_tx_irq(struct ixgbe_q_vector *q_vector,
 			tx_desc->wb.status = 0;
 			if (likely(tx_desc == eop_desc)) {
 				eop_desc = NULL;
+
+				ixgbe_tx_hwtstamp(q_vector, tx_buffer);
+
 				dev_kfree_skb_any(tx_buffer->skb);
 				tx_buffer->skb = NULL;
 
@@ -1399,6 +1616,9 @@ static bool ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector,
 		if (adapter->netdev->features & NETIF_F_RXHASH)
 			ixgbe_rx_hash(rx_desc, skb);
 
+		if (unlikely(staterr & IXGBE_RXDADV_STAT_TS))
+			ixgbe_rx_hwtstamp(q_vector, skb);
+
 		/* probably a little skewed due to removing CRC */
 		total_rx_bytes += skb->len;
 		total_rx_packets++;
@@ -5841,6 +6061,24 @@ static void ixgbe_check_hang_subtask(struct ixgbe_adapter *adapter)
 }
 
 /**
+ * ixgbe_check_cycle_overflow_subtask - read the clock every
+ * few seconds to prevent the timecounter from losing cycles
+ * due to cycle counter overflow.
+ * @adapter - pointer to the device adapter structure
+ */
+static void ixgbe_check_cycle_overflow_subtask(struct ixgbe_adapter *adapter)
+{
+	u64 ns;
+
+	if (!(adapter->flags2 & IXGBE_FLAG2_CYCLECOUNTER_RUNNING))
+		return;
+
+	/* read the cycle counter and update the clock skew. */
+	ns = timecounter_read(&adapter->clock);
+	timecompare_update(&adapter->compare, ns);
+}
+
+/**
  * ixgbe_watchdog_update_link - update the link status
  * @adapter - pointer to the device adapter structure
  * @link_speed - pointer to a u32 to store the link_speed
@@ -5922,6 +6160,9 @@ static void ixgbe_watchdog_link_is_up(struct ixgbe_adapter *adapter)
 		flow_rx = false;
 		break;
 	}
+
+	ixgbe_init_cyclecounter(adapter);
+
 	e_info(drv, "NIC Link is Up %s, Flow Control: %s\n",
 	       (link_speed == IXGBE_LINK_SPEED_10GB_FULL ?
 	       "10 Gbps" :
@@ -6268,6 +6509,7 @@ static void ixgbe_service_task(struct work_struct *work)
 	ixgbe_watchdog_subtask(adapter);
 	ixgbe_fdir_reinit_subtask(adapter);
 	ixgbe_check_hang_subtask(adapter);
+	ixgbe_check_cycle_overflow_subtask(adapter);
 
 	ixgbe_service_event_complete(adapter);
 }
@@ -6425,6 +6667,9 @@ static __le32 ixgbe_tx_cmd_type(u32 tx_flags)
 	if (tx_flags & IXGBE_TX_FLAGS_HW_VLAN)
 		cmd_type |= cpu_to_le32(IXGBE_ADVTXD_DCMD_VLE);
 
+	if (tx_flags & IXGBE_TX_FLAGS_TSTAMP)
+		cmd_type |= cpu_to_le32(IXGBE_ADVTXD_MAC_TSTAMP);
+
 	/* set segmentation enable bits for TSO/FSO */
 #ifdef IXGBE_FCOE
 	if ((tx_flags & IXGBE_TX_FLAGS_TSO) || (tx_flags & IXGBE_TX_FLAGS_FSO))
@@ -6800,6 +7045,11 @@ netdev_tx_t ixgbe_xmit_frame_ring(struct sk_buff *skb,
 		return NETDEV_TX_BUSY;
 	}
 
+	if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)) {
+		skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
+		tx_flags |= IXGBE_TX_FLAGS_TSTAMP;
+	}
+
 #ifdef CONFIG_PCI_IOV
 	if (adapter->flags & IXGBE_FLAG_SRIOV_ENABLED)
 		tx_flags |= IXGBE_TX_FLAGS_TXSW;
@@ -6952,11 +7202,187 @@ static int ixgbe_mdio_write(struct net_device *netdev, int prtad, int devad,
 	return hw->phy.ops.write_reg(hw, addr, devad, value);
 }
 
+/**
+ * ixgbe_hwtstamp_ioctl - control hardware time stamping
+ * @netdev:
+ * @ifreq:
+ * @cmd:
+ *
+ * Outgoing time stamping can be enabled and disabled. Play nice and
+ * disable it when requested, although it shouldn't case any overhead
+ * when no packet needs it. At most one packet in the queue may be
+ * marked for time stamping, otherwise it would be impossible to tell
+ * for sure to which packet the hardware time stamp belongs.
+ *
+ * Incoming time stamping has to be configured via the hardware
+ * filters. Not all combinations are supported, in particular event
+ * type has to be specified. Matching the kind of event packet is
+ * not supported, with the exception of "all V2 events regardless of
+ * level 2 or 4".
+ *
+ **/
+static int ixgbe_hwtstamp_ioctl(struct net_device *netdev,
+				struct ifreq *ifr, int cmd)
+{
+	struct ixgbe_adapter *adapter = netdev_priv(netdev);
+	struct ixgbe_hw *hw = &adapter->hw;
+	struct hwtstamp_config config;
+	u32 tsync_tx_ctl = IXGBE_TSYNCTXCTL_ENABLED;
+	u32 tsync_rx_ctl = IXGBE_TSYNCRXCTL_ENABLED;
+	u32 tsync_rx_mtrl = 0;
+	bool is_l4 = false;
+	bool is_l2 = false;
+	u32 regval;
+
+	if (copy_from_user(&config, ifr->ifr_data, sizeof(config)))
+		return -EFAULT;
+
+	/* reserved for future extensions */
+	if (config.flags)
+		return -EINVAL;
+
+	switch (config.tx_type) {
+	case HWTSTAMP_TX_OFF:
+		tsync_tx_ctl = 0;
+	case HWTSTAMP_TX_ON:
+		break;
+	default:
+		return -ERANGE;
+	}
+
+	switch (config.rx_filter) {
+	case HWTSTAMP_FILTER_NONE:
+		tsync_rx_ctl = 0;
+		break;
+	case HWTSTAMP_FILTER_PTP_V1_L4_SYNC:
+		tsync_rx_ctl |= IXGBE_TSYNCRXCTL_TYPE_L4_V1;
+		tsync_rx_mtrl = IXGBE_RXMTRL_V1_SYNC_MSG;
+		is_l4 = true;
+		break;
+	case HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ:
+		tsync_rx_ctl |= IXGBE_TSYNCRXCTL_TYPE_L4_V1;
+		tsync_rx_mtrl = IXGBE_RXMTRL_V1_DELAY_REQ_MSG;
+		is_l4 = true;
+		break;
+	case HWTSTAMP_FILTER_PTP_V2_SYNC:
+	case HWTSTAMP_FILTER_PTP_V2_L2_SYNC:
+	case HWTSTAMP_FILTER_PTP_V2_L4_SYNC:
+		tsync_rx_ctl |= IXGBE_TSYNCRXCTL_TYPE_L2_L4_V2;
+		tsync_rx_mtrl = IXGBE_RXMTRL_V2_SYNC_MSG;
+		is_l2 = true;
+		is_l4 = true;
+		config.rx_filter = HWTSTAMP_FILTER_SOME;
+		break;
+	case HWTSTAMP_FILTER_PTP_V2_DELAY_REQ:
+	case HWTSTAMP_FILTER_PTP_V2_L2_DELAY_REQ:
+	case HWTSTAMP_FILTER_PTP_V2_L4_DELAY_REQ:
+		tsync_rx_ctl |= IXGBE_TSYNCRXCTL_TYPE_L2_L4_V2;
+		tsync_rx_mtrl = IXGBE_RXMTRL_V2_DELAY_REQ_MSG;
+		is_l2 = true;
+		is_l4 = true;
+		config.rx_filter = HWTSTAMP_FILTER_SOME;
+		break;
+	case HWTSTAMP_FILTER_PTP_V2_L4_EVENT:
+	case HWTSTAMP_FILTER_PTP_V2_L2_EVENT:
+	case HWTSTAMP_FILTER_PTP_V2_EVENT:
+		tsync_rx_ctl |= IXGBE_TSYNCRXCTL_TYPE_EVENT_V2;
+		config.rx_filter = HWTSTAMP_FILTER_PTP_V2_EVENT;
+		is_l2 = true;
+		is_l4 = true;
+		break;
+	case HWTSTAMP_FILTER_PTP_V1_L4_EVENT:
+	case HWTSTAMP_FILTER_ALL:
+	default:
+		/*
+		 * register RXMTRL must be set, therefore it is not
+		 * possible to time stamp both V1 Sync and Delay_Req messages
+		 * and hardware does not support timestamping all packets
+		 * => return error
+		 */
+		return -ERANGE;
+	}
+
+	if (hw->mac.type == ixgbe_mac_82598EB) {
+		if (tsync_rx_ctl | tsync_tx_ctl)
+			return -ERANGE;
+		return 0;
+	}
+
+	/* define ethertype filter for timestamped packets */
+	if (is_l2)
+		IXGBE_WRITE_REG(hw, IXGBE_ETQF(3),
+				(IXGBE_ETQF_FILTER_EN | /* enable filter */
+				 IXGBE_ETQF_1588 | /* enable timestamping */
+				 ETH_P_1588));     /* 1588 eth protocol type */
+	else
+		IXGBE_WRITE_REG(hw, IXGBE_ETQF(3), 0);
+
+#define PTP_PORT 319
+	/* L4 Queue Filter[3]: filter by destination port and protocol */
+	if (is_l4) {
+		u32 ftqf = (IXGBE_FTQF_PROTOCOL_UDP /* UDP */
+			    | IXGBE_FTQF_POOL_MASK_EN /* Pool not compared */
+			    | IXGBE_FTQF_QUEUE_ENABLE);
+
+		ftqf |= ((IXGBE_FTQF_PROTOCOL_COMP_MASK /* protocol check */
+			  & IXGBE_FTQF_DEST_PORT_MASK /* dest check */
+			  & IXGBE_FTQF_SOURCE_PORT_MASK) /* source check */
+			 << IXGBE_FTQF_5TUPLE_MASK_SHIFT);
+
+		IXGBE_WRITE_REG(hw, IXGBE_L34T_IMIR(3),
+				(3 << IXGBE_IMIR_RX_QUEUE_SHIFT_82599 |
+				 IXGBE_IMIR_SIZE_BP_82599));
+
+		/* enable port check */
+		IXGBE_WRITE_REG(hw, IXGBE_SDPQF(3),
+				(htons(PTP_PORT) |
+				 htons(PTP_PORT) << 16));
+
+		IXGBE_WRITE_REG(hw, IXGBE_FTQF(3), ftqf);
+
+		tsync_rx_mtrl |= PTP_PORT << 16;
+	} else {
+		IXGBE_WRITE_REG(hw, IXGBE_FTQF(3), 0);
+	}
+
+	/* enable/disable TX */
+	regval = IXGBE_READ_REG(hw, IXGBE_TSYNCTXCTL);
+	regval &= ~IXGBE_TSYNCTXCTL_ENABLED;
+	regval |= tsync_tx_ctl;
+	IXGBE_WRITE_REG(hw, IXGBE_TSYNCTXCTL, regval);
+
+	/* enable/disable RX */
+	regval = IXGBE_READ_REG(hw, IXGBE_TSYNCRXCTL);
+	regval &= ~(IXGBE_TSYNCRXCTL_ENABLED | IXGBE_TSYNCRXCTL_TYPE_MASK);
+	regval |= tsync_rx_ctl;
+	IXGBE_WRITE_REG(hw, IXGBE_TSYNCRXCTL, regval);
+
+	/* define which PTP packets are time stamped */
+	IXGBE_WRITE_REG(hw, IXGBE_RXMTRL, tsync_rx_mtrl);
+
+	IXGBE_WRITE_FLUSH(hw);
+
+	adapter->hwtstamp_config = config;
+
+	/* clear TX/RX time stamp registers, just to be sure */
+	regval = IXGBE_READ_REG(hw, IXGBE_TXSTMPH);
+	regval = IXGBE_READ_REG(hw, IXGBE_RXSTMPH);
+
+	return copy_to_user(ifr->ifr_data, &config, sizeof(config)) ?
+		-EFAULT : 0;
+}
+
+
 static int ixgbe_ioctl(struct net_device *netdev, struct ifreq *req, int cmd)
 {
 	struct ixgbe_adapter *adapter = netdev_priv(netdev);
 
-	return mdio_mii_ioctl(&adapter->hw.phy.mdio, if_mii(req), cmd);
+	switch (cmd) {
+	case SIOCSHWTSTAMP:
+		return ixgbe_hwtstamp_ioctl(netdev, req, cmd);
+	default:
+		return mdio_mii_ioctl(&adapter->hw.phy.mdio, if_mii(req), cmd);
+	}
 }
 
 /**
@@ -7643,6 +8069,23 @@ static int __devinit ixgbe_probe(struct pci_dev *pdev,
 	/* pick up the PCI bus settings for reporting later */
 	hw->mac.ops.get_bus_info(hw);
 
+	/* initialize the cycle counter */
+	ixgbe_init_cyclecounter(adapter);
+
+	/*
+	 * Provide a timestamp synchronized against the system wall
+	 * clock. NIC time stamp reading requires ~3us per sample,
+	 * each sample was pretty stable even under load
+	 * => only require 10 samples for each offset comparison.
+	 */
+	if (adapter->flags2 & IXGBE_FLAG2_CYCLECOUNTER_RUNNING) {
+		memset(&adapter->compare, 0, sizeof(adapter->compare));
+		adapter->compare.source = &adapter->clock;
+		adapter->compare.target = ktime_get_real;
+		adapter->compare.num_samples = 10;
+		timecompare_update(&adapter->compare, 0);
+	}
+
 	/* print bus type/speed/width info */
 	e_dev_info("(PCI Express:%s:%s) %pM\n",
 		   (hw->bus.speed == ixgbe_bus_speed_5000 ? "5.0GT/s" :
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
index 6c5cca8..8692672 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_type.h
@@ -1837,6 +1837,36 @@ enum {
 #define IXGBE_RXDCTL_RLPML_EN   0x00008000
 #define IXGBE_RXDCTL_VME        0x40000000  /* VLAN mode enable */
 
+#define IXGBE_TSYNCTXCTL_VALID     0x00000001 /* Tx timestamp valid */
+#define IXGBE_TSYNCTXCTL_ENABLED   0x00000010 /* Tx timestamping enabled */
+
+#define IXGBE_TSYNCRXCTL_VALID     0x00000001 /* Rx timestamp valid */
+#define IXGBE_TSYNCRXCTL_TYPE_MASK 0x0000000E /* Rx type mask */
+#define IXGBE_TSYNCRXCTL_TYPE_L2_V2      0x00
+#define IXGBE_TSYNCRXCTL_TYPE_L4_V1      0x02
+#define IXGBE_TSYNCRXCTL_TYPE_L2_L4_V2   0x04
+#define IXGBE_TSYNCRXCTL_TYPE_EVENT_V2   0x0A
+#define IXGBE_TSYNCRXCTL_ENABLED   0x00000010 /* Rx Timestamping enabled */
+
+#define IXGBE_RXMTRL_V1_CTRLT_MASK 0x000000FF
+#define IXGBE_RXMTRL_V1_SYNC_MSG         0x00
+#define IXGBE_RXMTRL_V1_DELAY_REQ_MSG    0x01
+#define IXGBE_RXMTRL_V1_FOLLOWUP_MSG     0x02
+#define IXGBE_RXMTRL_V1_DELAY_RESP_MSG   0x03
+#define IXGBE_RXMTRL_V1_MGMT_MSG         0x04
+
+#define IXGBE_RXMTRL_V2_MSGID_MASK      0x0000FF00
+#define IXGBE_RXMTRL_V2_SYNC_MSG            0x0000
+#define IXGBE_RXMTRL_V2_DELAY_REQ_MSG       0x0100
+#define IXGBE_RXMTRL_V2_PDELAY_REQ_MSG      0x0200
+#define IXGBE_RXMTRL_V2_PDELAY_RESP_MSG     0x0300
+#define IXGBE_RXMTRL_V2_FOLLOWUP_MSG        0x0800
+#define IXGBE_RXMTRL_V2_DELAY_RESP_MSG      0x0900
+#define IXGBE_RXMTRL_V2_PDELAY_FOLLOWUP_MSG 0x0A00
+#define IXGBE_RXMTRL_V2_ANNOUNCE_MSG        0x0B00
+#define IXGBE_RXMTRL_V2_SIGNALLING_MSG      0x0C00
+#define IXGBE_RXMTRL_V2_MGMT_MSG            0x0D00
+
 #define IXGBE_FCTRL_SBP 0x00000002 /* Store Bad Packet */
 #define IXGBE_FCTRL_MPE 0x00000100 /* Multicast Promiscuous Ena*/
 #define IXGBE_FCTRL_UPE 0x00000200 /* Unicast Promiscuous Ena */
@@ -1966,6 +1996,7 @@ enum {
 #define IXGBE_RXDADV_STAT_FCSTAT_NODDP  0x00000010 /* 01: Ctxt w/o DDP */
 #define IXGBE_RXDADV_STAT_FCSTAT_FCPRSP 0x00000020 /* 10: Recv. FCP_RSP */
 #define IXGBE_RXDADV_STAT_FCSTAT_DDP    0x00000030 /* 11: Ctxt w/ DDP */
+#define IXGBE_RXDADV_STAT_TS            0x00010000 /* IEEE1588 Time Stamp */
 
 /* PSRTYPE bit definitions */
 #define IXGBE_PSRTYPE_TCPHDR    0x00000010
@@ -2243,6 +2274,7 @@ struct ixgbe_adv_tx_context_desc {
 /* Adv Transmit Descriptor Config Masks */
 #define IXGBE_ADVTXD_DTALEN_MASK      0x0000FFFF /* Data buf length(bytes) */
 #define IXGBE_ADVTXD_MAC_LINKSEC      0x00040000 /* Insert LinkSec */
+#define IXGBE_ADVTXD_MAC_TSTAMP       0x00080000 /* IEEE1588 time stamp */
 #define IXGBE_ADVTXD_IPSEC_SA_INDEX_MASK   0x000003FF /* IPSec SA index */
 #define IXGBE_ADVTXD_IPSEC_ESP_LEN_MASK    0x000001FF /* IPSec ESP length */
 #define IXGBE_ADVTXD_DTYP_MASK  0x00F00000 /* DTYP mask */
-- 
1.7.6.4

[net-next 6/6] ixgbe: change the eeprom version reported by ethtool

[Jeff Kirsher]

From: Emil Tantilov <emil.s.tantilov@intel.com>

Use 32bit value starting at offset 0x2d for displaying the firmware
version in ethtool. This should work for all current ixgbe HW

Signed-off-by: Emil Tantilov <emil.s.tantilov@intel.com>
Tested-by: Stephen Ko <stephen.s.ko@intel.com>
Signed-off-by: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
---
 drivers/net/ethernet/intel/ixgbe/ixgbe.h         |    3 ++-
 drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c |   13 +++++++------
 drivers/net/ethernet/intel/ixgbe/ixgbe_main.c    |    7 ++++---
 3 files changed, 13 insertions(+), 10 deletions(-)

diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe.h b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
index 9e6635e..4881807 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe.h
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe.h
@@ -518,7 +518,8 @@ struct ixgbe_adapter {
 	u64 rsc_total_count;
 	u64 rsc_total_flush;
 	u32 wol;
-	u16 eeprom_version;
+	u16 eeprom_verh;
+	u16 eeprom_verl;
 	u16 eeprom_cap;
 
 	int node;
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
index 7acfce3..70d58c3 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
@@ -889,21 +889,22 @@ static void ixgbe_get_drvinfo(struct net_device *netdev,
 {
 	struct ixgbe_adapter *adapter = netdev_priv(netdev);
 	char firmware_version[32];
+	u32 nvm_track_id;
 
 	strncpy(drvinfo->driver, ixgbe_driver_name,
 	        sizeof(drvinfo->driver) - 1);
 	strncpy(drvinfo->version, ixgbe_driver_version,
 	        sizeof(drvinfo->version) - 1);
 
-	snprintf(firmware_version, sizeof(firmware_version), "%d.%d-%d",
-	         (adapter->eeprom_version & 0xF000) >> 12,
-	         (adapter->eeprom_version & 0x0FF0) >> 4,
-	         adapter->eeprom_version & 0x000F);
+	nvm_track_id = (adapter->eeprom_verh << 16) |
+			adapter->eeprom_verl;
+	snprintf(firmware_version, sizeof(firmware_version), "0x%08x",
+		 nvm_track_id);
 
 	strncpy(drvinfo->fw_version, firmware_version,
-	        sizeof(drvinfo->fw_version));
+		sizeof(drvinfo->fw_version) - 1);
 	strncpy(drvinfo->bus_info, pci_name(adapter->pdev),
-	        sizeof(drvinfo->bus_info));
+		sizeof(drvinfo->bus_info) - 1);
 	drvinfo->n_stats = IXGBE_STATS_LEN;
 	drvinfo->testinfo_len = IXGBE_TEST_LEN;
 	drvinfo->regdump_len = ixgbe_get_regs_len(netdev);
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index c92c3a7..31f4f53 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -8066,6 +8066,10 @@ static int __devinit ixgbe_probe(struct pci_dev *pdev,
 	}
 	device_set_wakeup_enable(&adapter->pdev->dev, adapter->wol);
 
+	/* save off EEPROM version number */
+	hw->eeprom.ops.read(hw, 0x2e, &adapter->eeprom_verh);
+	hw->eeprom.ops.read(hw, 0x2d, &adapter->eeprom_verl);
+
 	/* pick up the PCI bus settings for reporting later */
 	hw->mac.ops.get_bus_info(hw);
 
@@ -8115,9 +8119,6 @@ static int __devinit ixgbe_probe(struct pci_dev *pdev,
 			   "is required.\n");
 	}
 
-	/* save off EEPROM version number */
-	hw->eeprom.ops.read(hw, 0x29, &adapter->eeprom_version);
-
 	/* reset the hardware with the new settings */
 	err = hw->mac.ops.start_hw(hw);
 
-- 
1.7.6.4

Re: [net-next 2/6 v2] igb: Check if subordinate VFs are assigned to virtual machines

[Joe Perches]

On Mon, 2011-10-17 at 05:20 -0700, Jeff Kirsher wrote:
> From: Greg Rose <gregory.v.rose@intel.com>

Trivia:

> diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
[]
> +	if (old_vfs) {
> +		dev_info(&pdev->dev, "%d pre-allocated VFs found - override "
> +			 "max_vfs setting of %d\n", old_vfs, max_vfs);
[]
> +		dev_err(&pdev->dev, "Unable to allocate memory for VF "
> +			"Data Storage\n");

The dev_err line isn't really necessary.
There is a generic allocation failure dump_stack.
Also, please don't split formats into multiple lines.

Re: [net-next 6/6] ixgbe: change the eeprom version reported by ethtool

[Joe Perches]

On Mon, 2011-10-17 at 05:21 -0700, Jeff Kirsher wrote:
> From: Emil Tantilov <emil.s.tantilov@intel.com>
> 
> Use 32bit value starting at offset 0x2d for displaying the firmware
> version in ethtool. This should work for all current ixgbe HW
[]
> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
[]
> -	snprintf(firmware_version, sizeof(firmware_version), "%d.%d-%d",
> -	         (adapter->eeprom_version & 0xF000) >> 12,
> -	         (adapter->eeprom_version & 0x0FF0) >> 4,
> -	         adapter->eeprom_version & 0x000F);
> +	nvm_track_id = (adapter->eeprom_verh << 16) |
> +			adapter->eeprom_verl;
> +	snprintf(firmware_version, sizeof(firmware_version), "0x%08x",
> +		 nvm_track_id);

Is ethtool output like proc output considered an abi
that should not be changed?

Re: [net-next 5/6] ixgbe: add hardware timestamping support

[Richard Cochran]

On Mon, Oct 17, 2011 at 05:21:01AM -0700, Jeff Kirsher wrote:
> The cyclecounter has the potential to miss a wrap-around of the
> systim register (this should occur no more often than every 35
> seconds) unless some activity regarding the cycle counter occurs at
> least once within this time. This version adds a cycle counter read
> every time the watchdog task is run, which should occur at least once
> within this timeframe. Any packets being timestamped will also count
> as a read due to the call to timecompare_update.

So, is this wrap around due to the fact that you are tied to the
system time via time_compare? Or, putting it another way, can't you
program the hardware time stamping unit so that the registers have
some reasonable resolution (like 64 bits worth of nanoseconds) and
just offer RAW timestamps?

I would really like to move away from the timecompare hacks and
towards a proper PHC->SYS PPS solution.

> This version fixes an issue regarding timecompare not updating
> detected skew after the clock offset is changed due to ptpd or outside
> influence from the OS. Now the skew detection is forced just before we
> hand a timestamp up to the kernel stack

Again, doing the update thing on every packet won't work for real
world PTP scenarios.

Thanks,
Richard

Re: [net-next 6/6] ixgbe: change the eeprom version reported by ethtool

[Ben Hutchings]

On Mon, 2011-10-17 at 08:57 -0700, Joe Perches wrote:
> On Mon, 2011-10-17 at 05:21 -0700, Jeff Kirsher wrote:
> > From: Emil Tantilov <emil.s.tantilov@intel.com>
> > 
> > Use 32bit value starting at offset 0x2d for displaying the firmware
> > version in ethtool. This should work for all current ixgbe HW
> []
> > diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
> []
> > -	snprintf(firmware_version, sizeof(firmware_version), "%d.%d-%d",
> > -	         (adapter->eeprom_version & 0xF000) >> 12,
> > -	         (adapter->eeprom_version & 0x0FF0) >> 4,
> > -	         adapter->eeprom_version & 0x000F);
> > +	nvm_track_id = (adapter->eeprom_verh << 16) |
> > +			adapter->eeprom_verl;
> > +	snprintf(firmware_version, sizeof(firmware_version), "0x%08x",
> > +		 nvm_track_id);
> 
> Is ethtool output like proc output considered an abi
> that should not be changed?

No-one should make any assumptions about the format of firmware_version
strings.  However they ought to be consistent with vendor documentation,
update programs, etc.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

RE: [net-next 6/6] ixgbe: change the eeprom version reported by ethtool

[Tantilov, Emil S]

>-----Original Message-----
>From: Ben Hutchings [mailto:bhutchings@solarflare.com]
>Sent: Monday, October 17, 2011 10:17 AM
>To: Joe Perches
>Cc: Kirsher, Jeffrey T; davem@davemloft.net; Tantilov, Emil S;
>netdev@vger.kernel.org; gospo@redhat.com; sassmann@redhat.com
>Subject: Re: [net-next 6/6] ixgbe: change the eeprom version reported by
>ethtool
>
>On Mon, 2011-10-17 at 08:57 -0700, Joe Perches wrote:
>> On Mon, 2011-10-17 at 05:21 -0700, Jeff Kirsher wrote:
>> > From: Emil Tantilov <emil.s.tantilov@intel.com>
>> >
>> > Use 32bit value starting at offset 0x2d for displaying the firmware
>> > version in ethtool. This should work for all current ixgbe HW
>> []
>> > diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
>b/drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c
>> []
>> > -	snprintf(firmware_version, sizeof(firmware_version), "%d.%d-%d",
>> > -	         (adapter->eeprom_version & 0xF000) >> 12,
>> > -	         (adapter->eeprom_version & 0x0FF0) >> 4,
>> > -	         adapter->eeprom_version & 0x000F);
>> > +	nvm_track_id = (adapter->eeprom_verh << 16) |
>> > +			adapter->eeprom_verl;
>> > +	snprintf(firmware_version, sizeof(firmware_version), "0x%08x",
>> > +		 nvm_track_id);
>>
>> Is ethtool output like proc output considered an abi
>> that should not be changed?
>
>No-one should make any assumptions about the format of firmware_version
>strings.  However they ought to be consistent with vendor documentation,
>update programs, etc.

The old value was only marginally useful as it was possible to have different 
images with the same version. The 32 bit value shown by this patch is what 
is being used by the FW team to record the revisions of the images.

The words used to hold the 32 bit value are not described in the datasheet
for 82598/9, but will be in X540 once it becomes available and I have requested that this information be added to the 82598/9 docs as well.

Thanks,
Emil

>
>Ben.
>
>--
>Ben Hutchings, Staff Engineer, Solarflare
>Not speaking for my employer; that's the marketing department's job.
>They asked us to note that Solarflare product names are trademarked.

Re: [net-next 0/6 v2][pull request] Intel Wired LAN Driver Updates

[David Miller]

From: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
Date: Mon, 17 Oct 2011 05:20:56 -0700

> The following series contains updates to ixgbe, igbvf and igb.
> This version of the series contains the following changes:
> 
> - igb fix/add check if subordinate VFs are assigned to VM's
> - igbvf fix for trunk VLAN
> - ixgbe 2 fixes for ethtool and 1 endianess fix
> 
> -v2 update the igb patch to resolve a variable initialization warning
> 
> The following are changes since commit fd38f734cb8200529e281338514945fcbff2364b:
>   igbvf: convert to ndo_fix_features
> and are available in the git repository at
>   git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/net-next.git

Patch #5 adding timestamp offloading for ixgbe has some legitimate
pushback from Richard Cochran, why don't we defer that patch while that
discussion transpires?

I'm perfectly fine with the other patches in this series.

Please respin the remaining 5 patches into a pull request and I'll
take them into net-next, thanks Jeff!

Re: [net-next 0/6 v2][pull request] Intel Wired LAN Driver Updates

[Jeff Kirsher]

[-- Attachment #1: Type: text/plain, Size: 1201 bytes --]

On Mon, 2011-10-17 at 15:49 -0700, David Miller wrote:
> From: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
> Date: Mon, 17 Oct 2011 05:20:56 -0700
> 
> > The following series contains updates to ixgbe, igbvf and igb.
> > This version of the series contains the following changes:
> > 
> > - igb fix/add check if subordinate VFs are assigned to VM's
> > - igbvf fix for trunk VLAN
> > - ixgbe 2 fixes for ethtool and 1 endianess fix
> > 
> > -v2 update the igb patch to resolve a variable initialization warning
> > 
> > The following are changes since commit fd38f734cb8200529e281338514945fcbff2364b:
> >   igbvf: convert to ndo_fix_features
> > and are available in the git repository at
> >   git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/net-next.git
> 
> Patch #5 adding timestamp offloading for ixgbe has some legitimate
> pushback from Richard Cochran, why don't we defer that patch while that
> discussion transpires?
> 
> I'm perfectly fine with the other patches in this series.
> 
> Please respin the remaining 5 patches into a pull request and I'll
> take them into net-next, thanks Jeff!
> 

Sounds good, I have that ready in an hour or so.

Thanks!

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [net-next 5/6] ixgbe: add hardware timestamping support

[Jacob Keller]

On Mon, Oct 17, 2011 at 9:44 AM, Richard Cochran
<richardcochran@gmail.com> wrote:
> On Mon, Oct 17, 2011 at 05:21:01AM -0700, Jeff Kirsher wrote:
>> The cyclecounter has the potential to miss a wrap-around of the
>> systim register (this should occur no more often than every 35
>> seconds) unless some activity regarding the cycle counter occurs at
>> least once within this time. This version adds a cycle counter read
>> every time the watchdog task is run, which should occur at least once
>> within this timeframe. Any packets being timestamped will also count
>> as a read due to the call to timecompare_update.
>
> So, is this wrap around due to the fact that you are tied to the
> system time via time_compare? Or, putting it another way, can't you
> program the hardware time stamping unit so that the registers have
> some reasonable resolution (like 64 bits worth of nanoseconds) and
> just offer RAW timestamps?

The wrap around is due to hardware limitations. The ixgbe devices
cannot support 64bits worth of nanoseconds and still have the ability
to adjust the frequency in parts per billion. A larger increment
increases the resolution available for frequency adjustments, but
decreases the time it takes for the cycle counter to wrap around.

>
> I would really like to move away from the timecompare hacks and
> towards a proper PHC->SYS PPS solution.
>

I agree that this is the correct approach. The timecompare
functionality does have issues.

>> This version fixes an issue regarding timecompare not updating
>> detected skew after the clock offset is changed due to ptpd or outside
>> influence from the OS. Now the skew detection is forced just before we
>> hand a timestamp up to the kernel stack
>
> Again, doing the update thing on every packet won't work for real
> world PTP scenarios.
>
Which is why the PHC solution is better. Work on implementing this
support is in progress. Out of curiosity, what is the sync rate for
the scenario that breaks this? I would like to try that rate out on my
setup.

> Thanks,
> Richard
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [net-next 5/6] ixgbe: add hardware timestamping support

[Richard Cochran]

On Wed, Oct 19, 2011 at 10:04:33AM -0700, Jacob Keller wrote:
> On Mon, Oct 17, 2011 at 9:44 AM, Richard Cochran <richardcochran@gmail.com> wrote:
> > So, is this wrap around due to the fact that you are tied to the
> > system time via time_compare? Or, putting it another way, can't you
> > program the hardware time stamping unit so that the registers have
> > some reasonable resolution (like 64 bits worth of nanoseconds) and
> > just offer RAW timestamps?
> 
> The wrap around is due to hardware limitations. The ixgbe devices
> cannot support 64bits worth of nanoseconds and still have the ability
> to adjust the frequency in parts per billion. A larger increment
> increases the resolution available for frequency adjustments, but
> decreases the time it takes for the cycle counter to wrap around.

Oh, well. That stinks.

I think you do want to offer ppb adjustment.

> > I would really like to move away from the timecompare hacks and
> > towards a proper PHC->SYS PPS solution.
> >
> 
> I agree that this is the correct approach. The timecompare
> functionality does have issues.

And these cards are highlighting timecompare weaknesses I had not even
thought of.

I expect that if you offer the RAW time stamps, then it should be
possible to have the time stamp values always correct (or nearly so)
even with a changing link speed. If the link speed change gives an
interrupt, then the ISR can reprogram the frequency compensation
registers and let the counter continue.

> > Again, doing the update thing on every packet won't work for real
> > world PTP scenarios.
> >
> Which is why the PHC solution is better. Work on implementing this
> support is in progress. Out of curiosity, what is the sync rate for
> the scenario that breaks this? I would like to try that rate out on my
> setup.

For the audio/video profile, they have a max of 32 sync packets per
second. Not sure about delay request rate, maybe 16 per second.

Thanks,
Richard

RE: [net-next 5/6] ixgbe: add hardware timestamping support

[Keller, Jacob E]

> -----Original Message-----
> From: Richard Cochran [mailto:richardcochran@gmail.com]
> Sent: Thursday, October 20, 2011 7:57 AM
> To: Jacob Keller
> Cc: Kirsher, Jeffrey T; davem@davemloft.net; Keller, Jacob E;
> netdev@vger.kernel.org; gospo@redhat.com; sassmann@redhat.com
> Subject: Re: [net-next 5/6] ixgbe: add hardware timestamping support
> 
> On Wed, Oct 19, 2011 at 10:04:33AM -0700, Jacob Keller wrote:
> > On Mon, Oct 17, 2011 at 9:44 AM, Richard Cochran
> <richardcochran@gmail.com> wrote:
> > > So, is this wrap around due to the fact that you are tied to the
> > > system time via time_compare? Or, putting it another way, can't you
> > > program the hardware time stamping unit so that the registers have
> > > some reasonable resolution (like 64 bits worth of nanoseconds) and
> > > just offer RAW timestamps?
> >
> > The wrap around is due to hardware limitations. The ixgbe devices
> > cannot support 64bits worth of nanoseconds and still have the ability
> > to adjust the frequency in parts per billion. A larger increment
> > increases the resolution available for frequency adjustments, but
> > decreases the time it takes for the cycle counter to wrap around.
> 
> Oh, well. That stinks.
> 
> I think you do want to offer ppb adjustment.
> 
Correct, which is why the cycle counter wraps around every 35 seconds.

> > > I would really like to move away from the timecompare hacks and
> > > towards a proper PHC->SYS PPS solution.
> > >
> >
> > I agree that this is the correct approach. The timecompare
> > functionality does have issues.
> 
> And these cards are highlighting timecompare weaknesses I had not even
> thought of.
> 
> I expect that if you offer the RAW time stamps, then it should be
> possible to have the time stamp values always correct (or nearly so)
> even with a changing link speed. If the link speed change gives an
> interrupt, then the ISR can reprogram the frequency compensation
> registers and let the counter continue.
> 

The cyclecounter is based off of the DMA clock on the NIC which changes frequency with the link speed. So at 10G link, the DMA ticks once every 6.4ns. The cycle counter gets a value (specified in the TIMINCA register) added to it every DMA tick. In order to allow for ppb adjustments to the cycle counter, I have the TIMINCA value be as many bits wide as possible. Then I use the cyclecounter/timecounter structures to detect wraparound and convert to a ns value.

If we return the raw cycle counter stamps directly, they would not be measured in nanoseconds, but in a division of the DMA clock tick (DMA clock tick / TIMINCA value). This means for the values I chose we are somewhere in the range of femto seconds or so. The problem is that all of the upper stack expects values as nanoseconds. We wouldn't be able to frequency adjust in ppb to get down to nanoseconds again.

This is due to a limitation in the way the hardware was designed, (Ideally it would allow for precise adjustments, but still provide a nanosecond counter. The 82580 igb device does this.)

> > > Again, doing the update thing on every packet won't work for real
> > > world PTP scenarios.
> > >
> > Which is why the PHC solution is better. Work on implementing this
> > support is in progress. Out of curiosity, what is the sync rate for
> > the scenario that breaks this? I would like to try that rate out on
> my
> > setup.
> 
> For the audio/video profile, they have a max of 32 sync packets per
> second. Not sure about delay request rate, maybe 16 per second.
> 

Thanks :)

> Thanks,
> Richard