[PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Muhammad Husaini Zulkifli]

The HW TX timestamps created by the NIC via socket options can be
requested using the current network timestamps generation capability of
SOF_TIMESTAMPING_TX_HARDWARE. The most common users of this socket flag
is PTP, however other packet applications that require tx timestamps might
also ask for it.

The problem is that, when there is a lot of traffic, there is a high chance
that the timestamps for a PTP packet will be lost if both PTP and Non-PTP
packets use the same SOF TIMESTAMPING TX HARDWARE causing the tx timeout.

DMA timestamps through socket options are not currently available to
the user. Because if the user wants to, they can configure the hwtstamp
config option to use the new introduced DMA Time Stamp flag through the
setsockopt().

With these additional socket options, users can continue to utilise
HW timestamps for PTP while specifying non-PTP packets to use DMA
timestamps if the NIC can support them.

Any socket application can be use to verify this.
TSN Ref SW application is been used for testing by changing as below:

	int timestamping_flags = SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH;

	strncpy(hwtstamp.ifr_name, opt->ifname, sizeof(hwtstamp.ifr_name)-1);
	hwtstamp.ifr_data = (void *)&hwconfig;
	hwconfig.tx_type = HWTSTAMP_TX_ON;
	hwconfig.flags = HWTSTAMP_FLAG_DMA_TIMESTAMP;
	hwconfig.rx_filter = HWTSTAMP_FILTER_ALL;

	if (ioctl(sock, SIOCSHWTSTAMP, &hwtstamp) < 0) {
		fprintf(stderr, "%s: %s\n", "ioctl", strerror(errno));
		exit(1);
	}

	if (setsockopt(sock, SOL_SOCKET, SO_TIMESTAMPING, &timestamping_flags,
			sizeof(timestamping_flags)) < 0)
		exit_with_error("setsockopt SO_TIMESTAMPING");

Muhammad Husaini Zulkifli (3):
  ethtool: Add new hwtstamp flag
  net-timestamp: Increase the size of tsflags
  net: sock: extend SO_TIMESTAMPING for DMA Fetch

Vinicius Costa Gomes (1):
  igc: Add support for DMA timestamp for non-PTP packets

 drivers/net/ethernet/intel/igc/igc.h         | 10 +++
 drivers/net/ethernet/intel/igc/igc_base.h    |  2 +-
 drivers/net/ethernet/intel/igc/igc_defines.h |  2 +
 drivers/net/ethernet/intel/igc/igc_ethtool.c |  5 +-
 drivers/net/ethernet/intel/igc/igc_main.c    | 24 ++++--
 drivers/net/ethernet/intel/igc/igc_ptp.c     | 83 ++++++++++++++++++++
 include/linux/skbuff.h                       |  3 +
 include/net/sock.h                           | 12 +--
 include/uapi/linux/ethtool.h                 |  3 +
 include/uapi/linux/ethtool_netlink.h         |  1 +
 include/uapi/linux/net_tstamp.h              | 11 ++-
 net/ethtool/common.c                         |  7 ++
 net/ethtool/common.h                         |  2 +
 net/ethtool/strset.c                         |  5 ++
 net/ethtool/tsinfo.c                         | 17 ++++
 net/socket.c                                 |  5 +-
 16 files changed, 175 insertions(+), 17 deletions(-)

--
2.17.1

[Intel-wired-lan] [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Muhammad Husaini Zulkifli]

The HW TX timestamps created by the NIC via socket options can be
requested using the current network timestamps generation capability of
SOF_TIMESTAMPING_TX_HARDWARE. The most common users of this socket flag
is PTP, however other packet applications that require tx timestamps might
also ask for it.

The problem is that, when there is a lot of traffic, there is a high chance
that the timestamps for a PTP packet will be lost if both PTP and Non-PTP
packets use the same SOF TIMESTAMPING TX HARDWARE causing the tx timeout.

DMA timestamps through socket options are not currently available to
the user. Because if the user wants to, they can configure the hwtstamp
config option to use the new introduced DMA Time Stamp flag through the
setsockopt().

With these additional socket options, users can continue to utilise
HW timestamps for PTP while specifying non-PTP packets to use DMA
timestamps if the NIC can support them.

Any socket application can be use to verify this.
TSN Ref SW application is been used for testing by changing as below:

	int timestamping_flags = SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH;

	strncpy(hwtstamp.ifr_name, opt->ifname, sizeof(hwtstamp.ifr_name)-1);
	hwtstamp.ifr_data = (void *)&hwconfig;
	hwconfig.tx_type = HWTSTAMP_TX_ON;
	hwconfig.flags = HWTSTAMP_FLAG_DMA_TIMESTAMP;
	hwconfig.rx_filter = HWTSTAMP_FILTER_ALL;

	if (ioctl(sock, SIOCSHWTSTAMP, &hwtstamp) < 0) {
		fprintf(stderr, "%s: %s\n", "ioctl", strerror(errno));
		exit(1);
	}

	if (setsockopt(sock, SOL_SOCKET, SO_TIMESTAMPING, &timestamping_flags,
			sizeof(timestamping_flags)) < 0)
		exit_with_error("setsockopt SO_TIMESTAMPING");

Muhammad Husaini Zulkifli (3):
  ethtool: Add new hwtstamp flag
  net-timestamp: Increase the size of tsflags
  net: sock: extend SO_TIMESTAMPING for DMA Fetch

Vinicius Costa Gomes (1):
  igc: Add support for DMA timestamp for non-PTP packets

 drivers/net/ethernet/intel/igc/igc.h         | 10 +++
 drivers/net/ethernet/intel/igc/igc_base.h    |  2 +-
 drivers/net/ethernet/intel/igc/igc_defines.h |  2 +
 drivers/net/ethernet/intel/igc/igc_ethtool.c |  5 +-
 drivers/net/ethernet/intel/igc/igc_main.c    | 24 ++++--
 drivers/net/ethernet/intel/igc/igc_ptp.c     | 83 ++++++++++++++++++++
 include/linux/skbuff.h                       |  3 +
 include/net/sock.h                           | 12 +--
 include/uapi/linux/ethtool.h                 |  3 +
 include/uapi/linux/ethtool_netlink.h         |  1 +
 include/uapi/linux/net_tstamp.h              | 11 ++-
 net/ethtool/common.c                         |  7 ++
 net/ethtool/common.h                         |  2 +
 net/ethtool/strset.c                         |  5 ++
 net/ethtool/tsinfo.c                         | 17 ++++
 net/socket.c                                 |  5 +-
 16 files changed, 175 insertions(+), 17 deletions(-)

--
2.17.1

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

[PATCH v1 1/4] ethtool: Add new hwtstamp flag

[Muhammad Husaini Zulkifli]

This add patch add a new DMA Time Stamp flag. User can configure
hwtstamp_config with this flag if they want to use DMA time stamp.

Suggested-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
---
 include/uapi/linux/ethtool.h         |  3 +++
 include/uapi/linux/ethtool_netlink.h |  1 +
 include/uapi/linux/net_tstamp.h      |  5 ++++-
 net/ethtool/common.c                 |  6 ++++++
 net/ethtool/common.h                 |  2 ++
 net/ethtool/strset.c                 |  5 +++++
 net/ethtool/tsinfo.c                 | 17 +++++++++++++++++
 7 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/include/uapi/linux/ethtool.h b/include/uapi/linux/ethtool.h
index fe9893d1485d..07ca8d496e1a 100644
--- a/include/uapi/linux/ethtool.h
+++ b/include/uapi/linux/ethtool.h
@@ -675,6 +675,7 @@ enum ethtool_link_ext_substate_module {
  * @ETH_SS_MSG_CLASSES: debug message class names
  * @ETH_SS_WOL_MODES: wake-on-lan modes
  * @ETH_SS_SOF_TIMESTAMPING: SOF_TIMESTAMPING_* flags
+ * @ETH_SS_HWTSTAMP_FLAG: timestamping flags
  * @ETH_SS_TS_TX_TYPES: timestamping Tx types
  * @ETH_SS_TS_RX_FILTERS: timestamping Rx filters
  * @ETH_SS_UDP_TUNNEL_TYPES: UDP tunnel types
@@ -700,6 +701,7 @@ enum ethtool_stringset {
 	ETH_SS_MSG_CLASSES,
 	ETH_SS_WOL_MODES,
 	ETH_SS_SOF_TIMESTAMPING,
+	ETH_SS_HWTSTAMP_FLAG,
 	ETH_SS_TS_TX_TYPES,
 	ETH_SS_TS_RX_FILTERS,
 	ETH_SS_UDP_TUNNEL_TYPES,
@@ -1367,6 +1369,7 @@ struct ethtool_ts_info {
 	__u32	cmd;
 	__u32	so_timestamping;
 	__s32	phc_index;
+	__u32	flag;
 	__u32	tx_types;
 	__u32	tx_reserved[3];
 	__u32	rx_filters;
diff --git a/include/uapi/linux/ethtool_netlink.h b/include/uapi/linux/ethtool_netlink.h
index 408a664fad59..58d073b5a6d2 100644
--- a/include/uapi/linux/ethtool_netlink.h
+++ b/include/uapi/linux/ethtool_netlink.h
@@ -452,6 +452,7 @@ enum {
 	ETHTOOL_A_TSINFO_UNSPEC,
 	ETHTOOL_A_TSINFO_HEADER,			/* nest - _A_HEADER_* */
 	ETHTOOL_A_TSINFO_TIMESTAMPING,			/* bitset */
+	ETHTOOL_A_TSINFO_FLAG,				/* bitset */
 	ETHTOOL_A_TSINFO_TX_TYPES,			/* bitset */
 	ETHTOOL_A_TSINFO_RX_FILTERS,			/* bitset */
 	ETHTOOL_A_TSINFO_PHC_INDEX,			/* u32 */
diff --git a/include/uapi/linux/net_tstamp.h b/include/uapi/linux/net_tstamp.h
index 55501e5e7ac8..4966d5ca521f 100644
--- a/include/uapi/linux/net_tstamp.h
+++ b/include/uapi/linux/net_tstamp.h
@@ -89,7 +89,10 @@ enum hwtstamp_flags {
 	HWTSTAMP_FLAG_BONDED_PHC_INDEX = (1<<0),
 #define HWTSTAMP_FLAG_BONDED_PHC_INDEX	HWTSTAMP_FLAG_BONDED_PHC_INDEX
 
-	HWTSTAMP_FLAG_LAST = HWTSTAMP_FLAG_BONDED_PHC_INDEX,
+	HWTSTAMP_FLAG_DMA_TIMESTAMP = (1<<1),
+#define HWTSTAMP_FLAG_DMA_TIMESTAMP	HWTSTAMP_FLAG_DMA_TIMESTAMP
+
+	HWTSTAMP_FLAG_LAST = HWTSTAMP_FLAG_DMA_TIMESTAMP,
 	HWTSTAMP_FLAG_MASK = (HWTSTAMP_FLAG_LAST - 1) | HWTSTAMP_FLAG_LAST
 };
 
diff --git a/net/ethtool/common.c b/net/ethtool/common.c
index 566adf85e658..f2a178d162ef 100644
--- a/net/ethtool/common.c
+++ b/net/ethtool/common.c
@@ -406,6 +406,12 @@ const char sof_timestamping_names[][ETH_GSTRING_LEN] = {
 };
 static_assert(ARRAY_SIZE(sof_timestamping_names) == __SOF_TIMESTAMPING_CNT);
 
+const char ts_flag_names[][ETH_GSTRING_LEN] = {
+	[const_ilog2(HWTSTAMP_FLAG_BONDED_PHC_INDEX)]	= "bonded-phc-index",
+	[const_ilog2(HWTSTAMP_FLAG_DMA_TIMESTAMP)]	= "dma-time-stamp",
+};
+static_assert(ARRAY_SIZE(ts_flag_names) == __HWTSTAMP_FLAG_CNT);
+
 const char ts_tx_type_names[][ETH_GSTRING_LEN] = {
 	[HWTSTAMP_TX_OFF]		= "off",
 	[HWTSTAMP_TX_ON]		= "on",
diff --git a/net/ethtool/common.h b/net/ethtool/common.h
index 2dc2b80aea5f..39fedceb82ca 100644
--- a/net/ethtool/common.h
+++ b/net/ethtool/common.h
@@ -13,6 +13,7 @@
 	ETHTOOL_LINK_MODE_ ## speed ## base ## type ## _ ## duplex ## _BIT
 
 #define __SOF_TIMESTAMPING_CNT (const_ilog2(SOF_TIMESTAMPING_LAST) + 1)
+#define __HWTSTAMP_FLAG_CNT (const_ilog2(HWTSTAMP_FLAG_LAST) + 1)
 
 struct link_mode_info {
 	int				speed;
@@ -33,6 +34,7 @@ extern const struct link_mode_info link_mode_params[];
 extern const char netif_msg_class_names[][ETH_GSTRING_LEN];
 extern const char wol_mode_names[][ETH_GSTRING_LEN];
 extern const char sof_timestamping_names[][ETH_GSTRING_LEN];
+extern const char ts_flag_names[][ETH_GSTRING_LEN];
 extern const char ts_tx_type_names[][ETH_GSTRING_LEN];
 extern const char ts_rx_filter_names[][ETH_GSTRING_LEN];
 extern const char udp_tunnel_type_names[][ETH_GSTRING_LEN];
diff --git a/net/ethtool/strset.c b/net/ethtool/strset.c
index 3f7de54d85fb..2c26cfece494 100644
--- a/net/ethtool/strset.c
+++ b/net/ethtool/strset.c
@@ -65,6 +65,11 @@ static const struct strset_info info_template[] = {
 		.count		= __SOF_TIMESTAMPING_CNT,
 		.strings	= sof_timestamping_names,
 	},
+	[ETH_SS_HWTSTAMP_FLAG] = {
+		.per_dev	= false,
+		.count		= __HWTSTAMP_FLAG_CNT,
+		.strings	= ts_flag_names,
+	},
 	[ETH_SS_TS_TX_TYPES] = {
 		.per_dev	= false,
 		.count		= __HWTSTAMP_TX_CNT,
diff --git a/net/ethtool/tsinfo.c b/net/ethtool/tsinfo.c
index 63b5814bd460..84aa15445944 100644
--- a/net/ethtool/tsinfo.c
+++ b/net/ethtool/tsinfo.c
@@ -50,6 +50,7 @@ static int tsinfo_reply_size(const struct ethnl_req_info *req_base,
 	int ret;
 
 	BUILD_BUG_ON(__SOF_TIMESTAMPING_CNT > 32);
+	BUILD_BUG_ON(__HWTSTAMP_FLAG_CNT > 32);
 	BUILD_BUG_ON(__HWTSTAMP_TX_CNT > 32);
 	BUILD_BUG_ON(__HWTSTAMP_FILTER_CNT > 32);
 
@@ -61,6 +62,14 @@ static int tsinfo_reply_size(const struct ethnl_req_info *req_base,
 			return ret;
 		len += ret;	/* _TSINFO_TIMESTAMPING */
 	}
+	if (ts_info->flag) {
+		ret = ethnl_bitset32_size(&ts_info->flag, NULL,
+					  __HWTSTAMP_FLAG_CNT,
+					  ts_flag_names, compact);
+		if (ret < 0)
+			return ret;
+		len += ret;	/* _TSINFO_FLAG */
+	}
 	if (ts_info->tx_types) {
 		ret = ethnl_bitset32_size(&ts_info->tx_types, NULL,
 					  __HWTSTAMP_TX_CNT,
@@ -100,6 +109,14 @@ static int tsinfo_fill_reply(struct sk_buff *skb,
 		if (ret < 0)
 			return ret;
 	}
+	if (ts_info->flag) {
+		ret = ethnl_put_bitset32(skb, ETHTOOL_A_TSINFO_FLAG,
+					 &ts_info->flag, NULL,
+					 __HWTSTAMP_FLAG_CNT,
+					 ts_flag_names, compact);
+		if (ret < 0)
+			return ret;
+	}
 	if (ts_info->tx_types) {
 		ret = ethnl_put_bitset32(skb, ETHTOOL_A_TSINFO_TX_TYPES,
 					 &ts_info->tx_types, NULL,
-- 
2.17.1

[Intel-wired-lan] [PATCH v1 1/4] ethtool: Add new hwtstamp flag

[Muhammad Husaini Zulkifli]

This add patch add a new DMA Time Stamp flag. User can configure
hwtstamp_config with this flag if they want to use DMA time stamp.

Suggested-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
---
 include/uapi/linux/ethtool.h         |  3 +++
 include/uapi/linux/ethtool_netlink.h |  1 +
 include/uapi/linux/net_tstamp.h      |  5 ++++-
 net/ethtool/common.c                 |  6 ++++++
 net/ethtool/common.h                 |  2 ++
 net/ethtool/strset.c                 |  5 +++++
 net/ethtool/tsinfo.c                 | 17 +++++++++++++++++
 7 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/include/uapi/linux/ethtool.h b/include/uapi/linux/ethtool.h
index fe9893d1485d..07ca8d496e1a 100644
--- a/include/uapi/linux/ethtool.h
+++ b/include/uapi/linux/ethtool.h
@@ -675,6 +675,7 @@ enum ethtool_link_ext_substate_module {
  * @ETH_SS_MSG_CLASSES: debug message class names
  * @ETH_SS_WOL_MODES: wake-on-lan modes
  * @ETH_SS_SOF_TIMESTAMPING: SOF_TIMESTAMPING_* flags
+ * @ETH_SS_HWTSTAMP_FLAG: timestamping flags
  * @ETH_SS_TS_TX_TYPES: timestamping Tx types
  * @ETH_SS_TS_RX_FILTERS: timestamping Rx filters
  * @ETH_SS_UDP_TUNNEL_TYPES: UDP tunnel types
@@ -700,6 +701,7 @@ enum ethtool_stringset {
 	ETH_SS_MSG_CLASSES,
 	ETH_SS_WOL_MODES,
 	ETH_SS_SOF_TIMESTAMPING,
+	ETH_SS_HWTSTAMP_FLAG,
 	ETH_SS_TS_TX_TYPES,
 	ETH_SS_TS_RX_FILTERS,
 	ETH_SS_UDP_TUNNEL_TYPES,
@@ -1367,6 +1369,7 @@ struct ethtool_ts_info {
 	__u32	cmd;
 	__u32	so_timestamping;
 	__s32	phc_index;
+	__u32	flag;
 	__u32	tx_types;
 	__u32	tx_reserved[3];
 	__u32	rx_filters;
diff --git a/include/uapi/linux/ethtool_netlink.h b/include/uapi/linux/ethtool_netlink.h
index 408a664fad59..58d073b5a6d2 100644
--- a/include/uapi/linux/ethtool_netlink.h
+++ b/include/uapi/linux/ethtool_netlink.h
@@ -452,6 +452,7 @@ enum {
 	ETHTOOL_A_TSINFO_UNSPEC,
 	ETHTOOL_A_TSINFO_HEADER,			/* nest - _A_HEADER_* */
 	ETHTOOL_A_TSINFO_TIMESTAMPING,			/* bitset */
+	ETHTOOL_A_TSINFO_FLAG,				/* bitset */
 	ETHTOOL_A_TSINFO_TX_TYPES,			/* bitset */
 	ETHTOOL_A_TSINFO_RX_FILTERS,			/* bitset */
 	ETHTOOL_A_TSINFO_PHC_INDEX,			/* u32 */
diff --git a/include/uapi/linux/net_tstamp.h b/include/uapi/linux/net_tstamp.h
index 55501e5e7ac8..4966d5ca521f 100644
--- a/include/uapi/linux/net_tstamp.h
+++ b/include/uapi/linux/net_tstamp.h
@@ -89,7 +89,10 @@ enum hwtstamp_flags {
 	HWTSTAMP_FLAG_BONDED_PHC_INDEX = (1<<0),
 #define HWTSTAMP_FLAG_BONDED_PHC_INDEX	HWTSTAMP_FLAG_BONDED_PHC_INDEX
 
-	HWTSTAMP_FLAG_LAST = HWTSTAMP_FLAG_BONDED_PHC_INDEX,
+	HWTSTAMP_FLAG_DMA_TIMESTAMP = (1<<1),
+#define HWTSTAMP_FLAG_DMA_TIMESTAMP	HWTSTAMP_FLAG_DMA_TIMESTAMP
+
+	HWTSTAMP_FLAG_LAST = HWTSTAMP_FLAG_DMA_TIMESTAMP,
 	HWTSTAMP_FLAG_MASK = (HWTSTAMP_FLAG_LAST - 1) | HWTSTAMP_FLAG_LAST
 };
 
diff --git a/net/ethtool/common.c b/net/ethtool/common.c
index 566adf85e658..f2a178d162ef 100644
--- a/net/ethtool/common.c
+++ b/net/ethtool/common.c
@@ -406,6 +406,12 @@ const char sof_timestamping_names[][ETH_GSTRING_LEN] = {
 };
 static_assert(ARRAY_SIZE(sof_timestamping_names) == __SOF_TIMESTAMPING_CNT);
 
+const char ts_flag_names[][ETH_GSTRING_LEN] = {
+	[const_ilog2(HWTSTAMP_FLAG_BONDED_PHC_INDEX)]	= "bonded-phc-index",
+	[const_ilog2(HWTSTAMP_FLAG_DMA_TIMESTAMP)]	= "dma-time-stamp",
+};
+static_assert(ARRAY_SIZE(ts_flag_names) == __HWTSTAMP_FLAG_CNT);
+
 const char ts_tx_type_names[][ETH_GSTRING_LEN] = {
 	[HWTSTAMP_TX_OFF]		= "off",
 	[HWTSTAMP_TX_ON]		= "on",
diff --git a/net/ethtool/common.h b/net/ethtool/common.h
index 2dc2b80aea5f..39fedceb82ca 100644
--- a/net/ethtool/common.h
+++ b/net/ethtool/common.h
@@ -13,6 +13,7 @@
 	ETHTOOL_LINK_MODE_ ## speed ## base ## type ## _ ## duplex ## _BIT
 
 #define __SOF_TIMESTAMPING_CNT (const_ilog2(SOF_TIMESTAMPING_LAST) + 1)
+#define __HWTSTAMP_FLAG_CNT (const_ilog2(HWTSTAMP_FLAG_LAST) + 1)
 
 struct link_mode_info {
 	int				speed;
@@ -33,6 +34,7 @@ extern const struct link_mode_info link_mode_params[];
 extern const char netif_msg_class_names[][ETH_GSTRING_LEN];
 extern const char wol_mode_names[][ETH_GSTRING_LEN];
 extern const char sof_timestamping_names[][ETH_GSTRING_LEN];
+extern const char ts_flag_names[][ETH_GSTRING_LEN];
 extern const char ts_tx_type_names[][ETH_GSTRING_LEN];
 extern const char ts_rx_filter_names[][ETH_GSTRING_LEN];
 extern const char udp_tunnel_type_names[][ETH_GSTRING_LEN];
diff --git a/net/ethtool/strset.c b/net/ethtool/strset.c
index 3f7de54d85fb..2c26cfece494 100644
--- a/net/ethtool/strset.c
+++ b/net/ethtool/strset.c
@@ -65,6 +65,11 @@ static const struct strset_info info_template[] = {
 		.count		= __SOF_TIMESTAMPING_CNT,
 		.strings	= sof_timestamping_names,
 	},
+	[ETH_SS_HWTSTAMP_FLAG] = {
+		.per_dev	= false,
+		.count		= __HWTSTAMP_FLAG_CNT,
+		.strings	= ts_flag_names,
+	},
 	[ETH_SS_TS_TX_TYPES] = {
 		.per_dev	= false,
 		.count		= __HWTSTAMP_TX_CNT,
diff --git a/net/ethtool/tsinfo.c b/net/ethtool/tsinfo.c
index 63b5814bd460..84aa15445944 100644
--- a/net/ethtool/tsinfo.c
+++ b/net/ethtool/tsinfo.c
@@ -50,6 +50,7 @@ static int tsinfo_reply_size(const struct ethnl_req_info *req_base,
 	int ret;
 
 	BUILD_BUG_ON(__SOF_TIMESTAMPING_CNT > 32);
+	BUILD_BUG_ON(__HWTSTAMP_FLAG_CNT > 32);
 	BUILD_BUG_ON(__HWTSTAMP_TX_CNT > 32);
 	BUILD_BUG_ON(__HWTSTAMP_FILTER_CNT > 32);
 
@@ -61,6 +62,14 @@ static int tsinfo_reply_size(const struct ethnl_req_info *req_base,
 			return ret;
 		len += ret;	/* _TSINFO_TIMESTAMPING */
 	}
+	if (ts_info->flag) {
+		ret = ethnl_bitset32_size(&ts_info->flag, NULL,
+					  __HWTSTAMP_FLAG_CNT,
+					  ts_flag_names, compact);
+		if (ret < 0)
+			return ret;
+		len += ret;	/* _TSINFO_FLAG */
+	}
 	if (ts_info->tx_types) {
 		ret = ethnl_bitset32_size(&ts_info->tx_types, NULL,
 					  __HWTSTAMP_TX_CNT,
@@ -100,6 +109,14 @@ static int tsinfo_fill_reply(struct sk_buff *skb,
 		if (ret < 0)
 			return ret;
 	}
+	if (ts_info->flag) {
+		ret = ethnl_put_bitset32(skb, ETHTOOL_A_TSINFO_FLAG,
+					 &ts_info->flag, NULL,
+					 __HWTSTAMP_FLAG_CNT,
+					 ts_flag_names, compact);
+		if (ret < 0)
+			return ret;
+	}
 	if (ts_info->tx_types) {
 		ret = ethnl_put_bitset32(skb, ETHTOOL_A_TSINFO_TX_TYPES,
 					 &ts_info->tx_types, NULL,
-- 
2.17.1

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

[PATCH v1 2/4] net-timestamp: Increase the size of tsflags

[Muhammad Husaini Zulkifli]

Increase the size of tsflags to support more SOF_TIMESTAMPING flags.
Current flag size can only support up to 16 flags only.

Suggested-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
---
 include/net/sock.h | 12 ++++++------
 net/socket.c       |  2 +-
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/net/sock.h b/include/net/sock.h
index 96a31026e35d..6be4d768d7dd 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -503,7 +503,7 @@ struct sock {
 #if BITS_PER_LONG==32
 	seqlock_t		sk_stamp_seq;
 #endif
-	u16			sk_tsflags;
+	u32			sk_tsflags;
 	u8			sk_shutdown;
 	atomic_t		sk_tskey;
 	atomic_t		sk_zckey;
@@ -1897,7 +1897,7 @@ void sk_send_sigurg(struct sock *sk);
 struct sockcm_cookie {
 	u64 transmit_time;
 	u32 mark;
-	u16 tsflags;
+	u32 tsflags;
 };
 
 static inline void sockcm_init(struct sockcm_cookie *sockc,
@@ -2728,7 +2728,7 @@ static inline void sock_recv_cmsgs(struct msghdr *msg, struct sock *sk,
 		sock_write_timestamp(sk, 0);
 }
 
-void __sock_tx_timestamp(__u16 tsflags, __u8 *tx_flags);
+void __sock_tx_timestamp(__u32 tsflags, __u8 *tx_flags);
 
 /**
  * _sock_tx_timestamp - checks whether the outgoing packet is to be time stamped
@@ -2739,7 +2739,7 @@ void __sock_tx_timestamp(__u16 tsflags, __u8 *tx_flags);
  *
  * Note: callers should take care of initial ``*tx_flags`` value (usually 0)
  */
-static inline void _sock_tx_timestamp(struct sock *sk, __u16 tsflags,
+static inline void _sock_tx_timestamp(struct sock *sk, __u32 tsflags,
 				      __u8 *tx_flags, __u32 *tskey)
 {
 	if (unlikely(tsflags)) {
@@ -2752,13 +2752,13 @@ static inline void _sock_tx_timestamp(struct sock *sk, __u16 tsflags,
 		*tx_flags |= SKBTX_WIFI_STATUS;
 }
 
-static inline void sock_tx_timestamp(struct sock *sk, __u16 tsflags,
+static inline void sock_tx_timestamp(struct sock *sk, __u32 tsflags,
 				     __u8 *tx_flags)
 {
 	_sock_tx_timestamp(sk, tsflags, tx_flags, NULL);
 }
 
-static inline void skb_setup_tx_timestamp(struct sk_buff *skb, __u16 tsflags)
+static inline void skb_setup_tx_timestamp(struct sk_buff *skb, __u32 tsflags)
 {
 	_sock_tx_timestamp(skb->sk, tsflags, &skb_shinfo(skb)->tx_flags,
 			   &skb_shinfo(skb)->tskey);
diff --git a/net/socket.c b/net/socket.c
index 7378375d3a5b..34ddb5d6889e 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -679,7 +679,7 @@ void sock_release(struct socket *sock)
 }
 EXPORT_SYMBOL(sock_release);
 
-void __sock_tx_timestamp(__u16 tsflags, __u8 *tx_flags)
+void __sock_tx_timestamp(__u32 tsflags, __u8 *tx_flags)
 {
 	u8 flags = *tx_flags;
 
-- 
2.17.1

[Intel-wired-lan] [PATCH v1 2/4] net-timestamp: Increase the size of tsflags

[Muhammad Husaini Zulkifli]

Increase the size of tsflags to support more SOF_TIMESTAMPING flags.
Current flag size can only support up to 16 flags only.

Suggested-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
---
 include/net/sock.h | 12 ++++++------
 net/socket.c       |  2 +-
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/include/net/sock.h b/include/net/sock.h
index 96a31026e35d..6be4d768d7dd 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -503,7 +503,7 @@ struct sock {
 #if BITS_PER_LONG==32
 	seqlock_t		sk_stamp_seq;
 #endif
-	u16			sk_tsflags;
+	u32			sk_tsflags;
 	u8			sk_shutdown;
 	atomic_t		sk_tskey;
 	atomic_t		sk_zckey;
@@ -1897,7 +1897,7 @@ void sk_send_sigurg(struct sock *sk);
 struct sockcm_cookie {
 	u64 transmit_time;
 	u32 mark;
-	u16 tsflags;
+	u32 tsflags;
 };
 
 static inline void sockcm_init(struct sockcm_cookie *sockc,
@@ -2728,7 +2728,7 @@ static inline void sock_recv_cmsgs(struct msghdr *msg, struct sock *sk,
 		sock_write_timestamp(sk, 0);
 }
 
-void __sock_tx_timestamp(__u16 tsflags, __u8 *tx_flags);
+void __sock_tx_timestamp(__u32 tsflags, __u8 *tx_flags);
 
 /**
  * _sock_tx_timestamp - checks whether the outgoing packet is to be time stamped
@@ -2739,7 +2739,7 @@ void __sock_tx_timestamp(__u16 tsflags, __u8 *tx_flags);
  *
  * Note: callers should take care of initial ``*tx_flags`` value (usually 0)
  */
-static inline void _sock_tx_timestamp(struct sock *sk, __u16 tsflags,
+static inline void _sock_tx_timestamp(struct sock *sk, __u32 tsflags,
 				      __u8 *tx_flags, __u32 *tskey)
 {
 	if (unlikely(tsflags)) {
@@ -2752,13 +2752,13 @@ static inline void _sock_tx_timestamp(struct sock *sk, __u16 tsflags,
 		*tx_flags |= SKBTX_WIFI_STATUS;
 }
 
-static inline void sock_tx_timestamp(struct sock *sk, __u16 tsflags,
+static inline void sock_tx_timestamp(struct sock *sk, __u32 tsflags,
 				     __u8 *tx_flags)
 {
 	_sock_tx_timestamp(sk, tsflags, tx_flags, NULL);
 }
 
-static inline void skb_setup_tx_timestamp(struct sk_buff *skb, __u16 tsflags)
+static inline void skb_setup_tx_timestamp(struct sk_buff *skb, __u32 tsflags)
 {
 	_sock_tx_timestamp(skb->sk, tsflags, &skb_shinfo(skb)->tx_flags,
 			   &skb_shinfo(skb)->tskey);
diff --git a/net/socket.c b/net/socket.c
index 7378375d3a5b..34ddb5d6889e 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -679,7 +679,7 @@ void sock_release(struct socket *sock)
 }
 EXPORT_SYMBOL(sock_release);
 
-void __sock_tx_timestamp(__u16 tsflags, __u8 *tx_flags)
+void __sock_tx_timestamp(__u32 tsflags, __u8 *tx_flags)
 {
 	u8 flags = *tx_flags;
 
-- 
2.17.1

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

[PATCH v1 3/4] net: sock: extend SO_TIMESTAMPING for DMA Fetch

[Muhammad Husaini Zulkifli]

This patch is to extend SO_TIMESTAMPING API to support the DMA Time
Stamp by adding a new flag SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH.
User space application can configure this flag into hwtstamp_config flag
if require to use the DMA Time Stamp for that socket application.

Suggested-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
---
 include/linux/skbuff.h          | 3 +++
 include/uapi/linux/net_tstamp.h | 6 ++++--
 net/ethtool/common.c            | 1 +
 net/socket.c                    | 3 +++
 4 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index f15d5b62539b..7a79ffa7799d 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -485,6 +485,9 @@ enum {
 
 	/* generate software time stamp when entering packet scheduling */
 	SKBTX_SCHED_TSTAMP = 1 << 6,
+
+	/* generate hardware DMA time stamp */
+	SKBTX_HW_DMA_TSTAMP = 1 << 7,
 };
 
 #define SKBTX_ANY_SW_TSTAMP	(SKBTX_SW_TSTAMP    | \
diff --git a/include/uapi/linux/net_tstamp.h b/include/uapi/linux/net_tstamp.h
index 4966d5ca521f..c9e113cea883 100644
--- a/include/uapi/linux/net_tstamp.h
+++ b/include/uapi/linux/net_tstamp.h
@@ -31,8 +31,9 @@ enum {
 	SOF_TIMESTAMPING_OPT_PKTINFO = (1<<13),
 	SOF_TIMESTAMPING_OPT_TX_SWHW = (1<<14),
 	SOF_TIMESTAMPING_BIND_PHC = (1 << 15),
+	SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH = (1 << 16),
 
-	SOF_TIMESTAMPING_LAST = SOF_TIMESTAMPING_BIND_PHC,
+	SOF_TIMESTAMPING_LAST = SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH,
 	SOF_TIMESTAMPING_MASK = (SOF_TIMESTAMPING_LAST - 1) |
 				 SOF_TIMESTAMPING_LAST
 };
@@ -45,7 +46,8 @@ enum {
 #define SOF_TIMESTAMPING_TX_RECORD_MASK	(SOF_TIMESTAMPING_TX_HARDWARE | \
 					 SOF_TIMESTAMPING_TX_SOFTWARE | \
 					 SOF_TIMESTAMPING_TX_SCHED | \
-					 SOF_TIMESTAMPING_TX_ACK)
+					 SOF_TIMESTAMPING_TX_ACK | \
+					 SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH)
 
 /**
  * struct so_timestamping - SO_TIMESTAMPING parameter
diff --git a/net/ethtool/common.c b/net/ethtool/common.c
index f2a178d162ef..24f8e7f9b4a5 100644
--- a/net/ethtool/common.c
+++ b/net/ethtool/common.c
@@ -403,6 +403,7 @@ const char sof_timestamping_names[][ETH_GSTRING_LEN] = {
 	[const_ilog2(SOF_TIMESTAMPING_OPT_PKTINFO)]  = "option-pktinfo",
 	[const_ilog2(SOF_TIMESTAMPING_OPT_TX_SWHW)]  = "option-tx-swhw",
 	[const_ilog2(SOF_TIMESTAMPING_BIND_PHC)]     = "bind-phc",
+	[const_ilog2(SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH)]	= "hardware-dma-transmit",
 };
 static_assert(ARRAY_SIZE(sof_timestamping_names) == __SOF_TIMESTAMPING_CNT);
 
diff --git a/net/socket.c b/net/socket.c
index 34ddb5d6889e..dfb9fa693a11 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -695,6 +695,9 @@ void __sock_tx_timestamp(__u32 tsflags, __u8 *tx_flags)
 			flags |= SKBTX_HW_TSTAMP_USE_CYCLES;
 	}
 
+	if (tsflags & SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH)
+		flags |= SKBTX_HW_DMA_TSTAMP;
+
 	if (tsflags & SOF_TIMESTAMPING_TX_SOFTWARE)
 		flags |= SKBTX_SW_TSTAMP;
 
-- 
2.17.1

[Intel-wired-lan] [PATCH v1 3/4] net: sock: extend SO_TIMESTAMPING for DMA Fetch

[Muhammad Husaini Zulkifli]

This patch is to extend SO_TIMESTAMPING API to support the DMA Time
Stamp by adding a new flag SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH.
User space application can configure this flag into hwtstamp_config flag
if require to use the DMA Time Stamp for that socket application.

Suggested-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Signed-off-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
---
 include/linux/skbuff.h          | 3 +++
 include/uapi/linux/net_tstamp.h | 6 ++++--
 net/ethtool/common.c            | 1 +
 net/socket.c                    | 3 +++
 4 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index f15d5b62539b..7a79ffa7799d 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -485,6 +485,9 @@ enum {
 
 	/* generate software time stamp when entering packet scheduling */
 	SKBTX_SCHED_TSTAMP = 1 << 6,
+
+	/* generate hardware DMA time stamp */
+	SKBTX_HW_DMA_TSTAMP = 1 << 7,
 };
 
 #define SKBTX_ANY_SW_TSTAMP	(SKBTX_SW_TSTAMP    | \
diff --git a/include/uapi/linux/net_tstamp.h b/include/uapi/linux/net_tstamp.h
index 4966d5ca521f..c9e113cea883 100644
--- a/include/uapi/linux/net_tstamp.h
+++ b/include/uapi/linux/net_tstamp.h
@@ -31,8 +31,9 @@ enum {
 	SOF_TIMESTAMPING_OPT_PKTINFO = (1<<13),
 	SOF_TIMESTAMPING_OPT_TX_SWHW = (1<<14),
 	SOF_TIMESTAMPING_BIND_PHC = (1 << 15),
+	SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH = (1 << 16),
 
-	SOF_TIMESTAMPING_LAST = SOF_TIMESTAMPING_BIND_PHC,
+	SOF_TIMESTAMPING_LAST = SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH,
 	SOF_TIMESTAMPING_MASK = (SOF_TIMESTAMPING_LAST - 1) |
 				 SOF_TIMESTAMPING_LAST
 };
@@ -45,7 +46,8 @@ enum {
 #define SOF_TIMESTAMPING_TX_RECORD_MASK	(SOF_TIMESTAMPING_TX_HARDWARE | \
 					 SOF_TIMESTAMPING_TX_SOFTWARE | \
 					 SOF_TIMESTAMPING_TX_SCHED | \
-					 SOF_TIMESTAMPING_TX_ACK)
+					 SOF_TIMESTAMPING_TX_ACK | \
+					 SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH)
 
 /**
  * struct so_timestamping - SO_TIMESTAMPING parameter
diff --git a/net/ethtool/common.c b/net/ethtool/common.c
index f2a178d162ef..24f8e7f9b4a5 100644
--- a/net/ethtool/common.c
+++ b/net/ethtool/common.c
@@ -403,6 +403,7 @@ const char sof_timestamping_names[][ETH_GSTRING_LEN] = {
 	[const_ilog2(SOF_TIMESTAMPING_OPT_PKTINFO)]  = "option-pktinfo",
 	[const_ilog2(SOF_TIMESTAMPING_OPT_TX_SWHW)]  = "option-tx-swhw",
 	[const_ilog2(SOF_TIMESTAMPING_BIND_PHC)]     = "bind-phc",
+	[const_ilog2(SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH)]	= "hardware-dma-transmit",
 };
 static_assert(ARRAY_SIZE(sof_timestamping_names) == __SOF_TIMESTAMPING_CNT);
 
diff --git a/net/socket.c b/net/socket.c
index 34ddb5d6889e..dfb9fa693a11 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -695,6 +695,9 @@ void __sock_tx_timestamp(__u32 tsflags, __u8 *tx_flags)
 			flags |= SKBTX_HW_TSTAMP_USE_CYCLES;
 	}
 
+	if (tsflags & SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH)
+		flags |= SKBTX_HW_DMA_TSTAMP;
+
 	if (tsflags & SOF_TIMESTAMPING_TX_SOFTWARE)
 		flags |= SKBTX_SW_TSTAMP;
 
-- 
2.17.1

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

[PATCH v1 4/4] igc: Add support for DMA timestamp for non-PTP packets

[Muhammad Husaini Zulkifli]

From: Vinicius Costa Gomes <vinicius.gomes@intel.com>

For PTP traffic, timestamp is retrieved from TXSTMP register.
For all other packets, DMA time stamp field of the Transmit
Descriptor Write-back is used.

If the TXSTAMPO register is used for both PTP and non-PTP packets,
there is a significant possibility that the time stamp for a PTP packet
will be lost when there is a lot of traffic.

This patch introduce to use the DMA Time Stamp for non PTP packet to
solve the current issue. User application can add new SOF_TIMESTAMPING flag
SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH when configure the
hwtstamp_config for the socket option if require DMA Time Stamp.

Before:

ptp4l: rms    2 max    5 freq  -3404 +/-   3 delay     1 +/-   0
ptp4l: rms    3 max    6 freq  -3400 +/-   3 delay     1 +/-   0
ptp4l: rms    2 max    4 freq  -3400 +/-   3 delay     1 +/-   0
ptp4l: timed out while polling for tx timestamp
ptp4l: increasing tx_timestamp_timeout may correct this issue,
but it is likely caused by a driver bug
ptp4l: port 1 (enp170s0.vlan): send peer delay response failed
ptp4l: port 1 (enp170s0.vlan): SLAVE to FAULTY on FAULT_DETECTED
ptp4l: port 1 (enp170s0.vlan): FAULTY to LISTENING on INIT_COMPLETE
ptp4l: port 1 (enp170s0.vlan): LISTENING to MASTER on
ANNOUNCE_RECEIPT_TIMEOUT_EXPIRES
ptp4l: selected local clock aa00aa.fffe.00aa00 as best master
ptp4l: port 1 (enp170s0.vlan): assuming the grand master role
ptp4l: port 1 (enp170s0.vlan): new foreign master 22bb22.fffe.bb22bb-1
ptp4l: selected best master clock 22bb22.fffe.bb22bb
ptp4l: port 1 (enp170s0.vlan): MASTER to UNCALIBRATED on RS_SLAVE
ptp4l: port 1 (enp170s0.vlan): UNCALIBRATED to SLAVE on
MASTER_CLOCK_SELECTED
ptp4l: rms   39 max   66 freq  -3355 +/-  45 delay     1 +/-   0
ptp4l: rms   20 max   36 freq  -3339 +/-  12 delay     1 +/-   0
ptp4l: rms   11 max   18 freq  -3371 +/-  11 delay     1 +/-   0
ptp4l: rms   10 max   16 freq  -3384 +/-   2 delay     1 +/-   0
ptp4l: rms    1 max    2 freq  -3375 +/-   2 delay     1 +/-   0
ptp4l: rms    3 max    6 freq  -3373 +/-   4 delay     0 +/-   0

After:

ptp4l: rms    3 max    4 freq  -3386 +/-   4 delay     0 +/-   0
ptp4l: rms    3 max    7 freq  -3380 +/-   3 delay     0 +/-   0
ptp4l: rms    3 max    6 freq  -3380 +/-   3 delay     0 +/-   0
ptp4l: rms    1 max    3 freq  -3381 +/-   2 delay     0 +/-   0
ptp4l: rms    3 max    5 freq  -3377 +/-   2 delay     0 +/-   0
ptp4l: rms    2 max    3 freq  -3377 +/-   2 delay     0 +/-   0
ptp4l: rms    3 max    6 freq  -3375 +/-   4 delay     0 +/-   0
ptp4l: rms    2 max    4 freq  -3380 +/-   2 delay     1 +/-   0
ptp4l: rms    4 max    7 freq  -3385 +/-   3 delay     0 +/-   0
ptp4l: rms    2 max    3 freq  -3384 +/-   2 delay     0 +/-   0
ptp4l: rms    4 max    7 freq  -3376 +/-   2 delay     0 +/-   0
ptp4l: rms    3 max    5 freq  -3376 +/-   4 delay     0 +/-   0
ptp4l: rms    3 max    5 freq  -3382 +/-   2 delay     0 +/-   0
ptp4l: rms    5 max    7 freq  -3389 +/-   2 delay     0 +/-   0
ptp4l: rms    3 max    4 freq  -3388 +/-   3 delay     1 +/-   0
ptp4l: rms    3 max    5 freq  -3387 +/-   4 delay     1 +/-   0
ptp4l: rms    5 max    8 freq  -3395 +/-   3 delay     1 +/-   0
ptp4l: rms    5 max    8 freq  -3399 +/-   4 delay     0 +/-   0
ptp4l: rms    2 max    5 freq  -3397 +/-   3 delay     1 +/-   0
ptp4l: rms    2 max    4 freq  -3397 +/-   3 delay     1 +/-   0
ptp4l: rms    2 max    3 freq  -3397 +/-   2 delay     1 +/-   0
ptp4l: rms    3 max    5 freq  -3391 +/-   2 delay     2 +/-   0

Test Setup:
back-to-back communication between Host and DUT. Host will act as
transmitter and DUT will become receiver. Host will generate the
packet using sample application with timestamping_flag of
SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH and hwtstamp_config flag of
HWTSTAMP_FLAG_DMA_TIMESTAMP.

Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Co-developed-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
Signed-off-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
Co-developed-by: Aravindhan Gunasekaran <aravindhan.gunasekaran@intel.com>
Signed-off-by: Aravindhan Gunasekaran <aravindhan.gunasekaran@intel.com>
Signed-off-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
---
 drivers/net/ethernet/intel/igc/igc.h         | 10 +++
 drivers/net/ethernet/intel/igc/igc_base.h    |  2 +-
 drivers/net/ethernet/intel/igc/igc_defines.h |  2 +
 drivers/net/ethernet/intel/igc/igc_ethtool.c |  5 +-
 drivers/net/ethernet/intel/igc/igc_main.c    | 24 ++++--
 drivers/net/ethernet/intel/igc/igc_ptp.c     | 83 ++++++++++++++++++++
 6 files changed, 119 insertions(+), 7 deletions(-)

diff --git a/drivers/net/ethernet/intel/igc/igc.h b/drivers/net/ethernet/intel/igc/igc.h
index 1e7e7071f64d..38a24b5260d1 100644
--- a/drivers/net/ethernet/intel/igc/igc.h
+++ b/drivers/net/ethernet/intel/igc/igc.h
@@ -348,6 +348,12 @@ extern char igc_driver_name[];
 #define IGC_I225_RX_LATENCY_1000	300
 #define IGC_I225_RX_LATENCY_2500	1485
 
+/* Transmit latency (for DMA timestamps) in nanosecond */
+#define IGC_I225_TX_DMA_LATENCY_10	13100
+#define IGC_I225_TX_DMA_LATENCY_100	1410
+#define IGC_I225_TX_DMA_LATENCY_1000	285
+#define IGC_I225_TX_DMA_LATENCY_2500	1485
+
 /* RX and TX descriptor control thresholds.
  * PTHRESH - MAC will consider prefetch if it has fewer than this number of
  *           descriptors available in its onboard memory.
@@ -410,6 +416,8 @@ enum igc_tx_flags {
 	/* olinfo flags */
 	IGC_TX_FLAGS_IPV4	= 0x10,
 	IGC_TX_FLAGS_CSUM	= 0x20,
+
+	IGC_TX_FLAGS_DMA_TSTAMP	= 0x200,
 };
 
 enum igc_boards {
@@ -627,6 +635,8 @@ void igc_ptp_reset(struct igc_adapter *adapter);
 void igc_ptp_suspend(struct igc_adapter *adapter);
 void igc_ptp_stop(struct igc_adapter *adapter);
 ktime_t igc_ptp_rx_pktstamp(struct igc_adapter *adapter, __le32 *buf);
+void igc_ptp_tx_dma_tstamp(struct igc_adapter *adapter,
+			   struct sk_buff *skb, u64 tstamp);
 int igc_ptp_set_ts_config(struct net_device *netdev, struct ifreq *ifr);
 int igc_ptp_get_ts_config(struct net_device *netdev, struct ifreq *ifr);
 void igc_ptp_tx_hang(struct igc_adapter *adapter);
diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h
index ce530f5fd7bd..672cf2d92165 100644
--- a/drivers/net/ethernet/intel/igc/igc_base.h
+++ b/drivers/net/ethernet/intel/igc/igc_base.h
@@ -16,7 +16,7 @@ union igc_adv_tx_desc {
 		__le32 olinfo_status;
 	} read;
 	struct {
-		__le64 rsvd;       /* Reserved */
+		__le64 dma_tstamp;
 		__le32 nxtseq_seed;
 		__le32 status;
 	} wb;
diff --git a/drivers/net/ethernet/intel/igc/igc_defines.h b/drivers/net/ethernet/intel/igc/igc_defines.h
index f7311aeb293b..baedf48b4e2e 100644
--- a/drivers/net/ethernet/intel/igc/igc_defines.h
+++ b/drivers/net/ethernet/intel/igc/igc_defines.h
@@ -312,6 +312,7 @@
 #define IGC_TXD_CMD_DEXT	0x20000000 /* Desc extension (0 = legacy) */
 #define IGC_TXD_CMD_VLE		0x40000000 /* Add VLAN tag */
 #define IGC_TXD_STAT_DD		0x00000001 /* Descriptor Done */
+#define IGC_TXD_STAT_TS_STAT    0x00000002 /* DMA Timestamp in packet */
 #define IGC_TXD_CMD_TCP		0x01000000 /* TCP packet */
 #define IGC_TXD_CMD_IP		0x02000000 /* IP packet */
 #define IGC_TXD_CMD_TSE		0x04000000 /* TCP Seg enable */
@@ -520,6 +521,7 @@
 /* Transmit Scheduling */
 #define IGC_TQAVCTRL_TRANSMIT_MODE_TSN	0x00000001
 #define IGC_TQAVCTRL_ENHANCED_QAV	0x00000008
+#define IGC_TQAVCTRL_1588_STAT_EN	0x00000004
 
 #define IGC_TXQCTL_QUEUE_MODE_LAUNCHT	0x00000001
 #define IGC_TXQCTL_STRICT_CYCLE		0x00000002
diff --git a/drivers/net/ethernet/intel/igc/igc_ethtool.c b/drivers/net/ethernet/intel/igc/igc_ethtool.c
index 8cc077b712ad..7d198fb6d619 100644
--- a/drivers/net/ethernet/intel/igc/igc_ethtool.c
+++ b/drivers/net/ethernet/intel/igc/igc_ethtool.c
@@ -1532,7 +1532,8 @@ static int igc_ethtool_get_ts_info(struct net_device *dev,
 			SOF_TIMESTAMPING_SOFTWARE |
 			SOF_TIMESTAMPING_TX_HARDWARE |
 			SOF_TIMESTAMPING_RX_HARDWARE |
-			SOF_TIMESTAMPING_RAW_HARDWARE;
+			SOF_TIMESTAMPING_RAW_HARDWARE |
+			SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH;
 
 		info->tx_types =
 			BIT(HWTSTAMP_TX_OFF) |
@@ -1541,6 +1542,8 @@ static int igc_ethtool_get_ts_info(struct net_device *dev,
 		info->rx_filters = BIT(HWTSTAMP_FILTER_NONE);
 		info->rx_filters |= BIT(HWTSTAMP_FILTER_ALL);
 
+		info->flag = HWTSTAMP_FLAG_DMA_TIMESTAMP;
+
 		return 0;
 	default:
 		return -EOPNOTSUPP;
diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c
index 97b9edb5153e..afcf98cd6117 100644
--- a/drivers/net/ethernet/intel/igc/igc_main.c
+++ b/drivers/net/ethernet/intel/igc/igc_main.c
@@ -1415,6 +1415,7 @@ static int igc_tso(struct igc_ring *tx_ring,
 static netdev_tx_t igc_xmit_frame_ring(struct sk_buff *skb,
 				       struct igc_ring *tx_ring)
 {
+	struct igc_adapter *adapter = netdev_priv(tx_ring->netdev);
 	u16 count = TXD_USE_COUNT(skb_headlen(skb));
 	__be16 protocol = vlan_get_protocol(skb);
 	struct igc_tx_buffer *first;
@@ -1445,16 +1446,14 @@ static netdev_tx_t igc_xmit_frame_ring(struct sk_buff *skb,
 	first->bytecount = skb->len;
 	first->gso_segs = 1;
 
-	if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)) {
-		struct igc_adapter *adapter = netdev_priv(tx_ring->netdev);
-
+	if (unlikely((skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) &&
+		     !(skb_shinfo(skb)->tx_flags & SKBTX_HW_DMA_TSTAMP))) {
 		/* FIXME: add support for retrieving timestamps from
 		 * the other timer registers before skipping the
 		 * timestamping request.
 		 */
 		if (adapter->tstamp_config.tx_type == HWTSTAMP_TX_ON &&
-		    !test_and_set_bit_lock(__IGC_PTP_TX_IN_PROGRESS,
-					   &adapter->state)) {
+		    !test_and_set_bit_lock(__IGC_PTP_TX_IN_PROGRESS, &adapter->state))	{
 			skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
 			tx_flags |= IGC_TX_FLAGS_TSTAMP;
 
@@ -1463,6 +1462,14 @@ static netdev_tx_t igc_xmit_frame_ring(struct sk_buff *skb,
 		} else {
 			adapter->tx_hwtstamp_skipped++;
 		}
+	} else if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_DMA_TSTAMP)) {
+		if (adapter->tstamp_config.tx_type == HWTSTAMP_TX_ON &&
+		    adapter->tstamp_config.flags == HWTSTAMP_FLAG_DMA_TIMESTAMP) {
+			skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
+			tx_flags |= IGC_TX_FLAGS_DMA_TSTAMP;
+		} else {
+			adapter->tx_hwtstamp_skipped++;
+		}
 	}
 
 	if (skb_vlan_tag_present(skb)) {
@@ -2741,6 +2748,13 @@ static bool igc_clean_tx_irq(struct igc_q_vector *q_vector, int napi_budget)
 		if (!(eop_desc->wb.status & cpu_to_le32(IGC_TXD_STAT_DD)))
 			break;
 
+		if (eop_desc->wb.status & cpu_to_le32(IGC_TXD_STAT_TS_STAT) &&
+		    tx_buffer->tx_flags & IGC_TX_FLAGS_DMA_TSTAMP) {
+			u64 tstamp = le64_to_cpu(eop_desc->wb.dma_tstamp);
+
+			igc_ptp_tx_dma_tstamp(adapter, tx_buffer->skb, tstamp);
+		}
+
 		/* clear next_to_watch to prevent false hangs */
 		tx_buffer->next_to_watch = NULL;
 
diff --git a/drivers/net/ethernet/intel/igc/igc_ptp.c b/drivers/net/ethernet/intel/igc/igc_ptp.c
index 8dbb9f903ca7..631972d7e97b 100644
--- a/drivers/net/ethernet/intel/igc/igc_ptp.c
+++ b/drivers/net/ethernet/intel/igc/igc_ptp.c
@@ -432,6 +432,29 @@ static void igc_ptp_systim_to_hwtstamp(struct igc_adapter *adapter,
 	}
 }
 
+static void igc_ptp_dma_time_to_hwtstamp(struct igc_adapter *adapter,
+					 struct skb_shared_hwtstamps *hwtstamps,
+					 u64 systim)
+{
+	struct igc_hw *hw = &adapter->hw;
+	u32 sec, nsec;
+
+	nsec = rd32(IGC_SYSTIML);
+	sec = rd32(IGC_SYSTIMH);
+
+	if (unlikely(nsec < (systim & 0xFFFFFFFF)))
+		--sec;
+
+	switch (adapter->hw.mac.type) {
+	case igc_i225:
+		memset(hwtstamps, 0, sizeof(*hwtstamps));
+		hwtstamps->hwtstamp = ktime_set(sec, systim & 0xFFFFFFFF);
+		break;
+	default:
+		break;
+	}
+}
+
 /**
  * igc_ptp_rx_pktstamp - Retrieve timestamp from Rx packet buffer
  * @adapter: Pointer to adapter the packet buffer belongs to
@@ -549,6 +572,28 @@ static void igc_ptp_enable_tx_timestamp(struct igc_adapter *adapter)
 	rd32(IGC_TXSTMPH);
 }
 
+static void igc_ptp_disable_dma_timestamp(struct igc_adapter *adapter)
+{
+	struct igc_hw *hw = &adapter->hw;
+	u32 tqavctrl;
+
+	tqavctrl = rd32(IGC_TQAVCTRL);
+	tqavctrl &= ~IGC_TQAVCTRL_1588_STAT_EN;
+
+	wr32(IGC_TQAVCTRL, tqavctrl);
+}
+
+static void igc_ptp_enable_dma_timestamp(struct igc_adapter *adapter)
+{
+	struct igc_hw *hw = &adapter->hw;
+	u32 tqavctrl;
+
+	tqavctrl = rd32(IGC_TQAVCTRL);
+	tqavctrl |= IGC_TQAVCTRL_1588_STAT_EN;
+
+	wr32(IGC_TQAVCTRL, tqavctrl);
+}
+
 /**
  * igc_ptp_set_timestamp_mode - setup hardware for timestamping
  * @adapter: networking device structure
@@ -562,9 +607,14 @@ static int igc_ptp_set_timestamp_mode(struct igc_adapter *adapter,
 	switch (config->tx_type) {
 	case HWTSTAMP_TX_OFF:
 		igc_ptp_disable_tx_timestamp(adapter);
+		igc_ptp_disable_dma_timestamp(adapter);
 		break;
 	case HWTSTAMP_TX_ON:
 		igc_ptp_enable_tx_timestamp(adapter);
+
+		/* Ensure that flag only can be used during HWTSTAMP_TX_ON */
+		if (config->flags == HWTSTAMP_FLAG_DMA_TIMESTAMP)
+			igc_ptp_enable_dma_timestamp(adapter);
 		break;
 	default:
 		return -ERANGE;
@@ -683,6 +733,39 @@ static void igc_ptp_tx_hwtstamp(struct igc_adapter *adapter)
 	dev_kfree_skb_any(skb);
 }
 
+void igc_ptp_tx_dma_tstamp(struct igc_adapter *adapter,
+			   struct sk_buff *skb, u64 tstamp)
+{
+	struct skb_shared_hwtstamps shhwtstamps;
+	int adjust = 0;
+
+	if (!(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS))
+		return;
+
+	igc_ptp_dma_time_to_hwtstamp(adapter, &shhwtstamps, tstamp);
+
+	switch (adapter->link_speed) {
+	case SPEED_10:
+		adjust = IGC_I225_TX_DMA_LATENCY_10;
+		break;
+	case SPEED_100:
+		adjust = IGC_I225_TX_DMA_LATENCY_100;
+		break;
+	case SPEED_1000:
+		adjust = IGC_I225_TX_DMA_LATENCY_1000;
+		break;
+	case SPEED_2500:
+		adjust = IGC_I225_TX_DMA_LATENCY_2500;
+		break;
+	}
+
+	shhwtstamps.hwtstamp =
+		ktime_add_ns(shhwtstamps.hwtstamp, adjust);
+
+	/* Notify the stack and free the skb after we've unlocked */
+	skb_tstamp_tx(skb, &shhwtstamps);
+}
+
 /**
  * igc_ptp_tx_work
  * @work: pointer to work struct
-- 
2.17.1

[Intel-wired-lan] [PATCH v1 4/4] igc: Add support for DMA timestamp for non-PTP packets

[Muhammad Husaini Zulkifli]

From: Vinicius Costa Gomes <vinicius.gomes@intel.com>

For PTP traffic, timestamp is retrieved from TXSTMP register.
For all other packets, DMA time stamp field of the Transmit
Descriptor Write-back is used.

If the TXSTAMPO register is used for both PTP and non-PTP packets,
there is a significant possibility that the time stamp for a PTP packet
will be lost when there is a lot of traffic.

This patch introduce to use the DMA Time Stamp for non PTP packet to
solve the current issue. User application can add new SOF_TIMESTAMPING flag
SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH when configure the
hwtstamp_config for the socket option if require DMA Time Stamp.

Before:

ptp4l: rms    2 max    5 freq  -3404 +/-   3 delay     1 +/-   0
ptp4l: rms    3 max    6 freq  -3400 +/-   3 delay     1 +/-   0
ptp4l: rms    2 max    4 freq  -3400 +/-   3 delay     1 +/-   0
ptp4l: timed out while polling for tx timestamp
ptp4l: increasing tx_timestamp_timeout may correct this issue,
but it is likely caused by a driver bug
ptp4l: port 1 (enp170s0.vlan): send peer delay response failed
ptp4l: port 1 (enp170s0.vlan): SLAVE to FAULTY on FAULT_DETECTED
ptp4l: port 1 (enp170s0.vlan): FAULTY to LISTENING on INIT_COMPLETE
ptp4l: port 1 (enp170s0.vlan): LISTENING to MASTER on
ANNOUNCE_RECEIPT_TIMEOUT_EXPIRES
ptp4l: selected local clock aa00aa.fffe.00aa00 as best master
ptp4l: port 1 (enp170s0.vlan): assuming the grand master role
ptp4l: port 1 (enp170s0.vlan): new foreign master 22bb22.fffe.bb22bb-1
ptp4l: selected best master clock 22bb22.fffe.bb22bb
ptp4l: port 1 (enp170s0.vlan): MASTER to UNCALIBRATED on RS_SLAVE
ptp4l: port 1 (enp170s0.vlan): UNCALIBRATED to SLAVE on
MASTER_CLOCK_SELECTED
ptp4l: rms   39 max   66 freq  -3355 +/-  45 delay     1 +/-   0
ptp4l: rms   20 max   36 freq  -3339 +/-  12 delay     1 +/-   0
ptp4l: rms   11 max   18 freq  -3371 +/-  11 delay     1 +/-   0
ptp4l: rms   10 max   16 freq  -3384 +/-   2 delay     1 +/-   0
ptp4l: rms    1 max    2 freq  -3375 +/-   2 delay     1 +/-   0
ptp4l: rms    3 max    6 freq  -3373 +/-   4 delay     0 +/-   0

After:

ptp4l: rms    3 max    4 freq  -3386 +/-   4 delay     0 +/-   0
ptp4l: rms    3 max    7 freq  -3380 +/-   3 delay     0 +/-   0
ptp4l: rms    3 max    6 freq  -3380 +/-   3 delay     0 +/-   0
ptp4l: rms    1 max    3 freq  -3381 +/-   2 delay     0 +/-   0
ptp4l: rms    3 max    5 freq  -3377 +/-   2 delay     0 +/-   0
ptp4l: rms    2 max    3 freq  -3377 +/-   2 delay     0 +/-   0
ptp4l: rms    3 max    6 freq  -3375 +/-   4 delay     0 +/-   0
ptp4l: rms    2 max    4 freq  -3380 +/-   2 delay     1 +/-   0
ptp4l: rms    4 max    7 freq  -3385 +/-   3 delay     0 +/-   0
ptp4l: rms    2 max    3 freq  -3384 +/-   2 delay     0 +/-   0
ptp4l: rms    4 max    7 freq  -3376 +/-   2 delay     0 +/-   0
ptp4l: rms    3 max    5 freq  -3376 +/-   4 delay     0 +/-   0
ptp4l: rms    3 max    5 freq  -3382 +/-   2 delay     0 +/-   0
ptp4l: rms    5 max    7 freq  -3389 +/-   2 delay     0 +/-   0
ptp4l: rms    3 max    4 freq  -3388 +/-   3 delay     1 +/-   0
ptp4l: rms    3 max    5 freq  -3387 +/-   4 delay     1 +/-   0
ptp4l: rms    5 max    8 freq  -3395 +/-   3 delay     1 +/-   0
ptp4l: rms    5 max    8 freq  -3399 +/-   4 delay     0 +/-   0
ptp4l: rms    2 max    5 freq  -3397 +/-   3 delay     1 +/-   0
ptp4l: rms    2 max    4 freq  -3397 +/-   3 delay     1 +/-   0
ptp4l: rms    2 max    3 freq  -3397 +/-   2 delay     1 +/-   0
ptp4l: rms    3 max    5 freq  -3391 +/-   2 delay     2 +/-   0

Test Setup:
back-to-back communication between Host and DUT. Host will act as
transmitter and DUT will become receiver. Host will generate the
packet using sample application with timestamping_flag of
SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH and hwtstamp_config flag of
HWTSTAMP_FLAG_DMA_TIMESTAMP.

Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Co-developed-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
Signed-off-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
Co-developed-by: Aravindhan Gunasekaran <aravindhan.gunasekaran@intel.com>
Signed-off-by: Aravindhan Gunasekaran <aravindhan.gunasekaran@intel.com>
Signed-off-by: Muhammad Husaini Zulkifli <muhammad.husaini.zulkifli@intel.com>
---
 drivers/net/ethernet/intel/igc/igc.h         | 10 +++
 drivers/net/ethernet/intel/igc/igc_base.h    |  2 +-
 drivers/net/ethernet/intel/igc/igc_defines.h |  2 +
 drivers/net/ethernet/intel/igc/igc_ethtool.c |  5 +-
 drivers/net/ethernet/intel/igc/igc_main.c    | 24 ++++--
 drivers/net/ethernet/intel/igc/igc_ptp.c     | 83 ++++++++++++++++++++
 6 files changed, 119 insertions(+), 7 deletions(-)

diff --git a/drivers/net/ethernet/intel/igc/igc.h b/drivers/net/ethernet/intel/igc/igc.h
index 1e7e7071f64d..38a24b5260d1 100644
--- a/drivers/net/ethernet/intel/igc/igc.h
+++ b/drivers/net/ethernet/intel/igc/igc.h
@@ -348,6 +348,12 @@ extern char igc_driver_name[];
 #define IGC_I225_RX_LATENCY_1000	300
 #define IGC_I225_RX_LATENCY_2500	1485
 
+/* Transmit latency (for DMA timestamps) in nanosecond */
+#define IGC_I225_TX_DMA_LATENCY_10	13100
+#define IGC_I225_TX_DMA_LATENCY_100	1410
+#define IGC_I225_TX_DMA_LATENCY_1000	285
+#define IGC_I225_TX_DMA_LATENCY_2500	1485
+
 /* RX and TX descriptor control thresholds.
  * PTHRESH - MAC will consider prefetch if it has fewer than this number of
  *           descriptors available in its onboard memory.
@@ -410,6 +416,8 @@ enum igc_tx_flags {
 	/* olinfo flags */
 	IGC_TX_FLAGS_IPV4	= 0x10,
 	IGC_TX_FLAGS_CSUM	= 0x20,
+
+	IGC_TX_FLAGS_DMA_TSTAMP	= 0x200,
 };
 
 enum igc_boards {
@@ -627,6 +635,8 @@ void igc_ptp_reset(struct igc_adapter *adapter);
 void igc_ptp_suspend(struct igc_adapter *adapter);
 void igc_ptp_stop(struct igc_adapter *adapter);
 ktime_t igc_ptp_rx_pktstamp(struct igc_adapter *adapter, __le32 *buf);
+void igc_ptp_tx_dma_tstamp(struct igc_adapter *adapter,
+			   struct sk_buff *skb, u64 tstamp);
 int igc_ptp_set_ts_config(struct net_device *netdev, struct ifreq *ifr);
 int igc_ptp_get_ts_config(struct net_device *netdev, struct ifreq *ifr);
 void igc_ptp_tx_hang(struct igc_adapter *adapter);
diff --git a/drivers/net/ethernet/intel/igc/igc_base.h b/drivers/net/ethernet/intel/igc/igc_base.h
index ce530f5fd7bd..672cf2d92165 100644
--- a/drivers/net/ethernet/intel/igc/igc_base.h
+++ b/drivers/net/ethernet/intel/igc/igc_base.h
@@ -16,7 +16,7 @@ union igc_adv_tx_desc {
 		__le32 olinfo_status;
 	} read;
 	struct {
-		__le64 rsvd;       /* Reserved */
+		__le64 dma_tstamp;
 		__le32 nxtseq_seed;
 		__le32 status;
 	} wb;
diff --git a/drivers/net/ethernet/intel/igc/igc_defines.h b/drivers/net/ethernet/intel/igc/igc_defines.h
index f7311aeb293b..baedf48b4e2e 100644
--- a/drivers/net/ethernet/intel/igc/igc_defines.h
+++ b/drivers/net/ethernet/intel/igc/igc_defines.h
@@ -312,6 +312,7 @@
 #define IGC_TXD_CMD_DEXT	0x20000000 /* Desc extension (0 = legacy) */
 #define IGC_TXD_CMD_VLE		0x40000000 /* Add VLAN tag */
 #define IGC_TXD_STAT_DD		0x00000001 /* Descriptor Done */
+#define IGC_TXD_STAT_TS_STAT    0x00000002 /* DMA Timestamp in packet */
 #define IGC_TXD_CMD_TCP		0x01000000 /* TCP packet */
 #define IGC_TXD_CMD_IP		0x02000000 /* IP packet */
 #define IGC_TXD_CMD_TSE		0x04000000 /* TCP Seg enable */
@@ -520,6 +521,7 @@
 /* Transmit Scheduling */
 #define IGC_TQAVCTRL_TRANSMIT_MODE_TSN	0x00000001
 #define IGC_TQAVCTRL_ENHANCED_QAV	0x00000008
+#define IGC_TQAVCTRL_1588_STAT_EN	0x00000004
 
 #define IGC_TXQCTL_QUEUE_MODE_LAUNCHT	0x00000001
 #define IGC_TXQCTL_STRICT_CYCLE		0x00000002
diff --git a/drivers/net/ethernet/intel/igc/igc_ethtool.c b/drivers/net/ethernet/intel/igc/igc_ethtool.c
index 8cc077b712ad..7d198fb6d619 100644
--- a/drivers/net/ethernet/intel/igc/igc_ethtool.c
+++ b/drivers/net/ethernet/intel/igc/igc_ethtool.c
@@ -1532,7 +1532,8 @@ static int igc_ethtool_get_ts_info(struct net_device *dev,
 			SOF_TIMESTAMPING_SOFTWARE |
 			SOF_TIMESTAMPING_TX_HARDWARE |
 			SOF_TIMESTAMPING_RX_HARDWARE |
-			SOF_TIMESTAMPING_RAW_HARDWARE;
+			SOF_TIMESTAMPING_RAW_HARDWARE |
+			SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH;
 
 		info->tx_types =
 			BIT(HWTSTAMP_TX_OFF) |
@@ -1541,6 +1542,8 @@ static int igc_ethtool_get_ts_info(struct net_device *dev,
 		info->rx_filters = BIT(HWTSTAMP_FILTER_NONE);
 		info->rx_filters |= BIT(HWTSTAMP_FILTER_ALL);
 
+		info->flag = HWTSTAMP_FLAG_DMA_TIMESTAMP;
+
 		return 0;
 	default:
 		return -EOPNOTSUPP;
diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c
index 97b9edb5153e..afcf98cd6117 100644
--- a/drivers/net/ethernet/intel/igc/igc_main.c
+++ b/drivers/net/ethernet/intel/igc/igc_main.c
@@ -1415,6 +1415,7 @@ static int igc_tso(struct igc_ring *tx_ring,
 static netdev_tx_t igc_xmit_frame_ring(struct sk_buff *skb,
 				       struct igc_ring *tx_ring)
 {
+	struct igc_adapter *adapter = netdev_priv(tx_ring->netdev);
 	u16 count = TXD_USE_COUNT(skb_headlen(skb));
 	__be16 protocol = vlan_get_protocol(skb);
 	struct igc_tx_buffer *first;
@@ -1445,16 +1446,14 @@ static netdev_tx_t igc_xmit_frame_ring(struct sk_buff *skb,
 	first->bytecount = skb->len;
 	first->gso_segs = 1;
 
-	if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)) {
-		struct igc_adapter *adapter = netdev_priv(tx_ring->netdev);
-
+	if (unlikely((skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP) &&
+		     !(skb_shinfo(skb)->tx_flags & SKBTX_HW_DMA_TSTAMP))) {
 		/* FIXME: add support for retrieving timestamps from
 		 * the other timer registers before skipping the
 		 * timestamping request.
 		 */
 		if (adapter->tstamp_config.tx_type == HWTSTAMP_TX_ON &&
-		    !test_and_set_bit_lock(__IGC_PTP_TX_IN_PROGRESS,
-					   &adapter->state)) {
+		    !test_and_set_bit_lock(__IGC_PTP_TX_IN_PROGRESS, &adapter->state))	{
 			skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
 			tx_flags |= IGC_TX_FLAGS_TSTAMP;
 
@@ -1463,6 +1462,14 @@ static netdev_tx_t igc_xmit_frame_ring(struct sk_buff *skb,
 		} else {
 			adapter->tx_hwtstamp_skipped++;
 		}
+	} else if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_DMA_TSTAMP)) {
+		if (adapter->tstamp_config.tx_type == HWTSTAMP_TX_ON &&
+		    adapter->tstamp_config.flags == HWTSTAMP_FLAG_DMA_TIMESTAMP) {
+			skb_shinfo(skb)->tx_flags |= SKBTX_IN_PROGRESS;
+			tx_flags |= IGC_TX_FLAGS_DMA_TSTAMP;
+		} else {
+			adapter->tx_hwtstamp_skipped++;
+		}
 	}
 
 	if (skb_vlan_tag_present(skb)) {
@@ -2741,6 +2748,13 @@ static bool igc_clean_tx_irq(struct igc_q_vector *q_vector, int napi_budget)
 		if (!(eop_desc->wb.status & cpu_to_le32(IGC_TXD_STAT_DD)))
 			break;
 
+		if (eop_desc->wb.status & cpu_to_le32(IGC_TXD_STAT_TS_STAT) &&
+		    tx_buffer->tx_flags & IGC_TX_FLAGS_DMA_TSTAMP) {
+			u64 tstamp = le64_to_cpu(eop_desc->wb.dma_tstamp);
+
+			igc_ptp_tx_dma_tstamp(adapter, tx_buffer->skb, tstamp);
+		}
+
 		/* clear next_to_watch to prevent false hangs */
 		tx_buffer->next_to_watch = NULL;
 
diff --git a/drivers/net/ethernet/intel/igc/igc_ptp.c b/drivers/net/ethernet/intel/igc/igc_ptp.c
index 8dbb9f903ca7..631972d7e97b 100644
--- a/drivers/net/ethernet/intel/igc/igc_ptp.c
+++ b/drivers/net/ethernet/intel/igc/igc_ptp.c
@@ -432,6 +432,29 @@ static void igc_ptp_systim_to_hwtstamp(struct igc_adapter *adapter,
 	}
 }
 
+static void igc_ptp_dma_time_to_hwtstamp(struct igc_adapter *adapter,
+					 struct skb_shared_hwtstamps *hwtstamps,
+					 u64 systim)
+{
+	struct igc_hw *hw = &adapter->hw;
+	u32 sec, nsec;
+
+	nsec = rd32(IGC_SYSTIML);
+	sec = rd32(IGC_SYSTIMH);
+
+	if (unlikely(nsec < (systim & 0xFFFFFFFF)))
+		--sec;
+
+	switch (adapter->hw.mac.type) {
+	case igc_i225:
+		memset(hwtstamps, 0, sizeof(*hwtstamps));
+		hwtstamps->hwtstamp = ktime_set(sec, systim & 0xFFFFFFFF);
+		break;
+	default:
+		break;
+	}
+}
+
 /**
  * igc_ptp_rx_pktstamp - Retrieve timestamp from Rx packet buffer
  * @adapter: Pointer to adapter the packet buffer belongs to
@@ -549,6 +572,28 @@ static void igc_ptp_enable_tx_timestamp(struct igc_adapter *adapter)
 	rd32(IGC_TXSTMPH);
 }
 
+static void igc_ptp_disable_dma_timestamp(struct igc_adapter *adapter)
+{
+	struct igc_hw *hw = &adapter->hw;
+	u32 tqavctrl;
+
+	tqavctrl = rd32(IGC_TQAVCTRL);
+	tqavctrl &= ~IGC_TQAVCTRL_1588_STAT_EN;
+
+	wr32(IGC_TQAVCTRL, tqavctrl);
+}
+
+static void igc_ptp_enable_dma_timestamp(struct igc_adapter *adapter)
+{
+	struct igc_hw *hw = &adapter->hw;
+	u32 tqavctrl;
+
+	tqavctrl = rd32(IGC_TQAVCTRL);
+	tqavctrl |= IGC_TQAVCTRL_1588_STAT_EN;
+
+	wr32(IGC_TQAVCTRL, tqavctrl);
+}
+
 /**
  * igc_ptp_set_timestamp_mode - setup hardware for timestamping
  * @adapter: networking device structure
@@ -562,9 +607,14 @@ static int igc_ptp_set_timestamp_mode(struct igc_adapter *adapter,
 	switch (config->tx_type) {
 	case HWTSTAMP_TX_OFF:
 		igc_ptp_disable_tx_timestamp(adapter);
+		igc_ptp_disable_dma_timestamp(adapter);
 		break;
 	case HWTSTAMP_TX_ON:
 		igc_ptp_enable_tx_timestamp(adapter);
+
+		/* Ensure that flag only can be used during HWTSTAMP_TX_ON */
+		if (config->flags == HWTSTAMP_FLAG_DMA_TIMESTAMP)
+			igc_ptp_enable_dma_timestamp(adapter);
 		break;
 	default:
 		return -ERANGE;
@@ -683,6 +733,39 @@ static void igc_ptp_tx_hwtstamp(struct igc_adapter *adapter)
 	dev_kfree_skb_any(skb);
 }
 
+void igc_ptp_tx_dma_tstamp(struct igc_adapter *adapter,
+			   struct sk_buff *skb, u64 tstamp)
+{
+	struct skb_shared_hwtstamps shhwtstamps;
+	int adjust = 0;
+
+	if (!(skb_shinfo(skb)->tx_flags & SKBTX_IN_PROGRESS))
+		return;
+
+	igc_ptp_dma_time_to_hwtstamp(adapter, &shhwtstamps, tstamp);
+
+	switch (adapter->link_speed) {
+	case SPEED_10:
+		adjust = IGC_I225_TX_DMA_LATENCY_10;
+		break;
+	case SPEED_100:
+		adjust = IGC_I225_TX_DMA_LATENCY_100;
+		break;
+	case SPEED_1000:
+		adjust = IGC_I225_TX_DMA_LATENCY_1000;
+		break;
+	case SPEED_2500:
+		adjust = IGC_I225_TX_DMA_LATENCY_2500;
+		break;
+	}
+
+	shhwtstamps.hwtstamp =
+		ktime_add_ns(shhwtstamps.hwtstamp, adjust);
+
+	/* Notify the stack and free the skb after we've unlocked */
+	skb_tstamp_tx(skb, &shhwtstamps);
+}
+
 /**
  * igc_ptp_tx_work
  * @work: pointer to work struct
-- 
2.17.1

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Andrew Lunn]

On Tue, Sep 27, 2022 at 09:06:52PM +0800, Muhammad Husaini Zulkifli wrote:
> The HW TX timestamps created by the NIC via socket options can be
> requested using the current network timestamps generation capability of
> SOF_TIMESTAMPING_TX_HARDWARE. The most common users of this socket flag
> is PTP, however other packet applications that require tx timestamps might
> also ask for it.
> 
> The problem is that, when there is a lot of traffic, there is a high chance
> that the timestamps for a PTP packet will be lost if both PTP and Non-PTP
> packets use the same SOF TIMESTAMPING TX HARDWARE causing the tx timeout.
> 
> DMA timestamps through socket options are not currently available to
> the user. Because if the user wants to, they can configure the hwtstamp
> config option to use the new introduced DMA Time Stamp flag through the
> setsockopt().
> 
> With these additional socket options, users can continue to utilise
> HW timestamps for PTP while specifying non-PTP packets to use DMA
> timestamps if the NIC can support them.

Although this is not actually for PTP, you probably should Cc: the PTP
maintainer for patches like this.

	   Andrew

Re: [Intel-wired-lan] [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Andrew Lunn]

On Tue, Sep 27, 2022 at 09:06:52PM +0800, Muhammad Husaini Zulkifli wrote:
> The HW TX timestamps created by the NIC via socket options can be
> requested using the current network timestamps generation capability of
> SOF_TIMESTAMPING_TX_HARDWARE. The most common users of this socket flag
> is PTP, however other packet applications that require tx timestamps might
> also ask for it.
> 
> The problem is that, when there is a lot of traffic, there is a high chance
> that the timestamps for a PTP packet will be lost if both PTP and Non-PTP
> packets use the same SOF TIMESTAMPING TX HARDWARE causing the tx timeout.
> 
> DMA timestamps through socket options are not currently available to
> the user. Because if the user wants to, they can configure the hwtstamp
> config option to use the new introduced DMA Time Stamp flag through the
> setsockopt().
> 
> With these additional socket options, users can continue to utilise
> HW timestamps for PTP while specifying non-PTP packets to use DMA
> timestamps if the NIC can support them.

Although this is not actually for PTP, you probably should Cc: the PTP
maintainer for patches like this.

	   Andrew
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Jakub Kicinski]

On Tue, 27 Sep 2022 21:06:52 +0800 Muhammad Husaini Zulkifli wrote:
> The HW TX timestamps created by the NIC via socket options can be
> requested using the current network timestamps generation capability of
> SOF_TIMESTAMPING_TX_HARDWARE. The most common users of this socket flag
> is PTP, however other packet applications that require tx timestamps might
> also ask for it.
> 
> The problem is that, when there is a lot of traffic, there is a high chance
> that the timestamps for a PTP packet will be lost if both PTP and Non-PTP
> packets use the same SOF TIMESTAMPING TX HARDWARE causing the tx timeout.
> 
> DMA timestamps through socket options are not currently available to
> the user. Because if the user wants to, they can configure the hwtstamp
> config option to use the new introduced DMA Time Stamp flag through the
> setsockopt().
> 
> With these additional socket options, users can continue to utilise
> HW timestamps for PTP while specifying non-PTP packets to use DMA
> timestamps if the NIC can support them.
> 
> Any socket application can be use to verify this.
> TSN Ref SW application is been used for testing by changing as below:

Very glad to see this being worked on!

High level tho, are we assuming that the existing HW timestamps are
always PTP-quality, i.e. captured when SFD crosses the RS layer, or
whatnot? I'm afraid some NICs already report PCI stamps as the HW ones.
So the existing HW stamps are conceptually of "any" type, if we want to
be 100% sure NIC actually stamps at the PHY we'd need another tx_type
to express that.

Same story on the Rx - what do you plan to do there? We'll need to
configure the filters per type, but that's likely to mean two new
filters, because the current one gives no guarantee.

Re: [Intel-wired-lan] [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Jakub Kicinski]

On Tue, 27 Sep 2022 21:06:52 +0800 Muhammad Husaini Zulkifli wrote:
> The HW TX timestamps created by the NIC via socket options can be
> requested using the current network timestamps generation capability of
> SOF_TIMESTAMPING_TX_HARDWARE. The most common users of this socket flag
> is PTP, however other packet applications that require tx timestamps might
> also ask for it.
> 
> The problem is that, when there is a lot of traffic, there is a high chance
> that the timestamps for a PTP packet will be lost if both PTP and Non-PTP
> packets use the same SOF TIMESTAMPING TX HARDWARE causing the tx timeout.
> 
> DMA timestamps through socket options are not currently available to
> the user. Because if the user wants to, they can configure the hwtstamp
> config option to use the new introduced DMA Time Stamp flag through the
> setsockopt().
> 
> With these additional socket options, users can continue to utilise
> HW timestamps for PTP while specifying non-PTP packets to use DMA
> timestamps if the NIC can support them.
> 
> Any socket application can be use to verify this.
> TSN Ref SW application is been used for testing by changing as below:

Very glad to see this being worked on!

High level tho, are we assuming that the existing HW timestamps are
always PTP-quality, i.e. captured when SFD crosses the RS layer, or
whatnot? I'm afraid some NICs already report PCI stamps as the HW ones.
So the existing HW stamps are conceptually of "any" type, if we want to
be 100% sure NIC actually stamps at the PHY we'd need another tx_type
to express that.

Same story on the Rx - what do you plan to do there? We'll need to
configure the filters per type, but that's likely to mean two new
filters, because the current one gives no guarantee.
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

Re: [PATCH v1 1/4] ethtool: Add new hwtstamp flag

[Jakub Kicinski]

On Tue, 27 Sep 2022 21:06:53 +0800 Muhammad Husaini Zulkifli wrote:
> --- a/include/uapi/linux/ethtool.h
> +++ b/include/uapi/linux/ethtool.h
> @@ -675,6 +675,7 @@ enum ethtool_link_ext_substate_module {
>   * @ETH_SS_MSG_CLASSES: debug message class names
>   * @ETH_SS_WOL_MODES: wake-on-lan modes
>   * @ETH_SS_SOF_TIMESTAMPING: SOF_TIMESTAMPING_* flags
> + * @ETH_SS_HWTSTAMP_FLAG: timestamping flags
>   * @ETH_SS_TS_TX_TYPES: timestamping Tx types
>   * @ETH_SS_TS_RX_FILTERS: timestamping Rx filters
>   * @ETH_SS_UDP_TUNNEL_TYPES: UDP tunnel types
> @@ -700,6 +701,7 @@ enum ethtool_stringset {
>  	ETH_SS_MSG_CLASSES,
>  	ETH_SS_WOL_MODES,
>  	ETH_SS_SOF_TIMESTAMPING,
> +	ETH_SS_HWTSTAMP_FLAG,
>  	ETH_SS_TS_TX_TYPES,
>  	ETH_SS_TS_RX_FILTERS,
>  	ETH_SS_UDP_TUNNEL_TYPES,
> @@ -1367,6 +1369,7 @@ struct ethtool_ts_info {
>  	__u32	cmd;
>  	__u32	so_timestamping;
>  	__s32	phc_index;
> +	__u32	flag;
>  	__u32	tx_types;
>  	__u32	tx_reserved[3];
>  	__u32	rx_filters;
> diff --git a/include/uapi/linux/ethtool_netlink.h b/include/uapi/linux/ethtool_netlink.h
> index 408a664fad59..58d073b5a6d2 100644
> --- a/include/uapi/linux/ethtool_netlink.h
> +++ b/include/uapi/linux/ethtool_netlink.h
> @@ -452,6 +452,7 @@ enum {
>  	ETHTOOL_A_TSINFO_UNSPEC,
>  	ETHTOOL_A_TSINFO_HEADER,			/* nest - _A_HEADER_* */
>  	ETHTOOL_A_TSINFO_TIMESTAMPING,			/* bitset */
> +	ETHTOOL_A_TSINFO_FLAG,				/* bitset */
>  	ETHTOOL_A_TSINFO_TX_TYPES,			/* bitset */
>  	ETHTOOL_A_TSINFO_RX_FILTERS,			/* bitset */
>  	ETHTOOL_A_TSINFO_PHC_INDEX,			/* u32 */

You can't add stuff into the middle of an enum or a struct in uAPI.
What's worse for the struct ethtool_ts_info you can't actually add
anything in, period. You can reuse reserved fields but even that
requires extra legwork. If the fields were not previously validated on
input to the kernel (ie. kernel didn't check they are zero) the ioctl
path can't use them, because some application may had been passing in
garbage.

Re: [Intel-wired-lan] [PATCH v1 1/4] ethtool: Add new hwtstamp flag

[Jakub Kicinski]

On Tue, 27 Sep 2022 21:06:53 +0800 Muhammad Husaini Zulkifli wrote:
> --- a/include/uapi/linux/ethtool.h
> +++ b/include/uapi/linux/ethtool.h
> @@ -675,6 +675,7 @@ enum ethtool_link_ext_substate_module {
>   * @ETH_SS_MSG_CLASSES: debug message class names
>   * @ETH_SS_WOL_MODES: wake-on-lan modes
>   * @ETH_SS_SOF_TIMESTAMPING: SOF_TIMESTAMPING_* flags
> + * @ETH_SS_HWTSTAMP_FLAG: timestamping flags
>   * @ETH_SS_TS_TX_TYPES: timestamping Tx types
>   * @ETH_SS_TS_RX_FILTERS: timestamping Rx filters
>   * @ETH_SS_UDP_TUNNEL_TYPES: UDP tunnel types
> @@ -700,6 +701,7 @@ enum ethtool_stringset {
>  	ETH_SS_MSG_CLASSES,
>  	ETH_SS_WOL_MODES,
>  	ETH_SS_SOF_TIMESTAMPING,
> +	ETH_SS_HWTSTAMP_FLAG,
>  	ETH_SS_TS_TX_TYPES,
>  	ETH_SS_TS_RX_FILTERS,
>  	ETH_SS_UDP_TUNNEL_TYPES,
> @@ -1367,6 +1369,7 @@ struct ethtool_ts_info {
>  	__u32	cmd;
>  	__u32	so_timestamping;
>  	__s32	phc_index;
> +	__u32	flag;
>  	__u32	tx_types;
>  	__u32	tx_reserved[3];
>  	__u32	rx_filters;
> diff --git a/include/uapi/linux/ethtool_netlink.h b/include/uapi/linux/ethtool_netlink.h
> index 408a664fad59..58d073b5a6d2 100644
> --- a/include/uapi/linux/ethtool_netlink.h
> +++ b/include/uapi/linux/ethtool_netlink.h
> @@ -452,6 +452,7 @@ enum {
>  	ETHTOOL_A_TSINFO_UNSPEC,
>  	ETHTOOL_A_TSINFO_HEADER,			/* nest - _A_HEADER_* */
>  	ETHTOOL_A_TSINFO_TIMESTAMPING,			/* bitset */
> +	ETHTOOL_A_TSINFO_FLAG,				/* bitset */
>  	ETHTOOL_A_TSINFO_TX_TYPES,			/* bitset */
>  	ETHTOOL_A_TSINFO_RX_FILTERS,			/* bitset */
>  	ETHTOOL_A_TSINFO_PHC_INDEX,			/* u32 */

You can't add stuff into the middle of an enum or a struct in uAPI.
What's worse for the struct ethtool_ts_info you can't actually add
anything in, period. You can reuse reserved fields but even that
requires extra legwork. If the fields were not previously validated on
input to the kernel (ie. kernel didn't check they are zero) the ioctl
path can't use them, because some application may had been passing in
garbage.
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

RE: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Zulkifli, Muhammad Husaini]

Hi Andrew,

Thanks for reviewing. 
Please see my reply in-line.

> -----Original Message-----
> From: Andrew Lunn <andrew@lunn.ch>
> Sent: Tuesday, 27 September, 2022 11:27 PM
> To: Zulkifli, Muhammad Husaini <muhammad.husaini.zulkifli@intel.com>
> Cc: intel-wired-lan@osuosl.org; netdev@vger.kernel.org;
> davem@davemloft.net; kuba@kernel.org; edumazet@google.com; Gomes,
> Vinicius <vinicius.gomes@intel.com>; Gunasekaran, Aravindhan
> <aravindhan.gunasekaran@intel.com>; Ahmad Tarmizi, Noor Azura
> <noor.azura.ahmad.tarmizi@intel.com>
> Subject: Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP
> packets
> 
> On Tue, Sep 27, 2022 at 09:06:52PM +0800, Muhammad Husaini Zulkifli
> wrote:
> > The HW TX timestamps created by the NIC via socket options can be
> > requested using the current network timestamps generation capability
> > of SOF_TIMESTAMPING_TX_HARDWARE. The most common users of this
> socket
> > flag is PTP, however other packet applications that require tx
> > timestamps might also ask for it.
> >
> > The problem is that, when there is a lot of traffic, there is a high
> > chance that the timestamps for a PTP packet will be lost if both PTP
> > and Non-PTP packets use the same SOF TIMESTAMPING TX HARDWARE
> causing the tx timeout.
> >
> > DMA timestamps through socket options are not currently available to
> > the user. Because if the user wants to, they can configure the
> > hwtstamp config option to use the new introduced DMA Time Stamp flag
> > through the setsockopt().
> >
> > With these additional socket options, users can continue to utilise HW
> > timestamps for PTP while specifying non-PTP packets to use DMA
> > timestamps if the NIC can support them.
> 
> Although this is not actually for PTP, you probably should Cc: the PTP
> maintainer for patches like this.

Thanks. Added Richard in the CC list.

> 
> 	   Andrew

Re: [Intel-wired-lan] [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Zulkifli, Muhammad Husaini]

Hi Andrew,

Thanks for reviewing. 
Please see my reply in-line.

> -----Original Message-----
> From: Andrew Lunn <andrew@lunn.ch>
> Sent: Tuesday, 27 September, 2022 11:27 PM
> To: Zulkifli, Muhammad Husaini <muhammad.husaini.zulkifli@intel.com>
> Cc: intel-wired-lan@osuosl.org; netdev@vger.kernel.org;
> davem@davemloft.net; kuba@kernel.org; edumazet@google.com; Gomes,
> Vinicius <vinicius.gomes@intel.com>; Gunasekaran, Aravindhan
> <aravindhan.gunasekaran@intel.com>; Ahmad Tarmizi, Noor Azura
> <noor.azura.ahmad.tarmizi@intel.com>
> Subject: Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP
> packets
> 
> On Tue, Sep 27, 2022 at 09:06:52PM +0800, Muhammad Husaini Zulkifli
> wrote:
> > The HW TX timestamps created by the NIC via socket options can be
> > requested using the current network timestamps generation capability
> > of SOF_TIMESTAMPING_TX_HARDWARE. The most common users of this
> socket
> > flag is PTP, however other packet applications that require tx
> > timestamps might also ask for it.
> >
> > The problem is that, when there is a lot of traffic, there is a high
> > chance that the timestamps for a PTP packet will be lost if both PTP
> > and Non-PTP packets use the same SOF TIMESTAMPING TX HARDWARE
> causing the tx timeout.
> >
> > DMA timestamps through socket options are not currently available to
> > the user. Because if the user wants to, they can configure the
> > hwtstamp config option to use the new introduced DMA Time Stamp flag
> > through the setsockopt().
> >
> > With these additional socket options, users can continue to utilise HW
> > timestamps for PTP while specifying non-PTP packets to use DMA
> > timestamps if the NIC can support them.
> 
> Although this is not actually for PTP, you probably should Cc: the PTP
> maintainer for patches like this.

Thanks. Added Richard in the CC list.

> 
> 	   Andrew
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

Re: [Intel-wired-lan] [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Zulkifli, Muhammad Husaini]

Hi Jakub,

Thanks for reviewing. 
Please see my reply in-line.

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Wednesday, 28 September, 2022 8:09 AM
> To: Zulkifli, Muhammad Husaini <muhammad.husaini.zulkifli@intel.com>
> Cc: intel-wired-lan@osuosl.org; netdev@vger.kernel.org;
> davem@davemloft.net; edumazet@google.com; Gomes, Vinicius
> <vinicius.gomes@intel.com>; Gunasekaran, Aravindhan
> <aravindhan.gunasekaran@intel.com>; Ahmad Tarmizi, Noor Azura
> <noor.azura.ahmad.tarmizi@intel.com>; Richard Cochran
> <richardcochran@gmail.com>
> Subject: Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP
> packets
> 
> On Tue, 27 Sep 2022 21:06:52 +0800 Muhammad Husaini Zulkifli wrote:
> > The HW TX timestamps created by the NIC via socket options can be
> > requested using the current network timestamps generation capability
> > of SOF_TIMESTAMPING_TX_HARDWARE. The most common users of this
> socket
> > flag is PTP, however other packet applications that require tx
> > timestamps might also ask for it.
> >
> > The problem is that, when there is a lot of traffic, there is a high
> > chance that the timestamps for a PTP packet will be lost if both PTP
> > and Non-PTP packets use the same SOF TIMESTAMPING TX HARDWARE
> causing the tx timeout.
> >
> > DMA timestamps through socket options are not currently available to
> > the user. Because if the user wants to, they can configure the
> > hwtstamp config option to use the new introduced DMA Time Stamp flag
> > through the setsockopt().
> >
> > With these additional socket options, users can continue to utilise HW
> > timestamps for PTP while specifying non-PTP packets to use DMA
> > timestamps if the NIC can support them.
> >
> > Any socket application can be use to verify this.
> > TSN Ref SW application is been used for testing by changing as below:
> 
> Very glad to see this being worked on!
> 
> High level tho, are we assuming that the existing HW timestamps are always
> PTP-quality, i.e. captured when SFD crosses the RS layer, or whatnot? I'm
> afraid some NICs already report PCI stamps as the HW ones.

Yes. HW timestamps always can be assume equivalent to PTP quality.
Could you provide additional information regarding SFD crosses the RS layer?
According to what I observed, The HW Timestamps will be requested if the application side 
specifies tx type = HWTSTAMP TX ON and timestamping flags = SOF TIMESTAMPING TX HARDWARE.
So it depends on how the application used it.

> So the existing HW stamps are conceptually of "any" type, if we want to be
> 100% sure NIC actually stamps at the PHY we'd need another tx_type to
> express that.

Yes, you're right. Are you suggesting that we add a new tx_type to specify
Only MAC/PHY timestamp ? Ex. HWTSTAMP_TX_PHY/MAC_ON.
Sorry about the naming here. Just so you know, the DMA timestamp does not
quite match the PTP's level timestamping. The DMA timestamp will be capture when
DMA request to fetch the data from the memory. 

> 
> Same story on the Rx - what do you plan to do there? We'll need to configure
> the filters per type, but that's likely to mean two new filters, because the
> current one gives no guarantee.

Current I225 HW only allow to retrieve the dma time for TX packets only. 
So as of now based on our HW, on RX side we just requesting rx filter to timestamps any incoming packets.
We always allocating additional bytes in the packet buffer for the receive packets for timestamp. 
It is a 1588 PTP level kind of timestamping accuracy here.

_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

RE: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Zulkifli, Muhammad Husaini]

Hi Jakub,

Thanks for reviewing. 
Please see my reply in-line.

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Wednesday, 28 September, 2022 8:09 AM
> To: Zulkifli, Muhammad Husaini <muhammad.husaini.zulkifli@intel.com>
> Cc: intel-wired-lan@osuosl.org; netdev@vger.kernel.org;
> davem@davemloft.net; edumazet@google.com; Gomes, Vinicius
> <vinicius.gomes@intel.com>; Gunasekaran, Aravindhan
> <aravindhan.gunasekaran@intel.com>; Ahmad Tarmizi, Noor Azura
> <noor.azura.ahmad.tarmizi@intel.com>; Richard Cochran
> <richardcochran@gmail.com>
> Subject: Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP
> packets
> 
> On Tue, 27 Sep 2022 21:06:52 +0800 Muhammad Husaini Zulkifli wrote:
> > The HW TX timestamps created by the NIC via socket options can be
> > requested using the current network timestamps generation capability
> > of SOF_TIMESTAMPING_TX_HARDWARE. The most common users of this
> socket
> > flag is PTP, however other packet applications that require tx
> > timestamps might also ask for it.
> >
> > The problem is that, when there is a lot of traffic, there is a high
> > chance that the timestamps for a PTP packet will be lost if both PTP
> > and Non-PTP packets use the same SOF TIMESTAMPING TX HARDWARE
> causing the tx timeout.
> >
> > DMA timestamps through socket options are not currently available to
> > the user. Because if the user wants to, they can configure the
> > hwtstamp config option to use the new introduced DMA Time Stamp flag
> > through the setsockopt().
> >
> > With these additional socket options, users can continue to utilise HW
> > timestamps for PTP while specifying non-PTP packets to use DMA
> > timestamps if the NIC can support them.
> >
> > Any socket application can be use to verify this.
> > TSN Ref SW application is been used for testing by changing as below:
> 
> Very glad to see this being worked on!
> 
> High level tho, are we assuming that the existing HW timestamps are always
> PTP-quality, i.e. captured when SFD crosses the RS layer, or whatnot? I'm
> afraid some NICs already report PCI stamps as the HW ones.

Yes. HW timestamps always can be assume equivalent to PTP quality.
Could you provide additional information regarding SFD crosses the RS layer?
According to what I observed, The HW Timestamps will be requested if the application side 
specifies tx type = HWTSTAMP TX ON and timestamping flags = SOF TIMESTAMPING TX HARDWARE.
So it depends on how the application used it.

> So the existing HW stamps are conceptually of "any" type, if we want to be
> 100% sure NIC actually stamps at the PHY we'd need another tx_type to
> express that.

Yes, you're right. Are you suggesting that we add a new tx_type to specify
Only MAC/PHY timestamp ? Ex. HWTSTAMP_TX_PHY/MAC_ON.
Sorry about the naming here. Just so you know, the DMA timestamp does not
quite match the PTP's level timestamping. The DMA timestamp will be capture when
DMA request to fetch the data from the memory. 

> 
> Same story on the Rx - what do you plan to do there? We'll need to configure
> the filters per type, but that's likely to mean two new filters, because the
> current one gives no guarantee.

Current I225 HW only allow to retrieve the dma time for TX packets only. 
So as of now based on our HW, on RX side we just requesting rx filter to timestamps any incoming packets.
We always allocating additional bytes in the packet buffer for the receive packets for timestamp. 
It is a 1588 PTP level kind of timestamping accuracy here.

RE: [PATCH v1 1/4] ethtool: Add new hwtstamp flag

[Zulkifli, Muhammad Husaini]

Hi Jakub,

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Wednesday, 28 September, 2022 8:12 AM
> To: Zulkifli, Muhammad Husaini <muhammad.husaini.zulkifli@intel.com>
> Cc: intel-wired-lan@osuosl.org; netdev@vger.kernel.org;
> davem@davemloft.net; edumazet@google.com; Gomes, Vinicius
> <vinicius.gomes@intel.com>; Gunasekaran, Aravindhan
> <aravindhan.gunasekaran@intel.com>; Ahmad Tarmizi, Noor Azura
> <noor.azura.ahmad.tarmizi@intel.com>
> Subject: Re: [PATCH v1 1/4] ethtool: Add new hwtstamp flag
> 
> On Tue, 27 Sep 2022 21:06:53 +0800 Muhammad Husaini Zulkifli wrote:
> > --- a/include/uapi/linux/ethtool.h
> > +++ b/include/uapi/linux/ethtool.h
> > @@ -675,6 +675,7 @@ enum ethtool_link_ext_substate_module {
> >   * @ETH_SS_MSG_CLASSES: debug message class names
> >   * @ETH_SS_WOL_MODES: wake-on-lan modes
> >   * @ETH_SS_SOF_TIMESTAMPING: SOF_TIMESTAMPING_* flags
> > + * @ETH_SS_HWTSTAMP_FLAG: timestamping flags
> >   * @ETH_SS_TS_TX_TYPES: timestamping Tx types
> >   * @ETH_SS_TS_RX_FILTERS: timestamping Rx filters
> >   * @ETH_SS_UDP_TUNNEL_TYPES: UDP tunnel types @@ -700,6 +701,7
> @@
> > enum ethtool_stringset {
> >  	ETH_SS_MSG_CLASSES,
> >  	ETH_SS_WOL_MODES,
> >  	ETH_SS_SOF_TIMESTAMPING,
> > +	ETH_SS_HWTSTAMP_FLAG,
> >  	ETH_SS_TS_TX_TYPES,
> >  	ETH_SS_TS_RX_FILTERS,
> >  	ETH_SS_UDP_TUNNEL_TYPES,
> > @@ -1367,6 +1369,7 @@ struct ethtool_ts_info {
> >  	__u32	cmd;
> >  	__u32	so_timestamping;
> >  	__s32	phc_index;
> > +	__u32	flag;
> >  	__u32	tx_types;
> >  	__u32	tx_reserved[3];
> >  	__u32	rx_filters;
> > diff --git a/include/uapi/linux/ethtool_netlink.h
> > b/include/uapi/linux/ethtool_netlink.h
> > index 408a664fad59..58d073b5a6d2 100644
> > --- a/include/uapi/linux/ethtool_netlink.h
> > +++ b/include/uapi/linux/ethtool_netlink.h
> > @@ -452,6 +452,7 @@ enum {
> >  	ETHTOOL_A_TSINFO_UNSPEC,
> >  	ETHTOOL_A_TSINFO_HEADER,			/* nest -
> _A_HEADER_* */
> >  	ETHTOOL_A_TSINFO_TIMESTAMPING,			/* bitset */
> > +	ETHTOOL_A_TSINFO_FLAG,				/* bitset */
> >  	ETHTOOL_A_TSINFO_TX_TYPES,			/* bitset */
> >  	ETHTOOL_A_TSINFO_RX_FILTERS,			/* bitset */
> >  	ETHTOOL_A_TSINFO_PHC_INDEX,			/* u32 */
> 
> You can't add stuff into the middle of an enum or a struct in uAPI.
> What's worse for the struct ethtool_ts_info you can't actually add anything
> in, period. You can reuse reserved fields but even that requires extra
> legwork. If the fields were not previously validated on input to the kernel (ie.
> kernel didn't check they are zero) the ioctl path can't use them, because
> some application may had been passing in garbage.

Noted. Will make the changes in V2 to move to the end.

Thanks! 

Re: [Intel-wired-lan] [PATCH v1 1/4] ethtool: Add new hwtstamp flag

[Zulkifli, Muhammad Husaini]

Hi Jakub,

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Wednesday, 28 September, 2022 8:12 AM
> To: Zulkifli, Muhammad Husaini <muhammad.husaini.zulkifli@intel.com>
> Cc: intel-wired-lan@osuosl.org; netdev@vger.kernel.org;
> davem@davemloft.net; edumazet@google.com; Gomes, Vinicius
> <vinicius.gomes@intel.com>; Gunasekaran, Aravindhan
> <aravindhan.gunasekaran@intel.com>; Ahmad Tarmizi, Noor Azura
> <noor.azura.ahmad.tarmizi@intel.com>
> Subject: Re: [PATCH v1 1/4] ethtool: Add new hwtstamp flag
> 
> On Tue, 27 Sep 2022 21:06:53 +0800 Muhammad Husaini Zulkifli wrote:
> > --- a/include/uapi/linux/ethtool.h
> > +++ b/include/uapi/linux/ethtool.h
> > @@ -675,6 +675,7 @@ enum ethtool_link_ext_substate_module {
> >   * @ETH_SS_MSG_CLASSES: debug message class names
> >   * @ETH_SS_WOL_MODES: wake-on-lan modes
> >   * @ETH_SS_SOF_TIMESTAMPING: SOF_TIMESTAMPING_* flags
> > + * @ETH_SS_HWTSTAMP_FLAG: timestamping flags
> >   * @ETH_SS_TS_TX_TYPES: timestamping Tx types
> >   * @ETH_SS_TS_RX_FILTERS: timestamping Rx filters
> >   * @ETH_SS_UDP_TUNNEL_TYPES: UDP tunnel types @@ -700,6 +701,7
> @@
> > enum ethtool_stringset {
> >  	ETH_SS_MSG_CLASSES,
> >  	ETH_SS_WOL_MODES,
> >  	ETH_SS_SOF_TIMESTAMPING,
> > +	ETH_SS_HWTSTAMP_FLAG,
> >  	ETH_SS_TS_TX_TYPES,
> >  	ETH_SS_TS_RX_FILTERS,
> >  	ETH_SS_UDP_TUNNEL_TYPES,
> > @@ -1367,6 +1369,7 @@ struct ethtool_ts_info {
> >  	__u32	cmd;
> >  	__u32	so_timestamping;
> >  	__s32	phc_index;
> > +	__u32	flag;
> >  	__u32	tx_types;
> >  	__u32	tx_reserved[3];
> >  	__u32	rx_filters;
> > diff --git a/include/uapi/linux/ethtool_netlink.h
> > b/include/uapi/linux/ethtool_netlink.h
> > index 408a664fad59..58d073b5a6d2 100644
> > --- a/include/uapi/linux/ethtool_netlink.h
> > +++ b/include/uapi/linux/ethtool_netlink.h
> > @@ -452,6 +452,7 @@ enum {
> >  	ETHTOOL_A_TSINFO_UNSPEC,
> >  	ETHTOOL_A_TSINFO_HEADER,			/* nest -
> _A_HEADER_* */
> >  	ETHTOOL_A_TSINFO_TIMESTAMPING,			/* bitset */
> > +	ETHTOOL_A_TSINFO_FLAG,				/* bitset */
> >  	ETHTOOL_A_TSINFO_TX_TYPES,			/* bitset */
> >  	ETHTOOL_A_TSINFO_RX_FILTERS,			/* bitset */
> >  	ETHTOOL_A_TSINFO_PHC_INDEX,			/* u32 */
> 
> You can't add stuff into the middle of an enum or a struct in uAPI.
> What's worse for the struct ethtool_ts_info you can't actually add anything
> in, period. You can reuse reserved fields but even that requires extra
> legwork. If the fields were not previously validated on input to the kernel (ie.
> kernel didn't check they are zero) the ioctl path can't use them, because
> some application may had been passing in garbage.

Noted. Will make the changes in V2 to move to the end.

Thanks! 
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Jakub Kicinski]

On Thu, 29 Sep 2022 02:35:29 +0000 Zulkifli, Muhammad Husaini wrote:
> > High level tho, are we assuming that the existing HW timestamps are always
> > PTP-quality, i.e. captured when SFD crosses the RS layer, or whatnot? I'm
> > afraid some NICs already report PCI stamps as the HW ones.  
> 
> Yes. HW timestamps always can be assume equivalent to PTP quality.
> Could you provide additional information regarding SFD crosses the RS layer?

I mean true PTP timestamps, rather than captured somewhere in the NIC
pipeline or at the DMA engine.

> According to what I observed, The HW Timestamps will be requested if the application side 
> specifies tx type = HWTSTAMP TX ON and timestamping flags = SOF TIMESTAMPING TX HARDWARE.
> So it depends on how the application used it.
> 
> > So the existing HW stamps are conceptually of "any" type, if we want to be
> > 100% sure NIC actually stamps at the PHY we'd need another tx_type to
> > express that.  
> 
> Yes, you're right. Are you suggesting that we add a new tx_type to specify
> Only MAC/PHY timestamp ? Ex. HWTSTAMP_TX_PHY/MAC_ON.

Perhaps we can call them HWTSTAMP_TX_PTP_* ? Was the general time
stamping requirement specified in IEEE 1588 or 802.1 (AS?)? 

Both MAC and PHY can provide the time stamps IIUC, so picking one of
those will not be entirely fortunate. In fact perhaps embedded folks
will use this opportunity to disambiguate the two..

> Sorry about the naming here. Just so you know, the DMA timestamp does not
> quite match the PTP's level timestamping. The DMA timestamp will be capture when
> DMA request to fetch the data from the memory. 
> 
> > 
> > Same story on the Rx - what do you plan to do there? We'll need to configure
> > the filters per type, but that's likely to mean two new filters, because the
> > current one gives no guarantee.  
> 
> Current I225 HW only allow to retrieve the dma time for TX packets only. 
> So as of now based on our HW, on RX side we just requesting rx filter to timestamps any incoming packets.
> We always allocating additional bytes in the packet buffer for the receive packets for timestamp. 
> It is a 1588 PTP level kind of timestamping accuracy here.

I see. I think datacenter NICs can provide DMA stamps for Rx as well.
Intel, Mellanox, Broadcom folks, could you confirm if your NIC can do Rx
DMA stamps?

Re: [Intel-wired-lan] [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Jakub Kicinski]

On Thu, 29 Sep 2022 02:35:29 +0000 Zulkifli, Muhammad Husaini wrote:
> > High level tho, are we assuming that the existing HW timestamps are always
> > PTP-quality, i.e. captured when SFD crosses the RS layer, or whatnot? I'm
> > afraid some NICs already report PCI stamps as the HW ones.  
> 
> Yes. HW timestamps always can be assume equivalent to PTP quality.
> Could you provide additional information regarding SFD crosses the RS layer?

I mean true PTP timestamps, rather than captured somewhere in the NIC
pipeline or at the DMA engine.

> According to what I observed, The HW Timestamps will be requested if the application side 
> specifies tx type = HWTSTAMP TX ON and timestamping flags = SOF TIMESTAMPING TX HARDWARE.
> So it depends on how the application used it.
> 
> > So the existing HW stamps are conceptually of "any" type, if we want to be
> > 100% sure NIC actually stamps at the PHY we'd need another tx_type to
> > express that.  
> 
> Yes, you're right. Are you suggesting that we add a new tx_type to specify
> Only MAC/PHY timestamp ? Ex. HWTSTAMP_TX_PHY/MAC_ON.

Perhaps we can call them HWTSTAMP_TX_PTP_* ? Was the general time
stamping requirement specified in IEEE 1588 or 802.1 (AS?)? 

Both MAC and PHY can provide the time stamps IIUC, so picking one of
those will not be entirely fortunate. In fact perhaps embedded folks
will use this opportunity to disambiguate the two..

> Sorry about the naming here. Just so you know, the DMA timestamp does not
> quite match the PTP's level timestamping. The DMA timestamp will be capture when
> DMA request to fetch the data from the memory. 
> 
> > 
> > Same story on the Rx - what do you plan to do there? We'll need to configure
> > the filters per type, but that's likely to mean two new filters, because the
> > current one gives no guarantee.  
> 
> Current I225 HW only allow to retrieve the dma time for TX packets only. 
> So as of now based on our HW, on RX side we just requesting rx filter to timestamps any incoming packets.
> We always allocating additional bytes in the packet buffer for the receive packets for timestamp. 
> It is a 1588 PTP level kind of timestamping accuracy here.

I see. I think datacenter NICs can provide DMA stamps for Rx as well.
Intel, Mellanox, Broadcom folks, could you confirm if your NIC can do Rx
DMA stamps?
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Gal Pressman]

On 29/09/2022 16:56, Jakub Kicinski wrote:
> On Thu, 29 Sep 2022 02:35:29 +0000 Zulkifli, Muhammad Husaini wrote:
>>> High level tho, are we assuming that the existing HW timestamps are always
>>> PTP-quality, i.e. captured when SFD crosses the RS layer, or whatnot? I'm
>>> afraid some NICs already report PCI stamps as the HW ones.  
>> Yes. HW timestamps always can be assume equivalent to PTP quality.
>> Could you provide additional information regarding SFD crosses the RS layer?
> I mean true PTP timestamps, rather than captured somewhere in the NIC
> pipeline or at the DMA engine.
>
>> According to what I observed, The HW Timestamps will be requested if the application side 
>> specifies tx type = HWTSTAMP TX ON and timestamping flags = SOF TIMESTAMPING TX HARDWARE.
>> So it depends on how the application used it.
>>
>>> So the existing HW stamps are conceptually of "any" type, if we want to be
>>> 100% sure NIC actually stamps at the PHY we'd need another tx_type to
>>> express that.  
>> Yes, you're right. Are you suggesting that we add a new tx_type to specify
>> Only MAC/PHY timestamp ? Ex. HWTSTAMP_TX_PHY/MAC_ON.
> Perhaps we can call them HWTSTAMP_TX_PTP_* ? Was the general time
> stamping requirement specified in IEEE 1588 or 802.1 (AS?)? 
>
> Both MAC and PHY can provide the time stamps IIUC, so picking one of
> those will not be entirely fortunate. In fact perhaps embedded folks
> will use this opportunity to disambiguate the two..
>
>> Sorry about the naming here. Just so you know, the DMA timestamp does not
>> quite match the PTP's level timestamping. The DMA timestamp will be capture when
>> DMA request to fetch the data from the memory. 
>>
>>> Same story on the Rx - what do you plan to do there? We'll need to configure
>>> the filters per type, but that's likely to mean two new filters, because the
>>> current one gives no guarantee.  
>> Current I225 HW only allow to retrieve the dma time for TX packets only. 
>> So as of now based on our HW, on RX side we just requesting rx filter to timestamps any incoming packets.
>> We always allocating additional bytes in the packet buffer for the receive packets for timestamp. 
>> It is a 1588 PTP level kind of timestamping accuracy here.
> I see. I think datacenter NICs can provide DMA stamps for Rx as well.
> Intel, Mellanox, Broadcom folks, could you confirm if your NIC can do Rx
> DMA stamps?

What exactly do you mean by DMA stamps?

Our NIC supports two modes of operation (both TX/RX):
- CQE timestamp (I think that's what you call DMA timestamp), where the
timestamp is written when the completion is being written/generated.
- Port timestamp (MAC timestamp), where the timstamp is written when the
packet is being sent to the wire, or received from the wire. This
doesn't account for the time the packet spent inside the NIC pipeline.

So I believe the answer to your question is yes :).

Re: [Intel-wired-lan] [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Gal Pressman]

On 29/09/2022 16:56, Jakub Kicinski wrote:
> On Thu, 29 Sep 2022 02:35:29 +0000 Zulkifli, Muhammad Husaini wrote:
>>> High level tho, are we assuming that the existing HW timestamps are always
>>> PTP-quality, i.e. captured when SFD crosses the RS layer, or whatnot? I'm
>>> afraid some NICs already report PCI stamps as the HW ones.  
>> Yes. HW timestamps always can be assume equivalent to PTP quality.
>> Could you provide additional information regarding SFD crosses the RS layer?
> I mean true PTP timestamps, rather than captured somewhere in the NIC
> pipeline or at the DMA engine.
>
>> According to what I observed, The HW Timestamps will be requested if the application side 
>> specifies tx type = HWTSTAMP TX ON and timestamping flags = SOF TIMESTAMPING TX HARDWARE.
>> So it depends on how the application used it.
>>
>>> So the existing HW stamps are conceptually of "any" type, if we want to be
>>> 100% sure NIC actually stamps at the PHY we'd need another tx_type to
>>> express that.  
>> Yes, you're right. Are you suggesting that we add a new tx_type to specify
>> Only MAC/PHY timestamp ? Ex. HWTSTAMP_TX_PHY/MAC_ON.
> Perhaps we can call them HWTSTAMP_TX_PTP_* ? Was the general time
> stamping requirement specified in IEEE 1588 or 802.1 (AS?)? 
>
> Both MAC and PHY can provide the time stamps IIUC, so picking one of
> those will not be entirely fortunate. In fact perhaps embedded folks
> will use this opportunity to disambiguate the two..
>
>> Sorry about the naming here. Just so you know, the DMA timestamp does not
>> quite match the PTP's level timestamping. The DMA timestamp will be capture when
>> DMA request to fetch the data from the memory. 
>>
>>> Same story on the Rx - what do you plan to do there? We'll need to configure
>>> the filters per type, but that's likely to mean two new filters, because the
>>> current one gives no guarantee.  
>> Current I225 HW only allow to retrieve the dma time for TX packets only. 
>> So as of now based on our HW, on RX side we just requesting rx filter to timestamps any incoming packets.
>> We always allocating additional bytes in the packet buffer for the receive packets for timestamp. 
>> It is a 1588 PTP level kind of timestamping accuracy here.
> I see. I think datacenter NICs can provide DMA stamps for Rx as well.
> Intel, Mellanox, Broadcom folks, could you confirm if your NIC can do Rx
> DMA stamps?

What exactly do you mean by DMA stamps?

Our NIC supports two modes of operation (both TX/RX):
- CQE timestamp (I think that's what you call DMA timestamp), where the
timestamp is written when the completion is being written/generated.
- Port timestamp (MAC timestamp), where the timstamp is written when the
packet is being sent to the wire, or received from the wire. This
doesn't account for the time the packet spent inside the NIC pipeline.

So I believe the answer to your question is yes :).
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Jakub Kicinski]

On Thu, 29 Sep 2022 17:46:04 +0300 Gal Pressman wrote:
> What exactly do you mean by DMA stamps?
> 
> Our NIC supports two modes of operation (both TX/RX):
> - CQE timestamp (I think that's what you call DMA timestamp), where the
> timestamp is written when the completion is being written/generated.
> - Port timestamp (MAC timestamp), where the timstamp is written when the
> packet is being sent to the wire, or received from the wire. This
> doesn't account for the time the packet spent inside the NIC pipeline.
> 
> So I believe the answer to your question is yes :).

Thanks! I think we should provide the config API for both Tx and Rx,
then.

Re: [Intel-wired-lan] [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Jakub Kicinski]

On Thu, 29 Sep 2022 17:46:04 +0300 Gal Pressman wrote:
> What exactly do you mean by DMA stamps?
> 
> Our NIC supports two modes of operation (both TX/RX):
> - CQE timestamp (I think that's what you call DMA timestamp), where the
> timestamp is written when the completion is being written/generated.
> - Port timestamp (MAC timestamp), where the timstamp is written when the
> packet is being sent to the wire, or received from the wire. This
> doesn't account for the time the packet spent inside the NIC pipeline.
> 
> So I believe the answer to your question is yes :).

Thanks! I think we should provide the config API for both Tx and Rx,
then.
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

RE: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Zulkifli, Muhammad Husaini]

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Thursday, 29 September, 2022 9:56 PM
> To: Zulkifli, Muhammad Husaini <muhammad.husaini.zulkifli@intel.com>
> Cc: intel-wired-lan@osuosl.org; netdev@vger.kernel.org;
> davem@davemloft.net; edumazet@google.com; Gomes, Vinicius
> <vinicius.gomes@intel.com>; Gunasekaran, Aravindhan
> <aravindhan.gunasekaran@intel.com>; Ahmad Tarmizi, Noor Azura
> <noor.azura.ahmad.tarmizi@intel.com>; Richard Cochran
> <richardcochran@gmail.com>; Gal Pressman <gal@nvidia.com>; Saeed
> Mahameed <saeed@kernel.org>; leon@kernel.org; Michael Chan
> <michael.chan@broadcom.com>; Andy Gospodarek <andy@greyhouse.net>
> Subject: Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP
> packets
> 
> On Thu, 29 Sep 2022 02:35:29 +0000 Zulkifli, Muhammad Husaini wrote:
> > > High level tho, are we assuming that the existing HW timestamps are
> > > always PTP-quality, i.e. captured when SFD crosses the RS layer, or
> > > whatnot? I'm afraid some NICs already report PCI stamps as the HW
> ones.
> >
> > Yes. HW timestamps always can be assume equivalent to PTP quality.
> > Could you provide additional information regarding SFD crosses the RS
> layer?
> 
> I mean true PTP timestamps, rather than captured somewhere in the NIC
> pipeline or at the DMA engine.

When SOF_TIMESTAMPING_TX_HARDWARE is been used, we guaranteed a PTP quality
Timestamps (timestamp capture when packet leave the wire upon sensing the SFD).
As of SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH, it is not a PTP quality because
the HW timestamp reported in this case, is a time when the data is DMA'ed into the NIC packet buffer.

> 
> > According to what I observed, The HW Timestamps will be requested if
> > the application side specifies tx type = HWTSTAMP TX ON and
> timestamping flags = SOF TIMESTAMPING TX HARDWARE.
> > So it depends on how the application used it.
> >
> > > So the existing HW stamps are conceptually of "any" type, if we want
> > > to be 100% sure NIC actually stamps at the PHY we'd need another
> > > tx_type to express that.
> >
> > Yes, you're right. Are you suggesting that we add a new tx_type to
> > specify Only MAC/PHY timestamp ? Ex. HWTSTAMP_TX_PHY/MAC_ON.
> 
> Perhaps we can call them HWTSTAMP_TX_PTP_* ? Was the general time
> stamping requirement specified in IEEE 1588 or 802.1 (AS?)?
> 
> Both MAC and PHY can provide the time stamps IIUC, so picking one of those
> will not be entirely fortunate. In fact perhaps embedded folks will use this
> opportunity to disambiguate the two..

With the help of SOF_TIMESTAMPING_TX_HARDWARE, we will get the 
PHY level timestamp(PTP quality) while using SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH,
we will get the timestamp at a point in the NIC pipeline.

Linuxptp application uses SOF_TIMESTAMPING_TX_HARDWARE for their socket option.
And this can guarantee a PTP quality timestamp. 

Can we just use a SOF_TIMESTAMPING to identify which timestamp that we want rather 
than creating a new tx_type?

> 
> > Sorry about the naming here. Just so you know, the DMA timestamp does
> > not quite match the PTP's level timestamping. The DMA timestamp will
> > be capture when DMA request to fetch the data from the memory.
> >
> > >
> > > Same story on the Rx - what do you plan to do there? We'll need to
> > > configure the filters per type, but that's likely to mean two new
> > > filters, because the current one gives no guarantee.
> >
> > Current I225 HW only allow to retrieve the dma time for TX packets only.
> > So as of now based on our HW, on RX side we just requesting rx filter to
> timestamps any incoming packets.
> > We always allocating additional bytes in the packet buffer for the receive
> packets for timestamp.
> > It is a 1588 PTP level kind of timestamping accuracy here.
> 
> I see. I think datacenter NICs can provide DMA stamps for Rx as well.
> Intel, Mellanox, Broadcom folks, could you confirm if your NIC can do Rx
> DMA stamps?

Re: [Intel-wired-lan] [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Zulkifli, Muhammad Husaini]

> -----Original Message-----
> From: Jakub Kicinski <kuba@kernel.org>
> Sent: Thursday, 29 September, 2022 9:56 PM
> To: Zulkifli, Muhammad Husaini <muhammad.husaini.zulkifli@intel.com>
> Cc: intel-wired-lan@osuosl.org; netdev@vger.kernel.org;
> davem@davemloft.net; edumazet@google.com; Gomes, Vinicius
> <vinicius.gomes@intel.com>; Gunasekaran, Aravindhan
> <aravindhan.gunasekaran@intel.com>; Ahmad Tarmizi, Noor Azura
> <noor.azura.ahmad.tarmizi@intel.com>; Richard Cochran
> <richardcochran@gmail.com>; Gal Pressman <gal@nvidia.com>; Saeed
> Mahameed <saeed@kernel.org>; leon@kernel.org; Michael Chan
> <michael.chan@broadcom.com>; Andy Gospodarek <andy@greyhouse.net>
> Subject: Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP
> packets
> 
> On Thu, 29 Sep 2022 02:35:29 +0000 Zulkifli, Muhammad Husaini wrote:
> > > High level tho, are we assuming that the existing HW timestamps are
> > > always PTP-quality, i.e. captured when SFD crosses the RS layer, or
> > > whatnot? I'm afraid some NICs already report PCI stamps as the HW
> ones.
> >
> > Yes. HW timestamps always can be assume equivalent to PTP quality.
> > Could you provide additional information regarding SFD crosses the RS
> layer?
> 
> I mean true PTP timestamps, rather than captured somewhere in the NIC
> pipeline or at the DMA engine.

When SOF_TIMESTAMPING_TX_HARDWARE is been used, we guaranteed a PTP quality
Timestamps (timestamp capture when packet leave the wire upon sensing the SFD).
As of SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH, it is not a PTP quality because
the HW timestamp reported in this case, is a time when the data is DMA'ed into the NIC packet buffer.

> 
> > According to what I observed, The HW Timestamps will be requested if
> > the application side specifies tx type = HWTSTAMP TX ON and
> timestamping flags = SOF TIMESTAMPING TX HARDWARE.
> > So it depends on how the application used it.
> >
> > > So the existing HW stamps are conceptually of "any" type, if we want
> > > to be 100% sure NIC actually stamps at the PHY we'd need another
> > > tx_type to express that.
> >
> > Yes, you're right. Are you suggesting that we add a new tx_type to
> > specify Only MAC/PHY timestamp ? Ex. HWTSTAMP_TX_PHY/MAC_ON.
> 
> Perhaps we can call them HWTSTAMP_TX_PTP_* ? Was the general time
> stamping requirement specified in IEEE 1588 or 802.1 (AS?)?
> 
> Both MAC and PHY can provide the time stamps IIUC, so picking one of those
> will not be entirely fortunate. In fact perhaps embedded folks will use this
> opportunity to disambiguate the two..

With the help of SOF_TIMESTAMPING_TX_HARDWARE, we will get the 
PHY level timestamp(PTP quality) while using SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH,
we will get the timestamp at a point in the NIC pipeline.

Linuxptp application uses SOF_TIMESTAMPING_TX_HARDWARE for their socket option.
And this can guarantee a PTP quality timestamp. 

Can we just use a SOF_TIMESTAMPING to identify which timestamp that we want rather 
than creating a new tx_type?

> 
> > Sorry about the naming here. Just so you know, the DMA timestamp does
> > not quite match the PTP's level timestamping. The DMA timestamp will
> > be capture when DMA request to fetch the data from the memory.
> >
> > >
> > > Same story on the Rx - what do you plan to do there? We'll need to
> > > configure the filters per type, but that's likely to mean two new
> > > filters, because the current one gives no guarantee.
> >
> > Current I225 HW only allow to retrieve the dma time for TX packets only.
> > So as of now based on our HW, on RX side we just requesting rx filter to
> timestamps any incoming packets.
> > We always allocating additional bytes in the packet buffer for the receive
> packets for timestamp.
> > It is a 1588 PTP level kind of timestamping accuracy here.
> 
> I see. I think datacenter NICs can provide DMA stamps for Rx as well.
> Intel, Mellanox, Broadcom folks, could you confirm if your NIC can do Rx
> DMA stamps?
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan

Re: [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Jakub Kicinski]

On Fri, 30 Sep 2022 08:52:38 +0000 Zulkifli, Muhammad Husaini wrote:
> > > Yes. HW timestamps always can be assume equivalent to PTP quality.
> > > Could you provide additional information regarding SFD crosses the RS  
> > > layer?
> > 
> > I mean true PTP timestamps, rather than captured somewhere in the NIC
> > pipeline or at the DMA engine.  
> 
> When SOF_TIMESTAMPING_TX_HARDWARE is been used, we guaranteed a PTP quality
> Timestamps (timestamp capture when packet leave the wire upon sensing the SFD).
> As of SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH, it is not a PTP quality because
> the HW timestamp reported in this case, is a time when the data is
> DMA'ed into the NIC packet buffer.

I understand that _your_ device does it right.
But there are vendors out there who treat SOF_TIMESTAMPING_TX_HARDWARE
as your new SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH.

> > > Yes, you're right. Are you suggesting that we add a new tx_type to
> > > specify Only MAC/PHY timestamp ? Ex. HWTSTAMP_TX_PHY/MAC_ON.  
> > 
> > Perhaps we can call them HWTSTAMP_TX_PTP_* ? Was the general time
> > stamping requirement specified in IEEE 1588 or 802.1 (AS?)?
> > 
> > Both MAC and PHY can provide the time stamps IIUC, so picking one of those
> > will not be entirely fortunate. In fact perhaps embedded folks will use this
> > opportunity to disambiguate the two..  
> 
> With the help of SOF_TIMESTAMPING_TX_HARDWARE, we will get the 
> PHY level timestamp(PTP quality) while using SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH,
> we will get the timestamp at a point in the NIC pipeline.
> 
> Linuxptp application uses SOF_TIMESTAMPING_TX_HARDWARE for their socket option.
> And this can guarantee a PTP quality timestamp. 
> 
> Can we just use a SOF_TIMESTAMPING to identify which timestamp that we want rather 
> than creating a new tx_type?

Hm, perhaps, yes, we can stick to that.

Re: [Intel-wired-lan] [PATCH v1 0/4] Add support for DMA timestamp for non-PTP packets

[Jakub Kicinski]

On Fri, 30 Sep 2022 08:52:38 +0000 Zulkifli, Muhammad Husaini wrote:
> > > Yes. HW timestamps always can be assume equivalent to PTP quality.
> > > Could you provide additional information regarding SFD crosses the RS  
> > > layer?
> > 
> > I mean true PTP timestamps, rather than captured somewhere in the NIC
> > pipeline or at the DMA engine.  
> 
> When SOF_TIMESTAMPING_TX_HARDWARE is been used, we guaranteed a PTP quality
> Timestamps (timestamp capture when packet leave the wire upon sensing the SFD).
> As of SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH, it is not a PTP quality because
> the HW timestamp reported in this case, is a time when the data is
> DMA'ed into the NIC packet buffer.

I understand that _your_ device does it right.
But there are vendors out there who treat SOF_TIMESTAMPING_TX_HARDWARE
as your new SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH.

> > > Yes, you're right. Are you suggesting that we add a new tx_type to
> > > specify Only MAC/PHY timestamp ? Ex. HWTSTAMP_TX_PHY/MAC_ON.  
> > 
> > Perhaps we can call them HWTSTAMP_TX_PTP_* ? Was the general time
> > stamping requirement specified in IEEE 1588 or 802.1 (AS?)?
> > 
> > Both MAC and PHY can provide the time stamps IIUC, so picking one of those
> > will not be entirely fortunate. In fact perhaps embedded folks will use this
> > opportunity to disambiguate the two..  
> 
> With the help of SOF_TIMESTAMPING_TX_HARDWARE, we will get the 
> PHY level timestamp(PTP quality) while using SOF_TIMESTAMPING_TX_HARDWARE_DMA_FETCH,
> we will get the timestamp at a point in the NIC pipeline.
> 
> Linuxptp application uses SOF_TIMESTAMPING_TX_HARDWARE for their socket option.
> And this can guarantee a PTP quality timestamp. 
> 
> Can we just use a SOF_TIMESTAMPING to identify which timestamp that we want rather 
> than creating a new tx_type?

Hm, perhaps, yes, we can stick to that.
_______________________________________________
Intel-wired-lan mailing list
Intel-wired-lan@osuosl.org
https://lists.osuosl.org/mailman/listinfo/intel-wired-lan