[PATCH] NTB: ntb_perf: Add more debugfs entries for ntb_perf

[PATCH] NTB: ntb_perf: Add more debugfs entries for ntb_perf

[Jiasen Lin]

Currently, read input and output buffer is not supported yet in
debugfs of ntb_perf. We can not confirm whether the output data is
transmitted to the input buffer at peer memory through NTB.

This patch add new entries in debugfs which implement interface to read
a part of input and out buffer. User can dump output and input data at
local and peer system by hexdump command, and then compare them manually.

Signed-off-by: Jiasen Lin <linjiasen@hygon.cn>
---
 drivers/ntb/test/ntb_perf.c | 59 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 59 insertions(+)

diff --git a/drivers/ntb/test/ntb_perf.c b/drivers/ntb/test/ntb_perf.c
index e9b7c2d..338c3ec 100644
--- a/drivers/ntb/test/ntb_perf.c
+++ b/drivers/ntb/test/ntb_perf.c
@@ -106,6 +106,8 @@ MODULE_DESCRIPTION("PCIe NTB Performance Measurement Tool");
 
 #define PERF_BUF_LEN 1024
 
+#define MAX_STR_LENGTH		16
+
 static unsigned long max_mw_size;
 module_param(max_mw_size, ulong, 0644);
 MODULE_PARM_DESC(max_mw_size, "Upper limit of memory window size");
@@ -1227,6 +1229,46 @@ static const struct file_operations perf_dbgfs_info = {
 	.read = perf_dbgfs_read_info
 };
 
+static ssize_t perf_dbgfs_read_inbuf(struct file *filep,
+				    char __user *ubuf,
+				    size_t size, loff_t *offp)
+{
+	struct perf_peer *peer = filep->private_data;
+	size_t buf_size;
+
+	if (!peer->inbuf)
+		return -ENXIO;
+
+	buf_size = min_t(size_t, size, peer->inbuf_size);
+	return simple_read_from_buffer(ubuf, size, offp,
+					peer->inbuf, buf_size);
+}
+
+static const struct file_operations perf_dbgfs_inbuf = {
+	.open = simple_open,
+	.read = perf_dbgfs_read_inbuf,
+};
+
+static ssize_t perf_dbgfs_read_outbuf(struct file *filep,
+				    char __user *ubuf,
+				    size_t size, loff_t *offp)
+{
+	struct perf_peer *peer = filep->private_data;
+	size_t buf_size;
+
+	if (!peer->outbuf)
+		return -ENXIO;
+
+	buf_size = min_t(size_t, size, peer->outbuf_size);
+	return simple_read_from_buffer(ubuf, size, offp,
+					peer->outbuf, buf_size);
+}
+
+static const struct file_operations perf_dbgfs_outbuf = {
+	.open = simple_open,
+	.read = perf_dbgfs_read_outbuf,
+};
+
 static ssize_t perf_dbgfs_read_run(struct file *filep, char __user *ubuf,
 				   size_t size, loff_t *offp)
 {
@@ -1318,6 +1360,9 @@ static const struct file_operations perf_dbgfs_tcnt = {
 static void perf_setup_dbgfs(struct perf_ctx *perf)
 {
 	struct pci_dev *pdev = perf->ntb->pdev;
+	struct perf_peer *peer;
+	int pidx;
+	char name[MAX_STR_LENGTH];
 
 	perf->dbgfs_dir = debugfs_create_dir(pci_name(pdev), perf_dbgfs_topdir);
 	if (!perf->dbgfs_dir) {
@@ -1334,6 +1379,20 @@ static void perf_setup_dbgfs(struct perf_ctx *perf)
 	debugfs_create_file("threads_count", 0600, perf->dbgfs_dir, perf,
 			    &perf_dbgfs_tcnt);
 
+	for (pidx = 0; pidx < perf->pcnt; pidx++) {
+		peer = &perf->peers[pidx];
+		if (!peer)
+			continue;
+		memset(name, 0, sizeof(name));
+		snprintf(name, sizeof(name), "%s_%u", "inbuf_info", pidx);
+		debugfs_create_file(name, 0600, perf->dbgfs_dir, peer,
+					&perf_dbgfs_inbuf);
+
+		memset(name, 0, sizeof(name));
+		snprintf(name, sizeof(name), "%s_%u", "outbuf_info", pidx);
+		debugfs_create_file(name, 0600, perf->dbgfs_dir, peer,
+					&perf_dbgfs_outbuf);
+	}
 	/* They are made read-only for test exec safety and integrity */
 	debugfs_create_u8("chunk_order", 0500, perf->dbgfs_dir, &chunk_order);
 
-- 
2.7.4

Re: [PATCH] NTB: ntb_perf: Add more debugfs entries for ntb_perf

[Logan Gunthorpe]

On 2019-12-16 7:42 p.m., Jiasen Lin wrote:
> Currently, read input and output buffer is not supported yet in
> debugfs of ntb_perf. We can not confirm whether the output data is
> transmitted to the input buffer at peer memory through NTB.
> 
> This patch add new entries in debugfs which implement interface to read
> a part of input and out buffer. User can dump output and input data at
> local and peer system by hexdump command, and then compare them manually.

Do we even initialize inbuf and outbuf? Probably not a good idea to
expose them to userspace if it's not initialized.

Really, ntb_tool should be used to check if memory windows work,
ntb_perf is just to see the maximum transfer rate.

Logan

Re: [PATCH] NTB: ntb_perf: Add more debugfs entries for ntb_perf

[Jiasen Lin]

On 2019/12/18 0:42, Logan Gunthorpe wrote:
> 
> 
> On 2019-12-16 7:42 p.m., Jiasen Lin wrote:
>> Currently, read input and output buffer is not supported yet in
>> debugfs of ntb_perf. We can not confirm whether the output data is
>> transmitted to the input buffer at peer memory through NTB.
>>
>> This patch add new entries in debugfs which implement interface to read
>> a part of input and out buffer. User can dump output and input data at
>> local and peer system by hexdump command, and then compare them manually.
> 
> Do we even initialize inbuf and outbuf? Probably not a good idea to
> expose them to userspace if it's not initialized.
> 

Good catch! Input buffer(peer->inbuf) should be initialized after call
dma_alloc_coherent in perf_setup_inbuf, but output buffer(peer->outbuf)
that point to NTB BARx MMIO address can not be initialized. The address 
of memory request that fall in NTB memory windown will be translated 
into an address that point to input buffer at peer memory system through 
NTB XLAT register in ntb_perf. In other words, only input buffer should 
be initialized.
Will fix it for v2 as a separate patch.

> Really, ntb_tool should be used to check if memory windows work,
> ntb_perf is just to see the maximum transfer rate.
> 

Yes, you are right! ntb_tool could be used to check memory window
by hexdmup mw and peer_mw after write mw_trans and peer_mw_trans,
but this is only one ntb client be probed although we can insmod both
ntb_tool.ko and ntb_perf.ko, That's to say, only one modulue can create
files in debugFS. So, I suggest that add debugfs entries that can read
input and output buffer in ntb_perf.

Thanks,
Jiasen Lin

> Logan
>