[PATCH v2 1/2] nvme-tcp: set the STABLE_WRITES flag when data digests are enabled

[PATCH v2 1/2] nvme-tcp: set the STABLE_WRITES flag when data digests are enabled

[Mikhail Skorzhinskii]

There was a few false alarms sighted on target side about wrong data
digest while performing high throughput load to XFS filesystem shared
through NVMoF TCP.

This flag tells the rest of the kernel to ensure that the data buffer does
not change while the write is in flight. It incurs a performance penalty,
so only enable it when it is actually needed, i.e. when we are calculating
data digests.

Although even with this change in place, ext2 users can steel experience
false positives, as ext2 is not respecting this flag. This may be apply to
vfat as well.

Signed-off-by: Mikhail Skorzhinskii <mskorzhinskiy at solarflare.com>
Signed-off-by: Mike Playle <mplayle at solarflare.com>

---

https://lwn.net/Articles/528031/
https://www.redhat.com/archives/dm-devel/2015-May/msg00158.html
---
 drivers/nvme/host/core.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 5417110cbf1b..f4340dc1d399 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -11,6 +11,7 @@
 #include <linux/hdreg.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
+#include <linux/backing-dev.h>
 #include <linux/list_sort.h>
 #include <linux/slab.h>
 #include <linux/types.h>
@@ -3304,6 +3305,10 @@ static int nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid)
 		goto out_free_ns;
 	}
 
+	if (ctrl->opts->data_digest)
+		ns->queue->backing_dev_info->capabilities
+			|= BDI_CAP_STABLE_WRITES;
+
 	blk_queue_flag_set(QUEUE_FLAG_NONROT, ns->queue);
 	if (ctrl->ops->flags & NVME_F_PCI_P2PDMA)
 		blk_queue_flag_set(QUEUE_FLAG_PCI_P2PDMA, ns->queue);
-- 
2.16.4

[PATCH v2 2/2] nvme-tcp: don't use sendpage for SLAB pages

[Mikhail Skorzhinskii]

According to commit a10674bf2406 ("tcp: detecting the misuse of .sendpage
for Slab objects") and previous discussion, tcp_sendpage should not be
used for pages that is managed by SLAB, as SLAB is not taking page
reference counters into consideration.

This change prevents sendpage calls for payload sending too, although this
is true only for admin commands, so actual data transfer performance
should be untouched.

Signed-off-by: Mikhail Skorzhinskii <mskorzhinskiy at solarflare.com>
---
 drivers/nvme/host/tcp.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 08a2501b9357..c4acdbdca520 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -860,7 +860,12 @@ static int nvme_tcp_try_send_data(struct nvme_tcp_request *req)
 		else
 			flags |= MSG_MORE;
 
-		ret = kernel_sendpage(queue->sock, page, offset, len, flags);
+		/* can't zcopy slab pages */
+		if (unlikely(PageSlab(page))) {
+			ret = sock_no_sendpage(queue->sock, page, offset, len, flags);
+		} else {
+			ret = kernel_sendpage(queue->sock, page, offset, len, flags);
+		}
 		if (ret <= 0)
 			return ret;
 
-- 
2.16.4

[PATCH v2 0/2] Miscellaneous changes for nvme-tcp v2

[Mikhail Skorzhinskii]

A few unrelated changes for NVMoF subsystem, that was found during testing
of NVMoF TCP.

patch 2: found during extensive testing on various filesystems
patch 3: found during testing on debug kernels with VM_DEBUG enabled

v2
  - Moved hint patch as separate patch;
  - Fix commit description in stable writes patch;
  - Simplify check for SLAB check;
  - Remove unnecessary replacments of sendpage calls;

Mike Playle (1):
  nvme-tcp: set the STABLE_WRITES flag when data digests are enabled

Mikhail Skorzhinskii (1):
  nvme-tcp: don't use sendpage for SLAB pages

 drivers/nvme/host/core.c | 5 +++++
 drivers/nvme/host/tcp.c  | 7 ++++++-
 2 files changed, 11 insertions(+), 1 deletion(-)

-- 
2.16.4

[PATCH v2 1/2] nvme-tcp: set the STABLE_WRITES flag when data digests are enabled

[Sagi Grimberg]

Reviewed-by: Sagi Grimberg <sagi at grimberg.me>

[PATCH v2 2/2] nvme-tcp: don't use sendpage for SLAB pages

[Sagi Grimberg]

Reviewed-by: Sagi Grimberg <sagi at grimberg.me>

[PATCH v2 1/2] nvme-tcp: set the STABLE_WRITES flag when data digests are enabled

[Christoph Hellwig]

Thanks,

applied to nvme-5.3.

[PATCH v2 2/2] nvme-tcp: don't use sendpage for SLAB pages

[Christoph Hellwig]

On Mon, Jul 08, 2019@12:31:29PM +0200, Mikhail Skorzhinskii wrote:
> This change prevents sendpage calls for payload sending too, although this
> is true only for admin commands, so actual data transfer performance
> should be untouched.

XFS can also send slab pages for data.

I've applied to patch to nvme-5.3, but removed the above sentence
from the commit log.