linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 1/3] net: introduce helper sendpage_ok() in include/linux/net.h
@ 2020-08-15  7:48 Coly Li
  2020-08-15  7:48 ` [PATCH 2/3] nvme-tcp: check page by sendpage_ok() before calling kernel_sendpage() Coly Li
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: Coly Li @ 2020-08-15  7:48 UTC (permalink / raw)
  To: linux-block, linux-nvme
  Cc: netdev, stable, linux-kernel, Coly Li, Chaitanya Kulkarni,
	Christoph Hellwig, Hannes Reinecke, Jan Kara, Jens Axboe,
	Mikhail Skorzhinskii, Philipp Reisner, Sagi Grimberg,
	Vlastimil Babka

The original problem was from nvme-over-tcp code, who mistakenly uses
kernel_sendpage() to send pages allocated by __get_free_pages() without
__GFP_COMP flag. Such pages don't have refcount (page_count is 0) on
tail pages, sending them by kernel_sendpage() may trigger a kernel panic
from a corrupted kernel heap, because these pages are incorrectly freed
in network stack as page_count 0 pages.

This patch introduces a helper sendpage_ok(), it returns true if the
checking page,
- is not slab page: PageSlab(page) is false.
- has page refcount: page_count(page) is not zero

All drivers who want to send page to remote end by kernel_sendpage()
may use this helper to check whether the page is OK. If the helper does
not return true, the driver should try other non sendpage method (e.g.
sock_no_sendpage()) to handle the page.

Signed-off-by: Coly Li <colyli@suse.de>
Cc: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Jan Kara <jack@suse.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Mikhail Skorzhinskii <mskorzhinskiy@solarflare.com>
Cc: Philipp Reisner <philipp.reisner@linbit.com>
Cc: Sagi Grimberg <sagi@grimberg.me>
Cc: Vlastimil Babka <vbabka@suse.com>
Cc: stable@vger.kernel.org
---
 include/linux/net.h | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/include/linux/net.h b/include/linux/net.h
index d48ff1180879..97e8f1a8a427 100644
--- a/include/linux/net.h
+++ b/include/linux/net.h
@@ -286,6 +286,21 @@ do {									\
 #define net_get_random_once_wait(buf, nbytes)			\
 	get_random_once_wait((buf), (nbytes))
 
+/*
+ * E.g. XFS meta- & log-data is in slab pages, or bcache meta
+ * data pages, or other high order pages allocated by
+ * __get_free_pages() without __GFP_COMP, which have a page_count
+ * of 0 and/or have PageSlab() set. We cannot use send_page for
+ * those, as that does get_page(); put_page(); and would cause
+ * either a VM_BUG directly, or __page_cache_release a page that
+ * would actually still be referenced by someone, leading to some
+ * obscure delayed Oops somewhere else.
+ */
+static inline bool sendpage_ok(struct page *page)
+{
+	return  (!PageSlab(page) && page_count(page) >= 1);
+}
+
 int kernel_sendmsg(struct socket *sock, struct msghdr *msg, struct kvec *vec,
 		   size_t num, size_t len);
 int kernel_sendmsg_locked(struct sock *sk, struct msghdr *msg,
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [PATCH 2/3] nvme-tcp: check page by sendpage_ok() before calling kernel_sendpage()
  2020-08-15  7:48 [PATCH 1/3] net: introduce helper sendpage_ok() in include/linux/net.h Coly Li
@ 2020-08-15  7:48 ` Coly Li
  2020-08-15  7:48 ` [PATCH 3/3] drbd: code cleanup by using sendpage_ok() to check page for kernel_sendpage() Coly Li
  2020-08-16  6:26 ` [PATCH 1/3] net: introduce helper sendpage_ok() in include/linux/net.h kernel test robot
  2 siblings, 0 replies; 4+ messages in thread
From: Coly Li @ 2020-08-15  7:48 UTC (permalink / raw)
  To: linux-block, linux-nvme
  Cc: netdev, stable, linux-kernel, Coly Li, Chaitanya Kulkarni,
	Christoph Hellwig, Hannes Reinecke, Jan Kara, Jens Axboe,
	Mikhail Skorzhinskii, Philipp Reisner, Sagi Grimberg,
	Vlastimil Babka

Currently nvme_tcp_try_send_data() doesn't use kernel_sendpage() to
send slab pages. But for pages allocated by __get_free_pages() without
__GFP_COMP, which also have refcount as 0, they are still sent by
kernel_sendpage() to remote end, this is problematic.

The new introduced helper sendpage_ok() checks both PageSlab tag and
page_count counter, and returns true if the checking page is OK to be
sent by kernel_sendpage().

This patch fixes the page checking issue of nvme_tcp_try_send_data()
with sendpage_ok(). If sendpage_ok() returns true, send this page by
kernel_sendpage(), otherwise use sock_no_sendpage to handle this page.

Signed-off-by: Coly Li <colyli@suse.de>
Cc: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Hannes Reinecke <hare@suse.de>
Cc: Jan Kara <jack@suse.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Mikhail Skorzhinskii <mskorzhinskiy@solarflare.com>
Cc: Philipp Reisner <philipp.reisner@linbit.com>
Cc: Sagi Grimberg <sagi@grimberg.me>
Cc: Vlastimil Babka <vbabka@suse.com>
Cc: stable@vger.kernel.org
---
 drivers/nvme/host/tcp.c | 7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 62fbaecdc960..902fe742762b 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -912,12 +912,11 @@ static int nvme_tcp_try_send_data(struct nvme_tcp_request *req)
 		else
 			flags |= MSG_MORE | MSG_SENDPAGE_NOTLAST;
 
-		/* can't zcopy slab pages */
-		if (unlikely(PageSlab(page))) {
-			ret = sock_no_sendpage(queue->sock, page, offset, len,
+		if (sendpage_ok(page)) {
+			ret = kernel_sendpage(queue->sock, page, offset, len,
 					flags);
 		} else {
-			ret = kernel_sendpage(queue->sock, page, offset, len,
+			ret = sock_no_sendpage(queue->sock, page, offset, len,
 					flags);
 		}
 		if (ret <= 0)
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* [PATCH 3/3] drbd: code cleanup by using sendpage_ok() to check page for kernel_sendpage()
  2020-08-15  7:48 [PATCH 1/3] net: introduce helper sendpage_ok() in include/linux/net.h Coly Li
  2020-08-15  7:48 ` [PATCH 2/3] nvme-tcp: check page by sendpage_ok() before calling kernel_sendpage() Coly Li
@ 2020-08-15  7:48 ` Coly Li
  2020-08-16  6:26 ` [PATCH 1/3] net: introduce helper sendpage_ok() in include/linux/net.h kernel test robot
  2 siblings, 0 replies; 4+ messages in thread
From: Coly Li @ 2020-08-15  7:48 UTC (permalink / raw)
  To: linux-block, linux-nvme
  Cc: netdev, stable, linux-kernel, Coly Li, Philipp Reisner, Sagi Grimberg

In _drbd_send_page() a page is checked by following code before sending
it by kernel_sendpage(),
        (page_count(page) < 1) || PageSlab(page)
If the check is true, this page won't be send by kernel_sendpage() and
handled by sock_no_sendpage().

This kind of check is exactly what macro sendpage_ok() does, which is
introduced into include/linux/net.h to solve a similar send page issue
in nvme-tcp code.

This patch uses macro sendpage_ok() to replace the open coded checks to
page type and refcount in _drbd_send_page(), as a code cleanup.

Signed-off-by: Coly Li <colyli@suse.de>
Cc: Philipp Reisner <philipp.reisner@linbit.com>
Cc: Sagi Grimberg <sagi@grimberg.me>
---
 drivers/block/drbd/drbd_main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index cb687ccdbd96..55dc0c91781e 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -1553,7 +1553,7 @@ static int _drbd_send_page(struct drbd_peer_device *peer_device, struct page *pa
 	 * put_page(); and would cause either a VM_BUG directly, or
 	 * __page_cache_release a page that would actually still be referenced
 	 * by someone, leading to some obscure delayed Oops somewhere else. */
-	if (drbd_disable_sendpage || (page_count(page) < 1) || PageSlab(page))
+	if (drbd_disable_sendpage || !sendpage_ok(page))
 		return _drbd_no_send_page(peer_device, page, offset, size, msg_flags);
 
 	msg_flags |= MSG_NOSIGNAL;
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [PATCH 1/3] net: introduce helper sendpage_ok() in include/linux/net.h
  2020-08-15  7:48 [PATCH 1/3] net: introduce helper sendpage_ok() in include/linux/net.h Coly Li
  2020-08-15  7:48 ` [PATCH 2/3] nvme-tcp: check page by sendpage_ok() before calling kernel_sendpage() Coly Li
  2020-08-15  7:48 ` [PATCH 3/3] drbd: code cleanup by using sendpage_ok() to check page for kernel_sendpage() Coly Li
@ 2020-08-16  6:26 ` kernel test robot
  2 siblings, 0 replies; 4+ messages in thread
From: kernel test robot @ 2020-08-16  6:26 UTC (permalink / raw)
  To: Coly Li, linux-block, linux-nvme
  Cc: kbuild-all, netdev, stable, linux-kernel, Coly Li,
	Chaitanya Kulkarni, Christoph Hellwig, Hannes Reinecke, Jan Kara

[-- Attachment #1: Type: text/plain, Size: 6804 bytes --]

Hi Coly,

I love your patch! Yet something to improve:

[auto build test ERROR on block/for-next]
[also build test ERROR on net/master net-next/master ipvs/master linus/master v5.8 next-20200814]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Coly-Li/net-introduce-helper-sendpage_ok-in-include-linux-net-h/20200816-090533
base:   https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git for-next
config: i386-randconfig-s002-20200816 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-15) 9.3.0
reproduce:
        # apt-get install sparse
        # sparse version: v0.6.2-168-g9554805c-dirty
        # save the attached .config to linux build tree
        make W=1 C=1 CF='-fdiagnostic-prefix -D__CHECK_ENDIAN__' ARCH=i386 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   In file included from net/core/stream.c:18:
   include/linux/net.h: In function 'sendpage_ok':
>> include/linux/net.h:301:30: error: implicit declaration of function 'page_count'; did you mean 'file_count'? [-Werror=implicit-function-declaration]
     301 |  return  (!PageSlab(page) && page_count(page) >= 1);
         |                              ^~~~~~~~~~
         |                              file_count
   In file included from include/linux/mm.h:27,
                    from include/linux/bvec.h:13,
                    from include/linux/skbuff.h:17,
                    from include/linux/tcp.h:17,
                    from net/core/stream.c:20:
   include/linux/page_ref.h: At top level:
>> include/linux/page_ref.h:70:19: error: static declaration of 'page_count' follows non-static declaration
      70 | static inline int page_count(struct page *page)
         |                   ^~~~~~~~~~
   In file included from net/core/stream.c:18:
   include/linux/net.h:301:30: note: previous implicit declaration of 'page_count' was here
     301 |  return  (!PageSlab(page) && page_count(page) >= 1);
         |                              ^~~~~~~~~~
   cc1: some warnings being treated as errors
--
   In file included from net/ipv6/ip6_fib.c:20:
   include/linux/net.h: In function 'sendpage_ok':
>> include/linux/net.h:301:30: error: implicit declaration of function 'page_count'; did you mean 'file_count'? [-Werror=implicit-function-declaration]
     301 |  return  (!PageSlab(page) && page_count(page) >= 1);
         |                              ^~~~~~~~~~
         |                              file_count
   In file included from include/linux/mm.h:27,
                    from include/linux/bvec.h:13,
                    from include/linux/skbuff.h:17,
                    from include/linux/if_ether.h:19,
                    from include/uapi/linux/ethtool.h:19,
                    from include/linux/ethtool.h:18,
                    from include/linux/netdevice.h:37,
                    from net/ipv6/ip6_fib.c:22:
   include/linux/page_ref.h: At top level:
>> include/linux/page_ref.h:70:19: error: static declaration of 'page_count' follows non-static declaration
      70 | static inline int page_count(struct page *page)
         |                   ^~~~~~~~~~
   In file included from net/ipv6/ip6_fib.c:20:
   include/linux/net.h:301:30: note: previous implicit declaration of 'page_count' was here
     301 |  return  (!PageSlab(page) && page_count(page) >= 1);
         |                              ^~~~~~~~~~
   net/ipv6/ip6_fib.c: In function 'fib6_add':
   net/ipv6/ip6_fib.c:1373:25: warning: variable 'pn' set but not used [-Wunused-but-set-variable]
    1373 |  struct fib6_node *fn, *pn = NULL;
         |                         ^~
   cc1: some warnings being treated as errors
--
   In file included from net/ipv6/udp.c:24:
   include/linux/net.h: In function 'sendpage_ok':
>> include/linux/net.h:301:30: error: implicit declaration of function 'page_count'; did you mean 'file_count'? [-Werror=implicit-function-declaration]
     301 |  return  (!PageSlab(page) && page_count(page) >= 1);
         |                              ^~~~~~~~~~
         |                              file_count
   In file included from include/linux/mm.h:27,
                    from include/linux/bvec.h:13,
                    from include/linux/skbuff.h:17,
                    from include/linux/if_ether.h:19,
                    from include/uapi/linux/ethtool.h:19,
                    from include/linux/ethtool.h:18,
                    from include/linux/netdevice.h:37,
                    from net/ipv6/udp.c:26:
   include/linux/page_ref.h: At top level:
>> include/linux/page_ref.h:70:19: error: static declaration of 'page_count' follows non-static declaration
      70 | static inline int page_count(struct page *page)
         |                   ^~~~~~~~~~
   In file included from net/ipv6/udp.c:24:
   include/linux/net.h:301:30: note: previous implicit declaration of 'page_count' was here
     301 |  return  (!PageSlab(page) && page_count(page) >= 1);
         |                              ^~~~~~~~~~
   net/ipv6/udp.c:1029:30: warning: no previous prototype for 'udp_v6_early_demux' [-Wmissing-prototypes]
    1029 | INDIRECT_CALLABLE_SCOPE void udp_v6_early_demux(struct sk_buff *skb)
         |                              ^~~~~~~~~~~~~~~~~~
   net/ipv6/udp.c:1070:29: warning: no previous prototype for 'udpv6_rcv' [-Wmissing-prototypes]
    1070 | INDIRECT_CALLABLE_SCOPE int udpv6_rcv(struct sk_buff *skb)
         |                             ^~~~~~~~~
   cc1: some warnings being treated as errors

vim +301 include/linux/net.h

   283	
   284	#define net_get_random_once(buf, nbytes)			\
   285		get_random_once((buf), (nbytes))
   286	#define net_get_random_once_wait(buf, nbytes)			\
   287		get_random_once_wait((buf), (nbytes))
   288	
   289	/*
   290	 * E.g. XFS meta- & log-data is in slab pages, or bcache meta
   291	 * data pages, or other high order pages allocated by
   292	 * __get_free_pages() without __GFP_COMP, which have a page_count
   293	 * of 0 and/or have PageSlab() set. We cannot use send_page for
   294	 * those, as that does get_page(); put_page(); and would cause
   295	 * either a VM_BUG directly, or __page_cache_release a page that
   296	 * would actually still be referenced by someone, leading to some
   297	 * obscure delayed Oops somewhere else.
   298	 */
   299	static inline bool sendpage_ok(struct page *page)
   300	{
 > 301		return  (!PageSlab(page) && page_count(page) >= 1);
   302	}
   303	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 28723 bytes --]

^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2020-08-16  6:53 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-15  7:48 [PATCH 1/3] net: introduce helper sendpage_ok() in include/linux/net.h Coly Li
2020-08-15  7:48 ` [PATCH 2/3] nvme-tcp: check page by sendpage_ok() before calling kernel_sendpage() Coly Li
2020-08-15  7:48 ` [PATCH 3/3] drbd: code cleanup by using sendpage_ok() to check page for kernel_sendpage() Coly Li
2020-08-16  6:26 ` [PATCH 1/3] net: introduce helper sendpage_ok() in include/linux/net.h kernel test robot

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).