[PATCH v2 0/3] Drivers: hv: vmbus: Miscellaneous improvements

[PATCH v2 0/3] Drivers: hv: vmbus: Miscellaneous improvements

[Andrea Parri]

Hi all,

The patchset:

  - refactors the VMBus negotiation code by introducing the table of
    VMBus protocol versions (patch 1/3),

  - enables VMBus protocol version 4.1, 5.1 and 5.2 (patch 2/3),

  - introduces a module parameter to cap the VMBus protocol versions
    which a guest can negotiate with the hypervisor (patch 3/3).

Thanks,
  Andrea

---

Changes since v1 ([1]):
  - remove the VERSION_INVAL macro (Vitaly Kuznetsov and Dexuan Cui)
  - make the table of VMBus protocol versions static (Dexuan Cui)
  - enable VMBus protocol version 4.1 (Michael Kelley)
  - introduce module parameter to cap the VMBus version (Dexuan Cui)

[1] https://lkml.kernel.org/r/20191007163115.26197-1-parri.andrea@gmail.com

Andrea Parri (3):
  Drivers: hv: vmbus: Introduce table of VMBus protocol versions
  Drivers: hv: vmbus: Enable VMBus protocol versions 4.1, 5.1 and 5.2
  Drivers: hv: vmbus: Add module parameter to cap the VMBus version

 drivers/hv/connection.c          | 68 ++++++++++++++++----------------
 drivers/hv/vmbus_drv.c           |  3 +-
 drivers/net/hyperv/netvsc.c      |  6 +--
 include/linux/hyperv.h           | 12 +++---
 net/vmw_vsock/hyperv_transport.c |  4 +-
 5 files changed, 48 insertions(+), 45 deletions(-)

-- 
2.23.0

[PATCH v2 1/3] Drivers: hv: vmbus: Introduce table of VMBus protocol versions

[Andrea Parri]

The technique used to get the next VMBus version seems increasisly
clumsy as the number of VMBus versions increases.  Performance is
not a concern since this is only done once during system boot; it's
just that we'll end up with more lines of code than is really needed.

As an alternative, introduce a table with the version numbers listed
in order (from the most recent to the oldest).  vmbus_connect() loops
through the versions listed in the table until it gets an accepted
connection or gets to the end of the table (invalid version).

Suggested-by: Michael Kelley <mikelley@microsoft.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
---
 drivers/hv/connection.c | 46 ++++++++++++++---------------------------
 drivers/hv/vmbus_drv.c  |  3 +--
 include/linux/hyperv.h  |  4 ----
 3 files changed, 17 insertions(+), 36 deletions(-)

diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
index 6e4c015783ffc..b1f805426e6b4 100644
--- a/drivers/hv/connection.c
+++ b/drivers/hv/connection.c
@@ -40,29 +40,17 @@ EXPORT_SYMBOL_GPL(vmbus_connection);
 __u32 vmbus_proto_version;
 EXPORT_SYMBOL_GPL(vmbus_proto_version);
 
-static __u32 vmbus_get_next_version(__u32 current_version)
-{
-	switch (current_version) {
-	case (VERSION_WIN7):
-		return VERSION_WS2008;
-
-	case (VERSION_WIN8):
-		return VERSION_WIN7;
-
-	case (VERSION_WIN8_1):
-		return VERSION_WIN8;
-
-	case (VERSION_WIN10):
-		return VERSION_WIN8_1;
-
-	case (VERSION_WIN10_V5):
-		return VERSION_WIN10;
-
-	case (VERSION_WS2008):
-	default:
-		return VERSION_INVAL;
-	}
-}
+/*
+ * Table of VMBus versions listed from newest to oldest.
+ */
+static __u32 vmbus_versions[] = {
+	VERSION_WIN10_V5,
+	VERSION_WIN10,
+	VERSION_WIN8_1,
+	VERSION_WIN8,
+	VERSION_WIN7,
+	VERSION_WS2008
+};
 
 int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)
 {
@@ -169,8 +157,8 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)
  */
 int vmbus_connect(void)
 {
-	int ret = 0;
 	struct vmbus_channel_msginfo *msginfo = NULL;
+	int i, ret = 0;
 	__u32 version;
 
 	/* Initialize the vmbus connection */
@@ -244,20 +232,18 @@ int vmbus_connect(void)
 	 * version.
 	 */
 
-	version = VERSION_CURRENT;
+	for (i = 0; i < ARRAY_SIZE(vmbus_versions); i++) {
+		version = vmbus_versions[i];
 
-	do {
 		ret = vmbus_negotiate_version(msginfo, version);
 		if (ret == -ETIMEDOUT)
 			goto cleanup;
 
 		if (vmbus_connection.conn_state == CONNECTED)
 			break;
+	}
 
-		version = vmbus_get_next_version(version);
-	} while (version != VERSION_INVAL);
-
-	if (version == VERSION_INVAL)
+	if (vmbus_connection.conn_state != CONNECTED)
 		goto cleanup;
 
 	vmbus_proto_version = version;
diff --git a/drivers/hv/vmbus_drv.c b/drivers/hv/vmbus_drv.c
index 391f0b225c9ae..a0cd65ab9a950 100644
--- a/drivers/hv/vmbus_drv.c
+++ b/drivers/hv/vmbus_drv.c
@@ -2215,8 +2215,7 @@ static int vmbus_bus_resume(struct device *dev)
 	 * We only use the 'vmbus_proto_version', which was in use before
 	 * hibernation, to re-negotiate with the host.
 	 */
-	if (vmbus_proto_version == VERSION_INVAL ||
-	    vmbus_proto_version == 0) {
+	if (!vmbus_proto_version) {
 		pr_err("Invalid proto version = 0x%x\n", vmbus_proto_version);
 		return -EINVAL;
 	}
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index b4a017093b697..c08b62dbd151f 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -192,10 +192,6 @@ static inline u32 hv_get_avail_to_write_percent(
 #define VERSION_WIN10	((4 << 16) | (0))
 #define VERSION_WIN10_V5 ((5 << 16) | (0))
 
-#define VERSION_INVAL -1
-
-#define VERSION_CURRENT VERSION_WIN10_V5
-
 /* Make maximum size of pipe payload of 16K */
 #define MAX_PIPE_DATA_PAYLOAD		(sizeof(u8) * 16384)
 
-- 
2.23.0

RE: [PATCH v2 1/3] Drivers: hv: vmbus: Introduce table of VMBus protocol versions

[Michael Kelley]

From: Andrea Parri <parri.andrea@gmail.com> Sent: Thursday, October 10, 2019 8:46 AM
> 
> The technique used to get the next VMBus version seems increasisly
> clumsy as the number of VMBus versions increases.  Performance is
> not a concern since this is only done once during system boot; it's
> just that we'll end up with more lines of code than is really needed.
> 
> As an alternative, introduce a table with the version numbers listed
> in order (from the most recent to the oldest).  vmbus_connect() loops
> through the versions listed in the table until it gets an accepted
> connection or gets to the end of the table (invalid version).
> 
> Suggested-by: Michael Kelley <mikelley@microsoft.com>
> Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
> ---
>  drivers/hv/connection.c | 46 ++++++++++++++---------------------------
>  drivers/hv/vmbus_drv.c  |  3 +--
>  include/linux/hyperv.h  |  4 ----
>  3 files changed, 17 insertions(+), 36 deletions(-)
> 
> @@ -244,20 +232,18 @@ int vmbus_connect(void)
>  	 * version.
>  	 */
> 
> -	version = VERSION_CURRENT;
> +	for (i = 0; i < ARRAY_SIZE(vmbus_versions); i++) {
> +		version = vmbus_versions[i];
> 
> -	do {
>  		ret = vmbus_negotiate_version(msginfo, version);
>  		if (ret == -ETIMEDOUT)
>  			goto cleanup;
> 
>  		if (vmbus_connection.conn_state == CONNECTED)
>  			break;
> +	}
> 
> -		version = vmbus_get_next_version(version);
> -	} while (version != VERSION_INVAL);
> -
> -	if (version == VERSION_INVAL)
> +	if (vmbus_connection.conn_state != CONNECTED)
>  		goto cleanup;
> 

This is a nit, but the loop exit path bugs me.  When a connection
is established, the loop is exited by the "break", and then
conn_state has to be tested again to decide whether the loop
exited due to getting a connection vs. hitting the end of the list.
Slightly cleaner in my mind would be:

	for (i=0; ; i++) {
		if (i == ARRAY_SIZE(vmbus_versions))
			goto cleanup;

		version  = vmbus_versions[i];
		ret = vmbus_negotiate_version(msginfo, version);
		if (ret == -ETIMEDOUT)
			goto cleanup;

		if (vmbus_connection.conn_state == CONNECTED)
			break;
	}

Michael

Re: [PATCH v2 1/3] Drivers: hv: vmbus: Introduce table of VMBus protocol versions

[Andrea Parri]

> > @@ -244,20 +232,18 @@ int vmbus_connect(void)
> >  	 * version.
> >  	 */
> > 
> > -	version = VERSION_CURRENT;
> > +	for (i = 0; i < ARRAY_SIZE(vmbus_versions); i++) {
> > +		version = vmbus_versions[i];
> > 
> > -	do {
> >  		ret = vmbus_negotiate_version(msginfo, version);
> >  		if (ret == -ETIMEDOUT)
> >  			goto cleanup;
> > 
> >  		if (vmbus_connection.conn_state == CONNECTED)
> >  			break;
> > +	}
> > 
> > -		version = vmbus_get_next_version(version);
> > -	} while (version != VERSION_INVAL);
> > -
> > -	if (version == VERSION_INVAL)
> > +	if (vmbus_connection.conn_state != CONNECTED)
> >  		goto cleanup;
> > 
> 
> This is a nit, but the loop exit path bugs me.  When a connection
> is established, the loop is exited by the "break", and then
> conn_state has to be tested again to decide whether the loop
> exited due to getting a connection vs. hitting the end of the list.
> Slightly cleaner in my mind would be:
> 
> 	for (i=0; ; i++) {
> 		if (i == ARRAY_SIZE(vmbus_versions))
> 			goto cleanup;
> 
> 		version  = vmbus_versions[i];
> 		ret = vmbus_negotiate_version(msginfo, version);
> 		if (ret == -ETIMEDOUT)
> 			goto cleanup;
> 
> 		if (vmbus_connection.conn_state == CONNECTED)
> 			break;
> 	}

Indeed.  I applied this locally, for the next iteration.  Thank you for
the review, Michael.

  Andrea

[PATCH v2 2/3] Drivers: hv: vmbus: Enable VMBus protocol versions 4.1, 5.1 and 5.2

[Andrea Parri]

Hyper-V has added VMBus protocol versions 5.1 and 5.2 in recent release
versions.  Allow Linux guests to negotiate these new protocol versions
on versions of Hyper-V that support them.  While on this, also allow
guests to negotiate the VMBus protocol version 4.1 (which was missing).

Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
---
 drivers/hv/connection.c          | 15 +++++++++------
 drivers/net/hyperv/netvsc.c      |  6 +++---
 include/linux/hyperv.h           |  8 +++++++-
 net/vmw_vsock/hyperv_transport.c |  4 ++--
 4 files changed, 21 insertions(+), 12 deletions(-)

diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
index b1f805426e6b4..2f6961ac8c996 100644
--- a/drivers/hv/connection.c
+++ b/drivers/hv/connection.c
@@ -44,8 +44,11 @@ EXPORT_SYMBOL_GPL(vmbus_proto_version);
  * Table of VMBus versions listed from newest to oldest.
  */
 static __u32 vmbus_versions[] = {
+	VERSION_WIN10_V5_2,
+	VERSION_WIN10_V5_1,
 	VERSION_WIN10_V5,
-	VERSION_WIN10,
+	VERSION_WIN10_V4_1,
+	VERSION_WIN10_V4,
 	VERSION_WIN8_1,
 	VERSION_WIN8,
 	VERSION_WIN7,
@@ -68,12 +71,12 @@ int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)
 	msg->vmbus_version_requested = version;
 
 	/*
-	 * VMBus protocol 5.0 (VERSION_WIN10_V5) requires that we must use
-	 * VMBUS_MESSAGE_CONNECTION_ID_4 for the Initiate Contact Message,
+	 * VMBus protocol 5.0 (VERSION_WIN10_V5) and higher require that we must
+	 * use VMBUS_MESSAGE_CONNECTION_ID_4 for the Initiate Contact Message,
 	 * and for subsequent messages, we must use the Message Connection ID
 	 * field in the host-returned Version Response Message. And, with
-	 * VERSION_WIN10_V5, we don't use msg->interrupt_page, but we tell
-	 * the host explicitly that we still use VMBUS_MESSAGE_SINT(2) for
+	 * VERSION_WIN10_V5 and higher, we don't use msg->interrupt_page, but we
+	 * tell the host explicitly that we still use VMBUS_MESSAGE_SINT(2) for
 	 * compatibility.
 	 *
 	 * On old hosts, we should always use VMBUS_MESSAGE_CONNECTION_ID (1).
@@ -399,7 +402,7 @@ int vmbus_post_msg(void *buffer, size_t buflen, bool can_sleep)
 		case HV_STATUS_INVALID_CONNECTION_ID:
 			/*
 			 * See vmbus_negotiate_version(): VMBus protocol 5.0
-			 * requires that we must use
+			 * and higher require that we must use
 			 * VMBUS_MESSAGE_CONNECTION_ID_4 for the Initiate
 			 * Contact message, but on old hosts that only
 			 * support VMBus protocol 4.0 or lower, here we get
diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
index d22a36fc7a7c6..d4c1a776b314a 100644
--- a/drivers/net/hyperv/netvsc.c
+++ b/drivers/net/hyperv/netvsc.c
@@ -624,11 +624,11 @@ void netvsc_device_remove(struct hv_device *device)
 	 * receive buffer GPADL. Do the same for send buffer.
 	 */
 	netvsc_revoke_recv_buf(device, net_device, ndev);
-	if (vmbus_proto_version < VERSION_WIN10)
+	if (vmbus_proto_version < VERSION_WIN10_V4)
 		netvsc_teardown_recv_gpadl(device, net_device, ndev);
 
 	netvsc_revoke_send_buf(device, net_device, ndev);
-	if (vmbus_proto_version < VERSION_WIN10)
+	if (vmbus_proto_version < VERSION_WIN10_V4)
 		netvsc_teardown_send_gpadl(device, net_device, ndev);
 
 	RCU_INIT_POINTER(net_device_ctx->nvdev, NULL);
@@ -650,7 +650,7 @@ void netvsc_device_remove(struct hv_device *device)
 	 * If host is Win2016 or higher then we do the GPADL tear down
 	 * here after VMBus is closed.
 	*/
-	if (vmbus_proto_version >= VERSION_WIN10) {
+	if (vmbus_proto_version >= VERSION_WIN10_V4) {
 		netvsc_teardown_recv_gpadl(device, net_device, ndev);
 		netvsc_teardown_send_gpadl(device, net_device, ndev);
 	}
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index c08b62dbd151f..a4f80e30b0207 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -182,15 +182,21 @@ static inline u32 hv_get_avail_to_write_percent(
  * 2 . 4  (Windows 8)
  * 3 . 0  (Windows 8 R2)
  * 4 . 0  (Windows 10)
+ * 4 . 1  (Windows 10 RS3)
  * 5 . 0  (Newer Windows 10)
+ * 5 . 1  (Windows 10 RS4)
+ * 5 . 2  (Windows Server 2019, RS5)
  */
 
 #define VERSION_WS2008  ((0 << 16) | (13))
 #define VERSION_WIN7    ((1 << 16) | (1))
 #define VERSION_WIN8    ((2 << 16) | (4))
 #define VERSION_WIN8_1    ((3 << 16) | (0))
-#define VERSION_WIN10	((4 << 16) | (0))
+#define VERSION_WIN10_V4 ((4 << 16) | (0))
+#define VERSION_WIN10_V4_1 ((4 << 16) | (1))
 #define VERSION_WIN10_V5 ((5 << 16) | (0))
+#define VERSION_WIN10_V5_1 ((5 << 16) | (1))
+#define VERSION_WIN10_V5_2 ((5 << 16) | (2))
 
 /* Make maximum size of pipe payload of 16K */
 #define MAX_PIPE_DATA_PAYLOAD		(sizeof(u8) * 16384)
diff --git a/net/vmw_vsock/hyperv_transport.c b/net/vmw_vsock/hyperv_transport.c
index c443db7af8d4a..cb0dbae4de14a 100644
--- a/net/vmw_vsock/hyperv_transport.c
+++ b/net/vmw_vsock/hyperv_transport.c
@@ -14,7 +14,7 @@
 #include <net/sock.h>
 #include <net/af_vsock.h>
 
-/* Older (VMBUS version 'VERSION_WIN10' or before) Windows hosts have some
+/* Older (VMBUS version 'VERSION_WIN10_V4' or before) Windows hosts have some
  * stricter requirements on the hv_sock ring buffer size of six 4K pages. Newer
  * hosts don't have this limitation; but, keep the defaults the same for compat.
  */
@@ -955,7 +955,7 @@ static int __init hvs_init(void)
 {
 	int ret;
 
-	if (vmbus_proto_version < VERSION_WIN10)
+	if (vmbus_proto_version < VERSION_WIN10_V4)
 		return -ENODEV;
 
 	ret = vmbus_driver_register(&hvs_drv);
-- 
2.23.0

RE: [PATCH v2 2/3] Drivers: hv: vmbus: Enable VMBus protocol versions 4.1, 5.1 and 5.2

[Michael Kelley]

From: Andrea Parri <parri.andrea@gmail.com> Sent: Thursday, October 10, 2019 8:46 AM
> 
> Hyper-V has added VMBus protocol versions 5.1 and 5.2 in recent release
> versions.  Allow Linux guests to negotiate these new protocol versions
> on versions of Hyper-V that support them.  While on this, also allow
> guests to negotiate the VMBus protocol version 4.1 (which was missing).
> 
> Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
> ---
>  drivers/hv/connection.c          | 15 +++++++++------
>  drivers/net/hyperv/netvsc.c      |  6 +++---
>  include/linux/hyperv.h           |  8 +++++++-
>  net/vmw_vsock/hyperv_transport.c |  4 ++--
>  4 files changed, 21 insertions(+), 12 deletions(-)
> 
> diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
> index c08b62dbd151f..a4f80e30b0207 100644
> --- a/include/linux/hyperv.h
> +++ b/include/linux/hyperv.h
> @@ -182,15 +182,21 @@ static inline u32 hv_get_avail_to_write_percent(
>   * 2 . 4  (Windows 8)
>   * 3 . 0  (Windows 8 R2)
>   * 4 . 0  (Windows 10)
> + * 4 . 1  (Windows 10 RS3)
>   * 5 . 0  (Newer Windows 10)
> + * 5 . 1  (Windows 10 RS4)
> + * 5 . 2  (Windows Server 2019, RS5)
>   */
> 
>  #define VERSION_WS2008  ((0 << 16) | (13))
>  #define VERSION_WIN7    ((1 << 16) | (1))
>  #define VERSION_WIN8    ((2 << 16) | (4))
>  #define VERSION_WIN8_1    ((3 << 16) | (0))
> -#define VERSION_WIN10	((4 << 16) | (0))
> +#define VERSION_WIN10_V4 ((4 << 16) | (0))

I would recommend not changing the symbol name for version 4.0.
The change makes it more consistent with the later VERSION_WIN10_*
symbols, but it doesn't fundamentally add any clarity and I'm not sure
it's worth the churn in the other files that have to be touched. It's a
judgment call, and that's just my input.

> +#define VERSION_WIN10_V4_1 ((4 << 16) | (1))
>  #define VERSION_WIN10_V5 ((5 << 16) | (0))
> +#define VERSION_WIN10_V5_1 ((5 << 16) | (1))
> +#define VERSION_WIN10_V5_2 ((5 << 16) | (2))
> 

Michael

Re: [PATCH v2 2/3] Drivers: hv: vmbus: Enable VMBus protocol versions 4.1, 5.1 and 5.2

[Andrea Parri]

> > @@ -182,15 +182,21 @@ static inline u32 hv_get_avail_to_write_percent(
> >   * 2 . 4  (Windows 8)
> >   * 3 . 0  (Windows 8 R2)
> >   * 4 . 0  (Windows 10)
> > + * 4 . 1  (Windows 10 RS3)
> >   * 5 . 0  (Newer Windows 10)
> > + * 5 . 1  (Windows 10 RS4)
> > + * 5 . 2  (Windows Server 2019, RS5)
> >   */
> > 
> >  #define VERSION_WS2008  ((0 << 16) | (13))
> >  #define VERSION_WIN7    ((1 << 16) | (1))
> >  #define VERSION_WIN8    ((2 << 16) | (4))
> >  #define VERSION_WIN8_1    ((3 << 16) | (0))
> > -#define VERSION_WIN10	((4 << 16) | (0))
> > +#define VERSION_WIN10_V4 ((4 << 16) | (0))
> 
> I would recommend not changing the symbol name for version 4.0.
> The change makes it more consistent with the later VERSION_WIN10_*
> symbols, but it doesn't fundamentally add any clarity and I'm not sure
> it's worth the churn in the other files that have to be touched. It's a
> judgment call, and that's just my input.

My fingers were itching:  ;-) I've reverted this change, following
your recommendation.

Thanks,
  Andrea

[PATCH v2 3/3] Drivers: hv: vmbus: Add module parameter to cap the VMBus version

[Andrea Parri]

Currently, Linux guests negotiate the VMBus version with Hyper-V
and use the highest available VMBus version they can connect to.
This has some drawbacks: by using the highest available version,
certain code paths are never executed and can not be tested when
the guest runs on the newest host.

Add the module parameter "max_version", to upper-bound the VMBus
versions guests can negotiate.

Suggested-by: Dexuan Cui <decui@microsoft.com>
Signed-off-by: Andrea Parri <parri.andrea@gmail.com>
---
 drivers/hv/connection.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/hv/connection.c b/drivers/hv/connection.c
index 2f6961ac8c996..f60d7330ff3fd 100644
--- a/drivers/hv/connection.c
+++ b/drivers/hv/connection.c
@@ -14,6 +14,7 @@
 #include <linux/wait.h>
 #include <linux/delay.h>
 #include <linux/mm.h>
+#include <linux/module.h>
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
 #include <linux/hyperv.h>
@@ -55,6 +56,16 @@ static __u32 vmbus_versions[] = {
 	VERSION_WS2008
 };
 
+/*
+ * Maximal VMBus protocol version guests can negotiate.  Useful to cap the
+ * VMBus version for testing and debugging purpose.
+ */
+static uint max_version = VERSION_WIN10_V5_2;
+
+module_param(max_version, uint, S_IRUGO);
+MODULE_PARM_DESC(max_version,
+		 "Maximal VMBus protocol version which can be negotiated");
+
 int vmbus_negotiate_version(struct vmbus_channel_msginfo *msginfo, u32 version)
 {
 	int ret = 0;
@@ -237,6 +248,8 @@ int vmbus_connect(void)
 
 	for (i = 0; i < ARRAY_SIZE(vmbus_versions); i++) {
 		version = vmbus_versions[i];
+		if (version > max_version)
+			continue;
 
 		ret = vmbus_negotiate_version(msginfo, version);
 		if (ret == -ETIMEDOUT)
-- 
2.23.0