[PATCH v7 0/3] Drivers: hv: vmbus: vmbus_requestor data structure for VMBus hardening

[PATCH v7 0/3] Drivers: hv: vmbus: vmbus_requestor data structure for VMBus hardening

[Andrea Parri (Microsoft)]

Currently, VMbus drivers use pointers into guest memory as request IDs
for interactions with Hyper-V. To be more robust in the face of errors
or malicious behavior from a compromised Hyper-V, avoid exposing
guest memory addresses to Hyper-V. Also avoid Hyper-V giving back a
bad request ID that is then treated as the address of a guest data
structure with no validation. Instead, encapsulate these memory
addresses and provide small integers as request IDs.

The first patch creates the definitions for the data structure, provides
helper methods to generate new IDs and retrieve data, and
allocates/frees the memory needed for vmbus_requestor.

The second and third patches make use of vmbus_requestor to send request
IDs to Hyper-V in storvsc and netvsc respectively.

This version of the series moves the allocation of the requst ID after
the data has been copied into the ring buffer in hv_ringbuffer_write()
to address a race with the request completion path pointed out by Juan.

  Andrea

Cc: James E.J. Bottomley <jejb@linux.ibm.com>
Cc: Martin K. Petersen <martin.petersen@oracle.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Jakub Kicinski <kuba@kernel.org>
Cc: linux-scsi@vger.kernel.org
Cc: netdev@vger.kernel.org

Andres Beltran (3):
  Drivers: hv: vmbus: Add vmbus_requestor data structure for VMBus
    hardening
  scsi: storvsc: Use vmbus_requestor to generate transaction IDs for
    VMBus hardening
  hv_netvsc: Use vmbus_requestor to generate transaction IDs for VMBus
    hardening

 drivers/hv/channel.c              | 174 ++++++++++++++++++++++++++++--
 drivers/hv/hyperv_vmbus.h         |   3 +-
 drivers/hv/ring_buffer.c          |  28 ++++-
 drivers/net/hyperv/hyperv_net.h   |  13 +++
 drivers/net/hyperv/netvsc.c       |  22 ++--
 drivers/net/hyperv/rndis_filter.c |   1 +
 drivers/scsi/storvsc_drv.c        |  26 ++++-
 include/linux/hyperv.h            |  23 ++++
 8 files changed, 272 insertions(+), 18 deletions(-)

-- 
2.25.1

[PATCH v7 1/3] Drivers: hv: vmbus: Add vmbus_requestor data structure for VMBus hardening

[Andrea Parri (Microsoft)]

From: Andres Beltran <lkmlabelt@gmail.com>

Currently, VMbus drivers use pointers into guest memory as request IDs
for interactions with Hyper-V. To be more robust in the face of errors
or malicious behavior from a compromised Hyper-V, avoid exposing
guest memory addresses to Hyper-V. Also avoid Hyper-V giving back a
bad request ID that is then treated as the address of a guest data
structure with no validation. Instead, encapsulate these memory
addresses and provide small integers as request IDs.

Signed-off-by: Andres Beltran <lkmlabelt@gmail.com>
Co-developed-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
Signed-off-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
---
Changes in v7:
	- Move the allocation of the request ID after the data has been
	  copied into the ring buffer.
Changes in v6:
        - Offset request IDs by 1 keeping the original initialization
          code.
Changes in v5:
        - Add support for unsolicited messages sent by the host with a
          request ID of 0.
Changes in v4:
        - Use channel->rqstor_size to check if rqstor has been
          initialized.
Changes in v3:
        - Check that requestor has been initialized in
          vmbus_next_request_id() and vmbus_request_addr().
Changes in v2:
        - Get rid of "rqstor" variable in __vmbus_open().

 drivers/hv/channel.c      | 174 ++++++++++++++++++++++++++++++++++++--
 drivers/hv/hyperv_vmbus.h |   3 +-
 drivers/hv/ring_buffer.c  |  28 +++++-
 include/linux/hyperv.h    |  22 +++++
 4 files changed, 218 insertions(+), 9 deletions(-)

diff --git a/drivers/hv/channel.c b/drivers/hv/channel.c
index 3ebda7707e46a..7f9e43f846fc0 100644
--- a/drivers/hv/channel.c
+++ b/drivers/hv/channel.c
@@ -112,6 +112,70 @@ int vmbus_alloc_ring(struct vmbus_channel *newchannel,
 }
 EXPORT_SYMBOL_GPL(vmbus_alloc_ring);
 
+/**
+ * request_arr_init - Allocates memory for the requestor array. Each slot
+ * keeps track of the next available slot in the array. Initially, each
+ * slot points to the next one (as in a Linked List). The last slot
+ * does not point to anything, so its value is U64_MAX by default.
+ * @size The size of the array
+ */
+static u64 *request_arr_init(u32 size)
+{
+	int i;
+	u64 *req_arr;
+
+	req_arr = kcalloc(size, sizeof(u64), GFP_KERNEL);
+	if (!req_arr)
+		return NULL;
+
+	for (i = 0; i < size - 1; i++)
+		req_arr[i] = i + 1;
+
+	/* Last slot (no more available slots) */
+	req_arr[i] = U64_MAX;
+
+	return req_arr;
+}
+
+/*
+ * vmbus_alloc_requestor - Initializes @rqstor's fields.
+ * Index 0 is the first free slot
+ * @size: Size of the requestor array
+ */
+static int vmbus_alloc_requestor(struct vmbus_requestor *rqstor, u32 size)
+{
+	u64 *rqst_arr;
+	unsigned long *bitmap;
+
+	rqst_arr = request_arr_init(size);
+	if (!rqst_arr)
+		return -ENOMEM;
+
+	bitmap = bitmap_zalloc(size, GFP_KERNEL);
+	if (!bitmap) {
+		kfree(rqst_arr);
+		return -ENOMEM;
+	}
+
+	rqstor->req_arr = rqst_arr;
+	rqstor->req_bitmap = bitmap;
+	rqstor->size = size;
+	rqstor->next_request_id = 0;
+	spin_lock_init(&rqstor->req_lock);
+
+	return 0;
+}
+
+/*
+ * vmbus_free_requestor - Frees memory allocated for @rqstor
+ * @rqstor: Pointer to the requestor struct
+ */
+static void vmbus_free_requestor(struct vmbus_requestor *rqstor)
+{
+	kfree(rqstor->req_arr);
+	bitmap_free(rqstor->req_bitmap);
+}
+
 static int __vmbus_open(struct vmbus_channel *newchannel,
 		       void *userdata, u32 userdatalen,
 		       void (*onchannelcallback)(void *context), void *context)
@@ -132,6 +196,12 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
 	if (newchannel->state != CHANNEL_OPEN_STATE)
 		return -EINVAL;
 
+	/* Create and init requestor */
+	if (newchannel->rqstor_size) {
+		if (vmbus_alloc_requestor(&newchannel->requestor, newchannel->rqstor_size))
+			return -ENOMEM;
+	}
+
 	newchannel->state = CHANNEL_OPENING_STATE;
 	newchannel->onchannel_callback = onchannelcallback;
 	newchannel->channel_callback_context = context;
@@ -228,6 +298,7 @@ static int __vmbus_open(struct vmbus_channel *newchannel,
 error_clean_ring:
 	hv_ringbuffer_cleanup(&newchannel->outbound);
 	hv_ringbuffer_cleanup(&newchannel->inbound);
+	vmbus_free_requestor(&newchannel->requestor);
 	newchannel->state = CHANNEL_OPEN_STATE;
 	return err;
 }
@@ -703,6 +774,9 @@ static int vmbus_close_internal(struct vmbus_channel *channel)
 		channel->ringbuffer_gpadlhandle = 0;
 	}
 
+	if (!ret)
+		vmbus_free_requestor(&channel->requestor);
+
 	return ret;
 }
 
@@ -783,7 +857,7 @@ int vmbus_sendpacket(struct vmbus_channel *channel, void *buffer,
 	/* in 8-bytes granularity */
 	desc.offset8 = sizeof(struct vmpacket_descriptor) >> 3;
 	desc.len8 = (u16)(packetlen_aligned >> 3);
-	desc.trans_id = requestid;
+	desc.trans_id = VMBUS_RQST_ERROR; /* will be updated in hv_ringbuffer_write() */
 
 	bufferlist[0].iov_base = &desc;
 	bufferlist[0].iov_len = sizeof(struct vmpacket_descriptor);
@@ -792,7 +866,7 @@ int vmbus_sendpacket(struct vmbus_channel *channel, void *buffer,
 	bufferlist[2].iov_base = &aligned_data;
 	bufferlist[2].iov_len = (packetlen_aligned - packetlen);
 
-	return hv_ringbuffer_write(channel, bufferlist, num_vecs);
+	return hv_ringbuffer_write(channel, bufferlist, num_vecs, requestid);
 }
 EXPORT_SYMBOL(vmbus_sendpacket);
 
@@ -834,7 +908,7 @@ int vmbus_sendpacket_pagebuffer(struct vmbus_channel *channel,
 	desc.flags = VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED;
 	desc.dataoffset8 = descsize >> 3; /* in 8-bytes granularity */
 	desc.length8 = (u16)(packetlen_aligned >> 3);
-	desc.transactionid = requestid;
+	desc.transactionid = VMBUS_RQST_ERROR; /* will be updated in hv_ringbuffer_write() */
 	desc.reserved = 0;
 	desc.rangecount = pagecount;
 
@@ -851,7 +925,7 @@ int vmbus_sendpacket_pagebuffer(struct vmbus_channel *channel,
 	bufferlist[2].iov_base = &aligned_data;
 	bufferlist[2].iov_len = (packetlen_aligned - packetlen);
 
-	return hv_ringbuffer_write(channel, bufferlist, 3);
+	return hv_ringbuffer_write(channel, bufferlist, 3, requestid);
 }
 EXPORT_SYMBOL_GPL(vmbus_sendpacket_pagebuffer);
 
@@ -878,7 +952,7 @@ int vmbus_sendpacket_mpb_desc(struct vmbus_channel *channel,
 	desc->flags = VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED;
 	desc->dataoffset8 = desc_size >> 3; /* in 8-bytes granularity */
 	desc->length8 = (u16)(packetlen_aligned >> 3);
-	desc->transactionid = requestid;
+	desc->transactionid = VMBUS_RQST_ERROR; /* will be updated in hv_ringbuffer_write() */
 	desc->reserved = 0;
 	desc->rangecount = 1;
 
@@ -889,7 +963,7 @@ int vmbus_sendpacket_mpb_desc(struct vmbus_channel *channel,
 	bufferlist[2].iov_base = &aligned_data;
 	bufferlist[2].iov_len = (packetlen_aligned - packetlen);
 
-	return hv_ringbuffer_write(channel, bufferlist, 3);
+	return hv_ringbuffer_write(channel, bufferlist, 3, requestid);
 }
 EXPORT_SYMBOL_GPL(vmbus_sendpacket_mpb_desc);
 
@@ -937,3 +1011,91 @@ int vmbus_recvpacket_raw(struct vmbus_channel *channel, void *buffer,
 				  buffer_actual_len, requestid, true);
 }
 EXPORT_SYMBOL_GPL(vmbus_recvpacket_raw);
+
+/*
+ * vmbus_next_request_id - Returns a new request id. It is also
+ * the index at which the guest memory address is stored.
+ * Uses a spin lock to avoid race conditions.
+ * @rqstor: Pointer to the requestor struct
+ * @rqst_add: Guest memory address to be stored in the array
+ */
+u64 vmbus_next_request_id(struct vmbus_requestor *rqstor, u64 rqst_addr)
+{
+	unsigned long flags;
+	u64 current_id;
+	const struct vmbus_channel *channel =
+		container_of(rqstor, const struct vmbus_channel, requestor);
+
+	/* Check rqstor has been initialized */
+	if (!channel->rqstor_size)
+		return VMBUS_NO_RQSTOR;
+
+	spin_lock_irqsave(&rqstor->req_lock, flags);
+	current_id = rqstor->next_request_id;
+
+	/* Requestor array is full */
+	if (current_id >= rqstor->size) {
+		spin_unlock_irqrestore(&rqstor->req_lock, flags);
+		return VMBUS_RQST_ERROR;
+	}
+
+	rqstor->next_request_id = rqstor->req_arr[current_id];
+	rqstor->req_arr[current_id] = rqst_addr;
+
+	/* The already held spin lock provides atomicity */
+	bitmap_set(rqstor->req_bitmap, current_id, 1);
+
+	spin_unlock_irqrestore(&rqstor->req_lock, flags);
+
+	/*
+	 * Cannot return an ID of 0, which is reserved for an unsolicited
+	 * message from Hyper-V.
+	 */
+	return current_id + 1;
+}
+EXPORT_SYMBOL_GPL(vmbus_next_request_id);
+
+/*
+ * vmbus_request_addr - Returns the memory address stored at @trans_id
+ * in @rqstor. Uses a spin lock to avoid race conditions.
+ * @rqstor: Pointer to the requestor struct
+ * @trans_id: Request id sent back from Hyper-V. Becomes the requestor's
+ * next request id.
+ */
+u64 vmbus_request_addr(struct vmbus_requestor *rqstor, u64 trans_id)
+{
+	unsigned long flags;
+	u64 req_addr;
+	const struct vmbus_channel *channel =
+		container_of(rqstor, const struct vmbus_channel, requestor);
+
+	/* Check rqstor has been initialized */
+	if (!channel->rqstor_size)
+		return VMBUS_NO_RQSTOR;
+
+	/* Hyper-V can send an unsolicited message with ID of 0 */
+	if (!trans_id)
+		return trans_id;
+
+	spin_lock_irqsave(&rqstor->req_lock, flags);
+
+	/* Data corresponding to trans_id is stored at trans_id - 1 */
+	trans_id--;
+
+	/* Invalid trans_id */
+	if (trans_id >= rqstor->size || !test_bit(trans_id, rqstor->req_bitmap)) {
+		spin_unlock_irqrestore(&rqstor->req_lock, flags);
+		return VMBUS_RQST_ERROR;
+	}
+
+	req_addr = rqstor->req_arr[trans_id];
+	rqstor->req_arr[trans_id] = rqstor->next_request_id;
+	rqstor->next_request_id = trans_id;
+
+	/* The already held spin lock provides atomicity */
+	bitmap_clear(rqstor->req_bitmap, trans_id, 1);
+
+	spin_unlock_irqrestore(&rqstor->req_lock, flags);
+	return req_addr;
+}
+EXPORT_SYMBOL_GPL(vmbus_request_addr);
diff --git a/drivers/hv/hyperv_vmbus.h b/drivers/hv/hyperv_vmbus.h
index 40e2b9f91163c..02f3e89888366 100644
--- a/drivers/hv/hyperv_vmbus.h
+++ b/drivers/hv/hyperv_vmbus.h
@@ -179,7 +179,8 @@ int hv_ringbuffer_init(struct hv_ring_buffer_info *ring_info,
 void hv_ringbuffer_cleanup(struct hv_ring_buffer_info *ring_info);
 
 int hv_ringbuffer_write(struct vmbus_channel *channel,
-			const struct kvec *kv_list, u32 kv_count);
+			const struct kvec *kv_list, u32 kv_count,
+			u64 requestid);
 
 int hv_ringbuffer_read(struct vmbus_channel *channel,
 		       void *buffer, u32 buflen, u32 *buffer_actual_len,
diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c
index 356e22159e834..e55d3c75e148c 100644
--- a/drivers/hv/ring_buffer.c
+++ b/drivers/hv/ring_buffer.c
@@ -248,7 +248,8 @@ void hv_ringbuffer_cleanup(struct hv_ring_buffer_info *ring_info)
 
 /* Write to the ring buffer. */
 int hv_ringbuffer_write(struct vmbus_channel *channel,
-			const struct kvec *kv_list, u32 kv_count)
+			const struct kvec *kv_list, u32 kv_count,
+			u64 requestid)
 {
 	int i;
 	u32 bytes_avail_towrite;
@@ -258,6 +259,8 @@ int hv_ringbuffer_write(struct vmbus_channel *channel,
 	u64 prev_indices;
 	unsigned long flags;
 	struct hv_ring_buffer_info *outring_info = &channel->outbound;
+	struct vmpacket_descriptor *desc = kv_list[0].iov_base;
+	u64 rqst_id = VMBUS_NO_RQSTOR;
 
 	if (channel->rescind)
 		return -ENODEV;
@@ -300,6 +303,22 @@ int hv_ringbuffer_write(struct vmbus_channel *channel,
 						     kv_list[i].iov_len);
 	}
 
+	/*
+	 * Allocate the request ID after the data has been copied into the
+	 * ring buffer.  Once this request ID is allocated, the completion
+	 * path could find the data and free it.
+	 */
+
+	if (desc->flags == VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED) {
+		rqst_id = vmbus_next_request_id(&channel->requestor, requestid);
+		if (rqst_id == VMBUS_RQST_ERROR) {
+			pr_err("No request id available\n");
+			return -EAGAIN;
+		}
+	}
+	desc = hv_get_ring_buffer(outring_info) + old_write;
+	desc->trans_id = (rqst_id == VMBUS_NO_RQSTOR) ? requestid : rqst_id;
+
 	/* Set previous packet start */
 	prev_indices = hv_get_ring_bufferindices(outring_info);
 
@@ -319,8 +338,13 @@ int hv_ringbuffer_write(struct vmbus_channel *channel,
 
 	hv_signal_on_write(old_write, channel);
 
-	if (channel->rescind)
+	if (channel->rescind) {
+		if (rqst_id != VMBUS_NO_RQSTOR) {
+			/* Reclaim request ID to avoid leak of IDs */
+			vmbus_request_addr(&channel->requestor, rqst_id);
+		}
 		return -ENODEV;
+	}
 
 	return 0;
 }
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index 38100e80360ac..45c7831ba0ef1 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -716,6 +716,22 @@ enum vmbus_device_type {
 	HV_UNKNOWN,
 };
 
+/*
+ * Provides request ids for VMBus. Encapsulates guest memory
+ * addresses and stores the next available slot in req_arr
+ * to generate new ids in constant time.
+ */
+struct vmbus_requestor {
+	u64 *req_arr;
+	unsigned long *req_bitmap; /* is a given slot available? */
+	u32 size;
+	u64 next_request_id;
+	spinlock_t req_lock; /* provides atomicity */
+};
+
+#define VMBUS_NO_RQSTOR U64_MAX
+#define VMBUS_RQST_ERROR (U64_MAX - 1)
+
 struct vmbus_device {
 	u16  dev_type;
 	guid_t guid;
@@ -940,8 +956,14 @@ struct vmbus_channel {
 	u32 fuzz_testing_interrupt_delay;
 	u32 fuzz_testing_message_delay;
 
+	/* request/transaction ids for VMBus */
+	struct vmbus_requestor requestor;
+	u32 rqstor_size;
 };
 
+u64 vmbus_next_request_id(struct vmbus_requestor *rqstor, u64 rqst_addr);
+u64 vmbus_request_addr(struct vmbus_requestor *rqstor, u64 trans_id);
+
 static inline bool is_hvsock_channel(const struct vmbus_channel *c)
 {
 	return !!(c->offermsg.offer.chn_flags &
-- 
2.25.1

[PATCH v7 2/3] scsi: storvsc: Use vmbus_requestor to generate transaction IDs for VMBus hardening

[Andrea Parri (Microsoft)]

From: Andres Beltran <lkmlabelt@gmail.com>

Currently, pointers to guest memory are passed to Hyper-V as
transaction IDs in storvsc. In the face of errors or malicious
behavior in Hyper-V, storvsc should not expose or trust the transaction
IDs returned by Hyper-V to be valid guest memory addresses. Instead,
use small integers generated by vmbus_requestor as requests
(transaction) IDs.

Signed-off-by: Andres Beltran <lkmlabelt@gmail.com>
Co-developed-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
Signed-off-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
Cc: "James E.J. Bottomley" <jejb@linux.ibm.com>
Cc: "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: linux-scsi@vger.kernel.org
---
Changes in v7:
	- Move the allocation of the request ID after the data has been
	  copied into the ring buffer (cf. 1/3).
Changes in v2:
        - Add casts to unsigned long to fix warnings on 32bit.

 drivers/scsi/storvsc_drv.c | 26 +++++++++++++++++++++++---
 1 file changed, 23 insertions(+), 3 deletions(-)

diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
index 8f5f5dc863a4a..0b9090d031fc0 100644
--- a/drivers/scsi/storvsc_drv.c
+++ b/drivers/scsi/storvsc_drv.c
@@ -399,6 +399,7 @@ static int storvsc_timeout = 180;
 static struct scsi_transport_template *fc_transport_template;
 #endif
 
+static struct scsi_host_template scsi_driver;
 static void storvsc_on_channel_callback(void *context);
 
 #define STORVSC_MAX_LUNS_PER_TARGET			255
@@ -698,6 +699,12 @@ static void handle_sc_creation(struct vmbus_channel *new_sc)
 
 	memset(&props, 0, sizeof(struct vmstorage_channel_properties));
 
+	/*
+	 * The size of vmbus_requestor is an upper bound on the number of requests
+	 * that can be in-progress at any one time across all channels.
+	 */
+	new_sc->rqstor_size = scsi_driver.can_queue;
+
 	ret = vmbus_open(new_sc,
 			 storvsc_ringbuffer_size,
 			 storvsc_ringbuffer_size,
@@ -1242,9 +1249,17 @@ static void storvsc_on_channel_callback(void *context)
 	foreach_vmbus_pkt(desc, channel) {
 		void *packet = hv_pkt_data(desc);
 		struct storvsc_cmd_request *request;
+		u64 cmd_rqst;
+
+		cmd_rqst = vmbus_request_addr(&channel->requestor,
+					      desc->trans_id);
+		if (cmd_rqst == VMBUS_RQST_ERROR) {
+			dev_err(&device->device,
+				"Incorrect transaction id\n");
+			continue;
+		}
 
-		request = (struct storvsc_cmd_request *)
-			((unsigned long)desc->trans_id);
+		request = (struct storvsc_cmd_request *)(unsigned long)cmd_rqst;
 
 		if (request == &stor_device->init_request ||
 		    request == &stor_device->reset_request) {
@@ -1265,6 +1280,12 @@ static int storvsc_connect_to_vsp(struct hv_device *device, u32 ring_size,
 
 	memset(&props, 0, sizeof(struct vmstorage_channel_properties));
 
+	/*
+	 * The size of vmbus_requestor is an upper bound on the number of requests
+	 * that can be in-progress at any one time across all channels.
+	 */
+	device->channel->rqstor_size = scsi_driver.can_queue;
+
 	ret = vmbus_open(device->channel,
 			 ring_size,
 			 ring_size,
@@ -1572,7 +1593,6 @@ static int storvsc_host_reset_handler(struct scsi_cmnd *scmnd)
 	struct vstor_packet *vstor_packet;
 	int ret, t;
 
-
 	stor_device = get_out_stor_device(device);
 	if (!stor_device)
 		return FAILED;
-- 
2.25.1

[PATCH v7 3/3] hv_netvsc: Use vmbus_requestor to generate transaction IDs for VMBus hardening

[Andrea Parri (Microsoft)]

From: Andres Beltran <lkmlabelt@gmail.com>

Currently, pointers to guest memory are passed to Hyper-V as
transaction IDs in netvsc. In the face of errors or malicious
behavior in Hyper-V, netvsc should not expose or trust the transaction
IDs returned by Hyper-V to be valid guest memory addresses. Instead,
use small integers generated by vmbus_requestor as requests
(transaction) IDs.

Signed-off-by: Andres Beltran <lkmlabelt@gmail.com>
Co-developed-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
Signed-off-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
Cc: "David S. Miller" <davem@davemloft.net>
Cc: Jakub Kicinski <kuba@kernel.org>
Cc: netdev@vger.kernel.org
---
Changes in v7:
	- Move the allocation of the request ID after the data has been
	  copied into the ring buffer (cf. 1/3).
Changes in v2:
        - Add casts to unsigned long to fix warnings on 32bit.
        - Use an inline function to get the requestor size.

 drivers/net/hyperv/hyperv_net.h   | 13 +++++++++++++
 drivers/net/hyperv/netvsc.c       | 22 ++++++++++++++++------
 drivers/net/hyperv/rndis_filter.c |  1 +
 include/linux/hyperv.h            |  1 +
 4 files changed, 31 insertions(+), 6 deletions(-)

diff --git a/drivers/net/hyperv/hyperv_net.h b/drivers/net/hyperv/hyperv_net.h
index 2181d4538ab70..4d2b2d48ff2a1 100644
--- a/drivers/net/hyperv/hyperv_net.h
+++ b/drivers/net/hyperv/hyperv_net.h
@@ -847,6 +847,19 @@ struct nvsp_message {
 
 #define NETVSC_XDP_HDRM 256
 
+#define NETVSC_MIN_OUT_MSG_SIZE (sizeof(struct vmpacket_descriptor) + \
+				 sizeof(struct nvsp_message))
+#define NETVSC_MIN_IN_MSG_SIZE sizeof(struct vmpacket_descriptor)
+
+/* Estimated requestor size:
+ * out_ring_size/min_out_msg_size + in_ring_size/min_in_msg_size
+ */
+static inline u32 netvsc_rqstor_size(unsigned long ringbytes)
+{
+	return ringbytes / NETVSC_MIN_OUT_MSG_SIZE +
+	       ringbytes / NETVSC_MIN_IN_MSG_SIZE;
+}
+
 struct multi_send_data {
 	struct sk_buff *skb; /* skb containing the pkt */
 	struct hv_netvsc_packet *pkt; /* netvsc pkt pending */
diff --git a/drivers/net/hyperv/netvsc.c b/drivers/net/hyperv/netvsc.c
index 41f5cf0bb9971..03e93e3ddbad8 100644
--- a/drivers/net/hyperv/netvsc.c
+++ b/drivers/net/hyperv/netvsc.c
@@ -50,7 +50,7 @@ void netvsc_switch_datapath(struct net_device *ndev, bool vf)
 
 	vmbus_sendpacket(dev->channel, init_pkt,
 			       sizeof(struct nvsp_message),
-			       (unsigned long)init_pkt,
+			       VMBUS_RQST_ID_NO_RESPONSE,
 			       VM_PKT_DATA_INBAND, 0);
 }
 
@@ -163,7 +163,7 @@ static void netvsc_revoke_recv_buf(struct hv_device *device,
 		ret = vmbus_sendpacket(device->channel,
 				       revoke_packet,
 				       sizeof(struct nvsp_message),
-				       (unsigned long)revoke_packet,
+				       VMBUS_RQST_ID_NO_RESPONSE,
 				       VM_PKT_DATA_INBAND, 0);
 		/* If the failure is because the channel is rescinded;
 		 * ignore the failure since we cannot send on a rescinded
@@ -213,7 +213,7 @@ static void netvsc_revoke_send_buf(struct hv_device *device,
 		ret = vmbus_sendpacket(device->channel,
 				       revoke_packet,
 				       sizeof(struct nvsp_message),
-				       (unsigned long)revoke_packet,
+				       VMBUS_RQST_ID_NO_RESPONSE,
 				       VM_PKT_DATA_INBAND, 0);
 
 		/* If the failure is because the channel is rescinded;
@@ -542,7 +542,7 @@ static int negotiate_nvsp_ver(struct hv_device *device,
 
 	ret = vmbus_sendpacket(device->channel, init_packet,
 				sizeof(struct nvsp_message),
-				(unsigned long)init_packet,
+				VMBUS_RQST_ID_NO_RESPONSE,
 				VM_PKT_DATA_INBAND, 0);
 
 	return ret;
@@ -599,7 +599,7 @@ static int netvsc_connect_vsp(struct hv_device *device,
 	/* Send the init request */
 	ret = vmbus_sendpacket(device->channel, init_packet,
 				sizeof(struct nvsp_message),
-				(unsigned long)init_packet,
+				VMBUS_RQST_ID_NO_RESPONSE,
 				VM_PKT_DATA_INBAND, 0);
 	if (ret != 0)
 		goto cleanup;
@@ -680,10 +680,19 @@ static void netvsc_send_tx_complete(struct net_device *ndev,
 				    const struct vmpacket_descriptor *desc,
 				    int budget)
 {
-	struct sk_buff *skb = (struct sk_buff *)(unsigned long)desc->trans_id;
+	struct sk_buff *skb;
 	struct net_device_context *ndev_ctx = netdev_priv(ndev);
 	u16 q_idx = 0;
 	int queue_sends;
+	u64 cmd_rqst;
+
+	cmd_rqst = vmbus_request_addr(&channel->requestor, (u64)desc->trans_id);
+	if (cmd_rqst == VMBUS_RQST_ERROR) {
+		netdev_err(ndev, "Incorrect transaction id\n");
+		return;
+	}
+
+	skb = (struct sk_buff *)(unsigned long)cmd_rqst;
 
 	/* Notify the layer above us */
 	if (likely(skb)) {
@@ -1422,6 +1431,7 @@ struct netvsc_device *netvsc_device_add(struct hv_device *device,
 		       netvsc_poll, NAPI_POLL_WEIGHT);
 
 	/* Open the channel */
+	device->channel->rqstor_size = netvsc_rqstor_size(netvsc_ring_bytes);
 	ret = vmbus_open(device->channel, netvsc_ring_bytes,
 			 netvsc_ring_bytes,  NULL, 0,
 			 netvsc_channel_cb, net_device->chan_table);
diff --git a/drivers/net/hyperv/rndis_filter.c b/drivers/net/hyperv/rndis_filter.c
index b81ceba38218c..10489ba44a090 100644
--- a/drivers/net/hyperv/rndis_filter.c
+++ b/drivers/net/hyperv/rndis_filter.c
@@ -1114,6 +1114,7 @@ static void netvsc_sc_open(struct vmbus_channel *new_sc)
 	/* Set the channel before opening.*/
 	nvchan->channel = new_sc;
 
+	new_sc->rqstor_size = netvsc_rqstor_size(netvsc_ring_bytes);
 	ret = vmbus_open(new_sc, netvsc_ring_bytes,
 			 netvsc_ring_bytes, NULL, 0,
 			 netvsc_channel_cb, nvchan);
diff --git a/include/linux/hyperv.h b/include/linux/hyperv.h
index 45c7831ba0ef1..32c03643577b4 100644
--- a/include/linux/hyperv.h
+++ b/include/linux/hyperv.h
@@ -731,6 +731,7 @@ struct vmbus_requestor {
 
 #define VMBUS_NO_RQSTOR U64_MAX
 #define VMBUS_RQST_ERROR (U64_MAX - 1)
+#define VMBUS_RQST_ID_NO_RESPONSE (U64_MAX - 2)
 
 struct vmbus_device {
 	u16  dev_type;
-- 
2.25.1

RE: [PATCH v7 1/3] Drivers: hv: vmbus: Add vmbus_requestor data structure for VMBus hardening

[Michael Kelley]

From: Andrea Parri (Microsoft) <parri.andrea@gmail.com> Sent: Monday, September 7, 2020 9:19 AM
> 
> From: Andres Beltran <lkmlabelt@gmail.com>
> 
> Currently, VMbus drivers use pointers into guest memory as request IDs
> for interactions with Hyper-V. To be more robust in the face of errors
> or malicious behavior from a compromised Hyper-V, avoid exposing
> guest memory addresses to Hyper-V. Also avoid Hyper-V giving back a
> bad request ID that is then treated as the address of a guest data
> structure with no validation. Instead, encapsulate these memory
> addresses and provide small integers as request IDs.
> 
> Signed-off-by: Andres Beltran <lkmlabelt@gmail.com>
> Co-developed-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
> Signed-off-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
> ---

[snip]

> --- a/drivers/hv/ring_buffer.c
> +++ b/drivers/hv/ring_buffer.c
> @@ -248,7 +248,8 @@ void hv_ringbuffer_cleanup(struct hv_ring_buffer_info *ring_info)
> 
>  /* Write to the ring buffer. */
>  int hv_ringbuffer_write(struct vmbus_channel *channel,
> -			const struct kvec *kv_list, u32 kv_count)
> +			const struct kvec *kv_list, u32 kv_count,
> +			u64 requestid)
>  {
>  	int i;
>  	u32 bytes_avail_towrite;
> @@ -258,6 +259,8 @@ int hv_ringbuffer_write(struct vmbus_channel *channel,
>  	u64 prev_indices;
>  	unsigned long flags;
>  	struct hv_ring_buffer_info *outring_info = &channel->outbound;
> +	struct vmpacket_descriptor *desc = kv_list[0].iov_base;
> +	u64 rqst_id = VMBUS_NO_RQSTOR;
> 
>  	if (channel->rescind)
>  		return -ENODEV;
> @@ -300,6 +303,22 @@ int hv_ringbuffer_write(struct vmbus_channel *channel,
>  						     kv_list[i].iov_len);
>  	}
> 
> +	/*
> +	 * Allocate the request ID after the data has been copied into the
> +	 * ring buffer.  Once this request ID is allocated, the completion
> +	 * path could find the data and free it.
> +	 */
> +
> +	if (desc->flags == VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED) {
> +		rqst_id = vmbus_next_request_id(&channel->requestor, requestid);
> +		if (rqst_id == VMBUS_RQST_ERROR) {
> +			pr_err("No request id available\n");
> +			return -EAGAIN;
> +		}
> +	}
> +	desc = hv_get_ring_buffer(outring_info) + old_write;
> +	desc->trans_id = (rqst_id == VMBUS_NO_RQSTOR) ? requestid : rqst_id;
> +

This is a nit, but the above would be clearer to me if written like this:

	flags = desc->flags;
	if (flags == VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED) {
		rqst_id = vmbus_next_request_id(&channel->requestor, requestid);
		if (rqst_id == VMBUS_RQST_ERROR) {
			pr_err("No request id available\n");
			return -EAGAIN;
		}
	} else {
		rqst_id = requestid;
	}
	desc = hv_get_ring_buffer(outring_info) + old_write;
	desc->trans_id = rqst_id;

The value of the flags field controls what will be used as the value for the
rqst_id.  Having another test to see which value will be used as the trans_id
somehow feels a bit redundant.  And then rqst_id doesn't have to be initialized.

>  	/* Set previous packet start */
>  	prev_indices = hv_get_ring_bufferindices(outring_info);
> 
> @@ -319,8 +338,13 @@ int hv_ringbuffer_write(struct vmbus_channel *channel,
> 
>  	hv_signal_on_write(old_write, channel);
> 
> -	if (channel->rescind)
> +	if (channel->rescind) {
> +		if (rqst_id != VMBUS_NO_RQSTOR) {

Of course, with my proposed change, the above test would also have to be for
the value of the flags field, which actually makes the code a bit more consistent.

Michael

> +			/* Reclaim request ID to avoid leak of IDs */
> +			vmbus_request_addr(&channel->requestor, rqst_id);
> +		}
>  		return -ENODEV;
> +	}
> 
>  	return 0;
>  }

RE: [PATCH v7 2/3] scsi: storvsc: Use vmbus_requestor to generate transaction IDs for VMBus hardening

[Michael Kelley]

From: Andrea Parri (Microsoft) <parri.andrea@gmail.com> Sent: Monday, September 7, 2020 9:19 AM
> 
> From: Andres Beltran <lkmlabelt@gmail.com>
> 
> Currently, pointers to guest memory are passed to Hyper-V as
> transaction IDs in storvsc. In the face of errors or malicious
> behavior in Hyper-V, storvsc should not expose or trust the transaction
> IDs returned by Hyper-V to be valid guest memory addresses. Instead,
> use small integers generated by vmbus_requestor as requests
> (transaction) IDs.
> 
> Signed-off-by: Andres Beltran <lkmlabelt@gmail.com>
> Co-developed-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
> Signed-off-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
> Cc: "James E.J. Bottomley" <jejb@linux.ibm.com>
> Cc: "Martin K. Petersen" <martin.petersen@oracle.com>
> Cc: linux-scsi@vger.kernel.org
> ---
> Changes in v7:
> 	- Move the allocation of the request ID after the data has been
> 	  copied into the ring buffer (cf. 1/3).
> Changes in v2:
>         - Add casts to unsigned long to fix warnings on 32bit.
> 
>  drivers/scsi/storvsc_drv.c | 26 +++++++++++++++++++++++---
>  1 file changed, 23 insertions(+), 3 deletions(-)
> 

Reviewed-by: Michael Kelley <mikelley@microsoft.com>

RE: [PATCH v7 3/3] hv_netvsc: Use vmbus_requestor to generate transaction IDs for VMBus hardening

[Michael Kelley]

From: Andrea Parri (Microsoft) <parri.andrea@gmail.com> Sent: Monday, September 7, 2020 9:19 AM
> 
> From: Andres Beltran <lkmlabelt@gmail.com>
> 
> Currently, pointers to guest memory are passed to Hyper-V as
> transaction IDs in netvsc. In the face of errors or malicious
> behavior in Hyper-V, netvsc should not expose or trust the transaction
> IDs returned by Hyper-V to be valid guest memory addresses. Instead,
> use small integers generated by vmbus_requestor as requests
> (transaction) IDs.
> 
> Signed-off-by: Andres Beltran <lkmlabelt@gmail.com>
> Co-developed-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
> Signed-off-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: Jakub Kicinski <kuba@kernel.org>
> Cc: netdev@vger.kernel.org
> ---
> Changes in v7:
> 	- Move the allocation of the request ID after the data has been
> 	  copied into the ring buffer (cf. 1/3).
> Changes in v2:
>         - Add casts to unsigned long to fix warnings on 32bit.
>         - Use an inline function to get the requestor size.
> 
>  drivers/net/hyperv/hyperv_net.h   | 13 +++++++++++++
>  drivers/net/hyperv/netvsc.c       | 22 ++++++++++++++++------
>  drivers/net/hyperv/rndis_filter.c |  1 +
>  include/linux/hyperv.h            |  1 +
>  4 files changed, 31 insertions(+), 6 deletions(-)
> 

Reviewed-by: Michael Kelley <mikelley@microsoft.com>

Re: [PATCH v7 1/3] Drivers: hv: vmbus: Add vmbus_requestor data structure for VMBus hardening

[Andrea Parri]

> > @@ -300,6 +303,22 @@ int hv_ringbuffer_write(struct vmbus_channel *channel,
> >  						     kv_list[i].iov_len);
> >  	}
> > 
> > +	/*
> > +	 * Allocate the request ID after the data has been copied into the
> > +	 * ring buffer.  Once this request ID is allocated, the completion
> > +	 * path could find the data and free it.
> > +	 */
> > +
> > +	if (desc->flags == VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED) {
> > +		rqst_id = vmbus_next_request_id(&channel->requestor, requestid);
> > +		if (rqst_id == VMBUS_RQST_ERROR) {
> > +			pr_err("No request id available\n");
> > +			return -EAGAIN;
> > +		}
> > +	}
> > +	desc = hv_get_ring_buffer(outring_info) + old_write;
> > +	desc->trans_id = (rqst_id == VMBUS_NO_RQSTOR) ? requestid : rqst_id;
> > +
> 
> This is a nit, but the above would be clearer to me if written like this:
> 
> 	flags = desc->flags;
> 	if (flags == VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED) {
> 		rqst_id = vmbus_next_request_id(&channel->requestor, requestid);
> 		if (rqst_id == VMBUS_RQST_ERROR) {
> 			pr_err("No request id available\n");
> 			return -EAGAIN;
> 		}
> 	} else {
> 		rqst_id = requestid;
> 	}
> 	desc = hv_get_ring_buffer(outring_info) + old_write;
> 	desc->trans_id = rqst_id;
> 
> The value of the flags field controls what will be used as the value for the
> rqst_id.  Having another test to see which value will be used as the trans_id
> somehow feels a bit redundant.  And then rqst_id doesn't have to be initialized.

Agreed, will apply in the next version.


> 
> >  	/* Set previous packet start */
> >  	prev_indices = hv_get_ring_bufferindices(outring_info);
> > 
> > @@ -319,8 +338,13 @@ int hv_ringbuffer_write(struct vmbus_channel *channel,
> > 
> >  	hv_signal_on_write(old_write, channel);
> > 
> > -	if (channel->rescind)
> > +	if (channel->rescind) {
> > +		if (rqst_id != VMBUS_NO_RQSTOR) {
> 
> Of course, with my proposed change, the above test would also have to be for
> the value of the flags field, which actually makes the code a bit more consistent.

Yes, indeed.  Thank you for the review and the suggestion.

  Andrea

RE: [PATCH v7 1/3] Drivers: hv: vmbus: Add vmbus_requestor data structure for VMBus hardening

[Michael Kelley]

From: Andrea Parri <parri.andrea@gmail.com> Sent: Tuesday, September 8, 2020 12:54 AM
> 
> > > @@ -300,6 +303,22 @@ int hv_ringbuffer_write(struct vmbus_channel *channel,
> > >  						     kv_list[i].iov_len);
> > >  	}
> > >
> > > +	/*
> > > +	 * Allocate the request ID after the data has been copied into the
> > > +	 * ring buffer.  Once this request ID is allocated, the completion
> > > +	 * path could find the data and free it.
> > > +	 */
> > > +
> > > +	if (desc->flags == VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED) {
> > > +		rqst_id = vmbus_next_request_id(&channel->requestor, requestid);
> > > +		if (rqst_id == VMBUS_RQST_ERROR) {
> > > +			pr_err("No request id available\n");
> > > +			return -EAGAIN;
> > > +		}
> > > +	}
> > > +	desc = hv_get_ring_buffer(outring_info) + old_write;
> > > +	desc->trans_id = (rqst_id == VMBUS_NO_RQSTOR) ? requestid : rqst_id;
> > > +
> >
> > This is a nit, but the above would be clearer to me if written like this:
> >
> > 	flags = desc->flags;
> > 	if (flags == VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED) {
> > 		rqst_id = vmbus_next_request_id(&channel->requestor, requestid);
> > 		if (rqst_id == VMBUS_RQST_ERROR) {
> > 			pr_err("No request id available\n");
> > 			return -EAGAIN;
> > 		}
> > 	} else {
> > 		rqst_id = requestid;
> > 	}
> > 	desc = hv_get_ring_buffer(outring_info) + old_write;
> > 	desc->trans_id = rqst_id;
> >
> > The value of the flags field controls what will be used as the value for the
> > rqst_id.  Having another test to see which value will be used as the trans_id
> > somehow feels a bit redundant.  And then rqst_id doesn't have to be initialized.
> 
> Agreed, will apply in the next version.
> 

In an offline conversation, Andrea has pointed out that my proposed changes
don't work.  After a second look, I'll agreed that Andrea's code is the best that
can be done, so my comments can be ignored.

Michael

Re: [PATCH v7 1/3] Drivers: hv: vmbus: Add vmbus_requestor data structure for VMBus hardening

[Andrea Parri]

On Mon, Sep 14, 2020 at 05:29:11PM +0000, Michael Kelley wrote:
> From: Andrea Parri <parri.andrea@gmail.com> Sent: Tuesday, September 8, 2020 12:54 AM
> > 
> > > > @@ -300,6 +303,22 @@ int hv_ringbuffer_write(struct vmbus_channel *channel,
> > > >  						     kv_list[i].iov_len);
> > > >  	}
> > > >
> > > > +	/*
> > > > +	 * Allocate the request ID after the data has been copied into the
> > > > +	 * ring buffer.  Once this request ID is allocated, the completion
> > > > +	 * path could find the data and free it.
> > > > +	 */
> > > > +
> > > > +	if (desc->flags == VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED) {
> > > > +		rqst_id = vmbus_next_request_id(&channel->requestor, requestid);
> > > > +		if (rqst_id == VMBUS_RQST_ERROR) {
> > > > +			pr_err("No request id available\n");
> > > > +			return -EAGAIN;
> > > > +		}
> > > > +	}
> > > > +	desc = hv_get_ring_buffer(outring_info) + old_write;
> > > > +	desc->trans_id = (rqst_id == VMBUS_NO_RQSTOR) ? requestid : rqst_id;
> > > > +
> > >
> > > This is a nit, but the above would be clearer to me if written like this:
> > >
> > > 	flags = desc->flags;
> > > 	if (flags == VMBUS_DATA_PACKET_FLAG_COMPLETION_REQUESTED) {
> > > 		rqst_id = vmbus_next_request_id(&channel->requestor, requestid);
> > > 		if (rqst_id == VMBUS_RQST_ERROR) {
> > > 			pr_err("No request id available\n");
> > > 			return -EAGAIN;
> > > 		}
> > > 	} else {
> > > 		rqst_id = requestid;
> > > 	}
> > > 	desc = hv_get_ring_buffer(outring_info) + old_write;
> > > 	desc->trans_id = rqst_id;
> > >
> > > The value of the flags field controls what will be used as the value for the
> > > rqst_id.  Having another test to see which value will be used as the trans_id
> > > somehow feels a bit redundant.  And then rqst_id doesn't have to be initialized.
> > 
> > Agreed, will apply in the next version.
> > 
> 
> In an offline conversation, Andrea has pointed out that my proposed changes
> don't work.  After a second look, I'll agreed that Andrea's code is the best that
> can be done, so my comments can be ignored.

Thanks for the confirmation, Michael.  So, I plan to keep this patch as
is for the next submission of the series (to be submitted shortly...).

Thanks,
  Andrea

RE: [PATCH v7 1/3] Drivers: hv: vmbus: Add vmbus_requestor data structure for VMBus hardening

[Michael Kelley]

From: Andrea Parri (Microsoft) <parri.andrea@gmail.com> Sent: Monday, September 7, 2020 9:19 AM

> 
> From: Andres Beltran <lkmlabelt@gmail.com>
> 
> Currently, VMbus drivers use pointers into guest memory as request IDs
> for interactions with Hyper-V. To be more robust in the face of errors
> or malicious behavior from a compromised Hyper-V, avoid exposing
> guest memory addresses to Hyper-V. Also avoid Hyper-V giving back a
> bad request ID that is then treated as the address of a guest data
> structure with no validation. Instead, encapsulate these memory
> addresses and provide small integers as request IDs.
> 
> Signed-off-by: Andres Beltran <lkmlabelt@gmail.com>
> Co-developed-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
> Signed-off-by: Andrea Parri (Microsoft) <parri.andrea@gmail.com>
> ---
> Changes in v7:
> 	- Move the allocation of the request ID after the data has been
> 	  copied into the ring buffer.
> Changes in v6:
>         - Offset request IDs by 1 keeping the original initialization
>           code.
> Changes in v5:
>         - Add support for unsolicited messages sent by the host with a
>           request ID of 0.
> Changes in v4:
>         - Use channel->rqstor_size to check if rqstor has been
>           initialized.
> Changes in v3:
>         - Check that requestor has been initialized in
>           vmbus_next_request_id() and vmbus_request_addr().
> Changes in v2:
>         - Get rid of "rqstor" variable in __vmbus_open().
> 
>  drivers/hv/channel.c      | 174 ++++++++++++++++++++++++++++++++++++--
>  drivers/hv/hyperv_vmbus.h |   3 +-
>  drivers/hv/ring_buffer.c  |  28 +++++-
>  include/linux/hyperv.h    |  22 +++++
>  4 files changed, 218 insertions(+), 9 deletions(-)
> 

With my previous comments shown to be incorrect, I'm good
with this code.

Reviewed-by: Michael Kelley <mikelley@microsoft.com>