From: boqun.feng@gmail.com
To: Michael Kelley <mikelley@microsoft.com>
Cc: "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
"linux-input@vger.kernel.org" <linux-input@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
KY Srinivasan <kys@microsoft.com>,
Haiyang Zhang <haiyangz@microsoft.com>,
Stephen Hemminger <sthemmin@microsoft.com>,
Wei Liu <wei.liu@kernel.org>, Jiri Kosina <jikos@kernel.org>,
Benjamin Tissoires <benjamin.tissoires@redhat.com>,
Dmitry Torokhov <dmitry.torokhov@gmail.com>,
"David S. Miller" <davem@davemloft.net>,
Jakub Kicinski <kuba@kernel.org>,
"James E.J. Bottomley" <jejb@linux.ibm.com>,
"Martin K. Petersen" <martin.petersen@oracle.com>
Subject: Re: [RFC 11/11] scsi: storvsc: Support PAGE_SIZE larger than 4K
Date: Thu, 23 Jul 2020 09:51:49 +0800 [thread overview]
Message-ID: <20200723015149.GE35358@debian-boqun.qqnc3lrjykvubdpftowmye0fmh.lx.internal.cloudapp.net> (raw)
In-Reply-To: <MW2PR2101MB1052B072CA85F82B74BE799FD7760@MW2PR2101MB1052.namprd21.prod.outlook.com>
On Thu, Jul 23, 2020 at 12:13:07AM +0000, Michael Kelley wrote:
> From: Boqun Feng <boqun.feng@gmail.com> Sent: Monday, July 20, 2020 6:42 PM
> >
> > Hyper-V always use 4k page size (HV_HYP_PAGE_SIZE), so when
> > communicating with Hyper-V, a guest should always use HV_HYP_PAGE_SIZE
> > as the unit for page related data. For storvsc, the data is
> > vmbus_packet_mpb_array. And since in scsi_cmnd, sglist of pages (in unit
> > of PAGE_SIZE) is used, we need convert pages in the sglist of scsi_cmnd
> > into Hyper-V pages in vmbus_packet_mpb_array.
> >
> > This patch does the conversion by dividing pages in sglist into Hyper-V
> > pages, offset and indexes in vmbus_packet_mpb_array are recalculated
> > accordingly.
> >
> > Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
> > ---
> > drivers/scsi/storvsc_drv.c | 27 +++++++++++++++++++++------
> > 1 file changed, 21 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/scsi/storvsc_drv.c b/drivers/scsi/storvsc_drv.c
> > index fb41636519ee..c54d25f279bc 100644
> > --- a/drivers/scsi/storvsc_drv.c
> > +++ b/drivers/scsi/storvsc_drv.c
> > @@ -1561,7 +1561,7 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct
> > scsi_cmnd *scmnd)
> > struct hv_host_device *host_dev = shost_priv(host);
> > struct hv_device *dev = host_dev->dev;
> > struct storvsc_cmd_request *cmd_request = scsi_cmd_priv(scmnd);
> > - int i;
> > + int i, j, k;
> > struct scatterlist *sgl;
> > unsigned int sg_count = 0;
> > struct vmscsi_request *vm_srb;
> > @@ -1569,6 +1569,8 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct
> > scsi_cmnd *scmnd)
> > struct vmbus_packet_mpb_array *payload;
> > u32 payload_sz;
> > u32 length;
> > + int subpage_idx = 0;
> > + unsigned int hvpg_count = 0;
> >
> > if (vmstor_proto_version <= VMSTOR_PROTO_VERSION_WIN8) {
> > /*
> > @@ -1643,23 +1645,36 @@ static int storvsc_queuecommand(struct Scsi_Host *host, struct
> > scsi_cmnd *scmnd)
> > payload_sz = sizeof(cmd_request->mpb);
> >
> > if (sg_count) {
> > - if (sg_count > MAX_PAGE_BUFFER_COUNT) {
> > + hvpg_count = sg_count * (PAGE_SIZE / HV_HYP_PAGE_SIZE);
>
> The above calculation doesn't take into account the offset in the
> first sglist or the overall length of the transfer, so the value of hvpg_count
> could be quite a bit bigger than it needs to be. For example, with a 64K
> page size and an 8 Kbyte transfer size that starts at offset 60K in the
> first page, hvpg_count will be 32 when it really only needs to be 2.
>
> The nested loops below that populate the pfn_array take the
> offset into account when starting, so that's good. But it will potentially
> leave allocated entries unused. Furthermore, the nested loops could
> terminate early when enough Hyper-V size pages are mapped to PFNs
> based on the length of the transfer, even if all of the last guest size
> page has not been mapped to PFNs. Like the offset at the beginning of
> first guest size page in the sglist, there's potentially an unused portion
> at the end of the last guest size page in the sglist.
>
Good point. I think we could calculate the exact hvpg_count as follow:
hvpg_count = 0;
cur_sgl = sgl;
for (i = 0; i < sg_count; i++) {
hvpg_count += HVPFN_UP(cur_sg->length)
cur_sgl = sg_next(cur_sgl);
}
> > + if (hvpg_count > MAX_PAGE_BUFFER_COUNT) {
> >
> > - payload_sz = (sg_count * sizeof(u64) +
> > + payload_sz = (hvpg_count * sizeof(u64) +
> > sizeof(struct vmbus_packet_mpb_array));
> > payload = kzalloc(payload_sz, GFP_ATOMIC);
> > if (!payload)
> > return SCSI_MLQUEUE_DEVICE_BUSY;
> > }
> >
> > + /*
> > + * sgl is a list of PAGEs, and payload->range.pfn_array
> > + * expects the page number in the unit of HV_HYP_PAGE_SIZE (the
> > + * page size that Hyper-V uses, so here we need to divide PAGEs
> > + * into HV_HYP_PAGE in case that PAGE_SIZE > HV_HYP_PAGE_SIZE.
> > + */
> > payload->range.len = length;
> > - payload->range.offset = sgl[0].offset;
> > + payload->range.offset = sgl[0].offset & ~HV_HYP_PAGE_MASK;
> > + subpage_idx = sgl[0].offset >> HV_HYP_PAGE_SHIFT;
> >
> > cur_sgl = sgl;
> > + k = 0;
> > for (i = 0; i < sg_count; i++) {
> > - payload->range.pfn_array[i] =
> > - page_to_pfn(sg_page((cur_sgl)));
> > + for (j = subpage_idx; j < (PAGE_SIZE / HV_HYP_PAGE_SIZE); j++) {
>
> In the case where PAGE_SIZE == HV_HYP_PAGE_SIZE, would it help the compiler
> eliminate the loop if local variable j is declared as unsigned? In that case the test in the
> for statement will always be false.
>
Good point! I did the following test:
test.c:
int func(unsigned int input, int *arr)
{
unsigned int i;
int result = 0;
for (i = input; i < 1; i++)
result += arr[i];
return result;
}
if I define i as "int", I got:
0000000000000000 <func>:
0: 85 ff test %edi,%edi
2: 7f 2c jg 30 <func+0x30>
4: 48 63 d7 movslq %edi,%rdx
7: f7 df neg %edi
9: 45 31 c0 xor %r8d,%r8d
c: 89 ff mov %edi,%edi
e: 48 8d 04 96 lea (%rsi,%rdx,4),%rax
12: 48 01 d7 add %rdx,%rdi
15: 48 8d 54 be 04 lea 0x4(%rsi,%rdi,4),%rdx
1a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1)
20: 44 03 00 add (%rax),%r8d
23: 48 83 c0 04 add $0x4,%rax
27: 48 39 d0 cmp %rdx,%rax
2a: 75 f4 jne 20 <func+0x20>
2c: 44 89 c0 mov %r8d,%eax
2f: c3 retq
30: 45 31 c0 xor %r8d,%r8d
33: 44 89 c0 mov %r8d,%eax
36: c3 retq
and when I define i as "unsigned int", I got:
0000000000000000 <func>:
0: 85 ff test %edi,%edi
2: 75 03 jne 7 <func+0x7>
4: 8b 06 mov (%rsi),%eax
6: c3 retq
7: 31 c0 xor %eax,%eax
9: c3 retq
So clearly it helps, I will change this in the next version.
Regards,
Boqun
> > + payload->range.pfn_array[k] =
> > + page_to_hvpfn(sg_page((cur_sgl))) + j;
> > + k++;
> > + }
> > cur_sgl = sg_next(cur_sgl);
> > + subpage_idx = 0;
> > }
> > }
> >
> > --
> > 2.27.0
>
next prev parent reply other threads:[~2020-07-23 1:51 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-21 1:41 [RFC 00/11] Hyper-V: Support PAGE_SIZE larger than 4K Boqun Feng
2020-07-21 1:41 ` [RFC 01/11] Drivers: hv: vmbus: Always use HV_HYP_PAGE_SIZE for gpadl Boqun Feng
2020-07-21 15:22 ` Wei Liu
2020-07-22 23:20 ` Boqun Feng
2020-07-21 1:41 ` [RFC 02/11] Drivers: hv: vmbus: Move __vmbus_open() Boqun Feng
2020-07-21 15:23 ` Wei Liu
2020-07-21 1:41 ` [RFC 03/11] Drivers: hv: vmbus: Introduce types of GPADL Boqun Feng
2020-07-22 23:25 ` Michael Kelley
2020-07-22 23:43 ` Boqun Feng
2020-07-22 23:56 ` Michael Kelley
2020-07-21 1:41 ` [RFC 04/11] Drivers: hv: Use HV_HYP_PAGE in hv_synic_enable_regs() Boqun Feng
2020-07-21 1:41 ` [RFC 05/11] Drivers: hv: vmbus: Move virt_to_hvpfn() to hyperv header Boqun Feng
2020-07-21 1:41 ` [RFC 06/11] hv: hyperv.h: Introduce some hvpfn helper functions Boqun Feng
2020-07-21 1:41 ` [RFC 07/11] hv_netvsc: Use HV_HYP_PAGE_SIZE for Hyper-V communication Boqun Feng
2020-07-21 1:41 ` [RFC 08/11] Input: hyperv-keyboard: Make ringbuffer at least take two pages Boqun Feng
2020-07-21 1:41 ` [RFC 09/11] HID: hyperv: " Boqun Feng
2020-07-22 23:36 ` Michael Kelley
2020-07-23 1:28 ` boqun.feng
2020-07-21 1:41 ` [RFC 10/11] Driver: hv: util: " Boqun Feng
2020-07-21 1:41 ` [RFC 11/11] scsi: storvsc: Support PAGE_SIZE larger than 4K Boqun Feng
2020-07-23 0:13 ` Michael Kelley
2020-07-23 1:51 ` boqun.feng [this message]
2020-07-23 2:26 ` Michael Kelley
2020-07-23 3:12 ` Boqun Feng
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200723015149.GE35358@debian-boqun.qqnc3lrjykvubdpftowmye0fmh.lx.internal.cloudapp.net \
--to=boqun.feng@gmail.com \
--cc=benjamin.tissoires@redhat.com \
--cc=davem@davemloft.net \
--cc=dmitry.torokhov@gmail.com \
--cc=haiyangz@microsoft.com \
--cc=jejb@linux.ibm.com \
--cc=jikos@kernel.org \
--cc=kuba@kernel.org \
--cc=kys@microsoft.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-input@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=martin.petersen@oracle.com \
--cc=mikelley@microsoft.com \
--cc=netdev@vger.kernel.org \
--cc=sthemmin@microsoft.com \
--cc=wei.liu@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).