From: Andrea Parri <parri.andrea@gmail.com>
To: Vitaly Kuznetsov <vkuznets@redhat.com>
Cc: Dexuan Cui <decui@microsoft.com>,
Michael Kelley <mikelley@microsoft.com>,
"K . Y . Srinivasan" <kys@microsoft.com>,
Haiyang Zhang <haiyangz@microsoft.com>,
Stephen Hemminger <sthemmin@microsoft.com>,
Wei Liu <wei.liu@kernel.org>,
linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org,
Boqun Feng <boqun.feng@gmail.com>
Subject: Re: [RFC PATCH 10/11] Drivers: hv: vmbus: Introduce the CHANNELMSG_MODIFYCHANNEL message type
Date: Sat, 28 Mar 2020 19:48:21 +0100 [thread overview]
Message-ID: <20200328184821.GA12184@andrea> (raw)
In-Reply-To: <87v9mr41j4.fsf@vitty.brq.redhat.com>
On Thu, Mar 26, 2020 at 03:46:23PM +0100, Vitaly Kuznetsov wrote:
> "Andrea Parri (Microsoft)" <parri.andrea@gmail.com> writes:
>
> > VMBus version 4.1 and later support the CHANNELMSG_MODIFYCHANNEL(22)
> > message type which can be used to request Hyper-V to change the vCPU
> > that a channel will interrupt.
> >
> > Introduce the CHANNELMSG_MODIFYCHANNEL message type, and define the
> > vmbus_send_modifychannel() function to send CHANNELMSG_MODIFYCHANNEL
> > requests to the host via a hypercall. The function is then used to
> > define a sysfs "store" operation, which allows to change the (v)CPU
> > the channel will interrupt by using the sysfs interface. The feature
> > can be used for load balancing or other purposes.
> >
> > One interesting catch here is that Hyper-V can *not* currently ACK
> > CHANNELMSG_MODIFYCHANNEL messages with the promise that (after the ACK
> > is sent) the channel won't send any more interrupts to the "old" CPU.
> >
> > The peculiarity of the CHANNELMSG_MODIFYCHANNEL messages is problematic
> > if the user want to take a CPU offline, since we don't want to take a
> > CPU offline (and, potentially, "lose" channel interrupts on such CPU)
> > if the host is still processing a CHANNELMSG_MODIFYCHANNEL message
> > associated to that CPU.
> >
> > It is worth mentioning, however, that we have been unable to observe
> > the above mentioned "race": in all our tests, CHANNELMSG_MODIFYCHANNEL
> > requests appeared *as if* they were processed synchronously by the
> > host.
>
> Hyper-V engineers never want to make our lifes easier :-)
Haha. I'd say more exciting! ;-) ;-)
> I can only think of a 'lazy' approach to setting channel CPU affinity:
> we actually re-assign it to the target CPU when we receive first
> interrupt there - but it's not very different from re-assigning it there
> but still handling interrupts on the old CPU like you do.
Interesting! I'm wondering whether it'd make sense to use a similar
approach to (lazily) "unblock" the "old" CPUs; let me think more...
> One more thing: it was already discussed several times but we never get
> to it. I think this question was even raised on Michael's latest
> 'Hyper-V on ARM' submission. What about implmenting a Hyper-V specific
> IRQ chip which would now support setting CPU affinity? The greatest
> benefit is that we don't need new tools to do e.g. load balancing,
> irqbalance will just work.
Thank you for the suggestions; TBH, I haven't considered such approach
so far (and I'd need more time to come up with an informed comment...)
OTOH, I had some initial investigations about the current (in-kernel)
balancing scheme/init_vp_index() and possible improvements/extensions
there... Hopefully another, follow-up series to come soon!
Thanks,
Andrea
next prev parent reply other threads:[~2020-03-28 18:48 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-25 22:54 [RFC PATCH 00/11] VMBus channel interrupt reassignment Andrea Parri (Microsoft)
2020-03-25 22:54 ` [RFC PATCH 01/11] Drivers: hv: vmbus: Always handle the VMBus messages on CPU0 Andrea Parri (Microsoft)
2020-03-26 14:05 ` Vitaly Kuznetsov
2020-03-28 18:50 ` Andrea Parri
2020-03-25 22:54 ` [RFC PATCH 02/11] Drivers: hv: vmbus: Don't bind the offer&rescind works to a specific CPU Andrea Parri (Microsoft)
2020-03-26 14:16 ` Vitaly Kuznetsov
2020-03-26 15:47 ` Andrea Parri
2020-03-26 17:26 ` Vitaly Kuznetsov
2020-03-28 17:08 ` Andrea Parri
2020-03-29 3:43 ` Michael Kelley
2020-03-30 12:24 ` Vitaly Kuznetsov
2020-04-03 12:04 ` Andrea Parri
2020-03-25 22:54 ` [RFC PATCH 03/11] Drivers: hv: vmbus: Replace the per-CPU channel lists with a global array of channels Andrea Parri (Microsoft)
2020-03-26 14:31 ` Vitaly Kuznetsov
2020-03-26 17:05 ` Andrea Parri
2020-03-26 17:43 ` Vitaly Kuznetsov
2020-03-28 18:21 ` Andrea Parri
2020-03-29 3:49 ` Michael Kelley
2020-03-30 12:45 ` Vitaly Kuznetsov
2020-04-03 13:38 ` Andrea Parri
2020-04-03 14:56 ` Vitaly Kuznetsov
2020-03-25 22:54 ` [RFC PATCH 04/11] hv_netvsc: Disable NAPI before closing the VMBus channel Andrea Parri (Microsoft)
2020-03-26 15:26 ` Stephen Hemminger
2020-03-26 17:55 ` Andrea Parri
2020-03-25 22:54 ` [RFC PATCH 05/11] hv_utils: Always execute the fcopy and vss callbacks in a tasklet Andrea Parri (Microsoft)
2020-03-25 22:55 ` [RFC PATCH 06/11] Drivers: hv: vmbus: Use a spin lock for synchronizing channel scheduling vs. channel removal Andrea Parri (Microsoft)
2020-03-25 22:55 ` [RFC PATCH 07/11] PCI: hv: Prepare hv_compose_msi_msg() for the VMBus-channel-interrupt-to-vCPU reassignment functionality Andrea Parri (Microsoft)
2020-03-25 22:55 ` [RFC PATCH 08/11] Drivers: hv: vmbus: Remove the unused HV_LOCALIZED channel affinity logic Andrea Parri (Microsoft)
2020-03-25 22:55 ` [RFC PATCH 09/11] Drivers: hv: vmbus: Synchronize init_vp_index() vs. CPU hotplug Andrea Parri (Microsoft)
2020-03-25 22:55 ` [RFC PATCH 10/11] Drivers: hv: vmbus: Introduce the CHANNELMSG_MODIFYCHANNEL message type Andrea Parri (Microsoft)
2020-03-26 14:46 ` Vitaly Kuznetsov
2020-03-28 18:48 ` Andrea Parri [this message]
2020-04-03 14:55 ` Andrea Parri
2020-03-25 22:55 ` [RFC PATCH 11/11] scsi: storvsc: Re-init stor_chns when a channel interrupt is re-assigned Andrea Parri (Microsoft)
2020-03-30 16:42 ` Michael Kelley
2020-03-30 18:55 ` Andrea Parri
2020-03-30 19:49 ` Michael Kelley
2020-04-03 13:41 ` Andrea Parri
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200328184821.GA12184@andrea \
--to=parri.andrea@gmail.com \
--cc=boqun.feng@gmail.com \
--cc=decui@microsoft.com \
--cc=haiyangz@microsoft.com \
--cc=kys@microsoft.com \
--cc=linux-hyperv@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mikelley@microsoft.com \
--cc=sthemmin@microsoft.com \
--cc=vkuznets@redhat.com \
--cc=wei.liu@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).