On Wed, 2020-12-02 at 18:34 +0000, Joao Martins wrote:
> On 12/2/20 4:47 PM, David Woodhouse wrote:
> > On Wed, 2020-12-02 at 13:12 +0000, Joao Martins wrote:
> > > On 12/2/20 11:17 AM, David Woodhouse wrote:
> > > > I might be more inclined to go for a model where the kernel handles the
> > > > evtchn_pending/evtchn_mask for us. What would go into the irq routing
> > > > table is { vcpu, port# } which get passed to kvm_xen_evtchn_send().
> > > 
> > > But passing port to the routing and handling the sending of events wouldn't it lead to
> > > unnecessary handling of event channels which aren't handled by the kernel, compared to
> > > just injecting caring about the upcall?
> > 
> > Well, I'm generally in favour of *not* doing things in the kernel that
> > don't need to be there.
> > 
> > But if the kernel is going to short-circuit the IPIs and VIRQs, then
> > it's already going to have to handle the evtchn_pending/evtchn_mask
> > bitmaps, and actually injecting interrupts.
> > 
> 
> Right. I was trying to point that out in the discussion we had
> in next patch. But true be told, more about touting the idea of kernel
> knowing if a given event channel is registered for userspace handling,
> rather than fully handling the event channel.
> 
> I suppose we are able to provide both options to the VMM anyway
> i.e. 1) letting them handle it enterily in userspace by intercepting
> EVTCHNOP_send, or through the irq route if we want kernel to offload it.

Right. The kernel takes what it knows about and anything else goes up
to userspace.

I do like the way you've handled the vcpu binding in userspace, and the
kernel just knows that a given port goes to a given target CPU.

> 
> > For the VMM
> > API I think we should follow the Xen model, mixing the domain-wide and
> > per-vCPU configuration. It's the best way to faithfully model the
> > behaviour a true Xen guest would experience.
> > 
> > So KVM_XEN_ATTR_TYPE_CALLBACK_VIA can be used to set one of
> >  • HVMIRQ_callback_vector, taking a vector#
> >  • HVMIRQ_callback_gsi for the in-kernel irqchip, taking a GSI#
> > 
> > And *maybe* in a later patch it could also handle
> >  • HVMIRQ_callback_gsi for split-irqchip, taking an eventfd
> >  • HVMIRQ_callback_pci_intx, taking an eventfd (or a pair, for EOI?)
> > 
> 
> Most of the Xen versions we were caring had callback_vector and
> vcpu callback vector (despite Linux not using the latter). But if you're
> dating back to 3.2 and 4.1 well (or certain Windows drivers), I suppose
> gsi and pci-intx are must-haves.

Note sure about GSI but PCI-INTX is definitely something I've seen in
active use by customers recently. I think SLES10 will use that.

> I feel we could just accommodate it as subtype in KVM_XEN_ATTR_TYPE_CALLBACK_VIA.
> Don't see the adavantage in having another xen attr type.

Yeah, fair enough.

> But kinda have mixed feelings in having kernel handling all event channels ABI,
> as opposed to only the ones userspace asked to offload. It looks a tad unncessary besides
> the added gain to VMMs that don't need to care about how the internals of event channels.
> But performance-wise it wouldn't bring anything better. But maybe, the former is reason
> enough to consider it.

Yeah, we'll see. Especially when it comes to implementing FIFO event
channels, I'd rather just do it in one place — and if the kernel does
it anyway then it's hardly difficult to hook into that.

But I've been about as coherent as I can be in email, and I think we're
generally aligned on the direction. I'll do some more experiments and
see what I can get working, and what it looks like.

I'm focusing on making the shinfo stuff all use kvm_map_gfn() first.