From: Jan Beulich <jbeulich@suse.com>
To: Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Cc: "julien@xen.org" <julien@xen.org>,
"sstabellini@kernel.org" <sstabellini@kernel.org>,
"Oleksandr Tyshchenko" <Oleksandr_Tyshchenko@epam.com>,
"Volodymyr Babchuk" <Volodymyr_Babchuk@epam.com>,
"Artem Mygaiev" <Artem_Mygaiev@epam.com>,
"andrew.cooper3@citrix.com" <andrew.cooper3@citrix.com>,
"george.dunlap@citrix.com" <george.dunlap@citrix.com>,
"paul@xen.org" <paul@xen.org>,
"Bertrand Marquis" <bertrand.marquis@arm.com>,
"Rahul Singh" <rahul.singh@arm.com>,
"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
"Roger Pau Monné" <roger.pau@citrix.com>
Subject: Re: [PATCH v4 02/11] vpci: cancel pending map/unmap on vpci removal
Date: Thu, 18 Nov 2021 16:41:06 +0100 [thread overview]
Message-ID: <ac0cc710-05b9-bdae-c31c-d159b4de0105@suse.com> (raw)
In-Reply-To: <2db325e9-0349-1cc3-1c4a-fefa048f181b@epam.com>
On 18.11.2021 16:21, Oleksandr Andrushchenko wrote:
> On 18.11.21 17:16, Jan Beulich wrote:
>> On 18.11.2021 16:11, Oleksandr Andrushchenko wrote:
>>> On 18.11.21 16:35, Jan Beulich wrote:
>>>> On 18.11.2021 15:14, Oleksandr Andrushchenko wrote:
>>>>> On 18.11.21 16:04, Roger Pau Monné wrote:
>>>>>> Indeed. In the physdevop failure case this comes from an hypercall
>>>>>> context, so maybe you could do the mapping in place using hypercall
>>>>>> continuations if required. Not sure how complex that would be,
>>>>>> compared to just deferring to guest entry point and then dealing with
>>>>>> the possible cleanup on failure.
>>>>> This will solve one part of the equation:
>>>>>
>>>>> pci_physdev_op
>>>>> pci_add_device
>>>>> init_bars -> modify_bars -> defer_map -> raise_softirq(SCHEDULE_SOFTIRQ)
>>>>> iommu_add_device <- FAILS
>>>>> vpci_remove_device -> xfree(pdev->vpci)
>>>>>
>>>>> But what about the other one, e.g. vpci_process_pending is triggered in
>>>>> parallel with PCI device de-assign for example?
>>>> Well, that's again in hypercall context, so using hypercall continuations
>>>> may again be an option. Of course at the point a de-assign is initiated,
>>>> you "only" need to drain requests (for that device, but that's unlikely
>>>> to be worthwhile optimizing for), while ensuring no new requests can be
>>>> issued. Again, for the device in question, but here this is relevant -
>>>> a flag may want setting to refuse all further requests. Or maybe the
>>>> register handling hooks may want tearing down before draining pending
>>>> BAR mapping requests; without the hooks in place no new such requests
>>>> can possibly appear.
>>> This can be probably even easier to solve as we were talking about
>>> pausing all vCPUs:
>> I have to admit I'm not sure. It might be easier, but it may also be
>> less desirable.
>>
>>> void vpci_cancel_pending(const struct pci_dev *pdev)
>>> {
>>> struct domain *d = pdev->domain;
>>> struct vcpu *v;
>>> int rc;
>>>
>>> while ( (rc = domain_pause_except_self(d)) == -ERESTART )
>>> cpu_relax();
>>>
>>> if ( rc )
>>> printk(XENLOG_G_ERR
>>> "Failed to pause vCPUs while canceling vPCI map/unmap for %pp %pd: %d\n",
>>> &pdev->sbdf, pdev->domain, rc);
>>>
>>> for_each_vcpu ( d, v )
>>> {
>>> if ( v->vpci.map_pending && (v->vpci.pdev == pdev) )
>>>
>>> This will prevent all vCPUs to run, but current, thus making it impossible
>>> to run vpci_process_pending in parallel with any hypercall.
>>> So, even without locking in vpci_process_pending the above should
>>> be enough.
>>> The only concern here is that domain_pause_except_self may return
>>> the error code we somehow need to handle...
>> Not just this. The -ERESTART handling isn't appropriate this way
>> either.
> Are you talking about cpu_relax()?
I'm talking about that spin-waiting loop as a whole.
>> For the moment I can't help thinking that draining would
>> be preferable over canceling.
> Given that cancellation is going to happen on error path or
> on device de-assign/remove I think this can be acceptable.
> Any reason why not?
It would seem to me that the correctness of a draining approach is
going to be easier to prove than that of a canceling one, where I
expect races to be a bigger risk. Especially something that gets
executed infrequently, if ever (error paths in particular), knowing
things are well from testing isn't typically possible.
Jan
next prev parent reply other threads:[~2021-11-18 15:41 UTC|newest]
Thread overview: 101+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-05 6:56 [PATCH v4 00/11] PCI devices passthrough on Arm, part 3 Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 01/11] vpci: fix function attributes for vpci_process_pending Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 02/11] vpci: cancel pending map/unmap on vpci removal Oleksandr Andrushchenko
2021-11-15 16:56 ` Jan Beulich
2021-11-16 7:32 ` Oleksandr Andrushchenko
2021-11-16 8:01 ` Jan Beulich
2021-11-16 8:23 ` Oleksandr Andrushchenko
2021-11-16 11:38 ` Jan Beulich
2021-11-16 13:27 ` Oleksandr Andrushchenko
2021-11-16 14:11 ` Jan Beulich
2021-11-16 13:41 ` Oleksandr Andrushchenko
2021-11-16 14:12 ` Jan Beulich
2021-11-16 14:24 ` Oleksandr Andrushchenko
2021-11-16 14:37 ` Oleksandr Andrushchenko
2021-11-16 16:09 ` Jan Beulich
2021-11-16 18:02 ` Julien Grall
2021-11-18 12:57 ` Oleksandr Andrushchenko
2021-11-17 8:28 ` Jan Beulich
2021-11-18 7:49 ` Oleksandr Andrushchenko
2021-11-18 8:36 ` Jan Beulich
2021-11-18 8:54 ` Oleksandr Andrushchenko
2021-11-18 9:15 ` Jan Beulich
2021-11-18 9:32 ` Oleksandr Andrushchenko
2021-11-18 13:25 ` Jan Beulich
2021-11-18 13:48 ` Oleksandr Andrushchenko
2021-11-18 14:04 ` Roger Pau Monné
2021-11-18 14:14 ` Oleksandr Andrushchenko
2021-11-18 14:35 ` Jan Beulich
2021-11-18 15:11 ` Oleksandr Andrushchenko
2021-11-18 15:16 ` Jan Beulich
2021-11-18 15:21 ` Oleksandr Andrushchenko
2021-11-18 15:41 ` Jan Beulich [this message]
2021-11-18 15:46 ` Oleksandr Andrushchenko
2021-11-18 15:53 ` Jan Beulich
2021-11-19 12:34 ` Oleksandr Andrushchenko
2021-11-19 13:00 ` Jan Beulich
2021-11-19 13:16 ` Oleksandr Andrushchenko
2021-11-19 13:25 ` Jan Beulich
2021-11-19 13:34 ` Oleksandr Andrushchenko
2021-11-22 14:21 ` Oleksandr Andrushchenko
2021-11-22 14:37 ` Jan Beulich
2021-11-22 14:45 ` Oleksandr Andrushchenko
2021-11-22 14:57 ` Jan Beulich
2021-11-22 15:02 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 03/11] vpci: make vpci registers removal a dedicated function Oleksandr Andrushchenko
2021-11-15 16:57 ` Jan Beulich
2021-11-16 8:02 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 04/11] vpci: add hooks for PCI device assign/de-assign Oleksandr Andrushchenko
2021-11-15 17:06 ` Jan Beulich
2021-11-16 9:38 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 05/11] vpci/header: implement guest BAR register handlers Oleksandr Andrushchenko
2021-11-19 11:58 ` Jan Beulich
2021-11-19 12:10 ` Oleksandr Andrushchenko
2021-11-19 12:37 ` Jan Beulich
2021-11-19 12:46 ` Oleksandr Andrushchenko
2021-11-19 12:49 ` Jan Beulich
2021-11-19 12:54 ` Oleksandr Andrushchenko
2021-11-19 13:02 ` Jan Beulich
2021-11-19 13:17 ` Oleksandr Andrushchenko
2021-11-23 15:14 ` Oleksandr Andrushchenko
2021-11-24 12:32 ` Roger Pau Monné
2021-11-24 12:36 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 06/11] vpci/header: handle p2m range sets per BAR Oleksandr Andrushchenko
2021-11-19 12:05 ` Jan Beulich
2021-11-19 12:13 ` Oleksandr Andrushchenko
2021-11-19 12:45 ` Jan Beulich
2021-11-19 12:50 ` Oleksandr Andrushchenko
2021-11-19 13:06 ` Jan Beulich
2021-11-19 13:19 ` Oleksandr Andrushchenko
2021-11-19 13:29 ` Jan Beulich
2021-11-19 13:38 ` Oleksandr Andrushchenko
2021-11-19 13:16 ` Jan Beulich
2021-11-19 13:41 ` Oleksandr Andrushchenko
2021-11-19 13:57 ` Jan Beulich
2021-11-19 14:09 ` Oleksandr Andrushchenko
2021-11-22 8:24 ` Jan Beulich
2021-11-22 8:31 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 07/11] vpci/header: program p2m with guest BAR view Oleksandr Andrushchenko
2021-11-19 12:33 ` Jan Beulich
2021-11-19 12:44 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 08/11] vpci/header: emulate PCI_COMMAND register for guests Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 09/11] vpci/header: reset the command register when adding devices Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 10/11] vpci: add initial support for virtual PCI bus topology Oleksandr Andrushchenko
2021-11-18 16:45 ` Jan Beulich
2021-11-24 11:28 ` Oleksandr Andrushchenko
2021-11-24 12:36 ` Roger Pau Monné
2021-11-24 12:43 ` Oleksandr Andrushchenko
2021-11-05 6:56 ` [PATCH v4 11/11] xen/arm: translate virtual PCI bus topology for guests Oleksandr Andrushchenko
2021-11-08 11:10 ` Jan Beulich
2021-11-08 11:16 ` Oleksandr Andrushchenko
2021-11-08 14:23 ` Roger Pau Monné
2021-11-08 15:28 ` Oleksandr Andrushchenko
2021-11-24 11:31 ` Oleksandr Andrushchenko
2021-11-19 13:56 ` [PATCH v4 00/11] PCI devices passthrough on Arm, part 3 Jan Beulich
2021-11-19 14:06 ` Oleksandr Andrushchenko
2021-11-19 14:23 ` Roger Pau Monné
2021-11-19 14:26 ` Oleksandr Andrushchenko
2021-11-20 9:47 ` Roger Pau Monné
2021-11-22 8:22 ` Jan Beulich
2021-11-22 8:34 ` Oleksandr Andrushchenko
2021-11-22 8:44 ` Jan Beulich
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ac0cc710-05b9-bdae-c31c-d159b4de0105@suse.com \
--to=jbeulich@suse.com \
--cc=Artem_Mygaiev@epam.com \
--cc=Oleksandr_Andrushchenko@epam.com \
--cc=Oleksandr_Tyshchenko@epam.com \
--cc=Volodymyr_Babchuk@epam.com \
--cc=andrew.cooper3@citrix.com \
--cc=bertrand.marquis@arm.com \
--cc=george.dunlap@citrix.com \
--cc=julien@xen.org \
--cc=paul@xen.org \
--cc=rahul.singh@arm.com \
--cc=roger.pau@citrix.com \
--cc=sstabellini@kernel.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).