From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59907) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e2Xpp-0004mB-4e for qemu-devel@nongnu.org; Thu, 12 Oct 2017 03:23:14 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e2Xpl-0006Zf-W0 for qemu-devel@nongnu.org; Thu, 12 Oct 2017 03:23:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50356) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e2Xpl-0006Yd-Mb for qemu-devel@nongnu.org; Thu, 12 Oct 2017 03:23:09 -0400 References: <20170628190047.26159-1-dgilbert@redhat.com> <20170628190047.26159-25-dgilbert@redhat.com> <2f4c1067-14a4-d943-0dff-790d705778f1@redhat.com> <20170707115353.GD2451@work-vm> <20171003132351.GA2935@work-vm> <0c312c2e-2ef1-07cb-10cc-69e21d38077d@redhat.com> <20171009121240.GK2374@work-vm> From: Maxime Coquelin Message-ID: <547c7c4a-2eed-3b83-2fac-f90c7dc07491@redhat.com> Date: Thu, 12 Oct 2017 09:22:56 +0200 MIME-Version: 1.0 In-Reply-To: <20171009121240.GK2374@work-vm> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [RFC 24/29] vhost+postcopy: Lock around set_mem_table List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: "Dr. David Alan Gilbert" Cc: qemu-devel@nongnu.org, a.perevalov@samsung.com, marcandre.lureau@redhat.com, mst@redhat.com, quintela@redhat.com, peterx@redhat.com, lvivier@redhat.com, aarcange@redhat.com On 10/09/2017 02:12 PM, Dr. David Alan Gilbert wrote: > * Maxime Coquelin (maxime.coquelin@redhat.com) wrote: >> >> >> On 10/03/2017 03:23 PM, Dr. David Alan Gilbert wrote: >>> * Dr. David Alan Gilbert (dgilbert@redhat.com) wrote: >>>> * Maxime Coquelin (maxime.coquelin@redhat.com) wrote: >>>>> >>>>> >>>>> On 06/28/2017 09:00 PM, Dr. David Alan Gilbert (git) wrote: >>>>>> From: "Dr. David Alan Gilbert" >>>>>> >>>>>> **HACK - better solution needed ** >>>>>> We have the situation where: >>>>>> >>>>>> qemu bridge >>>>>> >>>>>> send set_mem_table >>>>>> map memory >>>>>> a) mark area with UFD >>>>>> send reply with map addresses >>>>>> b) start using >>>>>> c) receive reply >>>>>> >>>>>> As soon as (a) happens qemu might start seeing faults >>>>>> from memory accesses (but doesn't until b); but it can't >>>>>> process those faults until (c) when it's received the >>>>>> mmap addresses. >>>>>> >>>>>> Make the fault handler spin until it gets the reply in (c). >>>>>> >>>>>> At the very least this needs some proper locks, but preferably >>>>>> we need to split the message. >>>>> >>>>> Yes, maybe the slave channel could be used to send the ufds with >>>>> a dedicated request? The backend would set the reply-ack flag, so that >>>>> it starts accessing the guest memory only when Qemu is ready to handle >>>>> faults. >>>> >>>> Yes, that would make life a lot easier. >>>> >>>>> Note that the slave channel support has not been implemented in Qemu's >>>>> libvhost-user yet, but this is something I can do if we feel the need. >>>> >>>> Can you tell me a bit about how the slave channel works? >>> >>> I've looked at the slave-channel; and I'm worried that it's not suitable >>> for this case. >>> The problem is that 'slave_read' is wired to a fd_handler that I think >>> is serviced by the main thread, >> I confirm, this is serviced by the main thread. >> >>> and while postcopy is running I don't >>> want to rely on the operation of the main thread (since it could be >>> blocked by a page fault). >> >> IIUC, you mean QEMU being blocked by a page fault. > > Yes. > >> In this case, I don't think this is an issue, because QEMU doesn't rely >> on the backend to handle the page fault, so the slave request can be >> handled only once QEMU has handled the fault. >> >> Maybe I am missing something? > > It feels delicate; with the vhost client blocked waiting for the ack > from the qemu to the registration reply on the slave, and some other > part blocked by a page fault, it makes it sound likely to hit deadlocks > even if I can't put my finger on one. Right, it is hard to be sure there is no risk of deadlock. >>> I could still use an explicit ack at that point though over the main >>> channel I think (or use the slave synchronously?). >> >> Can you please elaborate, I'm not sure to understand what you mean. > > In the world I'm currently working on I've got it just using the main > channel but: > settable -> client > settable-results -> qemu > ack -> client > > all over the main channel with each side waiting for the other. Ok, thanks for the clarification. Maxime > Dave > >> >> Thanks, >> Maxime >> >>> Dave >>> >>>> Dave >>>> >>>>> Maxime >>>> -- >>>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK >>> -- >>> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK >>> > -- > Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK >