From mboxrd@z Thu Jan  1 00:00:00 1970
From: liu ping fan <qemulist@gmail.com>
Subject: Re: [PATCH 13/15] hotplug: introduce qdev_unplug_complete() to remove
 device from views
Date: Thu, 9 Aug 2012 15:28:27 +0800
Message-ID: <CAJnKYQ=xWDSAk=+8duE2xo3uGyN2j_rBpYPsTRt=OgVuAMFnbA@mail.gmail.com>
References: <1344407156-25562-1-git-send-email-qemulist@gmail.com>
 <1344407156-25562-14-git-send-email-qemulist@gmail.com> <502236D8.3040902@redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org,
	Anthony Liguori <anthony@codemonkey.ws>,
	Avi Kivity <avi@redhat.com>,
	Jan Kiszka <jan.kiszka@siemens.com>,
	Marcelo Tosatti <mtosatti@redhat.com>,
	Stefan Hajnoczi <stefanha@gmail.com>,
	Blue Swirl <blauwirbel@gmail.com>,
	=?ISO-8859-1?Q?Andreas_F=E4rber?= <afaerber@suse.de>
To: Paolo Bonzini <pbonzini@redhat.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from mail-we0-f174.google.com ([74.125.82.174]:41914 "EHLO
	mail-we0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1755748Ab2HIH2s (ORCPT <rfc822;kvm@vger.kernel.org>);
	Thu, 9 Aug 2012 03:28:48 -0400
Received: by weyx8 with SMTP id x8so86791wey.19
        for <kvm@vger.kernel.org>; Thu, 09 Aug 2012 00:28:47 -0700 (PDT)
In-Reply-To: <502236D8.3040902@redhat.com>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

On Wed, Aug 8, 2012 at 5:52 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 08/08/2012 08:25, Liu Ping Fan ha scritto:
>> +void qdev_unplug_complete(DeviceState *dev, Error **errp)
>> +{
>> +    /* isolate from mem view */
>> +    qdev_unmap(dev);
>> +    qemu_lock_devtree();
>> +    /* isolate from device tree */
>> +    qdev_unset_parent(dev);
>> +    qemu_unlock_devtree();
>> +    object_unref(OBJECT(dev));
>
> Rather than deferring the free, you should defer the unref.  Otherwise
> the following can happen when you have "real" RCU access to the memory
> map on the read-side:
>
>     VCPU thread                    I/O thread
> =====================================================================
>     get MMIO request
>     rcu_read_lock()
>     walk memory map
>                                    qdev_unmap()
>                                    lock_devtree()
>                                    ...
>                                    unlock_devtree
>                                    unref dev -> refcnt=0, free enqueued
>     ref()

No ref() for dev here, while we have ref to flatview+radix in my patches.
I use rcu to protect radix+flatview+mr refered. As to dev, its ref has
inc when it is added into mem view -- that is
memory_region_add_subregion -> memory_region_get() {
if(atomic_add_and_return()) dev->ref++  }.
So not until reclaimer of mem view, the dev's ref is hold by mem view.
In a short word, rcu protect mem view, while device is protected by refcnt.

>     rcu_read_unlock()
>                                    free()
>     <dangling pointer!>
>
> If you defer the unref, you have instead
>
>     VCPU thread                    I/O thread
> =====================================================================
>     get MMIO request
>     rcu_read_lock()
>     walk memory map
>                                    qdev_unmap()
>                                    lock_devtree()
>                                    ...
>                                    unlock_devtree
>                                    unref is enqueued
>     ref() -> refcnt = 2
>     rcu_read_unlock()
>                                    unref() -> refcnt=1
>     unref() -> refcnt = 1
>
> So this also makes patch 14 unnecessary.
>
> Paolo
>
>> +}
>
>

From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from eggs.gnu.org ([208.118.235.92]:46248)
	by lists.gnu.org with esmtp (Exim 4.71)
	(envelope-from <qemulist@gmail.com>) id 1SzNAn-0003vk-8b
	for qemu-devel@nongnu.org; Thu, 09 Aug 2012 03:28:50 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
	(envelope-from <qemulist@gmail.com>) id 1SzNAm-0008Vz-6R
	for qemu-devel@nongnu.org; Thu, 09 Aug 2012 03:28:49 -0400
Received: from mail-wg0-f41.google.com ([74.125.82.41]:54436)
	by eggs.gnu.org with esmtp (Exim 4.71)
	(envelope-from <qemulist@gmail.com>) id 1SzNAm-0008Vv-1A
	for qemu-devel@nongnu.org; Thu, 09 Aug 2012 03:28:48 -0400
Received: by wgbds1 with SMTP id ds1so84961wgb.4
	for <qemu-devel@nongnu.org>; Thu, 09 Aug 2012 00:28:47 -0700 (PDT)
MIME-Version: 1.0
In-Reply-To: <502236D8.3040902@redhat.com>
References: <1344407156-25562-1-git-send-email-qemulist@gmail.com>
	<1344407156-25562-14-git-send-email-qemulist@gmail.com>
	<502236D8.3040902@redhat.com>
From: liu ping fan <qemulist@gmail.com>
Date: Thu, 9 Aug 2012 15:28:27 +0800
Message-ID: <CAJnKYQ=xWDSAk=+8duE2xo3uGyN2j_rBpYPsTRt=OgVuAMFnbA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Subject: Re: [Qemu-devel] [PATCH 13/15] hotplug: introduce
 qdev_unplug_complete() to remove device from views
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: kvm@vger.kernel.org, Jan Kiszka <jan.kiszka@siemens.com>, Marcelo Tosatti <mtosatti@redhat.com>, qemu-devel@nongnu.org, Blue Swirl <blauwirbel@gmail.com>, Avi Kivity <avi@redhat.com>, Anthony Liguori <anthony@codemonkey.ws>, Stefan Hajnoczi <stefanha@gmail.com>, =?ISO-8859-1?Q?Andreas_F=E4rber?= <afaerber@suse.de>

On Wed, Aug 8, 2012 at 5:52 PM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> Il 08/08/2012 08:25, Liu Ping Fan ha scritto:
>> +void qdev_unplug_complete(DeviceState *dev, Error **errp)
>> +{
>> +    /* isolate from mem view */
>> +    qdev_unmap(dev);
>> +    qemu_lock_devtree();
>> +    /* isolate from device tree */
>> +    qdev_unset_parent(dev);
>> +    qemu_unlock_devtree();
>> +    object_unref(OBJECT(dev));
>
> Rather than deferring the free, you should defer the unref.  Otherwise
> the following can happen when you have "real" RCU access to the memory
> map on the read-side:
>
>     VCPU thread                    I/O thread
> =====================================================================
>     get MMIO request
>     rcu_read_lock()
>     walk memory map
>                                    qdev_unmap()
>                                    lock_devtree()
>                                    ...
>                                    unlock_devtree
>                                    unref dev -> refcnt=0, free enqueued
>     ref()

No ref() for dev here, while we have ref to flatview+radix in my patches.
I use rcu to protect radix+flatview+mr refered. As to dev, its ref has
inc when it is added into mem view -- that is
memory_region_add_subregion -> memory_region_get() {
if(atomic_add_and_return()) dev->ref++  }.
So not until reclaimer of mem view, the dev's ref is hold by mem view.
In a short word, rcu protect mem view, while device is protected by refcnt.

>     rcu_read_unlock()
>                                    free()
>     <dangling pointer!>
>
> If you defer the unref, you have instead
>
>     VCPU thread                    I/O thread
> =====================================================================
>     get MMIO request
>     rcu_read_lock()
>     walk memory map
>                                    qdev_unmap()
>                                    lock_devtree()
>                                    ...
>                                    unlock_devtree
>                                    unref is enqueued
>     ref() -> refcnt = 2
>     rcu_read_unlock()
>                                    unref() -> refcnt=1
>     unref() -> refcnt = 1
>
> So this also makes patch 14 unnecessary.
>
> Paolo
>
>> +}
>
>