All of lore.kernel.org
 help / color / mirror / Atom feed
From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: Wei Liu <wei.liu@kernel.org>
Cc: linux-hyperv@vger.kernel.org,
	Andres Beltran <lkmlabelt@gmail.com>,
	Michael Kelley <mikelley@microsoft.com>,
	"Andrea Parri (Microsoft)" <parri.andrea@gmail.com>,
	Dexuan Cui <decui@microsoft.com>, Wei Liu <wei.liu@kernel.org>,
	Stephen Hemminger <sthemmin@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>,
	"K. Y. Srinivasan" <kys@microsoft.com>,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH] Drivers: hv: vmbus: Fix kernel crash upon unbinding a device from uio_hv_generic driver
Date: Wed, 01 Sep 2021 13:40:09 +0200	[thread overview]
Message-ID: <878s0go76u.fsf@vitty.brq.redhat.com> (raw)
In-Reply-To: <20210901112500.7q4oxjtesiuniop3@liuwe-devbox-debian-v2>

Wei Liu <wei.liu@kernel.org> writes:

> On Tue, Aug 31, 2021 at 04:39:16PM +0200, Vitaly Kuznetsov wrote:
>> The following crash happens when a never-used device is unbound from
>> uio_hv_generic driver:
>> 
>>  kernel BUG at mm/slub.c:321!
>>  invalid opcode: 0000 [#1] SMP PTI
>>  CPU: 0 PID: 4001 Comm: bash Kdump: loaded Tainted: G               X --------- ---  5.14.0-0.rc2.23.el9.x86_64 #1
>>  Hardware name: Microsoft Corporation Virtual Machine/Virtual Machine, BIOS 090008  12/07/2018
>>  RIP: 0010:__slab_free+0x1d5/0x3d0
>> ...
>>  Call Trace:
>>   ? pick_next_task_fair+0x18e/0x3b0
>>   ? __cond_resched+0x16/0x40
>>   ? vunmap_pmd_range.isra.0+0x154/0x1c0
>>   ? __vunmap+0x22d/0x290
>>   ? hv_ringbuffer_cleanup+0x36/0x40 [hv_vmbus]
>>   kfree+0x331/0x380
>>   ? hv_uio_remove+0x43/0x60 [uio_hv_generic]
>>   hv_ringbuffer_cleanup+0x36/0x40 [hv_vmbus]
>>   vmbus_free_ring+0x21/0x60 [hv_vmbus]
>>   hv_uio_remove+0x4f/0x60 [uio_hv_generic]
>>   vmbus_remove+0x23/0x30 [hv_vmbus]
>>   __device_release_driver+0x17a/0x230
>>   device_driver_detach+0x3c/0xa0
>>   unbind_store+0x113/0x130
>> ...
>> 
>> The problem appears to be that we free 'ring_info->pkt_buffer' twice:
>> first, when the device is unbound from in-kernel driver (netvsc in this
>> case) and second from hv_uio_remove(). Normally, ring buffer is supposed
>> to be re-initialized from hv_uio_open() but this happens when UIO device
>> is being opened and this is not guaranteed to happen.
>> 
>> Generally, it is OK to call hv_ringbuffer_cleanup() twice for the same
>> channel (which is being handed over between in-kernel drivers and UIO) even
>> if we didn't call hv_ringbuffer_init() in between. We, however, need to
>> avoid kfree() call for an already freed pointer.
>> 
>> Fixes: adae1e931acd ("Drivers: hv: vmbus: Copy packets sent by Hyper-V out of the ring buffer")
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
>>  drivers/hv/ring_buffer.c | 1 +
>>  1 file changed, 1 insertion(+)
>> 
>> diff --git a/drivers/hv/ring_buffer.c b/drivers/hv/ring_buffer.c
>> index 2aee356840a2..314015d9e912 100644
>> --- a/drivers/hv/ring_buffer.c
>> +++ b/drivers/hv/ring_buffer.c
>> @@ -245,6 +245,7 @@ void hv_ringbuffer_cleanup(struct hv_ring_buffer_info *ring_info)
>>  	mutex_unlock(&ring_info->ring_buffer_mutex);
>>  
>>  	kfree(ring_info->pkt_buffer);
>> +	ring_info->pkt_buffer = NULL;
>
> This certainly won't hurt.
>
> I would however like to wait till Andrea and Michael go over the
> reasoning of this patch before doing anything.
>

Thanks,

the counter-intuitive thing here is that the channel sometimes continues
to live after hv_ringbuffer_cleanup(): when we unbind a device from
in-kernel driver and give it to uio_hv_generic the channel is handed
over from one driver to another.

> Wei.
>
>>  	ring_info->pkt_buffer_size = 0;
>>  }
>>  
>> -- 
>> 2.31.1
>> 
>

-- 
Vitaly


  reply	other threads:[~2021-09-01 11:40 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-31 14:39 Vitaly Kuznetsov
2021-09-01 11:25 ` Wei Liu
2021-09-01 11:40   ` Vitaly Kuznetsov [this message]
2021-09-01 15:25 ` Andrea Parri
2021-09-02 17:09 ` Michael Kelley
2021-09-03 11:00 ` Wei Liu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=878s0go76u.fsf@vitty.brq.redhat.com \
    --to=vkuznets@redhat.com \
    --cc=decui@microsoft.com \
    --cc=haiyangz@microsoft.com \
    --cc=kys@microsoft.com \
    --cc=linux-hyperv@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lkmlabelt@gmail.com \
    --cc=mikelley@microsoft.com \
    --cc=parri.andrea@gmail.com \
    --cc=sthemmin@microsoft.com \
    --cc=wei.liu@kernel.org \
    --subject='Re: [PATCH] Drivers: hv: vmbus: Fix kernel crash upon unbinding a device from uio_hv_generic driver' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.