Re: BUG: KASAN: use-after-free in dec_rlimit_ucounts

From: ebiederm@xmission.com (Eric W. Biederman)
To: Qian Cai <quic_qiancai@quicinc.com>
Cc: Alexey Gladkov <legion@kernel.org>, Yu Zhao <yuzhao@google.com>,
	<linux-kernel@vger.kernel.org>
Subject: Re: BUG: KASAN: use-after-free in dec_rlimit_ucounts
Date: Thu, 18 Nov 2021 14:57:17 -0600	[thread overview]
Message-ID: <87k0h5rxle.fsf@email.froward.int.ebiederm.org> (raw)
In-Reply-To: <YZa4YbcOyjtD3+pL@fixkernel.com> (Qian Cai's message of "Thu, 18 Nov 2021 15:32:33 -0500")

Qian Cai <quic_qiancai@quicinc.com> writes:

> On Thu, Nov 18, 2021 at 01:46:05PM -0600, Eric W. Biederman wrote:
>> Is it possible?  Yes it is possible.  That is one place where
>> a use-after-free has shown up and I expect would show up in the
>> future.
>> 
>> That said it is hard to believe there is still a user-after-free in the
>> code.  We spent the last kernel development cycle pouring through and
>> correcting everything we saw until we ultimately found one very subtle
>> use-after-free.
>> 
>> If you have a reliable reproducer that you can share, we can look into
>> this and see if we can track down where the reference count is going
>> bad.
>> 
>> It tends to take instrumenting the entire life cycle every increment and
>> every decrement and then pouring through the logs to track down a
>> use-after-free.  Which is not something we can really do without a
>> reproducer.
>
> The reproducer is just to run trinity by an unprivileged user on defconfig
> with KASAN enabled (On linux-next, you can do "make defconfig debug.conf"
> [1], but dont think other debugging options are relevent here.)
>
> $ trinity -C 31 -N 10000000
>
> It is always reproduced on an arm64 server here within 5-minute so far.
> Some debugging progress so far. BTW, this could happen on user_shm_unlock()
> path as well.

Does this only happen on a single architecture?  If so I wonder if
perhaps some of the architectures atomic primitives are implemented
improperly.

Unfortunately I don't have any arm64 machines where I can easily test
this.

The call path you posted from user_shm_unlock is another path where
a use-after-free has show up in the past.

My blind guess would be that I made an implementation mistake in
inc_rlimit_get_ucounts or dec_rlimit_put_ucounts but I can't see it
right now.

Eric

>  Call trace:
>   dec_rlimit_ucounts
>   user_shm_unlock
>   (inlined by) user_shm_unlock at mm/mlock.c:854
>   shmem_lock
>   shmctl_do_lock
>   ksys_shmctl.constprop.0
>   __arm64_sys_shmctl
>   invoke_syscall
>   el0_svc_common.constprop.0
>   do_el0_svc
>   el0_svc
>   el0t_64_sync_handler
>   el0t_64_sync
>
> I noticed in dec_rlimit_ucounts(), dec == 0 and type ==
> UCOUNT_RLIMIT_MEMLOCK. 
>
> [1] https://lore.kernel.org/lkml/20211115134754.7334-1-quic_qiancai@quicinc.com/