[Question] audit_names use after delete in audit_filter_inodes

[Question] audit_names use after delete in audit_filter_inodes

[Chen Wandun]

Hi,
Recently, I hit a use after delete in audit_filter_inodes,

In this case, I found audit_names->list->next is dead000000000100, when enumerate each
audit_names on list context->names_list.

void audit_filter_inodes(struct task_struct *tsk, struct audit_context *ctx)
{
         struct audit_names *n;

         if (audit_pid && tsk->tgid == audit_pid)
                 return;

         rcu_read_lock();

         list_for_each_entry(n, &ctx->names_list, list) {
                 if (audit_filter_inode_name(tsk, n, ctx))
                         break;
         }
         rcu_read_unlock();
}

it seem like the audit_names was already delete from context->names_list.

In source code, there is no any protection on context->names_list when read and write,
is there any race in read and write?

Unfortunately, there is no way to reproduce it.

the call stack is below:
[321315.077117] CPU: 6 PID: 8944 Comm: DefSch0100 Tainted: G           OE  ----V-------   3.10.0-327.62.59.83.w75.x86_64 #1
[321315.077117] Hardware name: OpenStack Foundation OpenStack Nova, BIOS rel-1.8.1-0-g4adadbd-20170107_142945-9_64_246_229 04/01/2014
[321315.113772] task: ffff8804061c4500 ti: ffff8804021d8000 task.ti: ffff8804021d8000
[321315.113772] RIP: 0010:[<ffffffff8110f038>]  [<ffffffff8110f038>] audit_filter_inodes+0x68/0x130
[321315.113772] RSP: 0018:ffff8804021dbef0  EFLAGS: 00010297
[321315.113772] RAX: ffff88040632aa48 RBX: ffff88040632a800 RCX: 000000000000000a
[321315.113772] RDX: 00000000000000c0 RSI: ffff88040632a800 RDI: ffff8804061c4500
[321315.132068] RBP: ffff8804021dbf40 R08: 0000000000000000 R09: 0000000000000000
[321315.132068] R10: 00007fd38197ac00 R11: 0000000000000206 R12: dead000000000100
[321315.132068] R13: ffff8804061c4500 R14: 00000000ffffffff R15: ffff88040632a800
[321315.132068] FS:  00007fd38197b700(0000) GS:ffff88053c380000(0000) knlGS:0000000000000000
[321315.132068] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[321315.132068] CR2: 00007fe48936d156 CR3: 0000000098b50000 CR4: 00000000001407e0
[321315.149373] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[321315.149373] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[321315.149373] Stack:
[321315.149373]  ffff88040632aa48 ffff8804061c4500 ffff8804021dbf78 ffffffff819a6260
[321315.149373]  00000000aefb5477 ffff88040632a800 00007fd3d6e80690 ffff8804061c4500
[321315.149373]  00007fd38197ac70 00007fd3f218d008 ffff8804021dbf78 ffffffff8110f7d5
[321315.149373] Call Trace:
[321315.149373]  [<ffffffff8110f7d5>] __audit_syscall_exit+0x245/0x280
[321315.149373]  [<ffffffff8165316b>] sysret_audit+0x17/0x21
[321315.149373] Code: 84 be 00 00 00 4d 8b a7 48 02 00 00 49 8d 87 48 02 00 00 48 89 45 b0 49 39 c4 0f 84 a3 00 00 00 41 be ff ff ff ff 0f 1f 44 00 00 <49> 8b 44 24 20 83 e0 1f 48 c1 e0 04 4c 8d a8 e0 5e df 81 48 8b
[321315.149373] RIP  [<ffffffff8110f038>] audit_filter_inodes+0x68/0x130
[321315.149373]  RSP <ffff8804021dbef0>
[321315.196242] ---[ end trace e1b43c8e59447f0a ]---

I am unfamiliar with audit. I will be appreciated if you could give me some suggestion.

Thanks
ChenWandun

Re: [Question] audit_names use after delete in audit_filter_inodes

[Paul Moore]

On Wed, Aug 21, 2019 at 5:31 AM Chen Wandun <chenwandun@huawei.com> wrote:
>
> Hi,
> Recently, I hit a use after delete in audit_filter_inodes,

...

> the call stack is below:
> [321315.077117] CPU: 6 PID: 8944 Comm: DefSch0100 Tainted: G           OE  ----V-------   3.10.0-327.62.59.83.w75.x86_64 #1
> [321315.077117] Hardware name: OpenStack Foundation OpenStack Nova, BIOS rel-1.8.1-0-g4adadbd-20170107_142945-9_64_246_229 04/01/2014

It looks like this is a vendor kernel and not an upstream kernel, yes?
 Assuming that is the case I would suggest you contact your distro for
help/debugging/support; we simply don't know enough about your kernel
(what patches are included, how was it built/configured/etc.) to
comment with any certainty.

Linux Kernels based on v3.10.0 are extremely old from an upstream
perspective, with a number of fixes and changes to the audit subsystem
since v3.10.0 was released.  If you see the same problem on a modern
upstream kernel please let us know, we'll be happy to help.

-- 
paul moore
www.paul-moore.com

Re: [Question] audit_names use after delete in audit_filter_inodes

[Chen Wandun]

On 2019/8/21 23:36, Paul Moore wrote:
> On Wed, Aug 21, 2019 at 5:31 AM Chen Wandun <chenwandun@huawei.com> wrote:
>>
>> Hi,
>> Recently, I hit a use after delete in audit_filter_inodes,
> 
> ...
> 
>> the call stack is below:
>> [321315.077117] CPU: 6 PID: 8944 Comm: DefSch0100 Tainted: G           OE  ----V-------   3.10.0-327.62.59.83.w75.x86_64 #1
>> [321315.077117] Hardware name: OpenStack Foundation OpenStack Nova, BIOS rel-1.8.1-0-g4adadbd-20170107_142945-9_64_246_229 04/01/2014
> 
> It looks like this is a vendor kernel and not an upstream kernel, yes?

I analysed the upstream kernel about audit, and found there is no significant change
in audit_names add/read/delete since v3.10.

audit_names could be delete in __audit_syscall_exit, do_exit, copy_process
on upstream kernel(same as v3.10).

if we are reading audit_names, such as
	__audit_syscall_exit
		audit_filter_inodes
			read each audit_names ...
			

is there any situation could delete audit_names at the same time?

>   Assuming that is the case I would suggest you contact your distro for
> help/debugging/support; we simply don't know enough about your kernel
> (what patches are included, how was it built/configured/etc.) to
> comment with any certainty.
> 
> Linux Kernels based on v3.10.0 are extremely old from an upstream
> perspective, with a number of fixes and changes to the audit subsystem
> since v3.10.0 was released.  If you see the same problem on a modern
> upstream kernel please let us know, we'll be happy to help.
> 

Re: [Question] audit_names use after delete in audit_filter_inodes

[Paul Moore]

On Thu, Aug 22, 2019 at 3:22 AM Chen Wandun <chenwandun@huawei.com> wrote:
> On 2019/8/21 23:36, Paul Moore wrote:
> > On Wed, Aug 21, 2019 at 5:31 AM Chen Wandun <chenwandun@huawei.com> wrote:
> >>
> >> Hi,
> >> Recently, I hit a use after delete in audit_filter_inodes,
> >
> > ...
> >
> >> the call stack is below:
> >> [321315.077117] CPU: 6 PID: 8944 Comm: DefSch0100 Tainted: G           OE  ----V-------   3.10.0-327.62.59.83.w75.x86_64 #1
> >> [321315.077117] Hardware name: OpenStack Foundation OpenStack Nova, BIOS rel-1.8.1-0-g4adadbd-20170107_142945-9_64_246_229 04/01/2014
> >
> > It looks like this is a vendor kernel and not an upstream kernel, yes?
>
> I analysed the upstream kernel about audit, and found there is no significant change
> in audit_names add/read/delete since v3.10.

The number of changes in a Linux v3.10 based distro kernel and a
modern upstream are quite significant, regardless of what you might
see by comparing a limited number of functions/structs.  I once again
suggest you contact your distribution provider for support, or move to
a modern Linux kernel so that upstream can better understand your
problem.

-- 
paul moore
www.paul-moore.com