Linux-Fsdevel Archive on lore.kernel.org
 help / color / Atom feed
* WARNING in request_end
@ 2018-09-24 12:29 syzbot
  2018-09-24 14:44 ` Miklos Szeredi
                   ` (3 more replies)
  0 siblings, 4 replies; 11+ messages in thread
From: syzbot @ 2018-09-24 12:29 UTC (permalink / raw)
  To: linux-fsdevel, linux-kernel, miklos, syzkaller-bugs

Hello,

syzbot found the following crash on:

HEAD commit:    6bf4ca7fbc85 Linux 4.19-rc5
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=159149c6400000
kernel config:  https://syzkaller.appspot.com/x/.config?x=22a62640793a83c9
dashboard link: https://syzkaller.appspot.com/bug?extid=ef054c4d3f64cd7f7cec
compiler:       gcc (GCC) 8.0.1 20180413 (experimental)

Unfortunately, I don't have any reproducer for this crash yet.

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com

WARNING: CPU: 0 PID: 9445 at fs/fuse/dev.c:390 request_end+0x82e/0xaa0  
fs/fuse/dev.c:390
Kernel panic - not syncing: panic_on_warn set ...

CPU: 0 PID: 9445 Comm: syz-executor2 Not tainted 4.19.0-rc5+ #251
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS  
Google 01/01/2011
Call Trace:
kobject: '0:56' (00000000d57a9914): kobject_add_internal: parent: 'bdi',  
set: 'devices'
  __dump_stack lib/dump_stack.c:77 [inline]
  dump_stack+0x1c4/0x2b4 lib/dump_stack.c:113
kobject: '0:56' (00000000d57a9914): kobject_uevent_env
  panic+0x238/0x4e7 kernel/panic.c:184
kobject: '0:56' (00000000d57a9914): fill_kobj_path: path  
= '/devices/virtual/bdi/0:56'
  __warn.cold.8+0x163/0x1ba kernel/panic.c:536
  report_bug+0x254/0x2d0 lib/bug.c:186
  fixup_bug arch/x86/kernel/traps.c:178 [inline]
  do_error_trap+0x1fc/0x4d0 arch/x86/kernel/traps.c:296
  do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:316
  invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:993
RIP: 0010:request_end+0x82e/0xaa0 fs/fuse/dev.c:390
Code: 3c 03 0f 8f 6f fe ff ff 48 8b bd f0 fe ff ff e8 68 e7 39 ff e9 5e fe  
ff ff e8 8e 86 f6 fe 0f 0b e9 b0 fa ff ff e8 82 86 f6 fe <0f> 0b e9 f0 fa  
ff ff e8 36 71 c0 fe e8 61 e7 39 ff e9 5b fb ff ff
RSP: 0018:ffff88019df0f378 EFLAGS: 00010212
RAX: 0000000000040000 RBX: ffff8801d2ca6000 RCX: ffffc90006d53000
RDX: 0000000000000138 RSI: ffffffff82885d9e RDI: 0000000000000007
RBP: ffff88019df0f4a8 R08: ffff8801d1a6c200 R09: ffffed0034b53937
R10: ffffed0034b53937 R11: ffff8801a5a9c9bb R12: 1ffff10033be1e74
R13: ffff8801a5a9c940 R14: ffff8801d2ca6030 R15: ffff88019df0f480
  fuse_dev_do_write+0x192e/0x36e0 fs/fuse/dev.c:1915
  fuse_dev_write+0x19a/0x240 fs/fuse/dev.c:1939
  call_write_iter include/linux/fs.h:1808 [inline]
  new_sync_write fs/read_write.c:474 [inline]
  __vfs_write+0x6b8/0x9f0 fs/read_write.c:487
  vfs_write+0x1fc/0x560 fs/read_write.c:549
  ksys_write+0x101/0x260 fs/read_write.c:598
  __do_sys_write fs/read_write.c:610 [inline]
  __se_sys_write fs/read_write.c:607 [inline]
  __x64_sys_write+0x73/0xb0 fs/read_write.c:607
  do_syscall_64+0x1b9/0x820 arch/x86/entry/common.c:290
  entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x457679
Code: 1d b4 fb ff c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 48 89 f8 48 89 f7  
48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff  
ff 0f 83 eb b3 fb ff c3 66 2e 0f 1f 84 00 00 00 00
RSP: 002b:00007efd81affc78 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
RAX: ffffffffffffffda RBX: 00007efd81b006d4 RCX: 0000000000457679
RDX: 0000000000000090 RSI: 0000000020000500 RDI: 0000000000000003
RBP: 000000000072c040 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000246 R12: 00000000ffffffff
R13: 00000000004d8710 R14: 00000000004c50a2 R15: 0000000000000002
Kernel Offset: disabled
Rebooting in 86400 seconds..


---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with  
syzbot.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WARNING in request_end
  2018-09-24 12:29 WARNING in request_end syzbot
@ 2018-09-24 14:44 ` Miklos Szeredi
  2018-09-25  9:18   ` Kirill Tkhai
  2018-10-08  9:38 ` syzbot
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 11+ messages in thread
From: Miklos Szeredi @ 2018-09-24 14:44 UTC (permalink / raw)
  To: Kirill Tkhai; +Cc: syzbot, linux-fsdevel, linux-kernel, syzkaller-bugs

On Mon, Sep 24, 2018 at 2:29 PM, syzbot
<syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com> wrote:
> Hello,
>
> syzbot found the following crash on:
>
> HEAD commit:    6bf4ca7fbc85 Linux 4.19-rc5
> git tree:       upstream
> console output: https://syzkaller.appspot.com/x/log.txt?x=159149c6400000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=22a62640793a83c9
> dashboard link: https://syzkaller.appspot.com/bug?extid=ef054c4d3f64cd7f7cec
> compiler:       gcc (GCC) 8.0.1 20180413 (experimental)
>
> Unfortunately, I don't have any reproducer for this crash yet.
>
> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> Reported-by: syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com
>
> WARNING: CPU: 0 PID: 9445 at fs/fuse/dev.c:390 request_end+0x82e/0xaa0

And  there we have the bug likely caused by the set_bit(FR_SENT, ...)
not being inside the fpq->lock-ed region.

So that needs to be fixed anyway, apparently.

Thanks,
Miklos

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WARNING in request_end
  2018-09-24 14:44 ` Miklos Szeredi
@ 2018-09-25  9:18   ` Kirill Tkhai
  2018-09-25  9:38     ` Dmitry Vyukov
  0 siblings, 1 reply; 11+ messages in thread
From: Kirill Tkhai @ 2018-09-25  9:18 UTC (permalink / raw)
  To: Miklos Szeredi; +Cc: syzbot, linux-fsdevel, linux-kernel, syzkaller-bugs

On 24.09.2018 17:44, Miklos Szeredi wrote:
> On Mon, Sep 24, 2018 at 2:29 PM, syzbot
> <syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com> wrote:
>> Hello,
>>
>> syzbot found the following crash on:
>>
>> HEAD commit:    6bf4ca7fbc85 Linux 4.19-rc5
>> git tree:       upstream
>> console output: https://syzkaller.appspot.com/x/log.txt?x=159149c6400000
>> kernel config:  https://syzkaller.appspot.com/x/.config?x=22a62640793a83c9
>> dashboard link: https://syzkaller.appspot.com/bug?extid=ef054c4d3f64cd7f7cec
>> compiler:       gcc (GCC) 8.0.1 20180413 (experimental)
>>
>> Unfortunately, I don't have any reproducer for this crash yet.
>>
>> IMPORTANT: if you fix the bug, please add the following tag to the commit:
>> Reported-by: syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com
>>
>> WARNING: CPU: 0 PID: 9445 at fs/fuse/dev.c:390 request_end+0x82e/0xaa0
> 
> And  there we have the bug likely caused by the set_bit(FR_SENT, ...)
> not being inside the fpq->lock-ed region.
> 
> So that needs to be fixed anyway, apparently.

I can't confirm, since I haven't found yet the direct way, that set_bit() results
in this stack...

We have one more (unrelated) possible use-after-free here:

cpu0                                       cpu1
fuse_dev_do_write()                     fuse_dev_do_write()
  req = request_find(fpq, oh.unique)    ...
  spin_unlock(&fpq->lock)               ...
  ...                                   req = request_find(fpq, oh.unique)
  ...                                   spin_unlock(&fpq->lock)
  queue_interrupt(&fc->iq, req);        ...
  ...                                   ...
  ...                                   ...
request freed                           ...
...                                     queue_interrupt(&fc->iq, req); <- use after free

Something like below is needed:

@@ -1875,16 +1877,20 @@ static ssize_t fuse_dev_do_write(struct fuse_dev *fud,
 
 	/* Is it an interrupt reply? */
 	if (req->intr_unique == oh.unique) {
+		__fuse_get_request(req);
 		spin_unlock(&fpq->lock);
 
 		err = -EINVAL;
-		if (nbytes != sizeof(struct fuse_out_header))
+		if (nbytes != sizeof(struct fuse_out_header)) {
+			fuse_put_request(fc, req);
 			goto err_finish;
+		}
 
 		if (oh.error == -ENOSYS)
 			fc->no_interrupt = 1;
 		else if (oh.error == -EAGAIN)
 			queue_interrupt(&fc->iq, req);
+		fuse_put_request(fc, req);
 
 		fuse_copy_finish(cs);
 		return nbytes;

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WARNING in request_end
  2018-09-25  9:18   ` Kirill Tkhai
@ 2018-09-25  9:38     ` Dmitry Vyukov
  2018-09-25  9:49       ` Kirill Tkhai
  0 siblings, 1 reply; 11+ messages in thread
From: Dmitry Vyukov @ 2018-09-25  9:38 UTC (permalink / raw)
  To: Kirill Tkhai; +Cc: Miklos Szeredi, syzbot, linux-fsdevel, LKML, syzkaller-bugs

On Tue, Sep 25, 2018 at 11:18 AM, Kirill Tkhai <ktkhai@virtuozzo.com> wrote:
> On 24.09.2018 17:44, Miklos Szeredi wrote:
>> On Mon, Sep 24, 2018 at 2:29 PM, syzbot
>> <syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com> wrote:
>>> Hello,
>>>
>>> syzbot found the following crash on:
>>>
>>> HEAD commit:    6bf4ca7fbc85 Linux 4.19-rc5
>>> git tree:       upstream
>>> console output: https://syzkaller.appspot.com/x/log.txt?x=159149c6400000
>>> kernel config:  https://syzkaller.appspot.com/x/.config?x=22a62640793a83c9
>>> dashboard link: https://syzkaller.appspot.com/bug?extid=ef054c4d3f64cd7f7cec
>>> compiler:       gcc (GCC) 8.0.1 20180413 (experimental)
>>>
>>> Unfortunately, I don't have any reproducer for this crash yet.
>>>
>>> IMPORTANT: if you fix the bug, please add the following tag to the commit:
>>> Reported-by: syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com
>>>
>>> WARNING: CPU: 0 PID: 9445 at fs/fuse/dev.c:390 request_end+0x82e/0xaa0
>>
>> And  there we have the bug likely caused by the set_bit(FR_SENT, ...)
>> not being inside the fpq->lock-ed region.
>>
>> So that needs to be fixed anyway, apparently.
>
> I can't confirm, since I haven't found yet the direct way, that set_bit() results
> in this stack...
>
> We have one more (unrelated) possible use-after-free here:
>
> cpu0                                       cpu1
> fuse_dev_do_write()                     fuse_dev_do_write()
>   req = request_find(fpq, oh.unique)    ...
>   spin_unlock(&fpq->lock)               ...
>   ...                                   req = request_find(fpq, oh.unique)
>   ...                                   spin_unlock(&fpq->lock)
>   queue_interrupt(&fc->iq, req);        ...
>   ...                                   ...
>   ...                                   ...
> request freed                           ...
> ...                                     queue_interrupt(&fc->iq, req); <- use after free
>
> Something like below is needed:

There is a bunch of open bugs in fuse on syzbot dashboard, perhaps
it's one of them:

https://syzkaller.appspot.com/bug?id=19aabec97cbf73dd0475d6e599113a7861c4b306
https://syzkaller.appspot.com/bug?id=24aa489e6929205e40ec4aa52cd8f47897f2ad63
https://syzkaller.appspot.com/bug?id=400d6a977a0dbd8836d7c7ec8481782a674ee855
https://syzkaller.appspot.com/bug?id=ff9ab4a23afa7553fb79f745a92be87ba4144508
https://syzkaller.appspot.com/bug?id=d0f258de27b6d7ccecbba09385b3376cc4a12ffe
https://syzkaller.appspot.com/bug?id=e8077bce636d52d9c40e1ea904699c27b7454354


> @@ -1875,16 +1877,20 @@ static ssize_t fuse_dev_do_write(struct fuse_dev *fud,
>
>         /* Is it an interrupt reply? */
>         if (req->intr_unique == oh.unique) {
> +               __fuse_get_request(req);
>                 spin_unlock(&fpq->lock);
>
>                 err = -EINVAL;
> -               if (nbytes != sizeof(struct fuse_out_header))
> +               if (nbytes != sizeof(struct fuse_out_header)) {
> +                       fuse_put_request(fc, req);
>                         goto err_finish;
> +               }
>
>                 if (oh.error == -ENOSYS)
>                         fc->no_interrupt = 1;
>                 else if (oh.error == -EAGAIN)
>                         queue_interrupt(&fc->iq, req);
> +               fuse_put_request(fc, req);
>
>                 fuse_copy_finish(cs);
>                 return nbytes;
>
> --
> You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bugs+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/274aafd2-5076-6b14-f55e-360411fb8169%40virtuozzo.com.
> For more options, visit https://groups.google.com/d/optout.

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WARNING in request_end
  2018-09-25  9:38     ` Dmitry Vyukov
@ 2018-09-25  9:49       ` Kirill Tkhai
  0 siblings, 0 replies; 11+ messages in thread
From: Kirill Tkhai @ 2018-09-25  9:49 UTC (permalink / raw)
  To: Dmitry Vyukov; +Cc: Miklos Szeredi, syzbot, linux-fsdevel, LKML, syzkaller-bugs

On 25.09.2018 12:38, Dmitry Vyukov wrote:
> On Tue, Sep 25, 2018 at 11:18 AM, Kirill Tkhai <ktkhai@virtuozzo.com> wrote:
>> On 24.09.2018 17:44, Miklos Szeredi wrote:
>>> On Mon, Sep 24, 2018 at 2:29 PM, syzbot
>>> <syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com> wrote:
>>>> Hello,
>>>>
>>>> syzbot found the following crash on:
>>>>
>>>> HEAD commit:    6bf4ca7fbc85 Linux 4.19-rc5
>>>> git tree:       upstream
>>>> console output: https://syzkaller.appspot.com/x/log.txt?x=159149c6400000
>>>> kernel config:  https://syzkaller.appspot.com/x/.config?x=22a62640793a83c9
>>>> dashboard link: https://syzkaller.appspot.com/bug?extid=ef054c4d3f64cd7f7cec
>>>> compiler:       gcc (GCC) 8.0.1 20180413 (experimental)
>>>>
>>>> Unfortunately, I don't have any reproducer for this crash yet.
>>>>
>>>> IMPORTANT: if you fix the bug, please add the following tag to the commit:
>>>> Reported-by: syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com
>>>>
>>>> WARNING: CPU: 0 PID: 9445 at fs/fuse/dev.c:390 request_end+0x82e/0xaa0
>>>
>>> And  there we have the bug likely caused by the set_bit(FR_SENT, ...)
>>> not being inside the fpq->lock-ed region.
>>>
>>> So that needs to be fixed anyway, apparently.
>>
>> I can't confirm, since I haven't found yet the direct way, that set_bit() results
>> in this stack...
>>
>> We have one more (unrelated) possible use-after-free here:
>>
>> cpu0                                       cpu1
>> fuse_dev_do_write()                     fuse_dev_do_write()
>>   req = request_find(fpq, oh.unique)    ...
>>   spin_unlock(&fpq->lock)               ...
>>   ...                                   req = request_find(fpq, oh.unique)
>>   ...                                   spin_unlock(&fpq->lock)
>>   queue_interrupt(&fc->iq, req);        ...
>>   ...                                   ...
>>   ...                                   ...
>> request freed                           ...
>> ...                                     queue_interrupt(&fc->iq, req); <- use after free
>>
>> Something like below is needed:
> 
> There is a bunch of open bugs in fuse on syzbot dashboard, perhaps
> it's one of them:
> 
> https://syzkaller.appspot.com/bug?id=19aabec97cbf73dd0475d6e599113a7861c4b306
> https://syzkaller.appspot.com/bug?id=24aa489e6929205e40ec4aa52cd8f47897f2ad63
> https://syzkaller.appspot.com/bug?id=400d6a977a0dbd8836d7c7ec8481782a674ee855
> https://syzkaller.appspot.com/bug?id=ff9ab4a23afa7553fb79f745a92be87ba4144508
> https://syzkaller.appspot.com/bug?id=d0f258de27b6d7ccecbba09385b3376cc4a12ffe
> https://syzkaller.appspot.com/bug?id=e8077bce636d52d9c40e1ea904699c27b7454354

I can't find fuse_dev_do_write() there, but it's possible this race could appear
in another function. So, Dmitry, I won't add reference to one of tham. Let's
check, which will disappear in the future.

Thanks,
Kirill

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WARNING in request_end
  2018-09-24 12:29 WARNING in request_end syzbot
  2018-09-24 14:44 ` Miklos Szeredi
@ 2018-10-08  9:38 ` syzbot
  2019-03-23  7:50 ` syzbot
  2019-11-07 13:42 ` syzbot
  3 siblings, 0 replies; 11+ messages in thread
From: syzbot @ 2018-10-08  9:38 UTC (permalink / raw)
  To: dvyukov, ktkhai, linux-fsdevel, linux-kernel, miklos, syzkaller-bugs

syzbot has found a reproducer for the following crash on:

HEAD commit:    0238df646e62 Linux 4.19-rc7
git tree:       upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=16daaa85400000
kernel config:  https://syzkaller.appspot.com/x/.config?x=88e9a8a39dc0be2d
dashboard link: https://syzkaller.appspot.com/bug?extid=ef054c4d3f64cd7f7cec
compiler:       gcc (GCC) 8.0.1 20180413 (experimental)
userspace arch: i386
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=119bf2e6400000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=1760f806400000

IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com

8021q: adding VLAN 0 to HW filter on device team0
8021q: adding VLAN 0 to HW filter on device team0
WARNING: CPU: 1 PID: 7459 at fs/fuse/dev.c:390 request_end+0x82e/0xaa0  
fs/fuse/dev.c:390
Kernel panic - not syncing: panic_on_warn set ...

CPU: 1 PID: 7459 Comm: syz-executor659 Not tainted 4.19.0-rc7+ #176
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS  
Google 01/01/2011
Call Trace:
  __dump_stack lib/dump_stack.c:77 [inline]
  dump_stack+0x1c4/0x2b4 lib/dump_stack.c:113
  panic+0x238/0x4e7 kernel/panic.c:184
  __warn.cold.8+0x163/0x1ba kernel/panic.c:536
  report_bug+0x254/0x2d0 lib/bug.c:186
  fixup_bug arch/x86/kernel/traps.c:178 [inline]
  do_error_trap+0x1fc/0x4d0 arch/x86/kernel/traps.c:296
  do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:316
  invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:993
RIP: 0010:request_end+0x82e/0xaa0 fs/fuse/dev.c:390
Code: 3c 03 0f 8f 6f fe ff ff 48 8b bd f0 fe ff ff e8 78 63 3b ff e9 5e fe  
ff ff e8 1e f3 f7 fe 0f 0b e9 b0 fa ff ff e8 12 f3 f7 fe <0f> 0b e9 f0 fa  
ff ff e8 16 ca c1 fe e8 71 63 3b ff e9 5b fb ff ff
RSP: 0018:ffff8801c65e7328 EFLAGS: 00010293
RAX: ffff8801cd3362c0 RBX: ffff8801cba17000 RCX: ffffffff8286dd65
RDX: 0000000000000000 RSI: ffffffff8286e27e RDI: 0000000000000007
RBP: ffff8801c65e7458 R08: ffff8801cd3362c0 R09: ffffed00391cd5bf
R10: ffffed00391cd5bf R11: ffff8801c8e6adfb R12: 1ffff10038cbce6a
R13: ffff8801c8e6ad80 R14: ffff8801cba17030 R15: ffff8801c65e7430
  fuse_dev_do_write+0x192e/0x36e0 fs/fuse/dev.c:1915
  fuse_dev_write+0x19a/0x240 fs/fuse/dev.c:1939
  call_write_iter include/linux/fs.h:1808 [inline]
  new_sync_write fs/read_write.c:474 [inline]
  __vfs_write+0x6b8/0x9f0 fs/read_write.c:487
  vfs_write+0x1fc/0x560 fs/read_write.c:549
  ksys_write+0x101/0x260 fs/read_write.c:598
  __do_sys_write fs/read_write.c:610 [inline]
  __se_sys_write fs/read_write.c:607 [inline]
  __ia32_sys_write+0x71/0xb0 fs/read_write.c:607
  do_syscall_32_irqs_on arch/x86/entry/common.c:326 [inline]
  do_fast_syscall_32+0x34d/0xfb2 arch/x86/entry/common.c:397
  entry_SYSENTER_compat+0x70/0x7f arch/x86/entry/entry_64_compat.S:139
RIP: 0023:0xf7f43ca9
Code: 85 d2 74 02 89 0a 5b 5d c3 8b 04 24 c3 8b 0c 24 c3 8b 1c 24 c3 90 90  
90 90 90 90 90 90 90 90 90 90 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90  
90 90 90 eb 0d 90 90 90 90 90 90 90 90 90 90 90 90
RSP: 002b:00000000f7efd1fc EFLAGS: 00000246 ORIG_RAX: 0000000000000004
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00000000200002c0
RDX: 0000000000000050 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 00000000003d0f00 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
Kernel Offset: disabled
Rebooting in 86400 seconds..

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WARNING in request_end
  2018-09-24 12:29 WARNING in request_end syzbot
  2018-09-24 14:44 ` Miklos Szeredi
  2018-10-08  9:38 ` syzbot
@ 2019-03-23  7:50 ` syzbot
  2019-03-23 15:51   ` ebiederm
  2019-11-07 13:42 ` syzbot
  3 siblings, 1 reply; 11+ messages in thread
From: syzbot @ 2019-03-23  7:50 UTC (permalink / raw)
  To: dvyukov, ebiederm, ktkhai, linux-fsdevel, linux-kernel, miklos,
	mszeredi, syzkaller-bugs

syzbot has bisected this bug to:

commit 4ad769f3c346ec3d458e255548dec26ca5284cf6
Author: Eric W. Biederman <ebiederm@xmission.com>
Date:   Tue May 29 14:04:46 2018 +0000

     fuse: Allow fully unprivileged mounts

bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=16b4518b200000
start commit:   0238df64 Linux 4.19-rc7
git tree:       upstream
final crash:    https://syzkaller.appspot.com/x/report.txt?x=15b4518b200000
console output: https://syzkaller.appspot.com/x/log.txt?x=11b4518b200000
kernel config:  https://syzkaller.appspot.com/x/.config?x=88e9a8a39dc0be2d
dashboard link: https://syzkaller.appspot.com/bug?extid=ef054c4d3f64cd7f7cec
userspace arch: i386
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=119bf2e6400000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=1760f806400000

Reported-by: syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com
Fixes: 4ad769f3c346 ("fuse: Allow fully unprivileged mounts")

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WARNING in request_end
  2019-03-23  7:50 ` syzbot
@ 2019-03-23 15:51   ` ebiederm
  2019-03-23 19:48     ` Miklos Szeredi
  0 siblings, 1 reply; 11+ messages in thread
From: ebiederm @ 2019-03-23 15:51 UTC (permalink / raw)
  To: syzbot
  Cc: dvyukov, ktkhai, linux-fsdevel, linux-kernel, miklos, mszeredi,
	syzkaller-bugs

syzbot <syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com> writes:

> syzbot has bisected this bug to:

Nope.  syzbot got it wrong.

At most that commit will allow a larger class of users to mount fuse
and thus be able to reproduce the problem.

It does look like syzbot has found something concerning though.

Miklos any ideas?



> commit 4ad769f3c346ec3d458e255548dec26ca5284cf6
> Author: Eric W. Biederman <ebiederm@xmission.com>
> Date:   Tue May 29 14:04:46 2018 +0000
>
>     fuse: Allow fully unprivileged mounts
>
> bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=16b4518b200000
> start commit:   0238df64 Linux 4.19-rc7
> git tree:       upstream
> final crash:    https://syzkaller.appspot.com/x/report.txt?x=15b4518b200000
> console output: https://syzkaller.appspot.com/x/log.txt?x=11b4518b200000
> kernel config:  https://syzkaller.appspot.com/x/.config?x=88e9a8a39dc0be2d
> dashboard link: https://syzkaller.appspot.com/bug?extid=ef054c4d3f64cd7f7cec
> userspace arch: i386
> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=119bf2e6400000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=1760f806400000
>
> Reported-by: syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com
> Fixes: 4ad769f3c346 ("fuse: Allow fully unprivileged mounts")
>
> For information about bisection process see: https://goo.gl/tpsmEJ#bisection


From https://syzkaller.appspot.com/x/report.txt?x=15b4518b200000
> [  448.045793] ==================================================================
> [  448.053414] BUG: KASAN: use-after-free in fuse_dev_do_read.isra.24+0x166f/0x1be0
> [  448.060937] Read of size 8 at addr ffff8801cec98430 by task syz-executor0/9001
> [  448.068286] 
> [  448.069901] CPU: 1 PID: 9001 Comm: syz-executor0 Not tainted 4.16.0-rc6+ #1
> [  448.076990] Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
> [  448.086330] Call Trace:
> [  448.089107]  dump_stack+0x153/0x201
> [  448.092926]  ? arch_local_irq_restore+0x43/0x43
> [  448.097579]  ? printk+0x9a/0xc0
> [  448.100844]  ? show_regs_print_info+0xb/0xb
> [  448.105265]  print_address_description.cold.7+0x9/0x1c9
> [  448.110739]  kasan_report.cold.8+0x242/0x2fe
> [  448.115255]  ? fuse_dev_do_read.isra.24+0x166f/0x1be0
> [  448.120476]  __asan_report_load8_noabort+0x14/0x20
> [  448.125393]  fuse_dev_do_read.isra.24+0x166f/0x1be0
> [  448.130397]  ? debug_check_no_locks_freed+0x310/0x310
> [  448.135574]  ? end_requests+0x470/0x470
> [  448.139529]  ? print_usage_bug+0xc0/0xc0
> [  448.143576]  ? prepare_to_wait+0x4f0/0x4f0
> [  448.147932]  ? print_usage_bug+0xc0/0xc0
> [  448.152139]  ? __unqueue_futex+0x270/0x270
> [  448.156376]  ? add_lock_to_list.isra.29+0x4b0/0x4b0
> [  448.161703]  ? wake_up_q+0x9c/0xe0
> [  448.165236]  ? futex_wake+0x245/0x8a0
> [  448.169025]  ? find_held_lock+0x36/0x1c0
> [  448.173085]  ? aa_file_perm+0x319/0xda0
> [  448.177065]  ? lock_downgrade+0x900/0x900
> [  448.181241]  ? rcu_read_lock_bh_held+0xc0/0xc0
> [  448.185813]  ? debug_smp_processor_id+0x17/0x20
> [  448.190557]  ? rcu_is_watching+0x69/0x180
> [  448.194700]  ? __lock_is_held+0xb5/0x140
> [  448.198859]  ? rcu_dynticks_eqs_exit+0x70/0x70
> [  448.203436]  ? aa_file_perm+0x336/0xda0
> [  448.207393]  ? rcu_read_lock_bh_held+0xc0/0xc0
> [  448.211958]  ? aa_path_link+0x610/0x610
> [  448.215913]  ? rcu_dynticks_eqs_exit+0x70/0x70
> [  448.220485]  ? memset+0x31/0x40
> [  448.223752]  fuse_dev_read+0x185/0x240
> [  448.227665]  ? fuse_dev_splice_read+0x7a0/0x7a0
> [  448.232375]  ? find_held_lock+0x36/0x1c0
> [  448.236439]  __vfs_read+0x54a/0xd20
> [  448.240161]  ? debug_lockdep_rcu_enabled+0x77/0x90
> [  448.245069]  ? vfs_copy_file_range+0xb60/0xb60
> [  448.249737]  ? fsnotify_first_mark+0x280/0x280
> [  448.254360]  ? rw_verify_area+0xb8/0x2b0
> [  448.258411]  ? __fdget_raw+0x10/0x10
> [  448.262151]  vfs_read+0xf5/0x300
> [  448.265509]  SyS_read+0xf5/0x250
> [  448.268860]  ? kernel_write+0x130/0x130
> [  448.272823]  ? do_fast_syscall_32+0x151/0x1016
> [  448.277396]  do_fast_syscall_32+0x3d5/0x1016
> [  448.281797]  ? _raw_spin_unlock_irq+0x27/0x80
> [  448.286317]  ? trace_hardirqs_on_caller+0x421/0x5c0
> [  448.291337]  ? do_int80_syscall_32+0x9f0/0x9f0
> [  448.296277]  ? _raw_spin_unlock_irq+0x60/0x80
> [  448.300761]  ? finish_task_switch+0x1f4/0x890
> [  448.305411]  ? syscall_return_slowpath+0x215/0x4e0
> [  448.310337]  ? prepare_exit_to_usermode+0x300/0x300
> [  448.315348]  ? sysret32_from_system_call+0x5/0x3c
> [  448.320187]  ? trace_hardirqs_off_thunk+0x1a/0x1c
> [  448.325080]  entry_SYSENTER_compat+0x70/0x7f
> [  448.329492] RIP: 0023:0xf7f8fcb9
> [  448.332846] RSP: 002b:00000000f7f8b0cc EFLAGS: 00000296 ORIG_RAX: 0000000000000003
> [  448.340546] RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 0000000020001000
> [  448.347796] RDX: 00000000ffffff20 RSI: 0000000000000000 RDI: 0000000000000000
> [  448.355047] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
> [  448.362301] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> [  448.369595] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [  448.376890] 
> [  448.378514] Allocated by task 9010:
> [  448.382133]  save_stack+0x43/0xd0
> [  448.385681]  kasan_kmalloc+0xc7/0xe0
> [  448.389408]  kasan_slab_alloc+0x12/0x20
> [  448.393373]  kmem_cache_alloc+0x12e/0x790
> [  448.397518]  __fuse_request_alloc+0x23/0xc0
> [  448.401827]  __fuse_get_req+0x186/0x8d0
> [  448.405790]  fuse_simple_request+0x20/0x610
> [  448.410101]  fuse_do_setattr+0x820/0x1f60
> [  448.414262]  fuse_setattr+0x1a6/0x470
> [  448.418074]  notify_change+0x779/0xda0
> [  448.421942]  utimes_common.isra.1+0x3f8/0x7f0
> [  448.426420]  do_utimes+0x199/0x250
> [  448.430053]  compat_SyS_utimes+0x1f8/0x2e0
> [  448.434563]  do_fast_syscall_32+0x3d5/0x1016
> [  448.438956]  entry_SYSENTER_compat+0x70/0x7f
> [  448.443357] 
> [  448.444974] Freed by task 9010:
> [  448.448305]  save_stack+0x43/0xd0
> [  448.451740]  __kasan_slab_free+0x102/0x150
> [  448.455957]  kasan_slab_free+0xe/0x10
> [  448.459750]  kmem_cache_free+0x83/0x2d0
> [  448.463719]  fuse_request_free+0x77/0x90
> [  448.467762]  fuse_put_request+0x22a/0x2d0
> [  448.471901]  fuse_simple_request+0x38a/0x610
> [  448.476394]  fuse_do_setattr+0x820/0x1f60
> [  448.480525]  fuse_setattr+0x1a6/0x470
> [  448.484304]  notify_change+0x779/0xda0
> [  448.488342]  utimes_common.isra.1+0x3f8/0x7f0
> [  448.492918]  do_utimes+0x199/0x250
> [  448.496443]  compat_SyS_utimes+0x1f8/0x2e0
> [  448.500769]  do_fast_syscall_32+0x3d5/0x1016
> [  448.505172]  entry_SYSENTER_compat+0x70/0x7f
> [  448.509660] 
> [  448.511273] The buggy address belongs to the object at ffff8801cec98400
> [  448.511273]  which belongs to the cache fuse_request of size 448
> [  448.524116] The buggy address is located 48 bytes inside of
> [  448.524116]  448-byte region [ffff8801cec98400, ffff8801cec985c0)
> [  448.535897] The buggy address belongs to the page:
> [  448.540853] page:ffffea00073b2600 count:1 mapcount:0 mapping:ffff8801cec98000 index:0x0
> [  448.549166] flags: 0x2fffc0000000100(slab)
> [  448.553534] raw: 02fffc0000000100 ffff8801cec98000 0000000000000000 0000000100000008
> [  448.561407] raw: ffffea0007656660 ffffea00076359e0 ffff8801d4de8680 0000000000000000
> [  448.569270] page dumped because: kasan: bad access detected
> [  448.574960]
> [  448.576564] Memory state around the buggy address:
> [  448.581477]  ffff8801cec98300: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> [  448.588871]  ffff8801cec98380: fb fb fb fb fb fb fb fb fc fc fc fc fc fc fc fc
> [  448.596217] >ffff8801cec98400: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> [  448.603596]                                      ^
> [  448.608507]  ffff8801cec98480: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> [  448.615843]  ffff8801cec98500: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> [  448.623284] ==================================================================

Eric

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WARNING in request_end
  2019-03-23 15:51   ` ebiederm
@ 2019-03-23 19:48     ` Miklos Szeredi
  2019-03-23 20:16       ` syzbot
  0 siblings, 1 reply; 11+ messages in thread
From: Miklos Szeredi @ 2019-03-23 19:48 UTC (permalink / raw)
  To: Eric W. Biederman
  Cc: syzbot, dvyukov, ktkhai, linux-fsdevel, lkml, Miklos Szeredi,
	syzkaller-bugs

On Sat, Mar 23, 2019 at 4:52 PM Eric W. Biederman <ebiederm@xmission.com> wrote:
>
> syzbot <syzbot+ef054c4d3f64cd7f7cec@syzkaller.appspotmail.com> writes:
>
> > syzbot has bisected this bug to:
>
> Nope.  syzbot got it wrong.
>
> At most that commit will allow a larger class of users to mount fuse
> and thus be able to reproduce the problem.
>
> It does look like syzbot has found something concerning though.
>
> Miklos any ideas?

Dup of this?

bc78abbd55dd ("fuse: Fix use-after-free in fuse_dev_do_read()")

Let's test:

#syz test: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
bc78abbd55dd

Thanks,
Miklos

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WARNING in request_end
  2019-03-23 19:48     ` Miklos Szeredi
@ 2019-03-23 20:16       ` syzbot
  0 siblings, 0 replies; 11+ messages in thread
From: syzbot @ 2019-03-23 20:16 UTC (permalink / raw)
  To: dvyukov, ebiederm, ktkhai, linux-fsdevel, linux-kernel, miklos,
	mszeredi, syzkaller-bugs

Hello,

syzbot has tested the proposed patch but the reproducer still triggered  
crash:
WARNING in request_end

WARNING: CPU: 0 PID: 16992 at fs/fuse/dev.c:390 request_end+0x836/0xac0  
fs/fuse/dev.c:390
kobject: '0:49' (000000001562c524): kobject_uevent_env
Kernel panic - not syncing: panic_on_warn set ...

CPU: 0 PID: 16992 Comm: syz-executor3 Not tainted 4.19.0-rc5+ #1
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS  
Google 01/01/2011
Call Trace:
  __dump_stack lib/dump_stack.c:77 [inline]
  dump_stack+0x1db/0x2ce lib/dump_stack.c:113
  panic+0x263/0x51a kernel/panic.c:184
kobject: 'loop5' (0000000073db98f3): kobject_uevent_env
  __warn.cold+0x13b/0x1ba kernel/panic.c:536
  report_bug+0x263/0x2b0 lib/bug.c:186
kobject: 'loop5' (0000000073db98f3): fill_kobj_path: path  
= '/devices/virtual/block/loop5'
  fixup_bug arch/x86/kernel/traps.c:178 [inline]
  fixup_bug arch/x86/kernel/traps.c:173 [inline]
  do_error_trap+0x200/0x4e0 arch/x86/kernel/traps.c:296
kobject: '0:49' (000000001562c524): fill_kobj_path: path  
= '/devices/virtual/bdi/0:49'
  do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:316
  invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:993
kobject: '0:49' (000000001562c524): kobject_cleanup, parent           (null)
RIP: 0010:request_end+0x836/0xac0 fs/fuse/dev.c:390
Code: 3c 03 0f 8f 7d fe ff ff 48 8b bd 30 ff ff ff e8 b0 b4 3b ff e9 6c fe  
ff ff e8 a6 ad f8 fe 0f 0b e9 be fa ff ff e8 9a ad f8 fe <0f> 0b e9 fc fa  
ff ff e8 4e c7 c2 fe e8 a9 b4 3b ff e9 6a fb ff ff
RSP: 0018:ffff8801c099f5a8 EFLAGS: 00010293
RAX: ffff8801be90e040 RBX: 1ffff10038133eba RCX: ffffffff82858ce9
RDX: 0000000000000000 RSI: ffffffff828591f6 RDI: 0000000000000007
RBP: ffff8801c099f698 R08: ffff8801be90e040 R09: ffffed0037bc2c18
R10: ffffed0037bc2c17 R11: ffff8801bde160bb R12: ffff8801a5ca9800
R13: ffff8801bde16040 R14: ffff8801c099f670 R15: ffff8801a5ca9830
kobject: '0:49' (000000001562c524): calling ktype release
  fuse_dev_do_write+0x1888/0x3730 fs/fuse/dev.c:1917
kobject: '0:49': free name
kobject: '0:49' (000000005b47baa2): kobject_add_internal: parent: 'bdi',  
set: 'devices'
kobject: '0:49' (000000005b47baa2): kobject_uevent_env
  fuse_dev_write+0x191/0x240 fs/fuse/dev.c:1941
kobject: '0:49' (000000005b47baa2): fill_kobj_path: path  
= '/devices/virtual/bdi/0:49'
  call_write_iter include/linux/fs.h:1808 [inline]
  new_sync_write fs/read_write.c:474 [inline]
  __vfs_write+0x6e5/0xa80 fs/read_write.c:487
kobject: '0:56' (00000000a2a816b6): kobject_add_internal: parent: 'bdi',  
set: 'devices'
kobject: '0:56' (00000000a2a816b6): kobject_uevent_env
  vfs_write+0x20c/0x560 fs/read_write.c:549
  ksys_write+0x105/0x260 fs/read_write.c:598
kobject: '0:56' (00000000a2a816b6): fill_kobj_path: path  
= '/devices/virtual/bdi/0:56'
  __do_sys_write fs/read_write.c:610 [inline]
  __se_sys_write fs/read_write.c:607 [inline]
  __ia32_sys_write+0x71/0xb0 fs/read_write.c:607
  do_syscall_32_irqs_on arch/x86/entry/common.c:326 [inline]
  do_fast_syscall_32+0x333/0xf98 arch/x86/entry/common.c:397
kobject: '0:57' (000000002c3163ad): kobject_add_internal: parent: 'bdi',  
set: 'devices'
kobject: '0:57' (000000002c3163ad): kobject_uevent_env
  entry_SYSENTER_compat+0x70/0x7f arch/x86/entry/entry_64_compat.S:139
RIP: 0023:0xf7fa0cb9
Code: 85 d2 74 02 89 0a 5b 5d c3 8b 04 24 c3 8b 0c 24 c3 8b 1c 24 c3 90 90  
90 90 90 90 90 90 90 90 90 90 51 52 55 89 e5 0f 34 cd 80 <5d> 5a 59 c3 90  
90 90 90 eb 0d 90 90 90 90 90 90 90 90 90 90 90 90
RSP: 002b:00000000f7f5a0cc EFLAGS: 00000296 ORIG_RAX: 0000000000000004
RAX: ffffffffffffffda RBX: 0000000000000003 RCX: 00000000200002c0
RDX: 0000000000000050 RSI: 0000000000000000 RDI: 0000000000000000
kobject: '0:57' (000000002c3163ad): fill_kobj_path: path  
= '/devices/virtual/bdi/0:57'
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
Kernel Offset: disabled
Rebooting in 86400 seconds..


Tested on:

commit:         bc78abbd fuse: Fix use-after-free in fuse_dev_do_read()
git tree:        
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
console output: https://syzkaller.appspot.com/x/log.txt?x=175a556d200000
kernel config:  https://syzkaller.appspot.com/x/.config?x=eb49a17588446b34
compiler:       gcc (GCC) 9.0.0 20181231 (experimental)
userspace arch: i386


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: WARNING in request_end
  2018-09-24 12:29 WARNING in request_end syzbot
                   ` (2 preceding siblings ...)
  2019-03-23  7:50 ` syzbot
@ 2019-11-07 13:42 ` syzbot
  3 siblings, 0 replies; 11+ messages in thread
From: syzbot @ 2019-11-07 13:42 UTC (permalink / raw)
  To: dvyukov, ebiederm, ktkhai, linux-fsdevel, linux-kernel, miklos,
	mszeredi, syzkaller-bugs

syzbot suspects this bug was fixed by commit:

commit 4c316f2f3ff315cb48efb7435621e5bfb81df96d
Author: Miklos Szeredi <mszeredi@redhat.com>
Date:   Fri Sep 28 14:43:22 2018 +0000

     fuse: set FR_SENT while locked

bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=113124ba600000
start commit:   0238df64 Linux 4.19-rc7
git tree:       upstream
kernel config:  https://syzkaller.appspot.com/x/.config?x=88e9a8a39dc0be2d
dashboard link: https://syzkaller.appspot.com/bug?extid=ef054c4d3f64cd7f7cec
userspace arch: i386
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=119bf2e6400000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=1760f806400000

If the result looks correct, please mark the bug fixed by replying with:

#syz fix: fuse: set FR_SENT while locked

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, back to index

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-09-24 12:29 WARNING in request_end syzbot
2018-09-24 14:44 ` Miklos Szeredi
2018-09-25  9:18   ` Kirill Tkhai
2018-09-25  9:38     ` Dmitry Vyukov
2018-09-25  9:49       ` Kirill Tkhai
2018-10-08  9:38 ` syzbot
2019-03-23  7:50 ` syzbot
2019-03-23 15:51   ` ebiederm
2019-03-23 19:48     ` Miklos Szeredi
2019-03-23 20:16       ` syzbot
2019-11-07 13:42 ` syzbot

Linux-Fsdevel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-fsdevel/0 linux-fsdevel/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-fsdevel linux-fsdevel/ https://lore.kernel.org/linux-fsdevel \
		linux-fsdevel@vger.kernel.org
	public-inbox-index linux-fsdevel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-fsdevel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git