From: Jens Axboe <axboe@kernel.dk>
To: Matt Mullins <mmullins@fb.com>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
"linux-aio@kvack.org" <linux-aio@kvack.org>,
"linux-api@vger.kernel.org" <linux-api@vger.kernel.org>
Cc: "hch@lst.de" <hch@lst.de>, "jannh@google.com" <jannh@google.com>,
"viro@ZenIV.linux.org.uk" <viro@ZenIV.linux.org.uk>,
"avi@scylladb.com" <avi@scylladb.com>,
"jmoyer@redhat.com" <jmoyer@redhat.com>
Subject: Re: [PATCH 14/19] io_uring: add file set registration
Date: Sat, 9 Feb 2019 19:57:56 -0700 [thread overview]
Message-ID: <a5a59573-1a61-a2a8-a8a1-24d729ab141a@kernel.dk> (raw)
In-Reply-To: <9e3d7c97-f649-ddb5-4a6f-0ab66742fc86@kernel.dk>
On 2/9/19 7:34 PM, Jens Axboe wrote:
> On 2/9/19 6:11 PM, Matt Mullins wrote:
>> On Sat, 2019-02-09 at 17:47 -0700, Jens Axboe wrote:
>>> On 2/9/19 4:52 PM, Matt Mullins wrote:
>>>>> @@ -1292,6 +1338,154 @@ static int io_cqring_wait(struct io_ring_ctx *ctx, int min_events,
>>>>> return READ_ONCE(ring->r.head) == READ_ONCE(ring->r.tail) ? ret : 0;
>>>>> }
>>>>>
>>>>> +static void __io_sqe_files_unregister(struct io_ring_ctx *ctx)
>>>>> +{
>>>>> +#if defined(CONFIG_UNIX)
>>>>> + if (ctx->ring_sock) {
>>>>> + struct sock *sock = ctx->ring_sock->sk;
>>>>> + struct sk_buff *skb;
>>>>> +
>>>>> + while ((skb = skb_dequeue(&sock->sk_receive_queue)) != NULL)
>>>>
>>>> Something's still a bit messy with destruction. I get a traceback here
>>>> while running
>>>>
>>>> int main() {
>>>> struct io_uring_params uring_params = {
>>>> .flags = IORING_SETUP_SQPOLL | IORING_SETUP_IOPOLL,
>>>> };
>>>> int uring_fd =
>>>> syscall(425 /* io_uring_setup */, 16, &uring_params);
>>>>
>>>> const __s32 fds[] = {1};
>>>>
>>>> syscall(427 /* io_uring_register */, uring_fd,
>>>> IORING_REGISTER_FILES, fds, sizeof(fds) / sizeof(*fds));
>>>> }
>>>>
>>>> I end up with the following spew:
>>>>
>>>> [ 195.983322] WARNING: CPU: 1 PID: 1938 at ../net/unix/af_unix.c:500 unix_sock_destructor+0x97/0xc0
>>>> [ 195.989556] Modules linked in:
>>>> [ 195.992738] CPU: 1 PID: 1938 Comm: aio_buffered Tainted: G W 5.0.0-rc5+ #379
>>>> [ 196.000926] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1ubuntu1 04/01/2014
>>>> [ 196.008316] RIP: 0010:unix_sock_destructor+0x97/0xc0
>>>> [ 196.010912] Code: 3f 37 f3 ff 5b 5d be 00 02 00 00 48 c7 c7 6c 5b 9a 81 e9 8c 2a 71 ff 48 89 ef e8 c4 dc 87 ff eb be 0f 0b 48 83 7b 70 00 74 8b <0f> 0b 48 83 bb 68 02 00 00 00 74 89 0f 0b eb 85 48 89 de 48 c7 c7
>>>> [ 196.018887] RSP: 0018:ffffc900008a7d40 EFLAGS: 00010282
>>>> [ 196.020754] RAX: 0000000000000000 RBX: ffff8881351dd000 RCX: 0000000000000000
>>>> [ 196.022811] RDX: 0000000000000001 RSI: 0000000000000282 RDI: 00000000ffffffff
>>>> [ 196.024901] RBP: ffff8881351dd000 R08: 0000000000024120 R09: ffffffff819a97fe
>>>> [ 196.026977] R10: ffffea0004cf6800 R11: 00000000005b8d80 R12: ffffffff81294ec2
>>>> [ 196.029119] R13: ffff888134e27b40 R14: ffff88813bb307a0 R15: ffff888133d59910
>>>> [ 196.031071] FS: 00007f1a8a8c3740(0000) GS:ffff88813bb00000(0000) knlGS:0000000000000000
>>>> [ 196.033069] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>> [ 196.034438] CR2: 00007f1a8aba5920 CR3: 000000000260e004 CR4: 00000000003606a0
>>>> [ 196.036310] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>>> [ 196.038399] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>>>> [ 196.039794] Call Trace:
>>>> [ 196.040259] __sk_destruct+0x1c/0x150
>>>> [ 196.040964] ? io_sqe_files_unregister+0x32/0x70
>>>> [ 196.041841] unix_destruct_scm+0x76/0xa0
>>>> [ 196.042587] skb_release_head_state+0x38/0x60
>>>> [ 196.043401] skb_release_all+0x9/0x20
>>>> [ 196.044034] kfree_skb+0x2d/0xb0
>>>> [ 196.044603] io_sqe_files_unregister+0x32/0x70
>>>> [ 196.045385] io_ring_ctx_wait_and_kill+0xf6/0x1a0
>>>> [ 196.046220] io_uring_release+0x17/0x20
>>>> [ 196.046881] __fput+0x9d/0x1d0
>>>> [ 196.047421] task_work_run+0x7a/0x90
>>>> [ 196.048045] do_exit+0x301/0xc20
>>>> [ 196.048626] ? handle_mm_fault+0xf3/0x230
>>>> [ 196.049321] do_group_exit+0x35/0xa0
>>>> [ 196.049944] __x64_sys_exit_group+0xf/0x10
>>>> [ 196.050658] do_syscall_64+0x3d/0xf0
>>>> [ 196.051317] entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>> [ 196.052217] RIP: 0033:0x7f1a8aba5956
>>>> [ 196.052859] Code: Bad RIP value.
>>>> [ 196.053488] RSP: 002b:00007fffbdbcad38 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
>>>> [ 196.054902] RAX: ffffffffffffffda RBX: 00007f1a8ac975c0 RCX: 00007f1a8aba5956
>>>> [ 196.056124] RDX: 0000000000000000 RSI: 000000000000003c RDI: 0000000000000000
>>>> [ 196.057348] RBP: 0000000000000000 R08: 00000000000000e7 R09: ffffffffffffff78
>>>> [ 196.058573] R10: 00007fffbdbcabf8 R11: 0000000000000246 R12: 00007f1a8ac975c0
>>>> [ 196.059459] R13: 0000000000000001 R14: 00007f1a8aca0288 R15: 0000000000000000
>>>> [ 196.060731] ---[ end trace 8a7e42f9199e5f92 ]---
>>>> [ 196.062671] WARNING: CPU: 1 PID: 1938 at ../net/unix/af_unix.c:501 unix_sock_destructor+0xa3/0xc0
>>>> [ 196.064372] Modules linked in:
>>>> [ 196.064966] CPU: 1 PID: 1938 Comm: aio_buffered Tainted: G W 5.0.0-rc5+ #379
>>>> [ 196.066546] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1ubuntu1 04/01/2014
>>>> [ 196.068234] RIP: 0010:unix_sock_destructor+0xa3/0xc0
>>>> [ 196.068999] Code: c7 c7 6c 5b 9a 81 e9 8c 2a 71 ff 48 89 ef e8 c4 dc 87 ff eb be 0f 0b 48 83 7b 70 00 74 8b 0f 0b 48 83 bb 68 02 00 00 00 74 89 <0f> 0b eb 85 48 89 de 48 c7 c7 a0 c8 42 82 5b 5d e9 31 8c 75 ff 0f
>>>> [ 196.072577] RSP: 0018:ffffc900008a7d40 EFLAGS: 00010282
>>>> [ 196.073595] RAX: 0000000000000000 RBX: ffff8881351dd000 RCX: 0000000000000000
>>>> [ 196.074973] RDX: 0000000000000001 RSI: 0000000000000282 RDI: 00000000ffffffff
>>>> [ 196.076348] RBP: ffff8881351dd000 R08: 0000000000024120 R09: ffffffff819a97fe
>>>> [ 196.077709] R10: ffffea0004cf6800 R11: 00000000005b8d80 R12: ffffffff81294ec2
>>>> [ 196.079072] R13: ffff888134e27b40 R14: ffff88813bb307a0 R15: ffff888133d59910
>>>> [ 196.080441] FS: 00007f1a8a8c3740(0000) GS:ffff88813bb00000(0000) knlGS:0000000000000000
>>>> [ 196.082026] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>> [ 196.083131] CR2: 00007fbc19f96550 CR3: 0000000138d1e003 CR4: 00000000003606a0
>>>> [ 196.084505] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>>> [ 196.085823] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>>>> [ 196.087185] Call Trace:
>>>> [ 196.087662] __sk_destruct+0x1c/0x150
>>>> [ 196.088376] ? io_sqe_files_unregister+0x32/0x70
>>>> [ 196.089299] unix_destruct_scm+0x76/0xa0
>>>> [ 196.090059] skb_release_head_state+0x38/0x60
>>>> [ 196.090929] skb_release_all+0x9/0x20
>>>> [ 196.091550] kfree_skb+0x2d/0xb0
>>>> [ 196.092745] io_sqe_files_unregister+0x32/0x70
>>>> [ 196.093535] io_ring_ctx_wait_and_kill+0xf6/0x1a0
>>>> [ 196.094358] io_uring_release+0x17/0x20
>>>> [ 196.095029] __fput+0x9d/0x1d0
>>>> [ 196.095660] task_work_run+0x7a/0x90
>>>> [ 196.096307] do_exit+0x301/0xc20
>>>> [ 196.096808] ? handle_mm_fault+0xf3/0x230
>>>> [ 196.097504] do_group_exit+0x35/0xa0
>>>> [ 196.098126] __x64_sys_exit_group+0xf/0x10
>>>> [ 196.098836] do_syscall_64+0x3d/0xf0
>>>> [ 196.099460] entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>> [ 196.100334] RIP: 0033:0x7f1a8aba5956
>>>> [ 196.100958] Code: Bad RIP value.
>>>> [ 196.101293] RSP: 002b:00007fffbdbcad38 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
>>>> [ 196.101933] RAX: ffffffffffffffda RBX: 00007f1a8ac975c0 RCX: 00007f1a8aba5956
>>>> [ 196.102535] RDX: 0000000000000000 RSI: 000000000000003c RDI: 0000000000000000
>>>> [ 196.103137] RBP: 0000000000000000 R08: 00000000000000e7 R09: ffffffffffffff78
>>>> [ 196.103739] R10: 00007fffbdbcabf8 R11: 0000000000000246 R12: 00007f1a8ac975c0
>>>> [ 196.104526] R13: 0000000000000001 R14: 00007f1a8aca0288 R15: 0000000000000000
>>>> [ 196.105777] ---[ end trace 8a7e42f9199e5f93 ]---
>>>> [ 196.107535] unix: Attempt to release alive unix socket: 000000003b3c1a34
>>>>
>>>> which corresponds to the WARN_ONs:
>>>>
>>>> WARN_ON(!sk_unhashed(sk));
>>>> WARN_ON(sk->sk_socket);
>>>>
>>>> This doesn't seem to happen if I omit the call to io_uring_register.
>>>
>>> Huh, I can't reproduce that here, teardown seems to work just fine. It
>>> looks like the socket is getting torn down prematurely, when we free the
>>> skb. I wonder if you have some networking options I don't? What's your
>>> .config?
>>>
>>
>> Interesting. Attached is the config I'm using to build
>> af22d31f8b09fa36f57569c95f4943febaacb2b1. I'll keep playing with it on
>> my end, too, maybe I've got something bad in my ccache.
>
> Bingo, reproduces with your .config. Looks like the io_uring is released
> basically as soon as we queue the skb in the socket. I'll take a look at
> this tomorrow.
OK, I think I see it. Apparently with my options, the size of the skb is
0 when I pass in 0. With your options, it's non-zero, which wreaks havoc
on the ref counting.
The below should fix it. I'll fold this in now.
diff --git a/fs/io_uring.c b/fs/io_uring.c
index d7a10484d748..c8794a11de3e 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -2006,6 +2006,7 @@ static int __io_sqe_files_scm(struct io_ring_ctx *ctx, int nr, int offset)
fpl->max = fpl->count = nr;
UNIXCB(skb).fp = fpl;
+ refcount_add(skb->truesize, &ctx->ring_sock->sk->sk_wmem_alloc);
skb_queue_head(&ctx->ring_sock->sk->sk_receive_queue, skb);
for (i = 0; i < nr; i++)
--
Jens Axboe
WARNING: multiple messages have this Message-ID (diff)
From: Jens Axboe <axboe@kernel.dk>
To: Matt Mullins <mmullins@fb.com>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
"linux-aio@kvack.org" <linux-aio@kvack.org>,
"linux-api@vger.kernel.org" <linux-api@vger.kernel.org>
Cc: "hch@lst.de" <hch@lst.de>, "jannh@google.com" <jannh@google.com>,
"viro@ZenIV.linux.org.uk" <viro@ZenIV.linux.org.uk>,
"avi@scylladb.com" <avi@scylladb.com>,
"jmoyer@redhat.com" <jmoyer@redhat.com>
Subject: Re: [PATCH 14/19] io_uring: add file set registration
Date: Sat, 9 Feb 2019 19:57:56 -0700 [thread overview]
Message-ID: <a5a59573-1a61-a2a8-a8a1-24d729ab141a@kernel.dk> (raw)
In-Reply-To: <9e3d7c97-f649-ddb5-4a6f-0ab66742fc86@kernel.dk>
On 2/9/19 7:34 PM, Jens Axboe wrote:
> On 2/9/19 6:11 PM, Matt Mullins wrote:
>> On Sat, 2019-02-09 at 17:47 -0700, Jens Axboe wrote:
>>> On 2/9/19 4:52 PM, Matt Mullins wrote:
>>>>> @@ -1292,6 +1338,154 @@ static int io_cqring_wait(struct io_ring_ctx *ctx, int min_events,
>>>>> return READ_ONCE(ring->r.head) == READ_ONCE(ring->r.tail) ? ret : 0;
>>>>> }
>>>>>
>>>>> +static void __io_sqe_files_unregister(struct io_ring_ctx *ctx)
>>>>> +{
>>>>> +#if defined(CONFIG_UNIX)
>>>>> + if (ctx->ring_sock) {
>>>>> + struct sock *sock = ctx->ring_sock->sk;
>>>>> + struct sk_buff *skb;
>>>>> +
>>>>> + while ((skb = skb_dequeue(&sock->sk_receive_queue)) != NULL)
>>>>
>>>> Something's still a bit messy with destruction. I get a traceback here
>>>> while running
>>>>
>>>> int main() {
>>>> struct io_uring_params uring_params = {
>>>> .flags = IORING_SETUP_SQPOLL | IORING_SETUP_IOPOLL,
>>>> };
>>>> int uring_fd =
>>>> syscall(425 /* io_uring_setup */, 16, &uring_params);
>>>>
>>>> const __s32 fds[] = {1};
>>>>
>>>> syscall(427 /* io_uring_register */, uring_fd,
>>>> IORING_REGISTER_FILES, fds, sizeof(fds) / sizeof(*fds));
>>>> }
>>>>
>>>> I end up with the following spew:
>>>>
>>>> [ 195.983322] WARNING: CPU: 1 PID: 1938 at ../net/unix/af_unix.c:500 unix_sock_destructor+0x97/0xc0
>>>> [ 195.989556] Modules linked in:
>>>> [ 195.992738] CPU: 1 PID: 1938 Comm: aio_buffered Tainted: G W 5.0.0-rc5+ #379
>>>> [ 196.000926] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1ubuntu1 04/01/2014
>>>> [ 196.008316] RIP: 0010:unix_sock_destructor+0x97/0xc0
>>>> [ 196.010912] Code: 3f 37 f3 ff 5b 5d be 00 02 00 00 48 c7 c7 6c 5b 9a 81 e9 8c 2a 71 ff 48 89 ef e8 c4 dc 87 ff eb be 0f 0b 48 83 7b 70 00 74 8b <0f> 0b 48 83 bb 68 02 00 00 00 74 89 0f 0b eb 85 48 89 de 48 c7 c7
>>>> [ 196.018887] RSP: 0018:ffffc900008a7d40 EFLAGS: 00010282
>>>> [ 196.020754] RAX: 0000000000000000 RBX: ffff8881351dd000 RCX: 0000000000000000
>>>> [ 196.022811] RDX: 0000000000000001 RSI: 0000000000000282 RDI: 00000000ffffffff
>>>> [ 196.024901] RBP: ffff8881351dd000 R08: 0000000000024120 R09: ffffffff819a97fe
>>>> [ 196.026977] R10: ffffea0004cf6800 R11: 00000000005b8d80 R12: ffffffff81294ec2
>>>> [ 196.029119] R13: ffff888134e27b40 R14: ffff88813bb307a0 R15: ffff888133d59910
>>>> [ 196.031071] FS: 00007f1a8a8c3740(0000) GS:ffff88813bb00000(0000) knlGS:0000000000000000
>>>> [ 196.033069] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>> [ 196.034438] CR2: 00007f1a8aba5920 CR3: 000000000260e004 CR4: 00000000003606a0
>>>> [ 196.036310] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>>> [ 196.038399] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>>>> [ 196.039794] Call Trace:
>>>> [ 196.040259] __sk_destruct+0x1c/0x150
>>>> [ 196.040964] ? io_sqe_files_unregister+0x32/0x70
>>>> [ 196.041841] unix_destruct_scm+0x76/0xa0
>>>> [ 196.042587] skb_release_head_state+0x38/0x60
>>>> [ 196.043401] skb_release_all+0x9/0x20
>>>> [ 196.044034] kfree_skb+0x2d/0xb0
>>>> [ 196.044603] io_sqe_files_unregister+0x32/0x70
>>>> [ 196.045385] io_ring_ctx_wait_and_kill+0xf6/0x1a0
>>>> [ 196.046220] io_uring_release+0x17/0x20
>>>> [ 196.046881] __fput+0x9d/0x1d0
>>>> [ 196.047421] task_work_run+0x7a/0x90
>>>> [ 196.048045] do_exit+0x301/0xc20
>>>> [ 196.048626] ? handle_mm_fault+0xf3/0x230
>>>> [ 196.049321] do_group_exit+0x35/0xa0
>>>> [ 196.049944] __x64_sys_exit_group+0xf/0x10
>>>> [ 196.050658] do_syscall_64+0x3d/0xf0
>>>> [ 196.051317] entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>> [ 196.052217] RIP: 0033:0x7f1a8aba5956
>>>> [ 196.052859] Code: Bad RIP value.
>>>> [ 196.053488] RSP: 002b:00007fffbdbcad38 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
>>>> [ 196.054902] RAX: ffffffffffffffda RBX: 00007f1a8ac975c0 RCX: 00007f1a8aba5956
>>>> [ 196.056124] RDX: 0000000000000000 RSI: 000000000000003c RDI: 0000000000000000
>>>> [ 196.057348] RBP: 0000000000000000 R08: 00000000000000e7 R09: ffffffffffffff78
>>>> [ 196.058573] R10: 00007fffbdbcabf8 R11: 0000000000000246 R12: 00007f1a8ac975c0
>>>> [ 196.059459] R13: 0000000000000001 R14: 00007f1a8aca0288 R15: 0000000000000000
>>>> [ 196.060731] ---[ end trace 8a7e42f9199e5f92 ]---
>>>> [ 196.062671] WARNING: CPU: 1 PID: 1938 at ../net/unix/af_unix.c:501 unix_sock_destructor+0xa3/0xc0
>>>> [ 196.064372] Modules linked in:
>>>> [ 196.064966] CPU: 1 PID: 1938 Comm: aio_buffered Tainted: G W 5.0.0-rc5+ #379
>>>> [ 196.066546] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1ubuntu1 04/01/2014
>>>> [ 196.068234] RIP: 0010:unix_sock_destructor+0xa3/0xc0
>>>> [ 196.068999] Code: c7 c7 6c 5b 9a 81 e9 8c 2a 71 ff 48 89 ef e8 c4 dc 87 ff eb be 0f 0b 48 83 7b 70 00 74 8b 0f 0b 48 83 bb 68 02 00 00 00 74 89 <0f> 0b eb 85 48 89 de 48 c7 c7 a0 c8 42 82 5b 5d e9 31 8c 75 ff 0f
>>>> [ 196.072577] RSP: 0018:ffffc900008a7d40 EFLAGS: 00010282
>>>> [ 196.073595] RAX: 0000000000000000 RBX: ffff8881351dd000 RCX: 0000000000000000
>>>> [ 196.074973] RDX: 0000000000000001 RSI: 0000000000000282 RDI: 00000000ffffffff
>>>> [ 196.076348] RBP: ffff8881351dd000 R08: 0000000000024120 R09: ffffffff819a97fe
>>>> [ 196.077709] R10: ffffea0004cf6800 R11: 00000000005b8d80 R12: ffffffff81294ec2
>>>> [ 196.079072] R13: ffff888134e27b40 R14: ffff88813bb307a0 R15: ffff888133d59910
>>>> [ 196.080441] FS: 00007f1a8a8c3740(0000) GS:ffff88813bb00000(0000) knlGS:0000000000000000
>>>> [ 196.082026] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>>>> [ 196.083131] CR2: 00007fbc19f96550 CR3: 0000000138d1e003 CR4: 00000000003606a0
>>>> [ 196.084505] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>>> [ 196.085823] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
>>>> [ 196.087185] Call Trace:
>>>> [ 196.087662] __sk_destruct+0x1c/0x150
>>>> [ 196.088376] ? io_sqe_files_unregister+0x32/0x70
>>>> [ 196.089299] unix_destruct_scm+0x76/0xa0
>>>> [ 196.090059] skb_release_head_state+0x38/0x60
>>>> [ 196.090929] skb_release_all+0x9/0x20
>>>> [ 196.091550] kfree_skb+0x2d/0xb0
>>>> [ 196.092745] io_sqe_files_unregister+0x32/0x70
>>>> [ 196.093535] io_ring_ctx_wait_and_kill+0xf6/0x1a0
>>>> [ 196.094358] io_uring_release+0x17/0x20
>>>> [ 196.095029] __fput+0x9d/0x1d0
>>>> [ 196.095660] task_work_run+0x7a/0x90
>>>> [ 196.096307] do_exit+0x301/0xc20
>>>> [ 196.096808] ? handle_mm_fault+0xf3/0x230
>>>> [ 196.097504] do_group_exit+0x35/0xa0
>>>> [ 196.098126] __x64_sys_exit_group+0xf/0x10
>>>> [ 196.098836] do_syscall_64+0x3d/0xf0
>>>> [ 196.099460] entry_SYSCALL_64_after_hwframe+0x44/0xa9
>>>> [ 196.100334] RIP: 0033:0x7f1a8aba5956
>>>> [ 196.100958] Code: Bad RIP value.
>>>> [ 196.101293] RSP: 002b:00007fffbdbcad38 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
>>>> [ 196.101933] RAX: ffffffffffffffda RBX: 00007f1a8ac975c0 RCX: 00007f1a8aba5956
>>>> [ 196.102535] RDX: 0000000000000000 RSI: 000000000000003c RDI: 0000000000000000
>>>> [ 196.103137] RBP: 0000000000000000 R08: 00000000000000e7 R09: ffffffffffffff78
>>>> [ 196.103739] R10: 00007fffbdbcabf8 R11: 0000000000000246 R12: 00007f1a8ac975c0
>>>> [ 196.104526] R13: 0000000000000001 R14: 00007f1a8aca0288 R15: 0000000000000000
>>>> [ 196.105777] ---[ end trace 8a7e42f9199e5f93 ]---
>>>> [ 196.107535] unix: Attempt to release alive unix socket: 000000003b3c1a34
>>>>
>>>> which corresponds to the WARN_ONs:
>>>>
>>>> WARN_ON(!sk_unhashed(sk));
>>>> WARN_ON(sk->sk_socket);
>>>>
>>>> This doesn't seem to happen if I omit the call to io_uring_register.
>>>
>>> Huh, I can't reproduce that here, teardown seems to work just fine. It
>>> looks like the socket is getting torn down prematurely, when we free the
>>> skb. I wonder if you have some networking options I don't? What's your
>>> .config?
>>>
>>
>> Interesting. Attached is the config I'm using to build
>> af22d31f8b09fa36f57569c95f4943febaacb2b1. I'll keep playing with it on
>> my end, too, maybe I've got something bad in my ccache.
>
> Bingo, reproduces with your .config. Looks like the io_uring is released
> basically as soon as we queue the skb in the socket. I'll take a look at
> this tomorrow.
OK, I think I see it. Apparently with my options, the size of the skb is
0 when I pass in 0. With your options, it's non-zero, which wreaks havoc
on the ref counting.
The below should fix it. I'll fold this in now.
diff --git a/fs/io_uring.c b/fs/io_uring.c
index d7a10484d748..c8794a11de3e 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -2006,6 +2006,7 @@ static int __io_sqe_files_scm(struct io_ring_ctx *ctx, int nr, int offset)
fpl->max = fpl->count = nr;
UNIXCB(skb).fp = fpl;
+ refcount_add(skb->truesize, &ctx->ring_sock->sk->sk_wmem_alloc);
skb_queue_head(&ctx->ring_sock->sk->sk_receive_queue, skb);
for (i = 0; i < nr; i++)
--
Jens Axboe
--
To unsubscribe, send a message with 'unsubscribe linux-aio' in
the body to majordomo@kvack.org. For more info on Linux AIO,
see: http://www.kvack.org/aio/
Don't email: <a href=mailto:"aart@kvack.org">aart@kvack.org</a>
next prev parent reply other threads:[~2019-02-10 2:58 UTC|newest]
Thread overview: 77+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-09 21:13 [PATCHSET v14] io_uring IO interface Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 01/19] fs: add an iopoll method to struct file_operations Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 02/19] block: wire up block device iopoll method Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 03/19] block: add bio_set_polled() helper Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 04/19] iomap: wire up the iopoll method Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 05/19] Add io_uring IO interface Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-10 12:03 ` Thomas Gleixner
2019-02-10 12:03 ` Thomas Gleixner
2019-02-10 14:19 ` Jens Axboe
2019-02-10 14:19 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 06/19] io_uring: add fsync support Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 07/19] io_uring: support for IO polling Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 08/19] fs: add fget_many() and fput_many() Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 09/19] io_uring: use fget/fput_many() for file references Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 10/19] io_uring: batch io_kiocb allocation Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 11/19] block: implement bio helper to add iter bvec pages to bio Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 12/19] io_uring: add support for pre-mapped user IO buffers Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 13/19] net: split out functions related to registering inflight socket files Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 14/19] io_uring: add file set registration Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 23:52 ` Matt Mullins
2019-02-10 0:47 ` Jens Axboe
2019-02-10 0:47 ` Jens Axboe
2019-02-10 1:11 ` Matt Mullins
2019-02-10 2:34 ` Jens Axboe
2019-02-10 2:34 ` Jens Axboe
2019-02-10 2:57 ` Jens Axboe [this message]
2019-02-10 2:57 ` Jens Axboe
2019-02-10 19:55 ` Matt Mullins
2019-02-09 21:13 ` [PATCH 15/19] io_uring: add submission polling Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-09 21:13 ` [PATCH 16/19] io_uring: add io_kiocb ref count Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-10 9:19 ` Hannes Reinecke
2019-02-10 9:19 ` Hannes Reinecke
2019-02-09 21:13 ` [PATCH 17/19] io_uring: add support for IORING_OP_POLL Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-10 9:28 ` Hannes Reinecke
2019-02-10 9:28 ` Hannes Reinecke
2019-02-09 21:13 ` [PATCH 18/19] io_uring: allow workqueue item to handle multiple buffered requests Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-10 9:31 ` Hannes Reinecke
2019-02-10 9:31 ` Hannes Reinecke
2019-02-09 21:13 ` [PATCH 19/19] io_uring: add io_uring_event cache hit information Jens Axboe
2019-02-09 21:13 ` Jens Axboe
2019-02-10 9:36 ` Hannes Reinecke
2019-02-10 9:36 ` Hannes Reinecke
2019-02-10 13:39 ` Jens Axboe
2019-02-10 13:39 ` Jens Axboe
-- strict thread matches above, loose matches on Subject: below --
2019-02-11 19:00 [PATCHSET v15] io_uring IO interface Jens Axboe
2019-02-11 19:00 ` [PATCH 14/19] io_uring: add file set registration Jens Axboe
2019-02-11 19:00 ` Jens Axboe
2019-02-19 16:12 ` Jann Horn
2019-02-19 16:12 ` Jann Horn
2019-02-22 22:29 ` Jens Axboe
2019-02-22 22:29 ` Jens Axboe
2019-02-08 17:34 [PATCHSET v13] io_uring IO interface Jens Axboe
2019-02-08 17:34 ` [PATCH 14/19] io_uring: add file set registration Jens Axboe
2019-02-08 17:34 ` Jens Axboe
2019-02-08 20:26 ` Jann Horn
2019-02-08 20:26 ` Jann Horn
2019-02-09 0:16 ` Jens Axboe
2019-02-09 0:16 ` Jens Axboe
2019-02-09 9:50 ` Hannes Reinecke
2019-02-09 9:50 ` Hannes Reinecke
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a5a59573-1a61-a2a8-a8a1-24d729ab141a@kernel.dk \
--to=axboe@kernel.dk \
--cc=avi@scylladb.com \
--cc=hch@lst.de \
--cc=jannh@google.com \
--cc=jmoyer@redhat.com \
--cc=linux-aio@kvack.org \
--cc=linux-api@vger.kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=mmullins@fb.com \
--cc=viro@ZenIV.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.