On 06/02/2020 23:16, Jens Axboe wrote:
> On 2/6/20 1:00 PM, Pavel Begunkov wrote:
>> On 06/02/2020 22:56, Jens Axboe wrote:
>>> On 2/6/20 10:16 AM, Pavel Begunkov wrote:
>>>> On 06/02/2020 20:04, Pavel Begunkov wrote:
>>>>> On 06/02/2020 19:51, Pavel Begunkov wrote:
>>>>>> After defer, a request will be prepared, that includes allocating iovec
>>>>>> if needed, and then submitted through io_wq_submit_work() but not custom
>>>>>> handler (e.g. io_rw_async()/io_sendrecv_async()). However, it'll leak
>>>>>> iovec, as it's in io-wq and the code goes as follows:
>>>>>>
>>>>>> io_read() {
>>>>>> 	if (!io_wq_current_is_worker())
>>>>>> 		kfree(iovec);
>>>>>> }
>>>>>>
>>>>>> Put all deallocation logic in io_{read,write,send,recv}(), which will
>>>>>> leave the memory, if going async with -EAGAIN.
>>>>>>
>>>>> Interestingly, this will fail badly if it returns -EAGAIN from io-wq context.
>>>>> Apparently, I need to do v2.
>>>>>
>>>> Or not...
>>>> Jens, can you please explain what's with the -EAGAIN handling in
>>>> io_wq_submit_work()? Checking the code, it seems neither of
>>>> read/write/recv/send can return -EAGAIN from async context (i.e.
>>>> force_nonblock=false). Are there other ops that can do it?
>>>
>>> Nobody should return -EAGAIN with force_nonblock=false, they should
>>> end the io_kiocb inline for that.
>>>
>>
>> If so for those 4, then the patch should work well.
> 
> Maybe I'm dense, but I'm not seeing the leak? We have two cases here:
> 

There is an example:

1. submit a read, which need defer.

2. io_req_defer() allocates ->io and goes io_req_defer_prep() -> io_read_prep().
Let #vecs > UIO_FASTIOV, so the prep() in the presence of ->io will allocate iovec.
Note: that work.func is left io_wq_submit_work

3. At some point @io_wq calls io_wq_submit_work() -> io_issue_sqe() -> io_read(),

4. actual reading succeeds, and it's coming to finalisation and the following
code in particular.

if (!io_wq_current_is_worker())
	kfree(iovec);

5. Because we're in io_wq, the cleanup will not be performed, even though we're
returning with success. And that's a leak.

Do you see anything wrong with it?

> - The number of vecs is less than UIO_FASTIOV, in which case we use the
>   on-stack inline_vecs. If we need to go async, we copy that inline vec
>   to our async_ctx area.
> 
> - The number of vecs is more than UIO_FASTIOV, this is where iovec is
>   allocated by the vec import. If we make it to completion here, we
>   free it at the end of eg io_read(). If we need to go async, we stash
>   that pointer in our async_ctx area and free it when the work item
>   has run and completed.
> 

-- 
Pavel Begunkov