recvmmsg() behaviour with MSG_PEEK flag

recvmmsg() behaviour with MSG_PEEK flag

[Alexander Babayants]

Hello!

The behaviour of recvmmsg() with MSG_PEEK flag confuses me. I'd expect
it to peek multiple messages at once, but it seems to peek only the
first one, filling each of the provided struct msghdr with a copy of
the first message. I do not see if it is documented anywhere, is it a
bug or intended design?

What I want to achieve is to first peek into the socket, get the size
of each message, then allocate appropriate buffers and read the
messages with the second recvmmsg() call. This seems to be a
relatively common pattern for reading single messages via recvmsg(),
and I naively expected it to work with recvmmsg() too.

-- 
Regards,
Alexander Babayants.

Re: recvmmsg() behaviour with MSG_PEEK flag

[Eric Dumazet]

On Fri, Jul 15, 2022 at 1:03 PM Alexander Babayants
<babayants.alexander@gmail.com> wrote:
>
> Hello!
>
> The behaviour of recvmmsg() with MSG_PEEK flag confuses me. I'd expect
> it to peek multiple messages at once, but it seems to peek only the
> first one, filling each of the provided struct msghdr with a copy of
> the first message. I do not see if it is documented anywhere, is it a
> bug or intended design?
>
> What I want to achieve is to first peek into the socket, get the size
> of each message, then allocate appropriate buffers and read the
> messages with the second recvmmsg() call. This seems to be a
> relatively common pattern for reading single messages via recvmsg(),
> and I naively expected it to work with recvmmsg() too.
>

MSG_PEEK is a mistake really, it considerably increases the cost of
receiving packets,
because you need twice more system calls.

It also increases costs in the kernel, having to deal with it, even if
not used by fast applications.

Adding proper MSG_PEEK support to recvmmsg() is tricky, do_recvmmsg()
would probably have
to be completely reimplemented to not call ___sys_recvmsg(), or risk
some quadratic behavior
if each ___sys_recvmsg() has to skip over N datagrams

Re: recvmmsg() behaviour with MSG_PEEK flag

[Alexander Babayants]

Thanks for the prompt reply!

On Fri, Jul 15, 2022 at 14:21, Eric Dumazet <edumazet@google.com>:
>
...
> MSG_PEEK is a mistake really, it considerably increases the cost of
> receiving packets,
> because you need twice more system calls.

Sure, but I don't know the size of packets in advance, and I don't
control the sending side. So for me it's the choice between multiple
bad options -- either allocate some reasonable sized buffers and
truncate/drop packets that do not fit, or allocate 64k buffer for each
packet (UDP max packet size), or do two syscalls at a time. I thought
I could compensate for the cost of doing two syscalls by reading
multiple packets by syscall, but it seems that it's not possible with
the current implementation. Maybe I'm missing something and there are
other options?

> It also increases costs in the kernel, having to deal with it, even if
> not used by fast applications.
>
> Adding proper MSG_PEEK support to recvmmsg() is tricky, do_recvmmsg()
> would probably have
> to be completely reimplemented to not call ___sys_recvmsg(), or risk
> some quadratic behavior
> if each ___sys_recvmsg() has to skip over N datagrams

If there is no intention to fix the current behaviour, will it be OK
to submit a patch to man-pages with a note about it to the BUGS
section?

-- 
Regards,
Alexander Babayants.

Re: recvmmsg() behaviour with MSG_PEEK flag

[Jann Horn]

On Fri, Jul 15, 2022 at 1:03 PM Alexander Babayants
<babayants.alexander@gmail.com> wrote:
> The behaviour of recvmmsg() with MSG_PEEK flag confuses me. I'd expect
> it to peek multiple messages at once, but it seems to peek only the
> first one, filling each of the provided struct msghdr with a copy of
> the first message. I do not see if it is documented anywhere, is it a
> bug or intended design?
>
> What I want to achieve is to first peek into the socket, get the size
> of each message, then allocate appropriate buffers and read the
> messages with the second recvmmsg() call. This seems to be a
> relatively common pattern for reading single messages via recvmsg(),
> and I naively expected it to work with recvmmsg() too.

You might want to look at SO_PEEK_OFF in the socket.7 manpage. (Or
even better, as suggested before in the thread, rework your code to
not peek at all.)

recvmmsg() behaviour with MSG_PEEK flag

[Alexander Babayants]

(sorry in advance - I'm not sure it is the right mailing list for the
question, if it is not, feel free to redirect me)

The behaviour of recvmmsg() with MSG_PEEK flag confuses me. I'd expect
it to peek multiple messages at once, but it seems to peek only the
first one, filling each of the provided struct msghdr with a copy of
the first message. I do not see if it is documented anywhere, is it a
bug or intended design?

What I want to achieve is to first peek into the socket, get the size
of each message, then allocate appropriate buffers and read the
messages with the second recvmmsg() call. This seems to be a
relatively common pattern for reading single messages via recvmsg(),
and I naively expected it to work with recvmmsg() too.

-- 
Regards,
Alexander Babayants.