From: Ilya Maximets <i.maximets@ovn.org>
To: Flavio Leitner <fbl@sysclose.org>,
Maxime Coquelin <maxime.coquelin@redhat.com>
Cc: Ilya Maximets <i.maximets@ovn.org>,
dev@dpdk.org, Shahaf Shuler <shahafs@mellanox.com>,
David Marchand <david.marchand@redhat.com>,
Tiwei Bie <tiwei.bie@intel.com>,
Obrembski MichalX <michalx.obrembski@intel.com>,
Stokes Ian <ian.stokes@intel.com>
Subject: Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers
Date: Wed, 16 Oct 2019 16:08:54 +0200 [thread overview]
Message-ID: <ba266dc1-f578-c5e9-af4d-83a814f9138e@ovn.org> (raw)
In-Reply-To: <20191016110243.14005e1c@p50.lan>
On 16.10.2019 16:02, Flavio Leitner wrote:
> On Wed, 16 Oct 2019 15:46:15 +0200
> Maxime Coquelin <maxime.coquelin@redhat.com> wrote:
>
>> On 10/16/19 3:32 PM, Ilya Maximets wrote:
>>> On 16.10.2019 13:13, Maxime Coquelin wrote:
>>>>
>>>>
>>>> On 10/15/19 8:59 PM, Flavio Leitner wrote:
>>>>> The rte_vhost_dequeue_burst supports two ways of dequeuing data.
>>>>> If the data fits into a buffer, then all data is copied and a
>>>>> single linear buffer is returned. Otherwise it allocates
>>>>> additional mbufs and chains them together to return a multiple
>>>>> segments mbuf.
>>>>>
>>>>> While that covers most use cases, it forces applications that
>>>>> need to work with larger data sizes to support multiple segments
>>>>> mbufs. The non-linear characteristic brings complexity and
>>>>> performance implications to the application.
>>>>>
>>>>> To resolve the issue, add support to attach external buffer
>>>>> to a pktmbuf and let the host provide during registration if
>>>>> attaching an external buffer to pktmbuf is supported and if
>>>>> only linear buffer are supported.
>>>>>
>>>>> Signed-off-by: Flavio Leitner <fbl@sysclose.org>
>>>>> ---
>>>>> doc/guides/prog_guide/vhost_lib.rst | 35 +++++++++
>>>>> lib/librte_vhost/rte_vhost.h | 4 +
>>>>> lib/librte_vhost/socket.c | 22 ++++++
>>>>> lib/librte_vhost/vhost.c | 22 ++++++
>>>>> lib/librte_vhost/vhost.h | 4 +
>>>>> lib/librte_vhost/virtio_net.c | 109
>>>>> ++++++++++++++++++++++++---- 6 files changed, 182 insertions(+),
>>>>> 14 deletions(-)
>>>>>
>>>>> - Changelog:
>>>>> v5:
>>>>> - fixed to destroy mutex if incompatible flags
>>>>> v4:
>>>>> - allow to use pktmbuf if there is exact space
>>>>> - removed log message if the buffer is too big
>>>>> - fixed the length to include align padding
>>>>> - free allocated buf if shinfo fails
>>>>> v3:
>>>>> - prevent the new features to be used with zero copy
>>>>> - fixed sizeof() usage
>>>>> - fixed log msg indentation
>>>>> - removed/replaced asserts
>>>>> - used the correct virt2iova function
>>>>> - fixed the patch's title
>>>>> - OvS PoC code:
>>>>> https://github.com/fleitner/ovs/tree/rte_malloc-v3
>>>>> v2:
>>>>> - Used rte_malloc() instead of another mempool as suggested
>>>>> by Shahaf.
>>>>> - Added the documentation section.
>>>>> - Using driver registration to negotiate the features.
>>>>> - OvS PoC code:
>>>>>
>>>>> https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7
>>>>>
>>>>
>>>> Applied to dpdk-next-virtio/master.
>>>
>>> Thanks Maxime,
>>>
>>> But can we return back the mbuf allocation failure message?
>>
>> Good point, I missed it.
>>
>>> I mean:
>>>
>>> diff --git a/lib/librte_vhost/virtio_net.c
>>> b/lib/librte_vhost/virtio_net.c index 66f0c7206..f8af4e0b3 100644
>>> --- a/lib/librte_vhost/virtio_net.c
>>> +++ b/lib/librte_vhost/virtio_net.c
>>> @@ -1354,8 +1354,11 @@ virtio_dev_pktmbuf_alloc(struct virtio_net
>>> *dev, struct rte_mempool *mp,
>>> {
>>> struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
>>>
>>> - if (unlikely(pkt == NULL))
>>> + if (unlikely(pkt == NULL)) {
>>> + RTE_LOG(ERR, VHOST_DATA,
>>> + "Failed to allocate memory for mbuf.\n");
>>> return NULL;
>>> + }
>>>
>>> if (rte_pktmbuf_tailroom(pkt) >= data_len)
>>> return pkt;
>>> ---
>>>
>>> It's a hard failure that highlights some significant issues with
>>> memory pool size or a mbuf leak.
>>
>> I agree, we need this error message.
>
> If it runs out of memory and there are many packets coming, then it
> will flood the logs. Maybe we could use something to report in a more
> moderated way, for example:
>
> diff --git a/lib/librte_vhost/virtio_net.c b/lib/librte_vhost/virtio_net.c
> index da69ab1db..b1572b3a4 100644
> --- a/lib/librte_vhost/virtio_net.c
> +++ b/lib/librte_vhost/virtio_net.c
> @@ -1354,8 +1354,14 @@ virtio_dev_pktmbuf_alloc(struct virtio_net *dev, struct rte_mempool *mp,
> {
> struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
>
> - if (unlikely(pkt == NULL))
> + if (unlikely(pkt == NULL)) {
> + static int rate_lim = 0;
> +
> + if (rate_lim++ % 1000 == 0)
> + RTE_LOG...
> +
> return NULL;
> + }
>
> if (rte_pktmbuf_tailroom(pkt) >= data_len)
> return pkt;
>
>
> Or track this at mempool level and keep stats of failed requests and
> report that in OvS. That would cover other points too.
OVS is already rete-limiting DPDK logs. Basically, I introduced this
functionality to limit exactly this message.
See: 9fd38f6867df ("netdev-dpdk: Limit rate of DPDK logs.")
Best regards, Ilya Maximets.
next prev parent reply other threads:[~2019-10-16 14:09 UTC|newest]
Thread overview: 32+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-01 22:19 [dpdk-dev] [PATCH] vhost: add support to large linear mbufs Flavio Leitner
2019-10-01 23:10 ` Flavio Leitner
2019-10-02 4:45 ` Shahaf Shuler
2019-10-02 8:04 ` David Marchand
2019-10-02 9:00 ` Shahaf Shuler
2019-10-02 12:58 ` Flavio Leitner
2019-10-02 17:50 ` Shahaf Shuler
2019-10-02 18:15 ` Flavio Leitner
2019-10-03 16:57 ` Ilya Maximets
2019-10-03 21:25 ` Flavio Leitner
2019-10-02 7:51 ` Maxime Coquelin
2019-10-04 20:10 ` [dpdk-dev] [PATCH v2] vhost: add support for large buffers Flavio Leitner
2019-10-06 4:47 ` Shahaf Shuler
2019-10-10 5:12 ` Tiwei Bie
2019-10-10 12:12 ` Flavio Leitner
2019-10-11 17:09 ` [dpdk-dev] [PATCH v3] " Flavio Leitner
2019-10-14 2:44 ` Tiwei Bie
2019-10-15 16:17 ` [dpdk-dev] [PATCH v4] " Flavio Leitner
2019-10-15 17:41 ` Ilya Maximets
2019-10-15 18:44 ` Flavio Leitner
2019-10-15 18:59 ` [dpdk-dev] [PATCH v5] " Flavio Leitner
2019-10-16 10:02 ` Maxime Coquelin
2019-10-16 11:13 ` Maxime Coquelin
2019-10-16 13:32 ` Ilya Maximets
2019-10-16 13:46 ` Maxime Coquelin
2019-10-16 14:02 ` Flavio Leitner
2019-10-16 14:08 ` Ilya Maximets [this message]
2019-10-16 14:14 ` Flavio Leitner
2019-10-16 14:05 ` Ilya Maximets
2019-10-29 9:02 ` David Marchand
2019-10-29 12:21 ` Flavio Leitner
2019-10-29 16:19 ` David Marchand
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ba266dc1-f578-c5e9-af4d-83a814f9138e@ovn.org \
--to=i.maximets@ovn.org \
--cc=david.marchand@redhat.com \
--cc=dev@dpdk.org \
--cc=fbl@sysclose.org \
--cc=ian.stokes@intel.com \
--cc=maxime.coquelin@redhat.com \
--cc=michalx.obrembski@intel.com \
--cc=shahafs@mellanox.com \
--cc=tiwei.bie@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).