From mboxrd@z Thu Jan 1 00:00:00 1970 From: Olivier MATZ Subject: Re: [PATCH 2/2] mbuf: make sure userdata is initialized Date: Wed, 15 Jul 2015 12:10:43 +0200 Message-ID: <55A631A3.1000309@6wind.com> References: <1436485068-30609-1-git-send-email-stephen@networkplumber.org> <1436485068-30609-3-git-send-email-stephen@networkplumber.org> <20150710093217.GB10556@bricha3-MOBL3> <20150710084356.37d22b65@urahara> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Cc: dev@dpdk.org, Stephen Hemminger To: Stephen Hemminger , Bruce Richardson Return-path: Received: from mail-wi0-f171.google.com (mail-wi0-f171.google.com [209.85.212.171]) by dpdk.org (Postfix) with ESMTP id 9344B5A7A for ; Wed, 15 Jul 2015 12:10:50 +0200 (CEST) Received: by wibud3 with SMTP id ud3so36310668wib.0 for ; Wed, 15 Jul 2015 03:10:50 -0700 (PDT) In-Reply-To: <20150710084356.37d22b65@urahara> List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 07/10/2015 05:43 PM, Stephen Hemminger wrote: > On Fri, 10 Jul 2015 10:32:17 +0100 > Bruce Richardson wrote: > >> On Thu, Jul 09, 2015 at 04:37:48PM -0700, Stephen Hemminger wrote: >>> From: Stephen Hemminger >>> >>> For applications that use m->userdata the initialization can >>> be a signficant (10%) performance penalty. >>> >>> Rather than taking the cache penalty of initializing userdata >>> in the receive handling, do it in the place where mbuf is >>> already cache hot and being setup. >> >> Should the management of the userdata field not be the responsibility of the >> app itself, rather than having the PMD manage it? If the PMD does manage the >> userdata field, I would suggest taking the approach of having the field cleared >> by the mbuf library on free, rather than on RX. > > The problem with that is m->userdata is that when the application > gets the mbuf, touching the userdata field causes false cache > sharing and we see a significant performance drop. Internally the > userdata field is only used for few special cases and 0/NULL > is used to indicate that no metadata is present. > I agree with Bruce: the userdata field is in the second cache line, and we should avoid touching it in rx path. Resetting it in mbuf_free() may not be the solution either: in tx path, the mbufs are freed when we want to send a new mbuf on the hw ring entry, and the mbuf may not be in the cache anymore. I don't understand why doing it in the application induces a performance drop compared to the PMD. What do you mean by false cache sharing?