From mboxrd@z Thu Jan 1 00:00:00 1970 From: Thomas Monjalon Subject: Re: [PATCH v3 7/8] mk: sort object files when building deps lists Date: Tue, 27 Jun 2017 18:14:27 +0200 Message-ID: <42831069.yGv8R2R7xR@xps> References: <20170623181616.16981-1-lboccass@brocade.com> <2009930.OQrImQK4oz@xps> <1498575098.16925.5.camel@brocade.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7Bit Cc: dev@dpdk.org To: Luca Boccassi Return-path: Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) by dpdk.org (Postfix) with ESMTP id 2716F2C08 for ; Tue, 27 Jun 2017 18:14:29 +0200 (CEST) In-Reply-To: <1498575098.16925.5.camel@brocade.com> List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" 27/06/2017 16:51, Luca Boccassi: > On Tue, 2017-06-27 at 15:52 +0200, Thomas Monjalon wrote: > > 27/06/2017 12:43, Luca Boccassi: > > > On Tue, 2017-06-27 at 01:20 +0200, Thomas Monjalon wrote: > > > > 23/06/2017 20:41, lboccass@brocade.com: > > > > > From: Luca Boccassi > > > > > > > > > > In order to achieve reproducible builds, always use the same > > > > > order when listing object files to build dependencies lists. > > > > > > > > > > Signed-off-by: Luca Boccassi > > > > > --- > > > > > mk/rte.app.mk | 4 ++-- > > > > > mk/rte.hostapp.mk | 4 ++-- > > > > > mk/rte.shared.mk | 4 ++-- > > > > > 3 files changed, 6 insertions(+), 6 deletions(-) > > > > > > > > > > --- a/mk/rte.app.mk > > > > > +++ b/mk/rte.app.mk > > > > > @@ -263,8 +263,8 @@ LDLIBS_NAMES += $(patsubst -Wl$(comma)- > > > > > l%,lib%.a,$(filter -Wl$(comma)-l%,$(LDLIB > > > > > > > > > > # list of found libraries files (useful for deps). If not > > > > > found, > > > > > the > > > > > # library is silently ignored and dep won't be checked > > > > > -LDLIBS_FILES := $(wildcard $(foreach dir,$(LDLIBS_PATH),\ > > > > > - $(addprefix $(dir)/,$(LDLIBS_NAMES)))) > > > > > +LDLIBS_FILES := $(sort $(wildcard $(foreach > > > > > dir,$(LDLIBS_PATH),\ > > > > > + $(addprefix $(dir)/,$(LDLIBS_NAMES))))) > > > > > > > > You cannot sort libraries. > > > > Check - for instance - this comment above in this file: > > > > # Eliminate duplicates without sorting, only keep the last > > > > occurrence > > > > filter-libs = \ > > > > > > Not sure I follow - what's the reason for avoiding to sort the list > > > of > > > libs to link against? > > > > Sorry, the ordering issue is with LDLIBS not LDLIBS_FILES. > > > > > > Why sorting them? > > > > What is random in libraries list? > > > > > > The issue is that the output of wildcard is not fully > > > deterministic. It > > > can depend on the filesystem, and even on the locale settings [1]. > > > Before GNU Make 3.82 (2009) it used to automatically sort the > > > output, > > > but that was removed (to make it faster... sigh). [2] > > > > It is not a true wildcard here. It is just filtering files which > > do not exist. > > I think you do not need this patch for deterministic build. > > But then those lists are passed down in the .SECONDEXPANSION rule > right? I do not follow you. Please explain what is the benefit of the patch in the next version. > I am trying to find out an alternative solution. The problem to solve > is that the build system picks the public headers path (which is > embedded in the object files as notation and in the debug symbol) from > a seemingly random location between the make install path and the > source path (build/include/rte_foo.h vs lib/librte_foo/rte_foo.h) and > this makes the build not reproducible. > > Nonetheless, while I work more on the last 4 patches, could you please > have a look and eventually take patch 3 and 4? They are needed to > respectively have a deterministic list of files in the libdpdk.so > linker script and a list of source files in one of the example > documentation files. I think it's better to consider and apply the whole series making a reproducible build. The rule is to avoid splitting series without good reason, so tracking patches is easier.