DPDK-dev Archive on lore.kernel.org
 help / color / Atom feed
From: Andrzej Ostruszka <amo@semihalf.com>
To: Thomas Monjalon <thomas@monjalon.net>
Cc: dev@dpdk.org, Aaron Conole <aconole@redhat.com>,
	Michael Santana <msantana@redhat.com>,
	John McNamara <john.mcnamara@intel.com>,
	Marko Kovacevic <marko.kovacevic@intel.com>,
	David Hunt <david.hunt@intel.com>,
	Bruce Richardson <bruce.richardson@intel.com>,
	Vladimir Medvedkin <vladimir.medvedkin@intel.com>,
	Robert Sanford <rsanford@akamai.com>,
	Erik Gabriel Carrillo <erik.g.carrillo@intel.com>,
	mattias.ronnblom@ericsson.com, stephen@networkplumber.org
Subject: Re: [dpdk-dev] [PATCH v4 01/10] build: add an option to enable LTO build
Date: Mon, 28 Oct 2019 11:47:47 +0100
Message-ID: <1bb37fbc-bf02-9485-fcb5-0c342ccee4f6@semihalf.com> (raw)
In-Reply-To: <12578231.CQkhPFEoQU@xps>

Thank you Thomas for the comments.  Please see below for replies.

On 10/27/19 12:47 PM, Thomas Monjalon wrote:
> Adding John and Marko for doc review.
> 
> 22/10/2019 13:54, Andrzej Ostruszka:
>> --- /dev/null
>> +++ b/doc/guides/prog_guide/lto.rst
> 
> This new file is not included in the prog guide index.

Will add it (between "writing efficient code" and "profile app").

>> @@ -0,0 +1,36 @@
> 
> You are missing the mandatory SPDX and Copyright header.

Will add.

>> +Link Time Optimization
>> +======================
>> +
>> +The DPDK framework supports compilation with link time optimization
> 
> Instead of "The DPDK framework", I prefer just "DPDK".

OK

>> +turned on.  This depends obviously on the capabilities of the compiler
> 
> Not sure what you are talking about.
> If it is about fat objects, it is not obvious.
> 
>> +to do "whole program" optimization at link time and is available only

That was not a reference to some formally defined "capabilities" just an
"obvious" statement that in order to benefit from LTO one needs a
compiler "capable of" performing it.  I will rephrase this paragraph as:

-8<-------------
The DPDK supports compilation with link time optimization turned on.
This depends obviously on the ability of the compiler to do "whole
program" optimization at link time and is available only for compilers
that support that feature (gcc and icc).
To be more specific, compiler (in addition to performing LTO) have to
support creation of ELF objects containing both normal code and internal
representation (fat-lto-objects).  This is required since during build
some code is generated by parsing produced ELF objects (pmdinfogen).
-8<-------------

Will it suffice?  If not, then please suggest wording to be used.

>> +for compilers that support that feature (gcc and icc).  To be more
> 
> Please start a new sentence at the beginning of a line in the RsT file.
> It is easier to review and update.

OK

>> +specific compiler have to support creation of ELF objects containing
> 
> A comma is missing after "specific".

OK

>> +both normal code and internal representation (fat-lto-objects).  This is
>> +required since during build some code is generated by parsing produced
>> +ELF objects (pmdinfogen).
>> +
>> +The amount of performance gain that one can get from LTO depends on the
>> +compiler and the code that is being compiled.  However LTO is also
>> +useful for additional code analysis done by the compiler.  In particular
>> +due to interprocedural analysis compiler can produce additional warnings
>> +about variables that might be used uninitialized.  Some of these
>> +warnings might be "false positives" though and you might need to
>> +explicitly initialize variable in order to silence the compiler.
> 
> Any word about build speed degradation?

It is mentioned at the end of this file - I will move it here.

>> +Link time optimization can be enabled for whole DPDK framework by
>> +setting:
>> +
>> +.. code-block:: console
>> +    CONFIG_ENABLE_LTO=y
>> +
>> +in config file for the case of make based build and by:
>> +
>> +.. code-block:: console
>> +    meson build -Db_lto=true
>> +    ninja -C build
> 
> No need to add the ninja step here.

OK

>> +for the case of meson based build.
> 
> Better to describe the case (make or meson) before the code-block.

OK

>> +Please note that turning LTO on causes considerable extension of
>> +compilation time.
> 
> "compilation time" is not accurate.
> When referring to compilation + linking, it is better to use the words
> "build time".

OK

>> --- a/doc/guides/rel_notes/release_19_11.rst
>> +++ b/doc/guides/rel_notes/release_19_11.rst
>> @@ -56,6 +56,14 @@ New Features
>>       Also, make sure to start the actual text at the margin.
>>       =========================================================
>>  
>> +**Added build support for Link Time Optimization.**
>> +
>> + LTO is an optimization technique used by the compiler to perform whole
>> + program analysis and optimization at link time.  In order to do that
>> + compilers store their internal representation of the source code that
>> + the linker uses at the final stage of compilation process.
>> +
>> + See :doc:`../prog_guide/lto` for more information:
> 
> Please rebase and add a blank line when adding a paragraph.

OK

[...]
> I wanted to merge this series in 19.11-rc1 but I think it is better
> to move its merge in 19.11-rc2.

OK.  I'll send the next version today.

Thank you again for the comments.

Regards
Andrzej

  reply index

Thread overview: 110+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-17  7:57 [dpdk-dev] [PATCH v2 00/10] Add an option to use LTO for DPDK build Andrzej Ostruszka
2019-09-17  7:57 ` [dpdk-dev] [PATCH v2 01/10] build: add an option to enable LTO build Andrzej Ostruszka
2019-09-18 10:36   ` Bruce Richardson
2019-09-18 13:32   ` Ray Kinsella
2019-09-19 12:35     ` Andrzej Ostruszka
2019-09-19 13:28       ` Ray Kinsella
2019-09-19 15:16         ` Bruce Richardson
2019-09-20  7:38           ` Ray Kinsella
2019-09-23  7:23             ` Thomas Monjalon
2019-09-23  9:36               ` Ray Kinsella
2019-09-23 10:16                 ` Mattias Rönnblom
2019-09-23 12:03               ` Andrzej Ostruszka
2019-09-23 12:06                 ` Bruce Richardson
2019-09-23 13:02                   ` Andrzej Ostruszka
2019-09-23 16:13                     ` Bruce Richardson
2019-09-24  6:46                       ` Andrzej Ostruszka
2019-09-24 10:25                         ` Bruce Richardson
2019-09-24 11:52                           ` Andrzej Ostruszka
2019-09-24 12:11                             ` Bruce Richardson
2019-09-24 12:59                           ` Neil Horman
2019-09-24 16:01                             ` Ray Kinsella
2019-09-26 15:32                             ` Andrzej Ostruszka
2019-09-27 19:55                               ` Bruce Richardson
2019-09-23 12:16                 ` Ray Kinsella
2019-10-27 11:31     ` Thomas Monjalon
2019-10-28  8:36       ` Andrzej Ostruszka
2019-10-28  9:07         ` Thomas Monjalon
2019-10-28 12:12         ` Andrzej Ostruszka
2019-10-28 17:16           ` Thomas Monjalon
2019-09-17  7:57 ` [dpdk-dev] [PATCH v2 02/10] eventdev: fix possible use of uninitialized var Andrzej Ostruszka
2019-10-12 13:35   ` Jerin Jacob
2019-09-17  7:57 ` [dpdk-dev] [PATCH v2 03/10] app/eventdev: fix maybe-uninitialized warnings for LTO build Andrzej Ostruszka
2019-10-12 13:52   ` Jerin Jacob
2019-09-17  7:57 ` [dpdk-dev] [PATCH v2 04/10] event/octeontx2: " Andrzej Ostruszka
2019-09-17  7:57 ` [dpdk-dev] [PATCH v2 05/10] app/test: " Andrzej Ostruszka
2019-09-17  7:57 ` [dpdk-dev] [PATCH v2 06/10] net/dpaa2: fix possible use of uninitialized vars Andrzej Ostruszka
2019-09-17  7:57 ` [dpdk-dev] [PATCH v2 07/10] net/e1000: fix maybe-uninitialized warnings for LTO build Andrzej Ostruszka
2019-09-17  7:57 ` [dpdk-dev] [PATCH v2 08/10] net/i40e: " Andrzej Ostruszka
2019-09-17  7:57 ` [dpdk-dev] [PATCH v2 09/10] net/ifc: " Andrzej Ostruszka
2019-09-17  7:57 ` [dpdk-dev] [PATCH v2 10/10] net/qede: " Andrzej Ostruszka
     [not found] ` <20191021105707.25691-1-aostruszka@marvell.com>
     [not found]   ` <20191021105707.25691-2-aostruszka@marvell.com>
2019-10-21 12:59     ` [dpdk-dev] [PATCH v3 01/10] build: add an option to enable " Bruce Richardson
2019-10-22  8:53       ` Andrzej Ostruszka
2019-10-22 11:54 ` [dpdk-dev] [PATCH v4 00/10] Add an option to use LTO for DPDK build Andrzej Ostruszka
2019-10-22 11:54   ` [dpdk-dev] [PATCH v4 01/10] build: add an option to enable LTO build Andrzej Ostruszka
2019-10-22 12:45     ` Bruce Richardson
2019-10-27 11:47     ` Thomas Monjalon
2019-10-28 10:47       ` Andrzej Ostruszka [this message]
2019-10-28 11:03         ` Thomas Monjalon
2019-10-22 11:54   ` [dpdk-dev] [PATCH v4 02/10] eventdev: fix possible use of uninitialized var Andrzej Ostruszka
2019-10-22 11:54   ` [dpdk-dev] [PATCH v4 03/10] app/eventdev: clean LTO build warnings (maybe-uninitialized) Andrzej Ostruszka
2019-10-22 11:54   ` [dpdk-dev] [PATCH v4 04/10] event/octeontx2: " Andrzej Ostruszka
2019-10-22 11:54   ` [dpdk-dev] [PATCH v4 05/10] app/test: " Andrzej Ostruszka
2019-10-22 11:54   ` [dpdk-dev] [PATCH v4 06/10] net/dpaa2: fix possible use of uninitialized vars Andrzej Ostruszka
2019-10-22 11:54   ` [dpdk-dev] [PATCH v4 07/10] net/e1000: clean LTO build warnings (maybe-uninitialized) Andrzej Ostruszka
2019-10-22 11:54   ` [dpdk-dev] [PATCH v4 08/10] net/i40e: " Andrzej Ostruszka
2019-10-22 11:54   ` [dpdk-dev] [PATCH v4 09/10] net/ifc: " Andrzej Ostruszka
2019-10-22 11:54   ` [dpdk-dev] [PATCH v4 10/10] net/qede: " Andrzej Ostruszka
2019-10-22 12:48   ` [dpdk-dev] [PATCH v4 00/10] Add an option to use LTO for DPDK build Bruce Richardson
2019-10-22 13:03     ` Andrzej Ostruszka
2019-10-28 14:21   ` [dpdk-dev] [PATCH v5 00/11] " Andrzej Ostruszka
2019-10-28 14:21     ` [dpdk-dev] [PATCH v5 01/11] build: annotate versioned symbols with __vsym macro Andrzej Ostruszka
2019-10-29 10:49       ` Neil Horman
2019-10-28 14:21     ` [dpdk-dev] [PATCH v5 02/11] build: add an option to enable LTO build Andrzej Ostruszka
2019-10-28 14:21     ` [dpdk-dev] [PATCH v5 03/11] eventdev: fix possible use of uninitialized var Andrzej Ostruszka
2019-10-28 14:21     ` [dpdk-dev] [PATCH v5 04/11] app/eventdev: clean LTO build warnings (maybe-uninitialized) Andrzej Ostruszka
2019-10-28 14:21     ` [dpdk-dev] [PATCH v5 05/11] event/octeontx2: " Andrzej Ostruszka
2019-10-28 14:21     ` [dpdk-dev] [PATCH v5 06/11] app/test: " Andrzej Ostruszka
2019-10-28 14:21     ` [dpdk-dev] [PATCH v5 07/11] net/dpaa2: fix possible use of uninitialized vars Andrzej Ostruszka
2019-10-28 14:21     ` [dpdk-dev] [PATCH v5 08/11] net/e1000: clean LTO build warnings (maybe-uninitialized) Andrzej Ostruszka
2019-10-28 14:21     ` [dpdk-dev] [PATCH v5 09/11] net/i40e: " Andrzej Ostruszka
2019-10-28 14:21     ` [dpdk-dev] [PATCH v5 10/11] net/ifc: " Andrzej Ostruszka
2019-10-28 14:21     ` [dpdk-dev] [PATCH v5 11/11] net/qede: " Andrzej Ostruszka
2019-10-29 14:12     ` [dpdk-dev] [PATCH v6 00/12] Add an option to use LTO for DPDK build Andrzej Ostruszka
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 01/12] doc: fix description of versioning macros Andrzej Ostruszka
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 02/12] build: annotate versioned symbols with __vsym macro Andrzej Ostruszka
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 03/12] build: add an option to enable LTO build Andrzej Ostruszka
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 04/12] eventdev: fix possible use of uninitialized var Andrzej Ostruszka
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 05/12] app/eventdev: clean LTO build warnings (maybe-uninitialized) Andrzej Ostruszka
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 06/12] event/octeontx2: " Andrzej Ostruszka
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 07/12] app/test: " Andrzej Ostruszka
2019-11-01 17:15         ` Wang, Yipeng1
2019-11-04 13:48           ` Andrzej Ostruszka
2019-11-07 17:48             ` Wang, Yipeng1
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 08/12] net/dpaa2: fix possible use of uninitialized vars Andrzej Ostruszka
2019-11-04 11:46         ` Hemant Agrawal
2019-11-04 14:33           ` Andrzej Ostruszka
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 09/12] net/e1000: clean LTO build warnings (maybe-uninitialized) Andrzej Ostruszka
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 10/12] net/i40e: " Andrzej Ostruszka
2019-11-01  2:05         ` Xing, Beilei
2019-11-04 14:06           ` Andrzej Ostruszka
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 11/12] net/ifc: " Andrzej Ostruszka
2019-10-29 14:12       ` [dpdk-dev] [PATCH v6 12/12] net/qede: " Andrzej Ostruszka
2019-10-30  9:09       ` [dpdk-dev] [PATCH v6 00/12] Add an option to use LTO for DPDK build Andrzej Ostruszka
2019-10-30 14:23         ` Aaron Conole
2019-11-07 15:03       ` [dpdk-dev] [PATCH v7 " Andrzej Ostruszka
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 01/12] doc: fix description of versioning macros Andrzej Ostruszka
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 02/12] build: annotate versioned symbols with __vsym macro Andrzej Ostruszka
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 03/12] build: add an option to enable LTO build Andrzej Ostruszka
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 04/12] eventdev: fix possible use of uninitialized var Andrzej Ostruszka
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 05/12] app/eventdev: clean LTO build warnings (maybe-uninitialized) Andrzej Ostruszka
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 06/12] event/octeontx2: " Andrzej Ostruszka
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 07/12] app/test: " Andrzej Ostruszka
2019-11-07 17:53           ` Wang, Yipeng1
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 08/12] net/dpaa2: fix possible use of uninitialized vars Andrzej Ostruszka
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 09/12] net/e1000: clean LTO build warnings (maybe-uninitialized) Andrzej Ostruszka
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 10/12] net/i40e: " Andrzej Ostruszka
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 11/12] net/ifc: " Andrzej Ostruszka
2019-11-07 15:03         ` [dpdk-dev] [PATCH v7 12/12] net/qede: " Andrzej Ostruszka
2019-11-08 14:24         ` [dpdk-dev] [PATCH v7 00/12] Add an option to use LTO for DPDK build Thomas Monjalon
2019-11-01 21:33 ` [dpdk-dev] [PATCH v2 00/10] " Stephen Hemminger

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1bb37fbc-bf02-9485-fcb5-0c342ccee4f6@semihalf.com \
    --to=amo@semihalf.com \
    --cc=aconole@redhat.com \
    --cc=bruce.richardson@intel.com \
    --cc=david.hunt@intel.com \
    --cc=dev@dpdk.org \
    --cc=erik.g.carrillo@intel.com \
    --cc=john.mcnamara@intel.com \
    --cc=marko.kovacevic@intel.com \
    --cc=mattias.ronnblom@ericsson.com \
    --cc=msantana@redhat.com \
    --cc=rsanford@akamai.com \
    --cc=stephen@networkplumber.org \
    --cc=thomas@monjalon.net \
    --cc=vladimir.medvedkin@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

DPDK-dev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dpdk-dev/0 dpdk-dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dpdk-dev dpdk-dev/ https://lore.kernel.org/dpdk-dev \
		dev@dpdk.org
	public-inbox-index dpdk-dev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.dpdk.dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git