From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ferruh Yigit Subject: Re: [PATCH 1/2] eal: add macro to mark variable mostly read only Date: Wed, 18 Apr 2018 19:03:06 +0100 Message-ID: <291a43da-6c2d-f65b-374d-206a0f674db6@intel.com> References: <20180418153035.5972-1-pbhagavatula@caviumnetworks.com> <20180418175505.GA17954@ltp-pvn> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: dev@dpdk.org To: Pavan Nikhilesh , thomas@monjalon.net, jerin.jacob@caviumnetworks.com, techboard@dpdk.org Return-path: In-Reply-To: <20180418175505.GA17954@ltp-pvn> Content-Language: en-US List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 4/18/2018 6:55 PM, Pavan Nikhilesh wrote: > On Wed, Apr 18, 2018 at 06:43:11PM +0100, Ferruh Yigit wrote: >> On 4/18/2018 4:30 PM, Pavan Nikhilesh wrote: >>> Add macro to mark a variable to be mostly read only and place it in a >>> separate section. >>> >>> Signed-off-by: Pavan Nikhilesh >>> --- >>> >>> Group together mostly read only data to avoid cacheline bouncing, also >>> useful for auditing purposes. >>> >>> lib/librte_eal/common/include/rte_common.h | 5 +++++ >>> 1 file changed, 5 insertions(+) >>> >>> diff --git a/lib/librte_eal/common/include/rte_common.h b/lib/librte_eal/common/include/rte_common.h >>> index 6c5bc5a76..f2ff2e9e6 100644 >>> --- a/lib/librte_eal/common/include/rte_common.h >>> +++ b/lib/librte_eal/common/include/rte_common.h >>> @@ -114,6 +114,11 @@ static void __attribute__((constructor(prio), used)) func(void) >>> */ >>> #define __rte_noinline __attribute__((noinline)) >>> >>> +/** >>> + * Mark a variable to be mostly read only and place it in a separate section. >>> + */ >>> +#define __rte_read_mostly __attribute__((__section__(".read_mostly"))) >> > > Hi Ferruh, > >> Hi Pavan, >> >> Is the section ".read_mostly" treated specially [1] or is this just for grouping >> symbols together (to reduce cacheline bouncing)? > > The section .read_mostly is not treated specially it's just for grouping > symbols. I have encounter with a blog post claiming this is not working: " The problem with the above approach is that once all the __read_mostly variables are grouped into one section, the remaining "non-read-mostly" variables end-up together too. This increases the chances that two frequently used elements (in the "non-read-mostly" region) will end-up competing for the same position (or cache-line, the basic fixed-sized block for memory<-->cache transfers) in the cache. Thus frequent accesses will cause excessive cache thrashing on that particular cache-line thereby degrading the overall system performance. " https://thecodeartist.blogspot.com/2011/12/why-readmostly-does-not-work-as-it.html > >> >> [1] >> If this is special section, can you please point counter part in the kernel? > > The kernel has something similar[1] but they have a custom linker script to > arrange symbols. > > [1] https://github.com/torvalds/linux/blob/a27fc14219f2e3c4a46ba9177b04d9b52c875532/arch/x86/include/asm/cache.h#L11 > kernel commit id 54cb27a71f51d304342c79e62fd7667f2171062b > >> >> >>> + >>> /*********** Macros for pointer arithmetic ********/ >>> >>> /** >>> -- >>> 2.17.0 >>> >>