From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1031925AbbKEKcU (ORCPT ); Thu, 5 Nov 2015 05:32:20 -0500 Received: from foss.arm.com ([217.140.101.70]:36404 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1031170AbbKEKcT (ORCPT ); Thu, 5 Nov 2015 05:32:19 -0500 Date: Thu, 5 Nov 2015 10:32:15 +0000 From: Catalin Marinas To: Joonsoo Kim Cc: Robert Richter , Will Deacon , linux-kernel@vger.kernel.org, Robert Richter , Tirumalesh Chalamarla , linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH] arm64: Increase the max granular size Message-ID: <20151105103214.GP7637@e104818-lin.cambridge.arm.com> References: <1442944788-17254-1-git-send-email-rric@kernel.org> <20151105044014.GB20374@js1304-P5Q-DELUXE> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151105044014.GB20374@js1304-P5Q-DELUXE> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Nov 05, 2015 at 01:40:14PM +0900, Joonsoo Kim wrote: > On Tue, Sep 22, 2015 at 07:59:48PM +0200, Robert Richter wrote: > > From: Tirumalesh Chalamarla > > > > Increase the standard cacheline size to avoid having locks in the same > > cacheline. > > > > Cavium's ThunderX core implements cache lines of 128 byte size. With > > current granulare size of 64 bytes (L1_CACHE_SHIFT=6) two locks could > > share the same cache line leading a performance degradation. > > Increasing the size fixes that. > > Beside, slab-side bug, I don't think this argument is valid. > Even if this change is applied, statically allocated spinlock could > share the same cache line. The benchmarks didn't show any difference with or without this patch applied. What convinced me to apply it was this email: http://lkml.kernel.org/g/CAOZdJXUiRMAguDV+HEJqPg57MyBNqEcTyaH+ya=U93NHb-pdJA@mail.gmail.com On ARM we have a notion of cache writeback granule (CWG) which tells us "the maximum size of memory that can be overwritten as a result of the eviction of a cache entry that has had a memory location in it modified". What we actually needed was ARCH_DMA_MINALIGN to be 128 (currently defined to the L1_CACHE_BYTES value). However, this wouldn't have fixed the KMALLOC_MIN_SIZE, unless we somehow generate different kmalloc_caches[] and kmalloc_dma_caches[] and probably introduce a size_dma_index[]. > If two locks should not share the same cache line, you'd better to use > compiler attribute such as ____cacheline_aligned_in_smp in appropriate > place. We could decouple SMP_CACHE_BYTES from L1_CACHE_BYTES but see above for the other issue we had to solve. -- Catalin