From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758751Ab2JZNX1 (ORCPT ); Fri, 26 Oct 2012 09:23:27 -0400 Received: from mail-qc0-f174.google.com ([209.85.216.174]:38662 "EHLO mail-qc0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753984Ab2JZNXZ (ORCPT ); Fri, 26 Oct 2012 09:23:25 -0400 MIME-Version: 1.0 In-Reply-To: References: <20121025121617.617683848@chello.nl> <20121025124832.840241082@chello.nl> <5089F5B5.1050206@redhat.com> <508A0A0D.4090001@redhat.com> Date: Fri, 26 Oct 2012 06:23:24 -0700 Message-ID: Subject: Re: [PATCH 05/31] x86/mm: Reduce tlb flushes from ptep_set_access_flags() From: Michel Lespinasse To: Andi Kleen Cc: Linus Torvalds , Rik van Riel , Peter Zijlstra , Andrea Arcangeli , Mel Gorman , Johannes Weiner , Thomas Gleixner , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Ingo Molnar Content-Type: text/plain; charset=ISO-8859-1 X-System-Of-Record: true Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Oct 26, 2012 at 5:48 AM, Andi Kleen wrote: > Michel Lespinasse writes: > >> On Thu, Oct 25, 2012 at 9:23 PM, Linus Torvalds >> wrote: >>> On Thu, Oct 25, 2012 at 8:57 PM, Rik van Riel wrote: >>>> >>>> That may not even be needed. Apparently Intel chips >>>> automatically flush an entry from the TLB when it >>>> causes a page fault. I assume AMD chips do the same, >>>> because flush_tlb_fix_spurious_fault evaluates to >>>> nothing on x86. >>> >>> Yes. It's not architected as far as I know, though. But I agree, it's >>> possible - even likely - we could avoid TLB flushing entirely on x86. >> >> Actually, it is architected on x86. This was first described in the >> intel appnote 317080 "TLBs, Paging-Structure Caches, and Their >> Invalidation", last paragraph of section 5.1. Nowadays, the same >> contents are buried somewhere in Volume 3 of the architecture manual >> (in my copy: 4.10.4.1 Operations that Invalidate TLBs and >> Paging-Structure Caches) > > This unfortunately would only work for processes with no threads > because it only works on the current logical CPU. No, the point is, if we are only *increasing* permissions on a page, we can skip the remote TLB invalidations. Later on each remote CPU might possibly get a spurious fault on that page, but that spurious fault will resynchronize their TLBs for that page, so that the instruction retry after the fault won't fault again. It is often cheaper to let remote CPUs get an occasional spurious fault than to synchronize with them on every permission change. Of course, none of the above applies if we are *reducing* permissions on a page (we really can't skip TLB invalidations there) -- Michel "Walken" Lespinasse A program is never fully debugged until the last user dies. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from psmtp.com (na3sys010amx145.postini.com [74.125.245.145]) by kanga.kvack.org (Postfix) with SMTP id 1F7046B0072 for ; Fri, 26 Oct 2012 09:23:26 -0400 (EDT) Received: by mail-qc0-f169.google.com with SMTP id t2so1665902qcq.14 for ; Fri, 26 Oct 2012 06:23:25 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <20121025121617.617683848@chello.nl> <20121025124832.840241082@chello.nl> <5089F5B5.1050206@redhat.com> <508A0A0D.4090001@redhat.com> Date: Fri, 26 Oct 2012 06:23:24 -0700 Message-ID: Subject: Re: [PATCH 05/31] x86/mm: Reduce tlb flushes from ptep_set_access_flags() From: Michel Lespinasse Content-Type: text/plain; charset=ISO-8859-1 Sender: owner-linux-mm@kvack.org List-ID: To: Andi Kleen Cc: Linus Torvalds , Rik van Riel , Peter Zijlstra , Andrea Arcangeli , Mel Gorman , Johannes Weiner , Thomas Gleixner , Andrew Morton , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Ingo Molnar On Fri, Oct 26, 2012 at 5:48 AM, Andi Kleen wrote: > Michel Lespinasse writes: > >> On Thu, Oct 25, 2012 at 9:23 PM, Linus Torvalds >> wrote: >>> On Thu, Oct 25, 2012 at 8:57 PM, Rik van Riel wrote: >>>> >>>> That may not even be needed. Apparently Intel chips >>>> automatically flush an entry from the TLB when it >>>> causes a page fault. I assume AMD chips do the same, >>>> because flush_tlb_fix_spurious_fault evaluates to >>>> nothing on x86. >>> >>> Yes. It's not architected as far as I know, though. But I agree, it's >>> possible - even likely - we could avoid TLB flushing entirely on x86. >> >> Actually, it is architected on x86. This was first described in the >> intel appnote 317080 "TLBs, Paging-Structure Caches, and Their >> Invalidation", last paragraph of section 5.1. Nowadays, the same >> contents are buried somewhere in Volume 3 of the architecture manual >> (in my copy: 4.10.4.1 Operations that Invalidate TLBs and >> Paging-Structure Caches) > > This unfortunately would only work for processes with no threads > because it only works on the current logical CPU. No, the point is, if we are only *increasing* permissions on a page, we can skip the remote TLB invalidations. Later on each remote CPU might possibly get a spurious fault on that page, but that spurious fault will resynchronize their TLBs for that page, so that the instruction retry after the fault won't fault again. It is often cheaper to let remote CPUs get an occasional spurious fault than to synchronize with them on every permission change. Of course, none of the above applies if we are *reducing* permissions on a page (we really can't skip TLB invalidations there) -- Michel "Walken" Lespinasse A program is never fully debugged until the last user dies. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org