From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 079D4C04ABB for ; Thu, 13 Sep 2018 10:30:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 855D320882 for ; Thu, 13 Sep 2018 10:30:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 855D320882 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=de.ibm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727670AbeIMPjN (ORCPT ); Thu, 13 Sep 2018 11:39:13 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:36126 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726797AbeIMPjN (ORCPT ); Thu, 13 Sep 2018 11:39:13 -0400 Received: from pps.filterd (m0098413.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w8DASpEl094521 for ; Thu, 13 Sep 2018 06:30:21 -0400 Received: from e06smtp05.uk.ibm.com (e06smtp05.uk.ibm.com [195.75.94.101]) by mx0b-001b2d01.pphosted.com with ESMTP id 2mfn96jcbn-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Thu, 13 Sep 2018 06:30:21 -0400 Received: from localhost by e06smtp05.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Thu, 13 Sep 2018 11:30:19 +0100 Received: from b06cxnps4075.portsmouth.uk.ibm.com (9.149.109.197) by e06smtp05.uk.ibm.com (192.168.101.135) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Thu, 13 Sep 2018 11:30:17 +0100 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w8DAUGiI63635472 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 13 Sep 2018 10:30:16 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A14F1A4065; Thu, 13 Sep 2018 13:30:04 +0100 (BST) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 609C3A4059; Thu, 13 Sep 2018 13:30:04 +0100 (BST) Received: from mschwideX1 (unknown [9.152.212.164]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Thu, 13 Sep 2018 13:30:04 +0100 (BST) Date: Thu, 13 Sep 2018 12:30:14 +0200 From: Martin Schwidefsky To: Peter Zijlstra Cc: will.deacon@arm.com, aneesh.kumar@linux.vnet.ibm.com, akpm@linux-foundation.org, npiggin@gmail.com, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, linux@armlinux.org.uk, heiko.carstens@de.ibm.com Subject: Re: [RFC][PATCH 01/11] asm-generic/tlb: Provide a comment In-Reply-To: <20180913092811.894806629@infradead.org> References: <20180913092110.817204997@infradead.org> <20180913092811.894806629@infradead.org> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-TM-AS-GCONF: 00 x-cbid: 18091310-0020-0000-0000-000002C5C483 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18091310-0021-0000-0000-00002113200D Message-Id: <20180913123014.0d9321b8@mschwideX1> Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8bit X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2018-09-13_06:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=745 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1807170000 definitions=main-1809130108 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 13 Sep 2018 11:21:11 +0200 Peter Zijlstra wrote: > Write a comment explaining some of this.. > > Cc: Will Deacon > Cc: "Aneesh Kumar K.V" > Cc: Andrew Morton > Cc: Nick Piggin > Signed-off-by: Peter Zijlstra (Intel) > --- > include/asm-generic/tlb.h | 120 ++++++++++++++++++++++++++++++++++++++++++++-- > 1 file changed, 117 insertions(+), 3 deletions(-) > > --- a/include/asm-generic/tlb.h > +++ b/include/asm-generic/tlb.h > @@ -22,6 +22,119 @@ > > #ifdef CONFIG_MMU > > +/* > + * Generic MMU-gather implementation. > + * > + * The mmu_gather data structure is used by the mm code to implement the > + * correct and efficient ordering of freeing pages and TLB invalidations. > + * > + * This correct ordering is: > + * > + * 1) unhook page > + * 2) TLB invalidate page > + * 3) free page > + * > + * That is, we must never free a page before we have ensured there are no live > + * translations left to it. Otherwise it might be possible to observe (or > + * worse, change) the page content after it has been reused. > + * This first comment already includes the reason why s390 is probably better off with its own mmu-gather implementation. It depends on the situation if we have 1) unhook the page and do a TLB flush at the same time 2) free page or 1) unhook page 2) free page 3) final TLB flush of the whole mm A variant of the second order we had in the past is to do the mm TLB flush first, then the unhooks and frees of the individual pages. The are some tricky corners switching between the two variants, see finish_arch_post_lock_switch. The point is: we *never* have the order 1) unhook, 2) TLB invalidate, 3) free. If there is concurrency due to a multi-threaded application we have to do the unhook of the page-table entry and the TLB flush with a single instruction. -- blue skies, Martin. "Reality continues to ruin my life." - Calvin.