From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.0 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B2A15C433E1 for ; Fri, 22 May 2020 13:01:34 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8AB622072C for ; Fri, 22 May 2020 13:01:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WE6ke5zT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8AB622072C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id A9D5211FE8B0C; Fri, 22 May 2020 05:57:56 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=207.211.31.120; helo=us-smtp-1.mimecast.com; envelope-from=mpatocka@redhat.com; receiver= Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id EB63411FE8B0A for ; Fri, 22 May 2020 05:57:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590152490; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gFFp4PY2q6XngmMDcgxR3SERztb+l48+iSkbcR2vUwA=; b=WE6ke5zTieVUSNz9xKQTvi14tsHb7VomwS1jh1Cu2UAayn2OToX3ZWVcS5eOu7p8V9QvON 7iMkONqDDMeskyRGscjIc1QvtoYcVb2WxngEv3SVxZdju2HaS4NdoExBVaA+u7xtLYeKN9 UBiQ8957MkqPF8AQv763f1crRfaLfDY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-27-X2igJmbUOIG4NzTVRJjJzA-1; Fri, 22 May 2020 09:01:22 -0400 X-MC-Unique: X2igJmbUOIG4NzTVRJjJzA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 78A8B8015D2; Fri, 22 May 2020 13:01:20 +0000 (UTC) Received: from file01.intranet.prod.int.rdu2.redhat.com (file01.intranet.prod.int.rdu2.redhat.com [10.11.5.7]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A39505C1D0; Fri, 22 May 2020 13:01:19 +0000 (UTC) Received: from file01.intranet.prod.int.rdu2.redhat.com (localhost [127.0.0.1]) by file01.intranet.prod.int.rdu2.redhat.com (8.14.4/8.14.4) with ESMTP id 04MD1Jp8020003; Fri, 22 May 2020 09:01:19 -0400 Received: from localhost (mpatocka@localhost) by file01.intranet.prod.int.rdu2.redhat.com (8.14.4/8.14.4/Submit) with ESMTP id 04MD1Har019975; Fri, 22 May 2020 09:01:17 -0400 X-Authentication-Warning: file01.intranet.prod.int.rdu2.redhat.com: mpatocka owned process doing -bs Date: Fri, 22 May 2020 09:01:17 -0400 (EDT) From: Mikulas Patocka X-X-Sender: mpatocka@file01.intranet.prod.int.rdu2.redhat.com To: "Aneesh Kumar K.V" Subject: Re: [PATCH v2 3/5] libnvdimm/nvdimm/flush: Allow architecture to override the flush barrier In-Reply-To: <23e57565-be2a-a45c-f4d4-d8eca7262dea@linux.ibm.com> Message-ID: References: <20200513034705.172983-3-aneesh.kumar@linux.ibm.com> <87v9kspk3x.fsf@linux.ibm.com> <87d070f2vs.fsf@linux.ibm.com>

<20200522093127.GY25173@kitsune.suse.cz> <23e57565-be2a-a45c-f4d4-d8eca7262dea@linux.ibm.com> User-Agent: Alpine 2.02 (LRH 1266 2009-07-14) MIME-Version: 1.0 Content-ID: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Message-ID-Hash: FPN4GU63KYOJAQ4YYYUDMXQ5LNOKR22T X-Message-ID-Hash: FPN4GU63KYOJAQ4YYYUDMXQ5LNOKR22T X-MailFrom: mpatocka@redhat.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation X-Content-Filtered-By: Mailman/MimeDel 3.1.1 CC: =?ISO-8859-15?Q?Michal_Such=E1nek?= , Jan Kara , linux-nvdimm , alistair@popple.id.au, linuxppc-dev X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: TEXT/PLAIN; charset="iso-8859-15" Content-Transfer-Encoding: quoted-printable On Fri, 22 May 2020, Aneesh Kumar K.V wrote: > On 5/22/20 3:01 PM, Michal Such=C3=A1nek wrote: > > On Thu, May 21, 2020 at 02:52:30PM -0400, Mikulas Patocka wrote: > > >=20 > > >=20 > > > On Thu, 21 May 2020, Dan Williams wrote: > > >=20 > > > > On Thu, May 21, 2020 at 10:03 AM Aneesh Kumar K.V > > > > wrote: > > > > >=20 > > > > > > Moving on to the patch itself--Aneesh, have you audited other > > > > > > persistent > > > > > > memory users in the kernel? For example, drivers/md/dm-writeca= che.c > > > > > > does > > > > > > this: > > > > > >=20 > > > > > > static void writecache_commit_flushed(struct dm_writecache *wc,= bool > > > > > > wait_for_ios) > > > > > > { > > > > > > if (WC_MODE_PMEM(wc)) > > > > > > wmb(); <=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > > > else > > > > > > ssd_commit_flushed(wc, wait_for_ios); > > > > > > } > > > > > >=20 > > > > > > I believe you'll need to make modifications there. > > > > > >=20 > > > > >=20 > > > > > Correct. Thanks for catching that. > > > > >=20 > > > > >=20 > > > > > I don't understand dm much, wondering how this will work with > > > > > non-synchronous DAX device? > > > >=20 > > > > That's a good point. DM-writecache needs to be cognizant of things > > > > like virtio-pmem that violate the rule that persisent memory writes > > > > can be flushed by CPU functions rather than calling back into the > > > > driver. It seems we need to always make the flush case a dax_operat= ion > > > > callback to account for this. > > >=20 > > > dm-writecache is normally sitting on the top of dm-linear, so it would > > > need to pass the wmb() call through the dm core and dm-linear target = ... > > > that would slow it down ... I remember that you already did it this w= ay > > > some times ago and then removed it. > > >=20 > > > What's the exact problem with POWER? Could the POWER system have two = types > > > of persistent memory that need two different ways of flushing? > >=20 > > As far as I understand the discussion so far > >=20 > > - on POWER $oldhardware uses $oldinstruction to ensure pmem consisten= cy > > - on POWER $newhardware uses $newinstruction to ensure pmem consisten= cy > > (compatible with $oldinstruction on $oldhardware) >=20 > Correct. >=20 > > - on some platforms instead of barrier instruction a callback into the > > driver is issued to ensure consistency=20 >=20 > This is virtio-pmem only at this point IIUC. >=20 > -aneesh And does the virtio-pmem driver track which pages are dirty? Or does it=20 need to specify the range of pages to flush in the flush function? > > None of this is reflected by the dm driver. We could make a new dax method: void *(dax_get_flush_function)(void); This would return a pointer to "wmb()" on x86 and something else on Power. The method "dax_get_flush_function" would be called only once when=20 initializing the writecache driver (because the call would be slow because = it would have to go through the DM stack) and then, the returned function=20 would be called each time we need write ordering. The returned function=20 would do just "sfence; ret". Mikulas _______________________________________________ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-leave@lists.01.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.1 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87401C433E0 for ; Fri, 22 May 2020 13:05:37 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D9FF5206B6 for ; Fri, 22 May 2020 13:05:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WE6ke5zT"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CT9v5zjY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D9FF5206B6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 49T6DD2JwFzDr15 for ; Fri, 22 May 2020 23:05:32 +1000 (AEST) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=redhat.com (client-ip=207.211.31.120; helo=us-smtp-1.mimecast.com; envelope-from=mpatocka@redhat.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=WE6ke5zT; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=CT9v5zjY; dkim-atps=neutral Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 49T67h66h2zDqys for ; Fri, 22 May 2020 23:01:33 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590152490; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gFFp4PY2q6XngmMDcgxR3SERztb+l48+iSkbcR2vUwA=; b=WE6ke5zTieVUSNz9xKQTvi14tsHb7VomwS1jh1Cu2UAayn2OToX3ZWVcS5eOu7p8V9QvON 7iMkONqDDMeskyRGscjIc1QvtoYcVb2WxngEv3SVxZdju2HaS4NdoExBVaA+u7xtLYeKN9 UBiQ8957MkqPF8AQv763f1crRfaLfDY= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1590152491; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=gFFp4PY2q6XngmMDcgxR3SERztb+l48+iSkbcR2vUwA=; b=CT9v5zjY1m09twcwSV7WVziACNsj5m+eLyXfT6hJTqeiKkADxzSU1hGysBtwDEpx/tOMil S68K2qv0VczlLF2isNPfmIl/sfx99MIf0Iawm/PXM/bmK/XlegC6fRc6y5/c1Nw8mUt611 K3yLo5DpXl6mKAYb/wEshrv16IfQUg4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-27-X2igJmbUOIG4NzTVRJjJzA-1; Fri, 22 May 2020 09:01:22 -0400 X-MC-Unique: X2igJmbUOIG4NzTVRJjJzA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 78A8B8015D2; Fri, 22 May 2020 13:01:20 +0000 (UTC) Received: from file01.intranet.prod.int.rdu2.redhat.com (file01.intranet.prod.int.rdu2.redhat.com [10.11.5.7]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A39505C1D0; Fri, 22 May 2020 13:01:19 +0000 (UTC) Received: from file01.intranet.prod.int.rdu2.redhat.com (localhost [127.0.0.1]) by file01.intranet.prod.int.rdu2.redhat.com (8.14.4/8.14.4) with ESMTP id 04MD1Jp8020003; Fri, 22 May 2020 09:01:19 -0400 Received: from localhost (mpatocka@localhost) by file01.intranet.prod.int.rdu2.redhat.com (8.14.4/8.14.4/Submit) with ESMTP id 04MD1Har019975; Fri, 22 May 2020 09:01:17 -0400 X-Authentication-Warning: file01.intranet.prod.int.rdu2.redhat.com: mpatocka owned process doing -bs Date: Fri, 22 May 2020 09:01:17 -0400 (EDT) From: Mikulas Patocka X-X-Sender: mpatocka@file01.intranet.prod.int.rdu2.redhat.com To: "Aneesh Kumar K.V" Subject: Re: [PATCH v2 3/5] libnvdimm/nvdimm/flush: Allow architecture to override the flush barrier In-Reply-To: <23e57565-be2a-a45c-f4d4-d8eca7262dea@linux.ibm.com> Message-ID: References: <20200513034705.172983-3-aneesh.kumar@linux.ibm.com> <87v9kspk3x.fsf@linux.ibm.com> <87d070f2vs.fsf@linux.ibm.com>