From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EFFB8C433FE for ; Tue, 4 Oct 2022 15:32:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=LJpQSt6f76xKyg5mFA+8DKvdT5AduyXHQ9BWqHR6Jys=; b=jycTcn8rZW1cpMDfrRmpSWKoo+ ZFm5DcPmm9zrVCcmxGQm+sWWiHicC5EakKCAxlrNJIjDcvcvumZrHyf0fmERJH4IR57reksplcH1A nyGqQf2rxrQTYy/A7CUKTcDAByqh+hYZimP8voLvXQDaBBaTw3dTSKVh0ih+asc+CTlKoCHX9Iyz/ 9uvo5cXMIdf2lVKLyutuU7hWtXJX3TqQk+V5ekr/DPbCi3CvIbroBYlgCWJzmBwDFNRqD4mVy8rjW qO5LCdOnQzka6JjxUX70PyZQQWl7+NzIbrUwLLekgmjVJ0gURgIEO+5KUMJuVrK/xCnu6tgkXFBjz hScMnQUg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ofjuU-00A5KE-SA; Tue, 04 Oct 2022 15:32:42 +0000 Received: from mail-lj1-x22a.google.com ([2a00:1450:4864:20::22a]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ofjuR-00A5JC-Vj for linux-nvme@lists.infradead.org; Tue, 04 Oct 2022 15:32:41 +0000 Received: by mail-lj1-x22a.google.com with SMTP id a12so5724571ljr.7 for ; Tue, 04 Oct 2022 08:32:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date; bh=LJpQSt6f76xKyg5mFA+8DKvdT5AduyXHQ9BWqHR6Jys=; b=NBLGR9LigeFmBloKTsH3jAyJdA2BxTIkpCCdzRtXt35tT8t77dwX8+1lCCsIzW6m1K 10X2ruTGY9VTmD5WOseHk2qtASOcK9oNV2yeTVycxcIH1B27teASNTL8jkogyKcnG9ja lDYB/eUzTQtiog3LEROv008MKQyvrrmmHAUZqX9UiXsw5EsLKAQxgKS89IMkOyTy/SXW BoeX0Gt881L/c8+qW6ay+296XSBAnQVjvtl0f7QjPoQmqj+j2HqKBQ2galI9YkGZJgxw YPqPPDw1JEOrV60b/AHJzSy8+O+m/znCSzB9O4TyFBvZn2gzRfxRm/e3cyPKAHrrBq/A sPSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date; bh=LJpQSt6f76xKyg5mFA+8DKvdT5AduyXHQ9BWqHR6Jys=; b=krNw5M0zNDVaBUbQ5U5yuHGPWjj7klWU73Amse2WBoUKZZFyewmiCZevTw7wqAKuMZ 0OdTDjacuRm6GrXUyaXbFKPZ5fD8SXMZRY3mhrrmwfg+ptfoAmOWjwu4uHXrW3a7zeeZ tBXRCa1oCWKPq6p56R4QjtUsrPswBH2Lw1WGIiSLvro/2qLYf2TIW6deQW7ssafh+pAf 6yEzwU0kRtVY5chI8n4lNfPwrpplDozLqCnxMjoT18uvn+YKo5bRmnf2yO+kMmLrZka/ 73k5/VFoGCqltokR/ujREOJhaX+Z2TS9NAwYLs0PZfwXpZOdHfBGe236eP0YW0xjuALa zvCQ== X-Gm-Message-State: ACrzQf37J/2Cnd9kNKptOMfdTcwdjiCRyDE1x6JJqHhwso5YWaebCxw2 abU+gJhU0RMHQMx1eMpRbgg= X-Google-Smtp-Source: AMsMyM70KCRNNDKjPyPxNz/cIaHtp4wK32laKXHtSsO/3EwAzNBVS3tfMgR6hVFHSIO9qLn8bDdbpA== X-Received: by 2002:a05:651c:4ca:b0:26c:50e6:a9d3 with SMTP id e10-20020a05651c04ca00b0026c50e6a9d3mr8918382lji.318.1664897555773; Tue, 04 Oct 2022 08:32:35 -0700 (PDT) Received: from mobilestation ([95.79.133.202]) by smtp.gmail.com with ESMTPSA id bx30-20020a05651c199e00b0026acfbbcb7esm763543ljb.12.2022.10.04.08.32.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 04 Oct 2022 08:32:34 -0700 (PDT) Date: Tue, 4 Oct 2022 18:32:33 +0300 From: Serge Semin To: Jonathan Derrick Cc: Serge Semin , Jens Axboe , Keith Busch , Jens Axboe , Christoph Hellwig , Sagi Grimberg , Jonathan Derrick , Revanth Rajashekar , Rafael Antognolli , Scott Bauer , Alexey Malahov , Pavel Parkhomenko , Thomas Bogendoerfer , linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 3/3] block: sed-opal: Cache-line-align the cmd/resp buffers Message-ID: <20221004153233.bxevkwwi6kqdpyep@mobilestation> References: <20220929224648.8997-1-Sergey.Semin@baikalelectronics.ru> <20220929224648.8997-4-Sergey.Semin@baikalelectronics.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221004_083240_058123_C7070D1F X-CRM114-Status: GOOD ( 25.83 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org On Mon, Oct 03, 2022 at 12:24:08PM -0600, Jonathan Derrick wrote: > Hi > > On 9/29/2022 4:46 PM, Serge Semin wrote: > > In accordance with [1] the DMA-able memory buffers must be > > cacheline-aligned otherwise the cache writing-back and invalidation > > performed during the mapping may cause the adjacent data being lost. It's > > specifically required for the DMA-noncoherent platforms. Seeing the > > opal_dev.{cmd,resp} buffers are used for DMAs in the NVME and SCSI/SD > > drivers in framework of the nvme_sec_submit() and sd_sec_submit() methods > > respectively we must make sure the passed buffers are cacheline-aligned to > > prevent the denoted problem. > > > > [1] Documentation/core-api/dma-api.rst > > > > Fixes: 455a7b238cd6 ("block: Add Sed-opal library") > > Signed-off-by: Serge Semin > > --- > > block/sed-opal.c | 5 +++-- > > 1 file changed, 3 insertions(+), 2 deletions(-) > > > > diff --git a/block/sed-opal.c b/block/sed-opal.c > > index 9700197000f2..222acbd1f03a 100644 > > --- a/block/sed-opal.c > > +++ b/block/sed-opal.c > > @@ -73,6 +73,7 @@ struct parsed_resp { > > struct opal_resp_tok toks[MAX_TOKS]; > > }; > > +/* Presumably DMA-able buffers must be cache-aligned */ > > struct opal_dev { > > bool supported; > > bool mbr_enabled; > > @@ -88,8 +89,8 @@ struct opal_dev { > > u64 lowest_lba; > > size_t pos; > > - u8 cmd[IO_BUFFER_LENGTH]; > > - u8 resp[IO_BUFFER_LENGTH]; > > + u8 cmd[IO_BUFFER_LENGTH] ____cacheline_aligned; > > + u8 resp[IO_BUFFER_LENGTH] ____cacheline_aligned; > I'm with Christoph on this one. > When I see ____cacheline_aligned, I assume its for performance reasons, not > to work around a DMA limitation. Can we instead kmalloc (which provides > alignment) these buffers to make it more clear? May want to add that same > comment pointing out some architectures require these dma targets to be > cache aligned. Ok. I'll resend v3 with these buffers being kmalloc'ed. Please note the SED OPAL entry of the MAINTAINTER list contains your intel-email address, which bounces back the messages (so does the Revanth' one). I'll add your new address to my patchset' "To"-list, but if you want to get new OPAL-related patches sent directly to your linux.dev email address the entry should be updated. -Sergey > > > > struct parsed_resp parsed; > > size_t prev_d_len;