Re: [PATCH v2 3/3] block: sed-opal: Cache-line-align the cmd/resp buffers

From: Jonathan Derrick <jonathan.derrick@linux.dev>
To: Serge Semin <Sergey.Semin@baikalelectronics.ru>,
	Jens Axboe <axboe@kernel.dk>, Keith Busch <kbusch@kernel.org>,
	Jens Axboe <axboe@fb.com>, Christoph Hellwig <hch@lst.de>,
	Sagi Grimberg <sagi@grimberg.me>,
	Jonathan Derrick <jonathan.derrick@intel.com>,
	Revanth Rajashekar <revanth.rajashekar@intel.com>,
	Rafael Antognolli <Rafael.Antognolli@intel.com>,
	Scott Bauer <scott.bauer@intel.com>
Cc: Serge Semin <fancer.lancer@gmail.com>,
	Alexey Malahov <Alexey.Malahov@baikalelectronics.ru>,
	Pavel Parkhomenko <Pavel.Parkhomenko@baikalelectronics.ru>,
	Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
	linux-nvme@lists.infradead.org, linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 3/3] block: sed-opal: Cache-line-align the cmd/resp buffers
Date: Mon, 3 Oct 2022 12:24:08 -0600	[thread overview]
Message-ID: <cce74aec-61b1-d5eb-1b62-746e45ebfe69@linux.dev> (raw)
In-Reply-To: <20220929224648.8997-4-Sergey.Semin@baikalelectronics.ru>

Hi

On 9/29/2022 4:46 PM, Serge Semin wrote:
> In accordance with [1] the DMA-able memory buffers must be
> cacheline-aligned otherwise the cache writing-back and invalidation
> performed during the mapping may cause the adjacent data being lost. It's
> specifically required for the DMA-noncoherent platforms. Seeing the
> opal_dev.{cmd,resp} buffers are used for DMAs in the NVME and SCSI/SD
> drivers in framework of the nvme_sec_submit() and sd_sec_submit() methods
> respectively we must make sure the passed buffers are cacheline-aligned to
> prevent the denoted problem.
> 
> [1] Documentation/core-api/dma-api.rst
> 
> Fixes: 455a7b238cd6 ("block: Add Sed-opal library")
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> ---
>   block/sed-opal.c | 5 +++--
>   1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/block/sed-opal.c b/block/sed-opal.c
> index 9700197000f2..222acbd1f03a 100644
> --- a/block/sed-opal.c
> +++ b/block/sed-opal.c
> @@ -73,6 +73,7 @@ struct parsed_resp {
>   	struct opal_resp_tok toks[MAX_TOKS];
>   };
>   
> +/* Presumably DMA-able buffers must be cache-aligned */
>   struct opal_dev {
>   	bool supported;
>   	bool mbr_enabled;
> @@ -88,8 +89,8 @@ struct opal_dev {
>   	u64 lowest_lba;
>   
>   	size_t pos;
> -	u8 cmd[IO_BUFFER_LENGTH];
> -	u8 resp[IO_BUFFER_LENGTH];
> +	u8 cmd[IO_BUFFER_LENGTH] ____cacheline_aligned;
> +	u8 resp[IO_BUFFER_LENGTH] ____cacheline_aligned;
I'm with Christoph on this one.
When I see ____cacheline_aligned, I assume its for performance reasons, 
not to work around a DMA limitation. Can we instead kmalloc (which 
provides alignment) these buffers to make it more clear? May want to add 
that same comment pointing out some architectures require these dma 
targets to be cache aligned.


>   
>   	struct parsed_resp parsed;
>   	size_t prev_d_len;