Re: [PATCH] PCIe: limit Max Read Request Size on i.MX to 512 bytes

From: "Krzysztof Hałasa" <khalasa@piap.pl>
To: Richard Zhu <hongxing.zhu@nxp.com>
Cc: "Bjorn Helgaas" <helgaas@kernel.org>,
	"Krzysztof Wilczyński" <kw@linux.com>,
	"Bjorn Helgaas" <bhelgaas@google.com>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"Artem Lapkin" <email2tema@gmail.com>,
	"Neil Armstrong" <narmstrong@baylibre.com>,
	"Huacai Chen" <chenhuacai@gmail.com>,
	"Rob Herring" <robh@kernel.org>,
	"Lorenzo Pieralisi" <lorenzo.pieralisi@arm.com>,
	"Lucas Stach" <l.stach@pengutronix.de>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] PCIe: limit Max Read Request Size on i.MX to 512 bytes
Date: Mon, 16 Aug 2021 13:18:30 +0200	[thread overview]
Message-ID: <m3o89xzlh5.fsf@t19.piap.pl> (raw)
In-Reply-To: <VI1PR04MB5853728C0FD18D19901138048CFD9@VI1PR04MB5853.eurprd04.prod.outlook.com> (Richard Zhu's message of "Mon, 16 Aug 2021 07:49:52 +0000")

Hi Richard,

Please correct me if I got something wrong:

> Regarding my understanding, that there should be decomposition operations if the
>  Max_Read_Request_Size is larger than the Max_Payload_size specified
>  by RC port.

I think this means that, for example, a memory read request (a single
short physical TLP packet on PCIe, from the peripheral device to the
CPU) can request more data than Max_Payload_size (128 bytes on i.MX6).
In such case, up to 4 "completion" physical TLP packets are returned by
the CPU (up to 512 bytes, with each individual TLP up to 128 bytes as
per Max_Payload_size).

The device can't request more than 512 bytes, though - the CPU would not
service such request.

> The bit0 of AMBA Multiple Outbound Decomposed NP Sub-Requests Control Register(Offset:0x700 + 0x24)
>  had been set to be 1b1 in default.
>
> Note: The description of this bit.
> Enable AMBA Multiple Outbound Decomposed NP Sub- Requests.
> This bit when set to '0' disables the possibility of having multiple outstanding non-posted requests that
> were derived from decomposition of an outbound AMBA request. See Supported AXI Burst Operations for
> more details.

I think the above means that - when I disable the bit in question - I
can't issue memory read requests longer than 128 bytes (max payload
size).

This is not exactly clear to me:

> You should not clear this register unless your application master is
> requesting an amount of read data greater than Max_Read_Request_Size,
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Is "completing" such a request at all possible?
The device shouldn't request more data than its (not CPU's)
Max_Read_Request_Size. I. e. if 512 is written to RTL8111's
Max_Read_Request_Size, then the Realtek chip will request max 512 bytes
at a time.

> and the remote device (or switch) is reordering completions that have
> different tags

Fortunately, such completions don't seem to be reordered.

However, I'm not sure how setting a bit in the CPU registers could help
here. I think the only way *IF* the completions were reordered would be
setting MRRS = MPS (= 128 bytes on i.MX6) - so there is nothing that
could be reordered.
-- 
Krzysztof "Chris" Hałasa

Sieć Badawcza Łukasiewicz
Przemysłowy Instytut Automatyki i Pomiarów PIAP
Al. Jerozolimskie 202, 02-486 Warszawa