From: Dan Williams <dan.j.williams@intel.com>
To: linux-cxl@vger.kernel.org
Cc: Ira Weiny <ira.weiny@intel.com>,
Jonathan Cameron <Jonathan.Cameron@huawei.com>,
Ariel Sibley <ariel.sibley@microchip.com>,
Dan Carpenter <dan.carpenter@oracle.com>
Subject: [PATCH v3 00/13] cxl: Fix "mem_enable" handling
Date: Wed, 18 May 2022 16:34:09 -0700 [thread overview]
Message-ID: <165291684910.1426646.8615474651213855015.stgit@dwillia2-xfh> (raw)
Changes since v2 [1]:
- Drop the assumption that Memory_size=0 still decodes up to 256MB of
memory space (Ariel, and Jonathan)
- Move the s/out:/unlock:/ label rename a patch earlier (Jonathan)
- Drop ternary conditional trickery in a debug message (Jonathan)
[1]: https://lore.kernel.org/r/165283418817.1033989.11273676872054815459.stgit@dwillia2-xfh
---
Jonathan reports [2] that after he changed QEMU to stop setting
Mem_Enable (8.1.3.2 DVSEC CXL Control (Bit 2)) by default the following
problems arose:
1. Nothing in the Linux code actually sets Mem_Enable to 1.
2. Probing fails in mem.c as wait_for_media() checks for
info->mem_enabled (cached value of this bit).
The investigation turned up more issues:
- DVSEC ranges, to be valid, must be covered by a CFMWS entry. Current
code was blindly assuming that any non-zero value is valid.
- No driver consideration for clearing "mem_enabled" and / or HDM
Decoder Enable.
- The cxl_test mock override for cxl_hdm_decode_init() was hiding bugs
in this path.
The end goal of these reworks are to improve detection for cases where
platform firmware is actually operating in legacy CXL DVSEC Range mode,
take ownership for setting and clearing "mem_enable" and HDM Decoder
Enable, and cleanup the indirections / mocking for cxl_test.
The new flow is described in patch 14:
Previously, the cxl_mem driver was relying on platform-firmware to set
"mem_enable". That is an invalid assumption as there is no requirement
that platform-firmware sets the bit before the driver sees a device,
especially in hot-plug scenarios. Additionally, ACPI-platforms that
support CXL 2.0 devices also support the ACPI CEDT (CXL Early Discovery
Table). That table outlines the platform permissible address ranges for
CXL operation. So, there is a need for the driver to set "mem_enable",
and there is information available to determine the validity of the CXL
DVSEC Ranges. Note that the DVSEC Ranges can not be shut off completely.
They always decode at least 256MB if "mem_enable" is set and the HDM
Decoder capability is disabled.
Arrange for the driver to optionally enable the HDM Decoder Capability
if "mem_enable" was not set by platform firmware, or the CXL DVSEC Range
configuration was invalid. Be careful to only disable memory decode if
the kernel was the one to enable it. In other words, if CXL is backing
all of kernel memory at boot the device needs to maintain "mem_enable"
and "HDM Decoder enable" all the way up to handoff back to platform
firmware (e.g. ACPI S5 state entry may require CXL memory to stay
active).
Link: https://lore.kernel.org/r/20220426180832.00005f0b@Huawei.com [2]
---
Dan Williams (13):
cxl/mem: Drop mem_enabled check from wait_for_media()
cxl/pci: Consolidate wait_for_media() and wait_for_media_ready()
cxl/pci: Drop wait_for_valid() from cxl_await_media_ready()
cxl/mem: Fix cxl_mem_probe() error exit
cxl/mem: Validate port connectivity before dvsec ranges
cxl/pci: Move cxl_await_media_ready() to the core
cxl/mem: Consolidate CXL DVSEC Range enumeration in the core
cxl/mem: Skip range enumeration if mem_enable clear
cxl/mem: Merge cxl_dvsec_ranges() and cxl_hdm_decode_init()
cxl/pci: Drop @info argument to cxl_hdm_decode_init()
cxl/port: Move endpoint HDM Decoder Capability init to port driver
cxl/port: Reuse 'struct cxl_hdm' context for hdm init
cxl/port: Enable HDM Capability after validating DVSEC Ranges
drivers/cxl/core/pci.c | 364 +++++++++++++++++++++++++++++++++++++++++
drivers/cxl/cxlmem.h | 4
drivers/cxl/cxlpci.h | 2
drivers/cxl/mem.c | 115 -------------
drivers/cxl/pci.c | 184 ---------------------
drivers/cxl/port.c | 28 ++-
tools/testing/cxl/Kbuild | 3
tools/testing/cxl/mock_mem.c | 10 -
tools/testing/cxl/test/mem.c | 17 --
tools/testing/cxl/test/mock.c | 29 +++
10 files changed, 424 insertions(+), 332 deletions(-)
delete mode 100644 tools/testing/cxl/mock_mem.c
base-commit: e6829d1bd3c4b58296ee9e412f7ed4d6cb390192
next reply other threads:[~2022-05-18 23:34 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-05-18 23:34 Dan Williams [this message]
2022-05-18 23:34 ` [PATCH v3 01/13] cxl/mem: Drop mem_enabled check from wait_for_media() Dan Williams
2022-05-18 23:34 ` [PATCH v3 02/13] cxl/pci: Consolidate wait_for_media() and wait_for_media_ready() Dan Williams
2022-05-18 23:34 ` [PATCH v3 03/13] cxl/pci: Drop wait_for_valid() from cxl_await_media_ready() Dan Williams
2022-05-18 23:34 ` [PATCH v3 04/13] cxl/mem: Fix cxl_mem_probe() error exit Dan Williams
2022-05-18 23:34 ` [PATCH v3 05/13] cxl/mem: Validate port connectivity before dvsec ranges Dan Williams
2022-05-18 23:34 ` [PATCH v3 06/13] cxl/pci: Move cxl_await_media_ready() to the core Dan Williams
2022-05-18 23:34 ` [PATCH v3 07/13] cxl/mem: Consolidate CXL DVSEC Range enumeration in " Dan Williams
2022-05-18 23:34 ` [PATCH v3 08/13] cxl/mem: Skip range enumeration if mem_enable clear Dan Williams
2022-05-18 23:35 ` [PATCH v3 09/13] cxl/mem: Merge cxl_dvsec_ranges() and cxl_hdm_decode_init() Dan Williams
2022-05-18 23:35 ` [PATCH v3 10/13] cxl/pci: Drop @info argument to cxl_hdm_decode_init() Dan Williams
2022-05-18 23:35 ` [PATCH v3 11/13] cxl/port: Move endpoint HDM Decoder Capability init to port driver Dan Williams
2022-05-18 23:35 ` [PATCH v3 12/13] cxl/port: Reuse 'struct cxl_hdm' context for hdm init Dan Williams
2022-05-18 23:35 ` [PATCH v3 13/13] cxl/port: Enable HDM Capability after validating DVSEC Ranges Dan Williams
2022-05-19 22:38 ` [PATCH v4 " Dan Williams
2022-05-20 15:09 ` Jonathan Cameron
2022-05-20 17:35 ` Dan Williams
2022-05-20 18:30 ` [PATCH v5 " Dan Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=165291684910.1426646.8615474651213855015.stgit@dwillia2-xfh \
--to=dan.j.williams@intel.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=ariel.sibley@microchip.com \
--cc=dan.carpenter@oracle.com \
--cc=ira.weiny@intel.com \
--cc=linux-cxl@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.