From: Michael Ellerman <mpe@ellerman.id.au>
To: Alastair D'Silva <alastair@d-silva.org>,
'Dan Williams' <dan.j.williams@intel.com>
Cc: "'Aneesh Kumar K . V'" <aneesh.kumar@linux.ibm.com>,
"'Oliver O'Halloran'" <oohall@gmail.com>,
"'Benjamin Herrenschmidt'" <benh@kernel.crashing.org>,
"'Paul Mackerras'" <paulus@samba.org>,
"'Frederic Barrat'" <fbarrat@linux.ibm.com>,
"'Andrew Donnellan'" <ajd@linux.ibm.com>,
"'Arnd Bergmann'" <arnd@arndb.de>,
"'Greg Kroah-Hartman'" <gregkh@linuxfoundation.org>,
"'Vishal Verma'" <vishal.l.verma@intel.com>,
"'Dave Jiang'" <dave.jiang@intel.com>,
"'Ira Weiny'" <ira.weiny@intel.com>,
"'Andrew Morton'" <akpm@linux-foundation.org>,
"'Mauro Carvalho Chehab'" <mchehab+samsung@kernel.org>,
"'David S. Miller'" <davem@davemloft.net>,
"'Rob Herring'" <robh@kernel.org>,
"'Anton Blanchard'" <anton@ozlabs.org>,
"'Krzysztof Kozlowski'" <krzk@kernel.org>,
"'Mahesh Salgaonkar'" <mahesh@linux.vnet.ibm.com>,
"'Madhavan Srinivasan'" <maddy@linux.vnet.ibm.com>,
"'Cédric Le Goater'" <clg@kaod.org>,
"'Anju T Sudhakar'" <anju@linux.vnet.ibm.com>,
"'Hari Bathini'" <hbathini@linux.ibm.com>,
"'Thomas Gleixner'" <tglx@linutronix.de>,
"'Greg Kurz'" <groug@kaod.org>,
"'Nicholas Piggin'" <npiggin@gmail.com>,
"'Masahiro Yamada'" <yamada.masahiro@socionext.com>,
"'Alexey Kardashevskiy'" <aik@ozlabs.ru>,
"'Linux Kernel Mailing List'" <linux-kernel@vger.kernel.org>,
'linuxppc-dev' <linuxppc-dev@lists.ozlabs.org>,
'linux-nvdimm' <linux-nvdimm@lists.01.org>,
"'Linux MM'" <linux-mm@kvack.org>
Subject: RE: [PATCH v4 00/25] Add support for OpenCAPI Persistent Memory devices
Date: Thu, 02 Apr 2020 14:42:29 +1100 [thread overview]
Message-ID: <87imiituxm.fsf@mpe.ellerman.id.au> (raw)
In-Reply-To: <2d6901d60877$16aa7a90$43ff6fb0$@d-silva.org>
"Alastair D'Silva" <alastair@d-silva.org> writes:
>> -----Original Message-----
>> From: Dan Williams <dan.j.williams@intel.com>
>>
>> On Sun, Mar 29, 2020 at 10:23 PM Alastair D'Silva <alastair@d-silva.org>
>> wrote:
>> >
>> > This series adds support for OpenCAPI Persistent Memory devices on
>> > bare metal (arch/powernv), exposing them as nvdimms so that we can
>> > make use of the existing infrastructure. There already exists a driver
>> > for the same devices abstracted through PowerVM (arch/pseries):
>> > arch/powerpc/platforms/pseries/papr_scm.c
>> >
>> > These devices are connected via OpenCAPI, and present as LPC (lowest
>> coherence point) memory to the system, practically, that means that
>> memory on these cards could be treated as conventional, cache-coherent
>> memory.
>> >
>> > Since the devices are connected via OpenCAPI, they are not enumerated
>> via ACPI. Instead, OpenCAPI links present as pseudo-PCI bridges, with
>> devices below them.
>> >
>> > This series introduces a driver that exposes the memory on these cards as
>> nvdimms, with each card getting it's own bus. This is somewhat complicated
>> by the fact that the cards do not have out of band persistent storage for
>> metadata, so 1 SECTION_SIZE's (see SPARSEMEM) worth of storage is carved
>> out of the top of the card storage to implement the ndctl_config_* calls.
>>
>> Is it really tied to section-size? Can't that change based on the configured
>> page-size? It's not clear to me why that would be the choice, but I'll dig into
>> the implementation.
>>
>
> I had tried using PAGE_SIZE, but ran into problems carving off just 1 page and handing it to the kernel, while leaving the rest as pmem. That was a while ago though, so maybe I should retry it.
>
>> > The driver is not responsible for configuring the NPU (NVLink Processing
>> Unit) BARs to map the LPC memory from the card into the system's physical
>> address space, instead, it requests this to be done via OPAL calls (typically
>> implemented by Skiboot).
>>
>> Are OPAL calls similar to ACPI DSMs? I.e. methods for the OS to invoke
>> platform firmware services? What's Skiboot?
>>
>
> Yes, OPAL is the interface to firmware for POWER. Skiboot is the open-source (and only) implementation of OPAL.
https://github.com/open-power/skiboot
In particular the tokens for calls are defined here:
https://github.com/open-power/skiboot/blob/master/include/opal-api.h#L220
And you can grep for the token to find the implementation:
https://github.com/open-power/skiboot/blob/master/hw/npu2-opencapi.c#L2328
cheers
next prev parent reply other threads:[~2020-04-02 3:42 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-27 7:11 [PATCH v4 00/25] Add support for OpenCAPI Persistent Memory devices Alastair D'Silva
2020-03-27 7:11 ` [PATCH v4 01/25] powerpc/powernv: Add OPAL calls for LPC memory alloc/release Alastair D'Silva
2020-04-01 8:48 ` Dan Williams
2020-04-01 22:51 ` Alastair D'Silva
2020-03-27 7:11 ` [PATCH v4 02/25] mm/memory_hotplug: Allow check_hotplug_memory_addressable to be called from drivers Alastair D'Silva
2020-04-01 8:48 ` Dan Williams
2020-04-02 4:33 ` Alastair D'Silva
2020-03-27 7:11 ` [PATCH v4 03/25] powerpc/powernv: Map & release OpenCAPI LPC memory Alastair D'Silva
2020-04-01 8:48 ` Dan Williams
2020-04-02 4:36 ` Alastair D'Silva
2020-04-02 10:41 ` Benjamin Herrenschmidt
2020-04-03 4:27 ` Michael Ellerman
2020-03-27 7:11 ` [PATCH v4 04/25] ocxl: Remove unnecessary externs Alastair D'Silva
2020-04-01 8:48 ` Dan Williams
2020-03-27 7:11 ` [PATCH v4 05/25] ocxl: Address kernel doc errors & warnings Alastair D'Silva
2020-04-01 8:49 ` Dan Williams
2020-03-27 7:11 ` [PATCH v4 06/25] ocxl: Tally up the LPC memory on a link & allow it to be mapped Alastair D'Silva
2020-04-01 8:48 ` Dan Williams
2020-04-02 6:21 ` Andrew Donnellan
2020-03-27 7:11 ` [PATCH v4 07/25] ocxl: Add functions to map/unmap LPC memory Alastair D'Silva
2020-04-01 8:49 ` Dan Williams
2020-04-03 3:50 ` Alastair D'Silva
2020-03-27 7:11 ` [PATCH v4 08/25] ocxl: Emit a log message showing how much LPC memory was detected Alastair D'Silva
2020-04-01 8:49 ` Dan Williams
2020-04-02 1:29 ` Joe Perches
2020-04-03 3:52 ` Alastair D'Silva
2020-03-27 7:11 ` [PATCH v4 09/25] ocxl: Save the device serial number in ocxl_fn Alastair D'Silva
2020-03-27 7:11 ` [PATCH v4 10/25] nvdimm: Add driver for OpenCAPI Persistent Memory Alastair D'Silva
2020-03-29 2:56 ` Matthew Wilcox
2020-03-29 2:59 ` Matthew Wilcox
2020-04-01 8:49 ` Dan Williams
2020-04-01 19:35 ` Dan Williams
2020-03-27 7:11 ` [PATCH v4 11/25] powerpc: Enable the OpenCAPI Persistent Memory driver for powernv_defconfig Alastair D'Silva
2020-04-01 20:26 ` Dan Williams
2020-03-27 7:11 ` [PATCH v4 12/25] nvdimm/ocxl: Add register addresses & status values to the header Alastair D'Silva
2020-04-01 20:27 ` Dan Williams
2020-03-27 7:11 ` [PATCH v4 13/25] nvdimm/ocxl: Read the capability registers & wait for device ready Alastair D'Silva
2020-04-02 0:20 ` Dan Williams
2020-03-27 7:11 ` [PATCH v4 14/25] nvdimm/ocxl: Add support for Admin commands Alastair D'Silva
2020-04-02 6:41 ` Dan Williams
2020-03-27 7:11 ` [PATCH v4 15/25] nvdimm/ocxl: Register a character device for userspace to interact with Alastair D'Silva
2020-04-02 0:27 ` Dan Williams
2020-03-27 7:11 ` [PATCH v4 16/25] nvdimm/ocxl: Implement the Read Error Log command Alastair D'Silva
2020-04-03 0:54 ` Dan Williams
2020-03-27 7:11 ` [PATCH v4 17/25] nvdimm/ocxl: Add controller dump IOCTLs Alastair D'Silva
2020-03-27 7:11 ` [PATCH v4 18/25] nvdimm/ocxl: Add an IOCTL to report controller statistics Alastair D'Silva
2020-03-27 7:11 ` [PATCH v4 19/25] nvdimm/ocxl: Forward events to userspace Alastair D'Silva
2020-04-02 2:08 ` Dan Williams
2020-03-27 7:11 ` [PATCH v4 20/25] nvdimm/ocxl: Add an IOCTL to request controller health & perf data Alastair D'Silva
2020-03-27 7:11 ` [PATCH v4 21/25] nvdimm/ocxl: Implement the heartbeat command Alastair D'Silva
2020-03-27 7:11 ` [PATCH v4 22/25] nvdimm/ocxl: Add debug IOCTLs Alastair D'Silva
2020-03-27 7:12 ` [PATCH v4 23/25] nvdimm/ocxl: Expose SMART data via ndctl Alastair D'Silva
2020-03-27 7:12 ` [PATCH v4 24/25] nvdimm/ocxl: Expose the serial number & firmware version in sysfs Alastair D'Silva
2020-03-27 7:12 ` [PATCH v4 25/25] MAINTAINERS: Add myself & nvdimm/ocxl to ocxl Alastair D'Silva
2020-04-01 8:47 ` [PATCH v4 00/25] Add support for OpenCAPI Persistent Memory devices Dan Williams
2020-04-01 22:44 ` Alastair D'Silva
2020-04-02 3:42 ` Michael Ellerman [this message]
2020-04-02 3:50 ` Oliver O'Halloran
2020-04-02 10:06 ` Michael Ellerman
2020-04-02 11:10 ` Greg Kurz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87imiituxm.fsf@mpe.ellerman.id.au \
--to=mpe@ellerman.id.au \
--cc=aik@ozlabs.ru \
--cc=ajd@linux.ibm.com \
--cc=akpm@linux-foundation.org \
--cc=alastair@d-silva.org \
--cc=aneesh.kumar@linux.ibm.com \
--cc=anju@linux.vnet.ibm.com \
--cc=anton@ozlabs.org \
--cc=arnd@arndb.de \
--cc=benh@kernel.crashing.org \
--cc=clg@kaod.org \
--cc=dan.j.williams@intel.com \
--cc=dave.jiang@intel.com \
--cc=davem@davemloft.net \
--cc=fbarrat@linux.ibm.com \
--cc=gregkh@linuxfoundation.org \
--cc=groug@kaod.org \
--cc=hbathini@linux.ibm.com \
--cc=ira.weiny@intel.com \
--cc=krzk@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nvdimm@lists.01.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=maddy@linux.vnet.ibm.com \
--cc=mahesh@linux.vnet.ibm.com \
--cc=mchehab+samsung@kernel.org \
--cc=npiggin@gmail.com \
--cc=oohall@gmail.com \
--cc=paulus@samba.org \
--cc=robh@kernel.org \
--cc=tglx@linutronix.de \
--cc=vishal.l.verma@intel.com \
--cc=yamada.masahiro@socionext.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).