From: Vishal Verma <vishal.l.verma@intel.com>
To: linux-nvdimm@lists.01.org
Cc: Dave Hansen <dave.hansen@linux.intel.com>,
Pavel Tatashin <pasha.tatashin@soleen.com>
Subject: [ndctl PATCH v7 00/13] daxctl: add a new reconfigure-device command
Date: Wed, 24 Jul 2019 15:57:28 -0600 [thread overview]
Message-ID: <20190724215741.18556-1-vishal.l.verma@intel.com> (raw)
Changes in v7:
- Fix a couple of checkaptch type errors in the new lines added in v6 (Dan).
- Get rid of daxctl_dev_get_mode. daxctl_dev_get_memory is sufficient to
both check the mode and allocate the memory related structures on its
first call. (Dan)
- Due to the above, daxctl_dev_mode is now private to libdaxctl, and not
part of the API exported through libdaxctl.h
- Add a large enough buffer at init time to construct dynamic paths, and avoid
asprintf() type allocations for memory blocks at runtime (Dan).
Changes in v6:
- For memory block online/offline operations, the kernel responds with
an EINVAL for both 'real' errors, and if the memory was already in the
requested state. Since there is a TOCTOU hole between checking the
state and storing it, just perform a second check if the store results
in an error. If the check shows the state to be the same as the one
we're attempting, it means that another agent (usually udev) won the
race, but we don't care so long as the state change happened, so don't
report an error. (Fan Du)
Changes in v5:
- device.c: correctly set loglevel for daxctl_ctx for --verbose
- drop the subsys caching, its complexity started to exceed its
benefit. dax-class device models will simply error out during
reconfigure. (Dan)
- Add a note to the man page for the above.
- Clarify the onlining policy (online_movable) in the man page
- rename "numa_node" to "target_node" in device listings (Dan)
- When printing a device 'mode', assume devdax if !system-ram,
avoiding a "mode: unknown" situation which can be confusing. (Dan)
- Add a "state: disabled" attribute to the device listing if a driver
is not bound. This is more apt than the previous "mode: unknown"
listing.
- add an api to get 'dev->resource' parsing /proc/iomem as a
fallback for when the kernel doesn't provide the attribute (Dan)
- convert node_* apis to 'memory_* apis that act on a new daxctl_memory
object (Dan)
- online only memory sections belonging to the device in question by
cross referencing block indices with the dax device resource (Dan)
- Refuse to reconfigure a device that is already in the target mode.
Until now, reconfiguring a system-ram device back to system-ram would
result in a 'online memory may not be hot-removed' kernel warning.
- If the device was already in the system-ram mode, skip
disabling/enabling, but still try to online the memory unless the
--no-online option is in effect.
- In daxctl_unbind, also 'remove_id' to prevent devices automatically
binding to the kmem driver on a disable + re-enable, which can be
surprising (Dan).
- Rewrite the top half of daxctl/device.c to borrow elements from
ndctl/namespace.c so that it can support growing additional commands
that operate on devices (online-memory and offline-memory)
- Refactor the bottom half of daxctl/device.c so we only do the
disabling/offlining steps if the device was enabled.
- Add new commands to online and offline memory sections (Dan)
associated with a given dax device (Dan)
- Add a new test - daxctl-device.sh - to test daxctl reconfigure-device,
online-memory, and offline-memory commands.
- Add an example in documentation demonstrating how to use numactl
to bind a process to a node surfaced from a dax device (Andy Rudoff)
Changes in v4:
- Don't fail add_dax_dev for kmod failures. Instead fail only when the kmod
list is actually used, i.e. during daxctl-reconfigure-device
Changes in v3:
- In daxctl_dev_get_mode(), remove the subsystem warning, detect dax-class
and simply make it return devdax
Changes in v2:
- Add examples to the documentation page (Dave Hansen)
- Clarify documentation regarding the conversion from system-ram to devdax
- Remove any references to a persistent config from the documentation -
those can be added when the feature is added.
- device.c: validate option compatibility
- daxctl-list: display numa_node for device listings
- daxctl-list: display mode for device listings
- make the options more consistent by adding a '-O' short option
for --attempt-offline
Add a new daxctl-reconfigure-device command that lets us reconfigure DAX
devices back and forth between 'system-ram' and 'device-dax' modes. It
also includes facilities to online any newly hot-plugged memory
(default), and attempt to offline memory before converting away from the
system-ram mode (not default, requires a --attempt-offline option).
Currently missing from this series is a way to persistently store which
devices have been 'marked' for use as system-ram. This depends on a
config system overhaul in ndctl, and patches for those will follow
separately and are independent of this work.
Example invocations:
1. Reconfigure dax0.0 to system-ram mode, don’t online the memory
# daxctl reconfigure-device --mode=system-ram --no-online dax0.0
[
{
"chardev":"dax0.0",
"size":16777216000,
"target_node":2,
"mode":"system-ram"
}
]
2. Reconfigure dax0.0 to devdax mode, attempt to offline the memory
# daxctl reconfigure-device --human --mode=devdax --attempt-offline dax0.0
{
"chardev":"dax0.0",
"size":"15.63 GiB (16.78 GB)",
"target_node":2,
"mode":"devdax"
}
3. Reconfigure all dax devices on region0 to system-ram mode
# daxctl reconfigure-device --mode=system-ram --region=0 all
[
{
"chardev":"dax0.0",
"size":16777216000,
"target_node":2,
"mode":"system-ram"
},
{
"chardev":"dax0.1",
"size":16777216000,
"target_node":3,
"mode":"system-ram"
}
]
These patches can also be found in the 'kmem-pending' branch on github:
https://github.com/pmem/ndctl/tree/kmem-pending
Cc: Dan Williams <dan.j.williams@intel.com>
Cc: Dave Hansen <dave.hansen@linux.intel.com>
Cc: Pavel Tatashin <pasha.tatashin@soleen.com>
Vishal Verma (13):
libdaxctl: add interfaces to get ctx and check device state
libdaxctl: add interfaces to enable/disable devices
libdaxctl: add an interface to retrieve the device resource
libdaxctl: add a 'daxctl_memory' object for memory based operations
daxctl/list: add target_node for device listings
daxctl/list: display the mode for a dax device
daxctl: add a new reconfigure-device command
Documentation/daxctl: add a man page for daxctl-reconfigure-device
daxctl: add commands to online and offline memory
Documentation: Add man pages for daxctl-{on,off}line-memory
contrib/ndctl: fix region-id completions for daxctl
contrib/ndctl: add bash-completion for the new daxctl commands
test: Add a unit test for daxctl-reconfigure-device and friends
Documentation/daxctl/Makefile.am | 5 +-
.../daxctl/daxctl-offline-memory.txt | 72 ++
Documentation/daxctl/daxctl-online-memory.txt | 80 ++
.../daxctl/daxctl-reconfigure-device.txt | 139 ++++
Makefile.am | 3 +-
contrib/ndctl | 38 +-
daxctl/Makefile.am | 2 +
daxctl/builtin.h | 3 +
daxctl/daxctl.c | 3 +
daxctl/device.c | 476 ++++++++++++
daxctl/lib/Makefile.am | 5 +-
daxctl/lib/libdaxctl-private.h | 38 +
daxctl/lib/libdaxctl.c | 685 ++++++++++++++++++
daxctl/lib/libdaxctl.sym | 18 +
daxctl/libdaxctl.h | 16 +
test/Makefile.am | 3 +-
test/common | 19 +-
test/daxctl-devices.sh | 81 +++
util/iomem.c | 37 +
util/iomem.h | 12 +
util/json.c | 22 +
21 files changed, 1743 insertions(+), 14 deletions(-)
create mode 100644 Documentation/daxctl/daxctl-offline-memory.txt
create mode 100644 Documentation/daxctl/daxctl-online-memory.txt
create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt
create mode 100644 daxctl/device.c
create mode 100755 test/daxctl-devices.sh
create mode 100644 util/iomem.c
create mode 100644 util/iomem.h
--
2.20.1
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
next reply other threads:[~2019-07-24 22:00 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-07-24 21:57 Vishal Verma [this message]
2019-07-24 21:57 ` [ndctl PATCH v7 01/13] libdaxctl: add interfaces to get ctx and check device state Vishal Verma
2019-07-24 21:57 ` [ndctl PATCH v7 02/13] libdaxctl: add interfaces to enable/disable devices Vishal Verma
2019-07-24 21:57 ` [ndctl PATCH v7 03/13] libdaxctl: add an interface to retrieve the device resource Vishal Verma
2019-07-24 21:57 ` [ndctl PATCH v7 04/13] libdaxctl: add a 'daxctl_memory' object for memory based operations Vishal Verma
2019-07-24 21:57 ` [ndctl PATCH v7 05/13] daxctl/list: add target_node for device listings Vishal Verma
2019-07-24 21:57 ` [ndctl PATCH v7 06/13] daxctl/list: display the mode for a dax device Vishal Verma
2019-07-24 21:57 ` [ndctl PATCH v7 07/13] daxctl: add a new reconfigure-device command Vishal Verma
2019-07-24 21:57 ` [ndctl PATCH v7 08/13] Documentation/daxctl: add a man page for daxctl-reconfigure-device Vishal Verma
2019-07-26 2:46 ` Dan Williams
2019-07-26 23:14 ` Verma, Vishal L
2019-07-24 21:57 ` [ndctl PATCH v7 09/13] daxctl: add commands to online and offline memory Vishal Verma
2019-07-26 2:51 ` Dan Williams
2019-07-26 23:15 ` Verma, Vishal L
2019-07-24 21:57 ` [ndctl PATCH v7 10/13] Documentation: Add man pages for daxctl-{on, off}line-memory Vishal Verma
2019-07-26 2:52 ` Dan Williams
2019-07-24 21:57 ` [ndctl PATCH v7 11/13] contrib/ndctl: fix region-id completions for daxctl Vishal Verma
2019-07-26 2:55 ` Dan Williams
2019-07-26 23:30 ` Verma, Vishal L
2019-07-26 23:44 ` Dan Williams
2019-07-24 21:57 ` [ndctl PATCH v7 12/13] contrib/ndctl: add bash-completion for the new daxctl commands Vishal Verma
2019-07-26 2:56 ` Dan Williams
2019-07-24 21:57 ` [ndctl PATCH v7 13/13] test: Add a unit test for daxctl-reconfigure-device and friends Vishal Verma
2019-07-26 3:43 ` Dan Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190724215741.18556-1-vishal.l.verma@intel.com \
--to=vishal.l.verma@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=linux-nvdimm@lists.01.org \
--cc=pasha.tatashin@soleen.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).