Re: [v5 0/3] "Hotremove" persistent memory

["Verma, Vishal L" <vishal.l.verma@intel.com>]

[-- Attachment #1.1: Type: text/plain, Size: 2644 bytes --]

On Thu, 2019-05-02 at 18:36 -0400, Pavel Tatashin wrote:
> > Yes, here is the qemu config:
> > 
> > qemu-system-x86_64
> >         -machine accel=kvm
> >         -machine pc-i440fx-2.6,accel=kvm,usb=off,vmport=off,dump-guest-core=off,nvdimm
> >         -cpu Haswell-noTSX
> >         -m 12G,slots=3,maxmem=44G
> >         -realtime mlock=off
> >         -smp 8,sockets=2,cores=4,threads=1
> >         -numa node,nodeid=0,cpus=0-3,mem=6G
> >         -numa node,nodeid=1,cpus=4-7,mem=6G
> >         -numa node,nodeid=2
> >         -numa node,nodeid=3
> >         -drive file=/virt/fedora-test.qcow2,format=qcow2,if=none,id=drive-virtio-disk1
> >         -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x9,drive=drive-virtio-disk1,id=virtio-disk1,bootindex=1
> >         -object memory-backend-file,id=mem1,share,mem-path=/virt/nvdimm1,size=16G,align=128M
> >         -device nvdimm,memdev=mem1,id=nv1,label-size=2M,node=2
> >         -object memory-backend-file,id=mem2,share,mem-path=/virt/nvdimm2,size=16G,align=128M
> >         -device nvdimm,memdev=mem2,id=nv2,label-size=2M,node=3
> >         -serial stdio
> >         -display none
> > 
> > For the command list - I'm using WIP patches to ndctl/daxctl to add the
> > command I mentioned earlier. Using this command, I can reproduce the
> > lockdep issue. I thought I should be able to reproduce the issue by
> > onlining/offlining through sysfs directly too - something like:
> > 
> >    node="$(cat /sys/bus/dax/devices/dax0.0/target_node)"
> >    for mem in /sys/devices/system/node/node"$node"/memory*; do
> >      echo "offline" > $mem/state
> >    done
> > 
> > But with that I can't reproduce the problem.
> > 
> > I'll try to dig a bit deeper into what might be happening, the daxctl
> > modifications simply amount to doing the same thing as above in C, so
> > I'm not immediately sure what might be happening.
> > 
> > If you're interested, I can post the ndctl patches - maybe as an RFC -
> > to test with.
> 
> I could apply the patches and test with them. Also, could you please
> send your kernel config.
> 
Hi Pavel,

I've CC'd you on the patches mentioned above, and also pushed them to a
'kmem-pending' branch on github:

https://github.com/pmem/ndctl/tree/kmem-pending

After building ndctl from the above, you will want to run:

# daxctl reconfigure-device --mode=system-ram dax0.0
(this will also have onlined the memory sections)

# daxctl reconfigure-device --mode=devdax --attempt-offline dax0.0
(this triggers the lockdep warnings)

I've attached the kernel config here too (gzipped).

Thanks,
	-Vishal

[-- Attachment #1.2: config.gz --]
[-- Type: application/gzip, Size: 30367 bytes --]

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3270 bytes --]