qemu-emulated nfit devices disappeared

qemu-emulated nfit devices disappeared

[Christoph Hellwig]

[-- Attachment #1: Type: text/plain, Size: 737 bytes --]

I've update my typical test configs last week without saving the old
ones, and now the Qemu emulated NFIT devices don't show anyway more
despite the kernel showing a NFIT table:

# mesg | grep -i nfit
[    0.009184] ACPI: NFIT 0x00000000BFFE02FA 000198 (v01 BOCHS BXPCNFIT 00000001 BXPC 00000001)
[    0.009206] ACPI: Reserving NFIT table memory at [mem 0xbffe02fa-0xbffe0491]

# lsblk 
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda      8:0    0    8M  0 disk 
sr0     11:0    1 1024M  0 rom  
nullb0 250:0    0  250G  0 disk 
vda    252:0    0   10G  0 disk 
└─vda1 252:1    0   10G  0 part /

Any idea what might be missing in the attached config?  This seems to
be independent of the tested kernel version (5.14, 5.15, master).

[-- Attachment #2: config.gz --]
[-- Type: application/gzip, Size: 36548 bytes --]

Re: qemu-emulated nfit devices disappeared

[Dan Williams]

On Thu, Nov 4, 2021 at 12:27 AM Christoph Hellwig <hch@infradead.org> wrote:
>
> I've update my typical test configs last week without saving the old
> ones, and now the Qemu emulated NFIT devices don't show anyway more
> despite the kernel showing a NFIT table:
>
> # mesg | grep -i nfit
> [    0.009184] ACPI: NFIT 0x00000000BFFE02FA 000198 (v01 BOCHS BXPCNFIT 00000001 BXPC 00000001)
> [    0.009206] ACPI: Reserving NFIT table memory at [mem 0xbffe02fa-0xbffe0491]
>
> # lsblk
> NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
> sda      8:0    0    8M  0 disk
> sr0     11:0    1 1024M  0 rom
> nullb0 250:0    0  250G  0 disk
> vda    252:0    0   10G  0 disk
> └─vda1 252:1    0   10G  0 part /
>
> Any idea what might be missing in the attached config?  This seems to
> be independent of the tested kernel version (5.14, 5.15, master).

Can you share your qemu command line and the output of:

ndctl list -vvv

Re: qemu-emulated nfit devices disappeared

[Christoph Hellwig]

On Thu, Nov 04, 2021 at 09:00:01AM -0700, Dan Williams wrote:
> > Any idea what might be missing in the attached config?  This seems to
> > be independent of the tested kernel version (5.14, 5.15, master).
> 
> Can you share your qemu command line

qemu-system-x86_64 -enable-kvm \
	-machine pc,nvdimm=on \
	-m 4096,slots=8,maxmem=32G \
	-smp 4 \
	-kernel arch/x86/boot/bzImage \
	-append "root=/dev/vda1 console=ttyS0,115200n8" \
	-nographic \
	-drive if=virtio,file=/home/hch/images/jessie.img,cache=none \
	-object	memory-backend-file,id=mem0,mem-path=/home/hch/images/test.img,share=yes,prealloc=no,align=128M,size=10G \
	-device nvdimm,id=nvdimm0,memdev=mem0,slot=0,label-size=2M \
	-object memory-backend-file,id=mem1,mem-path=/home/hch/images/scratch.img,share=yes,prealloc=no,align=128M,size=10G \
	-device nvdimm,id=nvdimm1,memdev=mem1,slot=1,label-size=2M \


> and the output of:
> 
> ndctl list -vvv


[
  {
    "provider":"ACPI.NFIT",
    "dev":"ndbus0",
    "dimms":[
      {
        "dev":"nmem1",
        "id":"8680-57341200",
        "handle":2,
        "phys_id":0
      },
      {
        "dev":"nmem0",
        "id":"8680-56341200",
        "handle":1,
        "phys_id":0
      }
    ],
    "regions":[
      {
        "dev":"region1",
        "size":10603200512,
        "available_size":10603200512,
        "max_available_extent":10603200512,
        "type":"pmem",
        "iset_id":52512795602891997,
        "mappings":[
          {
            "dimm":"nmem1",
            "offset":0,
            "length":10603200512,
            "position":0
          }
        ],
        "persistence_domain":"unknown",
        "namespaces":[
          {
            "dev":"namespace1.0",
            "mode":"raw",
            "size":0,
            "uuid":"00000000-0000-0000-0000-000000000000",
            "sector_size":512,
            "state":"disabled"
          }
        ]
      },
      {
        "dev":"region0",
        "size":10603200512,
        "available_size":10603200512,
        "max_available_extent":10603200512,
        "type":"pmem",
        "iset_id":52512752653219036,
        "mappings":[
          {
            "dimm":"nmem0",
            "offset":0,
            "length":10603200512,
            "position":0
          }
        ],
        "persistence_domain":"unknown",
        "namespaces":[
          {
            "dev":"namespace0.0",
            "mode":"raw",
            "size":0,
            "uuid":"00000000-0000-0000-0000-000000000000",
            "sector_size":512,
            "state":"disabled"
          }
        ]
      }
    ]
  }
]

Re: qemu-emulated nfit devices disappeared

[Dan Williams]

On Thu, Nov 4, 2021 at 9:04 AM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Thu, Nov 04, 2021 at 09:00:01AM -0700, Dan Williams wrote:
> > > Any idea what might be missing in the attached config?  This seems to
> > > be independent of the tested kernel version (5.14, 5.15, master).
> >
> > Can you share your qemu command line
>
> qemu-system-x86_64 -enable-kvm \
>         -machine pc,nvdimm=on \
>         -m 4096,slots=8,maxmem=32G \
>         -smp 4 \
>         -kernel arch/x86/boot/bzImage \
>         -append "root=/dev/vda1 console=ttyS0,115200n8" \
>         -nographic \
>         -drive if=virtio,file=/home/hch/images/jessie.img,cache=none \
>         -object memory-backend-file,id=mem0,mem-path=/home/hch/images/test.img,share=yes,prealloc=no,align=128M,size=10G \
>         -device nvdimm,id=nvdimm0,memdev=mem0,slot=0,label-size=2M \
>         -object memory-backend-file,id=mem1,mem-path=/home/hch/images/scratch.img,share=yes,prealloc=no,align=128M,size=10G \
>         -device nvdimm,id=nvdimm1,memdev=mem1,slot=1,label-size=2M \
>
>
> > and the output of:
> >
> > ndctl list -vvv
>
>
> [
>   {
>     "provider":"ACPI.NFIT",
>     "dev":"ndbus0",
>     "dimms":[
>       {
>         "dev":"nmem1",
>         "id":"8680-57341200",
>         "handle":2,
>         "phys_id":0
>       },
>       {
>         "dev":"nmem0",
>         "id":"8680-56341200",
>         "handle":1,
>         "phys_id":0
>       }
>     ],
>     "regions":[
>       {
>         "dev":"region1",
>         "size":10603200512,
>         "available_size":10603200512,

Hmm, so the driver has 2 modes "labeled" and "label-less", in the
labeled mode it waits for an explicit:

    ndctl create-namespace

...to provision region capacity into a namespace. In label-less mode
it just assumes that the boundaries of the region are the boundaries
of the namespace. In this case it looks like the driver found a label
index block with no namespaces defined so it's waiting for one to be
created. Are you saying that the only thing you changed from a working
config with defined namespace to this one was a kernel change? I.e.
the content of those memory-backend files has not changed?


>         "max_available_extent":10603200512,
>         "type":"pmem",
>         "iset_id":52512795602891997,
>         "mappings":[
>           {
>             "dimm":"nmem1",
>             "offset":0,
>             "length":10603200512,
>             "position":0
>           }
>         ],
>         "persistence_domain":"unknown",
>         "namespaces":[
>           {
>             "dev":"namespace1.0",
>             "mode":"raw",
>             "size":0,
>             "uuid":"00000000-0000-0000-0000-000000000000",
>             "sector_size":512,
>             "state":"disabled"
>           }
>         ]
>       },
>       {
>         "dev":"region0",
>         "size":10603200512,
>         "available_size":10603200512,
>         "max_available_extent":10603200512,
>         "type":"pmem",
>         "iset_id":52512752653219036,
>         "mappings":[
>           {
>             "dimm":"nmem0",
>             "offset":0,
>             "length":10603200512,
>             "position":0
>           }
>         ],
>         "persistence_domain":"unknown",
>         "namespaces":[
>           {
>             "dev":"namespace0.0",
>             "mode":"raw",
>             "size":0,
>             "uuid":"00000000-0000-0000-0000-000000000000",
>             "sector_size":512,
>             "state":"disabled"
>           }
>         ]
>       }
>     ]
>   }
> ]

Re: qemu-emulated nfit devices disappeared

[Christoph Hellwig]

On Thu, Nov 04, 2021 at 10:09:37AM -0700, Dan Williams wrote:
> Hmm, so the driver has 2 modes "labeled" and "label-less", in the
> labeled mode it waits for an explicit:
> 
>     ndctl create-namespace
> 
> ...to provision region capacity into a namespace. In label-less mode
> it just assumes that the boundaries of the region are the boundaries
> of the namespace. In this case it looks like the driver found a label
> index block with no namespaces defined so it's waiting for one to be
> created. Are you saying that the only thing you changed from a working
> config with defined namespace to this one was a kernel change? I.e.
> the content of those memory-backend files has not changed?

Well, the config change is the only thing I though of as relevant.
The content of those files actually changes all the time, as I also
use them as the backing store for my qemu configs that use block
devices.  E.g. the previous run they did show up as NVMe devices.

Re: qemu-emulated nfit devices disappeared

[Christoph Hellwig]

On Thu, Nov 04, 2021 at 10:17:31AM -0700, Christoph Hellwig wrote:
> On Thu, Nov 04, 2021 at 10:09:37AM -0700, Dan Williams wrote:
> > Hmm, so the driver has 2 modes "labeled" and "label-less", in the
> > labeled mode it waits for an explicit:
> > 
> >     ndctl create-namespace
> > 
> > ...to provision region capacity into a namespace. In label-less mode
> > it just assumes that the boundaries of the region are the boundaries
> > of the namespace. In this case it looks like the driver found a label
> > index block with no namespaces defined so it's waiting for one to be
> > created. Are you saying that the only thing you changed from a working
> > config with defined namespace to this one was a kernel change? I.e.
> > the content of those memory-backend files has not changed?
> 
> Well, the config change is the only thing I though of as relevant.
> The content of those files actually changes all the time, as I also
> use them as the backing store for my qemu configs that use block
> devices.  E.g. the previous run they did show up as NVMe devices.

Ok, blowing the files away and recreating them makes the pmem
devices show up again.

Re: qemu-emulated nfit devices disappeared

[Dan Williams]

On Thu, Nov 4, 2021 at 10:20 AM Christoph Hellwig <hch@infradead.org> wrote:
>
> On Thu, Nov 04, 2021 at 10:17:31AM -0700, Christoph Hellwig wrote:
> > On Thu, Nov 04, 2021 at 10:09:37AM -0700, Dan Williams wrote:
> > > Hmm, so the driver has 2 modes "labeled" and "label-less", in the
> > > labeled mode it waits for an explicit:
> > >
> > >     ndctl create-namespace
> > >
> > > ...to provision region capacity into a namespace. In label-less mode
> > > it just assumes that the boundaries of the region are the boundaries
> > > of the namespace. In this case it looks like the driver found a label
> > > index block with no namespaces defined so it's waiting for one to be
> > > created. Are you saying that the only thing you changed from a working
> > > config with defined namespace to this one was a kernel change? I.e.
> > > the content of those memory-backend files has not changed?
> >
> > Well, the config change is the only thing I though of as relevant.
> > The content of those files actually changes all the time, as I also
> > use them as the backing store for my qemu configs that use block
> > devices.  E.g. the previous run they did show up as NVMe devices.
>
> Ok, blowing the files away and recreating them makes the pmem
> devices show up again.
>

Cool, yeah, I was going to guess the label area might have been
clobbered, but there was just enough left to force the driver into
labeled mode. You can verify with something like:

ndctl read-labels -I -j all

...to see if nvdimms have label index blocks.