Re: [SPDK] How to setup a QEMU system disk by using SPDK?

Re: [SPDK] How to setup a QEMU system disk by using SPDK?

[Wodkowski, PawelX]

[-- Attachment #1: Type: text/plain, Size: 4727 bytes --]

You can use SPDK nbd to expose lvol and write it while vhost is running. Then just stop exported nbd and pass lvol to qemu.

Pawel

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Bob Chen
Sent: Wednesday, June 6, 2018 12:28 PM
To: Wang, ChenX <chenx.wang(a)intel.com>
Cc: Storage Performance Development Kit <spdk(a)lists.01.org>; qemu block <qemu-block(a)nongnu.org>
Subject: Re: [SPDK] How to setup a QEMU system disk by using SPDK?


2018-06-06 16:40 GMT+08:00 Wang, ChenX <chenx.wang(a)intel.com<mailto:chenx.wang(a)intel.com>>:
Hi,  Bob

I have try to boot the os by QEMU UEFI, but failed, following is my qemu command:
. /qemu-system-x86_64 -smbios type=0,uefi=on -smp 8 -m 1024 -net user,hostfwd=tcp::10020-:22 \
-net nic -chardev socket,id=char0,path=vhost.0 -device vhost-user-blk-pci,chardev=char0,num-queues=4,bootindex=0  --enable-kvm

Would you be willing to share your commands and some tips for QEMU UEFI? Thanks so much. ☺

I was using the edk2-ovmf UEFI, https://github.com/tianocore/edk2. You may download the nightly build RPMs from https://www.kraxel.org/repos/jenkins/edk2/
Add a pflash drive to qemu's command-line, looks like:

-drive file=OVMF_CODE-pure-efi.fd,format=raw,if=pflash \
-m 4G -object memory-backend-file,id=mem0,size=4G,mem-path=/dev/hugepages,share=on -numa node,memdev=mem0 \
-chardev socket,id=spdk_vhost_scsi0,path=/var/tmp/vhost.0 \
-device vhost-user-scsi-pci,id=scsi0,chardev=spdk_vhost_scsi0,num_queues=4

However you still don't have a correct SPDK volume by far. Because most of the images you could download from the Internet are built on traditional BIOS boot-loader, thus cannot be directly copied. What I did is having the guest OS reinstalled from its attaching live CD-ROM.

Regarding to the import/export functionality, I think another app is needed to read/write lvol while interacting with vhost app simultaneously. Like qemu itself a little bit.
Perhaps someone could integrate it into the qemu-img command?


Thanks,
Chen

From: SPDK [mailto:spdk-bounces(a)lists.01.org<mailto:spdk-bounces(a)lists.01.org>] On Behalf Of Harris, James R
Sent: Wednesday, June 6, 2018 2:14 AM
To: Bob Chen <a175818323(a)gmail.com<mailto:a175818323(a)gmail.com>>; Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>; Cao, Gang <gang.cao(a)intel.com<mailto:gang.cao(a)intel.com>>; qemu block <qemu-block(a)nongnu.org<mailto:qemu-block(a)nongnu.org>>
Subject: Re: [SPDK] How to setup a QEMU system disk by using SPDK?



From: Bob Chen <a175818323(a)gmail.com<mailto:a175818323(a)gmail.com>>
Date: Monday, June 4, 2018 at 10:47 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>, "Cao, Gang" <gang.cao(a)intel.com<mailto:gang.cao(a)intel.com>>, James Harris <james.r.harris(a)intel.com<mailto:james.r.harris(a)intel.com>>, qemu block <qemu-block(a)nongnu.org<mailto:qemu-block(a)nongnu.org>>
Subject: Re: [SPDK] How to setup a QEMU system disk by using SPDK?


[…]


I suspect this is because all SPDK logical volumes are exposed with 4KB logical block size.  IIRC, Seabios legacy boot doesn’t support 4KB logical block devices.  UEFI boot should work but we don’t have automated tests set up for that yet.

I will have a try for UEFI boot-loader.


I think SPDK logical volumes could be exposed with 512 byte logical block size, if the underlying block device has a 512 byte logical block size.  Using 4KB LBA simplified the implementation a bit and all metadata has to use 4KB for atomicity.  But talking with Ben we don’t see any reason 512 byte for read/write/unmap operations wouldn’t work.  We may have to use feature bits though so that this change in behavior only applies to newly created logical volumes.

Bob – I’m curious if you can boot your VM with the SPDK logical volume as a secondary disk?

 Yes, we could. Although we were able to see the device in the VM, that disk with the guest OS inside still couldn't be mounted. Seemed like read-only or corrupted.

Because SPDK by far does not provide a method to import or export volume metadata to a external file, we came up with this thought and managed to copy data by `dd` and nbd. Not very straight forward...


Good news, UEFI works! Now Qemu can boot on the SPDK system disk successfully.

What we are expecting next is a more portable way to import/export volume.


Thanks for the update!  Would you be willing to add a section to the SPDK vhost doc (doc/vhost.md) with some tips for QEMU UEFI boot?

Regarding import/export – what kind of method do you have in mind?  I agree that a more straight forward method is warranted.

-Jim


[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 19409 bytes --]

Re: [SPDK] How to setup a QEMU system disk by using SPDK?

[Bob Chen]

[-- Attachment #1: Type: text/plain, Size: 5353 bytes --]

The last tricky issue of using SPDK as a system disk is how to deploy
files, for example, password and network scripts.

The host can't recognize its partitioning and filesystem, so deploying by
nbd is no longer working.

Later I'll see what I can do by using libguestfs tools.

2018-06-06 19:36 GMT+08:00 Wodkowski, PawelX <pawelx.wodkowski(a)intel.com>:

> You can use SPDK nbd to expose lvol and write it while vhost is running.
> Then just stop exported nbd and pass lvol to qemu.
>
>
>
> Pawel
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Bob Chen
> *Sent:* Wednesday, June 6, 2018 12:28 PM
> *To:* Wang, ChenX <chenx.wang(a)intel.com>
> *Cc:* Storage Performance Development Kit <spdk(a)lists.01.org>; qemu block
> <qemu-block(a)nongnu.org>
>
> *Subject:* Re: [SPDK] How to setup a QEMU system disk by using SPDK?
>
>
>
>
>
> 2018-06-06 16:40 GMT+08:00 Wang, ChenX <chenx.wang(a)intel.com>:
>
> Hi,  Bob
>
>
>
> I have try to boot the os by QEMU UEFI, but failed, following is my qemu
> command:
>
> . /qemu-system-x86_64 -smbios type=0,uefi=on -smp 8 -m 1024 -net
> user,hostfwd=tcp::10020-:22 \
>
> -net nic -chardev socket,id=char0,path=vhost.0 -device
> vhost-user-blk-pci,chardev=char0,num-queues=4,bootindex=0  --enable-kvm
>
>
>
> Would you be willing to share your commands and some tips for QEMU UEFI?
> Thanks so much. J
>
>
>
> I was using the edk2-ovmf UEFI, https://github.com/tianocore/edk2. You
> may download the nightly build RPMs from https://www.kraxel.org/
> repos/jenkins/edk2/
>
> Add a pflash drive to qemu's command-line, looks like:
>
>
>
> -drive file=OVMF_CODE-pure-efi.fd,format=raw,if=pflash \
>
> -m 4G -object memory-backend-file,id=mem0,size=4G,mem-path=/dev/hugepages,share=on
> -numa node,memdev=mem0 \
>
> -chardev socket,id=spdk_vhost_scsi0,path=/var/tmp/vhost.0 \
>
> -device vhost-user-scsi-pci,id=scsi0,chardev=spdk_vhost_scsi0,num_queues=4
>
>
>
> However you still don't have a correct SPDK volume by far. Because most of
> the images you could download from the Internet are built on traditional
> BIOS boot-loader, thus cannot be directly copied. What I did is having the
> guest OS reinstalled from its attaching live CD-ROM.
>
>
>
> Regarding to the import/export functionality, I think another app is
> needed to read/write lvol while interacting with vhost app simultaneously.
> Like qemu itself a little bit.
>
> Perhaps someone could integrate it into the qemu-img command?
>
>
>
>
>
> Thanks,
>
> Chen
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Harris,
> James R
> *Sent:* Wednesday, June 6, 2018 2:14 AM
> *To:* Bob Chen <a175818323(a)gmail.com>; Storage Performance Development
> Kit <spdk(a)lists.01.org>; Cao, Gang <gang.cao(a)intel.com>; qemu block <
> qemu-block(a)nongnu.org>
> *Subject:* Re: [SPDK] How to setup a QEMU system disk by using SPDK?
>
>
>
>
>
>
>
> *From: *Bob Chen <a175818323(a)gmail.com>
> *Date: *Monday, June 4, 2018 at 10:47 PM
> *To: *Storage Performance Development Kit <spdk(a)lists.01.org>, "Cao,
> Gang" <gang.cao(a)intel.com>, James Harris <james.r.harris(a)intel.com>, qemu
> block <qemu-block(a)nongnu.org>
> *Subject: *Re: [SPDK] How to setup a QEMU system disk by using SPDK?
>
>
>
>
>
> […]
>
>
>
>
> I suspect this is because all SPDK logical volumes are exposed with 4KB
> logical block size.  IIRC, Seabios legacy boot doesn’t support 4KB logical
> block devices.  UEFI boot should work but we don’t have automated tests set
> up for that yet.
>
>
>
> I will have a try for UEFI boot-loader.
>
>
>
>
> I think SPDK logical volumes could be exposed with 512 byte logical block
> size, if the underlying block device has a 512 byte logical block size.
> Using 4KB LBA simplified the implementation a bit and all metadata has to
> use 4KB for atomicity.  But talking with Ben we don’t see any reason 512
> byte for read/write/unmap operations wouldn’t work.  We may have to use
> feature bits though so that this change in behavior only applies to newly
> created logical volumes.
>
> Bob – I’m curious if you can boot your VM with the SPDK logical volume as
> a secondary disk?
>
>
>
>  Yes, we could. Although we were able to see the device in the VM, that
> disk with the guest OS inside still couldn't be mounted. Seemed like
> read-only or corrupted.
>
>
>
> Because SPDK by far does not provide a method to import or export volume
> metadata to a external file, we came up with this thought and managed to
> copy data by `dd` and nbd. Not very straight forward...
>
>
>
>
>
> Good news, UEFI works! Now Qemu can boot on the SPDK system disk
> successfully.
>
>
>
> What we are expecting next is a more portable way to import/export volume.
>
>
>
>
>
> Thanks for the update!  Would you be willing to add a section to the SPDK
> vhost doc (doc/vhost.md) with some tips for QEMU UEFI boot?
>
>
>
> Regarding import/export – what kind of method do you have in mind?  I
> agree that a more straight forward method is warranted.
>
>
>
> -Jim
>
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>
>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 16383 bytes --]

Re: [SPDK] How to setup a QEMU system disk by using SPDK?

[Bob Chen]

[-- Attachment #1: Type: text/plain, Size: 4226 bytes --]

2018-06-06 16:40 GMT+08:00 Wang, ChenX <chenx.wang(a)intel.com>:

> Hi,  Bob
>
>
>
> I have try to boot the os by QEMU UEFI, but failed, following is my qemu
> command:
>
> . /qemu-system-x86_64 -smbios type=0,uefi=on -smp 8 -m 1024 -net
> user,hostfwd=tcp::10020-:22 \
>
> -net nic -chardev socket,id=char0,path=vhost.0 -device
> vhost-user-blk-pci,chardev=char0,num-queues=4,bootindex=0  --enable-kvm
>
>
>
> Would you be willing to share your commands and some tips for QEMU UEFI?
> Thanks so much. J
>

I was using the edk2-ovmf UEFI, https://github.com/tianocore/edk2. You may
download the nightly build RPMs from
https://www.kraxel.org/repos/jenkins/edk2/
Add a pflash drive to qemu's command-line, looks like:

-drive file=OVMF_CODE-pure-efi.fd,format=raw,if=pflash \
-m 4G -object
memory-backend-file,id=mem0,size=4G,mem-path=/dev/hugepages,share=on -numa
node,memdev=mem0 \
-chardev socket,id=spdk_vhost_scsi0,path=/var/tmp/vhost.0 \
-device vhost-user-scsi-pci,id=scsi0,chardev=spdk_vhost_scsi0,num_queues=4

However you still don't have a correct SPDK volume by far. Because most of
the images you could download from the Internet are built on traditional
BIOS boot-loader, thus cannot be directly copied. What I did is having the
guest OS reinstalled from its attaching live CD-ROM.

Regarding to the import/export functionality, I think another app is needed
to read/write lvol while interacting with vhost app simultaneously. Like
qemu itself a little bit.
Perhaps someone could integrate it into the qemu-img command?


>
> Thanks,
>
> Chen
>
>
>
> *From:* SPDK [mailto:spdk-bounces(a)lists.01.org] *On Behalf Of *Harris,
> James R
> *Sent:* Wednesday, June 6, 2018 2:14 AM
> *To:* Bob Chen <a175818323(a)gmail.com>; Storage Performance Development
> Kit <spdk(a)lists.01.org>; Cao, Gang <gang.cao(a)intel.com>; qemu block <
> qemu-block(a)nongnu.org>
> *Subject:* Re: [SPDK] How to setup a QEMU system disk by using SPDK?
>
>
>
>
>
>
>
> *From: *Bob Chen <a175818323(a)gmail.com>
> *Date: *Monday, June 4, 2018 at 10:47 PM
> *To: *Storage Performance Development Kit <spdk(a)lists.01.org>, "Cao,
> Gang" <gang.cao(a)intel.com>, James Harris <james.r.harris(a)intel.com>, qemu
> block <qemu-block(a)nongnu.org>
> *Subject: *Re: [SPDK] How to setup a QEMU system disk by using SPDK?
>
>
>
>
>
> […]
>
>
>
>
> I suspect this is because all SPDK logical volumes are exposed with 4KB
> logical block size.  IIRC, Seabios legacy boot doesn’t support 4KB logical
> block devices.  UEFI boot should work but we don’t have automated tests set
> up for that yet.
>
>
>
> I will have a try for UEFI boot-loader.
>
>
>
>
> I think SPDK logical volumes could be exposed with 512 byte logical block
> size, if the underlying block device has a 512 byte logical block size.
> Using 4KB LBA simplified the implementation a bit and all metadata has to
> use 4KB for atomicity.  But talking with Ben we don’t see any reason 512
> byte for read/write/unmap operations wouldn’t work.  We may have to use
> feature bits though so that this change in behavior only applies to newly
> created logical volumes.
>
> Bob – I’m curious if you can boot your VM with the SPDK logical volume as
> a secondary disk?
>
>
>
>  Yes, we could. Although we were able to see the device in the VM, that
> disk with the guest OS inside still couldn't be mounted. Seemed like
> read-only or corrupted.
>
>
>
> Because SPDK by far does not provide a method to import or export volume
> metadata to a external file, we came up with this thought and managed to
> copy data by `dd` and nbd. Not very straight forward...
>
>
>
>
>
> Good news, UEFI works! Now Qemu can boot on the SPDK system disk
> successfully.
>
>
>
> What we are expecting next is a more portable way to import/export volume.
>
>
>
>
>
> Thanks for the update!  Would you be willing to add a section to the SPDK
> vhost doc (doc/vhost.md) with some tips for QEMU UEFI boot?
>
>
>
> Regarding import/export – what kind of method do you have in mind?  I
> agree that a more straight forward method is warranted.
>
>
>
> -Jim
>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 11480 bytes --]

Re: [SPDK] How to setup a QEMU system disk by using SPDK?

[Wang, ChenX]

[-- Attachment #1: Type: text/plain, Size: 2968 bytes --]

Hi,  Bob

I have try to boot the os by QEMU UEFI, but failed, following is my qemu command:
. /qemu-system-x86_64 -smbios type=0,uefi=on -smp 8 -m 1024 -net user,hostfwd=tcp::10020-:22 \
-net nic -chardev socket,id=char0,path=vhost.0 -device vhost-user-blk-pci,chardev=char0,num-queues=4,bootindex=0  --enable-kvm

Would you be willing to share your commands and some tips for QEMU UEFI? Thanks so much. ☺

Thanks,
Chen

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Harris, James R
Sent: Wednesday, June 6, 2018 2:14 AM
To: Bob Chen <a175818323(a)gmail.com>; Storage Performance Development Kit <spdk(a)lists.01.org>; Cao, Gang <gang.cao(a)intel.com>; qemu block <qemu-block(a)nongnu.org>
Subject: Re: [SPDK] How to setup a QEMU system disk by using SPDK?



From: Bob Chen <a175818323(a)gmail.com<mailto:a175818323(a)gmail.com>>
Date: Monday, June 4, 2018 at 10:47 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org<mailto:spdk(a)lists.01.org>>, "Cao, Gang" <gang.cao(a)intel.com<mailto:gang.cao(a)intel.com>>, James Harris <james.r.harris(a)intel.com<mailto:james.r.harris(a)intel.com>>, qemu block <qemu-block(a)nongnu.org<mailto:qemu-block(a)nongnu.org>>
Subject: Re: [SPDK] How to setup a QEMU system disk by using SPDK?


[…]


I suspect this is because all SPDK logical volumes are exposed with 4KB logical block size.  IIRC, Seabios legacy boot doesn’t support 4KB logical block devices.  UEFI boot should work but we don’t have automated tests set up for that yet.

I will have a try for UEFI boot-loader.


I think SPDK logical volumes could be exposed with 512 byte logical block size, if the underlying block device has a 512 byte logical block size.  Using 4KB LBA simplified the implementation a bit and all metadata has to use 4KB for atomicity.  But talking with Ben we don’t see any reason 512 byte for read/write/unmap operations wouldn’t work.  We may have to use feature bits though so that this change in behavior only applies to newly created logical volumes.

Bob – I’m curious if you can boot your VM with the SPDK logical volume as a secondary disk?

 Yes, we could. Although we were able to see the device in the VM, that disk with the guest OS inside still couldn't be mounted. Seemed like read-only or corrupted.

Because SPDK by far does not provide a method to import or export volume metadata to a external file, we came up with this thought and managed to copy data by `dd` and nbd. Not very straight forward...


Good news, UEFI works! Now Qemu can boot on the SPDK system disk successfully.

What we are expecting next is a more portable way to import/export volume.


Thanks for the update!  Would you be willing to add a section to the SPDK vhost doc (doc/vhost.md) with some tips for QEMU UEFI boot?

Regarding import/export – what kind of method do you have in mind?  I agree that a more straight forward method is warranted.

-Jim

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 10613 bytes --]

Re: [SPDK] How to setup a QEMU system disk by using SPDK?

[Harris, James R]

[-- Attachment #1: Type: text/plain, Size: 2022 bytes --]



From: Bob Chen <a175818323(a)gmail.com>
Date: Monday, June 4, 2018 at 10:47 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>, "Cao, Gang" <gang.cao(a)intel.com>, James Harris <james.r.harris(a)intel.com>, qemu block <qemu-block(a)nongnu.org>
Subject: Re: [SPDK] How to setup a QEMU system disk by using SPDK?


[…]


I suspect this is because all SPDK logical volumes are exposed with 4KB logical block size.  IIRC, Seabios legacy boot doesn’t support 4KB logical block devices.  UEFI boot should work but we don’t have automated tests set up for that yet.

I will have a try for UEFI boot-loader.


I think SPDK logical volumes could be exposed with 512 byte logical block size, if the underlying block device has a 512 byte logical block size.  Using 4KB LBA simplified the implementation a bit and all metadata has to use 4KB for atomicity.  But talking with Ben we don’t see any reason 512 byte for read/write/unmap operations wouldn’t work.  We may have to use feature bits though so that this change in behavior only applies to newly created logical volumes.

Bob – I’m curious if you can boot your VM with the SPDK logical volume as a secondary disk?

 Yes, we could. Although we were able to see the device in the VM, that disk with the guest OS inside still couldn't be mounted. Seemed like read-only or corrupted.

Because SPDK by far does not provide a method to import or export volume metadata to a external file, we came up with this thought and managed to copy data by `dd` and nbd. Not very straight forward...


Good news, UEFI works! Now Qemu can boot on the SPDK system disk successfully.

What we are expecting next is a more portable way to import/export volume.


Thanks for the update!  Would you be willing to add a section to the SPDK vhost doc (doc/vhost.md) with some tips for QEMU UEFI boot?

Regarding import/export – what kind of method do you have in mind?  I agree that a more straight forward method is warranted.

-Jim

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 6876 bytes --]

Re: [SPDK] How to setup a QEMU system disk by using SPDK?

[Bob Chen]

[-- Attachment #1: Type: text/plain, Size: 2560 bytes --]

2018-06-05 11:47 GMT+08:00 Bob Chen <a175818323(a)gmail.com>:

>
>
> 2018-06-05 2:58 GMT+08:00 Harris, James R <james.r.harris(a)intel.com>:
>
>>
>>
>> On 6/4/18, 5:52 AM, "SPDK on behalf of Stefan Hajnoczi" <
>> spdk-bounces(a)lists.01.org on behalf of stefanha(a)gmail.com> wrote:
>>
>>     On Mon, Jun 04, 2018 at 03:46:16PM +0800, Bob Chen wrote:
>>     > Hi, guys
>>     >
>>     > I have setup a SPDK lvol as a data disk, and run QEMU with it, as
>> per the
>>     > vhost documents instructed.
>>     >
>>     > Then I exposed a empty lvol to host nbd, and used `dd` to copy a
>> existing
>>     > raw disk into lvol, which contains a guest OS. But the guest was
>> not able
>>     > to boot.
>>     >
>>     > Was there any step that I missed?
>>
>>     Please post your QEMU command-line, spdk command-line, and the error
>>   message that you got.
>>
>> I suspect this is because all SPDK logical volumes are exposed with 4KB
>> logical block size.  IIRC, Seabios legacy boot doesn’t support 4KB logical
>> block devices.  UEFI boot should work but we don’t have automated tests set
>> up for that yet.
>>
>
> I will have a try for UEFI boot-loader.
>

>
>>
>> I think SPDK logical volumes could be exposed with 512 byte logical block
>> size, if the underlying block device has a 512 byte logical block size.
>> Using 4KB LBA simplified the implementation a bit and all metadata has to
>> use 4KB for atomicity.  But talking with Ben we don’t see any reason 512
>> byte for read/write/unmap operations wouldn’t work.  We may have to use
>> feature bits though so that this change in behavior only applies to newly
>> created logical volumes.
>>
>> Bob – I’m curious if you can boot your VM with the SPDK logical volume as
>> a secondary disk?
>
>
>  Yes, we could. Although we were able to see the device in the VM, that
> disk with the guest OS inside still couldn't be mounted. Seemed like
> read-only or corrupted.
>
> Because SPDK by far does not provide a method to import or export volume
> metadata to a external file, we came up with this thought and managed to
> copy data by `dd` and nbd. Not very straight forward...
>


Good news, UEFI works! Now Qemu can boot on the SPDK system disk
successfully.

What we are expecting next is a more portable way to import/export volume.



>
>
>> -Jim
>>
>>
>> _______________________________________________
>> SPDK mailing list
>> SPDK(a)lists.01.org
>> https://lists.01.org/mailman/listinfo/spdk
>>
>
>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 5746 bytes --]

Re: [SPDK] How to setup a QEMU system disk by using SPDK?

[Bob Chen]

[-- Attachment #1: Type: text/plain, Size: 2237 bytes --]

2018-06-05 2:58 GMT+08:00 Harris, James R <james.r.harris(a)intel.com>:

>
>
> On 6/4/18, 5:52 AM, "SPDK on behalf of Stefan Hajnoczi" <
> spdk-bounces(a)lists.01.org on behalf of stefanha(a)gmail.com> wrote:
>
>     On Mon, Jun 04, 2018 at 03:46:16PM +0800, Bob Chen wrote:
>     > Hi, guys
>     >
>     > I have setup a SPDK lvol as a data disk, and run QEMU with it, as
> per the
>     > vhost documents instructed.
>     >
>     > Then I exposed a empty lvol to host nbd, and used `dd` to copy a
> existing
>     > raw disk into lvol, which contains a guest OS. But the guest was not
> able
>     > to boot.
>     >
>     > Was there any step that I missed?
>
>     Please post your QEMU command-line, spdk command-line, and the error
>   message that you got.
>
> I suspect this is because all SPDK logical volumes are exposed with 4KB
> logical block size.  IIRC, Seabios legacy boot doesn’t support 4KB logical
> block devices.  UEFI boot should work but we don’t have automated tests set
> up for that yet.
>

I will have a try for UEFI boot-loader.


>
> I think SPDK logical volumes could be exposed with 512 byte logical block
> size, if the underlying block device has a 512 byte logical block size.
> Using 4KB LBA simplified the implementation a bit and all metadata has to
> use 4KB for atomicity.  But talking with Ben we don’t see any reason 512
> byte for read/write/unmap operations wouldn’t work.  We may have to use
> feature bits though so that this change in behavior only applies to newly
> created logical volumes.
>
> Bob – I’m curious if you can boot your VM with the SPDK logical volume as
> a secondary disk?


 Yes, we could. Although we were able to see the device in the VM, that
disk with the guest OS inside still couldn't be mounted. Seemed like
read-only or corrupted.

Because SPDK by far does not provide a method to import or export volume
metadata to a external file, we came up with this thought and managed to
copy data by `dd` and nbd. Not very straight forward...


> -Jim
>
>
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk
>

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 3167 bytes --]

Re: [SPDK] How to setup a QEMU system disk by using SPDK?

[Harris, James R]

[-- Attachment #1: Type: text/plain, Size: 1508 bytes --]



On 6/4/18, 5:52 AM, "SPDK on behalf of Stefan Hajnoczi" <spdk-bounces(a)lists.01.org on behalf of stefanha(a)gmail.com> wrote:

    On Mon, Jun 04, 2018 at 03:46:16PM +0800, Bob Chen wrote:
    > Hi, guys
    > 
    > I have setup a SPDK lvol as a data disk, and run QEMU with it, as per the
    > vhost documents instructed.
    > 
    > Then I exposed a empty lvol to host nbd, and used `dd` to copy a existing
    > raw disk into lvol, which contains a guest OS. But the guest was not able
    > to boot.
    > 
    > Was there any step that I missed?
    
    Please post your QEMU command-line, spdk command-line, and the error
  message that you got.

I suspect this is because all SPDK logical volumes are exposed with 4KB logical block size.  IIRC, Seabios legacy boot doesn’t support 4KB logical block devices.  UEFI boot should work but we don’t have automated tests set up for that yet.

I think SPDK logical volumes could be exposed with 512 byte logical block size, if the underlying block device has a 512 byte logical block size.  Using 4KB LBA simplified the implementation a bit and all metadata has to use 4KB for atomicity.  But talking with Ben we don’t see any reason 512 byte for read/write/unmap operations wouldn’t work.  We may have to use feature bits though so that this change in behavior only applies to newly created logical volumes. 

Bob – I’m curious if you can boot your VM with the SPDK logical volume as a secondary disk?

-Jim

Re: [SPDK] How to setup a QEMU system disk by using SPDK?

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 499 bytes --]

On Mon, Jun 04, 2018 at 03:46:16PM +0800, Bob Chen wrote:
> Hi, guys
> 
> I have setup a SPDK lvol as a data disk, and run QEMU with it, as per the
> vhost documents instructed.
> 
> Then I exposed a empty lvol to host nbd, and used `dd` to copy a existing
> raw disk into lvol, which contains a guest OS. But the guest was not able
> to boot.
> 
> Was there any step that I missed?

Please post your QEMU command-line, spdk command-line, and the error
message that you got.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [SPDK] How to setup a QEMU system disk by using SPDK?

[Cao, Gang]

[-- Attachment #1: Type: text/plain, Size: 825 bytes --]

Hi Bob,

Currently we used SPDK lvol as the data store in our testing. And have no try on this to make it the image bootable disk. May have some additional work to do support this functionality here.

Thanks,
Gang

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Bob Chen
Sent: Monday, June 4, 2018 3:46 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>; qemu block <qemu-block(a)nongnu.org>
Subject: [SPDK] How to setup a QEMU system disk by using SPDK?

Hi, guys

I have setup a SPDK lvol as a data disk, and run QEMU with it, as per the vhost documents instructed.

Then I exposed a empty lvol to host nbd, and used `dd` to copy a existing raw disk into lvol, which contains a guest OS. But the guest was not able to boot.

Was there any step that I missed?


Thanks, Bob

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 4808 bytes --]

[SPDK] How to setup a QEMU system disk by using SPDK?

[Bob Chen]

[-- Attachment #1: Type: text/plain, Size: 332 bytes --]

Hi, guys

I have setup a SPDK lvol as a data disk, and run QEMU with it, as per the
vhost documents instructed.

Then I exposed a empty lvol to host nbd, and used `dd` to copy a existing
raw disk into lvol, which contains a guest OS. But the guest was not able
to boot.

Was there any step that I missed?


Thanks, Bob

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 452 bytes --]