Re: xm/xl block-detach issue

["Sébastien Riccio" <sr@swisscenter.com>]

On 09.07.2011 16:34, Sébastien Riccio wrote:
> Hi dear list,
>
> I'm using xen 4.1.1 (built from tar.gz) with 2.6.39.2 kernel + blkback
> module and I'm getting into an issue trying to detach a block device
> from a domain.
>
> What I did:
>
> root@mybox:~# vhd-util create -s 50000 -n /cloud/data2/machines/vm1.vhd
> root@mybox:~# xl block-attach 0 tap:vhd:/cloud/local/vm1.vhd xvda rw 0
>
> everything fine, xvda shows up in the dom0 and I can access it.
>
> Now i want to detach it to reattach it to another domain:
>
> root@mybox:~# xl block-list 0
> Vdev  BE  handle state evt-ch ring-ref BE-path
> 51728 0   0      4     99     8        
> /local/domain/0/backend/vbd/0/51728
>
> root@mybox:~# xl block-detach 0 51728
> Error: Device 51728 not connected
> Usage: xm block-detach <Domain> <DevId> [-f|--force]
>
> Destroy a domain's virtual block device.
>
> game over :(
>
> Any idea what could be wrong ? Bug ?
>
> Thanks in advance for your advices.
>
> -- 
> Sébastien
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

Additions to my previous post:

As I can't detach the block device, the only way (i found) to get rid of 
it is
to reboot the xen host, but it's not acceptable for a near production
system.

I've tried other ways to do what I wished to do (mount a vhd file as a block
device in dom0 in order to be able to fdisk, format it and inject files 
in it.

Try no 2)

The readme of blktap2 that is shipped with xen 4.1.1 tar.gz says:

-----8<--------------------------------------8<---------------------------------
Mounting images in Dom0 using the blktap2 driver
===============================================
Tap (and blkback) disks are also mountable in Dom0 without requiring an
active VM to attach.

The syntax is -
   tapdisk2 -n <type>:<full path to file>

For example -
   tapdisk2  -n aio:/home/images/rawFile.img

When successful the location of the new device will be provided by
tapdisk2 to stdout and tapdisk2 will terminate.  From that point
forward control of the device is provided through sysfs in the
directory-

   /sys/class/blktap2/blktap#/

Where # is a blktap2 device number present in the path that tapdisk2
printed before terminating.  The sysfs interface is largely intuitive,
for example, to remove tap device 0 one would-

   echo 1 > /sys/class/blktap2/blktap0/remove
-----8<--------------------------------------8<---------------------------------

So let's try:

mybox# tapdisk2 -n vhd:/cloud/data2/machines/vm1.vhd
tapdisk2: invalid option -- 'n'
usage: tapdisk2 <-u uuid> <-c control socket>

Huh ? Seems it's no way either that path. Is it a bug or the README
is an outdated one ?

I've googled around trying to find a solution and it found out about 
tap-ctl,
with examples i found here:

http://permalink.gmane.org/gmane.comp.emulators.xen.devel/84223

Okay it's not as easy as a one-line command that the README shows about 
tapdisk2,
but I have a find a way, so let's go :)

mybox# tap-ctl allocate
mybox# tap-ctl spawn
mybox# tap-ctl list
mybox# tap-ctl attach -p 2546 -m 0
mybox# tap-ctl list
mybox# tap-ctl open -p 2546 -m -o -a vhd:/cloud/data2/machines/vm1.vhd

Ok seems I can now fdisk the device

mybox# fdisk /dev/xen/blktap-2/tapdev0

I've setup partition 1 and partition 2 with the scheme I wanted, wrote the
changes to it. Results:

Disk /dev/xen/blktap-2/tapdev0: 52.4 GB, 52428800000 bytes
255 heads, 63 sectors/track, 6374 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x440cbefb

                      Device Boot      Start         End      Blocks   
Id  System
/dev/xen/blktap-2/tapdev0p1               1        6267    50339646   
83  Linux
/dev/xen/blktap-2/tapdev0p2            6268        6374      859477+  
82  Linux swap / Solaris

Good.

Now let's format /dev/xen/blktap-2/tapdev0p1 and mount it so I can 
inject the files in the partition:

mybox# mkfs.ext3 /dev/xen/blktap-2/tapdev0p1
mke2fs 1.41.12 (17-May-2010)
Could not stat /dev/xen/blktap-2/tapdev0p1 --- No such file or directory

The device apparently does not exist; did you specify it correctly?

Hmmm, seems that /dev/xen/blktap-2/tapdev0p1 does not exists.

mybox# ls -la /dev/xen/blktap-2/
total 0
drwxr-xr-x 2 root root     100 Jul  9 17:32 .
drwxr-xr-x 3 root root      80 Jul  9 17:17 ..
crw------- 1 root root 252,  0 Jul  9 17:32 blktap0
crw------- 1 root root  10, 62 Jul  9 17:17 control
brw------- 1 root root 254,  0 Jul  9 18:19 tapdev0

Indeed. Does not exists. Now I remember fdisk told be that it wasn't able to
make the kernel to be aware of the changes until I reboot or I use partprobe
or kpartx.

Let's go:

mybox# partprobe /dev/xen/blktap-2/tapdev0
Error: Error informing the kernel about modifications to partition 
/dev/xen/blktap-2/tapdev0p1 -- Invalid argument.  This means Linux won't 
know about any changes you made to /dev/xen/blktap-2/tapdev0p1 until you 
reboot -- so you shouldn't mount it or use it in any way before rebooting.
Error: Failed to add partition 1 (Invalid argument)

hmm nope, let's go with partx :

mybox# partx -a /dev/xen/blktap-2/tapdev0
BLKPG: Invalid argument
error adding partition 1
BLKPG: Invalid argument
error adding partition 2
BLKPG: Invalid argument
error adding partition 3
BLKPG: Invalid argument
error adding partition 4

Neither with partx and kpartx. I'm stuck again :(

Of course it's not an acceptable solution if I have to reboot the whole 
xen host in order to be
able to access the partitions. (and I didn't even tried if it would 
work, as the main goal was
to get around the xm block-detach problem I had at first).

I've no more ideas about what to try, any ideas, hints, advices ?

Thanks a lot,
Sébastien