From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:56962) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UhL8b-0007gm-B7 for qemu-devel@nongnu.org; Tue, 28 May 2013 10:44:42 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1UhL8T-00087v-7O for qemu-devel@nongnu.org; Tue, 28 May 2013 10:44:33 -0400 Received: from aserp1040.oracle.com ([141.146.126.69]:37217) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1UhL8S-00087j-Vf for qemu-devel@nongnu.org; Tue, 28 May 2013 10:44:25 -0400 Date: Tue, 28 May 2013 10:44:10 -0400 From: Konrad Rzeszutek Wilk Message-ID: <20130528144410.GP724@phenom.dumpdata.com> References: <33183CC9F5247A488A2544077AF190206C8FCB9B@szxeml538-mbx.china.huawei.com> <33183CC9F5247A488A2544077AF190206C8FFCF4@szxeml538-mbx.china.huawei.com> <20130524122001.GC23160@stefanha-thinkpad.redhat.com> <33183CC9F5247A488A2544077AF190206C900F86@szxeml538-mbx.china.huawei.com> <20130527124419.GE23204@stefanha-thinkpad.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20130527124419.GE23204@stefanha-thinkpad.redhat.com> Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [Xen-devel] use O_DIRECT to open disk images for IDE failed under xen-4.1.2 and qemu upstream List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Stefan Hajnoczi Cc: Stefano Stabellini , Hanweidong , Luonengjun , "qemu-devel@nongnu.org" , Wangzhenguo , "Gonglei (Arei)" , "xen-devel@lists.xen.org" , "Huangweidong (Hardware)" On Mon, May 27, 2013 at 02:44:19PM +0200, Stefan Hajnoczi wrote: > On Sat, May 25, 2013 at 07:32:55AM +0000, Gonglei (Arei) wrote: > >=20 > >=20 > > > -----Original Message----- > > > From: Stefan Hajnoczi [mailto:stefanha@gmail.com] > > > Sent: Friday, May 24, 2013 8:20 PM > > > To: Gonglei (Arei) > > > Cc: Stefano Stabellini; Hanweidong; Luonengjun; qemu-devel@nongnu.o= rg; > > > Wangzhenguo; xen-devel@lists.xen.org; Huangweidong (Hardware) > > > Subject: Re: [Qemu-devel] use O_DIRECT to open disk images for IDE = failed > > > under xen-4.1.2 and qemu upstream > > >=20 > > > On Fri, May 24, 2013 at 02:59:05AM +0000, Gonglei (Arei) wrote: > > > > Hi, > > > > > > > > > > > > > > On Thu, 23 May 2013, Gonglei (Arei) wrote: > > > > > > Hi, all > > > > > > > > > > > > I use O_DIRECT to open disk images for IDE, but I'm failed. A= fter debug, I > > > get > > > > > the below logs: > > > > > > [2013-05-22 23:25:46] ide: CMD=3Dc8 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x00 : 0x08 > > > > > > [2013-05-22 23:25:46] bmdma: writeb 0x00 : 0x09 > > > > > > [2013-05-22 23:25:46] bmdma_cmd_writeb: 0x00000009 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:25:46] bmdma: readb 0x02 : 0x01 > > > > > > [2013-05-22 23:26:39] bmdma: writeb 0x00 : 0x08 > > > > > > [2013-05-22 23:26:39] bmdma_cmd_writeb: 0x00000008 > > > > > > [2013-05-22 23:26:56] =3D=3D=3D=3D=3D=3D offset:0 buf:0x7ff10= 0f21c00 count:512 > > > > > aio_offset:0 > > > > > > [2013-05-22 23:31:30] =3D=3D=3D=3D=3D=3D offset:0 buf:0x7ff10= 0f21c00 count:512 > > > > > aio_offset:0 > > > > > > [2013-05-22 23:31:30] =3D=3D=3D=3D=3D=3D handle_aiocb_rw_line= ar errno: -14 > > > > > > [2013-05-22 23:31:30] =3D=3D=3D=3D=3D=3D paio_complete errno=3D= 14 > > > > > > [2013-05-22 23:31:30] =3D=3D=3D=3D=3D=3D ide_dma_error!!! > > > > > > [2013-05-22 23:31:30] ide: read status addr=3D0x3f6 val=3D41 > > > > > > > > > > > > QEMU command line : > > > > > > qemu-system-i386 -xen-domid 837 -chardev > > > > > socket,id=3Dlibxl-cmd,path=3D/var/run/xen/qmp-libxl-837,server,= nowait -mon > > > > > chardev=3Dlibxl-cmd,mode=3Dcontrol -name suse11 -vnc 0.0.0.0:1 = -serial pty > > > -boot > > > > > order=3Dc -usb -usbdevice tablet -smp 2,maxcpus=3D2 -device > > > > > rtl8139,id=3Dnic0,netdev=3Dnet0,mac=3D00:16:3e:13:d3:72 -netdev > > > > > type=3Dtap,id=3Dnet0,ifname=3Dtap837.0,downscript=3Dno -M xenfv= -m 2040 -drive > > > > > > > > file=3D/mnt/sdd/image/suse.image,if=3Dide,index=3D0,media=3Ddisk,fo= rmat=3Draw,cache > > > > > =3Dnone > > > > > > > > > > > > errno 14 shows Bad Address. And I find QEMU_AIO_MISALIGNED fl= ag bit > > > is > > > > > not set through debug. > > > > > > > > > > > > /* > > > > > > * If O_DIRECT is used the buffer needs to be aligned on = a sector > > > > > > * boundary. Check if this is the case or tell the low-l= evel > > > > > > * driver that it needs to copy the buffer. > > > > > > */ > > > > > > if ((bs->open_flags & BDRV_O_NOCACHE)) { > > > > > > if (!bdrv_qiov_is_aligned(bs, qiov)) { //if the ad= dress is > > > > > aligned-512, will no meet the conditions > > > > > > type |=3D QEMU_AIO_MISALIGNED; > > > > > > #ifdef CONFIG_LINUX_AIO > > > > > > } else if (s->use_aio) { > > > > > > return laio_submit(bs, s->aio_ctx, s->fd, sector_= num, qiov, > > > > > > nb_sectors, cb, opaque, type); > > > > > > #endif > > > > > > > > > > > > Next process: > > > > > > static ssize_t handle_aiocb_rw(struct qemu_paiocb *aiocb) > > > > > > { > > > > > > ssize_t nbytes; > > > > > > char *buf; > > > > > > > > > > > > if (!(aiocb->aio_type & QEMU_AIO_MISALIGNED)) { > > > > > > /* > > > > > > * If there is just a single buffer, and it is proper= ly aligned > > > > > > * we can just use plain pread/pwrite without any pro= blems. > > > > > > */ > > > > > > if (aiocb->aio_niov =3D=3D 1) > > > > > > return handle_aiocb_rw_linear(aiocb, > > > > > aiocb->aio_iov->iov_base); //this way, and reports errno 14 nex= t > > > > > > > > > > > > Anyone have a good method to resolve this bug? Thanks! > > > > > > > > > > I know that this is not the answer you are looking for but why = do you > > > > > want to use O_DIRECT with IDE? > > > > > It should be perfectly safe to use write-back. > > > > > > > > A few days ago, I asked a question about the IDE FLUSH time of gu= est=EF=BC=9A > > > > http://lists.nongnu.org/archive/html/qemu-devel/2013-05/msg02642.= html > > > > > > > > finally I found that because Qemu use write-back flag to open dis= k images by > > > default. > > > > so I hope to use O_DIRECT to avoid meeting that problem, but I'm = failed > > > under Xen platform with Qemu upstream. > > >=20 > > > cache=3Dnone does not eliminate the need for flush. If you want to= do > > > that, try cache=3Dwritethrough or cache=3Ddirectsync. > > >=20 > > I had tried it, but I got the same error. >=20 > Perhaps that's because the busy bit also needs to be set correctly for > other IDE commands (like read/write requests). >=20 > > > Regarding the EFAULT you are seeing, did you check if the I/O buffe= r > > > address is valid? Try breaking in gdb and inspecting /proc//m= aps > > > or just x in gdb to see if the buffer is readable. > > >=20 > > > Stefan > >=20 > > Hi, Stefan > >=20 > > The follows are details in gdb: > > handle_aiocb_rw_linear (aiocb=3D0x13a0f80, buf=3D0x7f0a637d7c00 ) > > at /mnt/sdd/gonglei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:2= 13 > > 213 { > > (gdb) bt > > #0 handle_aiocb_rw_linear (aiocb=3D0x13a0f80, buf=3D0x7f0a637d7c00 <= Address 0x7f0a637d7c00 out of bounds>) > > at /mnt/sdd/gonglei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:2= 13 > > #1 0x000000000058ba4f in handle_aiocb_rw (aiocb=3D0x13a0f80) at /mnt= /sdd/gonglei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:254 > > #2 0x000000000058bde2 in aio_thread (unused=3D0x0) at /mnt/sdd/gongl= ei/xen-4.1.2/tools/qemu-xen/posix-aio-compat.c:351 > > #3 0x00007f0a6a1ec7b6 in start_thread () from /lib64/libpthread.so.0 > > #4 0x00007f0a68f0e9cd in clone () from /lib64/libc.so.6 > > #5 0x0000000000000000 in ?? () > > (gdb) x 0x7f0a637d7c00 > > 0x7f0a637d7c00: Cannot access memory at address 0x7f0a637d7c00 > > (gdb) n > > 214 ssize_t offset =3D 0; > > (gdb)=20 > > 217 while (offset < aiocb->aio_nbytes) { > > (gdb) n > > 218 if (aiocb->aio_type & QEMU_AIO_WRITE) > > (gdb)=20 > > 224 len =3D pread(aiocb->aio_fildes, > > (gdb)=20 > > 229 if (len =3D=3D -1 && errno =3D=3D EINTR) > > (gdb)=20 > > 231 else if (len =3D=3D -1) { > > (gdb)=20 > > 232 offset =3D -errno; > > (gdb) p errno > > $1 =3D 14 > >=20 > > linux-CGRmYS:~ # cat /proc/21837/maps=20 > > 00400000-00846000 r-xp 00000000 08:05 28601 = /usr/lib/xen/bin/qemu-system-i386 > > 00a45000-00a46000 r--p 00445000 08:05 28601 = /usr/lib/xen/bin/qemu-system-i386 > > 00a46000-00a92000 rw-p 00446000 08:05 28601 = /usr/lib/xen/bin/qemu-system-i386 > > 00a92000-01344000 rw-p 00000000 00:00 0 = [heap] > > 01344000-01345000 rw-p 00000000 00:00 0 = [heap] > > 01345000-01393000 rw-p 00000000 00:00 0 = [heap] > > 01393000-01394000 rw-p 00000000 00:00 0 = [heap] > > 01394000-013ef000 rw-p 00000000 00:00 0 = [heap] > > 7f0a636d0000-7f0a637d0000 rw-s 00000000 00:03 4026532176 = /proc/xen/privcmd > > 7f0a637d0000-7f0a638d0000 rw-s 00000000 00:03 4026532176 = /proc/xen/privcmd >=20 > Interesting, the memory is mapped read/write here. But it's shared > memory that belongs to /proc/xen/privcmd and maybe doesn't support > O_DIRECT? >=20 > I don't know the Xen side of things but why are we doing I/O to > /proc/xen/privcmd? ioctls are the only things one should be doing on that file.