[linux-lvm] why 'lvcreate' command blocked?

[linux-lvm] why 'lvcreate' command blocked?

[Busby.Cheung]

[-- Attachment #1: Type: text/plain, Size: 1525 bytes --]

Hi ALL，
       When I used the lvcreate cmd in my system, it blocked for a long time. My OS kernel version is linux-2.6.29, is there any bug?
       Here is the mesg:
INFO: task lvcreate:30225 blocked for more than 480 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
lvcreate      D ffff880028088780     0 30225  25700
 ffff88013e02a100 0000000000000086 0000000000000000 0000000000000000
 00000000000000d0 ffff880124879910 ffff880124879c70 00000001000aaa4a
 ffffffffffffffff ffffffffffffffff ffffffffffffffff 7fffffffffffffff
Call Trace:
 [<ffffffff804d4d5d>] io_schedule+0x5d/0x9e
 [<ffffffff802bfff5>] __blockdev_direct_IO+0x8a4/0xa3b
 [<ffffffff80272842>] __filemap_fdatawrite_range+0x54/0x5e
 [<ffffffff802bee5e>] blkdev_direct_IO+0x45/0x4a
 [<ffffffff802bed84>] blkdev_get_blocks+0x0/0x95
 [<ffffffff80272c28>] generic_file_direct_write+0xd2/0x18b
 [<ffffffff80272f65>] __generic_file_aio_write_nolock+0x284/0x3a5
 [<ffffffff802734d7>] generic_file_aio_read+0xe1/0x539
 [<ffffffff80273138>] generic_file_aio_write_nolock+0x34/0x80
 [<ffffffff8029d6db>] do_sync_write+0xc9/0x10c
 [<ffffffff803958e4>] tty_put_char+0x2b/0x2d
 [<ffffffff80249396>] autoremove_wake_function+0x0/0x2e
 [<ffffffff80326a0c>] selinux_file_permission+0x5a/0x120
 [<ffffffff8029ddd7>] vfs_write+0xad/0x136
 [<ffffffff8029e2f4>] sys_write+0x45/0x6e
 [<ffffffff8020bedb>] system_call_fastpath+0x16/0x1b

   I don't know whether it is the kernel's scheduling bug or the lvm2's bug, anyone knows?
 
Best Wishes!


 

[-- Attachment #2: Type: text/html, Size: 2149 bytes --]

Re: [linux-lvm] why 'lvcreate' command blocked?

[Stuart D. Gathman]

On Fri, 1 Jul 2011, Busby.Cheung wrote:

>        When I used the lvcreate cmd in my system, it blocked for a long
> time. My OS kernel version is linux-2.6.29, is there any bug?

There is a bug somewhere.  Your block device is hanging.  You don't
mention what kind of block device and what driver are used for your
PVs.  It would only be an LVM problem if the block device was itself
an LVM device. :-)

For instance, if your block device is iSCSI, and the SAN server is
down, or your block device is DRBD and the link is slow (has
happened to me), or your block device is a disk that is frantically
playing with bad sectors (has happened to me also), then LVM
would "hang".

--
 	      Stuart D. Gathman <stuart@bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Re: [linux-lvm] why 'lvcreate' command blocked?

[Ray Morris]

Please let me know if you figure it out. I have a server
with a similar problem which persists even after changing 
virtually every component of the system. One hint I learned
is that it can at times hang for serveral hours and then 
proceed. Just because it's been several minutes does not 
necesarily mean a 180 mile trip to the datacenter.
-- 
Ray Morris
support@bettercgi.com

Strongbox - The next generation in site security:
http://www.bettercgi.com/strongbox/

Throttlebox - Intelligent Bandwidth Control
http://www.bettercgi.com/throttlebox/

Strongbox / Throttlebox affiliate program:
http://www.bettercgi.com/affiliates/user/register.php




On Fri, 1 Jul 2011 14:49:09 +0800 (CST)
"Busby.Cheung" <chaimvy@163.com> wrote:

> Hi ALL，
>        When I used the lvcreate cmd in my system, it blocked for a
> long time. My OS kernel version is linux-2.6.29, is there any bug?
> Here is the mesg: INFO: task lvcreate:30225 blocked for more than 480
> seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
> this message. lvcreate      D ffff880028088780     0 30225  25700
>  ffff88013e02a100 0000000000000086 0000000000000000 0000000000000000
>  00000000000000d0 ffff880124879910 ffff880124879c70 00000001000aaa4a
>  ffffffffffffffff ffffffffffffffff ffffffffffffffff 7fffffffffffffff
> Call Trace:
>  [<ffffffff804d4d5d>] io_schedule+0x5d/0x9e
>  [<ffffffff802bfff5>] __blockdev_direct_IO+0x8a4/0xa3b
>  [<ffffffff80272842>] __filemap_fdatawrite_range+0x54/0x5e
>  [<ffffffff802bee5e>] blkdev_direct_IO+0x45/0x4a
>  [<ffffffff802bed84>] blkdev_get_blocks+0x0/0x95
>  [<ffffffff80272c28>] generic_file_direct_write+0xd2/0x18b
>  [<ffffffff80272f65>] __generic_file_aio_write_nolock+0x284/0x3a5
>  [<ffffffff802734d7>] generic_file_aio_read+0xe1/0x539
>  [<ffffffff80273138>] generic_file_aio_write_nolock+0x34/0x80
>  [<ffffffff8029d6db>] do_sync_write+0xc9/0x10c
>  [<ffffffff803958e4>] tty_put_char+0x2b/0x2d
>  [<ffffffff80249396>] autoremove_wake_function+0x0/0x2e
>  [<ffffffff80326a0c>] selinux_file_permission+0x5a/0x120
>  [<ffffffff8029ddd7>] vfs_write+0xad/0x136
>  [<ffffffff8029e2f4>] sys_write+0x45/0x6e
>  [<ffffffff8020bedb>] system_call_fastpath+0x16/0x1b
> 
>    I don't know whether it is the kernel's scheduling bug or the
> lvm2's bug, anyone knows? 
> Best Wishes!
> 
> 
>  

Re: [linux-lvm] why 'lvcreate' command blocked?

[Busby.Cheung]

[-- Attachment #1: Type: text/plain, Size: 1888 bytes --]

The PV device is a MD device, RAID5 on several SATA disks。
Linux kernel is 2.6.29. 
The SATA disks belong to a JBOD and the JBOD can be accessed by 2 controller(I use some methods to control multiple access). 
The OS on both of the controllers are the same, but one will hang and the other did not, so I think there is not any bad sectors on the disks.
Maybe there is something wrong with the MD (soft raid) driver, I should to check more information.
 
 > -----原始邮件-----
> 发件人: "Stuart D. Gathman" <stuart@bmsi.com>
> 发送时间: 2011年7月2日 星期六
> 收件人: "LVM general discussion and development" <linux-lvm@redhat.com>
> 抄送:
> 主题: Re: [linux-lvm] why 'lvcreate' command blocked?
>
> On Fri, 1 Jul 2011, Busby.Cheung wrote:
>
> >        When I used the lvcreate cmd in my system, it blocked for a long
> > time. My OS kernel version is linux-2.6.29, is there any bug?
>
> There is a bug somewhere.  Your block device is hanging.  You don't
> mention what kind of block device and what driver are used for your
> PVs.  It would only be an LVM problem if the block device was itself
> an LVM device. :-)
>
> For instance, if your block device is iSCSI, and the SAN server is
> down, or your block device is DRBD and the link is slow (has
> happened to me), or your block device is a disk that is frantically
> playing with bad sectors (has happened to me also), then LVM
> would "hang".
>
> --
>        Stuart D. Gathman <stuart@bmsi.com>
>      Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
> "Confutatis maledictis, flammis acribus addictis" - background song for
> a Microsoft sponsored "Where do you want to go from here?" commercial.
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

[-- Attachment #2: Type: text/html, Size: 2427 bytes --]

Re: [linux-lvm] why 'lvcreate' command blocked?

[Busby.Cheung]

[-- Attachment #1: Type: text/plain, Size: 2770 bytes --]

 I haven't figured it out. I should check more information and learn more.
 
 
 > -----原始邮件-----
> 发件人: "Ray Morris" <support@bettercgi.com>
> 发送时间: 2011年7月2日 星期六
> 收件人: "LVM general discussion and development" <linux-lvm@redhat.com>
> 抄送: chaimvy@163.com
> 主题: Re: [linux-lvm] why 'lvcreate' command blocked?
> 
> Please let me know if you figure it out. I have a server
> with a similar problem which persists even after changing 
> virtually every component of the system. One hint I learned
> is that it can at times hang for serveral hours and then 
> proceed. Just because it's been several minutes does not 
> necesarily mean a 180 mile trip to the datacenter.
> -- 
> Ray Morris
> support@bettercgi.com
> 
> Strongbox - The next generation in site security:
> http://www.bettercgi.com/strongbox/
> 
> Throttlebox - Intelligent Bandwidth Control
> http://www.bettercgi.com/throttlebox/
> 
> Strongbox / Throttlebox affiliate program:
> http://www.bettercgi.com/affiliates/user/register.php
> 
> 
> 
> 
> On Fri, 1 Jul 2011 14:49:09 +0800 (CST)
> "Busby.Cheung" <chaimvy@163.com> wrote:
> 
> > Hi ALL，
> >        When I used the lvcreate cmd in my system, it blocked for a
> > long time. My OS kernel version is linux-2.6.29, is there any bug?
> > Here is the mesg: INFO: task lvcreate:30225 blocked for more than 480
> > seconds. "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables
> > this message. lvcreate      D ffff880028088780     0 30225  25700
> >  ffff88013e02a100 0000000000000086 0000000000000000 0000000000000000
> >  00000000000000d0 ffff880124879910 ffff880124879c70 00000001000aaa4a
> >  ffffffffffffffff ffffffffffffffff ffffffffffffffff 7fffffffffffffff
> > Call Trace:
> >  [<ffffffff804d4d5d>] io_schedule+0x5d/0x9e
> >  [<ffffffff802bfff5>] __blockdev_direct_IO+0x8a4/0xa3b
> >  [<ffffffff80272842>] __filemap_fdatawrite_range+0x54/0x5e
> >  [<ffffffff802bee5e>] blkdev_direct_IO+0x45/0x4a
> >  [<ffffffff802bed84>] blkdev_get_blocks+0x0/0x95
> >  [<ffffffff80272c28>] generic_file_direct_write+0xd2/0x18b
> >  [<ffffffff80272f65>] __generic_file_aio_write_nolock+0x284/0x3a5
> >  [<ffffffff802734d7>] generic_file_aio_read+0xe1/0x539
> >  [<ffffffff80273138>] generic_file_aio_write_nolock+0x34/0x80
> >  [<ffffffff8029d6db>] do_sync_write+0xc9/0x10c
> >  [<ffffffff803958e4>] tty_put_char+0x2b/0x2d
> >  [<ffffffff80249396>] autoremove_wake_function+0x0/0x2e
> >  [<ffffffff80326a0c>] selinux_file_permission+0x5a/0x120
> >  [<ffffffff8029ddd7>] vfs_write+0xad/0x136
> >  [<ffffffff8029e2f4>] sys_write+0x45/0x6e
> >  [<ffffffff8020bedb>] system_call_fastpath+0x16/0x1b
> > 
> >    I don't know whether it is the kernel's scheduling bug or the
> > lvm2's bug, anyone knows? 
> > Best Wishes!
> > 
> > 
> >  

[-- Attachment #2: Type: text/html, Size: 3330 bytes --]