All of lore.kernel.org
 help / color / mirror / Atom feed
From: Dan C <dcodix@gmail.com>
To: linux-lvm@redhat.com
Subject: [linux-lvm] lvremove snapshot hangs LVM system
Date: Tue, 3 Apr 2012 17:05:47 +0200	[thread overview]
Message-ID: <CA+8+0ELd53bWR3JZbUvnDiQr0nsYZV1riqWPj9qT0ZXu007Vmg@mail.gmail.com> (raw)


[-- Attachment #1.1: Type: text/plain, Size: 1745 bytes --]

I've read about that in the mailing list, but all the messages were quite
old, and always referencing old kernels, so I decided to post it again.

My system is as follows:
Linux 2.6.32-131.17.1.el6.x86_64 #1 SMP Wed Oct 5 17:19:54 CDT 2011 x86_64
x86_64 x86_64 GNU/Linux

cman-3.0.12-23.el6.x86_64
corosynclib-1.2.3-21.el6.x86_64
corosync-1.2.3-21.el6.x86_64
lvm2-devel-2.02.88-3.el6.x86_64
lvm2-libs-2.02.88-3.el6.x86_64
lvm2-cluster-2.02.88-3.el6.x86_64
lvm2-2.02.88-3.el6.x86_64

I am running LVM as a cluster as:
clvmd -T30

The discs on the LVM groups are all on a SAN connected with FC.

I am working in a two-node cluster.

As I needed to use the snapshot features I have some volumes just active in
one of the nodes. As far as I know this is the only way to snapshot volumes.

Everything works fine. I am able to create volumes. I am able to put
volumes in "exclusive use" and it gives me the capability of snapshot.
Snapshooting works fine.

My problem comes when I "lvremove" a snapshot. It seems to work fine, and
the snapshot is removed, but, whatever LVM command I execute after that
hangs forever in an "uninterruptible sleep" (D).

After that, for a while, I can see some errors on the log(attached
messages.log). After a while those messages stop, but LVM command yet won't
respond.

The rest of the system works perfectly, the only problem is that I cannot
make modifications on LVM system.

The only way to solve the problem is by rebooting the machine.

As a note say this only happens "lvremoving" a snapshot. Removing a volume
works fine.

I don't know if I am doing something wrong, as I read that problem use to
happen with older versions but it was supposedly solved.

Does someone else have this problem?

Thanks a lot.

[-- Attachment #1.2: Type: text/html, Size: 1928 bytes --]

[-- Attachment #2: messages.log --]
[-- Type: application/octet-stream, Size: 8856 bytes --]

Apr  2 10:06:44 s02 lvm[18444]: No longer monitoring snapshot ofx--virtdisks--001-SL6.1--mysql--10G--clone--prod--02--2nd.snap201203060956
Apr  2 10:06:44 s02 lvm[18444]: No longer monitoring snapshot ofx--virtdisks--001-SL6.1--mysql--10G--clone--prod--02--2nd.snap201203191211
Apr  2 10:06:44 s02 lvm[18444]: No longer monitoring snapshot ofx--virtdisks--001-SL6.1--mysql--10G--clone--prod--02--2nd--autosnap--1333353742
Apr  2 10:06:44 s02 lvm[18444]: Monitoring snapshot ofx--virtdisks--001-SL6.1--mysql--10G--clone--prod--02--2nd.snap201203191211
Apr  2 10:06:44 s02 lvm[18444]: Monitoring snapshot ofx--virtdisks--001-SL6.1--mysql--10G--clone--prod--02--2nd--autosnap--1333353742
Apr  2 10:08:26 s02 libvirtd: Could not find keytab file: /etc/libvirt/krb5.tab: Permission denied
Apr  2 10:08:26 s02 libvirtd: Could not find keytab file: /etc/libvirt/krb5.tab: Permission denied
Apr  2 10:08:47 s02 libvirtd: Could not find keytab file: /etc/libvirt/krb5.tab: Permission denied
Apr  2 10:09:08 s02 libvirtd: Could not find keytab file: /etc/libvirt/krb5.tab: Permission denied
Apr  2 10:09:10 s02 kernel: INFO: task lvdisplay:29372 blocked for more than 120 seconds.
Apr  2 10:09:10 s02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr  2 10:09:10 s02 kernel: lvdisplay     D 0000000000000006     0 29372      1 0x00000084
Apr  2 10:09:10 s02 kernel: ffff88041b3f9b88 0000000000000086 ffff88041b3f9b48 ffffffffa00040bc
Apr  2 10:09:10 s02 kernel: ffff88041b3f9b58 00000000bfea3967 ffff88041b3f9b78 ffff880a24540200
Apr  2 10:09:10 s02 kernel: ffff88026c76b0b8 ffff88041b3f9fd8 000000000000f598 ffff88026c76b0b8
Apr  2 10:09:10 s02 kernel: Call Trace:
Apr  2 10:09:10 s02 kernel: [<ffffffffa00040bc>] ? dm_table_unplug_all+0x5c/0xd0 [dm_mod]
Apr  2 10:09:10 s02 kernel: [<ffffffff81098d19>] ? ktime_get_ts+0xa9/0xe0
Apr  2 10:09:10 s02 kernel: [<ffffffff814db743>] io_schedule+0x73/0xc0
Apr  2 10:09:10 s02 kernel: [<ffffffff811ac20e>] __blockdev_direct_IO+0x70e/0xc40
Apr  2 10:09:10 s02 kernel: [<ffffffff811a9e57>] blkdev_direct_IO+0x57/0x60
Apr  2 10:09:10 s02 kernel: [<ffffffff811a9020>] ? blkdev_get_blocks+0x0/0xc0
Apr  2 10:09:10 s02 kernel: [<ffffffff8110f19b>] generic_file_aio_read+0x6bb/0x700
Apr  2 10:09:10 s02 kernel: [<ffffffff8120c981>] ? avc_has_perm+0x71/0x90
Apr  2 10:09:10 s02 kernel: [<ffffffff812064af>] ? security_inode_permission+0x1f/0x30
Apr  2 10:09:10 s02 kernel: [<ffffffff8117269a>] do_sync_read+0xfa/0x140
Apr  2 10:09:10 s02 kernel: [<ffffffff8108e180>] ? autoremove_wake_function+0x0/0x40
Apr  2 10:09:10 s02 kernel: [<ffffffff811a93ec>] ? block_ioctl+0x3c/0x40
Apr  2 10:09:10 s02 kernel: [<ffffffff81185042>] ? vfs_ioctl+0x22/0xa0
Apr  2 10:09:10 s02 kernel: [<ffffffff81211edb>] ? selinux_file_permission+0xfb/0x150
Apr  2 10:09:10 s02 kernel: [<ffffffff81205346>] ? security_file_permission+0x16/0x20
Apr  2 10:09:10 s02 kernel: [<ffffffff811730c5>] vfs_read+0xb5/0x1a0
Apr  2 10:09:10 s02 kernel: [<ffffffff810d1b52>] ? audit_syscall_entry+0x272/0x2a0
Apr  2 10:09:10 s02 kernel: [<ffffffff81173201>] sys_read+0x51/0x90
Apr  2 10:09:10 s02 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
Apr  2 10:10:17 s02 dnsmasq-dhcp[3183]: DHCP packet received on br3 which has no address
Apr  2 10:11:10 s02 kernel: INFO: task lvdisplay:29372 blocked for more than 120 seconds.
Apr  2 10:11:10 s02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr  2 10:11:10 s02 kernel: lvdisplay     D 0000000000000006     0 29372      1 0x00000084
Apr  2 10:11:10 s02 kernel: ffff88041b3f9b88 0000000000000086 ffff88041b3f9b48 ffffffffa00040bc
Apr  2 10:11:10 s02 kernel: ffff88041b3f9b58 00000000bfea3967 ffff88041b3f9b78 ffff880a24540200
Apr  2 10:11:10 s02 kernel: ffff88026c76b0b8 ffff88041b3f9fd8 000000000000f598 ffff88026c76b0b8
Apr  2 10:11:10 s02 kernel: Call Trace:
Apr  2 10:11:10 s02 kernel: [<ffffffffa00040bc>] ? dm_table_unplug_all+0x5c/0xd0 [dm_mod]
Apr  2 10:11:10 s02 kernel: [<ffffffff81098d19>] ? ktime_get_ts+0xa9/0xe0
Apr  2 10:11:10 s02 kernel: [<ffffffff814db743>] io_schedule+0x73/0xc0
Apr  2 10:11:10 s02 kernel: [<ffffffff811ac20e>] __blockdev_direct_IO+0x70e/0xc40
Apr  2 10:11:10 s02 kernel: [<ffffffff811a9e57>] blkdev_direct_IO+0x57/0x60
Apr  2 10:11:10 s02 kernel: [<ffffffff811a9020>] ? blkdev_get_blocks+0x0/0xc0
Apr  2 10:11:10 s02 kernel: [<ffffffff8110f19b>] generic_file_aio_read+0x6bb/0x700
Apr  2 10:11:10 s02 kernel: [<ffffffff8120c981>] ? avc_has_perm+0x71/0x90
Apr  2 10:11:10 s02 kernel: [<ffffffff812064af>] ? security_inode_permission+0x1f/0x30
Apr  2 10:11:10 s02 kernel: [<ffffffff8117269a>] do_sync_read+0xfa/0x140
Apr  2 10:11:10 s02 kernel: [<ffffffff8108e180>] ? autoremove_wake_function+0x0/0x40
Apr  2 10:11:10 s02 kernel: [<ffffffff811a93ec>] ? block_ioctl+0x3c/0x40
Apr  2 10:11:10 s02 kernel: [<ffffffff81185042>] ? vfs_ioctl+0x22/0xa0
Apr  2 10:11:10 s02 kernel: [<ffffffff81211edb>] ? selinux_file_permission+0xfb/0x150
Apr  2 10:11:10 s02 kernel: [<ffffffff81205346>] ? security_file_permission+0x16/0x20
Apr  2 10:11:10 s02 kernel: [<ffffffff811730c5>] vfs_read+0xb5/0x1a0
Apr  2 10:11:10 s02 kernel: [<ffffffff810d1b52>] ? audit_syscall_entry+0x272/0x2a0
Apr  2 10:11:10 s02 kernel: [<ffffffff81173201>] sys_read+0x51/0x90
Apr  2 10:11:10 s02 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
Apr  2 10:12:57 s02 dnsmasq-dhcp[3183]: DHCP packet received on br3 which has no address
Apr  2 10:13:10 s02 kernel: INFO: task qemu-kvm:22570 blocked for more than 120 seconds.
Apr  2 10:13:10 s02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr  2 10:13:10 s02 kernel: qemu-kvm      D 0000000000000006     0 22570      1 0x00000080
Apr  2 10:13:10 s02 kernel: ffff8804ee3d5a68 0000000000000082 0000000000000000 ffffea000bbec910
Apr  2 10:13:10 s02 kernel: ffff8804ee3d59d8 ffffffff81012969 ffff8804ee3d5a18 00000001231112ef
Apr  2 10:13:10 s02 kernel: ffff880a2220c678 ffff8804ee3d5fd8 000000000000f598 ffff880a2220c678
Apr  2 10:13:10 s02 kernel: Call Trace:
Apr  2 10:13:10 s02 kernel: [<ffffffff81012969>] ? read_tsc+0x9/0x20
Apr  2 10:13:10 s02 kernel: [<ffffffff8110d3d0>] ? sync_page+0x0/0x50
Apr  2 10:13:10 s02 kernel: [<ffffffff814db743>] io_schedule+0x73/0xc0
Apr  2 10:13:10 s02 kernel: [<ffffffff8110d40d>] sync_page+0x3d/0x50
Apr  2 10:13:10 s02 kernel: [<ffffffff814dbfaf>] __wait_on_bit+0x5f/0x90
Apr  2 10:13:10 s02 kernel: [<ffffffff8110d5c3>] wait_on_page_bit+0x73/0x80
Apr  2 10:13:10 s02 kernel: [<ffffffff8108e1c0>] ? wake_bit_function+0x0/0x50
Apr  2 10:13:10 s02 kernel: [<ffffffff811232d5>] ? pagevec_lookup_tag+0x25/0x40
Apr  2 10:13:10 s02 kernel: [<ffffffff8110d9db>] wait_on_page_writeback_range+0xfb/0x190
Apr  2 10:13:10 s02 kernel: [<ffffffff8110dba8>] filemap_write_and_wait_range+0x78/0x90
Apr  2 10:13:10 s02 kernel: [<ffffffff811a0abe>] vfs_fsync_range+0x7e/0xe0
Apr  2 10:13:10 s02 kernel: [<ffffffff811a9501>] ? __invalidate_device+0x11/0x80
Apr  2 10:13:10 s02 kernel: [<ffffffff811a0b6b>] generic_write_sync+0x4b/0x50
Apr  2 10:13:10 s02 kernel: [<ffffffff811a95ee>] blkdev_aio_write+0x7e/0xa0
Apr  2 10:13:10 s02 kernel: [<ffffffff811a9570>] ? blkdev_aio_write+0x0/0xa0
Apr  2 10:13:10 s02 kernel: [<ffffffff8117241b>] do_sync_readv_writev+0xfb/0x140
Apr  2 10:13:10 s02 kernel: [<ffffffff8108e180>] ? autoremove_wake_function+0x0/0x40
Apr  2 10:13:10 s02 kernel: [<ffffffff81211edb>] ? selinux_file_permission+0xfb/0x150
Apr  2 10:13:10 s02 kernel: [<ffffffff81205346>] ? security_file_permission+0x16/0x20
Apr  2 10:13:10 s02 kernel: [<ffffffff811734df>] do_readv_writev+0xcf/0x1f0
Apr  2 10:13:10 s02 kernel: [<ffffffff8107ff76>] ? group_send_sig_info+0x56/0x70
Apr  2 10:13:10 s02 kernel: [<ffffffff8107ffcf>] ? kill_pid_info+0x3f/0x60
Apr  2 10:13:10 s02 kernel: [<ffffffff81173646>] vfs_writev+0x46/0x60
Apr  2 10:13:10 s02 kernel: [<ffffffff81173702>] sys_pwritev+0xa2/0xc0
Apr  2 10:13:10 s02 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
Apr  2 10:13:10 s02 kernel: INFO: task lvdisplay:29372 blocked for more than 120 seconds.
Apr  2 10:13:10 s02 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Apr  2 10:13:10 s02 kernel: lvdisplay     D 0000000000000006     0 29372      1 0x00000084
Apr  2 10:13:10 s02 kernel: ffff88041b3f9b88 0000000000000086 ffff88041b3f9b48 ffffffffa00040bc
Apr  2 10:13:10 s02 kernel: ffff88041b3f9b58 00000000bfea3967 ffff88041b3f9b78 ffff880a24540200
Apr  2 10:13:10 s02 kernel: ffff88026c76b0b8 ffff88041b3f9fd8 000000000000f598 ffff88026c76b0b8
Apr  2 10:13:10 s02 kernel: Call Trace:
Apr  2 10:13:10 s02 kernel: [<ffffffffa00040bc>] ? dm_table_unplug_all+0x5c/0xd0 [dm_mod]
Apr  2 10:13:10 s02 kernel: [<ffffffff81098d19>] ? ktime_get_ts+0xa9/0xe0
Apr  2 10:13:10 s02 kernel: [<ffffffff814db743>] io_schedule+0x73/0xc0

             reply	other threads:[~2012-04-03 15:05 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-04-03 15:05 Dan C [this message]
2012-04-24 15:27 [linux-lvm] lvremove snapshot hangs LVM system Da
2012-04-24 17:33 ` Ray Morris
2012-05-07  9:19 Da
2012-06-13 14:46 Da
2012-06-13 16:09 Da
2012-06-19 13:34 ` Da

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CA+8+0ELd53bWR3JZbUvnDiQr0nsYZV1riqWPj9qT0ZXu007Vmg@mail.gmail.com \
    --to=dcodix@gmail.com \
    --cc=linux-lvm@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.