* [linux-lvm] Does LVM RAID1 have TRIM support? @ 2014-08-24 16:08 Jarkko Oranen 2014-09-02 21:49 ` Brassow Jonathan 0 siblings, 1 reply; 8+ messages in thread From: Jarkko Oranen @ 2014-08-24 16:08 UTC (permalink / raw) To: linux-lvm Hello Yesterday I experimented a bit with my RAID configuration on a pair of SSDs, and it seems that LVM's native RAID does not have TRIM support... At least, when I try to run fstrim manually, it complains even though issue_discards is enabled. Plain LVs on top of an MD RAID PV do work, of course. Am I perhaps missing some configuration, or do RAID1 logical volumes simply not have support for TRIM yet? I'm running a fairly recent kernel (3.15.8) and lvm version says this: LVM version: 2.02.106(2) (2014-04-10) Library version: 1.02.85 (2014-04-10) Driver version: 4.27.0 As an aside, can anyone point me to documentation or other resources about the pros and cons of LVM native RAID1 setup (which I understand uses MD RAID internally?) vs. MD RAID PV + LVM. It seems I might be able to save some SSD space and only mirror the LVs I actually need to keep safe from crashes. -- Jarkko Oranen ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [linux-lvm] Does LVM RAID1 have TRIM support? 2014-08-24 16:08 [linux-lvm] Does LVM RAID1 have TRIM support? Jarkko Oranen @ 2014-09-02 21:49 ` Brassow Jonathan 2014-09-04 11:06 ` Jarkko Oranen 0 siblings, 1 reply; 8+ messages in thread From: Brassow Jonathan @ 2014-09-02 21:49 UTC (permalink / raw) To: oranenj+lvm, LVM general discussion and development On Aug 24, 2014, at 11:08 AM, Jarkko Oranen wrote: > Hello > > Yesterday I experimented a bit with my RAID configuration on a pair of > SSDs, and it seems that LVM's native RAID does not have TRIM support... > At least, when I try to run fstrim manually, it complains even though > issue_discards is enabled. Plain LVs on top of an MD RAID PV do work, of > course. > > Am I perhaps missing some configuration, or do RAID1 logical volumes > simply not have support for TRIM yet? I'm running a fairly recent kernel > (3.15.8) and lvm version says this: > > LVM version: 2.02.106(2) (2014-04-10) > Library version: 1.02.85 (2014-04-10) > Driver version: 4.27.0 TRIM is not yet supported in LVM RAID. However, if MD has a solid TRIM implementation, it should be simple to enable it for LVM. (This is because the MD kernel modules are used to perform RAID for LVM. There is only a thin wrapper layer (linux/drivers/md/dm-raid.c) in device-mapper used to set-up the device.) > As an aside, can anyone point me to documentation or other resources > about the pros and cons of LVM native RAID1 setup (which I understand > uses MD RAID internally?) vs. MD RAID PV + LVM. It seems I might be able > to save some SSD space and only mirror the LVs I actually need to keep > safe from crashes. I don't know if there is a specific list to point to out there, but I can give you a couple pros/cons. PRO: - use one volume manager instead of two - LVM is better suited to creating devices of varying sizes - leaving spare capacity for snapshots, etc CONS: - no trim support with RAID through LVM (although, I'm not sure of the state in MD) - no reshaping (changing from one RAID type to another) capability in LVM RAID. brassow ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [linux-lvm] Does LVM RAID1 have TRIM support? 2014-09-02 21:49 ` Brassow Jonathan @ 2014-09-04 11:06 ` Jarkko Oranen 2014-09-08 17:47 ` Brassow Jonathan 0 siblings, 1 reply; 8+ messages in thread From: Jarkko Oranen @ 2014-09-04 11:06 UTC (permalink / raw) To: LVM general discussion and development; +Cc: oranenj+lvm On Tue, 2014-09-02 at 16:49 -0500, Brassow Jonathan wrote: > On Aug 24, 2014, at 11:08 AM, Jarkko Oranen wrote: > > > Hello > > > > Yesterday I experimented a bit with my RAID configuration on a pair of > > SSDs, and it seems that LVM's native RAID does not have TRIM support... > > At least, when I try to run fstrim manually, it complains even though > > issue_discards is enabled. Plain LVs on top of an MD RAID PV do work, of > > course. > > > > Am I perhaps missing some configuration, or do RAID1 logical volumes > > simply not have support for TRIM yet? I'm running a fairly recent kernel > > (3.15.8) and lvm version says this: > > > > LVM version: 2.02.106(2) (2014-04-10) > > Library version: 1.02.85 (2014-04-10) > > Driver version: 4.27.0 > > TRIM is not yet supported in LVM RAID. However, if MD has a solid TRIM implementation, it should be simple to enable it for LVM. (This is because the MD kernel modules are used to perform RAID for LVM. There is only a thin wrapper layer (linux/drivers/md/dm-raid.c) in device-mapper used to set-up the device.) Thanks for the reply! Unfortunately, "simple" is relative. I have no experience with kernel hacking at all (or even much C programming, though I can read it), but I took a look at the code anyway to try to pinpoint the place that tells userspace the ioctl isn't supported. From my cursory browsing of the code I couldn't really tell how it works, but for the most part it seemed like it *should* just work... It does work just fine for linear volumes, and md/dm-linear.c doesn't seem to do anything special to enable discard/FITRIM support, while discard is specifically marked as supported in the md/linear.c and md/raid*.c files if the underlying device supports it. However, in dm.c there's the dm_blk_ioctl function which says the following: /* We only support devices that have a single target */ if (dm_table_get_num_targets(map) != 1) goto out; Intuitively it looks like this could prevent a raid1 target from working as it apparently consists of rmeta and rimage subcomponents, but I couldn't determine for certain if my intuition is correct. I did some testing, and discards work just fine on a linear volume spanning two different partitions, while they fail on a "RAID" mirror of two similar partitions. Clearly, two devices in a mapping are not a problem. > > > As an aside, can anyone point me to documentation or other resources > > about the pros and cons of LVM native RAID1 setup (which I understand > > uses MD RAID internally?) vs. MD RAID PV + LVM. It seems I might be able > > to save some SSD space and only mirror the LVs I actually need to keep > > safe from crashes. > > I don't know if there is a specific list to point to out there, but I can give you a couple pros/cons. > PRO: > - use one volume manager instead of two > - LVM is better suited to creating devices of varying sizes - leaving spare capacity for snapshots, etc > > CONS: > - no trim support with RAID through LVM (although, I'm not sure of the state in MD) > - no reshaping (changing from one RAID type to another) capability in LVM RAID. > > brassow > > _______________________________________________ > linux-lvm mailing list > linux-lvm@redhat.com > https://www.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [linux-lvm] Does LVM RAID1 have TRIM support? 2014-09-04 11:06 ` Jarkko Oranen @ 2014-09-08 17:47 ` Brassow Jonathan 2014-09-08 18:47 ` Jarkko Oranen 0 siblings, 1 reply; 8+ messages in thread From: Brassow Jonathan @ 2014-09-08 17:47 UTC (permalink / raw) To: LVM general discussion and development; +Cc: oranenj+lvm On Sep 4, 2014, at 6:06 AM, Jarkko Oranen wrote: >> TRIM is not yet supported in LVM RAID. However, if MD has a solid TRIM implementation, it should be simple to enable it for LVM. (This is because the MD kernel modules are used to perform RAID for LVM. There is only a thin wrapper layer (linux/drivers/md/dm-raid.c) in device-mapper used to set-up the device.) > > Thanks for the reply! Unfortunately, "simple" is relative. By "simple", I mean that from our side all we should need to do is set 'num_discard_bios = 1' in dm-raid.c. That is, if things are working fine in MD. We could even make that conditional depending on the type of RAID (e.g. discards for RAID1, but not RAID5). It could end up being that simple, but I need to know if MD has proper support for discards and all the kinks have been worked out. brassow ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [linux-lvm] Does LVM RAID1 have TRIM support? 2014-09-08 17:47 ` Brassow Jonathan @ 2014-09-08 18:47 ` Jarkko Oranen 2014-09-11 23:31 ` Brassow Jonathan 0 siblings, 1 reply; 8+ messages in thread From: Jarkko Oranen @ 2014-09-08 18:47 UTC (permalink / raw) To: LVM general discussion and development; +Cc: oranenj+lvm On Mon, 2014-09-08 at 12:47 -0500, Brassow Jonathan wrote: > On Sep 4, 2014, at 6:06 AM, Jarkko Oranen wrote: > > >> TRIM is not yet supported in LVM RAID. However, if MD has a solid TRIM implementation, it should be simple to enable it for LVM. (This is because the MD kernel modules are used to perform RAID for LVM. There is only a thin wrapper layer (linux/drivers/md/dm-raid.c) in device-mapper used to set-up the device.) > > > > Thanks for the reply! Unfortunately, "simple" is relative. > > By "simple", I mean that from our side all we should need to do is set 'num_discard_bios = 1' in dm-raid.c. That is, if things are working fine in MD. We could even make that conditional depending on the type of RAID (e.g. discards for RAID1, but not RAID5). It could end up being that simple, but I need to know if MD has proper support for discards and all the kinks have been worked out. > dm-raid.c already has ti->num_discard_bios = 1; in raid_ctr in the kernel source I'm looking at (3.15.6; not the most recent, but it's what I happen to have locally available). As I said, to me it looks like it should already work, but it doesn't, so I spent a bit of time trying to understand why. Digging at it again, eg. lsblk -D seems to report discard information identical to other discard-supporting block devices, suggesting that discard passthrough should work fine, but the actual ioctl fails (when testing via either fstrim or blkdiscard) with "not supported" for whatever reason. As far as I know, discard support in MD is stable, and has been so for a long while; my current setup consists of linear logical volumes on top of a software RAID PV created with mdadm, and I haven't had a single issue with discards. It seems RAID1 discard support was merged in January 2011 (git commit 5fc2ffeabb9ee0fc0e71ff16b49f34f0ed3d05b4), so it's definitely not new. -- Jarkko ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [linux-lvm] Does LVM RAID1 have TRIM support? 2014-09-08 18:47 ` Jarkko Oranen @ 2014-09-11 23:31 ` Brassow Jonathan 2014-09-12 7:24 ` Jarkko Oranen 0 siblings, 1 reply; 8+ messages in thread From: Brassow Jonathan @ 2014-09-11 23:31 UTC (permalink / raw) To: LVM general discussion and development; +Cc: oranenj+lvm On Sep 8, 2014, at 1:47 PM, Jarkko Oranen wrote: > On Mon, 2014-09-08 at 12:47 -0500, Brassow Jonathan wrote: >> On Sep 4, 2014, at 6:06 AM, Jarkko Oranen wrote: >> >>>> TRIM is not yet supported in LVM RAID. However, if MD has a solid TRIM implementation, it should be simple to enable it for LVM. (This is because the MD kernel modules are used to perform RAID for LVM. There is only a thin wrapper layer (linux/drivers/md/dm-raid.c) in device-mapper used to set-up the device.) >>> >>> Thanks for the reply! Unfortunately, "simple" is relative. >> >> By "simple", I mean that from our side all we should need to do is set 'num_discard_bios = 1' in dm-raid.c. That is, if things are working fine in MD. We could even make that conditional depending on the type of RAID (e.g. discards for RAID1, but not RAID5). It could end up being that simple, but I need to know if MD has proper support for discards and all the kinks have been worked out. >> > > dm-raid.c already has > > ti->num_discard_bios = 1; > > in raid_ctr in the kernel source I'm looking at (3.15.6; not the most > recent, but it's what I happen to have locally available). As I said, to > me it looks like it should already work, but it doesn't, so I spent a > bit of time trying to understand why. > > Digging at it again, eg. lsblk -D seems to report discard information > identical to other discard-supporting block devices, suggesting that > discard passthrough should work fine, but the actual ioctl fails (when > testing via either fstrim or blkdiscard) with "not supported" for > whatever reason. > > As far as I know, discard support in MD is stable, and has been so for a > long while; my current setup consists of linear logical volumes on top > of a software RAID PV created with mdadm, and I haven't had a single > issue with discards. It seems RAID1 discard support was merged in > January 2011 (git commit 5fc2ffeabb9ee0fc0e71ff16b49f34f0ed3d05b4), so > it's definitely not new. dm-raid.c does not have that - dm-raid1.c does. dm-raid1.c is a mirroring implementation specific to device-mapper. It is used by LVM for pvmove functionality (and is otherwise being slowly phased out). dm-raid.c is the device-mapper wrapper around the MD RAID personalities. It does not have discard support. I will ask Neil what the current state of discard support is in MD. I see that "support" was added in Oct of 2012 (starting at commit 2ff8cc2), but I remember them having problems with discards + any resync operations. If Neil says it is ok, I will turn it on. brassow ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [linux-lvm] Does LVM RAID1 have TRIM support? 2014-09-11 23:31 ` Brassow Jonathan @ 2014-09-12 7:24 ` Jarkko Oranen 2014-09-24 4:17 ` Brassow Jonathan 0 siblings, 1 reply; 8+ messages in thread From: Jarkko Oranen @ 2014-09-12 7:24 UTC (permalink / raw) To: LVM general discussion and development; +Cc: oranenj+lvm On Thu, 2014-09-11 at 18:31 -0500, Brassow Jonathan wrote: > dm-raid.c does not have that - dm-raid1.c does. dm-raid1.c is a mirroring implementation specific to device-mapper. It is used by LVM for pvmove functionality (and is otherwise being slowly phased out). dm-raid.c is the device-mapper wrapper around the MD RAID personalities. It does not have discard support. > > I will ask Neil what the current state of discard support is in MD. I see that "support" was added in Oct of 2012 (starting at commit 2ff8cc2), but I remember them having problems with discards + any resync operations. If Neil says it is ok, I will turn it on. > In the code that I'm looking at dm-raid.c definitely sets num_discard_bios to 1 on something... As does dm-raid1.c. However, Thanks for looking into it! > brassow > > _______________________________________________ > linux-lvm mailing list > linux-lvm@redhat.com > https://www.redhat.com/mailman/listinfo/linux-lvm > read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [linux-lvm] Does LVM RAID1 have TRIM support? 2014-09-12 7:24 ` Jarkko Oranen @ 2014-09-24 4:17 ` Brassow Jonathan 0 siblings, 0 replies; 8+ messages in thread From: Brassow Jonathan @ 2014-09-24 4:17 UTC (permalink / raw) To: LVM general discussion and development; +Cc: oranenj+lvm On Sep 12, 2014, at 2:24 AM, Jarkko Oranen wrote: > On Thu, 2014-09-11 at 18:31 -0500, Brassow Jonathan wrote: > >> dm-raid.c does not have that - dm-raid1.c does. dm-raid1.c is a mirroring implementation specific to device-mapper. It is used by LVM for pvmove functionality (and is otherwise being slowly phased out). dm-raid.c is the device-mapper wrapper around the MD RAID personalities. It does not have discard support. >> >> I will ask Neil what the current state of discard support is in MD. I see that "support" was added in Oct of 2012 (starting at commit 2ff8cc2), but I remember them having problems with discards + any resync operations. If Neil says it is ok, I will turn it on. >> > > In the code that I'm looking at dm-raid.c definitely sets > num_discard_bios to 1 on something... As does dm-raid1.c. However, > Thanks for looking into it! Heinz Mauelshagen is looking to get this included upstream now. Thanks for the poke! :) Initial status query: http://marc.info/?l=linux-raid&m=141047869821032&w=2 Patch posted for consideration: https://www.redhat.com/archives/dm-devel/2014-September/msg00126.html brassow ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2014-09-24 4:17 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-08-24 16:08 [linux-lvm] Does LVM RAID1 have TRIM support? Jarkko Oranen 2014-09-02 21:49 ` Brassow Jonathan 2014-09-04 11:06 ` Jarkko Oranen 2014-09-08 17:47 ` Brassow Jonathan 2014-09-08 18:47 ` Jarkko Oranen 2014-09-11 23:31 ` Brassow Jonathan 2014-09-12 7:24 ` Jarkko Oranen 2014-09-24 4:17 ` Brassow Jonathan
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.