* is there a COW inside the kernel ? @ 2006-03-03 8:29 roland 2006-03-03 13:33 ` Jan Engelhardt ` (3 more replies) 0 siblings, 4 replies; 8+ messages in thread From: roland @ 2006-03-03 8:29 UTC (permalink / raw) To: linux-kernel hello ! is there an equivalent of something like cowloop ( http://www.atconsultancy.nl/cowloop/total.html ) or md based cow device ( http://www.cl.cam.ac.uk/users/br260/doc/report.pdf ), i.e. a feature called "Copy On Write Blockdevice" inside the current or the near-future mainline kernel (besides UserModeLinux Arch)? i'm not sure - i think i remember having read that something like this can be probably done , but i don`t remember anymore what it was. i would find this useful for several purpose, but i don`t want to patch my system with 3rd party drivers or "non-standard" stuff - or even recompile the kernel. can someone help out with some information ? TIA roland k. system engineer ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: is there a COW inside the kernel ? 2006-03-03 8:29 is there a COW inside the kernel ? roland @ 2006-03-03 13:33 ` Jan Engelhardt 2006-03-03 14:19 ` Alasdair G Kergon ` (2 subsequent siblings) 3 siblings, 0 replies; 8+ messages in thread From: Jan Engelhardt @ 2006-03-03 13:33 UTC (permalink / raw) To: roland; +Cc: linux-kernel > is there an equivalent of something like > cowloop ( http://www.atconsultancy.nl/cowloop/total.html ) or md based cow > device ( http://www.cl.cam.ac.uk/users/br260/doc/report.pdf ), > i.e. a feature called "Copy On Write Blockdevice" inside the current or the > near-future mainline kernel (besides UserModeLinux Arch)? Not directly block-device-based cow, but unionfs is at least a filesystem-based cow. Jan Engelhardt -- ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: is there a COW inside the kernel ? 2006-03-03 8:29 is there a COW inside the kernel ? roland 2006-03-03 13:33 ` Jan Engelhardt @ 2006-03-03 14:19 ` Alasdair G Kergon 2006-03-03 14:28 ` Kevin Corry 2006-03-03 15:25 ` Jeff Dike 3 siblings, 0 replies; 8+ messages in thread From: Alasdair G Kergon @ 2006-03-03 14:19 UTC (permalink / raw) To: roland; +Cc: linux-kernel On Fri, Mar 03, 2006 at 09:29:02AM +0100, roland wrote: > is there an equivalent of something like > cowloop ( http://www.atconsultancy.nl/cowloop/total.html ) or md based cow > device ( http://www.cl.cam.ac.uk/users/br260/doc/report.pdf ), > i.e. a feature called "Copy On Write Blockdevice" inside the current or the > near-future mainline kernel (besides UserModeLinux Arch)? device-mapper snapshots? Documentation/device-mapper/snapshot.txt Alasdair -- agk@redhat.com ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: is there a COW inside the kernel ? 2006-03-03 8:29 is there a COW inside the kernel ? roland 2006-03-03 13:33 ` Jan Engelhardt 2006-03-03 14:19 ` Alasdair G Kergon @ 2006-03-03 14:28 ` Kevin Corry 2006-03-03 21:29 ` roland 2006-03-03 15:25 ` Jeff Dike 3 siblings, 1 reply; 8+ messages in thread From: Kevin Corry @ 2006-03-03 14:28 UTC (permalink / raw) To: linux-kernel; +Cc: roland On Fri March 3 2006 2:29 am, roland wrote: > hello ! > > is there an equivalent of something like > > cowloop ( http://www.atconsultancy.nl/cowloop/total.html ) or md based cow > device ( http://www.cl.cam.ac.uk/users/br260/doc/report.pdf ), > > i.e. a feature called "Copy On Write Blockdevice" inside the current or the > near-future mainline kernel (besides UserModeLinux Arch)? Device-Mapper has a snapshot module, which is used by LVM and EVMS. You can also use dmsetup if you want lower-level access than provided by the volume managers. To do the equivalent of the cowloop driver that you linked to above, you could do something like this: Say you have a read-only block-device (say a cd-rom) at /dev/hdc. And you have a small disk partition, /dev/hdb1, that you want to use for your "COW file". Run: cow_size=`blockdev --getsize /dev/hdc` chunk_size=64 # Size of each copied-on-write chunk, in 512 byte sectors cow_name="my_cow_dev" echo "0 $cow_size snapshot /dev/hdc /dev/hdb1 p $chunk_size" | \ dmsetup create $cow_name This will give you a device called /dev/mapper/$cow_name. Presuming /dev/hdc has a filesystem on it, you can mount /dev/mapper/$cow_name and get a read-write version of the filesystem on /dev/hdc, where updates to the filesystem will be stored on /dev/hdb1. The size of /dev/hdb1 can be significantly smaller than /dev/hdc, depending on the amount of writes you expect to happen on /dev/mapper/$cow_name. While this device is active, don't try to mount /dev/hdc read-write (assuming that's possible), or it will corrupt the view of /dev/mapper/$cow_name. If you need read-write access to both devices simultaneously, you'll probably just want to use LVM or EVMS and create snapshot volumes, since manually activating that kind of setup with dmsetup is incredibly tricky. Use "dmsetup remove $cow_name" to deactivate the device. > i would find this useful for several purpose, but i don`t want to patch my > system with 3rd party drivers or "non-standard" stuff - or even recompile > the kernel. This should work with any recent 2.6 kernel. You'll also need to have the device-mapper package installed, which should be available with any recent Linux distro. -- Kevin Corry kevcorry@us.ibm.com http://www.ibm.com/linux/ http://evms.sourceforge.net/ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: is there a COW inside the kernel ? 2006-03-03 14:28 ` Kevin Corry @ 2006-03-03 21:29 ` roland 2006-03-03 22:39 ` Kevin Corry 2006-03-04 5:35 ` Jon Masters 0 siblings, 2 replies; 8+ messages in thread From: roland @ 2006-03-03 21:29 UTC (permalink / raw) To: Kevin Corry, linux-kernel; +Cc: Jeff Dike, agk, jengelh hello ! thanks to all for providing that information. i think i will take a closer look on device-mapper, but i'm unsure if it`s perfectly suited. can i only use devices, not files for the cow? what about merging a cow-dev/file back to the r/o-dev/file ? cowloop can do this, and because it can use files, i don`t need to provide a fixed amount of diskspace for the cow. you have direct feedback about how big you cow grows.... i'm unsure about dm, but i will dig into the details. (wanted to learn more about dm anyway) any chance of cowloop being merged into mainline ? loop.c is 1343 lines of code, cowloop.c is around 1000 lines more so it`s "reasonable" small....and it has a _really_ nice user manual. regards roland ps: btw - mountlo looks really "freaky" - i like uml very much, but i think i need each bit of performance. will take a look at mountlo just for personal interest, though. ----- Original Message ----- From: "Kevin Corry" <kevcorry@us.ibm.com> To: <linux-kernel@vger.kernel.org> Cc: "roland" <devzero@web.de> Sent: Friday, March 03, 2006 3:28 PM Subject: Re: is there a COW inside the kernel ? > On Fri March 3 2006 2:29 am, roland wrote: >> hello ! >> >> is there an equivalent of something like >> >> cowloop ( http://www.atconsultancy.nl/cowloop/total.html ) or md based >> cow >> device ( http://www.cl.cam.ac.uk/users/br260/doc/report.pdf ), >> >> i.e. a feature called "Copy On Write Blockdevice" inside the current or >> the >> near-future mainline kernel (besides UserModeLinux Arch)? > > Device-Mapper has a snapshot module, which is used by LVM and EVMS. You > can > also use dmsetup if you want lower-level access than provided by the > volume > managers. To do the equivalent of the cowloop driver that you linked to > above, you could do something like this: > > Say you have a read-only block-device (say a cd-rom) at /dev/hdc. And you > have > a small disk partition, /dev/hdb1, that you want to use for your "COW > file". > Run: > > cow_size=`blockdev --getsize /dev/hdc` > chunk_size=64 # Size of each copied-on-write chunk, in 512 byte sectors > cow_name="my_cow_dev" > echo "0 $cow_size snapshot /dev/hdc /dev/hdb1 p $chunk_size" | \ > dmsetup create $cow_name > > This will give you a device called /dev/mapper/$cow_name. Presuming > /dev/hdc > has a filesystem on it, you can mount /dev/mapper/$cow_name and get a > read-write version of the filesystem on /dev/hdc, where updates to the > filesystem will be stored on /dev/hdb1. The size of /dev/hdb1 can be > significantly smaller than /dev/hdc, depending on the amount of writes you > expect to happen on /dev/mapper/$cow_name. While this device is active, > don't > try to mount /dev/hdc read-write (assuming that's possible), or it will > corrupt the view of /dev/mapper/$cow_name. If you need read-write access > to > both devices simultaneously, you'll probably just want to use LVM or EVMS > and > create snapshot volumes, since manually activating that kind of setup with > dmsetup is incredibly tricky. > > Use "dmsetup remove $cow_name" to deactivate the device. > >> i would find this useful for several purpose, but i don`t want to patch >> my >> system with 3rd party drivers or "non-standard" stuff - or even >> recompile >> the kernel. > > This should work with any recent 2.6 kernel. You'll also need to have the > device-mapper package installed, which should be available with any recent > Linux distro. > > -- > Kevin Corry > kevcorry@us.ibm.com > http://www.ibm.com/linux/ > http://evms.sourceforge.net/ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: is there a COW inside the kernel ? 2006-03-03 21:29 ` roland @ 2006-03-03 22:39 ` Kevin Corry 2006-03-04 5:35 ` Jon Masters 1 sibling, 0 replies; 8+ messages in thread From: Kevin Corry @ 2006-03-03 22:39 UTC (permalink / raw) To: linux-kernel; +Cc: roland On Fri March 3 2006 3:29 pm, roland wrote: > i think i will take a closer look on device-mapper, but i'm unsure if it`s > perfectly suited. > > can i only use devices, not files for the cow? Yes, Device-Mapper can only map to block-devices. If you need to do this with files, you could use losetup to create block-device from them. However, the COW device still won't "grow" automatically as you described. > what about merging a cow-dev/file back to the r/o-dev/file ? Device-Mapper does not directly support this, but EVMS provides a "rollback" function for reverting an origin volume back to the contents of its snapshot volume. -- Kevin Corry kevcorry@us.ibm.com http://www.ibm.com/linux/ http://evms.sourceforge.net/ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: is there a COW inside the kernel ? 2006-03-03 21:29 ` roland 2006-03-03 22:39 ` Kevin Corry @ 2006-03-04 5:35 ` Jon Masters 1 sibling, 0 replies; 8+ messages in thread From: Jon Masters @ 2006-03-04 5:35 UTC (permalink / raw) To: roland; +Cc: Kevin Corry, linux-kernel, Jeff Dike, agk, jengelh On 3/3/06, roland <devzero@web.de> wrote: > i think i will take a closer look on device-mapper, but i'm unsure if it`s > perfectly suited. It looks to me that you want to use something more like unionfs with COW/whiteout mode - there's an implementation based on FUSE available but I've not really looked at it. Obviously you might just want something in mainline, in which case you're limited to the FUSE idea or one of the others already suggested. > what about merging a cow-dev/file back to the r/o-dev/file ? This is why I suggest you look at the above. Jon. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: is there a COW inside the kernel ? 2006-03-03 8:29 is there a COW inside the kernel ? roland ` (2 preceding siblings ...) 2006-03-03 14:28 ` Kevin Corry @ 2006-03-03 15:25 ` Jeff Dike 3 siblings, 0 replies; 8+ messages in thread From: Jeff Dike @ 2006-03-03 15:25 UTC (permalink / raw) To: roland; +Cc: linux-kernel On Fri, Mar 03, 2006 at 09:29:02AM +0100, roland wrote: > hello ! > > is there an equivalent of something like > > cowloop ( http://www.atconsultancy.nl/cowloop/total.html ) or md based cow > device ( http://www.cl.cam.ac.uk/users/br260/doc/report.pdf ), > > i.e. a feature called "Copy On Write Blockdevice" inside the current or the > near-future mainline kernel (besides UserModeLinux Arch)? > can someone help out with some information ? Miklos Szeredi announced mountlo a few days ago - this uses a UML to export a filesystem to the host through FUSE. It's intended to allow non-privileged loopback mounting of normal file system images, but presumably will export a COW block device as well. I'm doing something similar, and using FUSE to export the entire UML filesystem to the host. These aren't specifically COW drivers, but they have the same effect as long as you have a UML with your COW device mounted. Jeff ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2006-03-04 5:35 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2006-03-03 8:29 is there a COW inside the kernel ? roland 2006-03-03 13:33 ` Jan Engelhardt 2006-03-03 14:19 ` Alasdair G Kergon 2006-03-03 14:28 ` Kevin Corry 2006-03-03 21:29 ` roland 2006-03-03 22:39 ` Kevin Corry 2006-03-04 5:35 ` Jon Masters 2006-03-03 15:25 ` Jeff Dike
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).