linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* btrfs seed question
@ 2017-10-12  0:47 Joseph Dunn
  2017-10-12  4:18 ` Anand Jain
  0 siblings, 1 reply; 11+ messages in thread
From: Joseph Dunn @ 2017-10-12  0:47 UTC (permalink / raw)
  To: linux-btrfs

After seeing how btrfs seeds work I wondered if it was possible to push
specific files from the seed to the rw device.  I know that removing
the seed device will flush all the contents over to the rw device, but
what about flushing individual files on demand?

I found that opening a file, reading the contents, seeking back to 0,
and writing out the contents does what I want, but I was hoping for a
bit less of a hack.

Is there maybe an ioctl or something else that might trigger a similar
action?

Thanks,
-Joseph

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: btrfs seed question
  2017-10-12  0:47 btrfs seed question Joseph Dunn
@ 2017-10-12  4:18 ` Anand Jain
  2017-10-12 13:20   ` Joseph Dunn
  0 siblings, 1 reply; 11+ messages in thread
From: Anand Jain @ 2017-10-12  4:18 UTC (permalink / raw)
  To: Joseph Dunn, linux-btrfs



On 10/12/2017 08:47 AM, Joseph Dunn wrote:
> After seeing how btrfs seeds work I wondered if it was possible to push
> specific files from the seed to the rw device.  I know that removing
> the seed device will flush all the contents over to the rw device, but
> what about flushing individual files on demand?
> 
> I found that opening a file, reading the contents, seeking back to 0,
> and writing out the contents does what I want, but I was hoping for a
> bit less of a hack.
> 
> Is there maybe an ioctl or something else that might trigger a similar
> action?

   You mean to say - seed-device delete to trigger copy of only the 
specified or the modified files only, instead of whole of seed-device ? 
What's the use case around this ?


Thanks, Anand


> Thanks,
> -Joseph
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: btrfs seed question
  2017-10-12  4:18 ` Anand Jain
@ 2017-10-12 13:20   ` Joseph Dunn
  2017-10-12 14:32     ` Chris Murphy
  2017-11-03  7:56     ` Kai Krakow
  0 siblings, 2 replies; 11+ messages in thread
From: Joseph Dunn @ 2017-10-12 13:20 UTC (permalink / raw)
  To: Anand Jain; +Cc: linux-btrfs

On Thu, 12 Oct 2017 12:18:01 +0800
Anand Jain <anand.jain@oracle.com> wrote:

> On 10/12/2017 08:47 AM, Joseph Dunn wrote:
> > After seeing how btrfs seeds work I wondered if it was possible to push
> > specific files from the seed to the rw device.  I know that removing
> > the seed device will flush all the contents over to the rw device, but
> > what about flushing individual files on demand?
> > 
> > I found that opening a file, reading the contents, seeking back to 0,
> > and writing out the contents does what I want, but I was hoping for a
> > bit less of a hack.
> > 
> > Is there maybe an ioctl or something else that might trigger a similar
> > action?  
> 
>    You mean to say - seed-device delete to trigger copy of only the 
> specified or the modified files only, instead of whole of seed-device ? 
> What's the use case around this ?
> 

Not quite.  While the seed device is still connected I would like to
force some files over to the rw device.  The use case is basically a
much slower link to a seed device holding significantly more data than
we currently need.  An example would be a slower iscsi link to the seed
device and a local rw ssd.  I would like fast access to a certain subset
of files, likely larger than the memory cache will accommodate.  If at
a later time I want to discard the image as a whole I could unmount the
file system or if I want a full local copy I could delete the
seed-device to sync the fs.  In the mean time I would have access to
all the files, with some slower (iscsi) and some faster (ssd) and the
ability to pick which ones are in the faster group at the cost of one
content transfer.

I'm not necessarily looking for a new feature addition, just if there is
some existing call that I can make to push specific files from the slow
mirror to the fast one.  If I had to push a significant amount of
metadata that would be fine, but the file contents feeding some
computations might be large and useful only to certain clients.

So far I found that I can re-write the file with the same contents and
thanks to the lack of online dedupe these writes land on the rw mirror
so later reads to that file should not hit the slower mirror.  By the
way, if I'm misunderstanding how the read process would work after the
file push please correct me.

I hope this makes sense but I'll try to clarify further if you have
more questions.

-Joseph

> 
> Thanks, Anand
> 
> 
> > Thanks,
> > -Joseph
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >   

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: btrfs seed question
  2017-10-12 13:20   ` Joseph Dunn
@ 2017-10-12 14:32     ` Chris Murphy
  2017-10-12 14:44       ` Joseph Dunn
  2017-11-03  7:56     ` Kai Krakow
  1 sibling, 1 reply; 11+ messages in thread
From: Chris Murphy @ 2017-10-12 14:32 UTC (permalink / raw)
  To: Joseph Dunn; +Cc: Anand Jain, Btrfs BTRFS

On Thu, Oct 12, 2017 at 2:20 PM, Joseph Dunn <jdunn14@gmail.com> wrote:
>
> On Thu, 12 Oct 2017 12:18:01 +0800
> Anand Jain <anand.jain@oracle.com> wrote:
>
> > On 10/12/2017 08:47 AM, Joseph Dunn wrote:
> > > After seeing how btrfs seeds work I wondered if it was possible to push
> > > specific files from the seed to the rw device.  I know that removing
> > > the seed device will flush all the contents over to the rw device, but
> > > what about flushing individual files on demand?
> > >
> > > I found that opening a file, reading the contents, seeking back to 0,
> > > and writing out the contents does what I want, but I was hoping for a
> > > bit less of a hack.
> > >
> > > Is there maybe an ioctl or something else that might trigger a similar
> > > action?
> >
> >    You mean to say - seed-device delete to trigger copy of only the
> > specified or the modified files only, instead of whole of seed-device ?
> > What's the use case around this ?
> >
>
> Not quite.  While the seed device is still connected I would like to
> force some files over to the rw device.  The use case is basically a
> much slower link to a seed device holding significantly more data than
> we currently need.  An example would be a slower iscsi link to the seed
> device and a local rw ssd.  I would like fast access to a certain subset
> of files, likely larger than the memory cache will accommodate.  If at
> a later time I want to discard the image as a whole I could unmount the
> file system or if I want a full local copy I could delete the
> seed-device to sync the fs.  In the mean time I would have access to
> all the files, with some slower (iscsi) and some faster (ssd) and the
> ability to pick which ones are in the faster group at the cost of one
> content transfer.


Multiple seeds?

Seed A has everything, is remote. Create sprout B also remotely,
deleting the things you don't absolutely need, then make it a seed.
Now via iSCSI you can mount both A and B seeds. Add local rw sprout C
to seed B, then delete B to move files to fast local storage.



-- 
Chris Murphy

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: btrfs seed question
  2017-10-12 14:32     ` Chris Murphy
@ 2017-10-12 14:44       ` Joseph Dunn
  2017-10-12 15:30         ` Chris Murphy
  2017-10-13  2:52         ` Anand Jain
  0 siblings, 2 replies; 11+ messages in thread
From: Joseph Dunn @ 2017-10-12 14:44 UTC (permalink / raw)
  To: Chris Murphy; +Cc: Anand Jain, Btrfs BTRFS

On Thu, 12 Oct 2017 15:32:24 +0100
Chris Murphy <lists@colorremedies.com> wrote:

> On Thu, Oct 12, 2017 at 2:20 PM, Joseph Dunn <jdunn14@gmail.com> wrote:
> >
> > On Thu, 12 Oct 2017 12:18:01 +0800
> > Anand Jain <anand.jain@oracle.com> wrote:
> >  
> > > On 10/12/2017 08:47 AM, Joseph Dunn wrote:  
> > > > After seeing how btrfs seeds work I wondered if it was possible to push
> > > > specific files from the seed to the rw device.  I know that removing
> > > > the seed device will flush all the contents over to the rw device, but
> > > > what about flushing individual files on demand?
> > > >
> > > > I found that opening a file, reading the contents, seeking back to 0,
> > > > and writing out the contents does what I want, but I was hoping for a
> > > > bit less of a hack.
> > > >
> > > > Is there maybe an ioctl or something else that might trigger a similar
> > > > action?  
> > >
> > >    You mean to say - seed-device delete to trigger copy of only the
> > > specified or the modified files only, instead of whole of seed-device ?
> > > What's the use case around this ?
> > >  
> >
> > Not quite.  While the seed device is still connected I would like to
> > force some files over to the rw device.  The use case is basically a
> > much slower link to a seed device holding significantly more data than
> > we currently need.  An example would be a slower iscsi link to the seed
> > device and a local rw ssd.  I would like fast access to a certain subset
> > of files, likely larger than the memory cache will accommodate.  If at
> > a later time I want to discard the image as a whole I could unmount the
> > file system or if I want a full local copy I could delete the
> > seed-device to sync the fs.  In the mean time I would have access to
> > all the files, with some slower (iscsi) and some faster (ssd) and the
> > ability to pick which ones are in the faster group at the cost of one
> > content transfer.  
> 
> 
> Multiple seeds?
> 
> Seed A has everything, is remote. Create sprout B also remotely,
> deleting the things you don't absolutely need, then make it a seed.
> Now via iSCSI you can mount both A and B seeds. Add local rw sprout C
> to seed B, then delete B to move files to fast local storage.
> 
Interesting thought.  I haven't tried working with multiple seeds but
I'll see what that can do.  I will say that this approach would require
more pre-planning meaning that the choice of fast files could not be
made based on current access patterns to tasks at hand.  This might
make sense for a core set of files, but it doesn't quite solve the
whole problem.

-Joseph

> 
> 
> -- 
> Chris Murphy

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: btrfs seed question
  2017-10-12 14:44       ` Joseph Dunn
@ 2017-10-12 15:30         ` Chris Murphy
  2017-10-12 15:50           ` Joseph Dunn
  2017-10-12 15:55           ` Austin S. Hemmelgarn
  2017-10-13  2:52         ` Anand Jain
  1 sibling, 2 replies; 11+ messages in thread
From: Chris Murphy @ 2017-10-12 15:30 UTC (permalink / raw)
  To: Joseph Dunn; +Cc: Chris Murphy, Anand Jain, Btrfs BTRFS

On Thu, Oct 12, 2017 at 3:44 PM, Joseph Dunn <jdunn14@gmail.com> wrote:
> On Thu, 12 Oct 2017 15:32:24 +0100
> Chris Murphy <lists@colorremedies.com> wrote:
>
>> On Thu, Oct 12, 2017 at 2:20 PM, Joseph Dunn <jdunn14@gmail.com> wrote:
>> >
>> > On Thu, 12 Oct 2017 12:18:01 +0800
>> > Anand Jain <anand.jain@oracle.com> wrote:
>> >
>> > > On 10/12/2017 08:47 AM, Joseph Dunn wrote:
>> > > > After seeing how btrfs seeds work I wondered if it was possible to push
>> > > > specific files from the seed to the rw device.  I know that removing
>> > > > the seed device will flush all the contents over to the rw device, but
>> > > > what about flushing individual files on demand?
>> > > >
>> > > > I found that opening a file, reading the contents, seeking back to 0,
>> > > > and writing out the contents does what I want, but I was hoping for a
>> > > > bit less of a hack.
>> > > >
>> > > > Is there maybe an ioctl or something else that might trigger a similar
>> > > > action?
>> > >
>> > >    You mean to say - seed-device delete to trigger copy of only the
>> > > specified or the modified files only, instead of whole of seed-device ?
>> > > What's the use case around this ?
>> > >
>> >
>> > Not quite.  While the seed device is still connected I would like to
>> > force some files over to the rw device.  The use case is basically a
>> > much slower link to a seed device holding significantly more data than
>> > we currently need.  An example would be a slower iscsi link to the seed
>> > device and a local rw ssd.  I would like fast access to a certain subset
>> > of files, likely larger than the memory cache will accommodate.  If at
>> > a later time I want to discard the image as a whole I could unmount the
>> > file system or if I want a full local copy I could delete the
>> > seed-device to sync the fs.  In the mean time I would have access to
>> > all the files, with some slower (iscsi) and some faster (ssd) and the
>> > ability to pick which ones are in the faster group at the cost of one
>> > content transfer.
>>
>>
>> Multiple seeds?
>>
>> Seed A has everything, is remote. Create sprout B also remotely,
>> deleting the things you don't absolutely need, then make it a seed.
>> Now via iSCSI you can mount both A and B seeds. Add local rw sprout C
>> to seed B, then delete B to move files to fast local storage.
>>
> Interesting thought.  I haven't tried working with multiple seeds but
> I'll see what that can do.  I will say that this approach would require
> more pre-planning meaning that the choice of fast files could not be
> made based on current access patterns to tasks at hand.  This might
> make sense for a core set of files, but it doesn't quite solve the
> whole problem.


I think the use case really dictates a dynamic solution that's smarter
than either of the proposed ideas (mine or yours). Basically you want
something that recognizes slow vs fast storage, and intelligently
populates fast storage with frequently used files.

Ostensibly this is the realm of dmcache. But I can't tell you whether
dmcache or via LVM tools, if it's possible to set the proper policy to
make it work for your use case. And also I have no idea how to set it
up after the fact, on an already created file system, rather than
block devices.

The hot vs cold files thing, is something I thought the VFS folks were
looking into.


-- 
Chris Murphy

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: btrfs seed question
  2017-10-12 15:30         ` Chris Murphy
@ 2017-10-12 15:50           ` Joseph Dunn
  2017-11-03  8:03             ` Kai Krakow
  2017-10-12 15:55           ` Austin S. Hemmelgarn
  1 sibling, 1 reply; 11+ messages in thread
From: Joseph Dunn @ 2017-10-12 15:50 UTC (permalink / raw)
  To: Chris Murphy; +Cc: Anand Jain, Btrfs BTRFS

On Thu, 12 Oct 2017 16:30:36 +0100
Chris Murphy <lists@colorremedies.com> wrote:

> On Thu, Oct 12, 2017 at 3:44 PM, Joseph Dunn <jdunn14@gmail.com> wrote:
> > On Thu, 12 Oct 2017 15:32:24 +0100
> > Chris Murphy <lists@colorremedies.com> wrote:
> >  
> >> On Thu, Oct 12, 2017 at 2:20 PM, Joseph Dunn <jdunn14@gmail.com> wrote:  
> >> >
> >> > On Thu, 12 Oct 2017 12:18:01 +0800
> >> > Anand Jain <anand.jain@oracle.com> wrote:
> >> >  
> >> > > On 10/12/2017 08:47 AM, Joseph Dunn wrote:  
> >> > > > After seeing how btrfs seeds work I wondered if it was possible to push
> >> > > > specific files from the seed to the rw device.  I know that removing
> >> > > > the seed device will flush all the contents over to the rw device, but
> >> > > > what about flushing individual files on demand?
> >> > > >
> >> > > > I found that opening a file, reading the contents, seeking back to 0,
> >> > > > and writing out the contents does what I want, but I was hoping for a
> >> > > > bit less of a hack.
> >> > > >
> >> > > > Is there maybe an ioctl or something else that might trigger a similar
> >> > > > action?  
> >> > >
> >> > >    You mean to say - seed-device delete to trigger copy of only the
> >> > > specified or the modified files only, instead of whole of seed-device ?
> >> > > What's the use case around this ?
> >> > >  
> >> >
> >> > Not quite.  While the seed device is still connected I would like to
> >> > force some files over to the rw device.  The use case is basically a
> >> > much slower link to a seed device holding significantly more data than
> >> > we currently need.  An example would be a slower iscsi link to the seed
> >> > device and a local rw ssd.  I would like fast access to a certain subset
> >> > of files, likely larger than the memory cache will accommodate.  If at
> >> > a later time I want to discard the image as a whole I could unmount the
> >> > file system or if I want a full local copy I could delete the
> >> > seed-device to sync the fs.  In the mean time I would have access to
> >> > all the files, with some slower (iscsi) and some faster (ssd) and the
> >> > ability to pick which ones are in the faster group at the cost of one
> >> > content transfer.  
> >>
> >>
> >> Multiple seeds?
> >>
> >> Seed A has everything, is remote. Create sprout B also remotely,
> >> deleting the things you don't absolutely need, then make it a seed.
> >> Now via iSCSI you can mount both A and B seeds. Add local rw sprout C
> >> to seed B, then delete B to move files to fast local storage.
> >>  
> > Interesting thought.  I haven't tried working with multiple seeds but
> > I'll see what that can do.  I will say that this approach would require
> > more pre-planning meaning that the choice of fast files could not be
> > made based on current access patterns to tasks at hand.  This might
> > make sense for a core set of files, but it doesn't quite solve the
> > whole problem.  
> 
> 
> I think the use case really dictates a dynamic solution that's smarter
> than either of the proposed ideas (mine or yours). Basically you want
> something that recognizes slow vs fast storage, and intelligently
> populates fast storage with frequently used files.
> 
> Ostensibly this is the realm of dmcache. But I can't tell you whether
> dmcache or via LVM tools, if it's possible to set the proper policy to
> make it work for your use case. And also I have no idea how to set it
> up after the fact, on an already created file system, rather than
> block devices.
> 
> The hot vs cold files thing, is something I thought the VFS folks were
> looking into.
> 
As a consumer of the file system data I tend to see things at a file
level rather than as blocks, but from a block level this does feel
dmcache-ish and I'll look into it.

I did try the multiple seeds approach and correct me if I'm wrong, but
once the files are deleted in the second seed they are no longer
accessible in anything sprouted from that.  The fully dynamic solution
would be nice, but I'm perfectly happy to pick the files for the ssd at
run time, just not at file system preparation time.  In any case, I may
fall back on inotify and overwriting file contents if I don't end up
with a better solution using dmcache or LVM tricks.

-Joseph Dunn

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: btrfs seed question
  2017-10-12 15:30         ` Chris Murphy
  2017-10-12 15:50           ` Joseph Dunn
@ 2017-10-12 15:55           ` Austin S. Hemmelgarn
  1 sibling, 0 replies; 11+ messages in thread
From: Austin S. Hemmelgarn @ 2017-10-12 15:55 UTC (permalink / raw)
  To: Chris Murphy, Joseph Dunn; +Cc: Anand Jain, Btrfs BTRFS

On 2017-10-12 11:30, Chris Murphy wrote:
> On Thu, Oct 12, 2017 at 3:44 PM, Joseph Dunn <jdunn14@gmail.com> wrote:
>> On Thu, 12 Oct 2017 15:32:24 +0100
>> Chris Murphy <lists@colorremedies.com> wrote:
>>
>>> On Thu, Oct 12, 2017 at 2:20 PM, Joseph Dunn <jdunn14@gmail.com> wrote:
>>>>
>>>> On Thu, 12 Oct 2017 12:18:01 +0800
>>>> Anand Jain <anand.jain@oracle.com> wrote:
>>>>
>>>>> On 10/12/2017 08:47 AM, Joseph Dunn wrote:
>>>>>> After seeing how btrfs seeds work I wondered if it was possible to push
>>>>>> specific files from the seed to the rw device.  I know that removing
>>>>>> the seed device will flush all the contents over to the rw device, but
>>>>>> what about flushing individual files on demand?
>>>>>>
>>>>>> I found that opening a file, reading the contents, seeking back to 0,
>>>>>> and writing out the contents does what I want, but I was hoping for a
>>>>>> bit less of a hack.
>>>>>>
>>>>>> Is there maybe an ioctl or something else that might trigger a similar
>>>>>> action?
>>>>>
>>>>>     You mean to say - seed-device delete to trigger copy of only the
>>>>> specified or the modified files only, instead of whole of seed-device ?
>>>>> What's the use case around this ?
>>>>>
>>>>
>>>> Not quite.  While the seed device is still connected I would like to
>>>> force some files over to the rw device.  The use case is basically a
>>>> much slower link to a seed device holding significantly more data than
>>>> we currently need.  An example would be a slower iscsi link to the seed
>>>> device and a local rw ssd.  I would like fast access to a certain subset
>>>> of files, likely larger than the memory cache will accommodate.  If at
>>>> a later time I want to discard the image as a whole I could unmount the
>>>> file system or if I want a full local copy I could delete the
>>>> seed-device to sync the fs.  In the mean time I would have access to
>>>> all the files, with some slower (iscsi) and some faster (ssd) and the
>>>> ability to pick which ones are in the faster group at the cost of one
>>>> content transfer.
>>>
>>>
>>> Multiple seeds?
>>>
>>> Seed A has everything, is remote. Create sprout B also remotely,
>>> deleting the things you don't absolutely need, then make it a seed.
>>> Now via iSCSI you can mount both A and B seeds. Add local rw sprout C
>>> to seed B, then delete B to move files to fast local storage.
>>>
>> Interesting thought.  I haven't tried working with multiple seeds but
>> I'll see what that can do.  I will say that this approach would require
>> more pre-planning meaning that the choice of fast files could not be
>> made based on current access patterns to tasks at hand.  This might
>> make sense for a core set of files, but it doesn't quite solve the
>> whole problem.
> 
> 
> I think the use case really dictates a dynamic solution that's smarter
> than either of the proposed ideas (mine or yours). Basically you want
> something that recognizes slow vs fast storage, and intelligently
> populates fast storage with frequently used files.
> 
> Ostensibly this is the realm of dmcache. But I can't tell you whether
> dmcache or via LVM tools, if it's possible to set the proper policy to
> make it work for your use case. And also I have no idea how to set it
> up after the fact, on an already created file system, rather than
> block devices.
It is possible with dm-cache, but not at the file level (as befits any 
good block layer, it doesn't care about files, just blocks).  Despite 
that, it should work reasonably fine (I've done this before with NBD and 
ATAoE devices, and it worked perfectly, so I would expect it to work 
just as well for iSCSI), and actually may do better than caching whole 
files locally depending on your workload.

As far as setup after the fact, the degree of difficulty depends on 
whether or not you want to use LVM.  Without LVM, you should have no 
issue just setting up a device-mapper table for caching, you just need 
enough room on the local SSD for both the cache data and cache metadata 
partition.  When you create the table using dmsetup 9and eventually in 
/etc/dmtab), it won't wipe the filesystem on the origin device, so you 
can easily add a cache to anything this way, even if it's an existing 
filesystem (though you will need to unmount the filesystem to add the 
cache).  All things considered, it's no worse than setting it up on a 
brand new device, the hardest part is making sure you get the device 
sizes right for the device mapper table.

With LVM however, it's somewhat more complicated, because it refuses to 
work with anything it's not managing already, so you would have to 
reprovision the iSCSI device as a PV, add it to the local VG, and then 
work with that.

If you don't mind reprovisioning, I would actually suggest bcache 
instead of LVM here though.  It's less complicated to add and remove 
caches, does somewhat better in recent versions of intelligently 
deciding what to cache locally, and is also significantly less likely to 
slow down the rest of your system than LVM (any management operations 
will have to hit that remote iSCSI device, and LVM does a lot more with 
data on the disk than bcache does).
> 
> The hot vs cold files thing, is something I thought the VFS folks were
> looking into.
I was under that impression too, but I haven't seen anything relating to 
it recently (though I'm not subscribed to the linux-vfs list, so there 
may be discussion there I'm not seeing).

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: btrfs seed question
  2017-10-12 14:44       ` Joseph Dunn
  2017-10-12 15:30         ` Chris Murphy
@ 2017-10-13  2:52         ` Anand Jain
  1 sibling, 0 replies; 11+ messages in thread
From: Anand Jain @ 2017-10-13  2:52 UTC (permalink / raw)
  To: Joseph Dunn, Chris Murphy; +Cc: Btrfs BTRFS



>>> Not quite.  While the seed device is still connected I would like to
>>> force some files over to the rw device.  The use case is basically a
>>> much slower link to a seed device holding significantly more data than
>>> we currently need.  An example would be a slower iscsi link to the seed
>>> device and a local rw ssd.  I would like fast access to a certain subset
>>> of files, likely larger than the memory cache will accommodate. >>> If at a later time I want to discard the image as a whole I could
 >>> unmount the file system or


   Interesting. If you had not brought the idea of using seed device,
   I would have still thinking about bcache device as a solution for
   the above use case.

   However bcache device won't help to solve the further use case
   requirements as below..


>>> if I want a full local copy I could delete the
>>> seed-device to sync the fs.  In the mean time I would have access to
>>> all the files, with some slower (iscsi) and some faster (ssd) and the
>>> ability to pick which ones are in the faster group at the cost of one
>>> content transfer.

   I am thinking why not indeed leverage seed/sprout for this.
   Let me see how I (or if anyone could) allocate time to experiment
   on this.

  Thanks for your use case !

-Anand

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: btrfs seed question
  2017-10-12 13:20   ` Joseph Dunn
  2017-10-12 14:32     ` Chris Murphy
@ 2017-11-03  7:56     ` Kai Krakow
  1 sibling, 0 replies; 11+ messages in thread
From: Kai Krakow @ 2017-11-03  7:56 UTC (permalink / raw)
  To: linux-btrfs

Am Thu, 12 Oct 2017 09:20:28 -0400
schrieb Joseph Dunn <jdunn14@gmail.com>:

> On Thu, 12 Oct 2017 12:18:01 +0800
> Anand Jain <anand.jain@oracle.com> wrote:
> 
> > On 10/12/2017 08:47 AM, Joseph Dunn wrote:  
> > > After seeing how btrfs seeds work I wondered if it was possible
> > > to push specific files from the seed to the rw device.  I know
> > > that removing the seed device will flush all the contents over to
> > > the rw device, but what about flushing individual files on demand?
> > > 
> > > I found that opening a file, reading the contents, seeking back
> > > to 0, and writing out the contents does what I want, but I was
> > > hoping for a bit less of a hack.
> > > 
> > > Is there maybe an ioctl or something else that might trigger a
> > > similar action?    
> > 
> >    You mean to say - seed-device delete to trigger copy of only the 
> > specified or the modified files only, instead of whole of
> > seed-device ? What's the use case around this ?
> >   
> 
> Not quite.  While the seed device is still connected I would like to
> force some files over to the rw device.  The use case is basically a
> much slower link to a seed device holding significantly more data than
> we currently need.  An example would be a slower iscsi link to the
> seed device and a local rw ssd.  I would like fast access to a
> certain subset of files, likely larger than the memory cache will
> accommodate.  If at a later time I want to discard the image as a
> whole I could unmount the file system or if I want a full local copy
> I could delete the seed-device to sync the fs.  In the mean time I
> would have access to all the files, with some slower (iscsi) and some
> faster (ssd) and the ability to pick which ones are in the faster
> group at the cost of one content transfer.
> 
> I'm not necessarily looking for a new feature addition, just if there
> is some existing call that I can make to push specific files from the
> slow mirror to the fast one.  If I had to push a significant amount of
> metadata that would be fine, but the file contents feeding some
> computations might be large and useful only to certain clients.
> 
> So far I found that I can re-write the file with the same contents and
> thanks to the lack of online dedupe these writes land on the rw mirror
> so later reads to that file should not hit the slower mirror.  By the
> way, if I'm misunderstanding how the read process would work after the
> file push please correct me.
> 
> I hope this makes sense but I'll try to clarify further if you have
> more questions.

You could try to wrap something like bcache ontop of the iscsi device,
then make it a read-mostly cache (like bcache write-around mode). This
probably involves rewriting the iscsi contents to add a bcache header.
You could try mdcache instead.

Then you sacrifice a few gigabytes of local SSD storage of the caching
layer.

I guess that you're sharing the seed device with different machines. As
bcache will add a protective superblock, you may need to thin-clone the
seed image on the source to have independent superblocks per each
bcache instance. Not sure how this applies to mdcache as I never used
it.

But the caching approach is probably the easiest way to go for you. And
it's mostly automatic once deployed: you don't have to manually
choose which files to move to the sprout...


-- 
Regards,
Kai

Replies to list-only preferred.


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: btrfs seed question
  2017-10-12 15:50           ` Joseph Dunn
@ 2017-11-03  8:03             ` Kai Krakow
  0 siblings, 0 replies; 11+ messages in thread
From: Kai Krakow @ 2017-11-03  8:03 UTC (permalink / raw)
  To: linux-btrfs

Am Thu, 12 Oct 2017 11:50:20 -0400
schrieb Joseph Dunn <jdunn14@gmail.com>:

> On Thu, 12 Oct 2017 16:30:36 +0100
> Chris Murphy <lists@colorremedies.com> wrote:
> 
> > On Thu, Oct 12, 2017 at 3:44 PM, Joseph Dunn <jdunn14@gmail.com>
> > wrote:  
> > > On Thu, 12 Oct 2017 15:32:24 +0100
> > > Chris Murphy <lists@colorremedies.com> wrote:
> > >    
>  [...]  
>  [...]  
>  [...]  
>  [...]  
>  [...]  
>  [...]  
>  [...]  
> > > Interesting thought.  I haven't tried working with multiple seeds
> > > but I'll see what that can do.  I will say that this approach
> > > would require more pre-planning meaning that the choice of fast
> > > files could not be made based on current access patterns to tasks
> > > at hand.  This might make sense for a core set of files, but it
> > > doesn't quite solve the whole problem.    
> > 
> > 
> > I think the use case really dictates a dynamic solution that's
> > smarter than either of the proposed ideas (mine or yours).
> > Basically you want something that recognizes slow vs fast storage,
> > and intelligently populates fast storage with frequently used files.
> > 
> > Ostensibly this is the realm of dmcache. But I can't tell you
> > whether dmcache or via LVM tools, if it's possible to set the
> > proper policy to make it work for your use case. And also I have no
> > idea how to set it up after the fact, on an already created file
> > system, rather than block devices.
> > 
> > The hot vs cold files thing, is something I thought the VFS folks
> > were looking into.
> >   
> As a consumer of the file system data I tend to see things at a file
> level rather than as blocks, but from a block level this does feel
> dmcache-ish and I'll look into it.
> 
> I did try the multiple seeds approach and correct me if I'm wrong, but
> once the files are deleted in the second seed they are no longer
> accessible in anything sprouted from that.  The fully dynamic solution
> would be nice, but I'm perfectly happy to pick the files for the ssd
> at run time, just not at file system preparation time.  In any case,
> I may fall back on inotify and overwriting file contents if I don't
> end up with a better solution using dmcache or LVM tricks.

Would "btrfs filesystem defrag" not rewrite files to local storage?

Using inotify, you could work with two queues "defrag" and "done". Add
files via inotify to the defrag queue unless it's already in one of the
queues. Then move it to the done queue while running defrag on each
files of the other queue.

You can then use the done queue as a seed for defrag queue on newly
provisioned systems.


-- 
Regards,
Kai

Replies to list-only preferred.


^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2017-11-03  8:04 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-10-12  0:47 btrfs seed question Joseph Dunn
2017-10-12  4:18 ` Anand Jain
2017-10-12 13:20   ` Joseph Dunn
2017-10-12 14:32     ` Chris Murphy
2017-10-12 14:44       ` Joseph Dunn
2017-10-12 15:30         ` Chris Murphy
2017-10-12 15:50           ` Joseph Dunn
2017-11-03  8:03             ` Kai Krakow
2017-10-12 15:55           ` Austin S. Hemmelgarn
2017-10-13  2:52         ` Anand Jain
2017-11-03  7:56     ` Kai Krakow

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).