All of lore.kernel.org
 help / color / mirror / Atom feed
From: Greg Freemyer <greg.freemyer@gmail.com>
To: Roland Dreier <rdreier@cisco.com>
Cc: Richard Sharpe <realrichardsharpe@gmail.com>,
	david@lang.hm, Markus Trippelsdorf <markus@trippelsdorf.de>,
	Matthew Wilcox <willy@linux.intel.com>,
	Hugh Dickins <hugh.dickins@tiscali.co.uk>,
	Nitin Gupta <ngupta@vflare.org>, Ingo Molnar <mingo@elte.hu>,
	Peter Zijlstra <peterz@infradead.org>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	linux-scsi@vger.kernel.org, linux-ide@vger.kernel.org,
	Linux RAID <linux-raid@vger.kernel.org>
Subject: Re: Discard support
Date: Fri, 14 Aug 2009 18:10:50 -0400	[thread overview]
Message-ID: <87f94c370908141510p1752183elf32c879c0510ebc4@mail.gmail.com> (raw)
In-Reply-To: <adafxbu3vqt.fsf@cisco.com>

On Fri, Aug 14, 2009 at 5:56 PM, Roland Dreier<rdreier@cisco.com> wrote:
>
>  > It seems to me that unmap is not all that different, why do we need to
>  > do it even close in time proximity to the deletes?  With a bitmap, we
>  > have total timing control of when the unmaps are forwarded down to the
>  > device.  I like that timing control much better than a cache and
>  > coalesce approach.
>
> The trouble I see with a bitmap is the amount of memory it consumes.  It
> seems that discards must be tracked on no bigger than 4KB sectors (and
> possibly even 512 byte sectors).  But even with 4KB, then, say, a 32 TB
> volume (just 16 * 2TB disks, or even lower end with thin provisioning)
> requires 1 GB of bitmap memory.  Which is a lot just to store, let alone
> walk over etc.

Have the filesystem guys created any efficient extent tree tracking solutions?

I mean a 16TB filesystem obviously has to track the freespace somehow
that does not require 1GB of ram.  Can that logic be leveraged in
block to track freespace?  That obviously assumes its not too cpu
intensive to do so.

If a leaf in the extent tracking tree becomes big enough, it could
even be sent down from the block layer and that leaf deleted.  ie. If
a leaf of the tree grows to represent X contiguous blocks, then a
discard could be sent down to the device and the leaf representing
those free blocks deleted.

The new topo info about block devices might be able to help optimize
the minimum size of a coalesced discard.

Greg
--
Greg Freemyer

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Greg Freemyer <greg.freemyer@gmail.com>
To: Roland Dreier <rdreier@cisco.com>
Cc: Richard Sharpe <realrichardsharpe@gmail.com>,
	david@lang.hm, Markus Trippelsdorf <markus@trippelsdorf.de>,
	Matthew Wilcox <willy@linux.intel.com>,
	Hugh Dickins <hugh.dickins@tiscali.co.uk>,
	Nitin Gupta <ngupta@vflare.org>, Ingo Molnar <mingo@elte.hu>,
	Peter Zijlstra <peterz@infradead.org>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	linux-scsi@vger.kernel.org, linux-ide@vger.kernel.org,
	Linux RAID <linux-raid@vger.kernel.org>
Subject: Re: Discard support
Date: Fri, 14 Aug 2009 18:10:50 -0400	[thread overview]
Message-ID: <87f94c370908141510p1752183elf32c879c0510ebc4@mail.gmail.com> (raw)
In-Reply-To: <adafxbu3vqt.fsf@cisco.com>

On Fri, Aug 14, 2009 at 5:56 PM, Roland Dreier<rdreier@cisco.com> wrote:
>
>  > It seems to me that unmap is not all that different, why do we need to
>  > do it even close in time proximity to the deletes?  With a bitmap, we
>  > have total timing control of when the unmaps are forwarded down to the
>  > device.  I like that timing control much better than a cache and
>  > coalesce approach.
>
> The trouble I see with a bitmap is the amount of memory it consumes.  It
> seems that discards must be tracked on no bigger than 4KB sectors (and
> possibly even 512 byte sectors).  But even with 4KB, then, say, a 32 TB
> volume (just 16 * 2TB disks, or even lower end with thin provisioning)
> requires 1 GB of bitmap memory.  Which is a lot just to store, let alone
> walk over etc.

Have the filesystem guys created any efficient extent tree tracking solutions?

I mean a 16TB filesystem obviously has to track the freespace somehow
that does not require 1GB of ram.  Can that logic be leveraged in
block to track freespace?  That obviously assumes its not too cpu
intensive to do so.

If a leaf in the extent tracking tree becomes big enough, it could
even be sent down from the block layer and that leaf deleted.  ie. If
a leaf of the tree grows to represent X contiguous blocks, then a
discard could be sent down to the device and the leaf representing
those free blocks deleted.

The new topo info about block devices might be able to help optimize
the minimum size of a coalesced discard.

Greg
--
Greg Freemyer

  reply	other threads:[~2009-08-14 22:10 UTC|newest]

Thread overview: 207+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2009-08-12 14:37 [PATCH] swap: send callback when swap slot is freed Nitin Gupta
2009-08-12 14:37 ` Nitin Gupta
2009-08-12 18:33 ` Peter Zijlstra
2009-08-12 22:13 ` Andrew Morton
2009-08-12 22:48 ` Hugh Dickins
2009-08-12 22:48   ` Hugh Dickins
2009-08-13  2:30   ` Nitin Gupta
2009-08-13  2:30     ` Nitin Gupta
2009-08-13  6:53     ` Peter Zijlstra
2009-08-13  6:53       ` Peter Zijlstra
2009-08-13 14:44       ` Nitin Gupta
2009-08-13 14:44         ` Nitin Gupta
2009-08-13 17:45     ` Hugh Dickins
2009-08-13 17:45       ` Hugh Dickins
2009-08-13  2:41   ` Nitin Gupta
2009-08-13  2:41     ` Nitin Gupta
2009-08-13  5:05     ` compcache as a pre-swap area (was: [PATCH] swap: send callback when swap slot is freed) Al Boldi
2009-08-13  5:05       ` Al Boldi
2009-08-13 17:31       ` Nitin Gupta
2009-08-13 17:31         ` Nitin Gupta
2009-08-14  4:02         ` Al Boldi
2009-08-14  4:02           ` Al Boldi
2009-08-14  4:53           ` compcache as a pre-swap area Nitin Gupta
2009-08-14  4:53             ` Nitin Gupta
2009-08-14 15:49             ` Al Boldi
2009-08-14 15:49               ` Al Boldi
2009-08-15 11:00               ` Al Boldi
2009-08-15 11:00                 ` Al Boldi
2009-08-13 15:13   ` Discard support (was Re: [PATCH] swap: send callback when swap slot is freed) Matthew Wilcox
2009-08-13 15:13     ` Matthew Wilcox
2009-08-13 15:17     ` david
2009-08-13 15:17       ` david
2009-08-13 15:26       ` Matthew Wilcox
2009-08-13 15:26         ` Matthew Wilcox
2009-08-13 15:43     ` James Bottomley
2009-08-13 15:43       ` James Bottomley
2009-08-13 18:22       ` Ric Wheeler
2009-08-13 18:22         ` Ric Wheeler
2009-08-13 16:13     ` Nitin Gupta
2009-08-13 16:13       ` Nitin Gupta
2009-08-13 16:26     ` Markus Trippelsdorf
2009-08-13 16:26       ` Markus Trippelsdorf
2009-08-13 16:33       ` david
2009-08-13 16:33         ` david
2009-08-13 18:15         ` Greg Freemyer
2009-08-13 18:15           ` Greg Freemyer
2009-08-13 18:15           ` Greg Freemyer
2009-08-13 19:18           ` James Bottomley
2009-08-13 19:18             ` James Bottomley
2009-08-13 20:31             ` Richard Sharpe
2009-08-13 20:31               ` Richard Sharpe
2009-08-13 20:31               ` Richard Sharpe
2009-08-14 22:03             ` Mark Lord
2009-08-14 22:03               ` Mark Lord
2009-08-14 22:54               ` Greg Freemyer
2009-08-14 22:54                 ` Greg Freemyer
2009-08-15 13:12                 ` Mark Lord
2009-08-15 13:12                   ` Mark Lord
2009-08-13 20:44           ` david
2009-08-13 20:44             ` david
2009-08-13 20:54             ` Bryan Donlan
2009-08-13 20:54               ` Bryan Donlan
2009-08-14 22:10               ` Mark Lord
2009-08-14 22:10                 ` Mark Lord
2009-08-14 23:21                 ` Chris Worley
2009-08-14 23:21                   ` Chris Worley
2009-08-14 23:45                   ` Matthew Wilcox
2009-08-14 23:45                     ` Matthew Wilcox
2009-08-15  0:19                     ` Chris Worley
2009-08-15  0:19                       ` Chris Worley
2009-08-15  0:30                       ` Greg Freemyer
2009-08-15  0:30                         ` Greg Freemyer
2009-08-15  0:38                         ` Chris Worley
2009-08-15  0:38                           ` Chris Worley
2009-08-15  0:38                           ` Chris Worley
2009-08-15  1:55                           ` Greg Freemyer
2009-08-15  1:55                             ` Greg Freemyer
2009-08-15  1:55                             ` Greg Freemyer
2009-08-15 13:20                           ` Mark Lord
2009-08-15 13:20                             ` Mark Lord
2009-08-16 22:52                             ` Chris Worley
2009-08-16 22:52                               ` Chris Worley
2009-08-17  2:03                               ` Mark Lord
2009-08-17  2:03                                 ` Mark Lord
2009-08-15 12:59                       ` James Bottomley
2009-08-15 12:59                         ` James Bottomley
2009-08-15 13:22                         ` Mark Lord
2009-08-15 13:22                           ` Mark Lord
2009-08-15 13:55                           ` James Bottomley
2009-08-15 13:55                             ` James Bottomley
2009-08-15 17:39                             ` jim owens
2009-08-15 17:39                               ` jim owens
2009-08-16 17:08                               ` Robert Hancock
2009-08-16 17:08                                 ` Robert Hancock
2009-08-16 14:05                             ` Alan Cox
2009-08-16 14:05                               ` Alan Cox
2009-08-16 14:16                               ` Mark Lord
2009-08-16 14:16                                 ` Mark Lord
2009-08-16 15:34                               ` Arjan van de Ven
2009-08-16 15:34                                 ` Arjan van de Ven
2009-08-16 15:44                                 ` Theodore Tso
2009-08-16 15:44                                   ` Theodore Tso
2009-08-16 17:28                                   ` Mark Lord
2009-08-16 17:28                                   ` Mark Lord
2009-08-16 17:28                                     ` Mark Lord
2009-08-16 17:37                                     ` Mark Lord
2009-08-16 17:37                                     ` Mark Lord
2009-08-16 17:37                                       ` Mark Lord
2009-08-16 17:37                                       ` Mark Lord
2009-08-17 16:30                                       ` Bill Davidsen
2009-08-17 16:30                                         ` Bill Davidsen
2009-08-17 16:56                                         ` jim owens
2009-08-17 16:56                                           ` jim owens
2009-08-17 17:14                                           ` Bill Davidsen
2009-08-17 17:14                                             ` Bill Davidsen
2009-08-17 17:37                                             ` jim owens
2009-08-17 17:37                                               ` jim owens
2009-08-16 17:37                                     ` Mark Lord
2009-08-16 17:37                                     ` Mark Lord
2009-08-16 17:28                                   ` Mark Lord
2009-08-16 17:28                                   ` Mark Lord
2009-08-16 15:52                                 ` James Bottomley
2009-08-16 15:52                                   ` James Bottomley
2009-08-16 16:32                                   ` Mark Lord
2009-08-16 16:32                                     ` Mark Lord
2009-08-16 16:32                                     ` Mark Lord
2009-08-16 18:07                                     ` James Bottomley
2009-08-16 18:07                                       ` James Bottomley
2009-08-16 18:19                                       ` Mark Lord
2009-08-16 18:19                                         ` Mark Lord
2009-08-16 18:19                                         ` Mark Lord
2009-08-16 18:24                                         ` James Bottomley
2009-08-16 18:24                                           ` James Bottomley
2009-08-17 16:37                                           ` Bill Davidsen
2009-08-17 16:37                                             ` Bill Davidsen
2009-08-17 16:37                                             ` Bill Davidsen
2009-08-17 17:08                                             ` Greg Freemyer
2009-08-17 17:08                                               ` Greg Freemyer
2009-08-17 17:19                                               ` James Bottomley
2009-08-17 17:19                                                 ` James Bottomley
2009-08-17 18:16                                                 ` Ric Wheeler
2009-08-17 18:16                                                   ` Ric Wheeler
2009-08-17 18:21                                                 ` Greg Freemyer
2009-08-17 18:21                                                   ` Greg Freemyer
2009-08-17 18:21                                                   ` Greg Freemyer
2009-08-17 19:18                                                   ` James Bottomley
2009-08-17 19:18                                                     ` James Bottomley
2009-08-17 20:19                                                     ` Mark Lord
2009-08-17 20:19                                                       ` Mark Lord
2009-08-17 20:28                                                       ` James Bottomley
2009-08-17 20:28                                                         ` James Bottomley
2009-08-17 20:28                                               ` Mark Lord
2009-08-17 20:28                                                 ` Mark Lord
2009-08-16 16:59                                   ` Christoph Hellwig
2009-08-16 16:59                                     ` Christoph Hellwig
2009-08-17  4:24                                     ` Douglas Gilbert
2009-08-17  4:24                                       ` Douglas Gilbert
2009-08-17 13:56                                     ` James Bottomley
2009-08-17 13:56                                       ` James Bottomley
2009-08-17 14:10                                       ` Matthew Wilcox
2009-08-17 14:10                                         ` Matthew Wilcox
2009-08-17 19:12                                         ` Christoph Hellwig
2009-08-17 19:12                                           ` Christoph Hellwig
2009-08-17 19:24                                           ` James Bottomley
2009-08-17 19:24                                             ` James Bottomley
2009-08-16 21:50                                   ` Discard support Roland Dreier
2009-08-16 21:50                                     ` Roland Dreier
2009-08-16 22:06                                     ` Jeff Garzik
2009-08-16 22:06                                       ` Jeff Garzik
2009-08-16 22:06                                       ` Jeff Garzik
2009-08-16 22:13                                     ` Theodore Tso
2009-08-16 22:13                                       ` Theodore Tso
2009-08-16 22:51                                       ` Mark Lord
2009-08-16 22:51                                         ` Mark Lord
2009-08-16 22:51                                       ` Mark Lord
2009-08-16 22:51                                       ` Mark Lord
2009-08-16 22:51                                       ` Mark Lord
2009-08-16 19:29                                 ` Discard support (was Re: [PATCH] swap: send callback when swap slot is freed) Alan Cox
2009-08-16 19:29                                   ` Alan Cox
2009-08-16 23:05                                   ` John Robinson
2009-08-16 23:05                                     ` John Robinson
2009-08-17  2:05                                     ` Mark Lord
2009-08-17  2:05                                       ` Mark Lord
2009-08-13 21:28             ` Greg Freemyer
2009-08-13 21:28               ` Greg Freemyer
2009-08-13 22:20               ` Richard Sharpe
2009-08-13 22:20                 ` Richard Sharpe
2009-08-13 22:20                 ` Richard Sharpe
2009-08-14  0:19                 ` Greg Freemyer
2009-08-14  0:19                   ` Greg Freemyer
     [not found]                   ` <46b8a8850908131758s781b07f6v2729483c0e50ae7a@mail.gmail.com>
2009-08-14 21:33                     ` Greg Freemyer
2009-08-14 21:33                       ` Greg Freemyer
2009-08-14 21:56                       ` Discard support Roland Dreier
2009-08-14 21:56                         ` Roland Dreier
2009-08-14 22:10                         ` Greg Freemyer [this message]
2009-08-14 22:10                           ` Greg Freemyer
2009-08-14 21:33                     ` Discard support (was Re: [PATCH] swap: send callback when swap slot is freed) Greg Freemyer
2009-08-14 21:33                     ` Greg Freemyer
2009-08-14 21:33                     ` Greg Freemyer
2009-08-13 17:19     ` Hugh Dickins
2009-08-13 17:19       ` Hugh Dickins
2009-08-13 18:08     ` Douglas Gilbert
2009-08-13 18:08       ` Douglas Gilbert
2009-08-17  2:55 ` [PATCH] swap: send callback when swap slot is freed KAMEZAWA Hiroyuki
2009-08-17  5:08   ` Nitin Gupta
2009-08-17  5:11     ` KAMEZAWA Hiroyuki
2009-08-22  7:34   ` Nai Xia

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87f94c370908141510p1752183elf32c879c0510ebc4@mail.gmail.com \
    --to=greg.freemyer@gmail.com \
    --cc=david@lang.hm \
    --cc=hugh.dickins@tiscali.co.uk \
    --cc=linux-ide@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=linux-raid@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=markus@trippelsdorf.de \
    --cc=mingo@elte.hu \
    --cc=ngupta@vflare.org \
    --cc=peterz@infradead.org \
    --cc=rdreier@cisco.com \
    --cc=realrichardsharpe@gmail.com \
    --cc=willy@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.