All of lore.kernel.org
 help / color / mirror / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: Jan Kara <jack@suse.cz>
Cc: Andreas Dilger <adilger@clusterfs.com>,
	sho@tnes.nec.co.jp, linux-ext4@vger.kernel.org,
	linux-fsdevel@vger.kernel.org
Subject: Re: [RFC][PATCH 2/3] Move the file data to the new blocks
Date: Thu, 8 Feb 2007 01:45:29 -0800	[thread overview]
Message-ID: <20070208014529.d990b502.akpm@linux-foundation.org> (raw)
In-Reply-To: <20070208092945.GA10973@duck.suse.cz>

On Thu, 8 Feb 2007 10:29:45 +0100 Jan Kara <jack@suse.cz> wrote:

> On Wed 07-02-07 12:56:59, Andrew Morton wrote:
> > On Wed, 7 Feb 2007 13:46:57 -0700
> > Andreas Dilger <adilger@clusterfs.com> wrote:
> > 
> > > On Feb 06, 2007  17:35 -0800, Andrew Morton wrote:
> > > > On Mon, 5 Feb 2007 14:12:04 +0100
> > > > Jan Kara <jack@suse.cz> wrote:
> > > > > > Move the blocks on the temporary inode to the original inode
> > > > > > by a page.
> > > > > > 1. Read the file data from the old blocks to the page
> > > > > > 2. Move the block on the temporary inode to the original inode
> > > > > > 3. Write the file data on the page into the new blocks
> > > > >   I have one thing - it's probably not good to use page cache for
> > > > > defragmentation.
> > > > 
> > > > Then it is no longer online defragmentation.  The issues with maintaining
> > > > correctness and coherency with ongoing VFS activity would be truly ghastly.
> > > > 
> > > > If we're worried about pagecache pollution then it would be better to control
> > > > that from userspace via fadvise().
> > > 
> > > It should be possible to have the online defrag tool lock the inode against
> > > any changes,
> > 
> > Sounds easy when you say it fast.  But how do we "lock" against, say, a
> > read pagefault?  Only by writing back then removing the pagecache page then
> > reinstantiating it as a locked, not-uptodate page and then removing it from
> > pagecache afterwards prior to unlocking it.  Or something.
> > 
> > I don't think we want to go there.
>   I though Andreas meant "any write changes" - i.e. you check that noone
> has open file descriptor for writing and block any new open for writing.
> That can be done quite easily.
>   Anyway, I agree with you that userspace solution to a possible page
> cache pollution is preferable after thinking about it for a while.
> As I've been thinking about it, we could actually do the copying
> from user space. We could do something like:
>   block any writes to file (as I described above)
>   craft new inode with blocks allocated as we want (using preallocation,
>     we should mostly have the kernel infrastructure we need)
>   copy data using splice syscall
>   call the kernel to switch data
> 

I don't think we need to block any writes to any file or anything.

To move a page within a file:

	fd = open(file);
	p = mmap(fd);
	the_page_was_in_core = mincore(p, offset);
	munmap(p);
	ioctl(fd, ..., new_block);

			<kernel>
			read_cache_page(inode, offset);
			lock_page(page);
			if (try_to_free_buffers(page)) {
				<relocate the page>
				set_page_dirty(page);
			}
			unlock_page(page);

	if (the_page_was_in_core) {
		sync_file_range(fd, offset SYNC_FILE_RANGE_WAIT_BEFORE|
						SYNC_FILE_RANGE_WRITE|
						SYNC_FILE_RANGE_WAIT_AFTER);
		fadvise(fd, offset, FADV_DONTNEED);
	}

completely coherent with pagecache, quite safe in the presence of mmap,
mlock, O_DIRECT, everything else.  Also fully journallable in-kernel.


  reply	other threads:[~2007-02-08  9:46 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2007-01-16 12:05 [RFC][PATCH 2/3] Move the file data to the new blocks sho
2007-02-05 13:12 ` Jan Kara
2007-02-05 22:06   ` Nathan Scott
2007-02-07  1:35   ` Andrew Morton
2007-02-07 20:46     ` Andreas Dilger
2007-02-07 20:56       ` Andrew Morton
2007-02-08  9:29         ` Jan Kara
2007-02-08  9:45           ` Andrew Morton [this message]
2007-02-08 10:21             ` Jan Kara
2007-02-08 10:32               ` Andrew Morton
2007-02-08 10:47                 ` Jan Kara
2007-02-12  3:11                   ` Theodore Tso
2007-02-07  1:33 ` Andrew Morton
2007-02-07  3:45   ` Eric Sandeen
2007-02-07  9:46     ` Takashi Sato
  -- strict thread matches above, loose matches on Subject: below --
2007-02-08  9:01 Takashi Sato
2006-12-22 10:30 sho
2006-11-09 11:10 sho

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20070208014529.d990b502.akpm@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=adilger@clusterfs.com \
    --cc=jack@suse.cz \
    --cc=linux-ext4@vger.kernel.org \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=sho@tnes.nec.co.jp \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.