From: Jesse Pollard <jesse@cats-chateau.net>
To: Maciej Zenczykowski <maze@cela.pl>
Cc: Florian Weimer <fw@deneb.enyo.de>, <Valdis.Kletnieks@vt.edu>,
Daniel Gryniewicz <dang@fprintf.net>,
linux-kernel mailing list <linux-kernel@vger.kernel.org>
Subject: Re: OT: why no file copy() libc/syscall ??
Date: Fri, 21 Nov 2003 10:34:25 -0600 [thread overview]
Message-ID: <03112110342501.29137@tabby> (raw)
In-Reply-To: <Pine.LNX.4.44.0311202206530.10515-100000@gaia.cela.pl>
On Thursday 20 November 2003 15:48, Maciej Zenczykowski wrote:
> Assume 'fast'copy(int fd_in, int fd_out) where fd_in and fd_out reference
> files. fd_in is opened for read and fd_out is opened for write. Ignore
> filepos locations in both fd's. fd_out must reference an empty/truncated
> file (if not then fail). Usually you'd call copy on fd_out straight out
> of a creat call (and thus this would be a non-issue).
>
> > 1. what happens if the copy is aborted?
>
> I'd say the copy operation should be 'atomic', either it succeeds (full
> copy) or fails (no changes to filesystems except for the truncate). An
> abort would obviously usually result in a failure (thus a possible revert,
> which is rather easy since it's likely just an truncate of whatever has
> already been copied) or if we've just finished and than a successful
> result.
Really? what happens if the abort is local to the system making the request?
what happens if the abort is on the remote server?
> > 2. what happens if the network drops while the remote server continues?
>
> If the remote server has enough data to perform the operation then it does
> complete it otherwise there ain't enough info anyway (afterall the
> entire idea of this is to fit the entire copy into a single copy
> instruction thus a single packet/command whatever, no extra data is
> passed)...
And back to aborts?
> > 3. what about buffer synchronization?
>
> If this is happening remotely then I don't see what requires sync???
Multiple hosts remote to the server that have afile open. Though this
already happens with NFS.
> > 4. what errors should be reported ?
>
> This is tougher:
>
> Tests first performed locally (if they can be) than request forwarded to
> remote end and tests performed remotely - return either error or
> ACCEPTED, at which point local end tells it to go ahead, (at this
> point the operation is effectively performed (unless an abort is
> signalled) regardless of network connectivity). On completion remote end
> will return info on completion or error code.
>
> a) operation not supported by kernel :) - ENOSYS
> b) fd_in/fd_out invalid file descriptor - EBADF
> c) fd_in/fd_out is directory - EISDIR
> d) can't read/write from/to fd_in/fd_out - EINVAL
> e) an error if fd_out ain't empty - ENOTEMPTY
> f) operation not supported by this combination of devices - EOPNOTSUPP
> [so you need to do it via usual loop]
> g) input file bigger then output file can be - EFBIG
> [ie copy of 5GB file from remote filesystem which supports it to
> another filesystem on the same server with 2GB max file size]
> h) low-level IO error - EIO - serious problems (i.e. HDD read/write error)
> i) out of disk space during copy - ENOSPC
> j) out of memory during copy - ENOMEM (unlikely, needed?)
> k) lost network connection - ENETRESET (unknown whether succeeded)
> or ENOLINK ?
> l) operation was aborted - EINTR [probably should be some other error
> code, not sure]
> m) success - either return 0 or the number of bytes copied
> [probably best to return the # of bytes copied, even if (for now?) we
> only accept full copies]
>
> Did I miss anything? What about non-blocking call? Basically as above but
> return INPROGRESS as soon as we tell remote end to go ahead... or perhaps
> don't support non-blocking call?
>
> > 5. what happens when the syscall is interupted? Especially if the remote
> > copy may take a while (I've seen some require an hour or more - worst
> > case: days due to a media error (completed after the disk was
> > replaced)).
>
> Well, if it's interrupted by a SIGINT or the like then return EINTR and
> the copy was not performed (ie we backed the copy out, unless net failure
> detected during abort then ENOLINK/ENETRESET).
Ooop - the copy is being done on the remote server.
> If it's a more normal signal than it should behave like any normal kernel
> restartable syscall (i.e. via ERESTARTNOHAND or something like that).
Again, the copy may be being made on the remote server.
> > 6. what about a client opening the copy before it is finished copying?
>
> The file copy is atomic and thus the file doesn't per se exist until the
> copy operation completes (or the file exists with zero size and is locked
> and can't be opened).
It does under all other methods of copying.
> Perhaps in the future we could support partial copies and restarting an
> interrupted copy, but let's first agree (or not) on the above.
>
> I think a copy syscall would be very useful. What I'd really like to see
> is some sort of block-hashed-space-compression with copy-on-write
> semantics file system for linux (for my 500 CD collection which probably
> has a 10-12 data duplicity factor).
It could be usefull. What you describe now is a migrating filesystem on a
server. And note that your COW is going from two different filesystems (hmm
or maybe a custom union mount?)...
Which is where the migrating filesystem. The served filesystem should already
know how to transfer a file from the archive.
next prev parent reply other threads:[~2003-11-21 16:35 UTC|newest]
Thread overview: 77+ messages / expand[flat|nested] mbox.gz Atom feed top
2003-11-11 1:05 OT: why no file copy() libc/syscall ?? Albert Cahalan
2003-11-11 3:50 ` Andreas Dilger
2003-11-11 4:03 ` Daniel Gryniewicz
2003-11-11 4:14 ` Valdis.Kletnieks
2003-11-11 6:00 ` Andreas Dilger
2003-11-11 8:58 ` Florian Weimer
2003-11-11 10:27 ` jw schultz
2003-11-11 20:08 ` Jan Harkes
2003-11-12 15:36 ` Jesse Pollard
2003-11-20 17:21 ` Florian Weimer
2003-11-20 19:08 ` Jesse Pollard
2003-11-20 19:12 ` Florian Weimer
2003-11-20 19:44 ` Justin Cormack
2003-11-20 20:44 ` Timothy Miller
2003-11-20 21:07 ` Andreas Dilger
2003-11-20 21:30 ` Timothy Miller
2003-11-20 21:49 ` Maciej Zenczykowski
2003-11-20 21:52 ` Timothy Miller
2003-11-20 21:58 ` Hua Zhong
2003-11-22 14:50 ` Pavel Machek
2003-11-22 19:50 ` Jamie Lokier
2003-11-22 23:07 ` Andreas Schwab
2003-11-21 16:24 ` Jesse Pollard
2003-11-20 21:48 ` Maciej Zenczykowski
2003-11-21 16:34 ` Jesse Pollard [this message]
2003-11-20 22:31 ` Xavier Bestel
2003-11-20 22:44 ` Andreas Dilger
2003-11-27 2:40 ` Robert White
2003-11-27 7:29 ` Nick Piggin
2003-11-27 9:15 ` David Lang
2003-11-27 8:56 ` Nick Piggin
2003-11-27 9:50 ` David Lang
2003-11-27 10:02 ` Jörn Engel
2003-11-27 10:58 ` David Lang
2003-12-01 16:20 ` Jesse Pollard
2003-11-11 8:52 ` Gábor Lénárt
2003-11-11 13:38 ` Rogier Wolff
2003-11-11 13:53 ` Jakub Jelinek
2003-11-11 13:58 ` David Woodhouse
2003-11-13 20:22 ` H. Peter Anvin
2003-11-13 23:39 ` Andrea Arcangeli
2003-11-14 0:04 ` jw schultz
2003-11-14 0:36 ` H. Peter Anvin
2003-11-14 1:10 ` Andrea Arcangeli
2003-11-14 1:15 ` H. Peter Anvin
2003-11-11 14:11 ` Albert Cahalan
2003-11-12 15:19 ` Jesse Pollard
2003-11-14 3:42 ` Albert Cahalan
[not found] <1068512710.722.161.camel@cube.suse.lists.linux.kernel>
[not found] ` <20031111133859.GA11115@bitwizard.nl.suse.lists.linux.kernel>
[not found] ` <20031111085323.M8854@devserv.devel.redhat.com.suse.lists.linux.kernel>
[not found] ` <bp0p5m$lke$1@cesium.transmeta.com.suse.lists.linux.kernel>
[not found] ` <20031113233915.GO1649@x30.random.suse.lists.linux.kernel>
[not found] ` <3FB4238A.40605@zytor.com.suse.lists.linux.kernel>
[not found] ` <20031114011009.GP1649@x30.random.suse.lists.linux.kernel>
[not found] ` <3FB42CC4.9030009@zytor.com.suse.lists.linux.kernel>
2003-11-14 15:26 ` Andi Kleen
2003-11-18 15:49 ` Jamie Lokier
2003-11-18 16:05 ` Andi Kleen
2003-11-18 16:25 ` Trond Myklebust
2003-11-19 13:30 ` Jesse Pollard
2003-11-18 16:58 ` H. Peter Anvin
2003-11-19 2:12 ` Linus Torvalds
2003-11-19 4:04 ` Chris Adams
[not found] <Qvw7.5Qf.9@gated-at.bofh.it>
[not found] ` <QxRl.17Y.9@gated-at.bofh.it>
[not found] ` <Qy0W.1sk.9@gated-at.bofh.it>
[not found] ` <QyaB.1GK.17@gated-at.bofh.it>
[not found] ` <QzSZ.4x1.1@gated-at.bofh.it>
[not found] ` <QCHh.X6.3@gated-at.bofh.it>
2003-11-11 9:51 ` Ihar 'Philips' Filipau
2003-11-11 10:41 ` jw schultz
[not found] ` <QH4e.eV.3@gated-at.bofh.it>
2003-11-11 14:11 ` Ihar 'Philips' Filipau
2003-11-11 15:02 ` Rogier Wolff
2003-11-11 15:31 ` Ihar 'Philips' Filipau
2003-11-11 20:22 ` Jan Harkes
2003-11-11 20:31 ` Valdis.Kletnieks
[not found] <QDtX.2dq.15@gated-at.bofh.it>
[not found] ` <QDtX.2dq.17@gated-at.bofh.it>
[not found] ` <QDtX.2dq.19@gated-at.bofh.it>
[not found] ` <QDtX.2dq.21@gated-at.bofh.it>
[not found] ` <QDtX.2dq.23@gated-at.bofh.it>
[not found] ` <QDtY.2dq.25@gated-at.bofh.it>
[not found] ` <QDtX.2dq.13@gated-at.bofh.it>
[not found] ` <QEg2.3zi.9@gated-at.bofh.it>
2003-11-11 12:43 ` Ihar 'Philips' Filipau
-- strict thread matches above, loose matches on Subject: below --
2003-11-10 12:09 Bradley Chapman
2003-11-10 18:47 ` Tomas Konir
2003-11-10 22:44 ` Derek Foreman
[not found] <QiyV.1k3.15@gated-at.bofh.it>
2003-11-10 12:08 ` Ihar 'Philips' Filipau
2003-11-10 13:29 ` Jesse Pollard
2003-11-10 14:22 ` Daniel Jacobowitz
2003-11-11 20:57 ` Jakob Oestergaard
2003-11-10 15:19 ` David Woodhouse
2003-11-10 16:15 ` Jesse Pollard
2003-11-11 12:00 ` davide.rossetti
2003-11-11 12:08 ` Andreas Schwab
2003-11-11 12:23 ` davide.rossetti
2003-11-10 11:33 Davide Rossetti
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=03112110342501.29137@tabby \
--to=jesse@cats-chateau.net \
--cc=Valdis.Kletnieks@vt.edu \
--cc=dang@fprintf.net \
--cc=fw@deneb.enyo.de \
--cc=linux-kernel@vger.kernel.org \
--cc=maze@cela.pl \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).