linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* fat problem in 2.4.2
@ 2001-02-25 20:08 James D Strandboge
  2001-02-25 20:20 ` Alan Cox
  0 siblings, 1 reply; 18+ messages in thread
From: James D Strandboge @ 2001-02-25 20:08 UTC (permalink / raw)
  To: linux-kernel

The bug with truncate in the fat filesystem that was present in 2.4.0,
and fixed with the 2.4.0-ac12 (or earlier) patch is still in the main
(unpatched) kernel, both 2.4.1 and 2.4.2.  The problem is that I cannot
apply the 2.4.0-ac patch to the newer kernels and I cannot patch up to
2.4.1 from 2.4.0 if I already applied the ac patch (so I obviously can't
get to 2.4.2).  I also can't patch from 2.4.0-ac12 to 2.4.1-ac20 using
only ac patches.  Neither of the ac patches for 2.4.1 and 2.4.2 have
this fix for fat, so giving the '-R' option to patch took away the fix.

I would be happy to extract the fix from 2.4.0-ac12, but I am not a
kernel developer and am not sure how many files were changed to fix this
bug.

I want to move up to 2.4.2 for the reiserfs updates and then eventually
to at least 2.4.2-ac2 for the loopback fix (but need the fat
functionality too).

Thanks for any help or suggestions.

James Strandboge
jstrand1@rochester.rr.com


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: fat problem in 2.4.2
  2001-02-25 20:08 fat problem in 2.4.2 James D Strandboge
@ 2001-02-25 20:20 ` Alan Cox
  2001-03-01 14:25   ` Peter Daum
  0 siblings, 1 reply; 18+ messages in thread
From: Alan Cox @ 2001-02-25 20:20 UTC (permalink / raw)
  To: James D Strandboge; +Cc: linux-kernel

> The bug with truncate in the fat filesystem that was present in 2.4.0,
> and fixed with the 2.4.0-ac12 (or earlier) patch is still in the main

It isnt a bug. The fix in 2.4-ac I've dropped. A program that assumes
ftruncating a file large will work is broken.

Alan


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: fat problem in 2.4.2
  2001-02-25 20:20 ` Alan Cox
@ 2001-03-01 14:25   ` Peter Daum
  2001-03-01 15:09     ` [PATCH] " Alexander Viro
  2001-03-01 18:19     ` Alan Cox
  0 siblings, 2 replies; 18+ messages in thread
From: Peter Daum @ 2001-03-01 14:25 UTC (permalink / raw)
  To: linux-kernel

On Sun, 25 Feb 2001, Alan Cox wrote:

> > The bug with truncate in the fat filesystem that was present in 2.4.0,
> > and fixed with the 2.4.0-ac12 (or earlier) patch is still in the main
>
> It isnt a bug. The fix in 2.4-ac I've dropped. A program that assumes
> ftruncating a file large will work is broken.

In that case, why was it changed for FAT only? Ext2 will still
happily enlarge a file by truncating it.

If the behavior has to be changed, wouldn't it be better to first
give people a chance to get programs, that rely on the old
behavior fixed, before enforcing the change?

Staroffice (the binary-only version; the new "open source"
version is not yet ready for real-world use) for example
currently doesn't write to FAT filesystems anymore - which is
pretty annoying for people who need it.

Is there somewhere a patch for the current kernel?

Regards,

       Peter Daum


^ permalink raw reply	[flat|nested] 18+ messages in thread

* [PATCH] Re: fat problem in 2.4.2
  2001-03-01 14:25   ` Peter Daum
@ 2001-03-01 15:09     ` Alexander Viro
  2001-03-02  0:21       ` Albert D. Cahalan
  2001-03-01 18:19     ` Alan Cox
  1 sibling, 1 reply; 18+ messages in thread
From: Alexander Viro @ 2001-03-01 15:09 UTC (permalink / raw)
  To: Peter Daum; +Cc: linux-kernel



On Thu, 1 Mar 2001, Peter Daum wrote:

> In that case, why was it changed for FAT only? Ext2 will still
> happily enlarge a file by truncating it.

Basically, the program depends on behaviour that was never guaranteed to
be there.

> Staroffice (the binary-only version; the new "open source"
> version is not yet ready for real-world use) for example
> currently doesn't write to FAT filesystems anymore - which is
> pretty annoying for people who need it.

Staroffice is non-portable and badly written, film at 11...

BTW, _some_ subset is doable on FAT. You can't always do it (bloody
thing doesn't support holes), but you can try the following (warning -
untested patch):

diff -urN S2/fs/fat/inode.c S2-fat/fs/fat/inode.c
--- S2/fs/fat/inode.c	Fri Feb 16 22:52:07 2001
+++ S2-fat/fs/fat/inode.c	Thu Mar  1 10:02:45 2001
@@ -897,6 +897,36 @@
 	unlock_kernel();
 }
 
+int generic_cont_expand(struct inode *inode, loff_t size)
+{
+	struct address_space *mapping = inode->i_mapping;
+	struct page *page;
+	unsigned long index, offset, limit;
+	int err;
+
+	limit = current->rlim[RLIMIT_FSIZE].rlim_cur;
+	if (limit != RLIM_INFINITY) {
+		if (size > limit) {
+			send_sig(SIGXFSZ, current, 0);
+			size = limit;
+		}
+	}
+	offset = (size & (PAGE_CACHE_SIZE-1)); /* Within page */
+	index = size >> PAGE_CACHE_SHIFT;
+	err = -ENOMEM;
+	page = grab_cache_page(mapping, index);
+	if (!page)
+		goto out;
+	err = mapping->a_ops->prepare_write(NULL, page, offset, offset);
+	if (!err)
+		err = mapping->a_ops->commit_write(NULL, page, offset, offset);
+	UnlockPage(page);
+	page_cache_release(page);
+	if (err > 0)
+		err = 0;
+out:
+	return err;
+}
 
 int fat_notify_change(struct dentry * dentry, struct iattr * attr)
 {
@@ -904,11 +934,17 @@
 	struct inode *inode = dentry->d_inode;
-	int error;
+	int error = 0;
 
-	/* FAT cannot truncate to a longer file */
+	/*
+	 * On FAT truncate to a longer file may fail with -ENOSPC. No
+	 * way to report it from fat_truncate(), so...
+	 */
 	if (attr->ia_valid & ATTR_SIZE) {
 		if (attr->ia_size > inode->i_size)
-			return -EPERM;
+			error = generic_cont_expand(inode, attr->ia_size);
 	}
+
+	if (error)
+		return error;
 
 	error = inode_change_ok(inode, attr);
 	if (error)

That said, if your only problem is Staroffice... <shrug> I would rather
rm the crapware and forget about it, but YMMV.
							Cheers,
								Al


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: fat problem in 2.4.2
  2001-03-01 14:25   ` Peter Daum
  2001-03-01 15:09     ` [PATCH] " Alexander Viro
@ 2001-03-01 18:19     ` Alan Cox
  2001-03-01 19:39       ` [CFT][PATCH] " Alexander Viro
  2001-03-02  7:45       ` ftruncate not extending files? bert hubert
  1 sibling, 2 replies; 18+ messages in thread
From: Alan Cox @ 2001-03-01 18:19 UTC (permalink / raw)
  To: gator; +Cc: linux-kernel

> In that case, why was it changed for FAT only? Ext2 will still
> happily enlarge a file by truncating it.

ftruncate() and truncate() may extend a file but they are not required to
do so.

> If the behavior has to be changed, wouldn't it be better to first
> give people a chance to get programs, that rely on the old
> behavior fixed, before enforcing the change?

A program relying on the old behaviour was violating standards. Also its been
this way for almost two years.

> Staroffice (the binary-only version; the new "open source"
> version is not yet ready for real-world use) for example
> currently doesn't write to FAT filesystems anymore - which is
> pretty annoying for people who need it.
> 
> Is there somewhere a patch for the current kernel?

You might be able to fish it out of old -ac kernel trees and debug it further.
Alternatively you could implement it in glibc of course, which is a nicer
solution


^ permalink raw reply	[flat|nested] 18+ messages in thread

* [CFT][PATCH] Re: fat problem in 2.4.2
  2001-03-01 18:19     ` Alan Cox
@ 2001-03-01 19:39       ` Alexander Viro
  2001-03-01 20:05         ` Linus Torvalds
  2001-03-01 20:56         ` Roman Zippel
  2001-03-02  7:45       ` ftruncate not extending files? bert hubert
  1 sibling, 2 replies; 18+ messages in thread
From: Alexander Viro @ 2001-03-01 19:39 UTC (permalink / raw)
  To: Alan Cox; +Cc: gator, linux-kernel



On Thu, 1 Mar 2001, Alan Cox wrote:

> > In that case, why was it changed for FAT only? Ext2 will still
> > happily enlarge a file by truncating it.
> 
> ftruncate() and truncate() may extend a file but they are not required to
> do so.
> 
> > If the behavior has to be changed, wouldn't it be better to first
> > give people a chance to get programs, that rely on the old
> > behavior fixed, before enforcing the change?
> 
> A program relying on the old behaviour was violating standards. Also its been
> this way for almost two years.
> 
> > Staroffice (the binary-only version; the new "open source"
> > version is not yet ready for real-world use) for example
> > currently doesn't write to FAT filesystems anymore - which is
> > pretty annoying for people who need it.
> > 
> > Is there somewhere a patch for the current kernel?
> 
> You might be able to fish it out of old -ac kernel trees and debug it further.
> Alternatively you could implement it in glibc of course, which is a nicer
> solution

Alan, fix is really quite simple. Especially if you have vmtruncate()
returning int (ac1 used to do it, I didn't check later ones). Actually
just a generic_cont_expand() done on expanding path in vmtruncate()
will be enough - it should be OK for all cases, including normal
filesystems. <grabbing -ac7>

OK, any brave soul to test that? All I can promise that it builds.
							Cheers,
								Al
diff -urN S2-ac7/fs/affs/file.c S2-ac7-truncate/fs/affs/file.c
--- S2-ac7/fs/affs/file.c	Fri Feb 16 22:52:08 2001
+++ S2-ac7-truncate/fs/affs/file.c	Thu Mar  1 14:22:54 2001
@@ -663,25 +663,6 @@
 	net_blocksize = blocksize - ((inode->i_sb->u.affs_sb.s_flags & SF_OFS) ? 24 : 0);
 	first = inode->i_size + net_blocksize -1;
 	do_div (first, net_blocksize);
-	if (inode->u.affs_i.i_lastblock < first - 1) {
-		/* There has to be at least one new block to be allocated */
-		if (!inode->u.affs_i.i_ec && alloc_ext_cache(inode)) {
-			/* XXX Fine! No way to indicate an error. */
-			return /* -ENOSPC */;
-		}
-		bh = affs_getblock(inode,first - 1);
-		if (!bh) {
-			affs_warning(inode->i_sb,"truncate","Cannot extend file");
-			inode->i_size = net_blocksize * (inode->u.affs_i.i_lastblock + 1);
-		} else if (inode->i_sb->u.affs_sb.s_flags & SF_OFS) {
-			tmp = inode->i_size;
-			rem = do_div(tmp, net_blocksize);
-			DATA_FRONT(bh)->data_size = cpu_to_be32(rem ? rem : net_blocksize);
-			affs_fix_checksum(blocksize,bh->b_data,5);
-			mark_buffer_dirty(bh);
-		}
-		goto out_truncate;
-	}
 	ekey = inode->i_ino;
 	ext  = 0;
 
diff -urN S2-ac7/fs/fat/inode.c S2-ac7-truncate/fs/fat/inode.c
--- S2-ac7/fs/fat/inode.c	Thu Mar  1 14:05:17 2001
+++ S2-ac7-truncate/fs/fat/inode.c	Thu Mar  1 14:21:45 2001
@@ -904,12 +904,6 @@
 	struct inode *inode = dentry->d_inode;
 	int error;
 
-	/* FAT cannot truncate to a longer file */
-	if (attr->ia_valid & ATTR_SIZE) {
-		if (attr->ia_size > inode->i_size)
-			return -EPERM;
-	}
-
 	error = inode_change_ok(inode, attr);
 	if (error)
 		return MSDOS_SB(sb)->options.quiet ? 0 : error;
diff -urN S2-ac7/mm/memory.c S2-ac7-truncate/mm/memory.c
--- S2-ac7/mm/memory.c	Thu Mar  1 14:05:20 2001
+++ S2-ac7-truncate/mm/memory.c	Thu Mar  1 14:21:05 2001
@@ -924,7 +924,32 @@
 		zap_page_range(mm, start, len);
 	} while ((mpnt = mpnt->vm_next_share) != NULL);
 }
-			      
+
+static int generic_vm_expand(struct address_space *mapping, loff_t size)
+{
+	struct page *page;
+	unsigned long index, offset;
+	int err;
+
+	if (!mapping->a_ops->prepare_write || !mapping->a_ops->commit_write)
+		return -ENOSYS;
+
+	offset = (size & (PAGE_CACHE_SIZE-1)); /* Within page */
+	index = size >> PAGE_CACHE_SHIFT;
+	err = -ENOMEM;
+	page = grab_cache_page(mapping, index);
+	if (!page)
+		goto out;
+	err = mapping->a_ops->prepare_write(NULL, page, offset, offset);
+	if (!err)
+		err = mapping->a_ops->commit_write(NULL, page, offset, offset);
+	UnlockPage(page);
+	page_cache_release(page);
+	if (err > 0)
+		err = 0;
+out:
+	return err;
+}
 
 /*
  * Handle all mappings that got truncated by a "truncate()"
@@ -939,6 +964,7 @@
 	unsigned long partial, pgoff;
 	struct address_space *mapping = inode->i_mapping;
 	unsigned long limit;
+	int err;
 
 	if (inode->i_size < offset)
 		goto do_expand;
@@ -976,10 +1002,14 @@
 			offset = limit;
 		}
 	}
-	inode->i_size = offset;
-	if (inode->i_op && inode->i_op->truncate)
+	err = generic_vm_expand(mapping, offset);
+	if (err == -ENOSYS) {
+		err = 0;
+		inode->i_size = offset;
+	}
+	if (!err && inode->i_op && inode->i_op->truncate)
 		inode->i_op->truncate(inode);
-	return 0;
+	return err;
 out:
 	return -EFBIG;
 }


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [CFT][PATCH] Re: fat problem in 2.4.2
  2001-03-01 19:39       ` [CFT][PATCH] " Alexander Viro
@ 2001-03-01 20:05         ` Linus Torvalds
  2001-03-01 20:52           ` Chris Mason
  2001-03-01 20:56         ` Roman Zippel
  1 sibling, 1 reply; 18+ messages in thread
From: Linus Torvalds @ 2001-03-01 20:05 UTC (permalink / raw)
  To: linux-kernel

In article <Pine.GSO.4.21.0103011345110.11577-100000@weyl.math.psu.edu>,
Alexander Viro  <viro@math.psu.edu> wrote:
>
>Alan, fix is really quite simple. Especially if you have vmtruncate()
>returning int (ac1 used to do it, I didn't check later ones). Actually
>just a generic_cont_expand() done on expanding path in vmtruncate()
>will be enough - it should be OK for all cases, including normal
>filesystems. <grabbing -ac7>
>
>OK, any brave soul to test that? All I can promise that it builds.

This looks like it would create a dummy block even for non-broken
filesystems (ie truncating a file to be larger on ext2 would create a
block, no?). While that would work, it would also waste disk-space.

		Linus

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [CFT][PATCH] Re: fat problem in 2.4.2
  2001-03-01 20:05         ` Linus Torvalds
@ 2001-03-01 20:52           ` Chris Mason
  0 siblings, 0 replies; 18+ messages in thread
From: Chris Mason @ 2001-03-01 20:52 UTC (permalink / raw)
  To: Linus Torvalds, linux-kernel



On Thursday, March 01, 2001 12:05:50 PM -0800 Linus Torvalds
<torvalds@transmeta.com> wrote:

> In article <Pine.GSO.4.21.0103011345110.11577-100000@weyl.math.psu.edu>,
> Alexander Viro  <viro@math.psu.edu> wrote:
>> 
>> Alan, fix is really quite simple. Especially if you have vmtruncate()
>> returning int (ac1 used to do it, I didn't check later ones). Actually
>> just a generic_cont_expand() done on expanding path in vmtruncate()
>> will be enough - it should be OK for all cases, including normal
>> filesystems. <grabbing -ac7>
>> 
>> OK, any brave soul to test that? All I can promise that it builds.
> 
> This looks like it would create a dummy block even for non-broken
> filesystems (ie truncating a file to be larger on ext2 would create a
> block, no?). While that would work, it would also waste disk-space.

Another idea is to create the hole for new_file_size + [one block], and
then truncate that block off the end of the file, leaving nothing but the
hole.  I'll never admit to suggesting it though.

-chris


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [CFT][PATCH] Re: fat problem in 2.4.2
  2001-03-01 19:39       ` [CFT][PATCH] " Alexander Viro
  2001-03-01 20:05         ` Linus Torvalds
@ 2001-03-01 20:56         ` Roman Zippel
  2001-03-01 21:07           ` Alexander Viro
  1 sibling, 1 reply; 18+ messages in thread
From: Roman Zippel @ 2001-03-01 20:56 UTC (permalink / raw)
  To: Alexander Viro; +Cc: Alan Cox, gator, linux-kernel

Hi,

On Thu, 1 Mar 2001, Alexander Viro wrote:

> +static int generic_vm_expand(struct address_space *mapping, loff_t size)
> +{
> +	struct page *page;
> +	unsigned long index, offset;
> +	int err;
> +
> +	if (!mapping->a_ops->prepare_write || !mapping->a_ops->commit_write)
> +		return -ENOSYS;
> +
> +	offset = (size & (PAGE_CACHE_SIZE-1)); /* Within page */
> +	index = size >> PAGE_CACHE_SHIFT;

For affs I did basically the same with a small difference:

	offset = ((size-1) & (PAGE_CACHE_SIZE-1)) + 1;
	index = (size-1) >> PAGE_CACHE_SHIFT;

That works fine here and allocates a page in the cache more likely to be
used.

bye, Roman


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [CFT][PATCH] Re: fat problem in 2.4.2
  2001-03-01 20:56         ` Roman Zippel
@ 2001-03-01 21:07           ` Alexander Viro
  2001-03-01 21:27             ` Roman Zippel
  0 siblings, 1 reply; 18+ messages in thread
From: Alexander Viro @ 2001-03-01 21:07 UTC (permalink / raw)
  To: Roman Zippel; +Cc: Alan Cox, gator, linux-kernel



On Thu, 1 Mar 2001, Roman Zippel wrote:

> Hi,
> 
> On Thu, 1 Mar 2001, Alexander Viro wrote:
> 
> > +static int generic_vm_expand(struct address_space *mapping, loff_t size)
> > +{
> > +	struct page *page;
> > +	unsigned long index, offset;
> > +	int err;
> > +
> > +	if (!mapping->a_ops->prepare_write || !mapping->a_ops->commit_write)
> > +		return -ENOSYS;
> > +
> > +	offset = (size & (PAGE_CACHE_SIZE-1)); /* Within page */
> > +	index = size >> PAGE_CACHE_SHIFT;
> 
> For affs I did basically the same with a small difference:
> 
> 	offset = ((size-1) & (PAGE_CACHE_SIZE-1)) + 1;
> 	index = (size-1) >> PAGE_CACHE_SHIFT;
> 
> That works fine here and allocates a page in the cache more likely to be
> used.

	The only difference is in the case when size is a multiple of
PAGE_CACHE_SHIFT, so most of the time it's the same, but yeah, this
variant is probably nicer.

	The problem with putting it into ->truncate() is obvious -
handling errors. And doing the thing before vmtruncate() (in
foo_notify_change(), whatever) is also PITA - you need to
reproduce the rlimit handling. Not nice...

	IOW, if it's worth doing at all it probably should be
on expanding path in vmtruncate() - limit checks are already
done, but old i_size is still not lost...
							Cheers,
								Al


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [CFT][PATCH] Re: fat problem in 2.4.2
  2001-03-01 21:07           ` Alexander Viro
@ 2001-03-01 21:27             ` Roman Zippel
  0 siblings, 0 replies; 18+ messages in thread
From: Roman Zippel @ 2001-03-01 21:27 UTC (permalink / raw)
  To: Alexander Viro; +Cc: Alan Cox, gator, linux-kernel

Hi,

On Thu, 1 Mar 2001, Alexander Viro wrote:

> 	IOW, if it's worth doing at all it probably should be
> on expanding path in vmtruncate() - limit checks are already
> done, but old i_size is still not lost...

The fs where it's important have mmu_private, that's what I use to decide
whether to expand or truncate.

bye, Roman


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: [PATCH] Re: fat problem in 2.4.2
  2001-03-01 15:09     ` [PATCH] " Alexander Viro
@ 2001-03-02  0:21       ` Albert D. Cahalan
  0 siblings, 0 replies; 18+ messages in thread
From: Albert D. Cahalan @ 2001-03-02  0:21 UTC (permalink / raw)
  To: Alexander Viro; +Cc: Peter Daum, linux-kernel

Alexander Viro writes:

[about file expansion by truncate]

> Basically, the program depends on behaviour that was never guaranteed
> to be there.

1. it is useful
2. it is documented in a few places AFAIK
3. it is portable enough for Star Office (Solaris I guess)

> BTW, _some_ subset is doable on FAT. You can't always do it (bloody
> thing doesn't support holes), but you can try the following (warning -
> untested patch):

Holes are nothing special. They are just a simple type of compression.
We are doing overcommit on filesystems, and need an OOD killer to
wipe out big files like the Star Office executable.

^ permalink raw reply	[flat|nested] 18+ messages in thread

* ftruncate not extending files?
  2001-03-01 18:19     ` Alan Cox
  2001-03-01 19:39       ` [CFT][PATCH] " Alexander Viro
@ 2001-03-02  7:45       ` bert hubert
  2001-03-02  9:57         ` Malcolm Beattie
                           ` (2 more replies)
  1 sibling, 3 replies; 18+ messages in thread
From: bert hubert @ 2001-03-02  7:45 UTC (permalink / raw)
  To: linux-kernel

On Thu, Mar 01, 2001 at 06:19:35PM +0000, Alan Cox wrote:
> > In that case, why was it changed for FAT only? Ext2 will still
> > happily enlarge a file by truncating it.
> 
> ftruncate() and truncate() may extend a file but they are not required to
> do so.

Stevens' example code assumes that it does. And given the number of people
that regard his work as Holy, it might not be a bad idea to implement Linux
so that it does what people think it does.

I would've sworn, based on the fact that I saw people do it, that ftruncate
was a legitimate way to extend a file - especially useful in combination
with mmap().

I don't really care where it is done, in glibc or in the kernel - but let's
honor this convention and not needlessly break code.

Regards,

bert hubert

-- 
http://www.PowerDNS.com      Versatile DNS Services  
Trilab                       The Technology People   
'SYN! .. SYN|ACK! .. ACK!' - the mating call of the internet

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: ftruncate not extending files?
  2001-03-02  7:45       ` ftruncate not extending files? bert hubert
@ 2001-03-02  9:57         ` Malcolm Beattie
  2001-03-02 10:56         ` Dmitry A. Fedorov
       [not found]         ` <mng==20010302095701.A4685@sable.ox.ac.uk>
  2 siblings, 0 replies; 18+ messages in thread
From: Malcolm Beattie @ 2001-03-02  9:57 UTC (permalink / raw)
  To: linux-kernel

bert hubert writes:
> I would've sworn, based on the fact that I saw people do it, that ftruncate
> was a legitimate way to extend a file

Well it's not SuSv2 standards compliant:

    http://www.opengroup.org/onlinepubs/007908799/xsh/ftruncate.html

    If the file previously was larger than length, the extra data is
    discarded. If it was previously shorter than length, it is
    unspecified whether the file is changed or its size increased. If
    ^^^^^^^^^^^
    the file is extended, the extended area appears as if it were
    zero-filled.

How "legitimate" relates to "SuSv2 standards compliant" is your call.

--Malcolm

-- 
Malcolm Beattie <mbeattie@sable.ox.ac.uk>
Unix Systems Programmer
Oxford University Computing Services

^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: ftruncate not extending files?
  2001-03-02  7:45       ` ftruncate not extending files? bert hubert
  2001-03-02  9:57         ` Malcolm Beattie
@ 2001-03-02 10:56         ` Dmitry A. Fedorov
  2001-03-02 14:15           ` Hugh Dickins
  2001-03-03 19:32           ` Kai Henningsen
       [not found]         ` <mng==20010302095701.A4685@sable.ox.ac.uk>
  2 siblings, 2 replies; 18+ messages in thread
From: Dmitry A. Fedorov @ 2001-03-02 10:56 UTC (permalink / raw)
  To: linux-kernel

On Fri, 2 Mar 2001, bert hubert wrote:

> > ftruncate() and truncate() may extend a file but they are not required to
> > do so.
> 
> I would've sworn, based on the fact that I saw people do it, that ftruncate
> was a legitimate way to extend a file - especially useful in combination
> with mmap().

lseek and write does it already and need not to duplicate with truncate().
Using truncate() to extend a file sounds very strange.

About mmap() SUSv2 says:

If the size of the mapped file changes after the call to mmap() as a
result of some other operation on the mapped file, the effect of
references to portions of the mapped region that correspond to added or
removed portions of the file is unspecified.
                                ^^^^^^^^^^^
What do you mean about "useful in combination with mmap()" ?


> I don't really care where it is done, in glibc or in the kernel - but let's
> honor this convention and not needlessly break code.

Let's don't connive to ill-formed programs.

--


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: ftruncate not extending files?
  2001-03-02 10:56         ` Dmitry A. Fedorov
@ 2001-03-02 14:15           ` Hugh Dickins
  2001-03-03 19:32           ` Kai Henningsen
  1 sibling, 0 replies; 18+ messages in thread
From: Hugh Dickins @ 2001-03-02 14:15 UTC (permalink / raw)
  To: Dmitry A. Fedorov; +Cc: linux-kernel

On Fri, 2 Mar 2001, Dmitry A. Fedorov wrote:
> On Fri, 2 Mar 2001, bert hubert wrote:
> 
> > > ftruncate() and truncate() may extend a file but they are not required to
> > > do so.
> > 
> > I would've sworn, based on the fact that I saw people do it, that ftruncate
> > was a legitimate way to extend a file - especially useful in combination
> > with mmap().
> 
> lseek and write does it already and need not to duplicate with truncate().
> Using truncate() to extend a file sounds very strange.
> 
> About mmap() SUSv2 says:
> 
> If the size of the mapped file changes after the call to mmap() as a
> result of some other operation on the mapped file, the effect of
> references to portions of the mapped region that correspond to added or
> removed portions of the file is unspecified.
>                                 ^^^^^^^^^^^
> What do you mean about "useful in combination with mmap()" ?
> 
> > I don't really care where it is done, in glibc or in the kernel - but let's
> > honor this convention and not needlessly break code.
> 
> Let's don't connive to ill-formed programs.

Those are not ill-formed programs, they are just ordinary programs.

ftruncate()-to-extend and mmap() go together like knife and fork:
you don't have to use either, and you can use each separately, but
it's conventional to use them together.  The awkward SIGBUS-beyond-EOF
specification of mmap() begs for a call to set filesize; often the man
page of one syscall references the other.

It would be perverse and unhelpful for Linux to offer a filesystem
which supports mmap() but not ftrunctate()-to-extend.

The SuSv2 quotations, above and in other mail, are just weasly.
They probably started off specifying the natural least-surprise
behaviours, which most support; but some vendor who failed to meet
that spec forced the wording to be changed to "unspecified".
That's no reason for Linux to be obstinate.

But I agree "ftruncate" is a confusing name for a syscall to
(truncate or) extend a file: like using a knife to spread glue.

Hugh


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: ftruncate not extending files?
       [not found]         ` <mng==20010302095701.A4685@sable.ox.ac.uk>
@ 2001-03-02 23:41           ` Jens-Uwe Mager
  0 siblings, 0 replies; 18+ messages in thread
From: Jens-Uwe Mager @ 2001-03-02 23:41 UTC (permalink / raw)


On Fri, 2 Mar 2001 10:17:14 GMT, Malcolm Beattie <mbeattie@sable.ox.ac.uk> wrote:
>bert hubert writes:
>> I would've sworn, based on the fact that I saw people do it, that ftruncate
>> was a legitimate way to extend a file
>
>Well it's not SuSv2 standards compliant:
>
>    http://www.opengroup.org/onlinepubs/007908799/xsh/ftruncate.html
>
>    If the file previously was larger than length, the extra data is
>    discarded. If it was previously shorter than length, it is
>    unspecified whether the file is changed or its size increased. If
>    ^^^^^^^^^^^
>    the file is extended, the extended area appears as if it were
>    zero-filled.
>
>How "legitimate" relates to "SuSv2 standards compliant" is your call.

It is interesting to compare the wording of the Solaris man page, it
sounds pretty much like the SuSv2 paragraph above, but without the
restriction that the result is unspecified, it guarantees the extending
is a legitimate operation. Sounds like the SuSv2 authors chickened out
on that issue, most of the Unix platforms I know (including SunOS 4&5,
HP/UX, IRIX, Tru64, AIX and various *BSD) do happily extend a file on
truncate.

-- 
Jens-Uwe Mager	<pgp-mailto:62CFDB25>


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: ftruncate not extending files?
  2001-03-02 10:56         ` Dmitry A. Fedorov
  2001-03-02 14:15           ` Hugh Dickins
@ 2001-03-03 19:32           ` Kai Henningsen
  1 sibling, 0 replies; 18+ messages in thread
From: Kai Henningsen @ 2001-03-03 19:32 UTC (permalink / raw)
  To: linux-kernel

hugh@veritas.com (Hugh Dickins)  wrote on 02.03.01 in <Pine.LNX.4.21.0103021347010.1321-100000@localhost.localdomain>:

> The SuSv2 quotations, above and in other mail, are just weasly.

The next version is less weasly. Right now it's still a draft; what it  
says in draft 5 is this (note the markers which show what's optional to  
which feature set; XSI aka SUS-next-generation _does_ require expansion):

15245 NAME
15246           ftruncate - truncate a file to a specified length
15247 SYNOPSIS
15248           #include <unistd.h>
15249           int ftruncate(int fildes, off_t length);
15250 DESCRIPTION
15251           If fildes is not a valid file descriptor open for writing, the ftruncate( ) function shall fail.
15252           If fildes refers to a regular file, the ftruncate( ) function shall cause the size of the file to be
15253           truncated to length. If the size of the file previously exceeded length, the extra data shall no
15254           longer be available to reads on the file. If the file previously was smaller than this size,
15255 XSI       ftruncate( ) shall either increase the size of the file or fail.  XSI-conformant systems shall increase
15256           the size of the file. If the file size is increased, the extended area shall appear as if it were zero- |
15257           filled. The value of the seek pointer shall not be modified by a call to ftruncate( ).                              |
15258           Upon successful completion, if fildes refers to a regular file, the ftruncate( ) function shall mark
15259           for update the st_ctime and st_mtime fields of the file and the S_ISUID and S_ISGID bits of the file
15260           mode may be cleared. If the ftruncate( ) function is unsuccessful, the file is unaffected.
15261 XSI       If the request would cause the file size to exceed the soft file size limit for the process, the
15262           request shall fail and the implementation shall generate the SIGXFSZ signal for the thread.                         |
15263           If fildes refers to a directory, ftruncate( ) shall fail.
15264           If fildes refers to any other file type, except a shared memory object, the result is unspecified.
15265 SHM       If fildes refers to a shared memory object, ftruncate( ) shall set the size of the shared memory
15266           object to length.
15267 MF|SHM If the effect of ftruncate( ) is to decrease the size of a shared memory object or memory mapped
15268           file and whole pages beyond the new end were previously mapped, then the whole pages
15269           beyond the new end shall be discarded.
15270 MPR       If the Memory Protection option is supported, references to discarded pages shall result in the
15271           generation of a SIGBUS signal; otherwise, the result of such references is undefined.
15272 MF|SHM If the effect of ftruncate( ) is to increase the size of a shared memory object, it is unspecified if the
15273           contents of any mapped pages between the old end-of-file and the new are flushed to the
15274           underlying object.
15275 RETURN VALUE
15276           Upon successful completion, ftruncate( ) shall return 0; otherwise, -1 shall be returned and errno
15277           set to indicate the error.
15278 ERRORS
15279           The ftruncate( ) function shall fail if:
15280           [EINTR]               A signal was caught during execution.
15281           [EINVAL]              The length argument was less than 0.
15282           [EFBIG] or [EINVAL]
15283                                 The length argument was greater than the maximum file size.
15284 XSI       [EFBIG]               The file is a regular file and length is greater than the offset maximum
15285                                 established in the open file description associated with fildes.
15286           [EIO]                 An I/O error occurred while reading from or writing to a file system.
15287              [EBADF] or [EINVAL]
15288                                   The fildes argument is not a file descriptor open for writing.
15289              [EINVAL]             The fildes argument references a file that was opened without write
15290                                   permission.
15291              [EROFS]              The named file resides on a read-only file system.
15292 EXAMPLES
15293              None.
15294 APPLICATION USAGE
15295              None.
15296 RATIONALE
15297              The ftruncate( ) function is part of IEEE Std 1003.1-200x as it was deemed to be more useful than |
15298              truncate( ). The truncate( ) function is provided as an XSI extension.
15299 FUTURE DIRECTIONS
15300              None.
15301 SEE ALSO
15302              open( ), truncate( ), the Base Definitions volume of IEEE Std 1003.1-200x, <unistd.h>                  |
15303 CHANGE HISTORY
15304              First released in Issue 4, Version 2.
15305 Issue 5
15306              Moved from X/OPEN UNIX extension to BASE and aligned with ftruncate( ) in the POSIX
15307              Realtime Extension. Specifically, the DESCRIPTION is extensively reworded and [EROFS] is
15308              added to the list of mandatory errors that can be returned by ftruncate( ).
15309              Large File Summit extensions are added.
15310 Issue 6
15311              The truncate( ) function has been split out into a separate reference page.
15312              The following new requirements on POSIX implementations derive from alignment with the
15313              Single UNIX Specification:
15314                * The DESCRIPTION is change to indicate that if the file size is changed, and if the file is a
15315                  regular file, the S_ISUID and S_ISGID bits in the file mode may be cleared.
15316              The following changes were made to align with the IEEE P1003.1a draft standard:
15317                * The DESCRIPTION text is updated.
15318              XSI-conformant systems are required to increase the size of the file if the file was previously
15319              smaller than the size requested.

You can get the whole draft by subscribing to the mailing list at
http://www.opengroup.org/austin/.

MfG Kai

^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2001-03-03 20:34 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2001-02-25 20:08 fat problem in 2.4.2 James D Strandboge
2001-02-25 20:20 ` Alan Cox
2001-03-01 14:25   ` Peter Daum
2001-03-01 15:09     ` [PATCH] " Alexander Viro
2001-03-02  0:21       ` Albert D. Cahalan
2001-03-01 18:19     ` Alan Cox
2001-03-01 19:39       ` [CFT][PATCH] " Alexander Viro
2001-03-01 20:05         ` Linus Torvalds
2001-03-01 20:52           ` Chris Mason
2001-03-01 20:56         ` Roman Zippel
2001-03-01 21:07           ` Alexander Viro
2001-03-01 21:27             ` Roman Zippel
2001-03-02  7:45       ` ftruncate not extending files? bert hubert
2001-03-02  9:57         ` Malcolm Beattie
2001-03-02 10:56         ` Dmitry A. Fedorov
2001-03-02 14:15           ` Hugh Dickins
2001-03-03 19:32           ` Kai Henningsen
     [not found]         ` <mng==20010302095701.A4685@sable.ox.ac.uk>
2001-03-02 23:41           ` Jens-Uwe Mager

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).