Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Ed Tomlinson]

CaT wrote:

> I believe the patch below will apply to both the above (I know it does
> to 2.5.66 and 2.4.20-pre2 mm/shmem.c does not look any different so it
> should be fine. :)
> 
> Anyways, what this patch does is allow you to specify the max amount of
> memory tmpfs can use as a percentage of available real ram. This (in my
> eyes) is useful so that you do not have to remember to change the
> setting if you want something other then 50% and some of your ram does
> (and you can't replacew it immediately).
> 
> Usage of this option is as follows:
> 
> tmpfs      /dev/shm tmpfs  rw,size=63%,noauto            0 0
> 
> This is taken from my working system and sets the tmpfs size to 63% of
> my real RAM (256MB). The end result is:
> 
> Filesystem           1k-blocks      Used Available Use% Mounted on
> /dev/shm/tmp            160868      6776    154092   5% /tmp
> 
> I've also tested remounting to silly values (and sane ones) and it all
> works fine with no oopses or freezes and the correct values appearing
> in df.
> 
> All up I feel safer using this then a hard value.

What does tmpfs have to do with ram size?  Its swappable.  This _might_ be
useful for ramfs but for tmpfs, IMHO, its not a good idea.

Ed Tomlinson

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Hugh Dickins]

On Tue, 1 Apr 2003, Ed Tomlinson wrote:
> 
> What does tmpfs have to do with ram size?  Its swappable.  This _might_ be
> useful for ramfs but for tmpfs, IMHO, its not a good idea.

The default size of a tmpfs filesystem is 50% of RAM.  That can be
overridden by setting the size explicitly, CaT is offering percent
instead.  Which is nice, and neatly done.

But I do agree with you that it seems a bit strange, not to take
swap into account at all.  That's one reason I never bothered to
add the feature CaT proposes before.  Just imagine you keep swap
at some % of your RAM, then it'll all scale together.

Hugh

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[CaT]

On Tue, Apr 01, 2003 at 08:38:32AM -0500, Ed Tomlinson wrote:
> CaT wrote:
> > tmpfs      /dev/shm tmpfs  rw,size=63%,noauto            0 0
> > 
> > This is taken from my working system and sets the tmpfs size to 63% of
> > my real RAM (256MB). The end result is:
> > 
> > Filesystem           1k-blocks      Used Available Use% Mounted on
> > /dev/shm/tmp            160868      6776    154092   5% /tmp
> 
> What does tmpfs have to do with ram size?  Its swappable.  This _might_ be
> useful for ramfs but for tmpfs, IMHO, its not a good idea.

Basically:

tmpfs size < ram size + swap size

now, lets say a fair bit of your ram dies. you can't repolace it but the
box will manage to run with your new ram size. if the tmpfs size is
static then you get

tmpfs size > new ram size + swap size

and so any process can fill up your ram+swap by writing to tmpfs when
you use tmpfs for /tmp.

By having the tmpfs size be a function of your ram size (as it is by
default at 50%) you wont get that. Currently my tmpfs size is 63%. If
half my ram dies I can still happily use my laptop without any fiddling
because its new size will by 63% of the new ram size (so it'll be around
80MB rather then 160MB). Now, it does mean my /tmp is smaller but
everything is still functional and it's bigger then my root partition,
which I'd rather nto be actively writing to and only has 20MB free by
design.

-- 
"Other countries of course, bear the same risk. But there's no doubt his
hatred is mainly directed at us. After all this is the guy who tried to
kill my dad."
        - George W. Bush Jr, Leader of the United States Regime
          September 26, 2002 (from a political fundraiser in Houston, Texas)

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Christoph Rohland]

Hi Ed,

On Tue, 01 Apr 2003, Ed Tomlinson wrote:
> What does tmpfs have to do with ram size?  Its swappable.  This
> _might_ be useful for ramfs but for tmpfs, IMHO, its not a good
> idea.

I agree and I think if you add this option it should adjust to a
percentage of (ram + swap). With this it would be a really nice
improvement.
I even had patches for this but to do it efficently you would need to
add some hooks to swapon and swapoff.

Greetings
		Christoph

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Hugh Dickins]

On Tue, 1 Apr 2003, Christoph Rohland wrote:
> On Tue, 01 Apr 2003, Ed Tomlinson wrote:
> > What does tmpfs have to do with ram size?  Its swappable.  This
> > _might_ be useful for ramfs but for tmpfs, IMHO, its not a good
> > idea.
> 
> I agree and I think if you add this option it should adjust to a
> percentage of (ram + swap). With this it would be a really nice
> improvement.
> I even had patches for this but to do it efficently you would need to
> add some hooks to swapon and swapoff.

You surprise me, Christoph, I'd expected you to approve of CaT's.

If tmpfs already defaulted to 50% of ram+swap, then I'd agree
with you.  But it has all along been in terms of RAM, so I think
it's better to continue in that way.  (We could add options to
allow +swap in too, but I'm not terribly interested.)

If people really wanted their tmpfs pages to go out to disk, I think
they'd be choosing a more sophisticated filesystem to manage that:
swap is a vital overflow area for tmpfs, not its home.

Hugh

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Christoph Rohland]

Hi Hugh,

On Tue, 1 Apr 2003, Hugh Dickins wrote:
> You surprise me, Christoph, I'd expected you to approve of CaT's.

look back into your mail store. I said this pretty early. 

> If tmpfs already defaulted to 50% of ram+swap, then I'd agree
> with you.  But it has all along been in terms of RAM, so I think
> it's better to continue in that way.  (We could add options to
> allow +swap in too, but I'm not terribly interested.)

No, I always thought the 50% RAM was a terrible hack. And IMHO it is
one of the biggest usability gaps of tmpfs. 

> If people really wanted their tmpfs pages to go out to disk, I think
> they'd be choosing a more sophisticated filesystem to manage that:
> swap is a vital overflow area for tmpfs, not its home.

I definitely want them to go there. It is a very sophisticated
mechinsm because by adding swap on _any_ storage I can grow my tmpfs
instances even temporarily.

If you now were able to take advantage of additional swap
automatically administration would be a no brainer. Also distributions
could add much saner defaults for /dev/shm or even use it for /tmp.

Greetings
		Christoph

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[William Lee Irwin III]

On Wed, Apr 02, 2003 at 09:20:50AM +0200, Christoph Rohland wrote:
> No, I always thought the 50% RAM was a terrible hack. And IMHO it is
> one of the biggest usability gaps of tmpfs. 

I've never had an issue with it. I just made sure it was large enough.


-- wli

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[CaT]

On Wed, Apr 02, 2003 at 09:20:50AM +0200, Christoph Rohland wrote:
> If you now were able to take advantage of additional swap
> automatically administration would be a no brainer. Also distributions

Sounds like what you want is a dynamically resizing tmpfs based on the
amount of memory (ram+swap) available. That's a much bigger goose to fry
I believe.

Now, even if the percentile patch took into account swap, you'd still
need to remount tmpfs in order to get it to take into account of any
swap you add on the fly.

> could add much saner defaults for /dev/shm or even use it for /tmp.

I use it for /tmp now just fine. :) It's sized at 63% of 256MB of RAM.

-- 
"Other countries of course, bear the same risk. But there's no doubt his
hatred is mainly directed at us. After all this is the guy who tried to
kill my dad."
        - George W. Bush Jr, Leader of the United States Regime
          September 26, 2002 (from a political fundraiser in Houston, Texas)

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Christoph Rohland]

On Thu, 3 Apr 2003, cat@zip.com.au wrote:
> Sounds like what you want is a dynamically resizing tmpfs based on
> the amount of memory (ram+swap) available. That's a much bigger
> goose to fry I believe.
> 
> Now, even if the percentile patch took into account swap, you'd
> still need to remount tmpfs in order to get it to take into account
> of any swap you add on the fly.

No, that's why I said you would need hooks into swapon and
swapoff. Then it would adjust on the fly. Else it's useless from the
usability point of view. With these hooks it's easy to do.

>> could add much saner defaults for /dev/shm or even use it for /tmp.
> 
> I use it for /tmp now just fine. :) It's sized at 63% of 256MB of
> RAM.

And I have set it to 400MB on a 256MB box just fine ;-) How could you
do these two setup generally without knowing your hardware. 20MB tmpfs
on a 40MB machine can be a desaster btw.

Greetings
		Christoph

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Jörn Engel]

On Wed, 2 April 2003 19:33:05 +0200, Christoph Rohland wrote:
> 
> No, that's why I said you would need hooks into swapon and
> swapoff. Then it would adjust on the fly. Else it's useless from the
> usability point of view. With these hooks it's easy to do.

Can you show me the easy part with this setup?
256MB RAM
512MB swap
50% tmpfs (384MB)
fill tmpfs completely
swapoff -a

Jörn

-- 
It's just what we asked for, but not what we want!
-- anonymous

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Hugh Dickins]

On Wed, 2 Apr 2003, Jörn Engel wrote:
> On Wed, 2 April 2003 19:33:05 +0200, Christoph Rohland wrote:
> > 
> > No, that's why I said you would need hooks into swapon and
> > swapoff. Then it would adjust on the fly. Else it's useless from the
> > usability point of view. With these hooks it's easy to do.
> 
> Can you show me the easy part with this setup?
> 256MB RAM
> 512MB swap
> 50% tmpfs (384MB)
> fill tmpfs completely
> swapoff -a

Don't blame Christoph for that, one of these days I'll face up to
my responsibilities and make swapoff fail (probably get itself
OOM-killed) instead of having everything else OOM-killed.

Hugh

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Jörn Engel]

On Wed, 2 April 2003 22:04:10 +0100, Hugh Dickins wrote:
> On Wed, 2 Apr 2003, Jörn Engel wrote:
> > 
> > Can you show me the easy part with this setup?
> > 256MB RAM
> > 512MB swap
> > 50% tmpfs (384MB)
> > fill tmpfs completely
> > swapoff -a
> 
> Don't blame Christoph for that, one of these days I'll face up to
> my responsibilities and make swapoff fail (probably get itself
> OOM-killed) instead of having everything else OOM-killed.

Ack. If you do that, I'd have to agree with Christoph.

Jörn

-- 
Homo Sapiens is a goal, not a description.
-- unknown

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Christoph Rohland]

Hi Hugh,

On Wed, 2 Apr 2003, Hugh Dickins wrote:
> Don't blame Christoph for that, one of these days I'll face up to
> my responsibilities and make swapoff fail (probably get itself
> OOM-killed) instead of having everything else OOM-killed.

The hooks for the accounting would solve this easily: the swapoff hook
would realize that there is not enough space left for the tmpfs
instance and simply return an error. So swapoff would fail.
We would not even stress the vm to swapin all pages until
realizingthat there is not enough memory left.

Greetings
		Christoph

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Hugh Dickins]

On Mon, 7 Apr 2003, Christoph Rohland wrote:
> On Wed, 2 Apr 2003, Hugh Dickins wrote:
> > Don't blame Christoph for that, one of these days I'll face up to
> > my responsibilities and make swapoff fail (probably get itself
> > OOM-killed) instead of having everything else OOM-killed.
> 
> The hooks for the accounting would solve this easily: the swapoff hook
> would realize that there is not enough space left for the tmpfs
> instance and simply return an error. So swapoff would fail.
> We would not even stress the vm to swapin all pages until
> realizingthat there is not enough memory left.

You're supposing that it's tmpfs causing this problem: not at all,
that's just an easy way to show it.  Take away tmpfs, and swapoff
is still liable to hang, with other tasks OOMing (even when strict
no-overcommit accounting: I've not yet added the check it needs).

Hugh

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Christoph Rohland]

Hi Hugh,

On Mon, 7 Apr 2003, Hugh Dickins wrote:
> You're supposing that it's tmpfs causing this problem: not at all,
> that's just an easy way to show it.  Take away tmpfs, and swapoff
> is still liable to hang, with other tasks OOMing (even when strict
> no-overcommit accounting: I've not yet added the check it needs).

No, I know tmpfs is not faulty :-)

But the hook would give tmpfs the chance to say stop before the mm
layer even tries to do something and where it actually fails to do it
right.

Greetings
		Christoph

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Henning P. Schmiedehausen]

=?iso-8859-1?Q?J=F6rn?= Engel <joern@wohnheim.fh-wedel.de> writes:

>On Wed, 2 April 2003 19:33:05 +0200, Christoph Rohland wrote:
>> 
>> No, that's why I said you would need hooks into swapon and
>> swapoff. Then it would adjust on the fly. Else it's useless from the
>> usability point of view. With these hooks it's easy to do.

>Can you show me the easy part with this setup?
>256MB RAM
>512MB swap
>50% tmpfs (384MB)
>fill tmpfs completely
>swapoff -a

# swapoff -a
swapoff: device is busy

# umount /tmpfs
# swapoff -a
#

... in an ideal world, of course.

	Regards
		Henning



-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen          INTERMETA GmbH
hps@intermeta.de        +49 9131 50 654 0   http://www.intermeta.de/

Java, perl, Solaris, Linux, xSP Consulting, Web Services 
freelance consultant -- Jakarta Turbine Development  -- hero for hire

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[CaT]

On Wed, Apr 02, 2003 at 07:33:05PM +0200, Christoph Rohland wrote:
> On Thu, 3 Apr 2003, cat@zip.com.au wrote:
> > Sounds like what you want is a dynamically resizing tmpfs based on
> > the amount of memory (ram+swap) available. That's a much bigger
> > goose to fry I believe.
> > 
> > Now, even if the percentile patch took into account swap, you'd
> > still need to remount tmpfs in order to get it to take into account
> > of any swap you add on the fly.
> 
> No, that's why I said you would need hooks into swapon and
> swapoff. Then it would adjust on the fly. Else it's useless from the

Ahh. Missed that. Sorry.

> usability point of view. With these hooks it's easy to do.

Yes. That'd solve that issue. You'd have it resize itself to your fixed
% of the new totals.

> >> could add much saner defaults for /dev/shm or even use it for /tmp.
> > 
> > I use it for /tmp now just fine. :) It's sized at 63% of 256MB of
> > RAM.
> 
> And I have set it to 400MB on a 256MB box just fine ;-) How could you

:)

> do these two setup generally without knowing your hardware. 20MB tmpfs

If you want to take swap into account you still can't but then this
patch changes nothing for you in either case. You'd still specify 400MB
or (now) 156%.

> on a 40MB machine can be a desaster btw.

It can be, depending on what you do with it, obviously. That's not an
argument against this and I'm not actually against it myself. I just
want the current patch in cos at least it's a step in the right
direction, it's simple and it ups the usability of tmpfs (and it stops
me from having to patch my kernel each and every time ;).

Personally I'd like a flag to set wether or not to take it into account
because it introduces a 2nd potential variable number into the equation
and I don't want my setting depending on swap suddenly dropping off Gods
green, somewhat polluted and messy, earth. I'd be in the same boat I was
in before the patch.

-- 
"Other countries of course, bear the same risk. But there's no doubt his
hatred is mainly directed at us. After all this is the guy who tried to
kill my dad."
        - George W. Bush Jr, Leader of the United States Regime
          September 26, 2002 (from a political fundraiser in Houston, Texas)

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[CaT]

[-- Attachment #1: Type: text/plain, Size: 1178 bytes --]

On Thu, Apr 03, 2003 at 03:35:43PM +1000, CaT wrote:
> > And I have set it to 400MB on a 256MB box just fine ;-) How could you
> 
> :)
> 
> > do these two setup generally without knowing your hardware. 20MB tmpfs
> 
> If you want to take swap into account you still can't but then this
> patch changes nothing for you in either case. You'd still specify 400MB
> or (now) 156%.

Well, I got bored.

size=<ram>%+<swap>%

So size=100%+50% will set the tmpfs size at 100% the size of your ram
+ 50% the size of your swap.

It's just an idea. It's still simple and gives you what you seem to want
(in part... there's no dynamic changes on swapon/off) and what I want
(the ability to set the size off just real ram).

It seems like a decent compromise so I'm sure someone'll hate it. :)

-- 
Martin's distress was in contrast to the bitter satisfaction of some
of his fellow marines as they surveyed the scene. "The Iraqis are sick
people and we are the chemotherapy," said Corporal Ryan Dupre. "I am
starting to hate this country. Wait till I get hold of a friggin' Iraqi.
No, I won't get hold of one. I'll just kill him."
	- http://www.informationclearinghouse.info/article2479.htm

[-- Attachment #2: add-percentile-ram+swap-size-tmpfs-2.5.66.patch --]
[-- Type: text/plain, Size: 1444 bytes --]

--- mm/shmem.c.old	Sun Mar 30 00:51:39 2003
+++ mm/shmem.c	Thu Apr  3 20:33:04 2003
@@ -34,6 +34,7 @@
 #include <linux/writeback.h>
 #include <linux/vfs.h>
 #include <asm/uaccess.h>
+#include <asm/div64.h>
 
 /* This magic number is used in glibc for posix shared memory */
 #define TMPFS_MAGIC	0x01021994
@@ -1630,6 +1631,25 @@
 		if (!strcmp(this_char,"size")) {
 			unsigned long long size;
 			size = memparse(value,&rest);
+			if (*rest == '%') {
+				size <<= PAGE_SHIFT;
+				size *= totalram_pages;
+				do_div(size, 100);
+				rest++;
+				if(*rest == '+') {
+					unsigned long long swpsize;
+					rest++;
+					value = rest;
+					swpsize = memparse(value,&rest);
+					if (*rest == '%') {
+						swpsize <<= PAGE_SHIFT;
+						swpsize *= nr_swap_pages;
+						do_div(swpsize, 100);
+						rest++;
+					}
+					size+=swpsize;
+				}
+			}
 			if (*rest)
 				goto bad_val;
 			*blocks = size >> PAGE_CACHE_SHIFT;
@@ -1695,7 +1715,6 @@
 	uid_t uid = current->fsuid;
 	gid_t gid = current->fsgid;
 	struct shmem_sb_info *sbinfo;
-	struct sysinfo si;
 	int err = -ENOMEM;
 
 	sbinfo = kmalloc(sizeof(struct shmem_sb_info), GFP_KERNEL);
@@ -1708,8 +1727,7 @@
 	 * Per default we only allow half of the physical ram per
 	 * tmpfs instance
 	 */
-	si_meminfo(&si);
-	blocks = inodes = si.totalram / 2;
+	blocks = inodes = totalram_pages / 2;
 
 #ifdef CONFIG_TMPFS
 	if (shmem_parse_options(data, &mode, &uid, &gid, &blocks, &inodes)) {

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Jörn Engel]

On Tue, 1 April 2003 18:27:47 +0200, Christoph Rohland wrote:
> 
> I agree and I think if you add this option it should adjust to a
> percentage of (ram + swap). With this it would be a really nice
> improvement.
> I even had patches for this but to do it efficently you would need to
> add some hooks to swapon and swapoff.

Which means we have two orthogonal problems to deal with. Adding the
percent option and "fixing" it to take swap into account.

Not completely orthogonal, though, since the "fix" actually changes
semantics and might break some peoples setups. Take 3x50% of RAM and
twice the RAM as swap, for example.

Jörn

-- 
They laughed at Galileo.  They laughed at Copernicus.  They laughed at
Columbus. But remember, they also laughed at Bozo the Clown.
-- unknown

RE: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Robert White]

This (using swap as part of the tmpfs type system) is what happens on a Sun.
I was disappointed (surprised even) in the Linux implementations because
mounting a truly temporary /tmp was what I wanted it for.

I would like to see a tmpfs (swapfs?) that did presume that files not in use
(lately?) would migrate out of my valuable RAM and onto the super-cheap swap
device.

IMHO of course.

Rob.

-----Original Message-----
From: linux-kernel-owner@vger.kernel.org
[mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Christoph
Rohland
Sent: Tuesday, April 01, 2003 8:28 AM
To: tomlins@cam.org
Cc: CaT; linux-kernel@vger.kernel.org; Hugh Dickins
Subject: Re: PATCH: allow percentile size of tmpfs (2.5.66 /
2.4.20-pre2)


Hi Ed,

On Tue, 01 Apr 2003, Ed Tomlinson wrote:
> What does tmpfs have to do with ram size?  Its swappable.  This
> _might_ be useful for ramfs but for tmpfs, IMHO, its not a good
> idea.

I agree and I think if you add this option it should adjust to a
percentage of (ram + swap). With this it would be a really nice
improvement.
I even had patches for this but to do it efficently you would need to
add some hooks to swapon and swapoff.

Greetings
		Christoph


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Hugh Dickins]

On Wed, 2 Apr 2003, Robert White wrote:

> This (using swap as part of the tmpfs type system) is what happens on a Sun.
> I was disappointed (surprised even) in the Linux implementations because
> mounting a truly temporary /tmp was what I wanted it for.

The Linux implementation of tmpfs _does_ use swap:
tmpfs data pages go out to swap under memory pressure.

> I would like to see a tmpfs (swapfs?) that did presume that files not in use
> (lately?) would migrate out of my valuable RAM and onto the super-cheap swap
> device.

RE: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Robert White]

hmm, I think, on reflection, that I knew that because of having encountered
the swapoff problem after putting a bunch of stuff in a tmpfs...

Regardless,

Since the tmpfs occupies both ram and swap, then having the absolute size
limit based only on ram seems odd.

IMHO of course 8-)

Rob.


-----Original Message-----
From: Hugh Dickins [mailto:hugh@veritas.com]
Sent: Thursday, April 03, 2003 12:35 AM
To: Robert White
Cc: Christoph Rohland; tomlins@cam.org; CaT;
linux-kernel@vger.kernel.org
Subject: RE: PATCH: allow percentile size of tmpfs (2.5.66 /
2.4.20-pre2)


On Wed, 2 Apr 2003, Robert White wrote:

> This (using swap as part of the tmpfs type system) is what happens on a
Sun.
> I was disappointed (surprised even) in the Linux implementations because
> mounting a truly temporary /tmp was what I wanted it for.

The Linux implementation of tmpfs _does_ use swap:
tmpfs data pages go out to swap under memory pressure.

> I would like to see a tmpfs (swapfs?) that did presume that files not in
use
> (lately?) would migrate out of my valuable RAM and onto the super-cheap
swap
> device.

RE: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Mikael Starvik]

>What does tmpfs have to do with ram size?  Its swappable.  This _might_ be
>useful for ramfs but for tmpfs, IMHO, its not a good idea.

All systems that uses tmpfs doesn't necessairly have a swap, 
tmpfs is used in several diskless embedded systems.

/Mikael

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Christoph Rohland]

Hi Mikael,

On Tue, 1 Apr 2003, Mikael Starvik wrote:
> All systems that uses tmpfs doesn't necessairly have a swap, 
> tmpfs is used in several diskless embedded systems.

But on these systems you better use ramfs.

Greetings
		Christoph

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Daniel Egger]

[-- Attachment #1: Type: text/plain, Size: 435 bytes --]

Am Die, 2003-04-01 um 18.24 schrieb Christoph Rohland:

> But on these systems you better use ramfs.

Just curious: Why? I'm using tmpfs on these systems and I'm rather
satisfied with it; especially the option to limit the amount of space
makes it rather useful. According to the documentation ramfs is most
useful as an educational example how to write filesystems not as a 
real filesystem...

-- 
Servus,
       Daniel

[-- Attachment #2: Dies ist ein digital signierter Nachrichtenteil --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Hugh Dickins]

On Tue, 1 Apr 2003, Daniel Egger wrote:
> Am Die, 2003-04-01 um 18.24 schrieb Christoph Rohland:
> 
> > But on these systems you better use ramfs.
> 
> Just curious: Why? I'm using tmpfs on these systems and I'm rather
> satisfied with it; especially the option to limit the amount of space
> makes it rather useful. According to the documentation ramfs is most
> useful as an educational example how to write filesystems not as a 
> real filesystem...

Simply because quite a lot of the tmpfs code is concerned with moving
pages between ram and swap: if you've limited ram and no swap, you may
not want to waste your ram on that code!  One day I might try applying
#ifdef CONFIG_SWAPs within mm/shmem.c; but I might well not, it could
get ugly, and looks rudimentary elsewhere - do we intend to get serious
about CONFIG_SWAP?

Hugh

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Daniel Egger]

[-- Attachment #1: Type: text/plain, Size: 674 bytes --]

Am Die, 2003-04-01 um 21.25 schrieb Hugh Dickins:

> Simply because quite a lot of the tmpfs code is concerned with moving
> pages between ram and swap: if you've limited ram and no swap, you may
> not want to waste your ram on that code!  One day I might try applying
> #ifdef CONFIG_SWAPs within mm/shmem.c; but I might well not, it could
> get ugly, and looks rudimentary elsewhere - do we intend to get serious
> about CONFIG_SWAP?

It would be more waste to have both ramfs and tmpfs in compiled form
since the whole system is intended to run on embedded systems with CF as
well as on faster machines with harddrive and swap.

-- 
Servus,
       Daniel

[-- Attachment #2: Dies ist ein digital signierter Nachrichtenteil --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Christoph Rohland]

Hi Daniel,

On 01 Apr 2003, Daniel Egger wrote:
> Just curious: Why? I'm using tmpfs on these systems and I'm rather
> satisfied with it; especially the option to limit the amount of
> space makes it rather useful. According to the documentation ramfs
> is most useful as an educational example how to write filesystems
> not as a real filesystem...

Uuh, now you are beating me with my old statements ;-)

tmpfs has the drawback that the in memory data structures are bigger
than ramfs'. But the core of tmpfs is always compiled in for anonymous
shared memory. And it has size limits. So you are probably right, that
tmpfs is the right choice. 

But you are arguing at a corner case. tmpfs is IMHO more often used on
machines with swap and (at least for me) the use of swap as store for
temporary data is the big point to use tmpfs. So the percentile should
take swap into account.

Greetings
		Christoph

RE: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Hua Zhong]

There is at least one case that ramfs works but tmpfs doesn't.

If you have a loopback file A, and the following will fail in 2.4:

mount -t tmpfs tmpfs /mnt/tmp
extract file A to /mnt/tmp/A
mount -t ext2 -o loop /mnt/tmp /mnt/loopback

You'll get "ioctl: LOOP_SET_FD: Invalid argument".

But ramfs works great.

Is this a bug or feature?

> Uuh, now you are beating me with my old statements ;-)
>
> tmpfs has the drawback that the in memory data structures are bigger
> than ramfs'. But the core of tmpfs is always compiled in for anonymous
> shared memory. And it has size limits. So you are probably right, that
> tmpfs is the right choice.
>
> But you are arguing at a corner case. tmpfs is IMHO more often used on
> machines with swap and (at least for me) the use of swap as store for
> temporary data is the big point to use tmpfs. So the percentile should
> take swap into account.
>
> Greetings
> 		Christoph
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

RE: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Hugh Dickins]

On Tue, 1 Apr 2003, Hua Zhong wrote:
> There is at least one case that ramfs works but tmpfs doesn't.
> 
> If you have a loopback file A, and the following will fail in 2.4:
> 
> mount -t tmpfs tmpfs /mnt/tmp
> extract file A to /mnt/tmp/A
> mount -t ext2 -o loop /mnt/tmp /mnt/loopback
> 
> You'll get "ioctl: LOOP_SET_FD: Invalid argument".
> 
> But ramfs works great.
> 
> Is this a bug or feature?

Feature: the way tmpfs allows a page to move between swap and memory
makes loopback harder to support than it is for all-in-memory ramfs.

I used to have the patches to enable loopback on tmpfs in 2.4 (well,
I still have them, but 2.4 and tmpfs have both moved on), but shifted
attention to 2.5.  One day I'll clean them up, but that day is not soon.

Hugh

RE: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Robert White]

I presume the below was a typo and In the second mount it should be

"mount -t ext2 -o loop /mnt/tmp/A /mnt/loopback"

instead of just

"mount -t ext2 -o loop /mnt/tmp /mnt/loopback"

Else wise the directory name "/mnt/tmp" would most certainly be a bad target
for a loopback mount.

Rob.


-----Original Message-----
From: linux-kernel-owner@vger.kernel.org
[mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Hua Zhong
Sent: Tuesday, April 01, 2003 11:55 PM
To: Christoph Rohland; Daniel Egger
Cc: linux-kernel@vger.kernel.org
Subject: RE: PATCH: allow percentile size of tmpfs (2.5.66 /
2.4.20-pre2)


mount -t tmpfs tmpfs /mnt/tmp
extract file A to /mnt/tmp/A
mount -t ext2 -o loop /mnt/tmp /mnt/loopback

You'll get "ioctl: LOOP_SET_FD: Invalid argument".

RE: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Hua Zhong]

Yes, you are absolutely right...

> I presume the below was a typo and In the second mount it should be
> 
> "mount -t ext2 -o loop /mnt/tmp/A /mnt/loopback"
> 
> instead of just
> 
> "mount -t ext2 -o loop /mnt/tmp /mnt/loopback"
> 
> Else wise the directory name "/mnt/tmp" would most certainly be a 
> bad target
> for a loopback mount.
> 
> Rob.
> 
> 
> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org
> [mailto:linux-kernel-owner@vger.kernel.org]On Behalf Of Hua Zhong
> Sent: Tuesday, April 01, 2003 11:55 PM
> To: Christoph Rohland; Daniel Egger
> Cc: linux-kernel@vger.kernel.org
> Subject: RE: PATCH: allow percentile size of tmpfs (2.5.66 /
> 2.4.20-pre2)
> 
> 
> mount -t tmpfs tmpfs /mnt/tmp
> extract file A to /mnt/tmp/A
> mount -t ext2 -o loop /mnt/tmp /mnt/loopback
> 
> You'll get "ioctl: LOOP_SET_FD: Invalid argument".
> 
> 

PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[CaT]

[-- Attachment #1: Type: text/plain, Size: 1347 bytes --]

I believe the patch below will apply to both the above (I know it does
to 2.5.66 and 2.4.20-pre2 mm/shmem.c does not look any different so it
should be fine. :)

Anyways, what this patch does is allow you to specify the max amount of
memory tmpfs can use as a percentage of available real ram. This (in my
eyes) is useful so that you do not have to remember to change the
setting if you want something other then 50% and some of your ram does
(and you can't replacew it immediately).

Usage of this option is as follows:

tmpfs      /dev/shm tmpfs  rw,size=63%,noauto            0 0

This is taken from my working system and sets the tmpfs size to 63% of
my real RAM (256MB). The end result is:

Filesystem           1k-blocks      Used Available Use% Mounted on
/dev/shm/tmp            160868      6776    154092   5% /tmp

I've also tested remounting to silly values (and sane ones) and it all
works fine with no oopses or freezes and the correct values appearing
in df.

All up I feel safer using this then a hard value.

Please apply. :)

-- 
"Other countries of course, bear the same risk. But there's no doubt his
hatred is mainly directed at us. After all this is the guy who tried to
kill my dad."
        - George W. Bush Jr, Leader of the United States Regime
          September 26, 2002 (from a political fundraiser in Houston, Texas)

[-- Attachment #2: add-perc-tmpfs-size-2.5.66.patch --]
[-- Type: text/plain, Size: 441 bytes --]

--- linux/mm/shmem.c.old	Sun Mar 30 00:51:39 2003
+++ linux/mm/shmem.c	Sun Mar 30 03:23:47 2003
@@ -1630,6 +1630,12 @@
 		if (!strcmp(this_char,"size")) {
 			unsigned long long size;
 			size = memparse(value,&rest);
+			if (*rest == '%') {
+				struct sysinfo si;
+				si_meminfo(&si);
+				size = (si.totalram << PAGE_CACHE_SHIFT) / 100 * size;
+				rest++;
+			}
 			if (*rest)
 				goto bad_val;
 			*blocks = size >> PAGE_CACHE_SHIFT;

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Xavier Bestel]

                        size = memparse(value,&rest);
+                       if (*rest == '%') {
+                               struct sysinfo si;
+                               si_meminfo(&si);
+                               size = (si.totalram << PAGE_CACHE_SHIFT) / 100 * size;

(si.totalram << PAGE_CACHE_SHIFT) * size / 100;
would have been better precision-wise.

	Xav

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Hugh Dickins]

On 1 Apr 2003, Xavier Bestel wrote:
>                         size = memparse(value,&rest);
> +                       if (*rest == '%') {
> +                               struct sysinfo si;
> +                               si_meminfo(&si);
> +                               size = (si.totalram << PAGE_CACHE_SHIFT) / 100 * size;
> 
> (si.totalram << PAGE_CACHE_SHIFT) * size / 100;
> would have been better precision-wise.

Hardly, it'll overflow in even more cases
than CaT's (si.totalram << PAGE_CACHE_SHIFT).
I'll take a look at this later, not right now.

Hugh

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[CaT]

On Tue, Apr 01, 2003 at 12:11:46PM +0100, Hugh Dickins wrote:
> On 1 Apr 2003, Xavier Bestel wrote:
> >                         size = memparse(value,&rest);
> > +                       if (*rest == '%') {
> > +                               struct sysinfo si;
> > +                               si_meminfo(&si);
> > +                               size = (si.totalram << PAGE_CACHE_SHIFT) / 100 * size;
> > 
> > (si.totalram << PAGE_CACHE_SHIFT) * size / 100;
> > would have been better precision-wise.
> 
> Hardly, it'll overflow in even more cases
> than CaT's (si.totalram << PAGE_CACHE_SHIFT).

Yes. I had it initially as Xavier suggested but after thinking about it
a bit I felt that making the value smaller and -then- bigger was safer.

> I'll take a look at this later, not right now.

It is still an unsigned long long int so (AFAIK) it wont overflow till
it hits 18,446,744,073,709,551,615. Now... if you have that much ram...
wow! :)

Basically it'll only overflow for astoundingly stupid values of size and
when it comes down to it, I think you could still make it overflow by
setting a REALLY large static value for size anyways (liek greater then
the above). Same issue as far as I can see.

-- 
"Other countries of course, bear the same risk. But there's no doubt his
hatred is mainly directed at us. After all this is the guy who tried to
kill my dad."
        - George W. Bush Jr, Leader of the United States Regime
          September 26, 2002 (from a political fundraiser in Houston, Texas)

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[Hugh Dickins]

On Wed, 2 Apr 2003, CaT wrote:
> > 
> > Hardly, it'll overflow in even more cases
> > than CaT's (si.totalram << PAGE_CACHE_SHIFT).
> 
> Yes. I had it initially as Xavier suggested but after thinking about it
> a bit I felt that making the value smaller and -then- bigger was safer.
> 
> > I'll take a look at this later, not right now.
> 
> It is still an unsigned long long int so (AFAIK) it wont overflow till
> it hits 18,446,744,073,709,551,615. Now... if you have that much ram...
> wow! :)

There's plenty of room in unsigned long long size, yes, but si.totalram
is only an unsigned long, so the arithmetic as you have it starts out
overflowing an unsigned long.

I don't know yet what it should say: RH2.96-110 is getting confused
by the do_div(size, 100) I have there (to respect Xavier's point),
and this is definitely _not_ worth adding a compiler dependency for.

Hugh

Re: PATCH: allow percentile size of tmpfs (2.5.66 / 2.4.20-pre2)

[CaT]

On Tue, Apr 01, 2003 at 03:43:09PM +0100, Hugh Dickins wrote:
> There's plenty of room in unsigned long long size, yes, but si.totalram
> is only an unsigned long, so the arithmetic as you have it starts out
> overflowing an unsigned long.

Ahh yes.

> I don't know yet what it should say: RH2.96-110 is getting confused
> by the do_div(size, 100) I have there (to respect Xavier's point),
> and this is definitely _not_ worth adding a compiler dependency for.

Would a cast be bad? ie:

--- linux/mm/shmem.c.old	Sun Mar 30 00:51:39 2003
+++ linux/mm/shmem.c	Sun Mar 30 03:23:47 2003
@@ -1630,6 +1630,12 @@
 		if (!strcmp(this_char,"size")) {
 			unsigned long long size;
 			size = memparse(value,&rest);
+			if (*rest == '%') {
+				struct sysinfo si;
+				si_meminfo(&si);
+				size = ((unsigned long long)si.totalram << PAGE_CACHE_SHIFT) / 100 * size;
+				rest++;
+			}
 			if (*rest)
 				goto bad_val;
 			*blocks = size >> PAGE_CACHE_SHIFT;
-- 
"Other countries of course, bear the same risk. But there's no doubt his
hatred is mainly directed at us. After all this is the guy who tried to
kill my dad."
        - George W. Bush Jr, Leader of the United States Regime
          September 26, 2002 (from a political fundraiser in Houston, Texas)