Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Ihar 'Philips' Filipau]

Roger Luethi wrote:
> On Sat, 27 Sep 2003 20:13:48 +0200, Ihar 'Philips' Filipau wrote:
> 
>>Roger Luethi wrote:
>>
>>>On Sun, 28 Sep 2003 01:26:34 +1000, Jason Lewis wrote:
>>>
>>>>0 12      0   3424    816   6008    0    0 19712     0 5519  3184  0 12  
>>>>0 87
>>>
>>>         ^^^^
>>>Looks like you don't have swap enabled. Are successful 2.4 runs with or
>>>without swap?
>>
>>   I'm running RH stock 2.4.20-20.9 without swap for around month.
>>   OOo, Mozilla, eDonkey & heaps of xterms. Even evaluation of VMware 
>>with Win2K inside was Ok.
>>   On average: much better experience.
> 
> Better than with swap? Or better than 2.6?
> 

   Better than with swap. This is production workstation - I cannot test 
something on it :-(

> 
>>$ free
>>             total       used       free     shared    buffers     cached
>>Mem:        513872     507128       6744          0      32784     341404
> 
> The initial post was about a 48 MB machine. 10% of what you have. The
> poster's system is paging like crazy -- since all dirty pages without a
> mapping are pinned in memory, it must shuffle around the rest.
> 

    Sorry, I even marked $subject as [OT].
    I'm answering the question '2.4 without swap' - Yes. It is. Works. 
No  problems.

    <rant>'Paging like crazy' became for me a synonym of Linux. It 
doesn't matter how much memory you have. Less == worse. Developers 
stopped testing VMM regression on low-memory computers long time ago. 
<sarcasm>We have now fashion for clusters and numas. And a lot of swap 
on very fast raids. <sarcasm size=+100%>After all it is cheap. Just 
couple of thousands greenbacks. </sarcasm> </sarcasm> It was really 
funny when developers on LKML were sugesting to buy another hdd for 
swap. Very funny.</rant>

    Unfortunately I'm not a specialist in VMM...
    As I see there is not that much edge case testing going around.

-- 
Ihar 'Philips' Filipau  / with best regards from Saarbruecken.
--
   "... and for $64000 question, could you get yourself vaguely
      familiar with the notion of on-topic posting?"
				-- Al Viro @ LKML

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[William Lee Irwin III]

On Sat, Sep 27, 2003 at 09:59:55PM +0200, Ihar 'Philips' Filipau wrote:
>    Sorry, I even marked $subject as [OT].
>    I'm answering the question '2.4 without swap' - Yes. It is. Works. 
> No  problems.
>    <rant>'Paging like crazy' became for me a synonym of Linux. It 
> doesn't matter how much memory you have. Less == worse. Developers 
> stopped testing VMM regression on low-memory computers long time ago. 
> <sarcasm>We have now fashion for clusters and numas. And a lot of swap 
> on very fast raids. <sarcasm size=+100%>After all it is cheap. Just 
> couple of thousands greenbacks. </sarcasm> </sarcasm> It was really 
> funny when developers on LKML were sugesting to buy another hdd for 
> swap. Very funny.</rant>
>    Unfortunately I'm not a specialist in VMM...
>    As I see there is not that much edge case testing going around.

It's known what has to be done for it. AFAICT upstream doesn't like
the answers and just says "throw hardware at it". I've written it off
as a lost cause, though I was at one time interested.


-- wli

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Roger Luethi]

On Sat, 27 Sep 2003 21:59:55 +0200, Ihar 'Philips' Filipau wrote:
>   Better than with swap. This is production workstation - I cannot test 
> something on it :-(

I don't think there's much risk involved in running 2.[56]. If it doesn't
boot, you can go back to 2.4. If it does boot, it won't eat your data.
YMMV, of course.

>    <rant>'Paging like crazy' became for me a synonym of Linux. It 
> doesn't matter how much memory you have. Less == worse. Developers 

Oh, it does matter. My workstation has 1 GB RAM and 2 GB swap and I hardly
see any problems with paging <g>.

> stopped testing VMM regression on low-memory computers long time ago. 

Many of the best Linux devs these days work for companies and organizations
that are in the business of selling or using big iron. They have people on
staff who test the kernel and whine if it fails to run well with a ton of
memory and a gazillion CPUs (plus people who actually fix it); it's not
their job to care about low end systems. However, you don't have to be a VM
hacker to run a bunch of benchmarks to compare performance and point out
regressions. People tend to listen if you offer some hard numbers instead
of a rant about how nobody cares about the low end.

Roger

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Roger Luethi]

On Sat, 27 Sep 2003 13:13:47 -0700, William Lee Irwin III wrote:
> >    As I see there is not that much edge case testing going around.
> 
> It's known what has to be done for it. AFAICT upstream doesn't like
> the answers and just says "throw hardware at it". I've written it off
> as a lost cause, though I was at one time interested.

Well, _I_ don't know that. What are the answers? And while we're at it,
what's the problem, exactly?

Roger

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Ihar 'Philips' Filipau]

Roger Luethi wrote:
> On Sat, 27 Sep 2003 21:59:55 +0200, Ihar 'Philips' Filipau wrote:
> 
>>  Better than with swap. This is production workstation - I cannot test 
>>something on it :-(
> 
> 
> I don't think there's much risk involved in running 2.[56]. If it doesn't
> boot, you can go back to 2.4. If it does boot, it won't eat your data.
> YMMV, of course.
> 

   Data corruption? My back-up of work?
   I cannot back-up my data since my comp by itself is backup :-)

> 
>>   <rant>'Paging like crazy' became for me a synonym of Linux. It 
>>doesn't matter how much memory you have. Less == worse. Developers 
> 
> 
> Oh, it does matter. My workstation has 1 GB RAM and 2 GB swap and I hardly
> see any problems with paging <g>.
> 

   Because your workload doesn't hit the 1GB limit.
   Actually we just do not have fast enough I/O + CPU to utilize 1GB of 
RAM efficiently.

   But if you will go into 128MB of RAM - you will see difference, where 
should be no difference.

   Let's say (my personal exp.) cp'ing of kernel source with 0.5/0.25 GB 
RAM dosn't differ. Aproximately the same time. 0.25GB little bit faster 
- but it can be written off to noise. But try to do the same cp with 
0.125GB - this cp (as of RH 2.4.20-20.9 +ext3 -swap) takes _*two*_ times 
longer. Should it be?

> 
>>stopped testing VMM regression on low-memory computers long time ago. 
> 
> Many of the best Linux devs these days work for companies and organizations
> that are in the business of selling or using big iron. They have people on

   Indeed.

-- 
Ihar 'Philips' Filipau  / with best regards from Saarbruecken.
--
   "... and for $64000 question, could you get yourself vaguely
      familiar with the notion of on-topic posting?"
				-- Al Viro @ LKML

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Mikulas Patocka]

> > Oh, it does matter. My workstation has 1 GB RAM and 2 GB swap and I hardly
> > see any problems with paging <g>.
> >
>
>    Because your workload doesn't hit the 1GB limit.
>    Actually we just do not have fast enough I/O + CPU to utilize 1GB of
> RAM efficiently.
>
>    But if you will go into 128MB of RAM - you will see difference, where
> should be no difference.
>
>    Let's say (my personal exp.) cp'ing of kernel source with 0.5/0.25 GB
> RAM dosn't differ. Aproximately the same time. 0.25GB little bit faster
> - but it can be written off to noise. But try to do the same cp with
> 0.125GB - this cp (as of RH 2.4.20-20.9 +ext3 -swap) takes _*two*_ times
> longer. Should it be?

Yes, it should. If you have 0.25GB, it can be copied into cache. If you
have 0.125GB, it doesn't fit there.

Mikulas

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Ihar 'Philips' Filipau]

Mikulas Patocka wrote:
>>>Oh, it does matter. My workstation has 1 GB RAM and 2 GB swap and I hardly
>>>see any problems with paging <g>.
>>
>>   Because your workload doesn't hit the 1GB limit.
>>   Actually we just do not have fast enough I/O + CPU to utilize 1GB of
>>RAM efficiently.
>>
>>   But if you will go into 128MB of RAM - you will see difference, where
>>should be no difference.
>>
>>   Let's say (my personal exp.) cp'ing of kernel source with 0.5/0.25 GB
>>RAM dosn't differ. Aproximately the same time. 0.25GB little bit faster
>>- but it can be written off to noise. But try to do the same cp with
>>0.125GB - this cp (as of RH 2.4.20-20.9 +ext3 -swap) takes _*two*_ times
>>longer. Should it be?
> 
> Yes, it should. If you have 0.25GB, it can be copied into cache. If you
> have 0.125GB, it doesn't fit there.
> 

   So you want to say to effectively copy (or whatever) 40GB harddrive I 
have to have 40GB of RAM? Ridiculous.
   Especially if copying is done in 4k lumps. (cp's default buffer)

<sarcasm flavour=sad> Hopefully not everyone shares your opinion. </sarcasm>

-- 
Ihar 'Philips' Filipau  / with best regards from Saarbruecken.
--
   "... and for $64000 question, could you get yourself vaguely
      familiar with the notion of on-topic posting?"
				-- Al Viro @ LKML

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Mikulas Patocka]

On Sun, 28 Sep 2003, Ihar 'Philips' Filipau wrote:

> Mikulas Patocka wrote:
> >>>Oh, it does matter. My workstation has 1 GB RAM and 2 GB swap and I hardly
> >>>see any problems with paging <g>.
> >>
> >>   Because your workload doesn't hit the 1GB limit.
> >>   Actually we just do not have fast enough I/O + CPU to utilize 1GB of
> >>RAM efficiently.
> >>
> >>   But if you will go into 128MB of RAM - you will see difference, where
> >>should be no difference.
> >>
> >>   Let's say (my personal exp.) cp'ing of kernel source with 0.5/0.25 GB
> >>RAM dosn't differ. Aproximately the same time. 0.25GB little bit faster
> >>- but it can be written off to noise. But try to do the same cp with
> >>0.125GB - this cp (as of RH 2.4.20-20.9 +ext3 -swap) takes _*two*_ times
> >>longer. Should it be?
> >
> > Yes, it should. If you have 0.25GB, it can be copied into cache. If you
> > have 0.125GB, it doesn't fit there.
> >
>
>    So you want to say to effectively copy (or whatever) 40GB harddrive I
> have to have 40GB of RAM? Ridiculous.

If you have plenty of RAM and you use cp command, you read data from disk
and copy them to RAM only, the kernel doesn't write them to disk.

If you copy more data than RAM, the data are read from disk and written to
disk again because RAM can't hold them. That's why you see 2 times
slowdown when you use machine with less RAM. It's perfectly OK.

Other unices (Solaris, IRIX) don't have dynamic cache that resizes
according to RAM, always copy data from disk to disk and so they are
order-of-magnitude slower than Linux. Did you ever tried to grep for
symbols in linux source tree unpacked on Solaris box with 512MB RAM? ---
it's just as slow as linux with very few RAM, because Solaris can't cache
data in RAM.

>    Especially if copying is done in 4k lumps. (cp's default buffer)

It doesn't matter what's copy buffer size (except for syscall overhead).
Kernel does both read-ahead and write-behind with request joining.

> <sarcasm flavour=sad> Hopefully not everyone shares your opinion. </sarcasm>

Mikulas

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Mr. James W. Laferriere]

	Hello Ihar ,

On Sun, 28 Sep 2003, Ihar 'Philips' Filipau wrote:
> Mikulas Patocka wrote:
> >>>Oh, it does matter. My workstation has 1 GB RAM and 2 GB swap and I hardly
> >>>see any problems with paging <g>.
> >>   Because your workload doesn't hit the 1GB limit.
> >>   Actually we just do not have fast enough I/O + CPU to utilize 1GB of
> >>RAM efficiently.
> >>   But if you will go into 128MB of RAM - you will see difference, where
> >>should be no difference.
> >>   Let's say (my personal exp.) cp'ing of kernel source with 0.5/0.25 GB
> >>RAM dosn't differ. Aproximately the same time. 0.25GB little bit faster
> >>- but it can be written off to noise. But try to do the same cp with
> >>0.125GB - this cp (as of RH 2.4.20-20.9 +ext3 -swap) takes _*two*_ times
> >>longer. Should it be?
> > Yes, it should. If you have 0.25GB, it can be copied into cache. If you
> > have 0.125GB, it doesn't fit there.
>    So you want to say to effectively copy (or whatever) 40GB harddrive I
> have to have 40GB of RAM? Ridiculous.
>    Especially if copying is done in 4k lumps. (cp's default buffer)
> <sarcasm flavour=sad> Hopefully not everyone shares your opinion. </sarcasm>
	If I am correct ,  I beleive he is speaking of the amount of
	MEMORY needed to cache the copy of file data WITHOUT a swap
	partit. or file .  Hth ,  JimL
-- 
       +------------------------------------------------------------------+
       | James   W.   Laferriere | System    Techniques | Give me VMS     |
       | Network        Engineer |     P.O. Box 854     |  Give me Linux  |
       | babydr@baby-dragons.com | Coudersport PA 16915 |   only  on  AXP |
       +------------------------------------------------------------------+

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Ihar 'Philips' Filipau]

Mikulas Patocka wrote:
> 
>>>
>>>Yes, it should. If you have 0.25GB, it can be copied into cache. If you
>>>have 0.125GB, it doesn't fit there.
>>
>>   So you want to say to effectively copy (or whatever) 40GB harddrive I
>>have to have 40GB of RAM? Ridiculous.
> 
> Other unices (Solaris, IRIX) don't have dynamic cache that resizes
> according to RAM, always copy data from disk to disk and so they are

   It is slow. But when someone compile something on our Linux cross 
compilation server - everyone else is out - "jerky" is not suffieciently 
hard word to describe how it works under load. (I hope 2.6 fixes this.)

   Sun Ultra 10 (Solaris 8) is magnitude slower, but when someone 
compiles something big - people still _*can*_ work on it. "find | xargs 
grep" is still ok. I notice usually background compilations by fact that 
:w takes visibly longer. But I can work.

   That's what matters. Try to work in vim if it is permanently get 
swaped out. _*Very*_ _*very*_ not nice.
   And there is no memory pressure - kernel just decided to enlarge I/O 
cache... 100% stupid.

   I personally prefer to have statical I/O cache - never saw it working 
reliably with dynamic allocation.

> order-of-magnitude slower than Linux. Did you ever tried to grep for
> symbols in linux source tree unpacked on Solaris box with 512MB RAM? ---
> it's just as slow as linux with very few RAM, because Solaris can't cache
> data in RAM.
> 

-- 
Ihar 'Philips' Filipau  / with best regards from Saarbruecken.
--
   "... and for $64000 question, could you get yourself vaguely
      familiar with the notion of on-topic posting?"
				-- Al Viro @ LKML

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Ihar 'Philips' Filipau]

Mr. James W. Laferriere wrote:
>>>Yes, it should. If you have 0.25GB, it can be copied into cache. If you
>>>have 0.125GB, it doesn't fit there.
>>
>>   So you want to say to effectively copy (or whatever) 40GB harddrive I
>>have to have 40GB of RAM? Ridiculous.
>>   Especially if copying is done in 4k lumps. (cp's default buffer)
>><sarcasm flavour=sad> Hopefully not everyone shares your opinion. </sarcasm>
> 
> 	If I am correct ,  I beleive he is speaking of the amount of
> 	MEMORY needed to cache the copy of file data WITHOUT a swap
> 	partit. or file .

   Probably I have misunderstood.
   Can you once again explain it to me - slowly.

-- 
Ihar 'Philips' Filipau  / with best regards from Saarbruecken.
--
   "... and for $64000 question, could you get yourself vaguely
      familiar with the notion of on-topic posting?"
				-- Al Viro @ LKML

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[William Lee Irwin III]

On Sun, Sep 28, 2003 at 07:54:07PM +0200, Ihar 'Philips' Filipau wrote:
>   That's what matters. Try to work in vim if it is permanently get 
> swaped out. _*Very*_ _*very*_ not nice.
>   And there is no memory pressure - kernel just decided to enlarge I/O 
> cache... 100% stupid.
>   I personally prefer to have statical I/O cache - never saw it working 
> reliably with dynamic allocation.

This is a different question from what I had in mind. There are some
controls in 2.6.x to control the relative tendencies to evict unmapped
pagecache vs. unmapping etc. anonymous memory. I'd try adjusting
/proc/sys/vm/swappiness and reporting how (in)effective that is, then
if that still doesn't work, various things can be adjusted.


-- wli

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Nick Piggin]

Roger Luethi wrote:

>On Sat, 27 Sep 2003 13:13:47 -0700, William Lee Irwin III wrote:
>
>>>   As I see there is not that much edge case testing going around.
>>>
>>It's known what has to be done for it. AFAICT upstream doesn't like
>>the answers and just says "throw hardware at it". I've written it off
>>as a lost cause, though I was at one time interested.
>>
>
>Well, _I_ don't know that. What are the answers? And while we're at it,
>what's the problem, exactly?
>
>

This particular problem was fixed nicely by getting the kernel to enable
swap, so I don't think its that bad of a problem.

Anyway, I think answers involve sizing data structures more effectively
on small memory boxes, more VM smarts in overload situations, and
probably most important for desktop use: light weight and unbloated
user level environment.

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[William Lee Irwin III]

On Mon, Sep 29, 2003 at 03:02:28PM +1000, Nick Piggin wrote:
> This particular problem was fixed nicely by getting the kernel to enable
> swap, so I don't think its that bad of a problem.
> Anyway, I think answers involve sizing data structures more effectively
> on small memory boxes, more VM smarts in overload situations, and
> probably most important for desktop use: light weight and unbloated
> user level environment.

The problem statement was unclear, and my original statement about
solutions accordingly inaccurate.


-- wli

Re: [OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Roger Luethi]

On Sat, 27 Sep 2003 20:13:48 +0200, Ihar 'Philips' Filipau wrote:
> Roger Luethi wrote:
> >On Sun, 28 Sep 2003 01:26:34 +1000, Jason Lewis wrote:
> >>0 12      0   3424    816   6008    0    0 19712     0 5519  3184  0 12  
> >>0 87
> >
> >          ^^^^
> >Looks like you don't have swap enabled. Are successful 2.4 runs with or
> >without swap?
> >
> 
>    I'm running RH stock 2.4.20-20.9 without swap for around month.
>    OOo, Mozilla, eDonkey & heaps of xterms. Even evaluation of VMware 
> with Win2K inside was Ok.
>    On average: much better experience.

Better than with swap? Or better than 2.6?

> $ free
>              total       used       free     shared    buffers     cached
> Mem:        513872     507128       6744          0      32784     341404

The initial post was about a 48 MB machine. 10% of what you have. The
poster's system is paging like crazy -- since all dirty pages without a
mapping are pinned in memory, it must shuffle around the rest.

Roger

[OT] No Swap. Re: [BUG 2.6.90-test5] kernel shits itself with 48mb ram under moderate load

[Ihar 'Philips' Filipau]

Roger Luethi wrote:
> On Sun, 28 Sep 2003 01:26:34 +1000, Jason Lewis wrote:
>> 0 12      0   3424    816   6008    0    0 19712     0 5519  3184  0 12  0 87
> 
>           ^^^^
> Looks like you don't have swap enabled. Are successful 2.4 runs with or
> without swap?
> 

    I'm running RH stock 2.4.20-20.9 without swap for around month.
    OOo, Mozilla, eDonkey & heaps of xterms. Even evaluation of VMware 
with Win2K inside was Ok.
    On average: much better experience.

$ free
              total       used       free     shared    buffers     cached
Mem:        513872     507128       6744          0      32784     341404
-/+ buffers/cache:     132940     380932
Swap:            0          0          0

$ vmstat
    procs                      memory      swap          io     system 
     cpu
  r  b  w   swpd   free   buff  cache   si   so    bi    bo   in    cs 
us sy id
  0  0  0      0   6744  32708 341292    0    2     6    24   17    39 
3  1 16


-- 
Ihar 'Philips' Filipau  / with best regards from Saarbruecken.
--
   "... and for $64000 question, could you get yourself vaguely
      familiar with the notion of on-topic posting?"
				-- Al Viro @ LKML