Re: [RFC] Optimization for use-once pages

Re: [RFC] Optimization for use-once pages

[Rik van Riel]

[ CC: to linux-kernel added back since the topic isn't flammable ;) ]

On Tue, 24 Jul 2001, Stuart MacDonald wrote:
> From: "Rik van Riel" <riel@conectiva.com.br>
> > If a page gets 2 1-byte reads in a microsecond, with this
> > patch it would get promoted to the active list, even though
> > it's really only used once.
>
> I hate to bother you directly, but I don't wish to start a flame
> war on lkml. How exactly would you explain two accesses as
> being "used once"?

Because they occur in a very short interval, an interval MUCH
shorter than the time scale in which the VM subsystem looks at
referenced bits, etc...

Generally a CPU doesn't read more than one cache line at a time,
so I guess all "single references" are in reality multiple
accesses very shortly following each other.

This seems to be generally accepted theory and practice in the
field of page replacement.

regards,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: [RFC] Optimization for use-once pages

[Stuart MacDonald]

From: "Rik van Riel" <riel@conectiva.com.br>
> Because they occur in a very short interval, an interval MUCH
> shorter than the time scale in which the VM subsystem looks at
> referenced bits, etc...

So what you're sayng is that any number of accesses, as long
as they all occur within the interval < VM subsystem scanning
interval, are all counted as one?

> This seems to be generally accepted theory and practice in the
> field of page replacement.

Okay, just seems odd to me, but IANAVMGuru.

..Stu

Re: [RFC] Optimization for use-once pages

[Mike Castle]

On Tue, Jul 24, 2001 at 02:07:32PM -0300, Rik van Riel wrote:
> > I hate to bother you directly, but I don't wish to start a flame
> > war on lkml. How exactly would you explain two accesses as
> > being "used once"?
> 
> Because they occur in a very short interval, an interval MUCH
> shorter than the time scale in which the VM subsystem looks at
> referenced bits, etc...


Would mmap() then a for(;;) loop over the range be an example of such a
use?

mrc
-- 
     Mike Castle      dalgoda@ix.netcom.com      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

Re: [RFC] Optimization for use-once pages

[Rik van Riel]

On Tue, 24 Jul 2001, Stuart MacDonald wrote:
> From: "Rik van Riel" <riel@conectiva.com.br>
> > Because they occur in a very short interval, an interval MUCH
> > shorter than the time scale in which the VM subsystem looks at
> > referenced bits, etc...
>
> So what you're sayng is that any number of accesses, as long
> as they all occur within the interval < VM subsystem scanning
> interval, are all counted as one?

Actually, the length of this interval could be even smaller
and is often a point of furious debating.

The 2Q algorithm seems to have solved this problem by not
using an interval, but a FIFO queue of small, fixed length.

> > This seems to be generally accepted theory and practice in the
> > field of page replacement.
>
> Okay, just seems odd to me, but IANAVMGuru.

Seems odd at first glance, true.

Let me give you an example:

- sequential access of a file
- script reads the file in 80-byte segments
  (parsing some arcane data structure)
- these segments are accessed in rapid succession
- each 80-byte segment is accessed ONCE

In this case, even though the data is accessed only
once, each page is touched PAGE_SIZE/80 times, with
one 80-byte read() each time.

I hope this example gives you some holding point to
make it easier and grasp this - somewhat counter
intuitive - concept.

regards,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: [RFC] Optimization for use-once pages

[Rik van Riel]

On Tue, 24 Jul 2001, Mike Castle wrote:
> On Tue, Jul 24, 2001 at 02:07:32PM -0300, Rik van Riel wrote:
> > > I hate to bother you directly, but I don't wish to start a flame
> > > war on lkml. How exactly would you explain two accesses as
> > > being "used once"?
> >
> > Because they occur in a very short interval, an interval MUCH
> > shorter than the time scale in which the VM subsystem looks at
> > referenced bits, etc...
>
> Would mmap() then a for(;;) loop over the range be an example of
> such a use?

Yes. The problem here, however, would be that we'll only find
the referenced bit in the page table some time later.

This is another reason for having a "new" queue like 2Q's A1in
queue. It means we can both count all accesses in the first short
period as one access AND we can avoid actually doing anything
special as these accesses happen....

regards,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: [RFC] Optimization for use-once pages

[Stuart MacDonald]

From: "Rik van Riel" <riel@conectiva.com.br>
> Actually, the length of this interval could be even smaller
> and is often a point of furious debating.

Which is why I was avoiding flames earlier; I sorta
figured this might be a hot issue.

> Let me give you an example:
>
> - sequential access of a file
> - script reads the file in 80-byte segments
>   (parsing some arcane data structure)
> - these segments are accessed in rapid succession
> - each 80-byte segment is accessed ONCE
>
> In this case, even though the data is accessed only
> once, each page is touched PAGE_SIZE/80 times, with
> one 80-byte read() each time.

I'd figured that sort of scenario was the basis for counting
them all as one. What about

- random access of same file
- script reads one arcane 80 byte struct
- script updates that struct say PAGE_SIZE/80
times, with one 80-byte write() each time

If they're all counted as one, would the page age
correctly, if the script happens to take a few seconds
break between another flurry of all-as-one updating?

..Stu

Re: [RFC] Optimization for use-once pages

[Mike Castle]

On Tue, Jul 24, 2001 at 02:51:03PM -0300, Rik van Riel wrote:
> Let me give you an example:
> 
> - sequential access of a file
> - script reads the file in 80-byte segments
>   (parsing some arcane data structure)
> - these segments are accessed in rapid succession
> - each 80-byte segment is accessed ONCE
> 
> In this case, even though the data is accessed only
> once, each page is touched PAGE_SIZE/80 times, with
> one 80-byte read() each time.

Of course, such a program is poorly written anyway.  That many individual
system calls!  At least rewrite to use fread().  ;->

Change the example to using mmap(), and one feels less likely throttle the
programmer.

mrc
-- 
     Mike Castle      dalgoda@ix.netcom.com      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

Re: [RFC] Optimization for use-once pages

[Rik van Riel]

On Tue, 24 Jul 2001, Mike Castle wrote:
> On Tue, Jul 24, 2001 at 02:51:03PM -0300, Rik van Riel wrote:
> > Let me give you an example:
>
> Of course, such a program is poorly written anyway.  That many
> individual system calls!

That's not the point I was trying to make ;))

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

adaptec aha152x.o stops working with AVA-1505 on 2.4.7

[Dr. Kelsey Hudson]

I compiled aha152x as a module and usually have it load at boot. However,
with kernel 2.4.7, the machine hangs in some in-between state when the
module is loaded. No oops or panic is printed -- the machine simply hangs
and does nothing. The keyboard is responsive however but it appears as
though nothing gets through to the kernel.

Furthermore, nothing is printed in the logs about any such error. I'm
wondering if this is somehow related to the recent SCSI layer cleanups?

The machine is an SMP Pentium III with 640 megabytes of RAM. If you need
any more details, e.g. .config, lspci, etc. let me know and I'll provide
them. For the moment, I have removed the controller card from the machine
simply because it was pissing me off to no extent and I didn't want to
deal with it any longer. Fortunately, the only device connected to it is
an optical disk drive that gets very infrequent use, so I can live without
it, for a while.

LMK what you come up with. TIA,

 Kelsey Hudson                                           khudson@ctica.com
 Software Engineer
 Compendium Technologies, Inc                               (619) 725-0771
---------------------------------------------------------------------------

Patch for better ip_dynaddr for 2.2.19

[Martin Devera]

I'm sorry that it is for old 2.2.x but I have 2.2.19 router
as MASQ box for our microwave line. I setup automatic backup
via ISDN but I find that masq connections stops working when
ISDN gets dialed.
It is because routing is changed and maddr field is bad in
ip_masq struct.
There is existing solution: sysctl_ip_dynaddr. But it replaces
maddr only on connections where no reply was recieved from outside.
It is true for machine connected via diald for example but when
changing LIVE route it simply doesn't work.
I patched 3 files with very short patch. It allows you to add
4 to sysctl_ip_dynaddr flag and it skips no-reply test.
It works well now.
Patched version is bacward compatible and there is no speed
penalty (well almost).
What about 2.2.20 inclusion ? It this correct place to send patch to ?

devik

--------
diff -ru linux/net/ipv4/ip_masq.c gate2/net/ipv4/ip_masq.c
--- linux/net/ipv4/ip_masq.c	Sun Mar 25 18:37:41 2001
+++ gate2/net/ipv4/ip_masq.c	Tue Jul 24 20:05:21 2001
@@ -50,6 +50,7 @@
  *	Kai Bankett		:	do not toss other IP protos in proto_doff()
  *	Dan Kegel		:	pointed correct NAT behavior for UDP streams
  *	Julian Anastasov	:	use daddr and dport as hash keys
+ *  Martin Devera 	:	extended sysctl_ip_dynaddr functionality
  *	
  */
 
@@ -1236,9 +1237,9 @@
                  *	in this tunnel and routing iface address has changed...
                  *	 "You are welcome, diald".
                  */
-                if ( sysctl_ip_dynaddr && ms->flags & IP_MASQ_F_NO_REPLY && maddr != ms->maddr) {
-
-                        if (sysctl_ip_dynaddr > 1) {
+                if ( sysctl_ip_dynaddr && (sysctl_ip_dynaddr & 4 || 
+					ms->flags & IP_MASQ_F_NO_REPLY) && maddr != ms->maddr) {
+                        if ((sysctl_ip_dynaddr & 3) > 1) {
                                 IP_MASQ_INFO( "ip_fw_masquerade(): change masq.addr from %d.%d.%d.%d to %d.%d.%d.%d\n",
                                        NIPQUAD(ms->maddr),NIPQUAD(maddr));
                         }
@@ -1527,9 +1528,9 @@
                  *	in this tunnel and routing iface address has changed...
                  *	 "You are welcome, diald".
                  */
-                if ( sysctl_ip_dynaddr && ms->flags & IP_MASQ_F_NO_REPLY && maddr != ms->maddr) {
-
-                        if (sysctl_ip_dynaddr > 1) {
+                if ( sysctl_ip_dynaddr && (sysctl_ip_dynaddr & 4 || 
+					ms->flags & IP_MASQ_F_NO_REPLY) && maddr != ms->maddr) {
+                        if ((sysctl_ip_dynaddr & 3) > 1) {
 				IP_MASQ_INFO( "ip_fw_masq_icmp(): change masq.addr %d.%d.%d.%d to %d.%d.%d.%d",
 				       NIPQUAD(ms->maddr), NIPQUAD(maddr));
 			}
diff -ru linux/net/ipv4/tcp_ipv4.c gate2/net/ipv4/tcp_ipv4.c
--- linux/net/ipv4/tcp_ipv4.c	Sun Mar 25 18:37:41 2001
+++ gate2/net/ipv4/tcp_ipv4.c	Tue Jul 24 20:12:06 2001
@@ -1889,7 +1889,7 @@
 	}
 
 	if (new_saddr != sk->saddr) {
-		if (sysctl_ip_dynaddr > 1) {
+		if ((sysctl_ip_dynaddr & 3) > 1) {
 			printk(KERN_INFO "tcp_v4_rebuild_header(): shifting sk->saddr "
 			       "from %d.%d.%d.%d to %d.%d.%d.%d\n",
 			       NIPQUAD(sk->saddr), 
--- linux/Documentation/networking/ip_dynaddr.txt	Sun Mar 25 18:31:57 2001
+++ gate2/Documentation/networking/ip_dynaddr.txt	Tue Jul 24 20:44:42 2001
@@ -23,6 +23,12 @@
      # echo 2 > /proc/sys/net/ipv4/ip_dynaddr
   To disable (default)
      # echo 0 > /proc/sys/net/ipv4/ip_dynaddr
+  To always rewrite MASQ connections instead only when no reply
+  packet is recieved (needed to be able to change routing, for
+  example when doing backup-line) add 4 to flag, like:
+  	 # echo 5 > /proc/sys/net/ipv4/ip_dynaddr
+  or
+  	 # echo 6 > /proc/sys/net/ipv4/ip_dynaddr
 
 Enjoy!
 
---------------

Question about termios parameters

[John Chris Wren]

	This may not be the best place to ask this, but I've done the research,
can't come up with an answer, and don't know a better group of people to
ask.

	I have an embedded Linux device (2.2.12 kernel, BlueCat distro) that uses a
serial port for the console.  When the box comes up, rc.sysinit starts the
application as a detached process (my_program&).  When the program spits out
periodic status reports, the \n is not being mapped to CR-LF (i.e., I'm
getting only linefeeds).

	Once you log on to the box (via login, into bash), the output becomes
correctly cooked.

	I've tried twiddling termios parameters for OPOST and ONLCR, but it has no
effect.  Trying to have the application run "stty -a" via a system() call
reports an error regarding it can't get the parameters for stdin.

	What parameters are required to be set for a detached process started via
init to correctly have it's output mapped from \n to CR-NL?

	--John