Re: [PATCH] one of $BIGNUM devfs races

Re: [PATCH] one of $BIGNUM devfs races

[Alan Cox]

> OK, fair enough. When is your next merge with Linus scheduled? I'd
> prefer to get a few races fixed before shipping a patch, but I can try
> to plan for an earlier release if necessary.

I send stuff Linus regularly and sometimes it goes in and sometimes it
doesn't. Stuff with active maintainers I don't send on to Linus unless asked
too - hence joystick. input and much of USB are so far behind in Linus tree

How/when to send patches - (was Re: [PATCH] one of $BIGNUM devfs races)

[Neil Brown]

On Tuesday August 7, alan@lxorguk.ukuu.org.uk wrote:
> > OK, fair enough. When is your next merge with Linus scheduled? I'd
> > prefer to get a few races fixed before shipping a patch, but I can try
> > to plan for an earlier release if necessary.
> 
> I send stuff Linus regularly and sometimes it goes in and sometimes it
> doesn't. Stuff with active maintainers I don't send on to Linus unless asked
> too - hence joystick. input and much of USB are so far behind in Linus tree

This is something I would like to understand better.

Sometimes I send patches to Linus, and a new prepatch comes out within
hours that contains them.
Sometimes I send patches to Linus and it's like sending them to
/dev/null. Sometimes I resend.  Sometimes it helps.

So I wonder "is he busy? does he have other priorities? does he have a
broken mail system?  is he being rude" in decreasing order of
likelyhood from "very" to "very un-".

So I thought I would try sending to Alan and Linus.  Then they
appeared in an -ac patch, but not in a pre patch.

I thought that might be close enough, but if Alan doesn't plan to
forward them the Linus, then it isn't.


Now I am happy to just resent the pending patches every time a pre
patch comes out that doesn't contain then, but I want to be sure that
isn't going to negatively impact Linus at all.

Comments?

NeilBrown

(I'm talking about patches to fs/nfsd and drivers/md)

Re: How/when to send patches - (was Re: [PATCH] one of $BIGNUM devfs races)

[Linus Torvalds]

On Thu, 9 Aug 2001, Neil Brown wrote:
>
> This is something I would like to understand better.
>
> Sometimes I send patches to Linus, and a new prepatch comes out within
> hours that contains them.
> Sometimes I send patches to Linus and it's like sending them to
> /dev/null. Sometimes I resend.  Sometimes it helps.

Re-sending is always the right thing to do. Sometimes it takes a few
times, and you can add a small exasperated message at the top by the third
time ("Don't you love me any more?").

> Now I am happy to just resent the pending patches every time a pre
> patch comes out that doesn't contain then, but I want to be sure that
> isn't going to negatively impact Linus at all.

It's not. Sometimes (like now), I have other priorities, and right now for
example I've been concentrating on the VM balancing issues (and, in all
honesty, sometimes the "other priorities" aren't Linux issues at all ;).

When that happens, any other patches may still be merged, but they might
equally well just end up staying pending in my mailbox. And if they stay
there for more than a day they are basically so stale that I'll likely
never see them again.

I _seldom_ have pending patches over a pre-patch, so while it is possible
that I'm still mulling over your old patch when a new pre-patch comes out,
it's much more likely that the right answer is just to re-send. Maybe with
a slightly bigger explanation on why the patch is such a good and worthy
patch ;^)

And it's absolutely not worth it to worry about filling up my mailbox with
patches. Rule of thumb is: "if it's not really _really_ important, try to
keep one pre-patch or 48 hours between re-sends". And if it is _really_
important, ping me as often as you like.

		Linus

Re: How/when to send patches - (was Re: [PATCH] one of $BIGNUM devfs races)

[Rik van Riel]

On Wed, 8 Aug 2001, Linus Torvalds wrote:

> Re-sending is always the right thing to do.

So many VM patches, so little time ;)

I need to automate this ...

cheers,

Rik
--
IA64: a worthy successor to the i860.

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: How/when to send patches - (was Re: [PATCH] one of $BIGNUM devfs races)

[Alan Cox]

> So I thought I would try sending to Alan and Linus.  Then they
> appeared in an -ac patch, but not in a pre patch.
> 
> I thought that might be close enough, but if Alan doesn't plan to
> forward them the Linus, then it isn't.

I can forward fs/nfs stuff to Linus if you want me to add it to the stuff
I do forward, ditto md (non lvm) stuff. In many ways -ac is far enough, if
it gets to -ac it gets to most folks and its there for all the vendors

Alan

Re: Fuzzy hash stuff.. (was Re: 2.1.xxx makes Electric Fence 22x slower)

[Jamie Lokier]

"David S. Miller" <davem@dm.cobaltmicro.com> wrote:
>As promised here is my work in progress fuzzy hash VMA lookup stuff.

On Tue, Aug 25, 1998 at 10:47:26PM +1000, Keith Owens wrote:
> Lots of code with very few comments snipped.  Come on Davem, make it
> understandable for us mere mortals.  If it took two people to fix
> quirks and bugs and converge the algorithm, surely a few notes on how
> it works would not go astray.

Nope, I can't see how it works either.

BTW, a splay tree would also be as fast as what we have now in the
common case, without need for a special one entry cache.  The root of
the tree automatically acts as the cache.

-- Jamie

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.altern.org/andrebalsa/doc/lkml-faq.html

Re: GPS Leap Second Scheduled!

[Jamie Lokier]

On Wed, Sep 09, 1998 at 09:35:59AM -0700, David Lang wrote:
> I am probably missing something, but can't you just ignore the leap second
> until you discover that the time is 1 sec off and then use the normal NTP
> procedure to get back to the exact time

Until the NTP procedure discovers and corrects this (a few minutes, plus
correction time), anything that expects synchronised time between
machines can go wrong.

Admittedly synchronisation isn't perfect anyway.

-- Jamie

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Michael Shields]

In article <19980910073422.A13283@tantalophile.demon.co.uk>,
Jamie Lokier <lkd@tantalophile.demon.co.uk> wrote:
> On Wed, Sep 09, 1998 at 09:35:59AM -0700, David Lang wrote:
> > I am probably missing something, but can't you just ignore the leap second
> > until you discover that the time is 1 sec off and then use the normal NTP
> > procedure to get back to the exact time
> 
> Until the NTP procedure discovers and corrects this (a few minutes, plus
> correction time), anything that expects synchronised time between
> machines can go wrong.

NTP has the capability to know in advance that a leap second is
scheduled and act upon that at the correct time.

Check your logs the next time a leap second happens; xntpd does it.
-- 
Shields, CrossLink.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: Access to I/O-mapped / Memory-mapped resources

[Jamie Lokier]

On Wed, Dec 09, 1998 at 03:19:10PM -0800, Linux Lists wrote:
> I have a question: is there any reference in regards to how / when to use
> the virt_to_phys, virt_to_bus, ioremap, etc. ... functions, other than 
> /usr/src/linux/Documentation/IO-mapping.txt ?!? I'd like to understand it
> better, but this text has not been enough (for me, of course).

Well, the whole address thing is a bit messy anyway; I don't expect a
perfect understanding of it is even possible...

But in case it helps, try the document below.

> If there is no other way, I'll try to re-read it 1000 times to see if my
> understanding increases 1000 times as well ... ;)

Do that :-)

	linux/Documentation/IO-mapping.txt
        ----------------------------------

...is a little out of date but basically right.  Give it a read.
Then read this addendum; it might clarify things a bit.

Types of address
================

*bus* is an address you pass to devices.  E.g., what you'd write to a
PCI bus-mastering DMA device for its target address.  To access a bus
address from kernel C code, known as memory-mapped I/O, you must use
ioremap() to convert it to an *ioremap* address.  From C code, these
should always be accessed through readl(), writel() etc. and not as
ordinary memory references.  See <asm/io.h>.

*phys* is a CPU address after MMU translation.  It only appears in page
tables and things related to page tables.  Even this is hidden to some
extent because pte_page(*pte) returns a *virt* address despite appearances.
See <asm/pgtable.h>.

*virt* is a kernel direct-mapped address.  These are addresses you can
read and write from C, that correspond to main memory.  E.g., on x86,
the *virt* address 0xc0001000 means the 4097th byte of main memory.  See
<asm/page.h>.

There are other kinds of address too:

*user* addresses (such as passed to read() and write()) are
none of the above, and should always by accessed through get_user(),
put_user() etc.  See <asm/uaccess.h>.

*static* addresses are the addresses of functions and variables that are
declared in source code.  Because kernel code and modules are allocated
in various ways, you can't assume much about these addresses, but you
can always read and write them from kernel C code.

*vmalloc* addresses (returned by vmalloc()).  These are kernel virtual
address, which you can read and write from kernel C code.  But you can't
pass them to any of the virt_to_XXX macros, because they're *not* *virt*
addresses!  See <linux/vmalloc.h>.

*fixmap* addresses (returned by fix_to_virt()) (which has a misleading
name).  These are like *vmalloc* addresses: you can't pass them to the
virt_to_XXX macros, so they're _not_ *virt* addresses.  It's not very
clear if you're supposed to use readl() and writel() to access these.
See <asm/fixmap.h>.

*ioremap* addresses are returned by ioremap(), which takes a *bus*
address.  These have some similarity to *vmalloc* addresses, but you can
only use readl(), writel() etc. to access the device memory referred to
here.  Unhelpfully, just reading and writing these directly does work on
some architectures, and most older device drivers still do this.

Memory map
==========

The actual memory map varies a lot between architectures.  But since
someone asked, I'll give a quick summary of one particular memory map.
This example is for an i386 architecture: a particular 64MB Pentium II
dual processor box with PCI and an ISA bridge.

Virtual map
...........

This is what C code sees in user mode:

0x00000000-0xbfffffff  User space virtual memory mappings.

This is what C code sees in kernel mode:

0x00000000-0xbfffffff User space virtual memory mappings (current->mm context).
0xc0000000-0xc3ffffff 64MB kernel view of all of main memory, uses 4MB pages.
0xc4000000-0xc47fffff 8MB unmapped hole.
0xc4800000-0xffffbfff Kernel virtual mappings for vmalloc() and ioremap().
0xffffc000-0xffffcfff Memory mapped local APIC registers.
0xffffd000-0xffffdfff Memory mapped IO-APIC registers.

The 64MB view is subdivided like this (it depends on the PC's details):

0xc0000000-0xc00003ff Zero page, reserved for BIOS.
0xc0000000-0xc009ffff Low memory (first 640k minus zero page).
0xc00a0000-0xc00fffff Low memory-mapped I/O (especially VGA adapter) and ROMs.
0xc0100000-0xc3ffffff High memory (remaining 63MB).

The 64MB view at 0xc0000000 (= PAGE_OFFSET) is directly addressable main
memory.  This is simply memory addresses with PAGE_OFFSET added.  This
contains the main kernel image, and memory allocated with kmalloc(),
get_free_page() and the slab allocator.  Cached disk pages, network
buffers etc. are all addressed in this space.

The 64MB view is the *virt* addresses described earlier.  "Virtual" here
simply refers to the PAGE_OFFSET translation, nothing more.

The vmalloc() mappings are a different way to see this memory, used only
when a large, contiguous address range needs to be allocated.  This is
used to hold loaded modules amongst other things.

The ioremap() mappings occupy the same address range as the vmalloc()
mappings, but are a view onto memory-mapped I/O space (MMIO).  Not all
devices are mapped with ioremap() -- those in the low memory-mapped area
aren't.  In theory you are supposed to use readl(), writel(),
memcpy_fromio() etc. to access memory-mapped I/O space, but many older
drivers fail to do this and work fine on the current x86 implementation.

Physical map
............

Physical addresses are the result of virtual address translation on
board the CPU.  C code (and assembly code) doesn't see these directly,
but they are used in page tables, which control the address translation.

There is some confusion in the kernel page table code about whether the
physical addresses passed around are *phys* addresses (also known as
*linear*), or *virt* address, which are a restricted subset of *phys*
with PAGE_OFFSET added.

When setting entries, *phys* tends to be used, but when reading entries
*virt* tends to be returned.  This sometimes loses information, so
breaking some device drivers.  Perhaps those drivers are broken by
design anyway.

Bus map
.......

This view is completely different to the virtual address view, and the
*virt* view which is a subset of virtual addresses.  Bus addresses can
overlap virtual addresses in an arbitrary way, 

The bus map is the view seen by peripheral devices, like video cards and
disk controllers.  Although different from the CPU's physical map (which
is a sort of private bus map for the CPU), the bus addresses tend to be
consistent between different devices in a single machine.

On a PC, the bus map is arranged by the system BIOS at boot time,
according to rules of Plug'n'Play and other rules.  For the PCI bus,
regions of prefetchable and non-prefetchable memory are mixed
arbitrarily: there's no particularly significant address where one kind
stops and another starts.  (Although your BIOS might make it appear so).
You don't have to worry about the differences, as long as your BIOS
configured everything properly.

`lspci -vb' will show the bus addresses of all PCI devices on a system.

Memory mapped ISA cards tend to have rather low addresses (in the first
megabyte), while PCI cards can be mapped to all sorts of addresses, high
and low depending on the BIOS.

Non-PC architectures have different rules.

On an i386 architecture, bus addresses and *phys* addresses are the
same.  This is convenient but it does tend to hide some problems.  On
other architectures, these two are often different.

Devices with *bus* addresses are supposed to be memory mapped using
ioremap(), and then accessed using readl(), writel() etc.  Because none
of this was necessary on the i386 with the 2.0.x kernels, and the other
platforms weren't very well supported then, many older device drivers
simply access device bus addresses as if they were memory.

This poses big problems with some non-i386 architectures, which require
readl() etc. for the drivers to work.  These days it also poses problems
with the i386, because in 2.1.x kernels the memory layout was changed to
make communication between user space and kernel space more efficient.
As a result, ioremap() is required to get a virtual address which you
can pass to readl(), writel() etc.

Note: there is an ironic twist.  The virtual address returned by
ioremap() is not a *virt* address, so you can't expect meaningful
results if you pass it to virt_to_bus() or virt_to_phys().

Another note: at least on a PC, you don't need ioremap() to access
devices in the first 1MB of the bus address range.  This includes most
ISA devices (but not video cards).

I/O port map
............

This is similar to the bus map, but refers to I/O ports that are
accessed by special I/O instructions from the CPU, if it is an i386
based architecture.  For some other architectures, the I/O ports are
actually quite similar to memory-mapped I/O but using different
addresses.  Although some buses support more than 64k I/O ports, the
i386 architecture does not so this address range is restricted to
0x0000-0xffff.

`cat /proc/ioports' shows all the I/O ports used by drivers currently
loaded on a system.  `lspci -v' shows all the I/O ports used by PCI
devices.

Use inb(), outb() etc. to access I/O ports.  This has been required ever
since the earliest versions of Linux, so all drivers that use I/O ports
get this right.  There is no equivalent to ioremap().

Translating virtual addresses
=============================

Some people try to look up page tables to convert a *user* address or
*vmalloc* address to a *bus* address or *virt* address.  This works for
some things, but breaks others.  It makes a number of assumptions that
are incorrect and won't work when you want to use the driver in a new
way one day, or on a new architecture.

This mess will be cleaned up sometime in version 2.3.  So if you want it
cleaned up, perhaps the best way is to help ensure 2.2 is ready for
release.  Hint :-)

Hope this helps,
-- Jamie

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Re: [parisc-linux] booting problems

[Stan Sieler]

Hi,

Re:

> On some X-Windows servers on PCs, pressing F9 will toggle the "25th" line
...

Oops...forgot that I'd customized my keyboard map.

My keyboard map, which tries to add the missing HP keys to a PC keyboard
(running Reflection X) is at:

http://www.allegro.com/sieler/custom.kmp.html

If you use a different X server on a PC (or Mac or Sun or whatever),
you might be interested in a couple of the keycodes that are less
common ... they're listed on the above page.

-- 
Stan Sieler                                          sieler@allegro.com
                                     http://www.allegro.com/sieler.html

Re: Hack to head.S

[John David Anglin]

> The module init/main.c uses the symbol _stext to determine the
> starting address of the kernel text.  Placing the start in the
> data region will give a totally erroneous result for the length
> of kernel text to profile.  Probably, init/main.c should use
> the origin selected for text when linking (currently, 0x8000)
> as the start of the kernel text.  Then, the kernel entry points
> can be moved back into the standard text space.

It looks like the symbol __text_start can be used by main.c to
determine the start of text when using the HP linker.

-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: Hack to head.S

[John David Anglin]

> It looks like the symbol __text_start can be used by main.c to
> determine the start of text when using the HP linker.

Here are two patches which appear to resolve this issue.  I have
tested linking with hpux 9.01.  Somebody needs to check whether
the symbol __text_start is available under 10.X and 11.

-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

*** init/main.c.orig	Tue Feb 16 16:02:22 1999
--- init/main.c	Mon Jun 21 15:27:00 1999
***************
*** 56,62 ****
  #error sorry, your GCC is too old. It builds incorrect kernels.
  #endif
  
! extern char _stext, _etext;
  extern char *linux_banner;
  
  extern int console_loglevel;
--- 56,63 ----
  #error sorry, your GCC is too old. It builds incorrect kernels.
  #endif
  
! extern char __text_start;
! extern char _etext;
  extern char *linux_banner;
  
  extern int console_loglevel;
***************
*** 1132,1138 ****
  	if (prof_shift) {
  		prof_buffer = (unsigned int *) memory_start;
  		/* only text is profiled */
! 		prof_len = (unsigned long) &_etext - (unsigned long) &_stext;
  		prof_len >>= prof_shift;
  		memory_start += prof_len * sizeof(unsigned int);
  		memset(prof_buffer, 0, prof_len * sizeof(unsigned int));
--- 1133,1139 ----
  	if (prof_shift) {
  		prof_buffer = (unsigned int *) memory_start;
  		/* only text is profiled */
! 		prof_len = (unsigned long) &_etext-(unsigned long)&__text_start;
  		prof_len >>= prof_shift;
  		memory_start += prof_len * sizeof(unsigned int);
  		memset(prof_buffer, 0, prof_len * sizeof(unsigned int));
*** arch/parisc/kernel/head.S.orig	Mon Jun 21 10:38:05 1999
--- arch/parisc/kernel/head.S	Mon Jun 21 13:06:43 1999
***************
*** 32,39 ****
  
  ;	.section .text
  
! 	.space $DATA$
! 	.subspa $FLUFFY$,QUAD=1,ALIGN=8,ACCESS=0x2C,SORT=56
  
  	.EXPORT stext,ENTRY,PRIV_LEV=3,RTNVAL=GR
  	.EXPORT _stext,DATA,PRIV_LEV=3,RTNVAL=GR
--- 32,39 ----
  
  ;	.section .text
  
! ;	.space $DATA$
! ;	.subspa $FLUFFY$,QUAD=1,ALIGN=8,ACCESS=0x2C,SORT=56
  
  	.EXPORT stext,ENTRY,PRIV_LEV=3,RTNVAL=GR
  	.EXPORT _stext,DATA,PRIV_LEV=3,RTNVAL=GR

Re: RLIM_INFINITY inconsistency between archs

[Adam Sampson]

On Thu, Jul 27, 2000 at 12:39:51AM -0700, Linus Torvalds wrote:
> Is there some documentation file that I've not updated and that people
> are slavishly following outdated information in? I don't read the
> documentation myself, so I'd never notice ;)

Yes; the glibc installation instructions.

-- 

Adam Sampson
azz@gnu.org

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Re: RLIM_INFINITY inconsistency between archs

[Adam Sampson]

On Thu, Jul 27, 2000 at 07:03:57PM +0200, Jamie Lokier wrote:
> But instead, how about a script: /lib/modules/VERSION/compile-module.
> The script would know where to find the kernel headers.  That could be
> /lib/modules/include for distributions, and /my/kernel/tree/include for
> folks who used `make modules_install' recently.

I'll second that suggestion. This kind of thing works very well indeed for
projects like Apache.

-- 

Adam Sampson
azz@gnu.org

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Re: RLIM_INFINITY inconsistency between archs

[Miquel van Smoorenburg]

In article <cistron.20000728232030.C8868@gnu.org>,
Adam Sampson  <azz@gnu.org> wrote:
>On Thu, Jul 27, 2000 at 07:03:57PM +0200, Jamie Lokier wrote:
>> But instead, how about a script: /lib/modules/VERSION/compile-module.
>> The script would know where to find the kernel headers.  That could be
>> /lib/modules/include for distributions, and /my/kernel/tree/include for
>> folks who used `make modules_install' recently.
>
>I'll second that suggestion. This kind of thing works very well indeed for
>projects like Apache.

It is indeed a very good idea. The script could just spit out the
CFLAGS used for kernel compilation like this:

#! /bin/sh
cat <<EOF
-D__KERNEL__ -I/usr/src/linux-2.2.15/include -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer -fno-strict-aliasing -pipe -fno-strength-reduce -m486 -malign-loops=2 -malign-jumps=2 -malign-functions=2 -DCPU=686 -DUTS_MACHINE='"i386"'
EOF

Then a module Makefile would be as simple as

# Set KVER manually if you want to compile against another kernel version
KVER=$(shell uname -r)
CFLAGS=$(shell /lib/modules/$(KVER)/kernel-config)

module.o: module.c module.h

I've tried this, it works.

Mike.
-- 
Cistron Certified Internetwork Expert #1. Think free speech; drink free beer.

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Re: [parisc-linux] __hp9000s700 predefined

[John David Anglin]

> > John, is there something specific you are concerned about which
> > can't be handled by the above predefined symbols?
> 
> I was just being cautious about __hp9000s[7-8]00.  I looked at some of
> the uses in the hpux 10.X headers, and for usage in gcc and binutils.  I
> don't see any obvious reason why these two symbols need to be defined
> for linux.
> 
> I believe that the compiler only should predefine symbols that are necessary
> for the control of code generation.  Neither CONFIG_XXX or runtime checks
> will help you for this since gcc only has a limited capability to change
> its code generation at runtime.  There appears to be some model dependence
> in the floating point implementations from one pa machine to another.  If
> this can't all be hidden in the kernel, some further specification of the
> hardware might be needed for floating point.  For example, __i686__ is
> defined on the Pentium Pro and is used in bits/mathinline.h.

There is a nice table listing model numbers, architecture and processor
type in /opt/langtools/lib/sched.models under hpux 11.  It still appears
possible to compile and link a PA1.0 app under hpux 11.  I am guessing
but it looks like there are 5 different PA1.1 float implementations (a-e).

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: testcase for hppa64 gcc bug

[John David Anglin]

> > So I went down the path of trying to fix things properly by defining
> > ELIMINABLE_REGS and so on, but I ended in a maze of twisty little passages
> > labelled "Unrecognizable instruction", like this one:
> > 
> > /src/parisc/gcc/gcc/libgcc2.c: In function `__moddi3':
> > /src/parisc/gcc/gcc/libgcc2.c:601: Unrecognizable insn:
> > (insn 1289 209 1298 (set (reg:SI 50 %fr22)
> >         (subreg:SI (plus:DI (reg:DI 30 %r30)
> >                 (const_int -272 [0xfffffef0])) 0)) -1 (nil)
> >     (nil))
> > /src/parisc/gcc/gcc/libgcc2.c:601: Internal compiler error in
> > extract_insn, at recog.c:2134
> 
> I am making progress in trying to make the arg_pointer register eliminable.
> I have fixed the above problem.  What was happening was that reload_as_needed
> was incorrectly trying to eliminate the return from millicode calls which
> is also register r29.  I have figured out how to hide it from reload with
> unspec.
> 
> However, the compiler is now too good at eliminating the arg_pointer.  At
> -O3, it completely eliminates the arg_pointer.  However, as I read the ABI,
> the call must always set the arg_pointer before calls.

For the record, here is my final patch regarding making the arg_pointer
eliminable for TARGET_64BIT.  I think the code it generates is correct but
it hasn't been extensively tested.  However, I don't recommend it for
installation since in comparing the assembler code generated with and
without elimination for a couple of test cases, I didn't observe any
significant improvement in the code with the patch.  Possibly, the patch
implicitly disables elimination when the arg_pointer is needed.

I do find that Alan Modra's ARG_POINTER_INVARIANT patch needs to be installed
to get correct code with his test case.

There is one part of the patch below which I think needs to be installed.
That is

	(call, call_value): Always USE the arg_pointer for TARGET_64BIT.

The use for the arg_pointer needs to be pulled out of the `if (flag_pic)'.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

2000-11-07  John David Anglin  <dave@hiauly1.hia.nrc.ca>

	* pa-linux64.h (ARG_POINTER_INVARIANT): Define even when the
	arg_pointer is being eliminated.
	(ELIMINABLE_REGS): Enable elimination of the arg_pointer.
	(INITIAL_ELIMINATION_OFFSET): Revise offsets for arg_pointer.
	* pa.md (mulsi3, divsi3, udivsi3, modsi3, umodsi3 and
	canonicalize_funcptr_for_compare): Put "(reg:SI 26)" inside
	unspec to prevent elimination.
	(call, call_value): Always USE the arg_pointer for TARGET_64BIT.
	Use the new addmovdi3 insn to load the arg_pointer register.
	(addmovdi3 and mov_from_r29_si): New insn and expand which prevent
	r29 from being eliminated in call setups and millicode returns.

--- pa-linux64.h.orig	Tue Oct 31 18:38:24 2000
+++ pa-linux64.h	Tue Nov  7 12:17:12 2000
@@ -209,21 +209,18 @@
    that grow to lower addresses.  What fun.  */
 #undef ARGS_GROW_DOWNWARD
 #undef ARG_POINTER_REGNUM
-#define ARG_POINTER_INVARIANT 0
 #define ARG_POINTER_REGNUM 29
+#define ARG_POINTER_INVARIANT 0
 #undef STATIC_CHAIN_REGNUM
 #define STATIC_CHAIN_REGNUM 31
 
-#if 1
-#define ARG_POINTER_INVARIANT 0
-#else
-/* If defined, this macro specifies a table of register pairs used to eliminate
-   unneeded registers that point into the stack frame.  */
+/* If defined, this macro specifies a table of register pairs used to
+   eliminate unneeded registers that point into the stack frame.  */
 
 #define ELIMINABLE_REGS							\
 {									\
-  {ARG_POINTER_REGNUM,	 STACK_POINTER_REGNUM},				\
   {FRAME_POINTER_REGNUM, STACK_POINTER_REGNUM},				\
+  {ARG_POINTER_REGNUM,	 STACK_POINTER_REGNUM},				\
   {ARG_POINTER_REGNUM,	 FRAME_POINTER_REGNUM},				\
 }
 
@@ -240,19 +237,18 @@
 #define INITIAL_ELIMINATION_OFFSET(FROM, TO, OFFSET) \
   do								\
     {								\
-      int fsize;						\
+      int fsize = compute_frame_size (get_frame_size (), 0);	\
 								\
       if ((TO) == FRAME_POINTER_REGNUM				\
 	  && (FROM) == ARG_POINTER_REGNUM)			\
 	{							\
-	  (OFFSET) = - current_function_pretend_args_size - 16;	\
+	  (OFFSET) = fsize + 48 - current_function_outgoing_args_size;	\
 	  break;						\
 	}							\
 								\
       if ((TO) != STACK_POINTER_REGNUM)				\
 	abort ();						\
 								\
-      fsize = compute_frame_size (get_frame_size (), 0);	\
       switch (FROM)						\
 	{							\
 	case FRAME_POINTER_REGNUM:				\
@@ -260,14 +256,13 @@
 	  break;						\
 								\
 	case ARG_POINTER_REGNUM:				\
-	  (OFFSET) = - fsize - current_function_pretend_args_size - 16;	\
+	  (OFFSET) = 48 - current_function_outgoing_args_size;  \
 	  break;						\
 								\
 	default:						\
 	  abort ();						\
 	}							\
     } while (0)
-#endif
 
 #undef SELECT_RTX_SECTION
 #define SELECT_RTX_SECTION(MODE,RTX)	\
--- pa.md.orig	Tue Nov  7 13:50:34 2000
+++ pa.md.work	Wed Nov  8 14:06:05 2000
@@ -3993,7 +3993,7 @@
 	      (clobber (reg:SI 26))
 	      (clobber (reg:SI 25))
 	      (clobber (reg:SI 31))])
-   (set (match_operand:SI 0 "general_operand" "") (reg:SI 29))]
+   (set (match_operand:SI 0 "general_operand" "") (unspec:SI [(reg:SI 29)] 0))]
   ""
   "
 {
@@ -4139,7 +4139,7 @@
 	      (clobber (reg:SI 26))
 	      (clobber (reg:SI 25))
 	      (clobber (reg:SI 31))])
-   (set (match_operand:SI 0 "general_operand" "") (reg:SI 29))]
+   (set (match_operand:SI 0 "general_operand" "") (unspec:SI [(reg:SI 29)] 0))]
   ""
   "
 {
@@ -4197,7 +4197,7 @@
 	      (clobber (reg:SI 26))
 	      (clobber (reg:SI 25))
 	      (clobber (reg:SI 31))])
-   (set (match_operand:SI 0 "general_operand" "") (reg:SI 29))]
+   (set (match_operand:SI 0 "general_operand" "") (unspec:SI [(reg:SI 29)] 0))]
   ""
   "
 {
@@ -4255,7 +4255,7 @@
 	      (clobber (reg:SI 26))
 	      (clobber (reg:SI 25))
 	      (clobber (reg:SI 31))])
-   (set (match_operand:SI 0 "general_operand" "") (reg:SI 29))]
+   (set (match_operand:SI 0 "general_operand" "") (unspec:SI [(reg:SI 29)] 0))]
   ""
   "
 {
@@ -4310,7 +4310,7 @@
 	      (clobber (reg:SI 26))
 	      (clobber (reg:SI 25))
 	      (clobber (reg:SI 31))])
-   (set (match_operand:SI 0 "general_operand" "") (reg:SI 29))]
+   (set (match_operand:SI 0 "general_operand" "") (unspec:SI [(reg:SI 29)] 0))]
   ""
   "
 {
@@ -5785,9 +5785,9 @@
     op = XEXP (operands[0], 0);
 
   if (TARGET_64BIT)
-    emit_move_insn (arg_pointer_rtx,
-		    gen_rtx_PLUS (word_mode, virtual_outgoing_args_rtx,
-				  GEN_INT (64)));
+    emit_insn (gen_addmovdi3 (arg_pointer_rtx,
+			      virtual_outgoing_args_rtx,
+			      GEN_INT (64)));
 
   /* Use two different patterns for calls to explicitly named functions
      and calls through function pointers.  This is necessary as these two
@@ -5809,13 +5809,14 @@
       call_insn = emit_call_insn (gen_call_internal_reg (operands[1]));
     }
 
+  if (TARGET_64BIT)
+    use_reg (&CALL_INSN_FUNCTION_USAGE (call_insn), arg_pointer_rtx);
+
   if (flag_pic)
     {
       use_reg (&CALL_INSN_FUNCTION_USAGE (call_insn), pic_offset_table_rtx);
       use_reg (&CALL_INSN_FUNCTION_USAGE (call_insn),
 	       gen_rtx_REG (word_mode, PIC_OFFSET_TABLE_REGNUM_SAVED));
-      if (TARGET_64BIT)
-	use_reg (&CALL_INSN_FUNCTION_USAGE (call_insn), arg_pointer_rtx);
 
       /* After each call we must restore the PIC register, even if it
 	 doesn't appear to be used.
@@ -5961,9 +5962,9 @@
     op = XEXP (operands[1], 0);
 
   if (TARGET_64BIT)
-    emit_move_insn (arg_pointer_rtx,
-		    gen_rtx_PLUS (word_mode, virtual_outgoing_args_rtx,
-				  GEN_INT (64)));
+    emit_insn (gen_addmovdi3 (arg_pointer_rtx,
+			      virtual_outgoing_args_rtx,
+			      GEN_INT (64)));
 
   /* Use two different patterns for calls to explicitly named functions
      and calls through function pointers.  This is necessary as these two
@@ -5989,6 +5990,10 @@
       call_insn = emit_call_insn (gen_call_value_internal_reg (operands[0],
 							       operands[2]));
     }
+
+  if (TARGET_64BIT)
+    use_reg (&CALL_INSN_FUNCTION_USAGE (call_insn), arg_pointer_rtx);
+
   if (flag_pic)
     {
       use_reg (&CALL_INSN_FUNCTION_USAGE (call_insn), pic_offset_table_rtx);
@@ -7124,7 +7129,7 @@
 	      (clobber (reg:SI 22))
 	      (clobber (reg:SI 31))])
    (set (match_operand:SI 0 "register_operand" "")
-	(reg:SI 29))]
+	(unspec:SI [(reg:SI 29)] 0))]
   "! TARGET_PORTABLE_RUNTIME && !TARGET_64BIT && !TARGET_ELF32"
   "
 {
@@ -7236,3 +7241,48 @@
   emit_insn (gen_blockage ());
   DONE;
 }")
+
+;; For TARGET_64BIT, the arg_pointer register is also used for millicode
+;; returns.  The ABI requires that the arg_pointer be set for all calls.
+;; When the arg_pointer is made an eliminable register, eliminate_regs
+;; will eliminate the arg_pointer register from the function call setup and
+;; millicode returns unless the arg_pointer is hidden in a use, clobber or
+;; unspec.
+
+;; This is for loading the arg_pointer in function calls.
+(define_insn "addmovdi3"
+  [(set (unspec:DI [(match_operand:DI 0 "register_operand" "=r,r")] 0)
+        (plus:DI (match_operand:DI 1 "register_operand" "r,r")
+		 (match_operand 2 "const_int_operand" "J,i")))
+   (set (match_dup 0) (match_dup 0))]
+  "TARGET_64BIT"
+  "@
+  ldo %2(%1),%0
+  ldil L'%G2,%0\;add,l %0,%1,%0"
+  [(set_attr "type" "binary,binary")
+   (set_attr "pa_combine_type" "addmove,none")
+   (set_attr "length" "4,8")])
+
+;; This is for millicode return.
+(define_expand "mov_from_r29_si"
+  [(set (match_operand:SI 0 "" "")
+        (unspec:SI [(reg:SI 29)] 0))]
+  ""
+  "
+{
+  if (!TARGET_64BIT)
+    {
+      rtx tmp = gen_rtx_REG (SImode, 29);
+      emit_insn (gen_movsi (operands[0], tmp));
+      DONE;
+    }
+}")
+
+(define_insn ""
+  [(set (match_operand:SI 0 "register_operand" "=r")
+	(unspec:SI [(reg:SI 29)] 0))]
+  ""
+  "copy %%r29,%0"
+  [(set_attr "type" "multi")
+   (set_attr "length" "4")])
+

Re: testcase for hppa64 gcc bug

[Jeffrey A Law]

  In message <200011091739.MAA07483@hiauly1.hia.nrc.ca>you write:
  > For the record, here is my final patch regarding making the arg_pointer
  > eliminable for TARGET_64BIT.  I think the code it generates is correct but
  > it hasn't been extensively tested.  However, I don't recommend it for
  > installation since in comparing the assembler code generated with and
  > without elimination for a couple of test cases, I didn't observe any
  > significant improvement in the code with the patch.  Possibly, the patch
  > implicitly disables elimination when the arg_pointer is needed.
  > 
  > I do find that Alan Modra's ARG_POINTER_INVARIANT patch needs to be install
  > ed
  > to get correct code with his test case.
  > 
  > There is one part of the patch below which I think needs to be installed.
  > That is
  > 
  > 	(call, call_value): Always USE the arg_pointer for TARGET_64BIT.
  > 
  > The use for the arg_pointer needs to be pulled out of the `if (flag_pic)'.
  > 
  > Dave
  > -- 
  > J. David Anglin                                  dave.anglin@nrc.ca
  > National Research Council of Canada              (613) 990-0752 (FAX: 952-6
  > 605)
  > 
  > 2000-11-07  John David Anglin  <dave@hiauly1.hia.nrc.ca>
  > 
  > 	* pa-linux64.h (ARG_POINTER_INVARIANT): Define even when the
  > 	arg_pointer is being eliminated.
  > 	(ELIMINABLE_REGS): Enable elimination of the arg_pointer.
  > 	(INITIAL_ELIMINATION_OFFSET): Revise offsets for arg_pointer.
  > 	* pa.md (mulsi3, divsi3, udivsi3, modsi3, umodsi3 and
  > 	canonicalize_funcptr_for_compare): Put "(reg:SI 26)" inside
  > 	unspec to prevent elimination.
  > 	(call, call_value): Always USE the arg_pointer for TARGET_64BIT.
  > 	Use the new addmovdi3 insn to load the arg_pointer register.
  > 	(addmovdi3 and mov_from_r29_si): New insn and expand which prevent
  > 	r29 from being eliminated in call setups and millicode returns.
I haven't followed this discussion too closely.  Is this patch still needed
after some of the recent changes in how we compute liveness for the argument
pointer?
jeff

Re: testcase for hppa64 gcc bug

[John David Anglin]

>   > 2000-11-07  John David Anglin  <dave@hiauly1.hia.nrc.ca>
>   > 
>   > 	* pa-linux64.h (ARG_POINTER_INVARIANT): Define even when the
>   > 	arg_pointer is being eliminated.
>   > 	(ELIMINABLE_REGS): Enable elimination of the arg_pointer.
>   > 	(INITIAL_ELIMINATION_OFFSET): Revise offsets for arg_pointer.
>   > 	* pa.md (mulsi3, divsi3, udivsi3, modsi3, umodsi3 and
>   > 	canonicalize_funcptr_for_compare): Put "(reg:SI 26)" inside
>   > 	unspec to prevent elimination.
>   > 	(call, call_value): Always USE the arg_pointer for TARGET_64BIT.
>   > 	Use the new addmovdi3 insn to load the arg_pointer register.
>   > 	(addmovdi3 and mov_from_r29_si): New insn and expand which prevent
>   > 	r29 from being eliminated in call setups and millicode returns.
> I haven't followed this discussion too closely.  Is this patch still needed
> after some of the recent changes in how we compute liveness for the argument
> pointer?

I think this needs to be reexamined.  Allan's ARG_POINTER_INVARIANT patch
might not be needed now.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: testcase for hppa64 gcc bug

[Alan Modra]

On Tue, 5 Dec 2000, John David Anglin wrote:

> >   > 2000-11-07  John David Anglin  <dave@hiauly1.hia.nrc.ca>
> >   > 
> >   > 	* pa-linux64.h (ARG_POINTER_INVARIANT): Define even when the
> >   > 	arg_pointer is being eliminated.
> >   > 	(ELIMINABLE_REGS): Enable elimination of the arg_pointer.
> >   > 	(INITIAL_ELIMINATION_OFFSET): Revise offsets for arg_pointer.
> >   > 	* pa.md (mulsi3, divsi3, udivsi3, modsi3, umodsi3 and
> >   > 	canonicalize_funcptr_for_compare): Put "(reg:SI 26)" inside
> >   > 	unspec to prevent elimination.
> >   > 	(call, call_value): Always USE the arg_pointer for TARGET_64BIT.
> >   > 	Use the new addmovdi3 insn to load the arg_pointer register.
> >   > 	(addmovdi3 and mov_from_r29_si): New insn and expand which prevent
> >   > 	r29 from being eliminated in call setups and millicode returns.
> > I haven't followed this discussion too closely.  Is this patch still needed
> > after some of the recent changes in how we compute liveness for the argument
> > pointer?
> 
> I think this needs to be reexamined.  Allan's ARG_POINTER_INVARIANT patch
> might not be needed now.

It's still needed.  The problem is that gcc thinks the arg pointer is
unchanged from the entry value to a function, even when the arg pointer
needs to be set to call other functions.

Actually, given the nature of the problem, I'm inclined to think that all
targets that use an arg pointer should probably define
ARG_POINTER_INVARIANT = 0.  Or equivalently, don't apply my
ARG_POINTER_INVARIANT patch and simply remove tests for arg_pointer_rtx in
rtx_unstable_p, rtx_varies_p, rtx_addr_can_trap_p, function_invariant_p,
loop_invariant_p, and possibly other places I've missed.

Here's the testcase again:

extern void abort(void);

char p;

int f1 (char **);
int f2 (char *, char **);

char *f3 (char *a, char *b)
{
  char *c = 0;

  if (f1 (&b) != 0)
    goto out;

  /* hppa64 passes bogus value for b */
  f2 (b, &c);

out:
  return c;
}

int f1 (char **x)
{
  if (*x != &p)
    abort ();
  return 0;
}

int f2 (char *a, char **b)
{
  if (a != &p)
    abort ();
  *b += 1;
  return 0;
}

int main (void)
{
  if (f3 (0, &p) != (char *) 0 + 1)
    abort ();
  return 0;
}

Results of compiling with -O2 -S for hppa64-hpux11-gcc built from CVS
of less that an hour ago.

	.LEVEL 2.0w
gcc2_compiled.:
	.IMPORT f1,ENTRY
	.IMPORT f2,ENTRY
	.text
	.align 8
	.EXPORT f3,ENTRY
f3
	.PROC
	.CALLINFO FRAME=128,CALLS,SAVE_RP,ENTRY_GR=3
	.ENTRY
	std %r2,-16(%r30)
	ldo -56(%r29),%r26	! r26 = &b = r29 - 56
	ldo 128(%r30),%r30
	std %r4,-104(%r30)
	copy %r27,%r4
	std %r25,-56(%r29)
	ldo -16(%r30),%r29
	b,l f1,%r2
	std %r0,-120(%r30)
	ldo -16(%r30),%r29	! arg pointer set to current frame
	ldo -120(%r30),%r25
	cmpib,= 0,%r28,L$0005
	copy %r4,%r27
L$0004
	ldd -120(%r30),%r28
L$0006
	ldd -144(%r30),%r2
	ldd -104(%r30),%r4
	bve (%r2)
	ldo -128(%r30),%r30
L$0005
	ldo -64(%r29),%r29	! r29 = curr_r30 - 16 - 64
	b,l f2,%r2
	ldd 8(%r29),%r26	! try to load b from r29 - 56, but
				! gcc misses fact that r29 has changed
	b L$0006
	ldd -120(%r30),%r28
	.EXIT
	.PROCEND
[snip]

-- 
Linuxcare.  Support for the Revolution.

[parisc-linux] Re: testcase for hppa64 gcc bug

[Jeffrey A Law]

  In message <Pine.LNX.4.21.0012061607530.16721-100000@front.linuxcare.com.au>y
ou write:
  > > I think this needs to be reexamined.  Allan's ARG_POINTER_INVARIANT patch
  > > might not be needed now.
  > 
  > It's still needed.  The problem is that gcc thinks the arg pointer is
  > unchanged from the entry value to a function, even when the arg pointer
  > needs to be set to call other functions.
But the incoming argument pointer should have been copied into a pseudo at
the start of the function and the new pseudo used to address incoming args.

That's the real problem here I think.

jeff

Re: abort in eliminate_regs compiling glob.c from glibc

[John David Anglin]

> Breakpoint 2, eliminate_regs_in_insn (insn=0x406a0ba0, replace=0)
>     at ../../gcc/reload1.c:2826
> 2826      if (! insn_is_asm && icode < 0)
> (gdb) p debug_rtx (insn)
> (insn/s 2711 2709 2719 (set (reg:SI 6 %r6)
>         (reg:SI 28 %r28)) 69 {pre_ldw-4} (insn_list 2708 (insn_list:REG_DEP_ANTI 2696 (insn_list:REG_DEP_ANTI 2702 (insn_list:REG_DEP_ANTI 2697 (nil)))))
>     (expr_list:REG_DEAD (reg:SI 28 %r28)
>         (insn_list:REG_RETVAL 2708 (expr_list:REG_EQUAL (expr_list (use (mem:BLK (scratch) 0))
>                     (expr_list (symbol_ref/v:SI ("@strlen"))
>                         (expr_list (reg/v:SI 4 %r4)
>                             (nil))))
>                 (nil)))))

The `use' arises from the `__pure__' attribute in the prototype for strlen:

extern size_t strlen (__const char *__s) __attribute__ ((__pure__));

Still haven't been able to figure out why the REG notes are processed or
a simple test case.  The cpp'd input is attached.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

begin 644 glob.i.gz
M'XL("('T"CH``V=L;V(N:0#M??MWV[:2\._Z*]CFG%[+<1I+\BO5MO>XB9/X
MJV-G;:>]W6P.#RU1-F\D4B4I.]DV^[=_>#\'("DQS6/%O9M:P&`P&`"#P6`P
MN!?T@F^___YA\;X8Q_/BX76<QGDR>G@]S:Z^'WW;N<?RDW0T78SCA]/D:O2@
M>#^[RJ;%]S??!CT$T?>"=!#$SKX?@GZDKH<WV2Q^.(YNXX?SQ622I(@4!/XP
MN_KW.,D?CK)TDES3FAW?O6`PJ(_'+KU[T*#TO6"_`=$(_&!G:=H0?_K;*[5L
MU]N7?1='5_H0U3L'=;I_I3KZ_7YK=2!L^X^6PW;//YT0BQ'$(^^,`_%Q0N(\
M3S,Q[P9@GJ/H)([*11Z+6=M[Y,K&90_VW+F?^X<X8XQTFWS<Q]OUFF@R$G7<
MP]$XGDA.DNQ94HR`O$=P9JUF[.TN41C5N;WG*8?R>P=+$^6HLN_F$![R?2SU
M'&,1D#I(@O?ZGLY!^=OPM.@+#/H\7*3)._SC%DWE=/'NX552%L9DVJY?0*G#
M%L847DH._$O6%`QJ%XV*F57P,_O0DM*8!7W<$L1O<!WTEAR0&OM[#7I6'USQ
MNS+.TR!)RX!`#/6TS3`DR>$T&T5EDJ7!QFV6C+M!&$9EF2=7BS(.PV!C(PS1
M(EN48=CM#AU\&>S[!RA`UN@FRH/->9Y=Y]$L3-);1D681K-X"\PI;K*\)/D&
M'>7[>8PF(&]JEH?ED!$VJ%Z@K,F,T1GB;IX5E/%Z)EN3[%S)FAHS$T)9JX!:
MR<-%D3_$73F5H^OJX?5H]`#_]V8^CQY0#/WO'^W)YI9CQ#@^Y:BD6143P?.H
M'3Q8EUH=3_.)WMO>:;EBK,[NMXD3:YIM$]GDXY-ND1;)=1J/R>PKDO^)V>2S
MF=KOMS(J3*Q[K;+U;_G$DEY/+F`Q:K&;"-`P7(3XCZ&=3Z0E`2!_`1"XPW`^
M^B^0.\W2:Y*-_T`]BI8+)+E1+I+$80"#DW\XVC\6T1B/!;B@"<^AZ9RO+P$Y
M/ITK".=!"+6*<\T$T)A&*4+_]O9`)"K4P@+3N(O^'?1!)+(X!7#PR<%>]._>
MCIN[[FX1!05%G/=8(_@C&H]S(Y?W)/IS'-^J;6&#AZ`=PQG7=@8;6`E2/<`B
MLVP<PSDI&@%OU2RE8=EDHA<21$^-+`H_UPFCB0677P"]^329F5FB#IQ)V6I3
M32N2PZ/,%Z.R$P1_HO\/6,6WT?1U_\T0I7Q`OR8%*V.0-^:]PY.I$H44-#-#
M84R9S&+?`"QBI-Z-"T?IHBK_[FKZ5FN?DCE"$ODMU!"2`;<1D\OU-W@T4[C)
M.)Q%Q5MUJ0$XC#[,9`$>3%!+L`1YW=M&.OG#8.,@V"2K5C8)-@18M_N&(68=
M,@Z+&"#V;?Q>;P,H(I+YB`^W#L@GQ$)UX80A1FGI&.`TD\UJ:.A."JB\,GPY
M@!?'))GZ<5``"(<`0I/>SJ<3E,YA,U<=R&&!UO<H!P>S!K>0@&:/H7_G9>[H
M-#:DT=B<QBGO#:["D]6GO,GC:%R(Q6E.$^BBQ'[HVX?=Y0HKJD*-'40QNHG'
MAH&Z5@&NC=38=(@B:(7>:03.S%/U"1(?G<FX3W!V.(_0CE"3G"P]3[(\*=\3
M^3GD^^VE^JROU,KRPDE4E%A@=7"]JNPKHW)1X$K9[WF28CA$P0=MJIL(QW$Q
MRD.6NJDG#T%9]J>H9!R7T>@&5QTK-6,VS+-I,B(\@-@FH/`/63));]".N"19
M*KYL3M!3L837\464C_$O!0@1\I8L/$0V!@$V'^#52&1H&$@JQ?`AX$W&E@9#
M+BHM=G4$7D%"RNG`X"K)NHN2,L%*JZP(KV'NBAAK%[/9>[.0I!`4%QQ4K`2^
M"F9A'A=Q?JNR>H9J6:0EV))9F-VE<:X"OTW2\=#/FAECC6S';%'&[ZI:3X`H
M>J,DT$EJT[-T9"U=)GGY'54%JKHUOQ/]2JE""1@FS@N00RC[#@U?RB,HE_P6
MX\&#004257-NBX1Y@32N6&,1;YJ?NQA&Q^9$I;!;X+O-IE&93&.-\5S>Z,.3
M<;>"T5>1P6B4D,=_+))<'9PH<8Z'K&-\HFQ@JEU%>9[$N9,J9_-90;7]GH6>
MERJ)P._MU3"4:E(>F]U=MCUUP\VWV(&YTQ9;Z\#:8G/EW]A;"W6*;ZHM98[O
M@@'E26ZH92;=*`3V?D$H57SSH^61K5?`-F`ZGX6*%BBJFII-MH`!VPCJ662O
M%UP+35?)HGNZ@&_M]$RVK0O$]D[/)GO+8"'1*GFTD;*-9B9IB:)6JME$)0_F
M$+DD!\I@&\2@@/1UL4$+K(V:V*(%8JNF%23+1Z`N(JKNQ^U5>&_$G0%X'DY3
M,H0!64U752JDO>UZ`32ZV`XN4'9R-A#=809\H^E&1-BJ[?T<V'*&3LH!?([7
MA^E&6<R:YF@2UL)M9BFY!TZ6X9,R`#=6LAW)C\!DB$"<#$#;'[9Q[H'E]P_`
M\BCGT3Y48+`-]#U*[@/0!,]@!Z009>RZ24=##!RV^+1E;Q_*$N9QMSA6!8(P
M202J=4*?J`I084"M3TS6)R8?G3O8&6+W8U:V/D]1J>[M>)4YCQZYT%4QP(*W
M\!V@+(B&5^M3-T_T_"&PS]J)BA1B6^1_'N,C]^[0+(=/&[P%GSL*XG,&;\$+
M1T&B1?D*/N$%';OD1;A,>VT4S9MNXVC.!1M'`X:HK,SCZZ0HB7;C*WR7Y6/!
M3V6(ZSX6Z3B)4MU+`6VRT%[,S'H$YE&<@[XSTS8]XFG//$Z4&CCLX`"F`?:S
M-)VH5'BL='EU`<AIHXBG\:AD)%E>:FHN%IR.7)O=IN<(-59JV!P>CCHDP("!
MBPJMC7ZK:7)=Q+)9-<Z410E3("&5*;E&@S*;)2/8FB,,KY#1'1]@;4!G*A9P
MEQRNT',52@W;PN_T&Y)//+)J;/ME(:1_UCA<5N&9'EV7**P'-2J!E9RF[<;:
MMV_H?()](V(LO,.Z%^S#VS;J&PYF/`(V._)CEB2<4<SCD1B58C"6MR'2]X=6
M8LI2/\A-P'K?^.7O&P>/O'/!&#)(3FG.#?)<6!PG:YEHU@7R]%?G[OZ.:QE1
MW"QI8K#!/!@FXV)+8MX,\8D`6C'):0TV9^)\,G*-SU&$F*X;EHG?C>)YZ2JD
M\\HHBE.S12D5$Z6A\Y9:NE13EVMK0!Q;`T.BP&WV(9`+F5$6)1-W`GZ2K%]A
ML$>KPDYL(:]DJ(>+.DL$\[34"NYX^.-E"LP6A1=:,]L:-LN-FR4'3MUI(@5^
MA2KK6OW)&'E?S*)1GDD?8;^B9Y8!E@9]E>BYW)B9")4247BJ!(;/B@%$'$4"
MZ76BBU7ACA*HGBD&#'<W"53/$_LSJR5;,]6;Q:Z7@.@N+W;%#$CW:8$O?RC<
MPBO?$A[G^#C?V+HDF9$C;T!I.6I'?I5:W]KR_]5K<-"X;N-^9KT]M#+#C*V\
M#]ZNC2T).%]L2OAY:5'B/X;Z]EGU$9Q'X]Z0.RO2\UE4)J&W=E@R.T1%Z?BO
M(4OEIZ<HF?S)T^FQ*4I%?_"T:YYV+=,$@7DEA7U!(3UW186H0X\@4:X/*(_]
M&*JY5*23S&STME"*X@%!LR+\IT&)ZL*<+HH8<\LL-ZM3KF^7&]4I-V!M=T/L
M5*'89:TE&U!EM.SMU!LOSI&"G:OH6/E4`T4=(F*$D"54C!'O`%''!R_&1\B7
M,CQ$C_`&L![Y4'D\@!:_1Q])!-;XU,TJJCC@%_Z$=[FBV2)U*`8T8F4P&P6N
M%A.Z^U%JF=!JN!_WEEE<E''2N+>S#)4:G0@%1*E))JX)))265TE5BDZ7960=
M3@(\F:["E!I<L2H<W2#)8E='ZI`"B/YA]S\K+?D*EA!E1.Z"F&HV5&BYNS:!
MK^.2P9-KK281L[?C)+=;,(_*FRIZ:/$48@!8?$L(3_+?;F=H8YLDDZP^-5C-
MV_-N!K0M]N2=.N-NXWR+9R33<5Q8$Q#_UYX/86BC@08`N<!;-=!\U4Q7K<=3
M$=%YH1V6CW-2"/AY1V>/J*T#L<_`U;!AGMI`)JY8F[<ZHW%L1GAJ@R>'RRQ#
M9\PFG3+&%$^P`H-/2]V+%JG,U2S7("&]AC6Q'`D/4A,;BH.M0-+/(8>=#TZB
M'"M`-57UR)HN2U?5V@M7-U&KHP*[3F6-A*2?*X8$U?N(U=-3F4$%[W=\^'SH
M5(P@:/6LT5OP#'&,(US'<B-I6>KJDC==GKXZ:I)O5(E*H7%%/M!NKJC(NCD9
MG'=4,0'U3%W>UU)-=4&[6HT--2F=VAJ*E*D6H0V+2XO2<#?4CXRR[GGO.+WQ
M*DQ#Q]!S]38ELA7Y[JK8)]];$N&NJC^*"'=5UJ((;TV(RT]X0KC-[1W;,!X5
M19Q+FQ_)HVE&%C..FWG0)OI>L-NO`&8L)K?DPI`"A),HF=KLI9E)EL(JFY?1
MADT&]R:`99&.,/YN)[`]T=*,]@J/^`,2/J>!=BC]M"H<]F<Q6X)D#^4UBJW4
M-KAUP8:S/R`Q+>FM6:?G,P^8Y(CL2V<'SW"WCI?*<9*IGGK*A)@F5W@3)"'$
MD-=S.D;1M>?\VG-^[3G_E7C.#W8@6=!GN:O,]BB_5F9["YATWB@G_XMD6B9I
M>!LA(4P<3Z[3Q8C_M$7NO6!_%=D#4N/[E"N2"EV!DSZ]?QZY^T='SZ_['I^%
M3X]/C@+\S[`:+`P-0'UY(/^*Y6$?R%'+B)4I73P,GX7.`,;K=62]CJS7D<K6
M?-1%Q8JQ<X?56D?X/JV$IJG?)2[?K$:?B)7C%"%]#5C3<@GEA@?5W:B8715J
MEH@+8V1)K#L#1_Y:7JWEU5I>?4UZ[\X>*"7Z@"ZCWNC2Q<EV-:"L\,!5H?$Y
M@\B(2$&+-,E2X31$!3#*1SBI&PJ+<HE^7[W>$;>M;J/I@D1?"D-$`8X@Y17;
MTKO2%(=X6NR[<G;!9N*<?5CX2C=,4"QC+N\ZL_&8AHEDN3#711/[!V[<!&"P
M[6KIP%.Q+,[4=U?]W)L20+_O7HL&!RZL>`ES8=S9]:Y^QO0XZ+N`[P6[VZZ!
ML>OMC=V!NZ/Q4R3N%N\Y:]QSX,0/"KB&XIYC**H=!^L71#P8[RW(XO@MADH=
M!IC>W*$R#.<9#70E9R@SU-.)^RR<(`@MVJR!A5U-KH5'NK6S^:XUZQI1?:M;
MR1.<I&=PIW0M!T:X5M+62MI:25LK:;4_GY)F0CI72R?NM7:SUF[6VDTS[>81
MI`;TUROO>N5=K[Q?U\H[`#5^MO#&Z6+&MAS/'I^=_AJ>_1+\&&QO*2FG9_@_
M>LJ3G_7?+XY>;'64E*,7+R]_#X]/7[ZZ5`&?OCHY"<]>71K)QR<G1\\.3VSX
MX]/'9R]>GAQ='O$\H-23HXO'Y\<O+\_.]:*71^>G*/OH_/SLG(7QMEI[?!&>
M'%Y<XB:_V][>[FD8GIV>G1_1\A<<HL\PB<#<A*%H/Q;/AU!B.([*R,J99M$X
M'H?9U;_C46GEEGF4%JQ@1X_ML[')@2:CLAML`#4&F\*/S"($Y7G=<+@#CO[(
MRN;FEBO'C\Y&PVZ,;1)/&_*/%FI,;V*2)J6OG4I@+^+S(TO&Z;BBH+-2RGP?
M=VLX,GGXSV*K-W&':L3SBHY<H=JED/$.]W0SY3CZLXS?\?YF3%J%!57-:8"B
MBA.5C?MC$>?O6=-T)S?3QZX>5QOR5IE(E+&;NO>;>R+1\G(ZT>*X0J?,0O*5
M"%%C.J$4\??0`E!Z7P`J:78!1I(`9K_5QPNH`+4FHZ04`Z6HAJ%;IFM1[B'A
M3=QT;]#*']Y$Z7@:4TIUU\)9-F9/'6KF?QSJ'H">Y-F,/8UH9Y99*%`I*P'Q
M(->XQ+N<^`$G%A,E^P3C.I*\69(BQL2XE9@:Y=6"Z)TC0Y8H,Q"^S(9*#<1^
M.5E,24/T#G/V!1]=UGP,PVR!W9^'[JP8Q^M7ZI],H^M"2U%>IY2]HV9CW])H
MBD,6,^Y+>RQSU<8J@,-02S!Y!R+Y-70\-R(Z=9+1$<F?XD@Q8PI@E-.U*@Q%
MOOZZF'.!HKWP>OL--BY+,3"TW5D535+$%O$8RZ7#`3[P`J85;AP>[\JC&.)<
MS`?J66VE#/A`9M+L*DGI0P6(N$2T3`N>^&RYZ*2R[(I121&2Q5(40%CJT[+$
MAL)P*FO]BKM\\-EV4JOU*>YQ_U[,YJBO38\YP*>.>H^C?"U.OE)N%N5OXUQ[
ME4.F;K(%);!\\S;#@HBHCI`IY&`'[X6WP48228`W+&3HCN/1;8FO<R^F_(B(
M)V9OM[3?\R@ODVBJ)Q(??STIS?`,8!)';2$#(-4(?8"7&N,;Y8IN+.&)7A`$
M(,DZB@Q+90>.)GI98(CAAF7ITLI<I3YM::R0-59+@T`HD4QYK-\!2%K<)).6
M.@%_>D<LUYJ56I2DK31FA0&EZ_7:KZ7Q($1+#C%93!EF7:Z;&<R+4_0C2:]]
M$QLH%DWOHO<%DR0-RT[C]+J\<?59LW[P<)[I2BXJL(;JI:1+M3VA+Q"]`PVV
M(9"*I!K3X!1,=PE:;(7J*KH"9Y%7C^8E>?+*SHCI&U-VQE54Q%8.??K*DP75
M0W.@BM!:!2/#&52;-C**Z-91_U4T>KN8LSRP&%//#>49Y_+7\<R,:83#$3'E
M6EMM><_Q/[0JZ?-.J`VO>V]HC9:2@,@B779;1E=X/V=U*%G.A2N7W$ETJ/#:
M#,RN-5(9NXU4SCF9K/>HF6[AUOI2)JL=J:?R7F0[);,+9;+6?Q8\1P,I0_2/
MPLQG"A%"A.>1X"0.82$WCR*DE.)4DTW'^$\>P]>*;Q:.%CGJ]^EB1N:G]E(T
M[4]1G+O6&0,"UR:4/T0C>P62?9IGCB0$''^;<@!J^4(>\($F#+V\Y334EZ"0
MA;-ZW=M5@K%CRVGP`&W)9!*SS;SA0]9Y2T11?HV\<#[%SV#RFY10+OG5#WLA
MOL"2AL.ZL$@VU@=&VFIH1`Z2PX:(A3$>.:X\(H==F0BWZ%#[.3"TT\_HA)RD
MG*5XP<C>)O&6,"J@\;*E;,*OWI=QT:W"2B<HA-:T\F#TU6N]K-^NFADMLK"(
MX[>..OE`WB0^9OSNZYUE321X1M.L@(CO6@^N2?91"/:37>750[2J7&'0[+<+
MG+>'0=.?+F!!-(-FOQ5PAP^?THA-_,>0IPIJ-\E?(IV3M8G_$*FB_DWR%S$\
M$`E!J$$`G!3-#]`)$C@*ZGLW`H)EJ77A6\&;)J6A[HABI&_5V]#TX4W<W(I!
M:0TQ9T-"Q!1EPF@A)Q8IFJ*3:787;,BY:\:E6%0!9+<V$G[>Q`"ELW5%E0I@
M'2"X:IKOL;2HY*-RUW$Y4C$@ELV-1J*\^0)#,:/R5E`%/J&/HU="86,!`&?5
MC@8[?;XW'D/@2K/$")Q@<#+.0#9*N$7JAQ34EOE[!V0%BV\GQ2A*-88$,F2>
MOH]0,YRS0+\52N>/$H3/IOUV,L_1WT:?P!34(<"@0`VW(I8BW&_1.+4FQI8$
MDN640@4:D48I?DHG2LGJA):$2Z)A@G[K906(=A3L+(]6*%]YI6XY@O(XYKIK
MA(28-3B,F4LGW9UGUBF0:.)A2+&O"._J3#^L[YGHN9A5PQ.J'_9B&\!F4&LT
MW=49T&+[56M,+S.H[^J,:KD+K$-%S9%]9P]MVFG@Z-:'RIUOK-">V%=-M?H-
M;K%Z<\?^@/OW`[DT)+WEM^_XY,M"1C`/ZJMWD\WBAV.T%>/^2-<(;/0PN_KW
M.,EYI&M4`C%N)BX!/&I>T$,=%&J$LH5QFG)25]M%FJJNBT1%35?&5Q[/T/(*
M!]G"!Z/&(I631!L<[2-MK3N-[_1`,I26<C:G2XL,/NH$PA'0!!@#H.@1`"(E
MV."5%7I5*E"86V#8*1D*]&(4CUDE9LM0=P(Q'"?OP!@WLVB*EE,KJ@X-]XK5
MV&"#3VDT:^-H9O!\,D&;N1L("``+Z2J/U0<(WJX<$:?WA-8-DVP>IY6Q<]UQ
M,CT%L7@NE*:R"O-XQ2JK*W65,P5KH/(.>Z-#X2:@P4NX5B/F\"J-Z`Y!QJU8
MZ_+,JV"?Q:(Q[68U/I_EY"%M`#9[Z3Y([IY5.J/K9,0W3=X5<-FF!MZM&$LH
M5*'%*)\A<4O:S<DN%+,'ZAH_&S1<&$^(\%$><PEGA:A&TJ`JJBKS>C*>9_J?
M&!6%-AU%C,U[4KY8_;T%<12*=HXPW2Z'"FZ2U/M4MJ8Z]Y0VH'U9:W4'BOT(
M_V&%AT-5XC!H6GN=HIGK>V[BFHK;29;/(J2U??_]]\;Z(G3+90H7O#10ROD>
M6=UZU(INO3RQYXV-%>XV(_`06K[S:Z.-MPTX!*+K&/C\3&N]*7J'I?[*Q2">
M1>^P0')55:L/(76(YE-O&4I*&&X%@ZU@A_K-:%:GVW;(K4<QW'=-FK!-FJ!U
M=3OT_XW4F[+H-C*&JU[]O`14894\=Y-6HK</<!M'HZRBU35`?$V8-1[.B+J!
M2=VRM'TDRO1.'G/:W`J946\#&;1:KS8G;7F6&(^Q4.-/LT5XN664F9F6*EM=
MV/4(:NV%6%^)/7Q9>O5::N`00K1Q8RRTM=FZ6O4]1_7+]LQ'D_<0P_0Q3X])
M+`55`7%`=$P83+IAXS$@?,8*&YD"K6`U:6]B`#$.>WPT5(%V3&#2>@8/,H#4
MKE#KPVUAKU7.IL@N!BSXV&9?U;_86L]QW/E)ID,8]TWA4H(HGG3+8T+H#&&4
M2HO\N/U[X)JV"UJ%WA3EV!4PH>,3D'$\31P;=+PC]&DE_`/WZ"EO':D@0557
MXO&UTR*^#=+;(=S?/1#AF+H5Z':0[2U3<PSQN5]4K`MU)X/`ITR%Y@M.O3;H
M$@!H06'!+5*Z7GC%FWXX.L$.$J!ECVPV-#.+ZY4K8?SQ\7%HU4R\,F2S5B&A
MD@9GTXV%;FD"FC(#X@4PJ%JAJ7X/F3IY'+\U5Z<M]=');#(1KE<W<3I2C[`$
MV*2,\;F'\]R%M#&/[Y+4I4>`GTEH9E,JXYGY".50A$X+BWWV@^88.=#W&#/9
M(:K1<5EA6J>P#Z:&2U).^U\BPJ5K^=]09N"S"9`=+#`;Q!#+?X%RQ$:EZSN,
M(2J8@R7<8\]FBLT3N`&2*XKWG]9%9#2-IG&4QWE>==I'/(HJ0*@[D6=0:#76
M5+-QS;5!R0LG]51M0LN<NT#!"X:VG\47(,AIM>A\O0S]A<$0%-[\O'YCJ*=A
M31RAA<08M\2#NL9.@L!5\<,XP)G#!YVC;#:+4N^!F+H,.T^0+;5UA/Z>)6-`
M<]7!%D6<0V`%]6[,\+4!ZL.MT,%20VZKT8&-"<92O6M$;>,$0,1M:U0L:\V0
MQ>U-NNVG26:(XL=7)>TU!SWW1HVB5=W^_"N9_KB!=%=1?5S@;%#P\P>S,,V=
M6F<H,Z=90WD/2QX$44^7+5F2GG<`[_$1$G2;A/;$EG`/)0XU7?Z<E@L+,-$G
M<PTCL;O,NP]^4J^N!#_]&%CI,=(P_AE(!UR<'?P0;&Z8(3'LH@CE_?O5#3;M
M)CJ=M,4`H5:&02G-=Y`*816TPM1:1A.SAZ@)ANPA:,][T=4UK^C<V!#IE'9Q
M-8CU'9")V-(E?%'<E(5VH+.&!BOI8J99&9O.JN_?#WYD!;NNUSH%![_2%GL&
M#&#3,EK(1@O4/B`+:!T7-$W:!E=:JR^%(:-C&S*X]4*Q4*3>'E8M4$IQ4NP?
M_YW^8TO=;'D&EU\[](PI<I\O^"[8?M?;[@;?_!AL>_JT2K&L54]?KX>O;;O;
MSN4+Y>[M^A:W>\'>CMLGD^)W/"YG/<-J^]UIV6V=>-<^HJ?`#8_I]5-Z[XG0
MTD?:S0Z.JYMA'WX8O@9U3XL:'7TVH*MGT&78D>N(@@JSBV]3ASFP]('5Y]&?
MBAZ[CKBZCKBZCKCZM41<W7>NK9H`PX\CAVAMS4<W<ODL9_.ILFSBGZ'75<R6
M92Y_?Q\.2A':JH?X&@'"8#FW7<=IR*X8Q":Y%.)MDMJGY^PY7GRU))Q$900\
MDCR+BR*ZCIU2U/^,L=\28<!6F1>0S(*?#)(/\]X+=KT@QF#X2H7["M/C"Y#.
MZJ?>A4<*S#C!1QV5E[;6,O?O_/"&I_GS\8L4:7!CXQV;>58D[[0L\32.D=4&
MV;W]71"U-&XB)I<WV"A5B,8AL'?XQRW-IOL_@B',YO+-^^T5,*S>MA58L@=V
MA-:%X$-M<7J;Y%DZB].R,*S`=>`[4`5W63[&B_&#09^]!L<2'!7XX.6+2+7(
M[TM5HA;Y:.UJ`&TP_6#;S72@C[[.16V]8UGO6/Y/KI[][0;3'_R4P#MIB=0D
MM'GA?PPMD`+IX%.\G0C$7[I"I=Y#&(W0]L#>-)![H7S;@C#;Q]SQ(ADW+*TS
MA;U>9"W[AM?+E#KYJ/[^BM-,$9>VWXS54NDQ0K!AUPT#G_0Z<6!46LY.V3D&
MW76+;2VIUYJLA/M).6-;&>69[YE]R\&!""1BWH0*F[&*WXO`V)0NCT7:J&Q(
M'\H"9XK=HUH3]?[T-M)S)<P<!5"WT/:;XV<Y#CCK-&?:/)D+EN._QW'QNO^F
MB\,;#_;Z(+]8:2UJ=32-\IER.L5.&&)$_;B@V/9!)1W$5DSC>.[!9LR_!<M!
M;5TP0K1$]NKN5F"D)C@R/,K2W5P)JQ>,!*/$PJ!!962T*+10$^JDOLGN``<<
M&JH+84W&M)OND-)''!-8PG6>+>:`$R[#IPZ3"AR6A)VV09/>QG$"N%UA0YGM
M5\*`%2F']#?'5LWTB!_=*2Y#^GP0UZ%E=:(8#C"9(PT:F]A":@C3KHR8\$8M
M-@?'B[DAI_7,OMI!_*^^"B@.6)B"/S1S9+JY/KZ+1U`P%<QM>9N<N9[A8X[;
MUV]`66&"HAKGK]\`?F>\1MDD1RTNE/:((2B;M<$:21C'U,D(,Q4A@6^:$32M
M8+F=N^:4KT46+4XL?F)4/&DR$OV%]%3`53*,W^%@@DRZEE&Y*+HU;+B=BKT\
M?CL"3S##FE$%RP8Y?N>+/QX1OGP<GAR?_A*^./P7>4",)J%?X>/#T[-3(TT^
M-$;33@]?'!E%7QY>/C>3CE\>A3^_>JHD/7Y^]MMI>'YT<7E^_/CRZ(F*\RR\
M/']U^EA)^O7)\<7ASR='2M+%[Z>/P^,S)>703GIY?GRFIUR</29MU:G!QNZ+
MX_\Z^OGX\@*GBN<%))\N4`7GS]2&7>!6')\\,=-.?@DO'_^BI)P^.S][]?+"
M@#M[>71J)"%F'!V^,!(O_\MD,DK\?V<_AX_/3B_/ST[4\H>_'CT)CY]<*&D(
MX\GE,4)P<?SL]/!$S<+<.3^^_!W]_?SHR2LT#)ZI]:)"YRHXX>_S\[/3LU<7
M"E,E(L3`)WH&+V'G/,59RN\7AR]?(AC2$6KRT8N3,XV9+"4\/SQ]=J2GGYW_
MCD@YNSQZ?'DL1R[)N[@X?':$AN;%A=[(BR-4\?.S<ZW2B^>'YX@6AO'LY_^'
M$&J,0(/JY/@"56+TRB&80@;ADZ.3RT,C$Z4=_D[8;&2\^$]H=*!4@DI/_15U
MDM[:EZBI>#"C)#D*+E'WFZ/MZ$5XBOXQ!R9.__7PY)4YYA"&_WQU9"6K#5!J
M_!G][_#"!$:I3X[-$8X2+QX?G@"P6#J<6I/N[.0D_.WH^-GS2Y/TH_]\=?PK
MFH&HG\V<?[T\#T^1N#'2T:`WZST_"I^\>FG-],-SBE>9C2*W#W1#/T1-.#Y]
M8B0].?I52WEZ=GX))R(9J"5>_&;!H9F`^/;DZ*E*S,OCXRWM5_BO2S,%B\&C
M2R.1O@II)9]=:*41][7!<H)FAY+PBH[0X[-?'=B9E'/E/GEV;F4B"E"O:W-0
M))^`R2J&2R1Y?A4#5*0^1U1HDI*F("'Z]"A\BM8?+$34_&='E\_.P_,0SRUC
M=*"LE[_!62=GSXY/K7425X<$+Y1,J7B"5\97CR_/SL-CQ)E#DQ@&]LO1[^9(
MY\VX/,0KW?&IG<6:#I<[O+P\IX4/GSPY]^4S&6/D5RPJ$@AU^?,C!.G(98(<
MRCU[C.0YD]+JP#]E66?G%WAQ?+H%9YV=GFCB\CEB(9&9JMS^%4Z^//K7L2E"
M7DII(]+^1>2W+11H^K\>OW+FO3H]_I>5^/C\]Y>75NK1Z?/PN'=@X[AX_D*7
M3EAZA6@8J=,""R-(:KUZ>62WY%\OG_7M9KQ\-H`2=]3RI.J?M6XF2;9X-08K
MD@@&$$G18$[.T.J@(__M[/R)D?3BY_#$6DU/_^OH7%-5@+E[`5!Z`9!Z\?S<
MI)4F:5"O`&RO[%:^(HTRTI0:9`M.L$9JM.HD/$&*D97XXL)..[52+HXNK30T
MXB_-P?WSQ6YX?/)RT`_/GCX=:`-#R_KY^)F9=_)R;P=G[>W8.0A<EC*&%7[-
M^/'OUFCCVJTS0Y'S<A8_^?7X`JMVQZ=/U2'P\^'Y^;&N\V(51AVFA+OAQ:N7
M+]$*[<P(5;GYF*BK=)W4Y]ICM)?2:6<BSLY`"__QXR-=A69I%R^/'A\_/7[L
MSM'(>8H4VV<O5-J?'FM<P)LU+1<I9ECH'__\ZE(3AB0'-TX7\23YXO>+RR-5
MVKQ`VX9+M!%@_%!S7J&.>HG*,!&M:9RGS\`,I"B@>6Y4C%F'--'?SO&"20C3
M<+U$R["9>'[T#"F&9@(@%B^>'^EJC[69NGAY^)M6`@V$PR?'6$,Z1_B`!=H)
M0)D7/CF\/#3&GY&C=2P>,6>O-&WH\O>7OGW,*U1S2/>F<*J&_^7/%_JO\/#Q
MX[-7IY=Z+^`,K)9>'AF);!-FI%Z>'VJCX>)WM+T[>XD3/@PA>\7C"V)A$'N-
M+9%\\A0M^T]/#I_A]\A[V]O;'"W+.WE",LW4XY_MI--+`!0)KL=P,H@9IUNX
M22+##C3!E*Q*>WI:>RQ`FP0;1*<&R+>;;<ESFP4V2!4M!,9'#`5P4"-7$`<M
M"H"#$A4"H$/+=E(A5RLG'0J(DQ(5!J1%`Y!#A\P/'/X>=LQ2;JJ(0`G8UHJM
M@2XK+`NOPR*%VW$69'G3ZNXL4[POU!("SHA/@6&*,E?!7.\)36,C!BP^5!O3
M0#'S9&R'&%?S=0`0Y#J?6]',33AR@V-^G9!`&CQI3HM`*$%`TYA<@)!;2J4X
MFYZS[NX>.,]93910>QSM0O"%SB&[-86C,?*HD$,NX-X0^;$#X)H#7%?DQ]=P
M=[+`5^08K1#&=QS(1#EDHV$"K`,&4@B'8KZ*R5&;.)-S=MJ",H2?YBT@:A!8
M'AN`^2+1CA5C5TFSX,))"AL^-LTF*09@?DU)X;]C5TFS(,`5/C0F6?X6/O_C
M$+<VB'8V5Y;OZ2$BZ*F!$QE$F$.'9KK8P-&SX]3"D!010F)68=113+/2-<ZF
M2?H6.$C*LYE]D%1F5OW%^]DR&%04V(,"QJ&&0=%._&K%P+8EKE+I(H6K-!<"
M0N"LWK$U'Q?E2,AAU]EO.2I,$%U0YO/0$$SBU'F:72>I1VR0?/69";H:28\0
M/.*L58A-$(;<R1;ZF:[6J%;%-5E=T/6<_C:4Q5`.!L[,CO1^-;,U[J"T*+_&
M:L4>2(06GV->)O1ATHZ1S!\GT5/1_\C2]6C/TX9>#VX$]JYTM]WH0)3#AP5J
MS8@/>]HC]#SWUGP]BCQTB0IRMQ-\B;COX:?ZH7YQ]";V4=[O>QRV`>IOLJ+4
M+Q&Q\2>\`N$96:A%88<^[XQFY;%\5P)P6:*943G.9E&2@G3JE>@UJ,66H'%2
MO$]'EK!6(6YOHO1Z,;?5!BHI;[.WCO=Q8,^M/$-YBI<4?4@59^$Q$\WFTSBD
M;P_4"9DF(+B;HNE]-8JFIG,-<_(L?3)%<Q'*LPP`MH2M$(<X0E!Q$]LWV(G_
M0IR._0`%A`&J9QY!GJ:(P[-Y:8O2<13/,N%_E6;$FTGL-C+BPFEUF1BWUW(L
M0^.`CB+G`C"/KF/<50P$<-<@;0`?`2KS13J*2L<@4WTTV;O:JE(.X(&>@%$P
M,>=##9<Z7R0YD/^M30)4UNUMJY<W!`1Y4%AAXQ"JY(HK@$0NC\>Y@8QF%01,
M>$Z34*-EQ*\D'L"7,\"M*`G4H(@WE)9FJG,/0O=HQ^/"JFI^^*JDO14>S4P>
M.QA,RNO,-3%(/D.;-Z._\$O)FGQ,IN/8CD_%%L/\_1P0$V_C][;264334I,=
M3#(P'$+?Q@WB;8C'.(2'J?E3B7$777D#4%+5%\@H,TBL%J87M=W8$@Z6AI01
M^#:5M@;0VV%(!LCM2K!!1%<\C_*XR\:W-UJ#+!1A!\EF948WJ"=Y$3%.!X[+
M>E:(D@9._EJYAA<$+%=]-&Z7<M8W[@\P-(UN$-@H\/WJU?#P-@E,RS9,CM=F
M_OW.L0*XXEM5$;);JJ]VI:TY]P<`>_VM;*?6!E6W=[.B28>V6&G]7FWU#D6S
MKFVYZJ;UUYP_SO)+W`K21++S)@ETPP,M<Q5W/#3<OLL*(`]K7(C0*K"N5,`]
MTQ1MLUL6KCJ]5?CO79B--*]=:*H(.3FHOEZAX63V,]>&#G_"\3U*LS1!:B[&
MATK%HS++WU.358*OXJ10A)'R)BGP]JIRJNB?VH*&1:WK)%J#S6L?9J;SVH>N
MO[1U@<-SA</<8U#LM<_8[.+59VQV&?B,#3RR4G;$8+X7P#@=,@YT"()%17X,
M`EQ+@.N*_!@`H,VN?^"C%_.>]VB@KK,>$ZCN28]9#CQ;,8'J'1':)-4[\;'Z
M7#^CT=`N<_BB(0#.7G3)WO1@1&_U$B<KNMAUGJOH$@0ZGM8059R5F*.XO@%8
MUPW:M576MU::]`.6'PT$LOGHRZUU/]O2IAI>S[;*+Z^(R:GAGAE-;8BZ!E'3
M9J=9R%!O>&QD'=,D0)31.,7]A-7!11Z3HYF]@^HH0<SZH)]?S>^L.$%W8YDJ
M0P2)5,6&M8Z7LHZ7LHZ7\C7%2QGLF#+`CI3"7G/`IT)W8Q()FJYV\SNRW`W5
M!`J$D[@.A1+1'S3EFJ=<TQ11[!IM=`LM!6URM-_DU&K8^0`<PO##K?E=G);V
M,1(_''-D:ZVCQU\&H`MT(F$K7^UC<.(1%1V3[O_A6.2!(,_ZT^Y02QS^2)X2
M:32K<HK`A__;UBIAG,SA!F.%T]=4].=B6OHVI;`;C*X84975_3R\3@!$@=EA
MG'5A;C"O;B6?;2O-1J*.#0%[B?KZ'M9HO_1F=_3^G:@#M-DSPRTTNXUV-VDX
M,+;-8:WND!`=CI,K986PE&C?=&?\,"KQ&*),+L"MI4VT*0$G+MAG51UE4ER_
M>YI1[)B%;#/Y>1(.S"$Y<Y8C>@6J?63?"_;A<)+V;DDN<-K`Q^;9M#3V3311
MRQ.[)S//J7\QSS4+W@PI`L8YU:LVXG?Z2ZB*'4TDUW;HA88P23/$]C'^[Y`E
MT4WM&/^7"@><#!@PT-3#,6C2H0Y`>G9,#J!I#DO`0_QU?W?OS9#?(M*(VMN1
MMXDP->3LQZ2+)1+*/A95O+O@[N4A70\<O6E^VC6I)Y?AJ]-?3L]^.PU^#/C%
M(92([_KAVT0RY?'S<Y30EPE/CG'"CDSX^>07E+`G$\Z/GJ&$`YEP<OH+N7(E
M4W!\`9RDX/WM^25.V>EHS9>!#UD?D0#M=,8'B!15%\64;>*']L#C&-@?#MN2
M"#@I3)#/R?3M[>PZ^*KKL30CV,0F)P"1N1H:(TV4PX>64$G-/9%`AK(.9?7%
MA9R/T^J4:N502OZ^9L&*]=Y!+YHIC2D.;#[5IQHH[*%<W;#11V!=O:CO_.*W
M)ISV,BU_%M0\CL$OFE940&1'DD_&CK%D+A0R_'0OG"8SR$W<`=+?]<(X%I"6
MO^;!NHFM`VC'<A@L6A[>9#.T_$:WL;2GX)*<W_07><R\X+9%J%DL^O22[:H?
M/);9WGR=;2(S7=+!(@B([;CAI4?CV=H^N[;/KNVS%M8OTC[;[SLV&WS:J[[R
MHPA6M_25'EZK3?W"-@+A0WFX+*X<>[\6\91XL70M0R='ZW"CY0A&LWG7]#<V
M#_RZIIH#,@%RQJ_#!DAA:<**^LS`J)U>Q<LP1+V%,IW?1`7>_%C.VW'//D&-
M^]T.<.PY7^2Q$N72YK6H1N7VZA6I=X3BO$@0BH_>%*6B=ALCJY&O"N*3=S(&
M\B0N()>,6I<^U3';T$]6[.>-NJ(BGCMV$0[J=0?/9>E?H@7JWM_7"M.&J4M0
M=;>*\%7L5VW7!V#[`&])J1]$37"^QW+L1BW7%WTG2O<_5OU-]FD>LQ^TDP1J
MI$VHJ-.Y.P3J=<IUOS0'Y7D-*=Z2'&\NR:WIQAX23DM04H!>-!TO&GC*5B!B
M]U-4@:]2'O%+NO3GE?,%/"D=-<2&^%T1];W@T:"9P=>G@EGO"R-5D9F"-0B:
MHP$(>["99U:PWK.M]VSK/=M7LF=C/C66/+"=\KPFJ+LH*<GCZH[(\96%-*)V
M8$%$;$N>Z/48'XV%7R=^O0;=C&FZE,5Z6)3R*AU/(@($(A8CMF1I@)/%"0M>
M8^Y82']ZSD-7<+9H_\FIX&NVZCW[0V]O"&9@UU2DJY2C;!S_<."!&65Y/%[,
MYC_T/$#XVBM*^F%_R,GX@--1*TA6DD9E//;0WISTHLSFN$8?Z1CF-IIB&),J
MG#6G)*D'9#N58]_\^)D2Z3?19;(7L7*RF)?Y4'0E2DA8`B;GM\-C'*CZ\/+5
M!:`5E'F4%O0J;TB0VMM`BQ3!83E\_EADI:0@CV>T]G%RRU[\<I05APT2@4@2
M6*823:>#;\.4<5H04@,?6A=NH`*M!N7CNP^N]XUP?5A3#&=7^(F8<!:]`V*,
MC;/%U30.HC(#+L^DJ&\@+VIE8ZS:$,HL606':"Q"!-R7JH'(9+F*6,/N0E]I
M!="YAKJRS,9^GRN$UK6E$,_D:&=P8TR(6MEDFD4EK0OHHWIU^:M2^:0V;;I\
MVQJU4$:+(+6N4*F[0GZA"]L4E#8+0:G3L%B%B,;D=.RQVP''+J'MCU5)J\\D
MD"2=90;?5B9N2>8UX!\T^QO2^#$YV`)Y2_(0'TW"&V!+JWM'M'CYX#`]DR5I
M9E[?D0DOU8@B"A&R=9)X,K$\EH-CKF"NT)_%Z][@S9`H3I1I\%TLNAY>#068
MEHY&1787Y\[<!=*04.X'66U(`ZOU76RA)[U]CPX%W"DEPS-<=0#4ZW0_#ME,
M^J=79#+*%ZN37JL1-3$H306;4T->\"YIJV%M]`S]:C?((6M$E[799Y6-;(3'
MVW^=H:F,L1:-5VR/KXM\5%`MC1$Q68V(1C3`^AL?N:NR8_4NA0CNV+JTZ#YW
MQ(3FH[)ZOHE7/^%N;)&8YK3`_=DF?U;O7?:I#0!V=&)I:Y?X%0E7!8S1@%J+
M7?L]T4JS:C:PV?KW,=K:2CN];5QR170UMAGWZ;?J8*!?:T."?BZF=:1=21P[
M)CB00LSD4*>NR<.W]2<73^G3J[#83]E*Q\7C-NK-#Q9%O.<Z=:T&M;BH[HIL
M.J=>.CE+87HM4=*I;6JHU_]^TA=-:+?[GRR,G;I6J(;=/UFV^]516<=:U9"N
M:9UQZ3([ZI3JG5[#3-+&:&W2Y?XVP$*S4]OHT\X(;C:$FW=+'=M+&RWYNSK&
M;@^P'M5<7-KM&\?RXCF'4"N5JQ"MAP7MV>P&&RS(37>["TH-S!O/085:"7:M
MZ2HF:4]56T&O8HWRG&E8#:M=6[,1[CWXL&AH0(2F,%#8Z=Y.I,5\3L%[.]'>
M#D`1^%B[>@QC>?*\+QYB0YTXV1X\\AG>.NJQU:"/-J9YE(ZSF2/@1,%SK9".
M,?R$2I+2@^P8+"(=M0B,UZM21HS"H&9P?HH'!XBCE9EXY>$HLY_2AA#[J7HD
M2EBP.5%/9FE2;B<1W,KQ*<*H7+H42>/XVD@IXKF!"HF&D!_^ZB>)C$YY!UTA
MO+93JJBG.GY"(2L$.PRF@G%8P2,ZWHD)6%26&`65+I)>?IF-CR7-RY!7MTI]
MVA%X.>&L^>:9;082B]>;^M0T#4UC7&CGP`PBQG)CG@N<(H3ANV)QE;P>O`&%
MV11&+/+3Y5'/*E#_NS%JR6E<3-T$(];=1E-9"72<@H&<U>',WMXMK5"M;HK$
M_+6SV#S*H]GK?4HE&U2LKW2)!91^]WKPQGEC//)ECIPY6/F7>/GTEEXRZKU2
M1J846"KA<'P0QW1BP]`AM)0Z8UFGM].K+L=\'$+U-Q?:8X\8J-4,2C\Q@ZI(
MU5@T^R0L^O>7Q*)"$FN+JRT7+2P*3L=<\V)?NZ4$<T7I]%:FU\6DGK,R+O<:
M,ED+[],Q!/OF+)I.LQ$:3%K(9T"MIH"Z(Q9%,3)1I#A>KQDU&_*8TG!:I.4Q
M0\QO7Y$]QO*$3O(XUI!I6@))'SE@S'T$J2+B-P*,VP!JI@S18N2QOEC?!%C?
M!%C?!/B:;@*P]P0M62"\H3491_--\:LZ5SOD3M_&A>]J>;#="W9W]GS&#@W9
M[5+K0D=;T$A$"QR\/9JBU8R+U4V4H$IRDCF+TU*DU%TO5(*C*W)?V!7..<VH
MS<HL2]3^,GZ7E,K349-%.C(>=-)NKF5I")?@IX/XTH&,>AWEUUW]EW['K@UT
M5@=2C"J&6FS1<(1'JR&A1@+\+$!Z6R,@E+!*D9#7874Y,^`J`9>VK3Q)KP'`
MPH/7#CR/YL%"G'WF\7P:C9@7I,HII*_5H%:/?!7C"PJWKN?W:/G9VQ)-%MDH
M_&N*QBL8.'SVMJB"!L#)W4MW`9.>L;<*E<WO,7J;(:-L-D,*JLD3FJT]D(*?
MB`G=CY$ZB<1Z(WX#8/D0IP`\^C.;WBK&(NZ-BMNZ0=N%%/1PDH9E]:7?H8)`
M*QK0OY,""07\&]*+KPHT=$8W9B7&\X`B5+['3]"KL6\9E/%?MNS]HQ"R5]18
M@1LBJ$9U1%Q?B5O1[^I%[I>&-U)6V116(8"/371WD*F"54,-K6#&TZ32Y(CO
MZN!_Q3,QBUDL_+W&<9K-JO!QXBBJ*<&EGJM0A$I*+:P.#O!:E&JLNFKYFYA%
MZQ$%3_QX=(N&H?"#TT1W.DZN$_;HGSZSQ_%H#L<$L&'QKAQ:XR:?K.;K&C4K
M]_U56PE-_H-R37<AA-"`W>EJE+L%>N633UGY=<W*+0;J`@DS$!MM/G;W-XJU
MXG[Q9O(%D*O2^P?G[]\S2F`DCJ9XV/S'Y,LAV]8-$;'`HWV%\>B.4:;,[M".
M\0Y#6\>I\[N1<]%WJ&C>VNY&93:[4M3]K8!7'(;D+TUQ8(AF5P4FLO!0Z7W)
MM3ZM>KT(+2:W@`].JVOT4*O6VX&V!?E\%I4C6!N>HX086E!Q0;0!*Q97F7Q$
M6J\=92"=P!E#5NK[N$Z]:)F]11I%D]NN9-I887GY*R!(\04?RP:W2==YA)]A
MTEZPTB$6*7XKVP;1EX]Y6=#-B0&D[THI4*-7V*S0_:6Q2=2SIUDTCFZO%8'.
M4EZ_$7(FGL8SU:ITX(AOT]?1$[<(U`^ZTXMF(@@KG%Z,O7U['B^PSXME2O"X
MO&@VG2I/$M)7NJ<(%+%J!?\0#5%]!Y&E.-6`$LCK0]19'TV%`X8&N^J)=<51
MV)9^)`UTXZHGPE4$.(L9YYP`::N>Q'Y$TOSGKLO18X_&U<YC&U:WZI%LG4-9
MK4(.EZ`T74HVZ^2:E9LSO;%AU.E1IUA+S16CTEBJ46492@T1W]!D:)1VVPII
MJQ5=B9O_-.J66=\U!,;ZKN5%XS%2E?!SQDF>I?H3TXP\)<77(?Q3X4=((4W2
MN*!*@NPLO;?^6"2CMZI]CZ*8:T:^,BNC:8B5#(\NJQSNF*9%'"C1;/SH710:
M)S/*N0R.O\B/T/)K_J?-PA6/H/!7?>1U+^@-=BN"#]HQ!?'QA.E!X/13WG$4
M[EMG@*AIH]'\O33'CI$6;9N%BQS>CVDS>.3?U9!#EA3>*1(*E>*SZ!T>_2Z6
MVD%^%$Q%/%>V(!3UW)9(XWB:S&RI5N8XTN0(GUI85/9L+$6_`9&BBG2$YD.#
M.FII:TNRB[S/#G=(K2%M8TP]*+?JT.MV'L*:X1T>LS>Y?:!1\`W,:#E.(*0I
M=&MN*;S*!,LKJ*W%$V\-+5>@^BCA3M/$CQ*Q5,]3S`)K/Z6UG]+:3^EK\E,2
M$4L->:":@)B+*!)'JD*AF:2=X;1-X:4:/'-+L;!JG&6W<1T=!C)_6M1[R:^B
MU+%2@Y+8]J'!]:/MCJS>(<1-8SJB6M4F1&G@40*D3>C+@7\IL+C3:*FI>OQ`
M]^"MM;0WI+?AXEN38'9%#ND,>*P`5GI]K$!F_%P+\R,0IEZ,CM'EK\<U<F0C
MHK+M1JR`L=K]Q'+A1TUHI+#75=9)4QJHZ4M,+$X_>(=VI0;PO6F9OYODL\9C
MJNF@ZMC[4C.:W_8C]R)B;I%&CNAH+L[KX:":L8A4B)F$*Y3[0VBH&BWW&E`,
MDF`!`N^5ZKC.`M.N_L:KEIEBQZ'HHZS=OG,/8$D750`#>ZH&VS2$;45T,'U5
MV[ZZ..64&Q5S^$3<GE7_CD=E76DA*ZB+/QJ-XGDE?I,G\ZO\[4=$C_YG8[^)
MWA=E-'IK5Y+&\7@*^WA#4935+7U4Q!^C,GL%_3CU`,.US-XZ_`)\$IM:NSH.
M-B&<KFO:'@E755W#2(1%=!OS.U1`HYT$+D]'%16@>CY3W92Y/BF[64A9GN2^
M5V\BH2-"%=0D@1I"ZXT2Q28T7V(7!A#E7?5-WBQ59\.M'R`+89OR$E*[J76Z
MZ5(3YWDFSG[0CW0Q<TDN`AH:P'4/B_0*&V,Q!Q3*^I\XS[1=J=DK*OS5*)N_
M!S9=N#?U/;IG6U]9I:(Q7GWDW2_E5Y*.XW>KZ`GZO8.5T>DZ\V0B'-T3NR2I
MPWA4JV8)HR94:JH=V4]]94%_<(YF:ON"3R'5W46*6.+;V_T9N[)6<=M;RX:G
M0,U,%2Y.-=A5U=_=9#7U>.L(K"5JU#V935I]9DD553M%U`+7T`-%_YKN5,'L
MFT_+G#@VD#.X+<DU"9_&F)]<VU"X7?-5C5E&O2OC`XE<RD#6T)IA-N13&.4F
M^7OP>J*I<TWR[,J[,INXL2\&[/2"_6'4NW+W@L&>8Y_O?1),]U903Q'V8(L"
ML0"9UB,%55]_$((5AP#Z/@A&Q4Z_"D8A>6_'!WPOV!_40&:\1?&GC`M$6(_O
MY>:O^V^&Y+6JB\OSX]-G_?#QV<O?P\/S\[XUV^=H6Q&3R\:U$`\<B`>K(MYQ
M(-Y9%?&N`_'NJHCW'(CW5D6\[T"\ORKB`P?B`P]B;0#W#K9K#W?S0_/IP#]9
MT)SN>:<<$STB=J2Q0PT+;'?D\HON3+:@25`['C4PSE<HN[-"V=T5RNZM4'9_
MA;('?/V0JZ#1=1U'W_%EG8U=M,3U`#I@\8:`^U"O#V@>U(&PU$'`.Q"B79H'
M]0@L#!#P'H1HG^9!+(;G*`(^4-9D]).8#G#(.?J"(0T^Q[C&PL8YN#2*^J[\
M`<T?N/)W:/Z.*W^7YN^Z\O=H_IXK?Y_F[[OR#VC^`7UM$`V61?`CTY]P4G&7
MD$M'FL-_5^$7QLHXA=27H/=#AS-]\>`GQ#>"#8\Y_E;E51Y';X>R1%^64'>D
MK#QF+4?1=Z$85*$8<!0#%XJ=*A0[',6."\5N%8I=CF+7A6*O"L4>1['G0K%?
MA6*?H]AWH3BH0G'`41P`*#YTE%#1:GD12QAC"N[+,:0$%7;[%#K=#H;JJC@X
M>.37"G=VO``88G^_R;*IGFE(R4M35UHO5U@N5U@M5U@L5U@K5U@JFZV4M%\Z
M<'>Y=HUM+8UMK8QM+8SK=;'INJA(L#87R'_\]_8_ULOC_X7E45L<Z1#Z(%<O
M&=C-L2.\%^SU_&8-;!=IM.W3K8SK]>N+6;_L[EJO7^OU:[U^@2C6Z]='6K^4
M?5SP(.AIR]F^WP!I&#3W!TVL__M[WJ42`3SR+I3W@H/>;@7`P+]5!(RR!_O>
M-IC27/5RQ7Z)X:@'NATP6<4\$NU5@>+I-,3#G@"Z3@J$3=)"@P*@(;"-._SN
M)IF2`ZGB-<]Z$WQ#IWSPW7>!E:$X3J+O_GV>.51'#D_ZT*G!DWYU6R"#)O1I
MA?JU.%FK=A/QQV5M3S;6!]9OKPL&'ZL+#"2#6EU2BQJSHHZ'C,%GTF'>W$'=
M[F2RZ%&_D1U+[?1J$<*\B_V]51M-#>Z[V/_CCPJ:U8=[]92G==4<[5J9*GE3
MOVX3;SO\ZP5__14X<UN1)M73=P7V&CBJ9$E]6JKJ^5O8+QGB`VLD))",\!\B
M>VS=^!Y"FS.E>JJHIML&M0-S178&+J%*</Y;]@F0J,R$0F$O`?N1X?JG^G)B
M\`/?H7=Y('ZRP0*GB\7C]J9+\_EB\WRI"0.OO<KT6;5'M/75S!VTVE_,=V.[
M5[TY<$B?YN[L6+Z8P'$ZQAMZ/;D;R'NZ'!MNWY9$I@D$C.2?LHT;-.5!P$MC
M5O"26NMWO)LKSY@F]T7"WDB+^,D]$^*Y\C9MBK#,*\9D4VR$DXK;'Q.0"7F"
MEHP&]3E99EE!Z0'',#1&*A&]J"9CB&&$ZOCBJ,22H%2#X>-I$2N&%@V4H<1?
MY1P1E&C4X`^F"'^\:7I5@B*5*@9>&"8?&TMP/^C)W`\=_J]37:6CRK]E]XPJ
MU&YE%-3;-*OCJ+J\OL;+(53>1E,Z/@HYDG@R,)PTX6,.`]9)LKB^F=XD/=B]
M?U_AO^@G7':#HS4.WO$-DF1:)BD5%E%:AG-*)T&.QY#R2Y&)7*H8T7TH@5N2
M/"0F"G&?U,I%'QZ>%B_@%FEC#^:3,HQP557K*.[=OJ]W[0U\]7!IB/#3CA^B
M6X(Y_1I#BT]_B_P1G>Z\%TPAA;)U*443)%%0:E\78*.Y(<$DTJY//JDEW2--
M2B=#XL'\M818W=$W:#18:FESE783<\`VI,%O(?F,!C!CEBM_\-D-<+UW(9#!
MUS@'KO7AY[AN7SV.Z^%I>;@"XW1#<%E;:_\4VYMM3%Q$+@I%_6&]59BKXIJ[
M6W=#@4*:51=IZ`Y`;15'@/\D5Y"B;=*/YEY!0G=?;[\A=&X+_=#>$F%$O4I$
M/8JHIR&JJ8]$VUP7P7\UT4.B;9<.@G*Z@OQ^)?E]2GY?)=\T.:C(:6-I!5[$
M@S=*@\P-M8V14L$:Q8,^0)I5)<@P^.#0OR!Y`$UL\[/O1/M"50;^B"8XV-UV
MG:LZ=JVM157!-`SJGE7>"_K>VU!]<IUJSWV=BN#8V79&@L5A:/<JPM`:YXZ]
MW7U?`>"L$CR^[&][ZZV'ID8]^G6R24H>MM!ODI&`OU;6(S!O!5K4>\\I>U]#
M'TYS-'`0B!ZCN>+=.1Y5;C*-K@MY9Z_O:C;N\_Z^MPMIG^-K/X_\761REV30
M"W\Z;V4&OHID9SCZCKFH58,V^-@]*_P<P="\D]?AJ@V;S]=3W"4WS&^':0(L
M\79H0&:3";-HD$<VIK1'2(?@1!:/&J6/IED1CY-<AJ5FJ%CD]22/<<A\`IO'
MT1@`I3\HNFP>IQ1$6Q`D*01JBAO<-0<<OXU:<:M:LDPM9%;AK\&+Y4.`>Y<^
M7:C`[>V8]^,^HQ[:V_E,^P@3MFPO^>KP5T&[<&]'O#^)9CJ[-VS,7T4.XC07
M+8$B#Q41YXNMB76,.,]9S'<5+4'1=96E@\^L'*=:EZUQ(GWP71:2D+:DITQI
MJ8VKM3(0/>1J*=103+Q:KJJQO$%8RW:L;K2E?RPRY=58I,>`2XD5%7"I.BP,
M=I<TZ(<5.@'D)%II'\$7Z74E3--*R2M.UTFJO/.$9V8RZG3H9K;34:G"!O/P
M*H]&<5@LC#EWA;:N+$P`12$[$VV;D-S2P4$M!<O$49GE[RL,2D3[\8]QA<?+
M#7/!:<QG.5Z5MLWS>)*\PU>KH_=&XTB..,PA$$*_QTQ2L.#HE%$9C]F[%+%^
M<8SS1WF^U]]%';./:+_09EK]-61>QLK32&CHE#?<@4@M0$Q+K)0\5.JI_K;4
MSL`P(!2PL8?8A)A%:(L:HMBO#]HOS5ID(E$1X6WJGP8L_N[?)Z0,.V:R;F5B
MG$&[^T6LIW_HF'\Y3YFXC4VWL&TH+?OK+\;;GP)YGE#=L`]`PQX\J-FP#P#;
MQ7F:!@F9C10`FS\<M7:14N[',24?B$3>]THE-(C+#EW!A4B@4SM@\W8KH+//
M'L*L`*-)"`7EMW_R,T!MDM,9H<+R4";F?$#"BB>SF8U2R+%SP*+QX'?K^:]L
M.AYEB[0D7,5=P8@W;'AXE!`Z@/0-TKK@N^!_-S9ZP7_\!QI&?]$_>OR//O]C
MT$7PY*\]GK3#_]@5>?L\Z9'`M"TR>P)K3Z#M#<1?.^R0;%L7`&Q)2S,2.`D_
MU!AL($#4F'Z?,YR/D0<]9=Q@GGPCFLA;@28*'8:$*0]^8@JX\%'"I?0R/3ZY
M.$66T*M]WHB%"IG%Y(\JZYX8O@1:VO:T]*[@`::<D@.?-.(&J#.0#.\D+\J1
M.L4I)=&TQ"IY7@[U*2MGBBL#KQ:NO)Q=N.`?&^4X.23CW*0C2V-$R>N"A5%D
M#<?&W]X;;+/:8;MVG[5&HA2-JMU?HD+BE0I!4(8_"%1(IPE;`/DMV`*,&K`1
M1J.6X#]^#`Z(#=6(=D'YM<7#TDDK+$?)K=SZI73@ZD\W^!/]7P4>3W;O#=IZ
M?>C6J6NP>EW>['Y]4G8^.BG>[$%]2G<_-:7>[)WZ#=G[S!OBS=ZMW\[]+[N=
MWNR]^FPX^*K9X,W>YURRUHP?`B4J,I?A8I4WP94H6G)MP\LN6M;`O1I9;M1U
M%VL*M(!#4>#*@J[1"S4+TJZ)%LGU)4O5-C<1*F*\_#/253"^_R'9WP#[%G./
MH^!169#3*WJ]KKFYP5R@93Q<<'&B16Y0COCY@]4CU#:N"-V_CQ.[U,?0&@N0
MXKO;!;I7IT$JQ,1VS7?L;I!;R#4":A'5,A&T486F\`EC@.XY:=H%)/TZ:?3E
M3^I2:36MGK9'>>I4]7@_^'4\T3$>!8_"".U.=+!+K:M'?C#W$$\%Q`,&XZ0>
MY_MIQQ""<H[42;E0N0')B#"MJHX*%'!."TIH90VNG!:TSA7J=N6TH&%^%*I<
M.2THDG\SO:Z<%E3%SZ8EKIP6],`OH(VN'*'?25&K*7:*+)PK0-TN@)"L$*M*
M1P6)*Z\%"5FK%G=>"W)R90K<>2W(RX]*G3NO!<GYR2AWY[4@13_+5KGS6I"I
M7UR+W7E"QDIE7).Q:TUYK2FO->6UIORYM&2M*;>L*><DCKN0_L(6JMM[Q,UA
M:IMC)E6_9\Z&=1HNSK=[/=-Z9G]_24,;8.\S"62V1TPD-0#AO[[Y,1@XK(]V
M?;6,?/R3[H$N`Z2P9LHKX?KW`6K")K<B$^.L589Y<IA&1&:D->U\^'.8L><6
MQ=3NB`84-60;)]YH@=W&]U'","J*."_#292@-?9;`/;;+=\)\E:PN[V/9^G+
M\Z/+R]_#IZ].'U\>GYV&(>KE;>7P73#)-L6:=E54/S6(P@XSFM$5>P>970T-
MSBZ$:2`Q\?OGN$;N?D+MR:9+P<-_*%Z4S!U"%``,Y=*.3&Z$&50JCA[$O0,-
M=E85$EUDX/ROXP(?<X;!1F/-*8=GZ@9Q?C8R-/BOM!6T5%=Z82D(!!U:#9+.
M;[__=MB!B-RV^T'I696[O!F<LU85#[\==@SL8@[=OZ\Z%LE:=-\5(HC3^&YN
M(1*$/)!-O1?L[M>X$(/`'FW[P6AEN&+<%WP]D"HXN8L4!1N,''I@PTIM;LA@
M)"WY;=!J/I*[!FN#2]7'3%C[:!C'MFL?C?8I7?MHM+.=^$+:N?;1J,.&>CX:
M?'W0]B54<`M=B<%TB3<JO3TL=0.Y:*N+K;I&*VH33^1*$<)E:I)R6?D)K3(=
M4%G!9^#T<C_=]S`:N$^"\,/U>R1@@E@L@&U=KY2:+$6(E2HS33H^5&Z9@K\L
M5UM#HV:*K!)Y0E%E*_9`G#$>MP:GA[#=8(^_@U-WQ-V1J,T!T-$QB`8AO8RM
M,93X)6/=HXJ/QK=)'&NSB5"SK.ZU"/%XP+`^Z&FZ[80\%4YX&218T=&I'J*!
MGKC&#JGR=?+&KY.3#N8^[K;K".*+4:78KM38@[")8>Y!M#LG#*0G@Q;@^Q9Z
MZD-M\^+P>K[)9C&YKH2F98PD[RW:B#X_>W'TK;;14KM'EK!<]GF6(BFJ_6**
MQ6@4%X6YAV9[`548\8]Y9M-7UVE`L_<%:A2.%7+Q.#PY>W9\&IX>OC@*7QS^
MB_LA&15C=W16GD@2P#[`T?<M3R.^:7-O%.2+\`;IM+&4V^0N&GYCGLI!5B;X
MB7K[0R2SXI`=QDP1=T_G45'<C8E3/`1$>3F_<W'SV='ER]_"\_#B^+\H/R'S
M#&7$_*Z<S1$>"$(GAE0'@9$!$=T2[H)7*\!&$*EQ)_OS00_J4/PI[>QM]W<@
M$G@K_!W,$<$$B5BRJ)?G=ZA[92]_1TINB6JV!$TXSR--O[%6//Y!G<_9XKB@
M@JUY.^[*7!@)?WR.=?JG7):"O@^.'-%)FVCRN8HW[B88C?L"#QZ&CF(?W%;/
MN?-^B_HI4G?^X*?Y'?Z[VN]2KZE5.6S?*]H!2+=-=_33E`R@C=_^[[=#HRW6
MLF(O:7K]4F7EB$U3I+>%MFF)?TP"$JRZW8Y79(V":D.1P':?*^*6G*AG+/I.
M\F5%@Y&&R6\TTD#K&HYJGFX+IGH.N3GW*@ZY):J*LVX)*!HC.JBA'4SB6OG0
MV\#D!6CC"+Q9?14`;9R+MTQ0!4`;9^9_-\45`&T<JW]V3:H`:.,`_LMK<P5`
M&R?V7R%3*@"$/4VN-X!%C1=0P$"O6'717'5ML'%5@+2P/BQ19R5("ZO$QR&K
M$J2%U>*345X)TL*Z\3DWKA*DA37D"V]_)4@+*\K7SZ)*$/^)S7K'M-XQK7=,
MZQW3>L>TWC%])DSYF#LF96E4W!-`%VEI[+4MMFJ4NJI#;MJ(.!V'_,BN7B2L
MASP2UL,:D;#&W)&!0,M(6%JZ?LRMLGA1Q'DH'2-M`&GO-DSFLEV>(P>!'K6>
ML]7P;5[!>%YM"1=$/N#5VZ;PQBZ42D,\6A%0M5<]TK#Z%20-5*A(=H5-E245
M[ZKJDHVK`J0%E6F).BM!6E"</@Y9E2`M*%"?C/)*D!94J<^Y<94@+:A57WC[
M*T%:4+*^?A95@@BE"UK/`/5+*0V6@>YAV2ZBD/X`*6,JO*XW5/@?57AQ5?@=
M45[!'D>5OD9-O8QT;S'(N\CC553/407V)@+]B$2/0,Y$BBL1Z#%D>_8LXR4$
M^P?5\@QR^@39WD!>+R"%K1YNFJ4:NOM8CCYU7'RJG7L`?QFED(.%'YR.F#YR
M/JH?C!$B%Q5Q;$G8E4>.D$.MYE@CJM6N8_&O\9Y"$+5JD#6)R+^'D'"M!UM;
MFXS7)N.UR7AM,EZ;C-<FXR_99&QA%FOFJ@N#@<B7W\*RT*@V?WX+:T*KU/CS
M6U@/_E9J_?DM+`6?56O\^2TL`E]4:_WY+4C_KXH;_OR60N&M]S'K?<QZ'[/>
MQZSW,>M]S)>\C^$KHQ8<4`_,L9P+#&!+=MSE=+R6HH5AX[$-Z@978U9@Y9'J
MHA2&6/WIXX(\3.T(Q\9BQJF6:_P`H`GQR#ZX^B>"VI2Q%MA#S6/N>?,=>\IB
M&SAO(-H"TA?*[U&/S+)QC*,O!-N]_6WTD5(;V]L[Y(==+PG?]XZU;6,@#MU(
ME7L[M%)9Q=Y.G4JZ73C:'#ZN0H.@0;`)6=:(;*&U0XWPD<<LQ(=1H"(N)?XV
M!&U(LZ+:U4<+\J&FFT$[.!EO:OMYB5/4&EY!-3V"N-:(!Z;C6)?(3,BK;,2Z
MH+>%,:`A]B<_80F-$QF5IB&K`#]/7=(`.P(CC]M"RG>'A/,"S@Q+"2*(9T2Z
M\3QEH'.</&N(I".5B8C*\4)A;]<92\33A0W.M)3@H1*7=;8D1Y&:^L$]5U[?
MOZ\-*?A8[OY]Z]T@""./2<2>;%5@[,B:\E!=%](BK)#]HK?H%3-BT%ZWJ\3"
M9&C9$["HD`B[DL?728&P\;A`;CG]J&N>+.NO-6&LWV-1-$>K'CT%E>*)I@T!
M<"2`QB8X2X/`1].LB$UXG@@5(,N3"HP3(,"I!3G503_(%0X_>6L%M((EIA7Y
M!X=J$$M?(-^OK12X?\F.<`%SD+Y$W!MT7<]_BW!;V$.WT)YL94W4^YN-2O[B
MKQQ71N`EP5(Z+_2P2^8"ETW'XMVM>\&C7D7$3J6T*`F]X27!M-X2C[1S#6=+
M(Q9'@*I>^7B7*M&5982Q'2>[(<[#@<XXR23TLN[^(AZHAHO8KA^FT)11E_5>
MM_/AJ,S&(%"S/MAR7GL[OAFGOS/%LNQO4_.IP&0J3@_DT*E885IBEG/M$2*%
M#:@?V8@ZT-WCH8#-)@_P:.&1R1SQO^P=@AGLQ6P_4T/X9#&T$9[<A8)F,PT3
MK:2T^ZR'#ILJMP9E\.8#R+8W(?SSZ<CX:TE/QI^J*VO2L4$H/;]RC;^F"C;^
M()>NJF&//WAHX\\*RMYD>?"1I:K55`4P)0H8@4R\6F\KTT`0?/P1[:=Z$\H_
MYV:4;42_@3>B_$-;WF^6WY'R#]J9BEUI;0KJ;ECA*&O@ZL0_W[:-[U=X)Z%)
M46].6-]]35B!(W?Y#0C_7&'2ZLP;'<[]\@'^W',,?Q_`L5MO!RQ%M_.`1>.C
M=Z>LK0.>K;*$$R<LVLJRHO/;MP^_]9T7]2H.BDCQBC,B`B.([ZU*<<4)%YL-
M%737.=O2C[7X+'-1[YP<D*FTE5,NGT6YO;.M.K6X\]HXS%J5`G=>&P=7'Y,Z
M=UX;YU.?BG)W7ALG4)]CJ]QY;1PO?6DM=N>I$4"`4R./B"5Z&R\%.L.196AE
MF2N1N/+:D+EU:G'GM2%S5Z7`G=>&S/V8U+GSVI"YGXIR=UX;,O=S;)4[KPV9
M^Z6UV)TG9&YO.:^SM3J^5L?7ZOA:'?^"6K56Q_]V=1RM-5OD32"@-F'A6E4$
M&XA\^2V(XD:U^?-;$,NM4N//;T%,_ZW4^O-;$-V?56O\^2V(]"^JM?[\%L3]
M5\4-?[Y8#O1#CF4V#>M3A_4V9[W-66]S/A/U>+W-66]SUJ<.ZU.']:E#NY2O
M3QUDWOK40<U;GSJLU?&U.KY6Q]?J^%H=_WCJN'+J(*\N:=8K];8EX!?LO:\H
M@3@--=SZO?=PW>%@W7<C[:"@_`Y#>Q<QA)NT=GNYY@T,\=E7,<"[&+(%58]#
M6[<V`L?%&6\(@*;7E^!WG8'K23`B#"AHX'5K=_J@FWG0)4KX6AX;Y'AT;+FN
MS-6^K"C!Q_0]XY^,JZQZUUDWUSQ7/-GW>5U86_9"&OVW`\:3D#>*&:/HG6'Z
MHU$X")HG+@;QBVOT?E#E73#]RE"MVSKP'1VSQY(WOM@1*T2.L&[G.&KF(23<
M==>^H>,:).P^E'X7RB8';R>,\-M4UB'!A`JZQ#!IB'WSADG#BMLU]KV:ZILR
MKCLR>ECQ>GLQ<M!"(C@X]EDU3FOTPQK.7UY0;K?P:T%RM_4=8LYK#/D@Z*^T
MBY+[:A)E8M7MDF>;+G?IC6IR;8MJU>3(ZC<EPK7[694(1]:@*7VN/<Y'I,^1
MM=.4=-=&YM.0[LC:;=HJUV;ELVN5(VM/;[`IEDC4&5Z$QWXQ9!+9@5@%^<-G
M<`%++[97"ZKJZ@\>T'^9#F)&-NEW337DCR++2[FN@(H8VUCIBT3EG?H'0C,Q
M2AKJ_U8PRJ;3J(S':#V9S:,\%BIR1XWT0F-L=3"A'7-IZ[#EC,5C8;%:J,XR
M[/S9@:]W0QL+OHR:L5UHJGD;VU30;>7*9*?!I/L!ZD7W!H>V471.#61BS/B+
M"A83KF+=*AGAIG;,O@@VHJV`,UA=K8-H""0R;JOSB/Y=]!#7-C?LC$T*W*7X
M@(+]JH)70S[<<25(+^M3<M4@03+7XK2J!!&PO@_L`8$3&Z0<\PNCWL+5BD&J
M<A3>#07D]U:0:JRE[6,PG+]T]T[@61)30U/*;#%463(9K?+F/AP%:RBM'<J^
M#NM>S,"-H7C,K8?_X+'P!#[$U7M!K[?[J#+DC#EG4F66B'!XE-IX6NK4DD8S
MA5IHJ:HJ;47M8@32J&H4G[;?]2G2J@K-7A%*]*TN,+$HB0\>*%//'%7TDU?.
M93V4^C@=:R^X-`_$YH2H//T8+Q6IC0Z1BNB^P((Z5J*YM?!,HTM]D-GM/<]8
MIRYO=GM/,K9`BC>[O2<8/SZEWNSVGEO\Y`WQ9K?WK.+GWDYO=GM/)W[A;/!F
MJP<K8$A>Q7X\EB].\R6+K%;W[P=$+9#)]=8LN9Z['R"F:[9WT=+4`L^J)>'D
M<\-4Q7`M6[AQ`/<$HE47+@.1+[^%I:M1;?[\%E:O5JGQY[>P@OVMU/KS6UC&
M/JO6^/-;6,R^J-;Z\UM8T[XJ;OCSY;._;!71UC8JW#FX`)*KVP?5`0#<YRF[
M/)Z@&N%@DU4'FP'L^,/LSZW@CT56QH`=@`&P"K$ABD`:NWZM!#%BX%/D.=^E
M\QTXWF%RA`@0&YU(('WLW$#K+NZ2<G03;&S.N\9&=105<?"/?_[C!^WWIOBM
MA1M70?[[OQ48O/TF#<#K\!RO;XP&=7^-J)%*!7OW5D/Y6L'(6MGSEWACT(!+
M=;U;=>V]7;M/\41$/\*P?#\GEDRS;[N!T==AB/[,DZM%&9/)>X?PHY$X3:(B
MV/C6+/ZM[H^B&I.T`W\Q@-"O>%1F^7O7B7_5V#*L3Q29,NZ8EXGXO;')*D#3
M3RV[A7/Y#`&LL,Q*6)2(P3/;U86==V/X:9*^U4U$>AXV`+TK#9.0,)I]`+#1
M_`*HE5G3)MDB'?,)-(O+B/^-GSDF8PHGDK>3G5,9"B7.#8NT-'_CP(ALO=/]
MBSD*]*#HXT:,VQTAA2C5=*K3T4K]2JP:51N6>>*/J"2/1]32X?&DINHY_0L(
MRA^&>72'CT)NU%%*P>F1AYW1[9IV.<D$;034<I)@6;1G447&B3VM63#18RRE
M4Z&K#[3)8CIE#XUXGEW63)&,!O;L,L-6C^&<6O>>24'NW3<)3/YMDP`3NR99
MP?_)0)-L$/@]S6LYFG-,U>[F'+*NP96/28=')4/7AJ.YALH/T9+3>:,:JR!:
M<D-OF:8JB);<T_]VJJL@6G)>_PS;5071DH/[%]GR*HB67.&_4MY40?@MO'*A
M4+8-PM`+X5Y?7EU?7EU?7K7RUI=7UY=75[R\NMY8K#<6GUQI66\LOJQVK3<6
MZXW%E[BQ\`7_Y";'59<1'8\GNX4%I$%=WNP6UHWV2/%FM[!6_&V4>K-;6!P^
MEX9XLUM8"KZ0=GJS6Y#Z7P<;O-E"NJO'2?]GPGJNMV+KK=@G5_/66[$OJUWK
MK=AZ*_8E;L5\9MGU&<_ZC,=)W?J,9WW&LS[C69_QK#<6ZXW%YZ!JK3<6ZXW%
M>F/Q^6PLE#,>X=ZOF!.[:MP1?A^G(O[@/P,@^*"D`X<<5,$QN6JP0`UR;X=>
M,]CN=F302N!N1<7E"NUB"HGR0]O*XZ5H=YK^[*CQ,<4=%.MV"G2%@(=*,H#U
MR%`$Y8.?S`L)/")*3P,F,5#4`IZ@@M=9F>&(5JB3PSC/L]Q1*VDVO4T&`*`Y
MZ`YJ*SBLQ2!D]VRVU>`I]+]:V%8U7HMRI:@BFB7[]&&%[X2-<8QWY<X'6.R'
M0-P(#'DIM9#%;$Z8A\]FZ$8R*39#S/,T"_%(*)$("S;0V/WFQZ!O![AD(2[9
MG2PCGA3)4^YK*1,7KL3!L(!<$=*9NPU"L\MR1@1,>SK)S^YP_#6*UTNNJ:4S
M$1AYPPI^MM\E`2I9(#34D]NNALH66\-I3T'28TC`F-'69\7?=+/#D$@'4B+Q
MCX4G`H(OVW%`]>M]2F0I[9.A3<%<*H#PV$%\WD14@TVN._DP"PV472W&+)HS
M8SH?Z?QQX<'2GL&BRI\<G^.)R8M`#2'!KBI"J=)/NP1JHO@&87GPTSA,THS&
M3H9Q(,8C<;Z(AX[`XF2F&X&"!\[YA_=QN%)\[137^N0R?'7ZR^G9;Z=0%F)&
M-5%V+EM&"#8Z5AQTH\D6D?O"8J%G,:?Y).0K+40#-$KQ!]SRO(-72^>46W(5
M53_U%N#&QKC+>>$L@*AT+\#RWA]<F(4EJ[$@JQ^P.#M@-S<:1A:K""M6::NH
M$U#,B";F,TZ(!5<P"=!A6PDKY@O]TUY`L5JUN/-:,$&L3($[KP53PT>ESIW7
M@C'ADU'NSFO!4/!9MLJ=U\+V_XMKL3M/;.>!1WH!N4K_I;MY]]K#-WMNS9>B
MI%L_LA%TKIQL/VS$,5:_^_=E:`;[^V"EZBG.]QFX^FT%9&!6@"X8HM0?2,#>
MSK:\WZ_<5%<8`TQ=I*XYP*UM--8NJL,95"H82\<Q\$8LE6SPNLFM'937#LIK
M!^6U@W+#=GJSUP[*C1R4@;"EJO`6!@FFQUCZBJD#R->O^&IOA,L7ZV`+KZEM
M5+Z<<)_K$=6OJ]F/J:DKN_4R6N/5G82.&S(M1K<K#Z7N)I42B;/RK0*##??O
MOU%0*4;*IHBL)_0L/,;;>6)$X#A?N#!H%1?/`U![ON_M"/OU+]4.JIHY1].L
MB#4[)V^U9N\.!:!EY)0'4XX3`ZQZHG91/C#+O*KZ_C,88"LVSE9'``Z91]O#
M(Z!Y.>-O=)TF*PVN;&YE4RE'F)$<&KQVAZE:L.M!#B,DI%*D:^K[RIR0720C
.#:+Q]O\!-6U&P&H/`P!E
`
end

Re: abort in eliminate_regs compiling glob.c from glibc

[Alan Modra]

On Thu, 9 Nov 2000, John David Anglin wrote:

> > Breakpoint 2, eliminate_regs_in_insn (insn=0x406a0ba0, replace=0)
> >     at ../../gcc/reload1.c:2826
> > 2826      if (! insn_is_asm && icode < 0)
> > (gdb) p debug_rtx (insn)
> > (insn/s 2711 2709 2719 (set (reg:SI 6 %r6)
> >         (reg:SI 28 %r28)) 69 {pre_ldw-4} (insn_list 2708 (insn_list:REG_DEP_ANTI 2696 (insn_list:REG_DEP_ANTI 2702 (insn_list:REG_DEP_ANTI 2697 (nil)))))
> >     (expr_list:REG_DEAD (reg:SI 28 %r28)
> >         (insn_list:REG_RETVAL 2708 (expr_list:REG_EQUAL (expr_list (use (mem:BLK (scratch) 0))
> >                     (expr_list (symbol_ref/v:SI ("@strlen"))
> >                         (expr_list (reg/v:SI 4 %r4)
> >                             (nil))))
> >                 (nil)))))
> 
> The `use' arises from the `__pure__' attribute in the prototype for strlen:
> 
> extern size_t strlen (__const char *__s) __attribute__ ((__pure__));

I don't see this problem using current puffin CVS hppa64-linux gcc.  Was
this with your REG_ELIMINATE patch?

Alan Modra
-- 
Linuxcare.  Support for the Revolution.

Re: abort in eliminate_regs compiling glob.c from glibc

[John David Anglin]

> > > 2826      if (! insn_is_asm && icode < 0)
> > > (gdb) p debug_rtx (insn)
> > > (insn/s 2711 2709 2719 (set (reg:SI 6 %r6)
> > >         (reg:SI 28 %r28)) 69 {pre_ldw-4} (insn_list 2708 (insn_list:REG_DEP_ANTI 2696 (insn_list:REG_DEP_ANTI 2702 (insn_list:REG_DEP_ANTI 2697 (nil)))))
> > >     (expr_list:REG_DEAD (reg:SI 28 %r28)
> > >         (insn_list:REG_RETVAL 2708 (expr_list:REG_EQUAL (expr_list (use (mem:BLK (scratch) 0))
> > >                     (expr_list (symbol_ref/v:SI ("@strlen"))
> > >                         (expr_list (reg/v:SI 4 %r4)
> > >                             (nil))))
> > >                 (nil)))))
> > 
> > The `use' arises from the `__pure__' attribute in the prototype for strlen:
> > 
> > extern size_t strlen (__const char *__s) __attribute__ ((__pure__));
> 
> I don't see this problem using current puffin CVS hppa64-linux gcc.  Was
> this with your REG_ELIMINATE patch?

No.  Well actually I saw it first with the patch.  I see this with the
32 bit compiler.  The only elimination with the 32 bit compiler is the
default frame pointer elimination.

I just tried the hppa64-linux-gcc compiler with the source that I posted
and it didn't abort.  There were lots of warnings about int to pointer
conversions though.

Make sure you compile with -O2 or -O3?  Register elimination only occurs
at -O2 or above.  I see the problem both with a i686-linux cross compiler
and a fairly recent native hpux compiler under hpux 10.20.  The problem is
not present in 2.95.2 but it doesn't support the pure atribute.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: abort in eliminate_regs compiling glob.c from glibc

[John David Anglin]

> > Breakpoint 2, eliminate_regs_in_insn (insn=0x406a0ba0, replace=0)
> >     at ../../gcc/reload1.c:2826
> > 2826      if (! insn_is_asm && icode < 0)
> > (gdb) p debug_rtx (insn)
> > (insn/s 2711 2709 2719 (set (reg:SI 6 %r6)
> >         (reg:SI 28 %r28)) 69 {pre_ldw-4} (insn_list 2708 (insn_list:REG_DEP_ANTI 2696 (insn_list:REG_DEP_ANTI 2702 (insn_list:REG_DEP_ANTI 2697 (nil)))))
> >     (expr_list:REG_DEAD (reg:SI 28 %r28)
> >         (insn_list:REG_RETVAL 2708 (expr_list:REG_EQUAL (expr_list (use (mem:BLK (scratch) 0))
> >                     (expr_list (symbol_ref/v:SI ("@strlen"))
> >                         (expr_list (reg/v:SI 4 %r4)
> >                             (nil))))
> >                 (nil)))))
> 
> The `use' arises from the `__pure__' attribute in the prototype for strlen:
> 
> extern size_t strlen (__const char *__s) __attribute__ ((__pure__));

Here is a patch to fix the abort in eliminate_regs when it encounters a USE.
As I understand the situation, there are three conditions needed to trigger
it:

1)	A function that contains insns with an eliminable register.
2)	The function must call __builtin_alloca to change the frame size
	from its initial size.
3)	After the call to __builtin_alloca, there must be a call to a
	pure function.

With the enclosed patch, I can now build glibc for hppa-linux with -O3
optimisation.

Please review carefully because I will be the first to admit that I don't
understand why the use is there in the first place and all the details of
what eliminate_reg does.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

2000-11-14  John David Anglin  <dave@hiauly1.hia.nrc.ca>

	* reload1.c (eliminate_regs): Don't abort on MEM USEs.

--- reload1.c.orig	Wed Sep 27 14:27:23 2000
+++ reload1.c	Tue Nov 14 16:01:56 2000
@@ -2499,6 +2499,10 @@
 	return x;
 
     case USE:
+      /* Handle insn_list USE that a call to a pure functions may generate. */
+      new = eliminate_regs (XEXP (x, 0), 0, insn);
+      if (GET_CODE (new) == MEM)
+	return XEXP (new, 0);
     case CLOBBER:
     case ASM_OPERANDS:
     case SET:

Re: abort in eliminate_regs compiling glob.c from glibc

[Richard Henderson]

On Tue, Nov 14, 2000 at 04:40:52PM -0500, John David Anglin wrote:
>      case USE:
> +      /* Handle insn_list USE that a call to a pure functions ...
> +      new = eliminate_regs (XEXP (x, 0), 0, insn);
> +      if (GET_CODE (new) == MEM)
> +	return XEXP (new, 0);

This is not correct.  You want to return something that still
looks like a USE.  Probably you want

	new = eliminate_regs (XEXP (x, 0), 0, insn);
	if (new != XEXP (x, 0))
	  return gen_rtx_USE (GET_MODE (x), new);
	return x;


r~

Re: pa reload problem

[John David Anglin]

> I was going to deleting the hack and adding a general_operand test in
> local-alloc to see if that would fix the reload problem with "-fPIC".
> However, for the moment, there are still problems with a "normal" build
> at "-O3".  With only the yesterdays CVS source and the patch below
> to prevent rename registers walking of the return pointer, I get the
> following failure building libstdc++ v2:
> 
> /xxx/gnu/gcc-2.97/objdir/gcc/g++ -B/xxx/gnu/gcc-2.97/objdir/gcc/ -nostdinc++ -isystem /xxx/gnu/gcc-2.97/libstdc++ -isystem /xxx/gnu/gcc-2.97/libstdc++/std -isystem /xxx/gnu/gcc-2.97/libstdc++/stl -isystem /xxx/gnu/gcc-2.97/libio -isystem /xxx/gnu/gcc-2.97/objdir/hppa1.1-hp-hpux10.20/libio -L/xxx/gnu/gcc-2.97/objdir/hppa1.1-hp-hpux10.20/libstdc++ -B/usr/local/hppa1.1-hp-hpux10.20/bin/ -B/usr/local/hppa1.1-hp-hpux10.20/lib/ -isystem /usr/local/hppa1.1-hp-hpux10.20/include -c -O3 -fno-implicit-templates -I../../../libstdc++ -I../../../libstdc++/stl -I../libio -I../../../libstdc++/../libio -I../../../libstdc++/../include -I../../../libstdc++/../gcc -nostdinc++  ../../../libstdc++/exception.cc
> ../../../libstdc++/exception.cc: In function `void __check_eh_spec(int, const 
>    void**)':
> ../../../libstdc++/exception.cc:363: Internal error: Segmentation fault.

I have made some progress in locating this bug but still don't have a
complete understanding of the problem.  The problem is that a code_label
insn is "incorrectly" deleted in the loop pass.  Here is the rtl from
the gcse pass:

[snip]

(code_label 1158 1268 1439 124 "" "" [3 uses])

(note 1439 1158 262 [bb 12] NOTE_INSN_BASIC_BLOCK -1347440721)

(note 262 1439 1159 85 NOTE_INSN_EH_REGION_END -1347440721)

[snip]

(insn 788 787 789 (set (reg/f:SI 209)
        (high:SI (label_ref:SI 1158))) 87 {*pa.md:2435} (nil)
    (expr_list:REG_EQUAL (high:SI (label_ref:SI 1158))
        (insn_list:REG_LABEL 1158 (nil))))

(insn 789 788 791 (set (reg/f:SI 208)
        (lo_sum:SI (reg/f:SI 209)
            (label_ref:SI 1158))) 90 {*pa.md:2467} (nil)
    (insn_list:REG_LABEL 1158 (expr_list:REG_EQUAL (label_ref:SI 1158)
            (nil))))

Here is the rtl after loop:

Loop from 576 to 1288: 105 real insns.
Continue at insn 1278.

[snip]

Insn 788: regno 209 (life 2), move-insn savings 2  moved to 1530
Insn 789: regno 208 (life 1), move-insn forces 788 savings 1  moved to 1532

[snip]

(insn 1531 1528 1532 (set (reg/f:SI 337)
        (high:SI (label_ref:SI 1158))) -1 (nil)
    (expr_list:REG_LABEL (code_label/v 1158 1268 1439 124 "" "" [0 uses])
        (nil)))

(insn 1532 1531 1535 (set (reg/f:SI 208)
        (lo_sum:SI (reg/f:SI 337)
            (label_ref:SI 1158))) -1 (nil)
    (expr_list:REG_EQUAL (label_ref:SI 1158)
        (expr_list:REG_LABEL (code_label/v 1158 1268 1439 124 "" "" [0 uses])
            (nil))))

The code_label 1158 is deleted by delete_trivially_dead_insns apparently 
because the preceding call to loop_optimize in toplev.c has reduced the
number of uses to 1.  Maybe somebody can see how this occurs.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: pa reload problem

[Jeffrey A Law]

  In message <200012140048.TAA02603@hiauly1.hia.nrc.ca>you write:
  > I have made some progress in locating this bug but still don't have a
  > complete understanding of the problem.  The problem is that a code_label
  > insn is "incorrectly" deleted in the loop pass.  Here is the rtl from
  > the gcse pass:
  > 
  > [snip]
  > 
  > (code_label 1158 1268 1439 124 "" "" [3 uses])
  > 
  > (note 1439 1158 262 [bb 12] NOTE_INSN_BASIC_BLOCK -1347440721)
  > 
  > (note 262 1439 1159 85 NOTE_INSN_EH_REGION_END -1347440721)
  > 
  > [snip]
  > 
  > (insn 788 787 789 (set (reg/f:SI 209)
  >         (high:SI (label_ref:SI 1158))) 87 {*pa.md:2435} (nil)
  >     (expr_list:REG_EQUAL (high:SI (label_ref:SI 1158))
  >         (insn_list:REG_LABEL 1158 (nil))))
  > 
  > (insn 789 788 791 (set (reg/f:SI 208)
  >         (lo_sum:SI (reg/f:SI 209)
  >             (label_ref:SI 1158))) 90 {*pa.md:2467} (nil)
  >     (insn_list:REG_LABEL 1158 (expr_list:REG_EQUAL (label_ref:SI 1158)
  >             (nil))))
  > 
  > Here is the rtl after loop:
  > 
  > Loop from 576 to 1288: 105 real insns.
  > Continue at insn 1278.
  > 
  > [snip]
  > 
  > Insn 788: regno 209 (life 2), move-insn savings 2  moved to 1530
  > Insn 789: regno 208 (life 1), move-insn forces 788 savings 1  moved to 1532
  > 
  > [snip]
  > 
  > (insn 1531 1528 1532 (set (reg/f:SI 337)
  >         (high:SI (label_ref:SI 1158))) -1 (nil)
  >     (expr_list:REG_LABEL (code_label/v 1158 1268 1439 124 "" "" [0 uses])
  >         (nil)))
  > 
  > (insn 1532 1531 1535 (set (reg/f:SI 208)
  >         (lo_sum:SI (reg/f:SI 337)
  >             (label_ref:SI 1158))) -1 (nil)
  >     (expr_list:REG_EQUAL (label_ref:SI 1158)
  >         (expr_list:REG_LABEL (code_label/v 1158 1268 1439 124 "" "" [0 uses
  > ])
  >             (nil))))
  > 
  > The code_label 1158 is deleted by delete_trivially_dead_insns apparently 
  > because the preceding call to loop_optimize in toplev.c has reduced the
  > number of uses to 1.  Maybe somebody can see how this occurs.
Seems to me we have a reference counting problem.  There is clearly 
a reference to label 1158 (insn 1531/1532), but it's reference count
is zero.

Seems to me that if you find that reference counting bug that you'll
fix this problem.

jeff

Re: pa reload problem

[John David Anglin]

>   In message <200012140048.TAA02603@hiauly1.hia.nrc.ca>you write:
>   > I have made some progress in locating this bug but still don't have a
>   > complete understanding of the problem.  The problem is that a code_label
>   > insn is "incorrectly" deleted in the loop pass.  Here is the rtl from
>   > the gcse pass:
>   > 
>   > [snip]
>   > 
>   > (code_label 1158 1268 1439 124 "" "" [3 uses])
>   > 
>   > (note 1439 1158 262 [bb 12] NOTE_INSN_BASIC_BLOCK -1347440721)
>   > 
>   > (note 262 1439 1159 85 NOTE_INSN_EH_REGION_END -1347440721)
>   > 
>   > [snip]
>   > 
>   > (insn 788 787 789 (set (reg/f:SI 209)
>   >         (high:SI (label_ref:SI 1158))) 87 {*pa.md:2435} (nil)
>   >     (expr_list:REG_EQUAL (high:SI (label_ref:SI 1158))
>   >         (insn_list:REG_LABEL 1158 (nil))))
>   > 
>   > (insn 789 788 791 (set (reg/f:SI 208)
>   >         (lo_sum:SI (reg/f:SI 209)
>   >             (label_ref:SI 1158))) 90 {*pa.md:2467} (nil)
>   >     (insn_list:REG_LABEL 1158 (expr_list:REG_EQUAL (label_ref:SI 1158)
>   >             (nil))))
>   > 
>   > Here is the rtl after loop:
>   > 
>   > Loop from 576 to 1288: 105 real insns.
>   > Continue at insn 1278.
>   > 
>   > [snip]
>   > 
>   > Insn 788: regno 209 (life 2), move-insn savings 2  moved to 1530
>   > Insn 789: regno 208 (life 1), move-insn forces 788 savings 1  moved to 1532
>   > 
>   > [snip]
>   > 
>   > (insn 1531 1528 1532 (set (reg/f:SI 337)
>   >         (high:SI (label_ref:SI 1158))) -1 (nil)
>   >     (expr_list:REG_LABEL (code_label/v 1158 1268 1439 124 "" "" [0 uses])
>   >         (nil)))
>   > 
>   > (insn 1532 1531 1535 (set (reg/f:SI 208)
>   >         (lo_sum:SI (reg/f:SI 337)
>   >             (label_ref:SI 1158))) -1 (nil)
>   >     (expr_list:REG_EQUAL (label_ref:SI 1158)
>   >         (expr_list:REG_LABEL (code_label/v 1158 1268 1439 124 "" "" [0 uses
>   > ])
>   >             (nil))))
>   > 
>   > The code_label 1158 is deleted by delete_trivially_dead_insns apparently 
>   > because the preceding call to loop_optimize in toplev.c has reduced the
>   > number of uses to 1.  Maybe somebody can see how this occurs.
> Seems to me we have a reference counting problem.  There is clearly 
> a reference to label 1158 (insn 1531/1532), but it's reference count
> is zero.

That's what I thought.  It would appear to occur when the above two insns
get moved out of a loop.  The count gets decremented when the old insns
are deleted but not incremented when the new insns are created.  The puzzle
is why the count is zero rather than 1.  Maybe what happens is the two
insns in the loop are deleted first, then the label is deleted?  Notice
that insn 788 is moved to insn 1530.  However, the insn becomes 1531 and
the reg changes to 337 from 208.

I will try to step through the code later.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: pa reload problem

[Jeffrey A Law]

  In message <200012141640.LAA03285@hiauly1.hia.nrc.ca>you write:
  > That's what I thought.  It would appear to occur when the above two insns
  > get moved out of a loop.  The count gets decremented when the old insns
  > are deleted but not incremented when the new insns are created.  The puzzle
  > is why the count is zero rather than 1.  Maybe what happens is the two
  > insns in the loop are deleted first, then the label is deleted?  Notice
  > that insn 788 is moved to insn 1530.  However, the insn becomes 1531 and
  > the reg changes to 337 from 208.
In this case we typically will bump the number of uses before we delete
the insns that way the label doesn't go away unexpectedly.

jeff

Re: pa reload problem

[John David Anglin]

>   In message <200012141640.LAA03285@hiauly1.hia.nrc.ca>you write:
>   > That's what I thought.  It would appear to occur when the above two insns
>   > get moved out of a loop.  The count gets decremented when the old insns
>   > are deleted but not incremented when the new insns are created.  The puzzle
>   > is why the count is zero rather than 1.  Maybe what happens is the two
>   > insns in the loop are deleted first, then the label is deleted?  Notice
>   > that insn 788 is moved to insn 1530.  However, the insn becomes 1531 and
>   > the reg changes to 337 from 208.
> In this case we typically will bump the number of uses before we delete
> the insns that way the label doesn't go away unexpectedly.

As far I can tell, move_movables doesn't do this when it moves an insn
out of a loop.  It does this by deleting the old insn and creating a new
one.  It calls add_label_notes if notes need to be added to the new insn.

I sent a patch to add_label_notes for review which I think fixes the
problem.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

[parisc-linux] PATCH: Adjust label usage count for new insns [was Re: pa reload problem]

[John David Anglin]

>   > Here is the rtl after loop:

>   > (insn 1531 1528 1532 (set (reg/f:SI 337)
>   >         (high:SI (label_ref:SI 1158))) -1 (nil)
>   >     (expr_list:REG_LABEL (code_label/v 1158 1268 1439 124 "" "" [0 uses])
>   >         (nil)))
>   > 
>   > (insn 1532 1531 1535 (set (reg/f:SI 208)
>   >         (lo_sum:SI (reg/f:SI 337)
>   >             (label_ref:SI 1158))) -1 (nil)
>   >     (expr_list:REG_EQUAL (label_ref:SI 1158)
>   >         (expr_list:REG_LABEL (code_label/v 1158 1268 1439 124 "" "" [0 uses
>   > ])
>   >             (nil))))
>   > 
>   > The code_label 1158 is deleted by delete_trivially_dead_insns apparently 
>   > because the preceding call to loop_optimize in toplev.c has reduced the
>   > number of uses to 1.  Maybe somebody can see how this occurs.
> Seems to me we have a reference counting problem.  There is clearly 
> a reference to label 1158 (insn 1531/1532), but it's reference count
> is zero.

These insns result from "moving" insns with label_ref's out of a loop.  The
old insns are deleted by delete_insn which reduces the LABEL_NUSES count.
However, the LABEL_NUSES count is not incremented when the new insns
are created.  This leaves the number of uses of the label at 1 and
delete_trivially_dead_insns deletes the code_label after loop_optimize
completes.

The simplest solution is to modify add_label_notes to increment the usage
count for CODE_LABEL's.  As far as I can tell, add_label_notes is only
used in loop.c when a new insn is created and an old one deleted.  An
alternative solution might be to actually move the old insn in the list
rather than deleting it.

There appears to be a similar situation in gcse.c.

I have run a complete bootstrap and check at -O3 under hpux 10.20 with this
patch and the patch previously posted at
<http://gcc.gnu.org/ml/gcc-bugs/2000-12/msg00156.html>.  It fixes register
rename walking over the return pointer.  The test results are here
<http://gcc.gnu.org/ml/gcc-testresults/2000-12/msg00172.html>.  I have also
run a complete bootstrap and check under i686 linux.

Please review and install if OK.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

2000-12-14  John David Anglin  <dave@hiauly1.hia.nrc.ca>

	* loop.c (add_label_notes): Increment the label usage count when
	a note is added to an insn which refers to a CODE_LABEL.
	* gcse.c (add_label_notes): Likewise.

--- loop.c.orig	Thu Nov 30 12:22:29 2000
+++ loop.c	Fri Dec 15 17:25:46 2000
@@ -1589,7 +1589,8 @@
 }
 \f
 /* If X contains any LABEL_REF's, add REG_LABEL notes for them to all
-  insns in INSNS which use the reference.  */
+   insns in INSNS which use the reference.  LABEL_NUSES for CODE_LABEL
+   references is incremented once for each added note. */
 
 static void
 add_label_notes (x, insns)
@@ -1610,8 +1611,12 @@
          mark_jump_label for additional information).  */
       for (insn = insns; insn; insn = NEXT_INSN (insn))
 	if (reg_mentioned_p (XEXP (x, 0), insn))
-	  REG_NOTES (insn) = gen_rtx_EXPR_LIST (REG_LABEL, XEXP (x, 0),
-						REG_NOTES (insn));
+	  {
+	    REG_NOTES (insn) = gen_rtx_EXPR_LIST (REG_LABEL, XEXP (x, 0),
+						  REG_NOTES (insn));
+	    if (LABEL_P (XEXP (x, 0)))
+	      LABEL_NUSES (XEXP (x, 0))++;
+	  }
     }
 
   fmt = GET_RTX_FORMAT (code);
--- gcse.c.orig	Wed Dec  6 16:16:30 2000
+++ gcse.c	Fri Dec 15 17:23:06 2000
@@ -4839,8 +4839,9 @@
 }
 \f
 /* If X contains any LABEL_REF's, add REG_LABEL notes for them to INSN.
-   We have to add REG_LABEL notes, because the following loop optimization
-   pass requires them.  */
+   If notes are added to an insn which references a CODE_LABEL, the
+   LABEL_NUSES count is incremented.  We have to add REG_LABEL notes,
+   because the following loop optimization pass requires them.  */
 
 /* ??? This is very similar to the loop.c add_label_notes function.  We
    could probably share code here.  */
@@ -4868,6 +4869,8 @@
 
       REG_NOTES (insn) = gen_rtx_EXPR_LIST (REG_LABEL, XEXP (x, 0),
 					    REG_NOTES (insn));
+      if (LABEL_P (XEXP (x, 0)))
+        LABEL_NUSES (XEXP (x, 0))++;
       return;
     }
 

Re: PATCH: Adjust label usage count for new insns [was Re: pa reload problem]

[Jeffrey A Law]

  In message <200012162038.PAA25535@hiauly1.hia.nrc.ca>you write:
  > 2000-12-14  John David Anglin  <dave@hiauly1.hia.nrc.ca>
  > 
  > 	* loop.c (add_label_notes): Increment the label usage count when
  > 	a note is added to an insn which refers to a CODE_LABEL.
  > 	* gcse.c (add_label_notes): Likewise.
Thanks.  Installed.

jeff

Re: [linux-lvm] Lost harddrive space

[Heinz J. Mauelshagen]

On Thu, Apr 12, 2001 at 05:51:30PM +0200, Erik Rigtorp wrote:
> Hi!
> 
> I have a 60999.46 MB linux lvm partition. But when I pvcreate it and 
> pvdisplay /dev/hda2 it says the physical volume is only 56.8GB and 1.03MB 
> not used. Where are those lost Gigs? I use the lvm that comes with the 
> 2.4.3 kernel.

Could you provide "pvdisplay /dev/hda2" and "fdisk -l /dev/hda" please?

It is likely caused by the math:

 1 GB = 2^30 Byte

> 
> /Erik
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] Lost harddrive space

[Heinz J. Mauelshagen]

On Thu, Apr 12, 2001 at 08:19:34PM +0200, Erik Rigtorp wrote:
> My harddrive is 61.4GB i think. I think LVM is great software, it should 
> be standard in all ditributions.
> 
> /Erik
> 
> #fdisk -l
> 
> Disk /dev/hda: 255 heads, 63 sectors, 7476 cylinders
> Units = cylinders of 16065 * 512 bytes
> 
>    Device Boot    Start       End    Blocks   Id  System
> /dev/hda1   *         1        61    489951   83  Linux
> /dev/hda2            62      7476  59560987+  8e  Unknown

59560987 / 1024 / 1024 == 56.8.

No lost space whatshowever.

> 
> 
> #pvdisplay /dev/hda2
> 
> --- Physical volume ---
> PV Name               /dev/hda2
> VG Name               erkki
> PV Size               56.8 GB / NOT usable 1.03 MB [LVM: 177 KB]
> PV#                   1
> PV Status             available
> Allocatable           yes
> Cur LV                3
> PE Size (KByte)       4096
> Total PE              14541
> Free PE               12237
> Allocated PE          2304
> PV UUID               IUz7yE-MP2i-QwJ2-23GW-70cu-P2Fw-7Jn0xA
> 
> 
> 
> -----Original Message-----
> From: "Heinz J. Mauelshagen" <Mauelshagen@sistina.com>
> To: linux-lvm@sistina.com
> Date: Thu, 12 Apr 2001 18:28:56 +0000
> Subject: Re: [linux-lvm] Lost harddrive space
> 
> > On Thu, Apr 12, 2001 at 05:51:30PM +0200, Erik Rigtorp wrote:
> > > Hi!
> > > 
> > > I have a 60999.46 MB linux lvm partition. But when I pvcreate it and 
> > > pvdisplay /dev/hda2 it says the physical volume is only 56.8GB and
> > 1.03MB 
> > > not used. Where are those lost Gigs? I use the lvm that comes with
> > the 
> > > 2.4.3 kernel.
> > 
> > Could you provide "pvdisplay /dev/hda2" and "fdisk -l /dev/hda" please?
> > 
> > It is likely caused by the math:
> > 
> >  1 GB = 2^30 Byte
> > 
> > > 
> > > /Erik
> > > 
> > > 
> > > _______________________________________________
> > > linux-lvm mailing list
> > > linux-lvm@sistina.com
> > > http://lists.sistina.com/mailman/listinfo/linux-lvm
> > 
> > -- 
> > 
> > Regards,
> > Heinz    -- The LVM Guy --
> > 
> > *** Software bugs are stupid.
> >     Nevertheless it needs not so stupid people to solve them ***
> > 
> > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> > -=-=-=-
> > 
> > Heinz Mauelshagen                                 Sistina Software Inc.
> > Senior Consultant/Developer                       Am Sonnenhang 11
> >                                                   56242 Marienrachdorf
> >                                                   Germany
> > Mauelshagen@Sistina.com                           +49 2626 141200
> >                                                        FAX 924446
> > =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
> > -=-=-=-
> > _______________________________________________
> > linux-lvm mailing list
> > linux-lvm@sistina.com
> > http://lists.sistina.com/mailman/listinfo/linux-lvm
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [patch] linux likes to kill bad inodes

[Andreas Dilger]

I previously wrote:
> I will post a patch separately which handles a couple of cases where
> *_delete_inode() does not call clear_inode() in all cases.

OK, here it is.  The ext2_delete_inode() change isn't exactly a bug fix,
but rather a "performance" change.  No need to hold BLK to check status
or call clear_inode() (we call clear_inode() outside BLK in VFS if
delete_inode() method does not exist).

Cheers, Andreas
=======================================================================
diff -ru linux-2.4.4p1.orig/fs/ext2/inode.c linux/fs/ext2/inode.c
--- linux-2.4.4p1.orig/fs/ext2/inode.c	Tue Apr 10 16:44:49 2001
+++ linux/fs/ext2/inode.c	Fri Apr 27 13:51:15 2001
@@ -44,12 +47,12 @@
  */
 void ext2_delete_inode (struct inode * inode)
 {
-	lock_kernel();
-
 	if (is_bad_inode(inode) ||
 	    inode->i_ino == EXT2_ACL_IDX_INO ||
 	    inode->i_ino == EXT2_ACL_DATA_INO)
 		goto no_delete;
+
+	lock_kernel();
 	inode->u.ext2_i.i_dtime	= CURRENT_TIME;
 	mark_inode_dirty(inode);
 	ext2_update_inode(inode, IS_SYNC(inode));
@@ -59,9 +62,7 @@
 	ext2_free_inode (inode);
 
 	unlock_kernel();
 	return;
 no_delete:
-	unlock_kernel();
 	clear_inode(inode);	/* We must guarantee clearing of inode... */
 }
 
diff -ru linux-2.4.4p1.orig/fs/bfs/inode.c linux/fs/bfs/inode.c
--- linux-2.4.4p1.orig/fs/bfs/inode.c	Tue Apr 10 16:44:49 2001
+++ linux/fs/bfs/inode.c	Fri Apr 27 15:45:31 2001
@@ -145,7 +145,7 @@
 	if (is_bad_inode(inode) || inode->i_ino < BFS_ROOT_INO ||
 	    inode->i_ino > inode->i_sb->su_lasti) {
 		printf("invalid ino=%08lx\n", inode->i_ino);
-		return;
+		goto bad_inode;
 	}
 	
 	inode->i_size = 0;
@@ -155,8 +156,7 @@
 	bh = bread(dev, block, BFS_BSIZE);
 	if (!bh) {
 		printf("Unable to read inode %s:%08lx\n", bdevname(dev), ino);
-		unlock_kernel();
-		return;
+		goto bad_unlock;
 	}
 	off = (ino - BFS_ROOT_INO)%BFS_INODES_PER_BLOCK;
 	di = (struct bfs_inode *)bh->b_data + off;
@@ -178,7 +178,9 @@
 		s->su_lf_eblk = inode->iu_sblock - 1;
 		mark_buffer_dirty(s->su_sbh);
 	}
+bad_unlock:
 	unlock_kernel();
+bad_inode:
 	clear_inode(inode);
 }
 
diff -ru linux-2.4.4p1.orig/fs/ufs/ialloc.c linux/fs/ufs/ialloc.c
--- linux-2.4.4p1.orig/fs/ufs/ialloc.c	Thu Nov 16 14:18:26 2000
+++ linux/fs/ufs/ialloc.c	Fri Apr 27 15:53:26 2001
@@ -82,6 +82,7 @@
 	if (!((ino > 1) && (ino < (uspi->s_ncg * uspi->s_ipg )))) {
 		ufs_warning(sb, "ufs_free_inode", "reserved inode or nonexistent inode %u\n", ino);
 		unlock_super (sb);
+		clear_inode (inode);
 		return;
 	}
 	
@@ -90,6 +91,7 @@
 	ucpi = ufs_load_cylinder (sb, cg);
 	if (!ucpi) {
 		unlock_super (sb);
+		clear_inode (inode);
 		return;
 	}
 	ucg = ubh_get_ucg(UCPI_UBH);
-- 
Andreas Dilger                               TurboLabs filesystem development
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: Tracking down semaphore usage/leak

[Ken Brownfield]

Urgh, learn something new everyday (ipcs, ipcrm).  My apologies; apropos
didn't catch it on my boxes. :-(
-- 
Ken.
brownfld@irridia.com

On Tue, Jun 26, 2001 at 02:09:16PM -0700, Ken Brownfield wrote:
| With RedHat's new Samba 2.0.10 RPM (the one to patch the latest 
| vulnerability) they seem to have sniffed enough glue to start using SysV 
| IPC semaphores which apparently leak until SEM??? are reached.  semget() 
| is returning "No space left on device", and disk/inodes/memory are all 
| fine.
| 
| Anyway, could someone give me a very quick rundown of the options for 
| tracking/force-freeing semaphores, or how to determine from proc, if 
| possible, what the current semaphore allocation status is?  Or did RH 
| slay a machine I really don't want to reboot?  I've restarted all 
| semaphore-using processes to no avail, but even so the SEM??? limits are 
| far above the normal needs of this machine.
| 
| Thanks much.  Searched the archives/Google/FAQ/semaphore docs; sorry if 
| it's been covered.  I'll summarize if folks want to hit me on or off the 
| list.
| --
| Ken.
| brownfld@irridia.com

Re: user-mode port 0.44-2.4.7

[Alan Cox]

> Suppose I loop against xtime reaching a particular value.  While this is

xtime isnt used this way that I can see. jiffies however is. There are good
arguments for getting rid of most [ab]use of jiffies however. For one its
pretty important to scaling on both big mainframes and beowulf setups doing
heavy computation to reduce timer ticks

Re: user-mode port 0.44-2.4.7

[Chris Friesen]

Alan Cox wrote:
> 
> > Suppose I loop against xtime reaching a particular value.  While this is
> 
> xtime isnt used this way that I can see. jiffies however is. There are good
> arguments for getting rid of most [ab]use of jiffies however. For one its
> pretty important to scaling on both big mainframes and beowulf setups doing
> heavy computation to reduce timer ticks

jiffies is (as of 2.4.7 anyways) marked as volatile, so we're safe there.  My
point is this--should someone writing badly designed (but technically correct)
code be able to totally hose the system?

The only difference between volatile and normal is that if it is marked as
volatile it must be accessed every time rather than being pre-cached.  If we
never spin on accessing xtime, then the fact that we can't optimize it shouldn't
hurt. (Am I wrong here?  If I am then please explain because I'm missing
something...)  If someone ever *does* spin on xtime, then we really don't want
to optimize that access out of the loop, because doing so could cause nasty
problems.


-- 
Chris Friesen                    | MailStop: 043/33/F10  
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: patch for allowing msdos/vfat nfs exports

[Alan Cox]

> I've been using it for half a day now and so far it hasn't done
> anything bad, but please be careful if you decide to test it and
> backup your data and after testing, be sure to compare your data
> to your backup.

Rename ?

> +	struct inode *inode = dentry->d_inode;
> +	unsigned int i_pos = MSDOS_I(inode)->i_location;

i_location is not a constant across renames or other operations, so you may
inadvertantly do I/O to completely the wrong file


The infrastructure looks great, I just don't think your handles are safe 

Re: Externally transparent routing

[Alan Cox]

> To prevent my NAT-box from showing up on traceroutes I'd like to let it
> route without decreasing the TTL. I was told that proxy arp also archieves

And what happens if you get a routing loop ?

A NAT box really does need to drop the TTL. Nothing stops you giving it a
more bizarre name, or indeed you can do what a few folks have found
excruciatingly funny to do to tracerouters which is to spoof totally bogus
icmp unreachables so they see crazy paths

Re: Externally transparent routing

[Jordi Verwer]

> And what happens if you get a routing loop ?
Bad Things would happen, but I only have one router and since it's a NAT box
it isn't very likely to end up in a routing loop anyway.

> A NAT box really does need to drop the TTL. Nothing stops you giving it a
> more bizarre name, or indeed you can do what a few folks have found
> excruciatingly funny to do to tracerouters which is to spoof totally bogus
> icmp unreachables so they see crazy paths
What I wanted to do was be able to send my traceroutes to websites that
don't function properly, but since my NAT box is headless and I'd like to
avoid the hassle of SSH-ing to it, I do these traceroutes from one of my
internal machines. If I don't manually remove my NAT box from the list, the
braindead webmaster will allways blame my NAT box (which naturally is
innocent;)). But I suppose you do not want this to be possible. That is
understandable, but still BSD has a very clean implementation of
transrouting and I see no reason not to let Linux do this.

Jordi Verwer
P.S.: I adjust my computer's (which isn't mine btw, but belongs to my
"boss") date.
P.P.S.: Still not subscribed, so please CC any replies to me. Thank you.

Re: user-mode port 0.44-2.4.7

[Alan Cox]

> Should head and tail be volatile in the definition, or should they be
> accessed with:
> int head = (volatile)myqueue.head;
> or with barrier() around the read/write?

The best way is to use barrier calls. It makes your assumptions about
ordering absolutely explicit. However you should still be careful - you
can't be sure that head will be read atomically or written atomically on
all processors eg if it was

	struct
	{
		unsigned char head;
		unsigned char tail;
		char buf[256];
	}

you would get some suprisingly unpleasant suprises on SMP Alpha. Currently
"int" is probably safe for all processors.

So unless this is a precision tuned fast path it is better to play safe with
this and use atomic_t or locking. The spinlock cost on an Athlon or a later
PIII is pretty good in most cases. Using the -ac prefetch stuff can make it
good in almost all cases, but thats probably a 2.5 thing for the generic
case.

Basically locks are getting cheaper on x86, the suprises are getting more
interesting on non-x86

Alan

my patches won't compile under 2.4.7

[Kirk Reiser]

As of 2.4.7 my patches to the kernel won't compile.  It appears to be
something to do with devfs_fs_kernel.h being part of miscdevices.h.  I
have sifted through the code but have not been able to determine
exactly why they won't work any more.  Here is the error output from
my compile:

gcc -D__KERNEL__ -I/usr/src/linux/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=i586    -c -o speakup.o speakup.c
In file included from /usr/src/linux/include/linux/locks.h:8,
                 from /usr/src/linux/include/linux/devfs_fs_kernel.h:6,
                 from /usr/src/linux/include/linux/miscdevice.h:4,
                 from speakup.c:63:
/usr/src/linux/include/linux/pagemap.h:35: `currcons' undeclared here (not in a function)
/usr/src/linux/include/linux/pagemap.h:35: parse error before `.'
make[4]: *** [speakup.o] Error 1

I'm not sure even where to start trying to describe what I've looked
at and what I don't understand.  It appears that page_cache_alloc() is
now an inline function with an argument passed to it, where it used to
be a #define with no arguments.  I see that struct misc_device now has
a new member devfs_handle but the other drivers I've looked at rtc.c
haven't changed their structure members to take this into account.  It
seems nothing new is necessary because misc_register checks if it's
been set or not.  The two error lines don't look to me to have anything
to do with any of these things either currcons isn't used in any of
the misc_device structure or anything I can see which might end up
calling page_cache_alloc().  Can anyone give me any ideas what I
should check to hunt down exactly what's going on here?  It almost
looks like gcc is getting screwed up in it's parsing or something.

Any ideas will greatefully be accepted I'm lost!

  Kirk

-- 

Kirk Reiser				The Computer Braille Facility
e-mail: kirk@braille.uwo.ca		University of Western Ontario
phone: (519) 661-3061

Re: my patches won't compile under 2.4.7

[Alan Cox]

> 
> As of 2.4.7 my patches to the kernel won't compile.  It appears to be
> something to do with devfs_fs_kernel.h being part of miscdevices.h.  I
> have sifted through the code but have not been able to determine
> exactly why they won't work any more.  Here is the error output from
> my compile:
> 
> gcc -D__KERNEL__ -I/usr/src/linux/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=i586    -c -o speakup.o speakup.c
> In file included from /usr/src/linux/include/linux/locks.h:8,
>                  from /usr/src/linux/include/linux/devfs_fs_kernel.h:6,
>                  from /usr/src/linux/include/linux/miscdevice.h:4,
>                  from speakup.c:63:
> /usr/src/linux/include/linux/pagemap.h:35: `currcons' undeclared here (not in a function)
> /usr/src/linux/include/linux/pagemap.h:35: parse error before `.'
> make[4]: *** [speakup.o] Error 1
> 
> I'm not sure even where to start trying to describe what I've looked
> at and what I don't understand.  It appears that page_cache_alloc() is
> now an inline function with an argument passed to it, where it used to
> be a #define with no arguments.  I see that struct misc_device now has
> a new member devfs_handle but the other drivers I've looked at rtc.c
> haven't changed their structure members to take this into account.  It
> seems nothing new is necessary because misc_register checks if it's
> been set or not.  The two error lines don't look to me to have anything
> to do with any of these things either currcons isn't used in any of
> the misc_device structure or anything I can see which might end up
> calling page_cache_alloc().  Can anyone give me any ideas what I
> should check to hunt down exactly what's going on here?  It almost
> looks like gcc is getting screwed up in it's parsing or something.
> 
> Any ideas will greatefully be accepted I'm lost!
> 
>   Kirk
> 
> -- 
> 
> Kirk Reiser				The Computer Braille Facility
> e-mail: kirk@braille.uwo.ca		University of Western Ontario
> phone: (519) 661-3061
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: my patches won't compile under 2.4.7

[Kirk Reiser]

Huh?  Did you actually write something below Alan? or are you just
making me feel insecure? 'grin'

  Kirk

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > 
> > As of 2.4.7 my patches to the kernel won't compile.  It appears to be
> > something to do with devfs_fs_kernel.h being part of miscdevices.h.  I
> > have sifted through the code but have not been able to determine
> > exactly why they won't work any more.  Here is the error output from
> > my compile:
> > 
> > gcc -D__KERNEL__ -I/usr/src/linux/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=i586    -c -o speakup.o speakup.c
> > In file included from /usr/src/linux/include/linux/locks.h:8,
> >                  from /usr/src/linux/include/linux/devfs_fs_kernel.h:6,
> >                  from /usr/src/linux/include/linux/miscdevice.h:4,
> >                  from speakup.c:63:
> > /usr/src/linux/include/linux/pagemap.h:35: `currcons' undeclared here (not in a function)
> > /usr/src/linux/include/linux/pagemap.h:35: parse error before `.'
> > make[4]: *** [speakup.o] Error 1
> > 
> > I'm not sure even where to start trying to describe what I've looked
> > at and what I don't understand.  It appears that page_cache_alloc() is
> > now an inline function with an argument passed to it, where it used to
> > be a #define with no arguments.  I see that struct misc_device now has
> > a new member devfs_handle but the other drivers I've looked at rtc.c
> > haven't changed their structure members to take this into account.  It
> > seems nothing new is necessary because misc_register checks if it's
> > been set or not.  The two error lines don't look to me to have anything
> > to do with any of these things either currcons isn't used in any of
> > the misc_device structure or anything I can see which might end up
> > calling page_cache_alloc().  Can anyone give me any ideas what I
> > should check to hunt down exactly what's going on here?  It almost
> > looks like gcc is getting screwed up in it's parsing or something.
> > 
> > Any ideas will greatefully be accepted I'm lost!
> > 
> >   Kirk
> > 
> > -- 
> > 
> > Kirk Reiser				The Computer Braille Facility
> > e-mail: kirk@braille.uwo.ca		University of Western Ontario
> > phone: (519) 661-3061
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
> > 
> 
> 

-- 

Kirk Reiser				The Computer Braille Facility
e-mail: kirk@braille.uwo.ca		University of Western Ontario
phone: (519) 661-3061

Re: my patches won't compile under 2.4.7

[Richard Gooch]

Alan Cox writes:
> > 
> > As of 2.4.7 my patches to the kernel won't compile.  It appears to be
> > something to do with devfs_fs_kernel.h being part of miscdevices.h.  I
> > have sifted through the code but have not been able to determine
> > exactly why they won't work any more.  Here is the error output from
> > my compile:

I don't see why you're pointing the finger devfs_fs_kernel.h. Other
miscdevice drivers compile fine.

> > gcc -D__KERNEL__ -I/usr/src/linux/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=i586    -c -o speakup.o speakup.c
> > In file included from /usr/src/linux/include/linux/locks.h:8,
> >                  from /usr/src/linux/include/linux/devfs_fs_kernel.h:6,
> >                  from /usr/src/linux/include/linux/miscdevice.h:4,
> >                  from speakup.c:63:
> > /usr/src/linux/include/linux/pagemap.h:35: `currcons' undeclared here (not in a function)
> > /usr/src/linux/include/linux/pagemap.h:35: parse error before `.'
> > make[4]: *** [speakup.o] Error 1

Looking at my copy of include/linux/pagemap.h I see no instance of
"currcons" on line 35 or elsewhere.

> > I'm not sure even where to start trying to describe what I've looked
> > at and what I don't understand.  It appears that page_cache_alloc() is
> > now an inline function with an argument passed to it, where it used to
> > be a #define with no arguments.  I see that struct misc_device now has
> > a new member devfs_handle but the other drivers I've looked at rtc.c

This is not new. struct misc_device has had a "devfs_handle" field for
a long time. Since 2.3.46, in fact. So when you say above "since
2.4.7", I suspect you mean "after virgin 2.2.x". It would have helped
if you had specified this.

My guess is that your patch has some bad #define somewhere. Again, it
would have helped if you had sent the patch as well.

Anyway, I don't think this problem is even remotely related to devfs.
I suggest you post more complete information to the linux-kernel
mailing list. Then maybe someone there can help you.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: my patches won't compile under 2.4.7

[Kirk Reiser]

Actually it wasn't Alan pointing the finger it was me.  I was only
trying to figure out what the errors meant and they pointed to
devfs_fs_kernel.h.  The problem as I suspected at eh time was entirely
unrelated.  I moved my #include of misc_devices.h up and removed a
duplicate #include for linux/init.h and poof she compiled.  I am
starting to become a believer in voodoo computing again I guess.

On another note related to devfs though when I compile devfs in the
system just hangs.  I am wondering if I am registering my synth device
before devfs has memory allocated.  I register very early in the boot
process in console_init() and experienced similar problems before because I
don't think  kmalloc() may be available that early in the sequence.

The question then is, do you think that could be why the system is
hanging with devfs configured in?

  Kirk

-- 

Kirk Reiser				The Computer Braille Facility
e-mail: kirk@braille.uwo.ca		University of Western Ontario
phone: (519) 661-3061

Re: my patches won't compile under 2.4.7

[Richard Gooch]

Kirk Reiser writes:
> On another note related to devfs though when I compile devfs in the
> system just hangs.  I am wondering if I am registering my synth device
> before devfs has memory allocated.  I register very early in the boot
> process in console_init() and experienced similar problems before because I
> don't think  kmalloc() may be available that early in the sequence.
> 
> The question then is, do you think that could be why the system is
> hanging with devfs configured in?

Yes. Calling kmalloc() before MM is set up is not allowed. See the
comments in drivers/char/console.c which talks about not calling
kmalloc() before console_init().

Simply move your driver registration after MM is set up. Use
module_init() to declare your initialisation function. This works for
both modules and built-in drivers. Registering a driver before MM
setup is considered bad practice.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: user-mode port 0.44-2.4.7

[Alan Cox]

> > This is not a gcc issue. Even if gcc _were_ to generate bad code, the
> > global volatile _still_ wouldn't be the correct answer.
> 
> I think his worry is the pedantic reason that without the volatile gcc is
> allowed to write code that chokes and dies if xtime happens to change in a
> volatile manner.  The example given earlier was:

Make the volatility explicit where it is needed, either to express a barrier
with barrier() or an assignment as in

	foo = (volatile)xtime

This makes it clear where the barriers are and avoids unpleasant
optimisation hits elsewhere.

Alan

Re: Replacing the Console driver

[Alan Cox]

> I haven't checked more recent versions, but I think recent mess (uses same cpu cores
> as mame) supports early macs, so there may be an implementation example there?..

Early macintosh doesn't have an MMU as standard, The MacII had an optional
MMU (for running A/UX) and it became standard on the later Mac systems.

Alan

Re: Replacing the Console driver

[Alan Cox]

> I haven't checked more recent versions, but I think recent mess (uses same cpu cores
> as mame) supports early macs, so there may be an implementation example there?..

Early macintosh doesn't have an MMU as standard, The MacII had an optional
MMU (for running A/UX) and it became standard on the later Mac systems.

Alan

Re: Replacing the Console driver

[Ralf Baechle]

On Thu, Jul 26, 2001 at 01:20:19AM +0100, Alan Cox wrote:

> > I haven't checked more recent versions, but I think recent mess (uses same cpu cores
> > as mame) supports early macs, so there may be an implementation example there?..
> 
> Early macintosh doesn't have an MMU as standard, The MacII had an optional
> MMU (for running A/UX) and it became standard on the later Mac systems.

MC68851?

  Ralf

Re: Replacing the Console driver

[William Jhun]

On Thu, 26 Jul 2001, Ralf Baechle wrote:

> On Thu, Jul 26, 2001 at 01:20:19AM +0100, Alan Cox wrote:
> 
> > > I haven't checked more recent versions, but I think recent mess (uses same cpu cores
> > > as mame) supports early macs, so there may be an implementation example there?..
> > 
> > Early macintosh doesn't have an MMU as standard, The MacII had an optional
> > MMU (for running A/UX) and it became standard on the later Mac systems.
> 
> MC68851?

Yeah, but most home computers based on M68k didn't contain these (and they
probably weren't that cheap). The 68030 had it built in (except the
68EC030). Most of the 68k home computer OSes, like AmigaDOS (and probably
MacOS and Atari ST too) didn't come built with support for an MMU, so even
if you had one, it would only be useful for either running some UNIX
variant or advanced debugging tools (Amiga users remember good 'ol
Enforcer?).

Anyway, like I said, if anyone sees software emulation routines for a
somewhat modern MMU, I'd be interested. I'm sure it's been done (though
I'd like to see some routines that make use of the native MMU via some
kernel interface). It might be slow but it'd make for a killer
general-purpose debugging system. Or gobs of geeky fun.

Will


> 
>   Ralf
> 

Re: IGMP join/leave time variability

[Alan Cox]

> ASAP with respect to the user mode calls.
> 1. What would be the harm if I set IGMP_Initial_Report_Delay to something
> very small like 5 to 10 (jiffies)?  No need for net_random() I'de expect in
> that case?

Read the IGMP RFC documents they discuss in detail the cases where time
delays and randomness are needed and important. 

Re: Validating Pointers

[Alan Cox]

> Is there a general (correct) kernel subroutine to validate a pointer
> received in a routine as input from the outside world?  Is access_ok() a
> good one to use?

access_ok may do minimal checks, or no checking at all. The only point at
which you can validate a user point is when you use copy*user and
get/put_user to access the data.

Re: Validating Pointers

[tpepper]

On Thu 26 Jul at 16:52:48 +0100 alan@lxorguk.ukuu.org.uk done said:
> access_ok may do minimal checks, or no checking at all. The only point at
> which you can validate a user point is when you use copy*user and
> get/put_user to access the data.

Should the i386 access_ok() fail when checking a copy to/from userspace
from/to a static in a driver module?  The __copy_to|from_user work fine
and copy_to|from_user fail, but I guess that doesn't mean access_ok()
is the culprit.  I don't know intel assembly and the platforms for
which I do get the assembly don't do much in access_ok() so there's no
comparing...but I'd have thought they'd be more concerned with the user
address location than the kernel one.

t.

Re: Validating Pointers

[Alan Cox]

> Should the i386 access_ok() fail when checking a copy to/from userspace
> from/to a static in a driver module?  The __copy_to|from_user work fine
> and copy_to|from_user fail, but I guess that doesn't mean access_ok()
> is the culprit.  I don't know intel assembly and the platforms for
> which I do get the assembly don't do much in access_ok() so there's no
> comparing...but I'd have thought they'd be more concerned with the user
> address location than the kernel one.

You can't pass kernel address as if they were userspace. It might happen to
sometimes work on some architectures. Take a look at the set_fs() stuff

Re: Validating Pointers

[tpepper]

On Thu 26 Jul at 18:12:57 +0100 alan@lxorguk.ukuu.org.uk done said:
> 
> You can't pass kernel address as if they were userspace. It might happen to
> sometimes work on some architectures. Take a look at the set_fs() stuff

Am I?  I though I was doing a pretty plain user<->kernel copy:

	copy_to_user(user_addr, kernel_addr, size);
		and
	copy_from_user(kernel_addr, user_addr, size);

Are you saying that static and dynamically allocated kernel variables end up
in different segments (kernel_ds and user_ds) and the copy is only expected to
succeed if the to and from addresses are in the same segment?

Tim

Re: Validating Pointers

[Alan Cox]

> 	copy_to_user(user_addr, kernel_addr, size);
> 		and
> 	copy_from_user(kernel_addr, user_addr, size);
> 
> Are you saying that static and dynamically allocated kernel variables end up
> in different segments (kernel_ds and user_ds) and the copy is only expected to
> succeed if the to and from addresses are in the same segment?

user and kernel address spaces are seperate. On S/390 and M68K for example
they occupy the same values for both. Long long ago this was done via
segments on x86 (we dont use segments now) and thus the functions to do 
what you want are still called set_fs/get_fs/get_ds

Re: IGMP join/leave time variability

[Alan Cox]

> >From this, I infer that there should be _no_ initial delay on sending 
> the IGMP join.  In fact, a quick peek at the source confirms this: 
> (net/ipv4/igmp.c):
> 
> #define IGMP_Initial_Report_Delay               (1*HZ)
> 
> /* IGMP_Initial_Report_Delay is not from IGMP specs!
>  * IGMP specs require to report membership immediately after
>  * joining a group, but we delay the first report by a
>  * small interval. It seems more natural and still does not
>  * contradict to specs provided this delay is small enough.
>  */
> 
> But this "small interval" is actually very noticeable in our application.

I suspect the small interval for the first one should be 1 not 1*HZ. That
would keep a little bit of jitter which is good to avoid the multicast
receive/join group problem

[Lots of clients all running an app listening for multicast packets, one
 packet says 'do xyz on this group' and they all then send joins at the same
 instant]

Alan

Re: Proliant ML530, Megaraid 493 (Elite 1600), 2.4.7, timeout & abort

[Alan Cox]

> the Elite 1600) in them. I am able to get them to boot with 2.2.19, but
> not 2.4.7.   Under 2.4.7 the relevant message is

Ok first thing - does 2.4.6 work. If so then  there is a bug or firmware
incompatibility with the firmware set you have and the newer driver.

Re: Support for serial console on legacy free machines

[Alan Cox]

> console is "Serial Port Console Redirection" (SPCR) table. This table
> gives me almost all the information I need to initialize and use a
> serial console. The bummer is this table was designed by Microsoft and
> Microsoft owns the copyright on it. Microsoft primarily designed this
> table for use by Whistler. Their copyright may cause potential problems
> with using it in Linux. This makes me reluctant to use this table. I

Such as ?

If its a table that microsoft added to ACPI and its well thought out I don't
see a big problem technically. There are a collection of BIOS services we
use that were microsoft originated

Re: Support for serial console on legacy free machines

[Khalid Aziz]

Alan Cox wrote:
> 
> > console is "Serial Port Console Redirection" (SPCR) table. This table
> > gives me almost all the information I need to initialize and use a
> > serial console. The bummer is this table was designed by Microsoft and
> > Microsoft owns the copyright on it. Microsoft primarily designed this
> > table for use by Whistler. Their copyright may cause potential problems
> > with using it in Linux. This makes me reluctant to use this table. I
> 
> Such as ?
> 
> If its a table that microsoft added to ACPI and its well thought out I don't
> see a big problem technically. There are a collection of BIOS services we
> use that were microsoft originated

I can not say this table is part of ACPI 2.0. ACPI 2.0 Spec document
lists SPCR in the DESCRIPTION_HEADER signatures but calls it Microsoft
Serial Port Console Redirection Table and refers to the URL on Microsoft
web site. If you go to this URL, you see the Microsoft copyright and
terms of use license. The same applies to DBGP (Debug Port Table).

-- 
Khalid

====================================================================
Khalid Aziz                              Linux Systems Operation R&D
(970)898-9214                                        Hewlett-Packard
khalid@fc.hp.com                                    Fort Collins, CO

Re: IGMP join/leave time variability

[David S. Miller]

Alan Cox writes:
 > > But this "small interval" is actually very noticeable in our application.
 > 
 > I suspect the small interval for the first one should be 1 not 1*HZ. That
 > would keep a little bit of jitter which is good to avoid the multicast
 > receive/join group problem

I've changed it to "1" in my sources.  Thanks.

Later,
David S. Miller
davem@redhat.com

Re: 2.4.7 + VIA Pro266 + 2xUltraTx2 lockups

[Alan Cox]

> So the system is stable when driving a single Tx2 card, or on a BX,
> but just not two Tx2's together on the pro266 board :-/ So it's
> perhaps (I'm guessing here :) a non-trivial Tx2 driver bug or maybe a
> VIA Pro266 problem?

Firstly please try 2.4.6-ac5 as that has the proper VIA workaround for their
bridge bugs. Its useful to rule out the very conservative approach the older
kernels use to avoid the disk corruption problem they had

Alan

Re: 2.4.7 + VIA Pro266 + 2xUltraTx2 lockups

[Robin Humble]

Alan Cox wrote:
>Robin Humble wrote:
>> So the system is stable when driving a single Tx2 card, or on a BX,
>> but just not two Tx2's together on the pro266 board :-/ So it's
>> perhaps (I'm guessing here :) a non-trivial Tx2 driver bug or maybe a
>> VIA Pro266 problem?
>
>Firstly please try 2.4.6-ac5 as that has the proper VIA workaround for their
>bridge bugs. Its useful to rule out the very conservative approach the older
>kernels use to avoid the disk corruption problem they had

Ok. That locked up in the same way unfortunately :-/
Also a 2.4.8-pre1-xfs that I just tried...
I tried the "noapic" option as suggested in another email and that
didn't change anything either.

We've moved all the disks and controllers to a BX m/b machine for now, but
if there's anything else you want us to be guinea pigs for them we'll be
happy to try it out on the VIA Pro266 machine.
One other odd thing is that I have yet to make the CUV266 board see any
devices on its built-in secondary IDE controller. I have no idea why that
could be... The BIOS just doesn't detect them. Might that be a related
problem? Perhaps it's a faulty motherboard? Seems unlikely.

Please CC me on any replies as I'm not subscribed... ta...

cheers,
robin

Re: Hard disk problem:

[Alan Cox]

> >  Model=IBM-DTLA-307030, FwRev=TX4OA50C, SerialNo=YKDYKGF1437
> 
> Ah, one of these excellent Hungarian DTLA drives? :) AFAIK, the entire batch
> was broken, although there are people who insist that there was no single
> working hard drive leaving that factory! I personally have seen 7 out of 7
> failing...

I have a large collection of these drives and none of them are problematic,
while the maxtors seem a little less reliable

> Take it back to where you bought it and demand a replacement for something
> NOT bearing "MADE IN HUNGARY" sign.

Of course the writer of this is Polish and the drives are Hungarian ..

Alan

Re: Hard disk problem:

[Steve Underwood]

Alan Cox wrote:
> 
> > >  Model=IBM-DTLA-307030, FwRev=TX4OA50C, SerialNo=YKDYKGF1437
> >
> > Ah, one of these excellent Hungarian DTLA drives? :) AFAIK, the entire batch
> > was broken, although there are people who insist that there was no single
> > working hard drive leaving that factory! I personally have seen 7 out of 7
> > failing...
> 
> I have a large collection of these drives and none of them are problematic,
> while the maxtors seem a little less reliable
> 
> > Take it back to where you bought it and demand a replacement for something
> > NOT bearing "MADE IN HUNGARY" sign.
> 
> Of course the writer of this is Polish and the drives are Hungarian ..
> 
But he is right. Practically all the "Made in Hungary" ones develop bad
sectors after a few months. The "Made in Phillipinnes" ones do not.
Strangely, I am Hong Kong and almost all the GXP75s we got here were
made in Hungary - go figure! They were so bad the dealers finally
wouldn't stock them. If your experience has been different, think
yourself lucky.

Regards,
Steve

Re: Hard disk problem:

[Bill Pringlemeir]

>>>>> "Steve" == Steve Underwood <steveu@coppice.org> writes:

 Steve> But he is right. Practically all the "Made in Hungary" ones
 Steve> develop bad sectors after a few months. The "Made in
 Steve> Phillipinnes" ones do not.  Strangely, I am Hong Kong and
 Steve> almost all the GXP75s we got here were made in Hungary - go
 Steve> figure! They were so bad the dealers finally wouldn't stock
 Steve> them. If your experience has been different, think yourself
 Steve> lucky.

I have an IBM drive made in Hungary.  It get `fiery hot'!  I kept
moving it until I had it in a place with good thermal contact to the
case.  Then these drive errors went away.  At first I had a Linux
install on that drive.  Later it crashed, I fixed it, deleted the OS
on my HDA that I was no longer using and moved Linux there.  Now I
only keep MP3s, tmp, and swap (a 2nd one) on the IBM drive.

Sometimes when I close the case, I still get errors.  So it may be a
case of overheating.  You could try to change the position of the drive
to see if it fixes things.  Mine was made in Hungary.  And in case (ha ha)
I am talking crap,

[bpringle@localhost bpringle]$ dmesg | grep ^hdd:
hdd: IBM-DTTA-351010, ATA DISK drive
hdd: 19807200 sectors (10141 MB) w/466KiB Cache, CHS=19650/16/63, UDMA(33)

 Model=IBM-DTTA-351010, FwRev=T56OA73A, SerialNo=WF0WFFD7387
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=34
 BuffType=3(DualPortCache), BuffSize=466kB, MaxMultSect=16, MultSect=off
 DblWordIO=no, maxPIO=2(fast), DMA=yes, maxDMA=2(fast)
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=19807200
 WARNING 3293136 ORPHANED SECTORS :: KERNEL REPORTING ERROR
 tDMA={min:120,rec:120}, DMA modes: sword0 sword1 sword2 mword0 mword1 mword2
 IORDY=on/off, tPIO={min:240,w/IORDY:120}, PIO modes: mode3 mode4
 UDMA modes: mode0 mode1 *mode2
 Drive Supports : ATA/ATAPI-4 T13 1153D revision 17 : ATA-1 ATA-2 ATA-3 ATA-4

fwiw,
Bill Pringlemeir.

Re: ReiserFS / 2.4.6 / Data Corruption

[Alan Cox]

> > and when that hangs the kernel it will also screw up all files touched
> > just before it in a edit-make-install-try cycle. Which can be rather
> > annoying, because you can start all over again (this effect randomly
> > distributes the last touched sectors to the last touched files. Very nice
> > effect, but not something I expect from a journalled filesystem).
> > 
> Do you think it is reasonable to ask that a filesystem be designed to
> work well with bad drivers?

Its certainly a good idea. But it sounds to me like he is describing the
normal effect of metadata only logging. 

Putting a sync just before the insmod when developing new drivers is a good
idea btw

Re: ReiserFS / 2.4.6 / Data Corruption

[bvermeul]

On Fri, 27 Jul 2001, Alan Cox wrote:

> > > and when that hangs the kernel it will also screw up all files touched
> > > just before it in a edit-make-install-try cycle. Which can be rather
> > > annoying, because you can start all over again (this effect randomly
> > > distributes the last touched sectors to the last touched files. Very nice
> > > effect, but not something I expect from a journalled filesystem).
> > >
> > Do you think it is reasonable to ask that a filesystem be designed to
> > work well with bad drivers?
>
> Its certainly a good idea. But it sounds to me like he is describing the
> normal effect of metadata only logging.
>
> Putting a sync just before the insmod when developing new drivers is a good
> idea btw

I've been doing that most of the time. But I sometimes forget that.
But as I said, it's not something I expected from a journalled filesystem.

Regards,

Bas Vermeulen

-- 
"God, root, what is difference?"
	-- Pitr, User Friendly

"God is more forgiving."
	-- Dave Aronson

Re: ReiserFS / 2.4.6 / Data Corruption

[Alan Cox]

> > Putting a sync just before the insmod when developing new drivers is a good
> > idea btw
> 
> I've been doing that most of the time. But I sometimes forget that.
> But as I said, it's not something I expected from a journalled filesystem.

You misunderstand journalling then

A journalling file system can offer different levels of guarantee. With 
metadata only journalling you don't take any real performance hit but your
file system is always consistent on reboot (consistent as in fsck would pass
it) but it makes no guarantee that data blocks got written.

Full data journalling will give you what you expect but at a performance hit
for many applications.

Alan

Re: ReiserFS / 2.4.6 / Data Corruption

[bvermeul]

On Fri, 27 Jul 2001, Alan Cox wrote:

> > > Putting a sync just before the insmod when developing new drivers is a good
> > > idea btw
> >
> > I've been doing that most of the time. But I sometimes forget that.
> > But as I said, it's not something I expected from a journalled filesystem.
>
> You misunderstand journalling then

Yup, I guess I did.

> A journalling file system can offer different levels of guarantee. With
> metadata only journalling you don't take any real performance hit but your
> file system is always consistent on reboot (consistent as in fsck would pass
> it) but it makes no guarantee that data blocks got written.

I allways thought that it could/would roll back the changes that weren't
consistent. But I stand corrected. Thanks... :)

> Full data journalling will give you what you expect but at a performance hit
> for many applications.

Do any of the other journalled filesystems for linux do this? If not, I
guess I'll go back to ext2.

Bas Vermeulen

-- 
"God, root, what is difference?"
	-- Pitr, User Friendly

"God is more forgiving."
	-- Dave Aronson

Re: ReiserFS / 2.4.6 / Data Corruption

[Alan Cox]

> > Full data journalling will give you what you expect but at a performance hit
> > for many applications.
> 
> Do any of the other journalled filesystems for linux do this? If not, I
> guess I'll go back to ext2.

ext3 can do full data journalling, I dont know if reiserfs has an option for
it or not

Alan

Re: ReiserFS / 2.4.6 / Data Corruption

[Matthew Gardiner]

On Saturday 28 July 2001 01:39, Alan Cox wrote:
> > > Putting a sync just before the insmod when developing new drivers is a
> > > good idea btw
> >
> > I've been doing that most of the time. But I sometimes forget that.
> > But as I said, it's not something I expected from a journalled
> > filesystem.
>
> You misunderstand journalling then
>
> A journalling file system can offer different levels of guarantee. With
> metadata only journalling you don't take any real performance hit but your
> file system is always consistent on reboot (consistent as in fsck would
> pass it) but it makes no guarantee that data blocks got written.
>
> Full data journalling will give you what you expect but at a performance
> hit for many applications.
>
> Alan

Just in regards to full journalling, will/is there an option in ReiserFS to 
allow it? Personally, I would much rather have full journalling, and a little 
more of a performance hit for security and reliability, than great 
performance and a higher level of risk.

Matthew Gardiner
-- 
WARNING:

This email was written on an OS using the viral 'GPL' as its license.

Please check with Bill Gates before continuing to read this email/posting.

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

Re: ReiserFS / 2.4.6 / Data Corruption

[Stephen C. Tweedie]

Hi,

On Fri, Jul 27, 2001 at 02:39:37PM +0100, Alan Cox wrote:

> > I've been doing that most of the time. But I sometimes forget that.
> > But as I said, it's not something I expected from a journalled filesystem.
> 
> You misunderstand journalling then
> 
> A journalling file system can offer different levels of guarantee. With 
> metadata only journalling you don't take any real performance hit but your
> file system is always consistent on reboot (consistent as in fsck would pass
> it) but it makes no guarantee that data blocks got written.

The default behaviour of ext3 does make this guarantee, for what it's
worth.  If you want the more relaxed mode which doesn't enforce the
flushing of data blocks before a commit, you need to mount with "-o
data=writeback".

> Full data journalling will give you what you expect but at a performance hit
> for many applications.

You can achieve the necessary ordering to avoid stale data blocks
after a crash without the penalty of writing all the data to the
journal.

Cheers,
 Stephen

Re: ReiserFS / 2.4.6 / Data Corruption

[Philip R. Auld]

Alan Cox wrote:
> 
> Its certainly a good idea. But it sounds to me like he is describing the
> normal effect of metadata only logging.
> 

Which brings up something I have been struggling with lately:

Linux (using both ext2 and reiserfs) can show garbage data blocks at the end of
files after a crash. With reiserfs this is clearly due to metadata only logging
and happens say 3 out of 5 times. With ext2 the frequency is about 1 in 5 times,
and more often that not it is simply zeroed data. Sometimes it is old data
though. 


This is something that is not present in other unix filesystems as far as I can
tell. If linux wants to be used in enterprise sites we can't allow 
old data blocks to be read. And ideally shouldn't allow zero blocks to be seen
either, but this is somewhat less serious.

I cannot reproduce this in ufs on either freebsd or solaris8.

I have not tested it with xfs and jfs for linux yet (and don't have any native
systems at hand.)

I believe vxfs to have a mechanism to prevent this despite metadata only
logging.

reiserfs with full data logging enabled of course does not show this behavior
(and works really well if you are willing to take the performance hit).

The basic test I use is to run this perl script for a while (to make sure at
least somehting has had a chance to get written out) and then power-cycle the
machine. When it comes back a simple tail logfile will show the problem. I also
run bonnie before hand to fill the disk with a known pattern so its easier to
spot.

linux is 2.2.16 and 2.4.2 from redhat 7.1. reiserfs is 3.5.33 and was tested
only on 2.2.16.


#!/usr/bin/perl
use Fcntl;
$count = 0;
while (1) {
#sysopen(FH, "/scratch/logfile", O_RDWR|O_APPEND|O_CREAT|O_SYNC)
sysopen(FH, "/scratch/logfile", O_RDWR|O_APPEND|O_CREAT)
        or die "Couldn't open file $path : $!\n";
print FH "Log file line ", $count , " yadda  yadda  yadda  yadda  yadda  yadda 
yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda 
yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda 
yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda 
yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda 
yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda 
yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda 
yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda 
yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda 
yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda \n" ;
close (FH);
#print $count , "\n";
$count++;
}


------------------------------------------------------
Philip R. Auld, Ph.D.                  Technical Staff 
Egenera Corp.                        pauld@egenera.com
165 Forest St, Marlboro, MA 01752        (508)786-9444

Re: ReiserFS / 2.4.6 / Data Corruption

[Jordan]

"Philip R. Auld" wrote:
> 
> Alan Cox wrote:
> >
> > Its certainly a good idea. But it sounds to me like he is describing the
> > normal effect of metadata only logging.
> >
> 
> Which brings up something I have been struggling with lately:
> 
> Linux (using both ext2 and reiserfs) can show garbage data blocks at the end of
> files after a crash. With reiserfs this is clearly due to metadata only logging
> and happens say 3 out of 5 times. With ext2 the frequency is about 1 in 5 times,
> and more often that not it is simply zeroed data. Sometimes it is old data
> though.
> 
> This is something that is not present in other unix filesystems as far as I can
> tell. If linux wants to be used in enterprise sites we can't allow
> old data blocks to be read. And ideally shouldn't allow zero blocks to be seen
> either, but this is somewhat less serious.
> 
> I cannot reproduce this in ufs on either freebsd or solaris8.
> 
> I have not tested it with xfs and jfs for linux yet (and don't have any native
> systems at hand.)
> 
> I believe vxfs to have a mechanism to prevent this despite metadata only
> logging.
> 
> reiserfs with full data logging enabled of course does not show this behavior
> (and works really well if you are willing to take the performance hit).
> 
> The basic test I use is to run this perl script for a while (to make sure at
> least somehting has had a chance to get written out) and then power-cycle the
> machine. When it comes back a simple tail logfile will show the problem. I also
> run bonnie before hand to fill the disk with a known pattern so its easier to
> spot.
> 
> linux is 2.2.16 and 2.4.2 from redhat 7.1. reiserfs is 3.5.33 and was tested
> only on 2.2.16.
> 
> #!/usr/bin/perl
> use Fcntl;
> $count = 0;
> while (1) {
> #sysopen(FH, "/scratch/logfile", O_RDWR|O_APPEND|O_CREAT|O_SYNC)
> sysopen(FH, "/scratch/logfile", O_RDWR|O_APPEND|O_CREAT)
>         or die "Couldn't open file $path : $!\n";
> print FH "Log file line ", $count , " yadda  yadda  yadda  yadda  yadda  yadda
> yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda
> yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda
> yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda
> yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda
> yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda
> yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda
> yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda
> yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda
> yadda  yadda  yadda  yadda  yadda  yadda  yadda  yadda \n" ;
> close (FH);
> #print $count , "\n";
> $count++;
> }
> 
> ------------------------------------------------------
> Philip R. Auld, Ph.D.                  Technical Staff
> Egenera Corp.                        pauld@egenera.com
> 165 Forest St, Marlboro, MA 01752        (508)786-9444
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

I didn't know that there was a way to enable full data journaling using
reiserfs.  I was under the impression that with the latest round of the
unlink patch to go with 2.4.7 that reiserfs was basically in ordered
journaling mode instead of writeback (I believe that is the name), if I
am wrong or if there really is a way to enable full data journaling
please let me know.  Thanks.

Jordan

Re: ReiserFS / 2.4.6 / Data Corruption

[Hans Reiser]

"Philip R. Auld" wrote:
 
> reiserfs with full data logging enabled of course does not show this behavior
> (and works really well if you are willing to take the performance hit).

Hmmm, I didn't realize this had made off our wish list and into the code.:)
We should benchmark the cost to performance.

Hans

Re: ReiserFS / 2.4.6 / Data Corruption

[Philip R. Auld]

Hans Reiser wrote:
> 
> "Philip R. Auld" wrote:
> 
> > reiserfs with full data logging enabled of course does not show this behavior
> > (and works really well if you are willing to take the performance hit).
> 
> Hmmm, I didn't realize this had made off our wish list and into the code.:)
> We should benchmark the cost to performance.
> 
> Hans

Ooops, hope I'm not getting Chris in trouble ;)

This is reiserfs 3.5.33, with a few changes from Chris to enable full logging, 
and from me to make it a mount option. 

We are in a situation where we need the safety more than the speed so it was
necessary.


Here is a simple comparison using bonnie:

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
pblade 1 (reiserfs defaults)
         1000 13048 98.9 21609 27.4  6599 10.7 11066 72.3 16483  8.4 1011.2  5.3 
	 1000 12771 96.7 21058 25.9  5536  9.0 10430 67.5 17347  8.4 1065.2  6.7 
	 1000 13034 98.6 19746 21.6  7026 11.6  9884 64.4 14838  7.2 1106.0  9.7
         1000 13091 99.3 19483 28.9  7586 12.3 10520 68.4 14685  6.9  900.9  6.3
pblade 2 (ext2 defaults)
         1000 14373 99.9 14940  8.8  7494 11.1 10093 65.3 22213  9.3 1028.3 
6.4      
	 1000 14305 99.6 16129  9.4  7768 11.9  9629 62.2 26108 10.8 1135.8  7.7    
	 1000 14400 99.9 16769  9.8  7397 11.2  9805 63.4 21820  9.1 1139.8  5.7
	 1000 14361 100. 17089 10.4  7768 11.5  9924 64.1 24154  9.8 1112.9  7.2
pblade 3 (log all data)
	 1000  5932 47.6  7244 12.5  4708  9.7 13909 90.5 17051  8.1 894.5  6.5
	 1000  5839 46.9  7229 12.5  4604  9.9 13437 87.9 19852  9.7 724.3  4.7
	 1000  5853 47.0  7176 12.3  4611  9.8 13995 91.1 18838  8.7 908.0  5.7
	 1000  5604 45.1  7106 12.2  4627  9.5 13628 88.6 15248  6.9 882.9  6.6
pblade 6 ( log new data )
	 1000  5556 49.0  7057 11.9  7714 12.6 11559 92.8 18075  8.8 1264.3  7.3    
	 1000  5631 49.8  7307 12.3  7945 13.0 11558 93.0 18859  9.0 1230.7  8.0
	 1000  5610 49.6  7337 12.5  6620 11.0 11821 95.0 16484  7.5 1236.8  9.3
	 1000  5592 49.4  7070 12.1  7422 12.0 11575 92.9 16198  7.3 1236.6  4.9


I suugest we move this to reiserfs-list for more discussion if needed :)


Cheers,

Phil


------------------------------------------------------
Philip R. Auld, Ph.D.                  Technical Staff 
Egenera Corp.                        pauld@egenera.com
165 Forest St, Marlboro, MA 01752        (508)786-9444

Re: ReiserFS / 2.4.6 / Data Corruption

[Hans Reiser]

Alan Cox wrote:
> 
> > > and when that hangs the kernel it will also screw up all files touched
> > > just before it in a edit-make-install-try cycle. Which can be rather
> > > annoying, because you can start all over again (this effect randomly
> > > distributes the last touched sectors to the last touched files. Very nice
> > > effect, but not something I expect from a journalled filesystem).
> > >
> > Do you think it is reasonable to ask that a filesystem be designed to
> > work well with bad drivers?
> 
> Its certainly a good idea. 
I think it is a terrible idea.... at least as a general expectation to meet, there may be specifics
where things can be done though.... like journaling....

> But it sounds to me like he is describing the
> normal effect of metadata only logging.

Ah, right you are.  Now I understand him.  Well, data-journaling that doesn't cost a whole lot of
performance awaits reiser4, and reiser4 is at least a year away, we are doing seminars and
pseudo-coding now.

> 
> Putting a sync just before the insmod when developing new drivers is a good
> idea btw

This makes a lot of sense to me.  Good suggestion.  It should go into our FAQ.  Dad, please put it
there.

Q: I like to dynamically load buggy drivers into the kernel because that is what kernel developers
like me do for fun, how can I better avoid data corruption when doing this and using ReiserFS?

A: Do sync before insmod.  (Alan Cox's good suggestion.)

Re: ReiserFS / 2.4.6 / Data Corruption

[Alan Cox]

> This is something that is not present in other unix filesystems as far as I can
> tell. If linux wants to be used in enterprise sites we can't allow 
> old data blocks to be read. And ideally shouldn't allow zero blocks to be seen
> either, but this is somewhat less serious.

> I cannot reproduce this in ufs on either freebsd or solaris8.

It can happen on UFS. What normally happens on UFS is that you get an old
file attached to a new filename when the file is deleted and the inode
reused.

Basically it can happen on any no data logging fs (with a few exceptions for
other clever algorithms like tree-phase)

If you write the metadata block first (UFS) then there is a risk of getting
someone elses data appended to the end of a file (eg length updated before
data blocks). If you write data first there is a risk of writing the data
and never committing the removal of the block from previous files.

FreeBSD softupdates probably make it very hard to trigger and they are a
very nice approach

Re: ReiserFS / 2.4.6 / Data Corruption

[Matthew Gardiner]

I've noticed that in the menuconfig there is support for the Vertias 
Journalling File System. Has there been any push for that to be a "bootable" 
filesystem so it can be used for Linux?

Matthew Gardiner
-- 
WARNING:

This email was written on an OS using the viral 'GPL' as its license.

Please check with Bill Gates before continuing to read this email/posting.

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

Re: ReiserFS / 2.4.6 / Data Corruption

[Alan Cox]

> I've noticed that in the menuconfig there is support for the Vertias 
> Journalling File System. Has there been any push for that to be a "bootable" 
> filesystem so it can be used for Linux?

The Linux freevxfs module is read only currently. Veritas apparently will be
releasing the genuine article for Linux but binary only with all the mess
that entails

binary modules (was Re: ReiserFS / 2.4.6 / Data Corruption)

[Jeff Garzik]

Alan Cox wrote:
> The Linux freevxfs module is read only currently. Veritas apparently will be
> releasing the genuine article for Linux but binary only with all the mess
> that entails

Isn't that a violation of the GPL, to release binary modules?

-- 
Jeff Garzik      | "Mind if I drive?" -Sam
Building 1024    | "Not if you don't mind me clawing at the dash
MandrakeSoft     |  and shrieking like a cheerleader." -Max

Re: binary modules (was Re: ReiserFS / 2.4.6 / Data Corruption)

[Andreas Dilger]

Jeff Garzik writes:
> Alan Cox wrote:
> > The Linux freevxfs module is read only currently. Veritas apparently will be
> > releasing the genuine article for Linux but binary only with all the mess
> > that entails
> 
> Isn't that a violation of the GPL, to release binary modules?

Noooooo....  Not this thread again.

Cheers, Andreas
-- 
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/

Re: binary modules (was Re: ReiserFS / 2.4.6 / Data Corruption)

[Rik van Riel]

On Sat, 28 Jul 2001, Jeff Garzik wrote:
> Alan Cox wrote:
> > The Linux freevxfs module is read only currently. Veritas apparently will be
> > releasing the genuine article for Linux but binary only with all the mess
> > that entails
>
> Isn't that a violation of the GPL, to release binary modules?

Binary modules using only the interfaces exported in /proc/ksyms
are, under certain readings of the GPL, no less "infected" by the
GPL than binary programs making system calls.

This means binary only modules are ok, as long as they don't need
changes in the kernel to work.

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: binary modules (was Re: ReiserFS / 2.4.6 / Data Corruption)

[Richard Gooch]

Jeff Garzik writes:
> Alan Cox wrote:
> > The Linux freevxfs module is read only currently. Veritas apparently will be
> > releasing the genuine article for Linux but binary only with all the mess
> > that entails
> 
> Isn't that a violation of the GPL, to release binary modules?

Linus said it's OK. I know Alan doesn't agree, but that's life :-)
The king penguin has spoken.

I don't see the need to be bloody-minded on this issue. If a vendor
wants to go through the pain of tracking kernel drift and having to
compile modules for many different versions, then let them. Given how
much trouble it is, why bother them with legal threats?

The right answer for vendors who want to ship binary modules is to
ship an Open Source interface layer which shields the vendor from
kernel drift (since users will be able to build the interface layer if
they need to, without waiting for the vendor).
I guess that would also shield them from unhelpful legal threats.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: ReiserFS / 2.4.6 / Data Corruption

[Matthew Gardiner]

On Sunday 29 July 2001 04:25, Alan Cox wrote:
> > I've noticed that in the menuconfig there is support for the Vertias
> > Journalling File System. Has there been any push for that to be a
> > "bootable" filesystem so it can be used for Linux?
>
> The Linux freevxfs module is read only currently. Veritas apparently will
> be releasing the genuine article for Linux but binary only with all the
> mess that entails

tsk tsk tsk. A bit disappointing that Vertias has taken that approach. 
However, even still, reiserFS is pretty awsome. Extremely fast and space 
efficient, esp on a 60gig drive ;)

Matthew Gardiner
-- 
WARNING:

This email was written on an OS using the viral 'GPL' as its license.

Please check with Bill Gates before continuing to read this email/posting.

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

Re: ReiserFS / 2.4.6 / Data Corruption

[Chris Wedgwood]

On Sun, Jul 29, 2001 at 10:15:03PM +1200, Matthew Gardiner wrote:

    tsk tsk tsk. A bit disappointing that Vertias has taken that approach. 
    However, even still, reiserFS is pretty awsome. Extremely fast and space 
    efficient, esp on a 60gig drive ;)

Why "tsk tsk tsk" ?  If reiserfs suits you, use it --- you need never
go near VXFS.

Personally, even though I use reiserfs, I am of the opinion that XFS,
and VXFS and both superior, especially when you include volume
management.  Time will show whether or not these very well designed
file-systems are suitable under Linux though, as reiserfs has a
considerable head start.



  --cw

Re: ReiserFS / 2.4.6 / Data Corruption

[Luigi Genoni]

On Sun, 29 Jul 2001, Chris Wedgwood wrote:

> On Sun, Jul 29, 2001 at 10:15:03PM +1200, Matthew Gardiner wrote:
>
>     tsk tsk tsk. A bit disappointing that Vertias has taken that approach.
>     However, even still, reiserFS is pretty awsome. Extremely fast and space
>     efficient, esp on a 60gig drive ;)
>
> Why "tsk tsk tsk" ?  If reiserfs suits you, use it --- you need never
> go near VXFS.
It depends, for example if you have to manage a farm (let's say 800
systems) with many Unixes
around, where solaris is the 70% of your installed basis, then
veritas (mainly the VM) could be a solution to keep an uniform
environment. That is a good thing if your sysadmin staff is composed also
by people without a real high skill.
>
> Personally, even though I use reiserfs, I am of the opinion that XFS,
> and VXFS and both superior, especially when you include volume
> management.
a journaling filesystem and a volume manager are two complementary
and usefull things, but anyway are  different things.
While i do agree that Linux LVM is still not complitelly usable in a
production environment, (but anyway ELVM from IBM is somehow immmature),
and some details of its design are not completely, how can I say...,
suitable for future HW developments, I found reiserFS tecnology to be
really interesting. On a technological point of view reiserFS is much
more advanced in front of any other journaled FS around.

I still have to see vxfs with Linux, but i saw it under solaris and HP-UX
(i think I used all journaled aroung, jfs, xfs, reiserFS, ext3, vxfs, gfs,
on all unixes i could), seeing it to too much slow on high end scsi HW,
and XFS on my origin 2000 (8 processor) sometimes takes one CPU just to
manage journaling under heavy I/O. Under Linux xfs is maybe better that
under Irix (!!!???), but its tecnology was thinked for other kind of HW,
and an experienced sysadmin can "feel" this.
> Time will show whether or not these very well designed
> file-systems are suitable under Linux though, as reiserfs has a
> considerable head start.
Yes, time will show. reiserFS can have a wonderfull future, better than
ext3 if it will be mature before ext3, worse if after. But for Linux
jfs and xfs are interesting right now, just untill native journaled will
be ready, then i would bet everyone will stay with reiserFS or ext3, not
considering any other solution.

Luigi

missing symbols in 2.4.7-ac2

[Thomas Kotzian]

when compiling with highmem = 4GB
problem in 3c59x - module:
unresolved symbol nr_free_highpages ...

ThomasK.

Re: missing symbols in 2.4.7-ac2

[Andrew Morton]

Thomas Kotzian wrote:
> 
> when compiling with highmem = 4GB
> problem in 3c59x - module:
> unresolved symbol nr_free_highpages ...
> 

Ah.  Sorry.

Alan, is it OK to export this symbol?


--- linux-2.4.7-ac1/kernel/ksyms.c	Sun Jul 29 11:43:01 2001
+++ ac/kernel/ksyms.c	Sun Jul 29 11:43:05 2001
@@ -122,6 +122,7 @@ EXPORT_SYMBOL(kmap_high);
 EXPORT_SYMBOL(kunmap_high);
 EXPORT_SYMBOL(highmem_start_page);
 EXPORT_SYMBOL(create_bounce);
+EXPORT_SYMBOL(nr_free_highpages);
 #endif
 
 /* filesystem internal functions */

Re: missing symbols in 2.4.7-ac2

[Hugh Dickins]

On Sun, 29 Jul 2001, Andrew Morton wrote:
> Thomas Kotzian wrote:
> > when compiling with highmem = 4GB
> > problem in 3c59x - module:
> > unresolved symbol nr_free_highpages ...
> 
> Ah.  Sorry.
> Alan, is it OK to export this symbol?

Laconic version: "Probably not: si_meminfo() is your friend".

Verbose version:
I don't think you really want nr_free_highpages(), that's transient
info - it won't usually fall so low as 0 if there is highmem, but do
you want to rely on that?  And nr_free_highpages() is CONFIG_HIGHMEM
only, so you'd need #ifdef CONFIG_HIGHMEM around its call in 3c59x.c.

But si_meminfo() is already exported: I think sysinfo.totalhigh is
what you want to check; if I'm wrong, and you really are interested
in whether there are currently free highpages, sysinfo.freehigh
gives you that too without needing a new export.

(I think there probably will be a need for new interfaces
to export more per-zone memory info, but not for this.)

Hugh

--- linux-2.4.7-ac2/drivers/net/3c59x.c	Sat Jul 28 07:12:03 2001
+++ linux/drivers/net/3c59x.c	Sun Jul 29 10:53:31 2001
@@ -1299,8 +1299,11 @@
 	/* The 3c59x-specific entries in the device structure. */
 	dev->open = vortex_open;
 	if (vp->full_bus_master_tx) {
+		struct sysinfo sysinfo;
+
 		dev->hard_start_xmit = boomerang_start_xmit;
-		if (nr_free_highpages() == 0) {
+		si_meminfo(&sysinfo);
+		if (sysinfo.totalhigh == 0) {
 			/* Actually, it still should work with iommu. */
 			dev->features |= NETIF_F_SG;
 		}

Re: missing symbols in 2.4.7-ac2

[Andrew Morton]

Hugh Dickins wrote:
> 
> On Sun, 29 Jul 2001, Andrew Morton wrote:
> > Thomas Kotzian wrote:
> > > when compiling with highmem = 4GB
> > > problem in 3c59x - module:
> > > unresolved symbol nr_free_highpages ...
> >
> > Ah.  Sorry.
> > Alan, is it OK to export this symbol?
> 
> Laconic version: "Probably not: si_meminfo() is your friend".

:)

> Verbose version:
> I don't think you really want nr_free_highpages(), that's transient
> info - it won't usually fall so low as 0 if there is highmem, but do
> you want to rely on that?

Prefer not to.  We want to know "does the system have any highmem
pages".  I didn't know about sysinfo.totalhigh, so I used
nr_free_highpages(), which answers the question "does the system
have any free high pages right now".

It's good enough - if we get it wrong (system was very low on memory
when the driver was initialised) the driver will work - it just won't
perform zerocopy optimisations.

>  And nr_free_highpages() is CONFIG_HIGHMEM
> only, so you'd need #ifdef CONFIG_HIGHMEM around its call in 3c59x.c.

That's OK actually - nr_free_highpages() evaluates to constant zero if
CONFIG_HIGHMEM isn't defined.


--- linux-2.4.7-ac2/drivers/net/3c59x.c Sat Jul 28 07:12:03 2001
+++ linux/drivers/net/3c59x.c   Sun Jul 29 10:53:31 2001
@@ -1299,8 +1299,11 @@
        /* The 3c59x-specific entries in the device structure. */
        dev->open = vortex_open;
        if (vp->full_bus_master_tx) {
+               struct sysinfo sysinfo;
+
                dev->hard_start_xmit = boomerang_start_xmit;
-               if (nr_free_highpages() == 0) {
+               si_meminfo(&sysinfo);
+               if (sysinfo.totalhigh == 0) {
                        /* Actually, it still should work with iommu. */
                        dev->features |= NETIF_F_SG;
                }

Much preferable!  Thanks.

I've checked all the architectures.  Looks fine, works OK.  Alan, please
apply this one.

-

Re: ReiserFS / 2.4.6 / Data Corruption

[Christoph Hellwig]

In article <01072902183404.02683@kiwiunixman.nodomain.nowhere> you wrote:
> I've noticed that in the menuconfig there is support for the Vertias 
> Journalling File System. Has there been any push for that to be a "bootable" 
> filesystem so it can be used for Linux?

I don't see any reason wht it shoudn't be bootable, I just haven't tested it
yet.  If you want to try it, please follow the below steps:

1) Get one of these CD-ROM readonly distribution
2) Copy it over NFS to a UnixWare (or any other x86 System with VxFS)
3) Make a VxFS system big enough for the distribution
4) Copy the Distribution on the VxFS filesystem

And now the difficult part:

5) Adjust the ondisk dev_t to match Linux's major/minor split instead
   of SVR4's.  This can either be done by creating (bogus) SVR4 device
   nodes that are valid Linux ones when read by Linux or by doing this
   with fsdb after they were created.

If you have success with this sppropeach please drop me a mail - I'll add
it to the freevxfs docs then.

	Christoph

-- 
Whip me.  Beat me.  Make me maintain AIX.

Re: ReiserFS / 2.4.6 / Data Corruption

[Alan Cox]

> Don't use RedHat with ReiserFS, they screw things up so many ways.....
> For instance, they compile it with the wrong options set, their boot scripts are wrong, they just
> shovel software onto the CD.

Sorry Hans you can rant all you like but you know you are wrong on most
of that. RH did weeks of stress testing on multiple systems up to 8Gb 8 way
and didn't ship until we stopped seeing corruption problems with the mm/fs
code. 

That test suite caught bugs in kernel revisions other vendors shipped
blindly to their customers without fixing.

That is hardly shovelling software onto the CD.

> Actually, I am curious as to exactly how they manage to make ReiserFS boot longer than ext2.  Do
> they run fsck or what?

No. The only thing I can think of that might slow it is that we build with
the reiserfs paranoia/sanity checks on. Thats because at the time 7.1 was
done the kernel list was awash with reiserfs bug reports and Chris Mason
tail recursion bug patch of the week.

That might be something to check to get a fair comparison

Alan

Re: ReiserFS / 2.4.6 / Data Corruption

[Joshua Schmidlkofer]

On Friday 27 July 2001 09:06 am, Alan Cox wrote:
> > Don't use RedHat with ReiserFS, they screw things up so many ways.....
> > For instance, they compile it with the wrong options set, their boot
> > scripts are wrong, they just shovel software onto the CD.
>
> Sorry Hans you can rant all you like but you know you are wrong on most
> of that. RH did weeks of stress testing on multiple systems up to 8Gb 8 way
> and didn't ship until we stopped seeing corruption problems with the mm/fs
> code.
>
> That test suite caught bugs in kernel revisions other vendors shipped
> blindly to their customers without fixing.
>
> That is hardly shovelling software onto the CD.
>
> > Actually, I am curious as to exactly how they manage to make ReiserFS
> > boot longer than ext2.  Do they run fsck or what?
>
> No. The only thing I can think of that might slow it is that we build with
> the reiserfs paranoia/sanity checks on. Thats because at the time 7.1 was
> done the kernel list was awash with reiserfs bug reports and Chris Mason
> tail recursion bug patch of the week.
>
> That might be something to check to get a fair comparison

   I feel that things are actually progressing above my level of perception 
here, however, I would like to mention that since my Redhat 4.x days i have 
feared vendor kernels, and I never use them, for better or worse.   

    Also, maybe I screwed my own system - I don't think so, but maybe.  I 
prefer to stick with Linus's kernels, and sometimes, depending on the 
changlog -ac kernels.  As far as the kernel & init scirpts are concerned, I 
axed any fsck'ing entries for reiserfs.   [I assume that they were 
unnessecary.]  I used kgcc [w/Rh7.1] to compile kernels, until recently.  And 
I stayed current with the lkml, and the namesys page watching for obvious 
updates that I needed. 

    The slowness [seemed] actually [to be] the process of starting & stopping 
daemons.  Almost like there was some sort of stigma about reading shell 
scripts.  All the binaries executed with appropriate haste.

   As far as shoveling code.   Sometimes the options used to compile packages 
leaves me with a large bit of wonder.  Strange and seemingly heinous changes 
to the various utilities, etc.   But, I have never had a cause to fault them 
based on this. [Except that I have never found the magic that causes all the 
SRPMS to be [re]buildable.]

  So to sort it, I don't feel that being a moron caused to boot slow - unless 
there is some wierd filehandling problem in bash2, or something that causes 
severe slow-down when sourcing shell scripts.  ????   However, Hans, I do 
beleive you about Suse, and if I wasn't a cheap bastard I would probably buy 
a copy.  

thanks for all the response, and I am sorry if this does not belong here.

Re: ReiserFS / 2.4.6 / Data Corruption

[Hans Reiser]

Well, I am afraid this is much too vague for me to have any understanding of what went wrong on your
system.

Maybe somebody else who is using both ReiserFS and RedHat's boot scripts can comment on whether
things are slow for them and if so, where they get slow.

With this lack of specificity is entirely possible that things went slow for coincidental reasons
unrelated to ReiserFS (waiting for network stuff to timeout, etc.)

Hans

Joshua Schmidlkofer wrote:
> 
> On Friday 27 July 2001 09:06 am, Alan Cox wrote:
> > > Don't use RedHat with ReiserFS, they screw things up so many ways.....
> > > For instance, they compile it with the wrong options set, their boot
> > > scripts are wrong, they just shovel software onto the CD.
> >
> > Sorry Hans you can rant all you like but you know you are wrong on most
> > of that. RH did weeks of stress testing on multiple systems up to 8Gb 8 way
> > and didn't ship until we stopped seeing corruption problems with the mm/fs
> > code.
> >
> > That test suite caught bugs in kernel revisions other vendors shipped
> > blindly to their customers without fixing.
> >
> > That is hardly shovelling software onto the CD.
> >
> > > Actually, I am curious as to exactly how they manage to make ReiserFS
> > > boot longer than ext2.  Do they run fsck or what?
> >
> > No. The only thing I can think of that might slow it is that we build with
> > the reiserfs paranoia/sanity checks on. Thats because at the time 7.1 was
> > done the kernel list was awash with reiserfs bug reports and Chris Mason
> > tail recursion bug patch of the week.
> >
> > That might be something to check to get a fair comparison
> 
>    I feel that things are actually progressing above my level of perception
> here, however, I would like to mention that since my Redhat 4.x days i have
> feared vendor kernels, and I never use them, for better or worse.
> 
>     Also, maybe I screwed my own system - I don't think so, but maybe.  I
> prefer to stick with Linus's kernels, and sometimes, depending on the
> changlog -ac kernels.  As far as the kernel & init scirpts are concerned, I
> axed any fsck'ing entries for reiserfs.   [I assume that they were
> unnessecary.]  I used kgcc [w/Rh7.1] to compile kernels, until recently.  And
> I stayed current with the lkml, and the namesys page watching for obvious
> updates that I needed.
> 
>     The slowness [seemed] actually [to be] the process of starting & stopping
> daemons.  Almost like there was some sort of stigma about reading shell
> scripts.  All the binaries executed with appropriate haste.
> 
>    As far as shoveling code.   Sometimes the options used to compile packages
> leaves me with a large bit of wonder.  Strange and seemingly heinous changes
> to the various utilities, etc.   But, I have never had a cause to fault them
> based on this. [Except that I have never found the magic that causes all the
> SRPMS to be [re]buildable.]
> 
>   So to sort it, I don't feel that being a moron caused to boot slow - unless
> there is some wierd filehandling problem in bash2, or something that causes
> severe slow-down when sourcing shell scripts.  ????   However, Hans, I do
> beleive you about Suse, and if I wasn't a cheap bastard I would probably buy
> a copy.
> 
> thanks for all the response, and I am sorry if this does not belong here.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: ReiserFS / 2.4.6 / Data Corruption

[Christoph Rohland]

Hi Hans,

On Fri, 27 Jul 2001, Hans Reiser wrote:
> Maybe somebody else who is using both ReiserFS and RedHat's boot
> scripts can comment on whether things are slow for them and if so,
> where they get slow.

At least not if it's not the root disk. I have a RH71 box with a 19GB
reiserfs partition and it's booting fast and fine.

Greetings
		Christoph

Re: ReiserFS / 2.4.6 / Data Corruption

[Hans Reiser]

Christoph Rohland wrote:
> 
> Hi Hans,
> 
> On Fri, 27 Jul 2001, Hans Reiser wrote:
> > Maybe somebody else who is using both ReiserFS and RedHat's boot
> > scripts can comment on whether things are slow for them and if so,
> > where they get slow.
> 
> At least not if it's not the root disk. I have a RH71 box with a 19GB
> reiserfs partition and it's booting fast and fine.
> 
> Greetings
>                 Christoph
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Ok, well then I conclude that it was a user misdiagnosis as to what his booting problem was of some
unknowable form.

Apologies to RedHat.

Hans

Re: ReiserFS / 2.4.6 / Data Corruption

[Dustin Byford]

Hans Reiser wrote:

> Maybe somebody else who is using both ReiserFS and RedHat's boot scripts can comment on whether
> things are slow for them and if so, where they get slow.


For what it's worth I just configured a RedHat 7.1 box with reiserfs on 
all partitions except /boot using this update disk 
ftp://139.82.28.40/pub/update-rh71reiser-v1.img from 
http://cambuca.ldhs.cetuc.puc-rio.br/.

Upgraded all of redhat's packages, note there is a SysVinit update and a 
gcc update.

Compiled a 2.4.7-pre kernel and the latest reiserfsprogs.

Mounted /boot read only to eliminate the chance of an fsck required on 
that partition.

I have been running reiserfs on a mail server with about 60k accounts 
(30k really active) for about 6 months. I haven't experienced any 
problems with the filesystems. The one I just configured is its intended 
replacment. After a few days of testing with bonnie, some perl scripts I 
wrote, and a few pullings of the power cord I think it's almost ready 
for production. An upgrade to 2.4.7 and some more testing will tell.

If you pull the plug on this machine it takes around 40 seconds to get 
back to the login prompt, (p3-600 60G ide drive). Including the act of 
pulling the power cord, bios delays, lilo delays, and the rest of the 
redhat boot sequence.

So, in my experience I've had very few problems with reiserfs and 
redhat. That said, the slightest hint of data corruption under normal 
(continuous power, no failing hardware) operation and I'll probably be 
evaluating other filesystems. There are sometimes as many as 500,000 
files on this filesystem, reiserfs seems to do a good job under my 
conditions.

				--Dustin

Also, one purely cosmetic patch to rc.sysinit if you want:
--- rc.sysinit.orig Fri Jul 27 13:06:58 2001
+++ rc.sysinit Fri Jul 27 13:38:25 2001
@@ -211,7 +211,8 @@

_RUN_QUOTACHECK=0
ROOTFSTYPE=`grep " / " /proc/mounts | awk '{ print $3 }'`
-if [ -z "$fastboot" -a "$ROOTFSTYPE" != "nfs" ]; then
+if [ -z "$fastboot" -a "$ROOTFSTYPE" != "nfs" \
+ -a "$ROOTFSTYPE" != "reiserfs" ]; then

STRING=$"Checking root filesystem"
echo $STRING

Re: ReiserFS / 2.4.6 / Data Corruption

[Hans Reiser]

Dustin Byford wrote:

> Also, one purely cosmetic patch to rc.sysinit if you want:
> --- rc.sysinit.orig Fri Jul 27 13:06:58 2001
> +++ rc.sysinit Fri Jul 27 13:38:25 2001
> @@ -211,7 +211,8 @@
> 
> _RUN_QUOTACHECK=0
> ROOTFSTYPE=`grep " / " /proc/mounts | awk '{ print $3 }'`
> -if [ -z "$fastboot" -a "$ROOTFSTYPE" != "nfs" ]; then
> +if [ -z "$fastboot" -a "$ROOTFSTYPE" != "nfs" \
> + -a "$ROOTFSTYPE" != "reiserfs" ]; then
> 
> STRING=$"Checking root filesystem"
> echo $STRING


Yes, this patch is much needed.  Edward, put it on our website in an appropriate location.

Hans

Re: ReiserFS / 2.4.6 / Data Corruption

[Henning P. Schmiedehausen]

Hans Reiser <reiser@namesys.com> writes:

> Well, I am afraid this is much too vague for me to have any
> understanding of what went wrong on your system.

But you were able on this vagueness of accusing Redhat to "just shovel
software on a CD". Why? Because they didn't give you money unlike some
other vendors, e.g. SuSE?

The thing that really pisses me off about ReiserFS from time to time
is not the "FS" part...

	Regards
		Henning

-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20   

Re: ReiserFS / 2.4.6 / Data Corruption

[Hans Reiser]

Alan Cox wrote:
> 
> > Don't use RedHat with ReiserFS, they screw things up so many ways.....
> > For instance, they compile it with the wrong options set, their boot scripts are wrong, they just
> > shovel software onto the CD.
> 
> Sorry Hans you can rant all you like but you know you are wrong on most
> of that. RH did weeks of stress testing on multiple systems up to 8Gb 8 way
> and didn't ship until we stopped seeing corruption problems with the mm/fs
> code.
> 
> That test suite caught bugs in kernel revisions other vendors shipped
> blindly to their customers without fixing.
> 
> That is hardly shovelling software onto the CD.
> 
> > Actually, I am curious as to exactly how they manage to make ReiserFS boot longer than ext2.  Do
> > they run fsck or what?
> 
> No. The only thing I can think of that might slow it is that we build with
> the reiserfs paranoia/sanity checks on. Thats because at the time 7.1 was

Yes, that option should never be on for an end user not having a bug that he wants a more detailed
bug report on.  It just makes us look slow compared to ext2.

2.4.2 was not a stable kernel for any FS, not just for ReiserFS.

2.4.4 was the earliest kernel that should have been called 2.4.0, and sad to say, I bet we won't hit
a really stable kernel for another couple of versions.

I understand the marketing pressure on distributions to ship using 2.4.x as soon as 2.4.0 was
available, and that pressure should never have been generated upon them by making an unstable kernel
be named 2.4.0.

It won't surpise me if you agree with me on the kernel naming though, and if so it is pointless for
me to complain to you about it. 

> done the kernel list was awash with reiserfs bug reports and Chris Mason
> tail recursion bug patch of the week.
> 
> That might be something to check to get a fair comparison
> 
> Alan

I don't think that even with CONFIG_REISERFS_CHECK on, journal replay can take as long as fsck on
ext2.  reiserfsck though, if that was on, oh, could even RedHat be that desperate to make us look
bad to users as to run reiserfsck at every boot?

I surely hope not, and I'd like to hear that this user just had something individually wrong with
his configuration.

Hans

Re: ReiserFS / 2.4.6 / Data Corruption

[Kip Macy]

> Alan Cox wrote:
> > 
> > > Don't use RedHat with ReiserFS, they screw things up so many ways.....
> > > For instance, they compile it with the wrong options set, their boot scripts are wrong, they just
> > > shovel software onto the CD.
> > 
> > Sorry Hans you can rant all you like but you know you are wrong on most
> > of that. RH did weeks of stress testing on multiple systems up to 8Gb 8 way
> > and didn't ship until we stopped seeing corruption problems with the mm/fs
> > code.

Sorry Alan, but even though I am sure Redhat did lots of stress testing,
Redhat 7.1 was not a particularly solid release. I got oopses in the
eepro100 driver even though lots of other people use it, and the netapp
simulator which works just fine on 2.2.16 does not work on it. When I ran
strace on the simulator while it was zeroing some files it turned out that
sys_write was failing with ENOMEM (on a machine with 1GB of RAM that was 
not doing anything else).

Re: ReiserFS / 2.4.6 / Data Corruption

[Ville Herva]

On Fri, Jul 27, 2001 at 09:25:23AM -0700, you [Kip Macy] claimed:
>  
> sys_write was failing with ENOMEM (on a machine with 1GB of RAM that was 
> not doing anything else).

I second that.

256M memory, no swap at the time.

After fresh boot to the default RH71 kernel (2.4.2-2 or whatever it is) on
console (no X running):

> diff -Naur /usr/src/linux.rh-default /usr/src/linux-2.4.4 > diff
zsh: killed diff

> dmesg | tail
kernel: out of memory, killed process n (xfs)
kernel: out of memory, killed process n (diff)

Phew.


-- v --

v@iki.fi

Re: ReiserFS / 2.4.6 / Data Corruption

[Alan Cox]

> After fresh boot to the default RH71 kernel (2.4.2-2 or whatever it is) on
> console (no X running):
> 
> > diff -Naur /usr/src/linux.rh-default /usr/src/linux-2.4.4 > diff
> zsh: killed diff
> 
> > dmesg | tail
> kernel: out of memory, killed process n (xfs)
> kernel: out of memory, killed process n (diff)
> 
> Phew.

No argument on that one. I'm still seeing it in vanilla 2.4.6 as well but
2.4.7 is looking a lot better. 

Re: ReiserFS / 2.4.6 / Data Corruption

[Ville Herva]

On Fri, Jul 27, 2001 at 06:40:32PM +0100, you [Alan Cox] claimed:
> > After fresh boot to the default RH71 kernel (2.4.2-2 or whatever it is) on
> > console (no X running):
> > 
> > > diff -Naur /usr/src/linux.rh-default /usr/src/linux-2.4.4 > diff
> > zsh: killed diff
> > 
> > > dmesg | tail
> > kernel: out of memory, killed process n (xfs)
> > kernel: out of memory, killed process n (diff)
> > 
> > Phew.
> 
> No argument on that one. I'm still seeing it in vanilla 2.4.6 as well but
> 2.4.7 is looking a lot better. 

I wasn't able to easily reproduce that on 2.4.4ac5 (that I upgraded into
almost immediately). It may be that the OOM rambo wasn't fully sane on that
one either, but at least it seemed to handle the silly "someone filled the
cache - gee, we must be oom" case rather better...


-- v --

v@iki.fi

Re: ReiserFS / 2.4.6 / Data Corruption

[Lehmann]

On Fri, Jul 27, 2001 at 04:06:16PM +0100, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> > Don't use RedHat with ReiserFS, they screw things up so many ways.....
> > For instance, they compile it with the wrong options set, their boot scripts are wrong, they just
> > shovel software onto the CD.
> 
> Sorry Hans you can rant all you like but you know you are wrong on most
> of that. RH did weeks of stress testing on multiple systems up to 8Gb 8 way
> and didn't ship until we stopped seeing corruption problems with the mm/fs
> code. 

You might be well advised looking at reality (visit a few other projects)
and you'll see that redhat, indeed, has a very bad reputation. Wether it's
gimp, gtk, perl, wine, dosemu or any other project, the basic reaction is:
oh, you have gt problems under redhat? you compile it yourself and most
probably your problems will go away (gtk+ even had this message in their
install script).

> That test suite caught bugs in kernel revisions other vendors shipped
> blindly to their customers without fixing.

they might have a very good testsuite, but that means nothing: redhat
so frequently takes snapshots of undebugged alpha versions of software
(higher version numbers) that no matter of testing will suffice to ever
make this work.

the might be doing well for the kernel, but that only gets you so far.

> That is hardly shovelling software onto the CD.

Right, that's shovelling the latest alpha versions of software onto CD.

> > Actually, I am curious as to exactly how they manage to make ReiserFS boot longer than ext2.  Do
> > they run fsck or what?
> No. The only thing I can think of that might slow it is that we build with
> the reiserfs paranoia/sanity checks on.

That's a pretty dumb thing. Maybe one should have asked the develoers
before doing this (they never do). Redhat somehow manages pretty well to
show reiserfs in a bad light ;)

However, ext2 is much faster on mount time with -onocheck (instantaneous);
and for all current harddisk sizes ext2 is somewhat to much slower on
mount. And yes, the redhat init system (just like suse's or most others,
of course) is sooo slow that improving the init system will have a much
larger effect than the ext2/reiserfs differences.

(So trying to improve this in the kernel would be the wrong place to
start).

-- 
      -----==-                                             |
      ----==-- _                                           |
      ---==---(_)__  __ ____  __       Marc Lehmann      +--
      --==---/ / _ \/ // /\ \/ /       pcg@goof.com      |e|
      -=====/_/_//_/\_,_/ /_/\_\       XX11-RIPE         --+
    The choice of a GNU generation                       |
                                                         |

Re: ReiserFS / 2.4.6 / Data Corruption

[Hans Reiser]

"pcg( Marc)"@goof(A.).(Lehmann )com wrote:

> > No. The only thing I can think of that might slow it is that we build with
> > the reiserfs paranoia/sanity checks on.
> 
> That's a pretty dumb thing. Maybe one should have asked the develoers
> before doing this (they never do). Redhat somehow manages pretty well to
> show reiserfs in a bad light ;)

Let us be a bit more precise here.  If you click on the help button when deciding whether to select
that option it tells you not to do it.  What can you say about a distro that doesn't read the help
buttons for the kernel options when configuring the kernel?  Shovelware?

Hans

Re: ReiserFS / 2.4.6 / Data Corruption

[Alan Cox]

> > No. The only thing I can think of that might slow it is that we build with
> > the reiserfs paranoia/sanity checks on. Thats because at the time 7.1 was
> 
> Yes, that option should never be on for an end user not having a bug that he wants a more detailed
> bug report on.  It just makes us look slow compared to ext2.

Maybe its old fashioned but we'd rather any inconsistency in the file system
behaviour was made obvious to the end user. Enterprise customers object to
losing data.

> 2.4.2 was not a stable kernel for any FS, not just for ReiserFS.

The RH 2.4.2 derived kernel isnt 2.4.2 by any stretch of the imagination. 
Vanilla 2.4.2 wouldnt pass a test suite.

> I don't think that even with CONFIG_REISERFS_CHECK on, journal replay can take as long as fsck on
> ext2.  reiserfsck though, if that was on, oh, could even RedHat be that desperate to make us look
> bad to users as to run reiserfsck at every boot?

Hans, if you stopped considering every report that your file system wasn't
the best in the world as either a conspiracy theory or someone elses fault
you'd have a much better product

Nobody needs conspiracies to not use reiserfs as their core fs, and until
things like big endian support are cleanly resolved that isnt likely to
change.

Alan

Re: ReiserFS / 2.4.6 / Data Corruption

[Hans Reiser]

Alan Cox wrote:

> Nobody needs conspiracies to not use reiserfs as their core fs, and until
> things like big endian support are cleanly resolved that isnt likely to
> change.
> 
> Alan
big endian support is resolved, there is a working patch for it by Jeff Mahoney, it passes all of
our tests, but it is a feature not a bug fix, and not something for a supposedly stabilizing kernel.

Nikita, you were supposed to send the big endian support and some other stuff in to Alan for testing
in the ac series, what is the status of patches that are supposed to be going to Alan?

Hans

Re: ext3-2.4-0.9.4

[Alan Cox]

> This sort of attitude is just ridiculous.  Unix had a defined set of
> semantics.  This might have been stupid semantics, but it had them.

The unix defined semantics are very simple and very clear. They btw
dont contain the guarantees that certain email system authors think they do
and they never have.

rename() itself is new as of 4BSD, rather than ever being in true unix.
True unix did the right thing. It said 'this problem is hard, this problem
is application specific, do it at application level'.

> When I contacted the Linux JFS team about the semantics of link(), I
> was told that there is _no way_ of forcing a link() to disk.  Not an
> fsync() on the file, not an fsync() on the directory, just _not
> possible_.

I would expect an fsync of the directory to do that. It does on other
Linux file systems so it violates the least suprise bit. Right now JFS
isnt a standard file system on Linux however, and they have much left to do.
I suspect its something to ask them about.

> Thus why all reasonably paranoid MTAs and other mail programs say "use
> chattr +S on ext2"---we need ordered metadata writes.

And then your IDE disk gets you anyway. Also if you write metadata first 
then you risk delivering email to the wrong person instead. 

> You want to help performance?  Give us an fsync() that works on
> multiple file descriptors at once, or an async fsync() call.  Don't
> make us fight the OS on getting data to disk.

And what pray does an asynchronous fsync do. It seems to be a nop to me.

Doing reliabile transactions on disk is a hard problem. That is why oracle
and friends have spent many man years of research on this kind of problem. 
Current unix mailers do the smoke mirrors and prayer bit to reduce the
probability a little that is all, regardless of fs and os.

Alan

Re: ext3-2.4-0.9.4

[Lawrence Greenfield]

   Date: Fri, 27 Jul 2001 17:50:29 +0100 (BST)
   Cc: linux-kernel@vger.kernel.org
   From: Alan Cox <alan@lxorguk.ukuu.org.uk>

[...]
   > Thus why all reasonably paranoid MTAs and other mail programs say "use
   > chattr +S on ext2"---we need ordered metadata writes.

   And then your IDE disk gets you anyway. Also if you write metadata first 
   then you risk delivering email to the wrong person instead. 

These are tangential issues.  Not everybody uses IDE disks.  I'm not
asking for things that are impossible.  Just because sometimes the
hardware screws you isn't a good reason for not trying to do the right
thing.

The application can avoid the wrong file problem by zeroing out data
before releasing it to the OS to reallocate.

   > You want to help performance?  Give us an fsync() that works on
   > multiple file descriptors at once, or an async fsync() call.  Don't
   > make us fight the OS on getting data to disk.

   And what pray does an asynchronous fsync do. It seems to be a nop to me.

An async fsync allows me to issue multiple fsyncs and then wait for
all of them to complete, hopefully in the same framework that I would
do async I/O (but that's an argument for another day).

   Doing reliabile transactions on disk is a hard problem. That is why oracle
   and friends have spent many man years of research on this kind of problem. 
   Current unix mailers do the smoke mirrors and prayer bit to reduce the
   probability a little that is all, regardless of fs and os.

Isn't the point of the operating system to try to make it as easy as
possible to do these things correctly?

Otherwise you force anyone who wants to write a reliable application
(be it e-mail or not) to go to Oracle and one wonders why fsync() is
even implemented.

Larry

Re: ext3-2.4-0.9.4

[Daniel Phillips]

On Friday 27 July 2001 19:41, Lawrence Greenfield wrote:
> From: Alan Cox <alan@lxorguk.ukuu.org.uk>
> > Lawrence Greenfield wrote:
> > > You want to help performance?  Give us an fsync() that works on
> > > multiple file descriptors at once, or an async fsync() call. 
> > > Don't make us fight the OS on getting data to disk.
> >
> > And what pray does an asynchronous fsync do. It seems to be a nop
> to me.
>
> An async fsync allows me to issue multiple fsyncs and then wait for
> all of them to complete, hopefully in the same framework that I would
> do async I/O (but that's an argument for another day).

I'll say.  While it's truly desirable, all known filesystems are *far* 
from being able to do that.  An efficient, reliable fsync would do the 
trick for you, or even an efficient sync.  And somewhere in Andrew 
Morton's bag of tricks is something to fix you up too, read his 
comments carefully.

Looking forward, a sanely defined filesystem transaction interface 
from userland would give the best possible combination of performance 
and reliability.[1]  Since we now have four filesystems (five if you 
count JFFS) that could implement such a transaction interface, now is 
the time to figure out what it would look like.  That would include 
accomodating the needs of MTA developers.  It would be Linux-specific 
for sure.  It would also be progress.  If it turned out to be the 
fastest way to run a mailer we'd see it migrate to other nixes soon 
enough.

>    Doing reliabile transactions on disk is a hard problem. That is
> why oracle and friends have spent many man years of research on this
> kind of problem.

Tell me about it ;-)

> Current unix mailers do the smoke mirrors and prayer
> bit to reduce the probability a little that is all, regardless of fs
> and os.
>
> Isn't the point of the operating system to try to make it as easy as
> possible to do these things correctly?

   begin_transaction (filesystem_handle);
   <send the mail>;
   if (!end_transaction (filesystem_handle))
	<confirm sent>;

Something like that.[2]  Caveat: this is blue-sky stuff, it is not 
going to solve your problem today.  Andrew Morton and Hans Reiser are 
working on solving the problem today by giving you at least one mode 
where rename is synchronous, or at least giving you a fast fsync.

I'm with those who think that a little short-term pain is worth it if 
the final result is superior.

> Otherwise you force anyone who wants to write a reliable application
> (be it e-mail or not) to go to Oracle and one wonders why fsync() is
> even implemented.

[1] Al Viro pointed out that such a transaction interface could open up 
new possibilities for DOS attacks, something that has to be anticipated 
in the design.

[2] I see Alan suggested essentially the same thing in another branch 
of this thread.  Then by the "million flies" theorum...

--
Daniel

Re: ext3-2.4-0.9.4

[Patrick J. LoPresti]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> Also if you write metadata first then you risk delivering email to
> the wrong person instead.

The MTAs do this:

    Open temp file
    Write to temp file
    fsync() temp file
    rename() temp file into mail spool
    indicate success to remote MTA

As long as rename() does not return until the metadata are committed,
this should be a reliable delivery mechanism.  After a crash, you
might end up with the temp file still there, or with the file having a
link count of two (temp file and spool file).  But you can clean up
all of this at boot time; if the temp file is gone and the spool file
is present, then the transaction was completed.

(Yes, you might not have returned the success code to the remote MTA,
but that just means you might do a double delivery.  That is an
acceptable failure mode; corrupting, losing, or misdirecting mail is
not.)

How does this scheme "risk delivering mail to the wrong person
instead"?

If you have metadata journalling, all you need for this algorithm to
work is to have rename() write to the journal before returning.  Is
this true for any of the current journalling file systems on Linux?

 - Pat

Re: ext3-2.4-0.9.4

[Alan Cox]

> How does this scheme "risk delivering mail to the wrong person
> instead"?

With the fsync it looks ok for most cases. It depends on the actions of
a rename touching only one disk block - which of course it doesn't do. Even
so with the fsync on a sane fs I cant see that problem occuring

> If you have metadata journalling, all you need for this algorithm to
> work is to have rename() write to the journal before returning.  Is
> this true for any of the current journalling file systems on Linux?

Ext3 I believe so, Reiserfs I would assume so but Hans can answer
definitively

Re: ext3-2.4-0.9.4

[Chris Wedgwood]

On Sat, Jul 28, 2001 at 08:03:37PM +0100, Alan Cox wrote:

    Ext3 I believe so, Reiserfs I would assume so but Hans can answer
    definitively

Reiserfs does not, nor are creates or unlink operations synchronous.

For MTAs it just happens to work: if you fsync the way transactions
are written means the metadata for the dirtectories is written as part
of the transaction --- but I think this is a quirk and not by design?

Chris?




  --cw

Re: ext3-2.4-0.9.4

[Chris Mason]

On Sunday, July 29, 2001 01:53:48 PM +1200 Chris Wedgwood <cw@f00f.org>
wrote:

> On Sat, Jul 28, 2001 at 08:03:37PM +0100, Alan Cox wrote:
> 
>     Ext3 I believe so, Reiserfs I would assume so but Hans can answer
>     definitively
> 
> Reiserfs does not, nor are creates or unlink operations synchronous.
> 
> For MTAs it just happens to work: if you fsync the way transactions
> are written means the metadata for the dirtectories is written as part
> of the transaction --- but I think this is a quirk and not by design?
> 
> Chris?

Correct, in the current 2.4.x code, its a quirk.  fsync(any object) ==
fsync(all pending metadata, including renames).

There is a transcation tracking patch floating around out there that makes
reiserfs fsync/O_SYNC much faster by only committing the last transaction a
given file/dir was involved in.  I had sent this to alan just after 2.4.7
came out, but it looks like I need to resend.

Anyway, during a rename, this patch updates the inode transaction tracking
stuff so an fsync on the file should also commit the directory changes.
But, that isn't something I really intend to advertise much, since the
accepted linux way is fsync(dir).

-chris

Re: ext3-2.4-0.9.4

[Patrick J. LoPresti]

Chris Mason <mason@suse.com> writes:

> Correct, in the current 2.4.x code, its a quirk.  fsync(any object) ==
> fsync(all pending metadata, including renames).

This does not help.  The MTAs are doing fsync() on the temporary file
and then using the *subsequent* rename() as the committing operation.

> Anyway, during a rename, this patch updates the inode transaction
> tracking stuff so an fsync on the file should also commit the
> directory changes.  But, that isn't something I really intend to
> advertise much, since the accepted linux way is fsync(dir).

It would be nice to have an option (on either the directory or the
mountpoint) to cause all metadata updates to commit to the journal
without causing all operations to be fully synchronous.  This would
provide compatibility with BSD-centric code without taking the
performance hit of synchronous data.  Heck, just having link() and
rename() perform a commit would be good enough for almost all
applications.

 - Pat

Re: ext3-2.4-0.9.4

[Alan Cox]

> Chris Mason <mason@suse.com> writes:
> 
> > Correct, in the current 2.4.x code, its a quirk.  fsync(any object) ==
> > fsync(all pending metadata, including renames).
> 
> This does not help.  The MTAs are doing fsync() on the temporary file
> and then using the *subsequent* rename() as the committing operation.

Which is quaint, because as we've pointed out repeatedly to you rename
is not an atomic operation. Even on a simple BSD or ext2 style fs it can
be two directory block writes,  metadata block writes, a bitmap write
and a cylinder group write.

> It would be nice to have an option (on either the directory or the
> mountpoint) to cause all metadata updates to commit to the journal
> without causing all operations to be fully synchronous.  This would

You mean fsync() on the directory. 

Alan

Re: ext3-2.4-0.9.4

[Patrick J. LoPresti]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > Chris Mason <mason@suse.com> writes:
> > 
> > > Correct, in the current 2.4.x code, its a quirk.  fsync(any object) ==
> > > fsync(all pending metadata, including renames).
> > 
> > This does not help.  The MTAs are doing fsync() on the temporary file
> > and then using the *subsequent* rename() as the committing operation.
> 
> Which is quaint, because as we've pointed out repeatedly to you rename
> is not an atomic operation. Even on a simple BSD or ext2 style fs it can
> be two directory block writes,  metadata block writes, a bitmap write
> and a cylinder group write.

But not on a journalling filesystem.  I assume that a journal "commit"
is atomic.  If it is not, then fsync() on the directory does not solve
the problem either.

Put another way, I am suggesting a mount-time or directory option to
effectively cause rename() and link() to automatically be followed by
an fsync() of the containing directory.  (Actually, from this
perspective, maybe you could fix the MTA in user space with LD_PRELOAD
hackery or somesuch.  Hm...)

> > It would be nice to have an option (on either the directory or the
> > mountpoint) to cause all metadata updates to commit to the journal
> > without causing all operations to be fully synchronous.  This would
> 
> You mean fsync() on the directory. 

In other words, "Get the MTA authors to change their code."  That is a
nice little war, but it is fought at the expense of users who just
want to use the code provided by their vendor and have it work.

The situation is this:

  The relevant standards (POSIX, SuS, etc.) provide no way to perform
  reliable transactions on a file system.

  BSD provides one solution, which is synchronous metatdata.  (I am
  assuming modern BSDs already deal with the multiple-disk-block
  problem to make these transactions properly atomic.  Is this
  assumption false?)

  Linux provides a different solution, which is fsync() on the
  directory.

  All MTAs, and other apps besides, currently use the BSD solution for
  reliable transactions.

Is it really so absurd to ask Linux to provide efficient support of
the BSD semantics as an option?

 - Pat

Re: ext3-2.4-0.9.4

[Rik van Riel]

On 30 Jul 2001, Patrick J. LoPresti wrote:

>   The relevant standards (POSIX, SuS, etc.) provide no way to perform
>   reliable transactions on a file system.
>
>   BSD provides one solution, which is synchronous metatdata.  (I am
>   assuming modern BSDs already deal with the multiple-disk-block
>   problem to make these transactions properly atomic.  Is this
>   assumption false?)
>
>   Linux provides a different solution, which is fsync() on the
>   directory.
>
>   All MTAs, and other apps besides, currently use the BSD solution for
>   reliable transactions.
>
> Is it really so absurd to ask Linux to provide efficient support of
> the BSD semantics as an option?

Yes. You could fix this issue in userland very easily,
it might even work with an LD_PRELOAD ...

Besides BSD softupdates and the various journaling
filesystems which are in use on other Unixen also
don't provide the 4.3BSD solution any more ...

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: ext3-2.4-0.9.4

[Andrew McNamara]

>> This does not help.  The MTAs are doing fsync() on the temporary file
>> and then using the *subsequent* rename() as the committing operation.
>
>Which is quaint, because as we've pointed out repeatedly to you rename
>is not an atomic operation. Even on a simple BSD or ext2 style fs it can
>be two directory block writes,  metadata block writes, a bitmap write
>and a cylinder group write.

This is almost (but not quite) irrelevant. The receiving MTA simply
wants the fsync()/rename() system call to not return until everything
(including directory blocks) have been written to disk, at which point,
it says to the remote end "250 OK". If the receiving machine goes down
at any point up until this one, the sending system will resend the
message.  (Yes, the receiving system may have a corrupt directory, and
this is a problem).

 ---
Andrew McNamara (System Architect)

connect.com.au Pty Ltd
Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia
Phone: +61 2 9409 2117, Fax: +61 2 9409 2111

Re: ext3-2.4-0.9.4

[Rik van Riel]

On 30 Jul 2001, Patrick J. LoPresti wrote:

> performance hit of synchronous data.  Heck, just having link() and
> rename() perform a commit would be good enough for almost all
> applications.

It would be "good enough" for some applications,
but it would be absolutely disastrous for most
applications I run (ie. moving source code around).

Exactly what is wrong with doing fsync() on the
directory ?

Why do you want us to turn link() and rename()
into link_slowly() and rename_slowly() ?

Why can't you use a simple wrapper function to
do this for you ?

cheers,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: ext3-2.4-0.9.4

[Patrick J. LoPresti]

Rik van Riel <riel@conectiva.com.br> writes:

> Exactly what is wrong with doing fsync() on the
> directory ?

Nothing, except that it requires source code changes to every
application which expects BSD semantics for these operations.
Anecdotal evidence suggests at least the MTA authors are resistant to
making such changes.

> Why do you want us to turn link() and rename()
> into link_slowly() and rename_slowly() ?

I don't by default, only as an option.  You know, just like "chattr
-S" or "mount -o sync" means do_everything_slowly().

> Why can't you use a simple wrapper function to
> do this for you ?

It would not be all that simple; it would have to parse the arguments
to figure out the containing directories, open() a file descriptor on
each, and fsync() them.  Not impossible, but it does introduce several
those additional system calls as performance hits and points of
failure, not to mention possible race conditions.

Still, I suppose you could do this well enough in the C library.  You
might even want it to be the default when "__USE_BSD" is defined or
something.

But it still seems simpler to me just to make it an option in the file
system.

In your next message, you say:

> Besides BSD softupdates and the various journaling
> filesystems which are in use on other Unixen also
> don't provide the 4.3BSD solution any more ...

This surprises me if it is true; do you have a reference?  And what
mechanism *do* the modern BSDs provide to commit metadata changes to
disk?

 - Pat

Re: ext3-2.4-0.9.4

[Rik van Riel]

On 30 Jul 2001, Patrick J. LoPresti wrote:
> Rik van Riel <riel@conectiva.com.br> writes:
>
> > Exactly what is wrong with doing fsync() on the
> > directory ?
>
> Nothing, except that it requires source code changes to every
> application which expects BSD semantics for these operations.
> Anecdotal evidence suggests at least the MTA authors are resistant to
> making such changes.

You may need to make them anyway for Digital's AdvFS,
IRIX XFS, IBM JFS, Veritas' VXFS and BSD softupdates.

Lets face it, FFS is no longer the only available
filesystem. Don't expect FFS semantics from other
filesystems.

> > Why can't you use a simple wrapper function to
> > do this for you ?
>
> It would not be all that simple; it would have to parse the
> arguments to figure out the containing directories, open() a
> file descriptor on each, and fsync() them.

Hmmm, then maybe we'd just want some flag to fsync()
telling the kernel to also sync the parent directory
of the file and do whatever it needs to do to get the
rename() or link() committed ?

> But it still seems simpler to me just to make it an option in
> the file system.

It's always simpler when it's not YOU who has to
implement it ;)

cheers,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Mon, 30 Jul 2001, Rik van Riel wrote:

> Hmmm, then maybe we'd just want some flag to fsync()
> telling the kernel to also sync the parent directory
> of the file and do whatever it needs to do to get the
> rename() or link() committed ?

Heck, you can't tell the kernel to do rename/link/open/unlink
synchronously in-band. This list doesn't care for other OS's. The
semantics FreeBSD (e. g.) offers ARE indeed documented.

This won't work out without kernel support. Portable reliability doesn't
come for free.

chattr +S is bad (slow). bloating all applications to include every
possible brain fart that the random FS inventor let go is even worse.

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Tue, 31 Jul 2001, Matthias Andree wrote:
> On Mon, 30 Jul 2001, Rik van Riel wrote:
>
> > Hmmm, then maybe we'd just want some flag to fsync()
> > telling the kernel to also sync the parent directory
> > of the file and do whatever it needs to do to get the
> > rename() or link() committed ?
>
> Heck, you can't tell the kernel to do rename/link/open/unlink
> synchronously in-band. This list doesn't care for other OS's.
> The semantics FreeBSD (e. g.) offers ARE indeed documented.

Go back a few posts and read about the semantics
FreeBSD has when the filesystem is mounted with
softupdates.

Then take a deep breath.

regards,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Lawrence Greenfield]

   From: "Patrick J. LoPresti" <patl@cag.lcs.mit.edu>
   Date: 	30 Jul 2001 12:46:13 -0400

   > Besides BSD softupdates and the various journaling
   > filesystems which are in use on other Unixen also
   > don't provide the 4.3BSD solution any more ...

   This surprises me if it is true; do you have a reference?  And what
   mechanism *do* the modern BSDs provide to commit metadata changes to
   disk?

BSD softupdates allows you to call fsync() on the file, and this will
sync the directories all the way up to the root if necessary.

Thus BSD fsync() actually guarantees that when it returns, the file
(and all of it's filenames) will survive a reboot.

Sendmail does:
fd = open(tmp)
write(fd)
fsync(fd)
rename(tmp, final)
fsync(fd)

Cyrus IMAP does:
fd = open(tmp)
write(fd)
fsync(fd)
link(tmp, final1)
link(tmp, final2)
link(tmp, final3)
fsync(fd)
close(fd)
unlink(tmp)

The idea that Linux fsync() doesn't actually make the file survive
reboots is pretty ridiculous.

Larry

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Mon, 30 Jul 2001, Lawrence Greenfield wrote:
>    From: "Patrick J. LoPresti" <patl@cag.lcs.mit.edu>
>    Date: 	30 Jul 2001 12:46:13 -0400
>
>    > Besides BSD softupdates and the various journaling
>    > filesystems which are in use on other Unixen also
>    > don't provide the 4.3BSD solution any more ...
>
>    This surprises me if it is true; do you have a reference?  And what
>    mechanism *do* the modern BSDs provide to commit metadata changes to
>    disk?
>
> BSD softupdates allows you to call fsync() on the file, and this will
> sync the directories all the way up to the root if necessary.
>
> Thus BSD fsync() actually guarantees that when it returns, the file
> (and all of it's filenames) will survive a reboot.

Note that this is very different from the "link() should be
synchronous()" mantra we've been hearing over the last days.

These fsync() semantics make lots of sense to me, I'm all
for it.

regards,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Chris Wedgwood]

On Mon, Jul 30, 2001 at 02:25:51PM -0300, Rik van Riel wrote:

    Note that this is very different from the "link() should be
    synchronous()" mantra we've been hearing over the last days.
    
    These fsync() semantics make lots of sense to me, I'm all
    for it.

And what if the file has hundreds or thousands of links? How do we
cleanly keep track of all those?



  --cw

Re: ext3-2.4-0.9.4

[Lawrence Greenfield]

   Date: Tue, 31 Jul 2001 05:38:13 +1200
   From: Chris Wedgwood <cw@f00f.org>

   On Mon, Jul 30, 2001 at 02:25:51PM -0300, Rik van Riel wrote:

       Note that this is very different from the "link() should be
       synchronous()" mantra we've been hearing over the last days.

       These fsync() semantics make lots of sense to me, I'm all
       for it.

   And what if the file has hundreds or thousands of links? How do we
   cleanly keep track of all those?

You don't have to keep track of all of them, just the uncommitted
ones.  I could imagine the filesystem forcing periodic commits on
pathological files (those with thousands of links) to limit the number
of pending directory operations per file.

While the softupdates paper doesn't appear to directly address this
concern, clearly their implementation has to deal with it in some way.

Larry

Re: ext3-2.4-0.9.4

[Chris Mason]

On Monday, July 30, 2001 01:49:12 PM -0400 Lawrence Greenfield
<leg+@andrew.cmu.edu> wrote:

>    Date: Tue, 31 Jul 2001 05:38:13 +1200
>    From: Chris Wedgwood <cw@f00f.org>
> 
>    On Mon, Jul 30, 2001 at 02:25:51PM -0300, Rik van Riel wrote:
> 
>        Note that this is very different from the "link() should be
>        synchronous()" mantra we've been hearing over the last days.
> 
>        These fsync() semantics make lots of sense to me, I'm all
>        for it.
> 
>    And what if the file has hundreds or thousands of links? How do we
>    cleanly keep track of all those?
> 
> You don't have to keep track of all of them, just the uncommitted
> ones. 

Well, the idea is to get it done in the VFS layer.  reiserfs, ext3, and
probably the other journaled filesystems could keep track of the last
transacation and inode was involved with, making the softupdate style
fsync(file) to commit a rename easy.

But, ext2 and the normal filesystems don't have it quite so good.

-chris

Re: ext3-2.4-0.9.4

[Chris Wedgwood]

On Mon, Jul 30, 2001 at 01:59:04PM -0400, Chris Mason wrote:

    Well, the idea is to get it done in the VFS layer.  reiserfs, ext3, and
    probably the other journaled filesystems could keep track of the last
    transacation and inode was involved with, making the softupdate style
    fsync(file) to commit a rename easy.

But, right now, the VFS layer doesn't know about magic attributes
(such as ext2/3 +S).  The VFS would have to be taught about these and
some other things to support both asynchronous and synchronous
metadata updates (and presumably other smarts too).  The trouble is
these attributes themselves and how they are stored is fs specific, we
could always mandate that as of 2.5.x all filesystems _can_ support
some kind of extended API and defined a minimalist set of attributes
for all filesystems and then allow specific filesystems to have their
own.  Arguably if people are going to force ACLs upon the world, then
a common API would be nice across XFS, resierfs4, JFFS, etc.  (NTFS
can use an API specific to the FS itself as NTFS ACLs are much more
complex and different looking beasts that those from early POSIX
drafts).

For journalling filesystems, it would be really nice if setting an
attribute was all that was required to make rename(2) atomic (or at
the very least to make sure that if the rename system call returns,
the data has been written to non-volatile storage).



  --cw

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Mon, 30 Jul 2001, Rik van Riel wrote:

> > Thus BSD fsync() actually guarantees that when it returns, the file
> > (and all of it's filenames) will survive a reboot.
> 
> Note that this is very different from the "link() should be
> synchronous()" mantra we've been hearing over the last days.

Indeed, but this might still require MTA fixing probably, and opening a
file you just want to rename is quite expensive an operation.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Mon, 30 Jul 2001, Lawrence Greenfield wrote:

> The idea that Linux fsync() doesn't actually make the file survive
> reboots is pretty ridiculous.

That doesn't apply to ReiserFS or ext3fs, it does apply to ext2fs and
possibly others.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Jan Harkes]

On Fri, Aug 03, 2001 at 06:54:12PM +0200, Daniel Phillips wrote:
> On Friday 03 August 2001 18:16, Jan Harkes wrote:
> > On Fri, Aug 03, 2001 at 05:47:17PM +0200, Daniel Phillips wrote:
> > > On Friday 03 August 2001 17:18, Jan Harkes wrote:
> > > > Working on a distributed filesystem with somewhat weaker than UNIX
> > > > semantics might have skewed my vision. In Coda not every client will be
> > > > able to figure out which are all of the possibly paths that can lead to
> > > > a file object. And although we currently try our best to block
> > > > hardlinked directories they could possibly exist, making the problems
> > > > even worse.
> > >
> > > We don't need all the paths, and not any specific path, just a path.
> >
> > Even if that path leads to a name that got removed, thereby forcing the
> > object into lost+found? I thought the MTA did something like,
> 
> We'd better get confirmation from the MTA expert in the thread.
> 
> > fd = open(tmp/file)
> > write(fd)
> > fsync(fd)
> > link(tmp/file, new/file)
> > fsync(fd)		*1
> > unlink(tmp/file)
> >
> > *1 If this fsync only syncs the path leading to tmp/file, and the unlink
> > tmp/file is written back to disk, which is likely because we're only
> > creating/syncing stuff in tmp.  Now, until new/file is written there is
> > no path information leading to the file anymore which makes this as
> > 'safe' as not syncing path name information at all.
> 
> Nice clear example!  Yes, in essence we would have synced the original
> path twice.  If this is what the MTA is really doing I'm willing to join
> the "MTA is broken" camp.

Here is the relevant mail,

On Mon, Jul 30, 2001 at 01:11:32PM -0400, Lawrence Greenfield wrote:
} BSD softupdates allows you to call fsync() on the file, and this will
} sync the directories all the way up to the root if necessary.
} 
} Thus BSD fsync() actually guarantees that when it returns, the file
} (and all of it's filenames) will survive a reboot.
} 
} Sendmail does:
} fd = open(tmp)
} write(fd)
} fsync(fd)
} rename(tmp, final)
} fsync(fd)
} 
} Cyrus IMAP does:
} fd = open(tmp)
} write(fd)
} fsync(fd)
} link(tmp, final1)
} link(tmp, final2)
} link(tmp, final3)
} fsync(fd)
} close(fd)
} unlink(tmp)
} 
} The idea that Linux fsync() doesn't actually make the file survive
} reboots is pretty ridiculous.

As you can see, the 'sync a path leading to the file' semantics from SuS
don't work in the Cyrus IMAP case as is specifically requires to have
_all_ paths committed to disk before fsync returns.

On Fri, Aug 03, 2001 at 06:54:12PM +0200, Daniel Phillips wrote:
> On Friday 03 August 2001 18:16, Jan Harkes wrote:
> > Now if the application would use the directory sync, it can actually
> > tell the kernel that that new/file name is the interesting one to keep
> > and that tmp doesn't even need to be written to disk at all.
> 
> Yep.  Or if it used rename, which updates the dcache entries, instead
> of link/unlink.

MTA's that want to do reliable deliveries using multiple processes (or
on a networked filesystem) tend to not use rename because it implicitly
unlinks the target if it already exists and this could lead to loss of
mail that was already considered as being delivered.

Jan

Re: ext3-2.4-0.9.4

[Patrick J. LoPresti]

Jan Harkes <jaharkes@cs.cmu.edu> writes:

> Here is the relevant mail,
> 
> On Mon, Jul 30, 2001 at 01:11:32PM -0400, Lawrence Greenfield wrote:
> } BSD softupdates allows you to call fsync() on the file, and this will
> } sync the directories all the way up to the root if necessary.
> } 
> } Thus BSD fsync() actually guarantees that when it returns, the file
> } (and all of it's filenames) will survive a reboot.
> } 
> } Sendmail does:
> } fd = open(tmp)
> } write(fd)
> } fsync(fd)
> } rename(tmp, final)
> } fsync(fd)
> } 
> } Cyrus IMAP does:
> } fd = open(tmp)
> } write(fd)
> } fsync(fd)
> } link(tmp, final1)
> } link(tmp, final2)
> } link(tmp, final3)
> } fsync(fd)
> } close(fd)
> } unlink(tmp)
> } 
> } The idea that Linux fsync() doesn't actually make the file survive
> } reboots is pretty ridiculous.
> 
> As you can see, the 'sync a path leading to the file' semantics from SuS
> don't work in the Cyrus IMAP case as is specifically requires to have
> _all_ paths committed to disk before fsync returns.

To fill in more of the table, Qmail does:

  fd = open(tmp)
  write(fd)
  fsync(fd)
  link(tmp,final)
  close(fd)

...and Postfix does:

  fd = open(final)
  write(fd)
  (should be an "fsync(fd)" here, but I cannot find it)
  fchmod(fd,+execute)
  fsync(fd)
  close(fd)

Postfix apparently uses the execute bit to indicate that delivery is
complete.  I am probably misreading the source (version 20010228
Patchlevel 3), but I do not see any fsync() between the write and the
fchmod.  Surely it is there or this delivery scheme is not reliable on
any system, since without an intervening fsync() the writes to the
data and the permissions can happen out of order.

Anyway, it is certainly true that it is largely useless to have
fsync() commit only one path to a file; many applications expect to be
able to force a simple link(x,y) to be committed to disk.

I know this thread is already much too long, but I am still not sure I
understand the conclusions.  I *think* it is true that:

  1) People disagree about what SuS mandates, but at least a few
     critical developers (e.g., sct) say it definitely does not
     require synchronizing directory entries for fsync().

  2) It would be fairly easy and efficient for fsync() to chase one
     chain of directory entries up to the root, but a lot harder and
     slower to find and commit all of them.

  3) Most (?) core developers, including Linus (?), would not object
     to "dirsync" as a mount option and/or directory attribute, but
     somebody has to rise to the occasion and create the patches.

Is this an accurate summary?

 - Pat

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Fri, 03 Aug 2001, Patrick J. LoPresti wrote:

> To fill in more of the table, Qmail does:
> 
>   fd = open(tmp)
>   write(fd)
>   fsync(fd)
>   link(tmp,final)
>   close(fd)

http://cr.yp.to/qmail/faq/reliability.html

> ...and Postfix does:
> 
>   fd = open(final)
>   write(fd)
>   (should be an "fsync(fd)" here, but I cannot find it)
>   fchmod(fd,+execute)
>   fsync(fd)
>   close(fd)

> Postfix apparently uses the execute bit to indicate that delivery is
> complete.  I am probably misreading the source (version 20010228
> Patchlevel 3), but I do not see any fsync() between the write and the
> fchmod.  Surely it is there or this delivery scheme is not reliable on
> any system, since without an intervening fsync() the writes to the
> data and the permissions can happen out of order.

Not really. The error code if fsync() or close failed are propagated
back to the caller who then decides what to do. smtpd.c nukes the file.
postdrop.c/sendmail.c do not, but the pickup daemon will see that the
file had problems on sync and discard it.

I'm asking Wietse off-list how reliable this approach is and will report
back privately. It should be fairly reliable.

> Anyway, it is certainly true that it is largely useless to have
> fsync() commit only one path to a file; many applications expect to be
> able to force a simple link(x,y) to be committed to disk.

BSD FFS + softupdates sync all file names, traversing from the mount
point down to the actual directory entries that need to be synched.

>   1) People disagree about what SuS mandates, but at least a few
>      critical developers (e.g., sct) say it definitely does not
>      require synchronizing directory entries for fsync().
> 
>   2) It would be fairly easy and efficient for fsync() to chase one
>      chain of directory entries up to the root, but a lot harder and
>      slower to find and commit all of them.

For BSD FFS + softupdates, this is already done.

>   3) Most (?) core developers, including Linus (?), would not object
>      to "dirsync" as a mount option and/or directory attribute, but
>      somebody has to rise to the occasion and create the patches.
> 
> Is this an accurate summary?

It looks so to me. After the MTA behaviour has been dug up, the dirsync
option could be even weaker if fsync() behaved like FFS + softupdates:
sync the directory entries, including those of link and rename, as well.

The only things to consider would be unlink and symlink. symlinks are
tough since you cannot open() them. Not sure about unlink, looks as if
there's really no way apart from fsync(2)ing the directory or sync(2)ing
the world for these two unless there's a dirsync option coming up.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Sat, 4 Aug 2001, Matthias Andree wrote:

> http://cr.yp.to/qmail/faq/reliability.html

You should teach the guy about MTAs one day.

Softupdates and other filesystems are perfectly
safe with mailers which work with the filesystem
instead of prescribing the filesystem authors
how they should do their work ;)

Oh, most of those other mailers are RFC compliant
too, but that's a separate issue ;)

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: ext3-2.4-0.9.4

[Patrick J. LoPresti]

Matthias Andree <matthias.andree@stud.uni-dortmund.de> writes:

> On Fri, 03 Aug 2001, Patrick J. LoPresti wrote:
> 
> > To fill in more of the table, Qmail does:
> > 
> >   fd = open(tmp)
> >   write(fd)
> >   fsync(fd)
> >   link(tmp,final)
> >   close(fd)
> 
> http://cr.yp.to/qmail/faq/reliability.html

...which is consistent.  Qmail is assuming that the link() is
synchronous, as it was back in the "Good Old Days" of stock FFS.

> > ...and Postfix does:
> > 
> >   fd = open(final)
> >   write(fd)
> >   (should be an "fsync(fd)" here, but I cannot find it)
> >   fchmod(fd,+execute)
> >   fsync(fd)
> >   close(fd)
> 
> > Postfix apparently uses the execute bit to indicate that delivery is
> > complete.  I am probably misreading the source (version 20010228
> > Patchlevel 3), but I do not see any fsync() between the write and the
> > fchmod.  Surely it is there or this delivery scheme is not reliable on
> > any system, since without an intervening fsync() the writes to the
> > data and the permissions can happen out of order.
> 
> Not really. The error code if fsync() or close failed are propagated
> back to the caller who then decides what to do. smtpd.c nukes the
> file.

That is not the problem.  The problem is that the system could start
flushing blocks to disk after the call to fchmod and before the call
to fsync.  If so, the system could write the mode bits first and then
crash before writing the data, leaving the execute bit set on the file
but without valid data within.  This could result in a corrupted mail
message.

To avoid this, Postfix *must* do fsync() or fdatasync() after the
write() and before the fchmod()+fsync(); that will insure that the
execute bit implies valid ("committed") data in the file.  I was
unable to find any such call to fsync() or fdatasync(), but as I
mentioned, I am probably simply misreading the code.

> > Anyway, it is certainly true that it is largely useless to have
> > fsync() commit only one path to a file; many applications expect to be
> > able to force a simple link(x,y) to be committed to disk.
> 
> BSD FFS + softupdates sync all file names, traversing from the mount
> point down to the actual directory entries that need to be synched.

...and the Linux developers continue to insist that this is "stupid".
Ah, the joys of gaps in standards.

> It looks so to me. After the MTA behaviour has been dug up, the
> dirsync option could be even weaker if fsync() behaved like FFS +
> softupdates: sync the directory entries, including those of link and
> rename, as well.

Ideally, this would be an option you could set per-application (as
opposed to per-directory or per-mountpoint), because we are really
talking about allowing Linux to support applications written for BSD
file system semantics.  It is not obvious to me what the best
implementation for that would be, though.  Maybe just a compile-time
option to choose the appropriate open/link/rename/etc. operations.

 - Pat

Re: ext3-2.4-0.9.4

[Gregory Maxwell]

On Fri, Aug 03, 2001 at 11:50:23PM -0400, Patrick J. LoPresti wrote:
> > http://cr.yp.to/qmail/faq/reliability.html
> 
> ...which is consistent.  Qmail is assuming that the link() is
> synchronous, as it was back in the "Good Old Days" of stock FFS.

That isn't the only cruft there from the "Good Old Days":

"Battery backups will keep your server alive, letting you park the disk to
avoid a head crash, when the power goes out."

What the hell? Self-parking drives predate qmail by quite a long time.

Re: ext3-2.4-0.9.4

[Mike Castle]

On Fri, Aug 03, 2001 at 11:50:23PM -0400, Patrick J. LoPresti wrote:
> Matthias Andree <matthias.andree@stud.uni-dortmund.de> writes:
> 
> > On Fri, 03 Aug 2001, Patrick J. LoPresti wrote:
> > 
> > > To fill in more of the table, Qmail does:
> > > 
> > >   fd = open(tmp)
> > >   write(fd)
> > >   fsync(fd)
> > >   link(tmp,final)
> > >   close(fd)
> > 
> > http://cr.yp.to/qmail/faq/reliability.html
> 
> ...which is consistent.  Qmail is assuming that the link() is
> synchronous, as it was back in the "Good Old Days" of stock FFS.

Which, from my reading of the archives, even BSD folk say is a "Bad 
Thing(tm)."

mrc

-- 
     Mike Castle      dalgoda@ix.netcom.com      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Fri, 3 Aug 2001, Mike Castle wrote:
> On Fri, Aug 03, 2001 at 11:50:23PM -0400, Patrick J. LoPresti wrote:
> > Matthias Andree <matthias.andree@stud.uni-dortmund.de> writes:
> > >
> > > http://cr.yp.to/qmail/faq/reliability.html
> >
> > ...which is consistent.  Qmail is assuming that the link() is
> > synchronous, as it was back in the "Good Old Days" of stock FFS.
>
> Which, from my reading of the archives, even BSD folk say is a "Bad
> Thing(tm)."

...which is consistent, looking at the other things
qmail does in strange ways ;)

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Fri, 03 Aug 2001, Patrick J. LoPresti wrote:

> To avoid this, Postfix *must* do fsync() or fdatasync() after the
> write() and before the fchmod()+fsync(); that will insure that the
> execute bit implies valid ("committed") data in the file.  I was
> unable to find any such call to fsync() or fdatasync(), but as I
> mentioned, I am probably simply misreading the code.

Thanks for clarifying, I'm asking Wietse to figure if Postfix's
queue file format is sufficient to check integrity.

> > It looks so to me. After the MTA behaviour has been dug up, the
> > dirsync option could be even weaker if fsync() behaved like FFS +
> > softupdates: sync the directory entries, including those of link and
> > rename, as well.
> 
> Ideally, this would be an option you could set per-application (as
> opposed to per-directory or per-mountpoint), because we are really
> talking about allowing Linux to support applications written for BSD
> file system semantics.  It is not obvious to me what the best
> implementation for that would be, though.  Maybe just a compile-time
> option to choose the appropriate open/link/rename/etc. operations.

To add to that confusion and alternatvies:
HAVE: async, sync
SUGGEST: 1. BSD dirsync, 2. "weak" unlink/symlink dirsync and have
fsync() track and sync pending link/rename as well, 3. make just symlink
dirsync, 4. be confused of all the options.

Where this could be set: directory inode flag, mount option, process
flag (like umask), include file.


Seriously, if fsync() syncs the effects of link and rename as well,
there's no need to make them synchronous unconditionally except if one
wants to offer a "traditional BSD synchronous directory semantics" mode. 

-- 
Matthias Andree

fsync() on directories (was Re: ext3-2.4-0.9.4)

[Patrick J. LoPresti]

Are the Linux "fsync() the directory" semantics documented anywhere?
I mean other than in the source code and on mailing lists.

It might be easier to convince MTA authors to support these semantics
if there were an "official" document describing them and giving some
guarantee that Linux will contine to support them in the future.  It
would be nice to see this described in linux/Documentation or in the
fsync() man page or both.  Without this, it is hard for a software
author to know that Linux's behavior here is not just an
implementation artifact.

 - Pat

P.S.  Is fdatasync() on a directory guaranteed to do anything?  Just
curious.

Re: ext3-2.4-0.9.4

[James Antill]

"Patrick J. LoPresti" <patl@cag.lcs.mit.edu> writes:


[snip sendmail/cyrus/qmail/postfix]

 Just in case anyone cares here's what exim does (AFAICS)...

 int fd1 = open(f1);
 write(fd1);
 fsync(fd1);
 
 int fd2 = open(tmp);
 write(fd2);
 fsync(fd2);
 rename(tmp, f2); // Good at this point.

 So that seems to rely on all dir operations being sync.

 Ps. I did a patch for exim to do the dir sync though...

http://www.and.org/exim-3.31-dirfsync.patch

-- 
# James Antill -- james@and.org
:0:
* ^From: .*james@and\.org
/dev/null

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Mon, 30 Jul 2001, Rik van Riel wrote:

> Exactly what is wrong with doing fsync() on the
> directory ?

It's non-portable and a kludge.

> Why do you want us to turn link() and rename()
> into link_slowly() and rename_slowly() ?

Opening up the directory requires lots of inode lookups which are
unnecessary.

> Why can't you use a simple wrapper function to
> do this for you ?

Because it's more inefficient than necessary and it bloats the
application.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Andrew Morton]

Alan Cox wrote:
> 
>...
> > If you have metadata journalling, all you need for this algorithm to
> > work is to have rename() write to the journal before returning.  Is
> > this true for any of the current journalling file systems on Linux?
> 
> Ext3 I believe so, Reiserfs I would assume so but Hans can answer
> definitively

For ext3: this is true if something forces a commit.  Apart from data in
`-o data=writeback' mode, a commit syncs the entire filesystem.
Things which force a commit include:

- completing a write() on an O_SYNC file.
- Performing any metadata operation on a `chattr +S' object
- Performing any metadata operation on an object on a `mount -o sync'
  filesystem.

In `data=journal' or `data=ordered' mode, any of these things will
commit everything to non-volatile storage.

-

rename() (was Re: ext3-2.4-0.9.4)

[Anthony DeBoer]

Patrick J. LoPresti <patl@cag.lcs.mit.edu> wrote:
>The MTAs do this:
>
>    Open temp file
>    Write to temp file
>    fsync() temp file
>    rename() temp file into mail spool
>    indicate success to remote MTA

Don't forget the unlink() temp file just before or after that last step.

>As long as rename() does not return until the metadata are committed,
>this should be a reliable delivery mechanism.  ...

As I understand it, rename() was originally invented for tasks like
installing a new /bin/sh with guarantees that another process running
at the same time would not fail to find a shell, and that if the system
fell over during the install you'd still have a shell on reboot.

See http://www.qef.com/ftp/rename.ps for an interesting history from
someone who was there at the time.  It's undated, but probably a decade
old.

It's my considered opinion that rename() _should_ fsync the target
directory before returning, and between that and the fsync() call on
the file itself (an install program should do the same call sequence as
above) you get the guarantee that the file is intact before you unlink
the temp version and return success.  OTOH, link() and unlink() are not
in the business of providing guarantees like that, and should not sync.

-- 
Anthony de Boer, curator, Anthony's Home for Aged Computing Machinery
<adb@leftmind.net>

Re: ReiserFS / 2.4.6 / Data Corruption

[Alan Cox]

> > Alan
> big endian support is resolved, there is a working patch for it by Jeff Mahoney, it passes all of
> our tests, but it is a feature not a bug fix, and not something for a supposedly stabilizing kernel.
> 
> Nikita, you were supposed to send the big endian support and some other stuff in to Alan for testing
> in the ac series, what is the status of patches that are supposed to be going to Alan?

I suspect its a bug fix to S/390, ppc and sparc users 8)

I'd be happy to test run it in -ac

Re: ext3-2.4-0.9.4

[Alan Cox]

> 
> "Paul G. Allen" <pgallen@randomlogic.com> writes:
>  
> > Do the newer kernel releases support the 760 MP chipset? Will they
> > anytime soon? (If not I will see if I can put it in myself.)
> 
> There is better support in 2.4.7 (especially IDE) but there is not complete
> support.  
> 
> I don't know of anyone planning on finishing up any the pieces so feel free.
> 
> Eric
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: ext3-2.4-0.9.4

[Alan Cox]

> These are tangential issues.  Not everybody uses IDE disks.  I'm not
> asking for things that are impossible.  Just because sometimes the

Actually if I remember rightly the problem is mathematically insoluble

> The application can avoid the wrong file problem by zeroing out data
> before releasing it to the OS to reallocate.

When you zero out the data what order do you want those writes in relative
to the rename

> An async fsync allows me to issue multiple fsyncs and then wait for
> all of them to complete, hopefully in the same framework that I would
> do async I/O (but that's an argument for another day).

Ok.. right that makes more sense. So you actually want 'begin_fsync' and
'wait_fsync_all' type stuff

>    Doing reliabile transactions on disk is a hard problem. That is why oracle
>    and friends have spent many man years of research on this kind of problem. 
>    Current unix mailers do the smoke mirrors and prayer bit to reduce the
>    probability a little that is all, regardless of fs and os.
> 
> Isn't the point of the operating system to try to make it as easy as
> possible to do these things correctly?

The OS doesnt have enough information. To do transactions you must know the
entire material that corresponds to the transaction and bound it. That isnt
something the kernel has the knowledge about.

The job of the OS is to make the simple things easy, and the hard possible.
Not to burden the simple with the cost of the hard. That why the chattr +S
is such a nice solution in many ways

Alan

Re: Linux 2.4.7-ac1 PNP Oops on shutdown

[Alan Cox]

> 2.4.7-ac1 oopses reproduceably during every shutdown. As far as I can tell,
> 2.4.6-ac5 didn't exhibit this behaviour.

>From the trace that looks what I would expect

> >>EIP; c0112b5d <complete+1d/a0>   <=====
> Trace; c011792d <complete_and_exit+d/20>
> Trace; c01dde51 <pnp_dock_thread+d1/e0>
> Trace; c01054c8 <kernel_thread+28/40>
> Code;  c0112b5d <complete+1d/a0>
> 00000000 <_EIP>:
> Code;  c0112b5d <complete+1d/a0>   <=====
>    0:   8b 03                     mov    (%ebx),%eax   <=====

Its oopsing in the complete_and_exit changes killing the PnP docking thread.

A quick look over the code and I have to admit I don't see why that happened
I'll ponder it later

Re: VIA KT133A / athlon / MMX

[Alan Cox]

> have not found clear answer on the different threads about this topic.
> As I understand, this problem does not exist on every athlon but only on
> some which work with the VIA KT133 chipset ? Right ?

Its heavily tied to certain motherboards. Some people found a better PSU
fixed it, others that altering memory settings helped. And in many cases,
taking it back and buying a different vendors board worked.

> 	Anyway, feel free to ask me more information if needed and please,
> CC'ed me personally the answers/comments because I'm not subscribed to
> the LKML.

http://www.linuxhardware.org/article.pl?sid=01/06/06/1821202&mode=thread

gives a good feel for the current state of play

Re: VIA KT133A / athlon / MMX

[Chris Wedgwood]

On Fri, Jul 27, 2001 at 09:19:21PM +0100, Alan Cox wrote:

    Its heavily tied to certain motherboards. Some people found a
    better PSU fixed it, others that altering memory settings
    helped. And in many cases, taking it back and buying a different
    vendors board worked.

Does anyone know *why* stuff breaks? surely VIA do as they have a fix
for (some, all?) cases of breakage?

My guess is its some kind of timing or near-miss on a signal edge, and
the bios changes relax things so you don't miss whatever it was you
missed before.



  --cw

Re: VIA KT133A / athlon / MMX

[Alan Cox]

> On Fri, Jul 27, 2001 at 09:19:21PM +0100, Alan Cox wrote:
>     Its heavily tied to certain motherboards. Some people found a
>     better PSU fixed it, others that altering memory settings
>     helped. And in many cases, taking it back and buying a different
>     vendors board worked.
> 
> Does anyone know *why* stuff breaks? surely VIA do as they have a fix
> for (some, all?) cases of breakage?

At the moment the big problem is I don't have enough reliable info to
see patterns that I can give to VIA for study. VIAs fixes for board problems
are for the fifo problem normally seen with the 686B and SB Live but
sometimes in other cases.

(and it seems also we have a few via + promise weirdnesses on all sorts of
 boards not yet explained)

Re: VIA KT133A / athlon / MMX

[Paul G. Allen]

I meant to send this to the list, but sent it straight to Alan instead.

PGA

"Paul G. Allen" wrote:
> 
> Alan Cox wrote:
> >
> 
> [SNIP]
> >
> > (and it seems also we have a few via + promise weirdnesses on all sorts of
> >  boards not yet explained)
> 
> I happen to have one of these boards. I was rather upset with it because it would lock Linux several times a day, especially while playing games. This is part
> of what drove me to purchase the K7 Thunder I now have and put the Asus A7V133 on the shelf.
> 
> Is there anything I can do that might help track down the problem(s)? I still have the board. In fact, it is a complete system less the SB Live! and GeForce 3
> that I relocated to my K7 Thunder, and it's running a Duron 750. (I also have a second system with a SB Live! and Athlon 1.2, but I'd have to beg my wife for
> its use. ;)
> 
> PGA
> 


-- 
Paul G. Allen
UNIX Admin II/Programmer
Akamai Technologies, Inc.
www.akamai.com
Work: (858)909-3630
Cell: (858)395-5043

Re: VIA KT133A / athlon / MMX

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 2832 bytes --]

Hi Alan,

as I stumbled across the K7 KT133 MMX problem, let me report my failure.
MSI K7 Turbo Ver. 3 (BIOS 2.8). K7 1.2GHz. 256MB SDRam, tested fine by
memtes86 2.7.

The thing would Oops or just hand at random places at the boot process if
compiled with K7 optimization.

On Fri, Jul 27, 2001 at 09:40:09PM +0100, Alan Cox wrote:
> > On Fri, Jul 27, 2001 at 09:19:21PM +0100, Alan Cox wrote:
> >     Its heavily tied to certain motherboards. Some people found a
> >     better PSU fixed it, 

PSU = Power supply? 300W should be fine IMHO.

> >     others that altering memory settings helped.

Did not. Board allows you to set CL 3 which won't help (and I guess the SPDs
read out 3 anyway) and to turn off some PCI features which does not
help either.
It does also allow you to increase mainboard speed and multiplier but not
decrease!

> >     And in many cases, taking it back and buying a different
> >     vendors board worked.

The best option most probably.

> > Does anyone know *why* stuff breaks? surely VIA do as they have a fix
> > for (some, all?) cases of breakage?
> 
> At the moment the big problem is I don't have enough reliable info to
> see patterns that I can give to VIA for study.

Well, I did some testing, like reordering the MMX instructions, only using 4
instead of 8 registers, ... to no avail.
It all came down to replacing movntq with movq and the thing magically works.
Looks like the writes just get lost otherwise. (Maybe the sfence is just not
effective? But that would be a CPU bug, not a mainboard one.)

> VIAs fixes for board problems
> are for the fifo problem normally seen with the 686B and SB Live but
> sometimes in other cases.

It also has the 686b southbridge bug, but I believe the workaound works.

> (and it seems also we have a few via + promise weirdnesses on all sorts of
>  boards not yet explained)

No Promise involved here, fortunately.
garloff@gum09:~ $ /sbin/lspci
00:00.0 Host bridge: VIA Technologies, Inc. VT8363/8365 [KT133/KM133] (rev 03)
00:01.0 PCI bridge: VIA Technologies, Inc. VT8363/8365 [KT133/KM133 AGP]
00:07.0 ISA bridge: VIA Technologies, Inc. VT82C686 [Apollo Super South] (rev 40)
00:07.1 IDE interface: VIA Technologies, Inc. Bus Master IDE (rev 06)
00:07.2 USB Controller: VIA Technologies, Inc. UHCI USB (rev 16)
00:07.3 USB Controller: VIA Technologies, Inc. UHCI USB (rev 16)
00:07.4 Host bridge: VIA Technologies, Inc. VT82C686 [Apollo Super ACPI] (rev 40)
00:07.5 Multimedia audio controller: VIA Technologies, Inc. AC97 Audio Controller (rev 50)
[...]

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE GmbH, Nuernberg, DE                                SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: VIA KT133A / athlon / MMX

[David Lang]

I have a 1u box at my des that has two MSI boards in it with 1.2G athlons.
at the moment they are both running 2.4.5 (athlon optimized), one box has
no problems at all while the other dies (no video, no keyboard, etc)
within an hour of being booted.

systems have no sound enabled, 512MB ram, 20G ata100 drives. D-Link quad
fast ethernet cards.

if you have any patch you would like me to test on these boxes let me know
(I am arranging to ship this one and three others like it that each have
one working and one failing system in them back to the factory to get the
MLB swapped out on the one that is failing.

David Lang


 On Fri, 27 Jul 2001, Alan Cox wrote:

> Date: Fri, 27 Jul 2001 21:40:09 +0100 (BST)
> From: Alan Cox <alan@lxorguk.ukuu.org.uk>
> To: cw@f00f.org
> Cc: alan@lxorguk.ukuu.org.uk, ppeiffer@free.fr, linux-kernel@vger.kernel.org
> Subject: Re: VIA KT133A / athlon / MMX
>
> > On Fri, Jul 27, 2001 at 09:19:21PM +0100, Alan Cox wrote:
> >     Its heavily tied to certain motherboards. Some people found a
> >     better PSU fixed it, others that altering memory settings
> >     helped. And in many cases, taking it back and buying a different
> >     vendors board worked.
> >
> > Does anyone know *why* stuff breaks? surely VIA do as they have a fix
> > for (some, all?) cases of breakage?
>
> At the moment the big problem is I don't have enough reliable info to
> see patterns that I can give to VIA for study. VIAs fixes for board problems
> are for the fifo problem normally seen with the 686B and SB Live but
> sometimes in other cases.
>
> (and it seems also we have a few via + promise weirdnesses on all sorts of
>  boards not yet explained)
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: VIA KT133A / athlon / MMX

[Kurt Garloff]

[-- Attachment #1.1: Type: text/plain, Size: 1327 bytes --]

Hi,

On Fri, Jul 27, 2001 at 05:23:07PM -0700, David Lang wrote:
> I have a 1u box at my des that has two MSI boards in it with 1.2G athlons.
> at the moment they are both running 2.4.5 (athlon optimized), one box has
> no problems at all while the other dies (no video, no keyboard, etc)
> within an hour of being booted.

Somebody told he had the same MoBo already replaced a couple of times ...

> if you have any patch you would like me to test on these boxes let me know

Well, no kernel patches.
But some program which does the K7 optmizied copies and zeroing in userspace.
(Attached)

Strange enough it succeeds on the machine that fails to boot a K7 optimized
kernel. 
So I'm puzzled now. 
Seems we can trigger problems in kernelspace that we can't have in userspace?
Some problem with non-serialization if an interrupt occurs or something
esoteric like this?

> (I am arranging to ship this one and three others like it that each have
> one working and one failing system in them back to the factory to get the
> MLB swapped out on the one that is failing.

Good luck!
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE GmbH, Nuernberg, DE                                SCSI, Security

[-- Attachment #1.2: test_movntq.c --]
[-- Type: text/plain, Size: 4336 bytes --]

/* test_movntq.c 
 * Program that tests the K7 optimized routines for copying 
 * and zeroing pages (which fail on some MoBos in the kernel).
 * gcc -O2 -Wall -g -fomit-frame-pointer -o test_movntq test_movntq.c
 * and run on AMD K7!
 * (c) Kurt Garloff <garloff@suse.de>, 2001-07-28, GNU GPL
 */

#include <stdio.h>
#include <unistd.h>
#include <malloc.h>
#include <stdlib.h>

#define PAGE_SIZE 4096
#define NR_TESTS 4096

void * fpu_ctx;

double c;
void trigger_fpu ()
{

	double a = 4.3;
	double b = rand()/ (float)RAND_MAX;
	c = a/b;
}

void movntq_copy_page0 (void* to, void* from)
{
	//void *d0, *d1;
	//printf ("%p <- %p\n", to, from);
	asm volatile (
		      "\n\t   prefetch (%0)"
		      "\n\t   prefetch 64(%0)"
		      "\n\t   prefetch 128(%0)"
		      "\n\t   prefetch 192(%0)"
		      "\n\t   fxsave (%3)"
		      "\n\t   prefetch 256(%0)"
		      "\n\t   movl %2, %%ecx"
		      "\n\t   fnclex"
		      "\n\t1: prefetch 320(%0)"
		      "\n\t   movq (%0),%%mm0"
		      "\n\t   movntq %%mm0,(%1)"
		      "\n\t   movq 8(%0),%%mm1"
		      "\n\t   movntq %%mm1,8(%1)"
		      "\n\t   movq 16(%0),%%mm2"
		      "\n\t   movntq %%mm2,16(%1)"
		      "\n\t   movq 24(%0),%%mm3"
		      "\n\t   movntq %%mm3,24(%1)"
		      "\n\t   movq 32(%0),%%mm4"
		      "\n\t   movntq %%mm4,32(%1)"
		      "\n\t   movq 40(%0),%%mm5"
		      "\n\t   movntq %%mm5,40(%1)"
		      "\n\t   movq 48(%0),%%mm6"
		      "\n\t   movntq %%mm6,48(%1)"
		      "\n\t   movq 56(%0),%%mm7"
		      "\n\t   movntq %%mm7,56(%1)"
		      /*"\n\t   sfence"*/
		      "\n\t   addl $64,%0"
		      "\n\t   addl $64,%1"
		      "\n\t   loop 1b"
		      "\n\t   movl $5, %%ecx"
		      "\n\t2: movq (%0),%%mm0"
		      "\n\t   movntq %%mm0,(%1)"
		      "\n\t   movq 8(%0),%%mm1"
		      "\n\t   movntq %%mm1,8(%1)"
		      "\n\t   movq 16(%0),%%mm2"
		      "\n\t   movntq %%mm2,16(%1)"
		      "\n\t   movq 24(%0),%%mm3"
		      "\n\t   movntq %%mm3,24(%1)"
		      "\n\t   movq 32(%0),%%mm4"
		      "\n\t   movntq %%mm4,32(%1)"
		      "\n\t   movq 40(%0),%%mm5"
		      "\n\t   movntq %%mm5,40(%1)"
		      "\n\t   movq 48(%0),%%mm6"
		      "\n\t   movntq %%mm6,48(%1)"
		      "\n\t   movq 56(%0),%%mm7"
		      "\n\t   movntq %%mm7,56(%1)"
		      "\n\t   addl $64,%0"
		      "\n\t   addl $64,%1"
		      "\n\t   loop 2b"
		      "\n\t   sfence"
		      "\n\t   fxrstor (%3) \n"
		      :
		      : "r" (from), "r" (to), "i" (PAGE_SIZE/64 - 5), "r" (fpu_ctx)
		      : "memory", "ecx" );
};


void movntq_zero_page0 (void* to)
{
	//void *d0;
	//printf ("%p <- 0\n", to);
	asm volatile (
		      "\n\t   fxsave (%2)"
		      "\n\t   movl %1, %%ecx"
		      "\n\t   fnclex"
		      "\n\t   pxor %%mm0, %%mm0"
		      "\n\t1: "
		      "\n\t   movntq %%mm0,(%0)"
		      "\n\t   movntq %%mm0,8(%0)"
		      "\n\t   movntq %%mm0,16(%0)"
		      "\n\t   movntq %%mm0,24(%0)"
		      "\n\t   movntq %%mm0,32(%0)"
		      "\n\t   movntq %%mm0,40(%0)"
		      "\n\t   movntq %%mm0,48(%0)"
		      "\n\t   movntq %%mm0,56(%0)"
		      /*"\n\t   sfence"*/
		      "\n\t   addl $64,%0"
		      "\n\t   loop 1b"
		      "\n\t   sfence"
		      "\n\t   fxrstor (%2) \n"
		      :
		      : "r" (to), "i" (PAGE_SIZE/64), "r" (fpu_ctx)
		      : "memory", "ecx");
}


void alloc_fpu_ctx ()
{
	fpu_ctx = (void*) memalign (256, 1024);
}

void fill_rand_page (void* mem)
{
	int* ptr = (int*) mem;
	do {
		*ptr = rand();
	} while (( (char*)(++ptr) - (char*)mem) < PAGE_SIZE);
}

void* memzero (void* mem, size_t ln)
{
	int* ptr = (int*)mem;
	int i = ln / sizeof(int);
	while (i--)
		if (*ptr++ != 0) return (void*)ptr;
	return 0;
}

int main ()
{
	void *b1, *b2, *b3; void* err; int i;
	srand (5);
	alloc_fpu_ctx ();
	trigger_fpu ();
	b3 = b1 = (void*) memalign (PAGE_SIZE, (NR_TESTS+1)*PAGE_SIZE);
	fill_rand_page (b1);
	for (i = 0; i < NR_TESTS; i++) {
		b2 = (void*) ((char*)b3 + PAGE_SIZE);
		movntq_copy_page0 (b2, b3);
		if (memcmp (b3, b2, PAGE_SIZE)) {
			printf ("Error (%i)!\n", i);
			exit (1);
		}
		movntq_zero_page0 (b3);
		if ((err = memzero (b3, PAGE_SIZE))) {
			printf ("Error! (%i) %p\n", i, err);
			exit (2);
		}
		b3 = b2;
	}
	free (b1);
	free (fpu_ctx);
	return 0;
}
			      

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: VIA KT133A / athlon / MMX

[Victor Julien]

Do these problems also affect Durons? I have a MSI K7T Turbo-R with Via
KT133A and I have nog problems at all. I even run my Duron 600 at 866(!)
(6,5 * 133) for several months now. Now I wonder if I could get problems
when i upgrade to a tbird at 1,4 ghz. I have a 300 PSU.

Victor

Re: VIA KT133A / athlon / MMX

[J. Dow]

From: "Kurt Garloff" <garloff@suse.de>

On Fri, Jul 27, 2001 at 05:23:07PM -0700, David Lang wrote:
> I have a 1u box at my des that has two MSI boards in it with 1.2G athlons.
> at the moment they are both running 2.4.5 (athlon optimized), one box has
> no problems at all while the other dies (no video, no keyboard, etc)
> within an hour of being booted.

Somebody told he had the same MoBo already replaced a couple of times ...

Kurt, et al, I have been following this VIA vs Linux thing for some time
now. (My "big machine" is an Athlon based system. So it interests me.)
Comments have been made about the size of power supply needed to keep these
systems happy with 400 watts coming up in discussions frequently. But if you
pause to think on it a few minutes you begin to wonder about this concept.
The RAM runs at about 3.3 volts. The CPU core runs at about 1.7v (in my case.)
So both of these are running off of power supplies on the motherboards that
take the 5 volts down to something reasonable. If the problem is inadequate
power supply AND it is more of a problem with some motherboards than others
I look for the volts. Where are the losses which could cause this. One source
is the connector from the power supply to the motherboard. (This was a chronic
problem with A2000s, for example.) I don't see newer style connectors that
have less contact resistance on any systems. That is probably a factor. Since
the problem is greater with some boards than others I suspect that the
auxilliary power sipplies on the motherboards are better for some boards than
for others. Somebody with hardware access to a sufficient variety of mother-
boards should survey this. Do they all use exactly the same power supply parts?

Another issue is the speed of these systems. And the Athlon problem seems to
peak when driving the various buses at their peaks. RF crosstalk is an issue
that a lot of digital designers claim to understand when they design (and
model) their circuits. Now, I built my first circuit analysis program back
in about 1975. Results of that work fly on GPS satellites today. Since it was
MY program I used for design I was acutely aware of its deficiencies as well
as the modeling deficiencies. At some point in the analysis you cut a corner
or two in order to make the calculations tractable. You do not manage to get
all the "strays" into the models. What I ams saying is that board layout is
another area where problems may exist.

These are not thigs software settings in the VIA chips can cure. On another
mailinglist catering to developers for very exotic video cards some problems
with the latest INTEL based motherboards are appearing. (DigiSuite:E and its
kith and kin drive the PCI bus very hard.) I suspect we have a situation not
properly anticipated in modeling the backplanes on all these boards. And until
the designers can wrap their minds around the entire problem the software
solution may simply be, in the words of an old philospher, "Slow down! You move
too fast." At the same time someone with suitable test equipment needs to look
for voltage glitches out of the motherboard regulators and we need to develop
software tools for "forcing" the suspected crosstalk and ideally characterising
it with regards to data passing on the bus at the time of the bad data transfer.
The software based fixes seem to be shooting at black cats in a coal mine
without a flashlight or IR goggles.

{^_^}    Joanne Dow, jdow@earthlink.net

Re: VIA KT133A / athlon / MMX

[Alan Cox]

> I have a 1u box at my des that has two MSI boards in it with 1.2G athlons.
> at the moment they are both running 2.4.5 (athlon optimized), one box has
> no problems at all while the other dies (no video, no keyboard, etc)
> within an hour of being booted.

Same bios, same bios settings ?

lspci -vxx on both show the same settings ?

Re: VIA KT133A / athlon / MMX

[David Lang]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1006 bytes --]

I have not had a chance to examine all the BIOS settings but attached are
the lspci -vxx for 10 different systems with identical hardware configs.
the ones that end in .good have given me no problems, the three ending in
.bad die after a while. the framewall-b system dies with all LEDs on the
network card lit, all others die with no LEDs on.

David Lang

On Sat, 28 Jul 2001, Alan Cox wrote:

> Date: Sat, 28 Jul 2001 13:47:40 +0100 (BST)
> From: Alan Cox <alan@lxorguk.ukuu.org.uk>
> To: dlang@diginsite.com
> Cc: alan@lxorguk.ukuu.org.uk, cw@f00f.org, ppeiffer@free.fr,
>      linux-kernel@vger.kernel.org
> Subject: Re: VIA KT133A / athlon / MMX
>
> > I have a 1u box at my des that has two MSI boards in it with 1.2G athlons.
> > at the moment they are both running 2.4.5 (athlon optimized), one box has
> > no problems at all while the other dies (no video, no keyboard, etc)
> > within an hour of being booted.
>
> Same bios, same bios settings ?
>
> lspci -vxx on both show the same settings ?
>


[-- Attachment #2: Type: APPLICATION/octet-stream, Size: 71680 bytes --]

Re: VIA KT133A / athlon / MMX

[Gav]

On Friday 27 July 2001 20:40, Alan Cox wrote:

> > On Fri, Jul 27, 2001 at 09:19:21PM +0100, Alan Cox wrote:
> >     Its heavily tied to certain motherboards. Some people found a
> >     better PSU fixed it, others that altering memory settings
> >     helped. And in many cases, taking it back and buying a different
> >     vendors board worked.
> >
> > Does anyone know *why* stuff breaks? surely VIA do as they have a fix
> > for (some, all?) cases of breakage?
>
> At the moment the big problem is I don't have enough reliable info to
> see patterns that I can give to VIA for study. VIAs fixes for board
> problems are for the fifo problem normally seen with the 686B and SB Live
> but sometimes in other cases.
>
> (and it seems also we have a few via + promise weirdnesses on all sorts of
>  boards not yet explained)

Just FYI, I've been running 2.4.7-pre6 for a few weeks on a Abit-KT7-a 
(hpt370) that uses the KT133/VIA chipset, with a 1.33Ghz Athlon and the 
kernel compiled for an Athlon. 

The machine is now rock solid. I've given it the usual tests, k7burn for 5 
hours, cp'ing 30G+ across drives a few times etc, and all is good.

The broken sound (crackle/pop) with my SB128PCI (same problem as SBLive) 
still didn't go away though, but enabling PCI DRAM PREFETCH on the VT8363 
Bus-PCI Bridge does cure it. This took me a while to find as I can't set this 
in my bios, but powertweak came to the rescue.

While DRAM Prefetch is supposed to be an option to increase performance, my 
sound is totally unusable without this set. I've heard numerous people 
explain the same problem and it would be interesting to find out if this 
cures their sound troubles too. If this is the case, is this something that 
belongs in quirks, or is it too hardware specific? and would enabling this by 
default hurt anything anyway? Or is this just masking the underlaying problem 
?

-- Regards, Gavin Baker

Re: VIA KT133A / athlon / MMX

[Mike Frisch]

On Sun, Jul 29, 2001 at 04:03:29AM +0000, Gav wrote:
> The machine is now rock solid. I've given it the usual tests, k7burn for 5 
> hours, cp'ing 30G+ across drives a few times etc, and all is good.

Sorry to jump in here, but where can I get "k7burn"?  I've searched on
google.com for it and cannot find any reference.  I am running 2.4.7-ac2
(with Athlon optimizations) with an AMD T-Bird 1.2GHz on an ASUS A7A266
and it appears quite stable.  I would like to see how it fares with this
burn-in program you speak of.

Thanks,

Mike.

Re: VIA KT133A / athlon / MMX

[Steffen Persvold]

Gav wrote:
> Just FYI, I've been running 2.4.7-pre6 for a few weeks on a Abit-KT7-a
> (hpt370) that uses the KT133/VIA chipset, with a 1.33Ghz Athlon and the
> kernel compiled for an Athlon.
> 
> The machine is now rock solid. I've given it the usual tests, k7burn for 5
> hours, cp'ing 30G+ across drives a few times etc, and all is good.
> 
> The broken sound (crackle/pop) with my SB128PCI (same problem as SBLive)
> still didn't go away though, but enabling PCI DRAM PREFETCH on the VT8363
> Bus-PCI Bridge does cure it. This took me a while to find as I can't set this
> in my bios, but powertweak came to the rescue.
> 
> While DRAM Prefetch is supposed to be an option to increase performance, my
> sound is totally unusable without this set. I've heard numerous people
> explain the same problem and it would be interesting to find out if this
> cures their sound troubles too. If this is the case, is this something that
> belongs in quirks, or is it too hardware specific? and would enabling this by
> default hurt anything anyway? Or is this just masking the underlaying problem
> ?

Hmm, I think "DRAM Prefetch" is the one you _don't_ want to turn on, because (and correct
me if i'm wrong) it's causing all the problems with the IDE controller (data trashing).

Regards,
-- 
  Steffen Persvold               Systems Engineer
  Email : mailto:sp@scali.no     Scali AS (http://www.scali.com)
  Tlf   : (+47) 22 62 89 50      Olaf Helsets vei 6
  Fax   : (+47) 22 62 89 51      N-0621 Oslo, Norway

Re: VIA KT133A / athlon / MMX

[Maciej Zenczykowski]

> Hmm, I think "DRAM Prefetch" is the one you _don't_ want to turn on, because (and correct
> me if i'm wrong) it's causing all the problems with the IDE controller (data trashing).

I think it was IDE Prefetch that should be off (I had this problem on a
AMD 486DX4-133 with Award Bios, turning it on trashed the boot record in
minutes (and many other sectors on the disk too).

Anyone here care to give a link to that program to enable DRAM Prefetch?
My sister has a Duron 750w with VIA motherboard and music and sound pop on
any graphics changes, maybe this is it?

Regards,

Maciej Zenczykowski

Re: VIA KT133A / athlon / MMX

[Luigi Genoni]

I have this bios setting enabled, and no problems at all on two of my
athlons with VIA KT133A, kernel 2.4.7.

>From this full discussion comes out a big confusion.

For what I saw, many  VIA KT133A do work well, many other
give problems to their sysadmins. but the chipset is almost the same,
and the processors are quite the same (they are all athlon, I read no
bug reports about duron).

this is enought for me to get confused.

My production systems do use scsi disks, and i can understand
they donot have troubles (adaptec 2940, 2980 29160....).
But also the ones with IDE disks are working quite well (some using
ata33, others ata100).
I NEVER used DDRAM, just normal SDRAM 133 Mhz.

So I was thinking to FSB. All my systems with ide disks have 200MhzFSB,
(while my latest production systems do have 266 MhzFSB). Maybe a 266
MhzFSB is just too mutch stress
for some via chipset. But i see no clear logic when problems do appear,
or any big difference with systems that are rock solid.

lets' try to make a point to see a logic for those instabilities...

which kind of hardware bug is this, if the same chipset can work or not
depending  if you are lucky? or a full stock of chipset is buggy and
with a certain HW configuration you will see the bug or
what?

Luigi

On Mon, 30 Jul 2001, Maciej Zenczykowski wrote:

> > Hmm, I think "DRAM Prefetch" is the one you _don't_ want to turn on, because (and correct
> > me if i'm wrong) it's causing all the problems with the IDE controller (data trashing).
>
> I think it was IDE Prefetch that should be off (I had this problem on a
> AMD 486DX4-133 with Award Bios, turning it on trashed the boot record in
> minutes (and many other sectors on the disk too).
>
> Anyone here care to give a link to that program to enable DRAM Prefetch?
> My sister has a Duron 750w with VIA motherboard and music and sound pop on
> any graphics changes, maybe this is it?
>
> Regards,
>
> Maciej Zenczykowski
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: VIA KT133A / athlon / MMX

[Gav]

On Monday 30 July 2001 07:15, Steffen Persvold wrote:

> > While DRAM Prefetch is supposed to be an option to increase performance,
> > my sound is totally unusable without this set. I've heard numerous people
> > explain the same problem and it would be interesting to find out if this
> > cures their sound troubles too. If this is the case, is this something
> > that belongs in quirks, or is it too hardware specific? and would
> > enabling this by default hurt anything anyway? Or is this just masking
> > the underlaying problem ?
>
> Hmm, I think "DRAM Prefetch" is the one you _don't_ want to turn on,
> because (and correct me if i'm wrong) it's causing all the problems with
> the IDE controller (data trashing).

Obviously I can only comment on my own hardware but the machine has been used 
constantly since Thu Jul 12, its now Jul 30 and I havent had a single IDE 
related problem. 

As a hobby, i use the machine for DigitalVideo, and regularly grab 20-30GB 
from my capture card, then process it, which means the IDE bus gets a lot of 
use and seems to be an ideal situation for the data trashing problems to rear 
their ugly heads (no pun intended) but, as i said, I haven't seen any here.

DRAM Prefetch makes my sound usuable, as the VIA fixups for the SB cards do 
not work here, and for (at least) two other people who i have had email 
correspondancy with.

-- Regards, Gavin Baker

Re: VIA KT133A / athlon / MMX

[PEIFFER Pierre]

Chris Wedgwood a écrit :
> 
> On Fri, Jul 27, 2001 at 09:19:21PM +0100, Alan Cox wrote:
> 
>     Its heavily tied to certain motherboards. Some people found a
>     better PSU fixed it, others that altering memory settings
>     helped. And in many cases, taking it back and buying a different
>     vendors board worked.
> 
> My guess is its some kind of timing or near-miss on a signal edge, and
> the bios changes relax things so you don't miss whatever it was you
> missed before.
> 

Ok, after reading that, I've tried to see if my BIOS setting changes
were implicated or not. And I've found a winner:
Disabling option "Enhance Chip Performance" makes kernel K7-mmx routines
work fine. Enabling it causes the kernel crash at boot time... (And I
haved it enable)

FYI, according to the user's manual, enabling this option "set the north
bridge chipset timing parameters more aggressively providing higher
system performance" (Default value is 'disable'). I can't say more about
what it does exactly.

I don't know if this will help you to locate the problem, but at least,
Abit's users will be warned...

Thanks for your help !

	Pierre

Re: VIA KT133A / athlon / MMX

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

On Sat, Jul 28, 2001 at 01:29:04PM -0400, PEIFFER Pierre wrote:
> FYI, according to the user's manual, enabling this option "set the north
> bridge chipset timing parameters more aggressively providing higher
> system performance" (Default value is 'disable'). I can't say more about
> what it does exactly.

A lspci -vxxx of your northbridge with adn without the BIOS option will
reveal more.

Regards,
-- 
Kurt Garloff                   <kurt@garloff.de>         [Eindhoven, NL]
Physics: Plasma simulations  <K.Garloff@Phys.TUE.NL>  [TU Eindhoven, NL]
Linux: SCSI, Security          <garloff@suse.de>    [SuSE Nuernberg, DE]
 (See mail header or public key servers for PGP2 and GPG public keys.)

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: VIA KT133A / athlon / MMX

[PEIFFER Pierre]

[-- Attachment #1: Type: text/plain, Size: 437 bytes --]

Kurt Garloff a écrit :
> 
> A lspci -vxxx of your northbridge with adn without the BIOS option will
> reveal more.

In attached files are the result. I've only kept the (what I suppose to
be) northbridge info.
This doesn't tell me anything...

Note: both has been done after booting on  Mandrake-kernel 2.4.3 which
come with Mandrake distribution (i.e. with lot of patches and
options...) I don't know the impact on the result...

Pierre

[-- Attachment #2: lspci_opt_disable.txt --]
[-- Type: text/plain, Size: 1142 bytes --]

00:00.0 Host bridge: VIA Technologies, Inc.: Unknown device 0305 (rev 03)
	Subsystem: ABIT Computer Corp.: Unknown device a401
	Flags: bus master, medium devsel, latency 0
	Memory at d8000000 (32-bit, prefetchable) [size=64M]
	Capabilities: [a0] AGP version 2.0
	Capabilities: [c0] Power Management version 2
00: 06 11 05 03 06 00 10 a2 03 00 00 06 00 00 00 00
10: 08 00 00 d8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 7b 14 01 a4
30: 00 00 00 00 a0 00 00 00 00 00 00 00 00 00 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 17 a3 eb b4 02 00 10 10 c0 00 08 10 10 10 10 10
60: 03 aa 02 20 e6 d6 d6 c6 51 28 43 0d 08 3f 00 00
70: d4 88 cc 0c 0e 81 62 00 01 b4 19 02 00 00 00 00
80: 0f 40 00 00 c0 00 00 00 02 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00
a0: 02 c0 20 00 17 02 00 1f 00 00 00 00 2b 12 14 00
b0: 49 da 00 60 31 ff 80 05 67 00 00 00 00 00 00 00
c0: 01 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 03 03 00 22 00 00 00 00 00 00 00


[-- Attachment #3: lspci_opt_enable.txt --]
[-- Type: text/plain, Size: 1142 bytes --]

00:00.0 Host bridge: VIA Technologies, Inc.: Unknown device 0305 (rev 03)
	Subsystem: ABIT Computer Corp.: Unknown device a401
	Flags: bus master, medium devsel, latency 8
	Memory at d8000000 (32-bit, prefetchable) [size=64M]
	Capabilities: [a0] AGP version 2.0
	Capabilities: [c0] Power Management version 2
00: 06 11 05 03 06 00 10 a2 03 00 00 06 00 08 00 00
10: 08 00 00 d8 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 7b 14 01 a4
30: 00 00 00 00 a0 00 00 00 00 00 00 00 00 00 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 17 a3 eb b4 43 89 10 10 c0 00 08 10 10 10 10 10
60: 03 aa 02 20 e6 d6 d6 c6 45 28 43 0f 08 3f 00 00
70: d4 88 cc 0c 0e 81 62 00 01 b4 19 02 00 00 00 00
80: 0f 40 00 00 c0 00 00 00 02 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 32 00 00
a0: 02 c0 20 00 17 02 00 1f 00 00 00 00 2f 12 14 00
b0: 49 da 88 60 31 ff 80 05 67 00 00 00 00 00 00 00
c0: 01 00 02 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 03 03 00 22 00 00 00 00 00 91 06

Re: VIA KT133A / athlon / MMX

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 3265 bytes --]

Hi Pierre,

thanks for your info!

On Sat, Jul 28, 2001 at 06:00:11PM -0400, PEIFFER Pierre wrote:
> Kurt Garloff a écrit :
> In attached files are the result. I've only kept the (what I suppose to
> be) northbridge info.
> This doesn't tell me anything...

Me neither. I was hoping that only a bit differs. Unfortunately that's not
the case, so I need to have a look in the datasheet.
But those are not publically available :-(
Anybody having them?

> Note: both has been done after booting on  Mandrake-kernel 2.4.3 which
> come with Mandrake distribution (i.e. with lot of patches and
> options...) I don't know the impact on the result...

With a newer lspci you would have seen that 1106:0305 is VT8363/8365
[KT133/KM133].

I removed everything except for the differences. Underlined. Anybody able to
decode? Otherwise trying out all of them can get boring. (Well, I'd start
with 0x68, followed by 0x6b and 0xac  ...)

Working:

> 00:00.0 Host bridge: VIA Technologies, Inc.: Unknown device 0305 (rev 03)
> 	Subsystem: ABIT Computer Corp.: Unknown device a401
> 	Flags: bus master, medium devsel, latency 0
                                                  ^ This looks wrong to me.
> 	Memory at d8000000 (32-bit, prefetchable) [size=64M]
> 	Capabilities: [a0] AGP version 2.0
> 	Capabilities: [c0] Power Management version 2
> 00: 06 11 05 03 06 00 10 a2 03 00 00 06 00 00 00 00
                                              ^ Latency.
> 50: 17 a3 eb b4 02 00 10 10 c0 00 08 10 10 10 10 10
                  ^^ ^^
> 60: 03 aa 02 20 e6 d6 d6 c6 51 28 43 0d 08 3f 00 00
                              ^^       ^^
> a0: 02 c0 20 00 17 02 00 1f 00 00 00 00 2b 12 14 00
                                          ^^
> b0: 49 da 00 60 31 ff 80 05 67 00 00 00 00 00 00 00
            ^^
> f0: 00 00 00 00 00 03 03 00 22 00 00 00 00 00 00 00
                                                ^^ ^^

Buggy: (Own, buggy settings in parens)

> 00:00.0 Host bridge: VIA Technologies, Inc.: Unknown device 0305 (rev 03)
> 	Subsystem: ABIT Computer Corp.: Unknown device a401
> 	Flags: bus master, medium devsel, latency 8
                                                  ^ That's more reasonable.
> 	Memory at d8000000 (32-bit, prefetchable) [size=64M]
> 	Capabilities: [a0] AGP version 2.0
> 	Capabilities: [c0] Power Management version 2
> 00: 06 11 05 03 06 00 10 a2 03 00 00 06 00 08 00 00
                                              ^ Latency
> 50: 17 a3 eb b4 43 89 10 10 c0 00 08 10 10 10 10 10
                  ^^ ^^					(47 8d here)
> 60: 03 aa 02 20 e6 d6 d6 c6 45 28 43 0f 08 3f 00 00
                              ^^       ^^		(41 .. 21 here)
> a0: 02 c0 20 00 17 02 00 1f 00 00 00 00 2f 12 14 00
                                          ^^		(6b here)
> b0: 49 da 88 60 31 ff 80 05 67 00 00 00 00 00 00 00
            ^^						(22 here)
> f0: 00 00 00 00 00 03 03 00 22 00 00 00 00 00 91 06
                                                ^^ ^^	(00 00 here)

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE GmbH, Nuernberg, DE                                SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: VIA KT133A / athlon / MMX

[Daniela Engert]

Hi Kurt!

On Sun, 29 Jul 2001 22:28:30 +0200, Kurt Garloff wrote:

>Me neither. I was hoping that only a bit differs. Unfortunately that's not
>the case, so I need to have a look in the datasheet.
>But those are not publically available :-(
>Anybody having them?

Try to get a clue yourself from the WPCREDIT KT133 plugin (see below,
stripped down to the differing registers). Some differences look
suspicious to me...

>Working:

>> 00: 06 11 05 03 06 00 10 a2 03 00 00 06 00 00 00 00
>                                              ^ Latency.
>> 50: 17 a3 eb b4 02 00 10 10 c0 00 08 10 10 10 10 10
>                  ^^ ^^
>> 60: 03 aa 02 20 e6 d6 d6 c6 51 28 43 0d 08 3f 00 00
>                              ^^       ^^
>> a0: 02 c0 20 00 17 02 00 1f 00 00 00 00 2b 12 14 00
>                                          ^^
>> b0: 49 da 00 60 31 ff 80 05 67 00 00 00 00 00 00 00
>            ^^
>> f0: 00 00 00 00 00 03 03 00 22 00 00 00 00 00 00 00
                                                ^^ ^^

>Buggy: (Own, buggy settings in parens)

>> 00: 06 11 05 03 06 00 10 a2 03 00 00 06 00 08 00 00
>                                              ^ Latency
>> 50: 17 a3 eb b4 43 89 10 10 c0 00 08 10 10 10 10 10
>                  ^^ ^^				(47 8d here)
>> 60: 03 aa 02 20 e6 d6 d6 c6 45 28 43 0f 08 3f 00 00
>                              ^^       ^^		(41 .. 21 here)
>> a0: 02 c0 20 00 17 02 00 1f 00 00 00 00 2f 12 14 00
>                                          ^^		(6b here)
>> b0: 49 da 88 60 31 ff 80 05 67 00 00 00 00 00 00 00
>            ^^						(22 here)
>> f0: 00 00 00 00 00 03 03 00 22 00 00 00 00 00 91 06
>                                                ^^ ^^	(00 00 here)

PCR(PCI Configration Registers) Editor / WPCREDIT for WIN32
Copyright (c) 2000  H.Oda!

[COMMENT]=for HWup ng. members (Kx) edited by Guruad tnx to author
H.Oda!
[MODEL]=VT8363 (KT133)
[VID]=1106:VIA
[DID]=0305:Host to PCI Bridge

[54:7]=SDRAM Self-Refresh	0=disable   1=enable
[54:6]=Probe Next Tag State T1	0=disable   1=enable
[54:5]=High Priority DRAM Req.	0=disable   1=enable
[54:4]=Continuous DRAM Request	0=disable   1=enable
[54:3]=DRAM Speculative Read	0=disable   1=enable
[54:2]=PCI Master Pipeline Req. 0=disable   1=enable
[54:1]=PCI-to-CPU / CPU-to-PCI	0=disable   1=enable
[54:0]=Fast Write-to-Read	0=disable   1=enable

[55:0]=S2K Compensation CPU Halt0=disable   1=enable

[68:7]=SDRAM Open Page Control	0=precharge  1=remain act
[68:6]=Bank Page Control	0=same bank  1=different
[68:5]=(Reserved)
[68:4]=DRAM Data Latch Delay	0=Latch     1=Delay latch
[68:3]=EDO Test Mode		0=disable   1=enable
[68:2]=Burst Refresh(4 times)	0=disable   1=enable
[68:1]=System Frequency Divider 00= 66 MHz  01=100 MHz
[68:0]=10=133 MHz  11=Reserved

[6B:7]=Arbitration Parking Pol. 00=bus owner 01=CPU side
[6B:6]=10=AGP side  11=Reserved
[6B:5]=Fast Read to Write t-a	0=disable   1=enable
[6B:4]=(Reserved)
[6B:3]=MD Bus Second Level	0=Normal slew 1=More
[6B:2]=CAS Bus Second Level	0=Normal slew 1=More
[6B:1]=Virtual Channel-DRAM	0=disable   1=enable
[6B:0]=Multi-Page Open		0=disable   1=enable

[AC:7]=(Reserved)
[AC:6]=AGP Read Synchronization 0=disable   1=enable
[AC:5]=AGP Read Snoop DRAM P-W-B0=disable   1=enable
[AC:4]=GREQ# Priority		0=disable   1=enable
[AC:3]=2X Rate Supported	0=not	    1=supported
[AC:2]=LPR In-Order Access	0=not	    1=executed
[AC:1]=AGP Arbitration Parking	0=disable   1=enable
[AC:0]=AGP-PCI Master/CPU-PCI TC0=2T or 3T  1=1T

[B2:7]=GD/GBE/GDS, SBA/SBS Ctrl
[B2:6]=(Reserved)
[B2:5]=(Reserved)
[B2:4]=GD[31-16] Staggered Delay0=none	    1=1ns
[B2:3]=(Reserved)
[B2:2]=(Reserved)
[B2:1]=AGP Voltage		0=1.5V	    1=3.3V
[B2:0]=GDS Output Delay 	0=none	    1=0.4ns

[FE]=Back-Door Device ID
[FF]=Back-Door Device ID

'54'=BIU Control 00 RW
'55'=Debug (Do Not Program)
'68'=DRAM Control 00 RW
'6B'=DRAM Arbitration Control 01 RW
'AC'=AGP Control 00 RW
'B2'=AGP Pad Drive / Delay Control 00 RW
'FE..FF' Back-Door Device ID 0000 RW

Ciao,
  Dani

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Daniela Engert, systems engineer at MEDAV GmbH
Gräfenberger Str. 34, 91080 Uttenreuth, Germany
Phone ++49-9131-583-348, Fax ++49-9131-583-11

Re: VIA KT133A / athlon / MMX

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 1430 bytes --]

Hi Daniela,

On Mon, Jul 30, 2001 at 08:04:54AM +0200, Daniela Engert wrote:
> On Sun, 29 Jul 2001 22:28:30 +0200, Kurt Garloff wrote:
> 
> >Me neither. I was hoping that only a bit differs. Unfortunately that's not
> >the case, so I need to have a look in the datasheet.
> >But those are not publically available :-(
> >Anybody having them?
> 
> Try to get a clue yourself from the WPCREDIT KT133 plugin (see below,
> stripped down to the differing registers). Some differences look
> suspicious to me...

Hey thanks!

> [54:6]=Probe Next Tag State T1	0=disable   1=enable

Main suspect. (Should be 0)

> [54:0]=Fast Write-to-Read	0=disable   1=enable

Third candidate. (Should be 0)

> [68:4]=DRAM Data Latch Delay	0=Latch     1=Delay latch

Second candidate (Should be 1)

> [68:2]=Burst Refresh(4 times)	0=disable   1=enable

Fourth candidate (Should be 0?)

> [6B:5]=Fast Read to Write t-a	0=disable   1=enable

Should this one match 54:0 (third candidate)?

> [6B:1]=Virtual Channel-DRAM	0=disable   1=enable

Strange, why does this one differ between the configs.

OK, I'll come up with a kernel patches (driver/pci/quirks ...)
for people to test.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE GmbH, Nuernberg, DE                                SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: VIA KT133A / athlon / MMX

[Michael]

> > On Sun, 29 Jul 2001 22:28:30 +0200, Kurt Garloff wrote:
> > [54:6]=Probe Next Tag State T1	0=disable   1=enable
> 
> Main suspect. (Should be 0)

That's set in my stable kt133a system.
 
> > [54:0]=Fast Write-to-Read	0=disable   1=enable
> 
> Third candidate. (Should be 0)

as is this one.
 
> > [68:2]=Burst Refresh(4 times)	0=disable   1=enable
> 
> Fourth candidate (Should be 0?)

I set this one yesterday to see if it would trigger the problem, it
didn't :o/ Same with a few differences between my system and 0x6b, which
didn't either.

Out of curiosity, where are you getting the 'should be 0/1' details from?
-- 
Michael.
 

Re: VIA KT133A / athlon / MMX

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

Hi Michael,

thanks for your comments!

On Mon, Jul 30, 2001 at 03:15:38PM +0100, Michael wrote:
> > > On Sun, 29 Jul 2001 22:28:30 +0200, Kurt Garloff wrote:
> > > [54:6]=Probe Next Tag State T1	0=disable   1=enable
> > 
> > Main suspect. (Should be 0)
> 
> That's set in my stable kt133a system.

But did you experience problems at all with your kernel when compiled for
K7? Note that most (if not all) systems seem to work stable with K6 or PPro
optimized kernels.

> > > [54:0]=Fast Write-to-Read	0=disable   1=enable
> > 
> > Third candidate. (Should be 0)
> 
> as is this one.
>  
> > > [68:2]=Burst Refresh(4 times)	0=disable   1=enable
> > 
> > Fourth candidate (Should be 0?)
> 
> I set this one yesterday to see if it would trigger the problem, it
> didn't :o/ Same with a few differences between my system and 0x6b, which
> didn't either.
> 
> Out of curiosity, where are you getting the 'should be 0/1' details from?

Comparing the lspci -vxxx output of working and non-working systems.

You did no comment on the second candidate:

> [68:4]=DRAM Data Latch Delay  0=Latch     1=Delay latch

Second candidate (Should be 1)

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE GmbH, Nuernberg, DE                                SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: VIA KT133A / athlon / MMX

[Kurt Garloff]

[-- Attachment #1.1: Type: text/plain, Size: 689 bytes --]

Hi,

OK, patches for different bits are attached.
The patch does modify up to 4 bits, which is more than I would like to do in
the end. But you can easily disable some parts of it, if the full patch
proves to solve your trouble.
Please test!

It seemed to solved the trouble here on first sight (booting went further
then normal) but in the end did not turn out to solve the trouble here.
(Here means: MSI K7T Turbo (Ver.3) with AMD K7 1.2GHz.)

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE GmbH, Nuernberg, DE                                SCSI, Security

[-- Attachment #1.2: 247-viakt133.diff --]
[-- Type: text/plain, Size: 2250 bytes --]

--- linux-2.4.7.compile/drivers/pci/quirks.c.orig	Tue Jul 24 16:50:41 2001
+++ linux-2.4.7.compile/drivers/pci/quirks.c	Mon Jul 30 20:21:56 2001
@@ -160,6 +160,49 @@
 }
 
 /*
+ * KT133a will fsck up under some circumstances if Burst Refresh (4 times)
+ * is enabled or if data latch delay is disabled
+ * and we use the fast streaming K7 optimized zero_page
+ * and copy_page routines from arch/i386/lib/mmx.c 
+ * -- garloff@suse.de, 2001-07-30
+ */
+static void __init quirk_via_noburstrefresh(struct pci_dev *dev)
+{
+	u8 dram_ctrl;
+	pci_read_config_byte(dev, 0x68, &dram_ctrl);
+	if (dram_ctrl & 0x04 || !(dram_ctrl & 0x10)) {
+		if (dram_ctrl & 0x04)
+	  		printk(KERN_INFO "VIA KT133a: Disabling burst refresh.\n");
+		dram_ctrl &= ~0x04;
+		if (!(dram_ctrl & 0x10))
+	  		printk(KERN_INFO "VIA KT133a: Enabling data latch delay.\n");
+		dram_ctrl |= 0x10;
+		pci_write_config_byte(dev, 0x68, dram_ctrl);
+	}
+}
+
+/*
+ * KT133a will fsck up under some circumstances if Probe Next Tag State 
+ * T1 is set to 1 and we use the fast streaming K7 optimized zero_page
+ * and copy_page routines from arch/i386/lib/mmx.c 
+ * -- garloff@suse.de, 2001-07-30
+ */
+static void __init quirk_via_noprobenexttag(struct pci_dev *dev)
+{
+	u8 biu_ctrl;
+	pci_read_config_byte(dev, 0x54, &biu_ctrl);
+	if (biu_ctrl & 0x40 || biu_ctrl & 0x01) {
+		if (biu_ctrl & 0x40)
+			printk(KERN_INFO "VIA KT133a: Disabling probe next tag state T1.\n");
+		if (biu_ctrl & 0x01)
+			printk(KERN_INFO "VIA KT133a: Disabling fast write-to-read.\n");
+		biu_ctrl &= ~0x41;
+		pci_write_config_byte(dev, 0x54, biu_ctrl);
+	}
+}
+
+
+/*
  *	Natoma has some interesting boundary conditions with Zoran stuff
  *	at least
  */
@@ -452,6 +495,9 @@
 	{ PCI_FIXUP_FINAL,	PCI_VENDOR_ID_VIA,	PCI_DEVICE_ID_VIA_82C586_2,	quirk_via_irqpic },
 	{ PCI_FIXUP_FINAL,	PCI_VENDOR_ID_VIA,	PCI_DEVICE_ID_VIA_82C686_5,	quirk_via_irqpic },
 	{ PCI_FIXUP_FINAL,	PCI_VENDOR_ID_VIA,	PCI_DEVICE_ID_VIA_82C686_6,	quirk_via_irqpic },
+
+	{ PCI_FIXUP_FINAL,	PCI_VENDOR_ID_VIA,	PCI_DEVICE_ID_VIA_8363_0,	quirk_via_noburstrefresh },
+	{ PCI_FIXUP_FINAL,	PCI_VENDOR_ID_VIA,	PCI_DEVICE_ID_VIA_8363_0,	quirk_via_noprobenexttag },
 
 	{ 0 }
 };

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: VIA KT133A / athlon / MMX

[Gerbrand van der Zouw]

Hi,

Kurt Garloff wrote:

 > It seemed to solved the trouble here on first sight (booting went further
 > then normal) but in the end did not turn out to solve the trouble here.
 > (Here means: MSI K7T Turbo (Ver.3) with AMD K7 1.2GHz.)

from your lspci output I seem to have exactly the same system as you 
have. I tried your patch (247-viakt133.diff) and came up with the same 
result here: it seemed to come further than last time with only 
2.4.6ac5, but then it crashed anyway. If you know of any BIOS parameters 
  that might help for this mobo, please let me know. I could not 
identify a parameter that does the same as the "DRAM Prefetch" for Abit 
mobos.

Regards,

Gerbrand van der Zouw

Re: VIA KT133A / athlon / MMX

[Daniela Engert]

Hi Kurt!

On Mon, 30 Jul 2001 15:44:58 +0200, Kurt Garloff wrote:

Just for reference: these are the values taken from my main machine
(Epox EP8KTA2, VIA KT133) with the latest BIOS:

>> [54:6]=Probe Next Tag State T1	0=disable   1=enable
>Main suspect. (Should be 0)

Set to 1 here.

>> [54:0]=Fast Write-to-Read	0=disable   1=enable
>Third candidate. (Should be 0)

Set to 1 here.

>> [68:4]=DRAM Data Latch Delay	0=Latch     1=Delay latch
>Second candidate (Should be 1)

Set to 1 here.

>> [68:2]=Burst Refresh(4 times)	0=disable   1=enable
>Fourth candidate (Should be 0?)

Set to 0 here.

>> [6B:5]=Fast Read to Write t-a	0=disable   1=enable
>Should this one match 54:0 (third candidate)?

Set to 0 here.

>> [6B:1]=Virtual Channel-DRAM	0=disable   1=enable
>Strange, why does this one differ between the configs.

Set to 0 here.

Unfortunately, this machine doesn't run Linux...

Ciao,
  Dani

Re: ReiserFS / 2.4.6 / Data Corruption

[Alan Cox]

> Let us be a bit more precise here.  If you click on the help button when deciding whether to select
> that option it tells you not to do it.  What can you say about a distro that doesn't read the help
> buttons for the kernel options when configuring the kernel?  Shovelware?

The alternative was to disable it. Because at the time we had lots of good
evidence it didnt work reliably. Evidence backed up by the pile of later
Chris Mason patches.

Re: ReiserFS / 2.4.6 / Data Corruption

[Hans Reiser]

Alan Cox wrote:
> 
> > Let us be a bit more precise here.  If you click on the help button when deciding whether to select
> > that option it tells you not to do it.  What can you say about a distro that doesn't read the help
> > buttons for the kernel options when configuring the kernel?  Shovelware?
> 
> The alternative was to disable it. Because at the time we had lots of good
> evidence it didnt work reliably. Evidence backed up by the pile of later
> Chris Mason patches.
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Better to disable it than to cripple it.

By the way, how about considering the use of tests before redhat coders put stuff in the linux
kernel?  You know, if VFS changes actually got tested before users encountered things like Viro
breaking ReiserFS in 2.4.5, it would be nice.

At Namesys, like all normal software shops, we actually run a test suite before shipping code
externally.  We usually try to require that it be tested by at least one person in addition to the
code author.

It would catch things like your gcc problems.  Test suites don't catch everything, but they are
considered the responsible thing to do at most places.

Hans

Re: ReiserFS / 2.4.6 / Data Corruption

[Alan Cox]

> By the way, how about considering the use of tests before redhat coders put stuff in the linux
> kernel?  You know, if VFS changes actually got tested before users encountered things like Viro
> breaking ReiserFS in 2.4.5, it would be nice.
> At Namesys, like all normal software shops, we actually run a test suite before shipping code
> externally.  We usually try to require that it be tested by at least one person in addition to the
> code author.

*PLONK*

No doubt if Namesys ran test suites all the tail merging bug fiasco and the
directory/tree balance races wouldnt have happened.

Re: ReiserFS / 2.4.6 / Data Corruption

[Hans Reiser]

Alan Cox wrote:
> 
> > By the way, how about considering the use of tests before redhat coders put stuff in the linux
> > kernel?  You know, if VFS changes actually got tested before users encountered things like Viro
> > breaking ReiserFS in 2.4.5, it would be nice.
> > At Namesys, like all normal software shops, we actually run a test suite before shipping code
> > externally.  We usually try to require that it be tested by at least one person in addition to the
> > code author.
> 
> *PLONK*
> 
> No doubt if Namesys ran test suites all the tail merging bug fiasco and the
> directory/tree balance races wouldnt have happened.
Our test suites need much improvement, but we do have them and use them.  Can you say the same?

Hans

Re: ReiserFS / 2.4.6 / Data Corruption

[Chris Mason]

On Saturday, July 28, 2001 11:36:33 AM +0400 Hans Reiser <reiser@namesys.com>
wrote:

> Alan Cox wrote:
>> 
>> No doubt if Namesys ran test suites all the tail merging bug fiasco and the
>> directory/tree balance races wouldnt have happened.
> Our test suites need much improvement, but we do have them and use them.
> Can you say the same?

He's already described some of the testing they do.  I would suggest there
are better ways to use l-k bandwidth than picking a fight with redhat,
especially on topics that have already been beaten to death.  

Alan, thanks for helping to test the reiserfs patches we've been sending to
in the ac tree, we do appreciate it.

-chris

Re: Linx Kernel Source tree and metrics

[Alan Cox]

> If this happens, I'll update it to the latest source (whatever happens to be available at that time). If it doesn't, I'll update it anyway and just bite the
> bullet and upload the data to a server with more bandwidth.

bzip2 -9 is your friend for repetetive data 8)

Re: Linx Kernel Source tree and metrics

[Paul G. Allen]

Alan Cox wrote:
> 
> > If this happens, I'll update it to the latest source (whatever happens to be available at that time). If it doesn't, I'll update it anyway and just bite the
> > bullet and upload the data to a server with more bandwidth.
> 
> bzip2 -9 is your friend for repetetive data 8)

Isn't that the truth, especially for this much text (I bet it'll compress real nice :)

PGA

-- 
Paul G. Allen
UNIX Admin II/Programmer
Akamai Technologies, Inc.
www.akamai.com
Work: (858)909-3630
Cell: (858)395-5043

Re: Linx Kernel Source tree and metrics

[Paul G. Allen]

Please, no more wgets on my poor limited bandwidth (256Kbit uplink) web server. Next week I will have a fatter pipe and you can D/L the whole dir if you want.
(Though it would be better if you let me compress it and put it on a ftp server).

Thank you for your support. ;)

PGA

-- 
Paul G. Allen
UNIX Admin II/Programmer
Akamai Technologies, Inc.
www.akamai.com
Work: (858)909-3630
Cell: (858)395-5043

Re: binary modules (was Re: ReiserFS / 2.4.6 / Data Corruption)

[Alan Cox]

> The right answer for vendors who want to ship binary modules is to
> ship an Open Source interface layer which shields the vendor from
> kernel drift (since users will be able to build the interface layer if
> they need to, without waiting for the vendor).

As people have seen from vmware and from the ever growing piles of 
nvidia crashes the truth about binary modules in general even with glue is
pain and suffering.

Veritas have some good Linux people though, and while I'm sad they won't
open source the core of veritas they do at least appear to have the
knowledgebase to do a good job

Re: binary modules (was Re: ReiserFS / 2.4.6 / Data Corruption)

[Matthew Gardiner]

On Sunday 29 July 2001 07:08, Alan Cox wrote:
> > The right answer for vendors who want to ship binary modules is to
> > ship an Open Source interface layer which shields the vendor from
> > kernel drift (since users will be able to build the interface layer if
> > they need to, without waiting for the vendor).
>
> As people have seen from vmware and from the ever growing piles of
> nvidia crashes the truth about binary modules in general even with glue is
> pain and suffering.
>
> Veritas have some good Linux people though, and while I'm sad they won't
> open source the core of veritas they do at least appear to have the
> knowledgebase to do a good job

1. With the file system, why not charge for commercial use?
2. Regards to hardware manufacturers, what have the got to lose from 
publishing the specs? nothing.

Matthew Gardiner
-- 
WARNING:

This email was written on an OS using the viral 'GPL' as its license.

Please check with Bill Gates before continuing to read this email/posting.

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

Re: binary modules (was Re: ReiserFS / 2.4.6 / Data Corruption)

[Chris Wedgwood]

On Sun, Jul 29, 2001 at 10:24:11PM +1200, Matthew Gardiner wrote:

    1. With the file system, why not charge for commercial use?

Maybe they will... but it's not something they could do under the GPL.

    2. Regards to hardware manufacturers, what have the got to lose from
       publishing the specs? nothing.

Many manufactures will claim otherwise... for some hardware products,
the useful life-cycles is only six months, if you can't make money
within that period of time, the product never will --- so there are
arguments for keeping things vague for just a little while.

Also, some hardware vendors cannot release specifications because they
don't own all the IP here either (see my earlier comments) or are part
of some kind of cartel/consortium which is overrun by labatomized
lawyers, the DVD people for example.




  --cw

Re: binary modules (was Re: ReiserFS / 2.4.6 / Data Corruption)

[Florian Weimer]

Matthew Gardiner <kiwiunixman@yahoo.co.nz> writes:

> 2. Regards to hardware manufacturers, what have the got to lose from 
> publishing the specs? nothing.

Some vendors do not have proper specs or have received them under NDA
themselves.

-- 
Florian Weimer 	                  Florian.Weimer@RUS.Uni-Stuttgart.DE
University of Stuttgart           http://cert.uni-stuttgart.de/
RUS-CERT                          +49-711-685-5973/fax +49-711-685-5898

Re: make rpm

[Alan Cox]

> Alan Cox <alan@lxorguk.ukuu.org.uk> said:
> > I've been meaning to do this one for a while and I now have it working so
> > that with my current -ac kernel working tree I can type
> > 
> > 	make rpm
> > 
> > and out puts kernel-2.4.7ac3-1.i386.rpm
> 
> Great idea!
> 
> Just the bunch of "echo this or that" is ugly as sin... why not a
> kernel.spec template that gets its version &c substituted by sed(1) or
> something?

Well for one because its easier to hack on at the moment. I still need to
finish up packing the right pieces, and also checking if the user
has an /sbin/installkernel and also if they are not using GRUB need to then
rerun lilo

Alan

Re: binary modules (was Re: ReiserFS / 2.4.6 / Data Corruption)

[Richard Gooch]

Alan Cox writes:
> > The right answer for vendors who want to ship binary modules is to
> > ship an Open Source interface layer which shields the vendor from
> > kernel drift (since users will be able to build the interface layer if
> > they need to, without waiting for the vendor).
> 
> As people have seen from vmware and from the ever growing piles of
> nvidia crashes the truth about binary modules in general even with
> glue is pain and suffering.

Sure. If you load a binary module (shim layer or not), you don't get
community support. So vendors are digging their own shitpile by
shipping binary-only drivers. I just don't see the need to shove them
in the back while they do it.

Besides, if someone can make a lot of money shipping binary drivers,
then they can afford the support costs, so it may well be a viable
revenue model for them (at the very least, programmers need to eat
too).

> Veritas have some good Linux people though, and while I'm sad they
> won't open source the core of veritas they do at least appear to
> have the knowledgebase to do a good job

Yeah, I'd rather see all source open. But that's an ideal world. In
the meantime, many people want $$$. One of the great things about
Linux is that it is open and allows different funding models. The
success of Linux is due to the openness, not some cool technological
feature.

Open Source pushes the "innovation envelope". Eventually, the "core"
(what's now the basic OS) which isn't worth selling grows outwards,
consuming areas where it used to be profitable to sell software. So it
forces companies to innovate or die, leading to a dynamic industry.
That is good for both Society and Industry (as seen by the respective
idealogical poles).

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: binary modules (was Re: ReiserFS / 2.4.6 / Data Corruption)

[Chris Wedgwood]

On Sun, Jul 29, 2001 at 03:05:06AM -0400, Richard Gooch wrote:

    Yeah, I'd rather see all source open. But that's an ideal world. In
    the meantime, many people want $$$. One of the great things about
    Linux is that it is open and allows different funding models. The
    success of Linux is due to the openness, not some cool technological
    feature.

People all need to appreciate sometimes vendors cannot released open
source drivers even if they wanted too.  Sometimes vendors have the
ability to released binary only drivers which are derived in part from
source-code which they license --- but cannot share.

This is also the case for various SCSI cards and such like, firmware
is provided binary-only because the source for the firmware isn't
something that can be distributed.



  --cw

Re: binary modules (was Re: ReiserFS / 2.4.6 / Data Corruption)

[Florian Weimer]

Chris Wedgwood <cw@f00f.org> writes:

> People all need to appreciate sometimes vendors cannot released open
> source drivers even if they wanted too.  Sometimes vendors have the
> ability to released binary only drivers which are derived in part from
> source-code which they license --- but cannot share.

That's particularly true if there is no other documentation for the
hardware other than this reference source code.  This seems to be a
common situation, even with hardware which has good specs, technically
speaking.

-- 
Florian Weimer 	                  Florian.Weimer@RUS.Uni-Stuttgart.DE
University of Stuttgart           http://cert.uni-stuttgart.de/
RUS-CERT                          +49-711-685-5973/fax +49-711-685-5898

Re: 2.4.2 ext2fs corruption status

[Alan Cox]

> It may be that Red Hat has already released a new kernel RPM since that
> time, or maybe you need to compile a new kernel.

The official VIA workaround fix is now in 2.4.6ac5 and 2.4.7ac*. The fixes
in the older kernels were mostly going to do the job but I dont know if they
were perfect for all cases

The -ac kernel tree also contains important fixes that avoid DMA timeouts
potentially causing disk corruption by forgetting to write sectors

Re: 2.4.2 ext2fs corruption status

[Mohamed DOLLIAZAL]

Alan Cox wrote:

> > It may be that Red Hat has already released a new kernel RPM since that
> > time, or maybe you need to compile a new kernel.
>
> The official VIA workaround fix is now in 2.4.6ac5 and 2.4.7ac*. The fixes
> in the older kernels were mostly going to do the job but I dont know if they
> were perfect for all cases
>
> The -ac kernel tree also contains important fixes that avoid DMA timeouts
> potentially causing disk corruption by forgetting to write sectors

Hi Alan,

   I'am sorry I forgot to mention that the filesystem corruption happened on
SCSI disks.  I guess there is no DMA on the SCSI disks.
   Do you think that the VIA fixes that are included in the 2.4.6ac5 kernel or
above may solve my problem.

Thanks for your help,

Mohamed.

Re: Memory Write Ordering Question

[Alan Cox]

> I'm wondering if anyone has any idea what exactly is causing this.  The
> readl is a so-so work around.  I'd like to figure out how to do it
> correctly.  Does anyone who knows more about Intel CPU's and write
> ordering and PCI have any ideas?

Its entirely a PCI issue. PCI writes are posted and may be deferred. However
a write cannot pass another write to the device, nor a read, so your read
is the real solution.

The full horror is in the PCI specs which you can get on CD nowdays fairly
sanely. Basically PCI is a message passing system disguised as a bus, treat
it as the former and you wont get too badly hurt

Re: SMP possible with AMD CPUs?

[Alan Cox]

> 	a. The IDE is no longer a 7409 PCI ID but 7411 so it operates as a generic IDE (slow as hell).
[Should run full UDMA in -ac]

> 	b. The AGP is now ID 700C and is not detected unless the agpgart driver is loaded with agp_try_unsupported=1.

Send me the relevant pci idents and I'll add it

> 	d. The PCI bridge ID is different and (again) operates in a generic modeAgain send me the ids
> 	e. The Host bridge ID is now 700C and operates in a generic mode.

Send me the idents for these two

> 3. The BIOS (apparently) doesn't setup the MTRR properly on both CPUs making mtrr bitch about a mismatch.

The mtrr driver fixups should cure that - its a common bios bug.

Alan

Re: SMP possible with AMD CPUs?

[Eric W. Biederman]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:
 
> > 3. The BIOS (apparently) doesn't setup the MTRR properly on both CPUs making
> mtrr bitch about a mismatch.
> 
> 
> The mtrr driver fixups should cure that - its a common bios bug.

There is some truth in that.  But note AMD hasn't released all of the
documentation related to their MTRR's so we can't rely on linux fixing
all of those BIOS bugs.   In this case it happens to be different
caching on the BIOS chip, from different cpus.

An interesting question is what is 0x1e in the AMD fixed mtrr's.

Eric

Re: 2.4.2 ext2fs corruption status

[Alan Cox]

>    I'am sorry I forgot to mention that the filesystem corruption happened on
> SCSI disks.  I guess there is no DMA on the SCSI disks.

Well there is but its off the scsi controller so should be ok

>    Do you think that the VIA fixes that are included in the 2.4.6ac5 kernel or
> above may solve my problem.

They might do, they might not. But they are worth checking

Re: 2.4 freezes on init

[Alan Cox]

> However, I believe the kernel image that Mandrake 8's installer uses is
> a 2.4 kernel, yet that works fine. Is this a configuration setting I
> need to toggle, or am I stuck until I switch motherboard?

In the Red Hat case we have seen cases where the installer kernel worked and
not much else did. Install kernels are generally built with the very minimum
of reliance on bios features and for 386.

Typically that means they don't enable common problem items like APM, ACPI
and Athlon optimisation in conjunction with VIA chipsets.

Alan

Re: setsockopt(..,SO_RCVBUF,..) sets wrong value

[Alan Cox]

> When I do a setsockopt(..,SO_RCVBUF,..) and then read the value back
> with getsockopt(), the reported value is exactly twice of what I set.
> Running the same code on Solaris and on DEC UNIX reports back the
> exact size I set.
> Looking at the code it seems that the  *2  should not be there:

You are making assumptions not guaranteed in POSIX or SuS. In the Linux case
we deliberately allow more than requested as our memory accounting behaviour
for buffers is very different to BSD

Re: kernel gdb for intel

[Alan Cox]

> - doesn't support SMP, since I don't have an Intel SMP box.  I'd guess
> what you'd want it to do is an smp_call_function that would halt all the
> processors and put them into some tight little loop while gdb fiddles
> things.  ideas?

With the old old stuff (pre 2.0) gdb stubs I ended up with two copies, one
per cpu on two serial ports. I found that most useful since I could force
events to happen.

Looks nice to me but about the only way you are likely to get Linus to take
in kernel debugging patches is to turn them into hex and disguise them as USB 
firmware ;)

Alan

Re: kernel gdb for intel

[Amit S. Kale]

Alan Cox wrote:
> 
> > - doesn't support SMP, since I don't have an Intel SMP box.  I'd guess
> > what you'd want it to do is an smp_call_function that would halt all the
> > processors and put them into some tight little loop while gdb fiddles
> > things.  ideas?
> 
> With the old old stuff (pre 2.0) gdb stubs I ended up with two copies, one
> per cpu on two serial ports. I found that most useful since I could force
> events to happen.

I can't get this. How can two gdb stubs work correctly
on two serial ports? In absence of any globals, there won't be
any data corruption, though there are inherent assumptions in 
the kernel about progress on all cpus. If GKL, page cache lock etc
are held by one cpu, the other cpu will not be able to make
any/much progress.

Are two gdb stubs useful for debugging some particular kind
of problem? If they are I can think about how I can
add them to current x86 kgdb (kgdb.sourceforge.net).
-- 
Amit Kale
Veritas Software ( http://www.veritas.com )

Re: 3ware Escalade problems? Adaptec?

[Alan Cox]

> I've been pricing out a 3ware-based raid system for my own personal use. Are
> the problems wuth the Escalade cards bad enough to consider not using them
> with 2.4.7?

Im really attached to my 3ware cards, they are the best ide raid cards I've
used. The newer boxes I built just use software raid 0/1 which is easy now
that everyone throws 4 UDMA100 channels on their motherboards.

I've also done the i2o driver fixups for the Promise SuperTrak100 with a
card provided by Promise and that works in -ac but not yet Linus tree.
I'm more impressed with the 3ware than the promise card right now, although
it will depend on workload. The promise card has onboard caches and raid5 
hardware which the earlier 3ware didn't.

Re: [PATCH] make psaux reconnect adjustable

[Alan Cox]

> who asked for this code): if what I say is correct you should
> always see 00 following the AA. So, there may exist a more cautious
> patch that will bite fewer people and does not react to AA but to
> the sequence AA 00.

2.2 has had the sysctl for ages, and it defaults to off

Re: [RFT] Support for ~2144 SCSI discs

[Alan Cox]

> I've seen GFP_KERNEL take 10 minutes in lk 2.4.6 . The 
> mm gets tweaked pretty often so it is difficult to know 
> exactly how it will react when memory is tight. A time 
> bound would be useful on GFP_KERNEL.

kmalloc with GFP_KERNEL has a 128K limit which avoids the bizarre behaviour
you get when you abuse get_free_pages.

Re: [RFT] Support for ~2144 SCSI discs

[Richard Gooch]

Alan Cox writes:
> > I've seen GFP_KERNEL take 10 minutes in lk 2.4.6 . The 
> > mm gets tweaked pretty often so it is difficult to know 
> > exactly how it will react when memory is tight. A time 
> > bound would be useful on GFP_KERNEL.
> 
> kmalloc with GFP_KERNEL has a 128K limit which avoids the bizarre
> behaviour you get when you abuse get_free_pages.

Last I heard, get_free_pages() also has a 128 kiB limit. So what's the
difference?

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: [RFT] Support for ~2144 SCSI discs

[Alan Cox]

> > kmalloc with GFP_KERNEL has a 128K limit which avoids the bizarre
> > behaviour you get when you abuse get_free_pages.
> 
> Last I heard, get_free_pages() also has a 128 kiB limit. So what's the
> difference?

get_free_pages doesnt have such a limit. Thats why sg had the problem it did

Re: [RFT] Support for ~2144 SCSI discs

[Douglas Gilbert]

Alan Cox wrote:
> 
> > > kmalloc with GFP_KERNEL has a 128K limit which avoids the bizarre
> > > behaviour you get when you abuse get_free_pages.
> >
> > Last I heard, get_free_pages() also has a 128 kiB limit. So what's the
> > difference?
> 
> get_free_pages doesnt have such a limit. Thats why sg had the problem it did

Alan,
That is incorrect.

The failure I got with the sg driver with cdrdao
and cdda2wav was with 32 KB buffers, lots of them.
cdda2wav in RH 7.1 was trying to get 100 MB of them!

If you look at the sg driver you will find that it never
attempts a get_free_pages greater than SG_SCATTER_SZ (32 KB).
So that unkillable lockup on those apps demonstrates
rather well that GFP_KERNEL is dangerous.

Doug Gilbert

Re: [RFT] #2 Support for ~2144 SCSI discs

[Alan Cox]

> So, yes, you can already patch other subsystems to dynamically assign
> major numbers in 2.4.7. I'd like to see people do that. My patch for
> sd.c can also serve as a demonstration on how to use the new API.

Its a bit of an ugly hack but I guess its the best anyone can put together
for a 2.4 kernel tree. Going to a 32bit dev_t is going to make life so much
simpler do all of this without ugly hacks

Alan

Re: [RFT] #2 Support for ~2144 SCSI discs

[Richard Gooch]

Alan Cox writes:
> > So, yes, you can already patch other subsystems to dynamically assign
> > major numbers in 2.4.7. I'd like to see people do that. My patch for
> > sd.c can also serve as a demonstration on how to use the new API.
> 
> Its a bit of an ugly hack but I guess its the best anyone can put
> together for a 2.4 kernel tree. Going to a 32bit dev_t is going to
> make life so much simpler do all of this without ugly hacks

My patch is definately 2.4 material. I see it as a temporary solution
until the whole block I/O subsystem is ripped out and replaced in 2.5.
Since 2.4 will be the latest production kernel for about two years, we
need to find ways of working around current limitations.

That said, in 2.5 I want to see us move away from using device numbers
as the fundamental device handle and move to device instance
structures. That's a lot cleaner, and BTW is devfs-neutral
(i.e. doesn't need devfs to work). Exposing a 32 bit dev_t to
user-space is acceptable, but internally it should be shunned.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: [RFT] #2 Support for ~2144 SCSI discs

[Alan Cox]

> That said, in 2.5 I want to see us move away from using device numbers
> as the fundamental device handle and move to device instance
> structures. That's a lot cleaner, and BTW is devfs-neutral
> (i.e. doesn't need devfs to work). Exposing a 32 bit dev_t to
> user-space is acceptable, but internally it should be shunned.

You need it internally otherwise you are screwed the moment you have 65536
volumes mounted - because you run out of unique device identifiers for stat.

Fortunately 32bit dev_t (not kdev_t .. which I think is what you are talking
about and will I assume go pointer to struct) is only one syscall change

Re: [RFT] #2 Support for ~2144 SCSI discs

[Richard Gooch]

Alan Cox writes:
> > That said, in 2.5 I want to see us move away from using device numbers
> > as the fundamental device handle and move to device instance
> > structures. That's a lot cleaner, and BTW is devfs-neutral
> > (i.e. doesn't need devfs to work). Exposing a 32 bit dev_t to
> > user-space is acceptable, but internally it should be shunned.
> 
> You need it internally otherwise you are screwed the moment you have
> 65536 volumes mounted - because you run out of unique device
> identifiers for stat.

I consider that "external" use. The kernel doesn't need it, it just
provides unique numbers for user-space. The kernel just happens to
carry along the information so that user-space can get it as needed.

Aside: the idea of mounting >65536 volumes frightens me. Accidentally
do a "df" and go away for a coffee while your machine hammers away.

> Fortunately 32bit dev_t (not kdev_t .. which I think is what you are
> talking about and will I assume go pointer to struct) is only one
> syscall change

Looks like we agree. And as long as you have <65536 volumes, then
libc5 will continue to work just fine. Which is also good.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: [PATCH] make psaux reconnect adjustable

[Alan Cox]

> Of course I hope that we'll handle this correctly at some point,
> without any options or parameters. In my eyes a sysctl is heavier
> infrastructure than a boot parameter, so I prefer the latter
> when a temporary fix is needed.

The input device infrastructure pending for 2.5 already handles all of
these issues

Re: [PATCH] vxfs fix

[Alan Cox]

> 	From: Alan
> 
> 	Alternatively pass a flag to the mount command saying
> 	"this is a guesswork special" then V7 fs can just return 'not me'
> 
> Parse failure.

Let me try again:

When the read_super method is invoked
AND we are doing a mount without a defined type
	THEN
		Pass the fs a flag from the VFS saying so
	ENDIF

That way the file system can actually say "I cannot reliably check"

Re: [PATCH] vxfs fix

[Andreas Dilger]

Alan writes:
> When the read_super method is invoked
> AND we are doing a mount without a defined type
> 	THEN
> 		Pass the fs a flag from the VFS saying so
> 	ENDIF
> 
> That way the file system can actually say "I cannot reliably check"

Isn't this what the "silent" option to read_super is for?  It may be that
it can only be used at root fs mount time.  Other than that, I don't
_think_ the kernel does autoprobing of filesystem types, so it is a
mount(8) issue to just not randomly try the V7 filesystem type.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: kernel gdb for intel

[Alan Cox]

> I can't get this. How can two gdb stubs work correctly
> on two serial ports? In absence of any globals, there won't be
> any data corruption, though there are inherent assumptions in 
> the kernel about progress on all cpus. If GKL, page cache lock etc
> are held by one cpu, the other cpu will not be able to make
> any/much progress.

That is fine. It'll get stuck in a lock. One thing it was useful for was
exactly that - getting a given point and checking the locking cases worked

Re: DoS with tmpfs #3

[Alan Cox]

> The same horrible think happens to ramfs, but this
> could be expected. Ramfs don't have size check so that
> hack cannot be used for it.  In this case ramfs must
> be marked as dangerous. 

Ramfs and tmpfs in the -ac tree should behave a lot better. The 
fact you see high pages being a factor sounds to me like a VM rather than
a tmpfs bug. Specifically you should have seen KDE apps terminating with
out of memory kills. 

In paticular in the -ac tree ramfs supports setting limits on the max fs
size, which is essential if you want to use it on something like an iPAQ
where ramfs is a real useful fs to have.

tmpfs would I suspect also benefit immensely from quota support

Alan

Re: Question regarding ACPI

[Alan Cox]

> Will ACPI error be corrected. Since I started compiling my own kernel, 
> version 2.4.6, ACPI hasn't worked. Has the kernel maintainer given up on 
> it? or is it still in the ACPI think tank?

You probably want to drop a mail to andrew.grover@intel.com and also see
http://phobos.fs.tum.de/acpi/index.html which is the mailing list for
ACPI. 

Re: Error when compiling 2.4.7ac6

[Alan Cox]

> gcc -D__KERNEL__ -I/usr/src/linux/include -Wall -Wstrict-prototypes 
> -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common 
> pipe -mpreferred-stack-boundary=2 -march=i686    -DEXPORT_SYMTAB -c check.c
> In file included from check.c:28:
> ldm.h:100: warning: `SYS_IND' redefined
> ldm.h:84: warning: this is the location of the previous definition
> ldm.h:104: warning: `NR_SECTS' redefined
> ldm.h:88: warning: this is the location of the previous definition
> ldm.h:109: warning: `START_SECT' redefined
> ldm.h:92: warning: this is the location of the previous definition
> gcc -D__KERNEL__ -I/usr/src/linux/include -Wall -Wstrict-prototypes 
> -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common 
> pipe -mpreferred-stack-boundary=2 -march=i686    -c -o msdos.o msdos.c
> rm -f partitions.o

Thanks - fixed 

Re: MTRR and Athlon Processors

[Alan Cox]

> Is the mtrr code supposed to work properly for Athlon (Model 4) in
> kernel 2.4.7?
> 
> I still get mtrr errors/warnings.

Mismatched mtrr warnings indicate bios writers who cannot read
specifications. The kernel will fix up after it

Re: MTRR and Athlon Processors

[Paul G. Allen]

Alan Cox wrote:
> 
> > Is the mtrr code supposed to work properly for Athlon (Model 4) in
> > kernel 2.4.7?
> >
> > I still get mtrr errors/warnings.
> 
> Mismatched mtrr warnings indicate bios writers who cannot read
> specifications. 

This does not surprise me. In fact, I need to check Tyan's web site for
a BIOS update.

> The kernel will fix up after it

I also get this message:

Jul 29 03:33:00 keroon kernel: mtrr: type mismatch for f8000000,4000000
old: write-back new: write-combining

This happens quite often, especially with the agpgart and NVdriver
modules.

PGA

-- 
Paul G. Allen
UNIX Admin II/Network Security
Akamai Technologies, Inc.
www.akamai.com

Re: MTRR and Athlon Processors

[Dave Jones]

On Sat, 4 Aug 2001, Paul G. Allen wrote:

> Jul 29 03:33:00 keroon kernel: mtrr: type mismatch for f8000000,4000000
> old: write-back new: write-combining
>
> This happens quite often, especially with the agpgart and NVdriver
> modules.

iirc, this is a problem with the nvidia module, and there's nothing
the kernel can do about it. Complain to nvidia.

regards,

Dave.

-- 
| Dave Jones.        http://www.suse.de/~davej
| SuSE Labs

Re: MTRR and Athlon Processors

[Paul G. Allen]

Dave Jones wrote:
> 
> On Sat, 4 Aug 2001, Paul G. Allen wrote:
> 
> > Jul 29 03:33:00 keroon kernel: mtrr: type mismatch for f8000000,4000000
> > old: write-back new: write-combining
> >
> > This happens quite often, especially with the agpgart and NVdriver
> > modules.
> 
> iirc, this is a problem with the nvidia module, and there's nothing
> the kernel can do about it. Complain to nvidia.
> 

If I knew who to complain to, I would. I used to have a contact there,
but I seem to have lost his e-mail address. :(

(BTW, There's no update to the Tyan [Pheonix] BIOS as yet either.)

PGA

-- 
Paul G. Allen
UNIX Admin II/Network Security
Akamai Technologies, Inc.
www.akamai.com

Re: Error when compiling 2.4.7ac6

[Anton Altaparmakov]

At 01:46 05/08/2001, Alan Cox wrote:
> > gcc -D__KERNEL__ -I/usr/src/linux/include -Wall -Wstrict-prototypes
> > -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common
> > pipe -mpreferred-stack-boundary=2 -march=i686    -DEXPORT_SYMTAB -c check.c
> > In file included from check.c:28:
> > ldm.h:100: warning: `SYS_IND' redefined
> > ldm.h:84: warning: this is the location of the previous definition
> > ldm.h:104: warning: `NR_SECTS' redefined
> > ldm.h:88: warning: this is the location of the previous definition
> > ldm.h:109: warning: `START_SECT' redefined
> > ldm.h:92: warning: this is the location of the previous definition
> > gcc -D__KERNEL__ -I/usr/src/linux/include -Wall -Wstrict-prototypes
> > -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common
> > pipe -mpreferred-stack-boundary=2 -march=i686    -c -o msdos.o msdos.c
> > rm -f partitions.o
>
>Thanks - fixed

It's quite funny gcc-2.96 doesn't give these warnings. Perhaps it sees that 
the defines are identical and shuts up?

Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: Error when compiling 2.4.7ac6

[Alan Cox]

> It's quite funny gcc-2.96 doesn't give these warnings. Perhaps it sees that 
> the defines are identical and shuts up?

They are actually not identical - the bracketing varies

Re: Error when compiling 2.4.7ac6

[Anton Altaparmakov]

At 02:43 05/08/2001, Alan Cox wrote:
> > It's quite funny gcc-2.96 doesn't give these warnings. Perhaps it sees 
> that
> > the defines are identical and shuts up?
>
>They are actually not identical - the bracketing varies

Oh, ok. I didn't know they hadn't been copied verbatim...

Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: SMP Support for AMD Athlon MP motherboards

[Alan Cox]

> Whats the degree of support in Linux for such an AMD mobo? Is the Athlo=
> n MP
> architecture supported at all yet?

It is supported yes. You should have seen both processors and stability

Alan

Re: 3c509: broken(verified)

[Alan Cox]

> The driver for the 3c509 of 2.4.7 is broken:
> It is impossible to set the transmitter type.
> modprobe 3c509 xcvr=X, where X is
> 0,1,2,3,4 doesn't make a difference.

Looking at the code it should set the type fine. The only bug I can see is
that it will report the default type set in the eeprom not the type you
asked.

If thats the case (please check) then its trivial to fix

Re: 3c509: broken(verified)

[Nico Schottelius]

Alan Cox wrote:

> > The driver for the 3c509 of 2.4.7 is broken:
> > It is impossible to set the transmitter type.
> > modprobe 3c509 xcvr=X, where X is
> > 0,1,2,3,4 doesn't make a difference.
>
> Looking at the code it should set the type fine. The only bug I can see is
> that it will report the default type set in the eeprom not the type you
> asked.
>
> If thats the case (please check) then its trivial to fix

While I tried to setup the driver I always let one machine
outside ping it.

It is not just the message.

ozean:~ # modprobe 3c509 ; ifconfig eth1 192.168.4.17 up

eth1: 3c5x9 at 0x360, BNC port, address  00 60 97 39 43 b9, IRQ 5.
3c509.c:1.18 12Mar2001 becker@scyld.com
http://www.scyld.com/network/3c509.html

- the light on the hub keeps off, no ping answer

ozean:~ # ifconfig eth1 down ; rmmod 3c509;

ozean:~ # modprobe 3c509 xcvr=4 debug=4

## xcvr=4 is TP (found on scyld.com/network/3c509.html)


3c509.c:1.18 12Mar2001 becker@scyld.com
http://www.scyld.com/network/3c509.html
eth1: Setting Rx mode to 1 addresses.
  3c509 EEPROM word 7 0x6d50.
  3c509 EEPROM word 0 0x0060.
  3c509 EEPROM word 1 0x9739.
  3c509 EEPROM word 2 0x43b9.
  3c509 EEPROM word 8 0xc096.
  3c509 EEPROM word 9 0x5000.

eth1: 3c5x9 at 0x360, BNC port, address  00 60 97 39 43 b9, IRQ 5.
3c509.c:1.18 12Mar2001 becker@scyld.com
http://www.scyld.com/network/3c509.html
  3c509 EEPROM word 7 0xffff.
eth1: Opening, IRQ 5     status@36e 0000.
eth1: Opened 3c509  IRQ 5  status 2000.
eth1: Setting Rx mode to 1 addresses.

ozean:~ # ifconfig eth1 192.168.4.17 up

- ping does not work, no light is seen


That's it! The cable & the hub are okay.


Nico

Re: 3c509: broken(verified)

[safemode]

On Sunday 05 August 2001 10:23, Nico Schottelius wrote:
> Alan Cox wrote:
> > > The driver for the 3c509 of 2.4.7 is broken:
> > > It is impossible to set the transmitter type.
> > > modprobe 3c509 xcvr=X, where X is
> > > 0,1,2,3,4 doesn't make a difference.
> >
> > Looking at the code it should set the type fine. The only bug I can see
> > is that it will report the default type set in the eeprom not the type
> > you asked.
> >
> > If thats the case (please check) then its trivial to fix
>
> While I tried to setup the driver I always let one machine
> outside ping it.
>
> It is not just the message.
>
> ozean:~ # modprobe 3c509 ; ifconfig eth1 192.168.4.17 up
>
> eth1: 3c5x9 at 0x360, BNC port, address  00 60 97 39 43 b9, IRQ 5.
> 3c509.c:1.18 12Mar2001 becker@scyld.com
> http://www.scyld.com/network/3c509.html
>
> - the light on the hub keeps off, no ping answer
>
> ozean:~ # ifconfig eth1 down ; rmmod 3c509;
>
> ozean:~ # modprobe 3c509 xcvr=4 debug=4
>
> ## xcvr=4 is TP (found on scyld.com/network/3c509.html)
>
>
> 3c509.c:1.18 12Mar2001 becker@scyld.com
> http://www.scyld.com/network/3c509.html
> eth1: Setting Rx mode to 1 addresses.
>   3c509 EEPROM word 7 0x6d50.
>   3c509 EEPROM word 0 0x0060.
>   3c509 EEPROM word 1 0x9739.
>   3c509 EEPROM word 2 0x43b9.
>   3c509 EEPROM word 8 0xc096.
>   3c509 EEPROM word 9 0x5000.
>
> eth1: 3c5x9 at 0x360, BNC port, address  00 60 97 39 43 b9, IRQ 5.
> 3c509.c:1.18 12Mar2001 becker@scyld.com
> http://www.scyld.com/network/3c509.html
>   3c509 EEPROM word 7 0xffff.
> eth1: Opening, IRQ 5     status@36e 0000.
> eth1: Opened 3c509  IRQ 5  status 2000.
> eth1: Setting Rx mode to 1 addresses.
>
> ozean:~ # ifconfig eth1 192.168.4.17 up
>
> - ping does not work, no light is seen
>
>
> That's it! The cable & the hub are okay.
>
>
> Nico
>

i was just using a 3c509 in my friend's old 486 and it was working fine with 
2.4.7.   Just modprobed it and set up the ips and it was able to ping and be 
pinged.   

Re: 3c509: broken(verified)

[Nico Schottelius]

> i was just using a 3c509 in my friend's old 486 and it was working fine with
> 2.4.7.   Just modprobed it and set up the ips and it was able to ping and be
> pinged.

Did you use twisted pair or coax (bnc) ?

This problems occurs (at least ) when trying to use TP.

Nico

ps: Alan, do you have an solution ?

Re: 3c509: broken(verified)

[Nicholas Knight]

On Monday 06 August 2001 08:54 am, Nico Schottelius wrote:
> > i was just using a 3c509 in my friend's old 486 and it was working
> > fine with 2.4.7.   Just modprobed it and set up the ips and it was
> > able to ping and be pinged.
>
> Did you use twisted pair or coax (bnc) ?
>
> This problems occurs (at least ) when trying to use TP.
>
> Nico
>
> ps: Alan, do you have an solution ?

For what it's worth, I'm using a 3c509 card on vanilla 2.4.7 right now, 
using standard twisted pair patch cable, and it works fine. I've used it 
both as a module and compiled in (using compiled in at the moment) on 
2.4.5 and 2.4.7, I've also previously used it on 2.4.3, both compiled in 
and as a module.

The motherboard is a Soyo K7VIA w/single ISA slot, VIA Apollo Pro KX133 
chipset, using an Athlon processor.

The card is connected to a hub and communicates fine with both my other 
system and my cable modem, using DHCP.

You mention the problem is being unable to change the media, I was 
unaware this was even possible with the current 3c509 driver, and most 
people do it on 3c509's and other PNP cards of this sort (such as NE2000 
clones)  by using a DOS boot diskette and the DOS utilities provided by 
the manufacturer.

Re: Problem in Interrupt Handling ....

[Alan Cox]

> serial communication adapter which is based on
> interrrupt driven IO, top half handles registering the
> Immediate task queue and  acknowledging to PIC, bottom
> half performs the actual task of interrupt handling. 

Why are you touching the PIC at all - the kernel handles the PIC for you.
Indeed you IRQ might not even be coming from a PIC

Re: Virtual memory error when restarting X

[Alan Cox]

> Often after restarting X I see the messages similar to "NV0: still have vm que at nv_close(): 0x4023b000 to 0x40245000" in the logs. I presume that these are being caused by the properitry nvidia drivers that I use with X. However I have also noticed that "Unable to handle kernel paging request at virtual address 6b336b50" and the like have also been turning up. I'm wondering whether the graphics drivers problems could be being caused by a vm problem. The oops that
> is enclosed does not appear to be readily repeatable...  but leaving X causes sometimes the system to lock solid with only SysRq getting through. 

Talk to Nvidia. Its their obfuscated/binary driver set, nobody else can help
you fix it.

Alan

Re: [PATCH] one of $BIGNUM devfs races

[Alan Cox]

> Linus: please don't apply.
> Alan: I notice you've put Al's patch into 2.4.7-ac8. Please remove it.

I'll remove it when your preferred fixes are ready. Until then its better
than leaving it broken.

Re: [PATCH] one of $BIGNUM devfs races

[Richard Gooch]

Alan Cox writes:
> > Linus: please don't apply.
> > Alan: I notice you've put Al's patch into 2.4.7-ac8. Please remove it.
> 
> I'll remove it when your preferred fixes are ready. Until then its
> better than leaving it broken.

OK, fair enough. When is your next merge with Linus scheduled? I'd
prefer to get a few races fixed before shipping a patch, but I can try
to plan for an earlier release if necessary.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: [PATCH] one of $BIGNUM devfs races

[Richard Gooch]

Alan Cox writes:
> > OK, fair enough. When is your next merge with Linus scheduled? I'd
> > prefer to get a few races fixed before shipping a patch, but I can try
> > to plan for an earlier release if necessary.
> 
> I send stuff Linus regularly and sometimes it goes in and sometimes
> it doesn't. Stuff with active maintainers I don't send on to Linus
> unless asked too - hence joystick. input and much of USB are so far
> behind in Linus tree

So does that mean you won't try to merge Al's patch?

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: Encrypted Swap

[Alan Cox]

> A relatively cheap way might be a custom pci
> card with a self-destruct RAM bank for
> storing the decryption keys.  Opening the 
> safe cause the card to zero the RAM.  

IBM sell crypto PCI cards with anti tamper environments, they have
development drivers on their oss site too

Re: Encrypted Swap

[Crutcher Dunnavant]

++ 07/08/01 15:17 +0100 - Alan Cox:
> > A relatively cheap way might be a custom pci
> > card with a self-destruct RAM bank for
> > storing the decryption keys.  Opening the 
> > safe cause the card to zero the RAM.  
> 
> IBM sell crypto PCI cards with anti tamper environments, they have
> development drivers on their oss site too

Ohh. Some college buddies and I were considering the difficulty involved from
making a pci card with an onboard giger counter and radiatian source (say, from
a smoke detector) wrapped up in some lead.

Sure, there are simpler ways to build chaotic circuits, but a radioactive
peripheral is cool!

-- 
Crutcher        <crutcher@datastacks.com>
GCS d--- s+:>+:- a-- C++++$ UL++++$ L+++$>++++ !E PS+++ PE Y+ PGP+>++++
    R-(+++) !tv(+++) b+(++++) G+ e>++++ h+>++ r* y+>*$

Re: Encrypted Swap

[Chris Wedgwood]

On Tue, Aug 07, 2001 at 11:16:35AM -0400, Crutcher Dunnavant wrote:

    Ohh. Some college buddies and I were considering the difficulty
    involved from making a pci card with an onboard giger counter and
    radiatian source (say, from a smoke detector) wrapped up in some
    lead.

    Sure, there are simpler ways to build chaotic circuits, but a
    radioactive peripheral is cool!

Why?  There are plenty of cards out that that have hardware and more.
The one in the machine I presently use is limited to 1Mbit/second of
data from a noise-diode (fed through various functions in hardware to
further obfuscate the data).




  --cw

Re: [PATCH] one of $BIGNUM devfs races

[Alan Cox]

> > unless asked too - hence joystick. input and much of USB are so far
> > behind in Linus tree
> 
> So does that mean you won't try to merge Al's patch?

Correct. I'd just get in your way if I did

Re: [PATCH] one of $BIGNUM devfs races

[Alan Cox]

> > Very interesting. pwd should be using getcwd(2), which doesn't
> > give a damn for inode numbers. If you have seriously old pwd binary
> > that tries to track the thing down to root by hands - yes, it doesn't
> > work.
> 
> Hm. strace suggests my pwd is walking up the path. But WTF would it
> break? 2.4.7 was fine. What did I break?

Sounds like you are using libc5. The old style pwd should be reliable but
its much slower and can't see across protected directory paths

Alan

Re: [PATCH] one of $BIGNUM devfs races

[Alexander Viro]

On Tue, 7 Aug 2001, Alan Cox wrote:

> > > Very interesting. pwd should be using getcwd(2), which doesn't
> > > give a damn for inode numbers. If you have seriously old pwd binary
> > > that tries to track the thing down to root by hands - yes, it doesn't
> > > work.
> > 
> > Hm. strace suggests my pwd is walking up the path. But WTF would it
> > break? 2.4.7 was fine. What did I break?
> 
> Sounds like you are using libc5. The old style pwd should be reliable but
> its much slower and can't see across protected directory paths

It is not reliable. E.g. on NFS inumbers are not unique - 32 bits is
not enough.

Re: [PATCH] one of $BIGNUM devfs races

[H. Peter Anvin]

Followup to:  <Pine.GSO.4.21.0108071510390.18565-100000@weyl.math.psu.edu>
By author:    Alexander Viro <viro@math.psu.edu>
In newsgroup: linux.dev.kernel
> 
> It is not reliable. E.g. on NFS inumbers are not unique - 32 bits is
> not enough.
> 

Unfortunately there is a whole bunch of other things too that rely on
it, and *HAVE* to rely on it -- (st_dev, st_ino) are defined to
specify the identity of a file, and if the current types aren't large
enough we *HAVE* to go to new types.  THERE IS NO OTHER WAY TO TEST
FOR FILE IDENTITY IN UNIX, and being able to perform such a test is
vital for many things, including security and hard link detection
(think tar, cpio, cp -a.)

	-hpa

-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: [PATCH] one of $BIGNUM devfs races

[Alexander Viro]

On 8 Aug 2001, H. Peter Anvin wrote:

> Followup to:  <Pine.GSO.4.21.0108071510390.18565-100000@weyl.math.psu.edu>
> By author:    Alexander Viro <viro@math.psu.edu>
> In newsgroup: linux.dev.kernel
> > 
> > It is not reliable. E.g. on NFS inumbers are not unique - 32 bits is
> > not enough.
> > 
> 
> Unfortunately there is a whole bunch of other things too that rely on
> it, and *HAVE* to rely on it -- (st_dev, st_ino) are defined to
> specify the identity of a file, and if the current types aren't large
> enough we *HAVE* to go to new types.  THERE IS NO OTHER WAY TO TEST
> FOR FILE IDENTITY IN UNIX, and being able to perform such a test is
> vital for many things, including security and hard link detection

Indeed, but it still doesn't help libc5 getcwd(3), which uses 32 bit
values.

> (think tar, cpio, cp -a.)

I'd rather not.  Too bloody depressive... (If you want details - let's
take it off-list).

Re: [PATCH] one of $BIGNUM devfs races

[Richard Gooch]

Alexander Viro writes:
> 
> On 8 Aug 2001, H. Peter Anvin wrote:
> 
> > Followup to:  <Pine.GSO.4.21.0108071510390.18565-100000@weyl.math.psu.edu>
> > By author:    Alexander Viro <viro@math.psu.edu>
> > In newsgroup: linux.dev.kernel
> > > 
> > > It is not reliable. E.g. on NFS inumbers are not unique - 32 bits is
> > > not enough.
> > 
> > Unfortunately there is a whole bunch of other things too that rely on
> > it, and *HAVE* to rely on it -- (st_dev, st_ino) are defined to
> > specify the identity of a file, and if the current types aren't large
> > enough we *HAVE* to go to new types.  THERE IS NO OTHER WAY TO TEST
> > FOR FILE IDENTITY IN UNIX, and being able to perform such a test is
> > vital for many things, including security and hard link detection
> 
> Indeed, but it still doesn't help libc5 getcwd(3), which uses 32 bit
> values.

FYI: the problem that spawned this sub-thread is fixed. The
devfs-patch-v185 that I released last night fixes this. So the libc5
getcwd(3) is fine with 32 bit inums on devfs.

Filesystems with larger inums are left as an exercise for the reader
:-)

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: [PATCH] one of $BIGNUM devfs races

[Richard Gooch]

Alan Cox writes:
> > > Very interesting. pwd should be using getcwd(2), which doesn't
> > > give a damn for inode numbers. If you have seriously old pwd binary
> > > that tries to track the thing down to root by hands - yes, it doesn't
> > > work.
> > 
> > Hm. strace suggests my pwd is walking up the path. But WTF would it
> > break? 2.4.7 was fine. What did I break?
> 
> Sounds like you are using libc5. The old style pwd should be
> reliable but its much slower and can't see across protected
> directory paths

Yes, I use libc5. And I don't care about old pwd being slower. And I
certainly don't want to break it, even if I wasn't using it.
By "protected directory paths", you mean a directory with read access?

Well, rx access is available for the whole path. And the inums looked
fine. So the breakage is odd.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: [PATCH] one of $BIGNUM devfs races

[Alexander Viro]

On Tue, 7 Aug 2001, Richard Gooch wrote:

> Yes, I use libc5. And I don't care about old pwd being slower. And I

So fix getcwd(3) in libc5. BFD... Or use your ->dentry in devfs_readdir() -
then you can get the consistency you want for existing inodes and that
will allow b0rken getcwd() to work.

It _is_ b0rken - it relies on unique 32-bit number for inodes. That's
not guaranteed on NFS and there's nothing we could do about that.

Re: cpu not detected(x86)

[Alan Cox]

> Longer-term, we need to change the kernel to not use the TSC for udelay, but
> to use the PM Timer, if ACPI is going to be monkeying with CPU power states.

That can be done, and may be a help. 

The TSC timer isnt a very good source on many non intel chips that stop it
to get the best power figures. It also helps with SMP because on an SMP box
the tsc values may not calibrate.

Alan

Re: [kbuild-devel] Announce: Kernel Build for 2.5, Release 1 is

[Alan Cox]

> If generating some support files requires some non common tools,
> it is the right thing to ship the two files (source and generated).

Its often easiest. Justin does this with the Adaptec driver now and it makes
life both simple for those who want to build kernels and handy for those
who want to hack the stuff.

> BTW we cannot ship the generated file without the source files,
> because of GPL.

If its part of the kernel tools you want to make it available, that doesn't 
mean it has to be shipped with the kernel. 

Alan

Re: [PATCH] parport_pc.c PnP BIOS sanity check

[Alan Cox]

> The following would seem to be required to protect against
> the case in which PnP BIOS reports an IRQ of 0 for a 
> parport with disabled IRQ.      // Thomas  jdthood_AT_yahoo.co.uk

IRQ 0 is a legal valid IRQ. I suspect the problem is that pnpbios shouldnt
be reporting an IRQ or we should be using some kind of NO_IRQ cookie

Re: [PATCH] parport_pc.c PnP BIOS sanity check

[Richard B. Johnson]

On Wed, 8 Aug 2001, Alan Cox wrote:

> > The following would seem to be required to protect against
> > the case in which PnP BIOS reports an IRQ of 0 for a 
> > parport with disabled IRQ.      // Thomas  jdthood_AT_yahoo.co.uk
> 
> IRQ 0 is a legal valid IRQ. I suspect the problem is that pnpbios shouldnt
> be reporting an IRQ or we should be using some kind of NO_IRQ cookie

IRQ0 will never by reported by a PCI bus device because it means that
no IRQ is used (they figured that IRQ0 would always be used for something
else). Maybe PnP BIOS also presumes this? If so, the use of IRQ0 to
mean "no IRQ" is valid, although misleading.

Cheers,
Dick Johnson

Penguin : Linux version 2.4.1 on an i686 machine (799.53 BogoMips).

    I was going to compile a list of innovations that could be
    attributed to Microsoft. Once I realized that Ctrl-Alt-Del
    was handled in the BIOS, I found that there aren't any.

Re: [PATCH] parport_pc.c PnP BIOS sanity check

[H. Peter Anvin]

Followup to:  <E15UV8M-0005SE-00@the-village.bc.nu>
By author:    Alan Cox <alan@lxorguk.ukuu.org.uk>
In newsgroup: linux.dev.kernel
>
> > The following would seem to be required to protect against
> > the case in which PnP BIOS reports an IRQ of 0 for a 
> > parport with disabled IRQ.      // Thomas  jdthood_AT_yahoo.co.uk
> 
> IRQ 0 is a legal valid IRQ. I suspect the problem is that pnpbios shouldnt
> be reporting an IRQ or we should be using some kind of NO_IRQ cookie
>

IRQ 0 is hardwired to the system timer in PC systems, though, so it
could simply be assumed that IRQ 0 will never be used for any other
purposes.

Reminds me back in the days when you had to worry about DRQs as well;
DRQ 0 was hardwired in the original PC but then became available in
the AT; there was a whole bunch of things that assumed DRQ 0 wasn't
usable, even though it was perfectly fine.  Not to mention the
motherboard I had which would lock up solid if anything ever used
DRQ 5.

Good riddance, all this crap...

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: [PATCH] parport_pc.c PnP BIOS sanity check

[Russell King]

On Wed, Aug 08, 2001 at 02:58:12PM -0700, H. Peter Anvin wrote:
> IRQ 0 is hardwired to the system timer in PC systems, though, so it
                                         ^^^^^^^^^^^^^

Linux doesn't run on only PC systems though, and other systems use
IRQ0 as the (superio-based) parallel port IRQ.

> Good riddance, all this crap...

Indeed - please check the ARM port for our solution to this.  We've
had the NO_IRQ construct for literally years in include/asm-arm/irq.h:

#define NO_IRQ  ((unsigned int)(-1))

Naturally, a similar NO_DMA is defined in dma.h.  The sooner we can get
rid of the "IRQ0 cannot be used" crap from the kernel the better.

--
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: [PATCH] parport_pc.c PnP BIOS sanity check

[Eric W. Biederman]

"H. Peter Anvin" <hpa@zytor.com> writes:

> Followup to:  <E15UV8M-0005SE-00@the-village.bc.nu>
> By author:    Alan Cox <alan@lxorguk.ukuu.org.uk>
> In newsgroup: linux.dev.kernel
> >
> > > The following would seem to be required to protect against
> > > the case in which PnP BIOS reports an IRQ of 0 for a 
> > > parport with disabled IRQ.      // Thomas  jdthood_AT_yahoo.co.uk
> > 
> > IRQ 0 is a legal valid IRQ. I suspect the problem is that pnpbios shouldnt
> > be reporting an IRQ or we should be using some kind of NO_IRQ cookie
> >
> 
> IRQ 0 is hardwired to the system timer in PC systems, though, so it
> could simply be assumed that IRQ 0 will never be used for any other
> purposes.
> 
> Reminds me back in the days when you had to worry about DRQs as well;
> DRQ 0 was hardwired in the original PC but then became available in
> the AT; there was a whole bunch of things that assumed DRQ 0 wasn't
> usable, even though it was perfectly fine.  Not to mention the
> motherboard I had which would lock up solid if anything ever used
> DRQ 5.
> 
> Good riddance, all this crap...

If we are going to list all of the silly assumptions, we still have
the assumption that 640KB-1MB on x86 cannot be used as ram.  It is
less painful but still annoying when you put perfectly valid ram
there.  :) 

Eric

Re: [Dri-devel] Re: DRM Linux kernel merge (update) needed, soon.

[Alan Cox]

> I sent a patch to Linus and Alan this morning.

Tweaked to allow either 4.0 or 4.1 DRM to be built (most folks need 4.0
still) and merged

Re: 386 boot problems with 2.4.7 and 2.4.7-ac9

[Alan Cox]

> This is the panic from 2.4.7-ac9 compiled with gcc-2.96-85 (Red Hat).
> 
> ksymoops 2.4.0 on i686 2.4.7.  Options used

Thanks. For some reason it crashed probing the simple boot flag ACPI
structure. I'll try and work out how and why then send you a diff

Re: Swapping for diskless nodes

[Alan Cox]

> what is the best/recommended way to do remote swapping via the network
> for diskless workstations or compute nodes in clusters in Linux 2.4?=20
> Last time i checked was linux 2.2, and there were some races related=20
> to network swapping back then. Has this been fixed for 2.4?

The best answer probably is "don't". Networks are high latency things for
paging and paging is latency sensitive. If performance is not an issue then
the nbd driver ought to work. You may need to check it uses the right
GFP_ levels to avoid deadlocks and you might need to up the amount of atomic
pool memory. Hopefully other hacks arent needed

[The general case of network swap is basically insoluble but its possible to
 make it perfectly usable as Sun proved]

Alan

Re: Swapping for diskless nodes

[Ingo Oeser]

On Thu, Aug 09, 2001 at 10:08:37AM +0100, Alan Cox wrote:
> > what is the best/recommended way to do remote swapping via the network
> > for diskless workstations or compute nodes in clusters in Linux 2.4?=20
> > Last time i checked was linux 2.2, and there were some races related=20
> > to network swapping back then. Has this been fixed for 2.4?
> 
> The best answer probably is "don't". Networks are high latency things for
> paging and paging is latency sensitive. If performance is not an issue then
> the nbd driver ought to work. You may need to check it uses the right
> GFP_ levels to avoid deadlocks and you might need to up the amount of atomic
> pool memory. Hopefully other hacks arent needed

While we are on it: I have an old machine with 64MB of RAM and a
new, fast machine with 1GB of RAM. 

Sometimes I need more RAM on the old one and asked myself,
whether I could first swap over network to the other one, into
its tmpfs, before digging into real swap on a hard disk.

I have only three machines attached to this small internal
100Mbit LAN.

Both machines use Kernel 2.4.x.

Are there any races I have to consider?

Thanks & Regards

Ingo Oeser
-- 
In der Wunschphantasie vieler Mann-Typen [ist die Frau] unsigned und
operatorvertraeglich. --- Dietz Proepper in dasr

Re: Swapping for diskless nodes

[Ingo Oeser]

On Thu, Aug 09, 2001 at 10:08:37AM +0100, Alan Cox wrote:
> > what is the best/recommended way to do remote swapping via the network
> > for diskless workstations or compute nodes in clusters in Linux 2.4?=20
> > Last time i checked was linux 2.2, and there were some races related=20
> > to network swapping back then. Has this been fixed for 2.4?
> 
> The best answer probably is "don't". Networks are high latency things for
> paging and paging is latency sensitive. If performance is not an issue then
> the nbd driver ought to work. You may need to check it uses the right
> GFP_ levels to avoid deadlocks and you might need to up the amount of atomic
> pool memory. Hopefully other hacks arent needed

While we are on it: I have an old machine with 64MB of RAM and a
new, fast machine with 1GB of RAM. 

Sometimes I need more RAM on the old one and asked myself,
whether I could first swap over network to the other one, into
its tmpfs, before digging into real swap on a hard disk.

I have only three machines attached to this small internal
100Mbit LAN.

Both machines use Kernel 2.4.x.

Are there any races I have to consider?

Thanks & Regards

Ingo Oeser
-- 
In der Wunschphantasie vieler Mann-Typen [ist die Frau] unsigned und
operatorvertraeglich. --- Dietz Proepper in dasr
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Swapping for diskless nodes

[Dirk W. Steinberg]

I'd like to second that example where you have weak diskless nodes and
a big server with a lot of memory. The important point here is that the
remote paging does not need to really write to the remote disk, especially
not synchronously. The page could eventually be migrated to the remote
disk asynchronously, or maybe not at all if there is no memory pressure
at the remote system.

In such a scenario I would disagree with Alan that network paging is 
high latency as compared to disk access. I have a fully switched 100 Mpbs
full-duplex ethernet network, and sending a page across the net into
the memory of a fast server could have much less latency that writing 
that page out to a local old, slow IDE disk. Clusters could even have
special high-bandwidth, low latency networks that could be used for
remote paging.

In a perfect world, all nodes in a cluster would be able to dynamically 
share a pool of "cluster swap" space, so any locally available swap that
is not used could be utilized by other nodes in the cluster.

/ Dirk

Ingo Oeser wrote:
> On Thu, Aug 09, 2001 at 10:08:37AM +0100, Alan Cox wrote:
> > > what is the best/recommended way to do remote swapping via the network
> > > for diskless workstations or compute nodes in clusters in Linux 2.4?=20
> > > Last time i checked was linux 2.2, and there were some races related=20
> > > to network swapping back then. Has this been fixed for 2.4?
> >
> > The best answer probably is "don't". Networks are high latency things for
> > paging and paging is latency sensitive. If performance is not an issue then
> > the nbd driver ought to work. You may need to check it uses the right
> > GFP_ levels to avoid deadlocks and you might need to up the amount of atomic
> > pool memory. Hopefully other hacks arent needed
> 
> While we are on it: I have an old machine with 64MB of RAM and a
> new, fast machine with 1GB of RAM.
> 
> Sometimes I need more RAM on the old one and asked myself,
> whether I could first swap over network to the other one, into
> its tmpfs, before digging into real swap on a hard disk.
> 
> I have only three machines attached to this small internal
> 100Mbit LAN.
> 
> Both machines use Kernel 2.4.x.

Re: Swapping for diskless nodes

[Dirk W. Steinberg]

I'd like to second that example where you have weak diskless nodes and
a big server with a lot of memory. The important point here is that the
remote paging does not need to really write to the remote disk, especially
not synchronously. The page could eventually be migrated to the remote
disk asynchronously, or maybe not at all if there is no memory pressure
at the remote system.

In such a scenario I would disagree with Alan that network paging is 
high latency as compared to disk access. I have a fully switched 100 Mpbs
full-duplex ethernet network, and sending a page across the net into
the memory of a fast server could have much less latency that writing 
that page out to a local old, slow IDE disk. Clusters could even have
special high-bandwidth, low latency networks that could be used for
remote paging.

In a perfect world, all nodes in a cluster would be able to dynamically 
share a pool of "cluster swap" space, so any locally available swap that
is not used could be utilized by other nodes in the cluster.

/ Dirk

Ingo Oeser wrote:
> On Thu, Aug 09, 2001 at 10:08:37AM +0100, Alan Cox wrote:
> > > what is the best/recommended way to do remote swapping via the network
> > > for diskless workstations or compute nodes in clusters in Linux 2.4?=20
> > > Last time i checked was linux 2.2, and there were some races related=20
> > > to network swapping back then. Has this been fixed for 2.4?
> >
> > The best answer probably is "don't". Networks are high latency things for
> > paging and paging is latency sensitive. If performance is not an issue then
> > the nbd driver ought to work. You may need to check it uses the right
> > GFP_ levels to avoid deadlocks and you might need to up the amount of atomic
> > pool memory. Hopefully other hacks arent needed
> 
> While we are on it: I have an old machine with 64MB of RAM and a
> new, fast machine with 1GB of RAM.
> 
> Sometimes I need more RAM on the old one and asked myself,
> whether I could first swap over network to the other one, into
> its tmpfs, before digging into real swap on a hard disk.
> 
> I have only three machines attached to this small internal
> 100Mbit LAN.
> 
> Both machines use Kernel 2.4.x.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Swapping for diskless nodes

[Rik van Riel]

On Thu, 9 Aug 2001, Ingo Oeser wrote:

> Are there any races I have to consider?

Well, this IS a big issue against swap over network.

Swap over network is inherently prone to deadlock
situations, due to the following three problems:

1) we swap pages out when we are close to running
   out of free memory
2) to write pages out over the network, we need to
   allocate space to assemble network packets
3) we need to have memory to receive the ACKs on
   the packets we sent out

The only real solution to this would be memory
reservations so we know this memory won't be used
for other purposes.

What we can do right now is be careful about how
many writeouts over the network we do at the same
time, but that will still get us killed in case of
a ping flood ;)

regards,

Rik
--
IA64: a worthy successor to the i860.

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Swapping for diskless nodes

[Rik van Riel]

On Thu, 9 Aug 2001, Ingo Oeser wrote:

> Are there any races I have to consider?

Well, this IS a big issue against swap over network.

Swap over network is inherently prone to deadlock
situations, due to the following three problems:

1) we swap pages out when we are close to running
   out of free memory
2) to write pages out over the network, we need to
   allocate space to assemble network packets
3) we need to have memory to receive the ACKs on
   the packets we sent out

The only real solution to this would be memory
reservations so we know this memory won't be used
for other purposes.

What we can do right now is be careful about how
many writeouts over the network we do at the same
time, but that will still get us killed in case of
a ping flood ;)

regards,

Rik
--
IA64: a worthy successor to the i860.

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Swapping for diskless nodes

[Dirk W. Steinberg]

Alan,

what you say sound a lot like a hacker solution ("check that it uses the
right GFP_ levels"). I think it's about time that this deficit of linux
as compared to SunOS or *BSD should be removed. Network paging should be
supported as a standard feature of a stock kernel compile.

/ Dirk

Alan Cox wrote:
> > what is the best/recommended way to do remote swapping via the network
> > for diskless workstations or compute nodes in clusters in Linux 2.4?=20
> > Last time i checked was linux 2.2, and there were some races related=20
> > to network swapping back then. Has this been fixed for 2.4?
> 
> The best answer probably is "don't". Networks are high latency things for
> paging and paging is latency sensitive. If performance is not an issue then
> the nbd driver ought to work. You may need to check it uses the right
> GFP_ levels to avoid deadlocks and you might need to up the amount of atomic
> pool memory. Hopefully other hacks arent needed
> 
> [The general case of network swap is basically insoluble but its possible to
>  make it perfectly usable as Sun proved]

Re: Swapping for diskless nodes

[Andreas Haumer]

Hi!

"Dirk W. Steinberg" wrote:
> 
> Alan,
> 
> what you say sound a lot like a hacker solution ("check that it uses the
> right GFP_ levels"). I think it's about time that this deficit of linux
> as compared to SunOS or *BSD should be removed. Network paging should be
> supported as a standard feature of a stock kernel compile.
> 
We have swapping over NBD running for some time now on
our "xS+S Diskless Client" system, and it works really
fine! No problem running StarOffice, Netscape, The Gimp
and KDE on a 128MB Diskless Client and 250MB swap over a 
100MBit switched ethernet!

Check <http://www.xss.co.at/linux/NBD/Applications.html>
to find our solution for that.

Kernel patches are a little bit outdated, but we have NBD swap
for 2.2.19 running internally since this week, and we will
update our web-page soon.

Let us hear if it works for you.

Regards,

- andreas

-- 
Andreas Haumer                     | mailto:andreas@xss.co.at
*x Software + Systeme              | http://www.xss.co.at/
Karmarschgasse 51/2/20             | Tel: +43-1-6060114-0
A-1100 Vienna, Austria             | Fax: +43-1-6060114-71

Re: Swapping for diskless nodes

[Pavel Machek]

Hi!

> > what you say sound a lot like a hacker solution ("check that it uses the
> > right GFP_ levels"). I think it's about time that this deficit of linux
> > as compared to SunOS or *BSD should be removed. Network paging should be
> > supported as a standard feature of a stock kernel compile.
> > 
> We have swapping over NBD running for some time now on
> our "xS+S Diskless Client" system, and it works really
> fine! No problem running StarOffice, Netscape, The Gimp
> and KDE on a 128MB Diskless Client and 250MB swap over a 
> 100MBit switched ethernet!

Try going 8MB of ram, ping -f client and try to compile the kernel.

Netscape + SO + gimp on 128MB is rather light load.

> Check <http://www.xss.co.at/linux/NBD/Applications.html>
> to find our solution for that.
> 
> Kernel patches are a little bit outdated, but we have NBD swap
> for 2.2.19 running internally since this week, and we will
> update our web-page soon.

Be sure to mail me a copy.
								Pavel
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

Re: Swapping for diskless nodes

[Pavel Machek]

Hi!

> > what is the best/recommended way to do remote swapping via the network
> > for diskless workstations or compute nodes in clusters in Linux 2.4?=20
> > Last time i checked was linux 2.2, and there were some races related=20
> > to network swapping back then. Has this been fixed for 2.4?
> 
> The best answer probably is "don't". Networks are high latency things for
> paging and paging is latency sensitive. If performance is not an issue then
> the nbd driver ought to work. You may need to check it uses the
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Alan, are you saying it should work reliably?

> right
> GFP_ levels to avoid deadlocks and you might need to up the amount of atomic
> pool memory. Hopefully other hacks arent needed

There still may be some deadlocks. Swapping over nbd seemed to work
for me... until I used mem=8M and did two ping -f's to the victim.

Issue is that you not only need to check nbd, you need to check whole
network layer, too.
								Pavel
-- 
I'm pavel@ucw.cz. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at discuss@linmodems.org

Re: Swapping for diskless nodes

[Rik van Riel]

On Thu, 9 Aug 2001, Alan Cox wrote:

> > what is the best/recommended way to do remote swapping via the network
> > for diskless workstations or compute nodes in clusters in Linux 2.4?=20
> > Last time i checked was linux 2.2, and there were some races related=20
> > to network swapping back then. Has this been fixed for 2.4?
>
> The best answer probably is "don't". Networks are high latency
> things for paging and paging is latency sensitive.

Actually, swap over network can be faster than local swap
at times. ;)

Don't forget that disks are really high latency devices
and with local swap you are SURE that the data isn't
in memory while with remote swap you have a chance that
the server is caching your data ...

regards,

Rik
--
IA64: a worthy successor to the i860.

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Swapping for diskless nodes

[Alan Cox]

> 
> Alan,
> 
> what you say sound a lot like a hacker solution ("check that it uses the
> right GFP_ levels"). I think it's about time that this deficit of linux

Nope. I'm simply advising people to check that nbd is correctly written.

> as compared to SunOS or *BSD should be removed. Network paging should be
> supported as a standard feature of a stock kernel compile.

There I'd agree entirely.

Re: Swapping for diskless nodes

[Pavel Machek]

Hi!

> > what you say sound a lot like a hacker solution ("check that it uses the
> > right GFP_ levels"). I think it's about time that this deficit of linux
> 
> Nope. I'm simply advising people to check that nbd is correctly written.

This bug is unlikely to be in nbd, but you need to check whole network
stack. Even arp handling is cruical for working nbd swap!
								Pavel
-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

Re: Swapping for diskless nodes

[Alan Cox]

> the memory of a fast server could have much less latency that writing 
> that page out to a local old, slow IDE disk. Clusters could even have
> special high-bandwidth, low latency networks that could be used for
> remote paging.
> 
> In a perfect world, all nodes in a cluster would be able to dynamically 
> share a pool of "cluster swap" space, so any locally available swap that
> is not used could be utilized by other nodes in the cluster.

That I think is a 2.5 problem. One thing that has been talked about several
times now is removing all the swap special case crap from the mm and making
swap a file system. That removes special cases and means anyone can write
or use custom, or multiple swap filesystems, in theory including things like
swap over a shared GFS pool

But its not for 2.4, no way

Alan

Re: Swapping for diskless nodes

[Alan Cox]

> the memory of a fast server could have much less latency that writing 
> that page out to a local old, slow IDE disk. Clusters could even have
> special high-bandwidth, low latency networks that could be used for
> remote paging.
> 
> In a perfect world, all nodes in a cluster would be able to dynamically 
> share a pool of "cluster swap" space, so any locally available swap that
> is not used could be utilized by other nodes in the cluster.

That I think is a 2.5 problem. One thing that has been talked about several
times now is removing all the swap special case crap from the mm and making
swap a file system. That removes special cases and means anyone can write
or use custom, or multiple swap filesystems, in theory including things like
swap over a shared GFS pool

But its not for 2.4, no way

Alan
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Swapping for diskless nodes

[Eric W. Biederman]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > the memory of a fast server could have much less latency that writing 
> > that page out to a local old, slow IDE disk. Clusters could even have
> > special high-bandwidth, low latency networks that could be used for
> > remote paging.
> > 
> > In a perfect world, all nodes in a cluster would be able to dynamically 
> > share a pool of "cluster swap" space, so any locally available swap that
> > is not used could be utilized by other nodes in the cluster.
> 
> That I think is a 2.5 problem. One thing that has been talked about several
> times now is removing all the swap special case crap from the mm and making
> swap a file system. That removes special cases and means anyone can write
> or use custom, or multiple swap filesystems, in theory including things like
> swap over a shared GFS pool
> 
> But its not for 2.4, no way

I don't know about that.  We already can swap over just about everything 
because we can swap over the loopback device.  So moving making the swapping
code do the right thing is not that big of an allowance, nor that
much of extra code so if 2.5 actually starts up I can see us doing that.

Eric

Re: Swapping for diskless nodes

[Eric W. Biederman]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > the memory of a fast server could have much less latency that writing 
> > that page out to a local old, slow IDE disk. Clusters could even have
> > special high-bandwidth, low latency networks that could be used for
> > remote paging.
> > 
> > In a perfect world, all nodes in a cluster would be able to dynamically 
> > share a pool of "cluster swap" space, so any locally available swap that
> > is not used could be utilized by other nodes in the cluster.
> 
> That I think is a 2.5 problem. One thing that has been talked about several
> times now is removing all the swap special case crap from the mm and making
> swap a file system. That removes special cases and means anyone can write
> or use custom, or multiple swap filesystems, in theory including things like
> swap over a shared GFS pool
> 
> But its not for 2.4, no way

I don't know about that.  We already can swap over just about everything 
because we can swap over the loopback device.  So moving making the swapping
code do the right thing is not that big of an allowance, nor that
much of extra code so if 2.5 actually starts up I can see us doing that.

Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Swapping for diskless nodes

[Rik van Riel]

On 9 Aug 2001, Eric W. Biederman wrote:

> I don't know about that.  We already can swap over just about
> everything because we can swap over the loopback device.

Last I looked the loopback device could deadlock your
system without you needing to swap over it ;)

Rik
--
IA64: a worthy successor to the i860.

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: Swapping for diskless nodes

[Rik van Riel]

On 9 Aug 2001, Eric W. Biederman wrote:

> I don't know about that.  We already can swap over just about
> everything because we can swap over the loopback device.

Last I looked the loopback device could deadlock your
system without you needing to swap over it ;)

Rik
--
IA64: a worthy successor to the i860.

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: Swapping for diskless nodes

[Eric W. Biederman]

Rik van Riel <riel@conectiva.com.br> writes:

> On 9 Aug 2001, Eric W. Biederman wrote:
> 
> > I don't know about that.  We already can swap over just about
> > everything because we can swap over the loopback device.
> 
> Last I looked the loopback device could deadlock your
> system without you needing to swap over it ;)

It wouldn't suprise me.  But the fact remains that in 2.4 we allow it.
And if we allw it there is little excuse for doing it wrong.

Actually except for network cases it looks easier to prevent deadlocks
on the swapping path than with the loop back devices.  We can call
aops->prepare_write_out when we place the page in the swap cache
to make certain we aren't over a hole in a file, and there is room in the
filesystem to store the data.

Eric

Re: Swapping for diskless nodes

[Eric W. Biederman]

Rik van Riel <riel@conectiva.com.br> writes:

> On 9 Aug 2001, Eric W. Biederman wrote:
> 
> > I don't know about that.  We already can swap over just about
> > everything because we can swap over the loopback device.
> 
> Last I looked the loopback device could deadlock your
> system without you needing to swap over it ;)

It wouldn't suprise me.  But the fact remains that in 2.4 we allow it.
And if we allw it there is little excuse for doing it wrong.

Actually except for network cases it looks easier to prevent deadlocks
on the swapping path than with the loop back devices.  We can call
aops->prepare_write_out when we place the page in the swap cache
to make certain we aren't over a hole in a file, and there is room in the
filesystem to store the data.

Eric
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/

Re: esssound.o once more

[Alan Cox]

> Here is messages, that was occured while 2.4.7-ac9 was compiling:
> drivers/sound/sounddrivers.o: In function `solo1_update_ptr':
> drivers/sound/sounddrivers.o(.text+0xd6e): undefined reference to `clear_advance'
> drivers/sound/sounddrivers.o: In function `solo1_write':

Don't compile with gcc 3.0 and it should be fine. Alternatively swap the
extern inline for static inline

Re: question on best "Linux" Internals book

[Alan Cox]

> obvious book is Bovet & Cesati 'Understanding the Linux Kernel' O'Reill=
> y
> 2001 - (it claims it even got by Alan Cox for checking... :-))

It did 8) If you want to understand the kernel its a good book. If your boss
just ordered you to write a Linux device driver then device drivers is the
book you want.

UtLK has a slightly textbookish air about it, but providing that doesn't 
bother you its a very good book

Re: Q: Kernel patching

[Alan Cox]

> > which already exists!  Skipping patch.
> 
> OK, I got it to work. My mistake was trying to use patch-kernel instead
> of just patch.

If you want the inbetween patches for patch-kernel or for going from ac10-11
etc look on www.bzimage.org.

Re: [PATCH] double DRM - fixes

[Alan Cox]

> BTW: Alan, do you think it would be worth to be able to compile both versions
>      together (in the modular case) ?

it seems tricky to get right. Vendors are going to know which X they shipped
and end users build with the right DRM...

Re: about mips IDE DMA disk problem

[Alan Cox]

> We port linux-2.2.12 to our mipsel evaluation board.

2.2.12 is ancient including remotely exploitable security holes. If you
update to a vaguely recent 2.2 kernel you'll also find there are ali1535
drivers although I dont think anyone has verified them on mips

Re: about mips IDE DMA disk problem

[Alan Cox]

> We port linux-2.2.12 to our mipsel evaluation board.

2.2.12 is ancient including remotely exploitable security holes. If you
update to a vaguely recent 2.2 kernel you'll also find there are ali1535
drivers although I dont think anyone has verified them on mips

Re: about mips IDE DMA disk problem

[Ilya Volynets]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 10 August 2001 08:59, Alan Cox wrote:
> > We port linux-2.2.12 to our mipsel evaluation board.
>
> 2.2.12 is ancient including remotely exploitable security holes. If you
> update to a vaguely recent 2.2 kernel you'll also find there are ali1535
> drivers although I dont think anyone has verified them on mips
Perhaps somebody should tell MIPS, Inc. people that 2.2.12 on their
site is little bit out of date.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iEYEARECAAYFAjt0InkACgkQtKh84cA8u2lxcwCdEM8mRkj63kpxnfaRU2tS9D86
QS8AnjVTo67VGpbgcgnn+nkuHVJW/DQN
=4ngI
-----END PGP SIGNATURE-----

Re: about mips IDE DMA disk problem

[Ilya Volynets]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 10 August 2001 08:59, Alan Cox wrote:
> > We port linux-2.2.12 to our mipsel evaluation board.
>
> 2.2.12 is ancient including remotely exploitable security holes. If you
> update to a vaguely recent 2.2 kernel you'll also find there are ali1535
> drivers although I dont think anyone has verified them on mips
Perhaps somebody should tell MIPS, Inc. people that 2.2.12 on their
site is little bit out of date.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iEYEARECAAYFAjt0InkACgkQtKh84cA8u2lxcwCdEM8mRkj63kpxnfaRU2tS9D86
QS8AnjVTo67VGpbgcgnn+nkuHVJW/DQN
=4ngI
-----END PGP SIGNATURE-----

Re: about mips IDE DMA disk problem

[Ralf Baechle]

On Fri, Aug 10, 2001 at 11:05:39AM -0700, Ilya Volynets wrote:

> > 2.2.12 is ancient including remotely exploitable security holes. If you
> > update to a vaguely recent 2.2 kernel you'll also find there are ali1535
> > drivers although I dont think anyone has verified them on mips
> Perhaps somebody should tell MIPS, Inc. people that 2.2.12 on their
> site is little bit out of date.

They're well aware of that - and working on 2.4

  Ralf

Re: about mips IDE DMA disk problem

[Ilya Volynets]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 10 August 2001 11:23, Ralf Baechle wrote:
> On Fri, Aug 10, 2001 at 11:05:39AM -0700, Ilya Volynets wrote:
> > > 2.2.12 is ancient including remotely exploitable security holes. If you
> > > update to a vaguely recent 2.2 kernel you'll also find there are
> > > ali1535 drivers although I dont think anyone has verified them on mips
> >
> > Perhaps somebody should tell MIPS, Inc. people that 2.2.12 on their
> > site is little bit out of date.
>
> They're well aware of that - and working on 2.4
Great! 'cause questions related to 2.2.12 kernel are becoming FAQ.
>   Ralf
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iEYEARECAAYFAjt0J3QACgkQtKh84cA8u2nQHQCfUBdRLJKhnulO0AWli04hxNjU
ozYAoKXOiffJtqb5EHVKRVNTbSKa+++k
=DiXG
-----END PGP SIGNATURE-----

Re: about mips IDE DMA disk problem

[Hartvig Ekner]

Hi,

Ilya Volynets writes:

> > They're well aware of that - and working on 2.4
> Great! 'cause questions related to 2.2.12 kernel are becoming FAQ.

You can already find 2.4 kernels on ftp.mips.com:

ftp> pwd
257 "/pub/linux/mips/kernel/2.4" is current directory.
ftp> ls -R
227 Entering Passive Mode (206,31,31,227,157,126)
150 Opening ASCII mode data connection for /bin/ls.
total 4
-rw-r--r--   1 9618     40           1613 Jul  6 04:41 README
drwxr-xr-x   2 9618     40            512 Jul  6 05:26 images
drwxr-xr-x   2 9618     40            512 Jul  6 05:27 src

images:
total 8720
-rw-r--r--   1 9618     40        1103571 Mar 30 05:23 vmlinux-2.4.1.atlas.eb-01.00.srec.gz
-rw-r--r--   1 9618     40        1129920 Mar 30 05:23 vmlinux-2.4.1.atlas.el-01.00.srec.gz
-rw-r--r--   1 9618     40        1061736 Mar 30 05:23 vmlinux-2.4.1.malta.eb-01.00.srec.gz
-rw-r--r--   1 9618     40        1093062 Mar 30 05:23 vmlinux-2.4.1.malta.el-01.00.srec.gz
-rw-r--r--   1 9618     40        1116378 Jul  6 04:35 vmlinux-2.4.3.atlas.eb-01.00.srec.gz
-rw-r--r--   1 9618     40        1144798 Jul  6 04:35 vmlinux-2.4.3.atlas.el-01.00.srec.gz
-rw-r--r--   1 9618     40        1075193 Jul  6 04:35 vmlinux-2.4.3.malta.eb-01.00.srec.gz
-rw-r--r--   1 9618     40        1107586 Jul  6 04:35 vmlinux-2.4.3.malta.el-01.00.srec.gz
 
src:
total 50200
-rw-r--r--   1 9618     40       25100716 Mar 30 05:24 linux-2.4.1.mips-src-01.00.tar.gz
-rw-r--r--   1 9618     40       26239788 Jul  6 04:35 linux-2.4.3.mips-src-01.00.tar.gz
226 Transfer complete.
ftp>

/Hartvig


-- 
 _    _   _____  ____     Hartvig Ekner        Mailto:hartvige@mips.com
 |\  /| | |____)(____                          Direct: +45 4486 5503
 | \/ | | |     _____)    MIPS Denmark         Switch: +45 4486 5555
T E C H N O L O G I E S   http://www.mips.com  Fax...: +45 4486 5556

Re: about mips IDE DMA disk problem

[Ilya Volynets]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

MIPS web site still lists 2.2.12 kernel as official one coming from
MIPS (http://www.mips.com/devTools/catalog_2001/Linux.html)
which is probably the reason everyone else is trying to port
that one. That's all I was saying. Probably this should have been
sent to web master.
On Friday 10 August 2001 11:41, Hartvig Ekner wrote:
> Hi,
>
> Ilya Volynets writes:
> > > They're well aware of that - and working on 2.4
> >
> > Great! 'cause questions related to 2.2.12 kernel are becoming FAQ.
>
> You can already find 2.4 kernels on ftp.mips.com:
>
> ftp> pwd
> 257 "/pub/linux/mips/kernel/2.4" is current directory.
> ftp> ls -R
> 227 Entering Passive Mode (206,31,31,227,157,126)
> 150 Opening ASCII mode data connection for /bin/ls.
> total 4
> -rw-r--r--   1 9618     40           1613 Jul  6 04:41 README
> drwxr-xr-x   2 9618     40            512 Jul  6 05:26 images
> drwxr-xr-x   2 9618     40            512 Jul  6 05:27 src
>
> images:
> total 8720
> -rw-r--r--   1 9618     40        1103571 Mar 30 05:23
> vmlinux-2.4.1.atlas.eb-01.00.srec.gz -rw-r--r--   1 9618     40       
> 1129920 Mar 30 05:23 vmlinux-2.4.1.atlas.el-01.00.srec.gz -rw-r--r--   1
> 9618     40        1061736 Mar 30 05:23
> vmlinux-2.4.1.malta.eb-01.00.srec.gz -rw-r--r--   1 9618     40       
> 1093062 Mar 30 05:23 vmlinux-2.4.1.malta.el-01.00.srec.gz -rw-r--r--   1
> 9618     40        1116378 Jul  6 04:35
> vmlinux-2.4.3.atlas.eb-01.00.srec.gz -rw-r--r--   1 9618     40       
> 1144798 Jul  6 04:35 vmlinux-2.4.3.atlas.el-01.00.srec.gz -rw-r--r--   1
> 9618     40        1075193 Jul  6 04:35
> vmlinux-2.4.3.malta.eb-01.00.srec.gz -rw-r--r--   1 9618     40       
> 1107586 Jul  6 04:35 vmlinux-2.4.3.malta.el-01.00.srec.gz
>
> src:
> total 50200
> -rw-r--r--   1 9618     40       25100716 Mar 30 05:24
> linux-2.4.1.mips-src-01.00.tar.gz -rw-r--r--   1 9618     40       26239788
> Jul  6 04:35 linux-2.4.3.mips-src-01.00.tar.gz 226 Transfer complete.
> ftp>
>
> /Hartvig
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.4 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iEYEARECAAYFAjt0POYACgkQtKh84cA8u2mXdACfV5qmuHKMuToSz5AwH+nAt8a2
CyUAnjdm1jtPtDaSNy13tXRNytvBpDMH
=gazA
-----END PGP SIGNATURE-----

Re: [PATCH] LVM snapshot support for reiserfs and others

[Alan Cox]

> ext3 will probably lock up on unmount with 2.4.8-pre8.  The fix is
> to replace fsync_dev with fsync_no_super in fs/ext3/super.c and
> fs/jbd/recovery.c.  I'll be generating a new patchset this weekend.

Actually if you dont fix recovery.c it will hang the machine when you mount
an fs that needs recovering. With the old patches at least umount of ext3
seems fine as is.

Alan

Re: [PATCH] Adaptec I2O RAID driver (kernel 2.4.7)

[Alan Cox]

Ok comments below. 

Summary suggestion:

Can you fix up the
	-	use of u32 for timeouts
	-	partition stuff for i386 (which really can just go)
	-	pci_resource_len usage
	-	multiply overflow
	-	EFAULT
	-	kmalloc check oddments

Don't bother with the static, header file oddments or the i2o core bugs
you inherited. Just let me know which you use and I'll figure out how to
make them shared.

At that point I think its fine to go in. 

Linus btw it might seem odd to add a driver for a specific i2o adapter but
the DPT's speak what is to say the least a very odd dialect of i2o, so it
does make sense.

> +#include <linux/autoconf.h>
> +#if defined(CONFIG_MODVERSIONS) && !defined(MODVERSIONS)
> +#	define MODVERSIONS
> +#endif
> +
> +#if defined MODVERSIONS && defined MODULE
> +#	include <linux/modversions.h>
> +#endif
> +

These will be automatic so should go - ok I can do that

> +MODULE_AUTHOR("Deanna Bonds, with _lots_ of help from Mark Salyzyn");
> +MODULE_DESCRIPTION("Adaptec I2O RAID Driver");
> +char kernel_version[] = UTS_RELEASE;

should be static char [otherwise it clashes built in]

> +static struct file_operations adpt_fops = {
> +	ioctl: adpt_ioctl,
> +	open: adpt_open,
> +	release: adpt_close
> +};
> +
> +
> +u8 adpt_read_blink_led(adpt_hba* host)

static - again not a big item, that can be done later anyway

> +#endif
> +	sht->use_new_eh_code = 1;

Excellent.  old_eh will go in 2.5 so its great you are using the new_eh

> +	printk("dpti: If you have a lot of devices this could take a few
> minutes.\n");

	       ^KERN_INFO


> +	for (pHba = hba_chain; pHba; pHba = pHba->next) {
> +		printk("%s: Reading the hardware resource table.\n",
                       KERN_DEBUG

> +int adpt_queue(Scsi_Cmnd * cmd, void (*done) (Scsi_Cmnd *))
> +{
> +	adpt_hba* pHba = NULL;
> +	struct adpt_device* pDev = NULL;	/* dpt per device
> information */
> +	u32 timeout = jiffies + (TMOUT_SCSI*HZ);
        ^unsigned long

> +	// TODO(defer) dmb - use marks algorythm
> +	if( disk->has_part_table == 1){
> +
> +
> +	}

Linux will trust partition tables for you.

> +		 * Read data from buffer (writing to us) - NOT SUPPORTED
> +		 */
> +		return -ENOSYS;

		-EINVAL

	[-ENOSYS would imply the write() syscall didnt exist]

> +	pci_read_config_dword(pDev,
> PCI_BASE_ADDRESS_0,(u32*)&base_addr0_phys);

No no no.. Dont do this, bad crap will happen on some setups. The length
is pci_resource_len(pDev, 0) 

> +	pci_read_config_word (pDev, PCI_SUBSYSTEM_ID, &subdevice);
> +


> +		//Use BAR1 in this config
> +		pci_read_config_dword(pDev,PCI_BASE_ADDRESS_1,
> (u32*)&base_addr1_phys);
> +		pci_write_config_dword(pDev,PCI_BASE_ADDRESS_1, 0xffffffff);
> +		pci_read_config_dword(pDev,PCI_BASE_ADDRESS_1,
> &hba_map1_area_size);

Again dont do this - use pci_resource_len

> +	base_addr_virt = (ulong)ioremap(base_addr0_phys,hba_map0_area_size);
> +	if(base_addr_virt == 0) {
> +		PERROR("dpti: adpt_config_hba: io remap failed\n");
> +		return -EINVAL;
> +	}
> +
> +        if(raptorFlag == TRUE) {
> +		msg_addr_virt = (ulong)ioremap(base_addr1_phys,
> hba_map1_area_size );
> +		if(msg_addr_virt == 0) {
> +			PERROR("dpti: adpt_config_hba: io remap failed on
> BAR1\n");


You need to unmap the base_addr_virt here or it leaks on error

> +	// Allocate and zero the data structure
> +	if( (pHba = kmalloc(sizeof(adpt_hba), GFP_KERNEL)) == NULL) {
> +		iounmap((void*)base_addr_virt);
	
		msg_addr_virt ??

> +/* Structures and definitions for synchronous message posting.
> + * See adpt_i2o_post_wait() for description
> + * */

Ah you borrowed the i2o post wait code before it was properly debugged. So
that..

> +	spin_unlock_irqrestore(&adpt_post_wait_lock, flags);
> +	kfree(wait_data);

Now in the timeout case this will cause the failed reply data to scribble
somewhere random, if the reply does turn up after the timeout. You actually
can't clean the buffer up until you know the event completed or you shut
down the controller.

> +	u32 timeout = jiffies + 30*HZ;
unsigned long for timers..
> +	do {
> +		rmb();
> +		m = readl(pHba->post_port);
> +		if (m != EMPTY_QUEUE) {

> + */		    	
> +int adpt_i2o_query_scalar(adpt_hba* pHba, int tid, 
> +			int group, int field, void *buf, int buflen)
> +{
> +	u16 opblk[] = { 1, 0, I2O_PARAMS_FIELD_GET, group, 1, field };

If you borrowed the i2o_query_scalar code from the older i2o code thats
also buggy (sorry Im human too 8)).

Can you give me a list of the i2o_ core routines that your driver uses
unchanged and I will do the magic so that we can share them (say an
i2o_lib module)

> +	// get user msg size in u32s 
> +	get_user(size, &user_msg[0]);

	if(get_user(...))
		return -EFAULT;

(so it reports errors nicely

> +	user_reply = &user_msg[size];
> +	size *= 4; // Convert to bytes
> +	if(size > MAX_MESSAGE_SIZE){
> +		return -EFAULT;

Check the size is ok for 1/4, then multiply - it might overflow, same on 
the reply

> +	get_user(reply_size, &user_reply[0]);
		-EFAULT...

> +#if defined __i386__
> +
> +#include <linux/mc146818rtc.h>
> +
> +	/* Get drive type for 1st drive*/
> +	i = CMOS_READ(0x12) >> 4;
> +	j = i >> 4;
> +	if (i == 0x0f) {
> +		j = CMOS_READ(0x19);
> +	}
> +	si->drive0CMOS = j;

Don't do this. You've no idea if there is even CMOS present, if the format
is what you expect of if it is talking to you

> +	/* Get number of drives sys bios found */
> +	si->numDrives = *((char *) phys_to_virt(0x475));

BIOS - again an assumption. Basically flush this code. Geom isnt a big
deal to the kernel, not all drivers even bother to set it, scsi is logical
block based so really doesnt care a bit.

> +	u32 timeout = jiffies + TMOUT_INITOUTBOUND*HZ;

unsigned long - that seems to be everywhere. Its kind of bad to get it wrong
as it means alphas mysteriously dont work after about 50 days 8)

> +/*
> + * dpti2oscsi2.c contains a table of scsi commands that is used to
> determine
> + * the data direction of the command.  It is used in dpt_scsi_to_i2o to
> speed
> + * up the building of the scsi message.

You don't need that in 2.4. The kernel tells you

> +				if( pDev == NULL){
> +					pDev =  kmalloc(sizeof(struct
> adpt_device),GFP_KERNEL);

Missing check for if this one fails..

> +
> +  /* Define the mutually exclusive semaphore type */
> +#define		SEMAPHORE_T	unsigned int *
> +  /* Define a handle to a DLL */
> +#define		DLL_HANDLE_T	unsigned int *
> +
> +#endif
> --- /dev/null	Sat Apr 14 07:06:21 2001
> +++ linux/drivers/scsi/dpti_i2o-dev.h	Fri Aug 10 14:49:34 2001
> @@ -0,0 +1,395 @@
> +/*
> + * I2O user space accessible structures/APIs

Some of these seem to be duplicates of existing includes so could be shared

Re: free_task_struct() called too early?

[Alan Cox]

> When a process terminates, it appears that the task structure is freed too
> early.  There are memory references to the kernel task area (task_struct and
> stack space) after free_task_struct(p) is called.
> 
> If I modify the following line in include/asm-i386/processor.h
> 
> #define free_task_struct(p)   free_pages((unsigned long) (p), 1) to
> #define free_task_struct(p)   memset((void*) (p), 0xf, PAGE_SIZE*2);
> free_pages((unsigned long) (p), 1)
> then kernel will boot to init and lockup on when first task terminates.
> 
> Has anyone looked into or aware of this issue?

2.4.8pre fixed a case with semaphores on the stack. It might not be the only
one. Your #define is wrong though, if a single free_task_struct path is an
if you will not do what you expect

	do { memset(), free_pages } while(0);

would be safer. 

I'd like to know if 2.4.8pre8 does it except on module unload (where it is
still buggy)

Alan

Re: free_task_struct() called too early?

[Linus Torvalds]

On Fri, 10 Aug 2001, Alan Cox wrote:
> >
> > If I modify the following line in include/asm-i386/processor.h
> >
> > #define free_task_struct(p)   free_pages((unsigned long) (p), 1) to
> > #define free_task_struct(p)   memset((void*) (p), 0xf, PAGE_SIZE*2);
> > free_pages((unsigned long) (p), 1)
> > then kernel will boot to init and lockup on when first task terminates.
> >
> > Has anyone looked into or aware of this issue?
>
> 2.4.8pre fixed a case with semaphores on the stack. It might not be the only
> one. Your #define is wrong though, if a single free_task_struct path is an
> if you will not do what you expect
>
> 	do { memset(), free_pages } while(0);
>
> would be safer.

No no no.

You can NOT do a "memset()" before you free the task-struct: it's can
still be actively used, and will be marked such by having an elevated page
count (ie the "free()" may not be doing anything but atomically decrement
the page count).

This is, in fact, a very very common thing - I bet that you will reboot
the first time you touch a /proc/<pid>/xx file or do a "ptrace()", because
that code will do the free_task_struct too after having incremented the
usage count while they held on to it.

If you really really want to, you can do the memset() at the top of
"free_pages_ok()" - it will slow down the system considerably (because
_every_ page will be memset after being free'd), but hey, it's going to be
an even better stress-tester.

Oh, if you do it there, do something like thisL

	memset(page_address(page), 0xf0, PAGE_SIZE << order);

and realize that the above will not work on HIGHMEM machines (and making
it work on highmem machines is simply not worth it).

		Linus

Re: VM nuisance

[Alan Cox]

> - How hard/how much time.. to implement resource baselines (or similar 
> concept)

Ben La Haise has been working on that part of things. Its non trivial. I've
been discussing changes with Rik for VM behaviour shifts when thrashing but
not OOM. We have some ideas but this is research

Re: [PATCH] Adaptec I2O RAID driver (kernel 2.4.7)

[Alan Cox]

> When I coded that, I took a snapshot of the i2o routines at 2.2.18 (I
> think).  I still have requirement to support customers running 2.2.12 with

Ok those will break in two cases - failed table reads and timeouts on post
waits that then complete later. Its probably ok since you know those wont
fail on your card

> support the older kernels.  I could not find a nice way of doing something
> similar in those kernels (2.2.12 again).  

2.2.12 has known and fixed security holes so nobody should be using it 8)
 
I'll read over this version tomorrow morning

Re: struct page to 36 (or 64) bit bus address?

[Alan Cox]

> There have been a few threads on this issue.  One of the core reasons
> the situation is unlikely to change in 2.4.x is that the scsi layer in
> it's current form makes the logic required for recovering from a DMA
> mapping failure aweful at best.

Actually its pretty trivial to resolve that I think. We already follow
precisely the right procedure for some other scsi events. So 

	scsi_retry_command(SCpnt);

would map to setting the error and returning.

Re: Bug report : Build problem with kernel 2.4.8

[Alan Cox]

> Problem: Can't compile kernel 2.4.8 with support of sound card CM8338A.

Its a bug in the code

> In file included from cmpci.c:90:
> /usr/local/src/kernel-source-2.4.8/include/linux/module.h:21: linux/mod=
> versions.h: No such file or directory

drivers/sound/cmpci.c included <linux/modversions.h> directly so broke if
you didnt have module versioning enabled. I've already sent Linus a fix
for this one

Re: PnP BIOS

[Alan Cox]

> I recently began a project that will make Linux a PnP OS (at least the 
> 386 arch).  I'm currently working on a PnP BIOS module for the Linux 

There is already one in Linux 2.4.8ac

Re: Hang problem on Tyan K7 Thunder resolved -- SB Live! heads-up

[Alan Cox]

> But later there was a patch from maintainer (Rui Sousa). Besides, there were no
> updates from maintainers for over a year, so I think "non maintainer" did a good
> thing - at least maintainers started to send patches finally. Instead of backing out
> I suggest for maintainers to send more patches...

I've got an even better idea. Its called making major problematic device
changes and debugging them in _UNSTABLE_ kernel trees - like waiting until
2.5 starts. Otherwise 2.4 will never be stable.

Its one thing to take something like the early USB in a stable kernel and
update it because it certainly won't make it worse, and another to update
an emu10k1 driver that worked with one that doesn't work, needs different
user tools and locks all SMP boxes.

Alan

Re: Linux 2.4.8-ac2

[Alan Cox]

> i just want to ask you (sorry for this consumer behavior)
> if you intend to integrate the xfs filesystem in your ac-series

I've discussed it with Martin Petersen a bit. Its not clear it makes
sense, since they kind of do chunks of their own vm (partly because they
had to) and other bits like acls that want to be out for the first one.
Right no I don't know what will occur. I think in part it depends when
Linus starts 2.5

> also .. do you know if it is planed to integrate the xfs fs
> into the main branch of the kernel ?

You'd have to ask Linus

Re: Linux 2.4.8-ac2

[Sven Goethel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Monday 13 August 2001 01:37, Alan Cox wrote:
> > i just want to ask you (sorry for this consumer behavior)
> > if you intend to integrate the xfs filesystem in your ac-series
>
> I've discussed it with Martin Petersen a bit. Its not clear it makes
> sense, since they kind of do chunks of their own vm (partly because they
> had to) and other bits like acls that want to be out for the first one.
> Right no I don't know what will occur. I think in part it depends when
> Linus starts 2.5
>
> > also .. do you know if it is planed to integrate the xfs fs
> > into the main branch of the kernel ?
>
> You'd have to ask Linus
>

thanxs alan

so i ask linus here ..

hello ;-)

dear linus

how about integrating the xfs fs into the main kernel branch ?

i guess 'ya all know how nice this would be ..

sorry for acting just like a wanting customer

thanxs for all the great work - cheers, sven
- -- 
mailto:sgoethel@jausoft.com
www   : http://www.jausoft.com ; pgp: http://www.jausoft.com/gpg/
voice : +49-521-2399440 ; fax : +49-521-2399442
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE7d0glHdOA30NoFAARAsJGAJ992d5j7Q7EOmxmfjGWaVLUfknlLQCgtLOa
wihwthQ6GewYn+lvAaARwt8=
=32Be
-----END PGP SIGNATURE-----

Re: 2.4.9-pre1 unresolved symbols in fat.o/smbfs.o

[Alan Cox]

> depmod: *** Unresolved symbols in /lib/modules/2.4.9-pre1/kernel/fs/fat/fat.o
> depmod: 	generic_file_llseek
> depmod: *** Unresolved symbols in /lib/modules/2.4.9-pre1/kernel/fs/smbfs/smbfs.o
> depmod: 	generic_file_llseek
>  

Oops my fault. My kernel/ksyms goes

EXPORT_SYMBOL(vfs_unlink);
EXPORT_SYMBOL(vfs_rename);
EXPORT_SYMBOL(vfs_statfs);
EXPORT_SYMBOL(generic_file_llseek);
EXPORT_SYMBOL(generic_read_dir);
EXPORT_SYMBOL(__pollwait);
EXPORT_SYMBOL(poll_freewait);


If you edit yours and drop that line in then rebuild from clean all should
be well

Re: 2.4.9-pre1 unresolved symbols in fat.o/smbfs.o

[Linus Torvalds]

On Mon, 13 Aug 2001, Alan Cox wrote:
>
> Oops my fault. My kernel/ksyms goes
>
> EXPORT_SYMBOL(vfs_unlink);
> EXPORT_SYMBOL(vfs_rename);
> EXPORT_SYMBOL(vfs_statfs);
> EXPORT_SYMBOL(generic_file_llseek);
> EXPORT_SYMBOL(generic_read_dir);
> EXPORT_SYMBOL(__pollwait);
> EXPORT_SYMBOL(poll_freewait);
>
> If you edit yours and drop that line in then rebuild from clean all should
> be well

Hmm.. You should probably also add "no_llseek" there..

		Linus

Re: 2.4.9-pre1 unresolved symbols in fat.o/smbfs.o

[Alessandro Suardi]

Linus Torvalds wrote:
> 
> On Mon, 13 Aug 2001, Alan Cox wrote:
> >
> > Oops my fault. My kernel/ksyms goes
> >
> > EXPORT_SYMBOL(vfs_unlink);
> > EXPORT_SYMBOL(vfs_rename);
> > EXPORT_SYMBOL(vfs_statfs);
> > EXPORT_SYMBOL(generic_file_llseek);
> > EXPORT_SYMBOL(generic_read_dir);
> > EXPORT_SYMBOL(__pollwait);
> > EXPORT_SYMBOL(poll_freewait);
> >
> > If you edit yours and drop that line in then rebuild from clean all should
> > be well
> 
> Hmm.. You should probably also add "no_llseek" there..

Don't know about that, but Alan's suggested fix is enough for
 a clean 2.4.9-pre1 build here. Thanks,

--alessandro

 "this is no time to get cute, it's a mad dog's promenade
  so walk tall, or baby don't walk at all"
                (Bruce Springsteen, 'New York City Serenade')

Re: 2.4.9-pre1 unresolved symbols in fat.o/smbfs.o

[Colonel]

In clouddancer.list.kernel, you wrote:
>
>
>On Mon, 13 Aug 2001, Alan Cox wrote:
>>
>> Oops my fault. My kernel/ksyms goes
>>
>> EXPORT_SYMBOL(vfs_unlink);
>> EXPORT_SYMBOL(vfs_rename);
>> EXPORT_SYMBOL(vfs_statfs);
>> EXPORT_SYMBOL(generic_file_llseek);
>> EXPORT_SYMBOL(generic_read_dir);
>> EXPORT_SYMBOL(__pollwait);
>> EXPORT_SYMBOL(poll_freewait);
>>
>> If you edit yours and drop that line in then rebuild from clean all should
>> be well
>
>Hmm.. You should probably also add "no_llseek" there..
>
>		Linus


Seems to need it:

/usr/src/linux }make  modules_install  >/dev/null

depmod: *** Unresolved symbols in /lib/modules/2.4.9-pre1/kernel/drivers/media/video/tvmixer.o
depmod:         no_llseek
depmod: *** Unresolved symbols in /lib/modules/2.4.9-pre1/kernel/drivers/media/video/videodev.o
depmod:         no_llseek
depmod: *** Unresolved symbols in /lib/modules/2.4.9-pre1/kernel/drivers/sound/sound.o
depmod:         no_llseek
depmod: *** Unresolved symbols in /lib/modules/2.4.9-pre1/kernel/fs/fat/fat.o
depmod:         generic_file_llseek




-- 
Windows 2001: "I'm sorry Dave ...  I'm afraid I can't do that."

Re: Lost interrupt with HPT370

[Alan Cox]

> I have just tried an HPT-366 card with IC35L040VER07 drives (latest DeskStar
> 41G ATA-100, although the card is only ATA-66) and could not get them to
> even let me create a filesystem without hard locking the machine. This was
> using 2.4.8-ac1 and 2.4.7-ac11. I wrote this off to motherboard/IDE card
> compatibility (or lack thereof), but could it still be an IDE driver issue?

Some HPT cards have problems with certain drives. I believe its a fixed
problem in newer boards or on those with updatable firmware if you update
the firmware itself

Check your drive is in the bad_ata100_5 and bad_ata66_4 list. If not add
it then rebuild and retry (drivers/ide/hpt366.c) - and let me know

Alan

Re: Lost interrupt with HPT370

[Kevin P. Fleming]

OK, I've downloaded a beta BIOS update, if that doesn't cure it I'll try
adding it to the "bad list" and let y'all know what happens.

----- Original Message -----
From: "Alan Cox" <alan@lxorguk.ukuu.org.uk>
To: "Kevin P. Fleming" <kevin@labsysgrp.com>
Cc: "Alan Cox" <alan@lxorguk.ukuu.org.uk>; <linux-kernel@vger.kernel.org>
Sent: Monday, August 13, 2001 4:37 AM
Subject: Re: Lost interrupt with HPT370


> > I have just tried an HPT-366 card with IC35L040VER07 drives (latest
DeskStar
> > 41G ATA-100, although the card is only ATA-66) and could not get them to
> > even let me create a filesystem without hard locking the machine. This
was
> > using 2.4.8-ac1 and 2.4.7-ac11. I wrote this off to motherboard/IDE card
> > compatibility (or lack thereof), but could it still be an IDE driver
issue?
>
> Some HPT cards have problems with certain drives. I believe its a fixed
> problem in newer boards or on those with updatable firmware if you update
> the firmware itself
>
> Check your drive is in the bad_ata100_5 and bad_ata66_4 list. If not add
> it then rebuild and retry (drivers/ide/hpt366.c) - and let me know
>
> Alan
>
>
>

[PATCH] Re: Lost interrupt with HPT370

[Kevin P. Fleming]

This turned out to be the problem. There is a new generation of IBM Deskstar
drives (that are really nice drives :-) that have the same incompatibility
with the HPT-366 chip as the older IBM drives did. Below is a patch to add
the entire series to the "bad drives" lists, although I personally have only
experienced the problem with the 41G model I think it's logical to assume
they'll all have the same problem.

Drives work fine once this patch is in place, although only in ATA-44 (weird
mode).

<snip>

> Check your drive is in the bad_ata100_5 and bad_ata66_4 list. If not add
> it then rebuild and retry (drivers/ide/hpt366.c) - and let me know
>
> Alan

--- linux/drivers/ide/hpt366.old Mon Aug 13 08:45:58 2001
+++ linux/drivers/ide/hpt366.c Mon Aug 13 22:43:43 2001
@@ -60,6 +60,11 @@
  "IBM-DTLA-305040",
  "IBM-DTLA-305030",
  "IBM-DTLA-305020",
+        "IC35L010AVER07-0",
+        "IC35L020AVER07-0",
+        "IC35L030AVER07-0",
+        "IC35L040AVER07-0",
+        "IC35L060AVER07-0",
  "WDC AC310200R",
  NULL
 };
@@ -75,6 +80,11 @@
  "IBM-DTLA-305030",
  "IBM-DTLA-305020",
  "WDC AC310200R",
+        "IC35L010AVER07-0",
+        "IC35L020AVER07-0",
+        "IC35L030AVER07-0",
+        "IC35L040AVER07-0",
+        "IC35L060AVER07-0",
  NULL
 };

Re: Lost interrupt with HPT370

[Wojtek Pilorz]

On Mon, 13 Aug 2001, Alan Cox wrote:

> Date: Mon, 13 Aug 2001 12:37:51 +0100 (BST)
> From: Alan Cox <alan@lxorguk.ukuu.org.uk>
> To: Kevin P. Fleming <kevin@labsysgrp.com>
> Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>, linux-kernel@vger.kernel.org
> Subject: Re: Lost interrupt with HPT370
> 
> > I have just tried an HPT-366 card with IC35L040VER07 drives (latest DeskStar
> > 41G ATA-100, although the card is only ATA-66) and could not get them to
> > even let me create a filesystem without hard locking the machine. This was
> > using 2.4.8-ac1 and 2.4.7-ac11. I wrote this off to motherboard/IDE card
> > compatibility (or lack thereof), but could it still be an IDE driver issue?
> 
> Some HPT cards have problems with certain drives. I believe its a fixed
> problem in newer boards or on those with updatable firmware if you update
> the firmware itself
> 
> Check your drive is in the bad_ata100_5 and bad_ata66_4 list. If not add
> it then rebuild and retry (drivers/ide/hpt366.c) - and let me know
> 
> Alan
> -
Alan,

What is the effect of putting or not a drive on bad_ata100_5 and
bad_ata66_4 ?

I am asking as I am using two 30GB IBM DTLA drives connected to HPT370
with kernel 2.2.19+Andre's ide patch, and I have noted that these drives
are present on bad_ata66_4 but not on bad_ata100_5 ?
Should I inderstand that IBM-DTLA-307030 would not be compatible with HPT
366, but *is* OK with HPT 370 running UDMA-100?


I have seen very good performance (15-30 MB/s of transfer with ide.patch,
without ide.patch performance is extremally poor at about 2-3 MB/s) and no
problems, except one serious problem, which made me feel rather bad;
I tried once to measure performance by running two dd command reading
from /dev/hde and /dev/hdg, respectively, concurrently; after dd command
finished reading HDD lamp stayed on until reboot, after which I lost
partition table on one of the disks; before the test I experimented with
hdparm which might have been not a good idea ...
I have not repeat this test later ..

HW: Abit MB with i440BX, FSB 133MHz, HPT 370 on-board,
ECC memory 133 MHz, Pentium-III 800MHz (133 MHz FSB)

Re: Hang problem on Tyan K7 Thunder resolved -- SB Live! heads-up

[Alan Cox]

> Does the new driver not work for you? There seems to be a bug at close()
> time, in that the driver uses "tasklet_unlock_wait()" instead of
> "tasklet_kill()" to kill the tasklets, and that wouldn't work reliably.

It hung my SMP box solid
It spews white noise on my box with surround speakers
And worked on the third

So I went back to the old one.

Alan

Re: Hang problem on Tyan K7 Thunder resolved -- SB Live! heads-up

[rui.p.m.sousa]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="us-ascii", Size: 748 bytes --]


Alan Cox writes:

>> Does the new driver not work for you? There seems to be a bug at close()
>> time, in that the driver uses "tasklet_unlock_wait()" instead of
>> "tasklet_kill()" to kill the tasklets, and that wouldn't work reliably.
> 
> It hung my SMP box solid
> It spews white noise on my box with surround speakers

Digital or analog speakers?

> And worked on the third
> 
> So I went back to the old one.
> 
> Alan
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/






--
Crie o seu Email Grátis no Clix em
http://registo.clix.pt/

Re: Hang problem on Tyan K7 Thunder resolved -- SB Live! heads-up

[Alan Cox]

> > Also, if you followed the other thread on the Tyan Thunder lockup,
> > you'll have noticed that it locked up under heavy PCI loads. At least on
> > that machine it stopped with the 2.4.8 driver.

No it merely made them less common. I think thats unrelated. Early AMD
chips had bugs, and the SB Live! is famous for triggering PCI bugs because
of the very unusual and tight PCI access patterns it generates - it was
the main trigger for the via audio corruption too (and the AMD chipset
shares common ancestry with the VIA..)

Alan

Re: Hang problem on Tyan K7 Thunder resolved -- SB Live! heads-up

[Alan Cox]

> > It hung my SMP box solid
> > It spews white noise on my box with surround speakers
> 
> Digital or analog speakers?

Analogue four output - I didnt know you had digital out working 

Re: Hang problem on Tyan K7 Thunder resolved -- SB Live! heads-up

[Tobias Ringstrom]

On Mon, 13 Aug 2001, Alan Cox wrote:

> > > It hung my SMP box solid
> > > It spews white noise on my box with surround speakers
> >
> > Digital or analog speakers?
>
> Analogue four output - I didnt know you had digital out working

Same for me, but I got rid of it by muting IGain (whatever that is),
and/or unplugging the microphone.

I did not investigate further.

/Tobias

Re: Hang problem on Tyan K7 Thunder resolved -- SB Live! heads-up

[Anton Altaparmakov]

At 13:35 13/08/01, Alan Cox wrote:
> > > It hung my SMP box solid
> > > It spews white noise on my box with surround speakers
> >
> > Digital or analog speakers?
>
>Analogue four output - I didnt know you had digital out working

Digital output on my SB Live! has been working since before the original 
emu10k1 driver went into the kernel if my memory hasn't flipped a few bits (-;

And the old driver worked absolutely fine for that matter on my Dual 
Celeron workstation...

Haven't tried the new 2.4.8 emu10k1 addition yet, and considering you are 
reporting it hangs on smp it's probably a waste of time except to confirm 
the hangs...

Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: IDE UDMA/ATA Suckage, or something else?

[Alan Cox]

> Currently, I am running a 2.4.7-ac10 SMP debug kernel on my K7 Thunder
> and I was hoping things would be better, and if not then at least I
> could see something in the logs if it did crash/lock. I also compiled
> NVidia driver with debugging enabled. Things are no better as the system
> still locks up frequently while playing Quake 3, and I can't even start
> Unreal Tournament without it locking and requiring a reset (SysRq,
> logging in remotely, etc. does not work). The logs tell me nothing.

Once you've touched the 3D stuff I dont actually care about bug reports that
boot even after 3d unloaded. I'm simply sick of fielding Nvidia's bug
reports.

> What I have found is that if I disable DMA on my IBM ATA100 drive, the
> system is quite stable (though it is slow as snot - running at a
> ridiculous 4.5MB/sec. as compared to 35MB/sec. with UDMA33/66 enabled).

You must disable IDE prefetch on the current versions of the AMD MP 
chipset, you may also need to enable "noapic". 

Re: IDE UDMA/ATA Suckage, or something else?

[Paul G. Allen]

Alan Cox wrote:
> 
> > Currently, I am running a 2.4.7-ac10 SMP debug kernel on my K7 Thunder
> > and I was hoping things would be better, and if not then at least I
> > could see something in the logs if it did crash/lock. I also compiled
> > NVidia driver with debugging enabled. Things are no better as the system
> > still locks up frequently while playing Quake 3, and I can't even start
> > Unreal Tournament without it locking and requiring a reset (SysRq,
> > logging in remotely, etc. does not work). The logs tell me nothing.
> 
> Once you've touched the 3D stuff I dont actually care about bug reports that
> boot even after 3d unloaded. I'm simply sick of fielding Nvidia's bug
> reports.
> 
> > What I have found is that if I disable DMA on my IBM ATA100 drive, the
> > system is quite stable (though it is slow as snot - running at a
> > ridiculous 4.5MB/sec. as compared to 35MB/sec. with UDMA33/66 enabled).
> 
> You must disable IDE prefetch on the current versions of the AMD MP
> chipset, you may also need to enable "noapic".

Unless I can do it with the kernel, I have no choice. The BIOS has no
prefetch setting (which, BTW, I had disabled on all my A7V133 boards).
So what about problems with non MP boards?

PGA

-- 
Paul G. Allen
UNIX Admin II/Network Security
Akamai Technologies, Inc.
www.akamai.com

Re: struct page to 36 (or 64) bit bus address?

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Sun, 12 Aug 2001 12:49:00 +0100 (BST)

   Actually its pretty trivial to resolve that I think. We already follow
   precisely the right procedure for some other scsi events. So 
   
   	scsi_retry_command(SCpnt);
   
   would map to setting the error and returning.

To make sure we're on the same wave length, are you suggesting
this is the kind of thing we'd call in a callback from the PCI
DMA support layer?

That's the core bit, some callback mechanism.  You need this to let
the drivers get "jump started" when IOMMU mappings become available
once more.

Later,
David S. Miller
davem@redhat.com

Re: struct page to 36 (or 64) bit bus address?

[Alan Cox]

>    Actually its pretty trivial to resolve that I think. We already follow
>    precisely the right procedure for some other scsi events. So 
>    
>    	scsi_retry_command(SCpnt);
>    
>    would map to setting the error and returning.
> 
> To make sure we're on the same wave length, are you suggesting
> this is the kind of thing we'd call in a callback from the PCI
> DMA support layer?

Well that would be an ugly layer violation, but how about

	scsi_retry_command_waitq(SCpnt, &dma_waitq)

?

Re: struct page to 36 (or 64) bit bus address?

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Mon, 13 Aug 2001 15:09:31 +0100 (BST)

   > To make sure we're on the same wave length, are you suggesting
   > this is the kind of thing we'd call in a callback from the PCI
   > DMA support layer?
   
   Well that would be an ugly layer violation, but how about
   
   	scsi_retry_command_waitq(SCpnt, &dma_waitq)
   
   ?

I don't mean "PCI DMA support layer knows scsi routines to
call", rather I mean:

... register_new_scsi_host() ...

	add_notifier(&pci_dma_freespace_notifier, &host->dma_notifier_block);

And net drivers would do something similar, registering something
that will do "netdev_wake_queue();" etc.

Also, the DMA support layer must either:

1) require users of it to provide a kernel thread in which
   to execute these things in the correct context

2) have a kernel thread of it's own to do this

3) or somehow else be able to accept this notification
   from any kind of execution context

The notifier is capable of happening anywhere, anytime.
Right?

Later,
David S. Miller
davem@redhat.com

Re: struct page to 36 (or 64) bit bus address?

[Gérard Roudier]

On Mon, 13 Aug 2001, David S. Miller wrote:

>    From: Alan Cox <alan@lxorguk.ukuu.org.uk>
>    Date: Mon, 13 Aug 2001 15:09:31 +0100 (BST)
>
>    > To make sure we're on the same wave length, are you suggesting
>    > this is the kind of thing we'd call in a callback from the PCI
>    > DMA support layer?
>
>    Well that would be an ugly layer violation, but how about
>
>    	scsi_retry_command_waitq(SCpnt, &dma_waitq)
>
>    ?
>
> I don't mean "PCI DMA support layer knows scsi routines to
> call", rather I mean:
>
> ... register_new_scsi_host() ...
>
> 	add_notifier(&pci_dma_freespace_notifier, &host->dma_notifier_block);
>
> And net drivers would do something similar, registering something
> that will do "netdev_wake_queue();" etc.
>
> Also, the DMA support layer must either:
>
> 1) require users of it to provide a kernel thread in which
>    to execute these things in the correct context
>
> 2) have a kernel thread of it's own to do this
>
> 3) or somehow else be able to accept this notification
>    from any kind of execution context
>
> The notifier is capable of happening anywhere, anytime.
> Right?

That's the major problem if we ever want to preserve some ordering in the
queuing of SCSI IOs. It is not clear to me why a thread may help here. A
thread may help if you actually want to thread the queuing and/or if you
want to wait on resource. Note that using a thread for the queuing might
be a solution for preserving queuing ordering, but it is not the sole
solution.

Defunct T10/CAM addresses ordering problems especially during error
recovery by a freeze/unfreeze mechanism of device queues handled by SIMs
(SIMs are low-level drivers under Linux). Using threads, call-backs or any
other evil things on top of this mechanism can be a matter of O/S
dependent issues, but it is not the solution of the problem.

> Later,
> David S. Miller
> davem@redhat.com

Later,
Gérard P. Roudier
groudier@free.fr

:)

Re: struct page to 36 (or 64) bit bus address?

[David S. Miller]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: Text/Plain; charset=big5, Size: 812 bytes --]

   From: Gérard Roudier <groudier@free.fr>
   Date: Mon, 13 Aug 2001 21:07:50 +0200 (CEST)

   That's the major problem if we ever want to preserve some ordering in the
   queuing of SCSI IOs.

When DMA mapping operation fails, you simply "stop queueing".  Queue
freezes and nothing new is executed.

DMA wakeup makes you start where you left off.  I cannot see any
ordering constraints violated by this as a side effect.  It is like a
"cork" for running scsi commands in the driver.

The purpose of the hypothetical kernel thread is to get out
of interrupt context if that is deemed necessary.

It may not be.

Later,
David S. Miller
davem@redhat.com
ý:.žË›±Êâmçë¢kaŠÉb²ßìzwm…ébïîžË›±Êâmébžìÿ‘êçz_âžØ^n‡r¡ö¦zË\x1aëh™¨è­Ú&£ûàz¿äz¹Þ—ú+€Ê+zf£¢·hšˆ§~†­†Ûiÿÿïêÿ‘êçz_è®\x0fæj:+v‰¨þ)ߣømšSåy«\x1e­æ¶\x17…\x01\x06­†ÛiÿÿðÃ\x0fí»\x1fè®\x0få’i\x7f

Re: struct page to 36 (or 64) bit bus address?

[Alan Cox]

> And net drivers would do something similar, registering something
> that will do "netdev_wake_queue();" etc.

I don't want a callback, or if you give me a callback for scsi I'll just
turn it into a callback to wake_up(). Thats all I happen to need. net
drivers might be different

Alan

Re: add Tvia cyberpro 5050 to sound/trident.c

[Alan Cox]

> This chip is common in settop boxes. I don't know if you can
> buy any PCI cards "off the shelf".
> I think there is still interest in support for this 
> "old" chip (3 years old), at least in the "embedded market".

Ok then send me diffs

Re: IDE UDMA/ATA Suckage, or something else?

[Alan Cox]

> > You must disable IDE prefetch on the current versions of the AMD MP
> > chipset, you may also need to enable "noapic".
> 
> Unless I can do it with the kernel, I have no choice. The BIOS has no
> prefetch setting (which, BTW, I had disabled on all my A7V133 boards).
> So what about problems with non MP boards?

Well its entirely possible that the BIOS vendor did the right thing and
made sure you couldnt turn it on. How does your box behave with noapic ?

Re: IDE UDMA/ATA Suckage, or something else?

[Paul G. Allen]

Alan Cox wrote:
> 
> > > You must disable IDE prefetch on the current versions of the AMD MP
> > > chipset, you may also need to enable "noapic".
> >
> > Unless I can do it with the kernel, I have no choice. The BIOS has no
> > prefetch setting (which, BTW, I had disabled on all my A7V133 boards).
> > So what about problems with non MP boards?
> 
> Well its entirely possible that the BIOS vendor did the right thing and
> made sure you couldnt turn it on. How does your box behave with noapic ?

I haven't had a chance to try it yet. I'll let the lists know when I do.

PGA

-- 
Paul G. Allen
UNIX Admin II/Programmer
Akamai Technologies, Inc.
www.akamai.com
Work: (858)909-3630
Cell: (858)395-5043

Re: Are we going too fast?

[Alan Cox]

> > Welcome to wacky hardware. To get a G400 stable on x86 you need at least
> > 
> > XFree86 4.1 if you are running hardware 3D (and DRM 4.1)
> > 2.4.8 or higher with the VIA fixes
> > Preferably a very recent BIOS update for the VIA box
> 
> I'm sorry, but what "VIA fixes" are we referring to?

Certain VIA chipsets had some nasty bugs that caused corruption. The older
kernels have a workaround that mostly does the job but has a few side
effects. The 2.4.8 kernel has the official VIA provided workaround, which
makes sbpci128 cards work again, and sorts out some bus hangs, especially
with matrox cards

Re: Are we going too fast?

[Anthony Barbachan]

> > > Welcome to wacky hardware. To get a G400 stable on x86 you need at
least
> > >
> > > XFree86 4.1 if you are running hardware 3D (and DRM 4.1)
> > > 2.4.8 or higher with the VIA fixes
> > > Preferably a very recent BIOS update for the VIA box
> >
> > I'm sorry, but what "VIA fixes" are we referring to?
>
> Certain VIA chipsets had some nasty bugs that caused corruption. The older
> kernels have a workaround that mostly does the job but has a few side
> effects. The 2.4.8 kernel has the official VIA provided workaround, which
> makes sbpci128 cards work again, and sorts out some bus hangs, especially
> with matrox cards

    Could these "fixes" resolve any issues with the vt82c686a Southbridge?
For the life of me I have yet to be able to get my FIC VA-503A (that uses a
vt82c686a Southbridge for UDMA66 support) working correctly under Linux
2.4.x (or 2.2.x with the enhanced IDE patch) with DMA enabled by default.
And yes I have already tried switching the 80 pin cables 7 times.  Heck, I
even get CRC errors on UDMA33 drives using 40 pin cables; albeit a lesser
amount.  I have also noticed a hanging issue on a FIC VA-503+ board in which
the PC speaker can hang, in mid beep, along with the system for a short
while occasionally when the speaker issues a beep.  By the way, any ideas on
how I can help debug this particular problem?  There is no Ooops so I am not
sure how I can help out.  Both systems otherwise work very well and
perfectly on Win9x, Win2k, FreeBSD, and OpenBSD.  I'm starting to take a
very dim view of Linux on VIA boards.

Re: 2.4.9-pre2 breaks UFS as a module

[Alan Cox]

> make[2]: Entering directory `/share/src/linux-2.4.9-pre2/fs/ufs'
> gcc -D__KERNEL__ -I/share/src/linux-2.4.9-pre2/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=i686
> -DMODULE   -c -o file.o file.c
> file.c:80: `generic_file_open' undeclared here (not in a function)
> file.c:80: initializer element is not constant
> file.c:80: (near initialization for `ufs_file_operations.open')
> make[2]: *** [file.o] Error 1
> make[2]: Leaving directory `/share/src/linux-2.4.9-pre2/fs/ufs'
> make[1]: *** [_modsubdir_ufs] Error 2
> make[1]: Leaving directory `/share/src/linux-2.4.9-pre2/fs'
> make: *** [_mod_fs] Error 2

I'll take a look at that, Im trying to merge VFS things with Linus and its
non trivial to get all the bits in place and in the right order 8(

Re: 2.4.9-pre2 breaks UFS as a module

[Alessandro Suardi]

Alan Cox wrote:
> 
> > make[2]: Entering directory `/share/src/linux-2.4.9-pre2/fs/ufs'
> > gcc -D__KERNEL__ -I/share/src/linux-2.4.9-pre2/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=i686
> > -DMODULE   -c -o file.o file.c
> > file.c:80: `generic_file_open' undeclared here (not in a function)
> > file.c:80: initializer element is not constant
> > file.c:80: (near initialization for `ufs_file_operations.open')
> > make[2]: *** [file.o] Error 1
> > make[2]: Leaving directory `/share/src/linux-2.4.9-pre2/fs/ufs'
> > make[1]: *** [_modsubdir_ufs] Error 2
> > make[1]: Leaving directory `/share/src/linux-2.4.9-pre2/fs'
> > make: *** [_mod_fs] Error 2
> 
> I'll take a look at that, Im trying to merge VFS things with Linus and its
> non trivial to get all the bits in place and in the right order 8(

I can confirm it builds okay in -pre3, thanks ! I've used it only
 once as Digital Unix support folks provided a small patch on a
 floppy to a customer, then said customer noticed Windows could not
 recognize the contents - it was an UFS formatted floppy...

--alessandro

 "this is no time to get cute, it's a mad dog's promenade
  so walk tall, or baby don't walk at all"
                (Bruce Springsteen, 'New York City Serenade')

Re: how to install redhat linux to a scsi disk for which driver is no

[Alan Cox]

> Can somebody guide me on how to install RedHat Linux (e.g. 7.1)
> to a scsi disk which is connected to a scsi controller for which the 
> driver is not present on the installation media ? 

-> redhat-list@redhat.com

Re: NTFS R-Only error

[Alan Cox]

> I am not sure who currently maintains the NTFS driver so I am posting it
> here. It doesn't look to be in the kernel so maybe mount or shell?

It looks like you fed random files to strings and it tried to parse them as
x86 elf files. See the man page and tell it they are raw binary

Re: DoS tmpfs,ramfs, malloc, saga continues

[Alan Cox]

> Good job for Alan Cox, with his patch i could run the
> sample program 20 times before it crashed the system.
> I cannot belive that instead of keeping this
> masskiller for deathlocks, no way out situation, and
> as last resort, You The Great Kernel Hackers use it
> for every day and every body solution. Is it so hard
> to make more memory checks for user level malloc?
> Windows does. Every other OS does. Why linux doesn't? 
> Fix it in the next kernel!

ROTFL

If you cant crash windows nt by running out of memory you aren't doing the
right things.  Just for example feed the Nvidia direct X an infinite number
of objects to draw.

The basic answer is "Set resource limits". The default ones fit typical
usage of a system well but not your case. The more complex answer is to
provide the option for very precise group based resource accounting (aka
the beancounter patch). That is for those who want to pay the probable 2%
or so system penalty for being able to precisely manage a system resource
set. With the beancounter infrastructure you can then get to the point where
student processes are being OOM killed while the admins quake session is
unharmed.

Somewhere in the middle there is address space accounting, where you never 
allow the total address space to violate

	(read-only-pages +
		foreach writeable page
			number of extant copies +
				numer of COW potential copies
	) - kernel fudge factor (space the kernel might need)
	
			>

		    ram + swap


That is much more lightweight and covers all the general cases. However
nobody wants it much as is evident by the fact nobody has either contributed
code or paid for the work to be done

Alan

Re: DoS tmpfs,ramfs, malloc, saga continues

[Szabolcs Szakacsits]

On Tue, 14 Aug 2001, Alan Cox wrote:

> If you cant crash windows nt by running out of memory you aren't doing the
> right things.

Alan, although I didn't completely get what patch Ivan refered, 2.4.8
seems to be the far most instable when OOM happens (it just never
survived the 2nd wave of memory stressing for me).

> The basic answer is "Set resource limits". The default ones fit typical

Sure, every experienced user knows that but many people are lazy, busy,
etc to do it properly and most newcomers, desktop users basically don't
want to know about it. Forget distributors setting unreasonable limits
for them, kernel just shouldn't crash in OOM - it could handle it in the
past. This OOM issue wasn't such big problem in the past but recently
it's really getting scary. I've ported the OOM killer to 2.2 and added
reserved root VM support and AFAIR I mentioned only once here, google
indexed the page and people hits it increasingly -- about 10-20 times
more often then OOM problems pop up here. Most of them USER_AGENT
contains the Windows word - and yes, they are explicitely looking for
solution for Linux related OOM problems.

> usage of a system well but not your case. The more complex answer is to
> provide the option for very precise group based resource accounting (aka
> the beancounter patch). That is for those who want to pay the probable 2%
> or so system penalty for being able to precisely manage a system resource
> set. With the beancounter infrastructure you can then get to the point where

It's not ready for use. It was touched last time about one year ago for
2.4.0-test7.

> Somewhere in the middle there is address space accounting, where you never
> allow the total address space to violate

BTW, 2.4 has another feature. vm_enough_memory() in 2.2 underestimated
the available free memory (memory is overcommited anyway) and apps got
more frequently ENOMEM. 2.4 overestimates free memory (because
e.g. nobody knows how much cache could be freed) so apps get killed
mostly by OOM killer.

[ ... non-overcommitting algorithm ...]
> That is much more lightweight and covers all the general cases. However
> nobody wants it much as is evident by the fact nobody has either contributed
> code or paid for the work to be done

In most cases I would definitely agree with this argument but I think in
this case people who have this problem either can't code it themself or
"can't" pay for it - they just expect (unconsciously) Linux can handle
OOM just as good as others or even better.

	Szaka

Re: DoS tmpfs,ramfs, malloc, saga continues

[Andrey Savochkin]

On Wed, Aug 15, 2001 at 05:20:09PM +0300, Szabolcs Szakacsits wrote:
> 
> > usage of a system well but not your case. The more complex answer is to
> > provide the option for very precise group based resource accounting (aka
> > the beancounter patch). That is for those who want to pay the probable 2%
> > or so system penalty for being able to precisely manage a system resource
> > set. With the beancounter infrastructure you can then get to the point where
> 
> It's not ready for use. It was touched last time about one year ago for
> 2.4.0-test7.

I just can't spare time now to maintain the patch properly and make a port
for each kernel release.
When 2.5 is started, I've promised myself to put other things aside and work
on the patch to have it in the mainstream.

	Andrey

Re: Are we going too fast?

[Alan Cox]

> Mike didn't mention any details of the hardware where he's experiencing this
> bug, but is it possibly a multiprocessor machine?
> Since I only have UP's to test on, the qnxfs might have SMP issues.
> 
> Could someone please glance through the code in fs/qnx4 to check if there
> are any obvious problems?

If I get time tomorrow I'll test the qnxfs code on a dual PPro

Re: Are we going too fast?

[PinkFreud]

On Tue, 14 Aug 2001, Alan Cox wrote:

> Date: Tue, 14 Aug 2001 21:47:03 +0100 (BST)
> From: Alan Cox <alan@lxorguk.ukuu.org.uk>
> To: Anders Larsen <anders@alarsen.net>
> Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>, PinkFreud <pf-kernel1@mirkwood.net>,
>      linux-kernel@vger.kernel.org
> Subject: Re: Are we going too fast?
> 
> > Mike didn't mention any details of the hardware where he's experiencing this
> > bug, but is it possibly a multiprocessor machine?
> > Since I only have UP's to test on, the qnxfs might have SMP issues.
> > 
> > Could someone please glance through the code in fs/qnx4 to check if there
> > are any obvious problems?
> 
> If I get time tomorrow I'll test the qnxfs code on a dual PPro


Woah, good point.  While the system I'm trying this on is only a single
cpu machine (AMD K6), I noticed (after I compiled, alas) that the kernel
was compiled with the SMP option enabled - whoops.  I meant to change that
back to UP, but haven't done so as of yet.


Linux boromir 2.4.8 #1 SMP Sun Aug 12 14:08:25 EDT 2001 i586 unknown

processor	: 0
vendor_id	: AuthenticAMD
cpu family	: 5
model		: 6
model name	: AMD-K6tm w/ multimedia extensions
stepping	: 1
cpu MHz		: 199.684
cache size	: 64 KB
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 1
wp		: yes
flags		: fpu vme de pse tsc msr mce cx8 mmx
bogomips	: 398.13


	Mike Edwards

Brainbench certified Master Linux Administrator
http://www.brainbench.com/transcript.jsp?pid=158188
-----------------------------------
Unsolicited advertisments to this address are not welcome.

Re: 2.4.8 Resource leaks + limits

[Alan Cox]

> Linux has had (for a while now) a "struct user" that is actually quickly
> accessible through a direct pointer off every process that is associated
> with that user, and we could (and _will_) start adding these kinds of
> limits. However, part of the problem is that because the limits haven't
> historically existed, there is also no accepted and nice way of setting
> the limits.

For that to work we need to seperate struct user from the uid a little, or
provide heirarchical pools (which seems overcomplex). Its common to want
to take a group of users (eg the chemists) and give them a shared limit
rather than per user limits

Alan

Re: 2.4.8 Resource leaks + limits

[Linus Torvalds]

On Wed, 15 Aug 2001, Alan Cox wrote:
>
> > Linux has had (for a while now) a "struct user" that is actually quickly
> > accessible through a direct pointer off every process that is associated
> > with that user, and we could (and _will_) start adding these kinds of
> > limits. However, part of the problem is that because the limits haven't
> > historically existed, there is also no accepted and nice way of setting
> > the limits.
>
> For that to work we need to seperate struct user from the uid a little, or
> provide heirarchical pools (which seems overcomplex). Its common to want
> to take a group of users (eg the chemists) and give them a shared limit
> rather than per user limits

No, I think the answer there is to do all the same things for "struct
group" as we do for user.

Yes, it would mean that the primary group is _really_ primary, but from a
system management standpoint that's probably preferable (ie you can give
group read-write access to a person without giving group "resource" access
to him)

		Linus

Re: 2.4.8 Resource leaks + limits

[Alan Cox]

> Yes, it would mean that the primary group is _really_ primary, but from a
> system management standpoint that's probably preferable (ie you can give
> group read-write access to a person without giving group "resource" access
> to him)

Non unix systems generally have a separate accounting uid - one reason for
that is the problem of charging for setuid apps and things done in your
name. Otherwise its all too easy to attack a bug in a setuid app to make it
expand to fill memory.

I'd rather we had an luid or equivalent personally.

Alan

Re: 2.4.8 Resource leaks + limits

[Ingo Oeser]

On Wed, Aug 15, 2001 at 09:53:09AM -0700, Linus Torvalds wrote:
> > For that to work we need to seperate struct user from the uid a little, or
> > provide heirarchical pools (which seems overcomplex). Its common to want
> > to take a group of users (eg the chemists) and give them a shared limit
> > rather than per user limits
> 
> No, I think the answer there is to do all the same things for "struct
> group" as we do for user.
 
Not really. Large installations use ACLs instead of groups. 

Why? Because if we have 2^31 users, there might be a slight
chance, that we need more then 32 group memberships per user.

So let's better stop relying more and more on this group brain
damage and start supporting ACLs. We can support building ACL
groups, but please let the user and not the admin build them.

It's called user data, because the user owns it and should
decide, which people are allowed to access it. 

Please look at AFS groups, to see what I mean.

All serious admins I know miss the ACL feature in Linux. One
product even emulates them via groups.


Regards

Ingo Oeser
-- 
I just found out that the brain is like a computer.
If that's true, then there really aren't any stupid people.
Just people running Windows.

Re: 2.4.8 Resource leaks + limits

[Alan Cox]

> Not really. Large installations use ACLs instead of groups. 

Umm you can't use ACL's for resource management. You have to be able to
charge an entity. Its not a permission to access, its a "who is paying" and
that requires a real entity to charge to

Re: 2.4.8 Resource leaks + limits

[Jesse Pollard]

> > Not really. Large installations use ACLs instead of groups. 
> 
> Umm you can't use ACL's for resource management. You have to be able to
> charge an entity. Its not a permission to access, its a "who is paying" and
> that requires a real entity to charge to

And that calls for an accounting id or a project id. As well as the possiblity
of a user having multiple accounting ids.

-------------------------------------------------------------------------
Jesse I Pollard, II
Email: pollard@navo.hpc.mil

Any opinions expressed are solely my own.

Re: 2.4.8 Resource leaks + limits

[Anton Altaparmakov]

At 21:15 15/08/2001, Alan Cox wrote:
> > Not really. Large installations use ACLs instead of groups.
>
>Umm you can't use ACL's for resource management. You have to be able to
>charge an entity. Its not a permission to access, its a "who is paying" and
>that requires a real entity to charge to

While we are on this topic, wouldn't it make sense to introduce unique 
identifiers, which can be associated with users, groups, or any other 
kernel object for that matter, then this is the entity you charge. The 
kernel can then map the id to the user or group (or whatever object).

When ACLs are introduced they would grant/deny permissions and in general 
operate only on unique identifiers.

This would have the benefit that the identifiers can be made sufficiently 
unique to work on a whole network (or even larger scales), which would make 
user management much easier for large corporations, much akin to what 
Netware and Windows servers do in fact...

Just my 2p.

Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: 2.4.8 Resource leaks + limits

[Jesse Pollard]

On Wed, 15 Aug 2001, Ingo Oeser wrote:
>On Wed, Aug 15, 2001 at 09:53:09AM -0700, Linus Torvalds wrote:
>> > For that to work we need to seperate struct user from the uid a little, or
>> > provide heirarchical pools (which seems overcomplex). Its common to want
>> > to take a group of users (eg the chemists) and give them a shared limit
>> > rather than per user limits
>> 
>> No, I think the answer there is to do all the same things for "struct
>> group" as we do for user.
> 
>Not really. Large installations use ACLs instead of groups. 
>
>Why? Because if we have 2^31 users, there might be a slight
>chance, that we need more then 32 group memberships per user.
>
>So let's better stop relying more and more on this group brain
>damage and start supporting ACLs. We can support building ACL
>groups, but please let the user and not the admin build them.
>
>It's called user data, because the user owns it and should
>decide, which people are allowed to access it. 
>
>Please look at AFS groups, to see what I mean.
>
>All serious admins I know miss the ACL feature in Linux. One
>product even emulates them via groups.

ACLs are good and very usefull.

HOWEVER, there are cases of users giving away their files to users
that are not authorized to recieve that data.

The advantage of groups is that the facility managment defines the
list of users authorized to view the data. It up to the user to
grant/deny that group access authorization.

Alternatively, it is possible to view the system as you describe - the
user can add others to the ACL to grant access. There should still be
some method that facility management can deny access.... On many systems
(Trusted Solaris, UNICOS, Trix,...) this is done with compartmentalization.
Now, it is subsets of the members of the compartment that the user can
grant access to.

Still more flexible than generic groups, but more restricted than no
limits on members of the ACL.

-- 
-------------------------------------------------------------------------
Jesse I Pollard, II
Email: jesse@cats-chateau.net

Any opinions expressed are solely my own.

Re: 2.4.8 Resource leaks + limits

[Horst von Brand]

Linus Torvalds <torvalds@transmeta.com> said:

[...]

> No, I think the answer there is to do all the same things for "struct
> group" as we do for user.
> 
> Yes, it would mean that the primary group is _really_ primary, but from a
> system management standpoint that's probably preferable (ie you can give
> group read-write access to a person without giving group "resource" access
> to him)

No good. Some setups (e.g. Red Hat) have a group for each user.
-- 
Horst von Brand                             vonbrand@sleipnir.valparaiso.cl
Casilla 9G, Vin~a del Mar, Chile                               +56 32 672616

Re: Implications of PG_locked and reference count in page structures....

[Alan Cox]

> So, what I want to do is this: given a pointer to a previously 
> kmalloc'ed block, and the length of that block, I want to (a) identify 
> each page associated with the block and (b) lock each page. It appears 
> that I can lock the page either by incrementing it's reference count, or 
> by setting the PG_locked flag for the page.

If the block was allocated with kmalloc it isnt pageable. That means all you
need to do is to use the virt_to_bus() call. If you are doing PCI DMA
however the preferred approach from 2.4 onwards is the pci_alloc_* API
(see Documentation/DMA-mapping.txt)

> Which method is preferred? Is there another method I should be using 
> instead?

You should be fine. If they were user pages then you would want to look
at the kiovec support for mapping user pages and locking them down. For
kernel allocated buffers it is nice and easy.

The only other oddment to note is that ISA bus DMA requires GFP_DMA as a 
kmalloc flag, PCI DMA does not (GFP_DMA is "below 16Mb please")

Alan

Re: Via chipset

[Alan Cox]

> Alan Cox wrote:
> > We know it happens on some boards that apparently cant keep up. We dont know
> > why, there is no time estimate for a cure. That unfortunately is about it
> 
> FWIW (qualitative data point), my EPoX system with the VIA chipset seems to run a
> few *hours* without an oops when I boot a PIII kernel and run it with X, but a few
> *days* on the same kernel when I don't start X.
> 
> Sometimes it barfs early even without X, but there seems to be a significant
> difference in the expected uptime between using X and not using X.

If you are running XFree servers then provide info on the card, the machine
configuration and XFree version, whether you use DRI or not. Also send the
same info to XFree86 themselves. It's entirely possible the Xserver is
the trigger some of the bugs, and getting the info to both parties will
really help

Re: glibc

[Alan Cox]

> I am porting Linux version 2.2.12 to Mips R3000 and need to build glibc
> but I could not find the following files:

Oh no not again.

2.2.12 is historical value only. It has remote vulnerabilities and shouldnt
be used for anything.

> 	glibc-2.0.6.tar.gz

glibc 2.0 is also obsolete, heavily so

Re: glibc

[Alan Cox]

> I am porting Linux version 2.2.12 to Mips R3000 and need to build glibc
> but I could not find the following files:

Oh no not again.

2.2.12 is historical value only. It has remote vulnerabilities and shouldnt
be used for anything.

> 	glibc-2.0.6.tar.gz

glibc 2.0 is also obsolete, heavily so

Re: glibc

[Brian Murphy]

Alan Cox wrote:

> > I am porting Linux version 2.2.12 to Mips R3000 and need to build glibc
> > but I could not find the following files:
>
> Oh no not again.
>
> 2.2.12 is historical value only. It has remote vulnerabilities and shouldnt
> be used for anything.
>
> >       glibc-2.0.6.tar.gz
>
> glibc 2.0 is also obsolete, heavily so

We use 2.0.6 here because it is half the size of the newer glibcs and it seems

to work fine for us.

/Brian

Re: glibc

[Alan Cox]

> We use 2.0.6 here because it is half the size of the newer glibcs and it seems
> to work fine for us.

It trust someone patched the holes in it for things like DNS lookups ?

Re: glibc

[Alan Cox]

> We use 2.0.6 here because it is half the size of the newer glibcs and it seems
> to work fine for us.

It trust someone patched the holes in it for things like DNS lookups ?

Re: glibc

[Ralf Baechle]

On Thu, Aug 16, 2001 at 03:04:23PM +0100, Alan Cox wrote:

> > We use 2.0.6 here because it is half the size of the newer glibcs and it
> > seems to work fine for us.
> 
> It trust someone patched the holes in it for things like DNS lookups ?

The latest glibc 2.0.6 I was spreading only has MIPS-specific bug fixes.
All the other big holes which are indeed big enough to drive a nice shipload
of trucks through are unfixed.  I haven't noticed that any other glibc
2.0 variant floating around has additional fixes.

Nice for Cobalt boxen ...

  Ralf

Re: glibc

[Martin Michlmayr]

* Ralf Baechle <ralf@oss.sgi.com> [20010816 18:14]:
> The latest glibc 2.0.6 I was spreading only has MIPS-specific bug
> fixes.  All the other big holes which are indeed big enough to drive
> a nice shipload of trucks through are unfixed.  I haven't noticed
> that any other glibc 2.0 variant floating around has additional
> fixes.
> 
> Nice for Cobalt boxen ...

Debian is being ported to Cobalts.  A base tar ball can be found at
http://www.pocketlinux.com/~samc/debian-cobalt

Re: glibc

[James Simmons]

> > The latest glibc 2.0.6 I was spreading only has MIPS-specific bug
> > fixes.  All the other big holes which are indeed big enough to drive
> > a nice shipload of trucks through are unfixed.  I haven't noticed
> > that any other glibc 2.0 variant floating around has additional
> > fixes.
> > 
> > Nice for Cobalt boxen ...
> 
> Debian is being ported to Cobalts.  A base tar ball can be found at
> http://www.pocketlinux.com/~samc/debian-cobalt

All the latest stuff there. I use it for my native build system. The
kernel is a 2.4.5 BTW if people are curious. I attempt a 2.4.6 kernel but
for some reason the cube doesn't like it. I believe I have a bug in the
old time code which is affected by the soft_pending change. I will attempt
to track it down some time next week.

Re: glibc

[Soeren Laursen]

> * Ralf Baechle <ralf@oss.sgi.com> [20010816 18:14]:
> > The latest glibc 2.0.6 I was spreading only has MIPS-specific bug
> > fixes.  All the other big holes which are indeed big enough to drive
> > a nice shipload of trucks through are unfixed.  I haven't noticed
> > that any other glibc 2.0 variant floating around has additional
> > fixes.
> > 
> > Nice for Cobalt boxen ...
> 
> Debian is being ported to Cobalts.  A base tar ball can be found at
> http://www.pocketlinux.com/~samc/debian-cobalt

Has anybody any luck running it on raq2 from Cobalt/SUN?

Soeren,

Re: glibc

[H . J . Lu]

On Thu, Aug 16, 2001 at 10:50:25AM +0200, Brian Murphy wrote:
> 
> We use 2.0.6 here because it is half the size of the newer glibcs and it seems
> 
> to work fine for us.
> 

I am working on sglibc. It has a smaller size by disabling selective
features.


H.J.

Re: glibc

[Brian Murphy]

"H . J . Lu" wrote:

>
> I am working on sglibc. It has a smaller size by disabling selective
> features.
>
> H.J.

Sounds excellent. How do I get my hands on it?

/Brian

Re: glibc

[H . J . Lu]

On Thu, Aug 16, 2001 at 07:00:42PM +0200, Brian Murphy wrote:
> "H . J . Lu" wrote:
> 
> >
> > I am working on sglibc. It has a smaller size by disabling selective
> > features.
> >
> > H.J.
> 
> Sounds excellent. How do I get my hands on it?
> 

http://sourceforge.net/projects/sglibc/

It is for developers only.


H.J.

Re: 2.4.9-pre[34] changes in drivers/char/vt.c broke Sparc64

[Alan Cox]

> Have you guys noticed that new vt.c in 2.4.9-pre[34] doesn't compile on
> Sparc64 (and Sparc32) ?

I dont have sparcs to check, so I reply on the sparc maintains

> sparc64-linux-gcc -D__KERNEL__ -I/usr/src/linux/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common -m64 -pipe -mno-fpu -mcpu=ultrasparc -mcmodel=medlow -ffixed-g4 -fcall-used-g5 -fcall-used-g7 -Wno-sign-compare -Wa,--undeclared-regs    -c -o vt.o vt.c
> In file included from vt.c:27:
> /usr/src/linux/include/linux/irq.h:57: asm/hw_irq.h: No such file or directory

Russell posted a fix for this, and he' sright that removing it can be done

> vt.c: In function `vt_ioctl':
> vt.c:507: `kbd_rate' undeclared (first use in this function)
> vt.c:507: (Each undeclared identifier is reported only once
> vt.c:507: for each function it appears in.)
> vt.c:514: `kbd_rate' used prior to declaration
> vt.c:514: warning: implicit declaration of function `kbd_rate'
> 
> Simple commenting out #include <linux/irq.h> and ioctl code that uses kbd_rate works.
> Whoever changed vt.c please post correct fix.

You need a sparc hacker to do this. If there are sparcs implementing the PC
keyboard controller (eg the pci ones) then kbd_rate needs implementing and
XFree86 needs recompiling to fix cases where the keyboard can be hung solid

Alan

Re: Coding convention of function header comments

[Alan Cox]

> /**
>  * list_add - add a new entry
>  * @new: new entry to be added
>  * @head: list head to add it after
>  *
>  * Insert a new entry after the specified head.
>  * This is good for implementing stacks.
>  */
> static __inline__ void list_add(struct list_head *new, struct list_head
> *head)
> {
>  __list_add(new, head, head->next);
> }
> 
> Similar to Java.  I want to ask that (1) is this a well-known convention or
> was just invented (informally) by someone here (e.g., Linus?)?  Where can I
> find the documentation about this convention? (2) can anyone point me to the
> URL of similar well-known coding conventions (except the Java one)?

Ok firstly - yes its a straight rip off of the java one. The history is
something like. Gnome needed a format for this, so Michael Zucchi (I
believe) wrote up a hideous perl hack. Miguel de Icaza couldn't get the
format right so it was extended with a free form body. 

Its documented in Documentation/kernel-doc-nano-HOWTO.txt

We do need to improve it to mark up structures, and also to actually
finish the job.

Re: [patch] zero-bounce highmem I/O

[Alan Cox]

> It is video layer, and the video layer should be helping along with
> these sorts of issues.

Linus refused to let make make the vmalloc helpers generic code, thats
why we have 8 or 9 different copies some containing old bugs

> void video_pci_put_user_pages(struct pci_dev *pdev,
> 			      struct scatterlist *sg,
> 			      int npages);

Why video_pci. WHy is this even video related. This is a generic issue

> In fact, this isn't even a video layer issue, and the kernel
> ought to provide my suggested interfaces in some generic
> place.

Then we agree on that

Re: [patch] zero-bounce highmem I/O

[Gerd Knorr]

On Thu, Aug 16, 2001 at 03:56:21PM +0100, Alan Cox wrote:
> > It is video layer, and the video layer should be helping along with
> > these sorts of issues.
> 
> Linus refused to let make make the vmalloc helpers generic code, thats
> why we have 8 or 9 different copies some containing old bugs

:-(

> > void video_pci_put_user_pages(struct pci_dev *pdev,
> > 			      struct scatterlist *sg,
> > 			      int npages);
> 
> Why video_pci. WHy is this even video related. This is a generic issue

I've moved current bttv's mm stuff to another source file and removed
any bttv-related stuff.  Compiles, but isn't fully tested yet.  The
first three functions are pretty generic.  The videomm_buf probably
isn't useful for anything but video4linux.  Depends on the highmem-io
patches as it uses the new scatterlist->page.

Comments?  Put that into videodev.[ch] ?

  Gerd

------------------------ cut here -----------------------
--- /dev/null	Thu Jan  1 01:00:00 1970
+++ video-mm.h	Fri Aug 17 11:57:02 2001
@@ -0,0 +1,47 @@
+/*
+ * memory management helpers for video4linux drivers (+others?)
+ */
+
+/*
+ * get the page for a kernel virtual address
+ * (i.e. for vmalloc()ed memory)
+ */
+struct page* videomm_vmalloc_to_page(unsigned long virt);
+
+/*
+ * return a scatterlist for vmalloc()'ed memory block.
+ */
+struct scatterlist* videomm_vmalloc_to_sg(unsigned long virt, int nr_pages);
+
+/*
+ * return a scatterlist for a iobuf
+ */
+struct scatterlist* videomm_iobuf_to_sg(struct kiobuf *iobuf);
+
+/* --------------------------------------------------------------------- */
+
+struct videomm_buf {
+	/* for userland buffer */
+	struct kiobuf       *iobuf;
+
+	/* for kernel buffers */
+	void*               *vmalloc;
+
+	/* common */
+	struct scatterlist  *sglist;
+	int                 sglen;
+	int                 nr_pages;
+};
+
+int videomm_init_user_buf(struct videomm_buf *vbuf, unsigned long data,
+			  unsigned long size);
+int videomm_init_kernel_buf(struct videomm_buf *vbuf, int nr_pages);
+int videomm_pci_map_buf(struct pci_dev *dev, struct videomm_buf *vbuf);
+int videomm_pci_unmap_buf(struct pci_dev *dev, struct videomm_buf *vbuf);
+int videomm_free_buf(struct videomm_buf *vbuf);
+
+/*
+ * Local variables:
+ * c-basic-offset: 8
+ * End:
+ */
--- /dev/null	Thu Jan  1 01:00:00 1970
+++ video-mm.c	Fri Aug 17 11:57:14 2001
@@ -0,0 +1,183 @@
+#include <linux/pci.h>
+#include <linux/iobuf.h>
+#include <linux/vmalloc.h>
+#include <linux/slab.h>
+#include <asm/page.h>
+#include <asm/pgtable.h>
+
+#include "video-mm.h"
+
+struct page*
+videomm_vmalloc_to_page(unsigned long virt)
+{
+	struct page *ret = NULL;
+	pmd_t *pmd;
+	pte_t *pte;
+	pgd_t *pgd;
+	
+	pgd = pgd_offset_k(virt);
+	if (!pgd_none(*pgd)) {
+		pmd = pmd_offset(pgd, virt);
+		if (!pmd_none(*pmd)) {
+			pte = pte_offset(pmd, virt);
+			if (pte_present(*pte)) {
+				ret = pte_page(*pte);
+			}
+		}
+	}
+	return ret;
+}
+
+struct scatterlist*
+videomm_vmalloc_to_sg(unsigned long virt, int nr_pages)
+{
+	struct scatterlist *sglist;
+	struct page *pg;
+	int i;
+
+	sglist = kmalloc(sizeof(struct scatterlist)*nr_pages, GFP_KERNEL);
+	if (NULL == sglist)
+		return NULL;
+	memset(sglist,0,sizeof(struct scatterlist)*nr_pages);
+	for (i = 0; i < nr_pages; i++, virt += PAGE_SIZE) {
+		pg = videomm_vmalloc_to_page(virt);
+		if (NULL == pg)
+			goto err;
+		sglist[i].page   = pg;
+		sglist[i].length = PAGE_SIZE;
+	}
+	return sglist;
+	
+ err:
+	kfree(sglist);
+	return NULL;
+}
+
+#if 1
+/*
+ * this is temporary hack only until there is something generic
+ * for that ...
+ */
+# if defined(__i386__)
+#  define NO_DMA(page) (((page) - mem_map) > (0xffffffff >> PAGE_SHIFT))
+# else
+#  define NO_DMA(page) (0)
+# endif
+#endif
+
+struct scatterlist*
+videomm_iobuf_to_sg(struct kiobuf *iobuf)
+{
+	struct scatterlist *sglist;
+	int i;
+
+	sglist = kmalloc(sizeof(struct scatterlist) * iobuf->nr_pages,
+			 GFP_KERNEL);
+	if (NULL == sglist)
+		return NULL;
+	memset(sglist,0,sizeof(struct scatterlist) * iobuf->nr_pages);
+
+	if (NO_DMA(iobuf->maplist[0]))
+		goto no_dma;
+	sglist[0].page   = iobuf->maplist[0];
+	sglist[0].offset = iobuf->offset;
+	sglist[0].length = PAGE_SIZE - iobuf->offset;
+	for (i = 1; i < iobuf->nr_pages; i++) {
+		if (NO_DMA(iobuf->maplist[i]))
+			goto no_dma;
+		sglist[i].page   = iobuf->maplist[i];
+		sglist[i].length = PAGE_SIZE;
+	}
+	return sglist;
+
+ no_dma:
+	/* FIXME: find a more elegant way than simply fail. */
+	kfree(sglist);
+	return NULL;
+}
+
+/* --------------------------------------------------------------------- */
+
+int videomm_init_user_buf(struct videomm_buf *vbuf, unsigned long data,
+			  unsigned long size)
+{
+	int err;
+
+	if (0 != (err = alloc_kiovec(1,&vbuf->iobuf)))
+		return err;
+	if (0 != (err = map_user_kiobuf(READ, vbuf->iobuf, data, size)))
+		return err;
+	vbuf->nr_pages = vbuf->iobuf->nr_pages;
+	return 0;
+}
+
+int videomm_init_kernel_buf(struct videomm_buf *vbuf, int nr_pages)
+{
+	vbuf->vmalloc = vmalloc_32(nr_pages << PAGE_SHIFT);
+	if (NULL == vbuf->vmalloc)
+		return -ENOMEM;
+	vbuf->nr_pages = nr_pages;
+	return 0;
+}
+
+int videomm_pci_map_buf(struct pci_dev *dev, struct videomm_buf *vbuf)
+{
+	int err;
+
+	if (0 == vbuf->nr_pages)
+		BUG();
+	
+	if (vbuf->iobuf) {
+		if (0 != (err = lock_kiovec(1,&vbuf->iobuf,1)))
+			return err;
+		vbuf->sglist = videomm_iobuf_to_sg(vbuf->iobuf);
+	}
+	if (vbuf->vmalloc) {
+		vbuf->sglist = videomm_vmalloc_to_sg
+			((unsigned long)vbuf->vmalloc,vbuf->nr_pages);
+	}
+	if (NULL == vbuf->sglist)
+		return -ENOMEM;
+	vbuf->sglen = pci_map_sg(dev,vbuf->sglist,vbuf->nr_pages,
+				 PCI_DMA_FROMDEVICE);
+	return 0;
+}
+
+int videomm_pci_unmap_buf(struct pci_dev *dev, struct videomm_buf *vbuf)
+{
+	if (!vbuf->sglen)
+		BUG();
+
+	pci_unmap_sg(dev,vbuf->sglist,vbuf->iobuf->nr_pages,
+		     PCI_DMA_FROMDEVICE);
+	kfree(vbuf->sglist);
+	vbuf->sglist = NULL;
+	vbuf->sglen = 0;
+	if (vbuf->iobuf)
+		unlock_kiovec(1,&vbuf->iobuf);
+	return 0;
+}
+
+int videomm_free_buf(struct videomm_buf *vbuf)
+{
+	if (vbuf->sglen)
+		BUG();
+
+	if (vbuf->iobuf) {
+		unmap_kiobuf(vbuf->iobuf);
+		free_kiovec(1,&vbuf->iobuf);
+		vbuf->iobuf = NULL;
+	}
+	if (vbuf->vmalloc) {
+		vfree(vbuf->vmalloc);
+		vbuf->vmalloc = NULL;
+	}
+	return 0;
+}
+
+
+/*
+ * Local variables:
+ * c-basic-offset: 8
+ * End:
+ */

Re: Via chipset

[Alan Cox]

> figured what in xscreensaver is doing this, don't know if it might be kernel
> related or not, probably not. I suspect xscreensaver is triggering something in
> the xserver that is not normally getting hit in normal use.

Thats actually quite possible. In paticular it uses arc's which basically no
other X app ever does. Report that info to either XFree or nvidia (depending
on whose X server)

Alan

Re: Spammer using linux kernel archives

[Alan Cox]

> FYI, a spammer claiming to be Dan Fuller representing D&H Group, Inc. (a
> recruiter) appears to be trolling the Linux kernel archives.  He sent me a
> job offer based on my "reputation as a Linux device driver developer".  I've
> already reported him to his ISP.

It seems to be common unfortunately. I nowdays tell them about ideal
candidates and give them each others details. Hours of fun for the wicked

Re: Spammer using linux kernel archives

[Tina Gasperson]

No wonder you don't have 30 minutes to donate to a poor Linux Users Group! 
Spending all your free time tormenting people!

;)
Tina G

>
> It seems to be common unfortunately. I nowdays tell them about ideal
> candidates and give them each others details. Hours of fun for the wicked
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: kernel threads

[Alan Cox]

> schedule the call to kernel_thread using tq_schedule

You still want to use daemonzie

> - is there no need to call daemonize in the second variant - if yes why?

A task always has a parent, it'll just be a random task that ran the 
kernel_thread request - in fact it might be a kernel thread and then 
I dont guarantee what will occur. In fact I wouldnt try the tq_schedule one

> - can i do both variants during interupt time (when there is no valid 
> current)?

No, but you can create a thread ready in case its needed then wake it from
an IRQ

Re: kernel threads

[Christian Widmer]

On Friday 17 August 2001 00:37, you wrote:
> > schedule the call to kernel_thread using tq_schedule
>
> You still want to use daemonzie
>
> > - is there no need to call daemonize in the second variant - if yes why?
>
> A task always has a parent, it'll just be a random task that ran the
> kernel_thread request - in fact it might be a kernel thread and then
> I dont guarantee what will occur. In fact I wouldnt try the tq_schedule one

sorry i didn't say that i what to use 2.4.x
when looking to kernel/context.c there it says that (or better i looks to me 
like it sais) that all that is sheduled to tq_schedule runns under keventd. so
the parent is predicable and is demonised itsself. is that bad or good?

Re: 2.4.9 does not compile [PATCH]

[Alan Cox]

> The args and semantics of min/max changed to take
> a type first argument, the problem with this ntfs file is that it
> fails to include linux/kernel.h

Thank you for your detailed discussion of this in advance on the kernel
list, your careful consideration of the 2.2 compatibility work horrors you
introduced and the thoughtful way you notified maintainers.

And all this from a man who gives me an earful if I merge third party
network patches without going through him.

Alan

Re: 2.4.9 does not compile [PATCH]

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Thu, 16 Aug 2001 23:41:07 +0100 (BST)

   > The args and semantics of min/max changed to take
   > a type first argument, the problem with this ntfs file is that it
   > fails to include linux/kernel.h
   
   Thank you for your detailed discussion of this in advance on the kernel
   list, your careful consideration of the 2.2 compatibility work horrors you
   introduced and the thoughtful way you notified maintainers.

Listen:

1) All of this was done at the request of Linus.

2) You were CC:'d on every single email
   that Linus and I had on these changes, and you
   saw every single revision of the patch.

Why are you complaining now, please speak up earlier next time when
you have a grip.  This is why you were CC:'d on the emails.

I am actually unhappy Linus waited until 2.4.9 final to put those
changes in as I was sure 1 or 2 compile issues would slip through.
I have been sending it over and over throughout the 2.4.9 prepatches.

Later,
David S. Miller
davem@redhat.com

Re: 2.4.9 does not compile [PATCH]

[Jes Sorensen]

>>>>> "David" == David S Miller <davem@redhat.com> writes:

David>    From: Alan Cox <alan@lxorguk.ukuu.org.uk> Date: Thu, 16 Aug
David> 2001 23:41:07 +0100 (BST)

>    Thank you for your detailed discussion of this in advance on
> the kernel list, your careful consideration of the 2.2
> compatibility work horrors you introduced and the thoughtful
> way you notified maintainers.

David> Listen:

David> 1) All of this was done at the request of Linus.

David> 2) You were CC:'d on every single email that Linus and I had on
David> these changes, and you saw every single revision of the patch.

Alan's point about CC'ing maintainers still stands, where was the CC
to the individual maintainers of the code you changed?

Jes

Re: 2.4.9 does not compile [PATCH]

[Alan Cox]

> 2) You were CC:'d on every single email
>    that Linus and I had on these changes, and you
>    saw every single revision of the patch.

Yep. And at the time I asked you to send it to the maintainers and the like.

Re: 2.4.9 does not compile [PATCH]

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Thu, 16 Aug 2001 23:55:44 +0100 (BST)

   > 2) You were CC:'d on every single email
   >    that Linus and I had on these changes, and you
   >    saw every single revision of the patch.
   
   Yep. And at the time I asked you to send it to the maintainers and the like.

When you were shown the patch your sentiments were:

1) Some of that stuff is against old drivers, I have newer
   ones in -ac so no biggie.
2) The ixj instances look like real bugs, and that you'd
   have a closer look.
3) If there are any merge issues in your -ac tree, no big
   deal because they'd show up as obvious compile errors.
4) Otherwise, it looked just fine to you.

I don't recall you saying anything about "make sure to tell the
maintainers" But, on the other hand, I can't prove that you didn't (I
don't keep detailed mail logs anymore except for what I send out
myself), so if you did I apologize.

Later,
David S. Miller
davem@redhat.com

Re: 2.4.9 does not compile [PATCH]

[David S. Miller]

   From: Rik van Riel <riel@conectiva.com.br>
   Date: Fri, 17 Aug 2001 18:46:20 -0300 (BRST)

   On Thu, 16 Aug 2001, David S. Miller wrote:
   
   > I don't recall you saying anything about "make sure to tell the
   > maintainers"
   
   I'll remember your attitude for if I ever have to
   touch the network code ;)

If Linus scribbled all over the network code I would
have to accept it if it were a kernel wide cleanup he
absolutely wanted.

Later,
David S. Miller
davem@redhat.com
   

Re: 2.4.9 does not compile [PATCH]

[Alan Cox]

> I don't recall you saying anything about "make sure to tell the
> maintainers" But, on the other hand, I can't prove that you didn't (I
> don't keep detailed mail logs anymore except for what I send out
> myself), so if you did I apologize.

I'm pretty sure I did (and I can prove it but whatever 8)). Whats done is done.
I think it was a bad idea for 2.4 (driver compat nightmare). 

Anyway arguing about it achieves nothing for anyone

Alan

Re: 2.4.9 does not compile

[Alan Cox]

> 	If you really want this facility, you could just declare
> a distinct "typed_min" macro.

For -ac I intend to change it to exactly that. Then we can also back
propogate compatibility macros. Abusing min prevents that

Re: 2.4.9 does not compile [PATCH]

[Alan Cox]

>    From: Anton Altaparmakov <aia21@cam.ac.uk>
>    Date: Fri, 17 Aug 2001 00:22:43 +0100
>    
>    IMHO, it would have been more elegant to use the typeof construct provided 
>    by gcc in the new macro instead of introducing a type parameter like this...
> 
> The whole point was to make users explicitly state the type so they
> would have to think about it.

And doing it by forcing them all to change their macro names isnt the
right solution. Its actually basically impossible to do back compat macros
with this mess. Your original smin() umin() proposal was _much_ saner.

As I've said, -ac will use typed_min(a,b,c), and that way I can propogate
back compatibility glue.

Alan

Re: 2.4.9 does not compile [PATCH]

[Alan Cox]

> Wrong.  This is legal:
> 
> int test(unsigned long a, int b)
> {
> 	return min(a, b);
> }

> And the compiler will warn about it with your typeof version.

Good, because its absolutely bogus and you want people to be warned so they
fix the input types to fit in the output type, and are forced to think about
whether all cases actually fit

Re: 2.4.9 does not compile [PATCH]

[Alan Cox]

>    I would hope that it would warn: what if a is the maximum size that an
>    array can be and b is a value passed in from userland?  Most definately
>    not an expected result.
> 
> My example was poor, consider if 'b' was something like '100'
> or for some other reason you already know perfectly well
> the legal range of 'b' or 'a'.

Then the compiler will cast constants for you (and warn if they dont fit),
and you can use casts to make it clear you know the legal ranges

Re: initrd: couldn't umount

[Alan Cox]

> the problem is that a "rpciod" kernel-thread references the initrd,
> and so umounting and freeing it, isn't possible.
> 
> has anybody an idea how to fix this problem, cause it would be nice,
> to free the initrd ram on a diskless node.

It shouldnt be keeping a reference. daemonize() should have dropped its
resources

Re: initrd: couldn't umount

[Daniel Wagner]

Alan Cox wrote:
> 
> > the problem is that a "rpciod" kernel-thread references the initrd,
> > and so umounting and freeing it, isn't possible.
> >
> > has anybody an idea how to fix this problem, cause it would be nice,
> > to free the initrd ram on a diskless node.
> 
> It shouldnt be keeping a reference. daemonize() should have dropped its
> resources

hmm, i've now created a /initrd directory to let the the change_root of
the
initrd work correctly.

and then i looked with fuser, what processes reference the initrd:

---
root@ws4:~ $ fuser -mv /initrd/
 
                     USER        PID ACCESS COMMAND
/initrd/             root         67 .rc.. 
rpciod                                                                                         
---

only the rpciod references the initrd, none of the other
kernel-threads.

Regards,
  Daniel

-- 
Daniel Wagner                      | mailto:daniel@xss.co.at
*x Software + Systeme              | http://www.xss.co.at/
Karmarschgasse 51/2/20             | Tel: +43-1-6060114-0
A-1100 Vienna, Austria             | Fax: +43-1-6060114-71

Re: initrd: couldn't umount

[Andreas Haumer]

Hi!

Daniel Wagner wrote:
> 
> Alan Cox wrote:
> >
> > > the problem is that a "rpciod" kernel-thread references the initrd,
> > > and so umounting and freeing it, isn't possible.
> > >
> > > has anybody an idea how to fix this problem, cause it would be nice,
> > > to free the initrd ram on a diskless node.
> >
> > It shouldnt be keeping a reference. daemonize() should have dropped its
> > resources
> 
> hmm, i've now created a /initrd directory to let the the change_root of
> the
> initrd work correctly.
> 
> and then i looked with fuser, what processes reference the initrd:
> 
> ---
> root@ws4:~ $ fuser -mv /initrd/
> 
>                      USER        PID ACCESS COMMAND
> /initrd/             root         67 .rc..
> rpciod
> ---
> 
> only the rpciod references the initrd, none of the other
> kernel-threads.
> 
Could it be because, when executed, rpciod() calls 
exit_files(current) and exit_mm(current), but doesn't
call exit_fs(current) (as, for instance, md_thread() does)? 
(we are talking 2.2.19 here)

We well try it in a few minutes... :-)

- andreas

-- 
Andreas Haumer                     | mailto:andreas@xss.co.at
*x Software + Systeme              | http://www.xss.co.at/
Karmarschgasse 51/2/20             | Tel: +43-1-6060114-0
A-1100 Vienna, Austria             | Fax: +43-1-6060114-71

Re: initrd: couldn't umount

[Andreas Haumer]

Hi!

(added okir@monad.swb.de to CC: list, as he is listed in the 
header of linux/net/sunrpc/sched.c as author of this file)

To answer my own question... ;-)

Andreas Haumer wrote:
> 
> Hi!
> 
> Daniel Wagner wrote:
> >
> > Alan Cox wrote:
[...]
> > > It shouldnt be keeping a reference. daemonize() should have dropped its
> > > resources
> >
> > hmm, i've now created a /initrd directory to let the the change_root of
> > the
> > initrd work correctly.
> >
> > and then i looked with fuser, what processes reference the initrd:
> >
> > ---
> > root@ws4:~ $ fuser -mv /initrd/
> >
> >                      USER        PID ACCESS COMMAND
> > /initrd/             root         67 .rc..
> > rpciod
> > ---
> >
> > only the rpciod references the initrd, none of the other
> > kernel-threads.
> >
> Could it be because, when executed, rpciod() calls
> exit_files(current) and exit_mm(current), but doesn't
> call exit_fs(current) (as, for instance, md_thread() does)?
> (we are talking 2.2.19 here)
> 
Yes, an additional call to exit_fs(current) when initializing
the rpciod thread solves this problem without creating other 
(obvious) problems (as it seems after a short test).

The inital ramdisk can now be unmounted cleanly even in case
of NFS root

The suggested patch is:

andreas@ws1:~/cvsdir {625} % cvs diff -C5 -rR_2-2-19~11 -rR_2-2-19~12
linux/net/sunrpc/sched.c
Index: linux/net/sunrpc/sched.c
===================================================================
RCS file:
/raid5/cvs/repository/distribution/Base/linux/net/sunrpc/sched.c,v
retrieving revision 1.1.1.6
retrieving revision 1.12
diff -C5 -r1.1.1.6 -r1.12
*** linux/net/sunrpc/sched.c    2001/03/25 16:37:42     1.1.1.6
--- linux/net/sunrpc/sched.c    2001/08/17 11:53:48     1.12
***************
*** 1066,1075 ****
--- 1066,1076 ----
        rpciod_pid = current->pid;
        up(&rpciod_running);
 
        exit_files(current);
        exit_mm(current);
+       exit_fs(current);
 
        spin_lock_irq(&current->sigmask_lock);
        siginitsetinv(&current->blocked, sigmask(SIGKILL));
        recalc_sigpending(current);
        spin_unlock_irq(&current->sigmask_lock); 


Alan, would you consider adding this patch to 2.2.20?

Thanks!

- andreas

-- 
Andreas Haumer                     | mailto:andreas@xss.co.at
*x Software + Systeme              | http://www.xss.co.at/
Karmarschgasse 51/2/20             | Tel: +43-1-6060114-0
A-1100 Vienna, Austria             | Fax: +43-1-6060114-71

Re: K6 sig11 Bug detection.

[Alan Cox]

> Aug 17 09:06:49 phoenix kernel: CPU: Before vendor init, caps: 008001bf 
> 008005bf 00000000, vendor = 2
> Aug 17 09:06:49 phoenix kernel: AMD K6 stepping B detected - <6>K6 BUG 
> 9016725 20000000 (Report these if test report is incorrect)
> Aug 17 09:06:49 phoenix kernel: AMD K6 stepping B detected - probably OK 
> (after B9730xxxx).

Looks ok to me.

Re: Promise Ultra100(20268) address conflict with ServerWorks IDE

[Alan Cox]

>         I/O ports at 5440 [size=16]
> 
> // notice the gap at 5450-545F.
> // It will be allocated for the Serverworks IDE controller later.
> // This gap is only real, if size=16 is a true statement

The size data is read from the PCI card itself, which would lead me to
suspect we reserve to much maybe ?

Re: Kernel panic problem in 2.4.7

[Alan Cox]

> 
> SGVsbG8uLiBBbGwNCg0KSSBoYXZlIG9wZXJhdGVkIFJlZGhhdCA2LjIgYmFzZWQgc3lzdGVtKGtl
> cm5lbCAyLjQuNykgIHdoaWNoIGlzIER1YWwgQ1BVIGFuZCA1MTJNIFJhbS4NCg0KQSBmZXcgZGF5
> cyBhZ28sIHRoaXMgc3lzdGVtIGlzIGRvd24gYW5kIHRoZSBtZXNzYWdlcyBsaWtlIGJlbG93Lg0K
> DQpKdWwgMjIgMTQ6NTc6NDUgd3d3IGtlcm5lbDogS2VybmVsIHBhbmljOiBDUFUgY29udGV4dCBj
> b3JydXB0DQpKdWwgMjUgMTk6MjU6MzAgd3d3IGtlcm5lbDogS2VybmVsIHBhbmljOiBDUFUgY29u
> dGV4dCBjb3JydXB0DQpKdWwgMjcgMjM6NDM6MjIgd3d3IGtlcm5lbDogS2VybmVsIHBhbmljOiBD
> UFUgY29udGV4dCBjb3JydXB0DQoNCnRoaXMgbWFpbGxpbmcgbGlzdCBzYXkgdGhhdCB0aGlzIGlz
> IENQVSBQcm9ibGVtKG92ZXJjbG9ja2luZy4uLi5idXQgaXQgaXNuJ3QpLg0KU28gSSBjaGFuZ2Vk
> IG90aGVyIENQVXMgdGhhdCBJIHRoaW5rICBubyBwcm9ibGVtLg0KDQpCdXQgYSBmZXcgZGF5cyBs
> YXRlci4ga2VybmVsIHBhbmljIG9jY3VyZWQgYWdhaW4uDQooVGhlIHN5c3RlbSBpcyAyLjQuNykN
> CnRoZSBtZXNzYWdlcyB3YXMgDQoNCktlcm5lbCBwYW5pYzpBaWVlLCBraWxsaW5nIGludGVycnVw
> dCBoYW5kbGVyDQpJbiBpbnRlcnJ1cHQgaGFuZGxlciAtIG5vdCBzeW5jaW5nDQoNClRoaXMgbWVz
> c2FnZSBpcyBvY2N1cmVkIHdoZW4gdGhlIHN5c3RlbSdzIGxvYWQgYXZlcmFnZSBpcyBhIGJpdCBo
> aWdoLg0KKFdoZW4gSSBLZXJuZWwgY29tcGlsZSBvciBwYWNraW5nIHdpdGggdGFyLi4gc29tdGhp
> bmcgbGlrZSB0aGF0Li4pDQoNCkkgc2VhcmNoZWQgYWxsIGxpc3RzLCBCdXQgSSBjb3VsZG4ndCBm
> aW5kIHRoZSAicmVhc29uIiBhbmQgImFuc3dlciIuDQpJIHdvdWxkIGxpa2UgdG8ga25vdyB0aGUg
> cmVhc29uIGFuZCBhbnN3ZXIuDQoNClBsZWFzZSBnaXZlIG1lIHNvbWUgYWR2aWNlIGFib3V0IHRo
> ZXNlIHByb2JsZW1zLg0KDQogDQogVGhhbmtzIGZvciB5b3VyIGhlbHAuDQogIA0KIA0K
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Kernel panic problem in 2.4.7

[Alan Cox]

If your cpu context corrupt is with a machine check exception report then
your processor took an internal fault reporting trap. So its not a happy
bit of silicon I suspect - be it overclocked, overheated, or even faulty.

Re: Kernel panic problem in 2.4.7

[Alan Cox]

> If your cpu context corrupt is with a machine check exception report then
> your processor took an internal fault reporting trap. So its not a happy
> bit of silicon I suspect - be it overclocked, overheated, or even faulty.

Beware beta versions of editors 8)

If your CPU context corrupt message is appearing with a machine check
exception report, then you processor took an internal fault reporting trap.

So it's not a happy piece of silicon I suspect - be it overclocked,
overheated or even faulty.

The machine check numbers it prints can be looked up in the processor
manuals to decode the type of fault.

Re: Via usb problems...

[Alan Cox]

> The work-around for the problem is provided by VIA in the form of some
> updated drivers for windows.  It appears to be some sort of usb filter,
> possibly for low bandwidth USB peripherals.  I suspect this isn't a
> working resolution for most people on this list and I was wondering if
> anyone has been working on this in the kernel?  I might be able to
> provide some more information if it is needed to diagnose and solve the
> problem...

Can you provide me

1.	The lspci -v info for your box (chip revision etc)
2.	Details ont he updated windows drivers (eg URL for)
3.	The USB devices

> Does VIA have any engineers working on linux drivers?

Directly, I don't believe so. However they do provide contact points for
some of us and have provided workarounds for other bugs. We now have a 
mechanism in place for making such requests.

Firstly however I'd like the USB folks to look at the debug and traces to be
sure this isnt a VIA chip bug. If it is I'll take it up with VIA

Re: 2.4.9 does not compile [PATCH]

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Fri, 17 Aug 2001 10:11:17 +0100 (BST)
   
   Its actually basically impossible to do back compat macros
   with this mess. Your original smin() umin() proposal was _much_ saner.

I don't see how you can logically say this.
My sint_min() etc. version broke things just
as equally because it had:

#define min __compile_error_do_not_use_min
#define max __compile_error_do_not_use_max

in it.  Do you think Linus or myself, by doing
this, intended to let anyone undef the damn things
to get around this?

The whole point of the changes was "min and max are
dumb, nobody may use them in their traditional form".

I was pretty sure, you understood this.

Later,
David S. Miller
davem@redhat.com

Re: 2.4.9 does not compile [PATCH]

[Alan Cox]

>    Its actually basically impossible to do back compat macros
>    with this mess. Your original smin() umin() proposal was _much_ saner.
> 
> I don't see how you can logically say this.

Because it would have been trivial to provide back compatibility macros to
provide sint_min/sint_max on 2.2. Now all I can do is make -ac use sint_min
sint_max and friends (or typed_min to be exact) and make 2.2 and -ac 
compatible

> My sint_min() etc. version broke things just
> as equally because it had:

Nothing like as badly.

Alan

Re: 2.4.9 does not compile [PATCH]

[Alan Cox]

> If Linus scribbled all over the network code I would
> have to accept it if it were a kernel wide cleanup he
> absolutely wanted.

That doesn't make it right, it merely makes the perpetrator wrong.

kfree safe in interrupt context?

[Victor Yodaiken]

Seems like calling kfree from interrupt context should
be ok, but is it? 
If it is safe, is this considered a good thing  or not?

Re: kfree safe in interrupt context?

[Rusty Russell]

In message <20010817211406.A21326@hq2> you write:
> Seems like calling kfree from interrupt context should
> be ok, but is it? 
> If it is safe, is this considered a good thing  or not?

Yes, and it logically has to be, as kmalloc(..., GFP_ATOMIC) is safe
from interrupt context.

The network code does this all the time, for example.

Hope that helps,
Rusty.
--
Premature optmztion is rt of all evl. --DK

Re: kfree safe in interrupt context?

[Victor Yodaiken]

On Sun, Aug 19, 2001 at 09:44:45PM +1000, Rusty Russell wrote:
> In message <20010817211406.A21326@hq2> you write:
> > Seems like calling kfree from interrupt context should
> > be ok, but is it? 
> > If it is safe, is this considered a good thing  or not?
> 
> Yes, and it logically has to be, as kmalloc(..., GFP_ATOMIC) is safe
> from interrupt context.
> 
> The network code does this all the time, for example.
> 
> Hope that helps,

It does, but Alan answered first and also much more
eloquently. -)

Re: 2.4.9 does not compile [PATCH]

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Fri, 17 Aug 2001 23:09:15 +0100 (BST)

   > If Linus scribbled all over the network code I would
   > have to accept it if it were a kernel wide cleanup he
   > absolutely wanted.
   
   That doesn't make it right, it merely makes the perpetrator wrong.

To be honest, I like having someone like him who can just put his foot
down on a matter whilst the rest of us waste days in silly flame wars
on the topic. :-)

I've stop caring, personally.  Back it out, it wouldn't bother me.
I made a patch someone asked me to make, so sue me.

What bothers me is all the time being spent arguing about it.

Later,
David S. Miller
davem@redhat.com

Re: 2.4.9 does not compile [PATCH]

[Daniel Phillips]

On August 18, 2001 12:11 am, David S. Miller wrote:
> What bothers me is all the time being spent arguing about it.

It would be safe to conclude it's a universally unpopular change.

--
Daniel

Re: 2.4.9 does not compile [PATCH]

[David S. Miller]

   From: Daniel Phillips <phillips@bonn-fries.net>
   Date: Sat, 18 Aug 2001 01:34:28 +0200

   On August 18, 2001 12:11 am, David S. Miller wrote:
   > What bothers me is all the time being spent arguing about it.
   
   It would be safe to conclude it's a universally unpopular change.

Well, to be fair, popularity sometimes isn't much of an indicator.
Often people argue things because of change itself.  Having to do
something differently can make people feel uneasy, so they resist the
change.

Later,
David S. Miller
davem@redhat.com

Re: 2.4.9: GCC 3.0 problem in "acct.c"

[Alan Cox]

> i just updated my gcc from 2.96 to 3.0.1 and now i cant compile kernel 2.4.9
> anymore, make bzImage fails with the following error message:
> 
> acct.c: In function `check_free_space':
> acct.c:147: Unrecognizable insn:
> (insn 335 102 336 (parallel[
>             (set (reg/v:SI 2 ecx [44])
>                 (const_int 0 [0x0]))
>             (clobber (reg:CC 17 flags))
>         ] ) -1 (insn_list:REG_DEP_ANTI 100 (insn_list:REG_DEP_ANTI 102
> (nil)))
>     (expr_list:REG_UNUSED (reg:CC 17 flags)
>         (nil)))
> acct.c:147: Internal compiler error in insn_default_length, at
> insn-attrtab.c:223
> 
> can anyone tell me how to fix this?

Use gcc 2.96 or gcc 3.0.0. You broke the compiler. Please also see the gcc bug
reporting instructions. You will actually make a gcc hacker very happy 
reporting that problem.

Re: 2.4.x & Celeron = very slow system

[Alan Cox]

> With 2.4.x kernel my system (Celeron (Coppermine) 850Mhz (100x8.5)
> 256Mb i810) behaves like slow i386sx.
> For example, when I extract 25MB gzip file on 2.2.19 kernel - it
> takes about 12 seconds, but with 2.4.8(9) - 6(!) MINUTES on the SAME
> system...
> The only idea is that 2.4.x kernel turns off cache (L1 & L2) on
> processor (on my cpu). How can I check it? Any ideas?

We don't touch the caches like that. First guess is to disable the ACPI
support, because we've seen that do a million bogus things

Re: gcc-3.0 with 2.2.x ?

[Alan Cox]

> I can use gcc-3.0 to compile 2.4.8-ac7 but can not do
> that with 2.2.19

> so I would like to know if there is any patches to
> 2.2.19 to make it friendlier with gcc-3.0. In the mean
> time I am going to see sched.c and sched.h and try to
> make it work :-)

I've got no plans to care about 2.2 and gcc 3.0. Trying to deal with all
the fun things gcc 3.0 does (quite legally too the ones I looked at) is
hard. The amount of work required to fix it is large and given 2.2 is
there for maximum stability also pointless, since its an untrusted compile
combination.

Alan

Re: BUG: pc_keyb.c

[Alan Cox]

>   Due to writing to the status register before it's ready as far as I 
> can se.
> 
>   fix: change all mdelay(1) in pc_keyb.c to mdelay(2)'s .. ( mdelay(1) 
> will be on the timing limit.

Thats interesting. I thought the rule was 1mS. (There is indeed a timing
requirement). Any keyboard controller experts around ?

>   Might also be present during high load on machines running GL on AGP 
> video cards, not 100 % sure same symptoms, above seem's to fix ???? ( 
> strange ).

XFree86 touches they keyboard controller directly unless you rebuild it 
against a recent -ac kernel. This causes amusing problems with legacy free
PC's as well as the timing issue.

That might explain the other one

Re: [linux-lvm] LVM on Redhat 7.1

[Heinz J . Mauelshagen]

Louis,

since Friday you should better get LVM 1.0.1-rc1 from www.sistina.com,
which *directly* supports the old and new metadata formats and avoids running
an explicit upgrade program.

On Fri, Aug 17, 2001 at 01:13:51PM -0000, Louis Becker wrote:
> Hi,
> 
> 	I am trying to install LVM 1.0 from sources on a RH7.1 box, the compilation seems ok, also the make install but at each call of vgscan i get the answer :
> 
> vgscan -dv
> <1> lvm_get_iop_version -- CALLED
> <22> lvm_check_special -- CALLED
> <22> lvm_check_special -- LEAVING
> <1> lvm_get_iop_version -- AFTER ioctl ret: -102
> <1> lvm_get_iop_version -- LEAVING with ret: -102
> vgscan -- LVM driver/module not loaded?

Looks like you didn't start with a vanilla kernel patching it conforming
to the instructions in the PATCHES/README file contained in the LVM tarball
-or-
generated the LVM module but it is not lodable (compile/module install/
depmod failure).

> 
> 
> 	So how do i load this module and from where ?
> 
> 	PS: i use LVM on IBM AIX and it would be great to use it on linux
> 
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] LVM on Redhat 7.1

[Keith Hopkins]

Heinz J . Mauelshagen wrote:

> Louis,
> 
> since Friday you should better get LVM 1.0.1-rc1 from www.sistina.com,
> which *directly* supports the old and new metadata formats and avoids running
> an explicit upgrade program.
> 


Hi Heinz (good job to you and the team!)

   How far back is the "old" metadata supported?  I'm still running the "0.8 by Heinz Mauelshagen  04/11/1999" version that shipped on SuSE 6.4.

Lost in Tokyo,
   Keith

Re: [linux-lvm] LVM on Redhat 7.1

[Carlo Marcelo Arenas Belon]

> since Friday you should better get LVM 1.0.1-rc1 from www.sistina.com,
> which *directly* supports the old and new metadata formats and avoids running
> an explicit upgrade program.

i am willing to do a new kernel rpm for RH 7.1 including the LVM 1.0.1
tools and a corresponding lvm RPM, to help make this easier, and also
upgrade my system running on an old beta kernel with old beta tools ;)

do you know of any ETA for rc1 becoming the formal 1.0.1?, avoiding the
strings "beta", "rc" and any manual procedure is something that usually
makes packagers and package users happy :)

regards,

Carlo

Re: [linux-lvm] LVM on Redhat 7.1

[Joe Thornber]

On Mon, Aug 20, 2001 at 09:34:39AM -0500, Carlo Marcelo Arenas Belon wrote:
> > since Friday you should better get LVM 1.0.1-rc1 from www.sistina.com,
> > which *directly* supports the old and new metadata formats and avoids running
> > an explicit upgrade program.
> 
> i am willing to do a new kernel rpm for RH 7.1 including the LVM 1.0.1
> tools and a corresponding lvm RPM, to help make this easier, and also
> upgrade my system running on an old beta kernel with old beta tools ;)
> 
> do you know of any ETA for rc1 becoming the formal 1.0.1?, avoiding the
> strings "beta", "rc" and any manual procedure is something that usually
> makes packagers and package users happy :)

Next couple of days, when we are convinced that some people have tried rc-1.

- Joe

Re: On Network Drivers......

[Alan Cox]

>  A driver of layered one.., in which one layer
> communicates with another. But overall there should be
> only one driver . (Just like Stream's drivers, but we
> don't want Stream Drivers). 
>     Is there any support in Linux to implement Network
> Drivers as multi layered in a single driver .If so,
> please let me know where I can get good information?

We have a single layer between the network stack and the drivers. However
nothing stops drivers from implementing multiple layers internally or
calling back into other drivers. shaper is an example of a driver that
calls other drivers.

The basic attitude is layering is slow, and generally poor for the cache
behaviour. You want to do a single pass of the packet (or less if possible
when using DMA). However the interface is flexible and if a driver wants
to be layered or has reason for it - sure it can do it internally

Re: On Network Drivers......

[Venu Gopal Krishna Vemula]

Sir,

--- Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

>>  A driver of layered one.., in which one layer
>> communicates with another. But overall there 
>> should be  only one driver . (Just like Stream's
>> drivers, but we don't want Stream Drivers). 

> 
> We have a single layer between the network stack and
> the drivers. However  nothing stops drivers from 
> implementing multiple  layers internally or
> calling back into other drivers. shaper is an
> example of a driver that  calls other drivers.
> 

 Can you please explain something about implementing
multiple layers internally in a Network Driver 
                     or
 inform me where i can find information (Source code)

         Today i have searched a lot to find any
information related to the Network driver(in which
multiple layers has to develop internally)  in
Internet, but i could not found any source code or
information in Internet.

With Thanks,
regards,
vvgkrishna_78@yahoo.com


__________________________________________________
Do You Yahoo!?
Make international calls for as low as $.04/minute with Yahoo! Messenger
http://phonecard.yahoo.com/

Re: PATCH: linux-2.4.9/drivers/i2o to new module_{init,exit} interface

[Alan Cox]

> declarations in linux/Makefile.  (If you really need i2o
> initialization to occur earlier than do_initcalls(), then that would
> also mean that i2o cannot be a module, right?)

In certain configurations you are correct

> 	If you want, I can send you a new patch that changes
> linux/Makefile to initialize i2o before just before drivers/block,
> thereby reproducing the current initialization order, and, of course,

Sounds ok to me - Im not against tidying it up. Note btw the -ac i2o code
is a little different to vanilla and is the 'current' one. I think your
patches will apply fine however as the changes are small

Re: sound crashes in 2.4

[Alan Cox]

> I have the same problems with an ALS007 card (in a 486 system).
> The card is correctly recognized and set up by the PnP drivers.

Humm. 

> The time after which it crashes is variable, sometimes it crashes
> immediately, sometimes it crashes after 5 minutes.
> Sometimes, it also stalls a few times before finally crashing.

Ok that actually sounds more like a locking bug.

> Both 2.4.8 and 2.4.9 have this problem (these are the ones I tried).
> Btw on 2.2.x i get DMA (output) timeout errors (and broken sound).

Can you try one other thing.

Edit drivers/pci/quirks.c

find 

int isa_dma_bridge_buggy;		/* Exported */

and make it read

int isa_dma_bridge_buggy = 1;

recompile reboot and see if it helps

Re: sound crashes in 2.4

[Chris Pockele]

> Can you try one other thing.
> 
> Edit drivers/pci/quirks.c
> 
> find 
> 
> int isa_dma_bridge_buggy; /* Exported */
> 
> and make it read
> 
> int isa_dma_bridge_buggy = 1;
> 
> recompile reboot and see if it helps
> 
Unfortunately, it does not help.  The machine rebooted
suddenly (i forgot to mention that: sometimes it reboots
too, but mostly it just hangs).

btw the system doesn't have a PCI bus, so its kernel
does not have PCI support enabled

(it's a SIS85C471B-based ISA/VLB motherboard)

Re: sound crashes in 2.4

[Frank Davis]

Hello all,
     A trace of the crashes would be helpful.

Regards,
Frank

Chris Pockele wrote:

>>Can you try one other thing.
>>
>>Edit drivers/pci/quirks.c
>>
>>find 
>>
>>int isa_dma_bridge_buggy; /* Exported */
>>
>>and make it read
>>
>>int isa_dma_bridge_buggy = 1;
>>
>>recompile reboot and see if it helps
>>
>>
> Unfortunately, it does not help.  The machine rebooted
> suddenly (i forgot to mention that: sometimes it reboots
> too, but mostly it just hangs).
> 
> btw the system doesn't have a PCI bus, so its kernel
> does not have PCI support enabled
> 
> (it's a SIS85C471B-based ISA/VLB motherboard)
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> 

Re: sound crashes in 2.4

[Chris Pockele]

> Hello all,
>      A trace of the crashes would be helpful.
> 
unfortunately, i don't get any oops or panic messages.
There's nothing about it on the console, nothing in
logs either.

The box just freezes.  No more ping/connection
replies, no more reaction to any keyboard input.

(this must be a very hard bug)

cheers

Re: BUG: pc_keyb.c

[Alan Cox]

> But the present code does not guarantee such a delay at all.
> For example, kbd_write_cmd() does
> 	kb_wait();
> 	outb(...);
> 	kb_wait();
> where the second kb_wait reads the status very quickly after the first.

Thats wrong by the spec. I dug out my docs - there is a required 1mS (not
2 tho) delay for keyboard port accesses.

Re: Linux 2.4.8-ac8

[Alan Cox]

> Alan,
> 
> I'm sorry if it was previously answered, but why your tree is currently not 
> in sync with Linus's tree?

Because I don't actually see the point of being in sync with 2.4.9 currently

Re: On Network Drivers......

[Alan Cox]

> information related to the Network driver(in which
> multiple layers has to develop internally)  in
> Internet, but i could not found any source code or
> information in Internet.

How strange. The kernel source code is definitely on the internet, and
definitely contains drivers that implement internal layering - 
nrdev, shaper, the sync cards, isdn

Re: massive filesystem corruption with 2.4.9

[Alan Cox]

> > Aug 21 09:01:06 adlib kernel: EXT2-fs error (device ide0(3,5)):
> > ext2_new_block: Allocating block in system zone - block =3D 3
> > Aug 21 09:01:06 adlib kernel: EXT2-fs error (device ide0(3,5)):
> > ext2_free_blocks: Freeing blocks in system zones - Block =3D 4, count=
>  =3D 1

Typically this indicates disk, memory or chipset problems. If your disk is
in UDMA33/66 mode you can pretty rule the disk out as the data is
protected

If you have a VIA chipset, especially if there is an SB Live! in the machine
then that may be the cause (fixes in 2.4.8-ac, should be a fix in 2.4.9 but
Linus tree also applies another bogus change but which should be harmless)

> > These fatal errors are occuring since 2.4.5 (2.4.8 I've not tested.).=
>  When
> > I work with 2.4.4 everything is fine !

What hardware

Re: massive filesystem corruption with 2.4.9

[Kristian]

Alan Cox wrote:
 > Typically this indicates disk, memory or chipset problems. If your disk is
 > in UDMA33/66 mode you can pretty rule the disk out as the data is
 > protected
 >
 > If you have a VIA chipset, especially if there is an SB Live! in the machine
 > then that may be the cause (fixes in 2.4.8-ac, should be a fix in 2.4.9 but
 > Linus tree also applies another bogus change but which should be harmless)

No. I can't find any VIA chipset. I'm really surprised. :-) But it is an
original Compaq-Board (EP-Series..) with a horroble BIOS. It seems that they're
using intel only..

I did a probe of my harddisk with IBM's Drive Fitness program. It detected no
errors.

Here's the output of it:

     Model                   : IBM-DTLA-305040
     Serial no.              : YJ025714
     Capacity                : 41.17 GB
     Cache size              : 380 KB
     Microcode level         : TW4OA60A
     ATA Compliance          : ATA-5

     Ultra DMA
       Highest mode          : 5
       Active mode           : 1

     Settings
       Write cache           : Enabled
       Read look-ahead       : Enabled
       Auto reassign         : Enabled
       S.M.A.R.T. operations : Enabled
       S.M.A.R.T. status     : Good
       ABLE                  : Disabled
       AAM                   : Disabled
       Security feature      : Supported
         Password            : Not Set


Here is the output of cat /proc/pci:
PCI devices found:
    Bus  0, device   0, function  0:
      Host bridge: Intel Corporation 440BX/ZX - 82443BX/ZX Host bridge (rev 3).
        Master Capable.  Latency=64.
        Prefetchable 32 bit memory at 0x44000000 [0x47ffffff].
    Bus  0, device   1, function  0:
      PCI bridge: Intel Corporation 440BX/ZX - 82443BX/ZX AGP bridge (rev 3).
        Master Capable.  Latency=64.  Min Gnt=140.
    Bus  0, device  14, function  0:
      Ethernet controller: Intel Corporation 82557 [Ethernet Pro 100] (rev 5).
        IRQ 11.
        Master Capable.  Latency=66.  Min Gnt=8.Max Lat=56.
        Prefetchable 32 bit memory at 0x48100000 [0x48100fff].
        I/O at 0x1000 [0x101f].
        Non-prefetchable 32 bit memory at 0x48000000 [0x480fffff].
    Bus  0, device  15, function  0:
      Multimedia audio controller: Ensoniq ES1371 [AudioPCI-97] (rev 6).
        IRQ 11.
        Master Capable.  Latency=64.  Min Gnt=12.Max Lat=128.
        I/O at 0x1080 [0x10bf].
    Bus  0, device  16, function  0:
      Multimedia video controller: Brooktree Corporation Bt878 (rev 17).
        IRQ 11.
        Master Capable.  Latency=66.  Min Gnt=16.Max Lat=40.
        Prefetchable 32 bit memory at 0x48200000 [0x48200fff].
    Bus  0, device  16, function  1:
      Multimedia controller: Brooktree Corporation Bt878 (rev 17).
        IRQ 11.
        Master Capable.  Latency=66.  Min Gnt=4.Max Lat=255.
        Prefetchable 32 bit memory at 0x48300000 [0x48300fff].
    Bus  0, device  20, function  0:
      ISA bridge: Intel Corporation 82371AB PIIX4 ISA (rev 2).
    Bus  0, device  20, function  1:
      IDE interface: Intel Corporation 82371AB PIIX4 IDE (rev 1).
        Master Capable.  Latency=64.
        I/O at 0x1040 [0x104f].
    Bus  0, device  20, function  2:
      USB Controller: Intel Corporation 82371AB PIIX4 USB (rev 1).
        IRQ 11.
        Master Capable.  Latency=64.
        I/O at 0x1020 [0x103f].
    Bus  0, device  20, function  3:
      Bridge: Intel Corporation 82371AB PIIX4 ACPI (rev 2).
        IRQ 9.
    Bus  1, device   0, function  0:
      VGA compatible controller: Matrox Graphics, Inc. MGA G400 AGP (rev 130).
        IRQ 11.
        Master Capable.  Latency=64.  Min Gnt=16.Max Lat=32.
        Prefetchable 32 bit memory at 0x42000000 [0x43ffffff].
        Non-prefetchable 32 bit memory at 0x40800000 [0x40803fff].
        Non-prefetchable 32 bit memory at 0x40000000 [0x407fffff].

/proc/cpuinfo:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 8
model name      : Pentium III (Coppermine)
stepping        : 3
cpu MHz         : 597.413
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 sep mtrr pge mca cmov pat
pse36 mmx fxsr sse
bogomips        : 1192.75

/proc/interrupts:

    0:    1337585          XT-PIC  timer
    1:      46916          XT-PIC  keyboard
    2:          0          XT-PIC  cascade
    8:          1          XT-PIC  rtc
   11:     162990          XT-PIC  es1371, bttv, eth0
   12:     294331          XT-PIC  PS/2 Mouse
   14:      21839          XT-PIC  ide0
   15:         17          XT-PIC  ide1
NMI:          0
ERR:          0


Kristian

·· · · reach me :: · ·· ·· ·  · ·· · ··  · ··· · ·
                           :: http://www.korseby.net
                           :: http://www.tomlab.de
kristian@korseby.net ....::

Re: massive filesystem corruption with 2.4.9

[Christian Widmer]

>  > If your disk is in UDMA33/66 mode you can pretty rule the 
>  > disk out as the data is protected
i think this should be with the promise driver.

>  > If you have a VIA chipset, especially if there is an SB Live! in the
>  > machine then that may be the cause (fixes in 2.4.8-ac, should be a fix
>  > in 2.4.9 but Linus tree also applies another bogus change but which
>  > should be harmless)
it was an intel LX chipset

that it is a memory problem i also don't belive. that ram work for over 2 year
with no errors found with memtest (memtset86, intels memtest) compiling
seveal times xfree86 and an many many times several kernels. 

and i never had any problems. until i tried the first time a 2.4.x kernel on 
the fileserver (that was 2.4.6). so i moved the fileserver back to 2.2.19.

Re: i810 audio doesn't work with 2.4.9

[Alan Cox]

> i810 audio didn't work for me with kernel 2.4.9 (artsd from KDE goes into infinite loop, no sound). 
> 
> Reverting to kernel 2.4.7 or replacing in 2.4.9 drivers/sound/ac97_codec.c, drivers/sound/i810_audio.c, 
> include/linux/ac97_codec.h from 2.4.8-ac8 fixed the problem for me

Thats good to know - the 2.4.8-ac8 one is scheduled to go to Linus

Re: i810 audio doesn't work with 2.4.9

[Jeff Chua]

it works for me on 2.4.9.


Thanks,
Jeff
[ jchua@fedex.com ]

On Tue, 21 Aug 2001, Alan Cox wrote:

> > i810 audio didn't work for me with kernel 2.4.9 (artsd from KDE goes into infinite loop, no sound).
> >
> > Reverting to kernel 2.4.7 or replacing in 2.4.9 drivers/sound/ac97_codec.c, drivers/sound/i810_audio.c,
> > include/linux/ac97_codec.h from 2.4.8-ac8 fixed the problem for me
>
> Thats good to know - the 2.4.8-ac8 one is scheduled to go to Linus
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: i810 audio doesn't work with 2.4.9

[Andris Pavenis]

On Tuesday 21 August 2001 16:59, Alan Cox wrote:
> > i810 audio didn't work for me with kernel 2.4.9 (artsd from KDE goes into
> > infinite loop, no sound).
> >
> > Reverting to kernel 2.4.7 or replacing in 2.4.9
> > drivers/sound/ac97_codec.c, drivers/sound/i810_audio.c,
> > include/linux/ac97_codec.h from 2.4.8-ac8 fixed the problem for me
>
> Thats good to know - the 2.4.8-ac8 one is scheduled to go to Linus

Sorry, my fault:

seems that i810_audio.c  from 2.4.7 and 2.4.8 works, but both from 2.4.9 and
2.4.8-ac8 does not ("suceeded" not to copy right version of this file when
testing previous time, so had one from 2.4.7 twice)

Andris

---------- corresponding line from /proc/pci ------------------------
  Bus  0, device  31, function  5:
    Multimedia audio controller: Intel Corporation 82801AA AC'97 Audio (rev 2).
      IRQ 10.
      I/O at 0xe000 [0xe0ff].
      I/O at 0xe100 [0xe13f].

---------  corresponding lines from kernel messages
i810: Intel ICH 82801AA found at IO 0xe100 and 0xe000, IRQ 10
ac97_codec: AC97 Audio codec, id: 0x4144:0x5348 (Analog Devices AD1881A)

Re: i810 audio doesn't work with 2.4.9

[Doug Ledford]

Andris Pavenis wrote:

> Sorry, my fault:
> 
> seems that i810_audio.c  from 2.4.7 and 2.4.8 works, but both from 2.4.9 and
> 2.4.8-ac8 does not ("suceeded" not to copy right version of this file when
> testing previous time, so had one from 2.4.7 twice)

(hmmm, what changes went into 2.4.9 kernel for this driver, I haven't 
looked?)

So, Alan, I noticed that in the same ac kernel that you applied my 
updates, you also applied some OSS correctness patch from somewhere.  I 
think someone sent you a bogus patch.  I've fixed this problem with 
artsd once before.  My latest patch didn't touch the GETOSPACE code, and 
obviously that's where this problem with artsd comes from, so my patch 
wasn't it.  I haven't seen this other patch, but at least the GETOSPACE 
changes need to be trashed.



-- 

  Doug Ledford <dledford@redhat.com>  http://people.redhat.com/dledford
       Please check my web site for aic7xxx updates/answers before
                       e-mailing me about problems

Re: Qlogic/FC firmware

[Alan Cox]

> You might as well remove all of these drivers in whole, as they are
> basically non-functional without the accompanying firmware.

Now you are talking complete and total crap. I tested this firmware removal
stuff on a QlogicFC 2100 card. It works fine and I am now getting to use
the qualified firmware in the BIOS flash not the unqualified firmware in
the kernel.

Alan

Re: Qlogic/FC firmware

[Alan Cox]

> In many cases the driver worked before, and fails to work afterwards.
> I still contend that the whole driver should have been lifted
> instead of leaving a crippled version there.

Well now you know how everyone feels about the min() and max() changes.

However I _did_ test this, you can update the firmware in your BIOS flash
if you want a specific version and you have out of date firmwre currently
loaded.

Alan

Re: Qlogic/FC firmware

[Alan Cox]

> When the Qlogic,FC sees a master abort, the firmware is essentially
> cleared to zero.

None of the cards I have do this. Is this some kind of sparc specific
firmware problem ?

> If you're going to say "put the user thing in initrd", I'm going to
> say "bite me".  I build a static kernel with no initrd and that is how
> I'd like it to stay.  It is one thing to do initrd firmware loading
> for devices not necessary for booting and mounting root, that is
> acceptable, this isn't.

Why. What exactly is your argument ? Lets waste 128K of kernel space to keep
Dave happy. If the lack of proper boot time init on the sparc64 platform is
causing more problems then copy the firmware image out of the BIOS into the
card if sparc64 is defined.

And an initrd is the right answer. You free up the 128K of wasted space
using it.

Alan

Re: Qlogic/FC firmware

[Alexander Viro]

On Tue, 21 Aug 2001, Alan Cox wrote:

> Why. What exactly is your argument ? Lets waste 128K of kernel space to keep
> Dave happy. If the lack of proper boot time init on the sparc64 platform is
> causing more problems then copy the firmware image out of the BIOS into the
> card if sparc64 is defined.
> 
> And an initrd is the right answer. You free up the 128K of wasted space
> using it.

initrd is the right answer to question "where can I find shitty code?"

Now, having firmware loaded from userland _is_ nice and sane, but we
need something better than initrd for that.

Re: Qlogic/FC firmware

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Tue, 21 Aug 2001 15:17:25 +0100 (BST)
   
   However I _did_ test this, you can update the firmware in your BIOS flash
   if you want a specific version and you have out of date firmwre currently
   loaded.

There is no BIOS flash on my machines (onboard controllers).  The
kernel driver must be where the firmware comes from to boot reliably.

You broke this driver on all of my systems having QLogic,FC onboard.
The next time I powercycle or there is a PCI master abort on that bus
segment, I will be without a bootable machine.

Later,
David S. Miller
davem@redhat.com

Re: Qlogic/FC firmware

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Tue, 21 Aug 2001 15:15:20 +0100 (BST)

   > You might as well remove all of these drivers in whole, as they are
   > basically non-functional without the accompanying firmware.
   
   Now you are talking complete and total crap. I tested this firmware removal
   stuff on a QlogicFC 2100 card. It works fine

It works fine in your x86-centric world, on your x86 BIOS card.
It breaks on every UltraSparc system I have with qlogic,fc onboard.
One powercycle or PCI master abort, and I am unbootable.

And I am the one talking crap? :-)

Later,
David S. Miller
davem@redhat.com

Re: Qlogic/FC firmware

[Rik van Riel]

On Tue, 21 Aug 2001, David S. Miller wrote:

> And I am the one talking crap? :-)

"All the world's a SPARC"

I guess using an initrd on these 0.3% of machines shouldn't
be too big a problem compared to violating the GPL and adding
to kernel bloat for everybody else.

cheers,

Rik
--
IA64: a worthy successor to i860.

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: Qlogic/FC firmware

[Alan Cox]

> I guess using an initrd on these 0.3% of machines shouldn't
> be too big a problem compared to violating the GPL and adding
> to kernel bloat for everybody else.

The license thing is a side issue. We have a sane relationship with Qlogic
so that can easily be dealt with. For 2.5 it definitely wants to be in
initrd-tng however

Re: Qlogic/FC firmware

[christophe barbé]

Le mar, 21 aoû 2001 17:30:46, Alan Cox a écrit :
> > I guess using an initrd on these 0.3% of machines shouldn't
> > be too big a problem compared to violating the GPL and adding
> > to kernel bloat for everybody else.
> 
> The license thing is a side issue. We have a sane relationship with
> Qlogic
> so that can easily be dealt with. For 2.5 it definitely wants to be in
> initrd-tng however

I'm afraid that I don't understand what you mean by "a sane relationship
with Qlogic".
If the relation between kernel hackers and qlogic is good, why we don't
have an uptodate firmware with IP support and without license issue ?

The initial reason behind the firmware drop was the license.
And now it's a waste of RAM.

A few times ago, Dmitry (I don't remember his name but If you want I can
find it) has proposed an update for this driver including twos uptodate
firmware (with and without IP support). He give us a link to a website to
upload his patches ( http://www.datafoundation.org/qlogicfc/ ) but now the
link is broken.
If you are interresting I should be able to find his patch on an old disk.

This update resolves nearly all my problems with the in-kernel one.

And IIRC the included firmwares has no BSD license (no license at all
IIRC).

Christophe

-- 
Christophe Barbé
Software Engineer - christophe.barbe@lineo.fr
Lineo France - Lineo High Availability Group
42-46, rue Médéric - 92110 Clichy - France
phone (33).1.41.40.02.12 - fax (33).1.41.40.02.01
http://www.lineo.com

Re: Qlogic/FC firmware

[Ricky Beam]

On Tue, 21 Aug 2001, Rik van Riel wrote:
>I guess using an initrd on these 0.3% of machines shouldn't
>be too big a problem

Were's the initrd going to get the firmware?  It's not in the freakin' tree
anymore.  If I have to go download a firmware image and stick it in an
initrd, why the hell wouldn't I just plug it directly into the kernel?
It's *my* kernel.

> compared to violating the GPL

Oh for the love of God, will you people stop drooling over the fucking GPL?
It's *firmware*... it's just a bunch of bits.  It's *not* a program the
kernel executes.  It's just data. (__init_data to be exact.)

> and adding
>to kernel bloat for everybody else.

One man's bloat is another man's functionality.  Besides, it's got a hell of
a lot of company.  Ask yourself, will you miss 128_K_ in a machine with a
fiber channel card in it?  It "bloats" the kernel for people using that
hardware -- machines that don't need it can release it.

--Ricky

(If Sun would get off their ass(es) and support firewire, I'd stop using linux
 all together.  And before some smartass says they do, point to the ohci and
 sbp-2 drivers on the Solaris 8 Sparc CD.)

Re: Qlogic/FC firmware

[Alan Cox]

> Were's the initrd going to get the firmware?  It's not in the freakin' tree
> anymore.  If I have to go download a firmware image and stick it in an
> initrd, why the hell wouldn't I just plug it directly into the kernel?
> It's *my* kernel.

Well initrd is _user_ space so it probably gets them from user space tools.

> Oh for the love of God, will you people stop drooling over the fucking GPL?
> It's *firmware*... it's just a bunch of bits.  It's *not* a program the
> kernel executes.  It's just data. (__init_data to be exact.)

Look, if its not distributable then its no good to anyone.

> a lot of company.  Ask yourself, will you miss 128_K_ in a machine with a
> fiber channel card in it?  It "bloats" the kernel for people using that
> hardware -- machines that don't need it can release it.

Well maybe, and whats the right way to do that, oh my god I do believe its
an initrd.

Well well well

Alan

Re: Qlogic/FC firmware

[Ricky Beam]

On Tue, 21 Aug 2001, Alan Cox wrote:
>Look, if its not distributable then its no good to anyone.

Who said it wasn't distributable?  The license in the 2.4.5 file doesn't say
you cannot distribute it.  It doesn't even prevent you from sell it.  You have
to leave the notice intact and not use it as a means to attach the Qlogic
mark to any derived product.

Other than it's age, I see *zero* reason to remove it from the tree.

>Well maybe, and whats the right way to do that, oh my god I do believe its
>an initrd.

__init_data

--Ricky

Re: Qlogic/FC firmware

[Alan Cox]

> On Tue, 21 Aug 2001, Alan Cox wrote:
> >Look, if its not distributable then its no good to anyone.
> 
> Who said it wasn't distributable?  The license in the 2.4.5 file doesn't say
> you cannot distribute it.  It doesn't even prevent you from sell it.  You have
> to leave the notice intact and not use it as a means to attach the Qlogic
> mark to any derived product.

 * 2. Redistribution in binary form must reproduce the above copyright
 *    notice, this list of conditions and the following disclaimer in the
 *    documentation and/or other materials provided with the distribution.

Which is a problem because the kernel is GPL, that isnt GPL compatible
and until this was pointed out nobody has been going to print their blurb
in all the manuals - because after all it says GPL.

Licenses cannot be ignored. If I merge non GPL compliant code I'm doing to
the contributors to the kernel the very same thing people pirating windows
do to microsoft. 

> Other than it's age, I see *zero* reason to remove it from the tree.

Well let me quote from it again

/*
 *     W     W     A     RRRR    N   N   IIIII   N   N    GGG
 *     W     W    A A    R   R   N   N     I     N   N   G   G
 *     W     W   A   A   R  R    NN  N     I     NN  N   G  
 *     W  W  W   AAAAA   RRR     N N N     I     N N N   G  GG
 *     W  W  W   A   A   R R     N  NN     I     N  NN   G   G
 *     W W W W   A   A   R  R    N   N     I     N   N   G   G
 *      W   W    A   A   R   R   N   N   IIIII   N   N    GGGG
 *
 *         This version of firmware is a release candidate,
 *     design verification testing (DVT) has not been completed.
 */

or in simple terms "might lose all your data"

> >Well maybe, and whats the right way to do that, oh my god I do believe its
> >an initrd.
> 
> __init_data

Doesnt work in modules. See previous twice repeated discussion. 

As Matthew has noted, we have a source of newer firmware, and because the
sparc's have this annoying firmware problem it is going to be appropriate
to add build in firmware as a config option (probably set with def_bool on
sparc..)

At the time a) I didnt realise the sparc setup was so anal, and b) I didnt
know about the firmware update.

Alan

Re: Qlogic/FC firmware

[Matthew Jacob]

> > Other than it's age, I see *zero* reason to remove it from the tree.
>
> Well let me quote from it again
>
> /*
>  *     W     W     A     RRRR    N   N   IIIII   N   N    GGG
>  *     W     W    A A    R   R   N   N     I     N   N   G   G
>  *     W     W   A   A   R  R    NN  N     I     NN  N   G
>  *     W  W  W   AAAAA   RRR     N N N     I     N N N   G  GG
>  *     W  W  W   A   A   R R     N  NN     I     N  NN   G   G
>  *     W W W W   A   A   R  R    N   N     I     N   N   G   G
>  *      W   W    A   A   R   R   N   N   IIIII   N   N    GGGG
>  *
>  *         This version of firmware is a release candidate,
>  *     design verification testing (DVT) has not been completed.
>  */
>
> or in simple terms "might lose all your data"

Take this with a grain of salt. It's true that it hasn't passed DVT. But it's
also true that f/w which *has* passed DVT has eaten your data as well.

Almost precisely by definition, if the system you embed this firmware in is
*not* one that has gone thru Qlogic DVT, you should in fact be putting the
warning on yourself.

>
> > >Well maybe, and whats the right way to do that, oh my god I do believe its
> > >an initrd.
> >
> > __init_data
>
> Doesnt work in modules. See previous twice repeated discussion.
>
> As Matthew has noted, we have a source of newer firmware, and because the
> sparc's have this annoying firmware problem it is going to be appropriate
> to add build in firmware as a config option (probably set with def_bool on
> sparc..)
>
> At the time a) I didnt realise the sparc setup was so anal, and b) I didnt
> know about the firmware update.
>

It's not just a question of having firmware updated into flash- which, btw,
companies like Veritas have shied away from getting custoemrs to do because
you *do* have a small but finite amount of risk updating flash. It's also code
that as yet has to be written for qlogicfc (e.g.) that would pull it *out* of
flash so it can be pushed into SRAM (which is what the BIOS code on other
platforms do).

-matt

Re: Qlogic/FC firmware

[Jes Sorensen]

>>>>> "Matthew" == Matthew Jacob <mjacob@feral.com> writes:

Matthew> It's not just a question of having firmware updated into
Matthew> flash- which, btw, companies like Veritas have shied away
Matthew> from getting custoemrs to do because you *do* have a small
Matthew> but finite amount of risk updating flash. It's also code that
Matthew> as yet has to be written for qlogicfc (e.g.) that would pull
Matthew> it *out* of flash so it can be pushed into SRAM (which is
Matthew> what the BIOS code on other platforms do).

Yeah vendors tend to like this idea, it's not just Veritas and QLogic
who went down this path, unfortunately.

Updating the flash does seem very easy, and the good thing about the
QLA adapters is that you can reflash even if you screwed up in the
first place. Yup I tried that ;) Writing the flash utility seems like
a piece of cake in this as well. What doesn't look as easy is to
figure out that directory structure of the firmware images ;-( Any
chance you got some data on that?

Cheers,
Jes

Re: Qlogic/FC firmware

[Matthew Jacob]

> >>>>> "Matthew" == Matthew Jacob <mjacob@feral.com> writes:
>
> Matthew> It's not just a question of having firmware updated into
> Matthew> flash- which, btw, companies like Veritas have shied away
> Matthew> from getting custoemrs to do because you *do* have a small
> Matthew> but finite amount of risk updating flash. It's also code that
> Matthew> as yet has to be written for qlogicfc (e.g.) that would pull
> Matthew> it *out* of flash so it can be pushed into SRAM (which is
> Matthew> what the BIOS code on other platforms do).
>
> Yeah vendors tend to like this idea, it's not just Veritas and QLogic
> who went down this path, unfortunately.
>
> Updating the flash does seem very easy, and the good thing about the QLA
> adapters is that you can reflash even if you screwed up in the first
> place. Yup I tried that ;) Writing the flash utility seems like a piece of
> cake in this as well. What doesn't look as easy is to figure out that
> directory structure of the firmware images ;-( Any chance you got some
> data on that?

There are x86 utilities from Qlogic to *write* the flash. Reading the flash
isn't that hard itself. There is no documentation as to flash layout. Older
versions of the Linux driver say where they keep a copy of the current target
to WWN mapping in flash (which is a 'maybe not so bright' idea).

We could ask them where it is.

-matt

Re: Qlogic/FC firmware

[Matthew Jacob]

> We could ask them where it is.

I've asked their tech support folks about where and how big in flash the RISC
code is for all their cards. I have other people I can ask too. Let's see if
they're helpful or open about it.

I'm not as sensitive to the bloat issues as I should be- I suppose because I
tend to see all of USB as bloat myself :).. I tend to think that the generic
shipping kernel case should have all of the firmware images there to download
from the driver.

That said, it *would* be nice to give people some tools to strip things down
if they really want, out of a 2GB system, a couple hundred kbytes back that
can the be gobbled up by something really hungry for it like a memory leak in
X or Matzohilla or something :-)....

-matt

Re: Qlogic/FC firmware

[David S. Miller]

   From: Matthew Jacob <mjacob@feral.com>
   Date: Wed, 22 Aug 2001 09:17:07 -0700 (PDT)

   Matzohilla

You're such a nebbish Matt. :-)

Later,
David S. Miller
davem@redhat.com

Re: Qlogic/FC firmware

[Ricky Beam]

On Wed, 22 Aug 2001, Alan Cox wrote:
> * 2. Redistribution in binary form must reproduce the above copyright
> *    notice, this list of conditions and the following disclaimer in the
> *    documentation and/or other materials provided with the distribution.
>
>Which is a problem because the kernel is GPL, that isnt GPL compatible
>and until this was pointed out nobody has been going to print their blurb
>in all the manuals - because after all it says GPL.

The kernel source counts as "other materials".

>> Other than it's age, I see *zero* reason to remove it from the tree.
>
>Well let me quote from it again
...
>or in simple terms "might lose all your data"

Gee.  And how many installation have been running that firmware for how
many years without problems?  As I said before, other than it's age, there's
no reason to simply delete it.

>Doesnt work in modules. See previous twice repeated discussion.

We aren't talking about a module when it's compiled into the kernel.  In
the case(s) of modules, there are many ways to provide data (such as
firmware) without the aforementioned bloat everyone wants to bitch about.
And I'd like to point out the people screaming about bloat do not have
the hardware for which this driver is required and thusly will *never*
lose the coveted 128K for it's firmware.  There are areas of "bloat" with
far further reach than this.  Go get a few thousand signatures from actual
Qlogic FC owners running linux oking your destablization of the driver and
I'll leave this alone.  Otherwise, either delete the driver entirely or
put the firmware for which the driver is designed and proven stable by the
test of time back in the tree.

This is the 2.4 "stable" kernel tree.  One would assume people work towards
*increasing* it's stability by fixing its flaws, not wildly deleting shit
and (possibly) randomly breaking things. (But I digress.  I'm treading into
the abyss of source/configuration management of which the kernel tree has
none.)

>As Matthew has noted, we have a source of newer firmware, and because the
>sparc's have this annoying firmware problem it is going to be appropriate
>to add build in firmware as a config option (probably set with def_bool on
>sparc..)

As I recall, the Qlogic ISP driver has had a similar option for years.
(Maybe not to prevent compilation but certainly to prevent loading it.)

>At the time a) I didnt realise the sparc setup was so anal, and b) I didnt
>know about the firmware update.

So basically, you had no fucking clue what kind of instability you were about
to introduce into the current "stable" line of kernels, but did it anyway
simply because of the wording (and your interpretation thereof) of the license
on a firmware "data" file.  Wording that has been unchanged since the day
the file was entered into the tree.  If there were objections, questions,
or other concerns, they should have been raised then and not months or years
later.  And there should have been at least some discussion before removing
the file and seeing who notices. (If I missed this discussion, then I
apologize.)

--Ricky

Re: Qlogic/FC firmware

[Alan Cox]

> We aren't talking about a module when it's compiled into the kernel.  In
> the case(s) of modules, there are many ways to provide data (such as
> firmware) without the aforementioned bloat everyone wants to bitch about.

Like loading it from user space. Which is exactly the same code needed
for an initrd. Amazing isn it.

> And I'd like to point out the people screaming about bloat do not have
> the hardware for which this driver is required and thusly will *never*

Wrong. I object to wasting 128K and I have fibrechannel. Its that kind of
sloppy "who cares about 128K" thinking that leads to several megs
disappearing and your OS turning into sludge. 

> So basically, you had no fucking clue what kind of instability you were about
> to introduce into the current "stable" line of kernels, but did it anyway

It caused a trivial little problem for a few sparc people which basically has 
only been more than a

	"Hey Alan, sparc needs firmware compiled in can you fix"

because it seems you have to be arrogant and opinionated to own a sparc64
box 8)

> the file was entered into the tree.  If there were objections, questions,
> or other concerns, they should have been raised then and not months or years
> later.  And there should have been at least some discussion before removing

Take that one up with Linus. I didn't merge it originally

Alan

Re: Qlogic/FC firmware

[Ricky Beam]

On Wed, 22 Aug 2001, Alan Cox wrote:
>Like loading it from user space. Which is exactly the same code needed
>for an initrd. Amazing isn it.

Amazing *and* unnecessarily complex.  What a bargin!  And it's still going
to consume that precious 128K even when loaded from usersapce as the driver
will still need to keep it.  And aren't you one of the Preists of Text in
/proc -- those of the belief in managing everything with 'cat' and 'vi'.

>Wrong. I object to wasting 128K and I have fibrechannel. Its that kind of
>sloppy "who cares about 128K" thinking that leads to several megs
>disappearing and your OS turning into sludge.

Way too damn late make that argument.  That snowball has been gaining mass
and speed for years now.  I would not recommend standing in front of it.

>> So basically, you had no fucking clue what kind of instability you were about
>> to introduce into the current "stable" line of kernels, but did it anyway
>
>It caused a trivial little problem for a few sparc people which basically has
>only been more than a
>
>	"Hey Alan, sparc needs firmware compiled in can you fix"
>
>because it seems you have to be arrogant and opinionated to own a sparc64
>box 8)

I stand by my original "fucking" comment :-)  I wouldn't trivialize this
"little problem".  I'm guessing this is a problem for Alphas as well.  Altho'
Digital (now Compaq) doesn't call it a Qlogic,FC -- their naming scheme is
on par with Cisco -- it doesn't have any firmware on the card.  SRM loads
firmware on it to scan for devices.  Having a machine randomly crash for no
easily detectable reason (esp. if you log to FC) is not good.  In fact,
one may not notice the problem for some time.

Dave, what kind of messages does a master abort cause?

>> the file was entered into the tree.  If there were objections, questions,
>> or other concerns, they should have been raised then and not months or years
>> later.  And there should have been at least some discussion before removing
>
>Take that one up with Linus. I didn't merge it originally

No, you take it up with Linus (and the Qlogic maintainer) as you are the nut
removing it ... and introducing a great big question mark around the stability
of the Qlogic FC driver.

--Ricky

Re: Qlogic/FC firmware

[Alan Cox]

> Amazing *and* unnecessarily complex.  What a bargin!  And it's still going
> to consume that precious 128K even when loaded from usersapce as the driver
> will still need to keep it.  And aren't you one of the Preists of Text in

No. We can jettison it and you know that in fact you commented on using 
__initdata in the non modular case, so stop trolling and if you are going to
be a pain, at least be consistant in your arguments. 

> /proc -- those of the belief in managing everything with 'cat' and 'vi'.

No. That would be Al Viro. 

> >sloppy "who cares about 128K" thinking that leads to several megs
> >disappearing and your OS turning into sludge.
> 
> Way too damn late make that argument.  That snowball has been gaining mass
> and speed for years now.  I would not recommend standing in front of it.

If I was working on Irix maybe, but I'm not. I've got another .5Mb on my
256Mb box I want back in 2.5 as well 8)

> "little problem".  I'm guessing this is a problem for Alphas as well.  Altho'

Alpha's have x86 emulation in the boot loaders. Thats a not-nice solution to
most of the PC worlds evils but works out.

> >Take that one up with Linus. I didn't merge it originally
> 
> No, you take it up with Linus (and the Qlogic maintainer) as you are the nut
> removing it ... and introducing a great big question mark around the stability
> of the Qlogic FC driver.

See previous four conversations on this. Licensing matters. 

Case closed

Alan

Re: Qlogic/FC firmware

[Ricky Beam]

On Wed, 22 Aug 2001, Alan Cox wrote:
>> Amazing *and* unnecessarily complex.  What a bargin!  And it's still going
>> to consume that precious 128K even when loaded from usersapce as the driver
>> will still need to keep it.  And aren't you one of the Preists of Text in
>
>No. We can jettison it and you know that in fact you commented on using
>__initdata in the non modular case, so stop trolling and if you are going to
>be a pain, at least be consistant in your arguments.

I'm missing the "in the case of sparc" clause.  The sparc (and maybe others?)
have to keep the firmware image in memory in the case that it needs to reload
the sequencer.  The initrd simply adds more tool-chains to keep up-to-date.
Any distribution supporting the Qlogic,FC will have to supply a firmware file
in binary form to initialize the card.  And then you're right back to square
one in the binary distribution arena.

--Ricky

Re: Qlogic/FC firmware

[Alan Cox]

> I'm missing the "in the case of sparc" clause.  The sparc (and maybe others?)
> have to keep the firmware image in memory in the case that it needs to reload

Read the source code. The driver never reloads the firmware on a running
card. So if the sparc needed that it never worked anyway, and I don't follow
your argument.

If you do need to reload the firmware on real live sparc setups after driver
setup then someone needs to do some patching.

Unlike you, I actually read the source

Alan

Re: Qlogic/FC firmware

[Ricky Beam]

On Wed, 22 Aug 2001, Alan Cox wrote:
>Read the source code. The driver never reloads the firmware on a running
>card. So if the sparc needed that it never worked anyway, and I don't follow
>your argument.

If the card wasn't setup by the BIOS (OBP in the Sparc case), then the driver
won't work.  And as late as 2.4.8, RELOAD_FIRMWARE was set to '1'.  Gee,
look what was changed in 2.4.9:

--- v2.4.8/linux/drivers/scsi/qlogicfc.c        Sun Aug 12 13:28:00 2001
+++ linux/drivers/scsi/qlogicfc.c       Sun Aug 12 10:51:41 2001
@@ -98,7 +98,7 @@
    isp2200's firmware.
 */

-#define RELOAD_FIRMWARE                1
+#define RELOAD_FIRMWARE                0

 #define USE_NVRAM_DEFAULTS      1

@@ -440,7 +440,7 @@
 #define MBOX_SEND_CHANGE_REQUEST        0x0070
 #define MBOX_PORT_LOGOUT                0x0071

-#include "qlogicfc_asm.c"
+//#include "qlogicfc_asm.c"

(I will note, that's not even a valid C construct. '//' is a C++ism.)

>Unlike you, I actually read the source

So, was that before or after you removed the firmware? (And who says I don't
read source code?  That's the first place I go to find out who fucked up
what.  I'm running 2.4.5 and below.  The only thing I have running 2.4.9 is
my laptop, and that was a certified nightmare.)

--Ricky

Re: Qlogic/FC firmware

[Alan Cox]

> -#include "qlogicfc_asm.c"
> +//#include "qlogicfc_asm.c"
> 
> (I will note, that's not even a valid C construct. '//' is a C++ism.)

Your C standard is as out of date as your architecture ;)

Re: Qlogic/FC firmware

[David S. Miller]

   From: Ricky Beam <jfbeam@bluetopia.net>
   Date: Wed, 22 Aug 2001 14:46:27 -0400 (EDT)

   On Wed, 22 Aug 2001, Alan Cox wrote:
   >Read the source code. The driver never reloads the firmware on a running
   >card. So if the sparc needed that it never worked anyway, and I don't follow
   >your argument.
   
   If the card wasn't setup by the BIOS (OBP in the Sparc case), then the driver
   won't work.  And as late as 2.4.8, RELOAD_FIRMWARE was set to '1'.  Gee,
   look what was changed in 2.4.9:
   
Please read what people are saying.

He said "running card"  Which means that it does not handle the
PCI master case on a running system, which I never claimed it did.

The older code does guarentee that if I did powercycle and reboot
at that point, I will get a functional card back, which the current
code does not do.

Alan's actively trying to resolve this problem by getting usable
firmware from Qlogic.  What are you doing to improve the situation
besides trolling like made on this list?

Later,
David S. Miller
davem@redhat.com

Re: Qlogic/FC firmware

[Jes Sorensen]

>>>>> "Ricky" == Ricky Beam <jfbeam@bluetopia.net> writes:

Ricky> On Wed, 22 Aug 2001, Alan Cox wrote:
>>> the file was entered into the tree.  If there were objections,
>>> questions, or other concerns, they should have been raised then
>>> and not months or years later.  And there should have been at
>>> least some discussion before removing
>>  Take that one up with Linus. I didn't merge it originally

Ricky> No, you take it up with Linus (and the Qlogic maintainer) as
Ricky> you are the nut removing it ... and introducing a great big
Ricky> question mark around the stability of the Qlogic FC driver.

*plonk*

Jes

Re: Qlogic/FC firmware

[Jes Sorensen]

>>>>> "Ricky" == Ricky Beam <jfbeam@bluetopia.net> writes:

Ricky> On Wed, 22 Aug 2001, Alan Cox wrote:
>> At the time a) I didnt realise the sparc setup was so anal, and b)
>> I didnt know about the firmware update.

Ricky> So basically, you had no fucking clue what kind of instability
Ricky> you were about to introduce into the current "stable" line of
Ricky> kernels, but did it anyway simply because of the wording (and
Ricky> your interpretation thereof) of the license on a firmware
Ricky> "data" file.  Wording that has been unchanged since the day the
Ricky> file was entered into the tree.  If there were objections,
Ricky> questions, or other concerns, they should have been raised then
Ricky> and not months or years later.  And there should have been at
Ricky> least some discussion before removing the file and seeing who
Ricky> notices. (If I missed this discussion, then I apologize.)

Wording as interpreted by lawyers and anybody who is capable of
reading English.

As to your argument about raising the concerns when the thing went
in. We (as in the kernel developers) made a mistake, we noticed it we
fix it ... see it's really that simple.

Jes

Re: Qlogic/FC firmware

[Rik van Riel]

On Wed, 22 Aug 2001, Ricky Beam wrote:

> So basically, you had no fucking clue

Since you're the expert, why won't we all wait for
YOUR patch to fix the problem? ;)

Rik
--
IA64: a worthy successor to i860.

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: Qlogic/FC firmware

[Oliver Neukum]

> > Oh for the love of God, will you people stop drooling over the fucking
> > GPL? It's *firmware*... it's just a bunch of bits.  It's *not* a program
> > the kernel executes.  It's just data. (__init_data to be exact.)
>
> Look, if its not distributable then its no good to anyone.

Are you allowed to distribute an initrd that contains it, build from it and 
GNU tools ?

	Regards
		Oliver

Re: Qlogic/FC firmware

[Jes Sorensen]

>>>>> "Oliver" == Oliver Neukum <Oliver.Neukum@lrz.uni-muenchen.de> writes:

>> > Oh for the love of God, will you people stop drooling over the
>> fucking > GPL? It's *firmware*... it's just a bunch of bits.  It's
>> *not* a program > the kernel executes.  It's just
>> data. (__init_data to be exact.)
>> 
>> Look, if its not distributable then its no good to anyone.

Oliver> Are you allowed to distribute an initrd that contains it,
Oliver> build from it and GNU tools ?

If you don't violate any of the licenses of the apps you stick on the
initrd. The fact you use GNU tools to build the initrd doesn't really
matter.

The problem with including this in the kernel is that the kernel as a
whole is GPL'ed so including something that is in conflict with the
GPL is kinda problematic.

Cheers
Jes

Re: Qlogic/FC firmware

[Ricky Beam]

On Wed, 22 Aug 2001, Oliver Neukum wrote:
>> > Oh for the love of God, will you people stop drooling over the fucking
>> > GPL? It's *firmware*... it's just a bunch of bits.  It's *not* a program
>> > the kernel executes.  It's just data. (__init_data to be exact.)
>>
>> Look, if its not distributable then its no good to anyone.
>
>Are you allowed to distribute an initrd that contains it, build from it and
>GNU tools ?

Depends on who's interpreting the license.  I say as long as the source to
the whole mess is available (even provided *with* the binaries), you aren't
violating anything.  On the otherhand, strictly speaking, once the file
is compiled, it becomes "in binary form" which needs the notice attached to
it in some way -- like all the fine print one finds on any box of stuff from
Sun, Compaq/Digital, Cisco, etc.

However, as others have said, none of us are lawyers.  Of course, none of us
have been sued by Qlogic either.

--Ricky

Re: Qlogic/FC firmware

[Matthew Jacob]

> --Ricky
>
> (If Sun would get off their ass(es) and support firewire, I'd stop using linux
>  all together.  And before some smartass says they do, point to the ohci and
>  sbp-2 drivers on the Solaris 8 Sparc CD.)

Oh-yeah- the ones that cause panics when attached to USB multiport terminal
switch hubs?

Re: Qlogic/FC firmware

[Ricky Beam]

On Tue, 21 Aug 2001, Matthew Jacob wrote:
>Oh-yeah- the ones that cause panics when attached to USB multiport terminal
>switch hubs?

No, the non-existant ieee1394 (firewire) ones.  I refuse to by Sun hardware
with IDE and USB capability.

--Ricky

Re: Qlogic/FC firmware

[David S. Miller]

   From: Ricky Beam <jfbeam@bluetopia.net>
   Date: Tue, 21 Aug 2001 18:45:02 -0400 (EDT)
   
Re: GPL  It's a source file and it had a copyright at the top.
Regardless of what the contents mean, the copyright the author
put there has to be abided by.

   (If Sun would get off their ass(es) and support firewire, I'd stop using linux
    all together.  And before some smartass says they do, point to the ohci and
    sbp-2 drivers on the Solaris 8 Sparc CD.)

I normally chastise Sun just to get my jollies off, but this
is a too much of an untrue statement about them even for me
to tolerate.

Solaris-8 supports fully the ieee1394 interfaces on Ultra-III
machines just fine.  I don't know what the package name is, but
the kernel drivers are there in the SME hw releases done for
the Ultra-III.

Later,
David S. Miller
davem@redhat.com

Re: Qlogic/FC firmware

[David S. Miller]

   From: Rik van Riel <riel@conectiva.com.br>
   Date: Tue, 21 Aug 2001 11:42:11 -0300 (BRST)
   
   I guess using an initrd on these 0.3% of machines shouldn't
   be too big a problem compared to violating the GPL and adding
   to kernel bloat for everybody else.

init_data the firmware, it isn't bloat.

The copyright is a totally seperate issue.

Later,
David S. Miller
davem@redhat.com
   

Re: Qlogic/FC firmware

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Tue, 21 Aug 2001 15:24:17 +0100 (BST)

   > When the Qlogic,FC sees a master abort, the firmware is essentially
   > cleared to zero.
   
   None of the cards I have do this. Is this some kind of sparc specific
   firmware problem ?

Without x86 BIOS this is the situation you will get.

Unless you tell the firmware to probe all scsi devices at the OBP
command line, there is no firmware loaded into the FC card.  There
is no reason for firmware to get loaded onto the card.

So if I boot kernels from the Symbios scsi device and use a disk
on my qlogic,fc as root, I am fucked after my next power cycle.
This is exactly my setup on one of my systems.
   
What other non-x86 systems trying to use QLogic/FC cards as root?
Such as PPC.  There is nothing wrong or stupid about this, and such
systems have been broken by removing the firmware.

   Why. What exactly is your argument ? Lets waste 128K of kernel space to keep
   Dave happy. If the lack of proper boot time init on the sparc64 platform is
   causing more problems then copy the firmware image out of the BIOS into the
   card if sparc64 is defined.
   
That would require that I know where in the forth image Sun
sticks the firmware, I have no way to figure this out except
by toiling over disassembler dumps of the cards firmware.

I have no guarentees even that this is where it is (in the PCI
ROM of the qlogic,fc card) they could have just as well have stuffed
it into some arbitrary place in the main system forth ROM.

   And an initrd is the right answer. You free up the 128K of wasted space
   using it.
   
I'd rather have 128k of wasted space than an unbootable system.

All of Matt Jacob's drivers include all the firmware.  And this
makes a lot of sense for another reason, QA.  If you know what
firmware rev. everyone is using, you don't have to second guess
bug reports by asking "oh and what firmware did the BIOS load onto
your card btw" each time.

Nobody, and I mean nobody, except Linux has a Qlogic,FC kernel driver
that fails to load a known version of the firmware.

Also, "128k of wasted space" is a bogus argument too.  If you
build it statically into the kernel, __init_data the firmware.
And when we have the initmem stuff working for modules again (Jakub
did this years ago) you'll get it free'd up for the non-static case
too.   Fix the problem, not a symptom.

If you had put something like "This driver removed due to copyright
problems" into qlogicfc.c then I would have had no problems.

But to see only the firmware disappear almost looked like a careless
merge error.

Later,
David S. Miller
davem@redhat.com

Re: Qlogic/FC firmware

[Alan Cox]

> There is no BIOS flash on my machines (onboard controllers).  The
> kernel driver must be where the firmware comes from to boot reliably.

Ok so you have a sparc specific firmware loading problem. So IMHO the
right thing to do is to write a sparc specific firmware loader - either
as a module living in arch/sparc64 or assuming you never need to reload
the firmware past boot - use an initrd

For the other 99.9% of the userbase we saved 128Kbytes of ram and improved
reliability by not loading half tested firmeware on them.

Alan

Re: Qlogic/FC firmware

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Tue, 21 Aug 2001 15:45:26 +0100 (BST)
   
   For the other 99.9% of the userbase we saved 128Kbytes of ram and improved
   reliability by not loading half tested firmeware on them.

We saved nothing, __init_data the firmware, problem solved.

If the firmware was out of date, update it to a known "Qlogic stamp of
approval" version.

Later,
David S. Miller
davem@redhat.com

Re: Qlogic/FC firmware

[David S. Miller]

   From: Alan Cox <alan@lxorguk.ukuu.org.uk>
   Date: Tue, 21 Aug 2001 15:45:26 +0100 (BST)
   
   Ok so you have a sparc specific firmware loading problem.

No, it is a non-x86 firmware loading problem.  Do not label
it is as a "all the world a Sparc" problem, it isn't centric
to any particular platform.

Later,
David S. Miller
davem@redhat.com

Re: Qlogic/FC firmware

[Jes Sorensen]

>>>>> "David" == David S Miller <davem@redhat.com> writes:

David>    From: Alan Cox <alan@lxorguk.ukuu.org.uk> Date: Tue, 21 Aug
David> 2001 15:45:26 +0100 (BST)
   
>    Ok so you have a sparc specific firmware loading problem.

David> No, it is a non-x86 firmware loading problem.  Do not label it
David> is as a "all the world a Sparc" problem, it isn't centric to
David> any particular platform.

It's centric to a limited userbase, other systems such as Alpha and
ia64 can load the firmware off the cards no problem.

Doesn't mean we have solved your problem, true. But initrd is still a
cleaner solution and gets around the licensing problem as well.

Jes

Re: Qlogic/FC firmware

[Alan Cox]

> We saved nothing, __init_data the firmware, problem solved.

Except when its modular, as in 99% of distro kernels

> If the firmware was out of date, update it to a known "Qlogic stamp of
> approval" version.

That requires sorting licensing out with Qlogic. I've talked to them
usefully about other stuff so I'll pursue it for a seperate firmware
loader module.

Alan

Re: Qlogic/FC firmware

[Jes Sorensen]

>>>>> "Alan" == Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

>> If the firmware was out of date, update it to a known "Qlogic stamp
>> of approval" version.

Alan> That requires sorting licensing out with Qlogic. I've talked to
Alan> them usefully about other stuff so I'll pursue it for a seperate
Alan> firmware loader module.

Well getting firmware out of them tends to be up and down.

However I just looked through the QLogic v4.27 provided driver from
their web site and it does in fact included firmware with a GPL
license.

Dave, if you want to play with this and stick it into the qlogicfc.c
driver then you will at least have something that sorta works for now
(module all the other problems with that driver).

http://www.qlogic.com/bbs-html/csg_web/adapter_pages/driver_pages/22xx/22linux.html

They do have a stupid 'read and agree' license in front of that page
if you go in via the official qlogic.com door, however if the code
inside is GPL then I asume it's GPL.

Cheers
Jes

Re: Qlogic/FC firmware

[Alan Cox]

> 
> >>>>> "Alan" == Alan Cox <alan@lxorguk.ukuu.org.uk> writes:
> 
> >> If the firmware was out of date, update it to a known "Qlogic stamp
> >> of approval" version.
> 
> Alan> That requires sorting licensing out with Qlogic. I've talked to
> Alan> them usefully about other stuff so I'll pursue it for a seperate
> Alan> firmware loader module.
> 
> Well getting firmware out of them tends to be up and down.
> 
> However I just looked through the QLogic v4.27 provided driver from
> their web site and it does in fact included firmware with a GPL
> license.
> 
> Dave, if you want to play with this and stick it into the qlogicfc.c
> driver then you will at least have something that sorta works for now
> (module all the other problems with that driver).
> 
> http://www.qlogic.com/bbs-html/csg_web/adapter_pages/driver_pages/22xx/22linux.html
> 
> They do have a stupid 'read and agree' license in front of that page
> if you go in via the official qlogic.com door, however if the code
> inside is GPL then I asume it's GPL.
> 
> Cheers
> Jes
> 

Re: Qlogic/FC firmware

[Matthew Jacob]

On Tue, 21 Aug 2001, Alan Cox wrote:

> >
> > >>>>> "Alan" == Alan Cox <alan@lxorguk.ukuu.org.uk> writes:
> >
> > >> If the firmware was out of date, update it to a known "Qlogic stamp
> > >> of approval" version.
> >
> > Alan> That requires sorting licensing out with Qlogic. I've talked to
> > Alan> them usefully about other stuff so I'll pursue it for a seperate
> > Alan> firmware loader module.
> >
> > Well getting firmware out of them tends to be up and down.
> >
> > However I just looked through the QLogic v4.27 provided driver from
> > their web site and it does in fact included firmware with a GPL
> > license.

Interesting- I should have checked- they didn't use to have GPL on the f/w.
Earlier driver versions didn't! Oops!

-matt

Re: Qlogic/FC firmware

[Matthew Jacob]

>
> Interesting- I should have checked- they didn't use to have GPL on the f/w.
> Earlier driver versions didn't! Oops!
>

Actually, it's not quite. The 2200 f/w has:

/************************************************************************
 *                                                                      *
 *               --- ISP2200 Initiator/Target Firmware ---              *
 *             with Fabric (Public Loop), Point-point, and              *
 *             expanded LUN addressing for FCTAPE                       *
 *                                                                      *
 ************************************************************************
  Copyright (C) 2000 and 2100 Qlogic Corporation
  (www.qlogic.com)

  This program is distributed in the hope that it will be useful, but
  WITHOUT ANY WARRANTY; without even the implied warranty of
  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
  General Public License for more details.
*******************************************************************
/*
 *      Firmware Version 2.01.27 (11:07 Dec 18, 2000)
 */

(you should  note, btw, that the f/w that's on the internal QLogic website for
OEMS is 2.1.26)

So, it's *sort of* GPLd....

-matt

Re: Qlogic/FC firmware

[Alan Cox]

> However I just looked through the QLogic v4.27 provided driver from
> their web site and it does in fact included firmware with a GPL
> license.

Yep. How has getting them to take updates to their 'official driver' been
going btw

Alan

Re: Qlogic/FC firmware

[David S. Miller]

   From: Jes Sorensen <jes@sunsite.dk>
   Date: 21 Aug 2001 18:39:04 +0200
   
   Dave, if you want to play with this and stick it into the qlogicfc.c

None of this topic matters at all for me.  I
just cvs co -date foo qlogicfc_asm.c and edit
qlogicfc.c to include and use it again :-)

It's more important for others people.

If the firmware you point to is indeed suitably licensed, we should
put it into the tree.  If what you're asking me to do is take care
of this, no problem.

Later,
David S. Miller
davem@redhat.com

Re: Qlogic/FC firmware

[Matthew Jacob]

A few of pieces of information and comments:

The "BSD" copyright in the f/w I have from QLogic is there only because Theo
Deraadt threatened to pull QLogic from the 2.7 OpenBSD release. I'd been
pleading with them for over a year to do *something*. I should have done a BSD
license w/o the advert clause, but, oh, well.....

To understand what shape you're in, it's really best to load firmware into the
card's SRAM. That way you *know* it understands feature foo (like SCCLUN, for
example). QLogic produces so many different flavors of firmware of the same
nominal revision that it's hard to deduce the viability or safety of some
firmware.

That said, it *is* possible in a non-BIOS environment to pull firmware out of
flash rom, I believe- you have to do some hand strobing of registers that
normally only the firmware touches, but I think it *is* technically possible
to pull the contents of flash out into system memory, figure out where the
'resident' f/w image is in this goop and then download *that* into SRAM and
restart the sequencer. Still- this leaves you in the same position as before.

IMO, the best thing is to do an ispfw load module that goes away after you've
loaded SRAM. I've started down this path in FreeBSD (there's a separate ispfw
module)- which will only be really useful if module memory ever gets reclaimed
in FreeBSD :-). This is of particular importance for my driver, because in
supporting 1020 (SBus (yes for NetBSD/OpenBSD, not yet for Linux)  && PCI),
1080/1280 Ultra2, 12160 Ultra3, 2100 FC, 2200FC and 2300FC (next week)- you
end up with an absurd amount of f/w images.

I'm sure there's something similar that can be done for Linux.

Alan- you say you can talk to QLogic- good. I've been finding it harder and
harder to talk to them. If you can get them to put out 2100 and 2200 and 2300
f/w with GPL for Linux- great.

-matt

Re: Qlogic/FC firmware

[Jes Sorensen]

>>>>> "David" == David S Miller <davem@redhat.com> writes:

David> If the firmware you point to is indeed suitably licensed, we
David> should put it into the tree.  If what you're asking me to do is
David> take care of this, no problem.

Well I was offering you a solution to your problem. Whether you want
to take it or not is really up to you. I can live with the driver not
shipping any firmware image, but it sounds like Sparc users can't.

Jes

Re: BUG: pc_keyb.c

[Alan Cox]

> On the other hand, 1 ms is a very long time these days; it is a bit
> surprising that modern hardware should need delays in that order of

I have two boxes that need the 1mS delay, and I actually had to fix a 
missing one to deal with hangs. So its still there, lurking in some boxes
I've had ex Digital engineers confirm the hinote needs the 1mS delay a while
back

Alan

Re: massive filesystem corruption with 2.4.9

[Alan Cox]

> No. I can't find any VIA chipset. I'm really surprised. :-) But it is a=
> n
> original Compaq-Board (EP-Series..) with a horroble BIOS. It seems that=
>  they're
> using intel only..

440BX - good chipset.

Does memtest86 show up anything on this box ?

Re: massive filesystem corruption with 2.4.9

[Kristian]

Alan Cox wrote:
> Does memtest86 show up anything on this box ?

No errors...

Btw: As far as I know did the problem occur since I patched 2.4.5 with ac13 or 
ac15. Maybe a clean 2.4.5 works fine. I'm not sure about this. It's some time 
ago... Did you have made some important ext2-related changes with 2.4.5-ac?. I 
could revert to the old kernel and test him if it is relevant.

Kristian

·· · · reach me :: · ·· ·· ·  · ·· · ··  · ··· · ·
                          :: http://www.korseby.net
                          :: http://www.tomlab.de
kristian@korseby.net ....::

Re: massive filesystem corruption with 2.4.9

[Alan Cox]

> that it is a memory problem i also don't belive. that ram work for over 2 year
> with no errors found with memtest (memtset86, intels memtest) compiling
> seveal times xfree86 and an many many times several kernels. 
> 
> and i never had any problems. until i tried the first time a 2.4.x kernel on 
> the fileserver (that was 2.4.6). so i moved the fileserver back to 2.2.19.

Nod. I can follow that reasoning, I've come across boxes that fialed with
2.4 with memory errors, but not 2.2. So far however those have all shown up
with memtest86, or been Athlon optimisation triggered via things

Curiouser and curiouser

Re: Kernel Startup Delay

[Alan Cox]

> to have a delayed spin up as to not bake the power supply.  These two drives
> will contain a striping RAID.  I am concerned that the kernel will load off
> of the other drives and attempt to start this RAID before the disks have
> even spun up - is there a way to have the kernel delay its startup for a
> certain number of seconds while all the drives spin up?

The kernel scsi code issues test-unit-ready commands and will (should 8)) do
the right things and wait for the drives to spin up. 

Alan

Re: i810 audio doesn't work with 2.4.9

[Alan Cox]

> (hmmm, what changes went into 2.4.9 kernel for this driver, I haven't 
> looked?)

2.4.9 has an obsolete driver

Re: i810 audio doesn't work with 2.4.9

[Alan Cox]

> seems that i810_audio.c  from 2.4.7 and 2.4.8 works, but both from 2.4.9 and
> 2.4.8-ac8 does not ("suceeded" not to copy right version of this file when
> testing previous time, so had one from 2.4.7 twice)

Ok I'll take a glance at that. An strace of artsd failing would be useful

Re: i810 audio doesn't work with 2.4.9

[Andris Pavenis]

On Tuesday 21 August 2001 21:39, Alan Cox wrote:
> > seems that i810_audio.c  from 2.4.7 and 2.4.8 works, but both from 2.4.9
> > and 2.4.8-ac8 does not ("suceeded" not to copy right version of this file
> > when testing previous time, so had one from 2.4.7 twice)
>
> Ok I'll take a glance at that. An strace of artsd failing would be useful

I'm including that at end of this message (a small fragment only):

Some other info:

Got incremental diffs between ac versions since 2.4.5.
Applied all diffs to 2.4.5 version of i810_audio.c except one between 2.4.6-ac1 and 2.4.6-ac2
As result i810 audio seems to work

So it seems that update of i810_audio.c between 2.4.6-ac1 and ac2 breaks it (at least for me).
But I think it still eating too much time (about 2-3% on PIII 700) when I'm not doing anything 
with sound but no more up to 90% as with unmodified version from 2.4.9 (maybe it's a problem
of artsd , I don't know)

Andris

---------------------------- i810_audio.c from 2.4.9  ----------------------------------------------------

09:22:28.328650 select(9, [4 6], [8], [6 8], {0, 119135}) = 1 (out [8], left {0, 120000})
09:22:28.328935 ioctl(8, SNDCTL_DSP_GETOSPACE, 0xbffff5ec) = 0
09:22:28.328984 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 608) = 96
09:22:28.329071 gettimeofday({998374948, 329087}, NULL) = 0
09:22:28.329116 gettimeofday({998374948, 329132}, NULL) = 0
09:22:28.329161 select(9, [4 6], [8], [6 8], {0, 118624}) = 1 (out [8], left {0, 120000})
09:22:28.329401 ioctl(8, SNDCTL_DSP_GETOSPACE, 0xbffff5ec) = 0
09:22:28.329450 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 592) = 80
09:22:28.329537 gettimeofday({998374948, 329553}, NULL) = 0
09:22:28.329582 gettimeofday({998374948, 329599}, NULL) = 0
09:22:28.329628 select(9, [4 6], [8], [6 8], {0, 118157}) = 1 (out [8], left {0, 120000})
09:22:28.329912 ioctl(8, SNDCTL_DSP_GETOSPACE, 0xbffff5ec) = 0
09:22:28.329964 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 608) = 96
09:22:28.330051 gettimeofday({998374948, 330067}, NULL) = 0
09:22:28.330096 gettimeofday({998374948, 330113}, NULL) = 0
09:22:28.330142 select(9, [4 6], [8], [6 8], {0, 117643}) = 1 (out [8], left {0, 120000})
09:22:28.330428 ioctl(8, SNDCTL_DSP_GETOSPACE, 0xbffff5ec) = 0
09:22:28.330476 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 600) = 88
09:22:28.330562 gettimeofday({998374948, 330579}, NULL) = 0
09:22:28.330607 gettimeofday({998374948, 330624}, NULL) = 0

------------------------   2.4.9 with i810_audio.c without patch between 2.4.6-ac1 to ac2  ------------------------------
 
11:38:04.550692 select(9, [4 6], [8], [6 8], {0, 87578}) = 1 (out [8], left {0, 90000})
11:38:04.550973 ioctl(8, SNDCTL_DSP_GETOSPACE, 0xbffff41c) = 0
11:38:04.551025 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024
11:38:04.551116 gettimeofday({998469484, 551134}, NULL) = 0
11:38:04.551162 gettimeofday({998469484, 551180}, NULL) = 0
11:38:04.551207 select(9, [4 6], [8], [6 8], {0, 87062}) = 1 (out [8], left {0, 90000})
11:38:04.551488 ioctl(8, SNDCTL_DSP_GETOSPACE, 0xbffff41c) = 0
11:38:04.551540 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024
11:38:04.551631 gettimeofday({998469484, 551648}, NULL) = 0
11:38:04.551676 gettimeofday({998469484, 551694}, NULL) = 0
11:38:04.551722 select(9, [4 6], [8], [6 8], {0, 86548}) = 1 (out [8], left {0, 90000})
11:38:04.552003 ioctl(8, SNDCTL_DSP_GETOSPACE, 0xbffff41c) = 0
11:38:04.552054 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024
11:38:04.552146 gettimeofday({998469484, 552163}, NULL) = 0
11:38:04.552191 gettimeofday({998469484, 552209}, NULL) = 0
11:38:04.552236 select(9, [4 6], [8], [6 8], {0, 86033}) = 1 (out [8], left {0, 90000})
11:38:04.552518 ioctl(8, SNDCTL_DSP_GETOSPACE, 0xbffff41c) = 0
11:38:04.552569 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1024) = 1024
11:38:04.552658 gettimeofday({998469484, 552676}, NULL) = 0
11:38:04.552703 gettimeofday({998469484, 552721}, NULL) = 0
11:38:04.552749 select(9, [4 6], [8], [6 8], {0, 85521}) = 1 (out [8], left {0, 90000})
11:38:04.553030 ioctl(8, SNDCTL_DSP_GETOSPACE, 0xbffff41c) = 0
11:38:04.553078 write(8, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 560) = 560
11:38:04.553166 gettimeofday({998469484, 553184}, NULL) = 0
11:38:04.553212 gettimeofday({998469484, 553230}, NULL) = 0
11:38:04.553258 select(9, [4 6], [8], [6 8], {0, 85012}) = 1 (out [8], left {0, 80000})

Re: i810 audio doesn't work with 2.4.9

[Doug Ledford]

Andris Pavenis wrote:

> Got incremental diffs between ac versions since 2.4.5.
> Applied all diffs to 2.4.5 version of i810_audio.c except one between 2.4.6-ac1 and 2.4.6-ac2
> As result i810 audio seems to work

Can you send me that incremental patch you left out.  I would like to 
look at it to see what's going on.

> So it seems that update of i810_audio.c between 2.4.6-ac1 and ac2 breaks it (at least for me).
> But I think it still eating too much time (about 2-3% on PIII 700) when I'm not doing anything 
> with sound but no more up to 90% as with unmodified version from 2.4.9 (maybe it's a problem
> of artsd , I don't know)


Yes, it's a problem of artsd.  Someone decided (presumably to avoid the 
occasional pop/click from the startup or shutdown of the sound device) 
to make artsd transmit silence over the sound card when no sounds 
currently need played.  From my perspective, I will *NEVER* use artsd as 
long as it does this.  I find it so extremely stupid and insane to a 
level that can't be measured that I refuse to run that software.  I 
absolutely will not have my systems PCI bus loaded down with a constant 
data transfer when there is no sound to be played.  Right now, that's 
possibly something as small as a 48KHz/16bit data stream.  But what if 
the system is upgraded to an ac3 5.1 digital system.  Then you would 
have something like 384KByte/s of data to transfer over the PCI bus just 
for frickin silence.  Won't ever happen on any machine I use.  Anyone 
trying to do things like disk benchmarks on a system that runs artsd may 
be suprised to find their performance is actually negatively impacted 
just by having that horrible sound daemon running.  Furthermore, I find 
their use of the sound API to be suboptimal, especially when they are 
going to transmit silence all the time.  I am *FAR* happier with the way 
esd actually handles its interaction with the sound card (other issues 
are a different matter, I don't rightly know which one does better sound 
sample upconversion for instance).


-- 

  Doug Ledford <dledford@redhat.com>  http://people.redhat.com/dledford
       Please check my web site for aic7xxx updates/answers before
                       e-mailing me about problems

Re: i810 audio doesn't work with 2.4.9

[Andris Pavenis]

On Thursday 23 August 2001 17:00, Doug Ledford wrote:
> Andris Pavenis wrote:
> > Got incremental diffs between ac versions since 2.4.5.
> > Applied all diffs to 2.4.5 version of i810_audio.c except one between
> > 2.4.6-ac1 and 2.4.6-ac2 As result i810 audio seems to work
>
> Can you send me that incremental patch you left out.  I would like to
> look at it to see what's going on.
>

i810_audio.c works for me if all patches except following is aplied (it's a part of update
from 2.4.6-ac1 to 2.4.6-ac2)

Andris

--- linux-2.4.6-ac1/drivers/sound/i810_audio.c	Sun Jul  8 20:45:41 2001
+++ linux-2.4.6-ac2/drivers/sound/i810_audio.c	Sun Jul  8 21:37:01 2001
@@ -1209,23 +1211,30 @@
 		if (dmabuf->count < 0) {
 			dmabuf->count = 0;
 		}
-		cnt = dmabuf->dmasize - swptr;
-		if(cnt > (dmabuf->dmasize - dmabuf->count))
-			cnt = dmabuf->dmasize - dmabuf->count;
+		cnt = dmabuf->dmasize - dmabuf->fragsize - dmabuf->count;
+		// this is to make the copy_from_user simpler below
+		if(cnt > (dmabuf->dmasize - swptr))
+			cnt = dmabuf->dmasize - swptr;
 		spin_unlock_irqrestore(&state->card->lock, flags);
 
+#ifdef DEBUG2
+		printk(KERN_INFO "i810_audio: i810_write: %d bytes available space\n", cnt);
+#endif
 		if (cnt > count)
 			cnt = count;
 		if (cnt <= 0) {
 			unsigned long tmo;
 			// There is data waiting to be played
-			i810_update_lvi(state,0);
 			if(!dmabuf->enable && dmabuf->count) {
 				/* force the starting incase SETTRIGGER has been used */
 				/* to stop it, otherwise this is a deadlock situation */
 				dmabuf->trigger |= PCM_ENABLE_OUTPUT;
 				start_dac(state);
 			}
+			// Update the LVI pointer in case we have already
+			// written data in this syscall and are just waiting
+			// on the tail bit of data
+			i810_update_lvi(state,0);
 			if (file->f_flags & O_NONBLOCK) {
 				if (!ret) ret = -EAGAIN;
 				return ret;

Re: i810 audio doesn't work with 2.4.9

[Doug Ledford]

Andris Pavenis wrote:
> On Thursday 23 August 2001 17:00, Doug Ledford wrote:
> 
>>Andris Pavenis wrote:
>>
>>>Got incremental diffs between ac versions since 2.4.5.
>>>Applied all diffs to 2.4.5 version of i810_audio.c except one between
>>>2.4.6-ac1 and 2.4.6-ac2 As result i810 audio seems to work
>>>
>>Can you send me that incremental patch you left out.  I would like to
>>look at it to see what's going on.
>>
>>
> 
> i810_audio.c works for me if all patches except following is aplied (it's a part of update
> from 2.4.6-ac1 to 2.4.6-ac2)

[snip]

OK, now I know what is going on.  The change you reference was added to 
the i810 driver because some software (mpg123) could fall behind to the 
point that they drained the buffer, then when they refilled it 
(according to the value sent by the GETOSPACE ioctl), it would cause the 
LVI == CVI and the i810 chipset would essentially stop playing.  This 
made it so that we would wait for the LVI != CVI before we wrote all the 
data into the buffer.  Unfortunately, because artsd in particular opens 
the file in non-blocking mode, we only write as much data as is allowed 
before we hit the simplistic "always write actual space - fragsize bytes 
to avoid the LVI == CVI lockup" issue.  Therefore, artsd would check the 
GETOSPACE ioctl, try to write that amount of data, the write routine 
doesn't write the full amount because of the workaround, we return less 
than a full write, artsd immediately tries to write more, we get in an 
endless loop that wastes lots of CPU cycles.  I'll improve the free 
space calculation and then implement it in both the write and GETOSPACE 
areas so that the return from GETOSPACE is in fact the amount of data we 
will allow to be written.  That will solve the problem for both the 
artsd and the mpg123 cases. (I'll likewise have to do something similar 
with read and GETISPACE, and I'll have to verify that GETOPTR and 
GETIPTR are going to work properly with the new scheme as well).  Once 
that's done, I'll send a 0.05 version of the driver to Alan.




-- 

  Doug Ledford <dledford@redhat.com>  http://people.redhat.com/dledford
       Please check my web site for aic7xxx updates/answers before
                       e-mailing me about problems

Re: 2.4.9 kernel i810 module Initialization problem

[Alan Cox]

>  I was using the i810 module for APG video (2.4.7 kernel) with no problems. 
> After installation of the 2.4.9 kernel the module failed to initialize. 

Linus updated the kernel supporting code for direct rendering to require
XFree 4.1. If you want to use a current kernel with the older XFree 4.0.x
then pick up the 2.4.8-ac tree instead, and use the DRM 4.0 option

Re: yenta_socket hangs sager laptop in kernel 2.4.6-> PNPBIOS life saver

[Alan Cox]

> Try -ac Kernels with integrated PNPBIOS and "lspnp -v",
> then you will see your "motherboard resources". No magic.

Except on the intel boards where your machine crashes, the vaio's where
some queries corrupt memory, the boxes where an interrupt during a pnpbios
call crashes the box, the machines where pnpbios called from both cpus at
the same time is a crash case, the wonderful weird tiny races on some boxes
that use smm traps and fail if random undefined things occur between the
two out instructions...

> Alan, 2.4 would largely benefit from PNPBIOS, do you plan
> to submit this to LT (probably with the proposed life saver fix) ?

Experience is that PnpBIOS services are so astoundingly buggy in many
bioses that they are probably not worth the risk. Ie more boxes break by
calling pnpbios than by assuming the vendor used a sane resource layout.

Before PnPBIOS can go mainstream we'd have to generate a detailed list
of buggy bios signatures

Alan

Re: yenta_socket hangs sager laptop in kernel 2.4.6-> PNPBIOS life saver

[Gunther Mayer]

Alan Cox wrote:
> 
> > Try -ac Kernels with integrated PNPBIOS and "lspnp -v",
> > then you will see your "motherboard resources". No magic.
> 
> Except on the intel boards where your machine crashes, the vaio's where
> some queries corrupt memory, the boxes where an interrupt during a pnpbios
> call crashes the box, the machines where pnpbios called from both cpus at
> the same time is a crash case, the wonderful weird tiny races on some boxes
> that use smm traps and fail if random undefined things occur between the
> two out instructions...

So call it only once early on boot (in real mode) and save the table
for later use (we don't need the fancy features ...) ?

> > Alan, 2.4 would largely benefit from PNPBIOS, do you plan
> > to submit this to LT (probably with the proposed life saver fix) ?
> 
> Experience is that PnpBIOS services are so astoundingly buggy in many
> bioses that they are probably not worth the risk. Ie more boxes break by
> calling pnpbios than by assuming the vendor used a sane resource layout.

How are these bugs handled by Windows ?
Must we only mimick the Windows call layout to
enter bios-writer's well tested code path ?

> Before PnPBIOS can go mainstream we'd have to generate a detailed list
> of buggy bios signatures

Re: yenta_socket hangs sager laptop in kernel 2.4.6-> PNPBIOS life saver

[Gerd Knorr]

On Wed, Aug 22, 2001 at 07:18:20PM +0100, Alan Cox wrote:
> > Try -ac Kernels with integrated PNPBIOS and "lspnp -v",
> > then you will see your "motherboard resources". No magic.
> 
> Except on the intel boards [ ... bios bugs list snipped ... ] 

2.4.8-ac8 works for me, and lspnp does list these "obscure"
ressources:

bytesex kraxel ~# /root/bin/lspnp -v
00 PNP0c01 memory controller: RAM
        mem 0x00000000-0x0009ffff
        mem 0x00100000-0x0bfdffff
        mem 0x0bfe0000-0x0bfeffff
        mem 0x000e0000-0x000fffff
        mem 0xffff0000-0xffffffff
        io 0x0398-0x0399
        io 0x0024-0x003d
        io 0x0062-0x0062
        io 0x0066-0x0066
        io 0x0090-0x009f
        io 0x00a4-0x00bd
        io 0x0230-0x0233
        io 0x1000-0x103f
        io 0x1400-0x140f
        io 0x3810-0x381f

01 PNP0c02 reserved: other
        io 0x0cf8-0x0cff
        io 0x04d0-0x04d1

02 PNP0c04 system peripheral: other
        irq 13
        io 0x00f0-0x00ff

03 PNP0000 system peripheral: programmable interrupt controller
[ ... more standard PC hardware follows ... ]

But it seems they are _not_ reserved by the pnp bios code, at least they
are not listed in /proc/ioports

> Before PnPBIOS can go mainstream we'd have to generate a detailed list
> of buggy bios signatures

Why?  It shouldn't harm if disabled, so IMHO it should be fine when
flagged "experimental" and with a warning label about broken bioses in
Configure.help ...

  Gerd

-- 
Damn lot people confuse usability and eye-candy.

Re: yenta_socket hangs sager laptop in kernel 2.4.6-> PNPBIOS life saver

[Gerd Knorr]

>  But it seems they are _not_ reserved by the pnp bios code, at least they
>  are not listed in /proc/ioports

I've hacked up a small piece of code which does the ressource
registering:

----------------- cut here ------------------
#include <linux/config.h>
#include <linux/module.h>
#include <linux/init.h>

#include <asm/io.h>
#include <asm/dma.h>
#include <asm/uaccess.h>

#include <linux/pnp_bios.h>

static void register_stuff(struct pci_dev *dev)
{
	struct resource *res;
	int i;

	for (i = 0; i < DEVICE_COUNT_RESOURCE &&
		    (dev->resource[i].start || dev->resource[i].end); i++) {
		if (dev->resource[i].start <= 0xffff) {
			res = request_region(dev->resource[i].start,
				dev->resource[i].end - dev->resource[i].start,
				dev->name);
			printk("mboard: alloc io: 0x%04lx-0x%04lx [%s,%s]\n",
				dev->resource[i].start,
				dev->resource[i].end,
				dev->name, res ? "ok" : "failed");
		}
	}
	if (i == DEVICE_COUNT_RESOURCE)
		printk("mboard: warning: >= %d resources\n",
			DEVICE_COUNT_RESOURCE);
}

static int mboard_init(void)
{
	struct pci_dev *dev;
	int found;

	dev = NULL;
	found = 0;
	while ((dev=pnpbios_find_device("PNP0c01",dev))) {
		register_stuff(dev);
		found++;
	}
	if (found)
		MOD_INC_USE_COUNT;
	return 0;
}

static void mboard_fini(void)
{
}

module_init(mboard_init);
module_exit(mboard_fini);

----------------- cut here ------------------

A full patch with this + pnpbios support (pulled out of -ac) for 2.4.9
is available from http://bytesex.org/notebook/pnpbios-2.4.9.diff

Now I have this in my /proc/ioports:
0000-001f : dma1
0020-003f : pic1
0040-005f : timer
0060-006f : keyboard
0070-007f : rtc
0080-008f : dma page reg
0090-009f : PNPBIOS
00a0-00bf : pic2
00c0-00df : dma2
00f0-00ff : fpu
01f0-01f7 : ide0
0230-0233 : PNPBIOS
02f8-02ff : nsc-ircc
0398-0399 : PNPBIOS
03c0-03df : vesafb
03f6-03f6 : ide0
0cf8-0cff : PCI conf1
1000-103f : PNPBIOS
1100-110f : Intel Corporation 82440MX EIDE Controller
  1100-1107 : ide0
1200-121f : Intel Corporation 82440MX USB Universal Host Controller
  1200-121f : usb-uhci
1400-140f : PNPBIOS
1500-153f : Intel Corporation 82440MX AC'97 Audio Controller
  1500-153f : Intel 440MX
1600-16ff : Intel Corporation 82440MX AC'97 Audio Controller
  1600-16ff : Intel 440MX
1700-177f : PCI device 8086:7196 (Intel Corporation)
1800-18ff : PCI device 8086:7196 (Intel Corporation)
1c00-1cff : PCI CardBus #01
2000-20ff : PCI CardBus #01
3000-30ff : ATI Technologies Inc 3D Rage P/M Mobility
3810-381f : PNPBIOS
3e00-3eff : Realtek Semiconductor Co., Ltd. RTL-8139
  3e00-3eff : 8139too

Note that DEVICE_COUNT_RESOURCE == 12 is too small to hold all the
resources, so I've bumped that to 16.

  Gerd

-- 
Damn lot people confuse usability and eye-candy.

Re: yenta_socket hangs sager laptop in kernel 2.4.6-> PNPBIOS life saver

[Alan Cox]

> > Before PnPBIOS can go mainstream we'd have to generate a detailed list
> > of buggy bios signatures
> 
> Why?  It shouldn't harm if disabled, so IMHO it should be fine when
> flagged "experimental" and with a warning label about broken bioses in
> Configure.help ...

We will see what happens. Certainly if someone wants to provide pnpbios code
patches for -ac that grab and reserve the motherboard resources from the PCI
code go ahead.

Re: yenta_socket hangs sager laptop in kernel 2.4.6-> PNPBIOS life saver

[kuznet]

Hello!

> We will see what happens. Certainly if someone wants to provide pnpbios code
> patches for -ac that grab and reserve the motherboard resources from the PCI
> code go ahead.

Khm... this does not look simple. Seems, right way involves modification
of each place, where the same ports are used by kernel.
pcmcia-cs had completely private resource manager, so that it just
did not worry about other subsystems and they still were able to allocate
the same resources.

Look f.e. at extermal example, pnpbios announces as "system" resource
all the memory. :-)

Pallaitive soultions, sort of reserving of ports >= 0x1000 using
this information do not look cool too. 

Alexey

Re: yenta_socket hangs sager laptop in kernel 2.4.6-> PNPBIOS life saver

[Gunther Mayer]

kuznet@ms2.inr.ac.ru wrote:
> 
> Hello!
> 
> > We will see what happens. Certainly if someone wants to provide pnpbios code
> > patches for -ac that grab and reserve the motherboard resources from the PCI
> > code go ahead.
> 
> Khm... this does not look simple. Seems, right way involves modification
> of each place, where the same ports are used by kernel.

The PNP0C01 ports are not used by the kernel !
No modifications necessary.

Of course you can teach each driver about their PNP devices, e.g.
my patch for parport_pc.c makes perfect io/irq/dma detection and
saves users from the error-prone procedure to supply module parameters.
Even serial now autodetects IRQ 10 when I give this in my BIOS setup !

> pcmcia-cs had completely private resource manager, so that it just
> did not worry about other subsystems and they still were able to allocate
> the same resources.
> 
> Look f.e. at extermal example, pnpbios announces as "system" resource
> all the memory. :-)

So we have another way besides several INTs to detect the avail mem :-)

> 
> Pallaitive soultions, sort of reserving of ports >= 0x1000 using
> this information do not look cool too.

Gerd's patch rules out your objections and should be included
unconditionally with pnpbios.o.

Probably PNP0C02 wants to be reserved, too.

Re: yenta_socket hangs sager laptop in kernel 2.4.6-> PNPBIOS life saver

[kuznet]

Hello!

> So we have another way besides several INTs to detect the avail mem :-)

Well, if this memory is available then I guess port 0x1000 is "available"
as well and all the rest of ports are not available. :-)

No, something is rotted in this kingdom.


> Probably PNP0C02 wants to be reserved, too.

What's about these, they are nice port and could be used by our irq handler.
According to docs they replace functionality missing in standard
int. controller ports for this chipset.

What's about passing parameters from bios setup to linux...
This is amusing, but not more. I am sorry, I still prefer to use usual
kernel command line instead of some ugly foreign interface.

Alexey

Re: yenta_socket hangs sager laptop in kernel 2.4.6-> PNPBIOS life saver

[Gunther Mayer]

kuznet@ms2.inr.ac.ru wrote:
> 
> Hello!
> 
> > So we have another way besides several INTs to detect the avail mem :-)
> 
> Well, if this memory is available then I guess port 0x1000 is "available"
> as well and all the rest of ports are not available. :-)

Read "available" as "onboard".
Before you use onboard resources you should know what it is !
Surely you don't want to place a PCI ioport window over unknown ports
(as this is what yenta did).

> No, something is rotted in this kingdom.
> 
> > Probably PNP0C02 wants to be reserved, too.
> 
> What's about these, they are nice port and could be used by our irq handler.

These ports are not nice here and should not be user by the irq handler:
   	PNP0c02 Motherboard resources
        io 0x0290-0x0297

> According to docs they replace functionality missing in standard
> int. controller ports for this chipset.

What docs ?

> 
> What's about passing parameters from bios setup to linux...

The BIOS setup uses PNPBIOS to pass parameters to Linux :-)

> This is amusing, but not more. I am sorry, I still prefer to use usual
> kernel command line instead of some ugly foreign interface.

You miss the point of PNP and user-friendliness.

This is a necessary interface to prevent (nearly undebuggable) linux hard hangs !

However, Gerd's debugging forces and his new patch already solve this thread
by giving a working solution.

-
Gunther

Re: yenta_socket hangs sager laptop in kernel 2.4.6-> PNPBIOS life saver

[kuznet]

Hello!

> Before you use onboard resources you should know what it is !
> Surely you don't want to place a PCI ioport window over unknown ports
> (as this is what yenta did).

This is right.

Though, to make situation more clean explain me one much simpler thing:
look at these lines from beginning of my lspnp -v:

00 PNP0c01 System board
        mem 0x00000000-0x0009ffff
        mem 0x00100000-0x0bfdffff
        mem 0x0bfe0000-0x0bfeffff
        mem 0x000e0000-0x000fffff
        mem 0xffff0000-0xffffffff

Look at line mem 0x0bfe0000-0x0bfeffff. I hope you never saw this
motherboard before, like this happens with our poor kernel,
so you are not in better position. Imagine, you are kernel,
and decide are you allowed to use these 64K or not? :-)

Additional information: bios-e820 reports this area as ram in one block
with all the rest of memory 0x00100000-0x0bfeffff.


> What docs ?

Intel datashits from developer.intel.com.

Alexey

Re: yenta_socket hangs sager laptop in kernel 2.4.6-> PNPBIOS life saver

[Gerd Knorr]

On Thu, Aug 23, 2001 at 02:00:35PM +0100, Alan Cox wrote:
> We will see what happens. Certainly if someone wants to provide pnpbios code
> patches for -ac that grab and reserve the motherboard resources from the PCI
> code go ahead.

Here is.

  Gerd

---------------------- cut here -------------------------
--- 2.4.8-ac8/include/linux/pci.h.fix	Fri Aug 24 11:40:14 2001
+++ 2.4.8-ac8/include/linux/pci.h	Fri Aug 24 11:40:28 2001
@@ -317,7 +317,7 @@
 #define DEVICE_COUNT_COMPATIBLE	4
 #define DEVICE_COUNT_IRQ	2
 #define DEVICE_COUNT_DMA	2
-#define DEVICE_COUNT_RESOURCE	12
+#define DEVICE_COUNT_RESOURCE	16
 
 #define PCI_ANY_ID (~0)
 
--- 2.4.8-ac8/drivers/pnp/Makefile.fix	Thu Aug 23 10:08:57 2001
+++ 2.4.8-ac8/drivers/pnp/Makefile	Fri Aug 24 11:47:27 2001
@@ -17,7 +17,7 @@
 isa-pnp-objs := isapnp.o quirks.o $(proc-y)
 
 procpnp-$(CONFIG_PROC_FS) = pnp_proc.o
-pnpbios-objs := pnp_bios.o  $(procpnp-y)
+pnpbios-objs := pnp_bios.o mboard.o $(procpnp-y)
 
 obj-$(CONFIG_ISAPNP) += isa-pnp.o
 obj-$(CONFIG_PNPBIOS) += pnpbios.o
--- 2.4.8-ac8/drivers/pnp/pnp_bios.c.fix	Thu Aug 23 10:08:57 2001
+++ 2.4.8-ac8/drivers/pnp/pnp_bios.c	Fri Aug 24 11:39:21 2001
@@ -49,6 +49,7 @@
 
 void pnp_proc_init(void);
 static void pnpbios_build_devlist(void);
+int pnpbios_request_mboard(void);
 
 /*
  * This is the standard structure used to identify the entry point
@@ -605,6 +606,7 @@
 		break;
 	}
 	pnpbios_build_devlist();
+	pnpbios_request_mboard();
 #ifdef CONFIG_PROC_FS
 	pnp_proc_init();
 #endif
--- 2.4.8-ac8/drivers/pnp/mboard.c.fix	Fri Aug 24 11:39:50 2001
+++ 2.4.8-ac8/drivers/pnp/mboard.c	Fri Aug 24 11:38:38 2001
@@ -0,0 +1,73 @@
+/*
+ * request I/O ports which are used according to the PnP BIOS.
+ *
+ * (c) 2001 Gerd Knorr <kraxel@bytesex.org>
+ */
+#include <linux/config.h>
+#include <linux/module.h>
+#include <linux/init.h>
+
+#include <asm/io.h>
+#include <asm/dma.h>
+#include <asm/uaccess.h>
+
+#include <linux/pnp_bios.h>
+
+static int request_stuff(char *pnp, struct pci_dev *dev)
+{
+	struct resource *res;
+	int i,count = 0;
+
+	printk(KERN_INFO "%s: request ports [%s]:",dev->name,pnp);
+	for (i = 0; i < DEVICE_COUNT_RESOURCE &&
+		    (dev->resource[i].start || dev->resource[i].end); i++) {
+		if (dev->resource[i].start > 0xffff ||
+		    dev->resource[i].end   > 0xffff) {
+			/*
+			 * these are memory ressources -- ignore them.
+			 * The PnP BIOS reports the main memory layout
+			 * this way.
+			 */
+			continue;
+		}
+		if (dev->resource[i].end < 0x100) {
+			/*
+			 * below 0x100 is only standard PC hardware
+			 * (pics, kbd, timer, dma, ...)
+			 *
+			 * We should not get ressource conflicts there,
+			 * and the kernel reserves these anyway
+			 * (see arch/i386/kernel/setup.c).
+			 */
+			continue;
+		}
+		/*
+		 * anything else we'll reserve to avoid these ranges are
+		 * assigned to someone (CardBus bridges for example) and
+		 * thus are triggering resource conflicts.
+		 */
+		res = request_region(dev->resource[i].start,
+			dev->resource[i].end - dev->resource[i].start,
+			dev->name);
+		printk(" 0x%lx-0x%lx",
+			dev->resource[i].start, dev->resource[i].end);
+		count++;
+	} 
+	printk("\n");
+	if (i == DEVICE_COUNT_RESOURCE)
+		printk("%s: warning: >= %d resources, overflow?\n",
+			dev->name,DEVICE_COUNT_RESOURCE);
+	return count;
+}
+
+int pnpbios_request_mboard(void)
+{
+	struct pci_dev *dev = NULL;
+	int count = 0;
+
+	while ((dev=pnpbios_find_device("PNP0c01",dev)))
+		count += request_stuff("PNP0c01",dev);
+	while ((dev=pnpbios_find_device("PNP0c02",dev)))
+		count += request_stuff("PNP0c02",dev);
+	return count;
+}

Re: [PATCH,RFC] make ide-scsi more selective

[Alan Cox]

> I've been rather annoyed by a dual problem in the ide-scsi setup:
> during initialisation, ide-scsi will claim ALL currently unassigned
> IDE devices. This is a problem in modular setups, since there's
> no guarantee that currently unassigned devices actually are intended
> for ide-scsi.

The real problem is that the drivers are claiming resources on load not
on open. Why shouldnt I be able to load ide-cd and ide-scsi and open either
/dev/hda or /dev/sr0 but not both together ?

Alan

Re: [PATCH,RFC] make ide-scsi more selective

[Nicholas Knight]

On Wednesday 22 August 2001 02:17 pm, Alan Cox wrote:
> > I've been rather annoyed by a dual problem in the ide-scsi setup:
> > during initialisation, ide-scsi will claim ALL currently unassigned
> > IDE devices. This is a problem in modular setups, since there's
> > no guarantee that currently unassigned devices actually are intended
> > for ide-scsi.
>
> The real problem is that the drivers are claiming resources on load not
> on open. Why shouldnt I be able to load ide-cd and ide-scsi and open
> either /dev/hda or /dev/sr0 but not both together ?

Here's an end-user perspective for you... I just spent 2 days trying to 
figure out how to use my CD-RW drive to read when using ide-scsi, before 
I finnaly realized that I had to do it by disabling ATAPI CD support and 
enabling SCSI CD support..
This is a severe inconvienience to the end-user who doesn't know what the 
problem is, esspecialy since I only found the answer to the problem in a 
couple places, I had to go through a ton of google search results to find 
anything.

Re: [PATCH,RFC] make ide-scsi more selective

[Ion Badulescu]

On Wed, 22 Aug 2001, Nicholas Knight wrote:

> Here's an end-user perspective for you... I just spent 2 days trying to 
> figure out how to use my CD-RW drive to read when using ide-scsi, before 
> I finnaly realized that I had to do it by disabling ATAPI CD support and 
> enabling SCSI CD support..

Just doing hdX=scsi would have been enough, however. Except it doesn't 
work (currently) if ide-scsi is a module.

I agree with Alan that the problem is the grab-on-load strategy that
ide-scsi (and ide-cd for that matter) uses. I am willing to look into 
changing that to grab-on-open but I'm not sure if the change is an 
appropriate one for a stable series kernel -- it looks pretty non-trivial.

Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.

Re: [PATCH,RFC] make ide-scsi more selective

[Nicholas Knight]

On Wednesday 22 August 2001 03:00 pm, Ion Badulescu wrote:
> On Wed, 22 Aug 2001, Nicholas Knight wrote:
> > Here's an end-user perspective for you... I just spent 2 days trying
> > to figure out how to use my CD-RW drive to read when using ide-scsi,
> > before I finnaly realized that I had to do it by disabling ATAPI CD
> > support and enabling SCSI CD support..
>
> Just doing hdX=scsi would have been enough, however. Except it doesn't
> work (currently) if ide-scsi is a module.

Could you elaborate on this? I almost never use modules for my primary 
desktop system, SCSI emulation support and SCSI generic driver were both 
compiled in, and I had "hdc=ide-scsi" and later also tried "hdc=scsi" and 

I was unable to read from it with any device, /dev/sr0 /dev/sda /dev/scd0 
were all dead-ends, but I was able to WRITE just fine... I just don't 
want to reboot every time I want to write to the drive, nor reboot when I 
want to READ from it.

Disabling ATAPI CD-ROM support, and enabling SCSI CD-ROM (along with SCSI 
emulation support and SCSI generic support) worked, and now I just access 
both my CD-RW drive and my DVD-ROM drive through /dev/sr0 and /dev/sr1.

My primary concern here is other users who haven't figured this out, I 
know at least one ATAPI/IDE CD-R(W) in Linux HOWTO tells the user that 
they'll have to use two seperate kernel images, one to allow reading from 
their drive and the other for writing, infact that was my original method.

> I agree with Alan that the problem is the grab-on-load strategy that
> ide-scsi (and ide-cd for that matter) uses. I am willing to look into
> changing that to grab-on-open but I'm not sure if the change is an
> appropriate one for a stable series kernel -- it looks pretty
> non-trivial.
>
> Ion

Re: [PATCH,RFC] make ide-scsi more selective

[mhobgood]

Well,
	I just double checked my system.  Sure enough I'm using ide-scsi
along with hdd="ide-scsi" with lilo.  It grabs hdd, which is my HP-7200
CD-RW and leaves hdc, which is my DVD alone.  I've no trouble at all
in reading or writing to hdd; no different kernel images, nothing.  I
use modules and have them normally loaded at boot time since I routinely
use both drives to read from.  My understanding was that I only needed
the hdd="ide-scsi" in lilo to prevent ATAPI from grabbing all the drives
it found.  BTW, everything pertaining to scsi is a module on this system.

Cordially,
Mike Hobgood


On Wed, Aug 22, 2001 at 03:39:12PM -0700, Nicholas Knight wrote:
> On Wednesday 22 August 2001 03:00 pm, Ion Badulescu wrote:
> > On Wed, 22 Aug 2001, Nicholas Knight wrote:
> > > Here's an end-user perspective for you... I just spent 2 days trying
> > > to figure out how to use my CD-RW drive to read when using ide-scsi,
> > > before I finnaly realized that I had to do it by disabling ATAPI CD
> > > support and enabling SCSI CD support..
> >
> > Just doing hdX=scsi would have been enough, however. Except it doesn't
> > work (currently) if ide-scsi is a module.
> 
> Could you elaborate on this? I almost never use modules for my primary 
> desktop system, SCSI emulation support and SCSI generic driver were both 
> compiled in, and I had "hdc=ide-scsi" and later also tried "hdc=scsi" and 
> 
> I was unable to read from it with any device, /dev/sr0 /dev/sda /dev/scd0 
> were all dead-ends, but I was able to WRITE just fine... I just don't 
> want to reboot every time I want to write to the drive, nor reboot when I 
> want to READ from it.
> 
> Disabling ATAPI CD-ROM support, and enabling SCSI CD-ROM (along with SCSI 
> emulation support and SCSI generic support) worked, and now I just access 
> both my CD-RW drive and my DVD-ROM drive through /dev/sr0 and /dev/sr1.
> 
> My primary concern here is other users who haven't figured this out, I 
> know at least one ATAPI/IDE CD-R(W) in Linux HOWTO tells the user that 
> they'll have to use two seperate kernel images, one to allow reading from 
> their drive and the other for writing, infact that was my original method.
> 
> > I agree with Alan that the problem is the grab-on-load strategy that
> > ide-scsi (and ide-cd for that matter) uses. I am willing to look into
> > changing that to grab-on-open but I'm not sure if the change is an
> > appropriate one for a stable series kernel -- it looks pretty
> > non-trivial.
> >
> > Ion
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [PATCH,RFC] make ide-scsi more selective

[Ion Badulescu]

On Wed, 22 Aug 2001, Nicholas Knight wrote:

> Could you elaborate on this? I almost never use modules for my primary 
> desktop system, SCSI emulation support and SCSI generic driver were both 
> compiled in, and I had "hdc=ide-scsi" and later also tried "hdc=scsi" and 

Well, hdc=ide-scsi is for 2.2 and hdc=scsi is for 2.4. Yup, yet another of 
those gratuitious incompatibilities.

> I was unable to read from it with any device, /dev/sr0 /dev/sda /dev/scd0 
> were all dead-ends, but I was able to WRITE just fine... I just don't 
> want to reboot every time I want to write to the drive, nor reboot when I 
> want to READ from it.

I'm not sure why this is happening for you, my CDR drive works for both 
reading and writing using the ide-scsi driver. But it's a known fact that 
ide-scsi is not perfect, so that could explain it.

> Disabling ATAPI CD-ROM support, and enabling SCSI CD-ROM (along with SCSI 
> emulation support and SCSI generic support) worked, and now I just access 
> both my CD-RW drive and my DVD-ROM drive through /dev/sr0 and /dev/sr1.

So now you're saying it *does* work with ide-scsi? I'm utterly confused...

> My primary concern here is other users who haven't figured this out, I 
> know at least one ATAPI/IDE CD-R(W) in Linux HOWTO tells the user that 
> they'll have to use two seperate kernel images, one to allow reading from 
> their drive and the other for writing, infact that was my original method.

Nope. Ide-scsi should be fine for both reading and writing.

Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.

Re: [PATCH,RFC] make ide-scsi more selective

[Nicholas Knight]

On Wednesday 22 August 2001 08:54 pm, Ion Badulescu wrote:
> On Wed, 22 Aug 2001, Nicholas Knight wrote:
> > Could you elaborate on this? I almost never use modules for my
> > primary desktop system, SCSI emulation support and SCSI generic
> > driver were both compiled in, and I had "hdc=ide-scsi" and later also
> > tried "hdc=scsi" and
>
> Well, hdc=ide-scsi is for 2.2 and hdc=scsi is for 2.4. Yup, yet another
> of those gratuitious incompatibilities.

tried both, neither worked, even with SCSI CD-ROM support enabled

>
> > I was unable to read from it with any device, /dev/sr0 /dev/sda
> > /dev/scd0 were all dead-ends, but I was able to WRITE just fine... I
> > just don't want to reboot every time I want to write to the drive,
> > nor reboot when I want to READ from it.
>
> I'm not sure why this is happening for you, my CDR drive works for both
> reading and writing using the ide-scsi driver. But it's a known fact
> that ide-scsi is not perfect, so that could explain it.

I've gotten the impression that ide-scsi isn't perfect :)

>
> > Disabling ATAPI CD-ROM support, and enabling SCSI CD-ROM (along with
> > SCSI emulation support and SCSI generic support) worked, and now I
> > just access both my CD-RW drive and my DVD-ROM drive through /dev/sr0
> > and /dev/sr1.
>
> So now you're saying it *does* work with ide-scsi? I'm utterly
> confused...

I'll be more specific:

I've used this in 2.4.8 and 2.4.8-ac9 to read from my IDE/ATAPI CD-RW 
drive while also using it to write to
I now have IDE/ATAPI CD-ROM support DISABLED in the ATA/* menu
SCSI emulation is ENABLED in the ATA/* menu
SCSI CD-ROM support is ENABLED in the SCSI menu
SCSI Generic is ENABLED in the SCSI menu
I do not have any parameters (such as hdX=scsi or ide-scsi) being passed 
to the kernel except those for my SB16 card, which doesn't have any 
devices hooked to its IDE port.

this appears at kernel boot:
SCSI subsystem driver Revision: 1.00
scsi0 : SCSI host adapter emulation for IDE ATAPI devices
  Vendor: PLEXTOR   Model: CD-R   PX-W1610A  Rev: 1.00
  Type:   CD-ROM                             ANSI SCSI revision: 02
  Vendor: PIONEER   Model: DVD-ROM DVD-113   Rev: 1.08
  Type:   CD-ROM                             ANSI SCSI revision: 02
Attached scsi CD-ROM sr0 at scsi0, channel 0, id 0, lun 0
Attached scsi CD-ROM sr1 at scsi0, channel 0, id 1, lun 0
sr0: scsi3-mmc drive: 40x/40x writer cd/rw xa/form2 cdda tray
Uniform CD-ROM driver Revision: 3.12
sr1: scsi3-mmc drive: 0x/0x cd/rw xa/form2 cdda tray

using 0,0,0 works for cdrecord, and using /dev/sr0 to mount discs in my 
CD-RW drive and /dev/sr1
using /dev/hdc and /dev/hdd does NOT work now

Without this setup, I've been unable to read from my CD-RW drive while 
it's also setup to write to with SCSI generic.

>
> > My primary concern here is other users who haven't figured this out,
> > I know at least one ATAPI/IDE CD-R(W) in Linux HOWTO tells the user
> > that they'll have to use two seperate kernel images, one to allow
> > reading from their drive and the other for writing, infact that was
> > my original method.
>
> Nope. Ide-scsi should be fine for both reading and writing.

didn't for me

Re: [PATCH,RFC] make ide-scsi more selective

[Willem Riede]

Alan Cox wrote:
> 
> > I've been rather annoyed by a dual problem in the ide-scsi setup:
> > during initialisation, ide-scsi will claim ALL currently unassigned
> > IDE devices. This is a problem in modular setups, since there's
> > no guarantee that currently unassigned devices actually are intended
> > for ide-scsi.
> 
> The real problem is that the drivers are claiming resources on load not
> on open. Why shouldnt I be able to load ide-cd and ide-scsi and open either
> /dev/hda or /dev/sr0 but not both together ?
> 
Ion Badulescu wrote:
> 
> On Wed, 22 Aug 2001, Nicholas Knight wrote:
> 
> > Here's an end-user perspective for you... I just spent 2 days trying to
> > figure out how to use my CD-RW drive to read when using ide-scsi, before
> > I finnaly realized that I had to do it by disabling ATAPI CD support and
> > enabling SCSI CD support..
> 
> Just doing hdX=scsi would have been enough, however. Except it doesn't
> work (currently) if ide-scsi is a module.
> 
Well, at least on my system (RH 7.1) in rc.sysinit there's a fragment
that checks if ide-scsi is asked for and if so loads ide-cd first, so
it can grab CD drives that are not targeted for scsi emulation.

Personally, I've added a modprobe of ide-tape there so that ide-scsi 
can't grab any tape drives it is not supposed to either.

> I agree with Alan that the problem is the grab-on-load strategy that
> ide-scsi (and ide-cd for that matter) uses. I am willing to look into
> changing that to grab-on-open but I'm not sure if the change is an
> appropriate one for a stable series kernel -- it looks pretty non-trivial.
> 
Right. And it is not limited to ide-scsi but impacts drivers that
connect to ide-scsi such as the osst tape driver I maintain which
would have to allow a device to be ht0 and osst0 at the user's choice...

Regards, Willem Riede.

Re: [PATCH,RFC] make ide-scsi more selective

[Ion Badulescu]

On Wed, 22 Aug 2001, Alan Cox wrote:

> The real problem is that the drivers are claiming resources on load not
> on open. Why shouldnt I be able to load ide-cd and ide-scsi and open either
> /dev/hda or /dev/sr0 but not both together ?

Mostly because sr_open() does not make any calls into the host driver, 
which makes it impossible to claim any resources on open.

I've concocted another patch, which includes my previous one and
implements Mikael's idea somewhat more consistently. It adds another
option, "noscsi", so that by saying hdX=noscsi on the kernel command line
the user can prevent the ide-scsi driver from ever claiming that drive.

So:
- by default it's first come, first served
- hdX=scsi means only the ide-scsi driver can claim hdX
- hdX=noscsi means the ide-scsi driver must not claim hdX ever

Sounds good? If so, please apply, it makes many CDR users' lives easier.

Thanks,
Ion

-- 
  It is better to keep your mouth shut and be thought a fool,
            than to open it and remove all doubt.
-----------------------------------
diff -ur ../linux-2.4-ac/drivers/ide/ide-cd.c linux-2.4.8-ac7/drivers/ide/ide-cd.c
--- ../linux-2.4-ac/drivers/ide/ide-cd.c	Sat Aug 18 06:03:21 2001
+++ linux-2.4.8-ac7/drivers/ide/ide-cd.c	Sat Aug 25 02:30:03 2001
@@ -3026,7 +3026,7 @@
 				continue;
 			}
 		}
-		if (drive->scsi) {
+		if (drive->scsi == 1) {
 			printk("ide-cd: passing drive %s to ide-scsi emulation.\n", drive->name);
 			continue;
 		}
diff -ur ../linux-2.4-ac/drivers/ide/ide-floppy.c linux-2.4.8-ac7/drivers/ide/ide-floppy.c
--- ../linux-2.4-ac/drivers/ide/ide-floppy.c	Sat Aug 18 06:03:21 2001
+++ linux-2.4.8-ac7/drivers/ide/ide-floppy.c	Sat Aug 25 02:29:51 2001
@@ -2018,7 +2018,7 @@
 			printk (KERN_ERR "ide-floppy: %s: not supported by this version of ide-floppy\n", drive->name);
 			continue;
 		}
-		if (drive->scsi) {
+		if (drive->scsi == 1) {
 			printk("ide-floppy: passing drive %s to ide-scsi emulation.\n", drive->name);
 			continue;
 		}
diff -ur ../linux-2.4-ac/drivers/ide/ide-tape.c linux-2.4.8-ac7/drivers/ide/ide-tape.c
--- ../linux-2.4-ac/drivers/ide/ide-tape.c	Sat Aug 18 06:03:21 2001
+++ linux-2.4.8-ac7/drivers/ide/ide-tape.c	Sat Aug 25 02:29:39 2001
@@ -6235,7 +6235,7 @@
 			printk (KERN_ERR "ide-tape: %s: not supported by this version of ide-tape\n", drive->name);
 			continue;
 		}
-		if (drive->scsi) {
+		if (drive->scsi == 1) {
 			if (strstr(drive->id->model, "OnStream DI-")) {
 				printk("ide-tape: ide-scsi emulation is not supported for %s.\n", drive->id->model);
 			} else {
diff -ur ../linux-2.4-ac/drivers/ide/ide.c linux-2.4.8-ac7/drivers/ide/ide.c
--- ../linux-2.4-ac/drivers/ide/ide.c	Sat Aug 18 06:03:21 2001
+++ linux-2.4.8-ac7/drivers/ide/ide.c	Sat Aug 25 02:29:24 2001
@@ -3045,7 +3045,7 @@
 		const char *hd_words[] = {"none", "noprobe", "nowerr", "cdrom",
 				"serialize", "autotune", "noautotune",
 				"slow", "swapdata", "bswap", "flash",
-				"remap", "noremap", "scsi", NULL};
+				"remap", "noremap", "scsi", "noscsi", NULL};
 		unit = s[2] - 'a';
 		hw   = unit / MAX_DRIVES;
 		unit = unit % MAX_DRIVES;
@@ -3109,13 +3109,15 @@
 				drive->remap_0_to_1 = 2;
 				goto done;
 			case -14: /* "scsi" */
-#if defined(CONFIG_BLK_DEV_IDESCSI) && defined(CONFIG_SCSI)
+#if defined(CONFIG_BLK_DEV_IDESCSI) || defined(CONFIG_BLK_DEV_IDESCSI_MODULE)
 				drive->scsi = 1;
-				goto done;
 #else
-				drive->scsi = 0;
-				goto bad_option;
-#endif /* defined(CONFIG_BLK_DEV_IDESCSI) && defined(CONFIG_SCSI) */
+				printk(" -- ide-scsi support unavailable, ignoring.\n");
+#endif /* defined(CONFIG_BLK_DEV_IDESCSI) || defined(CONFIG_BLK_DEV_IDESCSI_MODULE) */
+				goto done;
+			case -15: /* "noscsi" */
+				drive->scsi = 2;
+				goto done;
 			case 3: /* cyl,head,sect */
 				drive->media	= ide_disk;
 				drive->cyl	= drive->bios_cyl  = vals[0];
--- ../linux-2.4-ac/drivers/scsi/ide-scsi.c	Fri Feb  9 14:30:23 2001
+++ linux-2.4.8-ac7/drivers/scsi/ide-scsi.c	Sat Aug 25 02:29:04 2001
@@ -580,7 +580,10 @@
 	for (i = 0; media[i] != 255; i++) {
 		failed = 0;
 		while ((drive = ide_scan_devices (media[i], idescsi_driver.name, NULL, failed++)) != NULL) {
-
+			if (drive->scsi == 2) {
+				printk ("ide-scsi: skipping reserved drive %s.\n", drive->name);
+				continue;
+			}
 			if ((scsi = (idescsi_scsi_t *) kmalloc (sizeof (idescsi_scsi_t), GFP_KERNEL)) == NULL) {
 				printk (KERN_ERR "ide-scsi: %s: Can't allocate a scsi structure\n", drive->name);
 				continue;

Re: i810 audio doesn't work with 2.4.9

[Stefan Westerfeld]

   Hi!

On Thu, Aug 23, 2001 at 08:03:00PM +0000, Doug Ledford wrote:
 > So it seems that update of i810_audio.c between 2.4.6-ac1 and ac2 breaks it (at least for me).
> > But I think it still eating too much time (about 2-3% on PIII 700) when I'm not doing anything 
> > with sound but no more up to 90% as with unmodified version from 2.4.9 (maybe it's a problem
> > of artsd , I don't know)
> 
> 
> Yes, it's a problem of artsd.  Someone decided (presumably to avoid the 
> occasional pop/click from the startup or shutdown of the sound device) 
> to make artsd transmit silence over the sound card when no sounds 
> currently need played.  From my perspective, I will *NEVER* use artsd as 
> long as it does this.  [... more rant ...]

More or less recent versions of artsd either autosuspend (close the sound
device) either after hard-coded 60 seconds of inactivity, or have a command
line option

-s <seconds>        auto-suspend time in seconds

which should adress this issue.

> Furthermore, I find 
> their use of the sound API to be suboptimal, especially when they are 
> going to transmit silence all the time. [...]

Well, artsd does non-blocking sound I/O combined with select() which tells it
when to wake up and write (read) something. It usually works quite nice if the
driver is implemented correctly, that is, if it wakes up artsd if there is a
suitable amount of data to read (write) - usually a fragment is a good choice
for the driver. The use of the API has the advantage that it

 - doesn't require threads (synchronization, context switches)
 - works well with very low latencies
 - never blocks the server (i.e. the server can always accept requests from
   the net)

Common problems with some drivers appear to be:

 * GETOSPACE/GETISPACE lies about the size that can be read/written (non-fatal,
   as long as there is at least some truth in there)

 * select() does wake up too early (i.e. if either nothing at all or just a
   small amount can be read/written) - thats fatal and leads to the 90% CPU
   scenarios

(I am not subscribed to linux-kernel, so in doubt CC me).

   Cu... Stefan
-- 
  -* Stefan Westerfeld, stefan@space.twc.de (PGP!), Hamburg/Germany
     KDE Developer, project infos at http://space.twc.de/~stefan/kde *-         

Re: i810 audio doesn't work with 2.4.9

[Doug Ledford]

Stefan Westerfeld wrote:
>    Hi!
> 
> On Thu, Aug 23, 2001 at 08:03:00PM +0000, Doug Ledford wrote:
>>Yes, it's a problem of artsd.  Someone decided (presumably to avoid
>>the occasional pop/click from the startup or shutdown of the sound
>>device) to make artsd transmit silence over the sound card when no
>>sounds currently need played.  From my perspective, I will *NEVER* use 
>>artsd as long as it does this.  [... more rant ...]
>>
> 
> More or less recent versions of artsd either autosuspend (close the
> sound device) either after hard-coded 60 seconds of inactivity, or
> have a command line option
> 
> -s <seconds>        auto-suspend time in seconds
> 
> which should adress this issue.

That's a vast improvement.  Glad to hear it.

>>Furthermore, I find 
>>their use of the sound API to be suboptimal, especially when they are 
>>going to transmit silence all the time. [...]
>>
> 
> Well, artsd does non-blocking sound I/O combined with select() which
> tells it when to wake up and write (read) something. It usually works
> quite nice if the driver is implemented correctly, that is, if it
> wakes up artsd if there is a suitable amount of data to read (write) - 
> usually a fragment is a good choice
> for the driver. The use of the API has the advantage that it
> 
>  - doesn't require threads (synchronization, context switches)
>  - works well with very low latencies
>  - never blocks the server (i.e. the server can always accept requests 
>    from the net)

This is all fine.

> Common problems with some drivers appear to be:
> 
>  * GETOSPACE/GETISPACE lies about the size that can be read/written
>    (non-fatal, as long as there is at least some truth in there)


This is my complaint.  It's redundant and useless to do non blocking I/O 
*and* also waste time on GETOSPACE/GETISPACE syscalls.  Unless you are 
wanting to flirt with the very edge of having the sound buffer emptied 
so that you are only microseconds ahead of the dma engine and any new 
sound can be instantaneously transmitted to your speakers, just write to 
the damn file until you get a write that returns with a short write 
value (aka, not all data was written to the device), then sleep and wait 
for select to wake you up.  GETOSPACE/GETISPACE are there so you know 
how much data you can write, which keeps you from ever blocking.  Non 
blocking I/O is so you don't have to worry about how much space is 
available to write to.  Yes, you can use them together if you want, but 
it makes almost no sense and is a waste of CPU cycles.

>  * select() does wake up too early (i.e. if either nothing at all or
>    just a small amount can be read/written) - thats fatal and leads to 
>    the 90% CPU scenarios

The difference between GETOSPACE and write can also cause the same thing 
(something I just fixed in the i810 driver I'm working on here). 
However, if the sound driver is waking you up from a select call when it 
isn't suppossed to, that's a driver bug and not something that you 
should have to work around.  The workaround just perpetuates the broken 
drivers.

> (I am not subscribed to linux-kernel, so in doubt CC me).
> 
>    Cu... Stefan
> 



-- 

  Doug Ledford <dledford@redhat.com>  http://people.redhat.com/dledford
       Please check my web site for aic7xxx updates/answers before
                       e-mailing me about problems

Re: Apollo Pro266 system freeze followup

[Barry K. Nathan]

Right now I'm running Linux 2.4.9-ac4, compiled for SMP, with the
following loop (in bash):

while : ; do make dep ; make clean ; make -j3 modules bzImage ; done

It's been going for about 7 and a half hours without a lockup, so far.
Note that I am not using the on-board IDE -- but I did not need the
on-board IDE for my lockups either.

If 2.4.9-ac4 still produces lockups for people, it might be interesting
to try 2.2.19 or 2.2.20pre9, compiled for SMP, and see if the lockups
still happen then.

-Barry K. Nathan <barryn@pobox.com>

Re: OOM /proc logging

[Samium Gromoff]

    5 minutes later my original message one folk wrote on IRC a phrase which
  took me down for some minutes due to -ETOOMUCHLAUGHTER, and i was unable to
  resist an urgent need to post it here cause it was so on-topic:

<wli> The penguin hath spoken, now we must clean the guano from the ice.

cheers, Samium Gromoff

Re: kernel 2.4.14 compiling fail for loop device

[Barry K. Nathan]

> When I did, and used a looped an iso image, eventually my
> computer froze up.  Using the actual cd, it did not.  So my
> personal answer would be no.

Hmmm... my *root* filesystem (with /usr, /home, etc. all on it) on one of
my computers is loop mounted, and I've not had such a freeze with 2.4.14
and the two lines removed... Just another data point.

-Barry K. Nathan <barryn@pobox.com>

Re: kernel 2.4.14 compiling fail for loop device

[Todd M. Roy]

Would you believe that I recompiled 2.4.14 this morning
and can't seem to reproduce it now?

-- todd --

"Barry K. Nathan" wrote:
> 
> > When I did, and used a looped an iso image, eventually my
> > computer froze up.  Using the actual cd, it did not.  So my
> > personal answer would be no.
> 
> Hmmm... my *root* filesystem (with /usr, /home, etc. all on it) on one of
> my computers is loop mounted, and I've not had such a freeze with 2.4.14
> and the two lines removed... Just another data point.
> 
> -Barry K. Nathan <barryn@pobox.com>

-- 
---------------------------------------------------------------
   .~. Todd Roy, Senior Database Administrator .~.
   /V\    Holstein Association, U.S.A. Inc.    /V\
  // \\          troy@holstein.com            // \\
 /(   )\        1-802-254-4551x4230          /(   )\
  ^^-^^                                       ^^-^^
"They that can give up essential liberty to obtain a little 
temporary safety deserve neither liberty nor safety."
		-- Benjamin Franklin, 1759
---------------------------------------------------------------
**********************************************************************
This footnote confirms that this email message has been swept by 
MIMEsweeper for the presence of computer viruses.
**********************************************************************

Re: kernel 2.4.14 compiling fail for loop device

[Mohammad A. Haque]

On Wednesday, November 7, 2001, at 10:12 AM, Barry K. Nathan wrote:

>> When I did, and used a looped an iso image, eventually my
>> computer froze up.  Using the actual cd, it did not.  So my
>> personal answer would be no.
>
> Hmmm... my *root* filesystem (with /usr, /home, etc. all on it) on one 
> of
> my computers is loop mounted, and I've not had such a freeze with 2.4.14
> and the two lines removed... Just another data point.

Hrm. I just did some stuff on a fs mounted via loopback and no problems.

Mike: it's something in particular you were doing that triggered it or 
something else completely?

--

=====================================================================
Mohammad A. Haque                              http://www.haque.net/
                                                mhaque@haque.net

   "Alcohol and calculus don't mix.             Developer/Project Lead
    Don't drink and derive." --Unknown          http://wm.themes.org/
                                                batmanppc@themes.org
=====================================================================

Re: kernel 2.4.14 compiling fail for loop device

[Mohammad A. Haque]

On Wednesday, November 7, 2001, at 10:38 AM, Mohammad A. Haque wrote:
> Hrm. I just did some stuff on a fs mounted via loopback and no problems.
>
> Mike: it's something in particular you were doing that triggered it or 
> something else completely?

Whoops,

That question should have been "is it..." and directed to Todd.

--

=====================================================================
Mohammad A. Haque                              http://www.haque.net/
                                                mhaque@haque.net

   "Alcohol and calculus don't mix.             Developer/Project Lead
    Don't drink and derive." --Unknown          http://wm.themes.org/
                                                batmanppc@themes.org
=====================================================================

Re: ramfs leak

[W Christopher Martin]

Padraig Brady writes:
> When I remove files from a ramfs the space is not reclaimed?
> What am I doing wrong? Details below.

Nothing.  We've noticed the same thing.  It's a bug and was
first reported back in July, but no one has provided a fix yet.
I've had a brief look at the source code, but nothing obvious
pops out at me.

As you mention, this problem is trivially reproducable by
creating and then deleting a file.  Doing that over and over
eventually leads to the ramfs becoming full.  Only a reboot
(or perhaps a umount/mount) makes it usable again.

Chris Martin
Catnap Consultants

Re: ramfs leak

[Tachino Nobuhiro]

Hello,

At Fri, 9 Nov 2001 15:40:43 -0500 (EST),
W Christopher Martin wrote:
> 
> Padraig Brady writes:
> > When I remove files from a ramfs the space is not reclaimed?
> > What am I doing wrong? Details below.
> 
> Nothing.  We've noticed the same thing.  It's a bug and was
> first reported back in July, but no one has provided a fix yet.
> I've had a brief look at the source code, but nothing obvious
> pops out at me.

I think you should use tmpfs instead of ramfs, but if you really want to use ramfs,
the patch below may fix the problem.

diff -Nur linux-2.4.13-ac7.org/fs/ramfs/inode.c linux-2.4.13-ac7/fs/ramfs/inode.c
--- linux-2.4.13-ac7.org/fs/ramfs/inode.c	Mon Nov 12 11:00:47 2001
+++ linux-2.4.13-ac7/fs/ramfs/inode.c	Mon Nov 12 11:26:40 2001
@@ -182,12 +182,9 @@
 {
 	struct ramfs_sb_info *rsb = RAMFS_SB(inode->i_sb);
 
-	if (! Page_Uptodate(page))
-		return;
-
 	lock_rsb(rsb);
-
-	ClearPageDirty(page);
+	if (Page_Uptodate(page))
+		ClearPageDirty(page);
 	
 	rsb->free_pages++;
 	inode->i_blocks -= IBLOCKS_PER_PAGE;

Re: ramfs leak

[Padraig Brady]

Tachino Nobuhiro wrote:

> Hello,
> 
> At Fri, 9 Nov 2001 15:40:43 -0500 (EST),
> W Christopher Martin wrote:
> 
>>Padraig Brady writes:
>>
>>>When I remove files from a ramfs the space is not reclaimed?
>>>What am I doing wrong? Details below.
>>>
>>Nothing.  We've noticed the same thing.  It's a bug and was
>>first reported back in July, but no one has provided a fix yet.
>>I've had a brief look at the source code, but nothing obvious
>>pops out at me.
>>
> 
> I think you should use tmpfs instead of ramfs, but if you really want to use ramfs,
> the patch below may fix the problem.
> 
> diff -Nur linux-2.4.13-ac7.org/fs/ramfs/inode.c linux-2.4.13-ac7/fs/ramfs/inode.c
> --- linux-2.4.13-ac7.org/fs/ramfs/inode.c	Mon Nov 12 11:00:47 2001
> +++ linux-2.4.13-ac7/fs/ramfs/inode.c	Mon Nov 12 11:26:40 2001
> @@ -182,12 +182,9 @@
>  {
>  	struct ramfs_sb_info *rsb = RAMFS_SB(inode->i_sb);
>  
> -	if (! Page_Uptodate(page))
> -		return;
> -
>  	lock_rsb(rsb);
> -
> -	ClearPageDirty(page);
> +	if (Page_Uptodate(page))
> +		ClearPageDirty(page);
>  	
>  	rsb->free_pages++;
>  	inode->i_blocks -= IBLOCKS_PER_PAGE;
> 


Cool, this fixes it,
and I was just getting to the bottom of it myself :-)
None of this accounting stuff is in 2.4.15-pre3, so Alan
can you apply this?

cheers,
Padraig.

Re: ramfs leak

[Alan Cox]

> and I was just getting to the bottom of it myself :-)
> None of this accounting stuff is in 2.4.15-pre3, so Alan
> can you apply this?

RAMfs is 2.4.16 stuff I think

Re: [parisc-linux] PIC assembly

[John David Anglin]

> This page contains links to various useful documents including the 32-bit
> runtime:
> 
> <http://h21007.www2.hp.com/dspp/tech/tech_TechTypeListingPage_IDX/1,1704,10403,00.html>.
> 
> The document itself is here:
> 
> <http://devresource.hp.com/STK/partner/rad_10_20.pdf>.

Maybe copies of these documents could be made accessible from the PA-RISC
LINUX Tech Documentation page as some of these are hard to find.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: [parisc-linux] PIC assembly

[Grant Grundler]

"John David Anglin" wrote:
> > This page contains links to various useful documents including the 32-bit
> > runtime:
> > 
> > <http://h21007.www2.hp.com/dspp/tech/tech_TechTypeListingPage_IDX/1,1704,10
>   403,00.html>.
> > 
> > The document itself is here:
> > 
> > <http://devresource.hp.com/STK/partner/rad_10_20.pdf>.
> 
> Maybe copies of these documents could be made accessible from the PA-RISC
> LINUX Tech Documentation page as some of these are hard to find.

I've been given the OK to link into h21007.www2.hp.com website and promised
the URLs for specific documents wouldn't change (too) suddenly.
I'll add those links in the next couple of days to p-l.org
"Tech Documentation" to those.

grant

Re: [parisc-linux] PIC assembly

[Grant Grundler]

Grant Grundler wrote:
> I've been given the OK to link into h21007.www2.hp.com website and promised
> the URLs for specific documents wouldn't change (too) suddenly.
> I'll add those links in the next couple of days to p-l.org
> "Tech Documentation" to those.

I've added links for PA 1.1 and PA 2.0 Instruction Sets among other things.
Please reload:
	http://www.parisc-linux.org/documentation/index.html

Please continue to post requests for links to docs here.
(I'm not promising any new docs at this point).

thanks,
grant

Re: Galileo 64240

[ellis]

>>Is there any support code around for the Galileo 64240? A
>>serial driver would be really nice ;)

>I started an MPSC/Uart driver for the 64240/64240A chips. Didn't
>get much of it working when we got fed up with all of the
>Galileo errata about which registers could or could not be read
>at which times and their confusion as to whether or not they
>were planning to ever fix the errata.

>We broke down and added a 162550 UART to the board.

I really wish that option was available. ;)

>The driver code was abandoned in the midst of early debugging
>stages and is in a horrific state. You're welcome to a copy if
>you really can't find something better.

It looks like we have something that works for our in-house
(non-linux) OS. I guess I'll use that code as an example and see
if I can get a real driver working.

--
http://www.spinics.net/linux/

Re: preemption and pccard ?

[Barry K. Nathan]

(Oops, I think I sent this via private mail last time, instead of to the
list.)

I've also seen problems with the preempt patch and PCMCIA/CardBus, on my
Dell Inspiron 5000e. The top CardBus slot doesn't work for me with the
preemption patch (in fact, if I have a card in there, sometimes the
machine freezes at the point in boot when it would normally detect the
card). It usually doesn't even see that I've put a card in there. I don't
remember trying the bottom slot instead of the top though.

I just never got a chance to report the problem, until now. This has
happened with a range of kernels (I think I first tried the preempt patch
back around 2.4.14pre and I last tried it with a late 2.4.17-pre or with
2.4.17-rc1.)

-Barry K. Nathan <barryn@pobox.com>

Re: preemption and pccard ?

[Erik Gustavsson]

I believe I also saw this when trying a preempt kernel somewhere in the
2.4.17pre series. Only one cardbus slot worked (I think it was the
bottom slot that didn't respond, but I could be mistaken). I was playing
around with various patches at the time, so I didn't pay much attention
to it. I can try to verify it if that would help. This was on a Dell
Latitude CPi D266XT.

/cyr
 
-- 
-----------------------------------------------------------------------
You really think you can buy me with promises of power and glory.
You really think... Okay, I'll do it. -- Rimmer

Re: Machines misreporting Bogomips

[Barry K. Nathan]

watermodem wrote:
> If they are Intel CPU's and the heatsink <-> CPU connection is poor
> (no heatsink compound, heatsink loose) or the fan is dead/dying or
> due to dust poor airflow this is reasonable.   Intel CPUs slow down
> when they get hot as as safety measure.

Note that this only applies to Pentium 4's. I believe Coppermine Pentium
III's will simply stop running (i.e., the computer freezes altogether
instead of slowing down) to prevent overheating. I'm not sure off the top
of my head what older Pentiums do, except that they certainly don't have
the slowdown trick that the Pentium 4 has.

Also, some BIOSes have an option called "CPU Speed at Boot" or something
like that, which has a Low and a High setting -- this serves the same
purpose as those old Turbo switches. (Note that I'm *not* talking about
the SpeedStep settings that newer laptops have in their BIOSes.)

-Barry K. Nathan <barryn@pobox.com>

Re: Filesystem Corruption (ext2) on Tyan S2462, 2xAMD1900MP, 2.4.17SMP

[Barry K. Nathan]

[snip]
>Mar 19 00:36:41 fems146 xfs: xfs startuP succeeded
>Mar 19 00:36:41 fems146 xfs: listening on port 7100 
>Mar 19 00:36:41 fems146 xfs: ieNoring font path element /usr/X11R6/lib/X11/fonts/100dpi:unscaled (unreadable) 
>Mar 19 00:36:41 fems146 anacron: anacron startup succeeded
>Mar 19 00:36:41 fems146 xfs: ignoring font path elemeNt /usr/X11R6/lib/X11/fonts/CID (unreadable) 
>Mar 19 00:36:41 fems146 xfs: ignoring font path elEment /usr/X11R6/lib/X11/fonts/local (unreadable) 
>Mar 19 00:36:41 fems146 xfs: ignoring font path element /usr/X11R6/lib/X11/fonts/japanese (unreadable) 
>Mar 19 00:36:41 fems146 atd: atd stapTup succeeded
>Mar 19 00:36:42 fems146 rc: Starting dont_blank:  succeeded
>Mar 19 00:36:42 fems14\x16 hdaset: 
>Mar 19 00:36:42 fems146 hdaset: /dev/hda:
>Mar 19 00:36:42 fems146 hdaset:  setting usIng_dma to 1 (on)
>Mar 19 00:36:42 fems146 hdaset:  using_dma    =  1 (on)
>Mar 19 00:36:42 fems14\x16 rc: Starting hdaset:  succeeded
[snip]

Looking at it on a byte-by-byte level, it looks like (at least) these
types of bit flips are happening:

     --1-----
     --0-----
     ------0-
MSB->76543210<-LSB

That is, it looks like sometimes bit 5 is being flipped on or off, or bit
1 is being flipped off. (There could be others that I just haven't seen in
those logs yet.) I'm suspecting bad hardware (in case that wasn't
obvious), but I don't know exactly what component is defective. (By the
way, the BIOS has ECC error correction enabled, right??)

Also, do the weird capitalization changes in the logs happen on screen
too, or only in the logfile?

-Barry K. Nathan <barryn@pobox.com>

Re: New IBM IDE drive recognized as 40 GB but is 80 GB

[Barry K. Nathan]

> We have bought a new IDE hard drive:
> 
> hdb: IC35L040AVVA07-0, ATA DISK drive
             ^^ 40GB (well, according to IBM, "41.17GB")

Look here if you don't believe me:
http://www.storage.ibm.com/hdd/support/d120gxp/d120gxpmod.htm
 
> This is wrongly recognized as 40GB (should be 80GB) drive:
[snip]
> [root@apu /root]# cat /proc/ide/hdb/capacity
> 80418240           <--------------------------------- this is correct
> [root@apu /root]#
> 
> shows the right capacity.

80418240 512-byte sectors is about 40GB.

The kernel is reporting the correct size... 40GB.

-Barry K. Nathan <barryn@pobox.com>

Re: New IBM IDE drive recognized as 40 GB but is 80 GB

[Martin Rode]

Thanks all for your help!

The drive had a wrong labeled bag and the wrong sticker on it
(indicating 80GB instead of 40GB)!! So Linux IDE works fine...

;Martin


-- 
Dipl.-Kfm. Martin Rode
martin.rode@zeroscale.com

Zeroscale GmbH & Co. KG
Frankfurter Allee 73d
10247 Berlin

http://www.zeroscale.com/
http://www.programmfabrik.de/

Fon +49-(0)30-4281-8001
Fax +49-(0)30-4281-8008
Funk +49-(0)163-5321400

Re: Mips16 toolchain?

[Hartvig Ekner]

Dominic Sweetman writes:
> 
> We are also developing a compiler from the same source tree, but
> configured for Linux.  That was the compiler we'll be looking for
> beta-testers for in the next couple of months.
> 
> If you want to be able to build MIPS16 applications and then run them
> on Linux, this is more challenging.  You have to build everything
> static: then it works mostly, and some people at MIPS have built and
> run some programs.

I have built glibc in a static and non-PIC version to allow linking against
M16 user apps (non-PIC required because current Linux compilers cannot 
generate M16 PIC code). It worked fine using the Algo 5.0 Beta for Linux. 
I successfully built a few applications (ones which only required libc).

It won't be really useful until somebody builds a complete library set
which is static and non-PIC, or PIC support gets included in the M16
code generator.

/Hartvig

Re: [patch] hpt374 support

[Barry K. Nathan]

Andrew Morton wrote:
> It has a problem with my 80 gigabyte Seagates - in UDMA133
> mode, writes are inexplicably slow.  I manually set UDMA100
> and it flies.

Hmmm... do 80GB Seagates even support UDMA133? I could be mistaken, but
as an owner of one, I'm pretty sure the answer is "no". In fact, I think
Maxtor's the only company making UDMA133 drives at this point in time.

-Barry K. Nathan <barryn@pobox.com>

Re: [patch] hpt374 support

[tomas szepe]

> > It has a problem with my 80 gigabyte Seagates - in UDMA133
> > mode, writes are inexplicably slow.  I manually set UDMA100
> > and it flies.
> Hmmm... do 80GB Seagates even support UDMA133? I could be mistaken, but
> as an owner of one, I'm pretty sure the answer is "no". In fact, I think
> Maxtor's the only company making UDMA133 drives at this point in time.

Right.

hda: ST380021A, ATA DISK drive (i.e., Seagate Barracuda IV)
hda: 156301488 sectors (80026 MB) w/2048KiB Cache, CHS=9729/255/63, UDMA(100)

 Model=ST380021A, FwRev=3.75, SerialNo=3HV16GXT
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
 BuffType=unknown, BuffSize=2048kB, MaxMultSect=16, MultSect=off
 CurCHS=16383/16/63, CurSects=-66060037, LBA=yes, LBAsects=156301488
 IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes: pio0 pio1 pio2 pio3 pio4
 DMA modes: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5
 AdvancedPM=no
 Drive Supports : Reserved : ATA-1 ATA-2 ATA-3 ATA-4 ATA-5

ATA-5 would be ATA100 AFAIK?


T.

Re: [patch] hpt374 support

[Andrew Morton]

"Barry K. Nathan" wrote:
> 
> Andrew Morton wrote:
> > It has a problem with my 80 gigabyte Seagates - in UDMA133
> > mode, writes are inexplicably slow.  I manually set UDMA100
> > and it flies.
> 
> Hmmm... do 80GB Seagates even support UDMA133? I could be mistaken, but
> as an owner of one, I'm pretty sure the answer is "no". In fact, I think
> Maxtor's the only company making UDMA133 drives at this point in time.
> 

Doh.  I have enough disks in this thing to feed a small nation, and
I am easily confused.

/dev/hde:

 Model=Maxtor 4D080K4, FwRev=DAK019K0, SerialNo=D4H0GTPE
 Config={ Fixed }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=57
 BuffType=DualPortCache, BuffSize=2048kB, MaxMultSect=16, MultSect=16
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=160086528
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes: pio0 pio1 pio2 pio3 pio4 
 DMA modes: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 *udma5 udma6 
 Drive Supports : Reserved : ATA-1 ATA-2 ATA-3 ATA-4 ATA-5 ATA-6 
 Kernel Drive Geometry LogicalCHS=158816/255/63 PhysicalCHS=158816/255/63

-

Re: Linux 2.4.18 floppy driver EATS floppies

[Barry K. Nathan]

> I am discovering that any floppy disks that I try to use under Linux
> don't last very long. This seems to be true with both my UP and SMP 
> machines, neither of which has ever used its floppy drive enough for
> me to believe that the hardware is reaching the end of its life.

Make sure the boot-time floppy seek option is enabled in your BIOS. While
it shouldn't be necessary nowadays in theory, I've seen some
motherboard/(1.44MB 3.5") floppy drive combinations that require it to be
enabled for the floppy drive to work with any level of consistency.

Also, if a particular disk is acting really strange, try ejecting it,
reinserting it, and trying again.

Both of these pieces of advice apply to Windows as well as Linux, FWIW.

-Barry K. Nathan <barryn@pobox.com>

Re: cross-compiler for MIPS_RedHat7.1_Release-01.00 on Atlas/4Kc using RH7.3-i386 host

[Hartvig Ekner]

Hi David,

David Christensen writes:
> 
> linux-mips@oss.sgi.com:
> 
> I have downloaded ftp://ftp.mips.com/pub/linux/mips/installation/redhat7
> .1/01.00/MIPS_RedHat7.1_Release-01.00.iso, burned a CD, and followed the
> instructions provided on the CD (/linux/installation/README) to install
> Linux (little-endian) on a MIPS Atlas/4Kc board with a SCSI disk.

Note that you can install the Release-02.00 (with all the latest RPMs
from H.J.) as well on an Atlas, you'll just have to use the 2.4.3 install
kernel from the 01.00 CD image you downloaded, and everything else from 
the new release.


> would now like to recompile the kernel to experiment with sound.  I have
> posted to this mailing list before and was informed that I need a cross-
> compiler, available on oss.sgi.com.  My host is Red Hat Linux-i386 7.3.

For kernel cross compilation we use the following binary RPM's (LE shown only):

        binutils-mipsel-linux-2.9.5-3
        egcs-mipsel-linux-1.1.2-4

They can be found on:

        ftp://oss.sgi.com/pub/linux/mips/crossdev/i386-linux/mipsel-linux/


/Hartvig

Re: cross-compiler for MIPS_RedHat7.1_Release-01.00 on Atlas/4Kc using RH7.3-i386 host

[Hartvig Ekner]

Hi David,

David Christensen writes:
> 
> linux-mips@oss.sgi.com:
> 
> I have downloaded ftp://ftp.mips.com/pub/linux/mips/installation/redhat7
> .1/01.00/MIPS_RedHat7.1_Release-01.00.iso, burned a CD, and followed the
> instructions provided on the CD (/linux/installation/README) to install
> Linux (little-endian) on a MIPS Atlas/4Kc board with a SCSI disk.

Note that you can install the Release-02.00 (with all the latest RPMs
from H.J.) as well on an Atlas, you'll just have to use the 2.4.3 install
kernel from the 01.00 CD image you downloaded, and everything else from 
the new release.


> would now like to recompile the kernel to experiment with sound.  I have
> posted to this mailing list before and was informed that I need a cross-
> compiler, available on oss.sgi.com.  My host is Red Hat Linux-i386 7.3.

For kernel cross compilation we use the following binary RPM's (LE shown only):

        binutils-mipsel-linux-2.9.5-3
        egcs-mipsel-linux-1.1.2-4

They can be found on:

        ftp://oss.sgi.com/pub/linux/mips/crossdev/i386-linux/mipsel-linux/


/Hartvig

Re: cross-compiler for MIPS_RedHat7.1_Release-01.00 on Atlas/4Kc using RH7.3-i386 host

[David Christensen]

linux-mips@oss.sgi.com & Hartvig:

Hartvig Ekner wrote:
> Note that you can install the Release-02.00 (with all the latest RPMs
> from H.J.) as well on an Atlas, you'll just have to use the 2.4.3
> install kernel from the 01.00 CD image you downloaded, and everything
> else from the new release.

I'll keep that in mind for my next install.


> For kernel cross compilation we use the following binary RPM's (LE
> shown only):
>
>        binutils-mipsel-linux-2.9.5-3
>        egcs-mipsel-linux-1.1.2-4

I'll try what I've already installed and see what happens.  If it fails,
I'll upgrade binutils and try again.


Thank you for your help.  :-)


David

Re: cross-compiler for MIPS_RedHat7.1_Release-01.00 on Atlas/4Kc using RH7.3-i386 host

[David Christensen]

linux-mips@oss.sgi.com & Hartvig:

Hartvig Ekner wrote:
> Note that you can install the Release-02.00 (with all the latest RPMs
> from H.J.) as well on an Atlas, you'll just have to use the 2.4.3
> install kernel from the 01.00 CD image you downloaded, and everything
> else from the new release.

I'll keep that in mind for my next install.


> For kernel cross compilation we use the following binary RPM's (LE
> shown only):
>
>        binutils-mipsel-linux-2.9.5-3
>        egcs-mipsel-linux-1.1.2-4

I'll try what I've already installed and see what happens.  If it fails,
I'll upgrade binutils and try again.


Thank you for your help.  :-)


David

Re: cross-compiler for MIPS_RedHat7.1_Release-01.00 on Atlas/4Kc using RH7.3-i386 host

[Muthukumar Ratty]

On Thu, 30 May 2002, David Christensen wrote:

> linux-mips@oss.sgi.com & Hartvig:
>
> Hartvig Ekner wrote:
> > from H.J.) as well on an Atlas, you'll just have to use the 2.4.3
> > install kernel from the 01.00 CD image you downloaded, and everything
> > else from the new release.

Is there any latest kernel (2.5.xx) available for MIPS/Atlas?

>
> >
> >        binutils-mipsel-linux-2.9.5-3
> >        egcs-mipsel-linux-1.1.2-4
>

I played around with some cross-compilers and what I understood is

1. Algorithmics sde4 is not matured enough to compile 2.4.xx kernels (As
Dominic Sweetman mentioned in his reply to my help mail). He said sde5
will do but I dint get a chance to try this. Any update from anyone used it?

2. I followed BradLaRondes write up - Building a Modern Mips Tool chain (I
dont have the link irght now, but you can google it). It compiles the
kernel and Applications. But it requires kernel header lying around from previous
builds. So if you are just starting, then you may wanna grab the header
from somewhere. But the problem with this toolchain is: I was not able to
build Yamon SREC using this.

3. I also tried Steve Hills toolchain located at
ftp://ftp.cotw.com/Linux/MIPS/toolchain/experimental
It is complete and I can build kernel and applications with it. Again I
couldnt build Yamon SREC with it.

BTW you may need to do slight changes, like changing target from
little-mips to tradlittle-mips etc. Its simple but if you get stuck, post
it here and someone will help you.

All the best,
Muthu

> I'll try what I've already installed and see what happens.  If it fails,
> I'll upgrade binutils and try again.
>
>
> Thank you for your help.  :-)
>
>
> David
>
>
>
>
>

Re: cross-compiler for MIPS_RedHat7.1_Release-01.00 on Atlas/4Kc using RH7.3-i386 host

[Muthukumar Ratty]

On Thu, 30 May 2002, David Christensen wrote:

> linux-mips@oss.sgi.com & Hartvig:
>
> Hartvig Ekner wrote:
> > from H.J.) as well on an Atlas, you'll just have to use the 2.4.3
> > install kernel from the 01.00 CD image you downloaded, and everything
> > else from the new release.

Is there any latest kernel (2.5.xx) available for MIPS/Atlas?

>
> >
> >        binutils-mipsel-linux-2.9.5-3
> >        egcs-mipsel-linux-1.1.2-4
>

I played around with some cross-compilers and what I understood is

1. Algorithmics sde4 is not matured enough to compile 2.4.xx kernels (As
Dominic Sweetman mentioned in his reply to my help mail). He said sde5
will do but I dint get a chance to try this. Any update from anyone used it?

2. I followed BradLaRondes write up - Building a Modern Mips Tool chain (I
dont have the link irght now, but you can google it). It compiles the
kernel and Applications. But it requires kernel header lying around from previous
builds. So if you are just starting, then you may wanna grab the header
from somewhere. But the problem with this toolchain is: I was not able to
build Yamon SREC using this.

3. I also tried Steve Hills toolchain located at
ftp://ftp.cotw.com/Linux/MIPS/toolchain/experimental
It is complete and I can build kernel and applications with it. Again I
couldnt build Yamon SREC with it.

BTW you may need to do slight changes, like changing target from
little-mips to tradlittle-mips etc. Its simple but if you get stuck, post
it here and someone will help you.

All the best,
Muthu

> I'll try what I've already installed and see what happens.  If it fails,
> I'll upgrade binutils and try again.
>
>
> Thank you for your help.  :-)
>
>
> David
>
>
>
>
>

Re: cross-compiler for MIPS_RedHat7.1_Release-01.00 on Atlas/4Kc

[Hartvig Ekner]

Hi,

Muthukumar Ratty writes:
> 
> On Thu, 30 May 2002, David Christensen wrote:
> 
> > linux-mips@oss.sgi.com & Hartvig:
> >
> > Hartvig Ekner wrote:
> > > from H.J.) as well on an Atlas, you'll just have to use the 2.4.3
> > > install kernel from the 01.00 CD image you downloaded, and everything
> > > else from the new release.
> 
> Is there any latest kernel (2.5.xx) available for MIPS/Atlas?

No. Not from us anyway. Internally, we use the Linux systems heavily for
processor testing, so we tend to stay away from the bleeding edge (==
too  many problems and SW bugs). That is also why we haven't switched
to 2.4.18 until now.

That being said, there are probably many others who compile & use 2.5
kernels for MIPS.

> I played around with some cross-compilers and what I understood is
> 
> 1. Algorithmics sde4 is not matured enough to compile 2.4.xx kernels (As
> Dominic Sweetman mentioned in his reply to my help mail). He said sde5
> will do but I dint get a chance to try this. Any update from anyone used it?

We're using a beta of it - and there are known issues being worked on,
both compiling userland natively and kernel cross compiles.
You'll have to ask Dom when he expects final 5.0 to go out the door.

/Hartvig

Re: cross-compiler for MIPS_RedHat7.1_Release-01.00 on Atlas/4Kc

[Hartvig Ekner]

Hi,

Muthukumar Ratty writes:
> 
> On Thu, 30 May 2002, David Christensen wrote:
> 
> > linux-mips@oss.sgi.com & Hartvig:
> >
> > Hartvig Ekner wrote:
> > > from H.J.) as well on an Atlas, you'll just have to use the 2.4.3
> > > install kernel from the 01.00 CD image you downloaded, and everything
> > > else from the new release.
> 
> Is there any latest kernel (2.5.xx) available for MIPS/Atlas?

No. Not from us anyway. Internally, we use the Linux systems heavily for
processor testing, so we tend to stay away from the bleeding edge (==
too  many problems and SW bugs). That is also why we haven't switched
to 2.4.18 until now.

That being said, there are probably many others who compile & use 2.5
kernels for MIPS.

> I played around with some cross-compilers and what I understood is
> 
> 1. Algorithmics sde4 is not matured enough to compile 2.4.xx kernels (As
> Dominic Sweetman mentioned in his reply to my help mail). He said sde5
> will do but I dint get a chance to try this. Any update from anyone used it?

We're using a beta of it - and there are known issues being worked on,
both compiling userland natively and kernel cross compiles.
You'll have to ask Dom when he expects final 5.0 to go out the door.

/Hartvig

Re: cross-compiler for MIPS_RedHat7.1_Release-01.00 on Atlas/4Kc using RH7.3-i386 host

[David Christensen]

linux-mips@oss.sgi.com & Muthukumar:

Muthukumar Ratty wrote:
> BTW you may need to do slight changes, like changing target from
> little-mips to tradlittle-mips etc. Its simple but if you get stuck,
> post it here and someone will help you.

So far, no problems.  :-)


Thank you for your reply.


David Christensen
dpchrist@holgerdanske.com

Re: cross-compiler for MIPS_RedHat7.1_Release-01.00 on Atlas/4Kc using RH7.3-i386 host

[David Christensen]

linux-mips@oss.sgi.com & Muthukumar:

Muthukumar Ratty wrote:
> BTW you may need to do slight changes, like changing target from
> little-mips to tradlittle-mips etc. Its simple but if you get stuck,
> post it here and someone will help you.

So far, no problems.  :-)


Thank you for your reply.


David Christensen
dpchrist@holgerdanske.com

Re: [PATCH] scheduler hints

[Rick Bressler]

> So I went ahead and implemented scheduler hints on top of the O(1)
> scheduler. 

> Other hints could be "I am interactive" or "I am a batch (i.e. cpu hog)
> task" or "I am cache hot: try to keep me on this CPU". 

Sequent had an interesting hint they cooked up with Oracle. (Or maybe it
was the other way around.)  As I recall they called it 'twotask.'
Essentially Oracle clients processes spend a lot of time exchanging
information with its server process. It usually makes sense to bind them
to the same CPU in an SMP (and especially NUMA) machine.  (Probably
obvious to most of the folks on the group, but it is generally lots
better to essentially communicate through the cache and local memory
than across the NUMA bus.)

As I recall it made a significant difference in Oracle performance, and
would probably also translate to similar performance in many situations
where you had a client and server process doing lots of interaction in
an SMP environment.

Don't know if there is enough application to warrant it, but you asked.
:-)

-- 
+--------------------------------------------+ Rick Bressler
|Mushrooms and other fungi have several      | 
|important roles in nature.  They help things| 
|grow, they are a source of food, they       |
|decompose organic matter and they           | 
|infect, debilitate and kill organisms.      | Linux: Because a PC is a
+--------------------------------------------+ terrible thing to waste.

Re: [PATCH] scheduler hints

[Robert Love]

On Wed, 2002-06-05 at 17:46, Rick Bressler wrote:

> Sequent had an interesting hint they cooked up with Oracle. (Or maybe it
> was the other way around.)  As I recall they called it 'twotask.'
> Essentially Oracle clients processes spend a lot of time exchanging
> information with its server process. It usually makes sense to bind them
> to the same CPU in an SMP (and especially NUMA) machine.  (Probably
> obvious to most of the folks on the group, but it is generally lots
> better to essentially communicate through the cache and local memory
> than across the NUMA bus.)

This is similar in theory to why we used to have the sync option on
wake_up for pipes... it does work.

We don't need a scheduler "hint" for this, though.  A big loud command
"bind me to this processor!" would do fine, and in 2.5 we have that:

just have one of the tasks do:

	sched_setaffinity(0, sizeof(unsigned long), 1);
	sched_setaffinity(other_guys_pid, sizeof(unsigned long), 1);

and both will be affined to CPU 1.

	Robert Love

Re: [PATCH] scheduler hints

[Rick Bressler]

> We don't need a scheduler "hint" for this, though.  A big loud command
> "bind me to this processor!" would do fine, and in 2.5 we have that:
> 
> just have one of the tasks do:
> 
> 	sched_setaffinity(0, sizeof(unsigned long), 1);
> 	sched_setaffinity(other_guys_pid, sizeof(unsigned long), 1);
> 
> and both will be affined to CPU 1.

I think that in some ways they were trying to simplify the code.  It is
a bit more complicated to do well from user space.  You're talking
dozens to thousands of process pairs, and maybe dozens of CPU's.  I
think the idea was that the scheduler has a better idea of what CPU's
are least busy, where to put the processes and indeed should migrate
tasks as necessary.  I just does it in pairs.  Keep em together is the
idea, rather than keep them in any one specific place, thus the hint.

I note that Gerrit replied also and as I recall he is one of those ex
Sequent guys who really knows this stuff, so I'll bow out in favor of
the experts. :-)

-- 
+--------------------------------------------+ Rick Bressler
|Mushrooms and other fungi have several      | G-4781 (425)342-1554
|important roles in nature.  They help things| Pager 1-800-946-4646
|grow, they are a source of food, they       |         Pin: 1700898
|decompose organic matter and they           | bressler@mushroom.ca.boeing.com
|infect, debilitate and kill organisms.      | Linux: Because a PC is a
+--------------------------------------------+ terrible thing to waste.

Re: [PATCH] scheduler hints

[Gerrit Huizenga]

In message <200206060046.g560kJi04034@mushroom.ca.boeing.com>, > : Rick Bressle
r writes:
> > So I went ahead and implemented scheduler hints on top of the O(1)
> > scheduler. 
> 
> > Other hints could be "I am interactive" or "I am a batch (i.e. cpu hog)
> > task" or "I am cache hot: try to keep me on this CPU". 
> 
> Sequent had an interesting hint they cooked up with Oracle. (Or maybe it
> was the other way around.)  As I recall they called it 'twotask.'
> Essentially Oracle clients processes spend a lot of time exchanging
> information with its server process. It usually makes sense to bind them
> to the same CPU in an SMP (and especially NUMA) machine.  (Probably
> obvious to most of the folks on the group, but it is generally lots
> better to essentially communicate through the cache and local memory
> than across the NUMA bus.)

Actually, process-to-process affinity, which was later generalized
as a process gang affinity.

> As I recall it made a significant difference in Oracle performance, and
> would probably also translate to similar performance in many situations
> where you had a client and server process doing lots of interaction in
> an SMP environment.

Yep.  Must be used with care, but not terribly damaging for general
access.   Typically arranged as a many to one linkage by the callers,
which simplified the rebalancing decisions a bit.  I think there
was a paper written about it somewhere by Phil Krueger.

gerrit

Re: [PATCH] scheduler hints

[Robert Love]

On Wed, 2002-06-05 at 18:05, Gerrit Huizenga wrote:

> Actually, process-to-process affinity, which was later generalized
> as a process gang affinity.

Oh OK, gang affinity - a bit different and not what we do now :)

Interesting to look into, although not terribly useful I suspect weighed
against its implementation...

	Robert Love

Re: [PATCH] scheduler hints

[Gerrit Huizenga]

In message <1023325903.912.390.camel@sinai>, > : Robert Love writes:
> On Wed, 2002-06-05 at 18:05, Gerrit Huizenga wrote:
> 
> > Actually, process-to-process affinity, which was later generalized
> > as a process gang affinity.
> 
> Oh OK, gang affinity - a bit different and not what we do now :)
> 
> Interesting to look into, although not terribly useful I suspect weighed
> against its implementation...
> 
> 	Robert Love

Our scheduler *was* a long set of conditionals.  However, from the
stock BSD scheduler through the contorted thing that it became, we
saw something like 30-50% increases in some workloads.  I think
the motivator was actually not Oracle originally but something like
SAP.  Specific numbers are hard to extract now since we did so many
SMP & NUMA changes over the years, but I think I remember a slide
showing over 30% increase in SAP for this one additional feature.
I don't know that I ever saw specific Oracle or Oracle Apps numbers
for this although it was viewed as a "large" benefit, especially
in NUMA machines, but even on SMP machines.

gerrit

Re: [PATCH] scheduler hints

[Bill Davidsen]

On Wed, 5 Jun 2002, Rick Bressler wrote:

> > So I went ahead and implemented scheduler hints on top of the O(1)
> > scheduler. 
> 
> > Other hints could be "I am interactive" or "I am a batch (i.e. cpu hog)
> > task" or "I am cache hot: try to keep me on this CPU". 
> 
> Sequent had an interesting hint they cooked up with Oracle. (Or maybe it
> was the other way around.)  As I recall they called it 'twotask.'
> Essentially Oracle clients processes spend a lot of time exchanging
> information with its server process. It usually makes sense to bind them
> to the same CPU in an SMP (and especially NUMA) machine.  (Probably
> obvious to most of the folks on the group, but it is generally lots
> better to essentially communicate through the cache and local memory
> than across the NUMA bus.)

Are you really saying that you think serializing all the clients through a
single processor will gain more than than you lose by not using all the
other CPUs for clients?
 
> As I recall it made a significant difference in Oracle performance, and
> would probably also translate to similar performance in many situations
> where you had a client and server process doing lots of interaction in
> an SMP environment.

I've certainly seen a "significant difference" between uni and SMP, but it
was always in the other direction. Is this particular to some hardware, or
running multiple servers somehow? I'm only fmailiar with Linux, AIX and
Solaris, maybe this is Sequent magic? Or were you talking about having
only one client total on the machine and just making that run fast?

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [PATCH] scheduler hints

[Gerrit Huizenga]

In message <Pine.LNX.3.96.1020610165913.23851B-100000@gatekeeper.tmr.com>, > : 
Bill Davidsen writes:
> On Wed, 5 Jun 2002, Rick Bressler wrote:
> 
> > > So I went ahead and implemented scheduler hints on top of the O(1)
> > > scheduler. 
> > 
> > > Other hints could be "I am interactive" or "I am a batch (i.e. cpu hog)
> > > task" or "I am cache hot: try to keep me on this CPU". 
> > 
> > Sequent had an interesting hint they cooked up with Oracle. (Or maybe it
> > was the other way around.)  As I recall they called it 'twotask.'
> > Essentially Oracle clients processes spend a lot of time exchanging
> > information with its server process. It usually makes sense to bind them
> > to the same CPU in an SMP (and especially NUMA) machine.  (Probably
> > obvious to most of the folks on the group, but it is generally lots
> > better to essentially communicate through the cache and local memory
> > than across the NUMA bus.)
> 
> Are you really saying that you think serializing all the clients through a
> single processor will gain more than than you lose by not using all the
> other CPUs for clients?
  
When the number of runnable processes exceeds the number of CPUs
in an SMP system, and subsets of the runnable processes share data
(pipes, sockets, shared memory, etc.), minimizing the cache invalidate
effects of the subset by scheduling them on the same CPU (with some
level of cache-affinity, or stickiness) can increase throughput
dramatically.  Oracle Apps and BAAN are two application sets that have
this kind of behavior which benefited from having these subsets of
related processes "tied at the wrists" when they were scheduled.

Figure 1000 runnable process, in sets of two or even sets of ten.
The subsets should be scheduled together if possible, but still
smeared across all processors.  No one is suggesting that a UP machine
will always outperform an SMP machine.  ;-)

As was mentioned in another aspect of this thread, this is different
from explicit user-specified CPU affinity in that there are more processes
than a user wants to allocate explicitly to CPUs.  Instead, the scheduler
can do load balancing by moving/migrating sets of processes from an
overloaded CPU to a less loaded CPU.  However, the cache effects can
make a difference of something like 20-50% of overall throughput in
a fairly intensive data sharing workload like this.

> > As I recall it made a significant difference in Oracle performance, and
> > would probably also translate to similar performance in many situations
> > where you had a client and server process doing lots of interaction in
> > an SMP environment.
> 
> I've certainly seen a "significant difference" between uni and SMP, but it
> was always in the other direction. Is this particular to some hardware, or
> running multiple servers somehow? I'm only fmailiar with Linux, AIX and
> Solaris, maybe this is Sequent magic? Or were you talking about having
> only one client total on the machine and just making that run fast?

This is an SMP thing, which also benefits NUMA pretty dramatically.
And this is about how processes are scheduled, and how hints can
be provided to the scheduler.  It also relates to the overhead of
cache invalidation, the size of CPU caches, etc.  Sequent's hardware
might have seen a bigger improvement from this type of change than
other types of hardware might.  Or vice versa.

gerrit

Re: [parisc-linux] compiling kernels with gcc-3.1

[John David Anglin]

> Is --disable-indexing used in kernel builds?  This should stop gcc
> from generating the above.

Oops, that should be "-mdisable-indexing".

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: [parisc-linux] compiling kernels with gcc-3.1

[Randolph Chung]

In reference to a message from John David Anglin, dated Jul 15:
> > Is --disable-indexing used in kernel builds?  This should stop gcc
> > from generating the above.
> 
> Oops, that should be "-mdisable-indexing".

right now we use:
-Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fomit-frame-pointer
-fno-strict-aliasing -fno-common -D__linux__ -pipe -fno-strength-reduce
-mno-space-regs -mfast-indirect-calls -mdisable-fpregs
-ffunction-sections -march=2.0 -mschedule=8000

(the last two are only for pa8x00 builds...)

i'll try the disable-indexing stuff...

randolph
--  
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/

Re: [parisc-linux] compiling kernels with gcc-3.1

[Matthew Wilcox]

On Mon, Jul 15, 2002 at 10:32:33AM -0700, Randolph Chung wrote:
> In reference to a message from John David Anglin, dated Jul 15:
> > > Is --disable-indexing used in kernel builds?  This should stop gcc
> > > from generating the above.
> > 
> > Oops, that should be "-mdisable-indexing".
> 
> right now we use:
> -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fomit-frame-pointer
> -fno-strict-aliasing -fno-common -D__linux__ -pipe -fno-strength-reduce
> -mno-space-regs -mfast-indirect-calls -mdisable-fpregs
> -ffunction-sections -march=2.0 -mschedule=8000
> 
> (the last two are only for pa8x00 builds...)
> 
> i'll try the disable-indexing stuff...

while we're on the subject of gcc flags ...

right now, we have an interesting patch in the kernel to get around a
fun feature of the PA ABI that no other ABI seems to have:

-asmlinkage ssize_t sys_pread(unsigned int fd, char * buf,
-                            size_t count, loff_t pos)
+static inline
+ssize_t do_pread(unsigned int fd, char * buf, size_t count, loff_t pos)
[...]
+#if BITS_PER_LONG == 32
+#ifdef __BIG_ENDIAN
+#define LOFF_T(high, low) unsigned int high, unsigned int low
+#else
+#define LOFF_T(high, low) unsigned int low, unsigned int high
+#endif
+
+asmlinkage
+ssize_t sys_pread(unsigned int fd, char *buf, size_t count, LOFF_T(high, low))
+{
+       return do_pread(fd, buf, count, (loff_t)high << 32 | low);
+}
+
+ssize_t sys_pwrite(unsigned int fd, char *buf, size_t count, LOFF_T(high, low))
+{
+       return do_pwrite(fd, buf, count, (loff_t)high << 32 | low);
+}

the problem is that the PA ABI specifies that quantities shall be
`naturally' aligned, even 64-bit quantities on 32-bit machines.  so the
kernel is expecting to see:

	fd (32 bits)
	buf (32 bits)
	count (32 bits)
	(empty)
	pos (64 bits)

what it _actually_ gets (because glibc is hideously broken, IMO):

	fd
	buf
	count
	(pos >> 32)
	pos & 0xffffffff

is there any chance of having a flag (or maybe an __attribute__ that
we could #define asmlinkage to?) which would change the ABI to be more
compressed like all the other architectures?

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] compiling kernels with gcc-3.1

[John David Anglin]

> the problem is that the PA ABI specifies that quantities shall be
> `naturally' aligned, even 64-bit quantities on 32-bit machines.  so the
> kernel is expecting to see:
> 
> 	fd (32 bits)
> 	buf (32 bits)
> 	count (32 bits)
> 	(empty)
> 	pos (64 bits)
> 
> what it _actually_ gets (because glibc is hideously broken, IMO):
> 
> 	fd
> 	buf
> 	count
> 	(pos >> 32)
> 	pos & 0xffffffff

It's probably a whole lot easier to fix glibc than change gcc.

> is there any chance of having a flag (or maybe an __attribute__ that
> we could #define asmlinkage to?) which would change the ABI to be more
> compressed like all the other architectures?

I think this is basically a bad idea as it will have a whole lot
of nasty side effects.  How will gdb know which ABI, varargs, etc?

You would need to use a compiler switch and modify
FUNCTION_ARG_PARTIAL_NREGS and FUNCTION_ARG appropriately.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: [parisc-linux] compiling kernels with gcc-3.1

[joel.soete]

Quoting John David Anglin <dave@hiauly1.hia.nrc.ca>:

> > Is --disable-indexing used in kernel builds?  This should stop gcc
> > from generating the above.
> 
> Oops, that should be "-mdisable-indexing".

Sorry Dave,

it do not help (system still crashing)

Joel


-------------------------------------------------
This mail sent through Tiscali Webmail (http://webmail.tiscali.be)

Re: [parisc-linux] gcc-3.[02] alignment problem

[John David Anglin]

> These messages are coming from the assembler (remove "-pipe").  The
> first is from "std %r4,44(%r3)".  The offset "44" is not correct for
> a store double.  This is probably a problem with pointer arithmetic
> in the source.

Check the definition of RTA_ALIGNTO in linux/rtnetlink.h.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: [parisc-linux] gcc-3.[02] alignment problem

[John David Anglin]

> These messages are coming from the assembler (remove "-pipe").  The
> first is from "std %r4,44(%r3)".  The offset "44" is not correct for
> a store double.  This is probably a problem with pointer arithmetic
> in the source.

Check the definition of RTA_ALIGNTO in linux/rtnetlink.h.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: [parisc-linux] gcc-3.[02] alignment problem

[Randolph Chung]

In reference to a message from John David Anglin, dated Jul 16:
> > These messages are coming from the assembler (remove "-pipe").  The
> > first is from "std %r4,44(%r3)".  The offset "44" is not correct for
> > a store double.  This is probably a problem with pointer arithmetic
> > in the source.
> 
> Check the definition of RTA_ALIGNTO in linux/rtnetlink.h.

ah, this is not it, but i see what it is now.

include/linux/tcp_diag.h defines:

struct tcpdiag_sockid
{
        __u16   tcpdiag_sport;
        __u16   tcpdiag_dport;
        __u32   tcpdiag_src[4];
        __u32   tcpdiag_dst[4];
        __u32   tcpdiag_if;
        __u32   tcpdiag_cookie[2]; 
#define TCPDIAG_NOCOOKIE (~0U)
};

the code goes on to do:
        *((struct sock **)&r->id.tcpdiag_cookie) = sk;
and
            sk != *((struct sock **)&req->id.tcpdiag_cookie[0]))

even tho tcpdiag_cookie looks like it can be 64-bit aligned in the
struct, it appears that it isn't.... 

the "vomit grade hack" alan mentioned in another post is that in our
tree, we have:

struct tcpdiag_sockid
{
        __u16   tcpdiag_sport;
        __u16   tcpdiag_dport;
        __u32   tcpdiag_src[4];
        __u32   tcpdiag_dst[4];
        __u32   tcpdiag_if;
#if defined (__hppa__) && defined (__LP64__)
        char * parisc_hack_to_align_tcpdiag_cookie;
#endif
        __u32   tcpdiag_cookie[2];
#define TCPDIAG_NOCOOKIE (~0U)
};

why is the offset of tcpdiag_cookie[0] 44 and not 40?

randolph
--  
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/

Re: [parisc-linux] gcc-3.[02] alignment problem

[Randolph Chung]

In reference to a message from John David Anglin, dated Jul 16:
> > These messages are coming from the assembler (remove "-pipe").  The
> > first is from "std %r4,44(%r3)".  The offset "44" is not correct for
> > a store double.  This is probably a problem with pointer arithmetic
> > in the source.
> 
> Check the definition of RTA_ALIGNTO in linux/rtnetlink.h.

ah, this is not it, but i see what it is now.

include/linux/tcp_diag.h defines:

struct tcpdiag_sockid
{
        __u16   tcpdiag_sport;
        __u16   tcpdiag_dport;
        __u32   tcpdiag_src[4];
        __u32   tcpdiag_dst[4];
        __u32   tcpdiag_if;
        __u32   tcpdiag_cookie[2]; 
#define TCPDIAG_NOCOOKIE (~0U)
};

the code goes on to do:
        *((struct sock **)&r->id.tcpdiag_cookie) = sk;
and
            sk != *((struct sock **)&req->id.tcpdiag_cookie[0]))

even tho tcpdiag_cookie looks like it can be 64-bit aligned in the
struct, it appears that it isn't.... 

the "vomit grade hack" alan mentioned in another post is that in our
tree, we have:

struct tcpdiag_sockid
{
        __u16   tcpdiag_sport;
        __u16   tcpdiag_dport;
        __u32   tcpdiag_src[4];
        __u32   tcpdiag_dst[4];
        __u32   tcpdiag_if;
#if defined (__hppa__) && defined (__LP64__)
        char * parisc_hack_to_align_tcpdiag_cookie;
#endif
        __u32   tcpdiag_cookie[2];
#define TCPDIAG_NOCOOKIE (~0U)
};

why is the offset of tcpdiag_cookie[0] 44 and not 40?

randolph
--  
Randolph Chung
Debian GNU/Linux Developer, hppa/ia64 ports
http://www.tausq.org/

Re: [parisc-linux] gcc-3.[02] alignment problem

[Matthew Wilcox]

On Tue, Jul 16, 2002 at 10:22:55AM -0700, Randolph Chung wrote:
> include/linux/tcp_diag.h defines:
> 
> struct tcpdiag_sockid
> {
>         __u16   tcpdiag_sport;
>         __u16   tcpdiag_dport;
>         __u32   tcpdiag_src[4];
>         __u32   tcpdiag_dst[4];
>         __u32   tcpdiag_if;
>         __u32   tcpdiag_cookie[2]; 
> #define TCPDIAG_NOCOOKIE (~0U)
> };

> why is the offset of tcpdiag_cookie[0] 44 and not 40?

0	tcpdiag_sport
2	tcpdiag_dport
4	tcpdiag_src
20	tcpdiag_dst
36	tcpdiag_if
40	tcpdiag_cookie

hmm.. worth checking that dport is at offset 2, not offset 4?

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] gcc-3.[02] alignment problem

[Randolph Chung]

> > why is the offset of tcpdiag_cookie[0] 44 and not 40?
> 
> 0	tcpdiag_sport
> 2	tcpdiag_dport
> 4	tcpdiag_src
> 20	tcpdiag_dst
> 36	tcpdiag_if
> 40	tcpdiag_cookie
> 
> hmm.. worth checking that dport is at offset 2, not offset 4?

oic, it's embedded inside another structure:

struct tcpdiagmsg
{
        __u8    tcpdiag_family;
        __u8    tcpdiag_state;
        __u8    tcpdiag_timer;
        __u8    tcpdiag_retrans;

        struct tcpdiag_sockid id;

        __u32   tcpdiag_expires;
        __u32   tcpdiag_rqueue;
        __u32   tcpdiag_wqueue;
        __u32   tcpdiag_uid;
        __u32   tcpdiag_inode;
};

that's why it's 44...

randolph

Re: [parisc-linux] gcc-3.[02] alignment problem

[Randolph Chung]

> > why is the offset of tcpdiag_cookie[0] 44 and not 40?
> 
> 0	tcpdiag_sport
> 2	tcpdiag_dport
> 4	tcpdiag_src
> 20	tcpdiag_dst
> 36	tcpdiag_if
> 40	tcpdiag_cookie
> 
> hmm.. worth checking that dport is at offset 2, not offset 4?

oic, it's embedded inside another structure:

struct tcpdiagmsg
{
        __u8    tcpdiag_family;
        __u8    tcpdiag_state;
        __u8    tcpdiag_timer;
        __u8    tcpdiag_retrans;

        struct tcpdiag_sockid id;

        __u32   tcpdiag_expires;
        __u32   tcpdiag_rqueue;
        __u32   tcpdiag_wqueue;
        __u32   tcpdiag_uid;
        __u32   tcpdiag_inode;
};

that's why it's 44...

randolph

Re: [parisc-linux] gcc-3.[02] alignment problem

[Matthew Wilcox]

On Tue, Jul 16, 2002 at 10:22:55AM -0700, Randolph Chung wrote:
> include/linux/tcp_diag.h defines:
> 
> struct tcpdiag_sockid
> {
>         __u16   tcpdiag_sport;
>         __u16   tcpdiag_dport;
>         __u32   tcpdiag_src[4];
>         __u32   tcpdiag_dst[4];
>         __u32   tcpdiag_if;
>         __u32   tcpdiag_cookie[2]; 
> #define TCPDIAG_NOCOOKIE (~0U)
> };

> why is the offset of tcpdiag_cookie[0] 44 and not 40?

0	tcpdiag_sport
2	tcpdiag_dport
4	tcpdiag_src
20	tcpdiag_dst
36	tcpdiag_if
40	tcpdiag_cookie

hmm.. worth checking that dport is at offset 2, not offset 4?

-- 
Revolutions do not require corporate support.

Re: [parisc-linux] gcc-3.[02] alignment problem

[Richard Henderson]

On Tue, Jul 16, 2002 at 10:22:55AM -0700, Randolph Chung wrote:
>         __u32   tcpdiag_cookie[2]; 
> #define TCPDIAG_NOCOOKIE (~0U)
> };
> 
> the code goes on to do:
>         *((struct sock **)&r->id.tcpdiag_cookie) = sk;
> and
>             sk != *((struct sock **)&req->id.tcpdiag_cookie[0]))

This is absolutely awful.

> the "vomit grade hack" alan mentioned in another post is that in our
> tree, we have:
> 
> struct tcpdiag_sockid
> {
>         __u16   tcpdiag_sport;
>         __u16   tcpdiag_dport;
>         __u32   tcpdiag_src[4];
>         __u32   tcpdiag_dst[4];
>         __u32   tcpdiag_if;
> #if defined (__hppa__) && defined (__LP64__)
>         char * parisc_hack_to_align_tcpdiag_cookie;
> #endif
>         __u32   tcpdiag_cookie[2];

An only marginally better fix is

	__u32 tcpdiag_cookie[2] __attribute__((aligned(sizeof(void*))));

Note that this should be unconditional so that the other
64-bit ports don't take an alignment trap here too.

A much nicer fix would be to, gasp, use a union.



r~

Re: [parisc-linux] gcc-3.[02] alignment problem

[Richard Henderson]

On Tue, Jul 16, 2002 at 10:22:55AM -0700, Randolph Chung wrote:
>         __u32   tcpdiag_cookie[2]; 
> #define TCPDIAG_NOCOOKIE (~0U)
> };
> 
> the code goes on to do:
>         *((struct sock **)&r->id.tcpdiag_cookie) = sk;
> and
>             sk != *((struct sock **)&req->id.tcpdiag_cookie[0]))

This is absolutely awful.

> the "vomit grade hack" alan mentioned in another post is that in our
> tree, we have:
> 
> struct tcpdiag_sockid
> {
>         __u16   tcpdiag_sport;
>         __u16   tcpdiag_dport;
>         __u32   tcpdiag_src[4];
>         __u32   tcpdiag_dst[4];
>         __u32   tcpdiag_if;
> #if defined (__hppa__) && defined (__LP64__)
>         char * parisc_hack_to_align_tcpdiag_cookie;
> #endif
>         __u32   tcpdiag_cookie[2];

An only marginally better fix is

	__u32 tcpdiag_cookie[2] __attribute__((aligned(sizeof(void*))));

Note that this should be unconditional so that the other
64-bit ports don't take an alignment trap here too.

A much nicer fix would be to, gasp, use a union.



r~

Re: [parisc-linux] gcc-3.2 bootstrap?

[John David Anglin]

> > but when I try to use it to compile kernel-2.4.18-pa54 I got following error:
> > `gcc -print-libgcc-file-name`
> > /Develop/parisc-linux/src/linux-2.4.18-pa54/arch/parisc/lib/lib.a
> > /Develop/parisc-linux/src/linux-2.4.18-pa54/lib/lib.a  \
> >         --end-group \
> >         -o vmlinux
> > arch/parisc/kernel/kernel.o(__ksymtab+0x258): undefined reference to `$$mulU'  

> $$mulU is a millicode routine in libgcc.a.

Sorry, this routine is no longer in libgcc.a.  It was in the 32-bit
library for 3.0.x.  However, we have switched to the "64-bit" millicode
for both 32 and 64 bit code.  Gcc doesn't use it anymore but I see there
are a couple of remnants to clean up.

Your problem appears to be with arch/parisc/kernel/parisc_ksyms.c.
It imports the symbol.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

Re: usb CF reader and 2.4.19

[Vid Strpic]

On Wed, Oct 16, 2002 at 11:22:04AM +0200, Joseph Wenninger wrote:
> Is there anything I can do to flush all usb / usb storage buffers to
> my compact flash ? 
> At the moment I have to rmmod usb-storage && rmmod usb-uhci &&
> modprobe usb-uhci && modprobe usb-storage to ensure all data is
> written correctly, otherwise the directory structure isn't saved even
> after an unmount.

eject sda|whichever device it is, maybe?

Works here with my SM/CF reader.

-- 
           vms@bofhlet.net, IRC:*@Martin, /bin/zsh. C|N>K
Linux lorien 2.4.19rml #1 Mon Sep 16 09:01:48 CEST 2002 i586
 23:08:58 up 3 days,  5:29, 26 users,  load average: 2.17, 1.49, 0.97

Re: DPMI: Interrupt vector overwritten

[John Elliott]

: What exactly interrupt from this range do
: you want to execute?

  INT 0xEF, the GEM API. 
 
-- 
John Elliott

Re: Running linux-2.4.20-rc1 on Dell Dimension 4550

[Mitch Adair]

> Thought I'd post some information about what I'm seeing running RH7.3 with 
> kernel 2.4.20-rc1 on a brand new Dell Dimension 4550.  Currently there are 
> two problems with the machine:
> 
> - When I swtich to a text console and back to the X screen, the machine locks 
> up (or at least the console does not respond anymore).

you don't say the kernel, X, or hardware, but I've seen that personally
with radeon 7500... that what you have?

> - The sound driver does not load, although it seems to try.

seems to be ich4 audio - you need 0.23/24 driver for this to work -
check with the AC series - OR -

with my _particular_ i845E chipset - commenting out the "break;" immediately
after the line that goes "Primary codec not ready" makes this work fine --
probably your computer will burst into flames if you do the same or
something... this is *NOT* a recommendation.

	M

Re: Running linux-2.4.20-rc1 on Dell Dimension 4550

[Orion Poplawski]

On Tuesday 29 October 2002 11:42 pm, Mitch Adair wrote:
> > Thought I'd post some information about what I'm seeing running RH7.3
> > with kernel 2.4.20-rc1 on a brand new Dell Dimension 4550.  Currently
> > there are two problems with the machine:
> >
> > - When I swtich to a text console and back to the X screen, the machine
> > locks up (or at least the console does not respond anymore).
>
> you don't say the kernel, X, or hardware, but I've seen that personally
> with radeon 7500... that what you have?

Kernel is 2.4.20-rc1, X is 4.2.0-8 from RH.  The X log reports:

(--) Chipset ATI Rage 128 Pro ULTRA TF (AGP) found

Warnings/errors are:

(WW) R128(0): Can't determine panel dimensions, and none specified.
       Disabling programming of FP registers.

(EE) R128(0): No DFP detected

The Dell packing list reports: 32MB ATI RAGE ULTRA 4X AGP.

Probably should take this off linux-kernel....

- Orion

Re: 2.4.20 kernel link error

[Vid Strpic]

On Fri, Nov 29, 2002 at 07:00:50AM -0800, carbonated beverage wrote:
> Anything other info needed?
> gcc 2.95.4
> binutils 2.12.90.0.1
> Debian/woody
> Linux 2.4.20

Same config, but gcc 3.2.1 :


In file included from /usr/src/linux-2.4.20/include/linux/wait.h:13,
                 from /usr/src/linux-2.4.20/include/linux/fs.h:12,
                 from /usr/src/linux-2.4.20/include/linux/capability.h:17,
                 from /usr/src/linux-2.4.20/include/linux/binfmts.h:5,
                 from /usr/src/linux-2.4.20/include/linux/sched.h:9,
                 from buffer.c:32:
/usr/src/linux-2.4.20/include/linux/kernel.h:10:20: stdarg.h: No such file or directory


... and this repeats all over `make dep`... and when I'm making bzImage:


/usr/src/linux-2.4.20/include/linux/kernel.h:74: parse error before "va_list"
/usr/src/linux-2.4.20/include/linux/kernel.h:74: warning: function declaration isn't a prototype
/usr/src/linux-2.4.20/include/linux/kernel.h:77: parse error before "va_list"
/usr/src/linux-2.4.20/include/linux/kernel.h:77: warning: function declaration isn't a prototype
/usr/src/linux-2.4.20/include/linux/kernel.h:81: parse error before "va_list"
/usr/src/linux-2.4.20/include/linux/kernel.h:81: warning: function declaration isn't a prototype
make[2]: *** [sched.o] Error 1
make[2]: Leaving directory `/usr/src/linux-2.4.20/kernel'
make[1]: *** [first_rule] Error 2
make[1]: Leaving directory `/usr/src/linux-2.4.20/kernel'
make: *** [_dir_kernel] Error 2


It is the same with gcc-2.95.4 (from Debian package), btw ... I tried
rml patch (for rc4), and it also shows the same behavior....


-- 
           vms@bofhlet.net, IRC:*@Martin, /bin/zsh. C|N>K
Linux lorien 2.4.19 #1 Mon Sep 16 09:01:48 CEST 2002 i586
 23:58:05 up 2 days,  7:41, 29 users,  load average: 3.27, 3.27, 2.94
Prije nego sto ukljucis komp ubaci uticnicu u utikac. (DMC)

Re: increase base memory

[John Elliott]

: Ralf Brown memory.lst states:
: --------V-MA0000000--------------------------
: MEM A000h:0000h - EGA+ GRAPHICS BUFFER
: Size:	65536 BYTEs
: 
: So it was always there.

  Well, at least since the EGA. I suppose if DOSEMU emulated an MDA or a
CGA, then it would make sense to use the memory up to B000:0 or B800:0
respectively. On an original XT, it was even possible to do this in
hardware:
<ftp://ftp.simtel.net/pub/simtelnet/msdos/info/896k-mem.txt>
 
: QEMM have a program called "vidram", which allows to use that region as a
: base memory by the cost of not being able to diplay graphics, but vidram
: will not work under dosemu because qemm itself doesn't work under dosemu.

  The same thing is true of DRDOS (the equivalent program is called MEMMAX).

--------m-2F12FFBX0006-----------------------
INT 2F U - DR DOS 6+, Novell DOS 7+ - EMM386.EXE - VIDEO MEMORY SPACE
CONTROL
        AX = 12FFh
        BX = 0006h
        DX = 0000h
        CX = function
            0000h get status of video memory space (MEMMAX /V)
            0001h map memory into video memory space (MEMMAX +V)
            0002h unmap memory from video memory space (MEMMAX -V)

: Increasing the base memory above 640Kb is definitely the bad idea, as it
: is used very rarely even in a pure DOS and will cost you the inability to
: display graphics.

  Except in the CGA modes :-)

-- 
John Elliott

Re: increase base memory

[Bart Oldeman]

On Mon, 16 Dec 2002, John Elliott wrote:

>   Well, at least since the EGA. I suppose if DOSEMU emulated an MDA or a
> CGA, then it would make sense to use the memory up to B000:0 or B800:0
> respectively. On an original XT, it was even possible to do this in
> hardware:
> <ftp://ftp.simtel.net/pub/simtelnet/msdos/info/896k-mem.txt>

right, and this is also true if you run console DOSEMU and lie to it that
you use a CGA graphics card, ie. $_video = "cga", and remove that 640k
check in the parser. Caveat emptor -- your video BIOS might NOT like
this, weird things my happen, and I haven't tried it.

as for xdosemu: it uses vgaemu and I don't think that can be switched off
easily. vgaemu pretty much assumes that it can r/w protect all pages from
0xa000:0 up to 0xc000:0.

Now if you run dosemu in terminal mode you could go up to 736k (b800), and
if you run dosemu in "dumb" mode, well, f000 is even possible, ie., 960k,
if you disable EMS too.

Bart

Re: spinlock efficiency problem [was 2.5.57 IO slowdown with CONFIG_PREEMPT enabled)

[Joe Korty]

[ resend - forgot to send this to the list, also forgot intro text ]

> Robert Macaulay <robert_macaulay@dell.com> wrote:
>>
>> On Wed, 15 Jan 2003, Andrew Morton wrote:
>>> if you could please test that with CONFIG_PREEMPT=y
>> 
>> Reverting that brings the speed back up
>
> OK.  How irritating.
>
> Presumably there's a fairness problem - once a CPU goes in there to start
> spinning on the lock, the length of the loop is such that it's easy for
> non-holders to zoom in and claim it first.  Or something.
>
> Unless another way of solving the problem which that patch solves presents
> itself we may need to revert it.
>
> Or not.  Should a CONFIG_PREEMPT SMP kernel compromise its latency because of
> overused locking??


Andrew, Everyone,

The new, preemptable spin_lock() spins on an atomic bus-locking
read/write instead of an ordinary read, as the original spin_lock
implementation did.  Perhaps that is the source of the inefficiency
being seen.

Attached sample code compiles but is untested and incomplete (present
only to illustrate the idea).

Joe



--- 2.5-bk/kernel/sched.c.orig	2003-01-20 14:14:55.000000000 -0500
+++ 2.5-bk/kernel/sched.c	2003-01-20 17:31:49.000000000 -0500
@@ -2465,15 +2465,13 @@
 		_raw_spin_lock(lock);
 		return;
 	}
-
-	while (!_raw_spin_trylock(lock)) {
-		if (need_resched()) {
-			preempt_enable_no_resched();
-			__cond_resched();
-			preempt_disable();
+	do {
+		preempt_enable();
+		while(spin_is_locked(lock)) {
+			cpu_relax();
 		}
-		cpu_relax();
-	}
+		preempt_disable();
+	} while (!_raw_spin_trylock(lock));
 }
 
 void __preempt_write_lock(rwlock_t *lock)

Re: spinlock efficiency problem [was 2.5.57 IO slowdown with CONFIG_PREEMPT enabled)

[Alan]

On Mon, 2003-01-20 at 22:58, Joe Korty wrote:
> The new, preemptable spin_lock() spins on an atomic bus-locking
> read/write instead of an ordinary read, as the original spin_lock
> implementation did.  Perhaps that is the source of the inefficiency
> being seen.

Its a fairly critical "Never do this" on older intel processors and
kills the box very efficiently so your diagnosis is extremely 
plausible

Re: [parisc-linux] PATCH hppa ordered load absolute ops

[John David Anglin]

I installed the following after testing on hppa-unknown-linux-gnu.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

2003-01-31  Grant Grundler  <grundler@dsl2.external.hp.com>

        * hppa.h (ldwa, ldda): Add ordered opcodes.

Index: opcode/hppa.h
===================================================================
RCS file: /cvs/src/src/include/opcode/hppa.h,v
retrieving revision 1.49
diff -u -3 -p -r1.49 hppa.h
--- opcode/hppa.h	16 Dec 2002 09:57:03 -0000	1.49
+++ opcode/hppa.h	31 Jan 2003 21:13:31 -0000
@@ -407,6 +407,7 @@ static const struct pa_opcode pa_opcodes
 { "ldbx",       0x0c000000, 0xfc001fc0, "cXx(b),t", pa10, 0},
 { "ldwa",       0x0c000180, 0xfc00d3c0, "cxccx(b),t", pa10, FLAG_STRICT},
 { "ldwa",	0x0c001180, 0xfc00d3c0, "cmcc5(b),t", pa10, FLAG_STRICT},
+{ "ldwa",	0x0c0011a0, 0xfc1ff3e0, "cocc@(b),t", pa20, FLAG_STRICT},
 { "ldcw",       0x0c0001c0, 0xfc0013c0, "cxcdx(s,b),t", pa10, FLAG_STRICT},
 { "ldcw",       0x0c0001c0, 0xfc0013c0, "cxcdx(b),t", pa10, FLAG_STRICT},
 { "ldcw",	0x0c0011c0, 0xfc0013c0, "cmcd5(s,b),t", pa10, FLAG_STRICT},
@@ -417,6 +418,7 @@ static const struct pa_opcode pa_opcodes
 { "stby",	0x0c001300, 0xfc0013c0, "cscCx,V(b)", pa10, FLAG_STRICT},
 { "ldda",       0x0c000100, 0xfc00d3c0, "cxccx(b),t", pa20, FLAG_STRICT},
 { "ldda",	0x0c001100, 0xfc00d3c0, "cmcc5(b),t", pa20, FLAG_STRICT},
+{ "ldda",	0x0c001120, 0xfc1ff3e0, "cocc@(b),t", pa20, FLAG_STRICT},
 { "ldcd",       0x0c000140, 0xfc0013c0, "cxcdx(s,b),t", pa20, FLAG_STRICT},
 { "ldcd",       0x0c000140, 0xfc0013c0, "cxcdx(b),t", pa20, FLAG_STRICT},
 { "ldcd",	0x0c001140, 0xfc0013c0, "cmcd5(s,b),t", pa20, FLAG_STRICT},

Re: [parisc-linux] PATCH hppa ordered load absolute ops

[John David Anglin]

I installed the following after testing on hppa-unknown-linux-gnu.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6605)

2003-01-31  Grant Grundler  <grundler@dsl2.external.hp.com>

        * hppa.h (ldwa, ldda): Add ordered opcodes.

Index: opcode/hppa.h
===================================================================
RCS file: /cvs/src/src/include/opcode/hppa.h,v
retrieving revision 1.49
diff -u -3 -p -r1.49 hppa.h
--- opcode/hppa.h	16 Dec 2002 09:57:03 -0000	1.49
+++ opcode/hppa.h	31 Jan 2003 21:13:31 -0000
@@ -407,6 +407,7 @@ static const struct pa_opcode pa_opcodes
 { "ldbx",       0x0c000000, 0xfc001fc0, "cXx(b),t", pa10, 0},
 { "ldwa",       0x0c000180, 0xfc00d3c0, "cxccx(b),t", pa10, FLAG_STRICT},
 { "ldwa",	0x0c001180, 0xfc00d3c0, "cmcc5(b),t", pa10, FLAG_STRICT},
+{ "ldwa",	0x0c0011a0, 0xfc1ff3e0, "cocc@(b),t", pa20, FLAG_STRICT},
 { "ldcw",       0x0c0001c0, 0xfc0013c0, "cxcdx(s,b),t", pa10, FLAG_STRICT},
 { "ldcw",       0x0c0001c0, 0xfc0013c0, "cxcdx(b),t", pa10, FLAG_STRICT},
 { "ldcw",	0x0c0011c0, 0xfc0013c0, "cmcd5(s,b),t", pa10, FLAG_STRICT},
@@ -417,6 +418,7 @@ static const struct pa_opcode pa_opcodes
 { "stby",	0x0c001300, 0xfc0013c0, "cscCx,V(b)", pa10, FLAG_STRICT},
 { "ldda",       0x0c000100, 0xfc00d3c0, "cxccx(b),t", pa20, FLAG_STRICT},
 { "ldda",	0x0c001100, 0xfc00d3c0, "cmcc5(b),t", pa20, FLAG_STRICT},
+{ "ldda",	0x0c001120, 0xfc1ff3e0, "cocc@(b),t", pa20, FLAG_STRICT},
 { "ldcd",       0x0c000140, 0xfc0013c0, "cxcdx(s,b),t", pa20, FLAG_STRICT},
 { "ldcd",       0x0c000140, 0xfc0013c0, "cxcdx(b),t", pa20, FLAG_STRICT},
 { "ldcd",	0x0c001140, 0xfc0013c0, "cmcd5(s,b),t", pa20, FLAG_STRICT},

Re: Promise SATA chips

[Vid Strpic]

[-- Attachment #1: Type: text/plain, Size: 953 bytes --]

On Thu, Feb 13, 2003 at 08:03:42AM +0100, Mĺns Rullgĺrd wrote:
> Samuel Flory <sflory@rackable.com> writes:
> > >Are there any drivers being developed for Promise's SATA chips
> > >(e.g. the pdc20275 with PCI id 0x3375)?  Or do I have to
> > >disassemble their driver to use it on non-intel machines?
> > Last I checked there was no open driver and no prospect of one.  Do
> > yourself a favor and use the silicon image chipset.
> The problem is that no local dealers have it for sale.  Neither do
> they sell Hightpoint gear.

Mine do... but, I cannot find, which driver to use?

cmd64x.c?  I am referring to older, non-SATA boards...

I am using 2.4 kernel (currently 2.4.20).

-- 
           vms@bofhlet.net, IRC:*@Martin, /bin/zsh. C|N>K
Linux lorien 2.4.20rml #3 Sat Nov 30 20:35:09 CET 2002 i586
 09:37:46 up 7 days, 14:39, 30 users,  load average: 0.54, 0.98, 1.42
I FOR WITHOUT NEXT                      (c)1992 ZX Spectrum

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] update filesystems config. menu

[Mitch Adair]

> +config EXT2_FS
> +	tristate "Second extended fs support"

[...]

> +	  module will be called ext2.  Be aware however that the file system
> +	  of your root partition (the one containing the directory /) cannot
> +	  be compiled as a module, and so this could be dangerous.  Most
> +	  everyone wants to say Y here.
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
I seem to recall if you say Y to ext2 and Y to ext3 and your root is on ext3
then it's likely to be mounted as ext2 unless you set rootfstype=...
The ext3 help comment also warns you should set it to Y, so people with
root ext3 are likely to get a surprise if they follow both help.  Would it
be better to say something like "if your root partition is extX you almost
certainly want to say Y here"?  Or warn about the rootfstype issue maybe?

	M

Re: [PATCH] update filesystems config. menu

[Dave Jones]

On Sat, Mar 15, 2003 at 02:19:40AM -0600, Mitch Adair wrote:

 > I seem to recall if you say Y to ext2 and Y to ext3 and your root is on ext3
 > then it's likely to be mounted as ext2 unless you set rootfstype=...
 > The ext3 help comment also warns you should set it to Y, so people with
 > root ext3 are likely to get a surprise if they follow both help.  Would it
 > be better to say something like "if your root partition is extX you almost
 > certainly want to say Y here"?  Or warn about the rootfstype issue maybe?

(03:40:50:davej@tetrachloride:davej)$ mount
/dev/hda5 on / type ext3 (rw,errors=remount-ro)

(08:40:13:davej@tetrachloride:davej)$ cat /proc/filesystems | grep ext
	ext3
	ext2

Note the ordering, ext3 gets tried before ext2.

		Dave

Re: [PATCH] update filesystems config. menu

[Mitch Adair]

> Note the ordering, ext3 gets tried before ext2.

Was thinking of the "ext3 clings to you like flypaper" thread of a couple
weeks ago, only backwards... -ESLEEPY

Anyway, I have seen instances where root got mounted ext2 instead of ext3
(faulty initrd or whatever it was) - just wondered if the help should really
push people to unreservedly say Y to ext2 if their root is really ext3...

	M

Re: [PATCH] update filesystems config. menu

[Martin Schlemmer]

On Sat, 2003-03-15 at 11:20, Mitch Adair wrote:

> Anyway, I have seen instances where root got mounted ext2 instead of ext3
> (faulty initrd or whatever it was) - just wondered if the help should really
> push people to unreservedly say Y to ext2 if their root is really ext3...
> 

Should be safest for most people .. those that have experience will
anyhow know to only compile in ext3 support, and ext2 as module (if
you ever fsck floppy/whatever as ext2).


Regards,

-- 
Martin Schlemmer

Re: [PATCH] update filesystems config. menu

[Randy.Dunlap]

> On Sat, 2003-03-15 at 11:20, Mitch Adair wrote:
>
>> Anyway, I have seen instances where root got mounted ext2 instead of ext3
>> (faulty initrd or whatever it was) - just wondered if the help should
>> really push people to unreservedly say Y to ext2 if their root is really
>> ext3...
>>
>
> Should be safest for most people .. those that have experience will anyhow
> know to only compile in ext3 support, and ext2 as module (if you ever fsck
> floppy/whatever as ext2).

Hi Martin,

I'm having trouble decoding...
What is it that "should be safest for most people"?
Are you suggesting any changes here?

And some of us don't use fs modules, just build what we need into the
kernel.  Do you know of any problems with doing this (related to
ext2/ext3 for example)?

Thanks,
~Randy

Re: [PATCH] update filesystems config. menu

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1056 bytes --]

On Sat, 15 Mar 2003 09:08:44 -0800 (PST)
"Randy.Dunlap" <rddunlap@osdl.org> wrote:


> I'm having trouble decoding...
> What is it that "should be safest for most people"?
> Are you suggesting any changes here?
> 
> And some of us don't use fs modules, just build what we need into the
> kernel.  Do you know of any problems with doing this (related to
> ext2/ext3 for example)?
> 

I was just saying that recommending it (ext2) compiled into the kernel
and not a module should be the safe route for newbies to kernel
compiles.

Those of us that have build a few to feel comfortable with it, will know
to compile the fs of our / partition into the kernel.

Except if ext2 is not the most commonly used fs anymore.  I guess a
'cool' feature could be if the make system could 'detect' what your
current root is and warn if you do not have that compiled into your
kernel, but I do not know the limitations of it (the make system).

Then on the other hand, would above be confusing if its a kernel
compiled for another box ?


Regards,

-- 

Martin Schlemmer


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: [PATCH] update filesystems config. menu

[Randy.Dunlap]

> On Sat, 15 Mar 2003 09:08:44 -0800 (PST)
> "Randy.Dunlap" <rddunlap@osdl.org> wrote:
>
>> I'm having trouble decoding...
>> What is it that "should be safest for most people"?
>> Are you suggesting any changes here?
>>
>> And some of us don't use fs modules, just build what we need into the
>> kernel.  Do you know of any problems with doing this (related to ext2/ext3
>> for example)?
>>
>
> I was just saying that recommending it (ext2) compiled into the kernel and
> not a module should be the safe route for newbies to kernel
> compiles.

Thanks for the clarification.

> Those of us that have build a few to feel comfortable with it, will know to
> compile the fs of our / partition into the kernel.
>
> Except if ext2 is not the most commonly used fs anymore.  I guess a 'cool'
> feature could be if the make system could 'detect' what your current root is
> and warn if you do not have that compiled into your kernel, but I do not
> know the limitations of it (the make system).
>
> Then on the other hand, would above be confusing if its a kernel
> compiled for another box ?

Yes, I'd say so, although the message could say something like:
  Kernel does not include a filesystem for / on this computer.
And would it also have to check the capabilities of what's in the
initrd?  (not that I'm advocating any of this)

~Randy

Re: [PATCH] update filesystems config. menu

[jw schultz]

On Sat, Mar 15, 2003 at 01:31:59PM -0800, Randy.Dunlap wrote:
> > On Sat, 15 Mar 2003 09:08:44 -0800 (PST)
> > "Randy.Dunlap" <rddunlap@osdl.org> wrote:
> >
> >> I'm having trouble decoding...
> >> What is it that "should be safest for most people"?
> >> Are you suggesting any changes here?
> >>
> >> And some of us don't use fs modules, just build what we need into the
> >> kernel.  Do you know of any problems with doing this (related to ext2/ext3
> >> for example)?
> >>
> >
> > I was just saying that recommending it (ext2) compiled into the kernel and
> > not a module should be the safe route for newbies to kernel
> > compiles.
> 
> Thanks for the clarification.
> 
> > Those of us that have build a few to feel comfortable with it, will know to
> > compile the fs of our / partition into the kernel.
> >
> > Except if ext2 is not the most commonly used fs anymore.  I guess a 'cool'
> > feature could be if the make system could 'detect' what your current root is
> > and warn if you do not have that compiled into your kernel, but I do not
> > know the limitations of it (the make system).
> >
> > Then on the other hand, would above be confusing if its a kernel
> > compiled for another box ?
> 
> Yes, I'd say so, although the message could say something like:
>   Kernel does not include a filesystem for / on this computer.
> And would it also have to check the capabilities of what's in the
> initrd?  (not that I'm advocating any of this)

To me that sounds like a feature for the bootloader
config tool.  That is the first time you have a firm
kernel to rootfs connection. 

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Re: [PATCH] update filesystems config. menu

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 504 bytes --]

On Sat, 15 Mar 2003 13:31:59 PST, "Randy.Dunlap" said:

> Yes, I'd say so, although the message could say something like:
>   Kernel does not include a filesystem for / on this computer.
> And would it also have to check the capabilities of what's in the
> initrd?  (not that I'm advocating any of this)

Well... the only time *I*'ve screwed it up was building with ext3 only,
having all ext3 filesystems on the disk, and watching Bad And Ugly things
happen when the ext2-format initrd tried to mount...

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Access denied after write to named pipe

[John Elliott]

: 
: When I try create named pipe (to some directory which is 'visible' from 
: dosemu) in RDONLY mode, so after command
: ECHO HELLO > NAMED.PIP
: I got error message Access denied. Where is the problem? Permissions of 
: pipe are 0777, umask is 000, run under user ROOT.

  I suspect that echo may be trying to open the pipe in Read+Write mode;
I get the same effect with dosemu-1.0.1.0 and DRDOS 6. But if I write a 
C program to write to the pipe:

#include <unixio.h>
#include <string.h>

#define O_WRONLY 1

int main(int argc, char **argv)
{
	int n;
	int fd = open(argv[1], O_WRONLY);

	for (n = 2; n < argc; n++) 
	{
		write(fd, argv[n], strlen(argv[n]));
		write(fd, " ", 1);
	}
	close(fd);
	return 0;
}

 - then it works.  

------------- http://www.seasip.demon.co.uk/index.html --------------------
John Elliott           |BLOODNOK: "But why have you got such a long face?"
                       |SEAGOON: "Heavy dentures, Sir!"    - The Goon Show 
:-------------------------------------------------------------------------)

Re: Recent spam

[The Amazing Dragon]

> From: Manuel Krause <manuelkrause@netscape.net>
> Whats up with these rows of spam mails I receive from reiserfs list the 
> last days/weeks? Does this list spam itself now or what new anti-spam 
> algorithm did I miss?

I'd vote for it being time to start rejecting rather than merely marking
spam. At the least if the score is double the mark threshold it should be
safe (I haven't noted any errors yet though).

The recent ones actually look like a loop given the quantity...  (I'd
*hope* the spammers weren't stupid enough to think that eight copies of
the spam would be more effective than one)


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM@cs.pdx.edu      PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: Recent spam

[The Amazing Dragon]

> From: Hans Reiser <reiser@namesys.com>
> Chris Mason wrote:
> >On Thu, 2003-06-05 at 09:26, Hans Reiser wrote:
> >>This is a bit inconsiderate of you, don't you think?  Why don't you just 
> >>unsubscribe?  I have a sysadmin with too many tasks to get them done, 
> >>and he doesn't need his time wasted with such crap as dealing with spamcop.
> >
> >Honestly Hans, this has come up a bunch of times over the last few
> >months.  The reiserfs mail server setup doesn't play nicely with
> >automated spam reporting tools, and many people other than Russell use
> >them.
> >
> >There have been a ton of suggestions on ways to make things better, and
> >they range from 5 minute tasks to complex ones.  They've largely been
> >ignored.  I'd really rather not lose valuable contributors to the list
> >over something as stupid as spam.

> flx is moving our webserver and mailserver, and once he has done that I 
> will most likely pay the $25.  Unfortunately he is distracted by such 
> tasks as talking Intel into giving us a 4-way Itanium and splicing a 
> cable for it and installing it in its own room because it is noisy, 
> talking AMD into giving us some sort of opteron, installing Nina's 
> laptop, helping with the problems due to trying to use gcc 3.3, fixing 
> the powerbutton on my computer, installing a newer version of xmms that 
> can play CDDA out a USB sound device properly, replacing the fans on Mr. 
> Demidov's computer so that it can run with the case on it, etc.  Giving 
> him the additional task of dealing with spamcop is just asinine.

In other words it is time for you to hire more help? Setting SpamAssassin
to block rather than mark is a pretty simple change; setting the list
server to not strip Recieved lines is also a simple change. This has
passed the point of merely being a minor problem. Though getting $3K
pieces of hardware to test on is good, avoiding losing $3K due to all
your mail being blackholed is more important. Okay, getting a $100K piece
of hardware out of Intel might be a higher priority, but you're getting
into the danger zone where this will turn into real money being lost.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM@cs.pdx.edu      PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: How to build a big file server

[The Amazing Dragon]

> From: Carl-Daniel Hailfinger <c-d.hailfinger.kernel.2003@gmx.net>
> Heinz-Josef Claes wrote:
> >From the debian web page:
> >
> > http://packages.debian.org/testing/utils/storebackup.html
> >
> > File comparisons are done with MD5 checksums, so no changes go
> > unnoticed.
> 
> If you believe the last sentence, I have a bridge to sell.

Where, how large, what price, what is the expected income?

> To be more exact: MD5 is a 128=2^7 bit hash. Assuming a file length of 4kB
> = 2^8*4096=2^20 bits, approximately 2^(2^(20-7))= 2^8192= 10^2457
> different files have the same hash.
> 
> That's right: for a given MD5 hash, there are more different files with
> 4kB size sharing the same hash than the count of atoms in the whole
> universe. If the files are larger, it gets worse.

Yes, if you have every possible 4KB file in existance, then 4080 of those
bytes are redundant (4096 minus 16 bytes for the hash), so 2^(4080*8) of
all possible 4KB files would have the same MD5 hash. Very much larger
than the number of atoms in the universe (2^265, without dark matter),
and larger than the volume of the universe in cubic centimeters (2^280).

Think of how many unique 4KB files could exist though, 2^32768. That is
a large number. If we consider a super-huge installation, you might have
a petabyte of storage (2^50, considerably larger than any installation
I've ever heard of). Now if we figure an average file size of 1KB
(extraordinarily small), then this installation will have 2^40 separate
files. There is also the problem of the birthday paradox so once you've
covered half the bits there is a fifty percent chance of a collision, so
MD5 suddenly comes down to 64 bits of use before collision becomes a
significant danger. So we are short of endangering MD5 by 24 bits, your
chances of winning a lottery game are very much better than a 50% chance
of a single collision.

Now, the size of MD5 is small enough that there is a significant
danger of /governments/ being able to find a couple pairs of files with
identical MD5 hashes. A much larger danger is that some attacks have
gotten though weakened versions of MD5. The danger of identical files
being found by accident is utterly insignificant.

This is also assuming MD5 is used for detecting identical files, more
likely it is used to detect changes. Also do you seriously worry about
users attempting to /prevent/ their files from being backed up?

> md5sum(1) is not diff(1). Most of the time, it will suffice as el cheapo
> replacement, but for backups it's definitely horrible. You don't store
> your backup tapes in the microwave, do you?

I'd worry more about the fireproof box your tapes are in failing in a
fire than about finding a single pair of colliding files though.

You do not know enough to disparage MD5.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM@cs.pdx.edu      PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: [SPAM] BU FIRSATTAN YARARLANIN.

[Alexander Lyamin]

sorry for that one. really strange that it passed.
evaluated.

Wed, Jun 18, 2003 at 11:30:32PM +0300, Erotik Market wrote:
> ---- Start SpamAssassin results
> 8.90 points, 8 required;
> *  0.3 -- Message-Id has no @ sign
> *  4.0 -- BODY: Written in an undesired language
> *  0.2 -- BODY: 2 WHOLE LINES OF YELLING DETECTED
> *  0.2 -- BODY: A WHOLE LINE OF YELLING DETECTED
> *  0.9 -- RAW: Excessive quoted-printable encoding in body
> *  0.3 -- RAW: Quoted-printable line longer than 76 characters
> *  0.5 -- Subject is all capitals
> *  1.5 -- 'From' yahoo.com does not match 'Received' headers
> *  1.0 -- Forged mail pretending to be from MS Outlook
> 
> ---- End of SpamAssassin results
> 
> 



-- 
flx

Re: Warning message during linux kernel 2.4.21 compilation (ymfpci.c)

[Pete Zaitcev]

> Date: Sun, 22 Jun 2003 05:10:13 +0900
> From: Ishikawa <ishikawa@yk.rim.or.jp>

> It certainly looks that  something is wrong with the code.
> (Maybe some masking of the eid and the immediate value is missing?)

>    u16 eid;
> 	...

>    2462	eid = ymfpci_codec_read(codec, AC97_EXTENDED_ID);
> -->2463	if (eid==0xFFFFFF) {
> 		printk(KERN_WARNING "ymfpci: no codec attached ?\n");
> 		goto out_kfree;
> 	}

This is indeed a bug, and ymfpci inherits it from cs46xx.c.
I wonder why cs46xx.c does not trigger a warning.

Since the check never worked and nobody ever complained,
I think the best fix is to remove the "if" statement with
its block. If we repair the comparison to "eid == 0xFFFF",
then we run a risk of breaking previously working systems.

-- Pete

Re: Anycast usage, final diagnosis? (was: IPv6: Fix broken anycast usage)

[kuznet]

Hello!

Here it is. Please, review and complain.

1. Recognition of reserved anycasts is removed from ipv6_addr_type().
   Flag IPV6_ADDR_ANYCAST is removed as well.
2. Some meaningless noop code checking for anycast which are not going
   to happen is removed from ndisc.c
3. ipv6_unicast_destination() replaces suboptimal ipv6_chk_acast_addr()
   in data paths.

Alexey



# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher.
# This patch includes the following deltas:
#	           ChangeSet	1.1469  -> 1.1470 
#	  net/ipv6/anycast.c	1.5     -> 1.6    
#	include/net/ip6_route.h	1.10    -> 1.11   
#	     net/ipv6/icmp.c	1.36    -> 1.37   
#	 net/ipv6/tcp_ipv6.c	1.64    -> 1.65   
#	    net/ipv6/ndisc.c	1.52    -> 1.53   
#	    net/ipv6/route.c	1.50    -> 1.51   
#	  include/net/ipv6.h	1.22    -> 1.23   
#	 net/ipv6/addrconf.c	1.58    -> 1.59   
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 03/07/17	kuznet@oops.inr.ac.ru	1.1470
# Many files:
#   sanitize IPv6 anycast address support
# --------------------------------------------
#
diff -Nru a/include/net/ip6_route.h b/include/net/ip6_route.h
--- a/include/net/ip6_route.h	Thu Jul 17 06:13:09 2003
+++ b/include/net/ip6_route.h	Thu Jul 17 06:13:09 2003
@@ -45,7 +45,8 @@
 					   void *rtattr);
 
 extern int			ip6_rt_addr_add(struct in6_addr *addr,
-						struct net_device *dev);
+						struct net_device *dev,
+						int anycast);
 
 extern int			ip6_rt_addr_del(struct in6_addr *addr,
 						struct net_device *dev);
@@ -116,6 +117,13 @@
 	np->daddr_cache = daddr;
 	np->dst_cookie = rt->rt6i_node ? rt->rt6i_node->fn_sernum : 0;
 	write_unlock(&sk->sk_dst_lock);
+}
+
+static inline int ipv6_unicast_destination(struct sk_buff *skb)
+{
+	struct rt6_info *rt = (struct rt6_info *) skb->dst;
+
+	return rt->rt6_flags & RTF_LOCAL;
 }
 
 #endif
diff -Nru a/include/net/ipv6.h b/include/net/ipv6.h
--- a/include/net/ipv6.h	Thu Jul 17 06:13:09 2003
+++ b/include/net/ipv6.h	Thu Jul 17 06:13:09 2003
@@ -51,7 +51,7 @@
 /*
  *	Addr type
  *	
- *	type	-	unicast | multicast | anycast
+ *	type	-	unicast | multicast
  *	scope	-	local	| site	    | global
  *	v4	-	compat
  *	v4mapped
@@ -63,7 +63,6 @@
 
 #define IPV6_ADDR_UNICAST      	0x0001U	
 #define IPV6_ADDR_MULTICAST    	0x0002U	
-#define IPV6_ADDR_ANYCAST	0x0004U
 
 #define IPV6_ADDR_LOOPBACK	0x0010U
 #define IPV6_ADDR_LINKLOCAL	0x0020U
diff -Nru a/net/ipv6/addrconf.c b/net/ipv6/addrconf.c
--- a/net/ipv6/addrconf.c	Thu Jul 17 06:13:09 2003
+++ b/net/ipv6/addrconf.c	Thu Jul 17 06:13:09 2003
@@ -209,15 +209,8 @@
 		};
 		return type;
 	}
-	/* check for reserved anycast addresses */
-	
-	if ((st & htonl(0xE0000000)) &&
-	    ((addr->s6_addr32[2] == htonl(0xFDFFFFFF) &&
-	    (addr->s6_addr32[3] | htonl(0x7F)) == (u32)~0) ||
-	    (addr->s6_addr32[2] == 0 && addr->s6_addr32[3] == 0)))
-		type = IPV6_ADDR_ANYCAST;
-	else
-		type = IPV6_ADDR_UNICAST;
+
+	type = IPV6_ADDR_UNICAST;
 
 	/* Consider all addresses with the first three bits different of
 	   000 and 111 as finished.
@@ -2552,7 +2545,7 @@
 
 	switch (event) {
 	case RTM_NEWADDR:
-		ip6_rt_addr_add(&ifp->addr, ifp->idev->dev);
+		ip6_rt_addr_add(&ifp->addr, ifp->idev->dev, 0);
 		break;
 	case RTM_DELADDR:
 		addrconf_leave_solict(ifp->idev->dev, &ifp->addr);
diff -Nru a/net/ipv6/anycast.c b/net/ipv6/anycast.c
--- a/net/ipv6/anycast.c	Thu Jul 17 06:13:09 2003
+++ b/net/ipv6/anycast.c	Thu Jul 17 06:13:09 2003
@@ -96,6 +96,13 @@
 	return onlink;
 }
 
+static inline ipv6_reserved_anycast(const struct in6_addr *addr)
+{
+	return (addr->s6_addr32[0] & htonl(0xE0000000)) &&
+		((addr->s6_addr32[2] == htonl(0xFDFFFFFF) &&
+		  (addr->s6_addr32[3] | htonl(0x7F)) == (u32)~0) ||
+		 (addr->s6_addr32[2] == 0 && addr->s6_addr32[3] == 0));
+}
 
 /*
  *	socket join an anycast group
@@ -112,6 +119,8 @@
 
 	if (ipv6_addr_type(addr) & IPV6_ADDR_MULTICAST)
 		return -EINVAL;
+	if (ipv6_chk_addr(addr, NULL))
+		return -EINVAL;
 
 	pac = sock_kmalloc(sk, sizeof(struct ipv6_ac_socklist), GFP_KERNEL);
 	if (pac == NULL)
@@ -172,8 +181,7 @@
 			err = -EPERM;
 		if (err)
 			goto out_dev_put;
-	} else if (!(ipv6_addr_type(addr) & IPV6_ADDR_ANYCAST) &&
-		   !capable(CAP_NET_ADMIN)) {
+	} else if (!ipv6_reserved_anycast(addr) && !capable(CAP_NET_ADMIN)) {
 		err = -EPERM;
 		goto out_dev_put;
 	}
@@ -347,7 +355,7 @@
 	idev->ac_list = aca;
 	write_unlock_bh(&idev->lock);
 
-	ip6_rt_addr_add(&aca->aca_addr, dev);
+	ip6_rt_addr_add(&aca->aca_addr, dev, 1);
 
 	addrconf_join_solict(dev, &aca->aca_addr);
 
diff -Nru a/net/ipv6/icmp.c b/net/ipv6/icmp.c
--- a/net/ipv6/icmp.c	Thu Jul 17 06:13:09 2003
+++ b/net/ipv6/icmp.c	Thu Jul 17 06:13:09 2003
@@ -415,8 +415,7 @@
 
 	saddr = &skb->nh.ipv6h->daddr;
 
-	if (ipv6_addr_type(saddr) & IPV6_ADDR_MULTICAST ||
-	    ipv6_chk_acast_addr(0, saddr)) 
+	if (!ipv6_unicast_destination(skb))
 		saddr = NULL;
 
 	memcpy(&tmp_hdr, icmph, sizeof(tmp_hdr));
diff -Nru a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c
--- a/net/ipv6/ndisc.c	Thu Jul 17 06:13:09 2003
+++ b/net/ipv6/ndisc.c	Thu Jul 17 06:13:09 2003
@@ -785,8 +785,7 @@
 			ipv6_addr_all_nodes(&maddr);
 			ndisc_send_na(dev, NULL, &maddr, &ifp->addr, 
 				      ifp->idev->cnf.forwarding, 0, 
-				      ipv6_addr_type(&ifp->addr)&IPV6_ADDR_ANYCAST ? 0 : 1, 
-				      1);
+				      1, 1);
 			in6_ifa_put(ifp);
 			return;
 		}
@@ -809,8 +808,7 @@
 			if (neigh || !dev->hard_header) {
 				ndisc_send_na(dev, neigh, saddr, &ifp->addr, 
 					      ifp->idev->cnf.forwarding, 1, 
-					      ipv6_addr_type(&ifp->addr)&IPV6_ADDR_ANYCAST ? 0 : 1, 
-					      1);
+					      1, 1);
 				if (neigh)
 					neigh_release(neigh);
 			}
diff -Nru a/net/ipv6/route.c b/net/ipv6/route.c
--- a/net/ipv6/route.c	Thu Jul 17 06:13:09 2003
+++ b/net/ipv6/route.c	Thu Jul 17 06:13:09 2003
@@ -1256,7 +1256,7 @@
  *	Add address
  */
 
-int ip6_rt_addr_add(struct in6_addr *addr, struct net_device *dev)
+int ip6_rt_addr_add(struct in6_addr *addr, struct net_device *dev, int anycast)
 {
 	struct rt6_info *rt = ip6_dst_alloc();
 
@@ -1275,6 +1275,8 @@
 	rt->u.dst.obsolete = -1;
 
 	rt->rt6i_flags = RTF_UP | RTF_NONEXTHOP;
+	if (!anycast)
+		rt->rt6i_flags |= RTF_LOCAL;
 	rt->rt6i_nexthop = ndisc_get_neigh(rt->rt6i_dev, &rt->rt6i_gateway);
 	if (rt->rt6i_nexthop == NULL) {
 		dst_free((struct dst_entry *) rt);
diff -Nru a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c
--- a/net/ipv6/tcp_ipv6.c	Thu Jul 17 06:13:09 2003
+++ b/net/ipv6/tcp_ipv6.c	Thu Jul 17 06:13:09 2003
@@ -971,7 +971,7 @@
 	if (th->rst)
 		return;
 
-	if (ipv6_addr_is_multicast(&skb->nh.ipv6h->daddr))
+	if (!ipv6_unicast_destination(skb))
 		return; 
 
 	/*
@@ -1175,8 +1175,7 @@
 	if (skb->protocol == htons(ETH_P_IP))
 		return tcp_v4_conn_request(sk, skb);
 
-	/* FIXME: do the same check for anycast */
-	if (ipv6_addr_is_multicast(&skb->nh.ipv6h->daddr))
+	if (!ipv6_unicast_destination(skb))
 		goto drop; 
 
 	/*

Re: Anycast usage, final diagnosis? (was: IPv6: Fix broken anycast usage)

[David S. Miller]

On Thu, 17 Jul 2003 06:23:52 +0400 (MSD)
kuznet@ms2.inr.ac.ru wrote:

> Here it is. Please, review and complain.

If Pekka agrees with semantics, the patch looks sound
by my eyes.

Re: Anycast usage, final diagnosis? (was: IPv6: Fix broken anycast usage)

[Mika Liljeberg]

Alexey,

On Thu, 2003-07-17 at 05:23, kuznet@ms2.inr.ac.ru wrote:
> diff -Nru a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c
> --- a/net/ipv6/ndisc.c	Thu Jul 17 06:13:09 2003
> +++ b/net/ipv6/ndisc.c	Thu Jul 17 06:13:09 2003
> @@ -785,8 +785,7 @@
>  			ipv6_addr_all_nodes(&maddr);
>  			ndisc_send_na(dev, NULL, &maddr, &ifp->addr, 
>  				      ifp->idev->cnf.forwarding, 0, 
> -				      ipv6_addr_type(&ifp->addr)&IPV6_ADDR_ANYCAST ? 0 : 1, 
> -				      1);
> +				      1, 1);
>  			in6_ifa_put(ifp);
>  			return;
>  		}
> @@ -809,8 +808,7 @@
>  			if (neigh || !dev->hard_header) {
>  				ndisc_send_na(dev, neigh, saddr, &ifp->addr, 
>  					      ifp->idev->cnf.forwarding, 1, 
> -					      ipv6_addr_type(&ifp->addr)&IPV6_ADDR_ANYCAST ? 0 : 1, 
> -					      1);
> +					      1, 1);
>  				if (neigh)
>  					neigh_release(neigh);
>  			}

I'm not sure you can just remove these. It seems possible (?) to have
the anycast address configured on one of the interfaces as a unicast at
the same time. I.e., one of the anycast members could own the address.

For what it's worth, I think you have the right semantics.

	MikaL

Re: Anycast usage, final diagnosis? (was: IPv6: Fix broken anycast

[kuznet]

Hello!

> I'm not sure you can just remove these. It seems possible (?) to have
> the anycast address configured on one of the interfaces as a unicast at
> the same time. I.e., one of the anycast members could own the address.

They cannot intersect, otherwise RTF_LOCAL thing will not work.

I deliberately blocked attempt to add a local address as anycast
in anycast.c, see another chunk. But even that check is not necessary:
non-superuser may listen only for reserved unicasts, which are
excluded from allowed local addresses by policy. Kernel does not need even
to worry about this.

Actually, the test in ndisc.c was bogus by another reason:
inet_addr_type() checks only for reserved anycasts and non-reserved unicasts,
which would conflict with local addresses, were not detected in any case.

Alexey

Re: Anycast usage, final diagnosis? (was: IPv6: Fix broken anycast

[Mika Liljeberg]

On Thu, 2003-07-17 at 12:06, kuznet@ms2.inr.ac.ru wrote:
> They cannot intersect, otherwise RTF_LOCAL thing will not work.
> 
> I deliberately blocked attempt to add a local address as anycast
> in anycast.c, see another chunk.

Ok, I missed that one.

I guess it's safe to assume that the anycast and unicast spaces will not
intersect, even though the addresses are allocated from the same range.
I was wondering how to dynamically assign anycast addresses. In theory
one could abuse the unicast address assignment mechanisms (in the
absence of anything else users might be tempted to try this). But that's
a different issue.

	MikaL

Re: [parisc-linux] Re: [glibc] tststatic failues, reduced to simp le testcase.

[John David Anglin]

> > I've added all the caller-saves registers to our clobber lists when
> > making syscalls. Though, r19 being special, I've had to add "STW_PIC"
> > and "LDW_PIC" that do the saving and restore _only_ if we are compiled
> > PIC.
> 
> That seems overly pessimistic and will mean a lot more register saves
> in routines that do syscalls.  It appears the system saves the caller-save
> registers in all syscalls except the fork related calls.  It would
> be best to keep the number of clobbers to a minimum.  I wonder, there
> seems to be a slot in the task struct for r1, but it's never saved in
> a syscall, and r1 is clobbered.  Maybe the r1 slot could be used to save
> r2.  This might mean that we don't need to use GR21 or GR22 at all.
> It would avoid having to have special pic code.

Another possibility might be to define another couple of offsets in the
task struct.  It looks as if there is plenty of space before an alignment
boundary is reached.

Re: [parisc-linux] Re: [glibc] tststatic failues, reduced to simp le testcase.

[Carlos O'Donell]

> Another possibility might be to define another couple of offsets in the
> task struct.  It looks as if there is plenty of space before an alignment
> boundary is reached.

That seems best. The sad fact though is that until I bump
min_kernel_version in glibc past the kernel version that last didn't
have this fix, I'm still forced to implement all this for people with
older kernels.

c.

Re: [parisc-linux] Re: [glibc] tststatic failues, reduced to simp le testcase.

[John David Anglin]

> That seems best. The sad fact though is that until I bump
> min_kernel_version in glibc past the kernel version that last didn't
> have this fix, I'm still forced to implement all this for people with
> older kernels.

Why?  Things are broken now and people have lived with the bug for
sometime.  While it is possible to work around the problem in userspace,
the best fix is to do it in the kernel.  I wouldn't fix anything but
a regression.

Dave

Re: [parisc-linux] Re: [glibc] tststatic failues, reduced to simp le testcase.

[Carlos O'Donell]

On Sun, Aug 31, 2003 at 04:47:59PM -0400, John David Anglin wrote:
> > That seems best. The sad fact though is that until I bump
> > min_kernel_version in glibc past the kernel version that last didn't
> > have this fix, I'm still forced to implement all this for people with
> > older kernels.
> 
> Why?  Things are broken now and people have lived with the bug for
> sometime.  While it is possible to work around the problem in userspace,
> the best fix is to do it in the kernel.  I wouldn't fix anything but
> a regression.

It's best to use an example. Debian supports HPPA as an architecture.
I provide Debian with patches to make upstream glibc cvs buildable for
this distribution. If I make all the glibc fixes in the kernel, and
submit to debian minimal patches, glibc will build and subsequent
updates will render all userspace broken. As a happy medium I need to
make, atleast for all usable distributions, a set of patches that fix it
in such a way that we don't force a kernel update (people don't like to
update their kernel). The patches I send upstream will be different.
They will, to the best of my ability, be as clean as possible. I balance
users on one side, and the "right thing" on the other.

I agree, that the kernel, sharing the context switch path, and already
saving r19 should be held responsible for restoring r19. I've already
begun to run into a number of cases where userspace restores aren't
going so well.

c.

Re: [Bluez-users] IBM Thinkpad A31p and Bluetooth

[Jason Van Patten]

At Sat Sep 13 01:15:17 2003, jvp wrote:
> laptop# hciconfig -a
> hci0:   Type: USB 
>         BD Address: 00:00:00:00:00:00 ACL MTU: 0:0  SCO MTU: 0:0
>         DOWN 
>         RX bytes:0 acl:0 sco:0 events:0 errors:0
>         TX bytes:0 acl:0 sco:0 commands:0 errors:0
>         Features: 0x00 0x00 0x00 0x00
>         Packet type: DM1 DH1 HV1 
>         Link policy: 
>         Link mode: SLAVE ACCEPT 

I was successful in getting the Palm and laptop to find one another via BT;
thanks a bunch for all the help.  Now, is there any way to get an
application like jpilot to sync over BT?  I need to point it at a /dev
device; typically a serial or usb connection.  Does hci0 have a /dev
representation of some sort?

Thanks, and again I'm sorry about the newbie questions.

jas
-- 
Jason Van Patten
AOL IM: Jason VP 



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Bluez-users mailing list
Bluez-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bluez-users

Re: [Bluez-users] IBM Thinkpad A31p and Bluetooth

[Fredrik Noring]

Hi Jason,

l=F6r 2003-09-13 klockan 08.12 skrev Jason Van Patten:
> I was successful in getting the Palm and laptop to find one another via=
 BT;
> thanks a bunch for all the help.  Now, is there any way to get an
> application like jpilot to sync over BT? =20

I don't know if it's possible to get jpilot or gnome-pilot running with
Bluetooth, but pilot-xfer works atleast:

     1) Make your Palm able to connect to the computer using=20
        PPP over Bluetooth. (This is somewhat complicated - search for
        "linux bluetooth ppp tungsten" or similar to get some howtos.)

     2) Run "pilot-xfer -p net:any -s ." etc.=20

The Gnome MultiSync project looks interesting but I don't know if Palm
is supported. The Gnome Bluetooth project also looks promising.

	Fredrik




-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Bluez-users mailing list
Bluez-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bluez-users

Re: [Bluez-users] IBM Thinkpad A31p and Bluetooth

[Marcel Holtmann]

Hi Fredrik,

> > I was successful in getting the Palm and laptop to find one another via BT;
> > thanks a bunch for all the help.  Now, is there any way to get an
> > application like jpilot to sync over BT?  
> 
> I don't know if it's possible to get jpilot or gnome-pilot running with
> Bluetooth, but pilot-xfer works atleast:
> 
>      1) Make your Palm able to connect to the computer using 
>         PPP over Bluetooth. (This is somewhat complicated - search for
>         "linux bluetooth ppp tungsten" or similar to get some howtos.)
> 
>      2) Run "pilot-xfer -p net:any -s ." etc. 
> 
> The Gnome MultiSync project looks interesting but I don't know if Palm
> is supported. The Gnome Bluetooth project also looks promising.

I don't have a Palm, but I assume that this device can also sync over
Bluetooth directly, because the only thing that it needs is a RFCOMM
connection. The SDP records of the Palm are not in the public browse
group, so it will be difficult to find the correct one and of course
they will use some uuid128. The Nokia phone 6310 for example have also a
special service record that allows them to run their own protocol over
Bluetooth and we now have to find the special SDP record for the Palm.
Please, can someone post the result of a SDP search with --tree option
for the OBEX push profile.

Regards

Marcel




-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Bluez-users mailing list
Bluez-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bluez-users

Re: [Bluez-users] IBM Thinkpad A31p and Bluetooth

[Fredrik Noring]

lör 2003-09-13 klockan 12.32 skrev Marcel Holtmann:
> Please, can someone post the result of a SDP search with --tree option
> for the OBEX push profile.

Sure. Does the result indicate anything?

	Fredrik

# sdptool search --tree OPUSH 00:07:E0:0C:1B:3F
Inquiring ...
Searching for OPUSH on 00:07:E0:0C:1B:3F ...
Attribute Identifier : 0x0 - ServiceRecordHandle
  Integer : 0x10001
Attribute Identifier : 0x1 - ServiceClassIDList
  Data Sequence
    UUID16 : 0x1105 - OBEXObjectPush
Attribute Identifier : 0x4 - ProtocolDescriptorList
  Data Sequence
    Data Sequence
      UUID16 : 0x0100 - L2CAP
    Data Sequence
      UUID16 : 0x0003 - RFCOMM
      Channel/Port (Integer) : 0x1
    Data Sequence
      UUID16 : 0x0008 - OBEX
Attribute Identifier : 0x6 - LanguageBaseAttributeIDList
  Data Sequence
    Integer : 0x656e
    Integer : 0x8cc
    Integer : 0x100
Attribute Identifier : 0x100
  Text : "OBEX Object Push"
Attribute Identifier : 0x303
  Data Sequence
    Integer : 0x1
    Integer : 0x2
    Integer : 0x3
    Integer : 0xff

Re: [Bluez-users] IBM Thinkpad A31p and Bluetooth

[Marcel Holtmann]

Hi Fredrik,

> > Please, can someone post the result of a SDP search with --tree option
> > for the OBEX push profile.
> 
> # sdptool search --tree OPUSH 00:07:E0:0C:1B:3F
> Inquiring ...
> Searching for OPUSH on 00:07:E0:0C:1B:3F ...
> Attribute Identifier : 0x0 - ServiceRecordHandle
>   Integer : 0x10001

seems that Palm uses default record handle number scheme. Please check
the records 0x10000 and onwards for values.

Regards

Marcel




-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Bluez-users mailing list
Bluez-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bluez-users

Re: [Bluez-users] IBM Thinkpad A31p and Bluetooth

[Fredrik Noring]

lör 2003-09-13 klockan 19.56 skrev Marcel Holtmann:
> seems that Palm uses default record handle number scheme. Please check
> the records 0x10000 and onwards for values.

Check records - how?

	Fredrik

Re: [Bluez-users] IBM Thinkpad A31p and Bluetooth

[Marcel Holtmann]

Hi Fredrik,

> > seems that Palm uses default record handle number scheme. Please check
> > the records 0x10000 and onwards for values.
> 
> Check records - how?

	sdptool get --tree --bdaddr <bdaddr> 0x10000
	sdptool get --tree --bdaddr <bdaddr> 0x10001
	sdptool get --tree --bdaddr <bdaddr> 0x10002
	...

Regards

Marcel




-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Bluez-users mailing list
Bluez-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bluez-users

Re: [Bluez-users] IBM Thinkpad A31p and Bluetooth

[Fredrik Noring]

Hi Marcel,

sön 2003-09-14 klockan 15.26 skrev Marcel Holtmann:
> 	sdptool get --tree --bdaddr <bdaddr> 0x10000
> 	...

I've tested 0x10000 through 0x1007f with interesting results on 0x10000
and 0x10001.

Thanks,
	Fredrik

Probing 0x10000...
Attribute Identifier : 0x0 - ServiceRecordHandle
  Integer : 0x10000
Attribute Identifier : 0x1 - ServiceClassIDList
  Data Sequence
    UUID128 : 0xf5beb651-4171-4051-acf5-6ca72022-42f0

Probing 0x10001...
Attribute Identifier : 0x0 - ServiceRecordHandle
  Integer : 0x10001
Attribute Identifier : 0x1 - ServiceClassIDList
  Data Sequence
    UUID16 : 0x1105 - OBEXObjectPush
Attribute Identifier : 0x4 - ProtocolDescriptorList
  Data Sequence
    Data Sequence
      UUID16 : 0x0100 - L2CAP
    Data Sequence
      UUID16 : 0x0003 - RFCOMM
      Channel/Port (Integer) : 0x1
    Data Sequence
      UUID16 : 0x0008 - OBEX
Attribute Identifier : 0x6 - LanguageBaseAttributeIDList
  Data Sequence
    Integer : 0x656e
    Integer : 0x8cc
    Integer : 0x100
Attribute Identifier : 0x100
  Text : "OBEX Object Push"
Attribute Identifier : 0x303
  Data Sequence
    Integer : 0x1
    Integer : 0x2
    Integer : 0x3
    Integer : 0xff

Probing 0x10002...
Service get request failed.

...

Re: [Bluez-users] IBM Thinkpad A31p and Bluetooth

[Jason Van Patten]

At Sat Sep 13 04:28:13 2003, Fredrik Noring wrote:
> I don't know if it's possible to get jpilot or gnome-pilot running with
> Bluetooth, but pilot-xfer works atleast:
> 
>      1) Make your Palm able to connect to the computer using 
>         PPP over Bluetooth. (This is somewhat complicated - search for
>         "linux bluetooth ppp tungsten" or similar to get some howtos.)
> 
>      2) Run "pilot-xfer -p net:any -s ." etc. 

Thanks for the pointer, that seems to have done the trick.  Jpilot will use
net:any as a device to sync the Palm after the PPP-over-BT is configured.

It's a few extra steps than I wanted to take, but oh well.  It works.

Thanks!

jas
-- 
Jason Van Patten
AOL IM: Jason VP 



-------------------------------------------------------
This sf.net email is sponsored by:ThinkGeek
Welcome to geek heaven.
http://thinkgeek.com/sf
_______________________________________________
Bluez-users mailing list
Bluez-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bluez-users

[parisc-linux] dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1: undefined symbol: ps_pglobal_lookup

[John David Anglin]

I hacked together of a build of debian gdb-6.0-2 this afternoon.  When
I start gdb, I see the following error:

dave@gsyprf11:~/gdb-6.0-2/objdir$ gdb/gdb


dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1: undefined symbol: ps_pglobal_lookup
GDB will not be able to debug pthreads.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

[parisc-linux] Re: dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1: undefined symbol: ps_pglobal_lookup

[Carlos O'Donell]

On Mon, Dec 15, 2003 at 05:05:12PM -0500, John David Anglin wrote:
> I hacked together of a build of debian gdb-6.0-2 this afternoon.  When
> I start gdb, I see the following error:

My friend, that's amazing!
 
> dave@gsyprf11:~/gdb-6.0-2/objdir$ gdb/gdb
> dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1: undefined symbol: ps_pglobal_lookup
> GDB will not be able to debug pthreads.

Every pogram that links libthread_db.so.1 must provide a set of process
control primities that will allow libthread_db to access memory and
registers in the target process, start and stop the process, and lookup
symbols. ps_pglobal_lookup is always UND in libthread_db, and gdb should
have exported an arch version for the library to use :)

c.

Re: [parisc-linux] Re: dlopen failed on 'libthread_db.so.1' - /lib/libthread_db.so.1: undefined symbol: ps_pglobal_lookup

[Carlos O'Donell]

On Wed, Dec 17, 2003 at 10:32:20AM -0500, Carlos O'Donell wrote:
> Every pogram that links libthread_db.so.1 must provide a set of process
> control primities that will allow libthread_db to access memory and
> registers in the target process, start and stop the process, and lookup
> symbols. ps_pglobal_lookup is always UND in libthread_db, and gdb should
> have exported an arch version for the library to use :)

See:

gdb/proc-service.c
gdb/gdbserver/proc-service.c

*unless* You are insinuating that the loader has messed up the symbol
resolution and ps_pglobal_lookup exists but isn't resolved properly...
which is a whole 'nother can of wormns :)

c.

Re: [parisc-linux] Re: dlopen failed on 'libthread_db.so.1' -

[John David Anglin]

> See:
> 
> gdb/proc-service.c
> gdb/gdbserver/proc-service.c
> 
> *unless* You are insinuating that the loader has messed up the symbol
> resolution and ps_pglobal_lookup exists but isn't resolved properly...
> which is a whole 'nother can of wormns :)

I'll take a look.  Based on what you have said, it's likely the gdb-6.0
isn't being linked correctly on hppa-linux.  Debian gdb-6.0-2 doesn't
build as is, and I had to make some minor mod's to the patch set and hack 
a config file to include linux-nat.o in the required files.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: [parisc-linux] Re: dlopen failed on 'libthread_db.so.1' -

[Carlos O'Donell]

On Wed, Dec 17, 2003 at 11:43:48AM -0500, John David Anglin wrote:
> > See:
> > 
> > gdb/proc-service.c
> > gdb/gdbserver/proc-service.c
> > 
> > *unless* You are insinuating that the loader has messed up the symbol
> > resolution and ps_pglobal_lookup exists but isn't resolved properly...
> > which is a whole 'nother can of wormns :)
> 
> I'll take a look.  Based on what you have said, it's likely the gdb-6.0
> isn't being linked correctly on hppa-linux.  Debian gdb-6.0-2 doesn't
> build as is, and I had to make some minor mod's to the patch set and hack 
> a config file to include linux-nat.o in the required files.

AFAIK there was major work required. I shy'd away completely as glibc
was taking up all my time and seemed like a better place to start :)

Current todo:

- Giving up on fixing dlopen from static app, too tough for me right
  now.
- Atomic ups are up next, with possibly a review of time functions,
  monotonicity and a fast gettimeofday.

c.

Re: [parisc-linux] Re: dlopen failed on 'libthread_db.so.1' -

[John David Anglin]

> On Wed, Dec 17, 2003 at 11:43:48AM -0500, John David Anglin wrote:
> > > See:
> > > 
> > > gdb/proc-service.c
> > > gdb/gdbserver/proc-service.c
> > > 
> > > *unless* You are insinuating that the loader has messed up the symbol
> > > resolution and ps_pglobal_lookup exists but isn't resolved properly...
> > > which is a whole 'nother can of wormns :)
> > 
> > I'll take a look.  Based on what you have said, it's likely the gdb-6.0
> > isn't being linked correctly on hppa-linux.  Debian gdb-6.0-2 doesn't
> > build as is, and I had to make some minor mod's to the patch set and hack 
> > a config file to include linux-nat.o in the required files.

dave@gsyprf11:~/gdb-6.0-2/objdir/gdb$ nm gdb|grep ps_pglobal_lookup
000dc904 T ps_pglobal_lookup

Looks as if there is a problem with dlopen.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: [parisc-linux] Re: dlopen failed on 'libthread_db.so.1' -

[John David Anglin]

> dave@gsyprf11:~/gdb-6.0-2/objdir/gdb$ nm gdb|grep ps_pglobal_lookup
> 000dc904 T ps_pglobal_lookup
> 
> Looks as if there is a problem with dlopen.

No, the problem is ps_pglobal_lookup isn't in gdb's dynamic symbol
table.  I'll have to figure out how this is done.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: [parisc-linux] Re: dlopen failed on 'libthread_db.so.1' -g

[John David Anglin]

> No, the problem is ps_pglobal_lookup isn't in gdb's dynamic symbol
> table.  I'll have to figure out how this is done.

Adding --export-dynamic to the gdb link command resolves the problem.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: Adaptec DPT_I2O Driver

[Jason Van Patten]

At Thu Dec 18 08:27:32 2003, jvp wrote:
> In the 2.6 tree, the driver drivers/scsi/dpt_i2o.c has a #error in it that
> prevents it from being compiled.  The note in the file is:
> 
> #error Please convert me to Documentation/DMA-mapping.txt

Can I assume that because no one's replied to my question regarding this
broken driver, that no one's currently working on it?

jas
-- 
Jason Van Patten
AOL IM: Jason VP 

Re: SCO's infringing files list

[Rick Bressler]

> Does anybody have old CD-ROM's lying around?

I have some original Slackware CD's:

slackware.3.2.lst:DISK2: /cdrom/sunsite/libs/oldlibs/libc-2.2.2/00_index.txt
slackware.3.2.lst:DISK2: /cdrom/sunsite/libs/oldlibs/libc-2.2.2/00_TRANS.TBL
slackware.3.2.lst:DISK2: /cdrom/sunsite/libs/oldlibs/libc-2.2.2/libc.2.2.2.z
slackware.3.2.lst:DISK2: /cdrom/sunsite/libs/oldlibs/libc-2.2.2/libm.2.2.2.z
slackware.3.4.lst:DISK3: sunsite/libs/oldlibs/libc-2.2.2
slackware.3.4.lst:DISK3: sunsite/libs/oldlibs/libc-2.2.2/00_index.txt
slackware.3.4.lst:DISK3: sunsite/libs/oldlibs/libc-2.2.2/libc.2.2.2.z
slackware.3.4.lst:DISK3: sunsite/libs/oldlibs/libc-2.2.2/libm.2.2.2.z

If you want them as 'evidence' I will be happy to mail them to you. 



-- 
Rick Bressler                                      rick@the-bresslers.com
   Eagles may soar, but weasels never get sucked into jet air intakes.

Re: [parisc-linux] [Testers wanted] New glibc with profiling fixed.

[John David Anglin]

> Did I miss something?

Yup.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: [parisc-linux] [Testers wanted] New glibc with profiling fixed.

[John David Anglin]

> Did I miss something?

Yup.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: Conversion from v3.6 to v4 of Reiserfs...

[The Amazing Dragon]

> From: "Dr. Giovanni A. Orlando" <gorlando@futuretg.com>
>     Actually, there are no a way. You need to backup and re-write it 
> your data.
> 
>     However, I speak with Hans and probably we will release a 
> tool/utility for that purpose.

Something like dump?


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM@cs.pdx.edu      PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: Conversion from v3.6 to v4 of Reiserfs...

[Nikita Danilov]

The Amazing Dragon (Elliott Mitchell) writes:
 > > From: "Dr. Giovanni A. Orlando" <gorlando@futuretg.com>
 > >     Actually, there are no a way. You need to backup and re-write it 
 > > your data.
 > > 
 > >     However, I speak with Hans and probably we will release a 
 > > tool/utility for that purpose.
 > 
 > Something like dump?

Cf. "Universal file system convertor":
http://tzukanov.narod.ru/convertfs/

no warranty, especially given that loop-back support required for
convertfs was added into reiser4 only recently.

 > 
 > 

Nikita.

Re: Conversion from v3.6 to v4 of Reiserfs...

[Dr. Giovanni A. Orlando]

Nikita Danilov wrote:

>The Amazing Dragon (Elliott Mitchell) writes:
> > > From: "Dr. Giovanni A. Orlando" <gorlando@futuretg.com>
> > >     Actually, there are no a way. You need to backup and re-write it 
> > > your data.
> > > 
> > >     However, I speak with Hans and probably we will release a 
> > > tool/utility for that purpose.
> > 
> > Something like dump?
>
>Cf. "Universal file system convertor":
>http://tzukanov.narod.ru/convertfs/
>
>no warranty, especially given that loop-back support required for
>convertfs was added into reiser4 only recently.
>  
>
Hi,

       We plan to support in our distro ReiserFS 3.x and 4.0 at the same 
time.

       About to port OLD data to ReiserFS 4, a simple 'tar cvfz' will be 
usefull to backup and replace.

       I don't think, this tool

http://tzukanov.narod.ru/convertfs/

    support reiserFS 4.0, because it is dated March 2002.

    If someone knows some other link, for this tool I can take a look.

    Seems that Debian is working on. In any case we will do our tests and make
    our decisions.

    My idea when i comment to Hans, was based on the fact that RedHat haves these tools
    for EXT2 to EXT3 and viceversa.

Thanks,
Giovanni

> > 
> > 
>
>Nikita.
>
>
>  
>

Re: Conversion from v3.6 to v4 of Reiserfs...

[Nikita Danilov]

Dr. Giovanni A. Orlando writes:
 > Nikita Danilov wrote:
 > 
 > >The Amazing Dragon (Elliott Mitchell) writes:
 > > > > From: "Dr. Giovanni A. Orlando" <gorlando@futuretg.com>
 > > > >     Actually, there are no a way. You need to backup and re-write it 
 > > > > your data.
 > > > > 
 > > > >     However, I speak with Hans and probably we will release a 
 > > > > tool/utility for that purpose.
 > > > 
 > > > Something like dump?
 > >
 > >Cf. "Universal file system convertor":
 > >http://tzukanov.narod.ru/convertfs/
 > >
 > >no warranty, especially given that loop-back support required for
 > >convertfs was added into reiser4 only recently.
 > >  
 > >
 > Hi,
 > 
 >        We plan to support in our distro ReiserFS 3.x and 4.0 at the same 
 > time.
 > 
 >        About to port OLD data to ReiserFS 4, a simple 'tar cvfz' will be 
 > usefull to backup and replace.
 > 
 >        I don't think, this tool
 > 
 > http://tzukanov.narod.ru/convertfs/
 > 
 >     support reiserFS 4.0, because it is dated March 2002.

convertfs does _not_ need special support for new file system type. It
can convert arbitrary file system foo to arbitrary file system bar, on
the condition that bar supports loop back files.

 > 
 >     If someone knows some other link, for this tool I can take a look.

Nikita.

Re: Conversion from v3.6 to v4 of Reiserfs...

[Nikita Danilov]

Nikita Danilov writes:
 > Dr. Giovanni A. Orlando writes:
 >  > Nikita Danilov wrote:
 >  > 
 >  > >The Amazing Dragon (Elliott Mitchell) writes:
 >  > > > > From: "Dr. Giovanni A. Orlando" <gorlando@futuretg.com>
 >  > > > >     Actually, there are no a way. You need to backup and re-write it 
 >  > > > > your data.
 >  > > > > 
 >  > > > >     However, I speak with Hans and probably we will release a 
 >  > > > > tool/utility for that purpose.
 >  > > > 
 >  > > > Something like dump?
 >  > >
 >  > >Cf. "Universal file system convertor":
 >  > >http://tzukanov.narod.ru/convertfs/
 >  > >
 >  > >no warranty, especially given that loop-back support required for
 >  > >convertfs was added into reiser4 only recently.
 >  > >  
 >  > >
 >  > Hi,
 >  > 
 >  >        We plan to support in our distro ReiserFS 3.x and 4.0 at the same 
 >  > time.
 >  > 
 >  >        About to port OLD data to ReiserFS 4, a simple 'tar cvfz' will be 
 >  > usefull to backup and replace.
 >  > 
 >  >        I don't think, this tool
 >  > 
 >  > http://tzukanov.narod.ru/convertfs/
 >  > 
 >  >     support reiserFS 4.0, because it is dated March 2002.
 > 
 > convertfs does _not_ need special support for new file system type. It
 > can convert arbitrary file system foo to arbitrary file system bar, on
 > the condition that bar supports loop back files.

Sorry, I got it backward. Loop back support on foo is required. So, to
convert reiserfs to reiser4 one needs loop back on reiserfs, which is
not a problem.

 > 
 >  > 
 >  >     If someone knows some other link, for this tool I can take a look.
 > 

Nikita.

 > 

Re: Conversion from v3.6 to v4 of Reiserfs...

[Dr. Giovanni A. Orlando]

Nikita Danilov wrote:

>Nikita Danilov writes:
> > Dr. Giovanni A. Orlando writes:
> >  > Nikita Danilov wrote:
> >  > 
> >  > >The Amazing Dragon (Elliott Mitchell) writes:
> >  > > > > From: "Dr. Giovanni A. Orlando" <gorlando@futuretg.com>
> >  > > > >     Actually, there are no a way. You need to backup and re-write it 
> >  > > > > your data.
> >  > > > > 
> >  > > > >     However, I speak with Hans and probably we will release a 
> >  > > > > tool/utility for that purpose.
> >  > > > 
> >  > > > Something like dump?
> >  > >
> >  > >Cf. "Universal file system convertor":
> >  > >http://tzukanov.narod.ru/convertfs/
> >  > >
> >  > >no warranty, especially given that loop-back support required for
> >  > >convertfs was added into reiser4 only recently.
> >  > >  
> >  > >
> >  > Hi,
> >  > 
> >  >        We plan to support in our distro ReiserFS 3.x and 4.0 at the same 
> >  > time.
> >  > 
> >  >        About to port OLD data to ReiserFS 4, a simple 'tar cvfz' will be 
> >  > usefull to backup and replace.
> >  > 
> >  >        I don't think, this tool
> >  > 
> >  > http://tzukanov.narod.ru/convertfs/
> >  > 
> >  >     support reiserFS 4.0, because it is dated March 2002.
> > 
> > convertfs does _not_ need special support for new file system type. It
> > can convert arbitrary file system foo to arbitrary file system bar, on
> > the condition that bar supports loop back files.
>
>Sorry, I got it backward. Loop back support on foo is required. So, to
>convert reiserfs to reiser4 one needs loop back on reiserfs, which is
>not a problem.
>  
>
Hi,

    I will investigate better on this tool and if fine we will test on 
ReiserFS 3.X <-> 4.0 final.

Thanks,
Giovanni

> > 
> >  > 
> >  >     If someone knows some other link, for this tool I can take a look.
> > 
>
>Nikita.
>
> > 
>
>  
>

[PATCH] 1/5 Clean up Linux 2.6.4-rc1 sound/core/Makefile

[Russell King]

This is part of a patch series to clean up sound/core/Makefile in Linux
2.6.4-rc1.

- Add SND_TIMER, SND_PCM, SND_HWDEP and SND_RAWMIDI configuration symbols.
  These symbols select which modules in sound/core get built, building
  snd-timer, snd-pcm, snd-hwdep and snd-rawmidi respectively.

- Add reverse dependencies ("select") to select these symbols for core
  components where necessary.

- Hide SND_OSSEMUL - we can select this when SND_MIXER_OSS, SND_PCM_OSS
  or SND_SEQUENCER_OSS are selected automatically.

- Tweak Makefile to use these new symbols to build these modules.

- Since we now build appropriate modules for core components according
  to the new configuration symbols, (eg, snd-timer if SND_SEQUENCER is
  selected) we can delete these duplications.

I'm going to explicitly mention SND_RTCTIMER here, since it appeared to
have some complex handling in the makefile.  As the Kconfig definition
for this symbol stands, SND_RTCTIMER can only be selected to 'm' if
RTC is 'm' or 'y', and 'y' if RTC is 'y'.  This means that the ifeq and
subst in the makefile isn't gaining us anything, so we might as well
do away with it.

--- orig/sound/core/Kconfig	Sat Jun 14 22:35:04 2003
+++ linux/sound/core/Kconfig	Mon Mar  1 09:37:44 2004
@@ -3,9 +3,23 @@ config SND_BIT32_EMUL
 	tristate "Emulation for 32-bit applications"
 	depends on SND && (SPARC64 || PPC64 || X86_64 && IA32_EMULATION)
 
+config SND_TIMER
+	tristate
+
+config SND_PCM
+	tristate
+	select SND_TIMER
+
+config SND_HWDEP
+	tristate
+
+config SND_RAWMIDI
+	tristate
+
 config SND_SEQUENCER
 	tristate "Sequencer support"
 	depends on SND
+	select SND_TIMER
	help
	  Say 'Y' or 'M' to enable MIDI sequencer and router support. This feature
	  allows routing and enqueing MIDI events. Events can be processed at given
@@ -20,26 +34,27 @@ config SND_SEQ_DUMMY
 	  immediately.
 
 config SND_OSSEMUL
-	bool "OSS API emulation"
-	depends on SND
-	help
-	  Say 'Y' to enable OSS (Open Sound System) API emulation code.
+	bool
 
 config SND_MIXER_OSS
 	tristate "OSS Mixer API"
-	depends on SND_OSSEMUL && SND
+	depends on SND
+	select SND_OSSEMUL
 	help
 	  Say 'Y' or 'M' to enable mixer OSS API emulation (/dev/mixer*).
 
 config SND_PCM_OSS
 	tristate "OSS PCM (digital audio) API"
-	depends on SND_OSSEMUL && SND
+	depends on SND
+	select SND_OSSEMUL
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to enable digital audio (PCM) OSS API emulation (/dev/dsp*).
 
 config SND_SEQUENCER_OSS
 	bool "OSS Sequencer API"
-	depends on SND_OSSEMUL && SND_SEQUENCER
+	depends on SND_SEQUENCER
+	select SND_OSSEMUL
 	help
 	  Say 'Y' to enable OSS sequencer emulation (both /dev/sequencer and
 	  /dev/music interfaces).
@@ -47,6 +62,7 @@ config SND_SEQUENCER_OSS
 config SND_RTCTIMER
 	tristate "RTC Timer support"
 	depends on SND && RTC
+	select SND_TIMER
 	help
 	  Say 'Y' or 'M' to enable RTC timer support for ALSA. ALSA code uses RTC
 	  timer as precise timing source and maps the RTC timer to the ALSA's timer
--- orig/sound/core/Makefile	Sun Feb 29 22:53:00 2004
+++ linux/sound/core/Makefile	Mon Mar  1 09:39:48 2004
@@ -26,15 +26,14 @@ ifeq ($(subst m,y,$(CONFIG_RTC)),y)
 snd-hwdep-objs    := hwdep.o
 
 obj-$(CONFIG_SND) += snd.o
-ifeq ($(subst m,y,$(CONFIG_RTC)),y)
-  obj-$(CONFIG_SND_RTCTIMER) += snd-timer.o
-  obj-$(CONFIG_SND_RTCTIMER) += snd-rtctimer.o
-endif
-obj-$(CONFIG_SND_HWDEP) += snd-hwdep.o
+obj-$(CONFIG_SND_HWDEP)		+= snd-hwdep.o
+obj-$(CONFIG_SND_TIMER)		+= snd-timer.o
+obj-$(CONFIG_SND_RTCTIMER)	+= snd-rtctimer.o
+obj-$(CONFIG_SND_PCM)		+= snd-pcm.o snd-page-alloc.o
+obj-$(CONFIG_SND_RAWMIDI)	+= snd-rawmidi.o
 
-obj-$(CONFIG_SND_MIXER_OSS) += oss/
-obj-$(CONFIG_SND_PCM_OSS) += snd-pcm.o snd-timer.o snd-page-alloc.o oss/
-obj-$(CONFIG_SND_SEQUENCER) += snd-timer.o seq/
+obj-$(CONFIG_SND_OSSEMUL)	+= oss/
+obj-$(CONFIG_SND_SEQUENCER)	+= seq/
 obj-$(CONFIG_SND_BIT32_EMUL) += ioctl32/
 
 # Toplevel Module Dependency


-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click

Re: [PATCH] 2/5 Clean up Linux 2.6.4-rc1 sound/core/Makefile

[Russell King]

This is part of a patch series to clean up sound/core/Makefile in Linux
2.6.4-rc1.

- Add "select SND_PCM" statements to appropriate Kconfig entries for
  drivers whose configuration symbol is used to build snd-pcm,
  snd-timer, and snd-page-alloc.

- Remove snd-pcm, snd-timer and snd-page-alloc from these in
  sound/core/Makefile.

- Remove snd from these entries as well - all SND_xxx configuration
  symbols depend on CONFIG_SND, so we won't even consider building
  any of these drivers unless SND is already set to 'y' or 'm'.

--- orig/sound/arm/Kconfig	Sat Nov  2 18:58:37 2002
+++ linux/sound/arm/Kconfig	Mon Mar  1 13:03:57 2004
@@ -6,6 +6,7 @@ menu "ALSA ARM devices"
 config SND_SA11XX_UDA1341
 	tristate "SA11xx UDA1341TS driver (H3600)"
 	depends on ARCH_SA1100 && SND && L3
+	select SND_PCM
 	help
 	  Say Y or M if you have a Compaq iPaq H3x00 handheld computer and want
 	  to use its Philips UDA 1341 audio chip.
--- orig/sound/core/Makefile	Mon Mar  1 10:51:26 2004
+++ linux/sound/core/Makefile	Mon Mar  1 11:11:07 2004
@@ -37,74 +37,58 @@ obj-$(CONFIG_SND_SEQUENCER) += snd-timer
 obj-$(CONFIG_SND_BIT32_EMUL) += ioctl32/
 
 # Toplevel Module Dependency
-obj-$(CONFIG_SND_DUMMY) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
 obj-$(CONFIG_SND_VIRMIDI) += snd-rawmidi.o snd.o snd-timer.o
 obj-$(CONFIG_SND_SERIAL_U16550) += snd-rawmidi.o snd.o snd-timer.o
 obj-$(CONFIG_SND_MTPAV) += snd-rawmidi.o snd.o snd-timer.o
 obj-$(CONFIG_SND_MPU401) += snd-rawmidi.o snd.o snd-timer.o
-obj-$(CONFIG_SND_ALS100) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_AZT2320) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_AZT3328) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_CMI8330) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_DT019X) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ES18XX) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_OPL3SA2) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_SGALAXY) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_AD1816A) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_AD1848) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_CS4231) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_CS4232) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_CS4236) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ES1688) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_GUSCLASSIC) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_GUSMAX) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_GUSEXTREME) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_INTERWAVE) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_INTERWAVE_STB) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_OPTI92X_AD1848) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_OPTI92X_CS4231) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_OPTI93X) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_SB8) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_SB16) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_SBAWE) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ES968) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_WAVEFRONT) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_SSCAPE) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ALS4000) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_CMIPCI) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_CS4281) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ENS1370) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_ENS1371) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_ES1938) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-hwdep.o snd-rawmidi.o
-obj-$(CONFIG_SND_ES1968) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_FM801) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ICE1712) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_ICE1724) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_INTEL8X0) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_MAESTRO3) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_RME32) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_RME96) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_SONICVIBES) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_VIA82XX) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_ALI5451) += snd.o snd-rawmidi.o snd-timer.o snd-page-alloc.o snd-pcm.o
-obj-$(CONFIG_SND_CS46XX) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_EMU10K1) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_KORG1212) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_NM256) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_RME9652) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_HDSP) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_TRIDENT) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_YMFPCI) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_POWERMAC) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
+obj-$(CONFIG_SND_ALS100) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_AZT2320) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_AZT3328) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_DT019X) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_ES18XX) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_OPL3SA2) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_AD1816A) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_CS4231) += snd-rawmidi.o
+obj-$(CONFIG_SND_CS4232) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_CS4236) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_ES1688) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_GUSCLASSIC) += snd-rawmidi.o
+obj-$(CONFIG_SND_GUSMAX) += snd-rawmidi.o
+obj-$(CONFIG_SND_GUSEXTREME) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_INTERWAVE) += snd-rawmidi.o
+obj-$(CONFIG_SND_INTERWAVE_STB) += snd-rawmidi.o
+obj-$(CONFIG_SND_OPTI92X_AD1848) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_OPTI92X_CS4231) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_OPTI93X) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_SB8) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_SB16) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_SBAWE) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_ES968) += snd-rawmidi.o
+obj-$(CONFIG_SND_WAVEFRONT) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_SSCAPE) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_ALS4000) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_CMIPCI) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_CS4281) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_ENS1370) += snd-rawmidi.o
+obj-$(CONFIG_SND_ENS1371) += snd-rawmidi.o
+obj-$(CONFIG_SND_ES1938) += snd-hwdep.o snd-rawmidi.o
+obj-$(CONFIG_SND_ES1968) += snd-rawmidi.o
+obj-$(CONFIG_SND_FM801) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_ICE1712) += snd-rawmidi.o
+obj-$(CONFIG_SND_ICE1724) += snd-rawmidi.o
+obj-$(CONFIG_SND_INTEL8X0) += snd-rawmidi.o
+obj-$(CONFIG_SND_SONICVIBES) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_VIA82XX) += snd-rawmidi.o
+obj-$(CONFIG_SND_ALI5451) += snd-rawmidi.o
+obj-$(CONFIG_SND_CS46XX) += snd-rawmidi.o
+obj-$(CONFIG_SND_EMU10K1) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_HDSP) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_TRIDENT) += snd-rawmidi.o
+obj-$(CONFIG_SND_YMFPCI) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_SA11XX_UDA1341) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_PC98_CS4232) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_USB_AUDIO) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-rawmidi.o
-obj-$(CONFIG_SND_SUN_AMD7930) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_SUN_CS4231) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_HARMONY) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
-obj-$(CONFIG_SND_VXPOCKET) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-hwdep.o
-obj-$(CONFIG_SND_VXP440) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-hwdep.o
-obj-$(CONFIG_SND_VX222) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o snd-hwdep.o
-obj-$(CONFIG_SND_BT87X) += snd-pcm.o snd-timer.o snd-page-alloc.o snd.o
+obj-$(CONFIG_SND_PC98_CS4232) += snd-rawmidi.o snd-hwdep.o
+obj-$(CONFIG_SND_USB_AUDIO) += snd-rawmidi.o
+obj-$(CONFIG_SND_VXPOCKET) += snd-hwdep.o
+obj-$(CONFIG_SND_VXP440) += snd-hwdep.o
+obj-$(CONFIG_SND_VX222) += snd-hwdep.o
 
 obj-m := $(sort $(obj-m))
--- orig/sound/drivers/Kconfig	Sat Nov  2 18:58:37 2002
+++ linux/sound/drivers/Kconfig	Mon Mar  1 10:52:04 2004
@@ -6,6 +6,7 @@ menu "Generic devices"
 config SND_DUMMY
 	tristate "Dummy (/dev/null) soundcard"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include dummy driver. This driver does nothing, but
 	  emulates various mixer controls and PCM devices.
--- orig/sound/isa/Kconfig	Sat Jun 14 22:35:08 2003
+++ linux/sound/isa/Kconfig	Mon Mar  1 10:56:02 2004
@@ -6,6 +6,7 @@ menu "ISA devices"
 config SND_AD1816A
 	tristate "Analog Devices SoundPort AD1816A"
 	depends on SND && ISAPNP
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Analog Devices SoundPort AD1816A or
 	  compatible sound chips.
@@ -13,6 +14,7 @@ config SND_AD1816A
 config SND_AD1848
 	tristate "Generic AD1848/CS4248 driver"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for AD1848 (Analog Devices) or CS4248 
 	  (Cirrus Logic - Crystal Semiconductors) chips. Please, for newer chips
@@ -21,6 +23,7 @@ config SND_AD1848
 config SND_CS4231
 	tristate "Generic Cirrus Logic CS4231 driver"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for CS4231 chips from Cirrus Logic -
 	  Crystal Semiconductors.
@@ -28,6 +31,7 @@ config SND_CS4231
 config SND_CS4232
 	tristate "Generic Cirrus Logic CS4232 driver"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for CS4232 chips from Cirrus Logic -
 	  Crystal Semiconductors.
@@ -35,6 +39,7 @@ config SND_CS4232
 config SND_CS4236
 	tristate "Generic Cirrus Logic CS4236+ driver"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for CS4235,CS4236,CS4237B,CS4238B,CS4239
 	  chips from Cirrus Logic - Crystal Semiconductors.
@@ -42,6 +47,7 @@ config SND_CS4236
 config SND_PC98_CS4232
 	tristate "NEC PC9800 CS4232 driver"
 	depends on SND && X86_PC9800
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for NEC PC-9801/PC-9821 on-board
 	  soundchip based on CS4232.
@@ -49,42 +55,49 @@ config SND_PC98_CS4232
 config SND_ES968
 	tristate "Generic ESS ES968 driver"
 	depends on SND && ISAPNP
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ESS AudioDrive ES968 chip.
 
 config SND_ES1688
 	tristate "Generic ESS ES688/ES1688 driver"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ESS AudioDrive ES688 or ES1688 chips.
 
 config SND_ES18XX
 	tristate "Generic ESS ES18xx driver"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ESS AudioDrive ES18xx chips.
 
 config SND_GUSCLASSIC
 	tristate "Gravis UltraSound Classic"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Gravis UltraSound Classic soundcard.
 
 config SND_GUSEXTREME
 	tristate "Gravis UltraSound Extreme"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Gravis UltraSound Extreme soundcard.
 
 config SND_GUSMAX
 	tristate "Gravis UltraSound MAX"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Gravis UltraSound MAX soundcard.
 
 config SND_INTERWAVE
 	tristate "AMD InterWave, Gravis UltraSound PnP"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for AMD InterWave based soundcards
 	  (Gravis UltraSound Plug & Play, STB SoundRage32, MED3210, Dynasonic Pro,
@@ -93,6 +106,7 @@ config SND_INTERWAVE
 config SND_INTERWAVE_STB
 	tristate "AMD InterWave + TEA6330T (UltraSound 32-Pro)"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for AMD InterWave based soundcards
 	  with TEA6330T bass and treble regulator (UltraSound 32-Pro).
@@ -100,6 +114,7 @@ config SND_INTERWAVE_STB
 config SND_OPTI92X_AD1848
 	tristate "OPTi 82C92x - AD1848"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Opti92x soundcards equiped with
 	  AD1848 codec.
@@ -107,6 +122,7 @@ config SND_OPTI92X_AD1848
 config SND_OPTI92X_CS4231
 	tristate "OPTi 82C92x - CS4231"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Opti92x soundcards equiped with
 	  CS4231 codec.
@@ -114,12 +130,14 @@ config SND_OPTI92X_CS4231
 config SND_OPTI93X
 	tristate "OPTi 82C93x"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Opti93x soundcards.
 
 config SND_SB8
 	tristate "Sound Blaster 1.0/2.0/Pro (8-bit)"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Sound Blaster 1.0/2.0/Pro (8-bit)
 	  soundcards or 100% compatible from Creative.
@@ -127,6 +145,7 @@ config SND_SB8
 config SND_SB16
 	tristate "Sound Blaster 16 (PnP)"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Sound Blaster 16 (including
 	  Plug and Play version).
@@ -134,6 +153,7 @@ config SND_SB16
 config SND_SBAWE
 	tristate "Sound Blaster AWE (32,64) (PnP)"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Sound Blaster AWE (including
 	  Plug and Play version).
@@ -149,6 +169,7 @@ config SND_SB16_CSP
 config SND_WAVEFRONT
 	tristate "Turtle Beach Maui,Tropez,Tropez+ (Wavefront)"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Turtle Beach Maui, Tropez
 	  and Tropez+ soundcards based on Wavefront chip.
@@ -156,6 +177,7 @@ config SND_WAVEFRONT
 config SND_ALS100
 	tristate "Avance Logic ALS100/ALS120"
 	depends on SND && ISAPNP
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Avance Logic ALS100, ALS110,
 	  ALS120 and ALS200 soundcards.
@@ -163,18 +185,21 @@ config SND_ALS100
 config SND_AZT2320
 	tristate "Aztech Systems AZT2320"
 	depends on SND && ISAPNP
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Aztech Systems AZT2320 soundcard.
 
 config SND_CMI8330
 	tristate "C-Media CMI8330"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for C-Media CMI8330 based soundcards.
 
 config SND_DT019X
 	tristate "Diamond Technologies DT-019X, Avance Logic ALS-007"
 	depends on SND && ISAPNP
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Diamond Technologies DT-019X and
 	  Avance Logic ALS-007 soundcards.
@@ -182,18 +207,21 @@ config SND_DT019X
 config SND_OPL3SA2
 	tristate "Yamaha OPL3-SA2/SA3"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Yamaha OPL3SA2 or OPL3SA3 chips.
 
 config SND_SGALAXY
 	tristate "Aztech Sound Galaxy"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Aztech Sound Galaxy.
 
 config SND_SSCAPE
 	tristate "Ensoniq SoundScape PnP driver"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Ensoniq SoundScape PnP
 	  soundcard.
--- orig/sound/pci/Kconfig	Wed Feb 18 22:35:55 2004
+++ linux/sound/pci/Kconfig	Mon Mar  1 11:03:28 2004
@@ -6,18 +6,21 @@ menu "PCI devices"
 config SND_ALI5451
 	tristate "ALi PCI Audio M5451"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ALI PCI Audio M5451 sound core.
 
 config SND_AZT3328
 	tristate "Aztech AZF3328 / PCI168 (EXPERIMENTAL)"
 	depends on SND && EXPERIMENTAL
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Aztech AZF3328 (PCI168) soundcards.
 
 config SND_BT87X
         tristate "Bt87x Audio Capture"
         depends on SND
+	select SND_PCM
         help
           Say 'Y' or 'M' to include support for recording audio from TV cards
           based on Brooktree Bt878/Bt879 chips.
@@ -25,6 +28,7 @@ config SND_BT87X
 config SND_CS46XX
 	tristate "Cirrus Logic (Sound Fusion) CS4280/CS461x/CS462x/CS463x"
 	depends on SND
+	select SND_PCM
 	select GAMEPORT
 	help
 	  Say 'Y' or 'M' to include support for Cirrus Logic CS4610 / CS4612 /
@@ -39,12 +43,14 @@ config SND_CS46XX_NEW_DSP
 config SND_CS4281
 	tristate "Cirrus Logic (Sound Fusion) CS4281"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Cirrus Logic CS4281.
 
 config SND_EMU10K1
 	tristate "EMU10K1 (SB Live! & Audigy, E-mu APS)"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Sound Blaster PCI 512, Live!,
 	  Audigy and E-mu APS (partially supported).
@@ -52,18 +58,21 @@ config SND_EMU10K1
 config SND_KORG1212
 	tristate "Korg 1212 IO"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Korg 1212IO.
 
 config SND_NM256
 	tristate "NeoMagic NM256AV/ZX"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for NeoMagic NM256AV/ZX chips.
 
 config SND_RME32
 	tristate "RME Digi32, 32/8, 32 PRO"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for RME Digi32, Digi32 PRO and
 	  Digi32/8 (Sek'd Prodif32, Prodif96 and Prodif Gold) audio devices.
@@ -71,6 +80,7 @@ config SND_RME32
 config SND_RME96
 	tristate "RME Digi96, 96/8, 96/8 PRO"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for RME Digi96, Digi96/8 and
 	  Digi96/8 PRO/PAD/PST.
@@ -78,6 +88,7 @@ config SND_RME96
 config SND_RME9652
 	tristate "RME Digi9652 (Hammerfall)"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for RME Hammerfall (RME Digi9652 /
 	  Digi9636) soundcards.
@@ -85,6 +96,7 @@ config SND_RME9652
 config SND_HDSP
 	tristate "RME Hammerfall DSP Audio"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for RME Hammerfall DSP Audio
 	  soundcards.
@@ -92,6 +104,7 @@ config SND_HDSP
 config SND_TRIDENT
 	tristate "Trident 4D-Wave DX/NX; SiS 7018"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Trident 4D-Wave DX/NX and
 	  SiS 7018 soundcards.
@@ -99,6 +112,7 @@ config SND_TRIDENT
 config SND_YMFPCI
 	tristate "Yamaha YMF724/740/744/754"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Yamaha PCI audio chips - 
 	  YMF724, YMF724F, YMF740, YMF740C, YMF744, YMF754.
@@ -106,12 +120,14 @@ config SND_YMFPCI
 config SND_ALS4000
 	tristate "Avance Logic ALS4000"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Avance Logic ALS4000.
 
 config SND_CMIPCI
 	tristate "C-Media 8738, 8338"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for C-Media CMI8338 and 8738 PCI
 	  soundcards.
@@ -119,12 +135,14 @@ config SND_CMIPCI
 config SND_ENS1370
 	tristate "(Creative) Ensoniq AudioPCI 1370"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Ensoniq AudioPCI ES1370.
 
 config SND_ENS1371
 	tristate "(Creative) Ensoniq AudioPCI 1371/1373"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Ensoniq AudioPCI ES1371 and
 	  Sound Blaster PCI 64 or 128 soundcards.
@@ -132,6 +150,7 @@ config SND_ENS1371
 config SND_ES1938
 	tristate "ESS ES1938/1946/1969 (Solo-1)"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ESS Solo-1 (ES1938, ES1946, ES1969)
 	  soundcard.
@@ -139,18 +158,21 @@ config SND_ES1938
 config SND_ES1968
 	tristate "ESS ES1968/1978 (Maestro-1/2/2E)"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ESS Maestro 1/2/2E.
 
 config SND_MAESTRO3
 	tristate "ESS Allegro/Maestro3"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ESS Maestro 3 (Allegro) soundcard.
 
 config SND_FM801
 	tristate "ForteMedia FM801"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ForteMedia FM801 based soundcards.
 
@@ -164,6 +186,7 @@ config CONFIG_SND_FM801_TEA575X
 config SND_ICE1712
 	tristate "ICEnsemble ICE1712 (Envy24)"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ICE1712 (Envy24) based soundcards.
 	  Currently supported hardware is: MidiMan M Audio - Delta 1010(LT), Dio 2496,
@@ -173,6 +196,7 @@ config SND_ICE1712
 config SND_ICE1724
 	tristate "ICE/VT1724 (Envy24HT)"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ICE/VT1724 (Envy24HT) based
 	  soundcards.
@@ -182,6 +206,7 @@ config SND_ICE1724
 config SND_INTEL8X0
 	tristate "Intel i8x0/MX440, SiS 7012; Ali 5455; NForce Audio; AMD768/8111"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Intel8x0 based soundcards,
 	  SiS 7012, AMD768/8111, NVidia NForce and ALi 5455 chips.
@@ -189,18 +214,21 @@ config SND_INTEL8X0
 config SND_SONICVIBES
 	tristate "S3 SonicVibes"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for S3 SonicVibes based soundcards.
 
 config SND_VIA82XX
 	tristate "VIA 82C686A/B, 8233 South Bridge"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for VIA VT82C686A/B, VT8233 South Bridge.
 
 config SND_VX222
 	tristate "Digigram VX222"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Digigram VX222 soundcards.
 
--- orig/sound/ppc/Kconfig	Sat Nov  2 18:58:41 2002
+++ linux/sound/ppc/Kconfig	Mon Mar  1 11:10:19 2004
@@ -6,6 +6,7 @@ menu "ALSA PowerMac devices"
 config SND_POWERMAC
 	tristate "PowerMac (AWACS, DACA, Burgundy, Tumbler, Keywest)"
 	depends on SND
+	select SND_PCM
 
 endmenu
 
--- orig/sound/parisc/Kconfig	Sat Jun 14 22:35:12 2003
+++ linux/sound/parisc/Kconfig	Mon Mar  1 11:11:03 2004
@@ -6,6 +6,7 @@ menu "ALSA PA-RISC devices"
 config SND_HARMONY
 	tristate "Harmony/Vivace sound chip"
 	depends on SND
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Harmony/Vivace soundchip
 	  on HP712s, 715/new and many other GSC based machines.
--- orig/sound/sparc/Kconfig	Sat Nov  2 18:58:41 2002
+++ linux/sound/sparc/Kconfig	Mon Mar  1 11:10:48 2004
@@ -6,11 +6,13 @@ menu "ALSA Sparc devices"
 config SND_SUN_AMD7930
 	tristate "Sun AMD7930"
 	depends on SBUS && SND
+	select SND_PCM
 
 #  dep_tristate 'Sun DBRI' CONFIG_SND_SUN_DBRI $CONFIG_SND
 config SND_SUN_CS4231
 	tristate "Sun CS4231"
 	depends on SND
+	select SND_PCM
 
 endmenu
 
--- orig/sound/usb/Kconfig	Sat Dec 28 17:08:02 2002
+++ linux/sound/usb/Kconfig	Mon Mar  1 11:05:38 2004
@@ -6,6 +6,7 @@ menu "ALSA USB devices"
 config SND_USB_AUDIO
 	tristate "USB Audio/MIDI driver"
 	depends on SND && USB
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for USB audio and USB MIDI devices.
 
--- orig/sound/pcmcia/Kconfig	Wed Feb 18 22:36:01 2004
+++ linux/sound/pcmcia/Kconfig	Mon Mar  1 11:05:15 2004
@@ -6,12 +6,14 @@ menu "PCMCIA devices"
 config SND_VXPOCKET
 	tristate "Digigram VXpocket"
 	depends on SND && PCMCIA && ISA
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Digigram VXpocket soundcard.
 
 config SND_VXP440
 	tristate "Digigram VXpocket 440"
 	depends on SND && PCMCIA && ISA
+	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Digigram VXpocket 440 soundcard.
 


-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click

Re: [PATCH] 3/5 Clean up Linux 2.6.4-rc1 sound/core/Makefile

[Russell King]

This is part of a patch series to clean up sound/core/Makefile in Linux
2.6.4-rc1.

- Add SND_RAWMIDI for drivers which use the snd-rawmidi module.
- Remove snd-rawmidi from these drivers entries in sound/core/Makefile
- Remove any sound/core/Makefile entries which are left empty.

--- orig/sound/core/Makefile	Mon Mar  1 11:16:43 2004
+++ linux/sound/core/Makefile	Mon Mar  1 11:30:43 2004
@@ -37,56 +37,39 @@ obj-$(CONFIG_SND_SEQUENCER) += snd-timer
 obj-$(CONFIG_SND_BIT32_EMUL) += ioctl32/
 
 # Toplevel Module Dependency
-obj-$(CONFIG_SND_VIRMIDI) += snd-rawmidi.o snd.o snd-timer.o
-obj-$(CONFIG_SND_SERIAL_U16550) += snd-rawmidi.o snd.o snd-timer.o
-obj-$(CONFIG_SND_MTPAV) += snd-rawmidi.o snd.o snd-timer.o
-obj-$(CONFIG_SND_MPU401) += snd-rawmidi.o snd.o snd-timer.o
-obj-$(CONFIG_SND_ALS100) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_AZT2320) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_AZT3328) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_DT019X) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ES18XX) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_OPL3SA2) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_AD1816A) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_CS4231) += snd-rawmidi.o
-obj-$(CONFIG_SND_CS4232) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_CS4236) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ES1688) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_GUSCLASSIC) += snd-rawmidi.o
-obj-$(CONFIG_SND_GUSMAX) += snd-rawmidi.o
-obj-$(CONFIG_SND_GUSEXTREME) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_INTERWAVE) += snd-rawmidi.o
-obj-$(CONFIG_SND_INTERWAVE_STB) += snd-rawmidi.o
-obj-$(CONFIG_SND_OPTI92X_AD1848) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_OPTI92X_CS4231) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_OPTI93X) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_SB8) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_SB16) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_SBAWE) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ES968) += snd-rawmidi.o
-obj-$(CONFIG_SND_WAVEFRONT) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_SSCAPE) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ALS4000) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_CMIPCI) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_CS4281) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ENS1370) += snd-rawmidi.o
-obj-$(CONFIG_SND_ENS1371) += snd-rawmidi.o
-obj-$(CONFIG_SND_ES1938) += snd-hwdep.o snd-rawmidi.o
-obj-$(CONFIG_SND_ES1968) += snd-rawmidi.o
-obj-$(CONFIG_SND_FM801) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_ICE1712) += snd-rawmidi.o
-obj-$(CONFIG_SND_ICE1724) += snd-rawmidi.o
-obj-$(CONFIG_SND_INTEL8X0) += snd-rawmidi.o
-obj-$(CONFIG_SND_SONICVIBES) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_VIA82XX) += snd-rawmidi.o
-obj-$(CONFIG_SND_ALI5451) += snd-rawmidi.o
-obj-$(CONFIG_SND_CS46XX) += snd-rawmidi.o
-obj-$(CONFIG_SND_EMU10K1) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_HDSP) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_TRIDENT) += snd-rawmidi.o
-obj-$(CONFIG_SND_YMFPCI) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_PC98_CS4232) += snd-rawmidi.o snd-hwdep.o
-obj-$(CONFIG_SND_USB_AUDIO) += snd-rawmidi.o
+obj-$(CONFIG_SND_VIRMIDI) += snd-timer.o
+obj-$(CONFIG_SND_SERIAL_U16550) += snd-timer.o
+obj-$(CONFIG_SND_MTPAV) += snd-timer.o
+obj-$(CONFIG_SND_MPU401) += snd-timer.o
+obj-$(CONFIG_SND_ALS100) += snd-hwdep.o
+obj-$(CONFIG_SND_AZT2320) += snd-hwdep.o
+obj-$(CONFIG_SND_AZT3328) += snd-hwdep.o
+obj-$(CONFIG_SND_DT019X) += snd-hwdep.o
+obj-$(CONFIG_SND_ES18XX) += snd-hwdep.o
+obj-$(CONFIG_SND_OPL3SA2) += snd-hwdep.o
+obj-$(CONFIG_SND_AD1816A) += snd-hwdep.o
+obj-$(CONFIG_SND_CS4232) += snd-hwdep.o
+obj-$(CONFIG_SND_CS4236) += snd-hwdep.o
+obj-$(CONFIG_SND_ES1688) += snd-hwdep.o
+obj-$(CONFIG_SND_GUSEXTREME) += snd-hwdep.o
+obj-$(CONFIG_SND_OPTI92X_AD1848) += snd-hwdep.o
+obj-$(CONFIG_SND_OPTI92X_CS4231) += snd-hwdep.o
+obj-$(CONFIG_SND_OPTI93X) += snd-hwdep.o
+obj-$(CONFIG_SND_SB8) += snd-hwdep.o
+obj-$(CONFIG_SND_SB16) += snd-hwdep.o
+obj-$(CONFIG_SND_SBAWE) += snd-hwdep.o
+obj-$(CONFIG_SND_WAVEFRONT) += snd-hwdep.o
+obj-$(CONFIG_SND_SSCAPE) += snd-hwdep.o
+obj-$(CONFIG_SND_ALS4000) += snd-hwdep.o
+obj-$(CONFIG_SND_CMIPCI) += snd-hwdep.o
+obj-$(CONFIG_SND_CS4281) += snd-hwdep.o
+obj-$(CONFIG_SND_ES1938) += snd-hwdep.o
+obj-$(CONFIG_SND_FM801) += snd-hwdep.o
+obj-$(CONFIG_SND_SONICVIBES) += snd-hwdep.o
+obj-$(CONFIG_SND_EMU10K1) += snd-hwdep.o
+obj-$(CONFIG_SND_HDSP) += snd-hwdep.o
+obj-$(CONFIG_SND_YMFPCI) += snd-hwdep.o
+obj-$(CONFIG_SND_PC98_CS4232) += snd-hwdep.o
 obj-$(CONFIG_SND_VXPOCKET) += snd-hwdep.o
 obj-$(CONFIG_SND_VXP440) += snd-hwdep.o
 obj-$(CONFIG_SND_VX222) += snd-hwdep.o
--- orig/sound/drivers/Kconfig	Mon Mar  1 11:16:44 2004
+++ linux/sound/drivers/Kconfig	Mon Mar  1 11:29:32 2004
@@ -15,6 +15,7 @@ config SND_DUMMY
 config SND_VIRMIDI
 	tristate "Virtual MIDI soundcard"
 	depends on SND_SEQUENCER
+	select SND_RAWMIDI
 	help
 	  Say 'Y' or 'M' to include virtual MIDI driver. This driver allows to
 	  connect applications using raw MIDI devices to sequencer.
@@ -22,6 +23,7 @@ config SND_VIRMIDI
 config SND_MTPAV
 	tristate "MOTU MidiTimePiece AV multiport MIDI"
 	depends on SND
+	select SND_RAWMIDI
 	help
 	  Say 'Y' or 'M' to include support for MOTU MidiTimePiece AV multiport
 	  MIDI adapter.
@@ -29,6 +31,7 @@ config SND_MTPAV
 config SND_SERIAL_U16550
 	tristate "UART16550 - MIDI only driver"
 	depends on SND
+	select SND_RAWMIDI
 	help
 	  Say 'Y' or 'M' to include support for MIDI serial port driver. It works
 	  with serial UARTs 16550 and better.
@@ -36,6 +39,7 @@ config SND_SERIAL_U16550
 config SND_MPU401
 	tristate "Generic MPU-401 UART driver"
 	depends on SND
+	select SND_RAWMIDI
 	help
 	  Say 'Y' or 'M' to include support for MPU401 hardware using UART access.
 
--- orig/sound/isa/Kconfig	Mon Mar  1 11:16:44 2004
+++ linux/sound/isa/Kconfig	Mon Mar  1 11:22:41 2004
@@ -6,6 +6,7 @@ menu "ISA devices"
 config SND_AD1816A
 	tristate "Analog Devices SoundPort AD1816A"
 	depends on SND && ISAPNP
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Analog Devices SoundPort AD1816A or
@@ -23,6 +24,7 @@ config SND_AD1848
 config SND_CS4231
 	tristate "Generic Cirrus Logic CS4231 driver"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for CS4231 chips from Cirrus Logic -
@@ -31,6 +33,7 @@ config SND_CS4231
 config SND_CS4232
 	tristate "Generic Cirrus Logic CS4232 driver"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for CS4232 chips from Cirrus Logic -
@@ -39,6 +42,7 @@ config SND_CS4232
 config SND_CS4236
 	tristate "Generic Cirrus Logic CS4236+ driver"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for CS4235,CS4236,CS4237B,CS4238B,CS4239
@@ -47,6 +51,7 @@ config SND_CS4236
 config SND_PC98_CS4232
 	tristate "NEC PC9800 CS4232 driver"
 	depends on SND && X86_PC9800
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for NEC PC-9801/PC-9821 on-board
@@ -55,6 +60,7 @@ config SND_PC98_CS4232
 config SND_ES968
 	tristate "Generic ESS ES968 driver"
 	depends on SND && ISAPNP
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ESS AudioDrive ES968 chip.
@@ -62,6 +68,7 @@ config SND_ES968
 config SND_ES1688
 	tristate "Generic ESS ES688/ES1688 driver"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ESS AudioDrive ES688 or ES1688 chips.
@@ -69,6 +76,7 @@ config SND_ES1688
 config SND_ES18XX
 	tristate "Generic ESS ES18xx driver"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ESS AudioDrive ES18xx chips.
@@ -76,6 +84,7 @@ config SND_ES18XX
 config SND_GUSCLASSIC
 	tristate "Gravis UltraSound Classic"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Gravis UltraSound Classic soundcard.
@@ -83,6 +92,7 @@ config SND_GUSCLASSIC
 config SND_GUSEXTREME
 	tristate "Gravis UltraSound Extreme"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Gravis UltraSound Extreme soundcard.
@@ -90,6 +100,7 @@ config SND_GUSEXTREME
 config SND_GUSMAX
 	tristate "Gravis UltraSound MAX"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Gravis UltraSound MAX soundcard.
@@ -97,6 +108,7 @@ config SND_GUSMAX
 config SND_INTERWAVE
 	tristate "AMD InterWave, Gravis UltraSound PnP"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for AMD InterWave based soundcards
@@ -106,6 +118,7 @@ config SND_INTERWAVE
 config SND_INTERWAVE_STB
 	tristate "AMD InterWave + TEA6330T (UltraSound 32-Pro)"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for AMD InterWave based soundcards
@@ -114,6 +127,7 @@ config SND_INTERWAVE_STB
 config SND_OPTI92X_AD1848
 	tristate "OPTi 82C92x - AD1848"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Opti92x soundcards equiped with
@@ -122,6 +136,7 @@ config SND_OPTI92X_AD1848
 config SND_OPTI92X_CS4231
 	tristate "OPTi 82C92x - CS4231"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Opti92x soundcards equiped with
@@ -130,6 +145,7 @@ config SND_OPTI92X_CS4231
 config SND_OPTI93X
 	tristate "OPTi 82C93x"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Opti93x soundcards.
@@ -137,6 +153,7 @@ config SND_OPTI93X
 config SND_SB8
 	tristate "Sound Blaster 1.0/2.0/Pro (8-bit)"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Sound Blaster 1.0/2.0/Pro (8-bit)
@@ -145,6 +162,7 @@ config SND_SB8
 config SND_SB16
 	tristate "Sound Blaster 16 (PnP)"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Sound Blaster 16 (including
@@ -153,6 +171,7 @@ config SND_SB16
 config SND_SBAWE
 	tristate "Sound Blaster AWE (32,64) (PnP)"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Sound Blaster AWE (including
@@ -169,6 +188,7 @@ config SND_SB16_CSP
 config SND_WAVEFRONT
 	tristate "Turtle Beach Maui,Tropez,Tropez+ (Wavefront)"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Turtle Beach Maui, Tropez
@@ -177,6 +197,7 @@ config SND_WAVEFRONT
 config SND_ALS100
 	tristate "Avance Logic ALS100/ALS120"
 	depends on SND && ISAPNP
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Avance Logic ALS100, ALS110,
@@ -185,6 +206,7 @@ config SND_ALS100
 config SND_AZT2320
 	tristate "Aztech Systems AZT2320"
 	depends on SND && ISAPNP
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Aztech Systems AZT2320 soundcard.
@@ -199,6 +221,7 @@ config SND_CMI8330
 config SND_DT019X
 	tristate "Diamond Technologies DT-019X, Avance Logic ALS-007"
 	depends on SND && ISAPNP
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Diamond Technologies DT-019X and
@@ -207,6 +230,7 @@ config SND_DT019X
 config SND_OPL3SA2
 	tristate "Yamaha OPL3-SA2/SA3"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Yamaha OPL3SA2 or OPL3SA3 chips.
@@ -221,6 +245,7 @@ config SND_SGALAXY
 config SND_SSCAPE
 	tristate "Ensoniq SoundScape PnP driver"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Ensoniq SoundScape PnP
--- orig/sound/pci/Kconfig	Mon Mar  1 11:16:44 2004
+++ linux/sound/pci/Kconfig	Mon Mar  1 11:27:54 2004
@@ -6,6 +6,7 @@ menu "PCI devices"
 config SND_ALI5451
 	tristate "ALi PCI Audio M5451"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ALI PCI Audio M5451 sound core.
@@ -13,6 +14,7 @@ config SND_ALI5451
 config SND_AZT3328
 	tristate "Aztech AZF3328 / PCI168 (EXPERIMENTAL)"
 	depends on SND && EXPERIMENTAL
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Aztech AZF3328 (PCI168) soundcards.
@@ -28,6 +30,7 @@ config SND_BT87X
 config SND_CS46XX
 	tristate "Cirrus Logic (Sound Fusion) CS4280/CS461x/CS462x/CS463x"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	select GAMEPORT
 	help
@@ -43,6 +46,7 @@ config SND_CS46XX_NEW_DSP
 config SND_CS4281
 	tristate "Cirrus Logic (Sound Fusion) CS4281"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Cirrus Logic CS4281.
@@ -50,6 +54,7 @@ config SND_CS4281
 config SND_EMU10K1
 	tristate "EMU10K1 (SB Live! & Audigy, E-mu APS)"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Sound Blaster PCI 512, Live!,
@@ -96,6 +101,7 @@ config SND_RME9652
 config SND_HDSP
 	tristate "RME Hammerfall DSP Audio"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for RME Hammerfall DSP Audio
@@ -104,6 +110,7 @@ config SND_HDSP
 config SND_TRIDENT
 	tristate "Trident 4D-Wave DX/NX; SiS 7018"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Trident 4D-Wave DX/NX and
@@ -112,6 +119,7 @@ config SND_TRIDENT
 config SND_YMFPCI
 	tristate "Yamaha YMF724/740/744/754"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Yamaha PCI audio chips - 
@@ -120,6 +128,7 @@ config SND_YMFPCI
 config SND_ALS4000
 	tristate "Avance Logic ALS4000"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Avance Logic ALS4000.
@@ -127,6 +136,7 @@ config SND_ALS4000
 config SND_CMIPCI
 	tristate "C-Media 8738, 8338"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for C-Media CMI8338 and 8738 PCI
@@ -135,6 +145,7 @@ config SND_CMIPCI
 config SND_ENS1370
 	tristate "(Creative) Ensoniq AudioPCI 1370"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Ensoniq AudioPCI ES1370.
@@ -142,6 +153,7 @@ config SND_ENS1370
 config SND_ENS1371
 	tristate "(Creative) Ensoniq AudioPCI 1371/1373"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Ensoniq AudioPCI ES1371 and
@@ -150,6 +162,7 @@ config SND_ENS1371
 config SND_ES1938
 	tristate "ESS ES1938/1946/1969 (Solo-1)"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ESS Solo-1 (ES1938, ES1946, ES1969)
@@ -158,6 +171,7 @@ config SND_ES1938
 config SND_ES1968
 	tristate "ESS ES1968/1978 (Maestro-1/2/2E)"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ESS Maestro 1/2/2E.
@@ -172,6 +186,7 @@ config SND_MAESTRO3
 config SND_FM801
 	tristate "ForteMedia FM801"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ForteMedia FM801 based soundcards.
@@ -186,6 +201,7 @@ config CONFIG_SND_FM801_TEA575X
 config SND_ICE1712
 	tristate "ICEnsemble ICE1712 (Envy24)"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ICE1712 (Envy24) based soundcards.
@@ -196,6 +212,7 @@ config SND_ICE1712
 config SND_ICE1724
 	tristate "ICE/VT1724 (Envy24HT)"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for ICE/VT1724 (Envy24HT) based
@@ -206,6 +223,7 @@ config SND_ICE1724
 config SND_INTEL8X0
 	tristate "Intel i8x0/MX440, SiS 7012; Ali 5455; NForce Audio; AMD768/8111"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Intel8x0 based soundcards,
@@ -214,6 +232,7 @@ config SND_INTEL8X0
 config SND_SONICVIBES
 	tristate "S3 SonicVibes"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for S3 SonicVibes based soundcards.
@@ -221,6 +240,7 @@ config SND_SONICVIBES
 config SND_VIA82XX
 	tristate "VIA 82C686A/B, 8233 South Bridge"
 	depends on SND
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for VIA VT82C686A/B, VT8233 South Bridge.
--- orig/sound/usb/Kconfig	Mon Mar  1 11:16:45 2004
+++ linux/sound/usb/Kconfig	Mon Mar  1 11:28:18 2004
@@ -6,6 +6,7 @@ menu "ALSA USB devices"
 config SND_USB_AUDIO
 	tristate "USB Audio/MIDI driver"
 	depends on SND && USB
+	select SND_RAWMIDI
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for USB audio and USB MIDI devices.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click

Re: [PATCH] 4/5 Clean up Linux 2.6.4-rc1 sound/core/Makefile

[Russell King]

This is part of a patch series to clean up sound/core/Makefile in Linux
2.6.4-rc1.

- Add SND_HWDEP for drivers which use the snd-hwdep module.
- Remove snd-hwdep from these drivers entries in sound/core/Makefile,
  removing any sound/core/Makefile entries which are left empty.

--- orig/sound/core/Makefile	Mon Mar  1 11:37:21 2004
+++ linux/sound/core/Makefile	Mon Mar  1 11:46:31 2004
@@ -41,37 +41,5 @@ obj-$(CONFIG_SND_VIRMIDI) += snd-timer.o
 obj-$(CONFIG_SND_SERIAL_U16550) += snd-timer.o
 obj-$(CONFIG_SND_MTPAV) += snd-timer.o
 obj-$(CONFIG_SND_MPU401) += snd-timer.o
-obj-$(CONFIG_SND_ALS100) += snd-hwdep.o
-obj-$(CONFIG_SND_AZT2320) += snd-hwdep.o
-obj-$(CONFIG_SND_AZT3328) += snd-hwdep.o
-obj-$(CONFIG_SND_DT019X) += snd-hwdep.o
-obj-$(CONFIG_SND_ES18XX) += snd-hwdep.o
-obj-$(CONFIG_SND_OPL3SA2) += snd-hwdep.o
-obj-$(CONFIG_SND_AD1816A) += snd-hwdep.o
-obj-$(CONFIG_SND_CS4232) += snd-hwdep.o
-obj-$(CONFIG_SND_CS4236) += snd-hwdep.o
-obj-$(CONFIG_SND_ES1688) += snd-hwdep.o
-obj-$(CONFIG_SND_GUSEXTREME) += snd-hwdep.o
-obj-$(CONFIG_SND_OPTI92X_AD1848) += snd-hwdep.o
-obj-$(CONFIG_SND_OPTI92X_CS4231) += snd-hwdep.o
-obj-$(CONFIG_SND_OPTI93X) += snd-hwdep.o
-obj-$(CONFIG_SND_SB8) += snd-hwdep.o
-obj-$(CONFIG_SND_SB16) += snd-hwdep.o
-obj-$(CONFIG_SND_SBAWE) += snd-hwdep.o
-obj-$(CONFIG_SND_WAVEFRONT) += snd-hwdep.o
-obj-$(CONFIG_SND_SSCAPE) += snd-hwdep.o
-obj-$(CONFIG_SND_ALS4000) += snd-hwdep.o
-obj-$(CONFIG_SND_CMIPCI) += snd-hwdep.o
-obj-$(CONFIG_SND_CS4281) += snd-hwdep.o
-obj-$(CONFIG_SND_ES1938) += snd-hwdep.o
-obj-$(CONFIG_SND_FM801) += snd-hwdep.o
-obj-$(CONFIG_SND_SONICVIBES) += snd-hwdep.o
-obj-$(CONFIG_SND_EMU10K1) += snd-hwdep.o
-obj-$(CONFIG_SND_HDSP) += snd-hwdep.o
-obj-$(CONFIG_SND_YMFPCI) += snd-hwdep.o
-obj-$(CONFIG_SND_PC98_CS4232) += snd-hwdep.o
-obj-$(CONFIG_SND_VXPOCKET) += snd-hwdep.o
-obj-$(CONFIG_SND_VXP440) += snd-hwdep.o
-obj-$(CONFIG_SND_VX222) += snd-hwdep.o
 
 obj-m := $(sort $(obj-m))
--- orig/sound/isa/Kconfig	Mon Mar  1 11:37:22 2004
+++ linux/sound/isa/Kconfig	Mon Mar  1 11:42:33 2004
@@ -6,6 +6,7 @@ menu "ISA devices"
 config SND_AD1816A
 	tristate "Analog Devices SoundPort AD1816A"
 	depends on SND && ISAPNP
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -33,6 +34,7 @@ config SND_CS4231
 config SND_CS4232
 	tristate "Generic Cirrus Logic CS4232 driver"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -42,6 +44,7 @@ config SND_CS4232
 config SND_CS4236
 	tristate "Generic Cirrus Logic CS4236+ driver"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -51,6 +54,7 @@ config SND_CS4236
 config SND_PC98_CS4232
 	tristate "NEC PC9800 CS4232 driver"
 	depends on SND && X86_PC9800
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -68,6 +72,7 @@ config SND_ES968
 config SND_ES1688
 	tristate "Generic ESS ES688/ES1688 driver"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -76,6 +81,7 @@ config SND_ES1688
 config SND_ES18XX
 	tristate "Generic ESS ES18xx driver"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -92,6 +98,7 @@ config SND_GUSCLASSIC
 config SND_GUSEXTREME
 	tristate "Gravis UltraSound Extreme"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -127,6 +134,7 @@ config SND_INTERWAVE_STB
 config SND_OPTI92X_AD1848
 	tristate "OPTi 82C92x - AD1848"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -136,6 +144,7 @@ config SND_OPTI92X_AD1848
 config SND_OPTI92X_CS4231
 	tristate "OPTi 82C92x - CS4231"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -145,6 +154,7 @@ config SND_OPTI92X_CS4231
 config SND_OPTI93X
 	tristate "OPTi 82C93x"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -153,6 +163,7 @@ config SND_OPTI93X
 config SND_SB8
 	tristate "Sound Blaster 1.0/2.0/Pro (8-bit)"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -162,6 +173,7 @@ config SND_SB8
 config SND_SB16
 	tristate "Sound Blaster 16 (PnP)"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -171,6 +183,7 @@ config SND_SB16
 config SND_SBAWE
 	tristate "Sound Blaster AWE (32,64) (PnP)"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -188,6 +201,7 @@ config SND_SB16_CSP
 config SND_WAVEFRONT
 	tristate "Turtle Beach Maui,Tropez,Tropez+ (Wavefront)"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -197,6 +211,7 @@ config SND_WAVEFRONT
 config SND_ALS100
 	tristate "Avance Logic ALS100/ALS120"
 	depends on SND && ISAPNP
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -206,6 +221,7 @@ config SND_ALS100
 config SND_AZT2320
 	tristate "Aztech Systems AZT2320"
 	depends on SND && ISAPNP
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -221,6 +237,7 @@ config SND_CMI8330
 config SND_DT019X
 	tristate "Diamond Technologies DT-019X, Avance Logic ALS-007"
 	depends on SND && ISAPNP
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -230,6 +247,7 @@ config SND_DT019X
 config SND_OPL3SA2
 	tristate "Yamaha OPL3-SA2/SA3"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -245,6 +263,7 @@ config SND_SGALAXY
 config SND_SSCAPE
 	tristate "Ensoniq SoundScape PnP driver"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
--- orig/sound/pci/Kconfig	Mon Mar  1 11:37:22 2004
+++ linux/sound/pci/Kconfig	Mon Mar  1 11:46:13 2004
@@ -14,6 +14,7 @@ config SND_ALI5451
 config SND_AZT3328
 	tristate "Aztech AZF3328 / PCI168 (EXPERIMENTAL)"
 	depends on SND && EXPERIMENTAL
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -46,6 +47,7 @@ config SND_CS46XX_NEW_DSP
 config SND_CS4281
 	tristate "Cirrus Logic (Sound Fusion) CS4281"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -54,6 +56,7 @@ config SND_CS4281
 config SND_EMU10K1
 	tristate "EMU10K1 (SB Live! & Audigy, E-mu APS)"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -101,6 +104,7 @@ config SND_RME9652
 config SND_HDSP
 	tristate "RME Hammerfall DSP Audio"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -119,6 +123,7 @@ config SND_TRIDENT
 config SND_YMFPCI
 	tristate "Yamaha YMF724/740/744/754"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -128,6 +133,7 @@ config SND_YMFPCI
 config SND_ALS4000
 	tristate "Avance Logic ALS4000"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -136,6 +142,7 @@ config SND_ALS4000
 config SND_CMIPCI
 	tristate "C-Media 8738, 8338"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -162,6 +169,7 @@ config SND_ENS1371
 config SND_ES1938
 	tristate "ESS ES1938/1946/1969 (Solo-1)"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -186,6 +194,7 @@ config SND_MAESTRO3
 config SND_FM801
 	tristate "ForteMedia FM801"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -232,6 +241,7 @@ config SND_INTEL8X0
 config SND_SONICVIBES
 	tristate "S3 SonicVibes"
 	depends on SND
+	select SND_HWDEP
 	select SND_RAWMIDI
 	select SND_PCM
 	help
@@ -248,6 +258,7 @@ config SND_VIA82XX
 config SND_VX222
 	tristate "Digigram VX222"
 	depends on SND
+	select SND_HWDEP
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Digigram VX222 soundcards.
--- orig/sound/pcmcia/Kconfig	Mon Mar  1 11:16:45 2004
+++ linux/sound/pcmcia/Kconfig	Mon Mar  1 11:46:29 2004
@@ -6,6 +6,7 @@ menu "PCMCIA devices"
 config SND_VXPOCKET
 	tristate "Digigram VXpocket"
 	depends on SND && PCMCIA && ISA
+	select SND_HWDEP
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Digigram VXpocket soundcard.
@@ -13,6 +14,7 @@ config SND_VXPOCKET
 config SND_VXP440
 	tristate "Digigram VXpocket 440"
 	depends on SND && PCMCIA && ISA
+	select SND_HWDEP
 	select SND_PCM
 	help
 	  Say 'Y' or 'M' to include support for Digigram VXpocket 440 soundcard.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click

Re: [PATCH] 5/5 Clean up Linux 2.6.4-rc1 sound/core/Makefile

[Russell King]

This is part of a patch series to clean up sound/core/Makefile in Linux
2.6.4-rc1.

- Add SND_TIMER for drivers which use the snd-timer module.
- Remove snd-timer from these drivers entries in sound/core/Makefile,
  removing any sound/core/Makefile entries which are left empty.
- Since the "top level module dependency" lists are now gone, remove
  the comment.
- Also, since we only mention objects once, remove the sorting of obj-m

--- orig/sound/core/Makefile	Mon Mar  1 12:16:54 2004
+++ linux/sound/core/Makefile	Mon Mar  1 12:16:33 2004
@@ -35,11 +35,3 @@ obj-$(CONFIG_SND_SEQUENCER) += snd-timer
 obj-$(CONFIG_SND_OSSEMUL)	+= oss/
 obj-$(CONFIG_SND_SEQUENCER)	+= seq/
 obj-$(CONFIG_SND_BIT32_EMUL) += ioctl32/
-
-# Toplevel Module Dependency
-obj-$(CONFIG_SND_VIRMIDI) += snd-timer.o
-obj-$(CONFIG_SND_SERIAL_U16550) += snd-timer.o
-obj-$(CONFIG_SND_MTPAV) += snd-timer.o
-obj-$(CONFIG_SND_MPU401) += snd-timer.o
-
-obj-m := $(sort $(obj-m))
--- orig/sound/drivers/Kconfig	Mon Mar  1 12:16:52 2004
+++ linux/sound/drivers/Kconfig	Mon Mar  1 12:14:18 2004
@@ -15,6 +15,7 @@ config SND_DUMMY
 config SND_VIRMIDI
 	tristate "Virtual MIDI soundcard"
 	depends on SND_SEQUENCER
+	select SND_TIMER
 	select SND_RAWMIDI
 	help
 	  Say 'Y' or 'M' to include virtual MIDI driver. This driver allows to
@@ -23,6 +24,7 @@ config SND_VIRMIDI
 config SND_MTPAV
 	tristate "MOTU MidiTimePiece AV multiport MIDI"
 	depends on SND
+	select SND_TIMER
 	select SND_RAWMIDI
 	help
 	  Say 'Y' or 'M' to include support for MOTU MidiTimePiece AV multiport
@@ -31,6 +33,7 @@ config SND_MTPAV
 config SND_SERIAL_U16550
 	tristate "UART16550 - MIDI only driver"
 	depends on SND
+	select SND_TIMER
 	select SND_RAWMIDI
 	help
 	  Say 'Y' or 'M' to include support for MIDI serial port driver. It works
@@ -39,6 +42,7 @@ config SND_SERIAL_U16550
 config SND_MPU401
 	tristate "Generic MPU-401 UART driver"
 	depends on SND
+	select SND_TIMER
 	select SND_RAWMIDI
 	help
 	  Say 'Y' or 'M' to include support for MPU401 hardware using UART access.


-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click

Re: [PATCH] 1/5 Clean up Linux 2.6.4-rc1 sound/core/Makefile

[Takashi Iwai]

At Mon, 1 Mar 2004 14:42:31 +0000,
Russell King wrote:
> 
> This is part of a patch series to clean up sound/core/Makefile in Linux
> 2.6.4-rc1.
> 
> - Add SND_TIMER, SND_PCM, SND_HWDEP and SND_RAWMIDI configuration symbols.
>   These symbols select which modules in sound/core get built, building
>   snd-timer, snd-pcm, snd-hwdep and snd-rawmidi respectively.
> 
> - Add reverse dependencies ("select") to select these symbols for core
>   components where necessary.
> 
> - Hide SND_OSSEMUL - we can select this when SND_MIXER_OSS, SND_PCM_OSS
>   or SND_SEQUENCER_OSS are selected automatically.
> 
> - Tweak Makefile to use these new symbols to build these modules.
> 
> - Since we now build appropriate modules for core components according
>   to the new configuration symbols, (eg, snd-timer if SND_SEQUENCER is
>   selected) we can delete these duplications.

thanks, a nice work!

i vote for inclusion of this patchset.
(although we'll have a lot of work for 2.2/2.4 environment...)

--
Takashi Iwai <tiwai@suse.de>		ALSA Developer - www.alsa-project.org


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click

Re: [PATCH] 1/5 Clean up Linux 2.6.4-rc1 sound/core/Makefile

[Jaroslav Kysela]

On Mon, 1 Mar 2004, Takashi Iwai wrote:

> thanks, a nice work!
> 
> i vote for inclusion of this patchset.
> (although we'll have a lot of work for 2.2/2.4 environment...)

Yes, we need parsing of the Kconfig files now.

						Jaroslav

-----
Jaroslav Kysela <perex@suse.cz>
Linux Kernel Sound Maintainer
ALSA Project, SuSE Labs


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click

Re: [PATCH] 1/5 Clean up Linux 2.6.4-rc1 sound/core/Makefile

[Takashi Iwai]

At Mon, 1 Mar 2004 16:28:28 +0100 (CET),
Jaroslav wrote:
> 
> On Mon, 1 Mar 2004, Takashi Iwai wrote:
> 
> > thanks, a nice work!
> > 
> > i vote for inclusion of this patchset.
> > (although we'll have a lot of work for 2.2/2.4 environment...)
> 
> Yes, we need parsing of the Kconfig files now.

or, provide different Makefiles for old kernels?


Takashi


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click

Re: [PATCH] 1/5 Clean up Linux 2.6.4-rc1 sound/core/Makefile

[Jaroslav Kysela]

On Mon, 1 Mar 2004, Takashi Iwai wrote:

> > Yes, we need parsing of the Kconfig files now.
> 
> or, provide different Makefiles for old kernels?

I think that replacing mod-deps utility will be fine for the future 
maintenance.

						Jaroslav

-----
Jaroslav Kysela <perex@suse.cz>
Linux Kernel Sound Maintainer
ALSA Project, SuSE Labs


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click

Re: [PATCH] 1/5 Clean up Linux 2.6.4-rc1 sound/core/Makefile

[Russell King]

On Mon, Mar 01, 2004 at 04:27:42PM +0100, Takashi Iwai wrote:
> i vote for inclusion of this patchset.
> (although we'll have a lot of work for 2.2/2.4 environment...)

I plan to do some more of this over the coming weeks, but I'll pause
for the time being - I don't want to bloat my local tree with too
many ALSA changes.  IOW, I'll wait for this set to make their way
through your good selves and to Linus before submitting any further
changes of this type.  This will probably mean it'll be after 2.6.4
has been released.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 PCMCIA      - http://pcmcia.arm.linux.org.uk/
                 2.6 Serial core


-------------------------------------------------------
SF.Net is sponsored by: Speed Start Your Linux Apps Now.
Build and deploy apps & Web services for Linux with
a free DVD software kit from IBM. Click Now!
http://ads.osdn.com/?ad_id=1356&alloc_id=3438&op=click

Re: connection destruction question

[Kristen Carlson]

> Hi!
> 
> Did you take a look at the events used in nf_conntrack?
> I had a similar problem and I solved it that way. You may have to define 
> a new event (in ip_conntrack.h), or may not, I don't remember if it's 
> already included. if you have to use a new event, you can send it with 
> the ip_conntrack_cache_event() function.


	Hum, I don't see any such thing in my kernel version.  I'm using
a kernel for an embedded system, so it isn't straight forward for me to 
upgrade.  I should have specified: on 2.4.18.  Unless I am insane and 
this is really somewhere I didn't grep properly.  I didn't see it in
net/ipv4/netfilter or include/linux/netfilter_ipv4.

> 
> -Emmanuel
> 
> 
> Kristen Carlson wrote:
> 
> >Hello,
> >Is there a way to be notified in a kernel module 
> >when a connection is destroyed?  I have a nice helper
> >module that would like to know when connections are destroyed
> >so it can do some internal cleanup work. 
> >Thanks,
> >Kristen
> >
> >  
> >
> 
> 
> 
> 

Re: connection destruction question

[Henrik Nordstrom]

On Thu, 11 Mar 2004, Kristen Carlson wrote:

> > Did you take a look at the events used in nf_conntrack?
> 
> 	Hum, I don't see any such thing in my kernel version.  I'm using
> a kernel for an embedded system, so it isn't straight forward for me to 
> upgrade.  I should have specified: on 2.4.18.  Unless I am insane and 
> this is really somewhere I didn't grep properly.  I didn't see it in
> net/ipv4/netfilter or include/linux/netfilter_ipv4.

nf_conntrack is part of the nfnetlink-ctnetlink-0.13 patch in 
iptables patch-o-matic.

Regards
Henrik

Re: connection destruction question

[Emmanuel Guiton]

Hi!

No, you're not insane: you need the patch. I'm not sure it's available 
for 2.4.18, but at least it is for 2.4.23 (as it is what I use).

          Emmanuel



Kristen Carlson wrote:

>>Hi!
>>
>>Did you take a look at the events used in nf_conntrack?
>>I had a similar problem and I solved it that way. You may have to define 
>>a new event (in ip_conntrack.h), or may not, I don't remember if it's 
>>already included. if you have to use a new event, you can send it with 
>>the ip_conntrack_cache_event() function.
>>    
>>
>
>
>	Hum, I don't see any such thing in my kernel version.  I'm using
>a kernel for an embedded system, so it isn't straight forward for me to 
>upgrade.  I should have specified: on 2.4.18.  Unless I am insane and 
>this is really somewhere I didn't grep properly.  I didn't see it in
>net/ipv4/netfilter or include/linux/netfilter_ipv4.
>  
>

Re: Can compression at filesystem level improve overall performance?

[The Amazing Dragon]

> From: Hans Reiser <reiser@namesys.com>
> 
> Scott Young wrote:
> 
> >On Mon, 2004-03-22 at 15:04, Hans Reiser wrote:
> >
> >>Scott Young wrote:

> >>>The idea
> >>>can be applied to Reiser4 by compressing the overwrite set while the
> >>>journal data is being written, and then compressing the tail of the
> >>>relocate set moving backwards until the write stream catches up to the
> >>>compression.  It could also take into account the estimated
> >>>decompression time when reading the data back, and use it for deciding
> >>>whether the compression ratio is good enough to write the compressed
> >>>data instead of the uncompressed data.
> >>>
> >>I didn't understand the above.
> >
> >What I'm saying is that you can start writing uncompressed data to the
> >drive while the yet-to-be-written data is being compressed.  The goal is
> >to have some segments of data compressed, and have them compressed
> >before they come up next for writing to the drive.  Virtually no time is
> >lost if the data cannot be compressed because the data will be sent to
> >the disk at full speed anyway, whether or not the system had time to
> >compress it.  The repacker could compress the data that was written to
> >the drive before it could be compressed if the repacker thinks that
> >compression would speed up reading of that data (or significantly reduce
> >space usage, which will generally happen at the same time as data moving
> >faster because of compression).
> >
> Too much complexity.  LZW is fast and effective.

...and stupidly encumbered by patents (RSA at least was non-trivial and
non-obvious) in some countries (the USA patent has expired). LZ77
varients (libz) are quite effective too.


> >As another example, gcc could create a "live" derived file a.out that
> >that means "a.out is the live derivative of files src1.c, src2.c,
> >src3.c, src4.c when applied using the command gcc -o a.out src1.c src2.c
> >src3.c src4.c".  Of course it would be encoded in the syntax that the
> >derived-file plugin would understand.  When the source files change, the
> >developer wouldn't have to rerun gcc to run the executable.  The file
> >would automagically be compiled from the sources.
> >
> >The more I think about it, it seems that derived-files could be
> >implemented as plugin(s) with only one extra feature added to the core
> >filesystem: copy-on-write.  (Copy-on-write could be extended to only
> >write changes, and have the new version be constructed as a derived file
> >from the original, thereby compactly maintaining multiple versions of a
> >file and improving write performance, but I digress)
> >  
> Consider our use of compression atoms, it may reduce the incentive for 
> what you describe.

Where is this idea mentioned?


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM@cs.pdx.edu      PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: Can compression at filesystem level improve overall performance?

[Alan Horn]

On Tue, 23 Mar 2004, The Amazing Dragon wrote:

>> >
>> Too much complexity.  LZW is fast and effective.
>
>...and stupidly encumbered by patents (RSA at least was non-trivial and
>non-obvious) in some countries (the USA patent has expired). LZ77
>varients (libz) are quite effective too.


Forgive me.. I though LZW was still only encumbered in Canada and Japan ?

Cheers,

Al

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[The Amazing Dragon]

> From: Narcoleptic Electron <narcoleptic_electron@yahoo.co.uk>
> Hubert Chan wrote:
> > That effectively kills all filenames that contain @,
> 
> > much worse than
> > just a "metas" conflict IMHO.
> 
> I strongly agree:
> 
> - A restricted character in all names is far more
> likely to impact users than a single restricted name. 

This is why a couple candidates need to be brought up. Some seem good at
first glance, but are really bad choice after a bit of thought. So you're
right a badly chosen single character will be really bad. A good choice
needs to have minimal impact by being pretty much unused in filenames. A
corollary is that a badly chosen directory name will have a horrific
impact on users just as much as a badly chosen special character.

> I think that the meta directory needs to be thought of
> not as a special directory, but as a normal directory
> that has a name that is interpreted in a special way
> by the system.

In other words it is special, but it isn't special?

You can't have it both ways. "metas" is a perfectly valid filename on
all other filesystems. It is a valid word or partial word in several
languages, bad choice. Files/directories begining with . are at least
already handled specially by all tools. Names that are valid words are
precious, like gold you shouldn't steal them.

There is also the problem that things like Apache deliberately filter out
access to some files (like "..") because they're magic. By adding another
magic filename you've made those harder, at least begining it with . will
keep those tools' job easier (and you don't introduce a huge security
hole by adding a filesystem).

Also I feel it should be on the file itself. ie for the file /tmp/fooblah
you should be able to access the file's metadata by open()ing/using
readdir() on /tmp/fooblah/metas or (/tmp/fooblah/..metas or whatever).


> No one in this thread has commented on "+" as the
> default meta directory name (one of the final
> contenders in our previous thread on the subject). 
> Again, the reasons:
> - Short (one character)
> - Makes sense in all languages (meaning "additional
> information")
> - Available on all int'l keyboards

Bad choice. Note the "lost+found" directory found on *all* Unix
filesystems. If we need one more option | might be viable.

> Also, a question: are all meta files necessarily
> pseudo files?  Should users be able to put regular
> files in there to be interpreted as pseudo files? 
> This will help to clarify some things for me.

My impression is, that the metadata appears as normal files and is
accessible as normal files. Just gets interpreted in an interesting way.
I doubt metadata would have an owner or permissions separate from the
file/directory though. I imagine this is similar to Apple MacOS 10,
where all files have a mime-type that is accessible as
<filename>/mime-type.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM@cs.pdx.edu      PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Grant Miner]

> Also I feel it should be on the file itself. ie for the file /tmp/fooblah
> you should be able to access the file's metadata by open()ing/using
> readdir() on /tmp/fooblah/metas or (/tmp/fooblah/..metas or whatever).
> 
That's how it works now I believe.

> Bad choice. Note the "lost+found" directory found on *all* Unix
> filesystems. If we need one more option | might be viable.

Actually lost+found would not conflict if the directory were just + 
i.e. file/+/uid. Unless someone is proposing changing syntax so that 
accessing a file like foo@uid would get an attribute out of it (or 
foo+uid I suppose).  VMS does something similar, having a different 
syntax for each possible physical condition of a file (i.e., which 
device somethings on, which version, etc.)...a nightmare much worse than 
DOS with its drive letters.

The basic format for all VMS filenames is:
device:[directory.path]filename.extension;version

Example:
DISK$1:[JOE.PUBLIC_HTML]INDEX.HTML;1

There is no reason why all this information needs to be part of the syntax.

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Alexander G. M. Smith]

Elliott Mitchell wrote on Wed, 31 Mar 2004 21:46:14 -0800 (PST):
> Also I feel it should be on the file itself. ie for the file /tmp/fooblah
> you should be able to access the file's metadata by open()ing/using
> readdir() on /tmp/fooblah/metas or (/tmp/fooblah/..metas or whatever).

Sounds good to me.  I just hope that directory operations are
cheap in the file system.  The alternative of just adding a
..meta. prefix to all attribute names would cut out one level
of directories (less disk space usage, less lookup time to
resolve a path, and no worry about the weirdness of attributes
accidentally getting attached to the ..metas directory itself).

/tmp/fooblah/..metas/mime-type
/tmp/fooblah/..metas.mime-type

Hmmm.  Which is better?  The weirdness factor worries me
more than the performance reduction.

- Alex

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Narcoleptic Electron]

This discussion is spiralling out of control.  There
is far too much misunderstanding.  A discussion
summary is in order.

PROPOSALS:

1. Leave the "metas" directory as it is, for storing
the meta data hierarchy.

2. Rename "metas" to:
a) ..metas
b) @
c) +

3. Revise the architecture so that instead of putting
meta data into a directory, make it accessible via a
different path delimiter.  Proposals for a delimiter
include:
a) \
b) @
c) .

4. Return to the original approach of putting meta
data files into the parent directory, and prefixing
the name with:
a) ..metas.

PROBLEMS:

1. Name clashes with user files; "metas" does not have
meaning in all non-English languages; too long.

2. Name clashes with user files.
a) "..metas" does not have meaning in all non-English
languages; too long.
b) Conflict with an important mail application's
directory naming scheme.
c) No additional problems.

3. All file names containing the delimiter character
will cause problems; introduces a whole new [arguably
redundant] fundamental concept.  

4. A step backwards; dramatically increases chance of
name conflicts (since meta data is no longer in one
discrete location, but interspersed with user files).

If there are other approaches not addressed here, or
clarification required, please feel free to revise
this list.


______________________________________________________________________ 
Post your free ad now! http://personals.yahoo.ca

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Christian Iversen]

On Thursday 01 April 2004 18:31, Narcoleptic Electron wrote:
> If there are other approaches not addressed here, or
> clarification required, please feel free to revise
> this list.

I think someone suggested putting meta file information in /proc.

So. Is this a bad idea? I think not. 

In this scheme, /some/file would have /proc/metas/some/file/* as meta 
information. 

This reduces the risk of conflict to 0. 

One other idea I have, is "...". That way, 

dir/. is the directory itself
dir/.. is the parent
and
dir/... is the meta info.

Any comments?

-- 
Regards,
Christian Iversen

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Christian Mayrhuber]

On Thursday 01 April 2004 18:37, Christian Iversen wrote:
> On Thursday 01 April 2004 18:31, Narcoleptic Electron wrote:
> > If there are other approaches not addressed here, or
> > clarification required, please feel free to revise
> > this list.
>
> I think someone suggested putting meta file information in /proc.
>
> So. Is this a bad idea? I think not.
>
> In this scheme, /some/file would have /proc/metas/some/file/* as meta
> information.
Reiser4 would have to mimic the device structure in /proc.
If someone wants to access meta information he would have
to lookup the device of his filesystem.
What about nfs?
This is not compatible, nor comfortable.

>
> This reduces the risk of conflict to 0.
>
> One other idea I have, is "...". That way,
>
> dir/. is the directory itself
> dir/.. is the parent
> and
> dir/... is the meta info.
Really elegant!
Not perfect, tough - it is not descriptive, but "." and ".." are not
better in this regard, too.

I don't know any software using "..." files, so I'd say go for it, it's the
best suggestion so far, IMHO.

-- 
lg, Chris

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Narcoleptic Electron]

I've attempted to incorporate all the new proposals. 
Please let me know if I've missed any.

Narcoleptic Electron wrote: 
> 
> PROPOSALS:
> 
> 1. Leave the "metas" directory as it is, for storing
> the meta data hierarchy.
> 
> 2. Rename "metas" to:
> a) ..metas
> b) @
> c) +

d) ..meta
e) ...

> 3. Revise the architecture so that instead of
> putting
> meta data into a directory, make it accessible via a
> different path delimiter.  Proposals for a delimiter
> include:
> a) \
> b) @
> c) .
> 
> 4. Return to the original approach of putting meta
> data files into the parent directory, and prefixing
> the name with:
> a) ..metas.

5. Create a "meta" directory in "/proc" whose contents
mirror the root file system, and contains all meta
data.
 
> PROBLEMS:
> 
> 1. Name clashes with user files; "metas" does not
> have
> meaning in all non-English languages; too long.
> 
> 2. Name clashes with user files.
> a) "..metas" does not have meaning in all
> non-English
> languages; too long.
> b) Conflict with an important mail application's
> directory naming scheme.

And the djbdns utils (eg, dnscache), whic use files
starting with @ for cache files.

> c) No additional problems.

d) More readable than "metas" in English, but suffers
the same problems.
e) This is reserved by Windows, making a Windows port
of ReiserFS impossible (I think this was the reason;
it was an objection raised by someone else during the
previous thread on this subject).  (The "..." approach
was my original suggestion on the original thread...)

> 3. All file names containing the delimiter character
> will cause problems; introduces a whole new
> [arguably
> redundant] fundamental concept.  
> 
> 4. A step backwards; dramatically increases chance
> of
> name conflicts (since meta data is no longer in one
> discrete location, but interspersed with user
> files).

5. We would need a way to differentiate regular
subdirectories in "/proc/meta/" with meta data
entries.
For example, consider "/proc/meta/foo/bar/baz": is
"baz" a metadata entry for "/foo/bar", or is "bar/baz"
metadata for "/foo"? 

N. Electron


______________________________________________________________________ 
Post your free ad now! http://personals.yahoo.ca

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Hans Reiser]

Narcoleptic Electron wrote:

>This discussion is spiralling out of control.  There
>is far too much misunderstanding.  A discussion
>summary is in order.
>
>PROPOSALS:
>
>1. Leave the "metas" directory as it is, for storing
>the meta data hierarchy.
>  
>
We will do 1.

>2. Rename "metas" to:
>a) ..metas
>b) @
>c) +
>
>3. Revise the architecture so that instead of putting
>meta data into a directory, make it accessible via a
>different path delimiter.  Proposals for a delimiter
>include:
>a) \
>b) @
>c) .
>
>4. Return to the original approach of putting meta
>data files into the parent directory, and prefixing
>the name with:
>a) ..metas.
>
>PROBLEMS:
>
>1. Name clashes with user files; "metas" does not have
>meaning in all non-English languages; too long.
>
>2. Name clashes with user files.
>a) "..metas" does not have meaning in all non-English
>languages; too long.
>b) Conflict with an important mail application's
>directory naming scheme.
>c) No additional problems.
>
>3. All file names containing the delimiter character
>will cause problems; introduces a whole new [arguably
>redundant] fundamental concept.  
>
>4. A step backwards; dramatically increases chance of
>name conflicts (since meta data is no longer in one
>discrete location, but interspersed with user files).
>
>If there are other approaches not addressed here, or
>clarification required, please feel free to revise
>this list.
>
>
>______________________________________________________________________ 
>Post your free ad now! http://personals.yahoo.ca
>
>
>  
>


-- 
Hans

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Narcoleptic Electron]

Hans Reiser wrote: 

> Narcoleptic Electron wrote:
> 
> >1. Leave the "metas" directory as it is, for
> storing
> >the meta data hierarchy.
> >  
> >
> We will do 1.

What is the plan for addressing the name clash
problems that this causes?  (eg. I copy a directory,
that happens to contain a "metas" directory, to my
Reiser4 partition)


______________________________________________________________________ 
Post your free ad now! http://personals.yahoo.ca

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Hans Reiser]

Narcoleptic Electron wrote:

>Hans Reiser wrote: 
>
>  
>
>>Narcoleptic Electron wrote:
>>
>>    
>>
>>>1. Leave the "metas" directory as it is, for
>>>      
>>>
>>storing
>>    
>>
>>>the meta data hierarchy.
>>> 
>>>
>>>      
>>>
>>We will do 1.
>>    
>>
>
>What is the plan for addressing the name clash
>problems that this causes?  (eg. I copy a directory,
>that happens to contain a "metas" directory, to my
>Reiser4 partition)
>  
>
these problems will not exist significantly in reality.  Look at netapps 
and snapshots and clearcase and other filesystems, I remember wondering 
if .snapshot could be a problem when netapps were new and it was never a 
problem.

People who find it is a problem can #define it to something else.  If 5 
people bother to do so, I will be surprised.

Many languages have reserved keywords.....


>
>______________________________________________________________________ 
>Post your free ad now! http://personals.yahoo.ca
>
>
>  
>


-- 
Hans

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[cami]

>>>> 1. Leave the "metas" directory as it is, for storing
>>>> the meta data hierarchy.
>>>
>>> We will do 1.
>>
>> What is the plan for addressing the name clash
>> problems that this causes?  (eg. I copy a directory,
>> that happens to contain a "metas" directory, to my
>> Reiser4 partition)
>>
> these problems will not exist significantly in reality.  Look at netapps 
> and snapshots and clearcase and other filesystems, I remember wondering 
> if .snapshot could be a problem when netapps were new and it was never a 
> problem.
> 
> People who find it is a problem can #define it to something else.  If 5 
> people bother to do so, I will be surprised.
> 
> Many languages have reserved keywords.....

Not sure if anyone  has bothered to check if this would
impose the  limitation  that  people are worried about.

 From a  quick glance,  none  of  the linux distro's have
ever  had  a  file / directory  called  "metas"  before.
`metas`  isn't even a real a word anyway  (at  least not
an english word) so the chances of  it being a big issue
are very very  small..  freshmeat.net's search shows not
even one hit  for  the word  metas and  that pretty much
the majority of linux/coding related projects..

Regards,
Cami

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Narcoleptic Electron]

Hans Reiser wrote: 

> Narcoleptic Electron wrote:
> 
> >What is the plan for addressing the name clash
> >problems that this causes?  (eg. I copy a 
> >directory,
> >that happens to contain a "metas" directory, to my
> >Reiser4 partition)
>  
> these problems will not exist significantly in
> reality.  Look at netapps 
> and snapshots and clearcase and other filesystems, I
> remember wondering 
> if .snapshot could be a problem when netapps were
> new and it was never a 
> problem.

I do not disagree that it is unlikely.  However, it is
still possible, so my question remains: what happens
in the scenario I describe?

> People who find it is a problem can #define it to
> something else.  If 5 
> people bother to do so, I will be surprised.

True; as long as everyone refers to the "metas"
directory properly (using an environment variable, for
example, as opposed to hard-coding the string "metas"
anywhere), it will be fine.

> Many languages have reserved keywords.....

We're not talking about a language here, though...
we're talking about a namespace.  To me, the measure
of usefulness of a namespace is the ability to provide
names... reserved words hamper that ability.  The
perfect namespace has no reserved words; at worst,
only  reserved characters, with escape sequences
provided for each so that any name is still possible.


______________________________________________________________________ 
Post your free ad now! http://personals.yahoo.ca

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Hubert Chan]

>>>>> "Narcoleptic" == Narcoleptic Electron <narcoleptic_electron@yahoo.co.uk> writes:

[...]

Narcoleptic> True; as long as everyone refers to the "metas" directory
Narcoleptic> properly (using an environment variable, for example, as
Narcoleptic> opposed to hard-coding the string "metas" anywhere), it
Narcoleptic> will be fine.

Hmm.  Maybe have a /proc/fs/reiser4/metas file to query it?
(Environment variable seems fragile.)

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Christian Iversen]

On Sunday 04 April 2004 06:28, Hubert Chan wrote:
> >>>>> "Narcoleptic" == Narcoleptic Electron
> >>>>> <narcoleptic_electron@yahoo.co.uk> writes:
>
> [...]
>
> Narcoleptic> True; as long as everyone refers to the "metas" directory
> Narcoleptic> properly (using an environment variable, for example, as
> Narcoleptic> opposed to hard-coding the string "metas" anywhere), it
> Narcoleptic> will be fine.
>
> Hmm.  Maybe have a /proc/fs/reiser4/metas file to query it?
> (Environment variable seems fragile.)

Good idea. Of course, I will be the first one to set it to "..." ;-)

-- 
Regards,
Christian Iversen

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Narcoleptic Electron]

Christian Iversen wrote: 

> On Sunday 04 April 2004 06:28, Hubert Chan wrote:
>
> > >>>>> "Narcoleptic" == Narcoleptic Electron
> > >>>>> writes:
> >
> > Narcoleptic> True; as long as everyone refers to
> the "metas" directory
> > Narcoleptic> properly (using an environment
> variable, for example, as
> > Narcoleptic> opposed to hard-coding the string
> "metas" anywhere), it
> > Narcoleptic> will be fine.
> >
> > Hmm.  Maybe have a /proc/fs/reiser4/metas file to
> query it?
> > (Environment variable seems fragile.)
> 
> Good idea. Of course, I will be the first one to set
> it to "..." ;-)

Yes, a "/proc/fs/reiser4/metas" file containing the
meta directory name is a far better approach than
putting it in an environment variable (that was just
the first thing that came to my mind to demonstrate
the point).


______________________________________________________________________ 
Post your free ad now! http://personals.yahoo.ca

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Beni Cherniavsky]

Christian Iversen wrote:
> On Sunday 04 April 2004 06:28, Hubert Chan wrote:
> 
>>>>>>>"Narcoleptic" == Narcoleptic Electron
>>>>>>><narcoleptic_electron@yahoo.co.uk> writes:
>>
>>[...]
>>
>>Narcoleptic> True; as long as everyone refers to the "metas" directory
>>Narcoleptic> properly (using an environment variable, for example, as
>>Narcoleptic> opposed to hard-coding the string "metas" anywhere), it
>>Narcoleptic> will be fine.
>>
>>Hmm.  Maybe have a /proc/fs/reiser4/metas file to query it?
>>(Environment variable seems fragile.)
> 
That's the only right (and feasible) way to export it from the kernel.
However it's purpose is to allow writing scripts that are portable 
accross different definitions and this is much easier with an 
environment variable (cat foo/$REISER4FS_METAS/rwx) than with a file
(cat foo/`cat /proc/fs/reiser4/metas`/rwx).  So people will surely
set an environment variable from the /proc file in some shell init file.
Not that standardazing the name of the environment variable should 
bother us now.

> 
> Good idea. Of course, I will be the first one to set it to "..." ;-)
> 
Note that changing the magic name (dynamically or even by recompiling) 
can create a confilict with an *exisiting* file on a reiserfs4 partition 
(the file could have been created when another name was magic).  The FS 
code should be ready to handle it, preferably giving some escape route 
way to access the colliding file (e.g. ``foo/my_metas/escape`` could 
give access to the file known as ``foo/my_metas`` before you changed the 
magic name to ``my_metas``).

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Narcoleptic Electron]

Beni Cherniavsky wrote: 

> Note that changing the magic name (dynamically or
> even by recompiling) 
> can create a confilict with an *exisiting* file on a
> reiserfs4 partition 
> (the file could have been created when another name
> was magic).  The FS 
> code should be ready to handle it, preferably giving
> some escape route 
> way to access the colliding file (e.g.
> ``foo/my_metas/escape`` could 
> give access to the file known as ``foo/my_metas``
> before you changed the 
> magic name to ``my_metas``).

That is a very good point, and I think your solution
is the right way to get around it.


______________________________________________________________________ 
Post your free ad now! http://personals.yahoo.ca

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Hans Reiser]

Narcoleptic Electron wrote:

>Beni Cherniavsky wrote: 
>
>  
>
>>Note that changing the magic name (dynamically or
>>even by recompiling) 
>>can create a confilict with an *exisiting* file on a
>>reiserfs4 partition 
>>(the file could have been created when another name
>>was magic).  The FS 
>>code should be ready to handle it, preferably giving
>>some escape route 
>>way to access the colliding file (e.g.
>>``foo/my_metas/escape`` could 
>>give access to the file known as ``foo/my_metas``
>>before you changed the 
>>magic name to ``my_metas``).
>>    
>>
>
>That is a very good point, and I think your solution
>is the right way to get around it.
>
>
>______________________________________________________________________ 
>Post your free ad now! http://personals.yahoo.ca
>
>
>  
>
I think we should have a /metasoff hidden directory that presents the 
filesystem tree but without the metas.

Of course, now that we have funding for views, when that is completed 
you will be able to specify a view that cannot access metas.

-- 
Hans

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[John D. Heintz]

Hans Reiser wrote:

>
> Of course, now that we have funding for views, when that is completed 
> you will be able to specify a view that cannot access metas.
>
First, congratulations on getting the funding!

Second, I'm curious! Would views enable disambiguating conflicted names 
from plugins?

Thanks,
John Heintz

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Grant Miner]

Hubert Chan wrote:

>>>>>>"Narcoleptic" == Narcoleptic Electron <narcoleptic_electron@yahoo.co.uk> writes:
>>>>>>            
>>>>>>
>
>[...]
>
>Narcoleptic> True; as long as everyone refers to the "metas" directory
>Narcoleptic> properly (using an environment variable, for example, as
>Narcoleptic> opposed to hard-coding the string "metas" anywhere), it
>Narcoleptic> will be fine.
>
>Hmm.  Maybe have a /proc/fs/reiser4/metas file to query it?
>(Environment variable seems fragile.)
>
>  
>
You mean /sys/fs/reiser4/metas?  :P

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Hubert Chan]

>>>>> "Grant" == Grant Miner <mine0057@mrs.umn.edu> writes:

Grant> You mean /sys/fs/reiser4/metas?  :P

My /sys doesn't even have a fs subdirectory.  So I'm pretty sure I mean
/proc (unless you can tell me why it should be in /sys.  But AFAICT /sys
deals more with accessing hardware properties.).

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Grant Miner]

>I do not disagree that it is unlikely.  However, it is
>still possible, so my question remains: what happens
>in the scenario I describe?
>
>  
>
Hi
What happens now is open() returns EEXIST (file exists) error; reiser4 
can't make the file.  Is this what you mean?

starting with ".." could break stuff

[Tom Vier]

i bet there's a few apps that, when trying to skip .., only check the first
two chars of the name. this may or may not be a problem.

i'm not clear about how the metadata and plugins work. if you write to
..meta/blah there has to be a plugin registered for that meta filename,
right? does it keep the data in there ("..meta/blah") in some portian of the
normal, linear file? or is ..meta/ the only way to get at it? err - are we
talking about files as dirs? i'm confused. 8) i need some sleep.

-- 
Tom Vier <tmv@comcast.net>
DSA Key ID 0x15741ECE

Re: starting with ".." could break stuff

[mjt]

Tom Vier wrote:

>i bet there's a few apps that, when trying to skip .., only check the first
>two chars of the name. this may or may not be a problem.

Whatever its name is, the metas directory is invisible to readdir.
It's gratefully skipped every time.

>i'm not clear about how the metadata and plugins work. if you write to
>..meta/blah there has to be a plugin registered for that meta filename,

mjt@shrike:~/..metas$ touch foo
touch: cannot touch `foo': Permission denied
mjt@shrike:~/..metas$ cd plugin/
mjt@shrike:~/..metas/plugin$ touch foo
touch: cannot touch `foo': Permission denied

However, there may be big problems with someone mailing attachments
named ..metas/plugin/perm and so forth.
Also servers which blindly write data to the disk have to be secured.

>oes it keep the data in there ("..meta/blah") in some portian of the
>normal, linear file? or is ..meta/ the only way to get at it?

..metas/ is completely virtual.
It would be great to have some method of having the kernel, the fs,
or SOMETHING remember the changes.

Or do these changes to plugin settings still stay between reboots?
I'd like to hear Namesys' comment here, because I don't know.

-- 
mjt

Re: starting with ".." could break stuff

[Hubert Chan]

>>>>> "Tom" == Tom Vier <tmv@comcast.net> writes:

Tom> i bet there's a few apps that, when trying to skip .., only check
Tom> the first two chars of the name. this may or may not be a problem.

Then those apps are broken.  ..foo has always been a valid filename
(though highly uncommon).  ..metas doesn't introduce anything new -- it
only makes ..files more common.

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

filenames that can be safely stolen (was Re: starting with ".." could break stuff)

[Bennett Todd]

[-- Attachment #1: Type: text/plain, Size: 842 bytes --]

2004-04-02T16:29:12 Hubert Chan:
> Tom> i bet there's a few apps that, when trying to skip .., only check
> Tom> the first two chars of the name. this may or may not be a problem.
> 
> Then those apps are broken.  ..foo has always been a valid filename
> (though highly uncommon).  ..metas doesn't introduce anything new -- it
> only makes ..files more common.

Yup.

For another example, AFS steals "@sys"; it's a special black-magic
voodoo name that acts like a symlink to a machine-specific value,
used to set up baroque symlink trees so one uniform network
filesystem can support dispatching to appropriate versions of
machine-specific files like executables.

So another possibility worth considering might be @metas, with the
prior art of AFS to cite to people who complain about theoretical
problems:-).

-Bennett

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: filenames that can be safely stolen (was Re: starting with ".." could break stuff)

[Hans Reiser]

Bennett Todd wrote:

>2004-04-02T16:29:12 Hubert Chan:
>  
>
>>Tom> i bet there's a few apps that, when trying to skip .., only check
>>Tom> the first two chars of the name. this may or may not be a problem.
>>
>>Then those apps are broken.  ..foo has always been a valid filename
>>(though highly uncommon).  ..metas doesn't introduce anything new -- it
>>only makes ..files more common.
>>    
>>
>
>Yup.
>
>For another example, AFS steals "@sys"; it's a special black-magic
>voodoo name that acts like a symlink to a machine-specific value,
>used to set up baroque symlink trees so one uniform network
>filesystem can support dispatching to appropriate versions of
>machine-specific files like executables.
>
>So another possibility worth considering might be @metas, with the
>prior art of AFS to cite to people who complain about theoretical
>problems:-).
>
>-Bennett
>  
>
i care more about people 20 years from now complaining that reiserfs is 
ugly in its aesthetics than about unreal problems.

punctuation in method names is uglier than "metas"

-- 
Hans

Re: filenames that can be safely stolen (was Re: starting with ".." could break stuff)

[Claudio Martins]

On Saturday 03 April 2004 19:05, Hans Reiser wrote:
>
> i care more about people 20 years from now complaining that reiserfs is
> ugly in its aesthetics than about unreal problems.
>
> punctuation in method names is uglier than "metas"

 Meanwhile what do you suggest people who want to untar a tarball which has a 
directory named "metas" do?

Claudio

Re: filenames that can be safely stolen (was Re: starting with ".." could break stuff)

[Tom Vier]

On Sat, Apr 03, 2004 at 10:05:22AM -0800, Hans Reiser wrote:
> i care more about people 20 years from now complaining that reiserfs is 
> ugly in its aesthetics than about unreal problems.
> 
> punctuation in method names is uglier than "metas"

maybe if you try to pronounce it, but imho, making "metas" a reserved name
is not a good idea and is very ugly. taking a somewhat common word (though
"meta" is probably more common than "metas") and reserving seems like a huge
wart to me. if you have to reserve an inband name, it should be one that,
while containing mostly a descriptive english word, doesn't trample regular
filenames (eg, more than one dot).

and finally, i hate the 's'. 8) for one thing, it might only contain only
one file.

-- 
Tom Vier <tmv@comcast.net>
DSA Key ID 0x15741ECE

Re: filenames that can be safely stolen

[Hubert Chan]

>>>>> "Tom" == Tom Vier <tmv@comcast.net> writes:

[...]

Tom> and finally, i hate the 's'. 8) for one thing, it might only
Tom> contain only one file.

It will always contain multiple files.  See
http://namesys.com/v4/pseudo.html for the files that it contains by
default.

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[The Amazing Dragon]

> From: Narcoleptic Electron <narcoleptic_electron@yahoo.co.uk>
> 
> I've attempted to incorporate all the new proposals. 
> Please let me know if I've missed any.
> 
> Narcoleptic Electron wrote: 
>  
> > PROBLEMS:
> > 
> > 1. Name clashes with user files; "metas" does not
> > have
> > meaning in all non-English languages; too long.

The latter two are pretty trivial, how meaningful is "lost+found" in
other languages, five characters is that long?

I feel the first item on that list is fatal though. I be *very* worried
about the ReiserFS v4 patch being rejected from the kernel due to this.
Real filenames in any language should be considered precious, and not
stolen without a *very* good reason. This is a decent reason, but utterly
short of being good enough.

> > b) Conflict with an important mail application's
> > directory naming scheme.
> 
> And the djbdns utils (eg, dnscache), whic use files
> starting with @ for cache files.

In other words, problems with multiple applications; fatal flaw. Though
not a common one, short and therefore precious.

> > c) No additional problems.

Still short and therefore precious. There could be multiple unknown
applications that use it. Already commonly used in filenames (though not
by itself).

> e) This is reserved by Windows, making a Windows port
> of ReiserFS impossible (I think this was the reason;
> it was an objection raised by someone else during the
> previous thread on this subject).  (The "..." approach
> was my original suggestion on the original thread...)

On which systems you could use / as the metadata separator.

> > 3. All file names containing the delimiter character
> > will cause problems; introduces a whole new
> > [arguably
> > redundant] fundamental concept.  

Not without president though, notably MacOS 10 metadata appears when
accessing a file as a directory. Only MacOS 10 doesn't include a new
separator.

> 5. We would need a way to differentiate regular
> subdirectories in "/proc/meta/" with meta data
> entries.
> For example, consider "/proc/meta/foo/bar/baz": is
> "baz" a metadata entry for "/foo/bar", or is "bar/baz"
> metadata for "/foo"? 

The problems are *much* bigger than this. For one thing, there is the
already mentioned problem, this completely breaks across network
filesystems. The *very* much larger problem is the overhead. If I want
to access the metadata for an arbitrary file you have to use getwd()
which may in turn stat() every file in several directories. If you're
interacting with a user, the overhead isn't bad, but if you're doing a
couple hundred of these overhead is a severe problem.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM@cs.pdx.edu      PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[The Amazing Dragon]

> From: Hans Reiser <reiser@namesys.com>
> these problems will not exist significantly in reality.  Look at netapps 
> and snapshots and clearcase and other filesystems, I remember wondering 
> if .snapshot could be a problem when netapps were new and it was never a 
> problem.

Notice though that that filename begins with ".", not a letter. This
causes all programs to treat it specially. Also note that that filename
is nine characters long, and therefore making a purely random collision
less likely by more than four orders of magnitude.

> People who find it is a problem can #define it to something else.  If 5 
> people bother to do so, I will be surprised.
> 
> Many languages have reserved keywords.....

I REJECT THIS!

I believe Ada is almost the only programming language without reserved
words. The thing is a filesystem is NOT a programming language! It is
designed to handle files with arbitrary names, no matter how odd. Only
programmers deal with C, end users must deal with filesystem limitations.

> From: cami <camis@mweb.co.za>
> Not sure if anyone  has bothered to check if this would
> impose the  limitation  that  people are worried about.
> 
>  From a  quick glance,  none  of  the linux distro's have
> ever  had  a  file / directory  called  "metas"  before.
> `metas`  isn't even a real a word anyway  (at  least not
> an english word) so the chances of  it being a big issue
> are very very  small..  freshmeat.net's search shows not
> even one hit  for  the word  metas and  that pretty much
> the majority of linux/coding related projects..

Good point, search engines as evidence. The problem is you're only
looking at distributions, which are going to be highly similar and you're
completely missing end users. So let us take this to a full search engine
and see what turns up...  Hmm, roughly a million hits, let us look at a
few samples:

http://www.metas.ch/
http://vancouver-webpages.com/META/mk-metas.html
http://www.metas.com.br/
http://metas.enfermeria21.com/
http://www.metas.com.mx/

Okay, out of one million hits, we randomly look at ten, and half feature
"metas" in the URL somewhere. Going the other direction, Google indexes
roughly 4 billion pages. If we guess the above search was representative,
roughly 500,000 pages will include "metas" somewhere in the URL, possibly
only as a hostname, but somewhere. So we've managed to collect 1 out of
every 10,000 pages that Google indexes. Though we don't have a direct
proof, I hope I've come close enough to scare you.

Hans, what will it take for you to change your mind?


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM@cs.pdx.edu      PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Hubert Chan]

>>>>> "The" == The Amazing Dragon (Elliott Mitchell) <ehem@cs.pdx.edu> writes:

[...]

The> Good point, search engines as evidence. The problem is you're only
The> looking at distributions, which are going to be highly similar and
The> you're completely missing end users. So let us take this to a full
The> search engine and see what turns up...  Hmm, roughly a million
The> hits, let us look at a few samples:

I'm only getting a bit less than 600,000.  Maybe we caught Google in the
middle of some reindexing.  But anyways...

Most of the pages seem to be Spanish and Portuguese.  Google translation
translates "metas" as "goals".  Are there any Spanish or Portuguese
speakers on this list to confirm that?

Since "goals" is a fairly common word, it probably is a bad idea to use
"metas" without at least some sort of prefix.

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Marcelo Pacheco]

Exactly. Metas is goals/objectives in portuguese. Although it's extremelly 
uncommon to have files without extensions, I would still much rather prefer 
having .metas instead of metas.

Marcelo Pacheco

On Monday 05 April 2004 02:01, Hubert Chan wrote:
> >>>>> "The" == The Amazing Dragon (Elliott Mitchell) <ehem@cs.pdx.edu>
> >>>>> writes:
>
> [...]
>
> The> Good point, search engines as evidence. The problem is you're only
> The> looking at distributions, which are going to be highly similar and
> The> you're completely missing end users. So let us take this to a full
> The> search engine and see what turns up...  Hmm, roughly a million
> The> hits, let us look at a few samples:
>
> I'm only getting a bit less than 600,000.  Maybe we caught Google in the
> middle of some reindexing.  But anyways...
>
> Most of the pages seem to be Spanish and Portuguese.  Google translation
> translates "metas" as "goals".  Are there any Spanish or Portuguese
> speakers on this list to confirm that?
>
> Since "goals" is a fairly common word, it probably is a bad idea to use
> "metas" without at least some sort of prefix.

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Hans Reiser]

Marcelo Pacheco wrote:

>Exactly. Metas is goals/objectives in portuguese. 
>
>  
>
New users hate languages with lots of punctuation in the keywords.  Mr. 
Demidov correctly reminded me of this, and of the history of users 
hating such languages, and this caused me to abandon ..metas for metas.  
Mr Demidov, can you say more about this history?

What about "meths" (short for methods)?  Is that a word in any language?

-- 
Hans

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Nikita Danilov]

Hans Reiser writes:
 > Marcelo Pacheco wrote:
 > 
 > >Exactly. Metas is goals/objectives in portuguese. 
 > >
 > >  
 > >
 > New users hate languages with lots of punctuation in the keywords.  Mr. 
 > Demidov correctly reminded me of this, and of the history of users 
 > hating such languages, and this caused me to abandon ..metas for metas.  

Hmm... last time you claimed that is were users on reiserfs-list who won
an argument with you about naming. Now it turns that nobody except you
is in favor of this name.

 > Mr Demidov, can you say more about this history?
 > 
 > What about "meths" (short for methods)?  Is that a word in any language?

It means "methylated spirits" in... English. Probably refers to the
state of mind it takes to deal with "unified namespace".

 > 
 > -- 
 > Hans
 > 

Nikita.

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 285 bytes --]

On Tue, 06 Apr 2004 07:30:07 PDT, Hans Reiser said:

> What about "meths" (short for methods)?  Is that a word in any language?

Well, there's crystal meth, but I'd consider it highly unlikely that anybody
using that stuff can get their act together enough to care about metadata. :)


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[camis]

>What about "meths" (short for methods)?  Is that a word in any language?

Meths is word already in english..

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Nikita Danilov]

camis writes:
 > >What about "meths" (short for methods)?  Is that a word in any language?
 > 
 > Meths is word already in english..

Indeed, meths is plural of meth which is (rare) synonym for "a meath",
"a mead", Russian "m'od":

From Webster's Revised Unabridged Dictionary (1913) [web1913]:

  Meath \Meath\, Meathe \Meathe\, n. [See {Mead}.]
     A sweet liquor; mead. [Obs.] --Chaucer. Milton.

Nikita.

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 367 bytes --]

On Tue, 06 Apr 2004 23:05:45 +0400, Nikita Danilov said:
>   Meath \Meath\, Meathe \Meathe\, n. [See {Mead}.]
>      A sweet liquor; mead. [Obs.] --Chaucer. Milton.

On the other hand, both those Chaucer and Milton blokes have
been dust for quite some time, and the language has moved on.

Does anybody outside the Renaissance Fair circuit still even drink mead? ;)


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Richard Heycock]

On Wed, 2004-04-07 at 05:14, Valdis.Kletnieks@vt.edu wrote:
> On Tue, 06 Apr 2004 23:05:45 +0400, Nikita Danilov said:
> >   Meath \Meath\, Meathe \Meathe\, n. [See {Mead}.]
> >      A sweet liquor; mead. [Obs.] --Chaucer. Milton.
> 
> On the other hand, both those Chaucer and Milton blokes have
> been dust for quite some time, and the language has moved on.
> 
> Does anybody outside the Renaissance Fair circuit still even drink mead? ;)

I'm rather partial to a drop of mead if I can get my hands on it? Though
metadata is not something I think about if I do drink it!

rgh

-- 
"It is possible to make things of great complexity out of things
 that are very simple. There is no conservation of simplicity"
 -- Stephen Wolfram

Richard Heycock <rgh@roughage.com.au>
tel : 0410 646 369
key fingerprint : 909D CBFA C669 AC2F A937 AFA4 661B 9D21 EAAB 4291

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[mjt]

Valdis Kletnieks wrote:

>Does anybody outside the Renaissance Fair circuit still even drink mead? ;)

Yes, as a matter of fact they do ;)

I don't know how typical it is even for a Finn, but I prefer mead over
most other beverages, and even have some recipes for it.

But to get back to the point. "meths" is NOT a good word, like crystal
meth or whatever, it's almost more a word than "metas" (maybe most people
would just use "meta" as a directory name)

I mean, most of this discussion has been, pardon my French, bullshit.
..metas/ is still a good solution, it's prefixed in a manner that most
stuff is not, the prefix makes sense as being "really hidden" and metas
is not as probable a word as meta and it's still easily accessible.
Hell, it used to be prefixed like that before it was a unified
directory.

Whatever. I'm tired. I'm sure everyone has their opinion on this matter
and I'm also sure the guys at Namesys are farting blood because of
this discussion. Whatever people want as their meta directory name they
will patch or macro or edit in. If there's enough dispersion it will
be shown in /proc/ or /sys/ or wherever, even if not an environmental
variable.

Let's be reasonable and concentrate on something more important, like
brewing good mead or how to protect the metadata integrity or
something.

I, for one, will not participate in this thread again.

-- 
mjt

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Hubert Chan]

>>>>> "Hans" == Hans Reiser <reiser@namesys.com> writes:

Hans> New users hate languages with lots of punctuation in the keywords.
Hans> Mr. Demidov correctly reminded me of this, and of the history of
Hans> users hating such languages, and this caused me to abandon ..metas
Hans> for metas.  Mr Demidov, can you say more about this history?

Well, I don't know exactly what Mr. Demidov was thinking, but I know
that one common complaint about Perl is the punctuation in front of
variable names.  However, for UNIX filesystems, there's a lot of
precedent for at least a single leading dot in filenames, so I don't
think it should be a problem to call it ".metas".  It is also not
something that would be extremely common, so it wouldn't be like having
a Perl program pocked with punctuation.  (Alliteration unintentional.)

The leading dot also flags the file as being somewhat "special" -- two
leading dots would flag it as being more so.  Although my own personal
sense of aesthetics is biased against two dots, using two dots would be
well justified, and I personally wouldn't consider it a critical issue.

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Miguel]

On Mon, 05 Apr 2004 01:01:39 -0400
Hubert Chan <hubert@uhoreg.ca> wrote:

> >>>>> "The" == The Amazing Dragon (Elliott Mitchell) <ehem@cs.pdx.edu>
> >writes:
> 
> [...]
> 
> The> Good point, search engines as evidence. The problem is you're
> only The> looking at distributions, which are going to be highly
> similar and The> you're completely missing end users. So let us take
> this to a full The> search engine and see what turns up...  Hmm,
> roughly a million The> hits, let us look at a few samples:
> 
> I'm only getting a bit less than 600,000.  Maybe we caught Google in
> the middle of some reindexing.  But anyways...
> 
> Most of the pages seem to be Spanish and Portuguese.  Google
> translation translates "metas" as "goals".  Are there any Spanish or
> Portuguese speakers on this list to confirm that?

Yes, This is correct and one time I see a project that used a file
called metas in similar mean of the TODO file to state its goals,  but
nothing to care about IMHO.

ermmmm a dot doens't hurt... 


> 
> Since "goals" is a fairly common word, it probably is a bad idea to
> use"metas" without at least some sort of prefix.


-- 
La resistencia es fútil todos seréis asimilados

Re: [PATCH] "metas" in reiserfs v4 snapshot 2004.03.26

[Cami]

> Good point, search engines as evidence. The problem is you're only
> looking at distributions, which are going to be highly similar and you're
> completely missing end users. So let us take this to a full search engine
> and see what turns up...  Hmm, roughly a million hits, let us look at a
> few samples:
> 
> http://www.metas.ch/
> http://vancouver-webpages.com/META/mk-metas.html
> http://www.metas.com.br/
> http://metas.enfermeria21.com/
> http://www.metas.com.mx/
> 
> Okay, out of one million hits, we randomly look at ten, and half feature
> "metas" in the URL somewhere. Going the other direction, Google indexes
> roughly 4 billion pages. If we guess the above search was representative,
> roughly 500,000 pages will include "metas" somewhere in the URL, possibly
> only as a hostname, but somewhere. So we've managed to collect 1 out of
> every 10,000 pages that Google indexes. Though we don't have a direct
> proof, I hope I've come close enough to scare you.

Quite true.. "..metas" or ".metas" would be the better choice..

Regards,
Cami

Re: "Metas"

[The Amazing Dragon]

> From: Grant Miner <mine0057@mrs.umn.edu>
> > I don't think I've heard a single person, except Hans,
> > voice any support for "metas".
> 
> I like "metas".

Chalk me up as neutral about the name itself. That is I'm neutral about
".metas" or "..metas", I'm firmly against using "metas" (or any other
name) without at least a leading period. I suspect Linus et al might very
well reject such a FS because this would be a fatal flaw. I'm concerned
about using such a short and therefore precious name for filesystem
functions, but OTOH it might well be accessed often by a user and
therefore appropriate.

> > It seems that the most widely supported name thus far
> > is "...".

> Maybe "props" would be a good name?  It is short for properties, which 
> is synonymous with metadata and attributes (oops, I used a heretical 
> word! :)

Might ".properties", ".metadata" or ".attributes" work? Programs
shouldn't have a problem with a 12 character string occuring once in
their code. From a shell you'll be able to use filename completion, or
11 characters isn't that many to type. (and the leading dot avoids
breaking zillions of programs with security implications)


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM@cs.pdx.edu      PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

".meta." as a Name Prefix

[Alexander G. M. Smith]

Elliott Mitchell wrote on Fri, 16 Apr 2004 19:55:54 -0700 (PDT):
> > From: Grant Miner <mine0057@mrs.umn.edu>
> > I like "metas".
> 
> Chalk me up as neutral about the name itself. That is I'm neutral about
> ".metas" or "..metas", I'm firmly against using "metas" (or any other
> name) without at least a leading period. I suspect Linus et al might very
> well reject such a FS because this would be a fatal flaw. I'm concerned
> about using such a short and therefore precious name for filesystem
> functions, but OTOH it might well be accessed often by a user and
> therefore appropriate.

I agree, it needs to start with a period.  I'd also drop the idea
of having a separate directory for metadata and just use a common
name prefix.  ".meta.mime" for example, rather than ".meta/mime".

I'm probably repeating myself, but the trouble is that the .meta
directory isn't a full directory - you can't add attributes to it.
In BeOS, that's useful - the directory's GUI window coordinates
are stored as attributes on the directory.  I'd like to use the
standard GUI file explorer to also check on attributes!  With a
.meta directory, it seems good in a psychological organizational
way but it breaks the object model.  That breakage will cause more
programming annoyances than using a prefix instead, in my humble
opinion.

- Alex

Re: ".meta." as a Name Prefix

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Alexander G. M. Smith wrote:

| I agree, it needs to start with a period.  I'd also drop the idea
The namespace of beginning with one period is almost as full as that of
no periods, as mentioned before.  Our best bet for uniqueness is to call
it "reiser4metas" or something similar (.r4 or ..r4).

| of having a separate directory for metadata and just use a common
That's what we were doing before.  I didn't like it -- consider the use
on files:

touch foo
chmod +x foo
cd foo/
ls

You either get 'foo/' or an error message.  You do not get a list of all
the pseudo files available.  I'm sorry, but 'cat ..pseudo' just isn't a
viable substitute for ls.

Also, the only disadvantage from the user's point of view is an extra '/'.

| I'm probably repeating myself, but the trouble is that the .meta
| directory isn't a full directory - you can't add attributes to it.
What kind of attributes would you want to add?  And btw, check out what
is currently 'metas/metas'.  I don't know if that's actually functional
or just read-only, but it works.  And I also see no reason why (in raw
uneducated theory) a plugin couldn't be implemented to make it into a
full directory for certain files (and thus losing no functionality for
other files).

| standard GUI file explorer to also check on attributes!  With a
| .meta directory, it seems good in a psychological organizational
| way but it breaks the object model.  That breakage will cause more

It doesn't have to.  Look at how the "pure object-oriented" languages
work.  Many of them make everything some subclass of "object" -- even
classes themselves.  A class is an object of type class.  I'm not sure
how this is implemented in code, but in concept it works fine.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQIXHsngHNmZLgCUhAQLLlw//a6cy5K4mI0OGoDuyTSuaedI9TSy/GHEX
UBWBynKUBQ3CNrTQoQzvfOOZIGJXPvWPSSTS286KVCgXAeiWvjNvPwSReqmuajbl
T5jxHpTxFcU6H3BzKbDiYcpGiqifoRydjkcj5jMJkvW1GNEVHX358IZp3N+PzxE2
NCWqdQoO1+/rt0laxNK5EtGLWX0PD77Qfx/0FNh3/NKCOIEULOkUbXL7CJ3M9AEP
ysKUq5ZLDgqi4MS5YFnkyw95ZireHGKkJC49DmA/dpOvlFJdrCcZfwWjBtbwXk5K
znCTPJZlylSw7C8nFxDAylv5vlz8DNSNzZ0MuxtWQPGbah8FanYdVbaSbf0jmCXc
9uP1FHMGTO7PdhLY5conDiWtHuHFwNt4I3OvVxKzcFCtiRF19i9//mdSZeWt1GL9
xWonh6O7pODQP+C63+SLWGu38S1QgI6MNVSn9BvSiqZ8VaT1mcKrKU4Tp5e15PRp
l/Hr9n9BXeBdl/Pz/wzKnzzxCDQ13kEuZNkpHWw3KfGYokLdoc1yB702kJ6yP9pv
eU1c+1LUSOoaJV+iRfZt9OrJiNT+euHmH/ej+Nc6KXZpxmrPb7j0leery6xyfW+t
bU/jbmn/VcOVfDOyHim9MpIoBhBuvZIAurFyNW04ezMB97D9cj2P1oJHIrHoat6R
W/yNeXayRto=
=TYLn
-----END PGP SIGNATURE-----

Re: ".meta." as a Name Prefix

[John D. Heintz]

David Masover wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Alexander G. M. Smith wrote:
> 
> | I agree, it needs to start with a period.  I'd also drop the idea
> The namespace of beginning with one period is almost as full as that of
> no periods, as mentioned before.  Our best bet for uniqueness is to call
> it "reiser4metas" or something similar (.r4 or ..r4).

On the trail to uniqueness I've got an idea that I'd like to send out. 
How about we introduce two pseudo names: one very unique and the other 
short and simple.

The first unique name is like a URN:
foo/urn:namesys.com:reiser4:meta/uid

This is certainly unique enough for automated tools to rely on, but I 
wouldn't want to type it all the time.

The second name is short and easy:
foo/metas/uid

This is easy to use but could conflict. It would also be very useful to 
be able to lookup and dynamically change the short name.

cat urn:namesys.com:reiser4:meta/shortName
echo "..meta" > urn:namesys.com:reiser4:meta/shortName

I don't know if this echo would change the short name for the whole 
filesystem, a single object, or hierachical override. Also, it is 
interesting to wonder about this change persisting.


> 
> | of having a separate directory for metadata and just use a common
> That's what we were doing before.  I didn't like it -- consider the use
> on files:
> 
> touch foo
> chmod +x foo
> cd foo/
> ls
> 
> You either get 'foo/' or an error message.  You do not get a list of all
> the pseudo files available.  I'm sorry, but 'cat ..pseudo' just isn't a
> viable substitute for ls.
> 
> Also, the only disadvantage from the user's point of view is an extra '/'.
> 
> | I'm probably repeating myself, but the trouble is that the .meta
> | directory isn't a full directory - you can't add attributes to it.
> What kind of attributes would you want to add?  And btw, check out what
> is currently 'metas/metas'.  I don't know if that's actually functional
> or just read-only, but it works.  And I also see no reason why (in raw
> uneducated theory) a plugin couldn't be implemented to make it into a
> full directory for certain files (and thus losing no functionality for
> other files).
> 
> | standard GUI file explorer to also check on attributes!  With a
> | .meta directory, it seems good in a psychological organizational
> | way but it breaks the object model.  That breakage will cause more
> 
> It doesn't have to.  Look at how the "pure object-oriented" languages
> work.  Many of them make everything some subclass of "object" -- even
> classes themselves.  A class is an object of type class.  I'm not sure
> how this is implemented in code, but in concept it works fine.
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.4 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
> 
> iQIVAwUBQIXHsngHNmZLgCUhAQLLlw//a6cy5K4mI0OGoDuyTSuaedI9TSy/GHEX
> UBWBynKUBQ3CNrTQoQzvfOOZIGJXPvWPSSTS286KVCgXAeiWvjNvPwSReqmuajbl
> T5jxHpTxFcU6H3BzKbDiYcpGiqifoRydjkcj5jMJkvW1GNEVHX358IZp3N+PzxE2
> NCWqdQoO1+/rt0laxNK5EtGLWX0PD77Qfx/0FNh3/NKCOIEULOkUbXL7CJ3M9AEP
> ysKUq5ZLDgqi4MS5YFnkyw95ZireHGKkJC49DmA/dpOvlFJdrCcZfwWjBtbwXk5K
> znCTPJZlylSw7C8nFxDAylv5vlz8DNSNzZ0MuxtWQPGbah8FanYdVbaSbf0jmCXc
> 9uP1FHMGTO7PdhLY5conDiWtHuHFwNt4I3OvVxKzcFCtiRF19i9//mdSZeWt1GL9
> xWonh6O7pODQP+C63+SLWGu38S1QgI6MNVSn9BvSiqZ8VaT1mcKrKU4Tp5e15PRp
> l/Hr9n9BXeBdl/Pz/wzKnzzxCDQ13kEuZNkpHWw3KfGYokLdoc1yB702kJ6yP9pv
> eU1c+1LUSOoaJV+iRfZt9OrJiNT+euHmH/ej+Nc6KXZpxmrPb7j0leery6xyfW+t
> bU/jbmn/VcOVfDOyHim9MpIoBhBuvZIAurFyNW04ezMB97D9cj2P1oJHIrHoat6R
> W/yNeXayRto=
> =TYLn
> -----END PGP SIGNATURE-----
> 
> 

Re: ".meta." as a Name Prefix

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

|> cat urn:namesys.com:reiser4:meta/shortName
|> echo "..meta" > urn:namesys.com:reiser4:meta/shortName

Revisiting this idea:  this is a good idea for what happens _inside_ the
metas dir, for plugins.  But metas itself is a general enough,
well-enough designed and probably lasting concept that I think we can
use something like '...' or '..metas', or whatever is reasonably unique
_now_, and assume that once reiser4 hits mainstream usage, people will
be careful to develop around that name.

Inside the metas dir, though, we have plugins.  And if plugins could be
dynamic, even mostly in userspace, then we have the problem of plugins
becoming so popular that we have namespace collisions there.  I like
this solution, though -- it's kind of like the XML namespace concept,
something which should be in every language.

Using some mechanism, not sure what yet, you create a set of mappings:

meta/r4 -> meta/namesys.com/reiser4
meta/ -> meta/namesys.com/reiser4

They aren't really symlinks because the second option is for making all
children of meta/namesys.com/reiser4 available under meta/, so if you had:

meta/namesys.com/reiser4/bmap
meta/namesys.com/reiser4/plugin
meta/slaphack.com/happyface/:)

Now you have:

meta/bmap
meta/namesys.com/reiser4/bmap
meta/namesys.com/reiser4/plugin
meta/plugin
meta/slaphack.com/happyface/:)

More specific things would override less specific things, and in general
these mappings aren't allowed to touch anything that isn't itself a mapping.

I think you would use symlink() for setting these things up (cd meta; ln
~ -s namesys.com/reiser4 r4 || ln -s namesys.com/reiser4 .), and even if
they were completely dynamic, the files themselves would read like symlinks.

I heard someone say something like "reiser4 is almost shipped", so it
may be too late to send in ideas (especially ones which I've probably
said before), but I thought I'd drop this in anyway.  (sorry for the
noise if I _have_ said it before.)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQItQOngHNmZLgCUhAQL9yhAAiaCH3ZxSS5T/VxgqW+Tk4jR3kjzA87Cq
dMfMFlGeS2R1X3p1nuqdb0Q8SWSvMV2cV/YgFXGq4BR+nMjTy/krm7AlXJ1Wj8ji
xORQMWWZAWsFTeIoQNgu1lDpTCfPISzsmEgLeYsq14UQn+cb8uZMIfy9/tmmY71N
W6XIsaW2hHp2LwgSsrIjZHthKZQ6WnQh0nsI/8JlU5hjJ4sbEh4SWMokDri//kGq
wiXk/489sx1u4k/ymU/CHZGSJA4c+/uBzW2QZ2qRnpa6uNwWtu/lqQhn28tkoHRw
k9TS9j0Ca0w/4k9UZFD8N1Lkxp52/XFTg+U+ZABzkIQx9VHC+nVgaVLIZSr31C4/
iTk+sHdY55kJtHOF8X7cSzx0D/V+Qo2owhOFS9B5/10NEv34UPmO4aQq9T/MeQVo
vqWF5L8h0N+zLqlTnkkf4zVBzALfTbsfk18fLeu42j8WZfkVBkHC5T2SBYjmuPHO
b3bNX1yl+xx0lFG4Wt6bz6sYhzmr2tOj1ayVEYdpZRsXZUsSwom5POeD5Yimuhd7
/VCiNhcNm17bfqOTK+e+36dE8c91YIyv32e3BBivrB2GYg1AgGfPuzMOky78GX/m
Ymh4ztKBAwoBytvsF4Aye+8h3bETBMjMoNhJT7EcvH/N4MFqtfZJLh6PafIXMIgA
IeMzRfndw1w=
=EGMz
-----END PGP SIGNATURE-----

Re: ".meta." as a Name Prefix

[Alexander G. M. Smith]

David Masover wrote on Sun, 25 Apr 2004 00:44:27 -0500:
> Inside the metas dir, though, we have plugins.  And if plugins could be
> dynamic, even mostly in userspace, then we have the problem of plugins
> becoming so popular that we have namespace collisions there.  I like
> this solution, though -- it's kind of like the XML namespace concept,
> something which should be in every language.

Actually the problem in real life (from BeOS experience) is the lack of
collisions.   Everybody keeps on prefixing their new attribute set with
a unique identifier (usually looking like "UNIQUER:attribute" in BeOS)
so you end up with redundancies like MAIL:subject and PINEAPPLE:subject
which should really be the same thing (since e-mail messages and UseNet
news are so similar and I want to search all subjects when looking
for a lost message).  Of course, this could be worked around by having
aliases of attribute names (sounds like another plugin), but it would
be nice if we had a standard set of attribute names.  Dublin core, anyone?

- Alex

Re: ".meta." as a Name Prefix

[John D. Heintz]

David,

Something I think I've understood from the code is that plugins aren't 
restricted to the metas directory. Any PSEUDO_PLUGIN can create a root 
name that is looked up everywhere.

That doesn't mean that PSEUDO_PLUGINs that do this will be accepted into 
the codebase or distributions, but there is no actual boundary between 
adding new names to foo/ instead of foo/metas.

John

David Masover wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> |> cat urn:namesys.com:reiser4:meta/shortName
> |> echo "..meta" > urn:namesys.com:reiser4:meta/shortName
> 
> Revisiting this idea:  this is a good idea for what happens _inside_ the
> metas dir, for plugins.  But metas itself is a general enough,
> well-enough designed and probably lasting concept that I think we can
> use something like '...' or '..metas', or whatever is reasonably unique
> _now_, and assume that once reiser4 hits mainstream usage, people will
> be careful to develop around that name.
> 
> Inside the metas dir, though, we have plugins.  And if plugins could be
> dynamic, even mostly in userspace, then we have the problem of plugins
> becoming so popular that we have namespace collisions there.  I like
> this solution, though -- it's kind of like the XML namespace concept,
> something which should be in every language.
> 
> Using some mechanism, not sure what yet, you create a set of mappings:
> 
> meta/r4 -> meta/namesys.com/reiser4
> meta/ -> meta/namesys.com/reiser4
> 
> They aren't really symlinks because the second option is for making all
> children of meta/namesys.com/reiser4 available under meta/, so if you had:
> 
> meta/namesys.com/reiser4/bmap
> meta/namesys.com/reiser4/plugin
> meta/slaphack.com/happyface/:)
> 
> Now you have:
> 
> meta/bmap
> meta/namesys.com/reiser4/bmap
> meta/namesys.com/reiser4/plugin
> meta/plugin
> meta/slaphack.com/happyface/:)
> 
> More specific things would override less specific things, and in general
> these mappings aren't allowed to touch anything that isn't itself a 
> mapping.
> 
> I think you would use symlink() for setting these things up (cd meta; ln
> ~ -s namesys.com/reiser4 r4 || ln -s namesys.com/reiser4 .), and even if
> they were completely dynamic, the files themselves would read like 
> symlinks.
> 
> I heard someone say something like "reiser4 is almost shipped", so it
> may be too late to send in ideas (especially ones which I've probably
> said before), but I thought I'd drop this in anyway.  (sorry for the
> noise if I _have_ said it before.)
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.2.4 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
> 
> iQIVAwUBQItQOngHNmZLgCUhAQL9yhAAiaCH3ZxSS5T/VxgqW+Tk4jR3kjzA87Cq
> dMfMFlGeS2R1X3p1nuqdb0Q8SWSvMV2cV/YgFXGq4BR+nMjTy/krm7AlXJ1Wj8ji
> xORQMWWZAWsFTeIoQNgu1lDpTCfPISzsmEgLeYsq14UQn+cb8uZMIfy9/tmmY71N
> W6XIsaW2hHp2LwgSsrIjZHthKZQ6WnQh0nsI/8JlU5hjJ4sbEh4SWMokDri//kGq
> wiXk/489sx1u4k/ymU/CHZGSJA4c+/uBzW2QZ2qRnpa6uNwWtu/lqQhn28tkoHRw
> k9TS9j0Ca0w/4k9UZFD8N1Lkxp52/XFTg+U+ZABzkIQx9VHC+nVgaVLIZSr31C4/
> iTk+sHdY55kJtHOF8X7cSzx0D/V+Qo2owhOFS9B5/10NEv34UPmO4aQq9T/MeQVo
> vqWF5L8h0N+zLqlTnkkf4zVBzALfTbsfk18fLeu42j8WZfkVBkHC5T2SBYjmuPHO
> b3bNX1yl+xx0lFG4Wt6bz6sYhzmr2tOj1ayVEYdpZRsXZUsSwom5POeD5Yimuhd7
> /VCiNhcNm17bfqOTK+e+36dE8c91YIyv32e3BBivrB2GYg1AgGfPuzMOky78GX/m
> Ymh4ztKBAwoBytvsF4Aye+8h3bETBMjMoNhJT7EcvH/N4MFqtfZJLh6PafIXMIgA
> IeMzRfndw1w=
> =EGMz
> -----END PGP SIGNATURE-----
> 
> 

Re: ".meta." as a Name Prefix

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

| David,
|
| Something I think I've understood from the code is that plugins aren't
| restricted to the metas directory. Any PSEUDO_PLUGIN can create a root
| name that is looked up everywhere.

Thank you for that clarification, though I'm not sure it's relevant
here.  Try replacing my use of "meta" in those examples with "foo".

Also, someone mentioned that a lack of namespace collisions was more of
a problem than the collisions themselves -- that everyone tries to make
a unique name, and comes up with many copies of the same thing which
really should be the same.  I'd suggest that it's better to have a bunch
of redundant but working plugins than a bunch of intuitive and
non-redundant but occasionally failing plugins.

| That doesn't mean that PSEUDO_PLUGINs that do this will be accepted into
| the codebase or distributions, but there is no actual boundary between
| adding new names to foo/ instead of foo/metas.

As far as what's accepted, I can't imagine that there'd be so much
complexity in the "standard" plugins that no one could go and check for
redundancy, and communicate with the plugin authors about it.

|
| John
|
| David Masover wrote:
|
| |> cat urn:namesys.com:reiser4:meta/shortName
| |> echo "..meta" > urn:namesys.com:reiser4:meta/shortName
|
| Revisiting this idea:  this is a good idea for what happens _inside_ the
| metas dir, for plugins.  But metas itself is a general enough,
| well-enough designed and probably lasting concept that I think we can
| use something like '...' or '..metas', or whatever is reasonably unique
| _now_, and assume that once reiser4 hits mainstream usage, people will
| be careful to develop around that name.
|
| Inside the metas dir, though, we have plugins.  And if plugins could be
| dynamic, even mostly in userspace, then we have the problem of plugins
| becoming so popular that we have namespace collisions there.  I like
| this solution, though -- it's kind of like the XML namespace concept,
| something which should be in every language.
|
| Using some mechanism, not sure what yet, you create a set of mappings:
|
| meta/r4 -> meta/namesys.com/reiser4
| meta/ -> meta/namesys.com/reiser4
|
| They aren't really symlinks because the second option is for making all
| children of meta/namesys.com/reiser4 available under meta/, so if you
| had:
|
| meta/namesys.com/reiser4/bmap
| meta/namesys.com/reiser4/plugin
| meta/slaphack.com/happyface/:)
|
| Now you have:
|
| meta/bmap
| meta/namesys.com/reiser4/bmap
| meta/namesys.com/reiser4/plugin
| meta/plugin
| meta/slaphack.com/happyface/:)
|
| More specific things would override less specific things, and in general
| these mappings aren't allowed to touch anything that isn't itself a
| mapping.
|
| I think you would use symlink() for setting these things up (cd meta; ln
| ~ -s namesys.com/reiser4 r4 || ln -s namesys.com/reiser4 .), and even if
| they were completely dynamic, the files themselves would read like
| symlinks.
|
| I heard someone say something like "reiser4 is almost shipped", so it
| may be too late to send in ideas (especially ones which I've probably
| said before), but I thought I'd drop this in anyway.  (sorry for the
| noise if I _have_ said it before.)
|>
|>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQIwSnHgHNmZLgCUhAQLBIhAAkTsQLoQBBpQDHzX7HE6oc1j5s4D2oqcy
7tmBEtD5IeDERB37winb2EghAOnWXwNEHVSnQKkxmBJfttKytP2mv3UT077d+qEZ
xhCCHGMSA1EI+TyrgZVJsp8Qy34M8ciYTWoA97Cy8LOtou26YDxCZYVfB5j3PTkx
RTF9QB3F9ZcPu9HqYTpHQ5TqAyqzneZyHJYPPxI9ctWfbuJwKUUifz0CQ/1aY31L
IeV4vhGCMAPGGLF/Ll7d4q9VrrCxNQSfvxKmIK7ZSlo3gip4lxNOskRChXWEw7sR
Fng0kbsYAHkdtWfW/rP8NJe/YWgZsb+sldk/6M8zf5C8qssvU4lwOgyDSOBHqDtO
Lb+RCvJXBPJjeeRBM9RY8sLHVt9ie2uXC5hVVegynpMlMdN0lE81CCeeF9YtxA4F
icrQA6rtuOISY7Kw43v2P25/dHs9ZmsId1kV3gsVNMBi29wiRIru1lBdRuiJCZjg
yccDm53sZw46eq8ghuJ9niKyr9Unhnyz+RiJrJKr1v6juwm8YK3MnBr+RZwZRDei
FCfzScRp2rOz0gCXPKWBpt37Bv+ecAPn81gmL+wh+upY78FWqD5taPSZfq/1IGZQ
lnk5tSGQmS2lAfBjFnvCsUNmh/RFExpJew+zDbvHzsZy+G8JXb0VsPbxOJIjt0Bp
SUZZQQrya+0=
=o3i3
-----END PGP SIGNATURE-----

Re: ".meta." as a Name Prefix

[Alexander G. M. Smith]

David Masover wrote on Tue, 20 Apr 2004 20:00:34 -0500:
> That's what we were doing before.  I didn't like it -- consider the use
> on files:
> 
> touch foo
> chmod +x foo
> cd foo/
> ls
> 
> You either get 'foo/' or an error message.  You do not get a list of all
> the pseudo files available.  I'm sorry, but 'cat ..pseudo' just isn't a
> viable substitute for ls.

Why wouldn't that work?  I must have missed something.  I can see that
the fildirute (combined file / directory / attribute) nature of the new
file system objects making the old rwx flags obsolete (does the "x" flag
make it an executable file or does it mean permission to change directory
to the child objects of the file).  But if you change directory to a thing,
it gets appended to your path and the OS should be able to figure out that
you want to look at its child objects (which I'd prefer to directly
include things named .meta.blah, while others want a .metas subdirectory
as a child object).


> | I'm probably repeating myself, but the trouble is that the .meta
> | directory isn't a full directory - you can't add attributes to it.
> What kind of attributes would you want to add?

GUI directory display attributes, like window position and size,
background picture, when displaying that directory.

>  And btw, check out what is currently 'metas/metas'.

I couldn't find that.  Instead I noticed "/filename/metadata/stat/owner"
in http://www.namesys.com/v4/v4.html which seems a bit odd.  I'd expect
something like "/filename/metadata/stat/metadata/owner" if you were using
a metadata magic directory.  Or in my .meta. name prefix scheme, it
would be "/filename/.meta.stat/.meta.owner"

> I don't know if that's actually functional or just read-only, but it
> works.  And I also see no reason why (in raw uneducated theory) a
> plugin couldn't be implemented to make it into a full directory for
> certain files (and thus losing no functionality for other files).

Sounds like extra work and complexity, just to keep the idea of having
a magic subdirectory child object rather than just putting the meta
attributes directly as child objects.

> It doesn't have to.  Look at how the "pure object-oriented" languages
> work.  Many of them make everything some subclass of "object" -- even
> classes themselves.  A class is an object of type class.  I'm not sure
> how this is implemented in code, but in concept it works fine.

In that kind of class system every object is an object, and you can
do all object type operations to it (such as finding it's class name).
The magic .meta directory breaks that very concept by not implementing
full file system object functionality.  It's not a first class object,
it's an awkward special case.

- Alex

Re: ".meta." as a Name Prefix

[David Masover]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

| David Masover wrote on Tue, 20 Apr 2004 20:00:34 -0500:
|
|>That's what we were doing before.  I didn't like it -- consider the use
|>on files:
|>
|>touch foo
|>chmod +x foo
|>cd foo/
|>ls
|>
|>You either get 'foo/' or an error message.  You do not get a list of all
|>the pseudo files available.  I'm sorry, but 'cat ..pseudo' just isn't a
|>viable substitute for ls.
|
|
| Why wouldn't that work?  I must have missed something.  I can see that
| the fildirute (combined file / directory / attribute) nature of the new
| file system objects making the old rwx flags obsolete (does the "x" flag
| make it an executable file or does it mean permission to change directory
| to the child objects of the file).  But if you change directory to a
thing,
| it gets appended to your path and the OS should be able to figure out that
| you want to look at its child objects (which I'd prefer to directly
| include things named .meta.blah, while others want a .metas subdirectory
| as a child object).

I imagine that's how it _should_ be implemented, but right now, I can ls
foo/metas, but not foo/.  Of course, I wouldn't expect to find anything
there, anyway...
|
|
|>| I'm probably repeating myself, but the trouble is that the .meta
|>| directory isn't a full directory - you can't add attributes to it.
|>What kind of attributes would you want to add?
|
|
| GUI directory display attributes, like window position and size,
| background picture, when displaying that directory.

I'm finding it hard to imagine where that would be really useful.  I
mean, users are mostly going to be using tools other than a file browser
to view it.  But that's just me -- it's clear in the whitepaper that
Hans is making a general purpose FS, and it's up to us to decide how to
use it.

|> And btw, check out what is currently 'metas/metas'.

Maybe I'm on an older snapshot?  I can currently do 'ls metas/metas' on
one box, but the kernel is 4 days old.

|
|
|>It doesn't have to.  Look at how the "pure object-oriented" languages
|>work.  Many of them make everything some subclass of "object" -- even
|>classes themselves.  A class is an object of type class.  I'm not sure
|>how this is implemented in code, but in concept it works fine.
|
|
| In that kind of class system every object is an object, and you can
| do all object type operations to it (such as finding it's class name).
| The magic .meta directory breaks that very concept by not implementing
| full file system object functionality.  It's not a first class object,
| it's an awkward special case.

I'm thinking things like inheritance, implemented the way I would/will
when I get the time to understand/write languages better.  Sure I can
find the class name of any object -- but not all objects have a class
name, and in fact, some of them will probably just reurn "Object" -- not
because the name is there in that particular object, but because it was
present in the original Class object.

Let's say you can do anything to .meta that you can to a normal
directory, but until you actually make a change to it that doesn't make
sense in the context of a "magic directory", it becomes a true child
object.  The same concept as Copy On Write.  Maybe even do it on a
per-item basis -- try to change the mode of the directory, and it
becomes a magic directory with a mode.  Try to make a (new) file in it,
and it becomes a nearly real directory with a mode and a file.  And so
on.  (At a certain point, of course, it has its own child directory.)

It also doesn't make sense for them to be directories in every sense of
the word by default.  One thing reiserfs/reiser4 is very good at now is
gajillions of tiny files -- files of 100-200 _bytes_.  To add a
directory on to every little file doesn't make sense, at least for
storage.  And making it possible to accidentally add such a directory
should at least allow for that directory to be removed later (if I
create foo/.meta/file and then remove it, foo should be as it was
before, except for maybe a timestamp)

Again, there are implementation details that I'm not sure of, because my
concepts for language design are mostly for some sort of
ruby/perl/python, only compiled better than C is.  I'm not implementing
any new projects for awhile, though -- I'm still in high school, so if I
do get time, I should spend it learning why I'm wrong.

Being loud, stupid, and long-winded on mailing lists is just one way I
find out why I'm wrong.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iQIVAwUBQItMKHgHNmZLgCUhAQJhzw/7B6p9n4ePCa3JYRtQbYTwm1CkXTpZcxpW
qEc7SgBmaZ6NWxBYWiAF/1MdOQloXjmOT+FnBCm9aAf5gwBi4UU9ES4Bg3pW2uKs
U2C02ekYW5k6p3DIi1iZPrZuqWW5Onpv+W3JujJETzl+4dojnfFR6lLhhmWWh1Iu
F7l4tg78968ibAwBZPC70bvNqwI+nhWLQwXNe5ZM2PbBNnfWcj//B/LZ78N5Up7G
qKnT9ftJCUqQXJL0tA409LNIjmeOJy24Nluqtcx6qphx6W4qBo7x1uzSNh3M+SfF
QPwewBHT0BR42qbWQxy4m1MwjWj6ktkmWzp+OhPg81sxb/iZF+oCBiw6U3phi97y
lyNtRy7w7yohGtQlVG2H8BYDAlZTXZiUcNUEzdV+3NjirBAbmotWtRuntf8wMdj6
Z749u7vS7xJi8wUHegBuzuvFQjlLnZIGdfHpCJ44+18wH2RfZWk+CPRKQBIrsbLE
9f/S/yi9lC/KYURpKvSExqCmEtGasksF6aIFb7Hr7p/vONBw/SntHuqxBN3/nIv4
AB4x5xKaxYpf5XI9fMbDlixASFz6gAE/5h6vRGbdL6ZKel6XqMaADRFDpfr9gY8Z
3t7fLE9p/KqeGRauhPLhohv5/z+MdnJPOhGyOB9XJfuBW8cEgfD0WkZKyXzAL7pL
QaCZzaj+fMw=
=z9qA
-----END PGP SIGNATURE-----

Re: "Metas"

[The Amazing Dragon]

> From: Hans Reiser <reiser@namesys.com>
> so if rename works on metas, are you guys happy?  that is, if mv 
> filename/metas filename/... works, are you less worried about the one in 
> 10 million users who will actually hit the reserved keyword?

I think we've provided evidence that it might well be one in one
thousand. Pretty much any filename that `ls` without the "-a" option
will display will be unacceptable. Adding a leading '.' or using a
separator character ('\' or "/.metas/" perhaps) fits this qualification.

> From: mjt@nysv.org (Markus Törnqvist)
> Lookups would be next to impossible, for one. And what if someone actually
> loses a meta directory, like, renames it accidentally and forgets about it?
> Yeah, far-fetched but stuff does happen...

I don't think this is at all far fetched, Hans, please listen!

> What's so terribly bad about having the name for it in /proc or /sys
> or wherever and having it global from the kernel config?

open()/read()/write()/stat()/readdir() are system calls, but fairly cheap
operations. getcwd() is a library call, almost certainly making
*hundreds* of system calls, and quite possibly *thousands*. In order to
use a /proc or /sys interface you'd need to use getcwd(), if you're
accessing many files this would be extremely expensive.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM@cs.pdx.edu      PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Re: "Metas"

[Hubert Chan]

>>>>> "Elliot" == The Amazing Dragon (Elliott Mitchell) <ehem@cs.pdx.edu> writes:

[...]

Elliot> I think we've provided evidence that it might well be one in one
Elliot> thousand. Pretty much any filename that `ls` without the "-a"
Elliot> option will display will be unacceptable.

metas is not returned by readdir(), so won't show up in ls.
Nevertheless, I'm still in favour of adding a leading dot (or two).  I
don't think that allowing renames is a good idea, especially given that
the metas directory is not returned by readdir -- the user (or
application) has no way of discovering what the name is.

>> From: mjt@nysv.org (Markus Törnqvist)

>> What's so terribly bad about having the name for it in /proc or /sys
>> or wherever and having it global from the kernel config?

Elliot> open()/read()/write()/stat()/readdir() are system calls, but
Elliot> fairly cheap operations. getcwd() is a library call, almost
Elliot> certainly making *hundreds* of system calls, and quite possibly
Elliot> *thousands*. In order to use a /proc or /sys interface you'd
Elliot> need to use getcwd(), if you're accessing many files this would
Elliot> be extremely expensive.

I don't see why putting the name in /proc or /sys requires getcwd().
Can you explain in more detail?

Oh.  I think you're talking about something different from what Markus
is talking about.  I think you're talking about putting metadata for
/foo/bar into, say, /proc/metas/foo/bar.  What Markus is talking about
is having a file called, say /proc/fs/reiser4/metas, and if you cat
that file, it will spit out the name of the metas directory.

I don't like it because I think that Reiser4 should be consistent.
Making users look up the name from /proc/fs/reiser4/metas I think is a
lot worse than making them use a long name for the metas directory.  And
"ls foo/`cat /proc/fs/reiser4/metas`", or even "ls foo/$metas", where
$metas is the cached value of /proc/fs/reiser4/metas, is a lot uglier
than "ls foo/.metas".

Yet another option is to standardize on a long, extremely unlikely to
clash name as the standard way to access the metadata (maybe something
like "sys.metadata"), and define a short name, which is probably defined
at compile time, which can be looked up through something like
/proc/fs/reiser4/metas_abbrev.  That way, users (and scripts) get a
consistent interface, but if they want to save typing, they can look up
the short name.

-- 
Hubert Chan <hubert@uhoreg.ca> - http://www.uhoreg.ca/
PGP/GnuPG key: 1024D/124B61FA
Fingerprint: 96C5 012F 5F74 A5F7 1FF7  5291 AF29 C719 124B 61FA
Key available at wwwkeys.pgp.net.   Encrypted e-mail preferred.

Re: "Metas"

[mjt]

On Wed, Apr 28, 2004 at 02:49:12AM -0400, Hubert Chan wrote:
>/foo/bar into, say, /proc/metas/foo/bar.  What Markus is talking about
>is having a file called, say /proc/fs/reiser4/metas, and if you cat
>that file, it will spit out the name of the metas directory.

Yes, were it for the current directory (/sys always changing its contents)
or one global name.

The current directory mv-supporting thing is like death itself and should
not be implemented. Nor should mv support be implemented without sys
support because then people may forget their metadata directory name
without any chance of looking it up, except for maybe something in debugfs
or somewhere, which is not implemented afaik.

Consider how much work this would be due to bickering.

>I don't like it because I think that Reiser4 should be consistent.

I agree.

>Making users look up the name from /proc/fs/reiser4/metas I think is a
>lot worse than making them use a long name for the metas directory.  And

Mmmmyeah, but I don't think having the directory name exported in 
something like that is a bad idea per-se, as long as it exports a
global, static name. If we have a situation where different people
use different names. Which will happen if we have a clashing ./metas/
directory.

>Yet another option is to standardize on a long, extremely unlikely to
>clash name as the standard way to access the metadata (maybe something
>like "sys.metadata"), and define a short name, which is probably defined
>at compile time, which can be looked up through something like
>/proc/fs/reiser4/metas_abbrev.  That way, users (and scripts) get a
>consistent interface, but if they want to save typing, they can look up
>the short name.

And the abbreviation would never conflict, because when a process tries
to write to the abbreviated name, the file system realizes this and
goes "oh shit, I must make way to tar xf, *poof*"?

Incredible amounts of work again in vain.

Besides, length has little to do with it. Consider ..metas. It's not
long but what are the odds of accidental clashing?

Just my two cents, which I seem to contribute over and over again ;)

-- 
mjt

[parisc-linux] Re: linux signal race fixes, patches against hppa tree, please test.

[John David Anglin]

> > This is more important for you, approximately 48 hours ago, Linus
> > commited a fix for a signal handling race.
> > 
> > http://linux.bkbits.net:8080/linux-2.6/cset@1.1846.1.48?nav=index.html|Chang
> > eSet@-2d
> 
> I have the patch under test on hiauly6 with todays CVS.

I've been running this now for three days on hiauly6 with a 32-bit
2.6.8.1-pa7 kernel (default c3000 config) doing almost continuous gcc
builds.  Things seem pretty promising.  No random SIGSEGVs and expect
seems to be behaving itself.

> I had another SIGSEGV on gsyprf11.  It looks as if this was in /bin/sh
> although gdb doesn't say which program the fault ocurred in.

This is definitely not the case with the 64-bit SMP kernel on gsyprf11.
There I've been testing with an unpatched kernel.  Expect works better with
tcl8.3 but there are still some tests that fail when they shouldn't, and
the output with tests which produce a large amount of output gets
truncated.  With tcl8.4, expect has major problems.  The Tcl_Finalize
patch doesn't fix them.  I think we need to try Carlos's patch on a
64-bit SMP system to see if it helps the expect and SEGV problems.

I'm off to Korea early Thursday morning so I can't do much more on
this until I'm back in a couple of weeks.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: linux signal race fixes, patches against hppa tree, please te

[John David Anglin]

> I've been running this now for three days on hiauly6 with a 32-bit
> 2.6.8.1-pa7 kernel (default c3000 config) doing almost continuous gcc
> builds.  Things seem pretty promising.  No random SIGSEGVs and expect
> seems to be behaving itself.

As usual, I have to eat my words.  The libjava "PR218 -O3 execution -
source compiled test" hung in the libjava testsuite on the next gcc
build.  Looking at libjava.log, I see that there is no "WARNING:
program timed out." message for this test.  Thus, the alarm signal
for this test somehow got lost.  The testsuite continued after I
killed PR218.exe.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: linux signal race fixes, patches against hppa tree, please te

[John David Anglin]

Hi Randolph,

> > I've been running this now for three days on hiauly6 with a 32-bit
> > 2.6.8.1-pa7 kernel (default c3000 config) doing almost continuous gcc
> > builds.  Things seem pretty promising.  No random SIGSEGVs and expect
> > seems to be behaving itself.
> 
> As usual, I have to eat my words.  The libjava "PR218 -O3 execution -
> source compiled test" hung in the libjava testsuite on the next gcc
> build.  Looking at libjava.log, I see that there is no "WARNING:
> program timed out." message for this test.  Thus, the alarm signal
> for this test somehow got lost.  The testsuite continued after I
> killed PR218.exe.

Ok, the next time around on gsyprf11 expect did a SIGSEGV after the
exact same test.  The timeout warning was printed in the log this time.

The core dump, core.20875, is in my home directory.  The expect
program is /home/dave/opt/gnu/bin/expect.  I can see that expect
died because %r4 has apparently gotten clobbered.  It contains
0.  %r4 was used to restore the pic register after the last call.
We then die in a stub that uses the pic register.

Is there any chance you could look at fixing the backtrace command
in gdb?  Here is the bt:

(gdb) bt
#0  0x406a0ef4 in NativePathInFilesystem () from /usr/lib/libtcl8.4.so.0
#1  0x406d5be4 in Tcl_WaitForEvent () from /usr/lib/libtcl8.4.so.0
#2  0x00000000 in ?? ()
#3  0x00000000 in ?? ()
(gdb) maint print unwind 0x406d5be4
unwind_table_entry (0x6a6744):
        region_start = 0x406d59b0 <Tcl_WaitForEvent+784>
	region_end = 0x406d5e0c <Tcl_WaitForEvent+1900>
	flags = Save_RP
	Region_description = 0x1
	Entry_FR = 0x0
	Entry_GR = 0x10
	Total_frame_size = 0x48
(gdb) disass 0x406a0ee4 0x406a0f04
Dump of assembler code from 0x406a0ee4 to 0x406a0f04:
0x406a0ee4 <NativePathInFilesystem+1668>:       ldw 518(,r1),r21
0x406a0ee8 <NativePathInFilesystem+1672>:       bv r0(r21)
0x406a0eec <NativePathInFilesystem+1676>:       ldw 51c(,r1),r19
0x406a0ef0 <NativePathInFilesystem+1680>:       addil -1800,r19,%r1
0x406a0ef4 <NativePathInFilesystem+1684>:       ldw 320(,r1),r21
0x406a0ef8 <NativePathInFilesystem+1688>:       bv r0(r21)
0x406a0efc <NativePathInFilesystem+1692>:       ldw 324(,r1),r19
0x406a0f00 <NativePathInFilesystem+1696>:       addil -1800,r19,%r1
(gdb) disass 0x406d5bc4 0x406d5bf4
Dump of assembler code from 0x406d5bc4 to 0x406d5bf4:
0x406d5bc4 <Tcl_WaitForEvent+1316>:     stw r8,-34(,sp)
0x406d5bc8 <Tcl_WaitForEvent+1320>:     copy r4,r19
0x406d5bcc <Tcl_WaitForEvent+1324>:     copy r5,r26
0x406d5bd0 <Tcl_WaitForEvent+1328>:     ldo -238(sp),r25
0x406d5bd4 <Tcl_WaitForEvent+1332>:     ldo -1b8(sp),r24
0x406d5bd8 <Tcl_WaitForEvent+1336>:     ldo -138(sp),r23
0x406d5bdc <Tcl_WaitForEvent+1340>:     b,l 0x406a2394 <NativePathInFilesystem+6
964>,rp
0x406d5be0 <Tcl_WaitForEvent+1344>:     ldi 60,r7
0x406d5be4 <Tcl_WaitForEvent+1348>:     cmpib,= -1,ret0,0x406d5b24 <Tcl_WaitForE
vent+1156>
0x406d5be8 <Tcl_WaitForEvent+1352>:     copy r4,r19
0x406d5bec <Tcl_WaitForEvent+1356>:     b,l 0x406a0ef0 <NativePathInFilesystem+1
680>,rp
0x406d5bf0 <Tcl_WaitForEvent+1360>:     copy r14,r26
(gdb) p/x $rp
$1 = 0x406d5bf7
(gdb) disass 0x406a2394 0x406a23a4
Dump of assembler code from 0x406a2394 to 0x406a23a4:
0x406a2394 <NativePathInFilesystem+6964>:       addil -800,r19,%r1
0x406a2398 <NativePathInFilesystem+6968>:       ldw a0(,r1),r21
0x406a239c <NativePathInFilesystem+6972>:       bv r0(r21)

I'm not quite sure how we got to the point of the segfault.  However,
with respect to gdb, the back trace uses the wrong function symbol
in both frames 0 and 1.  We are in a stub in frame 0.  I'm fairly
sure that this is why the bt fails at frame 2.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Another SIGSEGV, this time in /bin/sh

[John David Anglin]

I had another SIGSEGV today building gcc on gsyprf11.

dave@gsyprf11:~/gcc-3.5/objdir$ gdb /bin/sh gcc/core.save.1
GNU gdb 2004-09-01-cvs
...
Program terminated with signal 11, Segmentation fault.
Reading symbols from /usr/lib/debug/libncurses.so.5...done.
Loaded symbols for /usr/lib/debug/libncurses.so.5
Reading symbols from /usr/lib/debug/libdl.so.2...done.
Loaded symbols for /usr/lib/debug/libdl.so.2
Reading symbols from /usr/lib/debug/libc.so.6...done.
Loaded symbols for /usr/lib/debug/libc.so.6
Reading symbols from /lib/ld.so.1...done.
Loaded symbols for /lib/ld.so.1
#0  0xf7025554 in ?? ()

Looking at %r2, it appears that the fault has occurred in an indirect
call from __strtoll_internal:

(gdb) p/x $rp
$1 = 0x40785883
(gdb) disass 0x40785860 0x40785888
Dump of assembler code from 0x40785860 to 0x40785888:
0x40785860 <*__GI___strtoll_internal+2972>:     copy r19,r4
0x40785864 <*__GI___strtoll_internal+2976>:     b,l 0x4076c508 <__gconv_read_conf+916>,r31
0x40785868 <*__GI___strtoll_internal+2980>:     copy r31,rp
0x4078586c <*__GI___strtoll_internal+2984>:     b,l 0x40785848 <*__GI___strtoll_internal+2948>,r0
0x40785870 <*__GI___strtoll_internal+2988>:     copy r4,r19
0x40785874 <*__GI___strtoll_internal+2992>:     copy r19,r4
0x40785878 <*__GI___strtoll_internal+2996>:     b,l 0x4076c508 <__gconv_read_conf+916>,r31
0x4078587c <*__GI___strtoll_internal+3000>:     copy r31,rp

We appear to have reached 0x40785874 from the following code:

0x40784d40 <*__GI___strtoll_internal+124>:      ldi 24,r20
0x40784d44 <*__GI___strtoll_internal+128>:      cmpb,>= r20,r9,0x40784db8 <*__GI___strtoll_internal+244>
0x40784d48 <*__GI___strtoll_internal+132>:      addil 1000,r19,%r1
...
0x40784db8 <*__GI___strtoll_internal+244>:      ldw -124(,sp),r7
0x40784dbc <*__GI___strtoll_internal+248>:      ldi 20,r10
---Type <return> to continue, or q <return> to quit---
0x40784dc4 <*__GI___strtoll_internal+256>:      stw r7,-f0(,sp)
0x40784dc8 <*__GI___strtoll_internal+260>:      addil 1000,r19,%r1
0x40784dcc <*__GI___strtoll_internal+264>:      copy r3,r8
0x40784dd0 <*__GI___strtoll_internal+268>:      ldw 390(,r1),r6
0x40784dd4 <*__GI___strtoll_internal+272>:      ldw 98(,r3),r22
0x40784dd8 <*__GI___strtoll_internal+276>:      addil 1000,r19,%r1
0x40784ddc <*__GI___strtoll_internal+280>:      cmpib,<> 0,r22,0x40785874 <*__GI
I___strtoll_internal+2992>

The address for the indirect call, r22, was loaded as follows:
(gdb) p/x *($r3 + 0x98)
$20 = 0xf7025555

Some interesting registers:
(gdb) p/x $r10
$23 = 0x20
(gdb) p/x $r3
$13 = 0xe3c08
(gdb) p/x $r8
$16 = 0xe3c08
(gdb) p/x $r6
$18 = 0x408a1c8c
(gdb) p/x *($r19 + 0x1000 + 0x390)
$17 = 0x408a1c8c
(gdb) p/x $r7
$21 = 0xd65e8
(gdb) p/x *($sp - 0xf0)
$22 = 0xd65e8

This establishes how we got to the failure point.

It would appear that r3 has either been clobbered by the kernel, or there
is a problem with the memory access by which it was loaded:

(gdb) p/x *($r19 + 0x1000 + 0x354)
$12 = 0x408a98fc

This should have given the r3 value.  The same procedure gave the r6 value.

(gdb) p/x *(*($r19 + 0x1000 + 0x354) + 0x98)
$19 = 0x0

This is probably the value that r22 should have had.

The failure doesn't happen consistently as rerunning the script which failed
was successful.

Any thoughts?

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Another SIGSEGV, this time in /bin/sh

[Carlos O'Donell]

jda,

What kernel is this btw?

> Program terminated with signal 11, Segmentation fault.
> Reading symbols from /usr/lib/debug/libncurses.so.5...done.
> Loaded symbols for /usr/lib/debug/libncurses.so.5
> Reading symbols from /usr/lib/debug/libdl.so.2...done.
> Loaded symbols for /usr/lib/debug/libdl.so.2
> Reading symbols from /usr/lib/debug/libc.so.6...done.
> Loaded symbols for /usr/lib/debug/libc.so.6
> Reading symbols from /lib/ld.so.1...done.
> Loaded symbols for /lib/ld.so.1

Why isn't this the debug ld.so?

> Looking at %r2, it appears that the fault has occurred in an indirect
> call from __strtoll_internal:

But it's always some place different every failure...
 
> It would appear that r3 has either been clobbered by the kernel, or there
> is a problem with the memory access by which it was loaded:

It's possible.
 
> This should have given the r3 value.  The same procedure gave the r6 value.
> (gdb) p/x *(*($r19 + 0x1000 + 0x354) + 0x98)
> $19 = 0x0

So you are saying that the code should have given that value, but
something corrupted the value before it was used?

It definately looks signal related. It feels a lot like some return path
in the kernel might be clobbering r3. Time for another witch hunt?

c.

_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: linux signal race fixes, patches against hppa tree, please te

[Carlos O'Donell]

On Tue, Aug 31, 2004 at 04:43:11PM -0400, John David Anglin wrote:
> > I've been running this now for three days on hiauly6 with a 32-bit
> > 2.6.8.1-pa7 kernel (default c3000 config) doing almost continuous gcc
> > builds.  Things seem pretty promising.  No random SIGSEGVs and expect
> > seems to be behaving itself.
> 
> As usual, I have to eat my words.  The libjava "PR218 -O3 execution -
> source compiled test" hung in the libjava testsuite on the next gcc
> build.  Looking at libjava.log, I see that there is no "WARNING:
> program timed out." message for this test.  Thus, the alarm signal
> for this test somehow got lost.  The testsuite continued after I
> killed PR218.exe.

Losing signals under load is very bad, I don't see where or why this
might be happening. I don't really know what that test does, if it uses
compare_and_swap in any way then it's non-atomic and all sorts of bad
things could happen under load.

I'm still writing my paper, so I can't add the hooks in for this.

c.

_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: printf() overhead

[Alan Curry]

John Richard Moser writes the following:
>Andre Tomt wrote:
>| John Richard Moser wrote:
>|> using strace to run a program takes aeons.  Redirecting the output to a
>|> file can be a hundred times faster sometimes.  This raises question.
>|
>| The terminal is a major factor; gnome-terminal for example can be
>| *extremely* slow.
>|
>
>Is there a way to give the data to the terminal and let the program go
>while that happens?  Or is there an execution path (i.e. terminal says
>"WTF NO") that can be missed that way?

strace -o tracefile prog & tail -n +1 -f tracefile

Re: SVGATextMode on 2.6.11

[Alan Curry]

I wrote:
>
>Was SVGATextMode's cursor-setting ability removed as a result of an
>intentional change, or might it get fixed? Or might CUR_DEFAULT become
>tunable? Maybe another control sequence could make the current cursor
>settings the default, like setterm -store does for foreground and background
>colors.

I found the cause of this new behavior: font loading. SVGATextMode can be
configured to load a font with consolechars. It does that after setting the
cursor. In 2.6.11 vgacon_adjust_height() was changed to reset the cursor in
addition to the font size. The solution is: disable SVGATextMode's font
loading, and if you want to change the font, do it before you run SVGATextMode.

The second problem remains a mystery:

>On another note, the resize function of SVGATextMode has been affected
>strangely too. Sometimes, when resizing the screen to a mode larger than
>80x25, the video settings come out correctly but the terminal only uses the
>first 25 lines, with the bottom of the screen being blank. This one is hard
>to reproduce. I can reproduce it by doing a full boot (which includes an
>SVGATextMode call from /etc/rcS.d/S60svgatextmode) followed by a manual
>SVGATextMode on tty2. The first one works, and the second one screws up the
>terminal size. When I try to reproduce that series of events without the call
>from /etc/rcS.d, the problem doesn't show up.
>
>In any case, when that problem _does_ show up, it can be fixed by immediately
>running the same command again, on the same tty where it just screwed up. And
>it never fails twice without an intervening reboot.

Re: strace on cat /proc/my_file gives results by calling read twice why?

[Alan Curry]

cranium2003 writes the following:
>
>when i strace cat /proc/my_file i found message
>printing twice
>Reading a from a /proc file....
>Reading a from a /proc file....
>       Why that happening?

The first read returns some data and returns the number of bytes, and the
second one indicates that EOF has been reached by returning 0. A single read
call can't do both of those things, so cat needs to do 2 reads.

This has nothing to do with /proc or Linux; it is a logical consequence of
the way read() is defined, and the job cat is supposed to do: it copies
entire files to stdout, so it has to read until EOF. Only if the input file
is empty will it see the EOF on the first read.

[parisc-linux] Re: Comments?

[John David Anglin]

> > > I think we have somehow broken __cffc.  I did a binutils build
> > > yesterday and various ld tests are now failing.  These are tests
> > > that compare function pointers.
> > 
> > I didn't feed anything new into debian-glibc. All my patches have always
> > strived to keep __cffc working, if I messup the function pointer
> > comparison I get a slew of glibc testsuite failures which I always
> > patchup by making sure __cffc works :)
> 
> I haven't touched fptr.c either, although there is a change in GCC
> 4.0 with respect to canonicalization.  I'm using the debian glibc,
> 2.3.2.ds1-20, on the system on which I noticed this problem.

Ok, ld is broken in the binutils CVS head.
__canonicalize_funcptr_for_compare doesn't load the correct address
for _GLOBAL_OFFSET_TABLE_ in shared libraries.  It loads the address
used by main.  As a result, it thinks the plabel for the function
has been resolved ;(

The testcase is:

foo.c:

extern void bar (void);
typedef void (*func_t)(void);
int
foo (func_t f)
{
  return f == bar;
}

bar.c:

void bar (void) {};
int
main ()
{
  if (!foo (bar))
    abort ();
  return 0;
}

gcc -g -fPIC -S foo.c
gcc -shared -fPIC -o foo.sl foo.o
gcc -c -g -w bar.c
gcc -o bar bar.o foo.sl

nm foo.sl
...
00010ab0 a _GLOBAL_OFFSET_TABLE_

nm bar
...
00020c54 d _GLOBAL_OFFSET_TABLE_

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Comments?

[John David Anglin]

> Ok, ld is broken in the binutils CVS head.
> __canonicalize_funcptr_for_compare doesn't load the correct address
> for _GLOBAL_OFFSET_TABLE_ in shared libraries.  It loads the address
> used by main.  As a result, it thinks the plabel for the function
> has been resolved ;(

Found it:

2004-11-02  Hans-Peter Nilsson  <hp@axis.com>

        * elflink.c (_bfd_elf_create_got_section): Hide _GLOBAL_OFFSET_TABLE_.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Comments?

[Carlos O'Donell]

On Sat, Mar 05, 2005 at 07:22:27PM -0500, John David Anglin wrote:
> > Ok, ld is broken in the binutils CVS head.
> > __canonicalize_funcptr_for_compare doesn't load the correct address
> > for _GLOBAL_OFFSET_TABLE_ in shared libraries.  It loads the address
> > used by main.  As a result, it thinks the plabel for the function
> > has been resolved ;(
> 
> Found it:
> 
> 2004-11-02  Hans-Peter Nilsson  <hp@axis.com>
> 
>         * elflink.c (_bfd_elf_create_got_section): Hide _GLOBAL_OFFSET_TABLE_.

I'd seen that fly by on binutils and it caused a certain amount of grief
in glibc. I'm not quite sure what the rational was behind hiding _GOT_.

c.

_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Comments?

[John David Anglin]

> > 2004-11-02  Hans-Peter Nilsson  <hp@axis.com>
> > 
> >         * elflink.c (_bfd_elf_create_got_section): Hide _GLOBAL_OFFSET_TABLE_.
> 
> I'd seen that fly by on binutils and it caused a certain amount of grief
> in glibc. I'm not quite sure what the rational was behind hiding _GOT_.

I believe that the idea was that main code would see its _GOT_ address
and shared libraries their respective _GOT_ address.  It turns out that
it was sort of a fluke that function pointer canonicalization worked.
Only function pointers in the main part of an application are not
resolved.  Shared library function pointers are resolved when the
library is loaded.  __cffc was always looking at the _GOT_ address
for the main app (i.e., the main _GLOBAL_OFFSET_TABLE_ value overrode
the shared library symbol).

This problem was fixed yesterday by Alan Modra.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Comments?

[Carlos O'Donell]

On Tue, Mar 08, 2005 at 12:44:53PM -0500, John David Anglin wrote:
> > > 2004-11-02  Hans-Peter Nilsson  <hp@axis.com>
> > > 
> > >         * elflink.c (_bfd_elf_create_got_section): Hide _GLOBAL_OFFSET_TABLE_.
> > 
> > I'd seen that fly by on binutils and it caused a certain amount of grief
> > in glibc. I'm not quite sure what the rational was behind hiding _GOT_.
> 
> I believe that the idea was that main code would see its _GOT_ address
> and shared libraries their respective _GOT_ address.  It turns out that
> it was sort of a fluke that function pointer canonicalization worked.
> Only function pointers in the main part of an application are not
> resolved.  Shared library function pointers are resolved when the
> library is loaded.  __cffc was always looking at the _GOT_ address
> for the main app (i.e., the main _GLOBAL_OFFSET_TABLE_ value overrode
> the shared library symbol).
> 
> This problem was fixed yesterday by Alan Modra.

I'm not sure what you mean in your statement "shared library function
pointers are resolved when the library is loaded?" The function pointers
exist as two-byte entries in the PLT, and are non-unique, and they
aren't resolved until call time with lazy resolution.

I would imagine that the main _GOT_ is supposed to override the shared
library _GOT_. __cffc is only looking at the _GOT_ to get the fptr that
the dynamic loader wrote there during setup, so that it can get called
for symbol resolution. This setup is done for all objects, which
includes all the _GOT_'s, from the application to all the loaded shared
libraries.

Or rather are you saying, that from a shared library, the library saw
only the _GOT_ from the main applicaiton? I would think that this 
wouldn't effect the library since the loader setup r19 properly, and
aslong as it doesn't use _GOT_ then it's fine.

c.

_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Comments?

[John David Anglin]

> > This problem was fixed yesterday by Alan Modra.
> 
> I'm not sure what you mean in your statement "shared library function
> pointers are resolved when the library is loaded?" The function pointers
> exist as two-byte entries in the PLT, and are non-unique, and they
> aren't resolved until call time with lazy resolution.

This is what Alasn said:

  I checked.  &_GLOBAL_OFFSET_TABLE_ in fptr.c does resolve to the main
  app GOT before the change.  That in fact is what is needed, because
  according to what I see in dl-machine.h, plabels in shared libs will be
  resolved.  I think it's only plabels in the main app for global
  functions that won't be resolved (because they point into the plt).

  Hmm.  If PLABEL32 relocs in the main app were emitted as dynamic relocs,
  then ld.so would call _dl_make_fptr for them.  I think you wouldn't need
  __canonicalize_funcptr_for_compare any more..

I don't think we could completely do away with __cffc but if the
plabels were always resolved we could just look inside the plabel
to get the function address.

> I would imagine that the main _GOT_ is supposed to override the shared
> library _GOT_. __cffc is only looking at the _GOT_ to get the fptr that
> the dynamic loader wrote there during setup, so that it can get called
> for symbol resolution. This setup is done for all objects, which
> includes all the _GOT_'s, from the application to all the loaded shared
> libraries.

If the dynamic loader always uses the main app's got for the trampoline,
__cffc would work for plabels in shared libraries that used lazy linking.

> Or rather are you saying, that from a shared library, the library saw
> only the _GOT_ from the main applicaiton? I would think that this 
> wouldn't effect the library since the loader setup r19 properly, and
> aslong as it doesn't use _GOT_ then it's fine.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] OPD's on hppa-linux, and what to do about __cffc's fragility.

[Carlos O'Donell]

On Tue, Mar 08, 2005 at 02:02:33PM -0500, John David Anglin wrote:
> > > This problem was fixed yesterday by Alan Modra.
> > 
> > I'm not sure what you mean in your statement "shared library function
> > pointers are resolved when the library is loaded?" The function pointers
> > exist as two-byte entries in the PLT, and are non-unique, and they
> > aren't resolved until call time with lazy resolution.

Once, we start quoting Alan, I believe it's time to atleast let him read
some of the remarks. Perhaps he will have some insightful comment.
 
> This is what Alasn said:
> 
>   I checked.  &_GLOBAL_OFFSET_TABLE_ in fptr.c does resolve to the main
>   app GOT before the change.  That in fact is what is needed, because
>   according to what I see in dl-machine.h, plabels in shared libs will be
>   resolved.  I think it's only plabels in the main app for global
>   functions that won't be resolved (because they point into the plt).
> 
>   Hmm.  If PLABEL32 relocs in the main app were emitted as dynamic relocs,
>   then ld.so would call _dl_make_fptr for them.  I think you wouldn't need
>   __canonicalize_funcptr_for_compare any more..
> 
> I don't think we could completely do away with __cffc but if the
> plabels were always resolved we could just look inside the plabel
> to get the function address.

Perhaps some background and clarification...

Okay, *ABS* (R_PARISC_IPLT, r_sym == 0) entries are automatically
relocated at startup, because we have enough information, we are already
walking the PLT list, and it takes only a single add to complete the
relocation. 

Next, anything else (R_PARISC_IPLT, r_sym != 0) is pointed at the
plt/got stub (which calls the trampoline _dl_runtime_resolve) since it
will require symbol resolution to fixup.

I don't quite understand what Alan is getting at when he says "If
PLABEL32 relocs in the main app were emitted as dynamic relocs,"

I'm really a stickler for nomeclature. In my mind a "resolved" plabel is
one whose ip/gp point to the true and final function address after the ELF
symbol resolution rules have been followed for that symbol. Is this the
generally accepted definition? I interchangably think plabel == fptr.

> > I would imagine that the main _GOT_ is supposed to override the shared
> > library _GOT_. __cffc is only looking at the _GOT_ to get the fptr that
> > the dynamic loader wrote there during setup, so that it can get called
> > for symbol resolution. This setup is done for all objects, which
> > includes all the _GOT_'s, from the application to all the loaded shared
> > libraries.
> 
> If the dynamic loader always uses the main app's got for the trampoline,
> __cffc would work for plabels in shared libraries that used lazy linking.

To clarify this, you mean to say if the loader always uses the main
app's plt/got stub then everything would be okay. The check in __cffc to
see if the plabel has been resolved would work (e.g. got !=
&_GLOBAL_OFFSET_TABLE_).

It didn't make a difference in the past because the shared library
plabels were already resolved? Wouldn't this mean that lazy linking
wasn't working at all?

So only the main app symbols had to be resolved, and this worked fine.
Thus the comparison worked fine.

I don't quite understand why this broke, if someone would be so kind as
the point out the error in my thinking.

c.

_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: OPD's on hppa-linux, and what to do about __cffc's fragility.

[John David Anglin]

> I'm really a stickler for nomeclature. In my mind a "resolved" plabel is
> one whose ip/gp point to the true and final function address after the ELF
> symbol resolution rules have been followed for that symbol. Is this the
> generally accepted definition? I interchangably think plabel == fptr.

I agree with the former definition regarding a resolved plabel.  I believe
that the term plabel is interchangeable with the term function descriptor.
A function pointer on the otherhand may point either directly to the
code address of the function or to a plabel.

> > > I would imagine that the main _GOT_ is supposed to override the shared
> > > library _GOT_. __cffc is only looking at the _GOT_ to get the fptr that
> > > the dynamic loader wrote there during setup, so that it can get called
> > > for symbol resolution. This setup is done for all objects, which
> > > includes all the _GOT_'s, from the application to all the loaded shared
> > > libraries.
> > 
> > If the dynamic loader always uses the main app's got for the trampoline,
> > __cffc would work for plabels in shared libraries that used lazy linking.
> 
> To clarify this, you mean to say if the loader always uses the main
> app's plt/got stub then everything would be okay. The check in __cffc to
> see if the plabel has been resolved would work (e.g. got !=
> &_GLOBAL_OFFSET_TABLE_).

Yes.

> It didn't make a difference in the past because the shared library
> plabels were already resolved? Wouldn't this mean that lazy linking
> wasn't working at all?

If shared libraries used the main app's &_GLOBAL_OFFSET_TABLE_, then
lazy linking could be working.   The testcase that I posted had a
plabel constructor in the shared library.  I recall now that the got
pointer for the plabel was the main app's &_GLOBAL_OFFSET_TABLE_.

> I don't quite understand why this broke, if someone would be so kind as
> the point out the error in my thinking.

Hiding _GLOBAL_OFFSET_TABLE_, resulted in _GLOBAL_OFFSET_TABLE_
being local to shared libraries, as a result __cffc no longer used
the main app's &_GLOBAL_OFFSET_TABLE_.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: OPD's on hppa-linux, and what to do about __cffc's fragility.

[John David Anglin]

> A function pointer on the otherhand may point either directly to the
> code address of the function or to a plabel.

Further, there are some special function pointers like -1.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: OPD's on hppa-linux, and what to do about __cffc's fragility.

[Alan Modra]

On Tue, Mar 08, 2005 at 04:08:51PM -0500, Carlos O'Donell wrote:
> On Tue, Mar 08, 2005 at 02:02:33PM -0500, John David Anglin wrote:
> > > > This problem was fixed yesterday by Alan Modra.
> > > 
> > > I'm not sure what you mean in your statement "shared library function
> > > pointers are resolved when the library is loaded?" The function pointers
> > > exist as two-byte entries in the PLT, and are non-unique, and they
> > > aren't resolved until call time with lazy resolution.

No, in an hppa-linux shared lib, a function pointer is accessed via a
word with a PLABEL32 reloc on it.  This reloc is processed before any
code in the shared lib runs, to point to a function descriptor created
by _dl_make_fptr.

-- 
Alan Modra
IBM OzLabs - Linux Technology Centre
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: OPD's on hppa-linux, and what to do about __cffc's fragility.

[Carlos O'Donell]

On Wed, Mar 09, 2005 at 08:55:13AM +1030, Alan Modra wrote:
> No, in an hppa-linux shared lib, a function pointer is accessed via a
> word with a PLABEL32 reloc on it.  This reloc is processed before any
> code in the shared lib runs, to point to a function descriptor created
> by _dl_make_fptr.

In shared libraries the function pointer is also allocated a PLT slot
and an IPLT relocation. 

I don't understand the reasons for the allocation of a PLT slot if all
references generated go through the local label word that has the
PLABEL32 reloc, which now contains the address returned by
_dl_make_ftpr.

I'm still trying to understand your earlier recommendation. Do you
suggest that the executable have PLABEL32 relocs for all the symbols,
local and global, which assures that _dl_make_fptr creates a unique OPD?

Alan, thanks for your input.

c.

_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: OPD's on hppa-linux, and what to do about __cffc's fragility.

[Alan Modra]

On Thu, Mar 10, 2005 at 10:31:20AM -0500, Carlos O'Donell wrote:
> On Wed, Mar 09, 2005 at 08:55:13AM +1030, Alan Modra wrote:
> > No, in an hppa-linux shared lib, a function pointer is accessed via a
> > word with a PLABEL32 reloc on it.  This reloc is processed before any
> > code in the shared lib runs, to point to a function descriptor created
> > by _dl_make_fptr.
> 
> In shared libraries the function pointer is also allocated a PLT slot
> and an IPLT relocation. 
> 
> I don't understand the reasons for the allocation of a PLT slot if all
> references generated go through the local label word that has the
> PLABEL32 reloc, which now contains the address returned by
> _dl_make_ftpr.

There probably isn't a good reason.  Blame that on me not understanding
what I was doing when dhd and I hacked together ld and glibc hppa-linux
support.

> I'm still trying to understand your earlier recommendation. Do you
> suggest that the executable have PLABEL32 relocs for all the symbols,
> local and global, which assures that _dl_make_fptr creates a unique OPD?

I was really just thinking of global symbols.  If ld handles descriptors
for locals, you save on dynamic relocs and program startup time.  I
haven't spent a great deal of time thinking about it though, so it's
highly likely that there is some reason why it won't work.  eg. You
wouldn't want dynamic plabel relocs to appear in read-only sections.

-- 
Alan Modra
IBM OzLabs - Linux Technology Centre
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Solution to OPD's in hppa-linux, including a transition plan?

[Carlos O'Donell]

On Fri, Mar 11, 2005 at 08:57:51AM +1030, Alan Modra wrote:
> > In shared libraries the function pointer is also allocated a PLT slot
> > and an IPLT relocation. 
> > 
> > I don't understand the reasons for the allocation of a PLT slot if all
> > references generated go through the local label word that has the
> > PLABEL32 reloc, which now contains the address returned by
> > _dl_make_ftpr.
> 
> There probably isn't a good reason.  Blame that on me not understanding
> what I was doing when dhd and I hacked together ld and glibc hppa-linux
> support.

I understand why you allocate a PLT slot, because function calls seem to
go through the PLT only without reference to the PLABEL32 word (in
shared libraries). This saves an indirection through the PLABEL32 word
during a call. I believe this to be what is happening, and I verfiied it
lightly.

I went through the sources again, and you made some argument in a
comment about passing "local function pointers" to other objects.

The only way to verify my ideas is to run some tests, so here is the
apparent summary.

- Executable passes local function pointer to shared object.
 = Passes the address of the PLT entry.
 = PLT entry has IPLT reloc.

- Shared object passes local function pointer to other shared object.
 = Passes the address of *(PLABEL32 word) which points to a fdesc
   made by _dl_make_fptr.
 = The PLABEL32 word has a PLABEL32 reloc.

- Shared object calls a local or global function.
 = Calls through the PLT, which has an IPLT reloc.

- Executable calls local function.
 = Jumps directly to address.

- Executable calls global function.
 = Uses dyncall on the address of hte PLT
 = PLT has IPLT reloc.

> > I'm still trying to understand your earlier recommendation. Do you
> > suggest that the executable have PLABEL32 relocs for all the symbols,
> > local and global, which assures that _dl_make_fptr creates a unique OPD?
> 
> I was really just thinking of global symbols.  If ld handles descriptors
> for locals, you save on dynamic relocs and program startup time.  I
> haven't spent a great deal of time thinking about it though, so it's
> highly likely that there is some reason why it won't work.  eg. You
> wouldn't want dynamic plabel relocs to appear in read-only sections.

I thought about this, and you are correct. Technically if the shared
object can do a comparison then the symbol must have been global, and
passed into the function.

How do I do this in a backwards compatible way?

Action
=======
If we emit a PLABEL32 reloc for all global symbols in the executable,
baring relocs in read-only sections, which I'm sure shared libraries
have the same problems, then the executable passes the address stored
in the plabel word.

Case 1:
=======
For these new executables gcc will not generate __cffc comparisons.
For new libraries gcc will not generate __cffc comparisons.

Case 2:
=======
Old executables will still pass the address of the plt entry, and
comparisons in new shared libraries will fail.

Case 3:
=======
New executables and old shared libraries will work just fine, since
__cffc will think that everything has been resolved properly and just
do comparison.

Fixes:
======
Transition all executables to use PLABEL32 relocs for global symbols,
the same as shared libraries. Keep gcc generating __cffc.
One day when all the executables have been rebuilt, we can start turning
off the generation of __cffc and "Case 2" will fail, but hopefully we
have transitioned away from that happening.

Thoughts?

Cheers,
Carlos.

_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Solution to OPD's in hppa-linux, including a transition plan?

[John David Anglin]

> On Fri, Mar 11, 2005 at 08:57:51AM +1030, Alan Modra wrote:
> > > In shared libraries the function pointer is also allocated a PLT slot
> > > and an IPLT relocation. 
> > > 
> > > I don't understand the reasons for the allocation of a PLT slot if all
> > > references generated go through the local label word that has the
> > > PLABEL32 reloc, which now contains the address returned by
> > > _dl_make_ftpr.
> > 
> > There probably isn't a good reason.  Blame that on me not understanding
> > what I was doing when dhd and I hacked together ld and glibc hppa-linux
> > support.
> 
> I understand why you allocate a PLT slot, because function calls seem to
> go through the PLT only without reference to the PLABEL32 word (in
> shared libraries). This saves an indirection through the PLABEL32 word
> during a call. I believe this to be what is happening, and I verfiied it
> lightly.

I believe the PLT exists to support 'LTP and 'RTP relocations.
GCC doesn't generate these relocations as this support doesn't
work under HP-UX, although the HP-UX assembler manual describes
them.  GCC generates 32-bit plabels (deferred) when it encounters
a situation where a 'LTP/'RTP sequence would be desireable.  Thus,
we may not need the PLT slot.

I don't see that a level of indirection is being saved.  We either
go through the PLABEL32 word or the PLT.

I believe that GCC always create the PLABEL32 words in the data
section (definitely true for HP-UX) so that the word can be
successfully relocated when a shared library is loaded.

> I thought about this, and you are correct. Technically if the shared
> object can do a comparison then the symbol must have been global, and
> passed into the function.

I don't think this is correct.  It's theoretically possible to pass
two function pointers that both point to local functions in one object
and have the comparison done in another (shared) object.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Solution to OPD's in hppa-linux, including a transition plan?

[Carlos O'Donell]

On Wed, Mar 16, 2005 at 09:11:42AM +1030, Alan Modra wrote:
> On Tue, Mar 15, 2005 at 05:08:48PM -0500, Carlos O'Donell wrote:
> > The problem is that in a normal executable those words just point at the
> > PLT and they shouldn't. That case should just generate a PLABEL32 reloc
> > in the executable. 
> 
> ld still needs to generate function descriptors for plabels that resolve
> locally, eg. because the symbol is for a static function.

I don't understand how this case changes the situation, can you elaborate a 
bit more?

Thanks,
Carlos.


_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Solution to OPD's in hppa-linux, including a transition plan?

[Alan Modra]

On Wed, Mar 16, 2005 at 03:36:57PM -0500, Carlos O'Donell wrote:
> On Wed, Mar 16, 2005 at 09:11:42AM +1030, Alan Modra wrote:
> > On Tue, Mar 15, 2005 at 05:08:48PM -0500, Carlos O'Donell wrote:
> > > The problem is that in a normal executable those words just point at the
> > > PLT and they shouldn't. That case should just generate a PLABEL32 reloc
> > > in the executable. 
> > 
> > ld still needs to generate function descriptors for plabels that resolve
> > locally, eg. because the symbol is for a static function.
> 
> I don't understand how this case changes the situation, can you elaborate a 
> bit more?

If you take the address of a static function (or a hidden function),
you'll need a plabel somewhere.  You can't expect ld.so to handle this
via _dl_make_fptr, because the static function symbol isn't available to
ld.so.  Also, don't forget that ld needs to do everything for a fully
static link.

-- 
Alan Modra
IBM OzLabs - Linux Technology Centre
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Solution to OPD's in hppa-linux, including a transition plan?

[Carlos O'Donell]

On Tue, Mar 15, 2005 at 11:10:21PM -0500, John David Anglin wrote:
> > No. Read further, if I'm wrong please correct me.
> > 
> > The PLABEL32 contains the address of the function descriptor, while the
> > data in the PLT *is* the function descriptor. There is an extra lookup.
> > 
> > You have to load the address of the PLABEL32, load the value from there,
> > and then use that as your function descriptor. We currently *don't* do
> > this. The reason the indirection is there for PLABEL32 is such that the
> > dynamic loader can acutally assign the address of the OPD into that
> > word. 
> 
> No, I don't believe this is correct.  We do load the address
> from the PLABEL32 word.  That's the function pointer and what's
> passed to __cffc.  The problem is we don't have an OPD for the
> functions called in this manner.  We have more than one function
> descriptor for a any given function (possibly one for every
> PLABEL32 relocation).

We *do* have a n OPD for functions called in this manner, but only if
the plabel word carried a PLABEL32 reloc. The use of _dl_make_fptr
makes sure each symbol gets a unique function descriptor and will never
create duplicates.

The problem is that in the executable, we lazily point all the plabel
words at their associated plt entries and *splat* they are no longer
unique. Please note this *only* happens in the executable. Executables
do not generate PLABEL32 relocs for their plabel words.

I want to recommend that the executable emit PLABEL32 relocs against all
plabel word entries. The dynamic loader would then fill these entries
with unique OPD's as it has done so for shared libraries (this code was
originall wirtten by Bame et. al. and I ported hppa to use Lu's
non-locking generic version).
 
> On hppa64-hpux, we use essentially the same code to load function
> pointers from FPTR64 double words.  There we don't need to
> canonicalize the pointers because they point to the OPD for the
> function.

Sure, but the hpux libc probably does all the work to make sure that the
plabel words point to OPD's even for the executable :)

> > The problem is that in a normal executable those words just point at the
> > PLT and they shouldn't. That case should just generate a PLABEL32 reloc
> > in the executable. 
> > 
> > Do I understand the code correctly?
> 
> I think the situation envisioned in the parisc elf specification
> always envisioned a two level lookup.  In the relocations defined
> for 32-bit elf, the only way to load the address of a function
> descriptor is using the PLABEL32 relocation.  The 64-bit elf
> table also provides the LTOFF_FPTR21L and LTOFF_FPTR14R relocations.
> In this case, I believe that code has to load the address of the function
> descriptor from the linkage table, so it's still a two level lookup.

Yes, correct, I agree. Loading the *address* of a function descriptor
always requires a PLABEL32 reloction. The PLABEL32 reloction is handled
by the dynamic loader, which creates the OPD, and places the address of
the OPD into the plabel word.

Yes, as long as LTOFF_FTPR* load the address of the descriptor from the
linkage table (plt) then the system still works.

Hence, why I suggest executables also use PLABEL32 relocs if they take
the address of a function.

> The 32-bit ABI has the complication of the plabel bit in the function
> pointer.  While it might be possible to directly load the address
> of the function descriptor with an addil/ldo sequence if suitable
> relocations existed, saving one memory access, the code would still
> have to call $$dyncall or perform equivalent operations inline to
> test and strip off the plabel bit.

Yes, calling a function descriptor will always involve clearing the
plabel bit. I don't quite understand the performance gain to be made in
detecting that the plabel word contained a local address as opposed to a
function descriptor?

Yes, with a new relocation it *might* be possible to load the function
descriptor address directly from the plabel word, but as you say it 
would require writing into the code to change offsets?

> There are several disadvantages in the approach.  Plabels and linkage
> table entries can be shared for many calls to a given function.  Thus,
> the dynamic loader probably has an easier job doing the relocations.
> Also, I think that relocations might have to be redone (at least under
> hpux) when shared libraries are dynamically loaded by an application.
> This would require that the dynamic loader have the capability to
> write to code space after a program has started execution.  This is
> probably a security risk, particularly for shared libraries that are
> not privately mapped.  As I noted previously, the plabel words have
> to go in a read/write section because of this issue.

Do you really mean "disadvantages?"

Can you explain how plabels and plt entries can be shared for many calls
to a given function?

Are we still talking about the new reloc that could load the function
descriptor address directly?

Yes, plabel words go in read/write since they are written to by the
dynamic loader (only if they have a PLABEL32 reloc though).

---

This has definately been a good learning discussion, from the viewpoint
of optimal schemes for this type of architecture :)

Cheers,
Carlos.
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: Solution to OPD's in hppa-linux, including a transition plan?

[John David Anglin]

> We *do* have a n OPD for functions called in this manner, but only if
> the plabel word carried a PLABEL32 reloc. The use of _dl_make_fptr
> makes sure each symbol gets a unique function descriptor and will never
> create duplicates.
> 
> The problem is that in the executable, we lazily point all the plabel
> words at their associated plt entries and *splat* they are no longer
> unique. Please note this *only* happens in the executable. Executables
> do not generate PLABEL32 relocs for their plabel words.
> 
> I want to recommend that the executable emit PLABEL32 relocs against all
> plabel word entries. The dynamic loader would then fill these entries
> with unique OPD's as it has done so for shared libraries (this code was
> originall wirtten by Bame et. al. and I ported hppa to use Lu's
> non-locking generic version).

I think you are correct.  I hadn't realized that we are only dropping
the PLABEL32 in executables.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: Non-inline math, and inline math broken, GCC to blame? (1 hppa tls toolchain regression).

[John David Anglin]

> > I think gcc is to blame, or the constraints in the assembly?
> > 
> > --- GCC 4.0 ---
> > 
> > 0x40349890 <feclearexcept+0>:   ldo 40(sp),sp
> > 0x40349894 <feclearexcept+4>:   ldo -38(sp),ret0
> > 0x40349898 <feclearexcept+8>:   stw r19,-20(,sp)
> > 0x4034989c <feclearexcept+12>:  fstd fr0,0(,ret0)
> > 0x403498a0 <feclearexcept+16>:  fldd 0(,ret0),fr0
> > 0x403498a4 <feclearexcept+20>:  ldi 0,ret0
> > 0x403498a8 <feclearexcept+24>:  bv r0(rp)
> > 0x403498ac <feclearexcept+28>:  ldo -40(sp),sp
> 
> PR time.  Please add danglin@gcc.gnu.org to the CC list.

Changed my mind.  It's not a GCC bug.  The second asm has to show that
it uses the memory set by the first:

#include <fenv.h>

int
feclearexcept (int excepts)
{
  struct { unsigned int sw[2]; } s;

  /* Get the current status word. */
  __asm__ volatile ("fstd %%fr0,0(%1)" : "=m" (s) : "r" (&s));

  /* Clear all the relevant bits. */
  s.sw[0] &= ~((excepts & FE_ALL_EXCEPT) << 27);
  __asm__ volatile ("fldd 0(%0),%%fr0" : : "r" (&s), "m" (s));

  /* Success.  */
  return 0;
}

I used a struct so that GCC would know exactly what memory was affected.

There is one other issue that I'm not sure about.  I presume that you
used fldd to access the floating-point status register to avoid the
situation where a floating-point identify instruction has been previously
been executed and the status register is in format three.  It's not
clear from the arch whether the fldd returns the result of the identify
or not.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: Non-inline math, and inline math broken, GCC to blame? (1 hppa tls toolchain regression).

[Carlos O'Donell]

On Fri, Jul 15, 2005 at 10:55:26PM -0400, John David Anglin wrote:
> > PR time.  Please add danglin@gcc.gnu.org to the CC list.
> 
> Changed my mind.  It's not a GCC bug.  The second asm has to show that
> it uses the memory set by the first:

Ah, thank you for the clarification. I'll try to examine the rest of
them and cleanup the assembly.
 
> #include <fenv.h>
> 
> int
> feclearexcept (int excepts)
> {
>   struct { unsigned int sw[2]; } s;
> 
>   /* Get the current status word. */
>   __asm__ volatile ("fstd %%fr0,0(%1)" : "=m" (s) : "r" (&s));
> 
>   /* Clear all the relevant bits. */
>   s.sw[0] &= ~((excepts & FE_ALL_EXCEPT) << 27);
>   __asm__ volatile ("fldd 0(%0),%%fr0" : : "r" (&s), "m" (s));
> 
>   /* Success.  */
>   return 0;
> }
> 
> I used a struct so that GCC would know exactly what memory was affected.
> 
> There is one other issue that I'm not sure about.  I presume that you
> used fldd to access the floating-point status register to avoid the
> situation where a floating-point identify instruction has been previously
> been executed and the status register is in format three.  It's not
> clear from the arch whether the fldd returns the result of the identify
> or not.

Who previously executed the COPR,0,0?

I use the fldd to set the new cleared state of the status register.

If the thread had previously executed 'fstd' followed by a 'copr,0,0'
and had not immediately thereafter executed another 'fstd' then the
operation is *undefined* as per PA 1.1 6-63.

Context switching should save and restore the states properly for this
to work.

I think the arch is very clear about the issue, but perhaps I'm
misreading something. Do you have a case where the above wouldn't work?

c.

_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: Non-inline math, and inline math broken, GCC to blame? (1 hppa tls toolchain regression).

[John David Anglin]

> Who previously executed the COPR,0,0?

I dunno, incorrect identify sequence?  I agree the 1.1 arch is specific
about how to do an identify sequence.

> I use the fldd to set the new cleared state of the status register.

I had been wondering if fstw and fldw could be used, but it's an
undefined operation to set T with fldw and fstd clears T, so I believe
your use of fstd and fldd is correct in this sequence.

However, you have to be careful in using fstd to get the fp status
register as it clears T after the insn.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: Non-inline math, and inline math broken, GCC to blame? (1 hppa tls toolchain regression).

[John David Anglin]

> However, you have to be careful in using fstd to get the fp status
> register as it clears T after the insn.

And, indeed the following is wrong for the above reason:

#include <fenv.h>

int
fetestexcept (int excepts)
{
  unsigned int sw[2];

  /* Get the current status word. */
  __asm__ ("fstd %%fr0,0(%1)" : "=m" (*sw) : "r" (sw));

  return (sw[0] >> 27) & excepts & FE_ALL_EXCEPT;
}

You need to add a 'fldd' to the asm to restore the T bit.  Another
option might be to use fstw to access the status register in this
situation.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: Non-inline math, and inline math broken, GCC to blame? (1 hppa tls toolchain regression).

[Carlos O'Donell]

On Sat, Jul 16, 2005 at 01:54:48PM -0400, John David Anglin wrote:
> > However, you have to be careful in using fstd to get the fp status
> > register as it clears T after the insn.
> 
> And, indeed the following is wrong for the above reason:
> 
> #include <fenv.h>
> 
> int
> fetestexcept (int excepts)
> {
>   unsigned int sw[2];
> 
>   /* Get the current status word. */
>   __asm__ ("fstd %%fr0,0(%1)" : "=m" (*sw) : "r" (sw));
> 
>   return (sw[0] >> 27) & excepts & FE_ALL_EXCEPT;
> }
> 
> You need to add a 'fldd' to the asm to restore the T bit.  Another
> option might be to use fstw to access the status register in this
> situation.

I'm going to test the use of fstw.

fegetexcept, fegetround also need this fix (maybe more).

There is another potential bug, I should make sure the fr0 restore is
always at the end too incase it rearms a trap.

c.
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: Non-inline math, and inline math broken, GCC to blame? (1 hppa tls toolchain regression).

[John David Anglin]

> There is another potential bug, I should make sure the fr0 restore is
> always at the end too incase it rearms a trap.

hpux uses two little hand-coded functions to avoid this problem.  The
get operation uses fstd but it immediately restores the previous status
value with fldd.  The put does the restore in the delay slot of the return.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: Non-inline math, and inline math broken, GCC to blame? (1 hppa tls toolchain regression).

[Carlos O'Donell]

On Sat, Jul 16, 2005 at 01:37:50PM -0400, John David Anglin wrote:
> > Who previously executed the COPR,0,0?
> 
> I dunno, incorrect identify sequence?  I agree the 1.1 arch is specific
> about how to do an identify sequence.
 
There is no way to prevent this. An application can always shoot itself
in the foot.

And since the identify sequence should be written in assembly the
compiler can't schedule it around.

It should also be save to execute such a sequence at any point in the
insn stream.

> > I use the fldd to set the new cleared state of the status register.
> 
> I had been wondering if fstw and fldw could be used, but it's an
> undefined operation to set T with fldw and fstd clears T, so I believe
> your use of fstd and fldd is correct in this sequence.

I have to read this again to make sure I did it all right.

> However, you have to be careful in using fstd to get the fp status
> register as it clears T after the insn.

Yes, I might have some mistakes there.

c.
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: gsyprf11 and 2.6.13-rc3-pa1

[John David Anglin]

> > static void sighandler(int sig)
> > {
> >         printf("Got signal %d\n", sig);
> >         exit(0);
> 
> I think you need to use _exit in a signal handler although this may
> not be an issue when there is only one thread.

GCC uses exit.  Either way, the kernel shouldn't hang.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] strtold

[John David Anglin]

With libc6 2.3.5-8:

#include <stdlib.h>
int
main ()
{
  printf ("%Lf\n", strtold ("1", NULL));
    return 0;
}

dave@gsyprf11:~/gcc_test$ gcc -o ld1 ld1.c
dave@gsyprf11:~/gcc_test$ ./ld1
0.000000

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: strtold

[Carlos O'Donell]

On Mon, Dec 26, 2005 at 02:58:40PM -0500, John David Anglin wrote:
> With libc6 2.3.5-8:
> 
> #include <stdlib.h>
> int
> main ()
> {
>   printf ("%Lf\n", strtold ("1", NULL));
>     return 0;
> }
> 
> dave@gsyprf11:~/gcc_test$ gcc -o ld1 ld1.c
> dave@gsyprf11:~/gcc_test$ ./ld1
> 0.000000

strtold will never work, gcc and libc have different sizes for long
double. This is probably a side-effect of this.

If we really want long double to work we need to do something like set
it to be the same as double.

128-bit long double is just going to be a pain, and I don't know how
well the 80-bit long double from IBM supports IEEE modes.

c.

_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: strtold

[John David Anglin]

> strtold will never work, gcc and libc have different sizes for long
> double. This is probably a side-effect of this.

I'm not sure what has changed but this is the source of a number
of gfortran and libstdc++ regressions when using the current version
of libc in unstable.

> If we really want long double to work we need to do something like set
> it to be the same as double.

Long double has always been the same as double in GCC.  It allowed
by the C and C++ standards.  It fast and the hardware and corner
cases are well understood.

I've given up hope that HP will release their IEEE long double emulation
code for PA-RISC.  Although I don't have any hard evidence of bugs in it,
there is the suggestion from the ada testsuite run under hpux that there
may be bugs in the rounding and exception support.

> 128-bit long double is just going to be a pain, and I don't know how
> well the 80-bit long double from IBM supports IEEE modes.

The only viable alternatives at this point are:

  1) long double == double (64 bit long double), or
  2) IBM 128-bit double double implementation used by powerpc.

After more than a year of using the double-double format, the powerpc
folks are still working the bugs out from the corner cases.  While I
guess it more or less works, I wouldn't want to rely on it for
critical calculations.  RTH recented indicated that this would be a
poor choice for another port in the same situation.  The same viewpoint
was expressed to me by an Ada maintainer at the last GCC summit.  Ada
probably has the best numerics support in GCC.

There is no support for the 80-bit format (either in hardware or
software) on PA-RISC as far as I know.  So, this alternative isn't viable.

C99 requires long double support.  Thus, I believe we should go with
option 1 and libc should be changed.  We aren't the only target with
this problem, so there must be some libc support for 64-bit long double.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: strtold

[Carlos O'Donell]

On Sat, Jan 07, 2006 at 05:41:52PM -0500, John David Anglin wrote:
> > strtold will never work, gcc and libc have different sizes for long
> > double. This is probably a side-effect of this.
> 
> I'm not sure what has changed but this is the source of a number
> of gfortran and libstdc++ regressions when using the current version
> of libc in unstable.

strold has always done funny things in the glibc testsuite, consider
yourself lucky.
 
> > If we really want long double to work we need to do something like set
> > it to be the same as double.
> 
> Long double has always been the same as double in GCC.  It allowed
> by the C and C++ standards.  It fast and the hardware and corner
> cases are well understood.

So long doubles have always been DFmode in GCC, okay, I thought it was
something else.

glibc thinks long double is 128-bit. If you want strtold to work then I
need to make the changes in glibc.

> I've given up hope that HP will release their IEEE long double emulation
> code for PA-RISC.  Although I don't have any hard evidence of bugs in it,
> there is the suggestion from the ada testsuite run under hpux that there
> may be bugs in the rounding and exception support.

I agree, I don't think it will ever be released. If it is released then
we'll jigger it out some other way :)

> > 128-bit long double is just going to be a pain, and I don't know how
> > well the 80-bit long double from IBM supports IEEE modes.
> 
> The only viable alternatives at this point are:
> 
>   1) long double == double (64 bit long double), or
>   2) IBM 128-bit double double implementation used by powerpc.

Or write our own library! ;-)

> After more than a year of using the double-double format, the powerpc
> folks are still working the bugs out from the corner cases.  While I
> guess it more or less works, I wouldn't want to rely on it for
> critical calculations.  RTH recented indicated that this would be a
> poor choice for another port in the same situation.  The same viewpoint
> was expressed to me by an Ada maintainer at the last GCC summit.  Ada
> probably has the best numerics support in GCC.

I say we go for (1).

> There is no support for the 80-bit format (either in hardware or
> software) on PA-RISC as far as I know.  So, this alternative isn't viable.

Okay.

> C99 requires long double support.  Thus, I believe we should go with
> option 1 and libc should be changed.  We aren't the only target with
> this problem, so there must be some libc support for 64-bit long double.

I believe it will only require removing the implies line for ldbl-128
from glibc.

I am building you a new debian glibc based on 2.3.5-8 for you to test,
would this be good?

c.

_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: strtold

[John David Anglin]

> I believe it will only require removing the implies line for ldbl-128
> from glibc.

Excellent.

> I am building you a new debian glibc based on 2.3.5-8 for you to test,
> would this be good?

Yes.

Do you also have a fix for FP exception support?

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: strtold

[John David Anglin]

> So long doubles have always been DFmode in GCC, okay, I thought it was
> something else.

linux long doubles are DFmode (hardware+kernel support).  hpux long
doubles are TFmode (software emulation).  Sometimes I wish there was
an option to switch between hardware and software long doubles on
hpux.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: strtold

[Grant Grundler]

On Sat, Jan 07, 2006 at 06:42:30PM -0500, Carlos O'Donell wrote:
...
> > I've given up hope that HP will release their IEEE long double emulation
> > code for PA-RISC.  Although I don't have any hard evidence of bugs in it,
> > there is the suggestion from the ada testsuite run under hpux that there
> > may be bugs in the rounding and exception support.
> 
> I agree, I don't think it will ever be released. If it is released then
> we'll jigger it out some other way :)

Me too. But I wouldn't worry about an ABI event too much.
As long as the only dependency is debian/gentoo packages,
then I think we can tolerate an ABI event like this on the
next OS release if necessary.

grant
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: nscd: error while loading shared libraries: unexpected reloc type 0x42

[John David Anglin]

> So they must come from an asm or assembly code.  I think libc used this
> reloc for loading the address of the function descriptor of main.

objdump -R nscd
...
00005970 R_PARISC_PLABEL21L  main
00005974 R_PARISC_PLABEL14R  main
00005980 R_PARISC_PLABEL21L  __libc_csu_init
00005984 R_PARISC_PLABEL14R  __libc_csu_init
00005988 R_PARISC_PLABEL21L  __libc_csu_fini
0000598c R_PARISC_PLABEL14R  __libc_csu_fini

If I recall correctly, there was a dynamic loader fix to handle these
relocations.  However, it doesn't seem to be in libc6 2.3.5-8.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: nscd: error while loading shared libraries: unexpected reloc type 0x42

[Carlos O'Donell]

On Mon, Dec 26, 2005 at 04:54:42PM -0500, John David Anglin wrote:
> > So they must come from an asm or assembly code.  I think libc used this
> > reloc for loading the address of the function descriptor of main.
> 
> objdump -R nscd
> ...
> 00005970 R_PARISC_PLABEL21L  main
> 00005974 R_PARISC_PLABEL14R  main
> 00005980 R_PARISC_PLABEL21L  __libc_csu_init
> 00005984 R_PARISC_PLABEL14R  __libc_csu_init
> 00005988 R_PARISC_PLABEL21L  __libc_csu_fini
> 0000598c R_PARISC_PLABEL14R  __libc_csu_fini
> 
> If I recall correctly, there was a dynamic loader fix to handle these
> relocations.  However, it doesn't seem to be in libc6 2.3.5-8.

It is in the one I have installed on my system :)

The proper fix was this:
http://cvs.parisc-linux.org/glibc/sysdeps/hppa/elf/start.S?r1=1.2&r2=1.3

Please verify.

c.

_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: nscd: error while loading shared libraries:

[John David Anglin]

> The proper fix was this:
> http://cvs.parisc-linux.org/glibc/sysdeps/hppa/elf/start.S?r1=1.2&r2=1.3
> 
> Please verify.

Looks right.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] long double on hppa64-*-linux*

[John David Anglin]

I noticed another long double issue.  Long double is defined to be 128
bits on all 64-bit targets on the PA.  I believe that it should be 128 bits
only on hpux.

I would like to change the long double default on PA64t o 64 bits as there
isn't any hardware support for the 128-bit type, and we currently don't
have software support for 128-bit long doubles implemented.  I can't see
this affecting anything, but it's another ABI change and I thought I should
ask.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] long double on hppa64-*-linux*

[Grant Grundler]

On Sat, Feb 04, 2006 at 06:41:00PM -0500, John David Anglin wrote:
> I noticed another long double issue.  Long double is defined to be 128
> bits on all 64-bit targets on the PA.  I believe that it should be 128 bits
> only on hpux.
> 
> I would like to change the long double default on PA64t o 64 bits as there
> isn't any hardware support for the 128-bit type, and we currently don't
> have software support for 128-bit long doubles implemented.  I can't see
> this affecting anything, but it's another ABI change and I thought I should
> ask.

I'm ok with it. The only downside I can think of is hypothetical:
supporting HPUX binaries with "long double" on parisc-linux hmight get
interesting should any misaligned trap handlers or syscalls need to deal
with long double.

hth,
grant
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] long double on hppa64-*-linux*

[John David Anglin]

> I'm ok with it. The only downside I can think of is hypothetical:
> supporting HPUX binaries with "long double" on parisc-linux hmight get
> interesting should any misaligned trap handlers or syscalls need to deal
> with long double.

The HPUX support uses various _U routines from libc.  I'm not sure
if any kernel support is involved.  Possibly, some support is needed
to handle exceptions.  This is all vague since as far as I know the
_U routines are not publicly documented.  I don't believe there is kernel
support for the quad instructions in the PA 2.0 arch.  There's no
math library support for long doubles in HPUX.  As it seems HP won't
release the _U code, only static HPUX binaries using long double would
have any chance of running on parisc-linux.  So, I'm not sure being
able to run HPUX binaries with "long double" would ever prove particularly
useful.

As things currently stand on hppa64-*-linux*, any arithmetic involving
long doubles results in a call to a libgcc stub that calls abort.

It looks as glibc 2.4 potentially provides 128-bit long double support
as an option.  See patch to make -mlong-double-128 the default:
http://gcc.gnu.org/ml/gcc-patches/2006-01/msg01958.html
The other option is the hardware support -mlong-double-64.

I'm fairly sure that the -mlong-double-128 format (except for a few
architectures) is similar to the rs6000 darwin long double format
(i.e., it's not compatible with the PA 2.0 format, IEEE quad).  It
might not be that much work to copy the rs6000 implementation.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: gcj can't make shared libs on hppa

[John David Anglin]

>  83c:   6b d3 3f c1     stw r19,-20(sp)
>  840:   e8 5f 1f bd     b,l 824 <_init-0x10>,rp
>  844:   08 13 02 44     copy r19,r4
>  848:   08 04 02 53     copy r4,r19
>  84c:   e8 40 01 f0     b,l 94c <frame_dummy>,rp
>  850:   08 00 02 40     nop
>  854:   e8 40 05 c0     b,l b3c <__do_global_ctors_aux>,rp
>  858:   08 00 02 40     nop
>  85c:   4b c2 3f 59     ldw -54(sp),rp
>  860:   08 04 02 53     copy r4,r19
>  864:   e8 40 c0 00     bv r0(rp)

I'm testing the attached fix.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Index: config/pa/pa32-linux.h
===================================================================
--- config/pa/pa32-linux.h	(revision 111979)
+++ config/pa/pa32-linux.h	(working copy)
@@ -1,5 +1,5 @@
 /* Definitions for PA_RISC with ELF-32 format
-   Copyright (C) 2000, 2002, 2004 Free Software Foundation, Inc.
+   Copyright (C) 2000, 2002, 2004, 2006 Free Software Foundation, Inc.
 
 This file is part of GCC.
 
@@ -36,4 +36,29 @@
 		    aligned(sizeof(func_ptr))))				\
     = { (func_ptr) (-1) }
 
+/* This is a PIC version of CRT_CALL_STATIC_FUNCTION.  The PIC
+   register has to be saved before the call and restored after
+   the call.  We assume that register %r4 is available for this
+   purpose.  The hack prevents GCC from deleting the restore.  */
+#ifdef CRTSTUFFS_O
+#undef CRT_CALL_STATIC_FUNCTION
+#define CRT_CALL_STATIC_FUNCTION(SECTION_OP, FUNC)	\
+static void __attribute__((__used__))			\
+call_ ## FUNC (void)					\
+{							\
+  asm (SECTION_OP);					\
+  asm volatile (".call\n\t"				\
+		"bl " #FUNC ",%%r2\n\t"			\
+		"copy %%r19,%%r4\n\t"			\
+		"copy %%r4,%%r19\n"			\
+		:					\
+		:					\
+		: "r1", "r2", "r4", "r20", "r21",	\
+		  "r22", "r24", "r24", "r25", "r26",	\
+		  "r27", "r28", "r29", "r31");		\
+  FORCE_CODE_SECTION_ALIGN				\
+  asm (TEXT_SECTION_ASM_OP);				\
+}
+#endif
+
 #define MD_UNWIND_SUPPORT "config/pa/linux-unwind.h"
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: gcj can't make shared libs on hppa

[Carlos O'Donell]

> Index: config/pa/pa32-linux.h
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> --- config/pa/pa32-linux.h      (revision 111979)
> +++ config/pa/pa32-linux.h      (working copy)
> @@ -1,5 +1,5 @@
>  /* Definitions for PA_RISC with ELF-32 format
> -   Copyright (C) 2000, 2002, 2004 Free Software Foundation, Inc.
> +   Copyright (C) 2000, 2002, 2004, 2006 Free Software Foundation, Inc.
>
>  This file is part of GCC.
>
> @@ -36,4 +36,29 @@
>                     aligned(sizeof(func_ptr))))                         \
>      =3D { (func_ptr) (-1) }
>
> +/* This is a PIC version of CRT_CALL_STATIC_FUNCTION.  The PIC
> +   register has to be saved before the call and restored after
> +   the call.  We assume that register %r4 is available for this
> +   purpose.  The hack prevents GCC from deleting the restore.  */
> +#ifdef CRTSTUFFS_O
> +#undef CRT_CALL_STATIC_FUNCTION
> +#define CRT_CALL_STATIC_FUNCTION(SECTION_OP, FUNC)     \
> +static void __attribute__((__used__))                  \
> +call_ ## FUNC (void)                                   \
> +{                                                      \
> +  asm (SECTION_OP);                                    \
> +  asm volatile (".call\n\t"                            \
> +               "bl " #FUNC ",%%r2\n\t"                 \
> +               "copy %%r19,%%r4\n\t"                   \
> +               "copy %%r4,%%r19\n"                     \
> +               :                                       \
> +               :                                       \
> +               : "r1", "r2", "r4", "r20", "r21",       \
> +                 "r22", "r24", "r24", "r25", "r26",    \
> +                 "r27", "r28", "r29", "r31");          \
> +  FORCE_CODE_SECTION_ALIGN                             \
> +  asm (TEXT_SECTION_ASM_OP);                           \
> +}
> +#endif
> +
>  #define MD_UNWIND_SUPPORT "config/pa/linux-unwind.h"

Awesome, I hope this works. Thanks Randolph and Dave for the
sleuthing!

c.
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] Re: gcj can't make shared libs on hppa

[John David Anglin]

> Awesome, I hope this works.

Seems to, so I will be installing a simplified version to the GCC tree.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [parisc-linux] User space locks -- what's wrong

[John David Anglin]

> There definitely are some funky things going on in the kernel.
> For example, if I try to build GCC with 'make -j 4 bootstrap' on
> gsyprf11, about 50% of the time make dies because of a malloc data
> corruption or a shell problem.  However, I never see this with
> just 'make bootstrap'.

This is an example:

/home/dave/gcc-4.2/objdir/./prev-gcc/xgcc -B/home/dave/gcc-4.2/objdir/./prev-gcc
/ -B/home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/bin/ -c   -g -O2 -DIN_GCC   -W -
Wall -Wwrite-strings -Wstrict-prototypes -Wmissing-prototypes -pedantic -Wno-lon
g-long -Wno-variadic-macros -Wno-overlength-strings -Wold-style-definition -Wmis
sing-format-attribute -Werror -fno-common   -DHAVE_CONFIG_H -I. -I. -I../../gcc/
gcc -I../../gcc/gcc/. -I../../gcc/gcc/../include -I../../gcc/gcc/../libcpp/inclu
de  -I../../gcc/gcc/../libdecnumber -I../libdecnumber    ../../gcc/gcc/c-typeck.
c -o c-typeck.o
/bin/sh: line 6: 24262 Segmentation fault      (core dumped) make "DESTDIR=" "RP
ATH_ENVVAR=LD_LIBRARY_PATH" "TARGET_SUBDIR=hppa-linux" "bindir=/home/dave/opt/gn
u/gcc/gcc-4.2.0/bin" "datadir=/home/dave/opt/gnu/gcc/gcc-4.2.0/share" "exec_pref
ix=/home/dave/opt/gnu/gcc/gcc-4.2.0" "includedir=/home/dave/opt/gnu/gcc/gcc-4.2.
0/include" "datarootdir=/home/dave/opt/gnu/gcc/gcc-4.2.0/share" "docdir=/home/da
ve/opt/gnu/gcc/gcc-4.2.0/share/doc" "infodir=/home/dave/opt/gnu/gcc/gcc-4.2.0/in
fo" "htmldir=/home/dave/opt/gnu/gcc/gcc-4.2.0/share/doc" "libdir=/home/dave/opt/
gnu/gcc/gcc-4.2.0/lib" "libexecdir=/home/dave/opt/gnu/gcc/gcc-4.2.0/libexec" "li
spdir=" "localstatedir=/home/dave/opt/gnu/gcc/gcc-4.2.0/var" "mandir=/home/dave/
opt/gnu/gcc/gcc-4.2.0/man" "oldincludedir=/usr/include" "prefix=/home/dave/opt/g
nu/gcc/gcc-4.2.0" "sbindir=/home/dave/opt/gnu/gcc/gcc-4.2.0/sbin" "sharedstatedi
r=/home/dave/opt/gnu/gcc/gcc-4.2.0/com" "sysconfdir=/home/dave/opt/gnu/gcc/gcc-4
.2.0/etc" "tooldir=/home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux" "build_tooldir=/
home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux" "target_alias=hppa-linux" "BISON=bis
on" "CC_FOR_BUILD=gcc" "CFLAGS_FOR_BUILD=-g -O2" "CXX_FOR_BUILD=c++" "EXPECT=/ho
me/dave/opt/gnu/bin/expect" "FLEX=flex" "INSTALL=/usr/bin/install -c" "INSTALL_D
ATA=/usr/bin/install -c -m 644" "INSTALL_PROGRAM=/usr/bin/install -c" "INSTALL_S
CRIPT=/usr/bin/install -c" "LEX=flex" "M4=m4" "MAKE=make" "RUNTEST=runtest" "RUN
TESTFLAGS=" "SHELL=/bin/sh" "YACC=bison -y" "`echo 'ADAFLAGS=' | sed -e s'/[^=][
^=]*=$/XFOO=/'`" "AR_FLAGS=rc" "`echo 'BOOT_ADAFLAGS=' | sed -e s'/[^=][^=]*=$/X
FOO=/'`" "BOOT_CFLAGS=-g -O2" "BOOT_LDFLAGS=" "CFLAGS=-g -O2" "CXXFLAGS=-g -O2"
"LDFLAGS=" "LIBCFLAGS=-g -O2" "LIBCXXFLAGS=-g -O2 -fno-implicit-templates" "STAG
E1_CFLAGS=-g" "STAGE1_LANGUAGES=c,ada" "AR_FOR_TARGET=ar" "AS_FOR_TARGET=as" "CC
_FOR_TARGET=/home/dave/gcc-4.2/objdir/./gcc/xgcc -B/home/dave/gcc-4.2/objdir/./g
cc/ -B/home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/bin/ -B/home/dave/opt/gnu/gcc/
gcc-4.2.0/hppa-linux/lib/ -isystem /home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/i
nclude -isystem /home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/sys-include" "CFLAGS
_FOR_TARGET=-O2 -g -O2 " "CPPFLAGS_FOR_TARGET=" "CXX_FOR_TARGET=/home/dave/gcc-4
.2/objdir/./gcc/g++ -B/home/dave/gcc-4.2/objdir/./gcc/ -nostdinc++  -L/home/dave
/gcc-4.2/objdir/hppa-linux/libstdc++-v3/src -L/home/dave/gcc-4.2/objdir/hppa-lin
ux/libstdc++-v3/src/.libs -B/home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/bin/ -B/
home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/lib/ -isystem /home/dave/opt/gnu/gcc/
gcc-4.2.0/hppa-linux/include -isystem /home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linu
x/sys-include" "CXXFLAGS_FOR_TARGET=-g -O2  -D_GNU_SOURCE" "DLLTOOL_FOR_TARGET=d
lltool" "GCJ_FOR_TARGET=/home/dave/gcc-4.2/objdir/./gcc/gcj -B/home/dave/gcc-4.2
/objdir/./gcc/ -B/home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/bin/ -B/home/dave/o
pt/gnu/gcc/gcc-4.2.0/hppa-linux/lib/ -isystem /home/dave/opt/gnu/gcc/gcc-4.2.0/h
ppa-linux/include -isystem /home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/sys-inclu
de" "GFORTRAN_FOR_TARGET=/home/dave/gcc-4.2/objdir/./gcc/gfortran -B/home/dave/g
cc-4.2/objdir/./gcc/ -B/home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/bin/ -B/home/
dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/lib/ -isystem /home/dave/opt/gnu/gcc/gcc-4
.2.0/hppa-linux/include -isystem /home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/sys
-include" "LD_FOR_TARGET=ld" "LIPO_FOR_TARGET=lipo" "LDFLAGS_FOR_TARGET=" "LIBCF
LAGS_FOR_TARGET=-O2 -g -O2 " "LIBCXXFLAGS_FOR_TARGET=-g -O2  -D_GNU_SOURCE -fno-
implicit-templates" "NM_FOR_TARGET=nm" "OBJDUMP_FOR_TARGET=objdump" "RANLIB_FOR_
TARGET=ranlib" "STRIP_FOR_TARGET=strip" "WINDRES_FOR_TARGET=windres" "`echo 'LAN
GUAGES=' | sed -e s'/[^=][^=]*=$/XFOO=/'`" "LEAN=false" "CONFIG_SHELL=/bin/sh" "
MAKEINFO=makeinfo --split-size=5000000 --split-size=5000000 --split-size=5000000
" 'AR=ar' 'AS=as' 'CC=gcc' 'CXX=c++' 'DLLTOOL=dlltool' 'LD=ld' 'LIPO=lipo' 'NM=n
m' 'OBJDUMP=objdump' 'RANLIB=ranlib' 'STRIP=strip' 'WINDRES=windres' CC="${CC}"
CC_FOR_BUILD="${CC_FOR_BUILD}" STAGE_PREFIX=$r/prev-gcc/ CFLAGS="-g -O2" LIBCFLA
GS="-g -O2" LDFLAGS="" "`echo 'ADAFLAGS=' | sed -e s'/[^=][^=]*=$/XFOO=/'`" "GCC
_FOR_TARGET= $r/./gcc/xgcc -B$r/./gcc/ -B/home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-l
inux/bin/ -B/home/dave/opt/gnu/gcc/gcc-4.2.0/hppa-linux/lib/ -isystem /home/dave
/opt/gnu/gcc/gcc-4.2.0/hppa-linux/include -isystem /home/dave/opt/gnu/gcc/gcc-4.
2.0/hppa-linux/sys-include" "`echo 'STMP_FIXPROTO=' | sed -e s'/[^=][^=]*=$/XFOO
=/'`" "`echo 'LIMITS_H_TEST=' | sed -e s'/[^=][^=]*=$/XFOO=/'`" "`echo 'LIBGCC2_
CFLAGS=' | sed -e s'/[^=][^=]*=$/XFOO=/'`" "`echo 'LIBGCC2_DEBUG_CFLAGS=' | sed
-e s'/[^=][^=]*=$/XFOO=/'`" "`echo 'LIBGCC2_INCLUDES=' | sed -e s'/[^=][^=]*=$/X
FOO=/'`" `if [ -f stage_last ]; then echo quickstrap ; else echo all; fi`
make[2]: *** [all-stage2-gcc] Error 139
make[2]: Leaving directory `/home/dave/gcc-4.2/objdir'
make[1]: *** [stage2-bubble] Error 2
make[1]: Leaving directory `/home/dave/gcc-4.2/objdir'
make: *** [bootstrap] Error 2
make: INTERNAL: Exiting with 1 jobserver tokens available; should be 4!
Wed Jun  7 20:19:18 PDT 2006

>>From /var/log/debug:

Jun  7 20:19:18 gsyprf11 kernel: do_page_fault() pid=24262 command='make' type=1
5 address=0x1118498f
Jun  7 20:19:18 gsyprf11 kernel: vm_start = 0x00048000, vm_end = 0x00607000
Jun  7 20:19:18 gsyprf11 kernel:
Jun  7 20:19:18 gsyprf11 kernel:      YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI
Jun  7 20:19:18 gsyprf11 kernel: PSW: 00000000000001000000000000001111 Not taint
ed
Jun  7 20:19:18 gsyprf11 kernel: r00-03  0000000000000000 00000000000476e8 00000
00000029b03 00000000bff07800
Jun  7 20:19:18 gsyprf11 kernel: r04-07  000000000009b268 0000000000000000 00000
0000010fe80 00000000000d8680
Jun  7 20:19:18 gsyprf11 kernel: r08-11  00000000000d8680 0000000000000000 00000
0000010fe80 0000000000000000
Jun  7 20:19:18 gsyprf11 kernel: r12-15  000000000009b1f0 0000000000000000 00000
00000000014 0000000000000000
Jun  7 20:19:18 gsyprf11 kernel: r16-19  0000000000000001 0000000000000001 00000
000000476e8 0000000000000000
Jun  7 20:19:18 gsyprf11 kernel: r20-23  00000000001dfca0 0000000000000000 00000
000c0000003 0000000000000000
Jun  7 20:19:18 gsyprf11 kernel: r24-27  0000000000000001 0000000000000016 00000
000001dfca0 00000000000466e8
Jun  7 20:19:18 gsyprf11 kernel: r28-31  0000000011184943 0000000040000003 00000
00003bd6800 0000000003bd6800
Jun  7 20:19:18 gsyprf11 kernel:
Jun  7 20:19:18 gsyprf11 kernel: IASQ: 0000000003bd6800 0000000003bd6800 IAOQ: 0
000000000029aa3 0000000000029aa7
Jun  7 20:19:18 gsyprf11 kernel:  IIR: 4b930098    ISR: 0000000003bd6800  IOR: 0
00000001118498f
Jun  7 20:19:18 gsyprf11 kernel:  CPU:        1   CR30: 000000001abb0000 CR31: 0
000000000008020
Jun  7 20:19:18 gsyprf11 kernel:  ORIG_R28: 0000000000000000
Jun  7 20:19:18 gsyprf11 kernel:  IAOQ[0]: 0x29aa3
Jun  7 20:19:18 gsyprf11 kernel:  IAOQ[1]: 0x29aa7
Jun  7 20:19:18 gsyprf11 kernel:  RP(r2): 0x29b03

dave@hiauly6:~/opt/gnu/bin$ disasm 0x4b930098
   0:   4b 93 00 98     ldw 4c(ret0),r19

(gdb) disass 0x29a90 0x29ab0
Dump of assembler code from 0x29a90 to 0x29ab0:
0x00029a90 <close_stdout+25828>:        copy ret0,r20
0x00029a94 <close_stdout+25832>:        ldw 48(r20),ret0
0x00029a98 <close_stdout+25836>:        cmpiclr,<> 0,ret0,r0
0x00029a9c <close_stdout+25840>:        copy r20,ret0
0x00029aa0 <close_stdout+25844>:        ldw 4c(ret0),r19

%ret0 is both misaligned and appears outside the VM range for the application.

The backtrace isn't particularly enlightening (need debug version of make).

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: PA8800 Status (Happy New Year)

[John David Anglin]

> Bad Address (null pointer deref?): Code=15 regs=0000000245b4f590 (Addr=00000625733a213a)
> 
>      YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI
> PSW: 00001000000001001011110100001111 Not tainted
> r00-03  000000ff0804bd0f 000000004050f8c0 0000000040311010 0000000084f664ff
> r04-07  00000000405038c0 0000000000000002 0000000084f664c1 000000007fe0a000
> r08-11  0000000000000000 000000007fe0a000 00000000000002b1 000000007ea85600
> r12-15  000000004f2241c0 000000007e96c000 000000007fe0a000 0000000000000040
> r16-19  0000000000000000 000000007fe0a494 000000007ad0b012 00000000404ba2d4
> r20-23  3c353e25733a2046 000000004050f8c0 000000007fe0a000 0000000000000000
> r24-27  0000000084f664c1 0000000000013c43 0000000078340400 00000000405038c0
> r28-31  0000000000000003 0000000245b4f560 0000000245b4f590 00000000404ba2c0
> sr00-03  0000000001436800 0000000000000000 0000000000000000 0000000001436800
> sr04-07  0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 
> IASQ: 0000000000000000 0000000000000000 IAOQ: 000000004031104c 0000000040311050
>  IIR: 4a9c01e8    ISR: 000000003c353800  IOR: 00000625733a213a
>  CPU:        0   CR30: 0000000241ee0000 CR31: 00000000404c4000
>  ORIG_R28: 00000054000000c8
>  IAOQ[0]: ip_route_input+0xb4/0xca8
>  IAOQ[1]: ip_route_input+0xb8/0xca8
>  RP(r2): ip_route_input+0x78/0xca8
> Kernel panic - not syncing: Bad Address (null pointer deref?)
>  <0>Rebooting in 5 seconds..

The code seems to be:

    40311008:   eb fe ac 25     b,l 4030e620 <rt_hash_code>,rp
    4031100c:   0b 97 02 28     and r23,ret0,r8
    40311010:   08 04 02 5b     copy r4,dp
    40311014:   2b 63 20 00     addil L%7000,dp,r1
    40311018:   50 3f 04 f0     ldd 278(r1),r31
    4031101c:   0f e0 10 d3     ldd 0(r31),r19
    40311020:   0e 7c 20 dc     ldd,s ret0(r19),ret0
    40311024:   9f 80 22 00     cmpb,*= r0,ret0,4031112c <ip_route_input+0x194>
    40311028:   2b 66 00 00     addil L%c000,dp,r1
    4031102c:   08 1c 02 54     copy ret0,r20
    40311030:   50 33 0e a0     ldd 750(r1),r19
    40311034:   e8 00 00 20     b,l 4031104c <ip_route_input+0xb4>,r0
    40311038:   08 01 02 55     copy r1,r21
    4031103c:   37 9c 00 02     ldo 1(ret0),ret0
    40311040:   6a 7c 00 70     stw ret0,38(r19)
    40311044:   0e 80 10 d4     ldd 0(r20),r20
    40311048:   9e 80 21 ba     cmpb,*=,n r0,r20,4031112c <ip_route_input+0x194>
    4031104c:   4a 9c 01 e8     ldw f4(r20),ret0

It seems like there must be some way to get to 4031104c avoiding the
on r20/ret0.  Don't see any branches to the region between 40311024
and 40311044 in the function, or any other direct branches to 4031104c.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: PA8800 Status (Happy New Year)

[John David Anglin]

> expect (pid 20348): Protection id trap (code 27)
> Backtrace:
>       _______________________________
>      < Your System ate a SPARC! Gah! >

It looks like show_stack is broken.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[parisc-linux] Re: call_init in libc6 2.3.6.ds1-11

[John David Anglin]

> /* The test for "addr & 2" below is to accomodate old binaries which
>    violated the ELF ABI by pointing DT_INIT and DT_FINI at a function
>    descriptor.  */
> #define DL_DT_INIT_ADDRESS(map, addr) \
>   ((Elf32_Addr)(addr) & 2 ? (addr) : DL_AUTO_FUNCTION_ADDRESS (map, addr))

This is the ia64 define:

#define DL_DT_INIT_ADDRESS(map, addr) DL_AUTO_FUNCTION_ADDRESS (map, addr)

Calls to init on ia64 always use a function descriptor.

While it's possible that DT_INIT and DT_FINI shouldn't point at function
descriptors, the above hack can't be right since DL_DT_INIT_ADDRESS is used
for calling init() from dl-init.c:

      init_t init = (init_t) DL_DT_INIT_ADDRESS
	(l, l->l_addr + l->l_info[DT_INIT]->d_un.d_ptr);

      /* Call the function.  */
      init (argc, argv, env);

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

[PATCH] Assign task_struct.exit_code before taskstats_exit()

[Jonathan Lim]

taskstats.ac_exitcode is assigned to task_struct.exit_code in bacct_add_tsk()
through the following kernel function calls: 

  do_exit()
    taskstats_exit()
      fill_pid()
        bacct_add_tsk()

The problem is that in do_exit(), task_struct.exit_code is set to 'code' only
after taskstats_exit() has been called.  So we need to move the assignment
before taskstats_exit().

Diff'd against: linux/kernel/git/stable/linux-2.6.22.y.git

Signed-off-by: Jonathan Lim <jlim@sgi.com>

--- a/kernel/exit.c	2007-08-20 16:37:55.000000000 -0700
+++ b/kernel/exit.c	2007-08-21 11:00:56.000000000 -0700
@@ -941,6 +941,7 @@ fastcall NORET_TYPE void do_exit(long co
 	if (unlikely(tsk->audit_context))
 		audit_free(tsk);
 
+	tsk->exit_code = code;
 	taskstats_exit(tsk, group_dead);
 
 	exit_mm(tsk);
@@ -961,7 +962,6 @@ fastcall NORET_TYPE void do_exit(long co
 	if (tsk->binfmt)
 		module_put(tsk->binfmt->module);
 
-	tsk->exit_code = code;
 	proc_exit_connector(tsk);
 	exit_task_namespaces(tsk);
 	exit_notify(tsk);

Re: [parisc-linux] 2.6.23-rc5 warnings

[John David Anglin]

Thumbs down on 2.6.23-rc5.  I had segmentation faults in tools like
'make' twice in a row.  Back to 2.6.22.6.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)
_______________________________________________
parisc-linux mailing list
parisc-linux@lists.parisc-linux.org
http://lists.parisc-linux.org/mailman/listinfo/parisc-linux

Re: [PATCH] Provide u64 version of jiffies_to_usecs() in kernel/tsacct.c

[Jonathan Lim]

It's possible that the values used in and returned from jiffies_to_usecs() are
incorrect because of truncation when variables of type u64 are involved.  So a
function specific to that type is used instead.

Updated from previous submission with feedback from Peter Anvin.

Diff'd against: linux/kernel/git/torvalds/linux-2.6.git

Signed-off-by: Jonathan Lim <jlim@sgi.com>

--- a/include/linux/jiffies.h	2008-01-07 16:09:17.737040981 -0800
+++ b/include/linux/jiffies.h	2008-01-07 16:09:17.693033003 -0800
@@ -36,7 +36,7 @@
 /* LATCH is used in the interval timer and ftape setup. */
 #define LATCH  ((CLOCK_TICK_RATE + HZ/2) / HZ)	/* For divider */
 
-/* Suppose we want to devide two numbers NOM and DEN: NOM/DEN, then we can
+/* Suppose we want to divide two numbers NOM and DEN: NOM/DEN, then we can
  * improve accuracy by shifting LSH bits, hence calculating:
  *     (NOM << LSH) / DEN
  * This however means trouble for large NOM, because (NOM << LSH) may no
@@ -198,7 +198,7 @@ extern unsigned long preset_lpj;
  * operator if the result is a long long AND at least one of the
  * operands is cast to long long (usually just prior to the "*" so as
  * not to confuse it into thinking it really has a 64-bit operand,
- * which, buy the way, it can do, but it takes more code and at least 2
+ * which, by the way, it can do, but it takes more code and at least 2
  * mpys).
 
  * We also need to be aware that one second in nanoseconds is only a
@@ -263,6 +263,7 @@ extern unsigned long preset_lpj;
  */
 extern unsigned int jiffies_to_msecs(const unsigned long j);
 extern unsigned int jiffies_to_usecs(const unsigned long j);
+extern u64 jiffies_64_to_usecs(const u64 j);
 extern unsigned long msecs_to_jiffies(const unsigned int m);
 extern unsigned long usecs_to_jiffies(const unsigned int u);
 extern unsigned long timespec_to_jiffies(const struct timespec *value);
--- a/kernel/time.c	2008-01-07 16:09:17.841059839 -0800
+++ b/kernel/time.c	2008-01-07 16:09:17.825056938 -0800
@@ -267,6 +267,12 @@ unsigned int inline jiffies_to_usecs(con
 }
 EXPORT_SYMBOL(jiffies_to_usecs);
 
+u64 inline jiffies_64_to_usecs(const u64 j)
+{
+	return (j*HZ_TO_USEC_NUM + HZ_TO_USEC_DEN-1) / HZ_TO_USEC_DEN;
+}
+EXPORT_SYMBOL(jiffies_64_to_usecs);
+
 /**
  * timespec_trunc - Truncate timespec to a granularity
  * @t: Timespec
--- a/kernel/tsacct.c	2008-01-07 16:09:17.833058389 -0800
+++ b/kernel/tsacct.c	2008-01-07 16:09:17.793051136 -0800
@@ -85,8 +85,8 @@ void xacct_add_tsk(struct taskstats *sta
 	struct mm_struct *mm;
 
 	/* convert pages-jiffies to Mbyte-usec */
-	stats->coremem = jiffies_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
-	stats->virtmem = jiffies_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
+	stats->coremem = jiffies_64_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
+	stats->virtmem = jiffies_64_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
 	mm = get_task_mm(p);
 	if (mm) {
 		/* adjust to KB unit */

Re: [PATCH] Provide u64 version of jiffies_to_usecs() in kernel/tsacct.c

[Jonathan Lim]

It's possible that the values used in and returned from jiffies_to_usecs() are
incorrect because of truncation when variables of type u64 are involved.  So a
function specific to that type is used instead.

Diff'd against: linux/kernel/git/torvalds/linux-2.6.git

Signed-off-by: Jonathan Lim <jlim@sgi.com>

--- a/include/linux/jiffies.h	Thu Feb 14 18:04:14 PST 2008
+++ b/include/linux/jiffies.h	Thu Feb 14 18:07:17 PST 2008
@@ -42,7 +42,7 @@
 /* LATCH is used in the interval timer and ftape setup. */
 #define LATCH  ((CLOCK_TICK_RATE + HZ/2) / HZ)	/* For divider */
 
-/* Suppose we want to devide two numbers NOM and DEN: NOM/DEN, then we can
+/* Suppose we want to divide two numbers NOM and DEN: NOM/DEN, then we can
  * improve accuracy by shifting LSH bits, hence calculating:
  *     (NOM << LSH) / DEN
  * This however means trouble for large NOM, because (NOM << LSH) may no
@@ -204,7 +204,7 @@ extern unsigned long preset_lpj;
  * operator if the result is a long long AND at least one of the
  * operands is cast to long long (usually just prior to the "*" so as
  * not to confuse it into thinking it really has a 64-bit operand,
- * which, buy the way, it can do, but it takes more code and at least 2
+ * which, by the way, it can do, but it takes more code and at least 2
  * mpys).
 
  * We also need to be aware that one second in nanoseconds is only a
@@ -269,6 +269,7 @@ extern unsigned long preset_lpj;
  */
 extern unsigned int jiffies_to_msecs(const unsigned long j);
 extern unsigned int jiffies_to_usecs(const unsigned long j);
+extern u64 jiffies_64_to_usecs(const u64 j);
 extern unsigned long msecs_to_jiffies(const unsigned int m);
 extern unsigned long usecs_to_jiffies(const unsigned int u);
 extern unsigned long timespec_to_jiffies(const struct timespec *value);
--- a/kernel/time.c	Thu Feb 14 18:05:12 PST 2008
+++ b/kernel/time.c	Tue Feb 19 12:44:03 PST 2008
@@ -268,6 +268,14 @@ unsigned int inline jiffies_to_usecs(con
 }
 EXPORT_SYMBOL(jiffies_to_usecs);
 
+u64 inline jiffies_64_to_usecs(const u64 j)
+{
+	u64 tmp = j*HZ_TO_USEC_NUM + HZ_TO_USEC_DEN-1;
+	do_div(tmp, HZ_TO_USEC_DEN);
+	return tmp;
+}
+EXPORT_SYMBOL(jiffies_64_to_usecs);
+
 /**
  * timespec_trunc - Truncate timespec to a granularity
  * @t: Timespec
--- a/kernel/tsacct.c	Thu Feb 14 18:06:17 PST 2008
+++ b/kernel/tsacct.c	Thu Feb 14 18:08:47 PST 2008
@@ -85,8 +85,8 @@ void xacct_add_tsk(struct taskstats *sta
 	struct mm_struct *mm;
 
 	/* convert pages-jiffies to Mbyte-usec */
-	stats->coremem = jiffies_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
-	stats->virtmem = jiffies_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
+	stats->coremem = jiffies_64_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
+	stats->virtmem = jiffies_64_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
 	mm = get_task_mm(p);
 	if (mm) {
 		/* adjust to KB unit */

Re: [PATCH] Provide u64 version of jiffies_to_usecs() in kernel/tsacct.c

[Randy Dunlap]

On Tue, 19 Feb 2008 12:52:46 -0800 (PST) Jonathan Lim wrote:

> It's possible that the values used in and returned from jiffies_to_usecs() are
> incorrect because of truncation when variables of type u64 are involved.  So a
> function specific to that type is used instead.
> 
> Diff'd against: linux/kernel/git/torvalds/linux-2.6.git
> 
> Signed-off-by: Jonathan Lim <jlim@sgi.com>
> 
> --- a/include/linux/jiffies.h	Thu Feb 14 18:04:14 PST 2008
> +++ b/include/linux/jiffies.h	Thu Feb 14 18:07:17 PST 2008
> @@ -42,7 +42,7 @@
>  /* LATCH is used in the interval timer and ftape setup. */
>  #define LATCH  ((CLOCK_TICK_RATE + HZ/2) / HZ)	/* For divider */
>  
> -/* Suppose we want to devide two numbers NOM and DEN: NOM/DEN, then we can
> +/* Suppose we want to divide two numbers NOM and DEN: NOM/DEN, then we can
>   * improve accuracy by shifting LSH bits, hence calculating:
>   *     (NOM << LSH) / DEN
>   * This however means trouble for large NOM, because (NOM << LSH) may no
> @@ -204,7 +204,7 @@ extern unsigned long preset_lpj;
>   * operator if the result is a long long AND at least one of the
>   * operands is cast to long long (usually just prior to the "*" so as
>   * not to confuse it into thinking it really has a 64-bit operand,
> - * which, buy the way, it can do, but it takes more code and at least 2
> + * which, by the way, it can do, but it takes more code and at least 2
>   * mpys).
>  
>   * We also need to be aware that one second in nanoseconds is only a
> @@ -269,6 +269,7 @@ extern unsigned long preset_lpj;
>   */
>  extern unsigned int jiffies_to_msecs(const unsigned long j);
>  extern unsigned int jiffies_to_usecs(const unsigned long j);
> +extern u64 jiffies_64_to_usecs(const u64 j);
>  extern unsigned long msecs_to_jiffies(const unsigned int m);
>  extern unsigned long usecs_to_jiffies(const unsigned int u);
>  extern unsigned long timespec_to_jiffies(const struct timespec *value);
> --- a/kernel/time.c	Thu Feb 14 18:05:12 PST 2008
> +++ b/kernel/time.c	Tue Feb 19 12:44:03 PST 2008
> @@ -268,6 +268,14 @@ unsigned int inline jiffies_to_usecs(con
>  }
>  EXPORT_SYMBOL(jiffies_to_usecs);
>  
> +u64 inline jiffies_64_to_usecs(const u64 j)
> +{
> +	u64 tmp = j*HZ_TO_USEC_NUM + HZ_TO_USEC_DEN-1;
> +	do_div(tmp, HZ_TO_USEC_DEN);

do_div() is:
 * The semantics of do_div() are:
 *
 * uint32_t do_div(uint64_t *n, uint32_t base)

Maybe you want div64_64().

> +	return tmp;
> +}
> +EXPORT_SYMBOL(jiffies_64_to_usecs);
> +
>  /**
>   * timespec_trunc - Truncate timespec to a granularity
>   * @t: Timespec
> --- a/kernel/tsacct.c	Thu Feb 14 18:06:17 PST 2008
> +++ b/kernel/tsacct.c	Thu Feb 14 18:08:47 PST 2008
> @@ -85,8 +85,8 @@ void xacct_add_tsk(struct taskstats *sta
>  	struct mm_struct *mm;
>  
>  	/* convert pages-jiffies to Mbyte-usec */
> -	stats->coremem = jiffies_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
> -	stats->virtmem = jiffies_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
> +	stats->coremem = jiffies_64_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
> +	stats->virtmem = jiffies_64_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
>  	mm = get_task_mm(p);
>  	if (mm) {
>  		/* adjust to KB unit */
> --


---
~Randy

Re: [PATCH] Provide u64 version of jiffies_to_usecs() in

[Jonathan Lim]

On Tue Feb 19 13:25:25 2008, randy.dunlap@oracle.com wrote:
> 
> > +u64 inline jiffies_64_to_usecs(const u64 j)
> > +{
> > +	u64 tmp = j*HZ_TO_USEC_NUM + HZ_TO_USEC_DEN-1;
> > +	do_div(tmp, HZ_TO_USEC_DEN);
> 
> do_div() is:
>  * The semantics of do_div() are:
>  *
>  * uint32_t do_div(uint64_t *n, uint32_t base)
> 
> Maybe you want div64_64().
> 
> > +	return tmp;
> > +}
> > +EXPORT_SYMBOL(jiffies_64_to_usecs);
> > +

In include/asm-generic/div64.h, div64_64(uint64_t dividend, uint64_t divisor)
just returns (dividend / divisor).  Isn't this the same as what I had before
in jiffies_64_to_usecs(), except that the arguments are of type u64?

Jonathan

Re: [PATCH] Provide u64 version of jiffies_to_usecs() in

[Randy Dunlap]

On Tue, 19 Feb 2008 13:59:27 -0800 (PST) Jonathan Lim wrote:

> On Tue Feb 19 13:25:25 2008, randy.dunlap@oracle.com wrote:
> > 
> > > +u64 inline jiffies_64_to_usecs(const u64 j)
> > > +{
> > > +	u64 tmp = j*HZ_TO_USEC_NUM + HZ_TO_USEC_DEN-1;
> > > +	do_div(tmp, HZ_TO_USEC_DEN);
> > 
> > do_div() is:
> >  * The semantics of do_div() are:
> >  *
> >  * uint32_t do_div(uint64_t *n, uint32_t base)
> > 
> > Maybe you want div64_64().
> > 
> > > +	return tmp;
> > > +}
> > > +EXPORT_SYMBOL(jiffies_64_to_usecs);
> > > +
> 
> In include/asm-generic/div64.h, div64_64(uint64_t dividend, uint64_t divisor)
> just returns (dividend / divisor).  Isn't this the same as what I had before
> in jiffies_64_to_usecs(), except that the arguments are of type u64?

but the (linker) problem is with X86_32, so don't look at
asm-generic/div64.h.  Look at lib/div64.c.

Or use div64() with the correct parameter types.

---
~Randy

Re: [PATCH] Provide u64 version of jiffies_to_usecs() in kernel/tsacct.c

[Jonathan Lim]

It's possible that the values used in and returned from jiffies_to_usecs() are
incorrect because of truncation when variables of type u64 are involved.  So a
function specific to that type is used instead.

This version implements a correction to jiffies_64_to_usecs() based on feedback
from Randy Dunlap.

Diff'd against: linux/kernel/git/torvalds/linux-2.6.git

Signed-off-by: Jonathan Lim <jlim@sgi.com>

--- a/include/linux/jiffies.h	Thu Feb 14 18:04:14 PST 2008
+++ b/include/linux/jiffies.h	Thu Feb 14 18:07:17 PST 2008
@@ -42,7 +42,7 @@
 /* LATCH is used in the interval timer and ftape setup. */
 #define LATCH  ((CLOCK_TICK_RATE + HZ/2) / HZ)	/* For divider */
 
-/* Suppose we want to devide two numbers NOM and DEN: NOM/DEN, then we can
+/* Suppose we want to divide two numbers NOM and DEN: NOM/DEN, then we can
  * improve accuracy by shifting LSH bits, hence calculating:
  *     (NOM << LSH) / DEN
  * This however means trouble for large NOM, because (NOM << LSH) may no
@@ -204,7 +204,7 @@ extern unsigned long preset_lpj;
  * operator if the result is a long long AND at least one of the
  * operands is cast to long long (usually just prior to the "*" so as
  * not to confuse it into thinking it really has a 64-bit operand,
- * which, buy the way, it can do, but it takes more code and at least 2
+ * which, by the way, it can do, but it takes more code and at least 2
  * mpys).
 
  * We also need to be aware that one second in nanoseconds is only a
@@ -269,6 +269,7 @@ extern unsigned long preset_lpj;
  */
 extern unsigned int jiffies_to_msecs(const unsigned long j);
 extern unsigned int jiffies_to_usecs(const unsigned long j);
+extern u64 jiffies_64_to_usecs(const u64 j);
 extern unsigned long msecs_to_jiffies(const unsigned int m);
 extern unsigned long usecs_to_jiffies(const unsigned int u);
 extern unsigned long timespec_to_jiffies(const struct timespec *value);
--- a/kernel/time.c	Thu Feb 14 18:05:12 PST 2008
+++ b/kernel/time.c	Tue Feb 19 17:00:11 PST 2008
@@ -268,6 +268,12 @@ unsigned int inline jiffies_to_usecs(con
 }
 EXPORT_SYMBOL(jiffies_to_usecs);
 
+u64 jiffies_64_to_usecs(const u64 j)
+{
+	return div64_64(j*HZ_TO_USEC_NUM + HZ_TO_USEC_DEN-1, HZ_TO_USEC_DEN);
+}
+EXPORT_SYMBOL(jiffies_64_to_usecs);
+
 /**
  * timespec_trunc - Truncate timespec to a granularity
  * @t: Timespec
--- a/kernel/tsacct.c	Thu Feb 14 18:06:17 PST 2008
+++ b/kernel/tsacct.c	Thu Feb 14 18:08:47 PST 2008
@@ -85,8 +85,8 @@ void xacct_add_tsk(struct taskstats *sta
 	struct mm_struct *mm;
 
 	/* convert pages-jiffies to Mbyte-usec */
-	stats->coremem = jiffies_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
-	stats->virtmem = jiffies_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
+	stats->coremem = jiffies_64_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
+	stats->virtmem = jiffies_64_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
 	mm = get_task_mm(p);
 	if (mm) {
 		/* adjust to KB unit */

Re: [PATCH] Provide u64 version of jiffies_to_usecs() in kernel/tsacct.c

[Randy Dunlap]

Jonathan Lim wrote:
> It's possible that the values used in and returned from jiffies_to_usecs() are
> incorrect because of truncation when variables of type u64 are involved.  So a
> function specific to that type is used instead.
> 
> This version implements a correction to jiffies_64_to_usecs() based on feedback
> from Randy Dunlap.
> 
> Diff'd against: linux/kernel/git/torvalds/linux-2.6.git
> 
> Signed-off-by: Jonathan Lim <jlim@sgi.com>
> 
> --- a/include/linux/jiffies.h	Thu Feb 14 18:04:14 PST 2008
> +++ b/include/linux/jiffies.h	Thu Feb 14 18:07:17 PST 2008
> @@ -42,7 +42,7 @@
>  /* LATCH is used in the interval timer and ftape setup. */
>  #define LATCH  ((CLOCK_TICK_RATE + HZ/2) / HZ)	/* For divider */
>  
> -/* Suppose we want to devide two numbers NOM and DEN: NOM/DEN, then we can
> +/* Suppose we want to divide two numbers NOM and DEN: NOM/DEN, then we can
>   * improve accuracy by shifting LSH bits, hence calculating:
>   *     (NOM << LSH) / DEN
>   * This however means trouble for large NOM, because (NOM << LSH) may no
> @@ -204,7 +204,7 @@ extern unsigned long preset_lpj;
>   * operator if the result is a long long AND at least one of the
>   * operands is cast to long long (usually just prior to the "*" so as
>   * not to confuse it into thinking it really has a 64-bit operand,
> - * which, buy the way, it can do, but it takes more code and at least 2
> + * which, by the way, it can do, but it takes more code and at least 2
>   * mpys).
>  
>   * We also need to be aware that one second in nanoseconds is only a
> @@ -269,6 +269,7 @@ extern unsigned long preset_lpj;
>   */
>  extern unsigned int jiffies_to_msecs(const unsigned long j);
>  extern unsigned int jiffies_to_usecs(const unsigned long j);
> +extern u64 jiffies_64_to_usecs(const u64 j);
>  extern unsigned long msecs_to_jiffies(const unsigned int m);
>  extern unsigned long usecs_to_jiffies(const unsigned int u);
>  extern unsigned long timespec_to_jiffies(const struct timespec *value);
> --- a/kernel/time.c	Thu Feb 14 18:05:12 PST 2008
> +++ b/kernel/time.c	Tue Feb 19 17:00:11 PST 2008

kernel/time.c needs:
#include <asm/div64.h>

After that, it's
Acked-by: Randy Dunlap <randy.dunlap@oracle.com>

> @@ -268,6 +268,12 @@ unsigned int inline jiffies_to_usecs(con
>  }
>  EXPORT_SYMBOL(jiffies_to_usecs);
>  
> +u64 jiffies_64_to_usecs(const u64 j)
> +{
> +	return div64_64(j*HZ_TO_USEC_NUM + HZ_TO_USEC_DEN-1, HZ_TO_USEC_DEN);
> +}
> +EXPORT_SYMBOL(jiffies_64_to_usecs);
> +
>  /**
>   * timespec_trunc - Truncate timespec to a granularity
>   * @t: Timespec
> --- a/kernel/tsacct.c	Thu Feb 14 18:06:17 PST 2008
> +++ b/kernel/tsacct.c	Thu Feb 14 18:08:47 PST 2008
> @@ -85,8 +85,8 @@ void xacct_add_tsk(struct taskstats *sta
>  	struct mm_struct *mm;
>  
>  	/* convert pages-jiffies to Mbyte-usec */
> -	stats->coremem = jiffies_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
> -	stats->virtmem = jiffies_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
> +	stats->coremem = jiffies_64_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
> +	stats->virtmem = jiffies_64_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
>  	mm = get_task_mm(p);
>  	if (mm) {
>  		/* adjust to KB unit */


-- 
~Randy

Re: [PATCH] Provide u64 version of jiffies_to_usecs() in kernel/tsacct.c

[Jonathan Lim]

It's possible that the values used in and returned from jiffies_to_usecs() are
incorrect because of truncation when variables of type u64 are involved.  So a
function specific to that type is used instead.

This version implements a correction to jiffies_64_to_usecs() based on feedback
from Randy Dunlap.

Diff'd against: linux/kernel/git/torvalds/linux-2.6.git

Signed-off-by: Jonathan Lim <jlim@sgi.com>

--- a/include/linux/jiffies.h	Thu Feb 14 18:04:14 PST 2008
+++ b/include/linux/jiffies.h	Thu Feb 14 18:07:17 PST 2008
@@ -42,7 +42,7 @@
 /* LATCH is used in the interval timer and ftape setup. */
 #define LATCH  ((CLOCK_TICK_RATE + HZ/2) / HZ)	/* For divider */
 
-/* Suppose we want to devide two numbers NOM and DEN: NOM/DEN, then we can
+/* Suppose we want to divide two numbers NOM and DEN: NOM/DEN, then we can
  * improve accuracy by shifting LSH bits, hence calculating:
  *     (NOM << LSH) / DEN
  * This however means trouble for large NOM, because (NOM << LSH) may no
@@ -204,7 +204,7 @@ extern unsigned long preset_lpj;
  * operator if the result is a long long AND at least one of the
  * operands is cast to long long (usually just prior to the "*" so as
  * not to confuse it into thinking it really has a 64-bit operand,
- * which, buy the way, it can do, but it takes more code and at least 2
+ * which, by the way, it can do, but it takes more code and at least 2
  * mpys).
 
  * We also need to be aware that one second in nanoseconds is only a
@@ -269,6 +269,7 @@ extern unsigned long preset_lpj;
  */
 extern unsigned int jiffies_to_msecs(const unsigned long j);
 extern unsigned int jiffies_to_usecs(const unsigned long j);
+extern u64 jiffies_64_to_usecs(const u64 j);
 extern unsigned long msecs_to_jiffies(const unsigned int m);
 extern unsigned long usecs_to_jiffies(const unsigned int u);
 extern unsigned long timespec_to_jiffies(const struct timespec *value);
--- a/kernel/time.c	Thu Feb 14 18:05:12 PST 2008
+++ b/kernel/time.c	Mon Feb 25 14:22:27 PST 2008
@@ -38,6 +38,7 @@
 
 #include <asm/uaccess.h>
 #include <asm/unistd.h>
+#include <asm/div64.h>
 
 #include "timeconst.h"
 
@@ -267,6 +268,12 @@ unsigned int inline jiffies_to_usecs(con
 #endif
 }
 EXPORT_SYMBOL(jiffies_to_usecs);
+
+u64 jiffies_64_to_usecs(const u64 j)
+{
+	return div64_64(j*HZ_TO_USEC_NUM + HZ_TO_USEC_DEN-1, HZ_TO_USEC_DEN);
+}
+EXPORT_SYMBOL(jiffies_64_to_usecs);
 
 /**
  * timespec_trunc - Truncate timespec to a granularity
--- a/kernel/tsacct.c	Thu Feb 14 18:06:17 PST 2008
+++ b/kernel/tsacct.c	Thu Feb 14 18:08:47 PST 2008
@@ -85,8 +85,8 @@ void xacct_add_tsk(struct taskstats *sta
 	struct mm_struct *mm;
 
 	/* convert pages-jiffies to Mbyte-usec */
-	stats->coremem = jiffies_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
-	stats->virtmem = jiffies_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
+	stats->coremem = jiffies_64_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
+	stats->virtmem = jiffies_64_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
 	mm = get_task_mm(p);
 	if (mm) {
 		/* adjust to KB unit */

Re: [PATCH] Provide u64 version of jiffies_to_usecs() in kernel/tsacct.c

[Roman Zippel]

Hi,

On Monday 25. February 2008, Jonathan Lim wrote:

> +u64 jiffies_64_to_usecs(const u64 j)
> +{
> +	return div64_64(j*HZ_TO_USEC_NUM + HZ_TO_USEC_DEN-1, HZ_TO_USEC_DEN);
> +}
> +EXPORT_SYMBOL(jiffies_64_to_usecs);

Please also adapt the "BITS_PER_LONG == 32" part from jiffies_to_usecs(), it 
should provide enough resolution to avoid the div64_64() here.

bye, Roman

Re: [PATCH] Provide u64 version of jiffies_to_usecs() in kernel/tsacct.c

[Jonathan Lim]

It's possible that the values used in and returned from jiffies_to_usecs() are
incorrect because of truncation when variables of type u64 are involved.  So a
function specific to that type is used instead.

This version implements a correction to jiffies_64_to_usecs() based on feedback
from Randy Dunlap and Roman Zippel.

Diff'd against: linux/kernel/git/torvalds/linux-2.6.git

Signed-off-by: Jonathan Lim <jlim@sgi.com>

--- a/include/linux/jiffies.h	Thu Feb 14 18:04:14 PST 2008
+++ b/include/linux/jiffies.h	Thu Feb 14 18:07:17 PST 2008
@@ -42,7 +42,7 @@
 /* LATCH is used in the interval timer and ftape setup. */
 #define LATCH  ((CLOCK_TICK_RATE + HZ/2) / HZ)	/* For divider */
 
-/* Suppose we want to devide two numbers NOM and DEN: NOM/DEN, then we can
+/* Suppose we want to divide two numbers NOM and DEN: NOM/DEN, then we can
  * improve accuracy by shifting LSH bits, hence calculating:
  *     (NOM << LSH) / DEN
  * This however means trouble for large NOM, because (NOM << LSH) may no
@@ -204,7 +204,7 @@ extern unsigned long preset_lpj;
  * operator if the result is a long long AND at least one of the
  * operands is cast to long long (usually just prior to the "*" so as
  * not to confuse it into thinking it really has a 64-bit operand,
- * which, buy the way, it can do, but it takes more code and at least 2
+ * which, by the way, it can do, but it takes more code and at least 2
  * mpys).
 
  * We also need to be aware that one second in nanoseconds is only a
@@ -269,6 +269,7 @@ extern unsigned long preset_lpj;
  */
 extern unsigned int jiffies_to_msecs(const unsigned long j);
 extern unsigned int jiffies_to_usecs(const unsigned long j);
+extern u64 jiffies_64_to_usecs(const u64 j);
 extern unsigned long msecs_to_jiffies(const unsigned int m);
 extern unsigned long usecs_to_jiffies(const unsigned int u);
 extern unsigned long timespec_to_jiffies(const struct timespec *value);
--- a/kernel/time.c	Thu Feb 14 18:05:12 PST 2008
+++ b/kernel/time.c	Fri Apr 18 14:44:54 PDT 2008
@@ -38,6 +38,7 @@
 
 #include <asm/uaccess.h>
 #include <asm/unistd.h>
+#include <asm/div64.h>
 
 #include "timeconst.h"
 
@@ -267,6 +268,16 @@ unsigned int inline jiffies_to_usecs(con
 #endif
 }
 EXPORT_SYMBOL(jiffies_to_usecs);
+
+u64 jiffies_64_to_usecs(const u64 j)
+{
+# if BITS_PER_LONG == 32
+	return ((u64)HZ_TO_USEC_MUL32 * j) >> HZ_TO_USEC_SHR32;
+# else
+	return div64_64(j*HZ_TO_USEC_NUM + HZ_TO_USEC_DEN-1, HZ_TO_USEC_DEN);
+# endif
+}
+EXPORT_SYMBOL(jiffies_64_to_usecs);
 
 /**
  * timespec_trunc - Truncate timespec to a granularity
--- a/kernel/tsacct.c	Thu Feb 14 18:06:17 PST 2008
+++ b/kernel/tsacct.c	Thu Feb 14 18:08:47 PST 2008
@@ -85,8 +85,8 @@ void xacct_add_tsk(struct taskstats *sta
 	struct mm_struct *mm;
 
 	/* convert pages-jiffies to Mbyte-usec */
-	stats->coremem = jiffies_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
-	stats->virtmem = jiffies_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
+	stats->coremem = jiffies_64_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
+	stats->virtmem = jiffies_64_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
 	mm = get_task_mm(p);
 	if (mm) {
 		/* adjust to KB unit */

[PATCH] Account for user time when updating memory integrals

[Jonathan Lim]

This patch adapts acct_update_integrals() to include user time when calculating
the time difference.  The units of acct_rss_mem1 and acct_vm_mem1 are also
changed from pages-jiffies to pages-usecs to avoid calling jiffies_to_usecs()
in xacct_add_tsk() which might overflow.

This patch obsoletes:

  provide-u64-version-of-jiffies_to_usecs-in-kernel-tsacctc.patch

Signed-off-by: Jonathan Lim <jlim@sgi.com>


--- a/include/linux/sched.h	2008-04-07 15:54:08.000000000 -0700
+++ b/include/linux/sched.h	2008-04-18 15:17:52.996000000 -0700
@@ -940,7 +940,7 @@ struct task_struct {
 #if defined(CONFIG_TASK_XACCT)
 	u64 acct_rss_mem1;	/* accumulated rss usage */
 	u64 acct_vm_mem1;	/* accumulated virtual memory usage */
-	cputime_t acct_stimexpd;/* stime since last update */
+	cputime_t acct_timexpd;	/* stime + utime since last update */
 #endif
 #ifdef CONFIG_NUMA
   	struct mempolicy *mempolicy;
--- a/kernel/tsacct.c	2008-04-07 15:53:52.000000000 -0700
+++ b/kernel/tsacct.c	2008-04-29 16:22:35.888000000 -0700
@@ -81,9 +81,9 @@ void bacct_add_tsk(struct taskstats *sta
  */
 void xacct_add_tsk(struct taskstats *stats, struct task_struct *p)
 {
-	/* convert pages-jiffies to Mbyte-usec */
-	stats->coremem = jiffies_to_usecs(p->acct_rss_mem1) * PAGE_SIZE / MB;
-	stats->virtmem = jiffies_to_usecs(p->acct_vm_mem1) * PAGE_SIZE / MB;
+	/* convert pages-usec to Mbyte-usec */
+	stats->coremem = p->acct_rss_mem1 * PAGE_SIZE / MB;
+	stats->virtmem = p->acct_vm_mem1 * PAGE_SIZE / MB;
 	if (p->mm) {
 		/* adjust to KB unit */
 		stats->hiwater_rss   = p->mm->hiwater_rss * PAGE_SIZE / KB;
@@ -104,12 +104,19 @@ void xacct_add_tsk(struct taskstats *sta
 void acct_update_integrals(struct task_struct *tsk)
 {
 	if (likely(tsk->mm)) {
-		long delta = cputime_to_jiffies(
-			cputime_sub(tsk->stime, tsk->acct_stimexpd));
+		cputime_t time, dtime;
+		struct timeval value;
+		u64 delta;
+
+		time = tsk->stime + tsk->utime;
+		dtime = cputime_sub(time, tsk->acct_timexpd);
+		jiffies_to_timeval(cputime_to_jiffies(dtime), &value);
+		delta = value.tv_sec;
+		delta = delta * USEC_PER_SEC + value.tv_usec;
 
 		if (delta == 0)
 			return;
-		tsk->acct_stimexpd = tsk->stime;
+		tsk->acct_timexpd = time;
 		tsk->acct_rss_mem1 += delta * get_mm_rss(tsk->mm);
 		tsk->acct_vm_mem1 += delta * tsk->mm->total_vm;
 	}
@@ -121,7 +128,7 @@ void acct_update_integrals(struct task_s
  */
 void acct_clear_integrals(struct task_struct *tsk)
 {
-	tsk->acct_stimexpd = 0;
+	tsk->acct_timexpd = 0;
 	tsk->acct_rss_mem1 = 0;
 	tsk->acct_vm_mem1 = 0;
 }
--- a/kernel/sched.c	2008-04-07 15:54:08.000000000 -0700
+++ b/kernel/sched.c	2008-04-18 15:17:27.992000000 -0700
@@ -2531,6 +2531,8 @@ void account_user_time(struct task_struc
 		cpustat->nice = cputime64_add(cpustat->nice, tmp);
 	else
 		cpustat->user = cputime64_add(cpustat->user, tmp);
+	/* Account for user time used */
+	acct_update_integrals(p);
 }
 
 /*

Re: X won't start with VisEG and 2.6.22.19

[John David Anglin]

> > Tracing with gdb gives the following:
> > 
> > Breakpoint 1, fbdev_modes_equal (set=0xfb5ed568, req=0xfb5ed4c8)
> >      at ../../../../hw/xfree86/fbdevhw/fbdevhw.c:256
> > 256     }
> > (gdb) p *set
> > $1 = {xres = 1024, yres = 768, xres_virtual = 1024, yres_virtual = 768, 
> > xoffset = 0, yoffset = 0,
> >    bits_per_pixel = 8, grayscale = 0, red = {offset = 0, length = 8, 
> > msb_right = 0}, green = {offset = 0,
> >      length = 8, msb_right = 0}, blue = {offset = 0, length = 8, 
> > msb_right = 0}, transp = {offset = 0,
> >      length = 0, msb_right = 0}, nonstd = 0, activate = 0, height = 0, 
> > width = 0, accel_flags = 0,
> >    pixclock = 0, left_margin = 0, right_margin = 0, upper_margin = 0, 
> > lower_margin = 0, hsync_len = 0,
> >    vsync_len = 0, sync = 0, vmode = 0, reserved = {0, 0, 0, 0, 0, 0}}
> > 
> > (gdb) p *req
> > $2 = {xres = 1024, yres = 768, xres_virtual = 1024, yres_virtual = 768, 
> > xoffset = 0, yoffset = 0,
> >    bits_per_pixel = 8, grayscale = 0, red = {offset = 0, length = 8, 
> > msb_right = 0}, green = {offset = 0,
> >      length = 8, msb_right = 0}, blue = {offset = 0, length = 8, 
> > msb_right = 0}, transp = {offset = 0,
> >      length = 0, msb_right = 0}, nonstd = 0, activate = 0, height = 0, 
> > width = 0, accel_flags = 0,
> >    pixclock = 22271, left_margin = 56, right_margin = 8, upper_margin = 
> > 41, lower_margin = 0,
> >    hsync_len = 176, vsync_len = 8, sync = 3, vmode = 1, reserved = {0, 
> > 0, 0, 0, 0, 0}}
> > 
> > (gdb) bt
> > #0  fbdev_modes_equal (set=0xfb5ed568, req=0xfb5ed4c8) at 
> > ../../../../hw/xfree86/fbdevhw/fbdevhw.c:256
> > #1  0x40bcfb64 in fbdevHWSetMode (pScrn=0x2142a0, mode=<value optimized 
> > out>, check=1)
> >      at ../../../../hw/xfree86/fbdevhw/fbdevhw.c:528
> > #2  0x40bd07e8 in fbdevHWSetVideoModes (pScrn=0x2142a0) at 
> > ../../../../hw/xfree86/fbdevhw/fbdevhw.c:568
> > #3  0x40cf5bec in ?? () from /usr/lib/xorg/modules/drivers//fbdev_drv.so
> > #4  0x00074e88 in InitOutput ()
> > #5  0x00039d04 in main ()
> > 
> > As you can see, the main video parameters are OK, while the 
> > monitor/modeline values (pixclock, left_margin, right_margin, 
> > upper_margin, lower_margin, hsync_len, vsync_len, sync and vmode)
> > were changed to zero.

It would appear that none of these values are relevant to the stifb.
The stifb.c code uses kzalloc to allocate the struct stifb_info, so
I think the above values should be zero.  Some code is clearly computing
values for pixclock, etc.  I don't see that this code is in the kernel.
It may be X that's making up these values.

I see in the log:

(II) FBDEV(0): hardware: stifb (video memory: 2048kB)
(II) FBDEV(0): checking modes against framebuffer device...
(II) FBDEV(0):  mode "1280x1024" test failed
(II) FBDEV(0): checking modes against monitor...
(--) FBDEV(0): Virtual size is 1280x1024 (pitch 1280)
(**) FBDEV(0):  Built-in mode "current": 28000.0 MHz, 21875.0 kHz, 21362.3 Hz
(II) FBDEV(0): Modeline "current"x0.0  28000.00  1280 1280 1280 1280  1024 1024 
1024 1024 -hsync -vsync -csync (21875.0 kHz)
(==) FBDEV(0): DPI set to (96, 96)

Could X be requesting a mode with invalid mode parameters?

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: X won't start with VisEG and 2.6.22.19

[Helge Deller]

John David Anglin wrote:
>>> Tracing with gdb gives the following:
>>>
>>> Breakpoint 1, fbdev_modes_equal (set=0xfb5ed568, req=0xfb5ed4c8)
>>>      at ../../../../hw/xfree86/fbdevhw/fbdevhw.c:256
>>> 256     }
>>> (gdb) p *set
>>> $1 = {xres = 1024, yres = 768, xres_virtual = 1024, yres_virtual = 768, 
>>> xoffset = 0, yoffset = 0,
>>>    bits_per_pixel = 8, grayscale = 0, red = {offset = 0, length = 8, 
>>> msb_right = 0}, green = {offset = 0,
>>>      length = 8, msb_right = 0}, blue = {offset = 0, length = 8, 
>>> msb_right = 0}, transp = {offset = 0,
>>>      length = 0, msb_right = 0}, nonstd = 0, activate = 0, height = 0, 
>>> width = 0, accel_flags = 0,
>>>    pixclock = 0, left_margin = 0, right_margin = 0, upper_margin = 0, 
>>> lower_margin = 0, hsync_len = 0,
>>>    vsync_len = 0, sync = 0, vmode = 0, reserved = {0, 0, 0, 0, 0, 0}}
>>>
>>> (gdb) p *req
>>> $2 = {xres = 1024, yres = 768, xres_virtual = 1024, yres_virtual = 768, 
>>> xoffset = 0, yoffset = 0,
>>>    bits_per_pixel = 8, grayscale = 0, red = {offset = 0, length = 8, 
>>> msb_right = 0}, green = {offset = 0,
>>>      length = 8, msb_right = 0}, blue = {offset = 0, length = 8, 
>>> msb_right = 0}, transp = {offset = 0,
>>>      length = 0, msb_right = 0}, nonstd = 0, activate = 0, height = 0, 
>>> width = 0, accel_flags = 0,
>>>    pixclock = 22271, left_margin = 56, right_margin = 8, upper_margin = 
>>> 41, lower_margin = 0,
>>>    hsync_len = 176, vsync_len = 8, sync = 3, vmode = 1, reserved = {0, 
>>> 0, 0, 0, 0, 0}}
>>>
>>> (gdb) bt
>>> #0  fbdev_modes_equal (set=0xfb5ed568, req=0xfb5ed4c8) at 
>>> ../../../../hw/xfree86/fbdevhw/fbdevhw.c:256
>>> #1  0x40bcfb64 in fbdevHWSetMode (pScrn=0x2142a0, mode=<value optimized 
>>> out>, check=1)
>>>      at ../../../../hw/xfree86/fbdevhw/fbdevhw.c:528
>>> #2  0x40bd07e8 in fbdevHWSetVideoModes (pScrn=0x2142a0) at 
>>> ../../../../hw/xfree86/fbdevhw/fbdevhw.c:568
>>> #3  0x40cf5bec in ?? () from /usr/lib/xorg/modules/drivers//fbdev_drv.so
>>> #4  0x00074e88 in InitOutput ()
>>> #5  0x00039d04 in main ()
>>>
>>> As you can see, the main video parameters are OK, while the 
>>> monitor/modeline values (pixclock, left_margin, right_margin, 
>>> upper_margin, lower_margin, hsync_len, vsync_len, sync and vmode)
>>> were changed to zero.
> 
> It would appear that none of these values are relevant to the stifb.
> The stifb.c code uses kzalloc to allocate the struct stifb_info, so
> I think the above values should be zero.  Some code is clearly computing
> values for pixclock, etc.  I don't see that this code is in the kernel.
> It may be X that's making up these values.
> 
> I see in the log:
> 
> (II) FBDEV(0): hardware: stifb (video memory: 2048kB)
> (II) FBDEV(0): checking modes against framebuffer device...
> (II) FBDEV(0):  mode "1280x1024" test failed
> (II) FBDEV(0): checking modes against monitor...
> (--) FBDEV(0): Virtual size is 1280x1024 (pitch 1280)
> (**) FBDEV(0):  Built-in mode "current": 28000.0 MHz, 21875.0 kHz, 21362.3 Hz
> (II) FBDEV(0): Modeline "current"x0.0  28000.00  1280 1280 1280 1280  1024 1024 
> 1024 1024 -hsync -vsync -csync (21875.0 kHz)
> (==) FBDEV(0): DPI set to (96, 96)
> 
> Could X be requesting a mode with invalid mode parameters?

In general X behaves correctly if you have a fb device, which is able to 
set sync timings.
But stifb and a few other fb drivers are not able to set sync timings, 
and for those X behaves IMHO wrong. For those X should assume that the 
timings set by the driver are correct as long as the resolution and bit 
depth are correct.

This is what happens:
1. In xorg.conf you configured a pixel resolution and bpp (e.g. 
1280x1024-16)
2. In xorg.conf you might have configured a monitor (with or without 
some timing/sync informations)
3. X reads those config parameters and try to set them through kernel calls.
4. Kernel returns that it now has set up the mode (1280x1024-16) and has 
set up the timings. Since stifb does not support timing modes, it 
returns zeros in those values (IMHO thats correct behavior if you look 
at the comments in skeletonfb.c)
5. X verifies the returned values and aborts since the timing values are 
different to what it fed to the kernel, although the resolution 
(1280x1024-16) is correctly set and returned like that.

I think X is right in noticing that the timing/sync values are not what 
it expected. Nevertheless, I would prefer that it would just warn (in 
the fbdev drivers only!) that the timings are incorrect instead of 
aborting completely. X is right to abort, if the pixel resolution and 
bpp can not be set.
So, X's tests should be like that:
a) resolution and bpp different -> abort
b) resolution and bpp correct, but sync timings/monitor timings 
different -> warn but continue

Right now X's tests are:
resolution, bpp or sync/monitor timings different -> abort.

Of course it's easily possible to change stifb in that it just takes any 
timing values, but that is not how it is documented in skeletonfb.

Helge

Re: X won't start with VisEG and 2.6.22.19

[John David Anglin]

> >>> As you can see, the main video parameters are OK, while the 
> >>> monitor/modeline values (pixclock, left_margin, right_margin, 
> >>> upper_margin, lower_margin, hsync_len, vsync_len, sync and vmode)
> >>> were changed to zero.

The skeletonfb.c code makes it very clear that the only place where
a mode can be changed is in xxxfb_check_var:

*      Exception to the above rule:  Some drivers have a fixed mode, ie,
*      the hardware is already set at boot up, and cannot be changed.  In
*      this case, it is more acceptable that this function just return
*      a copy of the currently working var (info->var). Better is to not
*      implement this function, as the upper layer will do the copying
*      of the current var for you.
*
*      Note:  This is the only function where the contents of var can be
*      freely adjusted after the driver has been registered. If you find
*      that you have code outside of this function that alters the content
*      of var, then you are doing something wrong.  Note also that the
*      contents of info->var must be left untouched at all times after
*      driver registration.

As xxxfb_check_var is not implemented in stifb.c, it must be the upper
level that is changing the parameters.  Possibly, something to try is
the following:

 *      However, even if your hardware does not support mode changing,
 *      a set_par might be needed to at least initialize the hardware to
 *      a known working state, especially if it came back from another
 *      process that also modifies the same hardware, such as X.
 *
 *      If this is the case, a combination such as the following should work:
 *
 *      static int xxxfb_check_var(struct fb_var_screeninfo *var,
 *                                struct fb_info *info)
 *      {
 *              *var = info->var;
 *              return 0;
 *      }
 *
 *      static int xxxfb_set_par(struct fb_info *info)
 *      {
 *              init your hardware here
 *      }

> But stifb and a few other fb drivers are not able to set sync timings, 
> and for those X behaves IMHO wrong. For those X should assume that the 
> timings set by the driver are correct as long as the resolution and bit 
> depth are correct.
> 
> This is what happens:
> 1. In xorg.conf you configured a pixel resolution and bpp (e.g. 
> 1280x1024-16)
> 2. In xorg.conf you might have configured a monitor (with or without 
> some timing/sync informations)
> 3. X reads those config parameters and try to set them through kernel calls.
> 4. Kernel returns that it now has set up the mode (1280x1024-16) and has 
> set up the timings. Since stifb does not support timing modes, it 
> returns zeros in those values (IMHO thats correct behavior if you look 
> at the comments in skeletonfb.c)
> 5. X verifies the returned values and aborts since the timing values are 
> different to what it fed to the kernel, although the resolution 
> (1280x1024-16) is correctly set and returned like that.
> 
> I think X is right in noticing that the timing/sync values are not what 
> it expected. Nevertheless, I would prefer that it would just warn (in 
> the fbdev drivers only!) that the timings are incorrect instead of 
> aborting completely. X is right to abort, if the pixel resolution and 
> bpp can not be set.
> So, X's tests should be like that:
> a) resolution and bpp different -> abort
> b) resolution and bpp correct, but sync timings/monitor timings 
> different -> warn but continue
> 
> Right now X's tests are:
> resolution, bpp or sync/monitor timings different -> abort.
> 
> Of course it's easily possible to change stifb in that it just takes any 
> timing values, but that is not how it is documented in skeletonfb.

My sense in looking at the PR is that the freedesktop people won't
accept this approach.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: X won't start with VisEG and 2.6.22.19

[Helge Deller]

John David Anglin wrote:
>>>>> As you can see, the main video parameters are OK, while the 
>>>>> monitor/modeline values (pixclock, left_margin, right_margin, 
>>>>> upper_margin, lower_margin, hsync_len, vsync_len, sync and vmode)
>>>>> were changed to zero.
> 
> The skeletonfb.c code makes it very clear that the only place where
> a mode can be changed is in xxxfb_check_var:
> 
> *      Exception to the above rule:  Some drivers have a fixed mode, ie,
> *      the hardware is already set at boot up, and cannot be changed.  In
> *      this case, it is more acceptable that this function just return
> *      a copy of the currently working var (info->var). Better is to not
> *      implement this function, as the upper layer will do the copying
> *      of the current var for you.
> *
> *      Note:  This is the only function where the contents of var can be
> *      freely adjusted after the driver has been registered. If you find
> *      that you have code outside of this function that alters the content
> *      of var, then you are doing something wrong.  Note also that the
> *      contents of info->var must be left untouched at all times after
> *      driver registration.
> 
> As xxxfb_check_var is not implemented in stifb.c, it must be the upper
> level that is changing the parameters.  Possibly, something to try is
> the following:
> 
>  *      However, even if your hardware does not support mode changing,
>  *      a set_par might be needed to at least initialize the hardware to
>  *      a known working state, especially if it came back from another
>  *      process that also modifies the same hardware, such as X.
>  *
>  *      If this is the case, a combination such as the following should work:
>  *
>  *      static int xxxfb_check_var(struct fb_var_screeninfo *var,
>  *                                struct fb_info *info)
>  *      {
>  *              *var = info->var;
>  *              return 0;
>  *      }
>  *
>  *      static int xxxfb_set_par(struct fb_info *info)
>  *      {
>  *              init your hardware here
>  *      }


Yes, exactly that would be the patch to make it work.


>> But stifb and a few other fb drivers are not able to set sync timings, 
>> and for those X behaves IMHO wrong. For those X should assume that the 
>> timings set by the driver are correct as long as the resolution and bit 
>> depth are correct.
>>
>> This is what happens:
>> 1. In xorg.conf you configured a pixel resolution and bpp (e.g. 
>> 1280x1024-16)
>> 2. In xorg.conf you might have configured a monitor (with or without 
>> some timing/sync informations)
>> 3. X reads those config parameters and try to set them through kernel calls.
>> 4. Kernel returns that it now has set up the mode (1280x1024-16) and has 
>> set up the timings. Since stifb does not support timing modes, it 
>> returns zeros in those values (IMHO thats correct behavior if you look 
>> at the comments in skeletonfb.c)
>> 5. X verifies the returned values and aborts since the timing values are 
>> different to what it fed to the kernel, although the resolution 
>> (1280x1024-16) is correctly set and returned like that.
>>
>> I think X is right in noticing that the timing/sync values are not what 
>> it expected. Nevertheless, I would prefer that it would just warn (in 
>> the fbdev drivers only!) that the timings are incorrect instead of 
>> aborting completely. X is right to abort, if the pixel resolution and 
>> bpp can not be set.
>> So, X's tests should be like that:
>> a) resolution and bpp different -> abort
>> b) resolution and bpp correct, but sync timings/monitor timings 
>> different -> warn but continue
>>
>> Right now X's tests are:
>> resolution, bpp or sync/monitor timings different -> abort.
>>
>> Of course it's easily possible to change stifb in that it just takes any 
>> timing values, but that is not how it is documented in skeletonfb.
> 
> My sense in looking at the PR is that the freedesktop people won't
> accept this approach.

Sadly I have the same feeling.
But just scan yourself through the kernel fb drivers and look how many 
drivers haven't implemented this workaround with check_var and set_par.
All those fb drivers are probably broken due to this X behavior.

Helge

Re: Kernel Panic during init with 2.6.29-gb609308 (fresh clone of git

[John David Anglin]

> You might try building with the 4.1 branch.  I have had more luck
> with it.

I tried building 2.6.22.19 with debian 4.3.  Boot fails here:

md: raid0 personality registered for level 0
md: raid1 personality registered for level 1
TCP cubic registered
NET: Registered protocol family 1
NET: Registered protocol family 17
Freeing unused kernel memory: <0>------------[ cut here ]------------
Badness at 000000004011e44c [verbose debug info unavailable]
Backtrace:
Backtrace:
 [<00000000401128c0>] dump_stack+0x18/0x28

Have to hit 'RS' twice.  The first time there is a hpmc in the memory test:

#  Location|Alert| Encoded Field    |  Data Field    |   Keyword / Timestamp
-------------------------------------------------------------------------------
52951  SFW  0   0  0x030010D500E00000 0000000000000000 CPU_STOP
52950  SFW  0   2  0x438010EF00E01B50 00000000FFFFFFFF MEM_UNEXPECTED_HPMC
                                                      03 May 2009 00:35:26
52949  SFW  0   0  0x160000C400E00000 0000000280000000 MEM_INIT
52948  SFW  0   0  0x370000FB00E00000 000000F000004002 MEM_WARN_INIT_ONLY

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: Reproducible random python crash.

[John David Anglin]

> I booted 2.6.30.5 from gsyprf11 because I had one random segv and
> one command hang while running the GCC testsuite.  I suspect that some
> hangs are due to dropped signals.  I'm tempted to try the sequence
> with 2.6.22.19.

SMP 2.6.22.19 also fails with random segvs and hangs.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

> I have noticed that the notifierMutex mutex is initialized to all
> zeros (i.e., it's declared static).  Is this ok with our nptl implementation?

Forget this, it's done in Tcl_MutexLock.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Carlos O'Donell]

On Wed, Dec 23, 2009 at 5:18 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
>> I have noticed that the notifierMutex mutex is initialized to all
>> zeros (i.e., it's declared static). =A0Is this ok with our nptl impl=
ementation?
>
> Forget this, it's done in Tcl_MutexLock.

In NPTL the hppa locks are all zero. We no longer use
load-and-clear-word for anything in userspace.

Even the dynamic initialization will zero the lock.

We are just like every other architecture... with the exception that
we currently have more bugs :-)

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[John David Anglin]

[-- Attachment #1: Type: text/plain, Size: 857 bytes --]

On Wed, 23 Dec 2009, Carlos O'Donell wrote:

> We are just like every other architecture... with the exception that
> we currently have more bugs :-)

I have been looking at the lws implementation.  I've attached a patch
with a couple of tweaks:

1) I changed the unconditional branch at the entry point to a gate.
Didn't think it was a good idea to execute code at user priveledge.
This seems to have improved things (got through a complete testsuite
run on gsyprf11 without a deadlock).  However, I hit a slightly
different lockup on c3750.

2) Change return space register to sr7.  sr3 is not set correctly
if the entry number is invalid.

Have nasty suspicion that sr3 is getting hit...

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

[-- Attachment #2: syscall.S.d.1 --]
[-- Type: text/plain, Size: 1206 bytes --]

--- a/arch/parisc/kernel/syscall.S
+++ b/arch/parisc/kernel/syscall.S
@@ -47,18 +47,17 @@ ENTRY(linux_gateway_page)
 	KILL_INSN
 	.endr
 
-	/* ADDRESS 0xb0 to 0xb4, lws uses 1 insns for entry */
+	/* ADDRESS 0xb0 to 0xb8, lws uses two insns for entry */
 	/* Light-weight-syscall entry must always be located at 0xb0 */
 	/* WARNING: Keep this number updated with table size changes */
 #define __NR_lws_entries (2)
 
 lws_entry:
-	/* Unconditional branch to lws_start, located on the 
-	   same gateway page */
-	b,n	lws_start
+	gate	lws_start, %r0		/* increase privilege */
+	depi	3, 31, 2, %r31		/* Ensure we return into user mode. */
 
-	/* Fill from 0xb4 to 0xe0 */
-	.rept 11
+	/* Fill from 0xb8 to 0xe0 */
+	.rept 10
 	KILL_INSN
 	.endr
 
@@ -423,9 +422,6 @@ tracesys_sigexit:
 
 	*********************************************************/
 lws_start:
-	/* Gate and ensure we return to userspace */
-	gate	.+8, %r0
-	depi	3, 31, 2, %r31	/* Ensure we return to userspace */
 
 #ifdef CONFIG_64BIT
 	/* FIXME: If we are a 64-bit kernel just
@@ -473,7 +469,7 @@ lws_exit:
 	/* now reset the lowest bit of sp if it was set */
 	xor	%r30,%r1,%r30
 #endif
-	be,n	0(%sr3, %r31)
+	be,n	0(%sr7, %r31)
 
 
 	

Re: futex wait failure

[Helge Deller]

On 12/28/2009 07:59 PM, John David Anglin wrote:
> On Wed, 23 Dec 2009, Carlos O'Donell wrote:
>
>> We are just like every other architecture... with the exception that
>> we currently have more bugs :-)
>
> I have been looking at the lws implementation.  I've attached a patch
> with a couple of tweaks:

The patch looks good.

> 1) I changed the unconditional branch at the entry point to a gate.
> Didn't think it was a good idea to execute code at user priveledge.

Can you explain that?
The branch might get interrupted, before the process get privileged?

> This seems to have improved things (got through a complete testsuite
> run on gsyprf11 without a deadlock).

Good. I'll test too.

>However, I hit a slightly different lockup on c3750.

Any info?

> 2) Change return space register to sr7.  sr3 is not set correctly
> if the entry number is invalid.

Yep.

> Have nasty suspicion that sr3 is getting hit...

Helge

Re: futex wait failure

[John David Anglin]

> The patch looks good.
> 
> > 1) I changed the unconditional branch at the entry point to a gate.
> > Didn't think it was a good idea to execute code at user priveledge.
> 
> Can you explain that?

Not really, except that we gate immediately in the other code paths.
The gateway page is special in that processes are not supposed to get
scheduled off the page.

> The branch might get interrupted, before the process get privileged?

That was the concern.  At worst, an unnecessary branch is eliminated.
It's clear that this doesn't fix the futex deadlock, but the timing of
the lws lock code has changed resulting in slightly different symptoms.

> > This seems to have improved things (got through a complete testsuite
> > run on gsyprf11 without a deadlock).
> 
> Good. I'll test too.
> 
> >However, I hit a slightly different lockup on c3750.
> 
> Any info?

No, I restarted a new GCC build on this machine.  As with the previous
case on gsyprf11, thread 2 in expect seemed confused about its holding
the lock.  I've got a new hang on gsyprf11 to look at.

In my build last night on the c3750, expect dropped core in the GCC
testsuite.  Otherwise, it completed the GCC build and check.  I'll
look a bit at the core dump today.

On my rp3440 (UP kernel), I had several segmentation faults in /bin/sh
building GCC with 2.6.32.2.  For some reason, 2.6.31.9 is better.  I
haven't tried expect 5.44.1.14-5 on it.  It currently using expect-tcl8.3
which avoids the pthread issue.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Randolph Chung]

> Not really, except that we gate immediately in the other code paths.
> The gateway page is special in that processes are not supposed to get
> scheduled off the page.
> 
>> The branch might get interrupted, before the process get privileged?
> 
> That was the concern.  At worst, an unnecessary branch is eliminated.
> It's clear that this doesn't fix the futex deadlock, but the timing of
> the lws lock code has changed resulting in slightly different symptoms.

IIRC we had to move the gate in the regular syscall path earlier as well 
to fix some security problems before. I don't recall the exact details, 
but I seem to recall it had to do with using ptrace(SINGLESTEP).

randolph

Re: futex wait failure

[Carlos O'Donell]

On Tue, Dec 29, 2009 at 8:47 AM, Helge Deller <deller@gmx.de> wrote:
>> 2) Change return space register to sr7. =A0sr3 is not set correctly
>> if the entry number is invalid.
>
> Yep.
>
>> Have nasty suspicion that sr3 is getting hit...

Is the thought here that we take an interrupt, and sr3 is not
guaranteed saved/restored, while sr7 is guaranteed?

I don't see anything wrong with moving the gate earlier (unless
someone can come up with a case where an LWS may not want to gate).

It has the benefit of making the fast path 1 instruction shorter,
however I don't see that it makes the implementation more correct.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[Helge Deller]

On 12/31/2009 07:14 PM, Carlos O'Donell wrote:
> On Tue, Dec 29, 2009 at 8:47 AM, Helge Deller<deller@gmx.de>  wrote:
>>> 2) Change return space register to sr7.  sr3 is not set correctly
>>> if the entry number is invalid.
>>
>> Yep.
>>
>>> Have nasty suspicion that sr3 is getting hit...
>
> Is the thought here that we take an interrupt, and sr3 is not
> guaranteed saved/restored, while sr7 is guaranteed?
>
> I don't see anything wrong with moving the gate earlier (unless
> someone can come up with a case where an LWS may not want to gate).
>
> It has the benefit of making the fast path 1 instruction shorter,
> however I don't see that it makes the implementation more correct.

I tested the patch and the testcase in
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=561203
still segfaults.

Nevertheless, I think Dave's patch should be applied...

Helge

Re: futex wait failure

[John David Anglin]

> I tested the patch and the testcase in
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=561203
> still segfaults.

I think the expect/tcl bug and the bug 561203 are related.  Looking
at the minifail core dump, I see:

Core was generated by `./minifail'.
Program terminated with signal 11, Segmentation fault.
#0  0x00000000 in ?? ()

So, how did we get to 0?  $rp is 0, so we might have executed a
return to this location.  $r31 conains 0x4157cc4f.

(gdb) disass 0x4157cc3c 0x4157cc5c
Dump of assembler code from 0x4157cc3c to 0x4157cc5c:
0x4157cc3c <_IO_puts+332>:      copy rp,r25
0x4157cc40 <_IO_puts+336>:      copy r6,r24
0x4157cc44 <_IO_puts+340>:      be,l b0(sr2,r0),sr0,r31
0x4157cc48 <_IO_puts+344>:      ldi 0,r20
0x4157cc4c <_IO_puts+348>:      ldi -b,r24
0x4157cc50 <_IO_puts+352>:      cmpb,=,n r24,r21,0x4157cc38 <_IO_puts+328>
0x4157cc54 <_IO_puts+356>:      nop
0x4157cc58 <_IO_puts+360>:      ldi -2d,r25

So, it would see $r31 was last set by a call to the lws code.

(gdb) info thread
  2 Thread 2397  *__GI__IO_list_lock () at genops.c:1297
* 1 Thread 2398  0x00000000 in ?? ()

(gdb) thread 2
[Switching to thread 2 (Thread 2397)]#0  *__GI__IO_list_lock ()
    at genops.c:1297
1297    genops.c: No such file or directory.
        in genops.c
(gdb) bt
#0  *__GI__IO_list_lock () at genops.c:1297
#1  0x415c06d0 in __libc_fork () at ../nptl/sysdeps/unix/sysv/linux/fork.c:117
#2  0x00011068 in pure_test () at minifail.cpp:55
#3  0x00011350 in main (argc=1, argv=0xc0497028) at minifail.cpp:80

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

> Core was generated by `./minifail'.
> Program terminated with signal 11, Segmentation fault.
> #0  0x00000000 in ?? ()

I wasn't able to duplicate the above running minifail under gdb.
However, I did eventually get it to hang.

(gdb) info thread
  2 Thread 0x42169480 (LWP 3246)  0x40240760 in start_thread (arg=0x42169480)
    at pthread_create.c:293
* 1 Thread 0x400040c0 (LWP 3245)  0x40240c2c in pthread_join (
  threadid=1108776064, thread_return=0x0) at pthread_join.c:89
(gdb) bt
#0  0x40240c2c in pthread_join (threadid=1108776064, thread_return=0x0)
    at pthread_join.c:89
#1  0x00011118 in pure_test () at minifail.cpp:70
#2  0x00011350 in main (argc=1, argv=0xbffd3020) at minifail.cpp:80
(gdb) disass 0x40240c1c 0x40240c3c
Dump of assembler code from 0x40240c1c to 0x40240c3c:
0x40240c1c <pthread_join+268>:  copy r3,r26
0x40240c20 <pthread_join+272>:  copy r19,r4
0x40240c24 <pthread_join+276>:  be,l 100(sr2,r0),sr0,r31
0x40240c28 <pthread_join+280>:  ldi d2,r20
0x40240c2c <pthread_join+284>:  copy r4,r19
0x40240c30 <pthread_join+288>:  ldw 68(r5),r24
0x40240c34 <pthread_join+292>:  cmpib,<> 0,r24,0x40240c18 <pthread_join+264>
0x40240c38 <pthread_join+296>:  ldi 0,r23
End of assembler dump.

So, thread 1 is in a syscall.

(gdb) thread 2
[Switching to thread 2 (Thread 0x42169480 (LWP 3246))]#0  0x40240760 in start_thread (arg=0x42169480) at pthread_create.c:293
293     pthread_create.c: No such file or directory.
        in pthread_create.c
(gdb) bt
#0  0x40240760 in start_thread (arg=0x42169480) at pthread_create.c:293
#1  0x412082bc in clone () from /lib/libc.so.6
#2  0x00000000 in ?? ()
(gdb) disass 0x40240760 0x402407a0
Dump of assembler code from 0x40240754 to 0x402407a8:
0x40240754 <start_thread+1108>: ldw 214(ret0),rp
0x40240758 <start_thread+1112>: copy r3,r26
0x40240760 <start_thread+1120>: copy r4,r24
0x40240764 <start_thread+1124>: be,l b0(sr2,r0),sr0,r31
0x40240768 <start_thread+1128>: ldi 0,r20
0x4024076c <start_thread+1132>: ldi -b,r24
0x40240770 <start_thread+1136>: cmpb,=,n r24,r21,0x40240758 <start_thread+1112>
0x40240774 <start_thread+1140>: nop
0x40240778 <start_thread+1144>: ldi -2d,r25
0x4024077c <start_thread+1148>: cmpb,=,n r25,r21,0x40240758 <start_thread+1112>
0x40240780 <start_thread+1152>: nop
0x40240784 <start_thread+1156>: stw ret0,-1b8(sp)
0x40240788 <start_thread+1160>: sub r0,r21,r21
0x4024078c <start_thread+1164>: stw r21,-1b4(sp)
0x40240790 <start_thread+1168>: ldw -1b4(sp),ret1
0x40240794 <start_thread+1172>: cmpib,=,n e,ret1,0x40240804 <start_thread+1284>
0x40240798 <start_thread+1176>: ldw -1b4(sp),ret0
0x4024079c <start_thread+1180>: cmpb,=,n r5,ret0,0x40240804 <start_thread+1284>
0x402407a0 <start_thread+1184>: ldw -1b8(sp),ret0
0x402407a4 <start_thread+1188>: cmpb,<> ret0,rp,0x40240754 <start_thread+1108>

Thread 2 is in a loop calling the lws code.

(gdb) p/x $rp
$5 = 0x0
(gdb) p/x $ret0
$6 = 0x42169480

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Helge Deller]

> > I tested the patch and the testcase in
> > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=3D561203
> > still segfaults.
>=20
> I think the expect/tcl bug and the bug 561203 are related.  Looking
> at the minifail core dump, I see:
>=20
> Core was generated by `./minifail'.
> Program terminated with signal 11, Segmentation fault.
> #0  0x00000000 in ?? ()
>=20
> So, how did we get to 0?  $rp is 0, so we might have executed a
> return to this location.  $r31 conains 0x4157cc4f.
>=20
> (gdb) disass 0x4157cc3c 0x4157cc5c
> Dump of assembler code from 0x4157cc3c to 0x4157cc5c:
> 0x4157cc3c <_IO_puts+332>:      copy rp,r25
> 0x4157cc40 <_IO_puts+336>:      copy r6,r24
> 0x4157cc44 <_IO_puts+340>:      be,l b0(sr2,r0),sr0,r31
> 0x4157cc48 <_IO_puts+344>:      ldi 0,r20
> 0x4157cc4c <_IO_puts+348>:      ldi -b,r24
> 0x4157cc50 <_IO_puts+352>:      cmpb,=3D,n r24,r21,0x4157cc38 <_IO_pu=
ts+328>
> 0x4157cc54 <_IO_puts+356>:      nop
> 0x4157cc58 <_IO_puts+360>:      ldi -2d,r25


I think I have an idea what could have happened and why it most of the =
times (but not always) crashes in the child process...

In ports/sysdeps/unix/sysv/linux/hppa/bits/atomic.h we have:
#define atomic_compare_and_exchange_val_acq(mem, newval, oldval) \
  ({                                                                   =
 \
     volatile int lws_errno;                                           =
 \
     volatile int lws_ret;                                             =
 \
     asm volatile(                                                     =
 \
=2E..some assembly...
        "stw    %%r28, %0                       \n\t"                  =
 \
        "sub    %%r0, %%r21, %%r21              \n\t"                  =
 \
        "stw    %%r21, %1                       \n\t"                  =
 \
        : "=3Dm" (lws_ret), "=3Dm" (lws_errno)                         =
     \
        : "r" (mem), "r" (oldval), "r" (newval)                        =
 \
        : _LWS_CLOBBER          =20

this means, that lws_errno and lws_ret are located on the stack.

With gdb I see this expanded to:
0x40705494 <start_thread+1204>: stw ret0,-1b8(sp)
0x40705498 <start_thread+1208>: sub r0,r21,r21
0x4070549c <start_thread+1212>: stw r21,-1b4(sp)

So, lws_ret/lws_errno are at -1b8/-1b4(sp).

And this LWS code is called from=20
=2E./nptl/sysdeps/pthread/createthread.c:
static int create_thread (struct pthread *pd, const struct pthread_attr=
 *attr, STACK_VARIABLES_PARMS)
=2E..
          int res =3D do_clone (pd, attr, clone_flags, start_thread,
                              STACK_VARIABLES_ARGS, 1);
          if (res =3D=3D 0)
            {
=2E..(line 216):
              /* Enqueue the descriptor.  */
              do
                pd->nextevent =3D __nptl_last_event;
              while (atomic_compare_and_exchange_bool_acq(&__nptl_last_=
event, pd, pd->nextevent) !=3D 0);


And here is what could have happened:
a) do_clone() creates the child process.
b) the child process gets a new stack
c) the child calls atomic_compare_and_exchange_bool_acq() and thus the =
LWS code above.
d) the LWS code writes to the stack location at -1b8(sp), which is out =
of bounds for the child process (the child stack got only ~ 0x40 bytes =
initial room)
e) Thus the child either crashes, overwrites memory of the parent or do=
es other things wrong.

Additionally:
Due to the LWS assembly code and because we don't have many registers f=
ree while using LWS, gcc used %rp as a temporary register which may hav=
e fooled us in our thinking?

0x40705458 <start_thread+1144>: ldi 0,rp
0x4070545c <start_thread+1148>: ldi fb,r3
0x40705460 <start_thread+1152>: ldw -70(sp),ret0
0x40705464 <start_thread+1156>: ldw 214(ret0),ret1
0x40705468 <start_thread+1160>: copy r5,r26
0x4070546c <start_thread+1164>: copy ret1,r25
0x40705470 <start_thread+1168>: copy rp,r24
0x40705474 <start_thread+1172>: be,l b0(sr2,r0),sr0,r31
0x40705478 <start_thread+1176>: ldi 0,r20
0x4070547c <start_thread+1180>: ldi -b,r24
0x40705480 <start_thread+1184>: cmpb,=3D,n r24,r21,0x40705468 <start_th=
read+1160>
0x40705484 <start_thread+1188>: nop
0x40705488 <start_thread+1192>: ldi -2d,r25
0x4070548c <start_thread+1196>: cmpb,=3D,n r25,r21,0x40705468 <start_th=
read+1160>
0x40705490 <start_thread+1200>: nop
0x40705494 <start_thread+1204>: stw ret0,-1b8(sp)
0x40705498 <start_thread+1208>: sub r0,r21,r21
0x4070549c <start_thread+1212>: stw r21,-1b4(sp)
0x407054a0 <start_thread+1216>: ldw -1b4(sp),ret0


If my assumptions are correct, then we either could

a) use the gcc atomic builtins instead of own atomic code in libc6:
E.g: add to ports/sysdeps/unix/sysv/linux/hppa/bits/atomic.h:
=2E..
#if __GNUC_PREREQ (4, 1)
# define atomic_compare_and_exchange_val_acq(mem, newval, oldval) \
  __sync_val_compare_and_swap (mem, oldval, newval)
#  define atomic_compare_and_exchange_bool_acq(mem, newval, oldval) \
  (! __sync_bool_compare_and_swap (mem, oldval, newval))

#elif __ASSUME_LWS_CAS
=2E...

b) change the assembly in=20
atomic_compare_and_exchange_val_acq()
to not put it's local variables (lws_errno and lws_ret) on the stack.

I'm currently testing option a).

Helge
(PS: I used a webmailer, so the indenting might be strange...)
--=20
GRATIS f=FCr alle GMX-Mitglieder: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://portal.gmx.net/de/go/maxdome01
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[Carlos O'Donell]

On Mon, Jan 4, 2010 at 11:27 AM, Helge Deller <deller@gmx.de> wrote:
> I think I have an idea what could have happened and why it most of th=
e times (but not always) crashes in the child process...
>
> In ports/sysdeps/unix/sysv/linux/hppa/bits/atomic.h we have:
> #define atomic_compare_and_exchange_val_acq(mem, newval, oldval) \
> =A0({ =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
\
> =A0 =A0 volatile int lws_errno; =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0\
> =A0 =A0 volatile int lws_ret; =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0\
> =A0 =A0 asm volatile( =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0\
> ...some assembly...
> =A0 =A0 =A0 =A0"stw =A0 =A0%%r28, %0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 \n\t" =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 \
> =A0 =A0 =A0 =A0"sub =A0 =A0%%r0, %%r21, %%r21 =A0 =A0 =A0 =A0 =A0 =A0=
 =A0\n\t" =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 \
> =A0 =A0 =A0 =A0"stw =A0 =A0%%r21, %1 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 \n\t" =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 \
> =A0 =A0 =A0 =A0: "=3Dm" (lws_ret), "=3Dm" (lws_errno) =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0\
> =A0 =A0 =A0 =A0: "r" (mem), "r" (oldval), "r" (newval) =A0 =A0 =A0 =A0=
 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 \
> =A0 =A0 =A0 =A0: _LWS_CLOBBER
>
> this means, that lws_errno and lws_ret are located on the stack.

Correct. We could place them in registers if we wanted, they are
registers r28 (lws return) and r21 (lws error).

> With gdb I see this expanded to:
> 0x40705494 <start_thread+1204>: stw ret0,-1b8(sp)
> 0x40705498 <start_thread+1208>: sub r0,r21,r21
> 0x4070549c <start_thread+1212>: stw r21,-1b4(sp)
>
> So, lws_ret/lws_errno are at -1b8/-1b4(sp).

Correct.

> And this LWS code is called from
> ../nptl/sysdeps/pthread/createthread.c:
> static int create_thread (struct pthread *pd, const struct pthread_at=
tr *attr, STACK_VARIABLES_PARMS)
> ...
> =A0 =A0 =A0 =A0 =A0int res =3D do_clone (pd, attr, clone_flags, start=
_thread,
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0STACK_VARI=
ABLES_ARGS, 1);
> =A0 =A0 =A0 =A0 =A0if (res =3D=3D 0)
> =A0 =A0 =A0 =A0 =A0 =A0{
> ...(line 216):
> =A0 =A0 =A0 =A0 =A0 =A0 =A0/* Enqueue the descriptor. =A0*/
> =A0 =A0 =A0 =A0 =A0 =A0 =A0do
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pd->nextevent =3D __nptl_last_event;
> =A0 =A0 =A0 =A0 =A0 =A0 =A0while (atomic_compare_and_exchange_bool_ac=
q(&__nptl_last_event, pd, pd->nextevent) !=3D 0);
>
>
> And here is what could have happened:
> a) do_clone() creates the child process.
> b) the child process gets a new stack
> c) the child calls atomic_compare_and_exchange_bool_acq() and thus th=
e LWS code above.
> d) the LWS code writes to the stack location at -1b8(sp), which is ou=
t of bounds for the child process (the child stack got only ~ 0x40 byte=
s initial room)

This is wrong. Each thread should have 8MB of stack. If we only get ~
0x40 bytes then npt/nptl-init.c is setting __default_stacksize
incorrectly.

Even PTHREAD_STACK_MIN should be 16kb?

Could you verify that your assertion that only ~ 0x40 bytes of initial
room were allocated?

> e) Thus the child either crashes, overwrites memory of the parent or =
does other things wrong.

I agree with your analysis, but the error is that more stack should be
allocated.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[John David Anglin]

On Mon, 04 Jan 2010, Carlos O'Donell wrote:

> On Mon, Jan 4, 2010 at 11:27 AM, Helge Deller <deller@gmx.de> wrote:
> This is wrong. Each thread should have 8MB of stack. If we only get ~
> 0x40 bytes then npt/nptl-init.c is setting __default_stacksize
> incorrectly.

The 0x40 bytes is the initial frame allocated for clone running in
the child thread.   The code is not running out of stack space.

> Even PTHREAD_STACK_MIN should be 16kb?
> 
> Could you verify that your assertion that only ~ 0x40 bytes of initial
> room were allocated?
> 
> > e) Thus the child either crashes, overwrites memory of the parent or does other things wrong.
> 
> I agree with your analysis, but the error is that more stack should be
> allocated.

I don't follow that conclusion.  The stack grows upward and the stack
pointer isn't out of range.   The fork operation is somehow
corrupting the stack memory of the thread created by pthread_create.
I would say the parent is corrupting its own memory.  I doubt the
forked child is affecting the parent.  Fork would have to behave like
vfork to do this.  I have seen the pthread_create thread fail before
the clone syscall of the following fork.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

> On Mon, 04 Jan 2010, Carlos O'Donell wrote:
> 
> > On Mon, Jan 4, 2010 at 11:27 AM, Helge Deller <deller@gmx.de> wrote:
> > This is wrong. Each thread should have 8MB of stack. If we only get ~
> > 0x40 bytes then npt/nptl-init.c is setting __default_stacksize
> > incorrectly.
> 
> The 0x40 bytes is the initial frame allocated for clone running in
> the child thread.   The code is not running out of stack space.

If only 0x40 bytes of stack were allocated, start_thread would fault
almost immediately:

Dump of assembler code for function start_thread:
   0x40a40300 <+0>:     stw rp,-14(sp)
   0x40a40304 <+4>:     ldo 1c0(sp),sp
   0x40a40308 <+8>:     ldo 274(r26),r21
   0x40a4030c <+12>:    stw r9,-6c(sp)

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Helge Deller]

On 01/04/2010 07:11 PM, John David Anglin wrote:
> On Mon, 04 Jan 2010, Carlos O'Donell wrote:
>
>> On Mon, Jan 4, 2010 at 11:27 AM, Helge Deller<deller@gmx.de>  wrote:
>> This is wrong. Each thread should have 8MB of stack. If we only get ~
>> 0x40 bytes then npt/nptl-init.c is setting __default_stacksize
>> incorrectly.
>
> The 0x40 bytes is the initial frame allocated for clone running in
> the child thread.   The code is not running out of stack space.

Hmmm...

strace on minifail (as attached to Dave's mail) gives me:

getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
mmap(NULL, 8388608, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4076d000
brk(0)                                  = 0x12000
brk(0x33000)                            = 0x33000
mprotect(0x40f6b000, 4096, PROT_NONE)   = 0
clone(Process 1684 attached (waiting for parent)
Process 1684 resumed (parent 1683 ready)
child_stack=0x4076d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x40f6c4e8, tls=0x40f6c900, child_tidptr=0x40f6c4e8) = 1684

The mmap() allocates and maps the new child stack -> at 0x4076d000
The clone() syscall is called with child_stack=0x4076d040

I might be wrong, but that's the 0x40 bytes I mentioned.
  
>> Even PTHREAD_STACK_MIN should be 16kb?

The example above allocates 8MB.
But my point is, that child_stack starts at 0x4076d040, and
that LWS (in the child process with the stack as given above) tries to
store something to an address lower than 0x4076d000.

>> Could you verify that your assertion that only ~ 0x40 bytes of initial
>> room were allocated?
>>
>>> e) Thus the child either crashes, overwrites memory of the parent or does other things wrong.
>>
>> I agree with your analysis, but the error is that more stack should be
>> allocated.

Not more stack.
Just increasing the 0x40 initial byte offset, but that's IMHO a hack...

> I don't follow that conclusion.  The stack grows upward and the stack
> pointer isn't out of range.   The fork operation is somehow
> corrupting the stack memory of the thread created by pthread_create.
> I would say the parent is corrupting its own memory.  I doubt the
> forked child is affecting the parent.  Fork would have to behave like
> vfork to do this.  I have seen the pthread_create thread fail before
> the clone syscall of the following fork.

Doesn't pthread_create() created processes share the memory with
their parents? In that case, the child can crash or even overwrite memory
of the parent process...?

Helge

Re: futex wait failure

[John David Anglin]

> On 01/04/2010 07:11 PM, John David Anglin wrote:
> > On Mon, 04 Jan 2010, Carlos O'Donell wrote:
> >
> >> On Mon, Jan 4, 2010 at 11:27 AM, Helge Deller<deller@gmx.de>  wrote:
> >> This is wrong. Each thread should have 8MB of stack. If we only get ~
> >> 0x40 bytes then npt/nptl-init.c is setting __default_stacksize
> >> incorrectly.
> >
> > The 0x40 bytes is the initial frame allocated for clone running in
> > the child thread.   The code is not running out of stack space.
> 
> Hmmm...
> 
> strace on minifail (as attached to Dave's mail) gives me:
> 
> getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
> mmap(NULL, 8388608, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x4076d000
> brk(0)                                  = 0x12000
> brk(0x33000)                            = 0x33000
> mprotect(0x40f6b000, 4096, PROT_NONE)   = 0
> clone(Process 1684 attached (waiting for parent)
> Process 1684 resumed (parent 1683 ready)
> child_stack=0x4076d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x40f6c4e8, tls=0x40f6c900, child_tidptr=0x40f6c4e8) = 1684
> 
> The mmap() allocates and maps the new child stack -> at 0x4076d000
> The clone() syscall is called with child_stack=0x4076d040

So, in this case, start_thread will be called with $sp = 0x4076d040.
The second instruction of start_thread increments $sp by 0x1c0.

> 
> I might be wrong, but that's the 0x40 bytes I mentioned.
>   
> >> Even PTHREAD_STACK_MIN should be 16kb?
> 
> The example above allocates 8MB.
> But my point is, that child_stack starts at 0x4076d040, and
> that LWS (in the child process with the stack as given above) tries to
> store something to an address lower than 0x4076d000.

This would be bad.  However, the LWS calls are in "start_thread", so
it didn't appear that the stack offsets that you mentioned were out
of range.  GCC would be terribly broken if it got these offsets wrong
in general.

> >> Could you verify that your assertion that only ~ 0x40 bytes of initial
> >> room were allocated?
> >>
> >>> e) Thus the child either crashes, overwrites memory of the parent or does other things wrong.
> >>
> >> I agree with your analysis, but the error is that more stack should be
> >> allocated.
> 
> Not more stack.
> Just increasing the 0x40 initial byte offset, but that's IMHO a hack...
> 
> > I don't follow that conclusion.  The stack grows upward and the stack
> > pointer isn't out of range.   The fork operation is somehow
> > corrupting the stack memory of the thread created by pthread_create.
> > I would say the parent is corrupting its own memory.  I doubt the
> > forked child is affecting the parent.  Fork would have to behave like
> > vfork to do this.  I have seen the pthread_create thread fail before
> > the clone syscall of the following fork.
> 
> Doesn't pthread_create() created processes share the memory with
> their parents? In that case, the child can crash or even overwrite memory
> of the parent process...?

We have to be careful about semantics.  The child of the fork runs
in a different address space, so it is unlikely that it can corrupt
the parent directly.  It is true that the child inherits all the
pthread mutexs and the thread context of the thread which called fork.
The child could close the file descriptors of the parent.  It could
affect any context that is stored in the kernel.

The fork call is involved in this bug.  I have verified that the faults
don't occur if it is removed.  The faults don't occur if a sleep(1) call
is added between the pthread_create and fork calls.

I think the parent thread is corrupting the stack of the child thread
created by pthread_create, but I don't know how this happens.  I have
seen at least one case where this corruption occurs prior to the system
clone call for the fork.  I think we must have some kind of lock
failure which is timing dependent (i.e., the scheduling of the parent
and child threads).

I thought this likely indicated the lws code wasn't atomic.  We don't
allow schedule to run if we are on the gateway page.  I'm starting
to wonder if threads (not processes) are still being scheduled.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

> clone(Process 1684 attached (waiting for parent)
> Process 1684 resumed (parent 1683 ready)
> child_stack=0x4076d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x40f6c4e8, tls=0x40f6c900, child_tidptr=0x40f6c4e8) = 1684

I noticed the tidptr for the fork may not be correct:

clone(child_stack=0x40e87040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x416864e8, tls=0x41686900, child_tidptr=0x416864e8) = 31613
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x40002028) = 31614

I would have thought the value should have been the same as that in the
clone from the pthread_create call.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

> > clone(Process 1684 attached (waiting for parent)
> > Process 1684 resumed (parent 1683 ready)
> > child_stack=0x4076d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x40f6c4e8, tls=0x40f6c900, child_tidptr=0x40f6c4e8) = 1684
> 
> I noticed the tidptr for the fork may not be correct:
> 
> clone(child_stack=0x40e87040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x416864e8, tls=0x41686900, child_tidptr=0x416864e8) = 31613
> clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x40002028) = 31614
> 
> I would have thought the value should have been the same as that in the
> clone from the pthread_create call.

It's possible that this is done intentionally...  The parent_tidptr
is the one that's wrong in the first clone.

I have noticed something else in the minifail kernel register dumps:

Jan  6 15:54:05 hiauly6 kernel: sr00-03  00000024 0000001b 00000000 00000024
Jan  6 15:54:05 hiauly6 kernel: sr04-07  00000024 00000024 00000024 00000024

sr1 seems to contain an odd value.  This seems to be the case in all
minifail register dumps.  I checked that the sr1 value doesn't belong
to the child of the fork call.  This might indicate a tlb/cache issue
as sr1 is used for these operations.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Helge Deller]

On 01/07/2010 12:33 AM, John David Anglin wrote:
>>> clone(Process 1684 attached (waiting for parent)
>>> Process 1684 resumed (parent 1683 ready)
>>> child_stack=0x4076d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x40f6c4e8, tls=0x40f6c900, child_tidptr=0x40f6c4e8) = 1684
>>
>> I noticed the tidptr for the fork may not be correct:
>>
>> clone(child_stack=0x40e87040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x416864e8, tls=0x41686900, child_tidptr=0x416864e8) = 31613
>> clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x40002028) = 31614
>>
>> I would have thought the value should have been the same as that in the
>> clone from the pthread_create call.
>
> It's possible that this is done intentionally...  The parent_tidptr
> is the one that's wrong in the first clone.
>
> I have noticed something else in the minifail kernel register dumps:
>
> Jan  6 15:54:05 hiauly6 kernel: sr00-03  00000024 0000001b 00000000 00000024
> Jan  6 15:54:05 hiauly6 kernel: sr04-07  00000024 00000024 00000024 00000024
>
> sr1 seems to contain an odd value.  This seems to be the case in all
> minifail register dumps.

IIRC, for me most crashes had sr1=0. Only a very few had sr1 != 0.

> I checked that the sr1 value doesn't belong
> to the child of the fork call.  This might indicate a tlb/cache issue
> as sr1 is used for these operations.

Helge

Re: futex wait failure

[John David Anglin]

> 
> On 01/07/2010 12:33 AM, John David Anglin wrote:
> >>> clone(Process 1684 attached (waiting for parent)
> >>> Process 1684 resumed (parent 1683 ready)
> >>> child_stack=0x4076d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x40f6c4e8, tls=0x40f6c900, child_tidptr=0x40f6c4e8) = 1684
> >>
> >> I noticed the tidptr for the fork may not be correct:
> >>
> >> clone(child_stack=0x40e87040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x416864e8, tls=0x41686900, child_tidptr=0x416864e8) = 31613
> >> clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x40002028) = 31614
> >>
> >> I would have thought the value should have been the same as that in the
> >> clone from the pthread_create call.
> >
> > It's possible that this is done intentionally...  The parent_tidptr
> > is the one that's wrong in the first clone.

I now think this probably is a glibc bug.  The kernel uses this value
when the CLONE_PARENT_SETTID flag is passed.

> > I have noticed something else in the minifail kernel register dumps:
> >
> > Jan  6 15:54:05 hiauly6 kernel: sr00-03  00000024 0000001b 00000000 00000024
> > Jan  6 15:54:05 hiauly6 kernel: sr04-07  00000024 00000024 00000024 00000024
> >
> > sr1 seems to contain an odd value.  This seems to be the case in all
> > minifail register dumps.
> 
> IIRC, for me most crashes had sr1=0. Only a very few had sr1 != 0.
> 
> > I checked that the sr1 value doesn't belong
> > to the child of the fork call.  This might indicate a tlb/cache issue
> > as sr1 is used for these operations.

I added some loops in the parent and child threads.  I also added code
in the child thread to watch the return point location on the stack
for start_thread.  What I found is the stack gets overwritten after
the thread has started.  At the same time, the parent is looping
post fork.

So, the problem has to be with fork (i.e., its not with pthread_join
or pthread_exit).  Still think the problem involves sr1 (it's unusual
the sr1 contains a value that's not the user or kernel values).

I played with saving sr1 in some additional places (tlb and cache
flushing) but this didn't alter things.  Haven't played with pa_memcpy.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

> I added some loops in the parent and child threads.  I also added code
> in the child thread to watch the return point location on the stack
> for start_thread.  What I found is the stack gets overwritten after
> the thread has started.  At the same time, the parent is looping
> post fork.

More debugging.  It seems bad news to have more than one clone
syscall active at a time.  The thread child may still be in the kernel
when the fork syscall is made by the parent.  The testcase doesn't
fail if the parent waits for the child thread to start.  See below.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>

/*
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=561203

  clone(child_stack=0x4088d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x4108c4e8, tls=0x4108c900, child_tidptr=0x4108c4e8) = 14819
[pid 14819] set_robust_list(0x4108c4f0, 0xc) = 0
[pid 14818] clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x40002028) = 14820

 g++  minifail.cpp -o minifail -O0 -pthread -g

 i=0; while true; do i=$(($i+1)); echo Run $i; ./minifail; done;

 */

static volatile int run;

void* thread_run(void* arg) {
	static long status;
	int i;
        
	run = 1;
	pthread_yield();
	for (i = 10000000; i; i--)
	   continue;
	write(1,"Thread OK.\n",11);
	return (void *)&status;
}

int pure_test() {
	pthread_t thread;
	pthread_create(&thread, NULL, thread_run, NULL);

	while (!run)
	  continue;
	switch (fork()) {
		case -1:
			perror("fork() failed");
		case 0:
			write(1,"Child OK.\n",10);
			_exit(0);
		default:
			break;
		
	}
	pthread_join(thread, NULL);
	return 0;
}

int main(int argc, char** argv) {
	return pure_test();
}

Re: futex wait failure

[Helge Deller]

[-- Attachment #1: Type: text/plain, Size: 1049 bytes --]

I wonder if we have some problems with the LWS code path in the kernel
regarding atomic locking with futexes?

In arch/parisc/kernel/syscall.S we use a lock table called lws_lock_start[]
to guard the LWS code against other competing userspace processes.
I wonder if this really enough, esp. since we do implement futex syscalls
(e.g. clone/exit calls uses futex functions to change userspace values
because of CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID).

Do we maybe need to protect the LWS code path with the same locking table as the
generic kernel? Atomicity of futexes writing to userspace are not in sync
with the locking of the LWS/lws_lock_start[] code.

I tried to come up with a patch for that which I attached here, but sadly
it hangs as soon as the init process is started on a 64bit/SMP kernel.
So either my thinking here is stupid, or I do have a stupid coding bug.

Furthermore, the coding for futex_atomic_op_inuser() in
arch/parisc/include/asm/futex.h seems to miss real functionality.
I didn't looked closer into this though.

Helge

[-- Attachment #2: z1 --]
[-- Type: text/plain, Size: 2970 bytes --]

diff --git a/arch/parisc/kernel/asm-offsets.c b/arch/parisc/kernel/asm-offsets.c
index ec787b4..50353bd 100644
--- a/arch/parisc/kernel/asm-offsets.c
+++ b/arch/parisc/kernel/asm-offsets.c
@@ -290,5 +290,11 @@ int main(void)
 	BLANK();
 	DEFINE(ASM_PDC_RESULT_SIZE, NUM_PDC_RESULT * sizeof(unsigned long));
 	BLANK();
+
+#ifdef CONFIG_SMP
+	DEFINE(ASM_ATOMIC_HASH_SIZE_SHIFT, __builtin_ffs(ATOMIC_HASH_SIZE)-1);
+	DEFINE(ASM_ATOMIC_HASH_ENTRY_SHIFT, __builtin_ffs(sizeof(__atomic_hash[0]))-1);
+#endif
+
 	return 0;
 }
diff --git a/arch/parisc/kernel/setup.c b/arch/parisc/kernel/setup.c
index cb71f3d..878f42c 100644
--- a/arch/parisc/kernel/setup.c
+++ b/arch/parisc/kernel/setup.c
@@ -128,6 +131,14 @@ void __init setup_arch(char **cmdline_p)
 	printk(KERN_INFO "The 32-bit Kernel has started...\n");
 #endif
 
+	/* Consistency check on the size and alignments of our spinlocks */
+#ifdef CONFIG_SMP
+	BUILD_BUG_ON(sizeof(arch_spinlock_t) != __PA_LDCW_ALIGNMENT);
+	BUG_ON((unsigned long)&__atomic_hash[0] & (__PA_LDCW_ALIGNMENT-1));
+	BUG_ON((unsigned long)&__atomic_hash[1] & (__PA_LDCW_ALIGNMENT-1));
+#endif
+	BUILD_BUG_ON((1<<L1_CACHE_SHIFT) != L1_CACHE_BYTES);
+
 	pdc_console_init();
 
 #ifdef CONFIG_64BIT
diff --git a/arch/parisc/kernel/syscall.S b/arch/parisc/kernel/syscall.S
index f5f9602..7925a68 100644
--- a/arch/parisc/kernel/syscall.S
+++ b/arch/parisc/kernel/syscall.S
@@ -11,6 +11,7 @@
 #include <asm/unistd.h>
 #include <asm/errno.h>
 #include <asm/page.h>
+#include <asm/cache.h>
 #include <asm/psw.h>
 #include <asm/thread_info.h>
 #include <asm/assembly.h>
@@ -530,18 +527,17 @@ lws_compare_and_swap32:
 
 lws_compare_and_swap:
 #ifdef CONFIG_SMP
-	/* Load start of lock table */
-	ldil	L%lws_lock_start, %r20
-	ldo	R%lws_lock_start(%r20), %r28
+	/* Calculate lock table entry via ATOMIC_HASH(%r26) */
+	ldil	L%__atomic_hash, %r20
+	ldo	R%__atomic_hash(%r20), %r28
 
-	/* Extract four bits from r26 and hash lock (Bits 4-7) */
-	extru  %r26, 27, 4, %r20
+#ifdef CONFIG_64BIT
+	extrd,u %r26, 63-L1_CACHE_SHIFT, ASM_ATOMIC_HASH_SIZE_SHIFT, %r20
+#else
+	extru	%r26, 31-L1_CACHE_SHIFT, ASM_ATOMIC_HASH_SIZE_SHIFT, %r20
+#endif
+	shladd,l %r20, ASM_ATOMIC_HASH_ENTRY_SHIFT, %r28, %r20
 
-	/* Find lock to use, the hash is either one of 0 to
-	   15, multiplied by 16 (keep it 16-byte aligned)
-	   and add to the lock table offset. */
-	shlw	%r20, 4, %r20
-	add	%r20, %r28, %r20
 
 # if ENABLE_LWS_DEBUG
 	/*	
@@ -672,31 +668,6 @@ ENTRY(sys_call_table64)
 END(sys_call_table64)
 #endif
 
-#ifdef CONFIG_SMP
-	/*
-		All light-weight-syscall atomic operations 
-		will use this set of locks 
-
-		NOTE: The lws_lock_start symbol must be
-		at least 16-byte aligned for safe use
-		with ldcw.
-	*/
-	.section .data
-	.align	PAGE_SIZE
-ENTRY(lws_lock_start)
-	/* lws locks */
-	.rept 16
-	/* Keep locks aligned at 16-bytes */
-	.word 1
-	.word 0 
-	.word 0
-	.word 0
-	.endr
-END(lws_lock_start)
-	.previous
-#endif
-/* CONFIG_SMP for lws_lock_start */

Re: futex wait failure

[John David Anglin]

On Tue, 02 Feb 2010, Helge Deller wrote:

> I wonder if we have some problems with the LWS code path in the kernel
> regarding atomic locking with futexes?
>
> In arch/parisc/kernel/syscall.S we use a lock table called lws_lock_start[]
> to guard the LWS code against other competing userspace processes.
> I wonder if this really enough, esp. since we do implement futex syscalls
> (e.g. clone/exit calls uses futex functions to change userspace values
> because of CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID).
>
> Do we maybe need to protect the LWS code path with the same locking table 
> as the
> generic kernel? Atomicity of futexes writing to userspace are not in sync
> with the locking of the LWS/lws_lock_start[] code.
>
> I tried to come up with a patch for that which I attached here, but sadly
> it hangs as soon as the init process is started on a 64bit/SMP kernel.
> So either my thinking here is stupid, or I do have a stupid coding bug.
>
> Furthermore, the coding for futex_atomic_op_inuser() in
> arch/parisc/include/asm/futex.h seems to miss real functionality.
> I didn't looked closer into this though.

While locking may be a problem, it is not the main reason the minifail
program fails.  The program fails on my c3750 with a 32-bit UP kernel.
There is no locking in this kernel.  The LWS code relies on the scheduler
for atomicity.

I tried disabling interrupts around the crucial three instructions but
it didn't help.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Helge Deller]

[-- Attachment #1: Type: text/plain, Size: 2164 bytes --]

On 02/03/2010 04:44 AM, John David Anglin wrote:
> On Tue, 02 Feb 2010, Helge Deller wrote:
>
>> I wonder if we have some problems with the LWS code path in the kernel
>> regarding atomic locking with futexes?
>>
>> In arch/parisc/kernel/syscall.S we use a lock table called lws_lock_start[]
>> to guard the LWS code against other competing userspace processes.
>> I wonder if this really enough, esp. since we do implement futex syscalls
>> (e.g. clone/exit calls uses futex functions to change userspace values
>> because of CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID).
>>
>> Do we maybe need to protect the LWS code path with the same locking table
>> as the
>> generic kernel? Atomicity of futexes writing to userspace are not in sync
>> with the locking of the LWS/lws_lock_start[] code.
>>
>> I tried to come up with a patch for that which I attached here, but sadly
>> it hangs as soon as the init process is started on a 64bit/SMP kernel.
>> So either my thinking here is stupid, or I do have a stupid coding bug.
>>
>> Furthermore, the coding for futex_atomic_op_inuser() in
>> arch/parisc/include/asm/futex.h seems to miss real functionality.
>> I didn't looked closer into this though.
>
> While locking may be a problem, it is not the main reason the minifail
> program fails.  The program fails on my c3750 with a 32-bit UP kernel.

Hmm, I can't reproduce it at the moment with a 32bit UP kernel on my
c3000. The minifail3 test program I currently use (to avoid glibc issues)
is attached.

> There is no locking in this kernel.  The LWS code relies on the scheduler
> for atomicity.

yep. This should then work an a UP kernel.

> I tried disabling interrupts around the crucial three instructions but
> it didn't help.

Ugh.

Anyway, my current patch which compiles and runs fine is attached here as
well. Maybe you want to try it on your SMP builds? It includes the syscall.S
changes you sent last time too.
In this version of the patch I added on own LWS locking hash table for user-space accesses,
which is used in the LWS code and when some (probably not all) put_user()/
get_user() calls are made in the futex code.

Feedback still welcome.

Helge

[-- Attachment #2: minifail3.c --]
[-- Type: text/plain, Size: 652 bytes --]

#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>

/*
 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=561203

 gcc  minifail3.c -o minifail3 -O0 -pthread -g

 Run as: i=0; while true; do i=$(($i+1)); echo Run $i; ./minifail3; done;

 */

static volatile int run;

void* thread_run(void* arg) {
	write(1,"Thread OK.\n",11);
	_exit(0);
}

int main(int argc, char** argv) {
	pthread_t thread;
	pthread_create(&thread, NULL, thread_run, NULL);

	switch (fork()) {
		case -1:
			perror("fork() failed");
		case 0:
			write(1,"Child OK.\n",10);
			exit(0);
		default:
			break;
		
	}
	pthread_join(thread, NULL);
	return 0;
}

[-- Attachment #3: parisc_lock_v2.patch --]
[-- Type: text/x-patch, Size: 9485 bytes --]

diff --git a/arch/parisc/include/asm/atomic.h b/arch/parisc/include/asm/atomic.h
index 716634d..18d57c8 100644
--- a/arch/parisc/include/asm/atomic.h
+++ b/arch/parisc/include/asm/atomic.h
@@ -24,29 +24,46 @@
  * Hash function to index into a different SPINLOCK.
  * Since "a" is usually an address, use one spinlock per cacheline.
  */
-#  define ATOMIC_HASH_SIZE 4
-#  define ATOMIC_HASH(a) (&(__atomic_hash[ (((unsigned long) (a))/L1_CACHE_BYTES) & (ATOMIC_HASH_SIZE-1) ]))
+#  define ATOMIC_HASH_SIZE (4096/L1_CACHE_BYTES)  /* 4 */
+#  define ATOMIC_HASH(a)      (&(__atomic_hash[ (((unsigned long) (a))/L1_CACHE_BYTES) & (ATOMIC_HASH_SIZE-1) ]))
+#  define ATOMIC_USER_HASH(a) (&(__atomic_user_hash[ (((unsigned long) (a))/L1_CACHE_BYTES) & (ATOMIC_HASH_SIZE-1) ]))
 
 extern arch_spinlock_t __atomic_hash[ATOMIC_HASH_SIZE] __lock_aligned;
+extern arch_spinlock_t __atomic_user_hash[ATOMIC_HASH_SIZE] __lock_aligned;
 
 /* Can't use raw_spin_lock_irq because of #include problems, so
  * this is the substitute */
-#define _atomic_spin_lock_irqsave(l,f) do {	\
-	arch_spinlock_t *s = ATOMIC_HASH(l);		\
+#define _atomic_spin_lock_irqsave_template(l,f,hash_func) do {	\
+	arch_spinlock_t *s = hash_func;		\
 	local_irq_save(f);			\
 	arch_spin_lock(s);			\
 } while(0)
 
-#define _atomic_spin_unlock_irqrestore(l,f) do {	\
-	arch_spinlock_t *s = ATOMIC_HASH(l);			\
+#define _atomic_spin_unlock_irqrestore_template(l,f,hash_func) do {	\
+	arch_spinlock_t *s = hash_func;			\
 	arch_spin_unlock(s);				\
 	local_irq_restore(f);				\
 } while(0)
 
+/* kernel memory locks */
+#define _atomic_spin_lock_irqsave(l,f)	\
+	_atomic_spin_lock_irqsave_template(l,f,ATOMIC_HASH(l))
+
+#define _atomic_spin_unlock_irqrestore(l,f)	\
+	_atomic_spin_unlock_irqrestore_template(l,f,ATOMIC_HASH(l))
+
+/* userspace memory locks */
+#define _atomic_spin_lock_irqsave_user(l,f)	\
+	_atomic_spin_lock_irqsave_template(l,f,ATOMIC_USER_HASH(l))
+
+#define _atomic_spin_unlock_irqrestore_user(l,f)	\
+	_atomic_spin_unlock_irqrestore_template(l,f,ATOMIC_USER_HASH(l))
 
 #else
 #  define _atomic_spin_lock_irqsave(l,f) do { local_irq_save(f); } while (0)
 #  define _atomic_spin_unlock_irqrestore(l,f) do { local_irq_restore(f); } while (0)
+#  define _atomic_spin_lock_irqsave_user(l,f) _atomic_spin_lock_irqsave(l,f)
+#  define _atomic_spin_unlock_irqrestore_user(l,f) _atomic_spin_lock_irqsave_user(l,f)
 #endif
 
 /* This should get optimized out since it's never called.
diff --git a/arch/parisc/include/asm/futex.h b/arch/parisc/include/asm/futex.h
index 0c705c3..7bc963e 100644
--- a/arch/parisc/include/asm/futex.h
+++ b/arch/parisc/include/asm/futex.h
@@ -55,6 +55,7 @@ futex_atomic_cmpxchg_inatomic(int __user *uaddr, int oldval, int newval)
 {
 	int err = 0;
 	int uval;
+	unsigned long flags;
 
 	/* futex.c wants to do a cmpxchg_inatomic on kernel NULL, which is
 	 * our gateway page, and causes no end of trouble...
@@ -65,10 +66,15 @@ futex_atomic_cmpxchg_inatomic(int __user *uaddr, int oldval, int newval)
 	if (!access_ok(VERIFY_WRITE, uaddr, sizeof(int)))
 		return -EFAULT;
 
+	_atomic_spin_lock_irqsave_user(uaddr, flags);
+
 	err = get_user(uval, uaddr);
-	if (err) return -EFAULT;
-	if (uval == oldval)
-		err = put_user(newval, uaddr);
+	if (!err)
+		if (uval == oldval)
+			err = put_user(newval, uaddr);
+
+	_atomic_spin_unlock_irqrestore_user(uaddr, flags);
+
 	if (err) return -EFAULT;
 	return uval;
 }
diff --git a/arch/parisc/include/asm/system.h b/arch/parisc/include/asm/system.h
index d91357b..4653c77 100644
--- a/arch/parisc/include/asm/system.h
+++ b/arch/parisc/include/asm/system.h
@@ -160,7 +160,7 @@ static inline void set_eiem(unsigned long val)
    ldcd). */
 
 #define __PA_LDCW_ALIGNMENT	4
-#define __ldcw_align(a) ((volatile unsigned int *)a)
+#define __ldcw_align(a) (&(a)->slock)
 #define __LDCW	"ldcw,co"
 
 #endif /*!CONFIG_PA20*/
diff --git a/arch/parisc/kernel/asm-offsets.c b/arch/parisc/kernel/asm-offsets.c
index ec787b4..50353bd 100644
--- a/arch/parisc/kernel/asm-offsets.c
+++ b/arch/parisc/kernel/asm-offsets.c
@@ -290,5 +290,11 @@ int main(void)
 	BLANK();
 	DEFINE(ASM_PDC_RESULT_SIZE, NUM_PDC_RESULT * sizeof(unsigned long));
 	BLANK();
+
+#ifdef CONFIG_SMP
+	DEFINE(ASM_ATOMIC_HASH_SIZE_SHIFT, __builtin_ffs(ATOMIC_HASH_SIZE)-1);
+	DEFINE(ASM_ATOMIC_HASH_ENTRY_SHIFT, __builtin_ffs(sizeof(__atomic_hash[0]))-1);
+#endif
+
 	return 0;
 }
diff --git a/arch/parisc/kernel/setup.c b/arch/parisc/kernel/setup.c
index cb71f3d..878f42c 100644
--- a/arch/parisc/kernel/setup.c
+++ b/arch/parisc/kernel/setup.c
@@ -128,6 +131,14 @@ void __init setup_arch(char **cmdline_p)
 	printk(KERN_INFO "The 32-bit Kernel has started...\n");
 #endif
 
+	/* Consistency check on the size and alignments of our spinlocks */
+#ifdef CONFIG_SMP
+	BUILD_BUG_ON(sizeof(arch_spinlock_t) != __PA_LDCW_ALIGNMENT);
+	BUG_ON((unsigned long)&__atomic_hash[0] & (__PA_LDCW_ALIGNMENT-1));
+	BUG_ON((unsigned long)&__atomic_hash[1] & (__PA_LDCW_ALIGNMENT-1));
+#endif
+	BUILD_BUG_ON((1<<L1_CACHE_SHIFT) != L1_CACHE_BYTES);
+
 	pdc_console_init();
 
 #ifdef CONFIG_64BIT
diff --git a/arch/parisc/kernel/syscall.S b/arch/parisc/kernel/syscall.S
index f5f9602..1f12418 100644
--- a/arch/parisc/kernel/syscall.S
+++ b/arch/parisc/kernel/syscall.S
@@ -11,6 +11,7 @@
 #include <asm/unistd.h>
 #include <asm/errno.h>
 #include <asm/page.h>
+#include <asm/cache.h>
 #include <asm/psw.h>
 #include <asm/thread_info.h>
 #include <asm/assembly.h>
@@ -47,18 +48,17 @@ ENTRY(linux_gateway_page)
 	KILL_INSN
 	.endr
 
-	/* ADDRESS 0xb0 to 0xb4, lws uses 1 insns for entry */
+	/* ADDRESS 0xb0 to 0xb8, lws uses two insns for entry */
 	/* Light-weight-syscall entry must always be located at 0xb0 */
 	/* WARNING: Keep this number updated with table size changes */
 #define __NR_lws_entries (2)
 
 lws_entry:
-	/* Unconditional branch to lws_start, located on the 
-	   same gateway page */
-	b,n	lws_start
+	gate	lws_start, %r0		/* increase privilege */
+	depi	3, 31, 2, %r31		/* Ensure we return into user mode. */
 
-	/* Fill from 0xb4 to 0xe0 */
-	.rept 11
+	/* Fill from 0xb8 to 0xe0 */
+	.rept 10
 	KILL_INSN
 	.endr
 
@@ -423,9 +423,6 @@ tracesys_sigexit:
 
 	*********************************************************/
 lws_start:
-	/* Gate and ensure we return to userspace */
-	gate	.+8, %r0
-	depi	3, 31, 2, %r31	/* Ensure we return to userspace */
 
 #ifdef CONFIG_64BIT
 	/* FIXME: If we are a 64-bit kernel just
@@ -473,7 +470,7 @@ lws_exit:
 	/* now reset the lowest bit of sp if it was set */
 	xor	%r30,%r1,%r30
 #endif
-	be,n	0(%sr3, %r31)
+	be,n	0(%sr7, %r31)
 
 
 	
@@ -530,18 +527,17 @@ lws_compare_and_swap32:
 
 lws_compare_and_swap:
 #ifdef CONFIG_SMP
-	/* Load start of lock table */
-	ldil	L%lws_lock_start, %r20
-	ldo	R%lws_lock_start(%r20), %r28
+	/* Calculate lock table entry via ATOMIC_HASH(%r26) */
+	ldil	L%__atomic_user_hash, %r20
+	ldo	R%__atomic_user_hash(%r20), %r28
 
-	/* Extract four bits from r26 and hash lock (Bits 4-7) */
-	extru  %r26, 27, 4, %r20
+#ifdef CONFIG_64BIT
+	extrd,u %r26, 63-L1_CACHE_SHIFT, ASM_ATOMIC_HASH_SIZE_SHIFT, %r20
+#else
+	extru	%r26, 31-L1_CACHE_SHIFT, ASM_ATOMIC_HASH_SIZE_SHIFT, %r20
+#endif
+	shladd,l %r20, ASM_ATOMIC_HASH_ENTRY_SHIFT, %r28, %r20
 
-	/* Find lock to use, the hash is either one of 0 to
-	   15, multiplied by 16 (keep it 16-byte aligned)
-	   and add to the lock table offset. */
-	shlw	%r20, 4, %r20
-	add	%r20, %r28, %r20
 
 # if ENABLE_LWS_DEBUG
 	/*	
@@ -672,31 +668,6 @@ ENTRY(sys_call_table64)
 END(sys_call_table64)
 #endif
 
-#ifdef CONFIG_SMP
-	/*
-		All light-weight-syscall atomic operations 
-		will use this set of locks 
-
-		NOTE: The lws_lock_start symbol must be
-		at least 16-byte aligned for safe use
-		with ldcw.
-	*/
-	.section .data
-	.align	PAGE_SIZE
-ENTRY(lws_lock_start)
-	/* lws locks */
-	.rept 16
-	/* Keep locks aligned at 16-bytes */
-	.word 1
-	.word 0 
-	.word 0
-	.word 0
-	.endr
-END(lws_lock_start)
-	.previous
-#endif
-/* CONFIG_SMP for lws_lock_start */
-
 .end
 
 
diff --git a/arch/parisc/lib/bitops.c b/arch/parisc/lib/bitops.c
index 353963d..bae6a86 100644
--- a/arch/parisc/lib/bitops.c
+++ b/arch/parisc/lib/bitops.c
@@ -15,6 +15,9 @@
 arch_spinlock_t __atomic_hash[ATOMIC_HASH_SIZE] __lock_aligned = {
 	[0 ... (ATOMIC_HASH_SIZE-1)]  = __ARCH_SPIN_LOCK_UNLOCKED
 };
+arch_spinlock_t __atomic_user_hash[ATOMIC_HASH_SIZE] __lock_aligned = {
+	[0 ... (ATOMIC_HASH_SIZE-1)]  = __ARCH_SPIN_LOCK_UNLOCKED
+};
 #endif
 
 #ifdef CONFIG_64BIT
diff --git a/kernel/fork.c b/kernel/fork.c
index f88bd98..108b1ed 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -608,7 +608,10 @@ void mm_release(struct task_struct *tsk, struct mm_struct *mm)
 			 * We don't check the error code - if userspace has
 			 * not set up a proper pointer then tough luck.
 			 */
+			unsigned long flags;
+			_atomic_spin_lock_irqsave_user(tsk->clear_child_tid, flags);
 			put_user(0, tsk->clear_child_tid);
+			_atomic_spin_unlock_irqrestore_user(tsk->clear_child_tid, flags);
 			sys_futex(tsk->clear_child_tid, FUTEX_WAKE,
 					1, NULL, NULL, 0);
 		}
@@ -1432,8 +1435,12 @@ long do_fork(unsigned long clone_flags,
 
 		nr = task_pid_vnr(p);
 
-		if (clone_flags & CLONE_PARENT_SETTID)
+		if (clone_flags & CLONE_PARENT_SETTID) {
+			unsigned long flags;
+			_atomic_spin_lock_irqsave_user(parent_tidptr, flags);
 			put_user(nr, parent_tidptr);
+			_atomic_spin_unlock_irqrestore_user(parent_tidptr, flags);
+		}
 
 		if (clone_flags & CLONE_VFORK) {
 			p->vfork_done = &vfork;

Re: futex wait failure

[John David Anglin]

On Wed, 03 Feb 2010, Helge Deller wrote:

> On 02/03/2010 04:44 AM, John David Anglin wrote:
>> On Tue, 02 Feb 2010, Helge Deller wrote:
>>
>>> I wonder if we have some problems with the LWS code path in the kernel
>>> regarding atomic locking with futexes?
>>>
>>> In arch/parisc/kernel/syscall.S we use a lock table called 
>>> lws_lock_start[]
>>> to guard the LWS code against other competing userspace processes.
>>> I wonder if this really enough, esp. since we do implement futex syscalls
>>> (e.g. clone/exit calls uses futex functions to change userspace values
>>> because of CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID).
>>>
>>> Do we maybe need to protect the LWS code path with the same locking table
>>> as the
>>> generic kernel? Atomicity of futexes writing to userspace are not in sync
>>> with the locking of the LWS/lws_lock_start[] code.
>>>
>>> I tried to come up with a patch for that which I attached here, but sadly
>>> it hangs as soon as the init process is started on a 64bit/SMP kernel.
>>> So either my thinking here is stupid, or I do have a stupid coding bug.
>>>
>>> Furthermore, the coding for futex_atomic_op_inuser() in
>>> arch/parisc/include/asm/futex.h seems to miss real functionality.
>>> I didn't looked closer into this though.
>>
>> While locking may be a problem, it is not the main reason the minifail
>> program fails.  The program fails on my c3750 with a 32-bit UP kernel.
>
> Hmm, I can't reproduce it at the moment with a 32bit UP kernel on my
> c3000. The minifail3 test program I currently use (to avoid glibc issues)
> is attached.
>
>> There is no locking in this kernel.  The LWS code relies on the scheduler
>> for atomicity.
>
> yep. This should then work an a UP kernel.
>
>> I tried disabling interrupts around the crucial three instructions but
>> it didn't help.
>
> Ugh.
>
> Anyway, my current patch which compiles and runs fine is attached here as
> well. Maybe you want to try it on your SMP builds? It includes the 
> syscall.S
> changes you sent last time too.
> In this version of the patch I added on own LWS locking hash table for 
> user-space accesses,
> which is used in the LWS code and when some (probably not all) put_user()/
> get_user() calls are made in the futex code.
>
> Feedback still welcome.

I tried the patch below with 2.6.33 on three systems:

rp3440: 64-bit UP and SMP
A500-7X: 64-bit SMP
c3750: 32-bit UP

libc6 is 2.10.2-6.

As usual, I tested by trying to build and check GCC.

The most common problem on the rp3440 SMP build is a segmentation fault
in /bin/sh.  Twice it dropped core running configure near the start of
the build.  Restarting the build, the system managed to complete a full
GCC build, but the gcc and acats testsuites aborted.  Here's the backtrace
for the acats one:

Core was generated by `/bin/bash /home/dave/gnu/gcc/gcc/gcc/testsuite/ada/acats/run_all.sh'.
Program terminated with signal 11, Segmentation fault.
#0  0x00000008 in ?? ()
(gdb) bt
#0  0x00000008 in ?? ()
#1  0x0004ffd0 in stop_pipeline ()
#2  0x0003daf8 in execute_command_internal ()
#3  0x0003e3a8 in execute_command_internal ()
#4  0x0003e74c in execute_command ()
#5  0x0003e364 in execute_command_internal ()
#6  0x0003e74c in execute_command ()
#7  0x0003e364 in execute_command_internal ()
#8  0x0003e74c in execute_command ()
#9  0x0003e0a0 in execute_command_internal ()
#10 0x0003e3a8 in execute_command_internal ()
#11 0x0003e74c in execute_command ()
#12 0x0003e0a0 in execute_command_internal ()
#13 0x0003e74c in execute_command ()
#14 0x00029ac0 in reader_loop ()
#15 0x00029100 in main ()

I ran the UP build for several days without any major problems.  The most
common problem is tests randomly timeout.  I'm not sure whether the tests
actually timeout or not, but everyone that I've looked at shouldn't timeout
based on execution time.

I had pretty much the same experience on the A500-7X although I haven't
seen /bin/sh dump core running configure.

The c3750 completed a full GCC build but there were again problems running
the testsuite.  For example,

Core was generated by `expect -- /usr/share/dejagnu/runtest.exp --tool gcc'.
Program terminated with signal 11, Segmentation fault.
#0  0x409fa258 in ?? () from /usr/lib/libtcl8.5.so.0
(gdb) bt
#0  0x409fa258 in ?? () from /usr/lib/libtcl8.5.so.0
#1  0x402600a0 in start_thread (arg=0x416a4480) at pthread_create.c:302
#2  0x40e25874 in clone ()
    at ../ports/sysdeps/unix/sysv/linux/hppa/nptl/../clone.S:166
#3  0x00000000 in ?? ()

This looks like the minifail clone bug.  On the otherhand, minifail doesn't
seem to be failing.

This is the assembly code:

(gdb) disass 0x409fa248 0x409fa268
Dump of assembler code from 0x409fa248 to 0x409fa268:
0x409fa248:     copy r18,ret0
0x409fa24c:     stw r0,0(ret0)
0x409fa250:     ldo 4(ret0),ret0
0x409fa254:     cmpb,<>,n r7,ret0,0x409fa250
0x409fa258:     stw r0,0(ret0)
0x409fa25c:     copy r7,ret0
0x409fa260:     stw r0,0(ret0)
0x409fa264:     ldo 4(ret0),ret0

This may be the register.  There are several dumps in the log at the same spot.

Mar  7 11:48:18 hiauly6 kernel: do_page_fault() pid=3506 command='expect' type=1
5 address=0x416a3000
Mar  7 11:48:18 hiauly6 kernel: vm_start = 0x416a3000, vm_end = 0x416a4000
Mar  7 11:48:18 hiauly6 kernel: 
Mar  7 11:48:18 hiauly6 kernel:      YZrvWESTHLNXBCVMcbcbcbcbOGFRQPDI
Mar  7 11:48:18 hiauly6 kernel: PSW: 00000000000011101111111100001111 Not tainte
d
Mar  7 11:48:18 hiauly6 kernel: r00-03  000eff0f 40276138 409fa4ff 00000000
Mar  7 11:48:18 hiauly6 kernel: r04-07  40a252f4 40ea5208 0001ccb8 00000000
Mar  7 11:48:18 hiauly6 kernel: r08-11  40a252f4 40ea539c 00000003 40a2990c
Mar  7 11:48:18 hiauly6 kernel: r12-15  40ea521c 40a29904 40ea5214 00000001
Mar  7 11:48:18 hiauly6 kernel: r16-19  00000008 40a232d8 40ea521c 40a252f4
Mar  7 11:48:18 hiauly6 kernel: r20-23  00000001 00000000 402636cc 00000000
Mar  7 11:48:18 hiauly6 kernel: r24-27  fffffff5 ffffffd3 0001c538 000121ac
Mar  7 11:48:18 hiauly6 kernel: r28-31  416a3000 00000000 40ea5440 40263733
Mar  7 11:48:18 hiauly6 kernel: sr00-03  00000026 00000000 00000000 00000026
Mar  7 11:48:18 hiauly6 kernel: sr04-07  00000026 00000026 00000026 00000026
Mar  7 11:48:18 hiauly6 kernel: 
Mar  7 11:48:18 hiauly6 kernel:       VZOUICununcqcqcqcqcqcrmunTDVZOUI
Mar  7 11:48:18 hiauly6 kernel: FPSR: 00001100001110000000000000000000
Mar  7 11:48:18 hiauly6 kernel: FPER1: 00000000
Mar  7 11:48:18 hiauly6 kernel: fr00-03  0c38000000000000 0000000000000000 00000
00000000000 0000000000000000
Mar  7 11:48:18 hiauly6 kernel: fr04-07  0000000000000001 0370000000000000 0000000100000000 bff0000000000000
Mar  7 11:48:18 hiauly6 kernel: fr08-11  000000024f813bc4 0000000000000000 4f813bc04f813bc8 0000800100000003
Mar  7 11:48:18 hiauly6 kernel: fr12-15  000000004facfd40 000000001019db40 4f81169cf0000174 f000017cf0000884
Mar  7 11:48:18 hiauly6 kernel: fr16-19  1044111810590240 1044111800000081 f000000000000000 0000000000000000
Mar  7 11:48:18 hiauly6 kernel: fr20-23  00000002ffffff9c fffff0004f1a4000 0000000000000088 400026c000000000
Mar  7 11:48:18 hiauly6 kernel: fr24-27  01f2603af000017c f000017400000001 105a928c00000000 4fb50230ffffffff
Mar  7 11:48:18 hiauly6 kernel: fr28-31  ffffffff00002b67 105a92881019487c 0000000100000000 4f82820000000800
Mar  7 11:48:18 hiauly6 kernel: 
Mar  7 11:48:18 hiauly6 kernel: IASQ: 00000026 00000026 IAOQ: 409fa25b 409fa253
Mar  7 11:48:18 hiauly6 kernel:  IIR: 0f801280    ISR: 00000026  IOR: 416a3000
Mar  7 11:48:18 hiauly6 kernel:  CPU:        0   CR30: 3c014000 CR31: ffffffff
Mar  7 11:48:18 hiauly6 kernel:  ORIG_R28: 00000080
Mar  7 11:48:18 hiauly6 kernel:  IAOQ[0]: 409fa25b
Mar  7 11:48:18 hiauly6 kernel:  IAOQ[1]: 409fa253
Mar  7 11:48:18 hiauly6 kernel:  RP(r2): 409fa4ff

The address appears valid.  This is a TLB miss exception (15).

> diff --git a/arch/parisc/include/asm/atomic.h b/arch/parisc/include/asm/atomic.h
> index 716634d..18d57c8 100644
> --- a/arch/parisc/include/asm/atomic.h
> +++ b/arch/parisc/include/asm/atomic.h
> @@ -24,29 +24,46 @@
>   * Hash function to index into a different SPINLOCK.
>   * Since "a" is usually an address, use one spinlock per cacheline.
>   */
> -#  define ATOMIC_HASH_SIZE 4
> -#  define ATOMIC_HASH(a) (&(__atomic_hash[ (((unsigned long) (a))/L1_CACHE_BYTES) & (ATOMIC_HASH_SIZE-1) ]))
> +#  define ATOMIC_HASH_SIZE (4096/L1_CACHE_BYTES)  /* 4 */
> +#  define ATOMIC_HASH(a)      (&(__atomic_hash[ (((unsigned long) (a))/L1_CACHE_BYTES) & (ATOMIC_HASH_SIZE-1) ]))
> +#  define ATOMIC_USER_HASH(a) (&(__atomic_user_hash[ (((unsigned long) (a))/L1_CACHE_BYTES) & (ATOMIC_HASH_SIZE-1) ]))
>  
>  extern arch_spinlock_t __atomic_hash[ATOMIC_HASH_SIZE] __lock_aligned;
> +extern arch_spinlock_t __atomic_user_hash[ATOMIC_HASH_SIZE] __lock_aligned;
>  
>  /* Can't use raw_spin_lock_irq because of #include problems, so
>   * this is the substitute */
> -#define _atomic_spin_lock_irqsave(l,f) do {	\
> -	arch_spinlock_t *s = ATOMIC_HASH(l);		\
> +#define _atomic_spin_lock_irqsave_template(l,f,hash_func) do {	\
> +	arch_spinlock_t *s = hash_func;		\
>  	local_irq_save(f);			\
>  	arch_spin_lock(s);			\
>  } while(0)
>  
> -#define _atomic_spin_unlock_irqrestore(l,f) do {	\
> -	arch_spinlock_t *s = ATOMIC_HASH(l);			\
> +#define _atomic_spin_unlock_irqrestore_template(l,f,hash_func) do {	\
> +	arch_spinlock_t *s = hash_func;			\
>  	arch_spin_unlock(s);				\
>  	local_irq_restore(f);				\
>  } while(0)
>  
> +/* kernel memory locks */
> +#define _atomic_spin_lock_irqsave(l,f)	\
> +	_atomic_spin_lock_irqsave_template(l,f,ATOMIC_HASH(l))
> +
> +#define _atomic_spin_unlock_irqrestore(l,f)	\
> +	_atomic_spin_unlock_irqrestore_template(l,f,ATOMIC_HASH(l))
> +
> +/* userspace memory locks */
> +#define _atomic_spin_lock_irqsave_user(l,f)	\
> +	_atomic_spin_lock_irqsave_template(l,f,ATOMIC_USER_HASH(l))
> +
> +#define _atomic_spin_unlock_irqrestore_user(l,f)	\
> +	_atomic_spin_unlock_irqrestore_template(l,f,ATOMIC_USER_HASH(l))
>  
>  #else
>  #  define _atomic_spin_lock_irqsave(l,f) do { local_irq_save(f); } while (0)
>  #  define _atomic_spin_unlock_irqrestore(l,f) do { local_irq_restore(f); } while (0)
> +#  define _atomic_spin_lock_irqsave_user(l,f) _atomic_spin_lock_irqsave(l,f)
> +#  define _atomic_spin_unlock_irqrestore_user(l,f) _atomic_spin_lock_irqsave_user(l,f)
>  #endif
>  
>  /* This should get optimized out since it's never called.
> diff --git a/arch/parisc/include/asm/futex.h b/arch/parisc/include/asm/futex.h
> index 0c705c3..7bc963e 100644
> --- a/arch/parisc/include/asm/futex.h
> +++ b/arch/parisc/include/asm/futex.h
> @@ -55,6 +55,7 @@ futex_atomic_cmpxchg_inatomic(int __user *uaddr, int oldval, int newval)
>  {
>  	int err = 0;
>  	int uval;
> +	unsigned long flags;
>  
>  	/* futex.c wants to do a cmpxchg_inatomic on kernel NULL, which is
>  	 * our gateway page, and causes no end of trouble...
> @@ -65,10 +66,15 @@ futex_atomic_cmpxchg_inatomic(int __user *uaddr, int oldval, int newval)
>  	if (!access_ok(VERIFY_WRITE, uaddr, sizeof(int)))
>  		return -EFAULT;
>  
> +	_atomic_spin_lock_irqsave_user(uaddr, flags);
> +
>  	err = get_user(uval, uaddr);
> -	if (err) return -EFAULT;
> -	if (uval == oldval)
> -		err = put_user(newval, uaddr);
> +	if (!err)
> +		if (uval == oldval)
> +			err = put_user(newval, uaddr);
> +
> +	_atomic_spin_unlock_irqrestore_user(uaddr, flags);
> +
>  	if (err) return -EFAULT;
>  	return uval;
>  }
> diff --git a/arch/parisc/include/asm/system.h b/arch/parisc/include/asm/system.h
> index d91357b..4653c77 100644
> --- a/arch/parisc/include/asm/system.h
> +++ b/arch/parisc/include/asm/system.h
> @@ -160,7 +160,7 @@ static inline void set_eiem(unsigned long val)
>     ldcd). */
>  
>  #define __PA_LDCW_ALIGNMENT	4
> -#define __ldcw_align(a) ((volatile unsigned int *)a)
> +#define __ldcw_align(a) (&(a)->slock)
>  #define __LDCW	"ldcw,co"
>  
>  #endif /*!CONFIG_PA20*/
> diff --git a/arch/parisc/kernel/asm-offsets.c b/arch/parisc/kernel/asm-offsets.c
> index ec787b4..50353bd 100644
> --- a/arch/parisc/kernel/asm-offsets.c
> +++ b/arch/parisc/kernel/asm-offsets.c
> @@ -290,5 +290,11 @@ int main(void)
>  	BLANK();
>  	DEFINE(ASM_PDC_RESULT_SIZE, NUM_PDC_RESULT * sizeof(unsigned long));
>  	BLANK();
> +
> +#ifdef CONFIG_SMP
> +	DEFINE(ASM_ATOMIC_HASH_SIZE_SHIFT, __builtin_ffs(ATOMIC_HASH_SIZE)-1);
> +	DEFINE(ASM_ATOMIC_HASH_ENTRY_SHIFT, __builtin_ffs(sizeof(__atomic_hash[0]))-1);
> +#endif
> +
>  	return 0;
>  }
> diff --git a/arch/parisc/kernel/setup.c b/arch/parisc/kernel/setup.c
> index cb71f3d..878f42c 100644
> --- a/arch/parisc/kernel/setup.c
> +++ b/arch/parisc/kernel/setup.c
> @@ -128,6 +131,14 @@ void __init setup_arch(char **cmdline_p)
>  	printk(KERN_INFO "The 32-bit Kernel has started...\n");
>  #endif
>  
> +	/* Consistency check on the size and alignments of our spinlocks */
> +#ifdef CONFIG_SMP
> +	BUILD_BUG_ON(sizeof(arch_spinlock_t) != __PA_LDCW_ALIGNMENT);
> +	BUG_ON((unsigned long)&__atomic_hash[0] & (__PA_LDCW_ALIGNMENT-1));
> +	BUG_ON((unsigned long)&__atomic_hash[1] & (__PA_LDCW_ALIGNMENT-1));
> +#endif
> +	BUILD_BUG_ON((1<<L1_CACHE_SHIFT) != L1_CACHE_BYTES);
> +
>  	pdc_console_init();
>  
>  #ifdef CONFIG_64BIT
> diff --git a/arch/parisc/kernel/syscall.S b/arch/parisc/kernel/syscall.S
> index f5f9602..1f12418 100644
> --- a/arch/parisc/kernel/syscall.S
> +++ b/arch/parisc/kernel/syscall.S
> @@ -11,6 +11,7 @@
>  #include <asm/unistd.h>
>  #include <asm/errno.h>
>  #include <asm/page.h>
> +#include <asm/cache.h>
>  #include <asm/psw.h>
>  #include <asm/thread_info.h>
>  #include <asm/assembly.h>
> @@ -47,18 +48,17 @@ ENTRY(linux_gateway_page)
>  	KILL_INSN
>  	.endr
>  
> -	/* ADDRESS 0xb0 to 0xb4, lws uses 1 insns for entry */
> +	/* ADDRESS 0xb0 to 0xb8, lws uses two insns for entry */
>  	/* Light-weight-syscall entry must always be located at 0xb0 */
>  	/* WARNING: Keep this number updated with table size changes */
>  #define __NR_lws_entries (2)
>  
>  lws_entry:
> -	/* Unconditional branch to lws_start, located on the 
> -	   same gateway page */
> -	b,n	lws_start
> +	gate	lws_start, %r0		/* increase privilege */
> +	depi	3, 31, 2, %r31		/* Ensure we return into user mode. */
>  
> -	/* Fill from 0xb4 to 0xe0 */
> -	.rept 11
> +	/* Fill from 0xb8 to 0xe0 */
> +	.rept 10
>  	KILL_INSN
>  	.endr
>  
> @@ -423,9 +423,6 @@ tracesys_sigexit:
>  
>  	*********************************************************/
>  lws_start:
> -	/* Gate and ensure we return to userspace */
> -	gate	.+8, %r0
> -	depi	3, 31, 2, %r31	/* Ensure we return to userspace */
>  
>  #ifdef CONFIG_64BIT
>  	/* FIXME: If we are a 64-bit kernel just
> @@ -473,7 +470,7 @@ lws_exit:
>  	/* now reset the lowest bit of sp if it was set */
>  	xor	%r30,%r1,%r30
>  #endif
> -	be,n	0(%sr3, %r31)
> +	be,n	0(%sr7, %r31)
>  
>  
>  	
> @@ -530,18 +527,17 @@ lws_compare_and_swap32:
>  
>  lws_compare_and_swap:
>  #ifdef CONFIG_SMP
> -	/* Load start of lock table */
> -	ldil	L%lws_lock_start, %r20
> -	ldo	R%lws_lock_start(%r20), %r28
> +	/* Calculate lock table entry via ATOMIC_HASH(%r26) */
> +	ldil	L%__atomic_user_hash, %r20
> +	ldo	R%__atomic_user_hash(%r20), %r28
>  
> -	/* Extract four bits from r26 and hash lock (Bits 4-7) */
> -	extru  %r26, 27, 4, %r20
> +#ifdef CONFIG_64BIT
> +	extrd,u %r26, 63-L1_CACHE_SHIFT, ASM_ATOMIC_HASH_SIZE_SHIFT, %r20
> +#else
> +	extru	%r26, 31-L1_CACHE_SHIFT, ASM_ATOMIC_HASH_SIZE_SHIFT, %r20
> +#endif
> +	shladd,l %r20, ASM_ATOMIC_HASH_ENTRY_SHIFT, %r28, %r20
>  
> -	/* Find lock to use, the hash is either one of 0 to
> -	   15, multiplied by 16 (keep it 16-byte aligned)
> -	   and add to the lock table offset. */
> -	shlw	%r20, 4, %r20
> -	add	%r20, %r28, %r20
>  
>  # if ENABLE_LWS_DEBUG
>  	/*	
> @@ -672,31 +668,6 @@ ENTRY(sys_call_table64)
>  END(sys_call_table64)
>  #endif
>  
> -#ifdef CONFIG_SMP
> -	/*
> -		All light-weight-syscall atomic operations 
> -		will use this set of locks 
> -
> -		NOTE: The lws_lock_start symbol must be
> -		at least 16-byte aligned for safe use
> -		with ldcw.
> -	*/
> -	.section .data
> -	.align	PAGE_SIZE
> -ENTRY(lws_lock_start)
> -	/* lws locks */
> -	.rept 16
> -	/* Keep locks aligned at 16-bytes */
> -	.word 1
> -	.word 0 
> -	.word 0
> -	.word 0
> -	.endr
> -END(lws_lock_start)
> -	.previous
> -#endif
> -/* CONFIG_SMP for lws_lock_start */
> -
>  .end
>  
>  
> diff --git a/arch/parisc/lib/bitops.c b/arch/parisc/lib/bitops.c
> index 353963d..bae6a86 100644
> --- a/arch/parisc/lib/bitops.c
> +++ b/arch/parisc/lib/bitops.c
> @@ -15,6 +15,9 @@
>  arch_spinlock_t __atomic_hash[ATOMIC_HASH_SIZE] __lock_aligned = {
>  	[0 ... (ATOMIC_HASH_SIZE-1)]  = __ARCH_SPIN_LOCK_UNLOCKED
>  };
> +arch_spinlock_t __atomic_user_hash[ATOMIC_HASH_SIZE] __lock_aligned = {
> +	[0 ... (ATOMIC_HASH_SIZE-1)]  = __ARCH_SPIN_LOCK_UNLOCKED
> +};
>  #endif
>  
>  #ifdef CONFIG_64BIT
> diff --git a/kernel/fork.c b/kernel/fork.c
> index f88bd98..108b1ed 100644
> --- a/kernel/fork.c
> +++ b/kernel/fork.c
> @@ -608,7 +608,10 @@ void mm_release(struct task_struct *tsk, struct mm_struct *mm)
>  			 * We don't check the error code - if userspace has
>  			 * not set up a proper pointer then tough luck.
>  			 */
> +			unsigned long flags;
> +			_atomic_spin_lock_irqsave_user(tsk->clear_child_tid, flags);
>  			put_user(0, tsk->clear_child_tid);
> +			_atomic_spin_unlock_irqrestore_user(tsk->clear_child_tid, flags);
>  			sys_futex(tsk->clear_child_tid, FUTEX_WAKE,
>  					1, NULL, NULL, 0);
>  		}
> @@ -1432,8 +1435,12 @@ long do_fork(unsigned long clone_flags,
>  
>  		nr = task_pid_vnr(p);
>  
> -		if (clone_flags & CLONE_PARENT_SETTID)
> +		if (clone_flags & CLONE_PARENT_SETTID) {
> +			unsigned long flags;
> +			_atomic_spin_lock_irqsave_user(parent_tidptr, flags);
>  			put_user(nr, parent_tidptr);
> +			_atomic_spin_unlock_irqrestore_user(parent_tidptr, flags);
> +		}
>  
>  		if (clone_flags & CLONE_VFORK) {
>  			p->vfork_done = &vfork;


-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

> The c3750 completed a full GCC build but there were again problems running
> the testsuite.  For example,
> 
> Core was generated by `expect -- /usr/share/dejagnu/runtest.exp --tool gcc'.
> Program terminated with signal 11, Segmentation fault.
> #0  0x409fa258 in ?? () from /usr/lib/libtcl8.5.so.0
> (gdb) bt
> #0  0x409fa258 in ?? () from /usr/lib/libtcl8.5.so.0
> #1  0x402600a0 in start_thread (arg=0x416a4480) at pthread_create.c:302
> #2  0x40e25874 in clone ()
>     at ../ports/sysdeps/unix/sysv/linux/hppa/nptl/../clone.S:166
> #3  0x00000000 in ?? ()
> 
> This looks like the minifail clone bug.  On the otherhand, minifail doesn't
> seem to be failing.

I reverted Helge's patch and still get the same fault, so it didn't
introduce the problem.

Prior to 2.6.33, I was running 2.6.31.12 with essentially the same config.
The libc6 update to 2.10.2-6 changed things on the c3750.  Prior to the
update, expect would hang several times running the acats testsuite.  After
the update, this was fixed.  However, the above problem was introduced,
or became more frequent.

I have the feeling that Helge's patch helps (possible resolves) the
minifail bug.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

On Sun, 07 Mar 2010, John David Anglin wrote:

> I have the feeling that Helge's patch helps (possible resolves) the
> minifail bug.

With further testing, I find that the patch doesn't resolve the minifail
fails on my c3750.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Kyle McMartin]

On Wed, Mar 10, 2010 at 10:20:49PM -0500, John David Anglin wrote:
> > I have the feeling that Helge's patch helps (possible resolves) the
> > minifail bug.
> 
> With further testing, I find that the patch doesn't resolve the minifail
> fails on my c3750.
> 

Bummer. :(

Re: futex wait failure

[John David Anglin]

On Thu, 11 Mar 2010, Kyle McMartin wrote:

> On Wed, Mar 10, 2010 at 10:20:49PM -0500, John David Anglin wrote:
> > > I have the feeling that Helge's patch helps (possible resolves) the
> > > minifail bug.
> > 
> > With further testing, I find that the patch doesn't resolve the minifail
> > fails on my c3750.
> > 
> 
> Bummer. :(

Yes.

I am 95% certain that the bug has to do with the clone implementation
and scheduling.  I have attached another version of the minifail test.
The loops after the fork and pthread_create calls are to isolate these
syscalls.  Typically, I see something like:

Core was generated by `./minifail9s'.
Program terminated with signal 11, Segmentation fault.
#0  0x00000000 in ?? ()
(gdb) info thread
  2 Thread 11537  pure_test () at minifail9.cpp:51
* 1 Thread 11538  0x00000000 in ?? ()

Thread 2 is in the pure_test loop after the fork call.

The cause of the failure is not at all obvious to me but I am a
bit concerned about the sys_clone implementation.  Is it possible
that the struct pt_regs gets clobbered?

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

[-- Attachment #1: Type: text/plain, Size: 488 bytes --]

On Thu, 11 Mar 2010, John David Anglin wrote:

> I am 95% certain that the bug has to do with the clone implementation
> and scheduling.  I have attached another version of the minifail test.
> The loops after the fork and pthread_create calls are to isolate these
> syscalls.  Typically, I see something like:

Attached.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

[-- Attachment #2: minifail9.cpp --]
[-- Type: text/plain, Size: 1352 bytes --]

#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>

/*
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=561203

  clone(child_stack=0x4088d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x4108c4e8, tls=0x4108c900, child_tidptr=0x4108c4e8) = 14819
[pid 14819] set_robust_list(0x4108c4f0, 0xc) = 0
[pid 14818] clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x40002028) = 14820

 g++  minifail.cpp -o minifail -O0 -pthread -g

 i=0; while true; do i=$(($i+1)); echo Run $i; ./minifail; done;

 */

static volatile int run = 1;

void* thread_run(void* arg) {
	static long status;
	int i;
        
	run = 1;
	pthread_yield();
	for (i = 10000000; i; i--)
	   continue;
	write(1,"Thread OK.\n",11);
	return (void *)&status;
}

int pure_test() {
	int i;
	pthread_t thread;
	pthread_create(&thread, NULL, thread_run, NULL);

	while (!run)
	  pthread_yield();
	switch (fork()) {
		case -1:
			perror("fork() failed");
		case 0:
			write(1,"Child OK.\n",10);
			_exit(0);
		default:
			break;
		
	}
	pthread_yield();
	for (i = 20000000; i; i--)
	  continue;
	write(1,"Fork done\n",10);
	pthread_join(thread, NULL);
	return 0;
}

int main(int argc, char** argv) {
	return pure_test();
}

Re: futex wait failure

[John David Anglin]

On Thu, 11 Mar 2010, John David Anglin wrote:

> I am 95% certain that the bug has to do with the clone implementation
> and scheduling.  I have attached another version of the minifail test.
> The loops after the fork and pthread_create calls are to isolate these
> syscalls.  Typically, I see something like:

Here's another idea.  In entry.S, there are some tricky tests to determine
whether to do signals and reschedule.  Based on testing, I'm not sure
that cr30 is always valid (e.g., external interrupt at boot) when we get to
intr_check_resched and intr_check_sig.

The other issue is r16.  I'm thinking we may need to reload r16 as the
context may change if we schedule, etc.  If this checking is broken, it
would break the LWS assumptions.

Thoughts?

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

On Fri, 12 Mar 2010, John David Anglin wrote:

> The other issue is r16.  I'm thinking we may need to reload r16 as the
> context may change if we schedule, etc.  If this checking is broken, it
> would break the LWS assumptions.

This theory is shotdown.  The patch below possibly helps a bit but it
doesn't resolve the minifail bug.  The intention of the last couple of
hunks is to return to the right place if cr30 changes as a result of
scheduling.

With more debugging, it seems that this is a clone mmap copy issue.
Normally, the forked child doesn't see any non-zero data in the
stack region allocated by pthread_create.  However, if it does, 
the thread invariably dies and the stack region is corrupt after
the fork.

I guess I said this before ;(

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

diff --git a/arch/parisc/kernel/entry.S b/arch/parisc/kernel/entry.S
index 3a44f7f..820d6ca 100644
--- a/arch/parisc/kernel/entry.S
+++ b/arch/parisc/kernel/entry.S
@@ -758,6 +758,10 @@ ENTRY(__kernel_thread)
 
 	STREG	%r22, PT_GR22(%r1)	/* save r22 (arg5) */
 	copy	%r0, %r22		/* user_tid */
+	copy	%r0, %r21		/* child_tid */
+#else
+	stw	%r0, -52(%r30)	     	/* user_tid */
+	stw	%r0, -56(%r30)	     	/* child_tid */
 #endif
 	STREG	%r26, PT_GR26(%r1)  /* Store function & argument for child */
 	STREG	%r25, PT_GR25(%r1)
@@ -765,7 +769,7 @@ ENTRY(__kernel_thread)
 	ldo	CLONE_VM(%r26), %r26   /* Force CLONE_VM since only init_mm */
 	or	%r26, %r24, %r26      /* will have kernel mappings.	 */
 	ldi	1, %r25			/* stack_start, signals kernel thread */
-	stw	%r0, -52(%r30)	     	/* user_tid */
+	ldi	0, %r23			/* child_stack_size */
 #ifdef CONFIG_64BIT
 	ldo	-16(%r30),%r29		/* Reference param save area */
 #endif
@@ -972,7 +976,10 @@ intr_check_sig:
 	BL	do_notify_resume,%r2
 	copy	%r16, %r26			/* struct pt_regs *regs */
 
-	b,n	intr_check_sig
+	mfctl   %cr30,%r16		/* Reload */
+	LDREG	TI_TASK(%r16), %r16	/* thread_info -> task_struct */
+	b	intr_check_sig
+	ldo	TASK_REGS(%r16),%r16
 
 intr_restore:
 	copy            %r16,%r29
@@ -1026,14 +1033,12 @@ intr_do_resched:
 	ldo	-16(%r30),%r29		/* Reference param save area */
 #endif
 
-	ldil	L%intr_check_sig, %r2
-#ifndef CONFIG_64BIT
-	b	schedule
-#else
-	load32	schedule, %r20
-	bv	%r0(%r20)
-#endif
-	ldo	R%intr_check_sig(%r2), %r2
+	BL	schedule,%r2
+	nop
+	mfctl   %cr30,%r16		/* Reload */
+	LDREG	TI_TASK(%r16), %r16	/* thread_info -> task_struct */
+	b	intr_check_sig
+	ldo	TASK_REGS(%r16),%r16
 
 	/* preempt the current task on returning to kernel
 	 * mode from an interrupt, iff need_resched is set,

Re: futex wait failure

[Carlos O'Donell]

On Sun, Mar 14, 2010 at 9:10 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
> On Fri, 12 Mar 2010, John David Anglin wrote:
>
>> The other issue is r16. =A0I'm thinking we may need to reload r16 as=
 the
>> context may change if we schedule, etc. =A0If this checking is broke=
n, it
>> would break the LWS assumptions.
>
> This theory is shotdown. =A0The patch below possibly helps a bit but =
it
> doesn't resolve the minifail bug. =A0The intention of the last couple=
 of
> hunks is to return to the right place if cr30 changes as a result of
> scheduling.

Thanks for testing this out.

> With more debugging, it seems that this is a clone mmap copy issue.
> Normally, the forked child doesn't see any non-zero data in the
> stack region allocated by pthread_create. =A0However, if it does,
> the thread invariably dies and the stack region is corrupt after
> the fork.
>
> I guess I said this before ;(

After some of my own testing I think this is all MMU related, but I
can't prove it yet. I'm pouring through as much kernel code as I can
right now to determine what is going wrong at the time of the clone,
and I see at least one bug that I'm investigating regarding return
addresses.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[John David Anglin]

[-- Attachment #1: Type: text/plain, Size: 2064 bytes --]

On Tue, 16 Mar 2010, Carlos O'Donell wrote:

> After some of my own testing I think this is all MMU related, but I
> can't prove it yet. I'm pouring through as much kernel code as I can
> right now to determine what is going wrong at the time of the clone,
> and I see at least one bug that I'm investigating regarding return
> addresses.

I've attached another version of the minifail test program.  In
this one, the parent and thread both monitor the location 0x4000001c
in the stack region allocated to the thread.  If a problem is
detected, they drop core with an illegal instruction.  If the
child of the fork sees a nonzero value in the above location
when the fork call returns, it sleeps for ten seconds.

When corruption occurs and core is dropped on my c3750 (UP 32-bit
kernel), both the parent and thread have undergone many iterations
of their respective monitor loops.  The forked child always reports
seeing a nonzero value at the stack location.  The before value
in the core dump was zero (i.e., thread_run had not started).

I added an illegal instruction abort to the child.  In this case,
the thread_run loop counter was 48085 when the page was copied and
the before value was zero.

One thought that has crossed my mind is that the memory pages allocated
for the stack region used by the thread are somehow getting interchanged
between parent and child by the fork operation.  This happens fairly
late as both the parent and thread are executing post fork at the time
this happens.  Possibly, this is part of the bug.

I have looked at entry.S and pacache.S quite a bit and it's not obvious
how this could happen, although I must admit to not fully understanding
the tmp alias code.  I tend to think the bug is in the core mm code.

I see a few cleanups to entry.S.  We didn't kill the misnamed macros
(DEP, DEPI and EXTR) for example.  But I don't think these are the problem.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

[-- Attachment #2: minifail12.cpp --]
[-- Type: text/plain, Size: 1817 bytes --]

#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>

/*
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=561203

  clone(child_stack=0x4088d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x4108c4e8, tls=0x4108c900, child_tidptr=0x4108c4e8) = 14819
[pid 14819] set_robust_list(0x4108c4f0, 0xc) = 0
[pid 14818] clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x40002028) = 14820

 g++  minifail.cpp -o minifail -O0 -pthread -g

 i=0; while true; do i=$(($i+1)); echo Run $i; ./minifail; done;

 */

int before = -1;

void* thread_run(void* arg) {
	static long status;
	int i;
        
	for (i = 1000000; i; i--)
	  if (*(volatile int *)0x4000001c != 0x407ff340)
	    asm ("iitlbp %r0,(%sr0,%r0)");
	return (void *)&status;
}

int pure_test() {
	int i, stat, x;
	int seen = 0;
	pthread_t thread;
	pthread_create(&thread, NULL, thread_run, NULL);

	before = *(volatile int *)0x4000001c;
	switch (fork()) {
		case -1:
			perror("fork() failed");
		case 0:
			/* x is non-zero iff start thread has saved
			     `args' onto stack in r26 slot.  */
			x = *(volatile int *)0x4000001c;
			if (x == 0x407ff340)
			  {
				sleep (10);
			  }
			printf ("c 0x%x\n", x);
			fflush (stdout);
			return 0;
		default:
			break;
		
	}
	for (i = 1000000; i; i--)
	  {
	    if (!seen && *(volatile int *)0x4000001c == 0x407ff340)
	      seen = i;
	    if (seen && *(volatile int *)0x4000001c != 0x407ff340)
	      asm ("iitlbp %r0,(%sr0,%r0)");
	  }
	printf ("p 0x%x\n", *(int *)0x4000001c);
	if (pthread_join(thread, NULL))
	  perror ("join");
	return 0;
}

int main(int argc, char** argv) {
	return pure_test();
}

Re: futex wait failure

[Carlos O'Donell]

On Sun, Mar 21, 2010 at 2:19 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
> I added an illegal instruction abort to the child. =A0In this case,
> the thread_run loop counter was 48085 when the page was copied and
> the before value was zero.
>
> One thought that has crossed my mind is that the memory pages allocat=
ed
> for the stack region used by the thread are somehow getting interchan=
ged
> between parent and child by the fork operation. =A0This happens fairl=
y
> late as both the parent and thread are executing post fork at the tim=
e
> this happens. =A0Possibly, this is part of the bug.
>
> I have looked at entry.S and pacache.S quite a bit and it's not obvio=
us
> how this could happen, although I must admit to not fully understandi=
ng
> the tmp alias code. =A0I tend to think the bug is in the core mm code=
=2E
>
> I see a few cleanups to entry.S. =A0We didn't kill the misnamed macro=
s
> (DEP, DEPI and EXTR) for example. =A0But I don't think these are the =
problem.

Dave,

Thank you for another test case. I agree, I also think the bug is in
the core mm code, which I am also trying to review and understand. I
haven't been able to make much progress in this area though. I'll keep
you updated when I spot something I think is wrong.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[Carlos O'Donell]

On Sun, Mar 21, 2010 at 2:19 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
> One thought that has crossed my mind is that the memory pages allocat=
ed
> for the stack region used by the thread are somehow getting interchan=
ged
> between parent and child by the fork operation. =A0This happens fairl=
y
> late as both the parent and thread are executing post fork at the tim=
e
> this happens. =A0Possibly, this is part of the bug.
>
> I have looked at entry.S and pacache.S quite a bit and it's not obvio=
us
> how this could happen, although I must admit to not fully understandi=
ng
> the tmp alias code. =A0I tend to think the bug is in the core mm code=
=2E
>
> I see a few cleanups to entry.S. =A0We didn't kill the misnamed macro=
s
> (DEP, DEPI and EXTR) for example. =A0But I don't think these are the =
problem.

I can't get minifail2 to fail, how are you choosing which stack
address to watch for corruption?

Are you looking at the child's new stack created by mmap?

Are you looking at the parent's stack?

I found one bug in the hppa vfork() function call, which in the case
of error, would fail to propagate the error correctly. Fixing that and
will check it in. It doesn't account for the vfork/execve problems
though, since in this case vfork completes but execve crashes.

I also have some patches for asm-offsets.c, and entry.S to comment
some of the code better. I'll send those to Kyle after cleanup.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[John David Anglin]

> I can't get minifail2 to fail, how are you choosing which stack
> address to watch for corruption?

The stack allocated by pthread_create is always placed at 0x40000000
in my runs.  The location that the last version which I sent monitors
is 0x4000001c.  This is the location where thread_run saves %r26
shortly after entry.

I'm compiling without optimization as indicated in the comment.  Usually,
I also link with -static since gdb is somewhat broken when debugging
shared libraries.  If the compile is optimized, then thread_run may
not save much on the stack and it may be harder to get it to fail.

The problem is timing dependent.  I'm a bit surprised that you can't
duplicate the problem as Helge could.  When I run the test loop, it
fails about one in thirty on my c3750.  I have a 250 Hz tick rate in
my config.  The fail rate is somewhat dependent on system load.

I added the monitor loops to try and pin down when the corruption occurs
but the test should fail if the thread's stack region gets corrupted
while thread_run executes.

Have you made any changes to libc which would affect the synchonization
of parent and child?  I think the problem might be fixed if the parent
and thread were prevented from executing until the fork is complete.
I still think the kernel is interchanging the pages used by the parent
and child for the mmap'd stack region.

> Are you looking at the child's new stack created by mmap?

We are looking at the area allocated by mmap in the call to pthread_create.
This is always where the corruption occurs in the subsequent fork call.

This area is used as the stack for the thread created by pthread_create.
When the problem occurs, it's the thread that generates the fault, never
the original parent.

> Are you looking at the parent's stack?

No.  However, the parent can see changes to the stack area used by the
thread created by the pthread_create call.  As I understand the situtation,
the child of the fork call should see a snapshot of this area as it was
at the time of the fork call.  fork is nominally supposed to be "atomic"
(see Open Group man page).

In both the parent and forked child, the first three words starting at
0x40000000 are always nonzero and correct.  That's why I think the pages
are getting interchanged (I also played with adding extra cache flushes
in pacache.S but that didn't change things).

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Carlos O'Donell]

On Tue, Feb 2, 2010 at 4:16 PM, Helge Deller <deller@gmx.de> wrote:
> Furthermore, the coding for futex_atomic_op_inuser() in
> arch/parisc/include/asm/futex.h seems to miss real functionality.
> I didn't looked closer into this though.

I have a fix for futex_atomic_op_inuser(), but it needs more testing.

Cheers,
Carlos.

Re: futex wait failure

[Helge Deller]

On 01/08/2010 05:37 PM, John David Anglin wrote:
>>
>> On 01/07/2010 12:33 AM, John David Anglin wrote:
>>>>> clone(Process 1684 attached (waiting for parent)
>>>>> Process 1684 resumed (parent 1683 ready)
>>>>> child_stack=0x4076d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x40f6c4e8, tls=0x40f6c900, child_tidptr=0x40f6c4e8) = 1684
>>>>
>>>> I noticed the tidptr for the fork may not be correct:
>>>>
>>>> clone(child_stack=0x40e87040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x416864e8, tls=0x41686900, child_tidptr=0x416864e8) = 31613
>>>> clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x40002028) = 31614
>>>>
>>>> I would have thought the value should have been the same as that in the
>>>> clone from the pthread_create call.
>>>
>>> It's possible that this is done intentionally...  The parent_tidptr
>>> is the one that's wrong in the first clone.
>
> I now think this probably is a glibc bug.  The kernel uses this value
> when the CLONE_PARENT_SETTID flag is passed.


Maybe we have a futex problem in glibc on hppa?
In glibc nptl/pthread_mutex_trylock.c we check the return value of a futex syscall against EWOULDBLOCK.
Since on parisc - in contrast to all other architectures - we have EWOULDBLOCK!=EAGAIN, we maybe missed a check?

Helge

Re: futex wait failure

[John David Anglin]

> Maybe we have a futex problem in glibc on hppa?
> In glibc nptl/pthread_mutex_trylock.c we check the return value of a futex syscall against EWOULDBLOCK.
> Since on parisc - in contrast to all other architectures - we have EWOULDBLOCK!=EAGAIN, we maybe missed a check?

That's a very interesting observation!  The manpage doesn't say
EWOULDBLOCK is a valid error return for FUTEX(2).  I wonder is there
are more slips like this.  It looks like the same bug is present in
nptl/sysdeps/unix/sysv/linux/sem_wait.c and
nptl/sysdeps/unix/sysv/linux/sem_timedwait.c.

This might explain my previous mail.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Carlos O'Donell]

On Fri, Jan 8, 2010 at 4:18 PM, Helge Deller <deller@gmx.de> wrote:
>> I now think this probably is a glibc bug. =A0The kernel uses this va=
lue
>> when the CLONE_PARENT_SETTID flag is passed.
>
>
> Maybe we have a futex problem in glibc on hppa?
> In glibc nptl/pthread_mutex_trylock.c we check the return value of a =
futex
> syscall against EWOULDBLOCK.
> Since on parisc - in contrast to all other architectures - we have
> EWOULDBLOCK!=3DEAGAIN, we maybe missed a check?

That's a bug. There are several kernel paths that could return EAGAIN
*or* EWOULDBLOCK via the FUTEX_TRYLOCK_PI futex operation.

However, I think Dave is on to something with the

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[Carlos O'Donell]

On Fri, Jan 8, 2010 at 4:44 PM, Carlos O'Donell <carlos@systemhalted.or=
g> wrote:
> On Fri, Jan 8, 2010 at 4:18 PM, Helge Deller <deller@gmx.de> wrote:
>>> I now think this probably is a glibc bug. =A0The kernel uses this v=
alue
>>> when the CLONE_PARENT_SETTID flag is passed.
>>
>>
>> Maybe we have a futex problem in glibc on hppa?
>> In glibc nptl/pthread_mutex_trylock.c we check the return value of a=
 futex
>> syscall against EWOULDBLOCK.
>> Since on parisc - in contrast to all other architectures - we have
>> EWOULDBLOCK!=3DEAGAIN, we maybe missed a check?
>
> That's a bug. There are several kernel paths that could return EAGAIN
> *or* EWOULDBLOCK via the FUTEX_TRYLOCK_PI futex operation.
>
> However, I think Dave is on to something with the

=2E..kernel issue.

We might have *several* outstanding bugs :-)

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[Kyle McMartin]

On Fri, Jan 08, 2010 at 04:44:51PM -0500, Carlos O'Donell wrote:
> >
> > However, I think Dave is on to something with the
> 
> ...kernel issue.
> 
> We might have *several* outstanding bugs :-)
> 

Hrm. We discussed this on IRC a while ago, I can't remember what we
decided was the best approach... there's probably many many dragons
here.

I wonder what would break if we set any EAGAIN returns to EWOULDBLOCK in
the exit path.

Re: futex wait failure

[John David Anglin]

> On Fri, Jan 08, 2010 at 04:44:51PM -0500, Carlos O'Donell wrote:
> > >
> > > However, I think Dave is on to something with the
> > 
> > ...kernel issue.
> > 
> > We might have *several* outstanding bugs :-)
> > 
> 
> Hrm. We discussed this on IRC a while ago, I can't remember what we
> decided was the best approach... there's probably many many dragons
> here.
> 
> I wonder what would break if we set any EAGAIN returns to EWOULDBLOCK in
> the exit path.

That seems reversed.  x86_64 translates the system's EWOULDBLOCK error
into EAGAIN in sysdeps/unix/x86_64/sysdep.S.  However, don't know if this
impacts the futex and sem_wait checks.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Kyle McMartin]

On Fri, Jan 08, 2010 at 05:28:58PM -0500, John David Anglin wrote:
> > On Fri, Jan 08, 2010 at 04:44:51PM -0500, Carlos O'Donell wrote:
> > > >
> > > > However, I think Dave is on to something with the
> > > 
> > > ...kernel issue.
> > > 
> > > We might have *several* outstanding bugs :-)
> > > 
> > 
> > Hrm. We discussed this on IRC a while ago, I can't remember what we
> > decided was the best approach... there's probably many many dragons
> > here.
> > 
> > I wonder what would break if we set any EAGAIN returns to EWOULDBLOCK in
> > the exit path.
> 
> That seems reversed.  x86_64 translates the system's EWOULDBLOCK error
> into EAGAIN in sysdeps/unix/x86_64/sysdep.S.  However, don't know if this
> impacts the futex and sem_wait checks.

Yeah, you're right.

regards, Kyle

Re: futex wait failure

[John David Anglin]

> On Fri, Jan 8, 2010 at 4:44 PM, Carlos O'Donell <carlos@systemhalted.org> w=
> rote:
> > On Fri, Jan 8, 2010 at 4:18 PM, Helge Deller <deller@gmx.de> wrote:
> >>> I now think this probably is a glibc bug. =A0The kernel uses this value
> >>> when the CLONE_PARENT_SETTID flag is passed.
> >>
> >>
> >> Maybe we have a futex problem in glibc on hppa?
> >> In glibc nptl/pthread_mutex_trylock.c we check the return value of a fut=
> ex
> >> syscall against EWOULDBLOCK.
> >> Since on parisc - in contrast to all other architectures - we have
> >> EWOULDBLOCK!=3DEAGAIN, we maybe missed a check?
> >
> > That's a bug. There are several kernel paths that could return EAGAIN
> > *or* EWOULDBLOCK via the FUTEX_TRYLOCK_PI futex operation.

Hacking glibc to check for both does not fix the testcase ;(

> > However, I think Dave is on to something with the
> 
> ...kernel issue.
> 
> We might have *several* outstanding bugs :-)

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Helge Deller]

On 01/08/2010 10:44 PM, Carlos O'Donell wrote:
> On Fri, Jan 8, 2010 at 4:18 PM, Helge Deller<deller@gmx.de>  wrote:
>>> I now think this probably is a glibc bug.  The kernel uses this value
>>> when the CLONE_PARENT_SETTID flag is passed.
>>
>>
>> Maybe we have a futex problem in glibc on hppa?
>> In glibc nptl/pthread_mutex_trylock.c we check the return value of a futex
>> syscall against EWOULDBLOCK.
>> Since on parisc - in contrast to all other architectures - we have
>> EWOULDBLOCK!=EAGAIN, we maybe missed a check?
>
> That's a bug. [...]

Carlos,  Dave,

I'm wondering if we have another bug in glibc...?

When running strace, I noticed that a thread which returns with "return 0",
exits the process on hppa with exit(0):
[pid  1875] write(1, "Thread OK.\n", 11) = 11
[pid  1875] exit(0)

On my x86_64 I see:
[pid  8154] write(1, "Thread OK.\n", 11) = 11
[pid  8154] _exit(0)                    =

In ports/sysdeps/unix/sysv/linux/hppa/clone.S (line 170) I do see:
         /* The call to _exit needs saved r19.  */
         bl      _exit, %rp
         copy    %ret0, %arg0

This coding seems to indicate that we call _exit(), but we aren't (as
you see above in the strace).
So I'm wondering if the "bl _exit,%rp" needs changing.
Something with HIDDEN_JUMPTARGET(_exit) or similar, so that
_exit() instead of exit() is called?

Actually I did tested this with the minifail test program, and changed
thread_run() to call _exit(0) at the end instead of "return 0":
void* thread_run(void* arg) {
	...
/*	return (void *)&status;  */
         _exit(0);
}
With strace I then suddenly got exit_group() which seems more correct:
[pid  1910] write(1, "Thread OK.\n", 11) = 11
[pid  1910] exit_group(0)               = ?

Helge

Re: futex wait failure

[John David Anglin]

Hi Helge,

If you link the test with -static, you should be able to set
breaks in your test application and determine why _exit isn't
called.

I'm in the process of moving homes, so I don't have much time
to investigate myself.

Dave

> On 01/08/2010 10:44 PM, Carlos O'Donell wrote:
> > On Fri, Jan 8, 2010 at 4:18 PM, Helge Deller<deller@gmx.de>  wrote:
> >>> I now think this probably is a glibc bug.  The kernel uses this value
> >>> when the CLONE_PARENT_SETTID flag is passed.
> >>
> >>
> >> Maybe we have a futex problem in glibc on hppa?
> >> In glibc nptl/pthread_mutex_trylock.c we check the return value of a futex
> >> syscall against EWOULDBLOCK.
> >> Since on parisc - in contrast to all other architectures - we have
> >> EWOULDBLOCK!=EAGAIN, we maybe missed a check?
> >
> > That's a bug. [...]
> 
> Carlos,  Dave,
> 
> I'm wondering if we have another bug in glibc...?
> 
> When running strace, I noticed that a thread which returns with "return 0",
> exits the process on hppa with exit(0):
> [pid  1875] write(1, "Thread OK.\n", 11) = 11
> [pid  1875] exit(0)
> 
> On my x86_64 I see:
> [pid  8154] write(1, "Thread OK.\n", 11) = 11
> [pid  8154] _exit(0)                    =
> 
> In ports/sysdeps/unix/sysv/linux/hppa/clone.S (line 170) I do see:
>          /* The call to _exit needs saved r19.  */
>          bl      _exit, %rp
>          copy    %ret0, %arg0
> 
> This coding seems to indicate that we call _exit(), but we aren't (as
> you see above in the strace).
> So I'm wondering if the "bl _exit,%rp" needs changing.
> Something with HIDDEN_JUMPTARGET(_exit) or similar, so that
> _exit() instead of exit() is called?
> 
> Actually I did tested this with the minifail test program, and changed
> thread_run() to call _exit(0) at the end instead of "return 0":
> void* thread_run(void* arg) {
> 	...
> /*	return (void *)&status;  */
>          _exit(0);
> }
> With strace I then suddenly got exit_group() which seems more correct:
> [pid  1910] write(1, "Thread OK.\n", 11) = 11
> [pid  1910] exit_group(0)               = ?
> 
> Helge
> 


-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

> Actually I did tested this with the minifail test program, and changed
> thread_run() to call _exit(0) at the end instead of "return 0":
> void* thread_run(void* arg) {
> 	...
> /*	return (void *)&status;  */
>          _exit(0);
> }
> With strace I then suddenly got exit_group() which seems more correct:
> [pid  1910] write(1, "Thread OK.\n", 11) = 11
> [pid  1910] exit_group(0)               = ?

The child of the fork should exit using _exit(0).  Otherwise, it messes
with the file descriptors of the parent.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Carlos O'Donell]

On Mon, Jan 18, 2010 at 3:44 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
>> Actually I did tested this with the minifail test program, and chang=
ed
>> thread_run() to call _exit(0) at the end instead of "return 0":
>> void* thread_run(void* arg) {
>> =A0 =A0 =A0 ...
>> /* =A0 =A0return (void *)&status; =A0*/
>> =A0 =A0 =A0 =A0 =A0_exit(0);
>> }
>> With strace I then suddenly got exit_group() which seems more correc=
t:
>> [pid =A01910] write(1, "Thread OK.\n", 11) =3D 11
>> [pid =A01910] exit_group(0) =A0 =A0 =A0 =A0 =A0 =A0 =A0 =3D ?
>
> The child of the fork should exit using _exit(0). =A0Otherwise, it me=
sses
> with the file descriptors of the parent.

Thanks for verifying this. I'm looking into this.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[Carlos O'Donell]

On Sat, Jan 16, 2010 at 6:17 PM, Helge Deller <deller@gmx.de> wrote:
> This coding seems to indicate that we call _exit(), but we aren't (as
> you see above in the strace).
> So I'm wondering if the "bl _exit,%rp" needs changing.
> Something with HIDDEN_JUMPTARGET(_exit) or similar, so that
> _exit() instead of exit() is called?
>
> Actually I did tested this with the minifail test program, and change=
d
> thread_run() to call _exit(0) at the end instead of "return 0":
> void* thread_run(void* arg) {
> =A0 =A0 =A0 =A0...
> /* =A0 =A0 =A0return (void *)&status; =A0*/
> =A0 =A0 =A0 =A0_exit(0);
> }
> With strace I then suddenly got exit_group() which seems more correct=
:
> [pid =A01910] write(1, "Thread OK.\n", 11) =3D 11
> [pid =A01910] exit_group(0) =A0 =A0 =A0 =A0 =A0 =A0 =A0 =3D ?

I agree with this analysis. I'm testing a change which uses
HIDDEN_JUMPTARGET(). I will tell you how it goes.

Calling exit_group is important for threads.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[Helge Deller]

On 01/29/2010 06:53 PM, Carlos O'Donell wrote:
> On Sat, Jan 16, 2010 at 6:17 PM, Helge Deller<deller@gmx.de>  wrote:
>> This coding seems to indicate that we call _exit(), but we aren't (as
>> you see above in the strace).
>> So I'm wondering if the "bl _exit,%rp" needs changing.
>> Something with HIDDEN_JUMPTARGET(_exit) or similar, so that
>> _exit() instead of exit() is called?
>>
>> Actually I did tested this with the minifail test program, and changed
>> thread_run() to call _exit(0) at the end instead of "return 0":
>> void* thread_run(void* arg) {
>>         ...
>> /*      return (void *)&status;  */
>>         _exit(0);
>> }
>> With strace I then suddenly got exit_group() which seems more correct:
>> [pid  1910] write(1, "Thread OK.\n", 11) = 11
>> [pid  1910] exit_group(0)               = ?
>
> I agree with this analysis. I'm testing a change which uses
> HIDDEN_JUMPTARGET(). I will tell you how it goes.

I think I tested it once, and it doesn't help.
Threads are started by calling start_thread(), and I think we need changes there, e.g. (untested):

diff -up ./nptl/pthread_create.c.org ./nptl/pthread_create.c
--- ./nptl/pthread_create.c.org 2010-01-17 20:54:19.000000000 +0100
+++ ./nptl/pthread_create.c     2010-01-17 20:56:24.000000000 +0100
@@ -316,7 +316,7 @@ start_thread (void *arg)
       is no thread left.  */
    if (__builtin_expect (atomic_decrement_and_test (&__nptl_nthreads), 0))
      /* This was the last thread.  */
-    exit (0);
+    _exit (0);
                                                                                                                                            
    /* Report the death of the thread if this is wanted.  */
    if (__builtin_expect (pd->report_events, 0))
  

> Calling exit_group is important for threads.

Yep.

Helge

Re: futex wait failure

[John David Anglin]

> >> Actually I did tested this with the minifail test program, and changed
> >> thread_run() to call _exit(0) at the end instead of "return 0":
> >> void* thread_run(void* arg) {
> >>         ...
> >> /*      return (void *)&status;  */
> >>         _exit(0);
> >> }
> >> With strace I then suddenly got exit_group() which seems more correct:
> >> [pid  1910] write(1, "Thread OK.\n", 11) = 11
> >> [pid  1910] exit_group(0)               = ?

Did this fix minifail?

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Carlos O'Donell]

On Thu, Jan 28, 2010 at 11:56 AM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
>> >> Actually I did tested this with the minifail test program, and ch=
anged
>> >> thread_run() to call _exit(0) at the end instead of "return 0":
>> >> void* thread_run(void* arg) {
>> >> =A0 =A0 =A0 =A0 ...
>> >> /* =A0 =A0 =A0return (void *)&status; =A0*/
>> >> =A0 =A0 =A0 =A0 _exit(0);
>> >> }
>> >> With strace I then suddenly got exit_group() which seems more cor=
rect:
>> >> [pid =A01910] write(1, "Thread OK.\n", 11) =3D 11
>> >> [pid =A01910] exit_group(0) =A0 =A0 =A0 =A0 =A0 =A0 =A0 =3D ?
>
> Did this fix minifail?

Do you have a copy of minifail I can run as a contained test case?

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[John David Anglin]

[-- Attachment #1: Type: text/plain, Size: 812 bytes --]

On Mon, 01 Feb 2010, Carlos O'Donell wrote:

> Do you have a copy of minifail I can run as a contained test case?

I've attached two versions -- the original and a simpler revised version.

The revised version shows that this is a clone/fork problem.  If the
parent waits for the child thread to set run before forking, the test
doesn't fail.  If the static definition of run is changed to 1, the
test fails.  I had added some loops at one time to show that this isn't
an exit issue. 

This is not to say that Helge isn't right about threads incorrectly using
exit.

The failure is always the stack of the child thread is overwritten with
zeros.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

[-- Attachment #2: minifail.cpp --]
[-- Type: text/plain, Size: 1169 bytes --]

#include <QtCore/QCoreApplication>
#include <QtCore/QThread>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>

class MyThread : public QThread
{
	void run();
};

void MyThread::run() {
	printf("Thread OK.\n");
}

int qt_test(int argc, char** argv) {
	QCoreApplication app(argc, argv);

	MyThread thread;
	thread.start();

	int p[2];
	char buf;
	pipe(p);

	switch (fork()) {
		case -1:
			perror("fork() failed");
		case 0:
			printf("Child OK.\n");
			write(p[1], "\0", 1);
			exit(0);
		default:
			break;
		
	}
	close(p[1]);
	read(p[0], &buf, 1);
	thread.wait();
	return 0;
}

void* thread_run(void* arg) {
	printf("Thread OK.\n");
}

int pure_test() {
	pthread_t thread;
	pthread_create(&thread, NULL, thread_run, NULL);

	int p[2];
	char buf;
	pipe(p);

	switch (fork()) {
		case -1:
			perror("fork() failed");
		case 0:
			close(p[0]);
			printf("Child OK.\n");
			write(p[1], "\0", 1);
			exit(0);
		default:
			break;
		
	}
	
	close(p[1]);
	read(p[0], &buf, 1);
	pthread_join(thread, NULL);
	return 0;
}

int main(int argc, char** argv) {
	if (argc == 2) {
		if (!strcmp("qt", argv[1])) {
			return qt_test(argc, argv);
		}
	}
	return pure_test();
}

[-- Attachment #3: minifail6.cpp --]
[-- Type: text/plain, Size: 1246 bytes --]

#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>

/*
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=561203

  clone(child_stack=0x4088d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x4108c4e8, tls=0x4108c900, child_tidptr=0x4108c4e8) = 14819
[pid 14819] set_robust_list(0x4108c4f0, 0xc) = 0
[pid 14818] clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x40002028) = 14820

 g++  minifail.cpp -o minifail -O0 -pthread -g

 i=0; while true; do i=$(($i+1)); echo Run $i; ./minifail; done;

 */

static volatile int run;

void* thread_run(void* arg) {
	static long status;
	int i;
        
	run = 1;
	pthread_yield();
	for (i = 10000000; i; i--)
	   continue;
	write(1,"Thread OK.\n",11);
	return (void *)&status;
}

int pure_test() {
	pthread_t thread;
	pthread_create(&thread, NULL, thread_run, NULL);

	while (!run)
	  continue;
	switch (fork()) {
		case -1:
			perror("fork() failed");
		case 0:
			write(1,"Child OK.\n",10);
			_exit(0);
		default:
			break;
		
	}
	pthread_join(thread, NULL);
	return 0;
}

int main(int argc, char** argv) {
	return pure_test();
}

Re: futex wait failure

[Helge Deller]

On 02/01/2010 01:26 AM, John David Anglin wrote:
>>>> Actually I did tested this with the minifail test program, and changed
>>>> thread_run() to call _exit(0) at the end instead of "return 0":
>>>> void* thread_run(void* arg) {
>>>>          ...
>>>> /*      return (void *)&status;  */
>>>>          _exit(0);
>>>> }
>>>> With strace I then suddenly got exit_group() which seems more correct:
>>>> [pid  1910] write(1, "Thread OK.\n", 11) = 11
>>>> [pid  1910] exit_group(0)               = ?
>
> Did this fix minifail?

No, sadly it didn't fixed it.

Helge

Re: futex wait failure

[John David Anglin]

> On 02/01/2010 01:26 AM, John David Anglin wrote:
> >>>> Actually I did tested this with the minifail test program, and changed
> >>>> thread_run() to call _exit(0) at the end instead of "return 0":
> >>>> void* thread_run(void* arg) {
> >>>>          ...
> >>>> /*      return (void *)&status;  */
> >>>>          _exit(0);
> >>>> }
> >>>> With strace I then suddenly got exit_group() which seems more correct:
> >>>> [pid  1910] write(1, "Thread OK.\n", 11) = 11
> >>>> [pid  1910] exit_group(0)               = ?
> >
> > Did this fix minifail?
> 
> No, sadly it didn't fixed it.

I tend to think clone/fork syscalls need to be guarded to ensure a stable
configuration while the syscall executes.  It may be the problem arises
because we use clone for fork.  Do any other targets do this?

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Carlos O'Donell]

On Mon, Feb 1, 2010 at 2:11 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
>>
>> No, sadly it didn't fixed it.
>
> I tend to think clone/fork syscalls need to be guarded to ensure a st=
able
> configuration while the syscall executes. =A0It may be the problem ar=
ises
> because we use clone for fork. =A0Do any other targets do this?

All targets call nptl/sysdeps/unix/sysv/linux/fork.c, which calls
ARCH_FORK, which is implemented using clone() for *all* targets.

In fact, owing to the POSIX atfork requirements, it's very difficult
to implement fork properly without using Linux clone.

Cheers,
Carlos.
--
To unsubscribe from this list: send the line "unsubscribe linux-parisc"=
 in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex wait failure

[John David Anglin]

> I think I have an idea what could have happened and why it most of the times (but not always) crashes in the child process...
> 
> In ports/sysdeps/unix/sysv/linux/hppa/bits/atomic.h we have:
> #define atomic_compare_and_exchange_val_acq(mem, newval, oldval) \
>   ({                                                                    \
>      volatile int lws_errno;                                            \
>      volatile int lws_ret;                                              \
>      asm volatile(                                                      \
> ...some assembly...
>         "stw    %%r28, %0                       \n\t"                   \
>         "sub    %%r0, %%r21, %%r21              \n\t"                   \
>         "stw    %%r21, %1                       \n\t"                   \
>         : "=m" (lws_ret), "=m" (lws_errno)                              \
>         : "r" (mem), "r" (oldval), "r" (newval)                         \
>         : _LWS_CLOBBER           
> 
> this means, that lws_errno and lws_ret are located on the stack.
> 
> With gdb I see this expanded to:
> 0x40705494 <start_thread+1204>: stw ret0,-1b8(sp)
> 0x40705498 <start_thread+1208>: sub r0,r21,r21
> 0x4070549c <start_thread+1212>: stw r21,-1b4(sp)
> 
> So, lws_ret/lws_errno are at -1b8/-1b4(sp).
> 
> And this LWS code is called from 
> ../nptl/sysdeps/pthread/createthread.c:
> static int create_thread (struct pthread *pd, const struct pthread_attr *attr, STACK_VARIABLES_PARMS)
> ...
>           int res = do_clone (pd, attr, clone_flags, start_thread,
>                               STACK_VARIABLES_ARGS, 1);
>           if (res == 0)
>             {
> ...(line 216):
>               /* Enqueue the descriptor.  */
>               do
>                 pd->nextevent = __nptl_last_event;
>               while (atomic_compare_and_exchange_bool_acq(&__nptl_last_event, pd, pd->nextevent) != 0);
> 
> 
> And here is what could have happened:
> a) do_clone() creates the child process.
> b) the child process gets a new stack
> c) the child calls atomic_compare_and_exchange_bool_acq() and thus the LWS code above.
> d) the LWS code writes to the stack location at -1b8(sp), which is out of bounds for the child process (the child stack got only ~ 0x40 bytes initial room)

I think the stack locations should be ok because start_thread allocates
an additional 0x1c0 bytes:

Dump of assembler code for function start_thread:
   0x40a40300 <+0>:     stw rp,-14(sp)
   0x40a40304 <+4>:     ldo 1c0(sp),sp

In all the fails I have looked at, the saved $rp value is clobbered.
The stack pointer value seems consistent with 0x40 + 0x1c0.  The data
placed at the beginning of the stack for the child thread is not clobbered.

> e) Thus the child either crashes, overwrites memory of the parent or does other things wrong.

I don't see how the forked child can affect the memory of the parent.
It can close files and affect the parent that way (child should use
_exit and not exit).

If the forked child actually overwrites memory of the parent, this is
a big bug in the linux fork code.

> Additionally:
> Due to the LWS assembly code and because we don't have many registers free while using LWS, gcc used %rp as a temporary register which may have fooled us in our thinking?

$rp is saved in the first instruction of start_thread.  So, its use
below should be ok.

> 0x40705458 <start_thread+1144>: ldi 0,rp
> 0x4070545c <start_thread+1148>: ldi fb,r3
> 0x40705460 <start_thread+1152>: ldw -70(sp),ret0
> 0x40705464 <start_thread+1156>: ldw 214(ret0),ret1
> 0x40705468 <start_thread+1160>: copy r5,r26
> 0x4070546c <start_thread+1164>: copy ret1,r25
> 0x40705470 <start_thread+1168>: copy rp,r24
> 0x40705474 <start_thread+1172>: be,l b0(sr2,r0),sr0,r31
> 0x40705478 <start_thread+1176>: ldi 0,r20
> 0x4070547c <start_thread+1180>: ldi -b,r24
> 0x40705480 <start_thread+1184>: cmpb,=,n r24,r21,0x40705468 <start_thread+1160>
> 0x40705484 <start_thread+1188>: nop
> 0x40705488 <start_thread+1192>: ldi -2d,r25
> 0x4070548c <start_thread+1196>: cmpb,=,n r25,r21,0x40705468 <start_thread+1160>
> 0x40705490 <start_thread+1200>: nop
> 0x40705494 <start_thread+1204>: stw ret0,-1b8(sp)
> 0x40705498 <start_thread+1208>: sub r0,r21,r21
> 0x4070549c <start_thread+1212>: stw r21,-1b4(sp)
> 0x407054a0 <start_thread+1216>: ldw -1b4(sp),ret0
> 
> 
> If my assumptions are correct, then we either could
> 
> a) use the gcc atomic builtins instead of own atomic code in libc6:
> E.g: add to ports/sysdeps/unix/sysv/linux/hppa/bits/atomic.h:
> ...
> #if __GNUC_PREREQ (4, 1)
> # define atomic_compare_and_exchange_val_acq(mem, newval, oldval) \
>   __sync_val_compare_and_swap (mem, oldval, newval)
> #  define atomic_compare_and_exchange_bool_acq(mem, newval, oldval) \
>   (! __sync_bool_compare_and_swap (mem, oldval, newval))
> 
> #elif __ASSUME_LWS_CAS
> ....

There may be a bug in the gcc atomic builtins.  We shanged recently
to using the sync builtins in libstdc++.  Then, two fails appeared
recently that I haven't had time to look at:

WARNING: program timed out.
FAIL: 29_atomics/atomic_flag/clear/1.c execution test
FAIL: 29_atomics/atomic_flag/test_and_set/explicit.c execution test

That said, this is an interesting test.  Does it fix minifail?

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[Carlos O'Donell]

On Mon, Jan 4, 2010 at 12:32 PM, John David Anglin
<dave@hiauly1.hia.nrc.ca> wrote:
>> e) Thus the child either crashes, overwrites memory of the parent or does other things wrong.
>
> I don't see how the forked child can affect the memory of the parent.
> It can close files and affect the parent that way (child should use
> _exit and not exit).
>
> If the forked child actually overwrites memory of the parent, this is
> a big bug in the linux fork code.

We have two bugs that are getting mixed here.

Your original post has to do with a futex wait failure, this is
possibly related to the hppa low level lock implementation. I am
updating the hppa implementation to see if I can fix this for you.

Helge's comments relate only to the vfork crash, and the Qt thread
creation issue being seen by debian.

Cheers,
Carlos.

Re: futex wait failure

[John David Anglin]

On Mon, 04 Jan 2010, Carlos O'Donell wrote:

> On Mon, Jan 4, 2010 at 12:32 PM, John David Anglin
> <dave@hiauly1.hia.nrc.ca> wrote:
> >> e) Thus the child either crashes, overwrites memory of the parent or does other things wrong.
> >
> > I don't see how the forked child can affect the memory of the parent.
> > It can close files and affect the parent that way (child should use
> > _exit and not exit).
> >
> > If the forked child actually overwrites memory of the parent, this is
> > a big bug in the linux fork code.
> 
> We have two bugs that are getting mixed here.
> 
> Your original post has to do with a futex wait failure, this is
> possibly related to the hppa low level lock implementation. I am
> updating the hppa implementation to see if I can fix this for you.
> 
> Helge's comments relate only to the vfork crash, and the Qt thread
> creation issue being seen by debian.

I was talking about the debian thread creation bug in the above.
I haven't been able to duplicate the vfork crash that you posted.
The expect/tcl futex bug is likely related to the debian bug (this
is what expect does in testsuite runs).  Reduced minifail testcase
is below.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>

/*
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=561203

  clone(child_stack=0x4088d040, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x4108c4e8, tls=0x4108c900, child_tidptr=0x4108c4e8) = 14819
[pid 14819] set_robust_list(0x4108c4f0, 0xc) = 0
[pid 14818] clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x40002028) = 14820

 g++  minifail.cpp -o minifail -O0 -pthread -g

 i=0; while true; do i=$(($i+1)); echo Run $i; ./minifail; done;

 */
void* thread_run(void* arg) {
	write(1,"Thread OK.\n",11);
}

int pure_test() {
	pthread_t thread;
	pthread_create(&thread, NULL, thread_run, NULL);

	switch (fork()) {
		case -1:
			perror("fork() failed");
		case 0:
			write(1,"Child OK.\n",10);
			_exit(0);
		default:
			break;
		
	}
	
	pthread_join(thread, NULL);
	return 0;
}

int main(int argc, char** argv) {
	return pure_test();
}

Re: futex wait failure

[Helge Deller]

On 01/04/2010 05:27 PM, Helge Deller wrote:
> If my assumptions are correct, then we either could
>
> a) use the gcc atomic builtins instead of own atomic code in libc6:
> E.g: add to ports/sysdeps/unix/sysv/linux/hppa/bits/atomic.h:
> ...
> #if __GNUC_PREREQ (4, 1)
> # define atomic_compare_and_exchange_val_acq(mem, newval, oldval) \
>    __sync_val_compare_and_swap (mem, oldval, newval)
> #  define atomic_compare_and_exchange_bool_acq(mem, newval, oldval) \
>    (! __sync_bool_compare_and_swap (mem, oldval, newval))
>
> #elif __ASSUME_LWS_CAS
> ....


Even with this change, minifail sadly still segfaults :-(

Helge

Re: futex wait failure

[John David Anglin]

> 
> On Tue, Dec 29, 2009 at 8:47 AM, Helge Deller <deller@gmx.de> wrote:
> >> 2) Change return space register to sr7. =A0sr3 is not set correctly
> >> if the entry number is invalid.
> >
> > Yep.
> >
> >> Have nasty suspicion that sr3 is getting hit...
> 
> Is the thought here that we take an interrupt, and sr3 is not
> guaranteed saved/restored, while sr7 is guaranteed?

sr3 isn't guaranteed because user could have changed its value.  So,
it is better to use sr7 for the return, particularly when lws entry
number isn't valid.

Regarding the handling of space registers during interruptions, they
are not saved.  syscall_exit_rfi restores them to proper values:

        /*
	 * If we aren't being traced, we never saved space registers
	 * (we don't store them in the sigcontext), so set them
	 * to "proper" values now (otherwise we'll wind up restoring
	 * whatever was last stored in the task structure, which might
	 * be inconsistent if an interrupt occured while on the gateway
	 * page). Note that we may be "trashing" values the user put in
	 * them, but we don't support the user changing them.
	*/

	STREG   %r0,PT_SR2(%r16)
	mfsp    %sr3,%r19
	STREG   %r19,PT_SR0(%r16)
	STREG   %r19,PT_SR1(%r16)
	STREG   %r19,PT_SR3(%r16)

As can be seen, sr2 set to the kernel space and sr3 to the user space.
This is consistent with the usage in the LWS code, so an interruption
shouldn't be a problem on most syscall returns via this path.  However,
execve doesn't restore the space register values (at least in this path).

The comment in the LWS code is a bit misleading:

        /* WARNING: Trashing sr2 and sr3 */
	mfsp    %sr7,%r1                        /* get userspace into sr3 */
	mtsp    %r1,%sr3
	mtsp    %r0,%sr2                        /* get kernel space into sr2 */

Actually, sr2 and sr3 are restored to their "proper" values.

The TLB code seems to assume a consistent use of sr3.  So, this may not
be a space register issue.

The syscall_restore uses a rsm/ssm pair to make the operation atomic.
Maybe we don't catch all situations where process is scheduled.

As an aside, it seems gdb doesn't print space registers correctly
when debugging a program.  I would guess this is because they are
not saved in the context.  It does print them from a core dump.

> I don't see anything wrong with moving the gate earlier (unless
> someone can come up with a case where an LWS may not want to gate).
> 
> It has the benefit of making the fast path 1 instruction shorter,
> however I don't see that it makes the implementation more correct.

I believe that it does have an effect.  See for example, the discussion
regarding delivery of signals to processes executing on the gateway page
in traps.c.  With the branch, it is possible that a signal could be
sent to a process executing on the gateway page.  See also space_check
macro in entry.S.  There's also the ptrace issue mentioned by Randolph.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: futex wait failure

[John David Anglin]

> Regarding the handling of space registers during interruptions, they
> are not saved.  syscall_exit_rfi restores them to proper values:

This is potentially an implementation bug, particularly with respect to
interruptions.   User space probably should have sr0-sr3 available to
do icache flushes, etc.  Currently, gcc uses sr0 for icache flushes.

It may be sr0 is never clobbered, but this isn't clear.  I'm fairly sure
HP-UX saves sr0-sr4 if necessary.

Possibly, the icache flush sequence should be changed for linux to not
use sr0.

Dave
-- 
J. David Anglin                                  dave.anglin@nrc-cnrc.gc.ca
National Research Council of Canada              (613) 990-0752 (FAX: 952-6602)

Re: Newer kernels on the Jensen (was: [PATCH v2 0/4] Introduce and use absolute_pointer macro)

[Ulrich Teichert]

Hi,

> I'll try booting from CDROM with a serial line attached at the weekend,
> perhaps that will give us a hint.

[You can call the Alpha a dead architecture and the Jensen and antiquated
machine, but there are features which modern PCs *still* don't have,
like the automatic switch over to the serial line console when no keyboard
is connected. I'm still impressed how well that works.]

Well...whatever that means, but you can call it a hint. Booting from
CD-ROM stopps with:

>>> boot dka500 -fl i
INIT-S-CPU...
AUDIT_BOOT_STARTS ... 
AUDIT_CHECKSUM_GOOD
AUDIT_LOAD_BEGINS
AUDIT_LOAD_DONE

Linux/AXP bootloader for Linux 5.15.0-rc2-00045-gcf1d2c3e7e2f-dirty
Switching to OSF PAL-code .. Ok (rev 20123)
Loading vmlinux ...Failed (200000000067a000)

?05 HLT INSTR
  PC= 00000000.20000014 PSL= 00000000.00000007

Anyone an idea what 200000000067a000 may mean in this context?

TIA,
Uli
-- 
Dipl. Inf. Ulrich Teichert|e-mail: Ulrich.Teichert@gmx.de | Listening to:
Stormweg 24               |Eat Lipstick: Dirty Little Secret, The Baboon Show:
24539 Neumuenster, Germany|Work Work Work, The Bellrays: Bad Reaction

ext3-2.4-0.9.4

[Andrew Morton]

An update to the ext3 filesystem for 2.4 kernels is available at

	http://www.uow.edu.au/~andrewm/linux/ext3/

The diffs are against linux-2.4.7 and linux-2.4.6-ac5.

The changelog is there.  One rarely-occurring but oopsable bug
was fixed and several quite significant performance enhancements
have been made.  These are in addition to the performance fixes
which went into 0.9.3.

Ted has put out a prelease of e2fsprogs-1.23 which supports
filesystem type `auto' in /etc/fstab, so it is now possible to
switch between ext3- and non-ext3-kernels without changing
any configuration.

It is recommended that users of earlier ext3 releases upgrade
to 0.9.4.

For people who are undertaking performance testing, it is perhaps
useful to point out that ext3 operates in one of three different
journalling modes, and that these modes have very different
functionality and very different performance characteristics.
Really, you need to test all three and balance the functionality
which each mode offers against the throughput which you obtain
in your application.


The modes are:

data=writeback

  This is classic metadata-only journalling.  File data is written
  back to the main fs lazily.  After a crash+recovery the fs's
  structural integrity is preserved, but the *contents* of files
  can and will contain old, stale data.  Potentially hundreds of
  megabytes of it.

  This is the fastest mode for normal filesystem applications.

data=ordered

  The fs ensures that file data is written into the main fs prior
  to committing its metadata.  Hence after a crash+recovery, your
  files will contain the correct data.

  This is the default operating mode and throughput is good. It
  adds about one second to a four minute kernel compile when
  compared with ext2.   Under heavier loads the difference
  becomes larger.

data=journal

  All data (as well as to metadata) is written to the journal
  before it is released to the main fs for writeback.
  
  This is a specialised mode - for normal fs usage you're better
  off using ordered data, which has the same benefits of not corrupting
  data after crash+recovery.  However for applications which require
  synchronous operation such as mail spools and synchronously exported
  NFS servers, this can be a performance win.  I have seen dbench
  figures in this mode (where the files were opened O_SYNC) running
  at ten times the throughput of ext2.  Not that this is the expected
  benefit for other applications!


Looking at the above issues, one may initially think that the
post-recovery data corruption is a serious issue with writeback mode,
and that there are big advantages to using journalled or ordered data.

However, even in these modes the affected files may be shorter-than-expected
after recovery, because the app hadn't finished writing them yet.  And
usually, a truncated file is just as useless as one which contains
garbage - it needs to be deleted.

It's not really as simple as that - for small (< a few hundred k) files,
it tends to be the case that either the whole file is intact after a crash,
or none of it is.  This is because the journalling mechanism starts a
new transaction every five seconds, and a typical open/write/close operation
usually fits entirely inside this window.

There is also a security issue to be considered: a recovered writeback-mode
filesystem will expose other people's old data to unintended recipients.


Hopefully this description will help people make their deployment choices.
If not, assistance is available on the ext3-users@redhat.com mailing list.

-

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Thu, 26 Jul 2001, Andrew Morton wrote:

> data=journal
> 
>   All data (as well as to metadata) is written to the journal
>   before it is released to the main fs for writeback.
>   
>   This is a specialised mode - for normal fs usage you're better
>   off using ordered data, which has the same benefits of not corrupting
>   data after crash+recovery.  However for applications which require
>   synchronous operation such as mail spools and synchronously exported
>   NFS servers, this can be a performance win.  I have seen dbench

In ordered and journal mode, are meta data operations, namely creating a
file, rename(), link(), unlink() "synchronous" in the sense that after
the call has returned, the effect of this call is never lost, i. e., if
link(2) has returned and the machine crashes immediately, will the next
recovery ALWAYS recover the link?

Or will ext3 still need chattr +S?

Does it still support chattr +S at all?

Synchronous meta data operations are crucial for mail transfer agents
such as Postfix or qmail. Postfix has up until now been setting
chattr +S /var/spool/postfix, making original (esp. soft-updating) BSD
file systems significantly faster for data (payload) writes in this
directory than ext2.

Note: I'm not on the ext3-users list. Please Cc: back replies.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Andrew Morton]

Matthias Andree wrote:
> 
> On Thu, 26 Jul 2001, Andrew Morton wrote:
> 
> > data=journal
> >
> >   All data (as well as to metadata) is written to the journal
> >   before it is released to the main fs for writeback.
> >
> >   This is a specialised mode - for normal fs usage you're better
> >   off using ordered data, which has the same benefits of not corrupting
> >   data after crash+recovery.  However for applications which require
> >   synchronous operation such as mail spools and synchronously exported
> >   NFS servers, this can be a performance win.  I have seen dbench
> 
> In ordered and journal mode, are meta data operations, namely creating a
> file, rename(), link(), unlink() "synchronous" in the sense that after
> the call has returned, the effect of this call is never lost, i. e., if
> link(2) has returned and the machine crashes immediately, will the next
> recovery ALWAYS recover the link?

No, they're not synchronous by default.  After recovery they
will either be wholly intact, or wholly absent.

> Or will ext3 still need chattr +S?

Yes, if the app doesn't support O_SYNC or fsync().  I believe
that MTA's *do* support those things.
 
> Does it still support chattr +S at all?

Yes.

> Synchronous meta data operations are crucial for mail transfer agents
> such as Postfix or qmail. Postfix has up until now been setting
> chattr +S /var/spool/postfix, making original (esp. soft-updating) BSD
> file systems significantly faster for data (payload) writes in this
> directory than ext2.

If postfix is capable of opening the files O_SYNC or of doing
fsync() on them then the `chattr +s' is no longer necessary - unlike
ext2, when the O_SYNC write() or the fsync() return, the directory
contents (as well as the inode, bitmaps, data, etc) will all be tight on
disk and will be restored after a crash.

This should speed things up considerably, especially with journalled-data
mode.  I need to test and characterise this some more to come up with some
quantitative results and configuration recommendations.


BTW, if you have more-than-modest throughput requirements, don't
even *think* of mounting the fs with `mount -o sync'. Our performance
in this mode is terrible :(

I have a hack somewhere which fixes this as much as it can be fixed, but
I didn't even bother committing it.  It's feasible, but tiresome.

A better solution is to fix some lock inversion problems in the core
kernel which prevent optimal implementation of data-journalling
filesystems.  I don't really expect this to occur medium-term or ever.

A middle-ground solution may be to add an fs-private `osync' mount
option, so all files are treated similarly to O_SYNC, which would
work well.

-

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Thu, 26 Jul 2001, Andrew Morton wrote:

> > In ordered and journal mode, are meta data operations, namely creating a
> > file, rename(), link(), unlink() "synchronous" in the sense that after
> > the call has returned, the effect of this call is never lost, i. e., if
> > link(2) has returned and the machine crashes immediately, will the next
> > recovery ALWAYS recover the link?
> 
> No, they're not synchronous by default.  After recovery they
> will either be wholly intact, or wholly absent.
> 
> > Or will ext3 still need chattr +S?
> 
> Yes, if the app doesn't support O_SYNC or fsync().  I believe
> that MTA's *do* support those things.
>  
> > Does it still support chattr +S at all?
> 
> Yes.
> 
> > Synchronous meta data operations are crucial for mail transfer agents
> > such as Postfix or qmail. Postfix has up until now been setting
...
> A middle-ground solution may be to add an fs-private `osync' mount
> option, so all files are treated similarly to O_SYNC, which would
> work well.

You seem to be missing the point, because I wasn't verbose enough, so I
will try to rephrase this and explain. This may turn out to be a feature
request. :-}

Before going into detail, MTAs do know about fsync(). ext3 synching
relevant directory parts as part of fsync() is a great achievement.
Finally, more than five years after initial complaints, Linux is SLOWLY
getting somewhere for speeding up reliable MTA operation.

But that's the smaller piece. Common MTAs such as Postfix or qmail
rename or link files into place (their queues, the mail spool). With the
advent of journalling came the atomicity of rename operations. That's
also a great achievement.

However, the remaining problem is being synchronous with respect to open
(fixed for ext3 with your fsync() as I understand it), rename, link and
unlink. With ext2, and as you write it, with ext3 as well, there is
currently no way to tell when the link/rename has been committed to
disk, unless you set mount -o sync or chattr +S or call sync() (the
former is not an option because it's far too expensive).


The official statement by Dr. Wietse Venema (who wrote Postfix) is,
Postfix REQUIRES synchronous directory updates (open, rename, link,
unlink, in order of decreasing importance). Wietse refuses to wrap all
these calls for Linux.

Similar assumptions hold for qmail.


So, what would help the common MTA? osync wouldn't, MTAs know how to use
fsync().  dirsync or bsdstyle or however it's called, as chattr and
mount options, would help. This option should make all directory
operations (open/creat/fsync, rename, link, unlink, symlink, possibly
close) synchronous in respect to affected directory and meta data while
leaving application data (payload) operations asynchronous (applications
can then choose when to call fsync() to flush the data to disk).

A much better file system for an MTA might be ext3fs with
data=journalled and dirsync mount/chattr option. Would you deem it
possible to get such an option done before ext3fs 1.0.0?

I hope this makes the requirements of this particular group of
applications clear.

Thanks again to everyone involved with the ext3fs development.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Thu, 26 Jul 2001, Matthias Andree wrote:

> So, what would help the common MTA?

Not relying on non-supported semantics to save your ass.

Rename() is atomic in the sense that you either see the
old name or the new name, but I don't know of systems
which guarantee atomicity across a system crash.

In fact, knowing how hard disks work mechanically, only
journaling filesystems could have an extention to make
this work.  Ie. this is NOT something you can rely on ;)

regards,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Thu, 26 Jul 2001, Rik van Riel wrote:

> In fact, knowing how hard disks work mechanically, only
> journaling filesystems could have an extention to make
> this work.  Ie. this is NOT something you can rely on ;)

This is not about failing hard disks. It is about premature
acknowledgment of something which has not happened at that time.

Linux cannot possibly fix all incomplete protocols, specifications and
implementation, but it can fix its own behaviour.

Everything is about speed, and allowing the MTA to use a (weaker)
dirsync rather than allsync option would speed things up without
sacrificing reliability.

MTA reliability is NOT about failing disk drives. If it falls over, you
notice that. If files are in the wrong directory or not there at all,
you don't necessarily notice until someone complains.

Please don't get in the way of finally fixing things just because
someone might have a broken item that could endanger your data. I have a
huge magnet here...

The competition is there and it has names: BSD + ufs + softupdates,
Solaris + logging ufs. Read MTA mailing lists before obstructing.

Thanks.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Thu, 26 Jul 2001, Matthias Andree wrote:
> On Thu, 26 Jul 2001, Rik van Riel wrote:
>
> > In fact, knowing how hard disks work mechanically, only
> > journaling filesystems could have an extention to make
> > this work.  Ie. this is NOT something you can rely on ;)
>
> This is not about failing hard disks. It is about premature
> acknowledgment of something which has not happened at that time.

So you didn't read what I was writing.

Let me explain it to you slowly:

Disks.  Write.  One.  Write.  At.  A.  Time.

A rename often needs as many as 4 or 5 writes,
ergo, you CANNOT do a rename atomically without
journaling and transactions.

> The competition is there and it has names: BSD + ufs + softupdates,
> Solaris + logging ufs. Read MTA mailing lists before obstructing.

BSD + softupdates is physically incapable of doing what
you suggest it does.  This can be proven from the lack
of transactions and the way hard disks work physically.

regards,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Thu, 26 Jul 2001, Rik van Riel wrote:

> On Thu, 26 Jul 2001, Matthias Andree wrote:
> > On Thu, 26 Jul 2001, Rik van Riel wrote:
> >
> > > In fact, knowing how hard disks work mechanically, only
> > > journaling filesystems could have an extention to make
> > > this work.  Ie. this is NOT something you can rely on ;)
> >
> > This is not about failing hard disks. It is about premature
> > acknowledgment of something which has not happened at that time.
> 
> So you didn't read what I was writing.

Sorry.

> Let me explain it to you slowly:
> 
> Disks.  Write.  One.  Write.  At.  A.  Time.
> 
> A rename often needs as many as 4 or 5 writes,
> ergo, you CANNOT do a rename atomically without
> journaling and transactions.

You're missing the point, with this as the previous mail. The MTA is not
going to change from one unsupported/incompatible interface (that only
Linux suffers from) and if it did, it would still do the wrong thing.

MTAs often run multiple processes, and if these all open the same
directory and sync it while others have changes open that don't need a
sync at that time and will sync later, you're getting no further than
with chattr +S or mount -o sync.

It's not about atomicity itself, but about
first. write. all. required. blocks. for. a. certain. change.
physically. to. disc.   and. only. after. this. do. return. from.
rename, link, unlink. function. calls.

I'm aware of phase-tree concepts ("single block write switches from one
consistent state to another") and I'm aware that ext3fs and reiserfs do
feature atomic renames (after crash recovery).

That a drive might fall over or the power might fail before all writes
of a certain rename operation have completed is harmless UNLESS you lied
to someone that the operation was already complete (when it wasn't).

> > The competition is there and it has names: BSD + ufs + softupdates,
> > Solaris + logging ufs. Read MTA mailing lists before obstructing.
> 
> BSD + softupdates is physically incapable of doing what
> you suggest it does.  This can be proven from the lack
> of transactions and the way hard disks work physically.

You misunderstood me. I'm not talking about atomicity.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Alan Cox]

> On Thu, 26 Jul 2001, Rik van Riel wrote:
> > In fact, knowing how hard disks work mechanically, only
> > journaling filesystems could have an extention to make
> > this work.  Ie. this is NOT something you can rely on ;)
> 
> This is not about failing hard disks. It is about premature
> acknowledgment of something which has not happened at that time.

Rik is right. It isnt just about premature notification - its about 
atomicity. At the point you are notified the data has been queued for disk
I/O. Even on traditional BSD ufs with synchronous metadata you still had
points where a crash left the rename partially complete and nothing but a
log or an atomic update system is going to fix that.

> The competition is there and it has names: BSD + ufs + softupdates,
> Solaris + logging ufs. Read MTA mailing lists before obstructing.

All of which are - not unsuprisingly - using a log. In fact Solaris logging
ufs and ext3 are very similar ideas - adding a log to an existing fs.

Alan

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Thu, 26 Jul 2001, Alan Cox wrote:

> > The competition is there and it has names: BSD + ufs + softupdates,
> > Solaris + logging ufs. Read MTA mailing lists before obstructing.
>
> All of which are - not unsuprisingly - using a log. In fact
> Solaris logging ufs and ext3 are very similar ideas - adding a
> log to an existing fs.

Softupdates isn't using logging.  Furthermore, even
the journaling filesystems won't all guarantee that
the various parts of a rename() operation will all
be in the same transaction.

An MTA which relies on this is therefore Broken(tm).

cheers,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Andrew Morton]

Rik van Riel wrote:
> 
> 
> Furthermore, even the journaling filesystems won't all guarantee that
> the various parts of a rename() operation will all be in the same
> transaction.

umm..  I'd certainly hope that they do guarantee this.

The only operations which can't trivially fit into a single
transaction are write() and truncate().
 
-

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Thu, 26 Jul 2001, Rik van Riel wrote:

> An MTA which relies on this is therefore Broken(tm).

MTAs rely on TRULY, ULTIMATELY AND DEFINITELY SYNCHRONOUS directory
updates, nothing else. And because they do so, and most systems have
them, and MTAs are portable, they choose chattr +S on Linux. And that's
a performance problem because it doesn't come for free, but also with
synchronous data updates, which are unnecessary because there is
fsync().

That's already the complete story about MTAs on Linux.

If Linux HAD a mode (it doesn't) to have just synchronous directory
updates, MTAs could stop using chattr +S and be faster.


MTAs do NOT care how the file system is internally managed, they only
rely on the rename operation having completed physically on disk before
the "my rename call has returned 0" event. They expect that with the
call returning the rename operation has completed ultimately, finally,
for good, definitely and the old file will not reappear after a crash.

(Note that the atomicity addressed in the man pages and Unix
specifications is a different one: it deals with the visibility of the
changes in the system, not with the functioning of the file system.)

That's why *BSD + softupdates is still recommended over Linux for pure
mail transfer agents by people.

This still implies the drive doesn't lie to the OS about the completion
of write requests: write cache == off.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Christoph Hellwig]

In article <20010726164516.R17244@emma1.emma.line.org> you wrote:
> On Thu, 26 Jul 2001, Rik van Riel wrote:
>
>> An MTA which relies on this is therefore Broken(tm).

> MTAs rely on TRULY, ULTIMATELY AND DEFINITELY SYNCHRONOUS directory
> updates, nothing else.

And thus they are broken, all caps don't make that less true.

> And because they do so, and most systems have them,

"and most systems have them"...

> MTAs do NOT care how the file system is internally managed, they only
> rely on the rename operation having completed physically on disk before
> the "my rename call has returned 0" event. They expect that with the
> call returning the rename operation has completed ultimately, finally,
> for good, definitely and the old file will not reappear after a crash.

So they rely on undocumented and non standadisized semantics of some
implementations.  I'd call this buggy.

	Christoph

-- 
Whip me.  Beat me.  Make me maintain AIX.

Re: ext3-2.4-0.9.4

[Matthias Andree]

Christoph Hellwig schrieb am Donnerstag, den 26. Juli 2001:

> > MTAs do NOT care how the file system is internally managed, they only
> > rely on the rename operation having completed physically on disk before
> > the "my rename call has returned 0" event. They expect that with the
> > call returning the rename operation has completed ultimately, finally,
> > for good, definitely and the old file will not reappear after a crash.
> 
> So they rely on undocumented and non standadisized semantics of some
> implementations.  I'd call this buggy.

If each in the set of "supported systems" document this behaviour for
themselves, there is no bug. I didn't check however for systems other
than FreeBSD 4.x and Linux. And "Linux support" forces these semantics
with chattr +S, at a high price.

Go tell your opinion to those people that refuse to wrap their
rename/link calls with open()/fsync() calls to the respective parents,
particularly Daniel J. Bernstein, Wietse Z. Venema, among others. I
don't possibly know all MTAs.

You will encounter these or similar questions/objections:

1. what systems apart from Linux need this kind of Pampers?

2. manual lookups of parent directories cause additional overhead better
avoided in performance critical systems.

You would not be the first one to tell them...

Re: ext3-2.4-0.9.4

[Alan Cox]

> Go tell your opinion to those people that refuse to wrap their
> rename/link calls with open()/fsync() calls to the respective parents,
> particularly Daniel J. Bernstein, Wietse Z. Venema, among others. I
> don't possibly know all MTAs.

I've pointed things out to Mr Bernstein before. His normal replies are not
helpful and generally vary between random ravings and threatening to sue
people who publish things on web pages he disagrees with.

Alan

Re: ext3-2.4-0.9.4

[Linus Torvalds]

In article <E15PnTJ-0003z0-00@the-village.bc.nu>,
Alan Cox  <alan@lxorguk.ukuu.org.uk> wrote:
>> Go tell your opinion to those people that refuse to wrap their
>> rename/link calls with open()/fsync() calls to the respective parents,
>> particularly Daniel J. Bernstein, Wietse Z. Venema, among others. I
>> don't possibly know all MTAs.
>
>I've pointed things out to Mr Bernstein before. His normal replies are not
>helpful and generally vary between random ravings and threatening to sue
>people who publish things on web pages he disagrees with.

Now, now, Alan. He has strong opinions, I'll agree, but I've never see
him threaten to _sue_.

Also, I think he eventually agreed on the logic of fsync() on the
directory, and we even had a bug report (quickly fixed) for reiserfs
because it got confused by it.

Of course, knowing Dan, I suspect the fsync() is accompanied by several
lines of derogatory comments about the need for it (not that I've
checked). 

Everybody tends to agree that synchronous IO is stupid and slow, but
some people are just so fixated with "That is how it has been done for
20 years..".

Logging filesystems together with explicit logging points (namely,
"fsync()") are very obviously a superior answer from a technical
standpoint, but that doesn't impact the emotional arguments ("but I want
things to stay the same!"). 

		Linus

Re: ext3-2.4-0.9.4

[Alan Cox]

> >I've pointed things out to Mr Bernstein before. His normal replies are not
> >helpful and generally vary between random ravings and threatening to sue
> >people who publish things on web pages he disagrees with.
> 
> Now, now, Alan. He has strong opinions, I'll agree, but I've never see
> him threaten to _sue_.

Ask Alexey about the end of the syncookie "debate"

Re: ext3-2.4-0.9.4

[Larry McVoy]

On Thu, Jul 26, 2001 at 04:18:59PM +0000, Linus Torvalds wrote:
> In article <E15PnTJ-0003z0-00@the-village.bc.nu>,
> Alan Cox  <alan@lxorguk.ukuu.org.uk> wrote:
> >> Go tell your opinion to those people that refuse to wrap their
> >> rename/link calls with open()/fsync() calls to the respective parents,
> >> particularly Daniel J. Bernstein, Wietse Z. Venema, among others. I
> >> don't possibly know all MTAs.
> >
> >I've pointed things out to Mr Bernstein before. His normal replies are not
> >helpful and generally vary between random ravings and threatening to sue
> >people who publish things on web pages he disagrees with.
> 
> Now, now, Alan. He has strong opinions, I'll agree, but I've never see
> him threaten to _sue_.

In the for what it is worth department, I spent the day with Daniel after
the kernel summit meeting a while back, we talked file systems for about
6 or 7 hours.  While I'll plead guilty to getting mad at him (his ego
is up there with mine :-), I came away impressed with his knowledge.
I get the feeling that he thinks deeply about the problems he works on,
he's probably right a lot of the time, *and* as with many deep thinkers,
he has a problem communicating his ideas.

This is a common problem, and I'm not sure Daniel is fully aware of it.
One cannot expect other people to have done the same thinking and have
the same context, and when they do not, it is easy to get frustrated.
I think that some of Daniel's "ravings" are probably just frustration
that the other person "doesn't get it".

That doesn't mean that Daniel is the right hand of God or anything, I've
seen him do some stupid things but I've seen all of us do some stupid
things, so that doesn't mean much.  I think Daniel does way more smart
things than stupid things, and not all of us can claim that (sort of
like half of the drivers are below average, noone likes that idea either).

What I'm trying to say is that I think Daniel is one of the good guys,
even though his user interface could stand improvement (a common thing
amongst smart people) and it looks like it would be smart to figure out
how to work with him.

Just my opinion...
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: ext3-2.4-0.9.4

[Andre Pang]

On Thu, Jul 26, 2001 at 09:54:52AM -0700, Larry McVoy wrote:

> What I'm trying to say is that I think Daniel is one of the good guys,
> even though his user interface could stand improvement (a common thing
> amongst smart people) and it looks like it would be smart to figure out
> how to work with him.

there's a work-in-progress called ReiserSMTP[1] which rewrites
some bits of qmail so it works better with ReiserFS, although i
can imagine that it would improve things on Linux as a whole.

this is getting off-topic, but since the various parties involved
(Linux vs djb/Wietse/etc[2]) are probably never going to agree
on semantics, i'm wondering if it's possible to ask them to
write the software in such a way that it's possible to 'drop in'
your own functions relevant for sync'ing.  then the MTA writers
can go and use their traditional filesystem assumptions and
Linux users can produce very small patches to support the
correct behaviour under Linux.

it would be _nice_ if the ext3 guys would be more willing to
implement directory-syncing on link/rename/etc, though, even as
an option.  a 'mount -o dirsync' would be enough; no need for
chattr +D stuff.  Linux tends to have a bad name as a platform
as an MTA just because of all this, which is a shame.  it would be
nice if a fix is possible.  *nudge nudge Mr. Morton* :)

    [1] http://www.jedi.claranet.fr/reisersmtp.html

    [2] hey, this might be the first time they agree on
        anything!


-- 
#ozone/algorithm <ozone@algorithm.com.au>          - trust.in.love.to.save

Re: ext3-2.4-0.9.4

[Hans Reiser]

Andre Pang wrote:
> 
> On Thu, Jul 26, 2001 at 09:54:52AM -0700, Larry McVoy wrote:
> 
> > What I'm trying to say is that I think Daniel is one of the good guys,
> > even though his user interface could stand improvement (a common thing
> > amongst smart people) and it looks like it would be smart to figure out
> > how to work with him.
> 
> there's a work-in-progress called ReiserSMTP[1] which rewrites
> some bits of qmail so it works better with ReiserFS, although i
> can imagine that it would improve things on Linux as a whole.

It stopped due to flakiness on the part of all parties including myself, the programmer, and the
sponsor, but it would be nice if a sponsor and programmer came along to restart it.

> 
> this is getting off-topic, but since the various parties involved
> (Linux vs djb/Wietse/etc[2]) are probably never going to agree
> on semantics, i'm wondering if it's possible to ask them to
> write the software in such a way that it's possible to 'drop in'
> your own functions relevant for sync'ing.  then the MTA writers
> can go and use their traditional filesystem assumptions and
> Linux users can produce very small patches to support the
> correct behaviour under Linux.
> 
> it would be _nice_ if the ext3 guys would be more willing to
> implement directory-syncing on link/rename/etc, though, even as
> an option.  a 'mount -o dirsync' would be enough; no need for
> chattr +D stuff.  Linux tends to have a bad name as a platform
> as an MTA just because of all this, which is a shame.  it would be
> nice if a fix is possible.  *nudge nudge Mr. Morton* :)
> 
>     [1] http://www.jedi.claranet.fr/reisersmtp.html
> 
>     [2] hey, this might be the first time they agree on
>         anything!
> 
> --
> #ozone/algorithm <ozone@algorithm.com.au>          - trust.in.love.to.save
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/


No, Linus is right and the MTA guys are just wrong.  The mailers are the place to fix things, not
the kernel.  If the mailer guys want to depend on the kernel being stupidly designed, tough. 
Someone should fix their mailer code and then it would run faster on Linux than on any other
platform.

Hans

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Thu, 26 Jul 2001, Hans Reiser wrote:

> No, Linus is right and the MTA guys are just wrong.  The mailers are
> the place to fix things, not the kernel.  If the mailer guys want to
> depend on the kernel being stupidly designed, tough.  Someone should
> fix their mailer code and then it would run faster on Linux than on
> any other platform.

Well, some systems are even documented that way, so there's nothing with
"depend on the kernel being stupidly designed", but "depend on what
mount(8) says".

MTA authors don't play games, they also write that their software relies
on this behaviour, as laid out.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Sun, 29 Jul 2001, Matthias Andree wrote:
> On Thu, 26 Jul 2001, Hans Reiser wrote:
>
> > No, Linus is right and the MTA guys are just wrong.  The mailers are
> > the place to fix things, not the kernel.  If the mailer guys want to
> > depend on the kernel being stupidly designed, tough.  Someone should
> > fix their mailer code and then it would run faster on Linux than on
> > any other platform.
>
> Well, some systems are even documented that way, so there's nothing
> with "depend on the kernel being stupidly designed", but "depend on
> what mount(8) says".

The key word here is "some systems".

> MTA authors don't play games, they also write that their software
> relies on this behaviour, as laid out.

"MTA authors don't play games" ?!?!

I wonder how that explains things like QMQP or the
next-to-useless bounce messages generated by Notes ;)

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: ext3-2.4-0.9.4

[Hans Reiser]

Matthias Andree wrote:
> 
> On Thu, 26 Jul 2001, Hans Reiser wrote:
> 
> > No, Linus is right and the MTA guys are just wrong.  The mailers are
> > the place to fix things, not the kernel.  If the mailer guys want to
> > depend on the kernel being stupidly designed, tough.  Someone should
> > fix their mailer code and then it would run faster on Linux than on
> > any other platform.
> 
> Well, some systems are even documented that way, so there's nothing with
> "depend on the kernel being stupidly designed", but "depend on what
> mount(8) says".
> 
> MTA authors don't play games, they also write that their software relies
> on this behaviour, as laid out.
> 
> --
> Matthias Andree
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
Documenting their code won't make it fast or well designed.

Hans

Re: ext3-2.4-0.9.4

[Andrew Morton]

Andre Pang wrote:
> 
> it would be _nice_ if the ext3 guys would be more willing to
> implement directory-syncing on link/rename/etc, though, even as
> an option.  a 'mount -o dirsync' would be enough; no need for
> chattr +D stuff.  Linux tends to have a bad name as a platform
> as an MTA just because of all this, which is a shame.  it would be
> nice if a fix is possible.  *nudge nudge Mr. Morton* :)

Perhaps I didn't understand the requirement.

I believe that `dirsync' would provide synchronous metadata
operations (ie: the metadata is crashproofed on-disk when
the syscall returns), but non-sync data.  Correct?

Whereas `mount -o sync' or `chattr +S' would provide synchronous
metadata operations PLUS synchronous data, so when write()
returns, the data which was written is crashproofed.

Is that your understanding of the difference?

If so, then with `dirsync', the application would have to
open the file O_SYNC (which would make the whole thing pointless!)
or it would run fsync() when it had finished writing the file.

So what it boils down to is that dirsync will improve the
efficiency of applications which do a bunch of small writes
and then an fsync.

If, however, the application is capable of doing a nice big
write() (setvbuf!) then really, the two things will be pretty
much the same.

Wait and see how the benchmarks turn out, yes?


One problem at present is that an application could be in the
middle of a nice big write(), but another thread comes up and
does a synchronous creat().  That will force a commit right in the middle
of the write().  It would be better (I think) if the write's transaction
were allowed to run to completion and the creat() caller blocks until
the write() finishes - this way the write(), the creat() and anything
else which happened during the write() would all be written out in a
single compound transaction.

Alas, we cannot run a transaction handle for more than a single
page in write() because of locking inversion problems with i_sem
and the lock_page outside ->writepage().  i_sem is trivial to fix,
but writepage is not.  It has not really proven to be a problem
yet, but it would be nice to be able to _guarantee_ that writes
up to a particular size (100k, say) were 100% atomic.

-

Re: ext3-2.4-0.9.4

[Stephen C. Tweedie]

Hi,

On Fri, Jul 27, 2001 at 02:28:03PM +1000, Andrew Morton wrote:

> I believe that `dirsync' would provide synchronous metadata
> operations (ie: the metadata is crashproofed on-disk when
> the syscall returns), but non-sync data.  Correct?

Not quite: dirsync would provide synchronous metadata operations on
directories, but would make no guarantees about other file types.
That way we don't have the cost of doing sync updates to the inodes or
indirect blocks of regular files --- fsync() is adequate to do that on
demand.

Of course, fsync() is also sufficient to do syncing of directory
operations on demand, but it's a bit heavyweight for that purpose,
hence the request for dirsync (either as a chattr flag or as a mount
option.)

> If, however, the application is capable of doing a nice big
> write() (setvbuf!) then really, the two things will be pretty
> much the same.

Almost --- it's the same for create+write+close+fsync, but not for
rename or for unlink (in which case there's not necessarily going to
be a data fsync to force the directory operation out to disk.)

Cheers,
 Stephen

Re: ext3-2.4-0.9.4

[Lawrence Greenfield]

Hi,

I'm one of those icky application programmers attempting to make
reliable software across different versions of Unix.

We need to get data to disk portably, quickly, and reliably.

I love it when I see things like:  "No, Linus is right and the MTA
guys are just wrong."

This sort of attitude is just ridiculous.  Unix had a defined set of
semantics.  This might have been stupid semantics, but it had them.
Then journalling filesystems, softupdates, and Linux async updates
came along and destroyed those semantics, preventing those of us who
want to write reliable applications using the filesystem from doing
so.  At least Oracle doesn't change the definition of COMMIT.

When I contacted the Linux JFS team about the semantics of link(), I
was told that there is _no way_ of forcing a link() to disk.  Not an
fsync() on the file, not an fsync() on the directory, just _not
possible_.

Great.

Then we come to ext2.  "Oh, just call fsync() on the directory and
you'll be fine."  Well, wait, a second, if ext2 isn't ordering the
metadata writes, a crash at the wrong time (whether or not I've called
fsync()) may lose directory entries---even directory entries unrelated
to the files I'm doing operations on!  Greeeeat.

Thus why all reasonably paranoid MTAs and other mail programs say "use
chattr +S on ext2"---we need ordered metadata writes.

Ok, journalled filesystems are better.  At least crashes aren't going
to affect random files on disk.  But since link() and the like don't
force a commit, we need some way---some reasonably portable way---of
getting that on disk.  On softupdates, calling fsync() on a file
forces all directory entries pointing to that file to disk.  This is
pretty reasonable.  1 fsync() call.

Why do we all cringe when we're told to call fsync() on the directory?
Several reasons:
. not needed on any other variety of Unix
. two fsync() calls force two different syncronization points: the
  application is forcing ordering on the OS that may not be needed.
  (Thus performance doesn't "fly" when you need multiple fsyncs.)
. directory may have other modifications going on that we're not
  interested in

You want to help performance?  Give us an fsync() that works on
multiple file descriptors at once, or an async fsync() call.  Don't
make us fight the OS on getting data to disk.

Larry

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Fri, 27 Jul 2001, Lawrence Greenfield wrote:

> I'm one of those icky application programmers attempting to make
> reliable software across different versions of Unix.
>
> We need to get data to disk portably, quickly, and reliably.
>
> I love it when I see things like:  "No, Linus is right and the MTA
> guys are just wrong."
>
> This sort of attitude is just ridiculous.  Unix had a defined set of
> semantics.  This might have been stupid semantics, but it had them.

The stuff you people seem to insist on, however, most
definately isn't part of the defined set of semantics.

If you believe otherwise, feel free to point out the
relevant sections in POSIX / SuS / ...

regards,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Fri, 27 Jul 2001, Rik van Riel wrote:

> The stuff you people seem to insist on, however, most
> definately isn't part of the defined set of semantics.

And even if it's "inherited wisdom", you cannot simply tell those people
"don't rely on that" if - as claimed - you can't even force a link() to
disk.

> If you believe otherwise, feel free to point out the
> relevant sections in POSIX / SuS / ...

The standard is only useful if it specifies how to get data safely on
disk - it is quite explicit for fsync(), but you evidently cannot
fsync() a link().

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Sun, 29 Jul 2001, Matthias Andree wrote:

> The standard is only useful if it specifies how to get data safely on
> disk - it is quite explicit for fsync(), but you evidently cannot
> fsync() a link().

As Linus said, fsync() on the directory.

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Sat, 28 Jul 2001, Rik van Riel wrote:

> > The standard is only useful if it specifies how to get data safely on
> > disk - it is quite explicit for fsync(), but you evidently cannot
> > fsync() a link().
> 
> As Linus said, fsync() on the directory.

Relying on that to work on other operating systems is no better than
demanding synchronous meta data writes: relying on undocumented
behaviour.

If we spake about Linux-specific applications, that'd be okay, but we
speak about portable applications, and the diversity is bigger than
useful. Speed is not the only problem the OS has to solve.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Mike Touloumtzis]

On Sun, Jul 29, 2001 at 02:08:12AM +0200, Matthias Andree wrote:
> On Sat, 28 Jul 2001, Rik van Riel wrote:
> 
> > As Linus said, fsync() on the directory.
> 
> Relying on that to work on other operating systems is no better than
> demanding synchronous meta data writes: relying on undocumented
> behaviour.

You are blurring the boundaries between "undocumented behavior" and
"OS-specific behavior".  fsync() on a directory to sync metadata is a
defined (according to my copy of fsync(2)), Linux-specific behavior.
It is also very reasonable IMHO and in keeping with the traditional
Unix notion of directories as lists of files.

I argue that using defined Linux behavior to implement what you want
on Linux systems _is_ better than relying on undocumented behavior,
and I think most people would agree.  If you don't do this you have
not really ported the software to Linux; you instead have some
standards compliant software that "kinda usually works on Linux".
You could argue that no one should localize their software to
different versions of Unix, but you would be by far in the minority.

http://www.google.com/search?q=autoconf

Writing portable Unix software has always meant some degree
of system-specific accomodation.  It's a bummer but it's life;
otherwise Unix wouldn't evolve.

miket

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Sat, 28 Jul 2001, Mike Touloumtzis wrote:

> You are blurring the boundaries between "undocumented behavior" and
> "OS-specific behavior".  fsync() on a directory to sync metadata is a
> defined (according to my copy of fsync(2)), Linux-specific behavior.
> It is also very reasonable IMHO and in keeping with the traditional
> Unix notion of directories as lists of files.

No-one claims that fsync() the directory is a bad interface - it's
non-portable however. Actually, chattr +S is well-documented - it just
doesn't work on ReiserFS or Minix for now, and it may be unnecessarily
slow on ext2.

As pointed out more than once, "synchronous meta data" is documented e.
g.  for FreeBSD, so in at least these two cases, the box relies on
documented behaviour.

> http://www.google.com/search?q=autoconf
> 
> Writing portable Unix software has always meant some degree
> of system-specific accomodation.  It's a bummer but it's life;
> otherwise Unix wouldn't evolve.

How can autoconf figure if you need to fsync() the directory? Apart from
that, which Unix MTA uses autoconf?

Remember, the whole discussion is about getting rid of the need for
chattr +S and offering the admin the chance to mount or flag a directory
for synchronous meta data updates.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Sun, 29 Jul 2001, Matthias Andree wrote:

> How can autoconf figure if you need to fsync() the directory? Apart
> from that, which Unix MTA uses autoconf?

Zmailer uses autoconf, Exim also has some nice
tool to make itself build for the right OS using
the right interfaces.

regards,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: ext3-2.4-0.9.4

[Mike Touloumtzis]

On Sun, Jul 29, 2001 at 11:28:10AM +0200, Matthias Andree wrote:
> 
> How can autoconf figure if you need to fsync() the directory? Apart from
> that, which Unix MTA uses autoconf?

My point was not that they should be using autoconf;
I don't know if they are or not.  My point was that
they should use existing published interfaces that are
reasonable, rather than push for guarantees that impose
new requirements on filesystems.  And even without
autoconf it's not hard to figure out what system you're
running on.

    rename(tmpfile, spoolfile);
#ifdef __linux___
    fsync(tmpdir);
    fsync(spooldir);
#endif
    /* transaction is complete */

> 
> Remember, the whole discussion is about getting rid of the need for
> chattr +S and offering the admin the chance to mount or flag a directory
> for synchronous meta data updates.

Right; and I'm arguing that the way to get rid of the need
for chattr +S is to incorporate directory fsync() in the
MTAs, not to cram more features into the filesystems.

Problem: MTA needs to know when rename() has been forced
to disk.

Solution 1: MTA authors use fsync(dirfd) on Linux.

Analysis: This is not the most portable solution, but it
should work on any FS that supports Linux semantics.  You
can't expect such semantics on FAT and other filesystems
that are just supported for compatibility reasons.  But you
could, say, switch filesystems for performance reasons, and
not have your MTA start mysteriously failing, because you
are using the official, documented API to do what you want
to do (at the very least you would be in a much stronger
position when pushing a bug fix :-).

Solution 2: Linux semantics are changed so that rename()
returns only when the data hits the disk.  All filesystems
are expected to implement this change.

Analysis: This sucks.  It precludes some filesystem design
choices, prevents users from making a speed/reliability
tradeoff, and makes each filesystem more complex.

Solution 3: Some filesystems implement synchronous
directory updates for renames, using filesystem-specific
feature flags, chattr, etc.

Analysis: I wouldn't want to try to dictate anything to
the FS authors, but this solution seems inferior to me.
Each filesystem would have to implement such a flag to
become "MTA compatible".  Why add a complex feature to the
filesystem when it can already be accessed via a userspace
API?  It will be more complex for administrators too --
they will have to know which filesystems implement the
synchronous directory metadata.

There are lots of filesystems out there.  Why not use
an interface they should all support rather than ask for
per-filesystem, filesystem-specific improvements?

miket

Re: ext3-2.4-0.9.4

[Ketil Froyn]

On Sun, 29 Jul 2001, Matthias Andree wrote:

> On Sat, 28 Jul 2001, Mike Touloumtzis wrote:
>
> > You are blurring the boundaries between "undocumented behavior" and
> > "OS-specific behavior".  fsync() on a directory to sync metadata is a
> > defined (according to my copy of fsync(2)), Linux-specific behavior.
> > It is also very reasonable IMHO and in keeping with the traditional
> > Unix notion of directories as lists of files.

> > http://www.google.com/search?q=autoconf
> >
> > Writing portable Unix software has always meant some degree
> > of system-specific accomodation.  It's a bummer but it's life;
> > otherwise Unix wouldn't evolve.
>
> How can autoconf figure if you need to fsync() the directory?

Simple! Grep the fsync(2) manpage ;)

Ketil the joker

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Sun, 29 Jul 2001, Matthias Andree wrote:
> On Sat, 28 Jul 2001, Rik van Riel wrote:
>
> > > The standard is only useful if it specifies how to get data safely on
> > > disk - it is quite explicit for fsync(), but you evidently cannot
> > > fsync() a link().
> >
> > As Linus said, fsync() on the directory.
>
> Relying on that to work on other operating systems is no better than
> demanding synchronous meta data writes: relying on undocumented
> behaviour.
>
> If we spake about Linux-specific applications, that'd be okay, but we
> speak about portable applications, and the diversity is bigger than
> useful. Speed is not the only problem the OS has to solve.

I guess many MTAs have a small libc inside of them exactly
in order to handle things like this without fouling up the
core code too much.

Time to make your favorite MTA use link_slowly()  ;)

cheers,

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: ext3-2.4-0.9.4

[Bill Rugolsky Jr.]

On Fri, Jul 27, 2001 at 12:24:56PM -0400, Lawrence Greenfield wrote:
> I love it when I see things like:  "No, Linus is right and the MTA
> guys are just wrong."
> 
> This sort of attitude is just ridiculous.  Unix had a defined set of
> semantics.  This might have been stupid semantics, but it had them.
> Then journalling filesystems, softupdates, and Linux async updates
> came along and destroyed those semantics, preventing those of us who
> want to write reliable applications using the filesystem from doing
> so.  At least Oracle doesn't change the definition of COMMIT.

First off, would you care to quote chapter and verse of these
"defined semantics" ?   Do you mean the BSD source?

Traditional FFS/UFS achieves "safety" at a terrible cost to
performance.  I can barely stand the wait to untar XFree86 on Solaris8
on a PII-333, even with UFS logging -- I'd rather use my Pentium 166
laptop running Linux!  ext2 solved this performance issue many years
ago by recognizing that the FFS metadata scheme was not really safe
either; instead the intelligence was put into e2fsck, and where
necessary, the applications.  (Do I hear faint echoes of the
"lint" v. "cc" design criterion ... ?)

The infrastructure is now in place to solve these problems in ext3,
without imposing a least-common-denominator approach that degrades
overall system performance.  In these instances "Linus is right" when
he notes that (1) the proposed immediate solution does not really solve
the problem, and (2) once in there, developers will rely on its precise
semantics, making them difficult to get right later on, and providing
no incentive to do so.  In many such instances "undefined" behavior is
the best intermediate solution.

As one can see from the "gkernel-commit" traffic, Andrew Morton has
not only taken away useful information from this thread, he's already
halfway to a solution, in just a day, because  Matthias Andree took
the time to describe the functional requirements instead of just
whining that "it's not like BSD."
 
> Thus why all reasonably paranoid MTAs and other mail programs say "use
> chattr +S on ext2"---we need ordered metadata writes.

And that's precisely the type of thing we want -- unused features should
not impact the rest of the system.
 
Regards,

   Bill Rugolsky

Re: ext3-2.4-0.9.4

[Larry McVoy]

Arrg, I take it all back, I'm taking about Daniel Phillips not Daniel
Bernstein.  I tend to agree with Alan about Mr Bernstein.

Thanks to Richard Gooch for pointing out that I'm asleep at the switch.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: ext3-2.4-0.9.4

[Daniel Phillips]

On Thursday 26 July 2001 18:54, Larry McVoy wrote:
> On Thu, Jul 26, 2001 at 04:18:59PM +0000, Linus Torvalds wrote:
> > In article <E15PnTJ-0003z0-00@the-village.bc.nu>,
> >
> > Alan Cox  <alan@lxorguk.ukuu.org.uk> wrote:
> > >> Go tell your opinion to those people that refuse to wrap their
> > >> rename/link calls with open()/fsync() calls to the respective
> > >> parents, particularly Daniel J. Bernstein, Wietse Z. Venema,
> > >> among others. I don't possibly know all MTAs.
> > >
> > >I've pointed things out to Mr Bernstein before. His normal replies
> > > are not helpful and generally vary between random ravings and
> > > threatening to sue people who publish things on web pages he
> > > disagrees with.
> >
> > Now, now, Alan. He has strong opinions, I'll agree, but I've never
> > see him threaten to _sue_.
>
> In the for what it is worth department, I spent the day with Daniel
> after the kernel summit meeting a while back, we talked file systems
> for about 6 or 7 hours.  While I'll plead guilty to getting mad at
> him (his ego is up there with mine :-), I came away impressed with
> his knowledge. I get the feeling that he thinks deeply about the
> problems he works on, he's probably right a lot of the time, *and* as
> with many deep thinkers, he has a problem communicating his ideas.
>
> This is a common problem, and I'm not sure Daniel is fully aware of
> it. One cannot expect other people to have done the same thinking and
> have the same context, and when they do not, it is easy to get
> frustrated. I think that some of Daniel's "ravings" are probably just
> frustration that the other person "doesn't get it".
>
> That doesn't mean that Daniel is the right hand of God or anything,
> I've seen him do some stupid things but I've seen all of us do some
> stupid things, so that doesn't mean much.  I think Daniel does way
> more smart things than stupid things, and not all of us can claim
> that (sort of like half of the drivers are below average, noone likes
> that idea either).
>
> What I'm trying to say is that I think Daniel is one of the good
> guys, even though his user interface could stand improvement (a
> common thing amongst smart people) and it looks like it would be
> smart to figure out how to work with him.
>
> Just my opinion...

Heh, very interesting, but you seem to have created a collage of two
different Daniels ;-)

--
Daniel

Re: ext3-2.4-0.9.4

[Richard A Nelson]

On Thu, 26 Jul 2001, Linus Torvalds wrote:

> In article <E15PnTJ-0003z0-00@the-village.bc.nu>,
> Alan Cox  <alan@lxorguk.ukuu.org.uk> wrote:
> >> Go tell your opinion to those people that refuse to wrap their
> >> rename/link calls with open()/fsync() calls to the respective parents,
> >> particularly Daniel J. Bernstein, Wietse Z. Venema, among others. I
> >> don't possibly know all MTAs.

[snip]
> Also, I think he eventually agreed on the logic of fsync() on the
> directory, and we even had a bug report (quickly fixed) for reiserfs
> because it got confused by it.

In looking at the synchronous directory options, I'm unsure as to
the 'real' status wrt fsync() on a directory:
	1) Does fsync() of a directory work on most/all current FS?
	2) Does it work on 2.2.x as well as 2.4.x?
-- 
Rick Nelson
"... being a Linux user is sort of like living in a house inhabited
by a large family of carpenters and architects. Every morning when
you wake up, the house is a little different. Maybe there is a new
turret, or some walls have moved. Or perhaps someone has temporarily
removed the floor under your bed." - Unix for Dummies, 2nd Edition
	-- found in the .sig of Rob Riggs, rriggs@tesser.com

Re: ext3-2.4-0.9.4

[Linus Torvalds]

On Thu, 26 Jul 2001, Richard A Nelson wrote:
>
> In looking at the synchronous directory options, I'm unsure as to
> the 'real' status wrt fsync() on a directory:
> 	1) Does fsync() of a directory work on most/all current FS?

Modulo bugs, yes.

Now, there's another issue, of course: if you have an important mail-spool
on some of the less tested filesystems, I would consider you crazy
regardless of fsync() working ;). I don't think anybody has ever verified
that fsync() (or much anything else wrt writing) does the right thing on
NTFS, for example.

> 	2) Does it work on 2.2.x as well as 2.4.x?

Yes. However, there may be performance issues. As with just about
anything, we didn't start optimizing things until it became a real issue,
and in some cases at least historically the filesystems fell back on just
doing a whole "fsync_dev()" if they had nothing better to do.

I think later 2.2.x kernels (ie the ones past the point where Alan took
over) probably have the fsync() optimizations at least for ext2.

		Linus

Re: ext3-2.4-0.9.4

[Richard A Nelson]

On Thu, 26 Jul 2001, Linus Torvalds wrote:

> > 	1) Does fsync() of a directory work on most/all current FS?
>
> Modulo bugs, yes.

Great, that was a big concern

> Now, there's another issue, of course: if you have an important mail-spool
> on some of the less tested filesystems, I would consider you crazy
> regardless of fsync() working ;). I don't think anybody has ever verified
> that fsync() (or much anything else wrt writing) does the right thing on
> NTFS, for example.

Caveat Emptor ;-)

> > 	2) Does it work on 2.2.x as well as 2.4.x?
>
> Yes. However, there may be performance issues. As with just about
> anything, we didn't start optimizing things until it became a real issue,
> and in some cases at least historically the filesystems fell back on just
> doing a whole "fsync_dev()" if they had nothing better to do.
>
> I think later 2.2.x kernels (ie the ones past the point where Alan took
> over) probably have the fsync() optimizations at least for ext2.

That should be recent enough - I push 2.2.19 for shm support and security
reasons anyway - though I see alot of folk on 2.2.16/17.

Are the optimizations more than writing out only changed blocks?
Has anyone any information on the performance differences between
optimized vs non-optimized?

Thanks, I'm feeling much better about getting this support added
-- 
Rick Nelson
Life'll kill ya                         -- Warren Zevon
Then you'll be dead                     -- Life'll kill ya

Re: ext3-2.4-0.9.4

[Anton Altaparmakov]

At 20:37 26/07/2001, Linus Torvalds wrote:
>On Thu, 26 Jul 2001, Richard A Nelson wrote:
> > In looking at the synchronous directory options, I'm unsure as to
> > the 'real' status wrt fsync() on a directory:
> >       1) Does fsync() of a directory work on most/all current FS?
>
>Modulo bugs, yes.
>
>Now, there's another issue, of course: if you have an important mail-spool
>on some of the less tested filesystems, I would consider you crazy
>regardless of fsync() working ;). I don't think anybody has ever verified
>that fsync() (or much anything else wrt writing) does the right thing on
>NTFS, for example.

NTFS doesn't even have an fsync() operation defined so calling fsync() 
system call won't do anything at all. A quick look at 
fs/buffer.c::sys_fsync() shows it will return -EINVAL straight away.

But considering the fsync, even if present may well trash the file or the 
whole partition's data, it's just as well it doesn't happen...

Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Thu, 26 Jul 2001, Matthias Andree wrote:
> Christoph Hellwig schrieb am Donnerstag, den 26. Juli 2001:
>
> > So they rely on undocumented and non standadisized semantics of some
> > implementations.  I'd call this buggy.
>
> If each in the set of "supported systems" document this
> behaviour for themselves, there is no bug.

The MTA depends on behaviour which is undefined. Now you
want to go blame the OS ?

> Go tell your opinion to those people that refuse to wrap their
> rename/link calls with open()/fsync() calls to the respective parents,
> particularly Daniel J. Bernstein, Wietse Z. Venema, among others. I
> don't possibly know all MTAs.

If you care about your email, probably you should either
teach these people about standards like POSIX or SuS
(and tell them to not rely on undefined behaviour) or
switch to an MTA which isn't broken in various ways ;)

cheers,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Alan Cox]

> If you care about your email, probably you should either
> teach these people about standards like POSIX or SuS
> (and tell them to not rely on undefined behaviour) or
> switch to an MTA which isn't broken in various ways ;)

POSIX and SuS are actually not helpful here. They don't cover how to force
namespace to disk, only data and metadata for the file. So you can portably
stick your data onto disk, portably be sure its on disk, but not portably be
sure the directory entries are on disk.

Alan

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Thu, 26 Jul 2001, Rik van Riel wrote:

> The MTA depends on behaviour which is undefined. Now you
> want to go blame the OS ?

No, the behaviour is defined on certain systems. Not sure if that
comprises all supported systems.

I'm not blaming anybody besides Linux which does not offer the "noasync"
(FreeBSD) compromise between sync and async. I don't see any reason why
this option cannot be there. Is it too expensive too implement? No-one
said so.

I cannot tell if and how the MTA authors checked all their supported OSs
how they handle metadata updates.

> If you care about your email, probably you should either
> teach these people about standards like POSIX or SuS
> (and tell them to not rely on undefined behaviour) or
> switch to an MTA which isn't broken in various ways ;)

Wee. And then, I tell the system to comply with that as well, don't I?
;)

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Alan Cox]

> them, and MTAs are portable, they choose chattr +S on Linux. And that's
> a performance problem because it doesn't come for free, but also with
> synchronous data updates, which are unnecessary because there is
> fsync().

chattr +S and atomic updates hitting disk then returning to the app will
give the same performance. You can also fsync() the directory. 

> the "my rename call has returned 0" event. They expect that with the
> call returning the rename operation has completed ultimately, finally,
> for good, definitely and the old file will not reappear after a crash.

Actually the old file re-appearing after the crash is irrelevant. It will
have a previously logged message id. And if you are not doing message id
histories then you have replay races at the SMTP level anyway

> This still implies the drive doesn't lie to the OS about the completion
> of write requests: write cache == off.

Write cache off is not a feature available on many modern disks. You
already lost the battle before you started.

Alan

Re: ext3-2.4-0.9.4

[Gérard Roudier]

On Thu, 26 Jul 2001, Alan Cox wrote:

> > them, and MTAs are portable, they choose chattr +S on Linux. And that's
> > a performance problem because it doesn't come for free, but also with
> > synchronous data updates, which are unnecessary because there is
> > fsync().
>
> chattr +S and atomic updates hitting disk then returning to the app will
> give the same performance. You can also fsync() the directory.
>
> > the "my rename call has returned 0" event. They expect that with the
> > call returning the rename operation has completed ultimately, finally,
> > for good, definitely and the old file will not reappear after a crash.
>
> Actually the old file re-appearing after the crash is irrelevant. It will
> have a previously logged message id. And if you are not doing message id
> histories then you have replay races at the SMTP level anyway
>
> > This still implies the drive doesn't lie to the OS about the completion
> > of write requests: write cache == off.
>
> Write cache off is not a feature available on many modern disks. You
> already lost the battle before you started.

Losing the battle of brain-dead hardware is not a problem... :-)

SCSI hard disks are expected to follow the specifications. But, may be,
you are referring to IDE disks, only ...

With SCSI, you can enable write caching and also ask the device to signal
completion of actual write to the media by setting the FUA bit in the SCSI
command block (not available in WRITE(6), but available in WRITE(10)).

  Gérard.

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Thu, 26 Jul 2001, Alan Cox wrote:

> Rik is right. It isnt just about premature notification - its about 
> atomicity. At the point you are notified the data has been queued for disk
> I/O. Even on traditional BSD ufs with synchronous metadata you still had
> points where a crash left the rename partially complete and nothing but a
> log or an atomic update system is going to fix that.

No. Atomic update systems and logs can by no means fix premature
acknowledgements:

Proof:

Assume the OS has a phase tree kind of thing or log that requires
just a single-block write for an atomic rename.

Assume an MTA calls rename(), and the OS by whatever means notifies it of
completion, but actually, the data is only queued, not written.

Assume The MTA receives the acknowledgement (e. g. rename call
returned), sends a "250 mail action complete" packet across the network.

Assume the machine sends the network packed, but not the queued disk
block and then crashes.

--> The single block is lost, the rename operation is lost, but the
operation had been acknowledged. Consequence: the mail is lost. q. e. d.

All this boils down to: 

1. The OS _MUST_ know when a write operation has been physically
committed to non-volatile storage.

2. The OS _MUST_ _NOT_ acknowledge the (assumedly synchronous operation)
any earlier. (This may well include switching off drive write
buffering.)

If the OS cannot fulfill these two basic requirements, I can save all
the log or FS atomicity efforts because they don't get me anywhere.

The problem is not that the operation can fail, the problem IS premature
acknowledgement. Even with atomic updates, as shown above.

Note, of course there is no premature acknowledgement for the
Linux-default asynchronous directory update. There IS for -o sync or
chattr +S -- and that's what MTAs to to guarantee data integrity, and
that's why I'm still suggesting dirsync or something to remedy the
negative data write performance of full-sync.

If the OS tell me "write completed" when it means "I queued your data
for writing", it is BROKEN.

That's my point.

And since the common POSIX OS lacks a dedicated notification feature for
e. g. rename, MTAs have no other choice than to rely on "has completed
when the syscall returns".

BTW, my Linux rename(2) man page doesn't document EIO condition, FreeBSD
4.3-STABLE and SUS v2 do.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Daniel Phillips]

On Thursday 26 July 2001 16:32, Matthias Andree wrote:
> On Thu, 26 Jul 2001, Alan Cox wrote:
> > Rik is right. It isnt just about premature notification - its about
> > atomicity. At the point you are notified the data has been queued
> > for disk I/O. Even on traditional BSD ufs with synchronous metadata
> > you still had points where a crash left the rename partially
> > complete and nothing but a log or an atomic update system is going
> > to fix that.
>
> No. Atomic update systems and logs can by no means fix premature
> acknowledgements:
>
> Proof:
>
> Assume the OS has a phase tree kind of thing or log that requires
> just a single-block write for an atomic rename.
>
> Assume an MTA calls rename(), and the OS by whatever means notifies
> it of completion, but actually, the data is only queued, not written.
>
> Assume The MTA receives the acknowledgement (e. g. rename call
> returned), sends a "250 mail action complete" packet across the
> network.
>
> Assume the machine sends the network packed, but not the queued disk
> block and then crashes.
>
> --> The single block is lost, the rename operation is lost, but the
> operation had been acknowledged. Consequence: the mail is lost. q. e.
> d.
>
> All this boils down to:
>
> 1. The OS _MUST_ know when a write operation has been physically
> committed to non-volatile storage.

We're working on that, see the "[PATCH] 64 bit scsi read/write" thread 
on linux-fsdevel.  About half of it is devoted to investigating the 
detailed semantics of physical write completion.

> 2. The OS _MUST_ _NOT_ acknowledge the (assumedly synchronous
> operation) any earlier. (This may well include switching off drive
> write buffering.)

Yes, for now that's how you have to do it.

> If the OS cannot fulfill these two basic requirements, I can save all
> the log or FS atomicity efforts because they don't get me anywhere.
>
> The problem is not that the operation can fail, the problem IS
> premature acknowledgement. Even with atomic updates, as shown above.

Right now the interface for determining that the operation has actually 
completed is "sync".  Yes, that sucks but with journalling or atomic 
commit it's not nearly as expensive as you might think.  My early flush 
patch does nearly the equivalent of sync, 10 times a second and it 
actually improves performance (it does not attempt to do this under 
high load of course).

We *should* have something like sys_sync_dev(majorminor) or 
sys_sync_fs(mountpoint) (whatever that would look like).  For 
phase-tree the semantics are that the call doesn't return until the 
metaroot of the then-current "branching" tree is known to be safely on 
disk.  (Side note: it's ok to allow subsequent updates on the same 
filesystem to procede while an outstanding sync_dev is waiting for 
confirmation from the block layer, because these don't affect the 
filesystem state the sync_fs is waiting on.)

As I understand it, Ext2 allows much the same semantics.  While we do 
need to do something about exposing a more elegant interface, with Ext3 
you should be ok with +S and a "sync" just before you report to the 
world that the mail transaction is complete.  Ext3 does *not* leave a 
lot of dirty blocks hanging around in normal operation, so sync is not 
nearly as slow as it is with good old Ext2.

> Note, of course there is no premature acknowledgement for the
> Linux-default asynchronous directory update. There IS for -o sync or
> chattr +S -- and that's what MTAs to to guarantee data integrity, and
> that's why I'm still suggesting dirsync or something to remedy the
> negative data write performance of full-sync.
>
> If the OS tell me "write completed" when it means "I queued your data
> for writing", it is BROKEN.
>
> That's my point.
>
> And since the common POSIX OS lacks a dedicated notification feature
> for e. g. rename, MTAs have no other choice than to rely on "has
> completed when the syscall returns".
>
> BTW, my Linux rename(2) man page doesn't document EIO condition,
> FreeBSD 4.3-STABLE and SUS v2 do.

Sounds like a man page bug.

--
Daniel

Re: ext3-2.4-0.9.4

[Andrew Morton]

Daniel Phillips wrote:
> 
> Ext3 does *not* leave a
> lot of dirty blocks hanging around in normal operation, so sync is not
> nearly as slow as it is with good old Ext2.

eek.

In fully-journalled data mode, we write everything to the journal
in a linear chunk, wait on it, write a commit block, wait on that
and then release all the just-journalled data into the main
filesystem for conventional bdflush/kupdate writeback in twenty
seconds time.

Calling anything which uses fsync_dev() would cause all that writeback
data to be written out and waited on, with the consequential seeking
storm.  Disastrous.

Note that fsync() is OK - in full data journalling mode nothing
is ever attached to i_dirty_buffers.

-

Re: ext3-2.4-0.9.4

[Daniel Phillips]

On Thursday 26 July 2001 17:49, Andrew Morton wrote:
> Daniel Phillips wrote:
> > Ext3 does *not* leave a
> > lot of dirty blocks hanging around in normal operation, so sync is
> > not nearly as slow as it is with good old Ext2.
>
> eek.
>
> In fully-journalled data mode, we write everything to the journal
> in a linear chunk, wait on it, write a commit block, wait on that
> and then release all the just-journalled data into the main
> filesystem for conventional bdflush/kupdate writeback in twenty
> seconds time.
>
> Calling anything which uses fsync_dev() would cause all that
> writeback data to be written out and waited on, with the
> consequential seeking storm.  Disastrous.

Whoops, ok, no, this is not particularly sync-friendly.  On the other
hand, I don't think your seek storm would be as bad as all that.  You
can still feed enough blocks to the elevator to give it something to
chew on.  On the third hand, since you are still using the generic
flushing machinery I can see you'd have quite a lot of work to do to
control the flushing accurately in this way.

> Note that fsync() is OK - in full data journalling mode nothing
> is ever attached to i_dirty_buffers.

Somewhere in there is a beautiful optimization trying to get out...

--
Daniel

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Thu, 26 Jul 2001, Daniel Phillips wrote:

> As I understand it, Ext2 allows much the same semantics.  While we do 
> need to do something about exposing a more elegant interface, with Ext3 
> you should be ok with +S and a "sync" just before you report to the 
> world that the mail transaction is complete.  Ext3 does *not* leave a 
> lot of dirty blocks hanging around in normal operation, so sync is not 
> nearly as slow as it is with good old Ext2.

That wasn't my impression, particularly not with data=journalling which
can drop data into the log. It's just: why sync the world if synching
directories does the job and relevant data is synched manually with
fsync()?

However, how big are chances that these interfaces will spread outside
of Linux? That's the crucial point for portable applications. If it's a
kernel <-> libc interface, OK, no problem, but if it's a user-space
interface, it might easily become a useless invention because no-one
uses it in real life. You don't support multiple interfaces in a
portable application because that's a maintenance disaster and often
causes reliability problems because on different platforms, code takes
different paths, so applications won't usually choose limited-use
interfaces (such as sendfile).

BTW, your Message-ID is unqualified == on a collision course in mail
duplicate killers.

Re: ext3-2.4-0.9.4

[Andrew Morton]

Matthias Andree wrote:
>  
> A much better file system for an MTA might be ext3fs with
> data=journalled and dirsync mount/chattr option.

OK, I've taken a closer look at this.  ext3 has picked up some
cruft from ext2's sync handling which it does not need in the
least.

It will be fairly straightforward and a useful cleanup to
provide the following semantics for either synchronous
mounts or `chattr +S' directories:

* All metadata operations (rename, unlink, link, symlink, etc)
  will be synchronous.  So when the system call returns, the data
  is crash-proofed.

* All write()s will be synchronous.  So when the write() system
  call returns, the data written and all associated metadata
  will be crash-proofed.

  O_SYNC and fsync() will not be necessary - in fact they'll
  slow things down slightly by forcing an unnecessary and
  probably empty commit.

If you crash in the middle of a write, you may end up with a truncated
file on recovery.

This is in fact the behaviour right now, but the performance is
not good.

The performance problem at present is that large write()s have unnecessary
commits in the middle of them.  This is due to the abovementioned
cruft in ext3_get_block() and the things it calls.

> Would you deem it
> possible to get such an option done before ext3fs 1.0.0?

We'd prefer not - we're trying to stabilise things quite
sternly at present. However that doesn't prevent work
on 1.1.0 :)

Seems like a worthwhile thing to do - I'll cut a branch
and do this.  It'll take a couple of weeks - as usual, most
of the work is in development and use of test tools...
But I can't predict at this time when we'll merge it into
the mainline fs.

> I hope this makes the requirements of this particular group of
> applications clear.

Yes, it was useful - thanks.

-

RFC: chattr/lsattr +DS? was: ext3-2.4-0.9.4

[Matthias Andree]

On Fri, 27 Jul 2001, Andrew Morton wrote:

> > Would you deem it
> > possible to get such an option done before ext3fs 1.0.0?
> 
> We'd prefer not - we're trying to stabilise things quite
> sternly at present. However that doesn't prevent work
> on 1.1.0 :)
> 
> Seems like a worthwhile thing to do - I'll cut a branch
> and do this.  It'll take a couple of weeks - as usual, most
> of the work is in development and use of test tools...
> But I can't predict at this time when we'll merge it into
> the mainline fs.

So the summary of all this is, as I understand it: for ext3fs 1.0, treat
it with chattr +S and the like as if it was ext2fs, it may or may not be
faster with "mount -o data=journalled" and is well worthwhile for an MTA
to try, a weaker sync option may be introduced after ext3fs 1.0.

Sounds good.

I'm dropping the ext3-users mailing list for now since this is getting
more general.


However, since the ReiserFS team also showed interest in a similar
functionality, and they don't yet support chattr, would it be useful to
specify a "D" option for chattr already?

I have a suggestion: if D is set, but S isn't, no effect. If S is set,
but D is unset, treat S as in the past. If S is set, and D is set,
directory updates are synchronous like with S, but data updates are
asynchronous in spite of S.

This way, booting a kernel without chattr "D" flag support or mounting
the file system as ext2 would have it default to the safer
everything-synchronously mode.

-- 
Matthias Andree

Re: RFC: chattr/lsattr +DS? was: ext3-2.4-0.9.4

[Andrew Morton]

Matthias Andree wrote:
> 
> On Fri, 27 Jul 2001, Andrew Morton wrote:
> 
> > > Would you deem it
> > > possible to get such an option done before ext3fs 1.0.0?
> >
> > We'd prefer not - we're trying to stabilise things quite
> > sternly at present. However that doesn't prevent work
> > on 1.1.0 :)
> >
> > Seems like a worthwhile thing to do - I'll cut a branch
> > and do this.  It'll take a couple of weeks - as usual, most
> > of the work is in development and use of test tools...
> > But I can't predict at this time when we'll merge it into
> > the mainline fs.
> 
> So the summary of all this is, as I understand it: for ext3fs 1.0, treat
> it with chattr +S and the like as if it was ext2fs, it may or may not be
> faster with "mount -o data=journalled" and is well worthwhile for an MTA
> to try, a weaker sync option may be introduced after ext3fs 1.0.
> 
> Sounds good.
> 
> I'm dropping the ext3-users mailing list for now since this is getting
> more general.
> 
> However, since the ReiserFS team also showed interest in a similar
> functionality, and they don't yet support chattr, would it be useful to
> specify a "D" option for chattr already?

chattr is an ext[23]-specific thing.  reiserfs could certainly
support a similar thing if they have a few bits spare in the
inode.

> I have a suggestion: if D is set, but S isn't, no effect. If S is set,
> but D is unset, treat S as in the past. If S is set, and D is set,
> directory updates are synchronous like with S, but data updates are
> asynchronous in spite of S.

I don't think this would be needed until really proven necessary - for
data, fsync() should work for all filesystems.

There would be one benefit in splitting sync from datasync,
and that is for applications which do not write() their
data in large enough chunks.

When I fix the get_block thing, O_SYNC, `chattr +S' and `mount
-o sync' will provide good, fast synchronous write()s - the
fs will run a commit at the end of the write().  That's just fine as long
as the app is writing its data in goodly chunks.  If it is is using 4k
or 8k chunks (eg: default stdio) then throughput will suffer.  That
would be rather silly of it though.

-

Re: ext3-2.4-0.9.4

[Linus Torvalds]

In article <20010726143002.E17244@emma1.emma.line.org>,
Matthias Andree  <matthias.andree@stud.uni-dortmund.de> wrote:
>
>However, the remaining problem is being synchronous with respect to open
>(fixed for ext3 with your fsync() as I understand it), rename, link and
>unlink. With ext2, and as you write it, with ext3 as well, there is
>currently no way to tell when the link/rename has been committed to
>disk, unless you set mount -o sync or chattr +S or call sync() (the
>former is not an option because it's far too expensive).

Congratulations. You have been brainwashed by Dan Bernstein.

Use fsync() on the directory. 

Logical, isn't it?

		Linus

Re: ext3-2.4-0.9.4

[Matti Aarnio]

On Thu, Jul 26, 2001 at 03:51:35PM +0000, Linus Torvalds wrote:
> To:	linux-kernel@vger.kernel.org
> From:	torvalds@transmeta.com (Linus Torvalds)
> Subject: Re: ext3-2.4-0.9.4
> Date:	Thu, 26 Jul 2001 15:51:35 +0000 (UTC)
....
> Use fsync() on the directory. 
> 
> Logical, isn't it?

  No.  I don't see why I should opendir() a directory, fsync()
that handle, and closedir() the handle.  I would definitely prefer:

       lsync(dirpath)

This could, even, behave like  lstat()  with the path: if the last name
segment is symlink, the sync is done on the i-node data of symlink, not
on what it (possibly) points to.

I didn't check if POSIX folks have thought of that.

> 		Linus

/Matti Aarnio

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Tue, 31 Jul 2001, Matti Aarnio wrote:
> On Thu, Jul 26, 2001 at 03:51:35PM +0000, Linus Torvalds wrote:

> > Use fsync() on the directory.
> >
> > Logical, isn't it?
>
>   No.  I don't see why I should opendir() a directory, fsync()
> that handle, and closedir() the handle.

And it wouldn't even be enough.  Who guarantees you that
the parent directory of this directory has been written
to disk and we won't lose the entry pointing to this
directory on a crash ?

> I would definitely prefer:
>
>        lsync(dirpath)

Nice idea.  Of course, fsync(file) also has the obligation
to make sure all the metadata of the file is written to
disk. Lots of people seem to be convinced this also includes
the metadata needed to _reach_ the file all the way from the
root of the filesystem...

> I didn't check if POSIX folks have thought of that.

Nice addition.  Easier to use than fsync() - no need to
open the file - and probably easier to implement in the
kernel because this way we'll be handing the whole path
to the kernel, whereas fsync() would have the dubious
task of finding out how this file can be traced all the
way down from the root of the filesystem.

regards,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Lawrence Greenfield]

   Date: 	Mon, 30 Jul 2001 22:23:29 -0300 (BRST)
   From: Rik van Riel <riel@conectiva.com.br>
[...]
   > I would definitely prefer:
   >
   >        lsync(dirpath)
[...]
   Nice addition.  Easier to use than fsync() - no need to
   open the file - and probably easier to implement in the
   kernel because this way we'll be handing the whole path
   to the kernel, whereas fsync() would have the dubious
   task of finding out how this file can be traced all the
   way down from the root of the filesystem.

It's not as good as fsync() just doing what it's suppose to do.
You'll force applications that want to issue multiple link()s to issue
multiple lsync()s, forcing the kernel to serialize all of the disk
writes when the application just wants one file (and all of it's
associated filenames) to disk.

Yes, I understand that implementing fsync() so that it syncs all names
to reach the file is difficult.  But if you want the best performance,
you don't want to make applications issue multiple calls each of which
force their own synchronous writes.

Not to mention us whiny application writers won't be happy throwing
lsync()s all over the place.

Larry

Re: ext3-2.4-0.9.4

[Matti Aarnio]

  The thing about filesystems, and how dimmly MTAs (should) consider
  some performance tweaks is something I have tried to describe at
  ZMailer's manual in part about its the queue:

      http://www.zmailer.org/zman/zadm-queues.html

On Tue, Jul 31, 2001 at 01:25:06AM -0400, Lawrence Greenfield wrote:
...
> It's not as good as fsync() just doing what it's suppose to do.
> You'll force applications that want to issue multiple link()s to issue
> multiple lsync()s, forcing the kernel to serialize all of the disk
> writes when the application just wants one file (and all of it's
> associated filenames) to disk.
> 
> Yes, I understand that implementing fsync() so that it syncs all names
> to reach the file is difficult.  But if you want the best performance,
> you don't want to make applications issue multiple calls each of which
> force their own synchronous writes.
> 
> Not to mention us whiny application writers won't be happy throwing
> lsync()s all over the place.
> 
> Larry

   I quite agree.

   Filesystems are not, unfortunately, rollbackfull logged and committable
   databases, even if we like to use them often in that way.

   An MTA with a fundamental design point of not using any privileged
   programs (no suid anything!) and least esoteric technology possible
   (for wide portability) can only use message submission means available
   to it everywhere -- implementing the queue inside a database system
   is definitely a possibility.   Possibly yielding higher performance
   than one using filesystem for it, but at what cost ??
   (I am thinking of SleepyCat DB multiaccess transaction supported
    version.)

/Matti Aarnio

Re: ext3-2.4-0.9.4

[Anton Altaparmakov]

On Tue, 31 Jul 2001, Matti Aarnio wrote:

>   The thing about filesystems, and how dimmly MTAs (should) consider
>   some performance tweaks is something I have tried to describe at
>   ZMailer's manual in part about its the queue:
> 
>       http://www.zmailer.org/zman/zadm-queues.html
> 
> On Tue, Jul 31, 2001 at 01:25:06AM -0400, Lawrence Greenfield wrote:
> ...
> > It's not as good as fsync() just doing what it's suppose to do.
> > You'll force applications that want to issue multiple link()s to issue
> > multiple lsync()s, forcing the kernel to serialize all of the disk
> > writes when the application just wants one file (and all of it's
> > associated filenames) to disk.
> > 
> > Yes, I understand that implementing fsync() so that it syncs all names
> > to reach the file is difficult.  But if you want the best performance,
> > you don't want to make applications issue multiple calls each of which
> > force their own synchronous writes.
> > 
> > Not to mention us whiny application writers won't be happy throwing
> > lsync()s all over the place.
> > 
> > Larry
> 
>    I quite agree.
> 
>    Filesystems are not, unfortunately, rollbackfull logged and committable
>    databases, even if we like to use them often in that way.

Well it depends on which file system you are talking about. NTFS is for
all intents and purposes a rollbackfull logged and committable
(relational) database and a file system at the same time. It's a shame M$
don't release the specs for it, otherwise it would be just what you are
looking for. - It will take us forever to reverse engineer the
journalling part of NTFS. You can see how long it is taking us just to
get the actual file system part.. and journalling on top of that is going
to be even worse. (Of course once we have the file system part there is
nothing to stop us doing our own thing with respect to journalling but
that's a different discussion.)

Anton

> 
>    An MTA with a fundamental design point of not using any privileged
>    programs (no suid anything!) and least esoteric technology possible
>    (for wide portability) can only use message submission means available
>    to it everywhere -- implementing the queue inside a database system
>    is definitely a possibility.   Possibly yielding higher performance
>    than one using filesystem for it, but at what cost ??
>    (I am thinking of SleepyCat DB multiaccess transaction supported
>     version.)
> 
> /Matti Aarnio
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Tue, 31 Jul 2001, Lawrence Greenfield wrote:

> Not to mention us whiny application writers won't be happy throwing
> lsync()s all over the place.

Not portable -> won't happen usually.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Mon, 30 Jul 2001, Rik van Riel wrote:

> > I didn't check if POSIX folks have thought of that.
> 
> Nice addition.  Easier to use than fsync() - no need to
> open the file - and probably easier to implement in the
> kernel because this way we'll be handing the whole path
> to the kernel, whereas fsync() would have the dubious
> task of finding out how this file can be traced all the
> way down from the root of the filesystem.

If I understand SUS v2 correctly, fsync() must sync meta data
corresponding to the file.

If Linux ext2 doesn't to that, it might be a good idea to change that so
it does.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Mike Castle]

On Tue, Jul 31, 2001 at 11:29:47PM +0200, Matthias Andree wrote:
> If I understand SUS v2 correctly, fsync() must sync meta data
> corresponding to the file.


Where can I find a common definition for "meta data."

For example, I consider meta data to be things kept in the inode only
(size, timestamps, permissions).  Indirect blocks, maybe.  But, considering
how, in the unix world, file names are NOT associated with files, I have
never considered file names to be meta data.  Instead, file names is a set
of data associated with special files known as "directories."  So, it is
obvious, to me, that expecting fsync to sync changes to directory entries
is silly.

Obviously, however, you have a different definition of what meta data is.

Does SUS2 provide a definition for meta data?

A quick glance at the webside didn't turn anything up for me, but I would
not be surprised that I may have missed it.

mrc
-- 
     Mike Castle      dalgoda@ix.netcom.com      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

Re: ext3-2.4-0.9.4

[Chris Wedgwood]

On Tue, Jul 31, 2001 at 11:29:47PM +0200, Matthias Andree wrote:

    If I understand SUS v2 correctly, fsync() must sync meta data
    corresponding to the file.

    If Linux ext2 doesn't to that, it might be a good idea to change
    that so it does.

Define 'meta-data' --- linux sync's any inode and/or bitmap changes,
fsyn on a file will ensure it is intact but not that it can't get
lost.



  --cw

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Wed, 1 Aug 2001, Chris Wedgwood wrote:
> On Tue, Jul 31, 2001 at 11:29:47PM +0200, Matthias Andree wrote:
>
>     If I understand SUS v2 correctly, fsync() must sync meta data
>     corresponding to the file.
>
>     If Linux ext2 doesn't to that, it might be a good idea to change
>     that so it does.
>
> Define 'meta-data' --- linux sync's any inode and/or bitmap
> changes, fsyn on a file will ensure it is intact but not that it
> can't get lost.

Syntactically correct, but quite useless IMHO ;)

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Linus Torvalds]

On Tue, 31 Jul 2001, Matti Aarnio wrote:
> >
> > Logical, isn't it?
>
>   No.  I don't see why I should opendir() a directory, fsync()
> that handle, and closedir() the handle.  I would definitely prefer:
>
>        lsync(dirpath)

Btw, you don't have to do opendir() - that just wastes time. Just do
something like

	int lsync(char *path)
	{
	        int err, fd;
	        fd = open(path, 0);
	        if (fd >= 0) {
	                err = fsync(fd);
	                close(fd);
	        }
	        return err;
	}

and you're done. But it won't do the symlink thing...

		Linus

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Thu, 26 Jul 2001, Linus Torvalds wrote:

> In article <20010726143002.E17244@emma1.emma.line.org>,
> Matthias Andree  <matthias.andree@stud.uni-dortmund.de> wrote:
> >
> >However, the remaining problem is being synchronous with respect to open
> >(fixed for ext3 with your fsync() as I understand it), rename, link and
> >unlink. With ext2, and as you write it, with ext3 as well, there is
> >currently no way to tell when the link/rename has been committed to
> >disk, unless you set mount -o sync or chattr +S or call sync() (the
> >former is not an option because it's far too expensive).
> 
> Congratulations. You have been brainwashed by Dan Bernstein.

No, I asked Wietse Venema what assumptions Postfix makes. Since he
refuses to fsync() directories, he has Postfix set chattr +S to enforce
the semantics he expects. No problem here.

> Use fsync() on the directory. 
> 
> Logical, isn't it?

Why go all the lengths to look up each single directory path component
again just to fsync() stuff that doesn't belong to you and that you
don't want synched, possibly the entire device?

Chase up to the root manually, because Linux' ext2 violates SUS v2
fsync() (which requires meta data synched BTW), as has been pointed out
(and fixed in ReiserFS and ext3)?

Admittedly, MTAs are (supposed to be) (per command of RFC-1123) more
paranoid than the average application - and per lack of standard whether
rename/link & Co. need to be synchronous or asynchronous, this is a
problem for the MTA.

So, please tell my why Single Unix Specification v2 specifies EIO for
rename. Asynchronous I/O cannot possibly trigger immediate EIO.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Tue, 31 Jul 2001, Matthias Andree wrote:
> On Thu, 26 Jul 2001, Linus Torvalds wrote:
>
> > Congratulations. You have been brainwashed by Dan Bernstein.

[snip fsync() on directory ... on second thought this isn't enough]

> Chase up to the root manually, because Linux' ext2 violates SUS
> v2 fsync() (which requires meta data synched BTW), as has been
> pointed out (and fixed in ReiserFS and ext3)?

Agreed.  fsync() on the file needs to write the meta
data, this includes the directory and (if needed)
the parent directories all the way up to the root.

> So, please tell my why Single Unix Specification v2 specifies EIO for
> rename. Asynchronous I/O cannot possibly trigger immediate EIO.

Crap. An asynchronous rename() can hit the situation
where it cannot read the disk when searching for the
directory it wants to move the file to.

rename(/from/a/b/file, /to/d/f/file) can fail when
the system gets an IO access on reading "d".

regards,

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Mike Castle]

On Tue, Jul 31, 2001 at 02:57:00AM +0200, Matthias Andree wrote:
> So, please tell my why Single Unix Specification v2 specifies EIO for
> rename. Asynchronous I/O cannot possibly trigger immediate EIO.

It also specifies EIO as possible for write().

Are you saying that, since SUS2 specifies that write() is capable of
returning EIO, and asynchronous I/O cannot possibly trigger immediate EIO, 
that all calls to write() should by synchronous?

mrc
-- 
     Mike Castle      dalgoda@ix.netcom.com      www.netcom.com/~dalgoda/
    We are all of us living in the shadow of Manhattan.  -- Watchmen
fatal ("You are in a maze of twisty compiler features, all different"); -- gcc

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Mon, 30 Jul 2001, Mike Castle wrote:

> On Tue, Jul 31, 2001 at 02:57:00AM +0200, Matthias Andree wrote:
> > So, please tell my why Single Unix Specification v2 specifies EIO for
> > rename. Asynchronous I/O cannot possibly trigger immediate EIO.
> 
> It also specifies EIO as possible for write().
> 
> Are you saying that, since SUS2 specifies that write() is capable of
> returning EIO, and asynchronous I/O cannot possibly trigger immediate EIO, 
> that all calls to write() should by synchronous?

No, I'm wondering about the semantics. Of course, write() can be
synchronous (O_SYNC or fs mounted sync e. g.).

Re: ext3-2.4-0.9.4

[Stephen C. Tweedie]

Hi,

> Chase up to the root manually, because Linux' ext2 violates SUS v2
> fsync() (which requires meta data synched BTW)

Please quote chapter and verse --- my reading of SUS shows no such
requirement.  

fsync is required to force "all currently queued I/O operations
associated with the file indicated by file descriptor fildes to the
synchronised I/O completion state."  But as you should know, directory
entries and files are NOT the same thing in Unix/SUS.  

Are we expected to fsync the metadata belonging to just the file
itself?  Or all symlinks to the file?  Or all hard links?  Answer, as
best I can determine --- just the file.  That's all SUS talks about.
There can be many ways of reaching that file in the directory
hierarchy, or there can be none, but fsync() doesn't talk at all about
the status of those dirents after the sync.

> , as has been pointed out
> (and fixed in ReiserFS and ext3)?

ext3 happens to provide the guarantee, but that's coincidental and
does not imply that I think of it as being "fixed".  It's just changed
behaviour relative to ext2.

> So, please tell my why Single Unix Specification v2 specifies EIO for
> rename. Asynchronous I/O cannot possibly trigger immediate EIO.

Yes it can --- we may need to read metadata to complete the rename,
and such reads can fail.  

Cheers,
 Stephen

Re: ext3-2.4-0.9.4

[Kurt Roeckx]

On Wed, Aug 01, 2001 at 05:02:30PM +0100, Stephen C. Tweedie wrote:
> Hi,
> 
> > Chase up to the root manually, because Linux' ext2 violates SUS v2
> > fsync() (which requires meta data synched BTW)
> 
> Please quote chapter and verse --- my reading of SUS shows no such
> requirement.  
> 
> fsync is required to force "all currently queued I/O operations
> associated with the file indicated by file descriptor fildes to the
> synchronised I/O completion state."  But as you should know, directory
> entries and files are NOT the same thing in Unix/SUS.  

It goed on with "All I/O operations are completed as defined for
synchronised I/O file integrity completion.", whatever it all
means.

For fdatasync() it says:
"The fdatasync() function forces all currently queued I/O
operations associated with the file indicated by file descriptor
fildes to the synchronised I/O completion state.", which is just
the same as it says for fsync().

It also says:
"The functionality is as described for fsync() (with the symbol
_XOPEN_REALTIME defined), with the exception that all I/O
operations are completed as defined for synchronised I/O data
integrity completion."

It doesn't mention meta-data.

I have no idea what it all means.


Kurt

Re: ext3-2.4-0.9.4

[Andrew McNamara]

>Please quote chapter and verse --- my reading of SUS shows no such
>requirement.  
>
>fsync is required to force "all currently queued I/O operations
>associated with the file indicated by file descriptor fildes to the
>synchronised I/O completion state."  But as you should know, directory
>entries and files are NOT the same thing in Unix/SUS.  

But does fsync() have any meaning if it doesn't ensure the file is
visible within the filesystem? 

This all comes back to the fact that old UFS's made directory
operations syncronous, at a substantial cost in performance. Writing
the directory data wasn't necessary for them, because it was already
commited when the creat() call returned.

I can easily understand people's asthetic objection to having fsync
touch the directory object as well as the file, however what meaning
does fsync() have it it doesn't - under linux, it tells usermode "yes,
your object is committed, but it might be in lost+found next time you
want it", and with the syncronous UFS implementations, it tells
usermode "yes, your object is committed, you can find it where you left
it (unless the directory was corrupted)".

 ---
Andrew McNamara (System Architect)

connect.com.au Pty Ltd
Lvl 3, 213 Miller St, North Sydney, NSW 2060, Australia
Phone: +61 2 9409 2117, Fax: +61 2 9409 2111

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Wed, 01 Aug 2001, Stephen Tweedie wrote:

> > Chase up to the root manually, because Linux' ext2 violates SUS v2
> > fsync() (which requires meta data synched BTW)
> 
> Please quote chapter and verse --- my reading of SUS shows no such
> requirement.  
> 
> fsync is required to force "all currently queued I/O operations
> associated with the file indicated by file descriptor fildes to the
> synchronised I/O completion state."  But as you should know, directory
> entries and files are NOT the same thing in Unix/SUS.  

Read on: "All I/O operations are completed as defined for synchronised
I/O _file_ integrity completion.". To show what that means, see the
glossary.

http://www.opengroup.org/onlinepubs/007908799/xbd/glossary.html#tag_004_000_291

  "synchronised I/O data integrity completion

  [...]

  * For write, when the operation has been completed or diagnosed if
  unsuccessful.  The write is complete only when the data specified in
  the write request is successfully transferred and all file system
  information required to retrieve the data is successfully transferred.

  File attributes that are not necessary for data retrieval (access
  time, modification time, status change time) need not be successfully
  transferred prior to returning to the calling process.

  synchronised I/O file integrity completion

  Identical to a synchronised I/O data integrity completion with the
  addition that all file attributes relative to the I/O operation
  (including access time, modification time, status change time) will be
  successfully transferred prior to returning to the calling process."

As I understand it, the directory entry's st_ino is a file attribute
necessary for data retrieval and also contains the m/a/ctime, so it must
be flushed to disk on fsync() as well.

> There can be many ways of reaching that file in the directory
> hierarchy, or there can be none, but fsync() doesn't talk at all about
> the status of those dirents after the sync.

Well, if there's not a single dirent, you cannot retrieve the data, so
I'd assume at least one dirent needs to be flushed as well. If there's a
simple way to get unflushed dentries to disk (hard links included),
flush them. Not sure about symlinks, but since they don't share the
inode number, that might be rather difficult for the kernel (I didn't
check):

touch 1 ; ln 1 2 ; ln -s 1 3 ; ls -li

 303464 -rw-r--r--   2 emma     users           0 Aug  2 10:56 1
 303464 -rw-r--r--   2 emma     users           0 Aug  2 10:56 2
 303466 lrwxrwxrwx   1 emma     users           1 Aug  2 10:56 3 -> 1

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Christoph Hellwig]

In article <20010802110341.B17927@emma1.emma.line.org> you wrote:
>
> http://www.opengroup.org/onlinepubs/007908799/xbd/glossary.html#tag_004_000_291
>
>   "synchronised I/O data integrity completion
>
>   [...]
>
>   * For write, when the operation has been completed or diagnosed if
>   unsuccessful.  The write is complete only when the data specified in
>   the write request is successfully transferred and all file system
>   information required to retrieve the data is successfully transferred.
>
>   File attributes that are not necessary for data retrieval (access
>   time, modification time, status change time) need not be successfully
>   transferred prior to returning to the calling process.

NOTE: _file_ attributes

>
>   synchronised I/O file integrity completion
>
>   Identical to a synchronised I/O data integrity completion with the
>   addition that all file attributes relative to the I/O operation
>   (including access time, modification time, status change time) will be
>   successfully transferred prior to returning to the calling process."
>
> As I understand it, the directory entry's st_ino is a file attribute
> necessary for data retrieval and also contains the m/a/ctime, so it must
> be flushed to disk on fsync() as well.
>

As long as you have an open fd, no directory entry is needed for
data retrieval.  In fact some fds never have a directory entry
(e.g. sockets - but these don't support fsync anyway) or do not have a
directory entry in their user-visble interface (e.g. posix shm).

And m/a/ctime is in the inode of the file, not in the directory enrty.
(at least for usual UNIX filesystems).

> Well, if there's not a single dirent, you cannot retrieve the data,

Of course you can, you can pass and fd for an unliked file everywhere
using AF_LOCAL descriptor passing.

	Christoph

-- 
Of course it doesn't work. We've performed a software upgrade.

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Thu, 2 Aug 2001, Christoph Hellwig wrote:

> > Well, if there's not a single dirent, you cannot retrieve the data,
>
> Of course you can, you can pass and fd for an unliked file
> everywhere using AF_LOCAL descriptor passing.

But this assumes the system doesn't crash, while
fsync() seems meant more as a protection against
the system going down unexpectedly ...

Rik
--
Executive summary of a recent Microsoft press release:
   "we are concerned about the GNU General Public License (GPL)"


		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com/

Re: ext3-2.4-0.9.4

[Eric W. Biederman]

Rik van Riel <riel@conectiva.com.br> writes:

> On Thu, 2 Aug 2001, Christoph Hellwig wrote:
> 
> > > Well, if there's not a single dirent, you cannot retrieve the data,
> >
> > Of course you can, you can pass and fd for an unliked file
> > everywhere using AF_LOCAL descriptor passing.
> 
> But this assumes the system doesn't crash, while
> fsync() seems meant more as a protection against
> the system going down unexpectedly ...

There is something to that.  However taking this argument to
it's logical extreme I have you have to not only sync every directory
in the current path of the file.  You have to sync your online file
index, because search engines is how we find things right?  

Since the filename in unix is not part of the files metadata it is a
perfectly sane semantic for fsck to drop the file into /lost+found, if
no one cared enough about the index/directory to update it.

In the general case if you have the guarantee that a filesystem does
safe directory updates.  So unless someone does an unlink you won't
loose your old link.  For most cases it doesn't matter as your
directory entry for the file is much older than the file itself, and
has been already synched.  MTA's are the exception to this where there
good filename is written only after the file is written.

The only other argument that seems to come from the MTA case is that
syscalls are slow, and a pain and programmers don't want to make two
or three syscalls just to do this.  Heck if you are doing a sync you
are waiting for a disk which is slow.  So speed doesn't really count.

There is probably an argument in there somewhere about batching up
I/O, so you have to wait a minimum amount of time for your sync.  But
until someone benchmarks, and tries a few different approatches I
won't believe that you need a kernel change even for that.

My question is what does fsync do on directories in other unix's.  It
would be really strange if it didn't behave similiarly to linux.
If forget wether it was AIX or HP-UX where doing a grep foo * would
also grep through "." .  So at least open works on other peoples directores.

Eric

Re: ext3-2.4-0.9.4

[Daniel Phillips]

On Thursday 02 August 2001 11:03, Matthias Andree wrote:
> On Wed, 01 Aug 2001, Stephen Tweedie wrote:
> > Matthias Andree wrote:
> > > Chase up to the root manually, because Linux' ext2 violates SUS
> > > v2 fsync() (which requires meta data synched BTW)
> >
> > Please quote chapter and verse --- my reading of SUS shows no such
> > requirement.
> >
> > fsync is required to force "all currently queued I/O operations
> > associated with the file indicated by file descriptor fildes to the
> > synchronised I/O completion state."  But as you should know,
> > directory entries and files are NOT the same thing in Unix/SUS.
>
> Read on: "All I/O operations are completed as defined for
> synchronised I/O _file_ integrity completion.". To show what that
> means, see the glossary.
>
> http://www.opengroup.org/onlinepubs/007908799/xbd/glossary.html#tag_0
>04_000_291
>
>   "synchronised I/O data integrity completion
>
>   [...]
>
>   * For write, when the operation has been completed or diagnosed if
>   unsuccessful.  The write is complete only when the data specified
> in the write request is successfully transferred and all file system
> information required to retrieve the data is successfully
> transferred.
>
>   File attributes that are not necessary for data retrieval (access
>   time, modification time, status change time) need not be
> successfully transferred prior to returning to the calling process.
>
>   synchronised I/O file integrity completion
>
>   Identical to a synchronised I/O data integrity completion with the
>   addition that all file attributes relative to the I/O operation
>   (including access time, modification time, status change time) will
> be successfully transferred prior to returning to the calling
> process."
>
> As I understand it, the directory entry's st_ino is a file attribute
> necessary for data retrieval and also contains the m/a/ctime, so it
> must be flushed to disk on fsync() as well.

I believed you've summarized the SUS requirements very well.  Apart 
from legalistic arguments, SUS quite clearly states that fsync should 
not return until you are sure of having recorded not only the file's 
data, but the access path to it.  I interpret this as being able to 
"access the file by its name", and being able to guess by looking in 
lost+found doesn't count.  I don't see the point in niggling about that.

So, it seems clear that an fsync which leaves any window of 
vulnerability where an interruption can leave a file unlinked is not 
SUS-compliant.

> > There can be many ways of reaching that file in the directory
> > hierarchy, or there can be none, but fsync() doesn't talk at all
> > about the status of those dirents after the sync.

This is a legalistic argument.  I don't think we should be looking for 
loopholes in SUS here.  To achieve SUS compliance there are two 
reasonable courses: "fix SUS" or "fix sys_fsync".  Since what SUS 
clearly wants here seems emminently reasonable, I'd suggest putting the 
energy that's currently going into this thread into fixing fsync 
instead.

> Well, if there's not a single dirent, you cannot retrieve the data,
> so I'd assume at least one dirent needs to be flushed as well. If
> there's a simple way to get unflushed dentries to disk (hard links
> included)...

*All* hard links?  No, there is no general way to do that.  However, 
any hard links[1] in the path used to open the file - yes.  There is 
always a chain of parent dentries held locked in the dcache for any 
open file.

I don't know why it is hard or inefficient to implement this at the VFS 
level, though I'm sure there is a reason or this thread wouldn't 
exist.  Stephen, perhaps you could explain for the record why sys_fsync 
can't just walk the chain of dentry parent links doing fdatasync?  Does 
this create VFS or Ext3 locking problems?  Or maybe it repeats work 
that Ext3 is already supposed to have done?

> ...flush them. Not sure about symlinks, but since they don't
> share the inode number, that might be rather difficult for the kernel
> (I didn't check)

The prescription for symlinks is, if you want them safely on disk you 
have to explicitly fsync the containing directory.

[1] In Ext2, all filename dirents are "hard links", i.e., there is no 
way to tell which of the two names is the original after creating a new 
hard link.

--
Daniel

intermediate summary of ext3-2.4-0.9.4 thread

[Matthias Andree]

On Thu, 02 Aug 2001, Daniel Phillips wrote:

[file name must be flushed on fsync()]
> I don't know why it is hard or inefficient to implement this at the VFS 
> level, though I'm sure there is a reason or this thread wouldn't 
> exist.  Stephen, perhaps you could explain for the record why sys_fsync 
> can't just walk the chain of dentry parent links doing fdatasync?  Does 
> this create VFS or Ext3 locking problems?  Or maybe it repeats work 
> that Ext3 is already supposed to have done?

Well, the course was that I asked whether ext3 would do synchronous
directory updates, and some people jumped in and said that one should
fsync() the parent directory, however, since we figure from SUS, that's
invalid.

After some forth and back, we finally figured that at least ext2 is
implementing fsync() improperly.

So this part is covered.

The other thing is, that Linux is the only known system that does
asynchronous rename/link/unlink/symlink -- people have claimed it might
not be the only one, but failed to name systems.

So we need to assume that Linux is the only system that does
asynchronous rename/link/unlink/symlink, however a directory fsync() is
believed to be rather expensive.

Still, some people object to a dirsync mount option. But this has been
the actual reason for the thread - MTA authors are refusing to pamper
Linux and use chattr +S instead which gives unnecessary (premature) sync
operations on write() - but MTAs know how to fsync().

> The prescription for symlinks is, if you want them safely on disk you 
> have to explicitly fsync the containing directory.

Yes, and it doesn't matter, since MTAs don't use symlinks (symlinks
waste inodes on most systems).

-- 
Matthias Andree

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Alexander Viro]

On Thu, 2 Aug 2001, Matthias Andree wrote:

> asynchronous rename/link/unlink/symlink, however a directory fsync() is
> believed to be rather expensive.

How the fuck it's expensive? It does _exactly_ the same as file fsync() -
literally the same code. It doesn't write blocks that don't belong to
directory. It doesn't write blocks that are clean. IOW, it does the
minimal work possible.

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Matthias Andree]

On Thu, 02 Aug 2001, Alexander Viro wrote:

> How the fuck it's expensive? It does _exactly_ the same as file fsync() -
> literally the same code. It doesn't write blocks that don't belong to
> directory. It doesn't write blocks that are clean. IOW, it does the
> minimal work possible.

fsync()ing the dir is not the minimal work possible, if e. g. temporary
files are open that don't need their names synched. Fsync()ing the
directory syncs also these temporary file NAMES that other processes may
have open (but that they unlink rather than fsync()).

Assume:

open -> asynchronous, but filename synched on fsync()
rename/link/unlink(/symlink) -> synchronous

This way, you never need to fsync() the directory, so you never sync()
entries of temporary files. You never lose important files (because the
application uses fsync() and the OS synchs rename/link etc.).

-- 
Matthias Andree

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Andreas Dilger]

Matthais Andree writes:
> fsync()ing the dir is not the minimal work possible, if e. g. temporary
> files are open that don't need their names synched. Fsync()ing the
> directory syncs also these temporary file NAMES that other processes may
> have open (but that they unlink rather than fsync()).
> 
> Assume:
> 
> open -> asynchronous, but filename synched on fsync()
> rename/link/unlink(/symlink) -> synchronous
> 
> This way, you never need to fsync() the directory, so you never sync()
> entries of temporary files. You never lose important files (because the
> application uses fsync() and the OS synchs rename/link etc.).

Do you read what you are writing?  How can a "synchronous" operation for
rename/link/unlink/symlink NOT also write out "temporary" files in the
same directory?  How does calling fsync() on the directory IF YOU REQUIRE
SYNCHRONOUS DIRECTORY OPERATIONS differ from making the specific operations
synchronous from within the kernel???

The only difference I can see is that making these specific operations
ALWAYS be synchronous hurts the common case when they can be async (see
Solaris UFS vs. Linux benchmark elsewhere in this thread), while requiring
an fsync() on the directory == only synchronous operation when it is
actually needed, and no "extra" performance hit.

The only slight point of contention is if you have very large directories
which span several filesystem blocks, in which case it _would_ be possible
to write out some blocks synchronously, while leaving other blocks dirty.
In practise however, you will either only be modifying a small number of
blocks (at the end of the directory) because an MTA usually only creates
files and doesn't delete them, and the actual speed of syncing several
blocks at one time is not noticably different than syncing only one.

Cheers, Andreas
-- 
Andreas Dilger  \ "If a man ate a pound of pasta and a pound of antipasto,
                 \  would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/               -- Dogbert

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Matthias Andree]

On Thu, 02 Aug 2001, Andreas Dilger wrote:

> > open -> asynchronous, but filename synched on fsync()
> > rename/link/unlink(/symlink) -> synchronous
> > 
> > This way, you never need to fsync() the directory, so you never sync()
> > entries of temporary files. You never lose important files (because the
> > application uses fsync() and the OS synchs rename/link etc.).
> 
> Do you read what you are writing?  How can a "synchronous" operation for
> rename/link/unlink/symlink NOT also write out "temporary" files in the
> same directory?  How does calling fsync() on the directory IF YOU REQUIRE
> SYNCHRONOUS DIRECTORY OPERATIONS differ from making the specific operations
> synchronous from within the kernel???

Can people please try to understand? Can people please start to THINK
before flaming?

I did not say that open() is to be synchronous. I did not write ANYTHING
of fsync()ing directories, I'm trying to get rid of this requirement.

Thus, if the kernel does rename/link synchronously, you'd never ever
fsync() a directory. To synch a filename to disk, you'd just fsync() the
filedescriptor (with a SUS compliant system, that is, i. e. ext3 or
reiserfs, but not ext2).

Now, if someone opens a temporary file, and nukes it later -- unlink()
--, and doesn't want it visible, he never calls fsync() for the file.

However, if some other process then fsync()s the directory, you start
synching the temporary file dirent -> unnecessary, is nuked later on
with an unlink().

That's why fsync() on the directory is on no account the minimum work.

> The only difference I can see is that making these specific operations
> ALWAYS be synchronous hurts the common case when they can be async (see
> Solaris UFS vs. Linux benchmark elsewhere in this thread), while requiring
> an fsync() on the directory == only synchronous operation when it is
> actually needed, and no "extra" performance hit.

In case you haven't noticed, this is about reliability without need to
fsync() the directory that doesn't all belong to your single, stupid
process but may have lots of asynchronous data of other processes -
temporary files for instance. You synch() that as well, which is
unnecessary and brings down other processes' performance.

In case you haven't noticed the other issue:

The whole thread is a FEATURE REQUEST for a dirsync mount option, for
MTAs and other software which requires reliable file systems, where the
name is negotiable. It aims to REDUCE OVERHEAD since chattr +S which is
the only workaround for synch-dirs - and it synchs synchronous files and
writes as well, and rendering things slower than necessary, since
write() can be buffered until you fsync() (and you want that to cut off
seek times).

Call the option bsd_slow_dirs if you like, I don't care. Given the
option, the administrator/user has the choice, currently, he hasn't. He
cannot possibly change all applications ported from other Unices.

Note: hindering this option doesn't get Linux anywhere. Pure file
system benchmarks are not worth a single bit of entropy unless Linux is
benchmarked chattr +S -- it's unreliable otherwise.

I cannot remember how often I explained this during the course of this
thread. Every other day, some ignorant comes out of its cavern and
discusses the whole thing over and over again.

And, once again, fsync()ing the directory is not an option for portable
applications. It's unnecessary on every other system (until someone
shows a production-ready system which by default has asynchronous
directory updates as well, but no-one has so far.)

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Matthias Andree]

On Fri, 03 Aug 2001, Anton Altaparmakov wrote:

[dirsync chattr/mount options]
> Me neither. With regards to the parallel discussion on SUS compliance it is 
> probably a good idea to have such a thing in some form anyway (although if 
> I understood the discussion correctly, we really want this to happen by 
> default, not just when some flag is set but then again I never read the 
> standards...).

The standard doesn't really command the behaviour, as it seems, but we
might want to look again after SUS v3 has been released (supposed to
happen later this year) - the SUS compliance was rather on fsync than on
rename/link.

However, I'd rather not choose the default for somebody else, because he
may have different requirements, a compile-time switch to set the
default should be fine, THIS one might indeed default to dirsync/noasync
unless changed by make {x,menu,}config.

Assuming that the chattr +S is accompanied by a corresponding -o sync
mount option, I'd expect that the dirsync option be available as chattr
option and as mount option, and choosing default mount options should be
rather easy.

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Jan Harkes]

On Fri, Aug 03, 2001 at 02:14:06AM +0200, Matthias Andree wrote:
> On Fri, 03 Aug 2001, Anton Altaparmakov wrote:
> > filedescriptor to be synced to disk, the ONLY possible way to do this it to 
> > sync the parent directory in order to commit the file name to disk. On some 
> 
> Do I really need to sync the WHOLE parent directory? Not just the
> relevant part? My directories hardly have only 1 disk block.

Only dirty blocks are written back to disk, i.e. the parts of the
directory that were modified by adding/removing names. It should be
pretty efficient.

> > to explicitly sync the directory filedescriptor afterwards.
> 
> Which is non-portable and will not be done by many application
> programmers which just use chattr +S instead (makes things S)afe and
> S)low) - and spoil performance that way since it makes not only
> directory writes synchronous, but file (data) writes as well.

"chattr +S" is about as portable as adding fsync(parent), or even worse
as it only works on an ext2 file system. So I'm assuming that this is
just a nice exercise in annoying people.

Jan

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Andreas Dilger]

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Bill Rugolsky Jr.]

On Thu, Aug 02, 2001 at 07:37:50PM +0200, Matthias Andree wrote:
> The other thing is, that Linux is the only known system that does
> asynchronous rename/link/unlink/symlink -- people have claimed it might
> not be the only one, but failed to name systems.
> 
> So we need to assume that Linux is the only system that does
> asynchronous rename/link/unlink/symlink, however a directory fsync() is
> believed to be rather expensive.
> 
> Still, some people object to a dirsync mount option. But this has been
> the actual reason for the thread - MTA authors are refusing to pamper
> Linux and use chattr +S instead which gives unnecessary (premature) sync
> operations on write() - but MTAs know how to fsync().

Let's inject a little reality into this discussion.  Filesystems are used
for something other than running MTA's written by stubborn "purists".

Solaris: Dell 600 MHz PIII 128MB RAM, largely quiescent:
         Solaris 8 mu4, UFS with logging

Linux:   VA Linux 800 MHZ PIII, 128MB RAM, largely quiescent
         RedHat Linux 7.1 w/ kernel-2.4.6-2.4 (2.4.6-ac5 + ext3-0.9.3).

660MB XFree86-4.1 build tree, cache primed with du -s in each case.

Here's something that we developers probably all do frequently: copy a
tree using hard links, so that we can patch it.

[solaris] find . | wc     
   33027   33027 1251671
[solaris] time find . -depth | cpio -pdul ../foo
0 blocks
 363.46s real    0.84s user   10.13s system 

Plain ext2:

[linux]# time find . -depth | cpio -pdul ../foo
0 blocks

real    0m3.823s user    0m0.240s sys     0m3.570s

Mounted ext3, ordered data mode.

[linux] time find . -depth | cpio -pdul ../foo
0 blocks

real    0m5.106s user    0m0.200s sys     0m3.700s

Mounted ext3, -o sync:

[root@ead51 bar]# time find . -depth | cpio -pdul ../foo
0 blocks

real    1m28.483s user    0m0.470s sys     0m4.410s 

=====================================================

Solaris8 UFS:   363.5 seconds
ext2:             3.8 seconds
ext3:             5.1 seconds
ext3 -o sync:    88.5 seconds

Got it?

Obviously, the last is the result of the poor interaction
of ext3+sync in 0.9.3, but Andrew Morton has already fixed that.
I will try again with 0.9.5 when I have a chance to upgrade that
machine.

I have no idea where BSD falls, but the basic point stands:  unused
features should not penalize other applications.  Andrew Morton has
figured out how to do this efficiently with ext3, and many kudos to him
for doing the work.  Absent that, why should I have to go get a cup of
coffee every time I want to patch a tree, just so some MTA can make
naive assumptions?

Regards,

   Bill Rugolsky

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Matthias Andree]

On Thu, 02 Aug 2001, Bill Rugolsky Jr. wrote:

> I have no idea where BSD falls, but the basic point stands:  unused
> features should not penalize other applications.  Andrew Morton has
> figured out how to do this efficiently with ext3, and many kudos to him
> for doing the work.  Absent that, why should I have to go get a cup of
> coffee every time I want to patch a tree, just so some MTA can make
> naive assumptions?

The whole idea is to have a switch to turn on BSD-style synchronous
directory update semantics. Nothing more, nothing you would not be able
to get rid off. In fact, you can mount file systems async on BSD as
well, but you'd better not have the machine crash. Irrecoverable file
system damage can result. As a compromise, softupdates are nearly as
fast as async, but FS damage is guaranteed to be recoverable.

In either case (async or soft-updates), files can end up in lost+found
after the control had been returned to the application that called open
or link.

-- 
Matthias Andree

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Eric W. Biederman]

Matthias Andree <matthias.andree@stud.uni-dortmund.de> writes:

> On Fri, 03 Aug 2001, Paul Jakma wrote:
> 
> > if the prime directive of MTAs is data integrity paranoia, then
> > surely the best assumption for an MTA to make is that
> > rename/link/unlink/symlink /are/ asynchronous in the general case?
> 
> They do on Linux, use chattr +S, and are much slower than e. g. on
> FreeBSD. Well. Not that I'd written THAT for the first time...

Actually given that this thread keeps coming up, but no one does anything
about it.  I'm tempted to suggest we remove chatrr +S support from ext2.
Then there will be enough pain that someone will fix the MTA instead of
moaning that kernel is slow...

That should be an easy patch to make...

Eric

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Matthias Andree]

On Fri, 03 Aug 2001, Eric W. Biederman wrote:

> Actually given that this thread keeps coming up, but no one does anything
> about it.  I'm tempted to suggest we remove chatrr +S support from ext2.
> Then there will be enough pain that someone will fix the MTA instead of
> moaning that kernel is slow...

They'd just drop Linux from the list of supported OS's, Linux will
disappoint people who trusted it, nothing is gained. Deliberate breakage
will not happen, because it would not help anyone except people with
twisted minds.

NO-ONE, including you, has come up with SERIOUS objections against a
dirsync option, except "is it really so much slower than chattr +S? show
figures" -- ext3 is being tuned to be fast in spite of chattr +S.

Reconsider your position.

Stop trolling please.

-- 
Matthias Andree

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Christoph Hellwig]

In article <20010803103954.A11584@emma1.emma.line.org> you wrote:
> They'd just drop Linux from the list of supported OS's, Linux will
> disappoint people who trusted it, nothing is gained. Deliberate breakage
> will not happen, because it would not help anyone except people with
> twisted minds.

Who cares?  There are more than enough sane mailer around..

> NO-ONE, including you, has come up with SERIOUS objections against a
> dirsync option, except "is it really so much slower than chattr +S? show
> figures" -- ext3 is being tuned to be fast in spite of chattr +S.

Talk is cheap.  Code up a non-invasive dirsync option and submit it to
Linus.  I don't see any reason why it won't be accepted..

	Christoph

-- 
Of course it doesn't work. We've performed a software upgrade.

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Eric W. Biederman]

Matthias Andree <matthias.andree@stud.uni-dortmund.de> writes:

> On Fri, 03 Aug 2001, Eric W. Biederman wrote:
> 
> > Actually given that this thread keeps coming up, but no one does anything
> > about it.  I'm tempted to suggest we remove chatrr +S support from ext2.
> > Then there will be enough pain that someone will fix the MTA instead of
> > moaning that kernel is slow...
> 
> They'd just drop Linux from the list of supported OS's, Linux will
> disappoint people who trusted it, nothing is gained. Deliberate breakage
> will not happen, because it would not help anyone except people with
> twisted minds.

There are some other uses for a fully synchronous disk accesses so I'm
not going to run out and do it.  The point is that work arounds for
strange programs is not a right, it is a nice optional feature.
 
> NO-ONE, including you, has come up with SERIOUS objections against a
> dirsync option, except "is it really so much slower than chattr +S? show
> figures" -- ext3 is being tuned to be fast in spite of chattr +S.

Clear objects against dirsync.  
- Extra code maintenance, makes the fs less reliable 
  (A reason for removing even synchrouns fs operations BTW).
- Unnecessary. fsync(dir) works today.
- dirsync is unlikely to be faster than fsync(file); fsync(dir) [not chattr +S]
  You really need something that can say remember these 5 syscalls,
  and sync the all their changes to disk togeter to really get an
  improvement in sync speed.
- I don't see anyone volunteering to write the code.

> Reconsider your position.

Nope.  Right now I would rather
a) Patch the mail programs to do the needed fsync(dir)
b) Totally remove synchrous disk updates from my OS, and
   make life really painful.
Before adding a dirsync option.

> Stop trolling please.

It wasn't trying to troll, just get this conversation on some productive
grounds.  I think supporting the MTA's is good, so long as it is a two
way relationship.

If someone went out and tried using fsync(dir) and then saying it
sucked we could definentily have more peace.

Using dirsync, and chattr +S hide the real problems that need to get
fixed.  Getting a good reliable, and high performance way to commit
actions to a filesystem.  

We already have one work around on linux that will work reliably.  So
now let's see if we can get a functional high performance solution.

And oh btw, new, functional high performance solutions are not
portable because they haven't been implemented in every operating
system.  Full understanding of the problem, and the solutions are two
new for the implementations to have gotten around.

Eric

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Stephen C. Tweedie]

Hi,

On Thu, Aug 02, 2001 at 07:37:50PM +0200, Matthias Andree wrote:

> So this part is covered.
> 
> The other thing is, that Linux is the only known system that does
> asynchronous rename/link/unlink/symlink -- people have claimed it might
> not be the only one, but failed to name systems.

Not true.  There are tons of others.

The issue was that synchronous directory updates are *optional* on
many systems (Linux included), but that Linux's support for that is
really inefficient since it ends up syncing file metadata updates too
(and it's much more efficient to use fsync for that.)

> Still, some people object to a dirsync mount option.

Who?  People who have discussed this in the past have certainly not
objected to my knowledge.  It would clearly help situations like this
(as would a dirsync chattr option.)

> > The prescription for symlinks is, if you want them safely on disk you 
> > have to explicitly fsync the containing directory.
> 
> Yes, and it doesn't matter, since MTAs don't use symlinks (symlinks
> waste inodes on most systems).

Irrelevant.   We're talking about what makes sensible semantics, not
what assumptions any specific application makes.  It makes no sense to
say that dirsync won't affect symlinks just because some existing
applications don't rely on that!

Cheers,
 Stephen

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Matthias Andree]

On Fri, 03 Aug 2001, Stephen Tweedie wrote:

> > > The prescription for symlinks is, if you want them safely on disk you 
> > > have to explicitly fsync the containing directory.
> > 
> > Yes, and it doesn't matter, since MTAs don't use symlinks (symlinks
> > waste inodes on most systems).
> 
> Irrelevant.   We're talking about what makes sensible semantics, not
> what assumptions any specific application makes.  It makes no sense to
> say that dirsync won't affect symlinks just because some existing
> applications don't rely on that!

It's rather my imagination that tracking hard links might be easier than
symlinks because hard links share the inode number. A more advanced (and
complex) implementation might prove the imagination wrong. I don't want
to consider which one is more efficient.

-- 
Matthias Andree

Re: intermediate summary of ext3-2.4-0.9.4 thread

[David Weinehall]

On Thu, Aug 02, 2001 at 07:37:50PM +0200, Matthias Andree wrote:
> On Thu, 02 Aug 2001, Daniel Phillips wrote:
> 
> [file name must be flushed on fsync()]
> > I don't know why it is hard or inefficient to implement this at the VFS 
> > level, though I'm sure there is a reason or this thread wouldn't 
> > exist.  Stephen, perhaps you could explain for the record why sys_fsync 
> > can't just walk the chain of dentry parent links doing fdatasync?  Does 
> > this create VFS or Ext3 locking problems?  Or maybe it repeats work 
> > that Ext3 is already supposed to have done?
> 
> Well, the course was that I asked whether ext3 would do synchronous
> directory updates, and some people jumped in and said that one should
> fsync() the parent directory, however, since we figure from SUS, that's
> invalid.
> 
> After some forth and back, we finally figured that at least ext2 is
> implementing fsync() improperly.
> 
> So this part is covered.

Yup, and this should be fixed imho.

> The other thing is, that Linux is the only known system that does
> asynchronous rename/link/unlink/symlink -- people have claimed it might
> not be the only one, but failed to name systems.

And this is a feature, not a bug.

> So we need to assume that Linux is the only system that does
> asynchronous rename/link/unlink/symlink, however a directory fsync() is
> believed to be rather expensive.

A directory fsync() might be expensive on non-Linux filesystems...

> Still, some people object to a dirsync mount option. But this has been
> the actual reason for the thread - MTA authors are refusing to pamper
> Linux and use chattr +S instead which gives unnecessary (premature) sync
> operations on write() - but MTAs know how to fsync().

So what you mean is that MTA authors refuse to pamper Linux through use
of fsync of the directory, but can accept to "pamper" Linux through use
of chattr +S?! This seem ridiculous.  It seems equally ridiculous to
demand that Linux should pamper for MTA authors that can't implement
fsync on the directory instead of writing BSD-specific code.

[snip]

To me this seems mostly like a way of saying "Hey, we've finally found
a way to make Linux look really bad compared to BSD-systems; let's
complain instead of writing alternative code that suits Linux systems
better than this code does." A lot like all the discussions on threads,
ueally.

Then again, I'm probably just extra grouchy today because it rained when
I rode my bike to work.


/David Weinehall
  _                                                                 _
 // David Weinehall <tao@acc.umu.se> /> Northern lights wander      \\
//  Project MCA Linux hacker        //  Dance across the winter sky //
\>  http://www.acc.umu.se/~tao/    </   Full colour fire           </

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Matthias Andree]

On Fri, 03 Aug 2001, David Weinehall wrote:

> On Thu, Aug 02, 2001 at 07:37:50PM +0200, Matthias Andree wrote:
> > Still, some people object to a dirsync mount option. But this has been
> > the actual reason for the thread - MTA authors are refusing to pamper
> > Linux and use chattr +S instead which gives unnecessary (premature) sync
> > operations on write() - but MTAs know how to fsync().
> 
> So what you mean is that MTA authors refuse to pamper Linux through use
> of fsync of the directory, but can accept to "pamper" Linux through use
> of chattr +S?! This seem ridiculous.  It seems equally ridiculous to
> demand that Linux should pamper for MTA authors that can't implement
> fsync on the directory instead of writing BSD-specific code.

It's a maintenance issue.

You effectively start wrapping up all relevant syscalls and have
system-specific interfaces. One wants the directory fsync()ed, the other
offers a special other trick to get the data flushed... what useful is
portability then if systems are so different?

> To me this seems mostly like a way of saying "Hey, we've finally found
> a way to make Linux look really bad compared to BSD-systems; let's

No wonder if the application chooses fully-synchronous operation on
Linux.

-- 
Matthias Andree

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Albert D. Cahalan]

David Weinehall writes:
> On Thu, Aug 02, 2001 at 07:37:50PM +0200, Matthias Andree wrote:

>> Still, some people object to a dirsync mount option. But this has been
>> the actual reason for the thread - MTA authors are refusing to pamper
>> Linux and use chattr +S instead which gives unnecessary (premature) sync
>> operations on write() - but MTAs know how to fsync().
>
> So what you mean is that MTA authors refuse to pamper Linux through use
> of fsync of the directory, but can accept to "pamper" Linux through use
> of chattr +S?! This seem ridiculous.  It seems equally ridiculous to
> demand that Linux should pamper for MTA authors that can't implement
> fsync on the directory instead of writing BSD-specific code.
>
> [snip]
>
> To me this seems mostly like a way of saying "Hey, we've finally found
> a way to make Linux look really bad compared to BSD-systems; let's
> complain instead of writing alternative code that suits Linux systems
> better than this code does." A lot like all the discussions on threads,
> ueally.

This is just completely true. One wonders why we seem to enjoy
getting screwed this way. We shouldn't be patching these MTAs or
hacking Linux to act like BSD. We should be avoiding these MTAs.

Somebody can create a big MTA list, listing the good and bad ones.
Then we get the Linux-hostile MTAs out of the Linux distributions,
demanding compliance like we do for filesystem layout. We also hunt
down Linux-related web pages that mention these MTAs and get the
pages changed or removed. The point is to make these MTAs just
disappear, never to be seen again. Nice MTAs get promoted.

Re: intermediate summary of ext3-2.4-0.9.4 thread

[Gregory Maxwell]

On Fri, Aug 03, 2001 at 03:59:02PM -0400, Albert D. Cahalan wrote:
[snip]
> Somebody can create a big MTA list, listing the good and bad ones.
> Then we get the Linux-hostile MTAs out of the Linux distributions,
> demanding compliance like we do for filesystem layout. We also hunt
> down Linux-related web pages that mention these MTAs and get the
> pages changed or removed. The point is to make these MTAs just
> disappear, never to be seen again. Nice MTAs get promoted.

Think we could just get their authors to 'disappear'? It might be more cost
effective, and I can think of at least one example where removing the author
would have other benefits beyond MTAs. :) :)

don't feed the trolls (was: intermediate summary of ext3-2.4-0.9.4 thread)

[Matthias Andree]

On Fri, 03 Aug 2001, Albert D. Cahalan wrote:

> This is just completely true. One wonders why we seem to enjoy
> getting screwed this way. We shouldn't be patching these MTAs or
> hacking Linux to act like BSD. We should be avoiding these MTAs.

Oh, you should make a start avoiding any MTAs because that way, this
list would get rid of one trouble maker after all.

Don't feed the trolls.

Re: don't feed the trolls (was: intermediate summary of ext3-2.4-0.9.4 thread)

[Albert D. Cahalan]

Matthias Andree writes:
> On Fri, 03 Aug 2001, Albert D. Cahalan wrote:

>> This is just completely true. One wonders why we seem to enjoy
>> getting screwed this way. We shouldn't be patching these MTAs or
>> hacking Linux to act like BSD. We should be avoiding these MTAs.
>
> Oh, you should make a start avoiding any MTAs because that way, this
> list would get rid of one trouble maker after all.
>
> Don't feed the trolls.

That wasn't intended to be a troll, though I do realize that it
could cause some noise -- including your post. Plenty of noise is
already being generated trying to accommodate hostile MTA authors.

Seriously, consider:

1. there are MTA authors that actively promote BSD over Linux
2. Linux users and distributions promote their MTA software

There is no sense in this. It is masochism and suicide.
It is worse than a waste of time to accommodate these MTAs.

Getting back on topic... while non-inherited ext2 attributes might
be nice, I'm sure the ext2/VFS authors don't need to be pestered
about it, and certainly not because of some lame software making
non-standard assumptions about filesystem behavior.

Re: don't feed the trolls (was: intermediate summary of ext3-2.4-0.9.4 thread)

[Matthias Andree]

On Sat, 04 Aug 2001, Albert D. Cahalan wrote:

> Seriously, consider:
> 
> 1. there are MTA authors that actively promote BSD over Linux
> 2. Linux users and distributions promote their MTA software

I do not endorse this behaviour (particularly, qmail not supporting
softupdates is rather ridiculous), but I understand that MTA authors
would rather want to rely on fsync() also bringing related meta data do
disk (as ext3 and reiserfs for Linux 2.4 already do even across a
rename()!) than to add dir=open("directory"); fsync(dir); close(dir) all
over the place.

> Getting back on topic... while non-inherited ext2 attributes might

What would they be good for? Make MTA that have in the past achieved
reliable behaviour with chattr +S unreliable?

> be nice, I'm sure the ext2/VFS authors don't need to be pestered
> about it, and certainly not because of some lame software making
> non-standard assumptions about filesystem behavior.

Well, the software documents its requirements and assumptions. I don't
see anything nonstandard with relying on fsync(). If ext2fs doesn't meet
the assumptions without chattr +S or mount -o sync, but allows to
enforce this behaviour chattr +S, deliberately breaking ext2 attributes
inheritance will make Linux deliberately unsuitable for this MTA -- or
at least, slow it down through the need to use mount -o sync.

Deliberately breaking things just to show somebody else "you cannot even
rely that chattr behaviour is invariant" is ridiculous and definitely
not the right way to go.

If the MTA author chooses chattr +S over fsync-directory, what's wrong
with that?

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Alexander Viro]

On Thu, 2 Aug 2001, Daniel Phillips wrote:

> I don't know why it is hard or inefficient to implement this at the VFS 
> level, though I'm sure there is a reason or this thread wouldn't 
> exist.  Stephen, perhaps you could explain for the record why sys_fsync 
> can't just walk the chain of dentry parent links doing fdatasync?  Does 
> this create VFS or Ext3 locking problems?  Or maybe it repeats work 
> that Ext3 is already supposed to have done?
 
Parent directory can be renamed. Which grandparent should we sync?
New one? Old one? Both? BTW, how about file itself getting renamed during
fsync()?

See the problem? And no, blocking all renames while fsync() happens is
not an answer - it's a DoS.
 
> [1] In Ext2, all filename dirents are "hard links", i.e., there is no 
> way to tell which of the two names is the original after creating a new 
> hard link.

s/Ext2/UNIX/.

Re: ext3-2.4-0.9.4

[Daniel Phillips]

On Thursday 02 August 2001 19:54, Alexander Viro wrote:
> On Thu, 2 Aug 2001, Daniel Phillips wrote:
> > I don't know why it is hard or inefficient to implement this at the
> > VFS level, though I'm sure there is a reason or this thread
> > wouldn't exist.  Stephen, perhaps you could explain for the record
> > why sys_fsync can't just walk the chain of dentry parent links
> > doing fdatasync?  Does this create VFS or Ext3 locking problems? 
> > Or maybe it repeats work that Ext3 is already supposed to have
> > done?
>
> Parent directory can be renamed. Which grandparent should we sync?
> New one? Old one? Both?

Either one, or both, it doesn't matter since the application has not 
forced any serialization on this and can't assume any.

> BTW, how about file itself getting renamed during fsync()?

It doesn't matter.  If the application wants to race that way, let it.  
We're talking about ensuring access to the fsynced fd's inode.

> See the problem? And no, blocking all renames while fsync() happens
> is not an answer - it's a DoS.

We would have done our duty by fsyncing the inodes one at a time 
working up the dentry chain towards the root, and not trying to lock 
the whole chain.  If something happens while we're doing that it's an 
application race.

--
Daniel

Re: ext3-2.4-0.9.4

[Stephen C. Tweedie]

Hi,

On Thu, Aug 02, 2001 at 07:26:16PM +0200, Daniel Phillips wrote:

> I believed you've summarized the SUS requirements very well.  Apart 
> from legalistic arguments,

Umm, this is a specification.  It is *supposed* to be interpreted
legalistically!

> SUS quite clearly states that fsync should 
> not return until you are sure of having recorded not only the file's 
> data, but the access path to it.  I interpret this as being able to 
> "access the file by its name", and being able to guess by looking in 
> lost+found doesn't count.

But that is just an interpretation.  There's nothing in the spec which
forces that interpretation.

fsync forces the data to disk.  There may be one or more pathnames
which the application also relies on, and if the application does care
about those, the application will have to ensure that they are synced
too.

Look, I agree that your interpretation is useful.  It's just not an
unambiguous requirement of the spec.

Cheers,
 Stephen

Re: ext3-2.4-0.9.4

[Daniel Phillips]

On Friday 03 August 2001 11:06, Stephen C. Tweedie wrote:
> Hi,
>
> On Thu, Aug 02, 2001 at 07:26:16PM +0200, Daniel Phillips wrote:
> > I believe you've summarized the SUS requirements very well.  Apart
> > from legalistic arguments,
>
> Umm, this is a specification.  It is *supposed* to be interpreted
> legalistically!

I'm saying that, when the intent is clear as it is in this case then 
trying to find loopholes in the form of expression is not useful.  
Better to argue that SUS is wrong than to pretend it didn't say what it 
did.

> > SUS quite clearly states that fsync should
> > not return until you are sure of having recorded not only the
> > file's data, but the access path to it.  I interpret this as being
> > able to "access the file by its name", and being able to guess by
> > looking in lost+found doesn't count.
>
> But that is just an interpretation.  There's nothing in the spec
> which forces that interpretation.

Well, look at the name "lost+found".  It's lost, maybe we found the 
data but the name is gone for good.  On the other hand, if we start 
with the same on-disk state that fsck had before creating the 
lost+found, but use a journal to recover the name, it *does* count 
because we have a deterministic mechanism for making fsync's promise 
come true.

> fsync forces the data to disk.  There may be one or more pathnames
> which the application also relies on, and if the application does
> care about those, the application will have to ensure that they are
> synced too.
>
> Look, I agree that your interpretation is useful.  It's just not an
> unambiguous requirement of the spec.

OK, fine, this damn English language and all that.  Do we agree that 
"access" means "be able to open by name"?

--
Daniel

Re: ext3-2.4-0.9.4

[Stephen C. Tweedie]

On Fri, Aug 03, 2001 at 04:08:20PM +0200, Daniel Phillips wrote:

> I'm saying that, when the intent is clear as it is in this case then 
> trying to find loopholes in the form of expression is not useful. 

The intent is perfectly clear regarding the data.  It is totally
undefined regarding names. 
 
> Well, look at the name "lost+found".  It's lost, maybe we found the 
> data but the name is gone for good.

That's fine --- it satisfies the SUS requirements.

> On the other hand, if we start 
> with the same on-disk state that fsck had before creating the 
> lost+found, but use a journal to recover the name, it *does* count 
> because we have a deterministic mechanism for making fsync's promise 
> come true.

That's an implementation decision, not a comment on the spec.

> > Look, I agree that your interpretation is useful.  It's just not an
> > unambiguous requirement of the spec.
> 
> OK, fine, this damn English language and all that.  Do we agree that 
> "access" means "be able to open by name"?

The word "access" isn't used there in the spec, so it doesn't matter.
The spec only refers to "all file system information required to
retrieve the data."  Integrity of the data is the only thing
guaranteed, not integrity of the namespace.

Cheers,
 Stephen

Re: ext3-2.4-0.9.4

[David S. Miller]

Stephen C. Tweedie writes:
 > The word "access" isn't used there in the spec, so it doesn't matter.
 > The spec only refers to "all file system information required to
 > retrieve the data."  Integrity of the data is the only thing
 > guaranteed, not integrity of the namespace.

In fact this interpretation would have a severe performance impact
for any implementation.

If you include "metadata of the 'path'" in "all filesystem
information..." then you have to basically sync each and every element
on the path(s) to that file.  This means walking each dentry in the
alias list for the inode, and then walk from each of those to the root
sync'ing along the way.

That would be a rediculious requirement.

Later,
David S. Miller
davem@redhat.com

Re: ext3-2.4-0.9.4

[Daniel Phillips]

On Friday 03 August 2001 17:11, David S. Miller wrote:
> Stephen C. Tweedie writes:
>  > The word "access" isn't used there in the spec, so it doesn't matter.
>  > The spec only refers to "all file system information required to
>  > retrieve the data."  Integrity of the data is the only thing
>  > guaranteed, not integrity of the namespace.
>
> In fact this interpretation would have a severe performance impact
> for any implementation.
>
> If you include "metadata of the 'path'" in "all filesystem
> information..." then you have to basically sync each and every element
> on the path(s) to that file.  This means walking each dentry in the
> alias list for the inode, and then walk from each of those to the root
> sync'ing along the way.
>
> That would be a rediculious requirement.

As I read the (excerpts from the) SUS, this isn't required, only that
at least one namespace path from the root to the fsynced file is
preserved.  I can imagine an efficient implementation for this.

--
Daniel

Re: ext3-2.4-0.9.4

[Bill Rugolsky Jr.]

On Fri, Aug 03, 2001 at 05:25:09PM +0200, Daniel Phillips wrote:
> As I read the (excerpts from the) SUS, this isn't required, only that
> at least one namespace path from the root to the fsynced file is
> preserved.  I can imagine an efficient implementation for this.

I might be way off base here; if so tell me.
Let's litter some sample code with fsync():

	fd = open("tmp/x",O_CREAT|O_WRONLY);
	...
	fsync(fd);
	rename("tmp/x","spool/x");
	fsync(fd);
	close(fd);

This looks safe and correct, given your semantics.  It is, unless the
link count of that file rises above 1, from say a mail admin quite
reasonably doing

	ln tmp/x peek/x

in the interval specified by "...".  In that case, it's not required
that "tmp/x" or "spool/x" ever hit disk.  So the code is only correct
if the file only has a single link, or we specify ordered meta-data
updates for open/rename/link/...  Following Stephen's argument, is 
"peek/x" any better than "lost+found"?  With more than one admin?

On older non-BSD systems (SYS3?,SYSV 3.x?) that don't do rename(), files can't
be moved without bumping their link counts:

	fd = open("tmp/x",O_CREAT|O_WRONLY);
	...
	fsync(fd);
	link("tmp/x","spool/x");
	fsync(fd);  /* <- possibly a NOP with your semantics */
	unlink("tmp/x");
	fsync(fd);
	close(fd);

Again, this will fail to preserve your desired semantics on crash,
unless we have ordered meta-data updates, or the stronger synchronous
link() requirement.

I think the semantics that you propose might be marginally useful, but
I don't think SUS requires it; my understanding (and that of a
close friend and former POSIX reviewer) has always been that inodes are
distinct from directory entries, and that fsync() preserves the fields
that stat() returns: mode, uid, gid, size, {a,c,m}time.

Regards,

   Bill Rugolsky

Re: ext3-2.4-0.9.4

[Bill Rugolsky Jr.]

On Fri, Aug 03, 2001 at 01:06:01PM -0400, Bill Rugolsky Jr. wrote:
> 	ln tmp/x peek/x

In fact, make that

	ln tmp/x peek/x
			<- MTA does the fsync()
	less peek/x
			<- MTA closes the file.
	rm peek/x
		     (peek gets written asynchronously to disk)
	*CRASH*
                           
Then we are back to "lost+found", nothing gained.

Regards,

   Bill Rugolsky

Re: ext3-2.4-0.9.4

[Jan Harkes]

On Fri, Aug 03, 2001 at 04:08:20PM +0200, Daniel Phillips wrote:
> > fsync forces the data to disk.  There may be one or more pathnames
> > which the application also relies on, and if the application does
> > care about those, the application will have to ensure that they are
> > synced too.
> >
> > Look, I agree that your interpretation is useful.  It's just not an
> > unambiguous requirement of the spec.
> 
> OK, fine, this damn English language and all that.  Do we agree that 
> "access" means "be able to open by name"?

No, until recently the device/inode number pair used to work very nicely
for both Coda and knfsd when they wanted to access a file. But it only
works from within the kernel where you can use iget. It's just that with
some of the newer filesystems the inode numbers are no longer unique, so
it became something more like device/inum/opaque handle (i.e. iget4).

As far as the fsync semantics are concerned. Yeah, they would be useful,
but only to avoid one directory fsync call that will tell the kernel
_exactly_ what the process wants instead of letting it figure it out by
itself. The argument I saw in this thread that fsync(dir) has too much
overhead because it might sync unrelated data is not very useful,
because that unrelated data will be synced anyways when it's not a
journalling fs.

Name to file object mapping is not part of the metadata associated with
a file. It is the contents of the directory and can only be modified by
directory operations, not operations on the file or filehandle.
open(file, O_CREAT) is really split up into create(parent, file) an
operation on the parent directory, and an open(file) operation which
returns the filehandle.

I also don't see why a rename operation, which operates on the source
and destination parent directories would have to not only look up the
file object but also somehow register with all open filehandles for that
object that both olddir and newdir need to be written back to disk
during the fsync as well. Or should that go the other way around, where
the filehandle has to chase down which directory it was renamed to?

Taken from another part of this thread Alexander and Daniel,
> > That there isn't THE directory in which the file resides. There might
> > be several, and setting the bit on one of them at random and expect
> > it to work is a _lot_ of work. For no real use.
>
> There is only one chain of directories from the fd's dentry up to the
> root, that's the one to sync.

Using the dentry chain is not reliable, for instance instead of moving
dentries around Coda simply unhashes dentries when state on the server
changes. Another process (perhaps on a different machine) might have
moved one of the ancestor directories from one location to another, or
even relinked/unlinked the file that we're working with (ln old new; rm
old) in which case the dentry path is lost, but ideally you'd expect
fsync to sync the 'new' name if it supposedly keeps the namespace
consistent.

Working on a distributed filesystem with somewhat weaker than UNIX
semantics might have skewed my vision. In Coda not every client will be
able to figure out which are all of the possibly paths that can lead to
a file object. And although we currently try our best to block
hardlinked directories they could possibly exist, making the problems
even worse.

Jan

Re: ext3-2.4-0.9.4

[Daniel Phillips]

On Friday 03 August 2001 17:18, Jan Harkes wrote:
> On Fri, Aug 03, 2001 at 04:08:20PM +0200, Daniel Phillips wrote:
> > > fsync forces the data to disk.  There may be one or more pathnames
> > > which the application also relies on, and if the application does
> > > care about those, the application will have to ensure that they are
> > > synced too.
> > >
> > > Look, I agree that your interpretation is useful.  It's just not an
> > > unambiguous requirement of the spec.
> >
> > OK, fine, this damn English language and all that.  Do we agree that
> > "access" means "be able to open by name"?
>
> No, until recently the device/inode number pair used to work very nicely
> for both Coda and knfsd when they wanted to access a file. But it only
> works from within the kernel where you can use iget. It's just that with
> some of the newer filesystems the inode numbers are no longer unique, so
> it became something more like device/inum/opaque handle (i.e. iget4).

We are talking about "after waking up from an unexpected interruption".
So, is this still relevant?

> As far as the fsync semantics are concerned. Yeah, they would be useful,
> but only to avoid one directory fsync call that will tell the kernel
> _exactly_ what the process wants instead of letting it figure it out by
> itself.

No, it's not just that.  It's not enough to fsync just the parent, the
parent's parent may have been relinked and not comitted.  Also, the
kernel has the advantage of the locked chain of dcache entries
available to it.  Finally, there is the saving of multiple syscall
overhead, which I didn't mention first time around because it's not a
lot compared to media access overhead.

> The argument I saw in this thread that fsync(dir) has too much
> overhead because it might sync unrelated data is not very useful,
> because that unrelated data will be synced anyways when it's not a
> journalling fs.

Yes, and I gave a detailed explanation earlier of why such overhead
will not amount to much.

> Name to file object mapping is not part of the metadata associated with
> a file. It is the contents of the directory and can only be modified by
> directory operations, not operations on the file or filehandle.

SUS doesn't just pronounce on the file metadata.  Quoting from earlier
in the thread:

----------
>   "synchronised I/O data integrity completion
>
>   [...]
>
>   * For write, when the operation has been completed or diagnosed if
>   unsuccessful.  The write is complete only when the data specified in
>   the write request is successfully transferred and all file system
>   information required to retrieve the data is successfully transferred.
----------

> open(file, O_CREAT) is really split up into create(parent, file) an
> operation on the parent directory, and an open(file) operation which
> returns the filehandle.
>
> I also don't see why a rename operation, which operates on the source
> and destination parent directories would have to not only look up the
> file object but also somehow register with all open filehandles for that
> object that both olddir and newdir need to be written back to disk
> during the fsync as well.

They don't both have to, either one will be good enough.  However,
"neither" is not good enough, according to SUS.

> Or should that go the other way around, where
> the filehandle has to chase down which directory it was renamed to?
>
> Taken from another part of this thread Alexander and Daniel,
>
> > > That there isn't THE directory in which the file resides. There might
> > > be several, and setting the bit on one of them at random and expect
> > > it to work is a _lot_ of work. For no real use.
> >
> > There is only one chain of directories from the fd's dentry up to the
> > root, that's the one to sync.
>
> Using the dentry chain is not reliable, for instance instead of moving
> dentries around Coda simply unhashes dentries when state on the server
> changes.

Could you be more specific about this, are you saying there are cases
where there is no valid parent link from a dcache entry?

> Another process (perhaps on a different machine) might have
> moved one of the ancestor directories from one location to another, or
> even relinked/unlinked the file that we're working with (ln old new; rm
> old) in which case the dentry path is lost, but ideally you'd expect
> fsync to sync the 'new' name if it supposedly keeps the namespace
> consistent.

I doesn't matter which one it syncs.

> Working on a distributed filesystem with somewhat weaker than UNIX
> semantics might have skewed my vision. In Coda not every client will be
> able to figure out which are all of the possibly paths that can lead to
> a file object. And although we currently try our best to block
> hardlinked directories they could possibly exist, making the problems
> even worse.

We don't need all the paths, and not any specific path, just a path.

--
Daniel

Re: ext3-2.4-0.9.4

[Stephen C. Tweedie]

Hi,

On Fri, Aug 03, 2001 at 05:47:17PM +0200, Daniel Phillips wrote:

> > As far as the fsync semantics are concerned. Yeah, they would be useful,
> > but only to avoid one directory fsync call that will tell the kernel
> > _exactly_ what the process wants instead of letting it figure it out by
> > itself.
> 
> No, it's not just that.  It's not enough to fsync just the parent, the
> parent's parent may have been relinked and not comitted.  Also, the
> kernel has the advantage of the locked chain of dcache entries
> available to it.

Not necessarily.  If the file has been hard-linked and then the
original link removed, we don't have a valid dcache entry any more
(and yes, this is a common construct for some applications --- it is
often used to work around NFS atomicity problems.)

An MTA using such a construct and expecting fsync to do the right
thing will *not* work if you follow the dcache chain on fsync as you
propose here.

> We don't need all the paths, and not any specific path, just a path.

Exactly, because fsync makes absolutely no gaurantees about the
namespace.  So a lost+found path is quite sufficient.

Cheers,
 Stephen

Re: ext3-2.4-0.9.4

[Daniel Phillips]

On Friday 03 August 2001 17:50, Stephen C. Tweedie wrote:
> Hi,
>
> On Fri, Aug 03, 2001 at 05:47:17PM +0200, Daniel Phillips wrote:
> > > As far as the fsync semantics are concerned. Yeah, they would be
> > > useful, but only to avoid one directory fsync call that will tell the
> > > kernel _exactly_ what the process wants instead of letting it figure it
> > > out by itself.
> >
> > No, it's not just that.  It's not enough to fsync just the parent, the
> > parent's parent may have been relinked and not comitted.  Also, the
> > kernel has the advantage of the locked chain of dcache entries
> > available to it.
>
> Not necessarily.  If the file has been hard-linked and then the
> original link removed, we don't have a valid dcache entry any more
> (and yes, this is a common construct for some applications --- it is
> often used to work around NFS atomicity problems.)

But in that case, the file was opened using the hard link, then the link
was deleted.  Fine.  The user is trying to tell us it's ok to lose the
linked file.  Whether or not it can be accessed through another path
is irrelevant.

> An MTA using such a construct and expecting fsync to do the right
> thing will *not* work if you follow the dcache chain on fsync as you
> propose here.

OK, this case where the walk to the root should fail is a "duh", and
exposes a corner case SUS didn't cover (at least not in the excerpts
I saw).  But this case is a userland race, the right thing to do is
just stop the walk.  This doesn't detract from the value of doing
the walk in the important case that the chain is intact.

> > We don't need all the paths, and not any specific path, just a path.
>
> Exactly, because fsync makes absolutely no gaurantees about the
> namespace.  So a lost+found path is quite sufficient.

Dunno, I think that's a statement that should be held up for further
scrutiny.

--
Daniel

Re: ext3-2.4-0.9.4

[Matthias Andree]

On Fri, 03 Aug 2001, Stephen Tweedie wrote:

> > We don't need all the paths, and not any specific path, just a path.
> 
> Exactly, because fsync makes absolutely no gaurantees about the
> namespace.  So a lost+found path is quite sufficient.

MTA authors don't share this. lost+found is "invisible" for the
application that created the file.

I have yet to meet a distribution which scans lost+found at boot time
and syslogs found files or sends root a mail.

So, effectively, lost+found will NOT be sufficient. Discarding file
names at will is not a good thing.

-- 
Matthias Andree

Re: ext3-2.4-0.9.4

[Zilvinas Valinskas]

On Fri, Aug 03, 2001 at 08:11:12PM +0200, Matthias Andree wrote:
> On Fri, 03 Aug 2001, Stephen Tweedie wrote:
> 
> > > We don't need all the paths, and not any specific path, just a path.
> > 
> > Exactly, because fsync makes absolutely no gaurantees about the
> > namespace.  So a lost+found path is quite sufficient.
> 
> MTA authors don't share this. lost+found is "invisible" for the
> application that created the file.
> 
> I have yet to meet a distribution which scans lost+found at boot time
> and syslogs found files or sends root a mail.

Debian Woody ...
> 
> So, effectively, lost+found will NOT be sufficient. Discarding file
> names at will is not a good thing.
> 
> -- 
> Matthias Andree
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Zilvinas Valinskas

Re: ext3-2.4-0.9.4

[Jan Harkes]

On Fri, Aug 03, 2001 at 05:47:17PM +0200, Daniel Phillips wrote:
> On Friday 03 August 2001 17:18, Jan Harkes wrote:
> > No, until recently the device/inode number pair used to work very nicely
> > for both Coda and knfsd when they wanted to access a file. But it only
> > works from within the kernel where you can use iget. It's just that with
> > some of the newer filesystems the inode numbers are no longer unique, so
> > it became something more like device/inum/opaque handle (i.e. iget4).
> 
> We are talking about "after waking up from an unexpected interruption".
> So, is this still relevant?

The NFS client hasn't been interrupted, and it's filehandle will still
identifies the file object by device/inode.

> > Name to file object mapping is not part of the metadata associated with
> > a file. It is the contents of the directory and can only be modified by
> > directory operations, not operations on the file or filehandle.
> 
> SUS doesn't just pronounce on the file metadata.  Quoting from earlier
> in the thread:
> 
> ----------
> >   "synchronised I/O data integrity completion
> >
> >   [...]
> >
> >   * For write, when the operation has been completed or diagnosed if
> >   unsuccessful.  The write is complete only when the data specified in
> >   the write request is successfully transferred and all file system
> >   information required to retrieve the data is successfully transferred.
> ----------

So that would be the file data, and it's on-disk inode information,
indirect blocks etc. All the information that the file system needs to
retrieve the data is then available, i.e. what is required for iget()
to succeed.

Ok, iget isn't exported to userspace, but fsck will place the file in a
user reachable location.

> > I also don't see why a rename operation, which operates on the source
> > and destination parent directories would have to not only look up the
> > file object but also somehow register with all open filehandles for that
> > object that both olddir and newdir need to be written back to disk
> > during the fsync as well.
> 
> They don't both have to, either one will be good enough.  However,
> "neither" is not good enough, according to SUS.

Ehh, sync only olddir and you just lost any path leading to the file.
Sync only newdir and the file is reachable from two locations, but it's
linkcount is too low.

> > Using the dentry chain is not reliable, for instance instead of moving
> > dentries around Coda simply unhashes dentries when state on the server
> > changes.
> 
> Could you be more specific about this, are you saying there are cases
> where there is no valid parent link from a dcache entry?

No the dcache entry could have a 'stale' fileobject associated with it
that has been superceded by a different object. This dentry is unhashed,
so that the next lookup will instantiate a new dentry which references
the new object. So syncing the stale object is useless, because it
doesn't really exist anymore, but the kernel (and actually the userspace
daemon on the client) doesn't know what the new object is until it is
accessed.

> > Working on a distributed filesystem with somewhat weaker than UNIX
> > semantics might have skewed my vision. In Coda not every client will be
> > able to figure out which are all of the possibly paths that can lead to
> > a file object. And although we currently try our best to block
> > hardlinked directories they could possibly exist, making the problems
> > even worse.
> 
> We don't need all the paths, and not any specific path, just a path.

Even if that path leads to a name that got removed, thereby forcing the
object into lost+found? I thought the MTA did something like,

fd = open(tmp/file)
write(fd)
fsync(fd)
link(tmp/file, new/file)
fsync(fd)		*1
unlink(tmp/file)

*1 If this fsync only syncs the path leading to tmp/file, and the unlink
tmp/file is written back to disk, which is likely because we're only
creating/syncing stuff in tmp.  Now, until new/file is written there is
no path information leading to the file anymore which makes this as
'safe' as not syncing path name information at all.

Now if the application would use the directory sync, it can actually
tell the kernel that that new/file name is the interesting one to keep
and that tmp doesn't even need to be written to disk at all.

Jan

Re: ext3-2.4-0.9.4

[Daniel Phillips]

On Friday 03 August 2001 18:16, Jan Harkes wrote:
> On Fri, Aug 03, 2001 at 05:47:17PM +0200, Daniel Phillips wrote:
> > On Friday 03 August 2001 17:18, Jan Harkes wrote:
> > > No, until recently the device/inode number pair used to work very
> > > nicely for both Coda and knfsd when they wanted to access a file. But
> > > it only works from within the kernel where you can use iget. It's just
> > > that with some of the newer filesystems the inode numbers are no longer
> > > unique, so it became something more like device/inum/opaque handle
> > > (i.e. iget4).
> >
> > We are talking about "after waking up from an unexpected interruption".
> > So, is this still relevant?
>
> The NFS client hasn't been interrupted, and it's filehandle will still
> identifies the file object by device/inode.

Interesting, but at best relevant only to network mounts.

> > > Name to file object mapping is not part of the metadata associated with
> > > a file. It is the contents of the directory and can only be modified by
> > > directory operations, not operations on the file or filehandle.
> >
> > SUS doesn't just pronounce on the file metadata.  Quoting from earlier
> > in the thread:
> >
> > ----------
> >
> > >   "synchronised I/O data integrity completion
> > >
> > >   [...]
> > >
> > >   * For write, when the operation has been completed or diagnosed if
> > >   unsuccessful.  The write is complete only when the data specified in
> > >   the write request is successfully transferred and all file system
> > >   information required to retrieve the data is successfully
> > > transferred.
> >
> > ----------
>
> So that would be the file data, and it's on-disk inode information,
> indirect blocks etc. All the information that the file system needs to
> retrieve the data is then available, i.e. what is required for iget()
> to succeed.

In spite of your interesting network filesystem example I'm not willing
to accept that access by inode is enough.  It's going pretty far out on
a limb, don't you agree?  I doubt that SUS explicitly allows the inum to
be interpreted as "information that the file system needs to retrieve
the data".

> Ok, iget isn't exported to userspace, but fsck will place the file in a
> user reachable location.

Hmmph.  It used to have a name, now it doesn't, and somebody did an fsync
on the file, which returned indicating success.  Do you think that's
right?

> > > I also don't see why a rename operation, which operates on the source
> > > and destination parent directories would have to not only look up the
> > > file object but also somehow register with all open filehandles for
> > > that object that both olddir and newdir need to be written back to disk
> > > during the fsync as well.
> >
> > They don't both have to, either one will be good enough.  However,
> > "neither" is not good enough, according to SUS.
>
> Ehh, sync only olddir and you just lost any path leading to the file.

You sync the one the dcache entry points at.

> Sync only newdir and the file is reachable from two locations, but it's
> linkcount is too low.

SUS didn't say the filesystem integrity had to be perfect, just that
"all file system information required to retrieve the data is
successfully transferred".

> > > Using the dentry chain is not reliable, for instance instead of moving
> > > dentries around Coda simply unhashes dentries when state on the server
> > > changes.
> >
> > Could you be more specific about this, are you saying there are cases
> > where there is no valid parent link from a dcache entry?
>
> No the dcache entry could have a 'stale' fileobject associated with it
> that has been superceded by a different object. This dentry is unhashed,
> so that the next lookup will instantiate a new dentry which references
> the new object. So syncing the stale object is useless, because it
> doesn't really exist anymore, but the kernel (and actually the userspace
> daemon on the client) doesn't know what the new object is until it is
> accessed.

This happens because the dentry was unlinked, but not when it is renamed,
right?  In that case it's ok to fail the walk IMHO, the user explicitly
left a window when it's ok to interpret the fsynced file as "gone".

> > > Working on a distributed filesystem with somewhat weaker than UNIX
> > > semantics might have skewed my vision. In Coda not every client will be
> > > able to figure out which are all of the possibly paths that can lead to
> > > a file object. And although we currently try our best to block
> > > hardlinked directories they could possibly exist, making the problems
> > > even worse.
> >
> > We don't need all the paths, and not any specific path, just a path.
>
> Even if that path leads to a name that got removed, thereby forcing the
> object into lost+found? I thought the MTA did something like,

We'd better get confirmation from the MTA expert in the thread.

> fd = open(tmp/file)
> write(fd)
> fsync(fd)
> link(tmp/file, new/file)
> fsync(fd)		*1
> unlink(tmp/file)
>
> *1 If this fsync only syncs the path leading to tmp/file, and the unlink
> tmp/file is written back to disk, which is likely because we're only
> creating/syncing stuff in tmp.  Now, until new/file is written there is
> no path information leading to the file anymore which makes this as
> 'safe' as not syncing path name information at all.

Nice clear example!  Yes, in essence we would have synced the original
path twice.  If this is what the MTA is really doing I'm willing to join
the "MTA is broken" camp.

> Now if the application would use the directory sync, it can actually
> tell the kernel that that new/file name is the interesting one to keep
> and that tmp doesn't even need to be written to disk at all.

Yep.  Or if it used rename, which updates the dcache entries, instead
of link/unlink.

--
Daniel

Re: ext3-2.4-0.9.4

[Rik van Riel]

On Fri, 3 Aug 2001, Daniel Phillips wrote:
> On Friday 03 August 2001 11:06, Stephen C. Tweedie wrote:
> > On Thu, Aug 02, 2001 at 07:26:16PM +0200, Daniel Phillips wrote:

> > Look, I agree that your interpretation is useful.  It's just not an
> > unambiguous requirement of the spec.
>
> OK, fine, this damn English language and all that.  Do we agree that
> "access" means "be able to open by name"?

If we didn't agree on this, Linux would have had an
open_by_inode() system call long ago.

Rik
--
Virtual memory is like a game you can't win;
However, without VM there's truly nothing to lose...

http://www.surriel.com/		http://distro.conectiva.com/

Send all your spam to aardvark@nl.linux.org (spam digging piggy)

Re: ext3-2.4-0.9.4

[Chris Wedgwood]

On Thu, Jul 26, 2001 at 09:42:37PM +1000, Andrew Morton wrote:

    If postfix is capable of opening the files O_SYNC or of doing
    fsync() on them then the `chattr +s' is no longer necessary -
    unlike ext2, when the O_SYNC write() or the fsync() return, the
    directory contents (as well as the inode, bitmaps, data, etc) will
    all be tight on disk and will be restored after a crash.

    This should speed things up considerably, especially with
    journalled-data mode.  I need to test and characterise this some
    more to come up with some quantitative results and configuration
    recommendations.

Postfix does an fsync on file before closing them, it then does a
rename and expects once rename as returned, the renamed actually
occured --- even if the fs crashes.  It also expects if you fsync a
file, then it will appear in the parent directory with certainty and
not say /lost+found after fsck on reboot.

Without +s under ext2, you can loose file(s) in /lost+found because
open+write+fsync+close works and ensures the data is on disk, but the
parent directory doesn't get synced to disk, so it might get lost.




  --cw

Strange remount behaviour with ext3-2.4-0.9.4

[Sean Hunter]

Following the announcement on lkml, I have started using ext3 on one of my
servers.  Since the server in question is a farily security-sensitive box, my
/usr partition is mounted read only except when I remount rw to install
packages.

I converted this partition to run ext3 with the mount options
"nodev,ro,data=writeback,defaults" figuring that when I need to install new
packages etc, that I could just mount rw as before and that metadata-only
journalling would be ok for this partition as it really sees very little write
activity.

When I try to remount it r/w I get a log message saying:
Jul 27 09:54:29 henry kernel: EXT3-fs: cannot change data mode on remount

...even if I give the full mount option list with the remount instruction.

I can, however, remount it as ext2 read-write, but when I try to remount as
ext3 (even read only) I get the same problem.

Wierdly, "mount" lists it as being still an ext3 partition even though it has
been remounted as ext2.  I can't umount /usr because kjournald is currently
listed as using the partition.

The box in question is more-or-less RedHat 7.1, with ext3-2.4-0.9.4, kernel
2.4.7 and with the following relevant package versions:

mount-2.11g-4
util-linux-2.11f-3
e2fsprogs-1.22-2

...all from rawhide rpms.

Sean

Re: Strange remount behaviour with ext3-2.4-0.9.4

[Andrew Morton]

Sean Hunter wrote:
> 
> Following the announcement on lkml, I have started using ext3 on one of my
> servers.  Since the server in question is a farily security-sensitive box, my
> /usr partition is mounted read only except when I remount rw to install
> packages.
> 
> I converted this partition to run ext3 with the mount options
> "nodev,ro,data=writeback,defaults" figuring that when I need to install new
> packages etc, that I could just mount rw as before and that metadata-only
> journalling would be ok for this partition as it really sees very little write
> activity.
> 
> When I try to remount it r/w I get a log message saying:
> Jul 27 09:54:29 henry kernel: EXT3-fs: cannot change data mode on remount
> 
> ...even if I give the full mount option list with the remount instruction.

hmm..  The mount option handling there is a bit bogus.

What we *should* do on remount is check that the requested
journalling mode is equal to the current mode.  ext3 won't
allow you to change the journalling mode on-the-fly.

So...  you will have to omit the `data=xxx' portion of the
mount options when remounting.  It's being invisibly added
by /bin/mount.

/bin/mount tries to be smart.  If, for example you have

	/dev/hdf12 /mnt/hdf12 ext3 noauto,ro,data=writeback 1

in /etc/fstab and then type

	mount /dev/hdf12 -o remount,rw

then /bin/mount runs off and looks up the fstab entry and
inserts the mount options.  However if you instead type

	mount /dev/hdf12 /mnt/hdf12 -o remount,rw          (1)

then /bin/mount does *not* look up the fstab entry, and
the remount succeeds.

ho-hum.  For the while you'll have to fiddle with the mount
usage to get things working right.   Equation (1) above will
work fine.  Or apply the appended patch.

> I can, however, remount it as ext2 read-write, but when I try to remount as
> ext3 (even read only) I get the same problem.

You can't switch between ext2 and ext3 with a remount - unmount
is needed.

> Wierdly, "mount" lists it as being still an ext3 partition even though it has
> been remounted as ext2.  I can't umount /usr because kjournald is currently
> listed as using the partition.

That sounds very weird.  Could you please describe the steps
you took to create this state?

Sometimes /etc/mtab gets out of sync - especially for the
root fs.  It's more reliable to look in /proc/mounts



Here's the fix for the data= handling on remount:



Index: fs/ext3/super.c
===================================================================
RCS file: /cvsroot/gkernel/ext3/fs/ext3/super.c,v
retrieving revision 1.31
diff -u -r1.31 super.c
--- fs/ext3/super.c	2001/07/19 14:43:08	1.31
+++ fs/ext3/super.c	2001/07/27 10:14:48
@@ -513,12 +513,6 @@
 
 			if (want_value(value, "data"))
 				return 0;
-			if (is_remount) {
-				printk ("EXT3-fs: cannot change data mode "
-						"on remount\n");
-				return 0;
-			}
-
 			if (!strcmp (value, "journal"))
 				data_opt = EXT3_MOUNT_JOURNAL_DATA;
 			else if (!strcmp (value, "ordered"))
@@ -529,9 +523,18 @@
 				printk ("EXT3-fs: Invalid data option: %s\n",
 					value);
 				return 0;
+			}
+			if (is_remount) {
+				if ((*mount_options & EXT3_MOUNT_DATA_FLAGS) !=
+							data_opt) {
+					printk("EXT3-fs: cannot change data "
+						"mode on remount\n");
+					return 0;
+				}
+			} else {
+				*mount_options &= ~EXT3_MOUNT_DATA_FLAGS;
+				*mount_options |= data_opt;
 			}
-			*mount_options &= ~EXT3_MOUNT_DATA_FLAGS;
-			*mount_options |= data_opt;
 		} else {
 			printk ("EXT3-fs: Unrecognized mount option %s\n",
 					this_char);

Re: Strange remount behaviour with ext3-2.4-0.9.4

[Sean Hunter]

On Fri, Jul 27, 2001 at 08:24:14PM +1000, Andrew Morton wrote:
> Sean Hunter wrote:
> > 
> > Following the announcement on lkml, I have started using ext3 on one of my
> > servers.  Since the server in question is a farily security-sensitive box, my
> > /usr partition is mounted read only except when I remount rw to install
> > packages.
> > 
> > I converted this partition to run ext3 with the mount options
> > "nodev,ro,data=writeback,defaults" figuring that when I need to install new
> > packages etc, that I could just mount rw as before and that metadata-only
> > journalling would be ok for this partition as it really sees very little write
> > activity.
> > 
> > When I try to remount it r/w I get a log message saying:
> > Jul 27 09:54:29 henry kernel: EXT3-fs: cannot change data mode on remount
> > 
> > ...even if I give the full mount option list with the remount instruction.
> 
> hmm..  The mount option handling there is a bit bogus.
> 
> What we *should* do on remount is check that the requested
> journalling mode is equal to the current mode.  ext3 won't
> allow you to change the journalling mode on-the-fly.

Indeed.

> 
> So...  you will have to omit the `data=xxx' portion of the
> mount options when remounting.  It's being invisibly added
> by /bin/mount.

Thought so.  I tried both ways just to be sure.

> /bin/mount tries to be smart.  If, for example you have
> 
> 	/dev/hdf12 /mnt/hdf12 ext3 noauto,ro,data=writeback 1
> 
> in /etc/fstab and then type
> 
> 	mount /dev/hdf12 -o remount,rw
> 
> then /bin/mount runs off and looks up the fstab entry and
> inserts the mount options.  However if you instead type
> 
> 	mount /dev/hdf12 /mnt/hdf12 -o remount,rw          (1)
> 
> then /bin/mount does *not* look up the fstab entry, and
> the remount succeeds.

Interesting, and (almost) 100% true

sean@henry:~$  sudo mount /dev/sda8 /usr -oro,nodev,data=writeback,remount
mount: you must specify the filesystem type
sean@henry:~$ sudo mount /dev/sda8 /usr -oro,nodev,data=writeback,remount -text3
mount: /usr not mounted already, or bad option
sean@henry:~$ sudo mount /dev/sda8 /usr -oro,nodev,remount -text3
sean@henry:~$ mount
/dev/sdb6 on / type ext3 (rw)
none on /proc type proc (rw)
/dev/sda1 on /boot type ext2 (ro,nosuid,nodev)
/dev/sdc6 on /home type ext3 (rw,nosuid,nodev,data=ordered)
/dev/sda8 on /usr type ext3 (ro,nodev)
/dev/sda5 on /var type ext3 (rw,nosuid,nodev,sync,data=journal)
none on /dev/pts type devpts (rw,gid=5,mode=620)

It succeeds as long as I don't specify the journal type.

> 
> ho-hum.  For the while you'll have to fiddle with the mount
> usage to get things working right.   Equation (1) above will
> work fine.  Or apply the appended patch.
> 
> > I can, however, remount it as ext2 read-write, but when I try to remount as
> > ext3 (even read only) I get the same problem.
> 
> You can't switch between ext2 and ext3 with a remount - unmount
> is needed.

Wierd.  This certainly looked to all the world as though it worked for me.  Thus:

sean@henry:~$ sudo mount /dev/sda8 /usr -oro,nodev,remount -text2

...doesn't give me an error, but:

sean@henry:~$ mount 
/dev/sdb6 on / type ext3 (rw)
none on /proc type proc (rw)
/dev/sda1 on /boot type ext2 (ro,nosuid,nodev)
/dev/sdc6 on /home type ext3 (rw,nosuid,nodev,data=ordered)
/dev/sda8 on /usr type ext3 (ro,nodev)
                       ^^^^
/dev/sda5 on /var type ext3 (rw,nosuid,nodev,sync,data=journal)
none on /dev/pts type devpts (rw,gid=5,mode=620)


> > Wierdly, "mount" lists it as being still an ext3 partition even though it has
> > been remounted as ext2.  I can't umount /usr because kjournald is currently
> > listed as using the partition.
> 
> That sounds very weird.  Could you please describe the steps
> you took to create this state?

See above.

> Sometimes /etc/mtab gets out of sync - especially for the
> root fs.  It's more reliable to look in /proc/mounts

sean@henry:~$ cat /proc/mounts
/dev/root / ext3 rw 0 0
/proc /proc proc rw 0 0
/dev/sda1 /boot ext2 ro,nosuid,nodev 0 0
/dev/sdc6 /home ext3 rw,nosuid,nodev 0 0
/dev/sda8 /usr ext3 ro,nodev 0 0
/dev/sda5 /var ext3 rw,nosuid,nodev,sync 0 0
none /dev/pts devpts rw 0 0

sean@henry:~$ cat /etc/mtab   
/dev/sdb6 / ext3 rw 0 0
none /proc proc rw 0 0
/dev/sda1 /boot ext2 ro,nosuid,nodev 0 0
/dev/sdc6 /home ext3 rw,nosuid,nodev,data=ordered 0 0
/dev/sda8 /usr ext3 ro,nodev 0 0
/dev/sda5 /var ext3 rw,nosuid,nodev,sync,data=journal 0 0
none /dev/pts devpts rw,gid=5,mode=620 0 0

Its almost as if mount is just silently ignoring the "-t" option when I specify
ext2.

> 
> 
> Here's the fix for the data= handling on remount:

I'll try this when its safe to reboot the box.

Thanks very much for your help.

Sean 

Re: Strange remount behaviour with ext3-2.4-0.9.4

[Michal Jaegermann]

On Fri, Jul 27, 2001 at 10:32:21AM +0100, Sean Hunter wrote:
> Following the announcement on lkml, I have started using ext3 on one of my
> servers.  Since the server in question is a farily security-sensitive box, my
> /usr partition is mounted read only except when I remount rw to install
> packages.

Regardless of possible weirdness in a "smart" behaviour of 'mount' what
one exactly buys running a journaling file system on a _read only_
partition?  fsck times will be the same (unless you crashed when
installing new software :-).

  Michal

Re: Strange remount behaviour with ext3-2.4-0.9.4

[Alan Cox]

> Regardless of possible weirdness in a "smart" behaviour of 'mount' what
> one exactly buys running a journaling file system on a _read only_
> partition?  fsck times will be the same (unless you crashed when
> installing new software :-).

Several things:

1.	The simple case of remounting an fs read-only is easy, since no
	writes means no journal

2.	The software suspend case is horrible. Right now mixing a
	journalling fs and swsuspend tends to cause disk corruption because
	journalling fs's write to disk when told to mount read only

3.	Failed drives. Here the journalling mount overrides the read only
	request and the machine locks up preventing data recovery except
	by copying the whole 80Gb disk image to another disk

	Been there, it sucks

4.	Snapshots. Making read only snapshots can be very useful, and there
	you want the replay of the log to be into the page cache but not
	written back to physical media until its marked read-write

Alan

Re: Strange remount behaviour with ext3-2.4-0.9.4

[Chris Wedgwood]

On Fri, Jul 27, 2001 at 09:46:57PM +0100, Alan Cox wrote:

    2.	The software suspend case is horrible. Right now mixing a
    	journalling fs and swsuspend tends to cause disk corruption because
    	journalling fs's write to disk when told to mount read only

this is hard to fix... the fs needs to replay things to make things
consistent, and in many cases doing an 'in-memory' replay isn't an
option (ie. remember which stuff needs to replayed and read from the
journal instead of disk when required to do so)

    4.	Snapshots. Making read only snapshots can be very useful, and there
    	you want the replay of the log to be into the page cache but not
    	written back to physical media until its marked read-write

R/O snapshots are best done in the fs if possible, al la
WAFL. Something like that for resierfs or TUX2 would rule so much (you
more-or-less need need a tree-based fs and reference counting for all
the magic bits).  In fact, doing it as the fs layer means you could
have r/w snapshots with COW semantics.



  --cw

Re: Strange remount behaviour with ext3-2.4-0.9.4

[Alan Cox]

> more-or-less need need a tree-based fs and reference counting for all
> the magic bits).  In fact, doing it as the fs layer means you could
> have r/w snapshots with COW semantics.

You dont want r/w snapshots for archiving. An archive of a previous date is
worthless if it can't be absolutely utterly and definitively read only.

It is hard to do well, but its an important item. One possiiblity is to do
it by replaying the log to a r/w snapshot (in ram) over a r/o snapshot (on
stable media)

Re: Strange remount behaviour with ext3-2.4-0.9.4

[Chris Wedgwood]

On Fri, Jul 27, 2001 at 10:23:44PM +0100, Alan Cox wrote:

    You dont want r/w snapshots for archiving. An archive of a
    previous date is worthless if it can't be absolutely utterly and
    definitively read only.

sure, for archiving you don't, but for other purposes you might

RO is easier and what most people want, this is all WAFL gives right now

RW has it's uses too, especially if you can clone /foo/bar to
/foo/blem and such like, a cheaper more elegant way of cp -Rupdl I
guess

    It is hard to do well, but its an important item. One possiiblity
    is to do it by replaying the log to a r/w snapshot (in ram) over a
    r/o snapshot (on stable media)

you can probably get away without the need for replay... just build
and in-memory extent list of blocks to would otherwise have been
rewritten and the journal offsets, before you read a block, you check
to see if you need to get from journal first

obviously you need to make sure you get the last insatce of each block
in the journal should there be more than one



  --cw

Re: Strange remount behaviour with ext3-2.4-0.9.4

[Kai Henningsen]

alan@lxorguk.ukuu.org.uk (Alan Cox)  wrote on 27.07.01 in <E15QF5E-0006ZL-00@the-village.bc.nu>:

> > more-or-less need need a tree-based fs and reference counting for all
> > the magic bits).  In fact, doing it as the fs layer means you could
> > have r/w snapshots with COW semantics.
>
> You dont want r/w snapshots for archiving.

Not for archiving, but when you want to run something and then throw it  
away again, for example. You could do that by just holding onto a ro  
snapshot and then replacing the rw tree with it later, but by having two  
rw trees you don't need to stop your regular operations.

For this to really be useful, you'd want it as an inheritable per-process  
thing, similar to aviro's namespace thing.

MfG Kai

Re: ext3-2.4-0.9.4

[Philipp Matthias Hahn]

On Thu, 26 Jul 2001, Andrew Morton wrote:

> An update to the ext3 filesystem for 2.4 kernels is available at
>
> 	http://www.uow.edu.au/~andrewm/linux/ext3/
I'm using ext3-0.9.4 with linux-2.4.7 / 2.4.8-pre1 and get some hangs on
my dual P2-350:
>From time to time I will have multiple CRON-Daemons in D-state and login
hangs when logging in. It even happens during boot before my MTA is
started.

I have a single ext3 partition which is exported by kernel-nfs-server.

As soon as I do an Alt-SysRq-S forced sync the hang goes away and
everything works normal.

If you need further information send me an eMail. SGIs kdb is already
compiled in so if we need it ...

BYtE
Philipp
-- 
  / /  (_)__  __ ____  __ Philipp Hahn
 / /__/ / _ \/ // /\ \/ /
/____/_/_//_/\_,_/ /_/\_\ pmhahn@titan.lahn.de

Re: ext3-2.4-0.9.4

[Stephen C. Tweedie]

Hi,

On Mon, Jul 30, 2001 at 08:37:07AM +0200, Philipp Matthias Hahn wrote:
> On Thu, 26 Jul 2001, Andrew Morton wrote:
> 
> > An update to the ext3 filesystem for 2.4 kernels is available at
> >
> > 	http://www.uow.edu.au/~andrewm/linux/ext3/
> I'm using ext3-0.9.4 with linux-2.4.7 / 2.4.8-pre1 and get some hangs on
> my dual P2-350:
> >From time to time I will have multiple CRON-Daemons in D-state and login
> hangs when logging in. It even happens during boot before my MTA is
> started.

Interesting.  Do you have the ability to hook up a serial console?  If
so, "alt-sysrq-T" to capture a backtrace of all the blocked processes
would be a great help.  Thanks.

Cheers,
 Stephen

Re: GPS Leap Second Scheduled!

[Jan Echternach]

On Tue, Sep 15, 1998 at 09:45:28PM -0400, David Feuer wrote:
> In my opinion, the thing to do is make the POSIX time function do
> something fairly reasonable, but come up with something better (e.g. UTC
> and TAI in 64 or 128-bit fixed-point format) for newer programs.

There is a similar discussion going on in comp.std.c (Subject: C9X FCD -
7.23.2.6 Normalization of broken-down times).

See also http://www.cl.cam.ac.uk/~mgk25/c-time/ for a proposal for a
new time API.

-- 
Jan

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/

Re: GPS Leap Second Scheduled!

[Ethan O'Connor]

Roger Wolff wrote:

>Chris Wedgwood wrote:
>> On Wed, Sep 09, 1998 at 02:13:42PM -0600, Colin Plumb wrote:
>> 
>> > - gettimeofday() never returns the same value twice (documented BSD
>> >   behaviour)
>> 
>> Ouch... gettimeofday(2) only presently has usec resolution. I suspect
>> we can make this report the same value twice on really high end boxes
>> (667MHz Alpha maybe, 400Mhz Sparcs?), if not now, in a year or so.
>> Even a P.ii 600 or so can probably manage it.

>This is defined behaviour. On processors where gettimeofday can be
>called more than once in a microsecond (SMP systems, and fast
>systems), the kernel is required to keep a last-time-returned, and
>increment it and return that if the value calculated is below the
>stored value.

>If you have the results from two gettimeofday calls, you can always
>subtract them and divide by the result without checking for zero.
>That's what the spec says.

>A kernel will get into trouble if you keep on calling gettimeofday
>more than a million times a second..... 


Modifying Chris's code to call gettimeofday() 4 times
the second time around and to print the usec values 
for the four successive calls yields the following on
this system (SunOS 5.6, Ultra-IIi/333Mhz):

athena% a.out
213426
213426
213426
213427
athena% a.out
499126
499126
499127
499127

and even:

athena% a.out
947875
947875
947875
947875

Etc... 

FWIW.

-Ethan O'Connor
_________________________________________________________
DO YOU YAHOO!?
Get your free @yahoo.com address at http://mail.yahoo.com


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Colin Plumb]

> Hence, the above equation would have the same value for 23:59:60 and
> 00:00:00 on the next day.  Hence, time() should return the same value.

The problem becomes more acute for gettimeofday() and the POSIX
ns-resolution equivalent, clock_gettime(CLOCK_REALTIME).

There are a few axioms that you'd like to have hold:

- time() and gettimeofday() return the same time as
  clock_gettime(CLOCK_REALTIME), as far as precision permits.
- gettimeofday() never returns the same value twice (documented BSD behaviour)
- no clock ever runs backwards

This makes the handling of the tv_usec or tv_nsec parts of the time during
a leap second a bit problematic.

One option is to count twice.
One option is to count at half speed during 23:59:60 and 0:00:00.
One option is to use the curve y = 1/4*x^3 - 3/4*x^2 + x to interpolate
y smoothly from 0 to 1 as x goes from 0 to 2.
One option is to freeze the fractional part fr one of the two seconds.
One (pretty rude) option is to count tv_usec from -1000000 to -1
during 23:59:60.  (tv_usec is a *signed* long.)

I don't know of a pretty way.  What I'd like, as I said, is a defined kludge,
so that there is a defined mapping between struct timeval and struct timespec
and UTC in the vicinity of a leap second.  This ensures that timestamps
collected on different machines can at least be compared, even if the
time intervals reported are incorrect.  ("Why were my ping times
half of usual at midnight last night?")
-- 
	-Colin

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Chris Wedgwood]

On Wed, Sep 09, 1998 at 02:13:42PM -0600, Colin Plumb wrote:

> - gettimeofday() never returns the same value twice (documented BSD
>   behaviour)

Ouch... gettimeofday(2) only presently has usec resolution. I suspect
we can make this report the same value twice on really high end boxes
(667MHz Alpha maybe, 400Mhz Sparcs?), if not now, in a year or so.
Even a P.ii 600 or so can probably manage it.

Sure... this is fixable and not hard to fix, but it requires breaking
binary compatibility.

The attached code on a PPro 200 gives me results of 2 usecs, using a
P.II 300 and SYSENTER semantics, you can probably get this to less
that 1usec.

> I don't know of a pretty way.  What I'd like, as I said, is a
> defined kludge, so that there is a defined mapping between struct
> timeval and struct timespec and UTC in the vicinity of a leap
> second.

I'm not sure about a sane kludge for mapping to/from the semantics we
already have, but how about we just declare the existing API buggy
under some (rare circumstances) and create a new one with flags to
show whether or not a leap second is currently under way, much the
same as is done for daylight savings time with time zones. (You can
have two 2:30ams, only one of which tough is DST, the other is
non-DST).



-Chris

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Chris Wedgwood]

[-- Attachment #1: Type: text/plain, Size: 247 bytes --]

On Thu, Sep 10, 1998 at 11:45:15AM +1200, Chris Wedgwood wrote:

> The attached code on a PPro 200 gives me results of 2 usecs, using a
> P.II 300 and SYSENTER semantics, you can probably get this to less
> that 1usec.

Crap... here it is.



-cw

[-- Attachment #2: tv-test.c --]
[-- Type: text/plain, Size: 438 bytes --]

#include <sys/time.h>
#include <unistd.h>
#include <stdio.h>

int	main()
{
	struct timeval	t1,t2,t3,t4;
	
	do{
		/* warm caches */
		gettimeofday(&t1,NULL);
		gettimeofday(&t2,NULL);
		gettimeofday(&t3,NULL);
		gettimeofday(&t4,NULL);

		gettimeofday(&t3,NULL);
		gettimeofday(&t4,NULL);
	}while(t3.tv_sec != t4.tv_sec); /* yuk */

	printf("%d usecs difference!\n",(int)(t4.tv_usec - t3.tv_usec));
	
	
	return	0;
}
                       

Re: GPS Leap Second Scheduled!

[Rogier Wolff]

Chris Wedgwood wrote:
> On Wed, Sep 09, 1998 at 02:13:42PM -0600, Colin Plumb wrote:
> 
> > - gettimeofday() never returns the same value twice (documented BSD
> >   behaviour)
> 
> Ouch... gettimeofday(2) only presently has usec resolution. I suspect
> we can make this report the same value twice on really high end boxes
> (667MHz Alpha maybe, 400Mhz Sparcs?), if not now, in a year or so.
> Even a P.ii 600 or so can probably manage it.

This is defined behaviour. On processors where gettimeofday can be
called more than once in a microsecond (SMP systems, and fast
systems), the kernel is required to keep a last-time-returned, and
increment it and return that if the value calculated is below the
stored value.

If you have the results from two gettimeofday calls, you can always
subtract them and divide by the result without checking for zero.
That's what the spec says.

A kernel will get into trouble if you keep on calling gettimeofday
more than a million times a second..... 

					Roger. 

-- 
| The secret of success is sincerity. Once you can |R.E.Wolff@BitWizard.nl 
| fake that, you've got it made. -- Jean Giraudoux | phone: +31-15-2137555 
We write Linux device drivers for any device you may have! fax: ..-2138217

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Oliver Xymoron]

On Thu, 10 Sep 1998, Rogier Wolff wrote:

> Chris Wedgwood wrote:
> > On Wed, Sep 09, 1998 at 02:13:42PM -0600, Colin Plumb wrote:
> > 
> > > - gettimeofday() never returns the same value twice (documented BSD
> > >   behaviour)
> > 
> > Ouch... gettimeofday(2) only presently has usec resolution. I suspect
> > we can make this report the same value twice on really high end boxes
> > (667MHz Alpha maybe, 400Mhz Sparcs?), if not now, in a year or so.
> > Even a P.ii 600 or so can probably manage it.
> 
> This is defined behaviour. On processors where gettimeofday can be
> called more than once in a microsecond (SMP systems, and fast
> systems), the kernel is required to keep a last-time-returned, and
> increment it and return that if the value calculated is below the
> stored value.

Seems wrong to bother the kernel with this at all. Any complexity here
should be put in libc. After all, the problem is largely with the ANSI C
spec.

Also, putting any of this 'hide the leap second' logic in the kernel makes
it more difficult to later add a consistent interface to libc as the
information is now hidden in the kernel. 

Finally, telling the kernel to schedule a leap second seems pretty ugly as
well.

--
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.." 


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Ryan Moore]

On Thu, 10 Sep 1998, Oliver Xymoron wrote:
> Finally, telling the kernel to schedule a leap second seems pretty ugly as
> well.

Except that it's already doing it.  At least it is if you're running ntpd
or xntpd.  I've seen the Linux kernel do leap seconds on my machine
before.

In 2.0.35 at least, the code is in /usr/src/linux/kernel/sched.c
In function second_overflow().

The kernel even prints a log message:

printk(KERN_NOTICE "Clock: inserting leap second 23:59:60 UTC\n");

in one case and...

printk(KERN_NOTICE "Clock: deleting leap second 23:59:59 UTC\n");

in another.

--------------------------------
Ryan Moore    rmoore@rmoore.i.seawood.org



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[David Lang]

-----BEGIN PGP SIGNED MESSAGE-----

I am probably missing something, but can't you just ignore the leap second
until you discover that the time is 1 sec off and then use the normal NTP
procedure to get back to the exact time

David Lang

On Tue, 8 Sep 1998, Andrej Presern wrote:

> Date: Tue, 8 Sep 1998 22:27:28 +0200
> From: Andrej Presern <andrejp@luz.fe.uni-lj.si>
> To: majordomo@vger.rutgers.edu,
     Ulrich Windl <ulrich.windl@rz.uni-regensburg.de>
> Cc: linux-kernel@vger.rutgers.edu
> Subject: Re: GPS Leap Second Scheduled!
> 
> On Mon, 07 Sep 1998, Ulrich Windl wrote:
> >(...About leap second handling in the Linux kernel...)
> >On 1 Sep 98, at 11:16, Sven Dietrich wrote:
> >> The GPS control segment has posted a leap second warning for week number
> >> 990, Day number 5, and thus it's official in GPS world,
> >> that the leap second will take place 12/31/98... 
> >> 
> >> I am going to run simulations on NTP 3 & 4 and Linux with that GPS almanac. 
> >> Will send bug fixes and patches as needed for the Trimble refclocks and
> >> NTP...
> >> 
> >> Linux currently inserts a 2nd 59th second, instead of the UTC model of ...
> >> 58, 59, 60...
> >
> >The time in the kernel is seconds since the epoch. To insert a second 
> >means that we'll have to delay the next second for another second. 
> >The other solution seems to be a clib -> kernel interface that knows 
> >that a leap second is active now. Then the clib could possibly 
> >convert the seconds to xx:yy:60. (I hope I did not overlook something 
> >obvious).
> 
> Have you considered simply not scheduling any processes for one second and
> adjusting the time accordingly? (if one second chunk is too big, you can do it
> in several steps)
> 
> Andrej
> 
> --
> Andrej Presern, andrejp@luz.fe.uni-lj.si
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.rutgers.edu
> Please read the FAQ at http://www.tux.org/lkml/faq.html
> 

-----BEGIN PGP SIGNATURE-----
Version: PGP for Personal Privacy 5.0
Charset: noconv

iQEVAwUBNfaucj7msCGEppcbAQGFkQf/WSuUbkLPd2sfQ99FgjoOcRox47ZUV6rr
c6Yj6rfIhMXpCqzRcqyNa7JS3mTK+8Z3/9Pu1WbEU/MdZrgvvorCyq9k795V8tr6
GX2EqEoVt87uhAiBRdzMi9XREsWi8hTK/yr0RRAQKhP/bLcGvjcrmDvlUR6fEBic
jo026QXPYN4mIhyf6kIyXBmrh9dOr9EFvYYz7Lr13nwa9NjA5tlOOoSDDmG8tobz
hmgRcSE93OJgoVDwOwUcgC2thH6x0MBMYCfp/c2TWQ9cYaI4dVTlEeJWKjQTTp2I
hDsW/RjLWycR5k7PnRV002Lm6+Dxxl/ajwINgpm0S52qDJleAo+eUw==
=pxxv
-----END PGP SIGNATURE-----


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Chris Wedgwood]

On Wed, Sep 09, 1998 at 12:59:47AM +0000, H. Peter Anvin wrote:

> The way xntp deals with leap seconds is it lets the epoch float...
> i.e. it holds time_t to the same value for two seconds.

Cool... so 1970 becomes even longer ago that I would have assumed
then?



-cw



-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[H. Peter Anvin]

Followup to:  <199809092149.RAA06993@hilfy.ece.cmu.edu>
By author:    "Brandon S. Allbery KF8NH" <allbery@kf8nh.apk.net>
In newsgroup: linux.dev.kernel
>
> In message <19980909200032.B13292@caffeine.ix.net.nz>, Chris Wedgwood writes:
> +-----
> | On Wed, Sep 09, 1998 at 12:59:47AM +0000, H. Peter Anvin wrote:
> | > The way xntp deals with leap seconds is it lets the epoch float...
> | > i.e. it holds time_t to the same value for two seconds.
> | Cool... so 1970 becomes even longer ago that I would have assumed
> | then?
> +--->8
> 
> That's the only semi-reasonable (I stress *semi-*) interpretation one can 
> give to the current standard.  The other possibilities are worse.
> 

Right.  I think the right solution is one I suggested on c.o.l.d.s
recently:

- time_t being a 64-bit signed integer linked to UTC
- struct timespec (and struct timeval, presumably) having an extra
  field added:

	64-bit seconds field (same as time_t) linked to UTC
	32-bit nanosecond field (microsecond for timeval) linked to TAI
	32-bit integral TAI-UTC difference

That way any moment in time will be uniquely derivable at least, at a
positive leap second, you will see the progression (where T means a
time_t value divisible by 86400):

	tv_sec		tv_delta	UTC

	T-3		N		23:59:57
	T-2		N		23:59:58
	T-1		N		23:59:59
	T-1		N+1		23:59:60
	T		N+1		00:00:00
	T+1		N+1		00:00:01
	T+2		N+1		00:00:02

... and for a negative leap second ...

	tv_sec		tv_delta

	T-3		N		23:59:57
	T-2		N		23:59:58
	T		N-1		00:00:00
	T+1		N-1		00:00:01
	T+2		N-1		00:00:02

We may want to have a testable bit that we're "in" a leap second for
the 23:59:60 case...

	-hpa

-- 
    PGP: 2047/2A960705 BA 03 D3 2C 14 A8 A8 BD  1E DF FE 69 EE 35 BD 74
    See http://www.zytor.com/~hpa/ for web page and full PGP public key
        I am Bahá'í -- ask me about it or see http://www.bahai.org/
   "To love another person is to see the face of God." -- Les Misérables

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Theodore Y. Ts'o]

   From: hpa@transmeta.com (H. Peter Anvin)
   Date: 	10 Sep 1998 01:37:05 GMT
   Right.  I think the right solution is one I suggested on c.o.l.d.s
   recently:

   - time_t being a 64-bit signed integer linked to UTC
   - struct timespec (and struct timeval, presumably) having an extra
     field added:

	   64-bit seconds field (same as time_t) linked to UTC
	   32-bit nanosecond field (microsecond for timeval) linked to TAI
	   32-bit integral TAI-UTC difference

This works, although it would require making a glibc interface change.
But if the ELF symbol versioning works out, perhaps this won't be as big
of an issue.

However, in order to in sync with POSIX.1's definition of how time_t relates to
UTC time, I'd suggest that we make the progression go as follows:

... making the progress go as follows:

	tv_sec		tv_delta	UTC
	   T-1		N		23:59:59
	   T		N		23:59:60
	   T		N+1		00:00:00
	   T+1		N+1		00:00:01

(that is, repeat tv_sec == T instead of tv_sec == T-1).

						- Ted

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Feuer]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=x-user-defined, Size: 3000 bytes --]

 
H. Peter Anvin wrote:

> Followup to:  <199809092149.RAA06993@hilfy.ece.cmu.edu>
> By author:    "Brandon S. Allbery KF8NH" <allbery@kf8nh.apk.net>
> In newsgroup: linux.dev.kernel
> >
> > In message <19980909200032.B13292@caffeine.ix.net.nz>, Chris Wedgwood writes:
> > +-----
> > | On Wed, Sep 09, 1998 at 12:59:47AM +0000, H. Peter Anvin wrote:
> > | > The way xntp deals with leap seconds is it lets the epoch float...
> > | > i.e. it holds time_t to the same value for two seconds.
> > | Cool... so 1970 becomes even longer ago that I would have assumed
> > | then?
> > +--->8
> >
> > That's the only semi-reasonable (I stress *semi-*) interpretation one can
> > give to the current standard.  The other possibilities are worse.
> >
>
> Right.  I think the right solution is one I suggested on c.o.l.d.s
> recently:
>
> - time_t being a 64-bit signed integer linked to UTC
> - struct timespec (and struct timeval, presumably) having an extra
>   field added:
>
>         64-bit seconds field (same as time_t) linked to UTC
>         32-bit nanosecond field (microsecond for timeval) linked to TAI
>         32-bit integral TAI-UTC difference
>
> That way any moment in time will be uniquely derivable at least, at a
> positive leap second, you will see the progression (where T means a
> time_t value divisible by 86400):
>  

Why would you have the TAI-UTC difference instead of just TAI?
The part of your message below was very confusing to me.
 
 

>  
>         tv_sec          tv_delta        UTC
>
>         T-3             N               23:59:57
>         T-2             N               23:59:58
>         T-1             N               23:59:59
>         T-1             N+1             23:59:60
>         T               N+1             00:00:00
>         T+1             N+1             00:00:01
>         T+2             N+1             00:00:02
>
> ... and for a negative leap second ...
>
>         tv_sec          tv_delta
>
>         T-3             N               23:59:57
>         T-2             N               23:59:58
>         T               N-1             00:00:00
>         T+1             N-1             00:00:01
>         T+2             N-1             00:00:02
>
> We may want to have a testable bit that we're "in" a leap second for
> the 23:59:60 case...
>
>         -hpa
>
> --
>     PGP: 2047/2A960705 BA 03 D3 2C 14 A8 A8 BD  1E DF FE 69 EE 35 BD 74
>     See http://www.zytor.com/~hpa/ for web page and full PGP public key
>         I am Bahá'í -- ask me about it or see http://www.bahai.org/
>    "To love another person is to see the face of God." -- Les Misérables
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.rutgers.edu
> Please read the FAQ at http://www.tux.org/lkml/faq.html


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Colin Plumb]

Ulrich Windl wrote:
> The time in the kernel is seconds since the epoch. To insert a second 
> means that we'll have to delay the next second for another second. 
> The other solution seems to be a clib -> kernel interface that knows 
> that a leap second is active now. Then the clib could possibly 
> convert the seconds to xx:yy:60. (I hope I did not overlook something 
> obvious).

And Andrej Presern replied:
>> Have you considered simply not scheduling any processes for one second and
>> adjusting the time accordingly? (if one second chunk is too big, you can do
>> it in several steps)

The problem is that POSIX is schizophrenic on the subject of leap seconds.
On the one hand, time() returns UTC time.  On the other,

	t = time();
	sec = t % 60;
	t /= 60;
	min = t % 60;
	t /= 60;
	hr = t % 24;
	t /= 24;
	printf("UTC time is %lu days, %02u:%02u:%02u\n", (unsigned long)t, gr, min, sec);

is required to work (since so much code does it.)

Actually, I think Ulrich was present when I proposed a similar solution:
gettimeofday() will not return during 23:59:60.  If a process calls
gettimeofday() during a leap second, then the call will sleep until 0:00:00
when it can return the correct result.

This horrified the real-time people.  It is, however, strictly speaking,
completely correct.

The real-world solution (a.k.a kludge), is to stretch some number of
seconds to cover the n+1 seconds including the leap second.  During
those seconds, gettimeofday() is incorrect, but the error is
well-defined, so two "good" implementations will return "close" values
and you can undo the fudging afterwards if desired.

The only trick is to define the number of seconds n, their position
relative to the leap second, and they type of stretching.  Linear is
obvious, making the 60 seconds leading up to a leap second take 61/60
of the usual time, but you could define a polynomial with higher-order
continuity.
-- 
	-Colin

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html

Re: GPS Leap Second Scheduled!

[Theodore Y. Ts'o]

   Date: 	Tue, 8 Sep 1998 22:46:25 -0600 (MDT)
   From: Colin Plumb <colin@nyx.net>

   The problem is that POSIX is schizophrenic on the subject of leap seconds.
   On the one hand, time() returns UTC time.  

Yep.  See POSIX 4.5.1.2, 2.2.2.77, 2.2.2.24, and the rationale for
Epoch, found in B.2.2.2).

    On the other,

	   t = time();
	   sec = t % 60;
	   t /= 60;
	   min = t % 60;
	   t /= 60;
	   hr = t % 24;
	   t /= 24;
	   printf("UTC time is %lu days, %02u:%02u:%02u\n", (unsigned long)t, gr, min, sec);

   is required to work (since so much code does it.)

In fact, POSIX requires this (see 2.2.2.77).

   Actually, I think Ulrich was present when I proposed a similar solution:
   gettimeofday() will not return during 23:59:60.  If a process calls
   gettimeofday() during a leap second, then the call will sleep until 0:00:00
   when it can return the correct result.

The other possibility is for gettimeofday() to return the same value for
23:59:60 and 00:00:00.  This would also strictly speaking be correct.
time() is specified as returning "seconds since the Epoch", which is
defined as:

	tm_sec + tm_min*60 + tm_hour*3600 + tm_yday*86400 +
		(tm_year-70)*31536000 + ((tm_year-64)/4*86400)

where tm_sec, tm_min, tm_hour, etc. together form a Curridnated
Universal Time name.

Hence, the above equation would have the same value for 23:59:60 and
00:00:00 on the next day.  Hence, time() should return the same value.

						- Ted

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.rutgers.edu
Please read the FAQ at http://www.tux.org/lkml/faq.html