Re: What's left over.

Re: What's left over.

[Andi Kleen]

Rusty Russell <rusty@rustcorp.com.au> writes:


> > > statfs64
> > 
> > I haven't even seen it.
> 
> It's fairly old, but Peter Chubb said there was some vendor interest
> for v. large devices.  Peter?

statfs64 is needed when you want to access large NFS servers (>2TB is 
becomming quite common for NAS) and want to have working "df" for them.

Currently it is scaled by wsize==blocksize, so it only breaks when
fileserversize/wsize > 2^31. For 1KB wsize it breaks with 2TB, with
4KB with 8TB etc. While 1KB wsize is arguably stupid (but happens sometimes
in practice). 8TB is not an unrealistic size for an NFS server these 
days.

I did an hack to scale the NFS block size in stat to make sure it fits
into 31bit, but statfs64 would be the correct solution for it really.

Also I would like to propose the nanosecond stat patches. It doesn't add
new system calls, but just uses spare fields in the existing stat64 
structure and closes a hole in make.

-Andi

Re: What's left over.

[Rusty Russell]

In message <p73hef2sepq.fsf@oldwotan.suse.de> you write:
> I did an hack to scale the NFS block size in stat to make sure it fits
> into 31bit, but statfs64 would be the correct solution for it really.

AFAICT the patches are not in shape at the moment, so I don't think it
fits "actively being pushed": unless someone chimes in, I'm removing
it.

> Also I would like to propose the nanosecond stat patches. It doesn't add
> new system calls, but just uses spare fields in the existing stat64 
> structure and closes a hole in make.

OK, I've added this one: sorry for missing it.  You might want to
split this into "core" and then updated the filesystems via their
maintainers during the freeze though: it's one *big* patch as it
stands.

Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: What's left over.

[Florian Weimer]

Oliver Xymoron <oxymoron@waste.org> writes:

> - /tmp-style symlink issues on shared directories
> - vast majority of software (including security tools) ACL-unaware
> - much harder to check for correctness

 - surprising inheritance of of the ACL of the directory

This is a known problem in NTFS land, and some people suggest that
per-directory ACLs are enough for everyone for exactly this reason.

Re: What's left over.

[yodaiken]

On Tue, Nov 05, 2002 at 12:09:17PM -0500, Bill Davidsen wrote:
> On Sun, 3 Nov 2002 yodaiken@fsmlabs.com wrote:
> 
> > On Sun, Nov 03, 2002 at 08:48:30AM -0500, Bill Davidsen wrote:
> > > Quite clearly SCO, Sun, and IBM have been doing this for years without
> > > offering dozens of options. I don't need it to sing and dance, I just need
> > > a way to put the dump where I can find it. I'm not going to put another
> > > box in at the end of a serial or parallel port, I don't have NVram, I do
> > > have lopts of disk, and so does almost everyone else. I have remote
> > > systems in wiring closets all over the country (all four time zones). They
> > > are at the end of open net connections, unreliable and untrusted. I don't
> > > want to bet that I have a working VPN, or that I can safely send all that
> > > data without it being read by someone other than me.
> > > 
> > > The AIX support has a group just to beat on dumps customers send. What
> > > more evidence is needed that people can and do use the capability.
> 
> > You paid someone for this for AIX. So the solution is obvious for Linux.
> 
> No, it's included in AIX, SCO and Solaris. And analysis is included in

None of those are free.

> support contracts. With all the stuff added to Linux to keep up with both
> M$ and commercial UNIX, I can't imagine why anyone would be against this.
> At least anyone who wanted Linux to compete in the commercial server
> market.

So buy your Linux from a vendor who supports it. 

Re: What's left over.

[Bill Davidsen]

On Sun, 3 Nov 2002 yodaiken@fsmlabs.com wrote:

> On Sun, Nov 03, 2002 at 08:48:30AM -0500, Bill Davidsen wrote:
> > Quite clearly SCO, Sun, and IBM have been doing this for years without
> > offering dozens of options. I don't need it to sing and dance, I just need
> > a way to put the dump where I can find it. I'm not going to put another
> > box in at the end of a serial or parallel port, I don't have NVram, I do
> > have lopts of disk, and so does almost everyone else. I have remote
> > systems in wiring closets all over the country (all four time zones). They
> > are at the end of open net connections, unreliable and untrusted. I don't
> > want to bet that I have a working VPN, or that I can safely send all that
> > data without it being read by someone other than me.
> > 
> > The AIX support has a group just to beat on dumps customers send. What
> > more evidence is needed that people can and do use the capability.

> You paid someone for this for AIX. So the solution is obvious for Linux.

No, it's included in AIX, SCO and Solaris. And analysis is included in
support contracts. With all the stuff added to Linux to keep up with both
M$ and commercial UNIX, I can't imagine why anyone would be against this.
At least anyone who wanted Linux to compete in the commercial server
market.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: What's left over.

[Andreas Gruenbacher]

On Friday 01 November 2002 01:54, john stultz wrote:
> I probably should just go read the specs. Anyone have a pointer, or care
> to explain what the differences are between AFS's ACLs and POSIX ACLs?

POSIX 1003.1e draft 17 (withdrawn) is available at 
<http://wt.xpilot.org/publications/posix.1e/>.

--Andreas.

Re: What's left over.

[Andreas Gruenbacher]

On Thursday 31 October 2002 07:21, Chris Wedgwood wrote:
> Don't get me wrong, I'm not against sane ACLs (POSIX ACLs are not) or
> EAs [...]

POSIX ACLs are more complicated than what would be inherently necessary, if we 
were in a situation where we could design from scratch. Unfortunately we are 
not in that situation. I've heard dozens of people complain about POSIX ACLs 
(and other kinds as well); nobody was able to come up with something truly 
better so far.

--Andreas.

Re: What's left over.

[Hugh Dickins]

On Sat, 2 Nov 2002, Horst von Brand wrote:
> Hugh Dickins <hugh@veritas.com> said:
> 
> > It's a real worry that writing a crash dump to disk might stomp in the
> > wrong place, but I don't recall it ever happening in practice.  But
> > occasionally, yes, a dump was not generated at all, or not completed.
> 
> How do you test that? Not in some contrieved situation, under real crashes.

Sorry for being unclear: by "in practice" I meant "under real crashes" i.e.
I was referring more to what we heard back from users than my own testing.

Hugh

Re: What's left over.

[Patrick Finnegan]

First I want to apologize to anyone I've pissed off too badly with this.
Another note - I have no relation to the LKCD developers, other than a
very satisfied, and sometimes excessivly vehement, user. I was about to
respond to this message in detail, but I dont need to put more Magnesium
on the flames.

Pat

On Mon, 4 Nov 2002, Rob Landley wrote:

> On Friday 01 November 2002 16:16, Patrick Finnegan wrote:
>
> > > It's not a fscking public service.  Linus has full control over his
> > > tree.  You have equally full control over your tree.  Linus can't
> > > tell you what patches to apply in your tree.  You can't tell Linus
> > > what patches he should apply to his.
> >
> > I'm sorry it _is_ a public service.  Once tens of people started
> > contributing to it, it became one.  This is like saying that the
> > Washington Monument belongs to the peole that maintain it, any building
> > belongs to the repair crews and janitors.
>
> You pay taxes to support the washington monument.  When's the last time you
> paid a tax to Linus?
>
> > I'm not saying that Linus is
> > necessarily a janitor, but when you consider how much of the Linux kernel
> > that he didn't write, you may relize that it's not just his kernel.
>
> He's the editor of a periodical publication.  A cross between an academic
> technical journal which people contribute to for professional reasons, and a
> hobbyist fanzine that people contribute to 'cause it's cool.  This is not a
> new thing, there are real-world precedents for this sort of relationship
> going back hundreds of years, to the invention of the printing press...
>
> Linus's editorial decisions are as final and unappealable as any other
> editorial decision at a magazine or newspaper.  You can publish your article
> elsewhere, and if it doesn't have the same prestige as the Harvard Law Review
> or the New England Journal of Medicine, tough.  They said no.
>
> And like ALL editors, his job isn't to write a significant portion of the
> articles in the publication, but to be a Sturgeon's Law filter throwing out
> 99% of the submissions in the slush pile, correcting the spelling and grammar
> of the remaining few, and trying to stitch them together into a coherent
> whole.
>
> Go track down somebody with a Journalism degree if you want to understand
> Linus's job.
>
> >  It
> > also belongs to every single person that has written even a single
> > line of code in it.
>
> If you get an article published in Time magazine, and you say that this gives
> you the right to print your own copies of Time and distribute them yourself,
> Time's lawyers are going to come after you.
>
> The GPL gives you the ability to do this, but it doesn't obligate the
> publication's editor to listen to you.  If next month's issue contains a huge
> rebuttal to one of your articles, calling you a boogerhead, tough.  The
> editor doesn't owe you anything as a previous contributor, and certainly
> doesn't owe you anything as someone from whom he did NOT take a submission.
>
> What Linus basically said was that if a significant number of distributions
> integrated it, he might take another look at the thing in the future.  But
> wasn't going into 2.5.
>
> Now, thanks to people pestering him beyond the Annoyance Event Horizon, he's
> got his fingers in his ears.  Congratulations.  Hopefully, he'll calm down a
> bit in a few months, but there's no guarantee.  In the mean time, the most
> productive thing to do is drop the topic and work on the Red Hat, SuSE, and
> Debian guys.  (Mandrake feeds from Red Hat, and SuSE is now making kernels
> for Connectiva and TurboLinux.  Gentoo and Slackware might be good to bug as
> well...)
>
> See if you can convince Alan Cox to pick up your patch.  That'll get you Red
> Hat, and the single largest concentration of roll-your-own kernel guys
> outside of Linus's own tree.
>
> Rob


--
Purdue Universtiy ITAP/RCS
Information Technology at Purdue
Research Computing and Storage
http://www-rcd.cc.purdue.edu

http://dilbert.com/comics/dilbert/archive/images/dilbert2040637020924.gif

Re: What's left over.

[Rob Landley]

On Monday 04 November 2002 14:58, Patrick Finnegan wrote:
> First I want to apologize to anyone I've pissed off too badly with this.

Sorry, didn't mean to bring up an old issue.  DSL was out over the weekend 
here (thank you Southwestern Bell), so some stuff queued up in my laptop's 
outbox...

Rob

-- 
http://penguicon.sf.net - Terry Pratchett, Eric Raymond, Pete Abrams, Illiad, 
CmdrTaco, liquid nitrogen ice cream, and caffienated jello.  Well why not?

Re: What's left over.

[Rob Landley]

On Friday 01 November 2002 16:16, Patrick Finnegan wrote:

> > It's not a fscking public service.  Linus has full control over his
> > tree.  You have equally full control over your tree.  Linus can't
> > tell you what patches to apply in your tree.  You can't tell Linus
> > what patches he should apply to his.
>
> I'm sorry it _is_ a public service.  Once tens of people started
> contributing to it, it became one.  This is like saying that the
> Washington Monument belongs to the peole that maintain it, any building
> belongs to the repair crews and janitors.

You pay taxes to support the washington monument.  When's the last time you 
paid a tax to Linus?

> I'm not saying that Linus is
> necessarily a janitor, but when you consider how much of the Linux kernel
> that he didn't write, you may relize that it's not just his kernel.

He's the editor of a periodical publication.  A cross between an academic 
technical journal which people contribute to for professional reasons, and a 
hobbyist fanzine that people contribute to 'cause it's cool.  This is not a 
new thing, there are real-world precedents for this sort of relationship 
going back hundreds of years, to the invention of the printing press...

Linus's editorial decisions are as final and unappealable as any other 
editorial decision at a magazine or newspaper.  You can publish your article 
elsewhere, and if it doesn't have the same prestige as the Harvard Law Review 
or the New England Journal of Medicine, tough.  They said no.

And like ALL editors, his job isn't to write a significant portion of the 
articles in the publication, but to be a Sturgeon's Law filter throwing out 
99% of the submissions in the slush pile, correcting the spelling and grammar 
of the remaining few, and trying to stitch them together into a coherent 
whole.

Go track down somebody with a Journalism degree if you want to understand 
Linus's job.

>  It
> also belongs to every single person that has written even a single
> line of code in it.

If you get an article published in Time magazine, and you say that this gives 
you the right to print your own copies of Time and distribute them yourself, 
Time's lawyers are going to come after you.

The GPL gives you the ability to do this, but it doesn't obligate the 
publication's editor to listen to you.  If next month's issue contains a huge 
rebuttal to one of your articles, calling you a boogerhead, tough.  The 
editor doesn't owe you anything as a previous contributor, and certainly 
doesn't owe you anything as someone from whom he did NOT take a submission.

What Linus basically said was that if a significant number of distributions 
integrated it, he might take another look at the thing in the future.  But 
wasn't going into 2.5.

Now, thanks to people pestering him beyond the Annoyance Event Horizon, he's 
got his fingers in his ears.  Congratulations.  Hopefully, he'll calm down a 
bit in a few months, but there's no guarantee.  In the mean time, the most 
productive thing to do is drop the topic and work on the Red Hat, SuSE, and 
Debian guys.  (Mandrake feeds from Red Hat, and SuSE is now making kernels 
for Connectiva and TurboLinux.  Gentoo and Slackware might be good to bug as 
well...)

See if you can convince Alan Cox to pick up your patch.  That'll get you Red 
Hat, and the single largest concentration of roll-your-own kernel guys 
outside of Linus's own tree.

Rob

-- 
http://penguicon.sf.net - Terry Pratchett, Eric Raymond, Pete Abrams, Illiad, 
CmdrTaco, liquid nitrogen ice cream, and caffienated jello.  Well why not?

Re: What's left over.

[yodaiken]

On Sun, Nov 03, 2002 at 08:48:30AM -0500, Bill Davidsen wrote:
> On 1 Nov 2002, Alan Cox wrote:
> 
> > On Fri, 2002-11-01 at 06:34, Bill Davidsen wrote:
> > >   From the standpoint of just the driver that's true. However, the remote
> > > machine and all the network bits between them are a string of single
> > > points of failure. Isn't it good that both disk and network can be
> > > supported.
> > 
> > My concerns are solely with things like the correctness of the disk
> > dumper. Its obviously a good way to do a lot more damage if it isnt done
> > carefully. Quite clearly your dump system wants to support multiple dump
> > targets so you can dump to pci battery backed ram, down the parallel
> > port to an analysing box etc
> 
> Quite clearly SCO, Sun, and IBM have been doing this for years without
> offering dozens of options. I don't need it to sing and dance, I just need
> a way to put the dump where I can find it. I'm not going to put another
> box in at the end of a serial or parallel port, I don't have NVram, I do
> have lopts of disk, and so does almost everyone else. I have remote
> systems in wiring closets all over the country (all four time zones). They
> are at the end of open net connections, unreliable and untrusted. I don't
> want to bet that I have a working VPN, or that I can safely send all that
> data without it being read by someone other than me.
> 
> The AIX support has a group just to beat on dumps customers send. What
> more evidence is needed that people can and do use the capability.
> 
> I had hoped that someone would do this for Linux, I never dreamed that

You paid someone for this for AIX. So the solution is obvious for Linux.

Re: What's left over.

[Bill Davidsen]

On Sat, 2 Nov 2002, Zwane Mwaikambo wrote:

> On Sat, 2 Nov 2002, Bill Davidsen wrote:
> 
> >   The thing is that Solaris, AIX, and ISC are written by commercial
> > companies, they realize that customers need to be able to debug systems
> > which don't have a screen, a serial printer, etc. They do have disk. 
> > 
> >   I was hoping Alan would push Redhat to put this in their Linux so we
> > could resolve some of the ongoing problems which don't write an oops to a
> > log, but I guess none of the developers has to actually support production
> > servers and find out why they crash.
> 
> Perhaps i'm being grossly naive here, but none of these presumably x86 
> productions servers don't have a serial port? Not even PCI/ISA slots to 
> add one? Serial would catch most of your oopsen anyway, and if you were 
> borked enough that serial couldn't get the entire output, i somehow doubt 
> dumping to disk could manage. And no i don't see anything wrong nor 
> consider it studly to use oopses only for debugging...

I have distributed servers in 15 locations, six states, four timezones. In
secure unattended locations like wiring closets. What do I do with the
serial port? Do I double my colocation costs and have another system there
to listen? Is the code on a sick system going to dial the modem on the
serial line amd establish a connection?

I have a mix of Linux, Solaris, and AIX systems deployed, and only the
Linux systems don't have this capability. Actually for the most part only
the Linux systems NEED it, that's another problem, but reliability would
go up if I could see the problem.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: What's left over.

[Bill Davidsen]

On 1 Nov 2002, Alan Cox wrote:

> On Fri, 2002-11-01 at 06:34, Bill Davidsen wrote:
> >   From the standpoint of just the driver that's true. However, the remote
> > machine and all the network bits between them are a string of single
> > points of failure. Isn't it good that both disk and network can be
> > supported.
> 
> My concerns are solely with things like the correctness of the disk
> dumper. Its obviously a good way to do a lot more damage if it isnt done
> carefully. Quite clearly your dump system wants to support multiple dump
> targets so you can dump to pci battery backed ram, down the parallel
> port to an analysing box etc

Quite clearly SCO, Sun, and IBM have been doing this for years without
offering dozens of options. I don't need it to sing and dance, I just need
a way to put the dump where I can find it. I'm not going to put another
box in at the end of a serial or parallel port, I don't have NVram, I do
have lopts of disk, and so does almost everyone else. I have remote
systems in wiring closets all over the country (all four time zones). They
are at the end of open net connections, unreliable and untrusted. I don't
want to bet that I have a working VPN, or that I can safely send all that
data without it being read by someone other than me.

The AIX support has a group just to beat on dumps customers send. What
more evidence is needed that people can and do use the capability.

I had hoped that someone would do this for Linux, I never dreamed that
it would be kept out of the kernel by people who clearly don't understand
the problems if distributed and clustered headless systems.

I guess the development folks are working on more important things like
xiafs and morse code dumps to the keyboard LEDs.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: What's left over.

[Horst von Brand]

Hugh Dickins <hugh@veritas.com> said:

[...]

> I dealt with crash dumps quite a lot over 10 years with SCO UNIX,
> OpenServer and UnixWare: which were addressing the PC market, not
> own hardware.

What I remember about hardware compatibility for SCO Unix and Solaris on
ia32 is _not_ funny. Lightyears from what Linux handles today without
breaking a sweat.

> It's a real worry that writing a crash dump to disk might stomp in the
> wrong place, but I don't recall it ever happening in practice.  But
> occasionally, yes, a dump was not generated at all, or not completed.

How do you test that? Not in some contrieved situation, under real crashes.
Don't just consider crashes in the official $DISTRIBUTION kernel, but in
Linus' BK tree, or some of the random, two-or-three-letter-trees of the day
(_that_ is where crashes happen, _that_ is where the info would be most
valuable). It gets _real_ hairy _real_ fast to make sure you don't scribble
over /home or /etc on the user's disk...

> Of course, you could argue that SCO's disk drivers were more stable :-)

If you only handle a few, thoroughly tested, high-end controllers and
disks, that is not too hard to do.
--
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: What's left over.

[Matt D. Robinson]

I'm not sure I understand your point, Horst.  There are four
primary mechanisms which would invoke a dump:

	die() (or die_if_kernel())
	panic()
	interrupt-driven dumps
	sysrq()

Assuming you call these functions, there is a single dump()
call that will perform dumping, the the dump_function_ptr
(which is assigned when the dump module is loaded) is set.
dump() is a simple function that basically says:

static inline void dump(char * str, struct pt_regs * regs)
{
	if (dump_function_ptr) {
		dump_function_ptr((char *)str, regs);
	}
}

str is for the panic() string, and regs are so you can create
a proper stack trace for the failing task on the correct CPU.

I don't see how that can can attributed to bloating the kernel.
If you don't panic(), the code is never invoked.  If you don't load
the dump module, dump_function_ptr isn't assigned.  It's meant
to be non-invasive, off to the side and called when required
(or requested).

There is some additional code put in the kernel to disable
interrupts, quiesce the system, and I think there are a few projects
that can probably use the same code base (such as the suspend-to-ram
project, which I was just informed about).  All of that is called
within the dump driver itself, otherwise it sits quietly off to
the side, never getting called.

Using the dump driver infrastructure is like writing any plain-jane
driver.  You set up the _open(), _close(), etc., functions,
assigning the ops table based on the dump method you want to use
(disk, network, mini-oopser, etc.)  This isn't that difficult,
and it should only be loaded for those customer systems that want
a specific dump style.

--Matt

Standard disclaimer:  I'm not trying anymore to get this into the
kernel at this time (via Linus).  This is purely for educating
those that aren't familiar with crash dumping for Linux.

On Sat, 2 Nov 2002, Horst von Brand wrote:
|>"Matt D. Robinson" <yakker@aparity.com> dijo:
|>
|>[...]
|>
|>> This isn't bloat.  If you want, it can be built as a module, and
|>> not as part of your kernel.  How can that be bloat?  People who
|>> build kernels can optionally build it in, but we're not asking
|>> that it be turned on by default, rather, built as a module so
|>> people can load it if they want to.  We made it into a module
|>> because 18 months ago you complained about it being bloat.  We
|>> addressed your concerns.
|>
|>Bloat is not just RAM/CPU/... usage when in use, it is much more about
|>developers who have to understand, work with, and consider how to use or
|>interface with the new code. Even more so when it is not builtin, as this
|>creates _two_ scenarios to consider.
|>
|>This is the sense of "bloat" that Linus is most worried about (and very
|>rightly so, IMVHO). At lesat that is my observation over the years.
|>

-- 

Re: What's left over.

[Horst von Brand]

"Matt D. Robinson" <yakker@aparity.com> dijo:

[...]

> This isn't bloat.  If you want, it can be built as a module, and
> not as part of your kernel.  How can that be bloat?  People who
> build kernels can optionally build it in, but we're not asking
> that it be turned on by default, rather, built as a module so
> people can load it if they want to.  We made it into a module
> because 18 months ago you complained about it being bloat.  We
> addressed your concerns.

Bloat is not just RAM/CPU/... usage when in use, it is much more about
developers who have to understand, work with, and consider how to use or
interface with the new code. Even more so when it is not builtin, as this
creates _two_ scenarios to consider.

This is the sense of "bloat" that Linus is most worried about (and very
rightly so, IMVHO). At lesat that is my observation over the years.
-- 
Dr. Horst H. von Brand                   User #22616 counter.li.org
Departamento de Informatica                     Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria              +56 32 654239
Casilla 110-V, Valparaiso, Chile                Fax:  +56 32 797513

Re: What's left over.

[Chris Wedgwood]

On Sat, Nov 02, 2002 at 09:35:17AM -0800, LA Walsh wrote:

> Then why do we need 'non-repudiation' w/r/t certificates?

we dont


  --cw

Re: What's left over.

[Alan Cox]

On Sat, 2002-11-02 at 18:55, Arnaldo Carvalho de Melo wrote:
> SPX was also removed (hey, it never worked anyway) and probably econet and
> ATM will be removed as well if nobody jumps to fix it (I mean net/atm, not
> drivers/atm, but I'm not sure the later will be useful without the former).

ATM is actively used by large numbers of people [1]. Its in the fix
rather than remove category. Econet should be trivial and might as well
just be marked CONFIG_OBSOLETE until someone does deal with it.

Alan
[1] PPPoATM is used for a large number of DSL connections

Re: What's left over.

[romieu]

Arnaldo Carvalho de Melo <acme@conectiva.com.br> :
[...]
> :-) I think that if you state that you plan to work on it RSN we can forget
> about removing it for now.

$*@#&%@ !

Will have to setup a burnproof testbed for ATM then.

--
Ueimor

Re: What's left over.

[Arnaldo Carvalho de Melo]

Em Sat, Nov 02, 2002 at 08:31:29PM +0000, Alan Cox escreveu:
> On Sat, 2002-11-02 at 18:55, Arnaldo Carvalho de Melo wrote:
> > SPX was also removed (hey, it never worked anyway) and probably econet and
> > ATM will be removed as well if nobody jumps to fix it (I mean net/atm, not
> > drivers/atm, but I'm not sure the later will be useful without the former).
 
> ATM is actively used by large numbers of people [1]. Its in the fix
> rather than remove category. Econet should be trivial and might as well
> just be marked CONFIG_OBSOLETE until someone does deal with it.

Oh, cool, way more motivation to fix that stuff 8)

Re: What's left over.

[Arnaldo Carvalho de Melo]

Em Sat, Nov 02, 2002 at 08:32:23PM +0100, romieu@fr.zoreil.com escreveu:
> Arnaldo Carvalho de Melo <acme@conectiva.com.br> :
> > Em Sat, Nov 02, 2002 at 08:19:17PM +0100, romieu@fr.zoreil.com escreveu:
> [...]
> > > What's the deadline ?
> > 
> > Plan was for 2.6.0
> 
> :o)
> Is there a lower bound for it's estimate arrival date ?

:-) I think that if you state that you plan to work on it RSN we can forget
about removing it for now.

- Arnaldo

Re: What's left over.

[romieu]

Arnaldo Carvalho de Melo <acme@conectiva.com.br> :
> Em Sat, Nov 02, 2002 at 08:19:17PM +0100, romieu@fr.zoreil.com escreveu:
[...]
> > What's the deadline ?
> 
> Plan was for 2.6.0

:o)
Is there a lower bound for it's estimate arrival date ?

--
Ueimor

Re: What's left over.

[Arnaldo Carvalho de Melo]

Em Sat, Nov 02, 2002 at 08:19:17PM +0100, romieu@fr.zoreil.com escreveu:
> > SPX was also removed (hey, it never worked anyway) and probably econet and
> > ATM will be removed as well if nobody jumps to fix it (I mean net/atm, not
> > drivers/atm, but I'm not sure the later will be useful without the former).
> 
> What's the deadline ?

Plan was for 2.6.0

- Arnaldo

Re: What's left over.

[romieu]

[Cc: changed]

Arnaldo Carvalho de Melo <acme@conectiva.com.br> :
> Em Sat, Nov 02, 2002 at 12:00:18AM -0500, Bill Davidsen escreveu:
[...]
> > Good point, that continues to disprove the theory that having one thing in
> > the kernel prevents development of a similar feature.
>
> SPX was also removed (hey, it never worked anyway) and probably econet and
> ATM will be removed as well if nobody jumps to fix it (I mean net/atm, not
> drivers/atm, but I'm not sure the later will be useful without the former).

What's the deadline ?

--
Ueimor

Re: What's left over.

[Arnaldo Carvalho de Melo]

Em Sat, Nov 02, 2002 at 12:00:18AM -0500, Bill Davidsen escreveu:
> On 1 Nov 2002, Alan Cox wrote:
> 
> > On Fri, 2002-11-01 at 06:36, Linus Torvalds wrote:
> > > This never works. Be honest. Nobody takes out features, they are stuck 
> > > once they get in. 
> > 
> > Linus I've asked a couple of times about killing sound/oss off now ALSA
> > is integrated 8) While you are on the rant how about that ;)
> 
> Good point, that continues to disprove the theory that having one thing in
> the kernel prevents development of a similar feature.

SPX was also removed (hey, it never worked anyway) and probably econet and
ATM will be removed as well if nobody jumps to fix it (I mean net/atm, not
drivers/atm, but I'm not sure the later will be useful without the former).

- Arnaldo

Re: What's left over.

[Geert Uytterhoeven]

On Fri, 1 Nov 2002, Hugh Dickins wrote:
> I dealt with crash dumps quite a lot over 10 years with SCO UNIX,
> OpenServer and UnixWare: which were addressing the PC market, not
> own hardware.
> 
> It's a real worry that writing a crash dump to disk might stomp in the
> wrong place, but I don't recall it ever happening in practice.  But
> occasionally, yes, a dump was not generated at all, or not completed.

IIRC, some years ago wuarchive.wustl.edu went down for a few days because the
machine paniced and dumped to the wrong partition...

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

RE: What's left over.

[LA Walsh]

	Then why do we need 'non-repudiation' w/r/t certificates?  Isn't
the
idea to provide a way to isolate "bugs" in the "security" system.  If
something
is written to a file by the group signon, who wrote it?

> -----Original Message-----
> From: linux-kernel-owner@vger.kernel.org 
> [mailto:linux-kernel-owner@vger.kernel.org] On Behalf Of 
> Alexander Viro
> Sent: Wednesday, October 30, 2002 11:43 PM
> Then give them all the same account and be done with that.  
> Effect will
> be the same.

Re: What's left over.

[Alan Cox]

On Sat, 2002-11-02 at 05:00, Bill Davidsen wrote:
> > Linus I've asked a couple of times about killing sound/oss off now ALSA
> > is integrated 8) While you are on the rant how about that ;)
> 
> Good point, that continues to disprove the theory that having one thing in
> the kernel prevents development of a similar feature.

Its preventing testing and its making parallel fixing hard to manage.
I'd really like to kill off the OSS drivers to make sure the ALSA ones
are tested and anything only in OSS does get ported over,

Re: What's left over.

[Alan Cox]

On Sat, 2002-11-02 at 05:17, Bill Davidsen wrote:
>   I was hoping Alan would push Redhat to put this in their Linux so we
> could resolve some of the ongoing problems which don't write an oops to a
> log, but I guess none of the developers has to actually support production
> servers and find out why they crash.

I think several Red Hat people would disagree very strongly. Red Hat
shipped with the kernel symbol decoding oops reporter for a good reason,
and also acquired netdump for a good reason. 

Re: What's left over.

[Michael Shuey]

On Fri, Nov 01, 2002 at 11:25:04PM +0100, Pavel Machek wrote:
> > Wouldn't you rather they neatly tftp'd dumps to a nominated central
> > server which noticed the arrival, did the initial processing with a perl
> > script and mailed you a summary ?
> 
> Out of interest, how does such "initial processing" look like?

Toss an email to root and the operations staff including the name of the
machine that crashed and the output of lcrash's "report" command, as well
as the location of the dumps (ie, where they were saved on the machine that
died and where they are on an optional netdump server).

-- 
Mike Shuey

Re: What's left over.

[Brad Hards]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 1 Nov 2002 13:01, Matt D. Robinson wrote:
<snip>
> Uh ... have you read the patches?  Do you see how few the
> changes are to non-dump code?  Do you know that most of those
> changes only get triggered in a crash situation anyway?
I applied the patches, and reported some issues.
http://marc.theaimsgroup.com/?l=linux-kernel&m=103520434201014&w=2
I see no signs that any of them have been addressed, although I haven't tried 
a really recent set.

> Breakage occurs when people change code areas that are used
> all the time, like VM, network, block layer, etc.
Actually, this is the area that Linux is best at. If you break it, some poor 
sod will hit the problem, and you'll know really soon.

> Look at the patches and tell me where we are causing overhead
> and and seriously potential breakage.  If you find problems,
> then tell us, don't just comment on breakage scenarios.

I'm a fairly typical user - I just have a couple of desktop machines and a 
server/firewall. 

I don't have 700 nodes in a cluster, and when my machines break, its normally 
something I did. Sometimes the desktop locks up (say every second month, 
unless I'm dicking with the kernel), but I reboot and everything is happy.

LKCD doesn't really seem to do anything for me - it wouldn't really worry me 
if it went in (since I don't have to maintain it - it isn't near any of my 
code), but I'd really prefer that having the _CONFIG option set to N didn't 
make the kernel any bigger, or change any code paths.

Is this unreasonable?

Brad

BTW: I admit that I'd be pretty pissed if Linus said that my code was 
"stupid", but life isn't reasonable or fair. Take a few days off LKCD, go for 
a few walks, and worry about how to get it integrated after that.


- -- 
http://linux.conf.au. 22-25Jan2003. Perth, Aust. I'm registered. Are you?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE9w6rCW6pHgIdAuOMRAlI5AJ48ELVdExIeCr5C5HtDpU5+1ZnuBQCdEji0
t4q2NjZQVGEumrz6b+CqEEs=
=xtYY
-----END PGP SIGNATURE-----

Re: What's left over.

[Zwane Mwaikambo]

On Sat, 2 Nov 2002, Bill Davidsen wrote:

>   The thing is that Solaris, AIX, and ISC are written by commercial
> companies, they realize that customers need to be able to debug systems
> which don't have a screen, a serial printer, etc. They do have disk. 
> 
>   I was hoping Alan would push Redhat to put this in their Linux so we
> could resolve some of the ongoing problems which don't write an oops to a
> log, but I guess none of the developers has to actually support production
> servers and find out why they crash.

Perhaps i'm being grossly naive here, but none of these presumably x86 
productions servers don't have a serial port? Not even PCI/ISA slots to 
add one? Serial would catch most of your oopsen anyway, and if you were 
borked enough that serial couldn't get the entire output, i somehow doubt 
dumping to disk could manage. And no i don't see anything wrong nor 
consider it studly to use oopses only for debugging...

	Zwane

-- 
function.linuxpower.ca

Re: What's left over.

[Bill Davidsen]

On Fri, 1 Nov 2002, Steven King wrote:

> On Friday 01 November 2002 11:18 am, Linus Torvalds wrote:
> 
> > To add insult to injury, you will not be able to actually _test_ any of
> > the real error paths in real life. Sure, you will be able to test forced
> > dumps on _your_ hardware, but while that is fine in the AIX model ("we
> > control the hardware, and charge the user five times what it is worth"),
> > again that doesn't mean _squat_ in the PC hardware space.
> 
>   On the other hand, ISC's system 5 r3 ran on commodity x86 hardware and the 
> crash dumper worked on the various disk hardware I had occasion to use it on 
> (mfm, scsi, ide), although one did need to make sure swap was larger than ram 
> or bad things would happen. 8-{.  

  The thing is that Solaris, AIX, and ISC are written by commercial
companies, they realize that customers need to be able to debug systems
which don't have a screen, a serial printer, etc. They do have disk. 

  I was hoping Alan would push Redhat to put this in their Linux so we
could resolve some of the ongoing problems which don't write an oops to a
log, but I guess none of the developers has to actually support production
servers and find out why they crash.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: What's left over.

[Bill Davidsen]

On 1 Nov 2002, Alan Cox wrote:

> On Fri, 2002-11-01 at 06:36, Linus Torvalds wrote:
> > This never works. Be honest. Nobody takes out features, they are stuck 
> > once they get in. 
> 
> Linus I've asked a couple of times about killing sound/oss off now ALSA
> is integrated 8) While you are on the rant how about that ;)

Good point, that continues to disprove the theory that having one thing in
the kernel prevents development of a similar feature.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: What's left over.

[Bill Davidsen]

On Fri, 1 Nov 2002, Craig I. Hagan wrote:

> If it becomes apparent through empirical data that crash dumps are a useful
> tool, I'm sure that Linus will become far more amenable. Until then, lets let
> him handle all of his other work which needs to get done.

Since he doesn't have the problem he will ignore the proof. Better be sure
we can generate ksymoops reports from the dump, so we can post them asking
for help. Anything else will get the old "I don't use that tool, can't
help." Or like Nvidia problems the "try it without the crash dump code,"
routine.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: What's left over.

[Karim Yaghmour]

Werner Almesberger wrote:
> Karim Yaghmour wrote:
> > Why not just have a simple backup stripped-down "hardened" copy of Linux
> > lying around in a physical RAM region not used by the copy of Linux
> > actually running.
> 
> Congratulations, you've just re-invented MCORE :-) That's exactly
> what they do on systems where rebooting through the firmware
> doesn't preserve RAM.

Oh well, can't have a freshmeat db in my head I guess ;) That said,
I like this approach since you don't need to care about new drivers
and so on ... but since it's already out there I guess it's
advantages have been covered elsewhere ...

Karim

===================================================
                 Karim Yaghmour
               karim@opersys.com
      Embedded and Real-Time Linux Expert
===================================================

Re: What's left over.

[Werner Almesberger]

Karim Yaghmour wrote:
> Why not just have a simple backup stripped-down "hardened" copy of Linux
> lying around in a physical RAM region not used by the copy of Linux
> actually running.

Congratulations, you've just re-invented MCORE :-) That's exactly
what they do on systems where rebooting through the firmware
doesn't preserve RAM.

- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         wa@almesberger.net /
/_http://www.almesberger.net/____________________________________________/

Re: What's left over.

[Karim Yaghmour]

Werner Almesberger wrote:
> But for a general solution, it seems more appropriate to me to solve
> the problem of moving the kernel data from the damaged system to an
> intact system only once, e.g. using the MCORE approach, than over
> and over again for all possible types of hardware and attachment
> methods.

This is just a random tangential thought here, but FWIW:

Why not just have a simple backup stripped-down "hardened" copy of Linux
lying around in a physical RAM region not used by the copy of Linux
actually running. Granted the running Linux doesn't do random physical
accesses when dying, the crash handler could then just boot that
secondary Linux which would then have a RAM disk containing the
appropriate scripts and binaries to handle the actual crash. Given the
cost of RAM these days, reserving a MB or two for this purpose should
probably not be that bad.

Karim

===================================================
                 Karim Yaghmour
               karim@opersys.com
      Embedded and Real-Time Linux Expert
===================================================

Re: What's left over.

[Rusty Russell]

In message <1036157418.12693.19.camel@irongate.swansea.linux.org.uk> you write:
> On Thu, 2002-10-31 at 21:02, Jeff Garzik wrote:
> > hosed/screaming, and various mid-layers are dying.  For LKCD to be of 
> > any use, it needs to _skip_ the block layer and talk directly to 
> > low-level drivers.
> 
> Rusty wrote a polled IDE driver that should handle some subset of that

Yes, patch has bitrotted but updating should be trivial.  There's
enough there that you get the idea though: frankly, it's noninvasive
enough for entry during the 2.6.x series, so it's been down on my
list:

	http://www.kernel.org/pub/linux/kernel/people/rusty/patches/Misc/oopser.patch.gz

I'd love someone to take this for a spin and tweak it up...
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: What's left over.

[Werner Almesberger]

[ Cc: trimmed ]

David Lang wrote:
> One question I have is how much of the driver problem you refer to is
> becouse of optimizations that the various drivers have, could you fall
> back to the simplest, works-with-everything,
> all-timeouts-longer-then-the-slowest-disk slug of a driver that could be
> used to do this dump?

Welcome to the wonderful world of code duplication. And don't forget
the "simplified" TCP/IP stack for network dumps. Uh, USB-attached
storage, anyone ? :-)

Special-case dump drivers make perfect sense in isolated cases (e.g.
narrowly specified boxes) or as a band-aid solution.

But for a general solution, it seems more appropriate to me to solve
the problem of moving the kernel data from the damaged system to an
intact system only once, e.g. using the MCORE approach, than over
and over again for all possible types of hardware and attachment
methods.

The only inherent weakness I see in MCORE is the need to reliably
reset a device, either to the point where it is operational (if
used in the process of dumping), or at least to the point where it
doesn't get in the way (if not used for the dump, e.g. video, HID,
etc.).

But this should still be significantly easier than introducing
"dumb" versions for all drivers. Besides, having a way for cleanly
shutting down or resetting devices is desirable in other contexts,
too (e.g. kexec).

- Werner (disclaimer: not affiliated with Mission Critical Linux,
	 any vendor, or any other form of gainful employment)

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         wa@almesberger.net /
/_http://www.almesberger.net/____________________________________________/

Re: What's left over.

[Pavel Machek]

Hi!

> > I'm a user, and I request that LKCD get merged into the kernel. :-)
> > Do you feel like donating a 700-port console server?  Right, so it's LKCD
> > for me then.
> 
> Wouldn't you rather they neatly tftp'd dumps to a nominated central
> server which noticed the arrival, did the initial processing with a perl
> script and mailed you a summary ?

Out of interest, how does such "initial processing" look like?

Of course I'd like perl script to tell me

"hey, at vicam.c:715 you are freeing memory that is still in use by
usb.c; that crashed your machine 5 times during last week",

but I guess your perl scripts can't do that, right?
								Pavel
-- 
When do you have heart between your knees?

Re: What's left over.

[romieu]

Linus Torvalds <torvalds@transmeta.com> :
[...]
> Maybe not. The only numbers I have is the slowness of PCI.

Issue 'openssl speed' and wait for more numbers.

Short lived hybrid sessions kill (not that this or any of the current 
reasons for asynchronous crypto really matters imho).

Instant benchmark:
                  sign    verify    sign/s verify/s
rsa 1024 bits   0.0148s   0.0008s     67.7   1198.6 (PIV 2GHz)
                  sign    verify    sign/s verify/s
rsa 1024 bits   0.0478s   0.0026s     20.9    381.6 (PII 350MHz)

The 'numbers' are in 1000s of bytes per second processed.
type              8 bytes  64 bytes  256 bytes  1024 bytes  8192 bytes
des ede3         3930.00k  4027.43k   4032.30k    4002.19k    3973.12k (PIV)
type              8 bytes  64 bytes  256 bytes  1024 bytes  8192 bytes
des ede3         1058.51k  1061.25k   1090.70k    1097.44k    1091.36k (PII)

blowfish is ~10x faster btw.

--
Ueimor 

Re: What's left over.

[Hugh Dickins]

On Fri, 1 Nov 2002, Linus Torvalds wrote:
> On Fri, 1 Nov 2002, Joel Becker wrote:
> > 
> > 	I always liked the AIX dumper choices.  You could either dump to
> > the swap area (and startup detects the dump and moves it to the
> > filesystem before swapon) or provide a dedicated dump partition.  The
> > latter was prefered.
> 
> Ehh.. That was on closed hardware that was largely designed with and for
> the OS.
>... 
> In other words: it's a huge risk to play with the disk when the system is
> already known to be unstable. The disk drivers tend to be one of the main
> issues even when everything else is _stable_, for chrissake!
> 
> To add insult to injury, you will not be able to actually _test_ any of 
> the real error paths in real life. Sure, you will be able to test forced 
> dumps on _your_ hardware, but while that is fine in the AIX model ("we 
> control the hardware, and charge the user five times what it is worth"), 
> again that doesn't mean _squat_ in the PC hardware space.

I dealt with crash dumps quite a lot over 10 years with SCO UNIX,
OpenServer and UnixWare: which were addressing the PC market, not
own hardware.

It's a real worry that writing a crash dump to disk might stomp in the
wrong place, but I don't recall it ever happening in practice.  But
occasionally, yes, a dump was not generated at all, or not completed.

Of course, you could argue that SCO's disk drivers were more stable :-)
which might or might not be a compliment to them.

Hugh

Re: What's left over.

[David Lang]

One question I have is how much of the driver problem you refer to is
becouse of optimizations that the various drivers have, could you fall
back to the simplest, works-with-everything,
all-timeouts-longer-then-the-slowest-disk slug of a driver that could be
used to do this dump?

David Lang

On Fri, 1 Nov 2002, Linus Torvalds wrote:

> Alan isn't worried about the "which sector do I write" kind of thing.
> That's the trivial part. Alan is worried about the fact that once you know
> which sector to write, actually _doing_ so is a really hard thing. You
> have bounce buffers, you have exceedingly complex drivers that work
> differently in PIO and DMA modes and are more likely than not the _cause_
> of a number of problems etc.
>
> And you have a situation where interrupts are not likely to work well
> (because you crashed with various locks held), so the regular driver
> simply isn't likely to work all that well.
>
> And you have a situation where there are hundreds of different kinds of
> device drivers for the disk.
>
> In other words, the AIX situation isn't even _remotely_ comparable. A
> large portion of the complexity in the PC stability space is in device
> drivers. It's the thing I worry most about for 2.6.x stabilization, by
> _far_.
>
> And if you get these things wrong, you're quite likely to stomp on your
> disk. Hard. You may be tryign to write the swap partition, but if the
> driver gets confused, you just overwrote all your important data. At which
> point it doesn't matter if your filesystem is journaling or not, since you
> just potentially overwrote it.
>
> In other words: it's a huge risk to play with the disk when the system is
> already known to be unstable. The disk drivers tend to be one of the main
> issues even when everything else is _stable_, for chrissake!
>
> To add insult to injury, you will not be able to actually _test_ any of
> the real error paths in real life. Sure, you will be able to test forced
> dumps on _your_ hardware, but while that is fine in the AIX model ("we
> control the hardware, and charge the user five times what it is worth"),
> again that doesn't mean _squat_ in the PC hardware space.
>
> See?
>
> 		Linus
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: What's left over.

[Steven King]

On Friday 01 November 2002 11:18 am, Linus Torvalds wrote:

> To add insult to injury, you will not be able to actually _test_ any of
> the real error paths in real life. Sure, you will be able to test forced
> dumps on _your_ hardware, but while that is fine in the AIX model ("we
> control the hardware, and charge the user five times what it is worth"),
> again that doesn't mean _squat_ in the PC hardware space.

  On the other hand, ISC's system 5 r3 ran on commodity x86 hardware and the 
crash dumper worked on the various disk hardware I had occasion to use it on 
(mfm, scsi, ide), although one did need to make sure swap was larger than ram 
or bad things would happen. 8-{.  

Re: What's left over.

[Shawn]

On 11/01, Shawn said something like:
> > Pat, the public service that Linus provides is doing exactly what he does.
> > He's acting as a filter.  You may or may not agree with the things he
> 
> cat name-your.patch | Linus --please-dont-delete-your-inbox-again

Maybe "piping" things to Linus is a little rude... :O

--
Shawn Leas
core@enodev.com

While I was gone, somebody rearranged on the furniture in my
bedroom.  They put it in _exactly_ the same place it was.
When I told my roommate, he said: Do I know you?
						-- Stephen Wright

Re: What's left over.

[Linus Torvalds]

On Fri, 1 Nov 2002, Joel Becker wrote:
> 
> 	I always liked the AIX dumper choices.  You could either dump to
> the swap area (and startup detects the dump and moves it to the
> filesystem before swapon) or provide a dedicated dump partition.  The
> latter was prefered.
> 	Either of these methods merely require the dumper to correctly
> write to one disk partition.  This is about as simple as you are going
> to get in disk dumping.

Ehh.. That was on closed hardware that was largely designed with and for
the OS.

Alan isn't worried about the "which sector do I write" kind of thing.  
That's the trivial part. Alan is worried about the fact that once you know
which sector to write, actually _doing_ so is a really hard thing. You
have bounce buffers, you have exceedingly complex drivers that work
differently in PIO and DMA modes and are more likely than not the _cause_
of a number of problems etc.

And you have a situation where interrupts are not likely to work well
(because you crashed with various locks held), so the regular driver
simply isn't likely to work all that well.

And you have a situation where there are hundreds of different kinds of 
device drivers for the disk.

In other words, the AIX situation isn't even _remotely_ comparable. A
large portion of the complexity in the PC stability space is in device
drivers. It's the thing I worry most about for 2.6.x stabilization, by 
_far_.

And if you get these things wrong, you're quite likely to stomp on your
disk. Hard. You may be tryign to write the swap partition, but if the
driver gets confused, you just overwrote all your important data. At which
point it doesn't matter if your filesystem is journaling or not, since you
just potentially overwrote it.

In other words: it's a huge risk to play with the disk when the system is
already known to be unstable. The disk drivers tend to be one of the main
issues even when everything else is _stable_, for chrissake!

To add insult to injury, you will not be able to actually _test_ any of 
the real error paths in real life. Sure, you will be able to test forced 
dumps on _your_ hardware, but while that is fine in the AIX model ("we 
control the hardware, and charge the user five times what it is worth"), 
again that doesn't mean _squat_ in the PC hardware space.

See?

		Linus

Re: What's left over.

[John Alvord]

On Fri, 1 Nov 2002 10:23:01 -0800 (PST), "Shane R. Stixrud"
<shane@stixrud.org> wrote:

>
>On Fri, 1 Nov 2002, Patrick Finnegan wrote:
>> 
>> I'm sorry it _is_ a public service.  Once tens of people started   
>> contributing to it, it became one.  This is like saying that the
>> Washington Monument belongs to the peole that maintain it, any building
>> belongs to the repair crews and janitors.  I'm not saying that Linus is
>> necessarily a janitor, but when you consider how much of the Linux kernel
>> that he didn't write, you may relize that it's not just his kernel.  It
>> also belongs to every single person that has written even a single
>> line of code in it.
>>
>
>The logic you seem to be missing is, the Washington Monument is a
>physical object.  Linus's source tree is a collection of "copied" parts 
>from other peoples source trees.  You obviously see his source copy 
>as special, more so then say my copy.  This is true _ONLY_ because 
>Linus's copy commands more respect then yours or mine.  
>If you think about it, the respect Linus's copy has is _PURELY_ 
>the result of his past _choices_ over how he maintains it.
>
>
>In effect you are saying: 
>
>Patrick: "Everyone trusts your source tree, I think LKCD 
>is SUPER DUPER important and should get the exposure and trust 
>that being in your tree commands." 
>
>Linus: "I think LKCD is a bad idea, until I am convinced otherwise I 
>will not merge it."  
>
>Patrick: "You are wrong, LKCD should be in your copy of the kernel source.
>It is your Job Linus, to add things to _your_ copy which others find 
>important, what you think is secondary."
>
>
>You cannot have it both ways, either Linus's tree is a dumping 
>grounds for all ideas (both good and bad) or it is a place for good 
>ideas (good defined by Linus) where people who trust Linus's judgment can 
>work from.
>
>In truth you can have it both ways.  Take Linus's existing copy, add the 
>features you think are important.  If your choices prove to be superior. 
>you can expect that people (over time) will begin to trust/respect your 
>copy more then Linus's.

This also explains why Linus said it was a vendor push situation. If
vendors pick it up, find it useful (as I am sure they will), and tell
Linus about that usage... LKCD will become part of the mainline tree.
I suspect for most vendors, it would be part of their extra cost
"server" package and the Linux/390 package... It clearly has the
potential to enhance service and buyers of server packages need it.

If along the way, significant numbers of "big users" like Purdue adopt
it, use it, and reflect back to L-K the diagnostic successes and fixes
which result, that could speed the decision. If Linus has a tough bug,
installs LKCD, sends the dump to a wizzard and gets a fix, that would
definitely speed the decision.

john alvord

Re: What's left over.

[Shawn]

On 11/01, Larry McVoy said something like:
> On Fri, Nov 01, 2002 at 11:16:20AM -0500, Patrick Finnegan wrote:
> > On Fri, 1 Nov 2002, Alexander Viro wrote:
> > > It's not a fscking public service.  Linus has full control over his
> > > tree.  You have equally full control over your tree.  Linus can't
> > > tell you what patches to apply in your tree.  You can't tell Linus
> > > what patches he should apply to his.
> > 
> > I'm sorry it _is_ a public service.  Once tens of people started
> > contributing to it, it became one.  
> 
> Pat, the public service that Linus provides is doing exactly what he does.
> He's acting as a filter.  You may or may not agree with the things he

cat name-your.patch | Linus --please-dont-delete-your-inbox-again

--
Shawn Leas
core@enodev.com

My friend has a baby.  I'm recording all the noises he makes so later I can
ask him what he meant.
						-- Stephen Wright

Re: What's left over.

[Joel Becker]

On Fri, Nov 01, 2002 at 01:26:44PM +0000, Alan Cox wrote:
> My concerns are solely with things like the correctness of the disk
> dumper. Its obviously a good way to do a lot more damage if it isnt done
> carefully.

	I always liked the AIX dumper choices.  You could either dump to
the swap area (and startup detects the dump and moves it to the
filesystem before swapon) or provide a dedicated dump partition.  The
latter was prefered.
	Either of these methods merely require the dumper to correctly
write to one disk partition.  This is about as simple as you are going
to get in disk dumping.

Joel

-- 

"You must remember this:
 A kiss is just a kiss,
 A sigh is just a sigh.
 The fundamental rules apply
 As time goes by."

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: What's left over.

[Shane R. Stixrud]

On Fri, 1 Nov 2002, Patrick Finnegan wrote:
> 
> I'm sorry it _is_ a public service.  Once tens of people started   
> contributing to it, it became one.  This is like saying that the
> Washington Monument belongs to the peole that maintain it, any building
> belongs to the repair crews and janitors.  I'm not saying that Linus is
> necessarily a janitor, but when you consider how much of the Linux kernel
> that he didn't write, you may relize that it's not just his kernel.  It
> also belongs to every single person that has written even a single
> line of code in it.
>

The logic you seem to be missing is, the Washington Monument is a
physical object.  Linus's source tree is a collection of "copied" parts 
from other peoples source trees.  You obviously see his source copy 
as special, more so then say my copy.  This is true _ONLY_ because 
Linus's copy commands more respect then yours or mine.  
If you think about it, the respect Linus's copy has is _PURELY_ 
the result of his past _choices_ over how he maintains it.


In effect you are saying: 

Patrick: "Everyone trusts your source tree, I think LKCD 
is SUPER DUPER important and should get the exposure and trust 
that being in your tree commands." 

Linus: "I think LKCD is a bad idea, until I am convinced otherwise I 
will not merge it."  

Patrick: "You are wrong, LKCD should be in your copy of the kernel source.
It is your Job Linus, to add things to _your_ copy which others find 
important, what you think is secondary."


You cannot have it both ways, either Linus's tree is a dumping 
grounds for all ideas (both good and bad) or it is a place for good 
ideas (good defined by Linus) where people who trust Linus's judgment can 
work from.

In truth you can have it both ways.  Take Linus's existing copy, add the 
features you think are important.  If your choices prove to be superior. 
you can expect that people (over time) will begin to trust/respect your 
copy more then Linus's.

-- 
Shane R. Stixrud        "Nothing would please me more than being able to 
shane@stixrud.org       hire ten programmers and deluge the hobby market 
                        with good software." -- Bill Gates 1976

                        We are still waiting ....

Re: What's left over.

[Matt Porter]

On Fri, Nov 01, 2002 at 10:50:45AM -0500, Gerald Britton wrote:
> On Fri, Nov 01, 2002 at 03:25:01PM +0000, Linus Torvalds wrote:
> > The question I have is whether such external hardware is even worth it
> > any more for any standard crypto work.  With a regular PCI bus
> > fundamentally limiting throughput to something like a maximum of 66MB/s
> > (copy-in and copy-out, and that's so theoretical that it's not even
> > funny - I'd be surprised if RL throughput copying back and forth over a
> > PCI bus is more than 25-30MB/s), I suspect that you can do most crypto
> > faster on the CPU directly these days. 
> 
> This may be true of a typical workstation or large server, but your router
> may not have such a modern CPU in it.  Crypto accelerators are likely a
> much bigger win on embedded routers or other small appliances with CPUs such
> as the AMD Elan or other 486 to Pentium class processors.

Yes, and as a tangent, the same class of embedded devices also benefit
from TCP/IP offload facilities.  The same argument against a crypto-api
supporting crypto hardware has been used in the past to argue against
a Linux kernel TCP/IP hardware offload layer.  The argument is
completely invalid once one considers the typically lower speed of an
embedded processor going into a crypto or network-edge device.

Even better, synthesizable SoC designs like IBM PPC4xx and reconfigurable
processors architectures have opened further the concept of an on-chip
crypto or tcp/ip offload macro cell which virtually eliminates PCI
speed/latency concerns for these assist engines.  It should be no
surprise that embedded Linux is highly desired in these application
specific processors.

Regards,
-- 
Matt Porter
porter@cox.net
This is Linux Country. On a quiet night, you can hear Windows reboot.

Re: What's left over.

[Nicolas Pitre]

On Fri, 1 Nov 2002, Patrick Finnegan wrote:

> What I'm going to say may not be popular, and probably won't win me
> friends, but here it is anyhow:
> 
> On Fri, 1 Nov 2002, Alexander Viro wrote:
> 
> > On Fri, 1 Nov 2002, Patrick Finnegan wrote:
> >
> > > No, vendor == people who sold or gave us the softare.  Right now, Linus is
> > > acting like he's a big evil corporation that won't add the change no
> > > matter what we say:
> >
> > ... to his tree.  Geez, why could that be?  Maybe because you don't have
> > any rights to decide what patches does anybody else apply to their trees?
> >
> > It's not a fscking public service.  Linus has full control over his
> > tree.  You have equally full control over your tree.  Linus can't
> > tell you what patches to apply in your tree.  You can't tell Linus
> > what patches he should apply to his.
> 
> I'm sorry it _is_ a public service.  Once tens of people started
> contributing to it, it became one.  This is like saying that the
> Washington Monument belongs to the peole that maintain it, any building
> belongs to the repair crews and janitors.  

But then would you agree seeing anybody, and I mean anybody, coming along 
with a "good idea" for alteration to the Washington Monument and let them do 
what they want?

> I'm not saying that Linus is
> necessarily a janitor, but when you consider how much of the Linux kernel
> that he didn't write, you may relize that it's not just his kernel.  It
> also belongs to every single person that has written even a single
> line of code in it.

It is _his_ copy of the kernel, just as you have your own copy.

Linus' tree is known to be the main reference tree, no more.

If your patch is so valuable (and I don't mean it's not), you should be able
to convince vendors to include it in their own tree.  If _then_ it happens
to be a major feature with a large user base I'm sure it'll make the
reference tree.  But in the mean time a few scattered users isn't enough.


Nicolas

Re: What's left over.

[Larry McVoy]

On Fri, Nov 01, 2002 at 11:16:20AM -0500, Patrick Finnegan wrote:
> On Fri, 1 Nov 2002, Alexander Viro wrote:
> > It's not a fscking public service.  Linus has full control over his
> > tree.  You have equally full control over your tree.  Linus can't
> > tell you what patches to apply in your tree.  You can't tell Linus
> > what patches he should apply to his.
> 
> I'm sorry it _is_ a public service.  Once tens of people started
> contributing to it, it became one.  

Pat, the public service that Linus provides is doing exactly what he does.
He's acting as a filter.  You may or may not agree with the things he
lets in or does not.  That's fine, if you think you can do a better job
you have that option.  i can imagine your answer is "I think he's doing
a fine job except for my project which isn't getting in" or something
like that.  That's a bummer for you but keep the big picture in mind.
Linus is the glue which keeps the Linux world from turning into the
BSD mess.  He is the acknowledged leader.  Without him we have a bunch
of semi-leaders, with him we have a real leader.  The fact that Linus
is here, leading this herd of cats, is a gift to the world.  Try and
imagine Linux without him, it's not a pretty picture.

So figure out a way to work with him, don't stress him out, he's a
critical resource without a viable replacement.
-- 
---
Larry McVoy            	 lm at bitmover.com           http://www.bitmover.com/lm 

Re: What's left over.

[Patrick Finnegan]

What I'm going to say may not be popular, and probably won't win me
friends, but here it is anyhow:

On Fri, 1 Nov 2002, Alexander Viro wrote:

> On Fri, 1 Nov 2002, Patrick Finnegan wrote:
>
> > No, vendor == people who sold or gave us the softare.  Right now, Linus is
> > acting like he's a big evil corporation that won't add the change no
> > matter what we say:
>
> ... to his tree.  Geez, why could that be?  Maybe because you don't have
> any rights to decide what patches does anybody else apply to their trees?
>
> It's not a fscking public service.  Linus has full control over his
> tree.  You have equally full control over your tree.  Linus can't
> tell you what patches to apply in your tree.  You can't tell Linus
> what patches he should apply to his.

I'm sorry it _is_ a public service.  Once tens of people started
contributing to it, it became one.  This is like saying that the
Washington Monument belongs to the peole that maintain it, any building
belongs to the repair crews and janitors.  I'm not saying that Linus is
necessarily a janitor, but when you consider how much of the Linux kernel
that he didn't write, you may relize that it's not just his kernel.  It
also belongs to every single person that has written even a single
line of code in it.

BTW, "My opinions do not represent the opinions of my employer" for at
least this email..

Pat
--
Purdue Universtiy ITAP/RCS
Information Technology at Purdue
Research Computing and Storage
http://www-rcd.cc.purdue.edu

http://dilbert.com/comics/dilbert/archive/images/dilbert2040637020924.gif

Re: What's left over.

[Erik Andersen]

On Fri Nov 01, 2002 at 03:25:01PM +0000, Linus Torvalds wrote:
> funny - I'd be surprised if RL throughput copying back and forth over a
> PCI bus is more than 25-30MB/s), I suspect that you can do most crypto
> faster on the CPU directly these days. 
> 
> Maybe not. The only numbers I have is the slowness of PCI.

It may be faster on your beefy 8 CPU boxes.  But many people are
creating, for example, little wireless access points with 200 Mhz
StrongArm CPUs and similar little devices that lack the major CPU
horsepower of big-iron system.  Such boxes would be far better
off offloading crypto to a little crypto chip, right?

 -Erik

--
Erik B. Andersen             http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

Re: What's left over.

[Michael Clark]

On 11/01/02 23:25, Linus Torvalds wrote:
> In article <20021031194351.GA24676@tapu.f00f.org>,
> Chris Wedgwood  <cw@f00f.org> wrote:
> 
>>On Thu, Oct 31, 2002 at 10:49:10AM -0800, Linus Torvalds wrote:
>>
>>
>>>Any hardware that needs to go off and think about how to encrypt
>>>something sounds like it's so slow as to be unusable. I suspect that
>>>anything that is over the PCI bus is already so slow (even if it
>>>adds no extra cycles of its own) that you're better off using the
>>>CPU for the encryption rather than some external hardware.
>>
>>Except almost all hardware out there that does this stuff is async to
>>some extent...
> 
> 
> That's not my argument.  I realize that external hardware on a PCI bus
> _has_ to be asynchronous, simply because it is so slow. 
> 
> The question I have is whether such external hardware is even worth it
> any more for any standard crypto work.  With a regular PCI bus
> fundamentally limiting throughput to something like a maximum of 66MB/s
> (copy-in and copy-out, and that's so theoretical that it's not even
> funny - I'd be surprised if RL throughput copying back and forth over a
> PCI bus is more than 25-30MB/s), I suspect that you can do most crypto
> faster on the CPU directly these days. 
> 
> Maybe not. The only numbers I have is the slowness of PCI.

A 1GHz PIII will do about 8MBytes/sec of 3DES

Plug in a 2.4Gbs broadcom crypto chip into a 64bit PCI-X slot with the
same CPU and you should be capable of doing at least 10 times that.

Stuff like RSA is much slower (and benefits more from hardware)

BTW - there are some outdated cryptolib patches with an async
interface around somewhere (along with patches for freeswan to use
the async api).

I guess the crypto guys like Chris will add the async API if they need
it (which they do i think ;).

~mc

Re: What's left over.

[Gerald Britton]

On Fri, Nov 01, 2002 at 03:25:01PM +0000, Linus Torvalds wrote:
> The question I have is whether such external hardware is even worth it
> any more for any standard crypto work.  With a regular PCI bus
> fundamentally limiting throughput to something like a maximum of 66MB/s
> (copy-in and copy-out, and that's so theoretical that it's not even
> funny - I'd be surprised if RL throughput copying back and forth over a
> PCI bus is more than 25-30MB/s), I suspect that you can do most crypto
> faster on the CPU directly these days. 

This may be true of a typical workstation or large server, but your router
may not have such a modern CPU in it.  Crypto accelerators are likely a
much bigger win on embedded routers or other small appliances with CPUs such
as the AMD Elan or other 486 to Pentium class processors.

				-- Gerald

Re: What's left over.

[bert hubert]

On Fri, Nov 01, 2002 at 03:25:01PM +0000, Linus Torvalds wrote:


> (copy-in and copy-out, and that's so theoretical that it's not even
> funny - I'd be surprised if RL throughput copying back and forth over a
> PCI bus is more than 25-30MB/s), I suspect that you can do most crypto
> faster on the CPU directly these days. 

I'd be amazed of current CPUs would be able to do asymmetric encryption at
anywhere within an order of magnitude of those rates.

Symmetric encryption is something else. This is the reason many encryption
products (ie, pgp) only use asymmetric encryption for encrypting a symmetric
session key, and not encrypting the entire message.

Regards,

bert hubert

-- 
http://www.PowerDNS.com          Versatile DNS Software & Services
http://lartc.org           Linux Advanced Routing & Traffic Control HOWTO

Re: What's left over.

[Richard B. Johnson]

On Fri, 1 Nov 2002, Patrick Finnegan wrote:
>
[SNIPPED...] 
> You know, pissing off core developers of projects that have been shown to
> be (1) desired (2) potentially useful in Linux, even as an aid to other
> Linux subsystem developers and (3) emperically show to be useful for other
> Free *nixes such as the BSDs, is not what I would be doing as a project
> maintainer.  Of course, I'm not Linus...
> 
> Pat

Maybe somebody should at least say what it is that is:
"(1) desired (2) potentially useful in Linux, even as an aid to
other..."

It might be that you guys are so close to the project that you
lose sight of the fact that others, including Linus, might not
understand how important it is. It is quite possible that somebody
has developed a lot of excellent code that has absolutely no use
to anybody except a small group of intellectuals who use the
kernel to write poetry. In that case, regardless of how excellent
it is, it really should not be in the standard kernel. OTH, it
might be useful to the whole world, but nobody has bothered to
explain how this may be so.


Cheers,
Dick Johnson
Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).
   Bush : The Fourth Reich of America

Re: What's left over.

[Patrick Finnegan]

On Fri, 1 Nov 2002, Alexander Viro wrote:

> On Fri, 1 Nov 2002, Patrick Finnegan wrote:
>
> > No, vendor == people who sold or gave us the softare.  Right now, Linus is
> > acting like he's a big evil corporation that won't add the change no
> > matter what we say:
>
> ... to his tree.  Geez, why could that be?  Maybe because you don't have
> any rights to decide what patches does anybody else apply to their trees?
>
> It's not a fscking public service.  Linus has full control over his
> tree.  You have equally full control over your tree.  Linus can't
> tell you what patches to apply in your tree.  You can't tell Linus
> what patches he should apply to his.
>
> "I'm not satisfied with this tree, I'll try that one" is perfectly OK.
> "I'm not satisfied with either, so bend the fsck over and change your
> tree the way I want" is _NOT_.

Yes, I recognise it's his right.  But what bothers me is that he says "I
want users to say they want it" and when user say they want it hey says
"It's a vendor thing, no users want it."

Linus, if you say you're going to listen, please try and listen.  This is
annoying and dissatisfying to all of us when you say you'll listen and you
blatantly ignore people.  Your tree is your tree, for now it's going to be
patching our own kernel, and then possibly moving to another vendor who
listens to their users.

Pat
--
Purdue Universtiy ITAP/RCS
Information Technology at Purdue
Research Computing and Storage
http://www-rcd.cc.purdue.edu

http://dilbert.com/comics/dilbert/archive/images/dilbert2040637020924.gif

Re: What's left over.

[Linus Torvalds]

In article <20021031194351.GA24676@tapu.f00f.org>,
Chris Wedgwood  <cw@f00f.org> wrote:
>On Thu, Oct 31, 2002 at 10:49:10AM -0800, Linus Torvalds wrote:
>
>> Any hardware that needs to go off and think about how to encrypt
>> something sounds like it's so slow as to be unusable. I suspect that
>> anything that is over the PCI bus is already so slow (even if it
>> adds no extra cycles of its own) that you're better off using the
>> CPU for the encryption rather than some external hardware.
>
>Except almost all hardware out there that does this stuff is async to
>some extent...

That's not my argument.  I realize that external hardware on a PCI bus
_has_ to be asynchronous, simply because it is so slow. 

The question I have is whether such external hardware is even worth it
any more for any standard crypto work.  With a regular PCI bus
fundamentally limiting throughput to something like a maximum of 66MB/s
(copy-in and copy-out, and that's so theoretical that it's not even
funny - I'd be surprised if RL throughput copying back and forth over a
PCI bus is more than 25-30MB/s), I suspect that you can do most crypto
faster on the CPU directly these days. 

Maybe not. The only numbers I have is the slowness of PCI.

		Linus

Re: What's left over.

[Alexander Viro]

On Fri, 1 Nov 2002, Patrick Finnegan wrote:

> No, vendor == people who sold or gave us the softare.  Right now, Linus is
> acting like he's a big evil corporation that won't add the change no
> matter what we say:

... to his tree.  Geez, why could that be?  Maybe because you don't have
any rights to decide what patches does anybody else apply to their trees?

It's not a fscking public service.  Linus has full control over his
tree.  You have equally full control over your tree.  Linus can't
tell you what patches to apply in your tree.  You can't tell Linus
what patches he should apply to his.

"I'm not satisfied with this tree, I'll try that one" is perfectly OK.
"I'm not satisfied with either, so bend the fsck over and change your
tree the way I want" is _NOT_.

Re: What's left over.

[Patrick Finnegan]

On Fri, 1 Nov 2002, Henning P. Schmiedehausen wrote:

> Patrick Finnegan <pat@purdueriots.com> writes:
>
> >Because I'm not a vendor, and I want it.
>
> You don't have a vendor, but roll your own kernels? Tough, so you're
> are a "vendor". Surprise, surprise.
>
> Replace "vendor" with "people who roll up and distribute kernels". So
> one vendor (Linus) refuses to integrate LKCD. Tough. Use another

I'm confused, you just said (1) I'm a vendor and then (2) Linus is my
vendor.  And besides, we don't distribute the kernels - we install them on
our own machines, and say 'done'.  The lack of distribution (at least IMO)
should make us not be a vendor.

> one. Think USP here. Think diversity. Think competition. Maybe "that
> vendor" (Linus) will catch up one day. Maybe not. Maybe "competition"
> is not on his agenda. So what?

This isn't about competition.  It's about integrating a core useful
feature that has been shown to be emperically useful by every other person
who writes an OS kernel.

> Get SuSE. They will integrate everything and their grand mother in
> their kernels.

That's not really an option at the moment.  We have a disto vendor
(RedHat) and were dissatisfied with its kernels so we are trying to use
*the*official* kernel (Linus's kernel).

> Gee, most people seem to think that "vendor" means "big evil
> corporation in Redmont, WA".

No, vendor == people who sold or gave us the softare.  Right now, Linus is
acting like he's a big evil corporation that won't add the change no
matter what we say:

On Thu, 31 Oct 2002, Linus Torvalds wrote:

> On Thu, 31 Oct 2002, Matt D. Robinson wrote:
> >
> > Sure, but why should they have to?  What technical reason is there
> > for not including it, Linus?
<snipped reasons that are imho incorrect>
> To me this says "LKCD is stupid". Which means that I'm not going to
> apply it

On Thu, 31 Oct 2002, Linus Torvalds wrote:

> Don't bother to ask me to merge the thing, that only makes me get even
> more fed up with the whole discussion.

On Thu, 31 Oct 2002, Linus Torvalds wrote:

> And imnsho, debugging the kernel on a source level is the way to do it.
>
> Which is why it's not going to be me who merges it.

On Fri, 1 Nov 2002, Linus Torvalds wrote:

> You got to hear my comment now, several times: convince somebody _else_.
<snip>
> What's so hard to understand about the "vendor-driven" thing, and why do
> people continue to argue about it?

You know, considering the volume of people on this list that have been
saying "I want it, Linus, please integrated it"  and:

On Thu, 31 Oct 2002, Matt D. Robinson wrote:

> I hate Linus' ego, I hate this whole damn discussion, and I find
> it very irritating that I have to go through this process after
> many people have created, enhanced and used LKCD for three years,
> and this is where we're at.
>
> To spend the last month and a half finalizing things for Linus,
> sending this to him on multiple occasions, asking for his comments
> and inclusion, asking for his feedback (as well as others), and
> not hearing _one damn word_ from Linus all that time, and for him
> to wait until now to just say "LKCD is stupid" is insulting.

You know, pissing off core developers of projects that have been shown to
be (1) desired (2) potentially useful in Linux, even as an aid to other
Linux subsystem developers and (3) emperically show to be useful for other
Free *nixes such as the BSDs, is not what I would be doing as a project
maintainer.  Of course, I'm not Linus...

Pat
--
Purdue Universtiy ITAP/RCS
Information Technology at Purdue
Research Computing and Storage
http://www-rcd.cc.purdue.edu

http://dilbert.com/comics/dilbert/archive/images/dilbert2040637020924.gif

Re: What's left over.

[Patrick Finnegan]

On Fri, 1 Nov 2002, Craig I. Hagan wrote:

> > Talk is cheap.
> >
> > I've not seen a _single_ bug-report with a fix that attributed the
> > existing LKCD patches. I might be more impressed if I had.
> >
> > The basic issue is that we don't put patches in in the hope that they will
> > prove themselves later. Your argument is fundamentally flawed.
>
> comment from userspace:
>
> I'm going to have to side with Linus here despite my desire to see LKCD
> merged.

I'll have to disagree with what you're saying, because:

> However, we need to show him the money. This means:
>
> 	* making sure that the patches are kept up to date

They are being kept up to date, and aparently have been for some time.

> 	* keep the LKCD patches in the list/community spotlight in a positive
> 		manner ("please test this!", or  "please use this when
> 		looking for help debugging a system problem"). Perhaps
> 		a 2.5.x-lkcd bk tree or something like that.

Umm, and the difference between maintaining a set of patches per kernel
version and something using bitkeeper (or heaven forbid, CVS)?  Even
Linus didn't starting using source code management until somewhat
recently.

> 	* make documentation/HOWTO's available for folks so that
> 		they'll know how to generate a crashdump
> 		and run a some utilities against it to generate
> 		a synopsis which can be submitted for debugging

Have you seen http://lkcd.sf.net ?  They have that there.   I've
successfully walked through their well-written tutorials and produced
crashdumps from machines that have failed.

> 	* most important: squash a whole lot of bugs with
> 		said dumps!

Perhaps people are but they're not posting the bugs to the list...

> If it becomes apparent through empirical data that crash dumps are a useful
> tool, I'm sure that Linus will become far more amenable. Until then, lets let
> him handle all of his other work which needs to get done.

The data is there, perhaps not for Linux, but for other Unixes -
including ones like the BSDs.  Crashdumps are an invaluable resource for
finding bugs that involve things like hardware that doesn't conform
exactly to specs, or deadlocks, or...

Pat
--
Purdue Universtiy ITAP/RCS
Information Technology at Purdue
Research Computing and Storage
http://www-rcd.cc.purdue.edu

http://dilbert.com/comics/dilbert/archive/images/dilbert2040637020924.gif

Re: What's left over.

[Alan Cox]

On Thu, 2002-10-31 at 21:02, Jeff Garzik wrote:
> hosed/screaming, and various mid-layers are dying.  For LKCD to be of 
> any use, it needs to _skip_ the block layer and talk directly to 
> low-level drivers.

Rusty wrote a polled IDE driver that should handle some subset of that

Re: What's left over.

[Alan Cox]

On Fri, 2002-11-01 at 06:27, Bill Davidsen wrote:
>   You're not listening! Screw the vendors! The users want this enough to
> be patching it into their kernels now.

Welcome to free software. If you can make a case for it go sell people
suitable kernels, build an "LKCD kernel site" whatever.

Re: What's left over.

[Alan Cox]

On Fri, 2002-11-01 at 06:36, Linus Torvalds wrote:
> This never works. Be honest. Nobody takes out features, they are stuck 
> once they get in. 

Linus I've asked a couple of times about killing sound/oss off now ALSA
is integrated 8) While you are on the rant how about that ;)

Re: What's left over.

[Alan Cox]

On Fri, 2002-11-01 at 06:34, Bill Davidsen wrote:
>   From the standpoint of just the driver that's true. However, the remote
> machine and all the network bits between them are a string of single
> points of failure. Isn't it good that both disk and network can be
> supported.

My concerns are solely with things like the correctness of the disk
dumper. Its obviously a good way to do a lot more damage if it isnt done
carefully. Quite clearly your dump system wants to support multiple dump
targets so you can dump to pci battery backed ram, down the parallel
port to an analysing box etc

Re: What's left over.

[Miquel van Smoorenburg]

In article <1036103335.25512.40.camel@bip>,
Xavier Bestel  <xavier.bestel@free.fr> wrote:
>Le jeu 31/10/2002 à 23:57, Pavel Machek a écrit :
>
>> This seems like a pretty common situation to me, and current solutions
>> are not nice. [I guess ~/bin/ with --x and
>> ~/bin/my-secret-password-only-jarka-and-mj-knows/phonebook would solve
>> the problem, but...!]
>
>Can't even this be spied from /proc/*/fd ?

Or ptrace, /proc/pid/mem, etc. If you can execute a binary, it
has to be loaded into memory in a process running as you, so
you can read it.

Mike.

Re: What's left over.

[Henning P. Schmiedehausen]

Bill Davidsen <davidsen@tmr.com> writes:

>  You're not listening! Screw the vendors! The users want this enough to
                         ^^^^^^^^^^^^^^^^^^
>be patching it into their kernels now.

[...]

>  I also have Solaris and AIX servers, and if they go down I send a crash
>dump to the vendor who can then provide support. Big difference. Visible
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

q.e.d. End of Discussion.

	Regard
		Henning


-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20   

Re: What's left over.

[Henning P. Schmiedehausen]

Patrick Finnegan <pat@purdueriots.com> writes:

>> What's so hard to understand about the "vendor-driven" thing, and why do
>> people continue to argue about it?

>Because I'm not a vendor, and I want it.

So get your vendor to integrate it. 

You don't have a vendor, but roll your own kernels? Tough, so you're
are a "vendor". Surprise, surprise.

Replace "vendor" with "people who roll up and distribute kernels". So
one vendor (Linus) refuses to integrate LKCD. Tough. Use another
one. Think USP here. Think diversity. Think competition. Maybe "that
vendor" (Linus) will catch up one day. Maybe not. Maybe "competition"
is not on his agenda. So what?

Get SuSE. They will integrate everything and their grand mother in
their kernels.

Gee, most people seem to think that "vendor" means "big evil
corporation in Redmont, WA".

	Regards
		Henning

-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20   

Re: What's left over.

[Craig I. Hagan]

> Talk is cheap.
> 
> I've not seen a _single_ bug-report with a fix that attributed the
> existing LKCD patches. I might be more impressed if I had. 
> 
> The basic issue is that we don't put patches in in the hope that they will
> prove themselves later. Your argument is fundamentally flawed.

comment from userspace:

I'm going to have to side with Linus here despite my desire to see LKCD merged.
However, we need to show him the money. This means:

	* making sure that the patches are kept up to date

	* keep the LKCD patches in the list/community spotlight in a positive
		manner ("please test this!", or  "please use this when
		looking for help debugging a system problem"). Perhaps
		a 2.5.x-lkcd bk tree or something like that.

	* make documentation/HOWTO's available for folks so that
		they'll know how to generate a crashdump
		and run a some utilities against it to generate
		a synopsis which can be submitted for debugging

	* most important: squash a whole lot of bugs with
		said dumps!

If it becomes apparent through empirical data that crash dumps are a useful
tool, I'm sure that Linus will become far more amenable. Until then, lets let
him handle all of his other work which needs to get done.

-- craig



	  .-    ... . -.-. .-. . -    -- . ... ... .- --. .

			    Craig I. Hagan
			   hagan(at)cih.com

Re: What's left over.

[Joe Thornber]

On Fri, Nov 01, 2002 at 08:14:16AM +1100, Rusty Russell wrote:
> In message <20021031101558.GB7487@fib011235813.fsnet.co.uk> you write:
> > On Thu, Oct 31, 2002 at 02:00:31PM +1100, Rusty Russell wrote:
> > > They have, IIRC.  Interestingly, it was less invasive (existing source
> > > touched) than the LVM2/DM patch you merged.
> > 
> > FUD.  I added to three areas of existing code:
> 
> [ 40-line detailed explanation snipped ]
> 
> Woah!  War's over dude!  We won!

:)

Sorry, it wasn't meant to be an agressive email.  However comments
like this do get picked up out of context and passed around until they
become the accepted truth.  I'm still trying to work out where 'dm
can't handle mirroring or raid' rumour came from.

- Joe

Re: What's left over.

[Linus Torvalds]

On Fri, 1 Nov 2002, Bill Davidsen wrote:
> 
>   If you really believed the stuff you say you'd put it in and promise to
> take it out if people didn't find it useful or there were inherent
> limitations.

This never works. Be honest. Nobody takes out features, they are stuck 
once they get in. Which is exactly why my job is to say "no", and why 
there is no "accepted unless proven bad". 

> It would probably take 10-30% off the time to a stable release.

Talk is cheap.

I've not seen a _single_ bug-report with a fix that attributed the
existing LKCD patches. I might be more impressed if I had. 

The basic issue is that we don't put patches in in the hope that they will
prove themselves later. Your argument is fundamentally flawed.

		Linus

Re: What's left over.

[Bill Davidsen]

On Thu, 31 Oct 2002, Linus Torvalds wrote:

> 
> On Thu, 31 Oct 2002, Chris Friesen wrote:
> > 
> > How do you deal with netdump when your network driver is what caused the 
> > crash?
> 
> Actually, from a driver perspective, _the_ most likely driver to crash is 
> the disk driver. 
> 
> That's from years of experience. The network drivers are a lot simpler,
> the hardware is simpler and more standardized, and doesn't do as many
> things. It's just plain _easier_ to write a network driver than a disk
> driver.
> 
> Ask anybody who has done both.

  From the standpoint of just the driver that's true. However, the remote
machine and all the network bits between them are a string of single
points of failure. Isn't it good that both disk and network can be
supported.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: What's left over.

[Bill Davidsen]

On Thu, 31 Oct 2002, Linus Torvalds wrote:

> 
> On Wed, 30 Oct 2002, Matt D. Robinson wrote:
> 
> > Linus Torvalds wrote:
> > > > Crash Dumping (LKCD)
> > > 
> > > This is definitely a vendor-driven thing. I don't believe it has any
> > > relevance unless vendors actively support it.
> > 
> > There are people within IBM in Germany, India and England, as well as
> > a number of companies (Intel, NEC, Hitachi, Fujitsu), as well as SGI
> > that are PAID to support this.
> 
> That's fine. And since they are paid to support it, they can apply the 
> patches.  
> 
> What I'm saying by "vendor driven" is that it has no relevance for the 
> standard kernel, and since it has no relevance to that, then I have no 
> incentives to merge it. The crash dump is only useful with people who 
> actively look at the dumps, and I don't know _anybody_ outside of the 
> specialized vendors you mention who actually do that.

  You're not listening! Screw the vendors! The users want this enough to
be patching it into their kernels now.

> 
> I will merge it when there are real users who want it - usually as a
> result of having gotten used to it through a vendor who supports it. (And
> by "support" I do not mean "maintain the patches", but "actively uses it"
> to work out the users problems or whatever).

  Did you not read the input from the developers? From the people who have
headless clusters?

  I have Linux systems in fifteen locations, six states, for timezones.
They oops from time to time, and I can't get any clue why, because (a)
they have no console, (b) most are in secure locations like locked wiring
closets with no one to read a console, and (c) the systems are thousands
of miles away. I don't need a debugger, I'd love to just have ksysoops
output! And given the reality of using the network, I don't make kcore
world readable, I'm not about to send that information over a few
thousand miles of open net to save writing it to disk.

  I also have Solaris and AIX servers, and if they go down I send a crash
dump to the vendor who can then provide support. Big difference. Visible
even to management, who see a real support issue.

> 
> Horse before the cart and all that thing.
> 
> People have to realize that my kernel is not for random new features.

  Supportablility is not a "random new feature," it's something which was
developed because users had a need (not by a vendor looking for a feature
to advertize), and if you would read the mail it's mostly coming from
people who want to use the feature. This is a whole new kernel series, it
will be stable a hell of a lot sooner if people can find problems!

  Notice that developers want it, vendors want to provide it, and end
users want to be able to get support. In fact, other than one person who
had doubts about the implementation being optimal, your voice is the only
one I hear against it. That should tell you something.

  Sometimes the best way to lead is to look at where everyone is going on
their own, jump in front, and yell "Follow me!" a few times. If you put
half the energy into improving the implementation that you put into
telling us we're all wrong it would be a better kernel.



On Thu, 31 Oct 2002, Linus Torvalds wrote:

> 
> [ Ok, this is a really serious email. If you don't get it, don't bother 
>   emailing me. Instead, think about it for an hour, and if you still don't 
>   get it, ask somebody you know to explain it to you. ]
> 
> On Thu, 31 Oct 2002, Matt D. Robinson wrote:
> > 
> > Sure, but why should they have to?  What technical reason is there
> > for not including it, Linus?
> 
> There are many:
> 
>  - bloat kills:
> 
> 	My job is saying "NO!"
> 
> 	In other words: the question is never EVER "Why shouldn't it be
> 	accepted?", but it is always "Why do we really not want to live 
> 	without this?"

  I suspect that you have not had to make any significant part of your
living administering systems, certainly not recently. Lack of this tool is
a one-to-one mapping to "no clue" if you can't get information from the
console.
 
>  - included features kill off (potentially better) projects.
> 
> 	There's a big "inertia" to features. It's often better to keep 
> 	features _off_ the standard kernel if they may end up being
> 	further developed in totally new directions.

  Yes, you can clearly see how that worked with ext2 stifling development
of... wait a minute, rethink that argument. This feature is years old, and
seems to be ready to add new destinations for the data, disk, net, high
memory, what elese is there? Once the data is saved people will be able to
develop any additional tools they want to read the raw data.
 
> 	In particular when it comes to this project, I'm told about
> 	"netdump", which doesn't try to dump to a disk, but over the net.
> 	And quite frankly, my immediate reaction is to say "Hell, I
> 	_never_ want the dump touching my disk, but over the network
> 	sounds like a great idea".

  You have this idea that the dump will go over a high reliability path,
and that's an option, but not in all cases true.

> To me this says "LKCD is stupid". Which means that I'm not going to apply 
> it, and I'm going to need some real reason to do so - ie being proven 
> wrong in the field.

  You've been proven wrong, you just don't want to look at the proof! You
can't say it doesn't work, it does. You can't say the (users, vendors,
developers} don't want it, because they do. You can't say it's untested,
it's been in use for several years, and you seem willing to take reiser4,
which isn't even finsished yet!

> (And don't get me wrong - I don't mind getting proven wrong. I change my 
> opinions the way some people change underwear. And I think that's ok).

  If you really believed the stuff you say you'd put it in and promise to
take it out if people didn't find it useful or there were inherent
limitations. It would probably take 10-30% off the time to a stable
release.
 
> > I completely don't understand your reasoning here.
> 
> Tough. That's YOUR problem.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: What's left over.

[James Morris]

On 31 Oct 2002, Alan Cox wrote:

> Chris is write that crypto api is misdesigned if we want to use hardware
> cryptocards

Hardware support was not an initial goal, as the requirements are not yet 
fully known.

>From Documentation/crypto/api-intro.txt:

  An asynchronous scheduling interface is in planning but not yet
  implemented, as we need to further analyze the requirements of all of
  the possible hardware scenarios (e.g. IPsec NIC offload).

Hardware accelerators are generally a known issue, with already proven 
solutions (e.g. the OpenBSD crypto queue).  We don't know much about IPSec 
NIC offload yet, however.


- James
-- 
James Morris
<jmorris@intercode.com.au>

Re: What's left over.

[Patrick Finnegan]

On Fri, 1 Nov 2002, Linus Torvalds wrote:

> But no, it wasn't the answer you wanted.  So you refuse to listen.  And
> yes, I get irritated too.  So right now I won't touch LKCD with a
> ten-foot pole, if only because I've been mail-bombed by people who argue
> for it when I have better things to do than to explain myself over and
> over again.

Maybe it's because users are wanting it in the mainline kernel...  Notice
I said 'users' not 'vendors' or 'the code's maintainers'.

> What's so hard to understand about the "vendor-driven" thing, and why do
> people continue to argue about it?

Because I'm not a vendor, and I want it.

Pat
--
Purdue Universtiy ITAP/RCS
Information Technology at Purdue
Research Computing and Storage
http://www-rcd.cc.purdue.edu

http://dilbert.com/comics/dilbert/archive/images/dilbert2040637020924.gif

Re: What's left over.

[Linus Torvalds]

In article <Pine.LNX.4.44.0210311923460.24182-100000@nakedeye.aparity.com>,
Matt D. Robinson <yakker@aparity.com> wrote:
>
>To spend the last month and a half finalizing things for Linus,
>sending this to him on multiple occasions, asking for his comments
>and inclusion, asking for his feedback (as well as others), and
>not hearing _one damn word_ from Linus all that time, and for him
>to wait until now to just say "LKCD is stupid" is insulting.

You got to hear my comment now, several times: convince somebody _else_.

But no, it wasn't the answer you wanted.  So you refuse to listen.  And
yes, I get irritated too.  So right now I won't touch LKCD with a
ten-foot pole, if only because I've been mail-bombed by people who argue
for it when I have better things to do than to explain myself over and
over again. 

What's so hard to understand about the "vendor-driven" thing, and why do
people continue to argue about it? 

			Linus

Re: What's left over.

[Matt D. Robinson]

On Thu, 31 Oct 2002, Jeff Garzik wrote:
|>Re-read my other post(s) -- I have said repeatedly that LKCD's 
|>infrastructure is decent.  But it's completely pointless to merge a 
|>decent infrastructure unless the users are up to snuff.  It's much 
|>smarter to keep the infrastructure out of the kernel until the low-level 
|>dump drivers are hammered out and stable, because that gives you more 
|>freedom to change the API.

This is where we disagree.  Without the base infrastructure, this
becomes an even larger and larger patch which needs testing and
verification with a massive number of configurations for each new
kernel release.  Do you know how much testing we go through for each
new kernel release?  Do you know that we actually try this stuff
out with panic(), die(), interrupt and sysrq() dumps before we send
it off?  Do you know we try this for SMP and UP?

If Linus would at least take the infrastructure patches and leave
out the drivers/dump code, that might be a good start.  Just take
the base code.  Just take the patches for panic.c, dump_ipi(), or
the rest of the other base kernel components,  But no.  Instead,
Linus just says "LKCD is stupid".

I also think you have completely misrepresented the LKCD user base,
but I'm sure our opinion on who those LKCD users are is different
and it's pointless to argue one person's experiences over another's.

I hate Linus' ego, I hate this whole damn discussion, and I find
it very irritating that I have to go through this process after
many people have created, enhanced and used LKCD for three years,
and this is where we're at.

To spend the last month and a half finalizing things for Linus,
sending this to him on multiple occasions, asking for his comments
and inclusion, asking for his feedback (as well as others), and
not hearing _one damn word_ from Linus all that time, and for him
to wait until now to just say "LKCD is stupid" is insulting.

--Matt

Re: What's left over.

[Jeff Garzik]

Matt D. Robinson wrote:

>On Thu, 31 Oct 2002, Jeff Garzik wrote:
>|>Linus Torvalds wrote:
>|>[yes, I realize the LKCD merge debate is over, bear with me :)]
>
>For Linus, it is.
>
>|>That said, I used to be an LKCD cheerleader until a couple people made 
>|>some good points to me:  it is not nearly low-level enough to truly be 
>|>of use in crash situations.  netdump can work if your interrupts are 
>|>hosed/screaming, and various mid-layers are dying.  For LKCD to be of 
>|>any use, it needs to _skip_ the block layer and talk directly to 
>|>low-level drivers.
>
>Just to clarify, LKCD is NOT block based dumping, OR net based
>dumping, or anything.  It's an infrastructure for dumping that
>lets you, the user, the distributor, the customer, whatever,
>make the decision for what's right for you.  Yes, we provide
>disk based dumping now, and are including the net dump code
>very soon, as well as some of these other smaller dump methods.
>
>Has ANYONE other than Christoph and Stephen H. done a full review of
>the LKCD patch set before commenting?  Or are people just making
>this stuff up as they go along?  A ton of things have changed
>over the past year just because people complained about only doing
>disk dumping.  And then to hear this ...
>  
>
You are confusing review with perspective.  I've read 
http://lkcd.sourceforge.net/download/latest/ before, and just checked it 
again tonight before posting.

My view is:  LKCD becomes useful to merge when the average user can do 
"safe" disk dumps.  netdumps are better for corporate customers, but for 
average users, disk dumps are _the_ method which is easiest, most 
accessible, and thus most helpful to kernel hackers debugging their 
problems.  LKCD has a dump block dev driver, but it's not even close to 
being low-level enough to be "safe".

Re-read my other post(s) -- I have said repeatedly that LKCD's 
infrastructure is decent.  But it's completely pointless to merge a 
decent infrastructure unless the users are up to snuff.  It's much 
smarter to keep the infrastructure out of the kernel until the low-level 
dump drivers are hammered out and stable, because that gives you more 
freedom to change the API.


>|>So, I think the stock kernel does need some form of disk dumping, 
>|>regardless of any presence/absence of netdump.  But LKCD isn't
>|>there yet...
>
>Please read the patches and decide again.  If you want the latest
>net dump patch, let me know.
>  
>

I have.  Nothing has changed.  Stable, polling, low-level disk dumps are 
not in the LKCD patches.

IMO, net dump is what corporate customers and network admins want.  And 
overall, net dumps are probably easier and much safer than disk dumps, 
from an implementor's perspective.  However, disk dumps are what the 
average kernel hacker will find most useful, because it is the easiest 
for end users, and thus will generate a higher number of quality bug 
reports.

    Jeff

Re: What's left over.

[Matt D. Robinson]

On Thu, 31 Oct 2002, Shawn wrote:
|>On 10/31, Matt D. Robinson said something like:
|>> On Thu, 31 Oct 2002, Linus Torvalds wrote:
|>> |>On Wed, 30 Oct 2002, Matt D. Robinson wrote:
|>> |>That's fine. And since they are paid to support it, they can apply the 
|>> |>patches.  
|>> 
|>> We want to see this in the kernel, frankly, because it's a pain
|>> in the butt keeping up with your kernel revisions and everything
|>> else that goes in that changes.  And I'm sure SuSE, UnitedLinux and
|>> (hopefully) Red Hat don't want to spend their time having to roll
|>> this stuff in each and every time you roll a new kernel.
|>
|>I share some of your sentiment, but honestly, think about it.
|>
|>Linus has to "keep up" with all the changees coming into his inbox as
|>well, and the more features, the more breakage that can happen when
|>Linus accepts a patch.

Uh ... have you read the patches?  Do you see how few the
changes are to non-dump code?  Do you know that most of those
changes only get triggered in a crash situation anyway?

Breakage occurs when people change code areas that are used
all the time, like VM, network, block layer, etc.

Look at the patches and tell me where we are causing overhead
and and seriously potential breakage.  If you find problems,
then tell us, don't just comment on breakage scenarios.

|>Really, Linus wants to push some of his maintanance overhead to distros,
|>who get paid to do it, but also to provide sexy bullet point items for
|>users, so they buy "Linux" stuff.

Sure, then remove all of the extra filesystems, sound drivers,
etc., that are bulking up the kernel distribution now and give
them to the distributors to include.

|>You try to find a better balance.

If I could think of a better balance to ease his load, I would.
He's already made his mind up.  It doesn't mean it won't end up
merged by someone else (or everyone else for that matter).

--Matt

Re: What's left over.

[Matt D. Robinson]

On Thu, 31 Oct 2002, Jeff Garzik wrote:
|>Linus Torvalds wrote:
|>[yes, I realize the LKCD merge debate is over, bear with me :)]

For Linus, it is.

|>That said, I used to be an LKCD cheerleader until a couple people made 
|>some good points to me:  it is not nearly low-level enough to truly be 
|>of use in crash situations.  netdump can work if your interrupts are 
|>hosed/screaming, and various mid-layers are dying.  For LKCD to be of 
|>any use, it needs to _skip_ the block layer and talk directly to 
|>low-level drivers.

Just to clarify, LKCD is NOT block based dumping, OR net based
dumping, or anything.  It's an infrastructure for dumping that
lets you, the user, the distributor, the customer, whatever,
make the decision for what's right for you.  Yes, we provide
disk based dumping now, and are including the net dump code
very soon, as well as some of these other smaller dump methods.

Has ANYONE other than Christoph and Stephen H. done a full review of
the LKCD patch set before commenting?  Or are people just making
this stuff up as they go along?  A ton of things have changed
over the past year just because people complained about only doing
disk dumping.  And then to hear this ...

|>So, I think the stock kernel does need some form of disk dumping, 
|>regardless of any presence/absence of netdump.  But LKCD isn't
|>there yet...

Please read the patches and decide again.  If you want the latest
net dump patch, let me know.

|>    Jeff

--Matt

Re: What's left over.

[Werner Almesberger]

john stultz wrote:
> I thought you were suggesting some crazy ACL
> symlink like: "Make file foo's ACL be the same as file blah's ACL" and
> if I then go and add some untrusted user to blah's ACL it would then
> automatically change foo's ACL.

Well, with "foo" getting the ACL from "bar", changing the ACL of
"bar" would change "foo", but not vice versa. Of course, the idea
is that you're careful when changing "bar", just like you'd be
careful with your SSH keys.

> Eh, as long as the ACLs are per-file, I can't ever accidentally give
> access to a file I didn't mean to. The corner cases of "remove my
> ex-friend from all my files" could be annoying, but could be done w/ the
> equiv of chgrp -r 

chgrp -r gets nasty if you have files which are stored off-line.
On the other hand, using the concept that ACEs add rights, but
never take them away, even an off-line "ACL link target" would
fail on the safe side, by not adding more rights.

> I probably should just go read the specs. Anyone have a pointer, or care
> to explain what the differences are between AFS's ACLs and POSIX ACLs?

I've forgotten most things I knew about AFS ACLs (I used them at
IBM about eight years ago), but http://acl.bestbits.at/ and in
particular http://acl.bestbits.at/cgi-man/acl.5 seem to have
everything about POSIX ACLs. They're not very complicated.

- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         wa@almesberger.net /
/_http://www.almesberger.net/____________________________________________/

Re: What's left over.

[john stultz]

On Thu, 2002-10-31 at 14:54, Werner Almesberger wrote:
> john stultz wrote:
> > Ugh, that seems dangerous. Too many forgotten ACL links and then I could
> > accidentally give a vague acquaintance access to all my data meant for
> > close friends. 
> 
> The idea is that you'd typically have (a) (small number of) specific
> location(s) where you keep your files representing groups, e.g.
> $HOME/acls/ for your personal lists, maybe ~project/acls/ for
> projects, etc.

Oh! Ok, that's exactly like the user-definable ACL groups I was
describing. My mistake, I thought you were suggesting some crazy ACL
symlink like: "Make file foo's ACL be the same as file blah's ACL" and
if I then go and add some untrusted user to blah's ACL it would then
automatically change foo's ACL. That just seemed a bit out there, but it
was just my mis-interpretation. Sorry :)

> If you think already this is dangerous, then you should be
> terrified by regular, non-aggregateable ACLs ;-)

Eh, as long as the ACLs are per-file, I can't ever accidentally give
access to a file I didn't mean to. The corner cases of "remove my
ex-friend from all my files" could be annoying, but could be done w/ the
equiv of chgrp -r 

> I'm not saying that ACLs aren't useful, only that the lack of
> aggregateability makes them hard to maintain, so that people
> frequently fall back to setup scripts that simple re-create
> their ACL configuration. Once you're at this point, ACLs have
> lost much of their usefulness, and you might as well use some
> suid program that creates groups for you.

Hmmm. I'm way out of my realm of competency here. I just know ACLs were
*really* useful w/ AFS. 

I probably should just go read the specs. Anyone have a pointer, or care
to explain what the differences are between AFS's ACLs and POSIX ACLs?

thanks
-john

Re: What's left over.

[James Simmons]

> > Fbdev Rewrite
>
> This one is just huge, and I have little personal judgement on it.

The size has been cut in half now that the issue of AGP being intialized
to late is on hold. We can discuss that move post-halloween. All that is
in the fbdev tree are fbdev changes. So it is safe to pull it.

Re: What's left over.

[Pavel Machek]

Hi!

> > This seems like a pretty common situation to me, and current solutions
> > are not nice. [I guess ~/bin/ with --x and
> > ~/bin/my-secret-password-only-jarka-and-mj-knows/phonebook would solve
> > the problem, but...!]
> 
> Can't even this be spied from /proc/*/fd ?

Not sure... Its true that if users are not carefull (i.e. do 

cat ~/bin/my-secret-password-only-jarka-and-mj-knows/phonebook

it can be seen on ps -aux ;-).
							Pavel
-- 
When do you have heart between your knees?

Re: What's left over.

[Pavel Machek]

Hi!

> > > ext2/ext3 ACLs and Extended Attributes
> > 
> > I don't know why people still want ACL's. There were noises about them for 
> > samba, but I'v enot heard anything since. Are vendors using this?
> 
> Because People Are Stupid(tm).  Because it's cheaper to put "ACL support: yes"
> in the feature list under "Security" than to make sure than userland can cope
> with anything more complex than  "Me Og.  Og see directory.  Directory Og's.
> Nobody change it".  C.f. snake oil, P.T.Barnum and esp. LSM users

Okay... Have ~/bin/phonebook and I'd like it to be rw- to me, r-- to
jarka and mj, and --- to everyone else. How do I do that without ACLs?
Adding a group is root-only operation.

This seems like a pretty common situation to me, and current solutions
are not nice. [I guess ~/bin/ with --x and
~/bin/my-secret-password-only-jarka-and-mj-knows/phonebook would solve
the problem, but...!]
								Pavel
-- 
When do you have heart between your knees?

Re: What's left over.

[Werner Almesberger]

john stultz wrote:
> Ugh, that seems dangerous. Too many forgotten ACL links and then I could
> accidentally give a vague acquaintance access to all my data meant for
> close friends. 

The idea is that you'd typically have (a) (small number of) specific
location(s) where you keep your files representing groups, e.g.
$HOME/acls/ for your personal lists, maybe ~project/acls/ for
projects, etc.

If you think already this is dangerous, then you should be
terrified by regular, non-aggregateable ACLs ;-)

I'm not saying that ACLs aren't useful, only that the lack of
aggregateability makes them hard to maintain, so that people
frequently fall back to setup scripts that simple re-create
their ACL configuration. Once you're at this point, ACLs have
lost much of their usefulness, and you might as well use some
suid program that creates groups for you.

- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         wa@almesberger.net /
/_http://www.almesberger.net/____________________________________________/

Re: What's left over.

[Pavel Machek]

Hi!

> > Without ACLs, if Sally, Joe and Bill need rw access to a file/dir, just
> > create another group with just those three people in.  Over time, of
> 
> If Sally, Joe and Bill need rw access to a directory, and Joe and Bill
> are using existing userland (any OS I'd seen), then Sally can easily
> fuck them into the next month and not in a good way.

Do you mean symlink attack?

> _That_ is the real problem.  Until that is solved (i.e. until all
> userland is written up to the standards allegedly followed in writing
> suid-root programs wrt hostile filesystem modifications) NO mechanism
> will help you.  ACLs, huge groups, whatever - setups with that sort
> of access allowed are NOT SUSTAINABLE with the current userland(s).

So userland needs to be improved. It already needs that modifications
because of /tmp. Is there any new issue there?
								Pavel
-- 
When do you have heart between your knees?

RE: What's left over.

[Perez-Gonzalez, Inaky]

> THAT is what I mean by vendor-driven. If vendors decide they 
> really want the patches, and I actually start seeing noises on 
> linux-kernel or getting
> requests for it being merged from _users_ rather than developers, then
> that means that the vendor is on to something.

I am a user and I use it; I'd like it. I am a developer and I use it. I'd
love it. Forget my intel.com paying my paycheck.

Inaky Perez-Gonzalez -- Not speaking for Intel - opinions are my own [or my
fault]

Re: What's left over.

[Werner Almesberger]

Jeff Garzik wrote:
> That said, I used to be an LKCD cheerleader until a couple people made 
> some good points to me:  it is not nearly low-level enough to truly be 
> of use in crash situations.

I'm not so convinced about this. I like the Mission Critical
approach: save the dump to memory, then either boot through the
firmware or through bootimg (nowadays, that would be kexec),
then retrieve the dump from memory, and do whatever you like
with it.

The huge advantage here is that you don't need a ton of
specialized dump drivers and/or have much of the original kernel
infrastructure to be in a usable state. The rebooted system will
typically be stable enough to offer the full range of utilities,
including up to date drivers for all possible devices, so you
can safely write to disk, scp all the mess to your support
critter, or post an automatic flame to linux-kernel :-)

The weak points of the Mission Critical design are that early
memory allocation in the kernel needs to be tightly controlled,
that architectures that wipe CPU caches on reboot need to
commit them to memory before the firmware restart, and that
drivers need to be able to recover from an "unclean" hardware
state. (I think we'll see much of the latter happen as kexec
advances. The other two issues aren't really special.)

Actually, at the RAS BOF I thought that IBM were developing LKCD
in this direction, and had also eliminated a few not so elegant
choices of Mission Critical's original design. I haven't looked
at the LKCD code, but the descriptions sound as if all the
special-case cruft seems to be back again, which I would find a
little disappointing.

There might be a case for specialized low-overhead dump handlers
for small embedded systems and such, but they're probably better
maintained outside of the mainstream kernel. (They're more like
firmware anyway.)

- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         wa@almesberger.net /
/_http://www.almesberger.net/____________________________________________/

Re: What's left over.

[john stultz]

On Thu, 2002-10-31 at 13:49, Werner Almesberger wrote:
> john stultz wrote:
> > groups for each project, I have no clue how anyone would be able to
> > handle the (unix)group management required. ACLs let the users
> > themselves manage what people got what access to their data.
> 
> Note that POSIX ACLs don't seem to solve this either: they only
> let you control access in terms of existing users or groups.

I've never looked at the POSIX ACL spec, so forgive my ignorance.
 
> IMHO, this is one of the standard pitfalls of ACLs: if they don't
> let you aggregate information, you quickly end up with huge ACLs
> hanging off every file, and each of those ACLs wants to be
> carefully maintained. I've seen a lot of this in my VMS days.
> (Unix is a bit better, because you can control access at a
> directory level, while VMS needs the ACL on each file, because
> you can open files directly by VMS' equivalent to an inode
> number, without traversing the directory hierarchy. Of course,
> many users didn't know that :-)

While it would be nice to have user-definable ACL groups ("my friends"
or "History 255 TAs") in addition to existing users and groups, I still
don't find this to be critical. Sure, it adds (possibly quite a bit of)
extra data to every file, but it gives you the granularity you need for
the situation I described.  It seems like user-definable ACL groups
would be a nice extra feature on top of existing users or groups, but
not a necessity.

> To make ACLs truly scalable, it would be nice to be able to
> express permissions in terms of access to other filesystem
> objects. E.g. "everybody who can read file ~me/acls/my_friends
> can write the directory on which this ACE hangs". This should
> work like a symlink, i.e. if I add new friends to my_friends, I
> don't have to update all my ACLs.

Ugh, that seems dangerous. Too many forgotten ACL links and then I could
accidentally give a vague acquaintance access to all my data meant for
close friends. 

Regardless, while ACLs do result in extra data per file being used, it
is my understanding that ACLs allow you to solve problems that currently
aren't solvable w/o administrator intervention. In my experience using
them w/ AFS, they have been extremely useful. 


-john

Re: What's left over.

[Xavier Bestel]

Le jeu 31/10/2002 à 23:57, Pavel Machek a écrit :

> This seems like a pretty common situation to me, and current solutions
> are not nice. [I guess ~/bin/ with --x and
> ~/bin/my-secret-password-only-jarka-and-mj-knows/phonebook would solve
> the problem, but...!]

Can't even this be spied from /proc/*/fd ?

Re: What's left over.

[Shawn]

On 10/31, Matt D. Robinson said something like:
> On Thu, 31 Oct 2002, Linus Torvalds wrote:
> |>On Wed, 30 Oct 2002, Matt D. Robinson wrote:
> |>That's fine. And since they are paid to support it, they can apply the 
> |>patches.  
> 
> We want to see this in the kernel, frankly, because it's a pain
> in the butt keeping up with your kernel revisions and everything
> else that goes in that changes.  And I'm sure SuSE, UnitedLinux and
> (hopefully) Red Hat don't want to spend their time having to roll
> this stuff in each and every time you roll a new kernel.

I share some of your sentiment, but honestly, think about it.

Linus has to "keep up" with all the changees coming into his inbox as
well, and the more features, the more breakage that can happen when
Linus accepts a patch.

Really, Linus wants to push some of his maintanance overhead to distros,
who get paid to do it, but also to provide sexy bullet point items for
users, so they buy "Linux" stuff.

You try to find a better balance.

--
Shawn Leas
core@enodev.com

I installed a skylight in my apartment...
The people who live above me are furious!
						-- Stephen Wright

Re: What's left over.

[Oliver Xymoron]

On Thu, Oct 31, 2002 at 03:59:34PM -0500, Dave Anderson wrote:
>
> > To me this says "LKCD is stupid". Which means that I'm not going to apply
> > it, and I'm going to need some real reason to do so - ie being proven
> > wrong in the field.
> >
> > (And don't get me wrong - I don't mind getting proven wrong. I change my
> > opinions the way some people change underwear. And I think that's ok).
> 
> It would be most unfortunate if the existance of netdump is used as a
> reason to deny LKCD's inclusion, or to simply dismiss LKCD as stupid.

What he really wants is for Andrew or Alan or someone else he trusts
to merge it, get actual field results, and declare it useful. If
people start visibly passing around crash dump results on l-k and
solving problems with them, that'll help too. Until then all he has is
his gut feel to go on.

-- 
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.." 

Re: What's left over.

[Werner Almesberger]

[ Cc: trimmed ]

john stultz wrote:
> groups for each project, I have no clue how anyone would be able to
> handle the (unix)group management required. ACLs let the users
> themselves manage what people got what access to their data.

Note that POSIX ACLs don't seem to solve this either: they only
let you control access in terms of existing users or groups.

IMHO, this is one of the standard pitfalls of ACLs: if they don't
let you aggregate information, you quickly end up with huge ACLs
hanging off every file, and each of those ACLs wants to be
carefully maintained. I've seen a lot of this in my VMS days.
(Unix is a bit better, because you can control access at a
directory level, while VMS needs the ACL on each file, because
you can open files directly by VMS' equivalent to an inode
number, without traversing the directory hierarchy. Of course,
many users didn't know that :-)

To make ACLs truly scalable, it would be nice to be able to
express permissions in terms of access to other filesystem
objects. E.g. "everybody who can read file ~me/acls/my_friends
can write the directory on which this ACE hangs". This should
work like a symlink, i.e. if I add new friends to my_friends, I
don't have to update all my ACLs.

- Werner

-- 
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         wa@almesberger.net /
/_http://www.almesberger.net/____________________________________________/

Re: What's left over.

[Rusty Russell]

In message <3DC171FF.5000803@nortelnetworks.com> you write:
> Ideally I would like to see a dump framework that can have a number of 
> possible dump targets.  We should be able to dump to any combination of 
> network, serial, disk, flash, unused ram that isn't wiped over restarts, 
> etc...

Both the lkcd and ide mini-oopser have that (although the mini-oopser
has only x86-ide for now).

The mini-oopser has different aims than LCKD: they want to debug one
system, I want to make sure we're reaping OOPS reports from those 99%
of desktop users who run X and simply reboot when their machine
crashes once a month.

I did *not* put the mini-oopser on the Snowball list, because I don't
have time to polish it.

Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: What's left over.

[James Simmons]

> > > On Thu, 31 Oct 2002, Rusty Russell wrote:
> > > > Fbdev Rewrite
> > >
> > > This one is just huge, and I have little personal judgement on it.
> >
> > It's been around for a while.  Geert, Russell?
>
> It's huge because it moves a lot of files around:
>   1. drivers/char/agp/ -> drivers/video/agp/
>   2. drivers/char/drm/ -> drivers/video/drm/
>   3. console related files in drivers/video/ -> drivers/video/console/
>
> (1) and (2) should be reverted, but apparently they aren't reverted in the
> patch at http://phoenix.infradead.org/~jsimmons/fbdev.diff.gz yet. The patch
> also seems to remove some drivers. Haven't checked the bk repo yet.
>
> James, can you please fix that (and the .Config files)?

Done. I have a new version of that patch at the same place. It is against
2.5.45.

http://phoenix.infradead.org/~jsimmons/fbdev.diff.gz

Its still pretty big. We can save the moving of the agp code for post
halloween.

Re: What's left over.

[Rusty Russell]

In message <20021031101558.GB7487@fib011235813.fsnet.co.uk> you write:
> On Thu, Oct 31, 2002 at 02:00:31PM +1100, Rusty Russell wrote:
> > They have, IIRC.  Interestingly, it was less invasive (existing source
> > touched) than the LVM2/DM patch you merged.
> 
> FUD.  I added to three areas of existing code:

[ 40-line detailed explanation snipped ]

Woah!  War's over dude!  We won!

I used Rusty's Unreliable Intrusiveness-o-meter (number of existing
non-config files touched), as I said.

I didn't read code or anything so unscientific or accurate.  But both
DM and EVMS were way down on the "intrusiveness" list.

Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: What's left over.

[john stultz]

On Wed, 2002-10-30 at 19:28, Stephen Frost wrote:
> The feeling I got on this was the ability to let users define their own
> groups.  Perhaps I'm not following it closely enough but that was the
> impression I got in terms of "what this does for us"; I'm probably
> missing other things.  Just that ability would be nice in my view
> though.  Isn't it something that's been in AFS for a long time too?
> I've got a few friends who've played with AFS before (at CMU and the
> like) and really enjoyed the ACLs there.

Yea, I haven't looked at the submitted implementation, but at CMU ACLs
were critical to be able to selectively share data between a very large
set of users w/o bugging an administrator. Given multiple classes per
semester which had multiple group projects, where you may have different
groups for each project, I have no clue how anyone would be able to
handle the (unix)group management required. ACLs let the users
themselves manage what people got what access to their data.

How else can I fix my partner's bugs (or vice-versa), give the clumsy TA
read only access, and let the cheat across the hall figure it out for
himself? (There may very well be a good solution to this w/o ACLs but
I've not seen it in use.)

So yea, I'd love to see a common ACLs API.
-john

Re: What's left over.

[Jeff Garzik]

Linus Torvalds wrote:

>	In particular when it comes to this project, I'm told about
>	"netdump", which doesn't try to dump to a disk, but over the net.
>	And quite frankly, my immediate reaction is to say "Hell, I
>	_never_ want the dump touching my disk, but over the network
>	sounds like a great idea".
>  
>

[yes, I realize the LKCD merge debate is over, bear with me :)]

I'm sort of in the middle on this issue:  The existence of netdump does 
not imply that disk dumps are a bad thing.

netdumps require a net dump server, and it is simply not realistic at 
all to assume that users seeing crashes will always have a netdump 
server set up in advance, or even have multiple machines to make that 
possible.  Disk dumps are valuable because their requirements are very 
low, and because of all the user-support reasons that Andrew Morton 
mentioned in this thread.

That said, I used to be an LKCD cheerleader until a couple people made 
some good points to me:  it is not nearly low-level enough to truly be 
of use in crash situations.  netdump can work if your interrupts are 
hosed/screaming, and various mid-layers are dying.  For LKCD to be of 
any use, it needs to _skip_ the block layer and talk directly to 
low-level drivers.

So, I think the stock kernel does need some form of disk dumping, 
regardless of any presence/absence of netdump.  But LKCD isn't there yet...

    Jeff

Re: What's left over.

[Dave Anderson]

On Thu, 31 Oct 2002, Linus Torvalds wrote:

>  - included features kill off (potentially better) projects.
>
>         There's a big "inertia" to features. It's often better to keep
>         features _off_ the standard kernel if they may end up being
>         further developed in totally new directions.
>
>         In particular when it comes to this project, I'm told about
>         "netdump", which doesn't try to dump to a disk, but over the net.
>         And quite frankly, my immediate reaction is to say "Hell, I
>         _never_ want the dump touching my disk, but over the network
>         sounds like a great idea".
>
> To me this says "LKCD is stupid". Which means that I'm not going to apply
> it, and I'm going to need some real reason to do so - ie being proven
> wrong in the field.
>
> (And don't get me wrong - I don't mind getting proven wrong. I change my
> opinions the way some people change underwear. And I think that's ok).

It would be most unfortunate if the existance of netdump is used as a
reason to deny LKCD's inclusion, or to simply dismiss LKCD as stupid.

On Thu, 31 Oct 2002, Matt D. Robinson wrote:

> We want to see this in the kernel, frankly, because it's a pain
> in the butt keeping up with your kernel revisions and everything
> else that goes in that changes.  And I'm sure SuSE, UnitedLinux and
> (hopefully) Red Hat don't want to spend their time having to roll
> this stuff in each and every time you roll a new kernel.

While Red Hat advocates Ingo's netdump option, we have customer
requests that are requiring us to look at LKCD disk-based dumps as an
alternative, co-existing dump mechanism.  Since the two methods are not mutually
exclusive, LKCD will never kill off netdump -- nor certainly vice-versa.  We're
all just looking for a better means to be able to
provide support to our customers, not to mention its value as a
development aid.

Dave Anderson
Red Hat, Inc.

Re: What's left over.

[Jeff Garzik]

Alan Cox wrote:

>On Thu, 2002-10-31 at 18:28, Nicholas Wourms wrote:
>  
>
>>>problems most people don't have?  What next, some kind of misdesigned
>>>in-kernel CryptoAPI?
>>>      
>>>
>>Get over it!  If you haven't noticed, CryptoAPI is merged already.  The only 
>>    
>>
>
>Chris is write that crypto api is misdesigned if we want to use hardware
>cryptocards
>  
>

I'll reserve judgement until we actually get access to some decent [made 
in the past few years] hardware crypto cards, and take a hard look at 
their PCI bus utilization... until then it is mostly vague handwaving...

[vendors - any takers?]

Re: What's left over.

[Bernhard Kaindl]

On Thu, 31 Oct 2002, Andrew Morton wrote:
>
> We'll be spending the next six months stabilising and hardening
> the used-to-be-2.5 kernel.  If grunts like me can get hold a
> copy of the other person's kernel image from time-of-crash, that
> has a ton of value.

Exactly, sometimes you don't even need the dump itself, The user
who has the dump just types lcrash and report -w file.txt and
lcrash writes a consolidated report with the most interesting
information from the dump to the file.txt and he can sent it
to you and you've much information you often miss in problem
reports.

The report consists of: time when the dump was created, time
when the report was created, the architecture, the hostname,
kernel version and compile time, the kernel (dmesg) buffer
with all the oopses logged into it, a short task list with
process adress, id's, state, flags, cpu and process name,
and finally a full CPU dump of every CPU with all registers,
current process and function and a symbolic stack backtrace
of the CPU.

Sometimes this is all you need to know and if you need to
know e.g. the stack backtrace of a not running process at
the time of the dump, you can ask the user to simply run
trace <process address> and lcrash gives you the backtrace
of this process:

lcrash> t[race] 0x1408000
================================================================
STACK TRACE FOR TASK: 0x1408000 (kjournald)

 STACK:
 0 schedule+894 [0x3164e]
 1 interruptible_sleep_on+174 [0x31eae]
 2 journal_revoke+<ERROR> [0x10889c0c]
 3 kernel_thread+70 [0x18c1e]

showing the full task scruct, a sub-struct or a field is also simple:

p[rint] ((struct task_struct *)0x1408000)->pending
struct sigpending {
        head = (nil)
        tail = 0x1598700
        signal = sigset_t {
                sig = {
                        [0] 0
                }
        }
}

"feels" a bit like gdb

> (Disclaimer: I've never used lkcd.  I'm assuming that it's
> possible to gdb around in a dump)

I don't know if there is an lkcd->ELF core converter yet, but
it should be doable. However, lcrash is quite powerful, it comes
with sial, an integrated C interpreter that permits easy access to the
symbol and type information, obviosly, it allows to write code like this:

        void
        showprocs()
        {
        	struct proc* p;
                for(p=*(struct proc**)procs; p; p=p->p_next)
                        do something...
                }
        }

Looks nice... :-)

I also don't know if (k)gdb knows about tasks, lcrash at least
knows about them and this may when you look into a specific
task(I'm not an expert)

Of cource lcrash can do dissembing, find symbols,
> So.  _If_ lkcd gives me gdb-able images from time-of-crash, I'd
> like it please.  And I'm the grunt who spent nearly two years
> doing not much else apart from working 2.3/2.4 oops reports.

Maybe the lkcd people can do so, but I think they can also give
a hands-on workshop to lcrash.

You can use lcrash also on running system to browse around,
learn and save dumps from it without interrupting it, you
just need lcrash, the System.map and the Kerntypes file from
kernel for using it.

> Oh, and as Rusty has pointed out, we lose a *lot* of oops reports
> because users are in X and the backtrace doesn't make it to the
> logs.

Yep, I think it would be good even if Linus just accepts the
infrastructure patch of lkcd which needs to be in the kernel,
the vafourite dump method module can then be downloaded, compiled
installed and configured without much pain, I think that people
can start using it in a broader range without the hassle of
needing to patching and booting a special kernel.

Bernd

PS: lcrash is only one of the many frontends, as I've read in
this thread, there is also Dave Anderson's "crash" tool which
works with LKCD dumps, netdump dumps, etc. There is also qlcrash,
an qt frontend for lcrash for people who like to click!

Re: What's left over.

[Chris Wedgwood]

On Thu, Oct 31, 2002 at 10:49:10AM -0800, Linus Torvalds wrote:

> Any hardware that needs to go off and think about how to encrypt
> something sounds like it's so slow as to be unusable. I suspect that
> anything that is over the PCI bus is already so slow (even if it
> adds no extra cycles of its own) that you're better off using the
> CPU for the encryption rather than some external hardware.

Except almost all hardware out there that does this stuff is async to
some extent...

I'm just speaking as someone who has (sadly) done this a couple of
times already for commercial real-world products.  I'm no expert, I
don't claim to be and admit there is still plenty to learn...

... that said, having access to lots of hardware, both our own and
other peoples, almost all of it needs to be driven asynchronously to
get good performance (or by a large number of threads).

Re: What's left over.

[Michael Shuey]

On Thu, Oct 31, 2002 at 07:04:31PM +0000, Alan Cox wrote:
> On Thu, 2002-10-31 at 17:13, Michael Shuey wrote:
> > I'm a user, and I request that LKCD get merged into the kernel. :-)
> > Do you feel like donating a 700-port console server?  Right, so it's LKCD
> > for me then.
> 
> Wouldn't you rather they neatly tftp'd dumps to a nominated central
> server which noticed the arrival, did the initial processing with a perl
> script and mailed you a summary ?

Generally speaking, no.

A tftp server doesn't provide enough security (specifically authentication).
It would need to be accessible from clusters in multiple buildings and on
multiple networks (some of which must be public).

I've seen more network adapter issues than drive controller issues.  In
particular, some vendors (Compaq, listen up) can't implement an eepro100 to
save their asses, especially on older hardware.

>From time to time bandwidth issues and/or network splits can prevent dumps
from being reliably delivered.

Right now we use the presence of a local dump to indicate that a machine
should not join the PBS pool (and begin to run more jobs) on a reboot.  I'd
rather not have the nodes check a central server to see if it's okay to run
jobs.  And no, I don't want machines to stay down after a crash - many nodes
are in distant corners of campus and it's cold outside. :-)  If I can fix the
problem through software I'd prefer that the problematic host be up, rather
than having to walk over to it just to hit reset and load a new kernel.

That said, it would be really nice if LKCD would log dumps to both the swap
device and to a remote server.  That way if the machine crashed because of
disk failure I'd still have an uncorrupted dump image (and could then notice
all the little errors coming back out of the swap device).  A tool to
automatically analyze a dump and email back summaries would be much more
useful, though.  If someone were to write such a widget, that'd be swell. :-)

Right now I'm less concerned with getting dumps to exactly the right place
and a bit more concerned with getting dumps in the main kernel at all.

-- 
Mike Shuey

Re: What's left over.

[Alan Cox]

On Thu, 2002-10-31 at 18:28, Nicholas Wourms wrote:
> > problems most people don't have?  What next, some kind of misdesigned
> > in-kernel CryptoAPI?
> 
> Get over it!  If you haven't noticed, CryptoAPI is merged already.  The only 

Chris is write that crypto api is misdesigned if we want to use hardware
cryptocards

Re: What's left over.

[Nicholas Wourms]

Alan Cox wrote:
> On Thu, 2002-10-31 at 18:28, Nicholas Wourms wrote:
> 
>>>problems most people don't have?  What next, some kind of misdesigned
>>>in-kernel CryptoAPI?
>>
>>Get over it!  If you haven't noticed, CryptoAPI is merged already.  The only 
> 
> 
> Chris is write that crypto api is misdesigned if we want to use hardware
> cryptocards
> 

Alan,

Thanks for setting me straight, your assertion is correct, 
of course.  I was under the impression that the CryptoAPI 
code was merged initially for IPSEC support and would be 
revamped and expanded at a later date to support a wide 
variety of interfaces?

Cheers,
Nicholas

Re: What's left over.

[Nicholas Wourms]

Alexander Viro wrote:
> 
> On Thu, 31 Oct 2002, Nicholas Wourms wrote:
> 
> 
>>slow or you are out of diskspace, too bad!  I'm sure I'm not the only one 
>>who is tired of hearing people whine about "bloat" wrt the sources and 
>>demanding that features they don't use be ignored.  No one (non-core) 
> 
> 
> One look at the From:
> understanding has blossomed
> .procmailrc grows
> 

Your point is?

Re: What's left over.

[Alan Cox]

On Thu, 2002-10-31 at 17:13, Michael Shuey wrote:
> I'm a user, and I request that LKCD get merged into the kernel. :-)
> Do you feel like donating a 700-port console server?  Right, so it's LKCD
> for me then.

Wouldn't you rather they neatly tftp'd dumps to a nominated central
server which noticed the arrival, did the initial processing with a perl
script and mailed you a summary ?

Re: What's left over.

[Alexander Viro]

On Thu, 31 Oct 2002, Nicholas Wourms wrote:

> slow or you are out of diskspace, too bad!  I'm sure I'm not the only one 
> who is tired of hearing people whine about "bloat" wrt the sources and 
> demanding that features they don't use be ignored.  No one (non-core) 

One look at the From:
understanding has blossomed
.procmailrc grows

Re: What's left over.

[Alan Cox]

On Thu, 2002-10-31 at 18:10, Chris Friesen wrote:
> > To me this says "LKCD is stupid". Which means that I'm not going to apply 
> > it, and I'm going to need some real reason to do so - ie being proven 
> > wrong in the field.
> 
> How do you deal with netdump when your network driver is what caused the 
> crash?

Netdump drives the system itself. Any dump driver has to as it cant
assume the system is in a remotely sane state

Re: What's left over.

[Rik van Riel]

On Thu, 31 Oct 2002, Linus Torvalds wrote:

> 	In particular when it comes to this project, I'm told about
> 	"netdump", which doesn't try to dump to a disk, but over the net.

And guess what ?   Netdump is one of various LKCD dump methods ...

regards,

Rik
-- 
A: No.
Q: Should I include quotations after my reply?

http://www.surriel.com/		http://distro.conectiva.com/

Re: What's left over.

[Linus Torvalds]

On Thu, 31 Oct 2002, Chris Wedgwood wrote:
> 
> It's synchronous and assume everything is synchronous.  Lots of
> hardware (most) doesn't work that way.

Think of it another way: many users will likely _require_ atomic
encryption / decryption (done in softirq contexts etc), and thus a 
synchronous interface. Also, it simplifies the code and makes it more 
efficient.

Any hardware that needs to go off and think about how to encrypt something
sounds like it's so slow as to be unusable. I suspect that anything that
is over the PCI bus is already so slow (even if it adds no extra cycles of
its own) that you're better off using the CPU for the encryption rather
than some external hardware.

In short, from what I can tell, there is no huge actual reason to ever
allow a asynchronous interface. Such interfaces are likely fine for things
like network cards that can do encryption on their own on outgoing or
incoming packets, but that is not a general-purpose encryption engine, and
would not merit being part of an encryption library anyway.

[ Such a card is just a way to _avoid_ using the encryption library - the
  same way we can avoid using the checksumming stuff for network cards 
  that can do their own checksums ]

We'll see. I'd rather have a simpler interface that works for all relevant
cases today, and then if external crypto chips end up being common and
sufficiently efficient, we can always re-consider. Are the DMA-over-PCI
roundtrip (and resulting cache invalidations) overheads really worth the
extra hardware?

		Linus

Re: What's left over.

[John Alvord]

On Thu, 31 Oct 2002 09:54:54 -0800 (PST), Linus Torvalds
<torvalds@transmeta.com> wrote:


>Guys, why do you even bother trying to convince me? If you are right, you 
>will be able to convince other people, and that's the whole point of open 
>source.
>
>Being "vendor-driven" is _not_ a bad thing. It only means that _I_ am not
>personally convinced. I'm only one person.

It sounds to me like there needs to be L-K traffic when problems are
solved using LKCD.

Personally I love crash dumps... in 33 years of computing I have spent
a total of 1-2 years doing nothing but enhancing and developing
post-processing facilities. The true benefit is not just the "crashed
here, add a null check nonsense". It is the ability to examine the
whole system state. With an inboard trace table, you can even go back
in time. You can look at call stacks, locks held, state of allocated
memory, etc etc. If you save callstacks and time with allocated
memory, you can track down storage growth problems. I have spent weeks
winkling problems out of crash dumps, solving problems the developers
didn't even know existed.

With the right facility you can take crash dump snapshots and keep on
running. It is a great tool for understanding a system.

But until there is a flow of results - good quality fixes - resulting
from such analysis, I can see exactly why LT is doubtful. 

john alvord

Re: What's left over.

[Nicholas Wourms]

Chris Wedgwood wrote:

> On Wed, Oct 30, 2002 at 11:48:23PM -0700, Dax Kelson wrote:
> 
>> Technically speaking you can achieve ACL like permissions/behavior
>> using the historical UNIX security model by creating a group EACH
>> time you run into a unique case permission scenario.
> 
> I'm not arguing against this... I'm claiming POSIX ACLs are mostly
> brain-dead and almost worthless (broken by committee pressure and too
> many people making stupid concessions).
> 
> If we must have ACLs, why not do it right?
> 
>> Without ACLs, if Sally, Joe and Bill need rw access to a file/dir,
>> just create another group with just those three people in.  Over
>> time, of course, this leads to massive group proliferation.  Without
>> Tim Hockin's patch, 32 groups is maximum number of groups a user can
>> be a member of.
> 
> How many people actually need this level of complexity?
> 
> Why are we adding all this shit and bloat because of perceived
> problems most people don't have?  What next, some kind of misdesigned
> in-kernel CryptoAPI?

Get over it!  If you haven't noticed, CryptoAPI is merged already.  The only 
bloat ACLs cause is the size of the source tarball.  If your connection is 
slow or you are out of diskspace, too bad!  I'm sure I'm not the only one 
who is tired of hearing people whine about "bloat" wrt the sources and 
demanding that features they don't use be ignored.  No one (non-core) 
feature will be useful to everyone, that is a given fact.  The point is 
that while you see no use for it, there are many others out there who do.  
ACLs are something which have existed in the Solaris/BSD world for a long 
time now, and people who have admin these boxen find ACLs to be quite 
useful.

Cheers,
Nicholas

Re: What's left over.

[Linus Torvalds]

On Thu, 31 Oct 2002, Oliver Xymoron wrote:
> 
> Perhaps not the best analogy.

Heh. I like my analogies bad. The best analogies should make you go 
"huh!" - kind of like a pink poodle in a tutu.

		Linus

Re: What's left over.

[Linus Torvalds]

On Thu, 31 Oct 2002, Chris Friesen wrote:
> 
> How do you deal with netdump when your network driver is what caused the 
> crash?

Actually, from a driver perspective, _the_ most likely driver to crash is 
the disk driver. 

That's from years of experience. The network drivers are a lot simpler,
the hardware is simpler and more standardized, and doesn't do as many
things. It's just plain _easier_ to write a network driver than a disk
driver.

Ask anybody who has done both.

But that's not the real issue. The real issue is that I have no personal
incentives to try to merge the thing, and as a result I think I'm the
wrong person to do so. I've told people over and over again that I think
this is a "vendor merge", and I'm fed up with people not _getting_ it.

Don't bother to ask me to merge the thing, that only makes me get even
more fed up with the whole discussion. This is open source, guys. Anybody 
can merge it. Because I don't particularly believe in it doesn't mean that 
it cannot be used. It only means that I want to see users flock to it and 
show my beliefs wrong.

		Linus

Re: What's left over.

[Patrick Finnegan]

On Thu, 31 Oct 2002, Linus Torvalds wrote:

>
> On Thu, 31 Oct 2002, Matt D. Robinson wrote:
> >
> > This isn't bloat.  If you want, it can be built as a module, and
> > not as part of your kernel.  How can that be bloat?
>
> I don't care one _whit_ about the size of the binary. I don't maintain
> binaries, adn the binary can be gigabytes for all I care.
>
> The only thing I care about is source code. So the "build it as a module
> and it is not bloat" argument is a total nonsense thing as far as I'm
> concerned.

So, you don't like bloat, such as having 22 different file systems (only
including the ones that can be placed on disk, not things like devfs or
smbfs...). That's more filesystems than I have dollars in my wallet at
the moment.   For the amount of utility that this code provides, it's
definately not 'bloat'.

> Anyway, new code is always bloat to me, unless I see people using them.

HEY!!! WE'RE USING IT!!!

> Guys, why do you even bother trying to convince me? If you are right, you
> will be able to convince other people, and that's the whole point of open
> source.

Now this sounds more like something I'd hear from Sun trying to get a fix
for a version of Solaris without having to buy a new one.  I thought the
whole point of Free Software was sharing with the community, and doing
what's best for the community.

> Being "vendor-driven" is _not_ a bad thing. It only means that _I_ am not
> personally convinced. I'm only one person.

That's the same as claiming that George W. Bush is just one person....

So I'll plea yet again, please add LKCD!

Pat
--
Purdue Universtiy ITAP/RCS
Information Technology at Purdue
Research Computing and Storage
http://www-rcd.cc.purdue.edu

http://dilbert.com/comics/dilbert/archive/images/dilbert2040637020924.gif

Re: What's left over.

[Oliver Xymoron]

On Thu, Oct 31, 2002 at 09:25:21AM -0800, Linus Torvalds wrote:
> (And don't get me wrong - I don't mind getting proven wrong. I change my 
> opinions the way some people change underwear. And I think that's ok).

As in 'sometimes not even when hundreds of people start haranguing me
about it in public forums'? 

Perhaps not the best analogy.

-- 
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.." 

Re: What's left over.

[Andrew Morton]

Linus Torvalds wrote:
> 
> [ lkcd ]
>

We'll be spending the next six months stabilising and hardening
the used-to-be-2.5 kernel.  If grunts like me can get hold a
copy of the other person's kernel image from time-of-crash, that
has a ton of value.

(Disclaimer: I've never used lkcd.  I'm assuming that it's
possible to gdb around in a dump)

>         In particular when it comes to this project, I'm told about
>         "netdump", which doesn't try to dump to a disk, but over the net.

It could help.  But like serial console, the random person whose
kernel just died often can't be bothered setting it up, or simply
doesn't have the gear, or the crash is not repeatable.


So.  _If_ lkcd gives me gdb-able images from time-of-crash, I'd
like it please.  And I'm the grunt who spent nearly two years
doing not much else apart from working 2.3/2.4 oops reports.


Oh, and as Rusty has pointed out, we lose a *lot* of oops reports
because users are in X and the backtrace doesn't make it to the
logs.  Rusty has a little app which dumps just the oops report to
disk somewhere.    Want that too.

Re: What's left over.

[Chris Wedgwood]

On Thu, Oct 31, 2002 at 09:31:09AM -0500, Jeff Garzik wrote:

> What's wrong with our current 2.5.45 crypto api?

It's synchronous and assume everything is synchronous.  Lots of
hardware (most) doesn't work that way.


  --cw

Re: What's left over.

[Chris Friesen]

Linus Torvalds wrote:

> 	In particular when it comes to this project, I'm told about
> 	"netdump", which doesn't try to dump to a disk, but over the net.
> 	And quite frankly, my immediate reaction is to say "Hell, I
> 	_never_ want the dump touching my disk, but over the network
> 	sounds like a great idea".
> 
> To me this says "LKCD is stupid". Which means that I'm not going to apply 
> it, and I'm going to need some real reason to do so - ie being proven 
> wrong in the field.

How do you deal with netdump when your network driver is what caused the 
crash?

Ideally I would like to see a dump framework that can have a number of 
possible dump targets.  We should be able to dump to any combination of 
network, serial, disk, flash, unused ram that isn't wiped over restarts, 
etc...

Chris

-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: What's left over.

[Nicholas Wourms]

Trever L. Adams wrote:

> On Wed, 2002-10-30 at 21:31, Linus Torvalds wrote:
> 
>> > ext2/ext3 ACLs and Extended Attributes
>> 
>> I don't know why people still want ACL's. There were noises about them
>> for samba, but I'v enot heard anything since. Are vendors using this?
>> 
> 
> I am sure I don't count (not being a vendor), but Intermezzo offers
> support for this (they are waiting on feature freeze to redo it to 2.5
> according to an email I have).  I want this stuff.  Yes, u+g+w is nice,
> but good ACLs are even better.  Please, if this is technically correct
> in implementation, do put it in.
> 

I agree, having them is far better then the standard u+g+w that's been 
around for ages.  I think it gives the "finer" grain of control over your 
system that a lot of users may desire.  Not to mention the fact that ACL's 
are well supported by the recently merged XFS.  If I'm not mistaken, AFS 
uses them as well.  I *really* don't see the overhead cost here in terms of 
compiled kernel size when they are turned off.  As for the size of the 
source tarball, who cares?  People should quit whining about the size of 
the sources and get over it!  Storage is cheap and broadband is in 
widespread use.

Cheers,
Nicholas

Re: What's left over.

[Oliver Xymoron]

On Thu, Oct 31, 2002 at 09:38:41AM -0800, Linus Torvalds wrote:
> 
> Note that as far as ACL's go, enough people have convinced me that we want 
> them, with clear real-life issues. So don't worry about them, I'll merge 
> it.

Ok, so now lets work on a Documentation/filesystems patch pointing
out a few of the common pitfalls, as I definitely agree they invite
some grave mistakes and are best avoided in most scenarios.

- /tmp-style symlink issues on shared directories
- vast majority of software (including security tools) ACL-unaware
- much harder to check for correctness

Al, I'm sure you have more..

-- 
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.." 

Re: What's left over.

[Linus Torvalds]

On Thu, 31 Oct 2002, Matt D. Robinson wrote:
> 
> This isn't bloat.  If you want, it can be built as a module, and
> not as part of your kernel.  How can that be bloat? 

I don't care one _whit_ about the size of the binary. I don't maintain 
binaries, adn the binary can be gigabytes for all I care.

The only thing I care about is source code. So the "build it as a module 
and it is not bloat" argument is a total nonsense thing as far as I'm 
concerned. 

Anyway, new code is always bloat to me, unless I see people using them.

Guys, why do you even bother trying to convince me? If you are right, you 
will be able to convince other people, and that's the whole point of open 
source.

Being "vendor-driven" is _not_ a bad thing. It only means that _I_ am not
personally convinced. I'm only one person.

		Linus

Re: What's left over.

[Matt D. Robinson]

On Thu, 31 Oct 2002, Linus Torvalds wrote:
|>[ Ok, this is a really serious email. If you don't get it, don't bother 
|>  emailing me. Instead, think about it for an hour, and if you still don't 
|>  get it, ask somebody you know to explain it to you. ]

Thanks for the response.  I don't think I need an hour.  This is
pretty simple.

|>On Thu, 31 Oct 2002, Matt D. Robinson wrote:
|>> 
|>> Sure, but why should they have to?  What technical reason is there
|>> for not including it, Linus?
|>
|>There are many:
|>
|> - bloat kills:
|>
|>	My job is saying "NO!"
|>
|>	In other words: the question is never EVER "Why shouldn't it be
|>	accepted?", but it is always "Why do we really not want to live 
|>	without this?"

This isn't bloat.  If you want, it can be built as a module, and
not as part of your kernel.  How can that be bloat?  People who
build kernels can optionally build it in, but we're not asking
that it be turned on by default, rather, built as a module so
people can load it if they want to.  We made it into a module
because 18 months ago you complained about it being bloat.  We
addressed your concerns.

Some people, particularly large SSI configurations, can't live
without this.  You shouldn't crash once.  Crashing twice, or
more often, is inexcusable.

|> - included features kill off (potentially better) projects.
|>
|>	There's a big "inertia" to features. It's often better to keep 
|>	features _off_ the standard kernel if they may end up being
|>	further developed in totally new directions.

I can't argue against this ... to do so would mean that you don't
accept any new features for 2.5, and there are a lot of projects
like mine that need to go in, although I do understand your concerns.

|>	In particular when it comes to this project, I'm told about
|>	"netdump", which doesn't try to dump to a disk, but over the net.
|>	And quite frankly, my immediate reaction is to say "Hell, I
|>	_never_ want the dump touching my disk, but over the network
|>	sounds like a great idea".

We've integrated the "netdump" capabilities as a dump method
for LKCD.  It's an option for dumping, just like all the other
dump methods available to people?  Want to dump to disk?  Use
LKCD.  Want to dump on the network?  USE LKCD.  What's wrong
with that?

We've created a net dump method that allows you to dump across the
network from Mohammed Abbas (modified from Ingo's netconsole dump).
It integrates into LKCD beautifully.  If you want that patch with
the rest of our LKCD patches, we can include it, no problem.

|>To me this says "LKCD is stupid". Which means that I'm not going to apply 
|>it, and I'm going to need some real reason to do so - ie being proven 
|>wrong in the field.

Hopefully some of this changes your mind.

|>(And don't get me wrong - I don't mind getting proven wrong. I change my 
|>opinions the way some people change underwear. And I think that's ok).
|>
|>> I completely don't understand your reasoning here.
|>
|>Tough. That's YOUR problem.

It is.  I lose sleep because this is my problem.  I lose time on
the weekends because this is my problem.

If you've _reviewed_ the LKCD patches and still have the opinions
you've mentioned above, then I'll consider this your position and
be done with it.  Otherwise, please accept the code.

We'll keep doing our best to keep up with your kernels in the
meantime.

|>		Linus

--Matt

Re: What's left over.

[Linus Torvalds]

On Thu, 31 Oct 2002, Alexander Viro wrote:
> 
> 	No.  I'm saying that ACLs do not have a point until at least basic
> userland gets ready for setups people want ACLs for.  Adding features that
> can't be used until $BIG_WORK is done is idiocy in the best case and
> danger in the worst.  Especially since $BIG_WORK does not depend on these
> features.

I think samba alone counts as enough user-land usage. 

And if it turns out nobody else ever wants to use them, that's fine too.

		Linus

Re: What's left over.

[Linus Torvalds]

Note that as far as ACL's go, enough people have convinced me that we want 
them, with clear real-life issues. So don't worry about them, I'll merge 
it.

		Linus

Re: What's left over.

[Richard Gooch]

Alexander Viro writes:
> On Thu, 31 Oct 2002, Stephen Wille Padnos wrote:
> 
> > >Then give them all the same account and be done with that.  Effect will
> > >be the same.
> > 
> > Unless I'm missing something, that only works if all the users need 
> > *exactly* the same permissions to all files, which isn't a good assumption.
> 
> That's the point.  In practice shared writable access to a directory
> can be easily elevated to full control of each others' accounts,
         ^^^^^^
While that may be true in theory, in practice it's not necessarily the
case. Many people don't have the expertise to make use of such
exploits. And before you say that they can download a pre-cooked
exploit kit, let me tell you that there are plenty of people who don't
have the time or inclination to do that.

I've seen you talk about these kinds of things before, and you always
seem to be talking about the typical nightmarish undergrad CS lab
where the kids spend all their time trying to crack each other and the
system. And I'm not saying that these don't exist: I've seen it.

But there are other environments (say a research lab with grad
students, post-docs and faculty) where the inhabitants either don't
have the skills or don't have the interest in cracking accounts.
Everyone is too busy doing their own research. Cracking the mysteries
of the universe seems to be more interesting.

So group write access and ACL's *can* lead to wanton cracking, but for
many environments it's not an issue. For many, the dangers lie outside
the firewall, not inside.

Note that I'm not specifically advocating ACL's, I'm just letting you
know that the problem you're concerned about is, for good reason, not
a problem for everyone.

I will note that one appealing aspect of ACL's is that they do not
require administrator intervention. That's good for a user who just
wants to set something up without having to wait for the sysadmin.
It's also good for the sysadmin (excepting control freaks) who doesn't
want to do things that the users can (or should) actually be doing by
themselves.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: What's left over.

[Alexander Viro]

On Thu, 31 Oct 2002, Stephen Frost wrote:

> So you're not really arguing against ACLs, you're complaining that
> userspace is broken when there's shared write access.  That's fine,
> userspace should be fixed, inclusion of ACLs into the kernel shouldn't
> be denied because of this.  ACLs should be optional, of course, and if
> you want them some really noisy warnings about the problems of shared
> writeable area with current userspace tools.  Of course, that same
> warning should probably be included in 'groupadd'.

	No.  I'm saying that ACLs do not have a point until at least basic
userland gets ready for setups people want ACLs for.  Adding features that
can't be used until $BIG_WORK is done is idiocy in the best case and
danger in the worst.  Especially since $BIG_WORK does not depend on these
features.

Re: What's left over.

[Linus Torvalds]

[ Ok, this is a really serious email. If you don't get it, don't bother 
  emailing me. Instead, think about it for an hour, and if you still don't 
  get it, ask somebody you know to explain it to you. ]

On Thu, 31 Oct 2002, Matt D. Robinson wrote:
> 
> Sure, but why should they have to?  What technical reason is there
> for not including it, Linus?

There are many:

 - bloat kills:

	My job is saying "NO!"

	In other words: the question is never EVER "Why shouldn't it be
	accepted?", but it is always "Why do we really not want to live 
	without this?"

 - included features kill off (potentially better) projects.

	There's a big "inertia" to features. It's often better to keep 
	features _off_ the standard kernel if they may end up being
	further developed in totally new directions.

	In particular when it comes to this project, I'm told about
	"netdump", which doesn't try to dump to a disk, but over the net.
	And quite frankly, my immediate reaction is to say "Hell, I
	_never_ want the dump touching my disk, but over the network
	sounds like a great idea".

To me this says "LKCD is stupid". Which means that I'm not going to apply 
it, and I'm going to need some real reason to do so - ie being proven 
wrong in the field.

(And don't get me wrong - I don't mind getting proven wrong. I change my 
opinions the way some people change underwear. And I think that's ok).

> I completely don't understand your reasoning here.

Tough. That's YOUR problem.

		Linus

Re: What's left over.

[Matt D. Robinson]

On Thu, 31 Oct 2002, Linus Torvalds wrote:
|>On Wed, 30 Oct 2002, Matt D. Robinson wrote:
|>That's fine. And since they are paid to support it, they can apply the 
|>patches.  

Sure, but why should they have to?  What technical reason is there
for not including it, Linus?

I completely don't understand your reasoning here.  I use it for my
home, not for work, and that's important for me.  And not everyone
can spend their evenings rolling up the next set of patches for
a distribution.  Yes, vendors want it, they need it, but there are
plenty of people like me that want this in too!

We want to see this in the kernel, frankly, because it's a pain
in the butt keeping up with your kernel revisions and everything
else that goes in that changes.  And I'm sure SuSE, UnitedLinux and
(hopefully) Red Hat don't want to spend their time having to roll
this stuff in each and every time you roll a new kernel.

I mean, PLEASE, Linus, what do we have to do?  There are so many
interests in this stuff, and I really, truly don't get what's wrong
with putting this in the kernel?

Have you looked at it?  Have you looked at how it is now structure
to be non-invasive?  How it will allow other kernel developers to
generate their own dumping methods?  I mean, we sent you E-mails
weeks ago, and you didn't respond to any of them with even a word
of acknowledgement of receipt.

|>What I'm saying by "vendor driven" is that it has no relevance for the 
|>standard kernel, and since it has no relevance to that, then I have no 
|>incentives to merge it. The crash dump is only useful with people who 
|>actively look at the dumps, and I don't know _anybody_ outside of the 
|>specialized vendors you mention who actually do that.

I do.  Others like myself do.  And not just for development
purposes.  I don't like to see my system crash after installing one
of your new kernels and not be able to figure out what's wrong.
The nice thing is that LKCD there, it works, and I can just look
at the crash report instead of wishing that my console buffer
didn't just scroll off.  Oh, I know, I'll just wait for it to
happen again ... yeah, like that's real intelligent.

|>I will merge it when there are real users who want it - usually as a
|>result of having gotten used to it through a vendor who supports it. (And
|>by "support" I do not mean "maintain the patches", but "actively uses it"
|>to work out the users problems or whatever).
|>
|>Horse before the cart and all that thing.
|>
|>People have to realize that my kernel is not for random new features. The
|>stuff I consider important are things that people use on their own, or
|>stuff that is the base for other work. Quite often I want vendors to merge
|>patches _they_ care about long long before I will merge them (examples of
|>this are quite common, things like reiserfs and ext3 etc).

Other vendors have merged LKCD a long time ago and use it, and
expect it to be there.  And users like myself find it valuable on
their desktops, their servers, etc.  I mean, there's someone using
this at Purdue that's responded to you, just another kernel user
that likes to have this stuff there automatically.

|>THAT is what I mean by vendor-driven. If vendors decide they really want
|>the patches, and I actually start seeing noises on linux-kernel or getting
|>requests for it being merged from _users_ rather than developers, then
|>that means that the vendor is on to something.

TurboLinux, MonteVista, Veritas, SuSE, and UnitedLinux have LKCD.
With the most recent changes, I think Red Hat can put LKCD in now
such that it isn't invasive to their distribution.

I think SuSE has already expressed a desire to have this in.  If
you want to hear from others, I'll asked them to respond to you.

|>		Linus

--Matt

Re: What's left over.

[Michael Shuey]

I'm a user, and I request that LKCD get merged into the kernel. :-)

On Thu, Oct 31, 2002 at 07:46:08AM -0800, Linus Torvalds wrote:
> What I'm saying by "vendor driven" is that it has no relevance for the 
> standard kernel, and since it has no relevance to that, then I have no 
> incentives to merge it. The crash dump is only useful with people who 
> actively look at the dumps, and I don't know _anybody_ outside of the 
> specialized vendors you mention who actually do that.

I actively look at LKCD dumps.  I have no affiliation with SGI, IBM, or any
of the previously mentioned companies.  I'm not aware of any vendors providing
pre-patched kernels with LKCD; right now my only option for reasonable crash
data is to patch and build my own kernel.

> I will merge it when there are real users who want it - usually as a
> result of having gotten used to it through a vendor who supports it. (And
> by "support" I do not mean "maintain the patches", but "actively uses it"
> to work out the users problems or whatever).

Here at Purdue University we're building several Linux clusters.  LKCD is
most useful to help find in-kernel problems.  Most of the time our crashes
are due to a flakey stick of RAM or a dying disk (or controller), but LKCD
dumps are still useful.  With a crash dump I can analyze the cause of the
crash after the fact, but without a dump my only option to get _any_ crash
data is to leave a console plugged into each node of my clusters.

Do you feel like donating a 700-port console server?  Right, so it's LKCD
for me then.

> People have to realize that my kernel is not for random new features. The
> stuff I consider important are things that people use on their own, or
> stuff that is the base for other work. Quite often I want vendors to merge
> patches _they_ care about long long before I will merge them (examples of
> this are quite common, things like reiserfs and ext3 etc).
> 
> THAT is what I mean by vendor-driven. If vendors decide they really want
> the patches, and I actually start seeing noises on linux-kernel or getting
> requests for it being merged from _users_ rather than developers, then
> that means that the vendor is on to something.

I understand that Linux can't have random new features (especially going into
a feature-freeze).  However, any additions that provide better debugging info
are (in my opinion, at any rate) worth it.  Every other UNIX I've used (with
the possible exception of an early Ultrix) has some facility to inspect the
kernel - all have _at_least_ dumps that get written to a swap disk on a crash
and many have an in-core debugger.  Running gdb on a live kernel from a
remote machine isn't unheard of, at least with other OSes.  Unfortunately,
only aid you'll get in debugging a Linux kernel is the source code.  Sure,
you can add a mess of printk's all over suspect code, and yes, the console
gets a register dump on a panic, but that really isn't enough.  Some times
it's nice to be able to walk through the kernel's data structures and figure
out just what was going on when things died.  I get this with LKCD.

To that end, it'd be nice if the trace toolkit and SGI's kernel debugger were
added.  No, I haven't used them, but then I don't do much kernel development
either.  I'd bet that LTT and the kernel debugger would be very useful to
those who do, though.

-- 
Mike Shuey

Re: What's left over.

[Stephen Frost]

[-- Attachment #1: Type: text/plain, Size: 1280 bytes --]

* Alexander Viro (viro@math.psu.edu) wrote:
> On Thu, 31 Oct 2002, Stephen Wille Padnos wrote:
> > Unless I'm missing something, that only works if all the users need 
> > *exactly* the same permissions to all files, which isn't a good assumption.
> 
> That's the point.  In practice shared writable access to a directory can be
> easily elevated to full control of each others' accounts, since most of
> userland code is written in implicit assumption that nothing bad happens with
> directory structure under it.  And there is nothing kernel can do about that -
> attacker does action you had explicitly allowed and your program goes bonkers
> since it can't cope with that.  Mechanism used to allow that action doesn't
> enter the picture - be it ACLs, groups or something else.

So you're not really arguing against ACLs, you're complaining that
userspace is broken when there's shared write access.  That's fine,
userspace should be fixed, inclusion of ACLs into the kernel shouldn't
be denied because of this.  ACLs should be optional, of course, and if
you want them some really noisy warnings about the problems of shared
writeable area with current userspace tools.  Of course, that same
warning should probably be included in 'groupadd'.

	Stephen

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: What's left over.

[Patrick Finnegan]

On Thu, 31 Oct 2002, Linus Torvalds wrote:

>
> On Wed, 30 Oct 2002, Matt D. Robinson wrote:
>
> > Linus Torvalds wrote:
> > > > Crash Dumping (LKCD)
> > >
> > > This is definitely a vendor-driven thing. I don't believe it has any
> > > relevance unless vendors actively support it.
> >
> > There are people within IBM in Germany, India and England, as well as
> > a number of companies (Intel, NEC, Hitachi, Fujitsu), as well as SGI
> > that are PAID to support this.

To add to that list, here at Purdue University, we actively look at crash
dumps on other architectures, such as IBM AIX, and are starting to do the
same on Linux machines, after discovery of LKCD.

> What I'm saying by "vendor driven" is that it has no relevance for the
> standard kernel, and since it has no relevance to that, then I have no
> incentives to merge it. The crash dump is only useful with people who
> actively look at the dumps, and I don't know _anybody_ outside of the
> specialized vendors you mention who actually do that.

This has much relevance for the standard kernel, as much relevance as gdb
has for people using applications.  While a majority of non-techno-geek
end-users probably don't care about the patch, I'm certain that there are
plenty of organizations out there like Purdue that WANT lkcd to become a
standard part of the Linux kernel.   Until then, we're forced to do our
own kernel patching every time we push out a new kernel.

> I will merge it when there are real users who want it - usually as a
> result of having gotten used to it through a vendor who supports it. (And
> by "support" I do not mean "maintain the patches", but "actively uses it"
> to work out the users problems or whatever).

We actively use it.

> People have to realize that my kernel is not for random new features. The
> stuff I consider important are things that people use on their own, or
> stuff that is the base for other work. Quite often I want vendors to merge
> patches _they_ care about long long before I will merge them (examples of
> this are quite common, things like reiserfs and ext3 etc).

LKCD isn't a 'random new feature'.  It's something that is present in
nearly ever other "Unix" on the market. (Yes I know Unix != Linux).  It's
a feature that should have been integrated by now IMHO.

> THAT is what I mean by vendor-driven. If vendors decide they really want
> the patches, and I actually start seeing noises on linux-kernel or getting
> requests for it being merged from _users_ rather than developers, then
> that means that the vendor is on to something.

Again, we're the end-user, not the vendor, and we're trying to drive to
have it included.  I've talked with outher sys admins in my department
here at Purdue, and have gotten a unanimous response that "It would be a
good and useful feature to have."

Pat
--
Purdue Universtiy ITAP/RCS
Information Technology at Purdue
Research Computing and Storage
http://www-rcd.cc.purdue.edu

http://dilbert.com/comics/dilbert/archive/images/dilbert2040637020924.gif

Re: What's left over.

[Stephen Frost]

[-- Attachment #1: Type: text/plain, Size: 1240 bytes --]

* Oliver Xymoron (oxymoron@waste.org) wrote:
> On Wed, Oct 30, 2002 at 09:43:29PM -0500, Alexander Viro wrote:
> > Because People Are Stupid(tm).  Because it's cheaper to put "ACL support: yes"
> > in the feature list under "Security" than to make sure than userland can cope
> > with anything more complex than  "Me Og.  Og see directory.  Directory Og's.
> > Nobody change it".  C.f. snake oil, P.T.Barnum and esp. LSM users
> 
> It's nearly useless in a Unix-only context, true, however there's a rather
> serious impedance mismatch for serving files to Windows that this
> addresses. Emulating ACLs on the fly with groups to fit into the
> Windows model is mostly doable but ain't pretty. 

It's only nearly useless if you have some desire as an admin to
constantly be creating groups and changing group lists for users.  This
is not a feature which is useful only when serving files to Windows
machines, not even nearly.  AFS, Solaris, Irix etc have support for ACLs
and have a great deal of people who use them.  The simple yet common
situation of one user who wants to give even just read access to
another specific user for a given file is a pain in the ass to deal with
given the current structure.

	Stephen

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: What's left over.

[Alexander Viro]

On Thu, 31 Oct 2002, Stephen Wille Padnos wrote:

> >Then give them all the same account and be done with that.  Effect will
> >be the same.
> >  
> >
> 
> Unless I'm missing something, that only works if all the users need 
> *exactly* the same permissions to all files, which isn't a good assumption.

That's the point.  In practice shared writable access to a directory can be
easily elevated to full control of each others' accounts, since most of
userland code is written in implicit assumption that nothing bad happens with
directory structure under it.  And there is nothing kernel can do about that -
attacker does action you had explicitly allowed and your program goes bonkers
since it can't cope with that.  Mechanism used to allow that action doesn't
enter the picture - be it ACLs, groups or something else.

Re: What's left over.

[Dr. Greg Wettstein]

On Oct 30,  6:31pm, Linus Torvalds wrote:
} Subject: Re: What's left over.

> > ext2/ext3 ACLs and Extended Attributes
>
> I don't know why people still want ACL's. There were noises about
> them for samba, but I'v enot heard anything since. Are vendors using
> this?

I can offer a perspective from someone who has been struggling to get
Linux competitive in real-life enterprise situations.

ACL's are an issue for Linux (and Samba) in order for the combination
to sustain competitiveness against Novell and NT in the desktop
fileservices domain.  The harsh reality of life is that file and
document sharing is a way of life in the environments where Novell
dominates.  The appearance of ACL's and desktop support for their
management in NT would tend to confirm this.

Without the granularity of ACL's it becomes too difficult to establish
the types of permission environments needed to support what most
administrative and department support personnel (ie, secretaries) seem
to desire.

The patches also begin implementing a common API framework which
multiple filesystems seem to be able to leverage.  At least the rumor
appears to be that the instrastructure allows common toolsets to be
used for both ext2/3, XFS and perhaps other filesystems which want to
implement ACL's.

Its a compilation option and if set to default minimizes the impact on
people who don't need or want the infrastructure.  Ted also has his
fingers in the project which probably means that it isn't going to get
neglected.

Just my 2 cents.

Best wishes for a productive weekend to everyone.

Greg

}-- End of excerpt from Linus Torvalds

As always,
Dr. G.W. Wettstein, Ph.D.   Enjellic Systems Development, LLC.
4206 N. 19th Ave.           Specializing in information infra-structure
Fargo, ND  58102            development.
PH: 701-281-4950            WWW: http://www.enjellic.com
FAX: 701-281-3949           EMAIL: greg@enjellic.com
------------------------------------------------------------------------------
"Open source code is not guaranteed nor does it come with a warranty."
                                -- the Alexis de Tocqueville Institute

"I guess that's in contrast to proprietary software, which comes with
 a money-back guarantee, and free on-site repairs if any bugs are found."
                                -- Rary

Re: What's left over.

[Henning P. Schmiedehausen]

Linus Torvalds <torvalds@transmeta.com> writes:

>> ext2/ext3 ACLs and Extended Attributes

>I don't know why people still want ACL's. There were noises about them for 
>samba, but I'v enot heard anything since. Are vendors using this?

CIFS/SMB. Replacing Windows Fileservers. Supporting the required Windows
semantics. World domination.

That's one patch I personally consider really important. Getting the API in
place and a couple of FSses supporting it. The rest is up to user space.

	Regards
		Henning

-- 
Dipl.-Inf. (Univ.) Henning P. Schmiedehausen       -- Geschaeftsfuehrer
INTERMETA - Gesellschaft fuer Mehrwertdienste mbH     hps@intermeta.de

Am Schwabachgrund 22  Fon.: 09131 / 50654-0   info@intermeta.de
D-91054 Buckenhof     Fax.: 09131 / 50654-20   

Re: What's left over.

[Oliver Xymoron]

On Wed, Oct 30, 2002 at 09:43:29PM -0500, Alexander Viro wrote:
> 
> 
> On Wed, 30 Oct 2002, Linus Torvalds wrote:
> 
> > > ext2/ext3 ACLs and Extended Attributes
> > 
> > I don't know why people still want ACL's. There were noises about them for 
> > samba, but I'v enot heard anything since. Are vendors using this?
> 
> Because People Are Stupid(tm).  Because it's cheaper to put "ACL support: yes"
> in the feature list under "Security" than to make sure than userland can cope
> with anything more complex than  "Me Og.  Og see directory.  Directory Og's.
> Nobody change it".  C.f. snake oil, P.T.Barnum and esp. LSM users

It's nearly useless in a Unix-only context, true, however there's a rather
serious impedance mismatch for serving files to Windows that this
addresses. Emulating ACLs on the fly with groups to fit into the
Windows model is mostly doable but ain't pretty. 

-- 
 "Love the dolphins," she advised him. "Write by W.A.S.T.E.." 

Re: What's left over.

[Stephen Wille Padnos]

Alexander Viro wrote:

>On 31 Oct 2002, Dax Kelson wrote:
>
>>I think the normal intent is to let Sally, Joe, and Bill have their own
>>private directory protected from THE REST OF THE USERS.
>>
>>If a member of your trusted circle goes rogue, then, yup you are screwed
>>for the moment. It shouldn't last a whole month though.
>>
>>That is what backups, and employment termination is for.
>>    
>>
>
>Then give them all the same account and be done with that.  Effect will
>be the same.
>  
>

Unless I'm missing something, that only works if all the users need 
*exactly* the same permissions to all files, which isn't a good assumption.

Example:  Sally is an accountant, Joe and Bill are engineers.

Bill and Joe are working on a project, and Sally is cost control for 
that project - they all need access to the project files.  Bill and Joe 
do not need access to officer salary data, but Sally does.  Bill and Joe 
need access to other projects (not necessarily the same ones), but Sally 
doesn't.  Oops.

- Steve

Re: What's left over.

[Jamie Lokier]

Richard J Moore wrote:
> With the two it is possible to implant tracepoints without having to
> code up specific printks: kprobes can be used to implant a probe,
> the probe handler can call LTT to record the event.

Hey, that _is_ useful.  Me like.  Me spent many times wondering what
gets called when, and hunting heisenbugs masked by printk slowness.

-- Jamie

Re: What's left over.

[Linus Torvalds]

On Wed, 30 Oct 2002, Matt D. Robinson wrote:

> Linus Torvalds wrote:
> > > Crash Dumping (LKCD)
> > 
> > This is definitely a vendor-driven thing. I don't believe it has any
> > relevance unless vendors actively support it.
> 
> There are people within IBM in Germany, India and England, as well as
> a number of companies (Intel, NEC, Hitachi, Fujitsu), as well as SGI
> that are PAID to support this.

That's fine. And since they are paid to support it, they can apply the 
patches.  

What I'm saying by "vendor driven" is that it has no relevance for the 
standard kernel, and since it has no relevance to that, then I have no 
incentives to merge it. The crash dump is only useful with people who 
actively look at the dumps, and I don't know _anybody_ outside of the 
specialized vendors you mention who actually do that.

I will merge it when there are real users who want it - usually as a
result of having gotten used to it through a vendor who supports it. (And
by "support" I do not mean "maintain the patches", but "actively uses it"
to work out the users problems or whatever).

Horse before the cart and all that thing.

People have to realize that my kernel is not for random new features. The
stuff I consider important are things that people use on their own, or
stuff that is the base for other work. Quite often I want vendors to merge
patches _they_ care about long long before I will merge them (examples of
this are quite common, things like reiserfs and ext3 etc).

THAT is what I mean by vendor-driven. If vendors decide they really want
the patches, and I actually start seeing noises on linux-kernel or getting
requests for it being merged from _users_ rather than developers, then
that means that the vendor is on to something.

		Linus

Re: What's left over.

[Lars Marowsky-Bree]

On 2002-10-31T14:56:27,
   Richard J Moore <richardj_moore@uk.ibm.com> said:

> >> Crash Dumping (LKCD)
> >This is definitely a vendor-driven thing. I don't believe it has any
> >relevance unless vendors actively support it.

As time to repair is critical for availability (obviously) and having a good
crash dump will help reduce this, I'd also like to point out that such a
dumping framework is very important. Please, merge it.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
Principal Squirrel 
SuSE Labs - Research & Development, SuSE Linux AG
  
"If anything can go wrong, it will." "Chance favors the prepared (mind)."
  -- Capt. Edward A. Murphy            -- Louis Pasteur

Re: What's left over.

[Richard J Moore]

>> Crash Dumping (LKCD)
>
>This is definitely a vendor-driven thing. I don't believe it has any
>relevance unless vendors actively support it.

I can't argue with the fact you want to view lkcd this way. However as a
developer I have found a crash dump facility indispensable for certain
problems, particularly those that involve multiple processors where to use
more invasive techniques such as an interactive debugger can make the
problem unreproducible. It's also worth pointing out that each of the
serviceability tools (dump, trace, probes) complements each other. They are
every so much more powerful when used as a set: lkcd can capture a trace
buffer, whose contents would otherwise be lost; kprobes enables LTT to
implant tracepoints dynamically; krpobes + lkcd allows a crash dump to be
triggered for complex and specific conditions that are difficult to
reproduce. Without such tools, data gathering for complex problems becomes
a problem in itself.  A problem doesn't necessarily have to be reproducible
to make it necessary to solve.


Richard

Re: What's left over.

[Alan Cox]

On Thu, 2002-10-31 at 14:26, Jeff Garzik wrote:
> Yeah, historically we have avoided things like this.
> kcalloc gets proposed every year or so too.

I would like to see both of these in because tons of kernel fixing that
has been done through audits has been about


	get_user(a, ...)
	kmalloc(a * sizeof(b), ..)

We end up with loads of ugly  > MAXINT/sizeof(foo) if checks in the code
that ought to be in one place

Re: What's left over

[Suparna Bhattacharya]

On Thu, Oct 31, 2002 at 02:39:23AM +0000, Linus Torvalds wrote:
> 
> On Thu, 31 Oct 2002, Rusty Russell wrote:
> > 
> > 	Here is the list of features which have are being actively
> > pushed, not NAK'ed, and are not in 2.5.45.  There are 13 of them, as
> > appropriate for Halloween.
> 
> I'm unlikely to be able to merge everything by tomorrow, so I will 
> consider tomorrow a submission deadline to me, rather than a merge 
> deadline. That said, I merged everything I'm sure I want to merge today, 
> and the rest I simply haven't had time to look at very much.
> 
> 
> > Crash Dumping (LKCD)
> 
> This is definitely a vendor-driven thing. I don't believe it has any 
> relevance unless vendors actively support it.
> 

Linus,

I wish you could have made it to the OLS RAS BOF and seen this for
yourself - the vendor support, the need and the drive towards a 
unified and flexible dumping framework. 

The problem with dump has not been lack of vendor interest. There
wouldn't have been multiple dump type implementations floating around 
if there wasn't a need  --  LKCD, Mission Critical dump, Ingo's
network dump, kmsgdump, Rusty's oops dumper to cite some. The difficulty
has been technical and hence the diversity of approaches that different
projects came up with to tackle the problem (arising from slightly
different priorities and environments in each case). The second has
been related to preferences in the kind of user level analysis tools.

And the LKCD project has been evolving to address these very 
problems to bring the best of these worlds together and also allow
flexibility on the choice of analysis tools !

Mission critical Linux project code base for example is now being 
maintained as part of the LKCD project. Either lcrash or mission 
critical linux crash can be used for analysing LKCD dumps. 

And on the kernel side of things:

(a) The dump driver interface in LKCD has been specifically 
    designed to enable different kinds of dumping mechanisms and 
    targets to be supported -- generic block, network dump , 
    polled-IDE (Rusty style) etc, even alternate dump targets failover 
    and multiple dump devices in the future if required. We are also 
    experimenting with a memory dump driver to save dump to memory 
    and dump after a memory preserving soft-boot, reusing the mission 
    critical mcore technique.
(b) Selective dumping, for different levels of dump data - one
    option that was added recently would dump all kernel pages
    and is likely to be commonly used (gzip compressed dump). Its
    pretty easy to extend to more selectivity or different levels
    and the dump also occurs in passes from more critical data to 
    less critical.
    (The page in use flag was added to help with this)
(c) The core pieces which touch the kernel as such just add basic 
    infrastructure that is needed in the kernel for any dumping 
    facility. Includes:
	- Enabling IPI to collect CPU state on all processors in the
	  system right when dump is triggered (may not be a normal
	  situation, so NMIs where supported are the best option)
	- Ability to quiesce (silence) the system before dumping 
	  (and if in non-disruptive mode, then restore it back)
	- Calls into dump from kernel paths (panic, oops, sysrq
	  etc). 
	- Exports of symbols to help with physical memory 
	  traversal and verification

As Matt has said there is an active development community behind 
LKCD and lot of the drive for that has come from companies who use it 
and are really hoping hard that it becomes part of the mainline.

BTW, the code has also been scrutinised and reviewed over
lkml as well and undergone iterations of releases following 
that. Anything else there that you think needs to be fixed please
do let us know.

Regards
Suparna

Re: What's left over.

[Richard J Moore]

>> Linux Trace Toolkit (LTT)
>
>I don't know what this buys us.

If you consider developer productivity useful then LTT has definite
benefits especially when combined with kprobes. With the two it is possible
to implant tracepoints without having to code up specific printks: kprobes
can be used to implant a probe, the probe handler can call LTT to record
the event.

Why call LTT instead of having a printk in the probe handler? - for
performance reasons, for latency reasons, because kprobes can implant
probes absolutely anywhere in the system, for analysis reasons - LTT trace
data can be post processed and massaged in a number of ways using the
visualizer tools. Yes you can do some of this using printk directly, but
you can be into a whole heap more work and it will certainly take longer to
implant a temporary tracepoint, recompile, run, remove, recompile the using
the dynamic trace technique of LTT+kprobes.


Richard

Re: What's left over.

[Jeff Garzik]

Chris Wedgwood wrote:

>problems most people don't have?  What next, some kind of misdesigned
>in-kernel CryptoAPI?
>  
>


Ok, I'll allow myself to be trolled.

What's wrong with our current 2.5.45 crypto api?

Re: What's left over.

[Jeff Garzik]

Joe Thornber wrote:

>ii) vcalloc, this *didn't* get merged, and will probably end up getting
>    moved into dm.h.
>

Yeah, historically we have avoided things like this.

kcalloc gets proposed every year or so too.

>iii) ioctl32 support: people have argued against an ioctl interface,
>     and I'm inclined to agree with them, which is why I'm going to
>     publish an fs interface shortly.  However, given that we are
>     currently using an ioctl interface how do we avoid adding support for
>     32bit userland/64 kernel space ?  If EVMS isn't touching these
>     files does that mean they're not supporting these architectures ?
>
>        arch/mips64/kernel/ioctl32.c
>        arch/ppc64/kernel/ioctl32.c
>        arch/s390x/kernel/ioctl32.c
>        arch/sparc64/kernel/ioctl32.c
>  
>

Well, I'll note that ALSA compartmentalizes their ioctl32 handling 
within their own subsystem, which seems like a decent solution.

That said, [maybe I'm biased <g>], using an fs interface allows one to 
completely eliminate an ioctl32 interface.  That would be the direction 
I would greatly prefer by the time 2.5.x hits the code freeze.

Best regards, and congrats for getting it merged,

    Jeff

Re: What's left over.

[Chris Friesen]

Linus Torvalds wrote:
>>Linux Trace Toolkit (LTT
> I don't know what this buys us.

I'd like to add a request for this to be in mainstream.  The benefits 
have already been stated in this thread, and it has been used here to 
good effect.

>>Crash Dumping (LKCD
> This is definitely a vendor-driven thing. I don't believe it has any 
> relevance unless vendors actively support it.

I'd like to see this too.  The more debug information the better as far 
as I'm concerned.


>>Hires Timer
> This one is likely another "vendor push" thing.

It doesn't hurt performance when turned off, and allows for 
finer-grained timing when turned on.  What's not to like?  I can't 
comment on the actual code, but I really like the idea.


Chris


-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: What's left over.

[mbs]

> > POSIX Timer API
>
> I think I'll do at least the API, but there were some questions about the
> config options here, I think.

I think george just posted a config optionless patch.

WOOHOO!  Thanks!

>
> > Hires Timers
>
> This one is likely another "vendor push" thing.
>

I work for a vendor who really wants this.  

we have customers who demand it.

I am sure we are not alone (mvista? concurrent? any embedded space people for 
whom 10msec is not good enough and the extra overhead of a higer frequency 
fixed interval timer is unacceptable please speak up, if we don't get it in 
now, we probably won't get it for 2 years.)

-- 
/**************************************************
**   Mark Salisbury       ||      mbs@mc.com     **
**************************************************/

Re: What's left over.

[Geert Uytterhoeven]

On Thu, 31 Oct 2002, Rusty Russell wrote:
> In message <Pine.LNX.4.44.0210301823120.1396-100000@home.transmeta.com> you wri
> te:
> > On Thu, 31 Oct 2002, Rusty Russell wrote:
> > > Fbdev Rewrite
> > 
> > This one is just huge, and I have little personal judgement on it.
> 
> It's been around for a while.  Geert, Russell?

It's huge because it moves a lot of files around:
  1. drivers/char/agp/ -> drivers/video/agp/
  2. drivers/char/drm/ -> drivers/video/drm/
  3. console related files in drivers/video/ -> drivers/video/console/

(1) and (2) should be reverted, but apparently they aren't reverted in the
patch at http://phoenix.infradead.org/~jsimmons/fbdev.diff.gz yet. The patch
also seems to remove some drivers. Haven't checked the bk repo yet.

James, can you please fix that (and the .Config files)?

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: What's left over.

[Trever L. Adams]

On Wed, 2002-10-30 at 21:31, Linus Torvalds wrote:

> > ext2/ext3 ACLs and Extended Attributes
> 
> I don't know why people still want ACL's. There were noises about them for 
> samba, but I'v enot heard anything since. Are vendors using this?
> 

I am sure I don't count (not being a vendor), but Intermezzo offers
support for this (they are waiting on feature freeze to redo it to 2.5
according to an email I have).  I want this stuff.  Yes, u+g+w is nice,
but good ACLs are even better.  Please, if this is technically correct
in implementation, do put it in.

Thank you,
Trever

Re: What's left over.

[Joe Thornber]

On Thu, Oct 31, 2002 at 02:00:31PM +1100, Rusty Russell wrote:
> > > EVMS
> > 
> > Not for the feature freeze, there are some noises that imply that SuSE may 
> > push it in their kernels. 
> 
> They have, IIRC.  Interestingly, it was less invasive (existing source
> touched) than the LVM2/DM patch you merged.

FUD.  I added to three areas of existing code:

i) Every man and his dog uses mempools in conjuction with slabs, so
   rather than having everyone redefining their own alloc/free
   functions I added the following huge functions to mempool.c.  In no
   way were they mandatory.

    /*
     * A commonly used alloc and free fn.
     */
    void *mempool_alloc_slab(int gfp_mask, void *pool_data)
    {
            kmem_cache_t *mem = (kmem_cache_t *) pool_data;
            return kmem_cache_alloc(mem, gfp_mask);
    }

    void mempool_free_slab(void *element, void *pool_data)
    {
            kmem_cache_t *mem = (kmem_cache_t *) pool_data;
            kmem_cache_free(mem, element);
    }

ii) vcalloc, this *didn't* get merged, and will probably end up getting
    moved into dm.h.

iii) ioctl32 support: people have argued against an ioctl interface,
     and I'm inclined to agree with them, which is why I'm going to
     publish an fs interface shortly.  However, given that we are
     currently using an ioctl interface how do we avoid adding support for
     32bit userland/64 kernel space ?  If EVMS isn't touching these
     files does that mean they're not supporting these architectures ?

        arch/mips64/kernel/ioctl32.c
        arch/ppc64/kernel/ioctl32.c
        arch/s390x/kernel/ioctl32.c
        arch/sparc64/kernel/ioctl32.c


So given that (ii) didn't get merged, which of (i) and (iii) were you
objecting to ?

- Joe

Re: What's left over.

[Lech Szychowski]

> Yes, people use it.  Not quite sure why though, I guess ACLs
> buy some flexibility over the user/group/other model but if
> the "unlimited groups" patch goes in (is in?) I'm happy ;)

Correct me if I'm wrong but I believe a process has to be
restarted to have its group membership list changed? 

That's a huge difference from ACL behavior which allow for changes to
file access rights without the need to restart the accessing process.

-- 
	Leszek.

-- lech7@pse.pl 2:480/33.7          -- REAL programmers use INTEGERS --
-- speaking just for myself...

Re: What's left over.

[Ville Herva]

On Thu, Oct 31, 2002 at 10:23:32AM +0100, you [Geert Uytterhoeven] wrote:
> 
> Except on m68k, where we've had a feature to store all kernel messages in an
> unused portion of memory (e.g. some Chip RAM on Amiga) and recover them after
> reboot since ages.

There was similar thing for x86 as well:

http://www.tux.org/hypermail/linux-kernel/1999week27/0782.html

Of course it never went to mainline (and I don't know how well it worked.)
>From what I understand, lkcd can support such method easily.


-- v --

v@iki.fi

Re: What's left over.

[Geert Uytterhoeven]

On Thu, 31 Oct 2002, Ville Herva wrote:
> On Wed, Oct 30, 2002 at 06:31:36PM -0800, you [Linus Torvalds] wrote:
> > > Crash Dumping (LKCD)
> > 
> > This is definitely a vendor-driven thing. I don't believe it has any 
> > relevance unless vendors actively support it.
> 
> I don't think this is just a vendor thing. Currently, linux doesn't have any
> way of saving the crash dump when the box crashes. So if it crashes, the
> user needs to write the oops down by hand (error prone, the interesting part
> has often scrolled off screen), or attach a serial console (then he needs to
> reproduce it - not always possible, and actually majority of people (home
> users) don't have second box and the cable. Nor the motivation.)

Except on m68k, where we've had a feature to store all kernel messages in an
unused portion of memory (e.g. some Chip RAM on Amiga) and recover them after
reboot since ages.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: What's left over.

[Ville Herva]

On Wed, Oct 30, 2002 at 06:31:36PM -0800, you [Linus Torvalds] wrote:
> 
> > Crash Dumping (LKCD)
> 
> This is definitely a vendor-driven thing. I don't believe it has any 
> relevance unless vendors actively support it.

I don't think this is just a vendor thing. Currently, linux doesn't have any
way of saving the crash dump when the box crashes. So if it crashes, the
user needs to write the oops down by hand (error prone, the interesting part
has often scrolled off screen), or attach a serial console (then he needs to
reproduce it - not always possible, and actually majority of people (home
users) don't have second box and the cable. Nor the motivation.)

So, imho some kind of way of semi-automatically save the dumps is needed. If
vendors even support it - great - but it has value to mainline kernel as
well, as people can submit more accurate error reports. Besides, if it goes
in mainline, I believe vendors are likely to support it. (Why wouldn't they?
Currently there just isn't a standard way of doing this.)

There are a bunch of patches for this sort of thing (Willy Tarreau's
kmsgdump for dumping to floppy, Ingo's netconsole, Rusty's oopser for
dumping to ide device...), but lkcd is a more general framework, and can
support different ways of dumping.

I know you are not keen on kernel debuggers, but I can't see what's
fundamentally wrong with being able to save the crucial info when a crash
happens...


-- v --

v@iki.fi

Re: What's left over.

[Alexander Viro]

On 31 Oct 2002, Dax Kelson wrote:

> I think the normal intent is to let Sally, Joe, and Bill have their own
> private directory protected from THE REST OF THE USERS.
> 
> If a member of your trusted circle goes rogue, then, yup you are screwed
> for the moment. It shouldn't last a whole month though.
> 
> That is what backups, and employment termination is for.

Then give them all the same account and be done with that.  Effect will
be the same.

Re: What's left over.

[Dax Kelson]

On Thu, 2002-10-31 at 00:10, Alexander Viro wrote:
> 
> 
> On 30 Oct 2002, Dax Kelson wrote:
> 
> > Without ACLs, if Sally, Joe and Bill need rw access to a file/dir, just
> > create another group with just those three people in.  Over time, of
> 
> If Sally, Joe and Bill need rw access to a directory, and Joe and Bill
> are using existing userland (any OS I'd seen), then Sally can easily
> fuck them into the next month and not in a good way.

I think the normal intent is to let Sally, Joe, and Bill have their own
private directory protected from THE REST OF THE USERS.

If a member of your trusted circle goes rogue, then, yup you are screwed
for the moment. It shouldn't last a whole month though.

That is what backups, and employment termination is for.

Dax

Re: What's left over.

[Alexander Viro]

On 30 Oct 2002, Dax Kelson wrote:

> Without ACLs, if Sally, Joe and Bill need rw access to a file/dir, just
> create another group with just those three people in.  Over time, of

If Sally, Joe and Bill need rw access to a directory, and Joe and Bill
are using existing userland (any OS I'd seen), then Sally can easily
fuck them into the next month and not in a good way.

_That_ is the real problem.  Until that is solved (i.e. until all
userland is written up to the standards allegedly followed in writing
suid-root programs wrt hostile filesystem modifications) NO mechanism
will help you.  ACLs, huge groups, whatever - setups with that sort
of access allowed are NOT SUSTAINABLE with the current userland(s).

Re: What's left over.

[Chris Wedgwood]

On Wed, Oct 30, 2002 at 11:48:23PM -0700, Dax Kelson wrote:

> Technically speaking you can achieve ACL like permissions/behavior
> using the historical UNIX security model by creating a group EACH
> time you run into a unique case permission scenario.

I'm not arguing against this... I'm claiming POSIX ACLs are mostly
brain-dead and almost worthless (broken by committee pressure and too
many people making stupid concessions).

If we must have ACLs, why not do it right?

> Without ACLs, if Sally, Joe and Bill need rw access to a file/dir,
> just create another group with just those three people in.  Over
> time, of course, this leads to massive group proliferation.  Without
> Tim Hockin's patch, 32 groups is maximum number of groups a user can
> be a member of.

How many people actually need this level of complexity?

Why are we adding all this shit and bloat because of perceived
problems most people don't have?  What next, some kind of misdesigned
in-kernel CryptoAPI?



  --cw

Re: What's left over.

[Dax Kelson]

On Wed, 2002-10-30 at 23:22, Chris Wedgwood wrote:
> On Thu, Oct 31, 2002 at 01:06:54AM -0200, Rik van Riel wrote:
> 
> > Personally I do think either the unlimited groups patch or ACLs are
> > needed in order to sanely run a large anoncvs setup.
> 
> Processes need to be a member of 20+ groups to make anoncvs work?
> Sounds like anoncvs is broken then.

Technically speaking you can achieve ACL like permissions/behavior using
the historical UNIX security model by creating a group EACH time you run
into a unique case permission scenario.

Without ACLs, if Sally, Joe and Bill need rw access to a file/dir, just
create another group with just those three people in.  Over time, of
course, this leads to massive group proliferation.  Without Tim Hockin's
patch, 32 groups is maximum number of groups a user can be a member of.

Dax

Re: What's left over.

[Matt D. Robinson]

Linus Torvalds wrote:
> > Crash Dumping (LKCD)
> 
> This is definitely a vendor-driven thing. I don't believe it has any
> relevance unless vendors actively support it.

There are people within IBM in Germany, India and England, as well as
a number of companies (Intel, NEC, Hitachi, Fujitsu), as well as SGI
that are PAID to support this.  In addition, Global Services at IBM
uses this as a front-line method for resolving customer problems.
If you're looking for names of people to sign up to support it
(both vendors and non-vendors), I can make that list up for you.

There are a number of us (developers, support staff, and other
interested parties) who bend over backwards, day in and day out
to make sure this stuff works and helps people, even if it isn't
kernel developers (directly -- indirectly, you get bug reports that
are sane and useful).

It's not sexy kernel stuff, but it is very important, and if you'd
like, I can have representatives from at least 10 major corporations
(Fortune 500 companies) contact you to request that this go in.

We're generating 2.5.45 patches now, and we ask that you include
the patches when they are posted.

I don't know what else to say except that people really want this
stuff and all of us in the LKCD community work really hard together
to make this project useful for everyone.

Please include this in your next snapshot.

--Matt

P.S.  Copying some of the users and developers.

Re: What's left over.

[Chris Wedgwood]

On Thu, Oct 31, 2002 at 01:06:54AM -0200, Rik van Riel wrote:

> Personally I do think either the unlimited groups patch or ACLs are
> needed in order to sanely run a large anoncvs setup.

Processes need to be a member of 20+ groups to make anoncvs work?
Sounds like anoncvs is broken then.


  --cw

Re: What's left over.

[Chris Wedgwood]

On Wed, Oct 30, 2002 at 10:19:54PM -0500, tridge@samba.org wrote:

> Eventually I'd like to see a combination of LSM with a new ACL
> system give the ability to support full NT ACLs on Linux (which is
> also needed for full nfsv4 support), but that is way too much to do
> for the 2.6 kernel.

Add bloat to make windows clients happy?

> Extended attributes are also important as they give a place to store
> all the extra DOS info that has no other logical place in a posix
> filesystem. For example, we can put the 'read only', 'archive',
> 'hidden' and 'system' attributes there. If we don't have extended
> attributes then we need to use a nasty kludge where these map to
> various unix permission bits, but the mapping is terrible and
> doesn't give the correct semantics (especially for things like read
> only on directories).

More bloat that does really solve Linux problems... sounds like nasty
hacks to make winduhs hacks work better.

Don't get me wrong, I'm not against sane ACLs (POSIX ACLs are not) os
EAs, but justification of "it makes windows clients easier" is pretty
horrendous IMO.

I'd would at some point like to see decent ACLs, but I don't want to
see 'windows ACLs' and all the SID nonsense.



  --cw

Re: What's left over.

[Dax Kelson]

On Wed, 2002-10-30 at 19:31, Linus Torvalds wrote:
> 
> > ext2/ext3 ACLs and Extended Attributes
> 
> I don't know why people still want ACL's. There were noises about them for 
> samba, but I'v enot heard anything since. Are vendors using this?
> 

I teach Linux classes to corporate IT guys (~300 or so this year) and
many of them are migrating from Solaris or deploying Linux along side
Solaris.

Solaris has had ACLs since 2.5.1 (1996), and EAs since 2.9 (May 2002).

Having ACL in Linux is a VERY COMMON REQUEST that I hear from the
students.

FWIW.

Dax Kelson
Guru Labs

Re: What's left over.

[Patrick Finnegan]

On Thu, 31 Oct 2002, Christoph Hellwig wrote:

> On Wed, Oct 30, 2002 at 11:20:42PM -0500, Patrick Finnegan wrote:
> > Specifically, the interoperation with IBM's JFS LVM and MS's LVM will be
>
> JFS has no lvm, it just sits on any blockdevice.  The support for Windows
> dynamic disks actually layers ontop of the MD driver..

To be more specific, I'm talking about AIX's JFS, not linux's JFS...

--
Purdue Universtiy ITAP/RCS
Information Technology at Purdue
Research Computing and Storage
http://www-rcd.cc.purdue.edu

http://dilbert.com/comics/dilbert/archive/images/dilbert2040637020924.gif

Re: What's left over.

[Christoph Hellwig]

On Wed, Oct 30, 2002 at 11:20:42PM -0500, Patrick Finnegan wrote:
> Specifically, the interoperation with IBM's JFS LVM and MS's LVM will be

JFS has no lvm, it just sits on any blockdevice.  The support for Windows
dynamic disks actually layers ontop of the MD driver..

Re: What's left over.

[Patrick Finnegan]

I'm kind of new here, but I'll present my case in hope that someone
listens to me.



On Wed, 30 Oct 2002, Linus Torvalds wrote:

> On Thu, 31 Oct 2002, Rusty Russell wrote:
>
> > Crash Dumping (LKCD)
>
> This is definitely a vendor-driven thing. I don't believe it has any
> relevance unless vendors actively support it.

This is something that we're just starting to use in my department in
Purdue - we work with clustering, and LKCD will let us determine why our
nodes decide to kernel panic since it's generally not worthwhile to
connect a head to each machine.

I see LKCD as having a big impact by allowing kernels to be debugged after
they have panic'd (and thus don't send out a message to syslog).  It can
especially be usful in compute farms, or other scenerios where it's
difficut or cost prohibitive to connect a console (or console server) to
each individual machine.

> > EVMS
>
> Not for the feature freeze, there are some noises that imply that SuSE may
> push it in their kernels.

I think that the integration between RAID and LVM is a good thing, and
EVMS's 'plug-in module' architecture will help tremendously to bring
interoperation with other systems' volume management subsystems.
Specifically, the interoperation with IBM's JFS LVM and MS's LVM will be
helpful for people trying to migrate their servers over from those OS's to
GNU/Linux.

-- Pat

Purdue University ITAP/RCS
Information Technology at Purdue
Research Computing and Storage
http://www-rcd.cc.purdue.edu

Re: What's left over.

[Andreas Dilger]

On Oct 30, 2002  18:31 -0800, Linus Torvalds wrote:
> On Thu, 31 Oct 2002, Rusty Russell wrote:
> > ext2/ext3 ACLs and Extended Attributes
> 
> I don't know why people still want ACL's. There were noises about them for 
> samba, but I've not heard anything since. Are vendors using this?

I don't really care about ACLs so much one way or the other, but we
DEFINITELY use EAs with Lustre, so at the minimum if we could have
that part of the changes I'd be happy.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: What's left over.

[tridge]

> XFS doesn't have ACLs either in plain 2.5.

The existing NAS boxes that use Linux and XFS tend to base their
kernels on the 2.4-xfs tree from cvs on sgi.com. It works well and the
SGI guys have been very good about fixing problems when they crop up.

I think that the biggest beneficiary of adding extended attributes and
ACLs into ext3 for 2.6 would be more casual users (home, small office
etc) as they will then be able to use ACLs in Samba without the pain
of switching to a different kernel.

Cheers, Tridge

--
http://samba.org/~tridge/

Re: What's left over.

[Christoph Hellwig]

On Thu, Oct 31, 2002 at 02:00:31PM +1100, Rusty Russell wrote:
> > I don't know why people still want ACL's. There were noises about them for 
> > samba, but I'v enot heard anything since. Are vendors using this?
> 
> SAMBA needs them, which is why serious Samba boxes use XFS.  Tridge,
> Ted?

XFS doesn't have ACLs either in plain 2.5.

> > Not for the feature freeze, there are some noises that imply that SuSE may 
> > push it in their kernels. 
> 
> They have, IIRC.  Interestingly, it was less invasive (existing source
> touched) than the LVM2/DM patch you merged.

But that only because dm added stuff to the generic code where we
told it. It's a lot more code than dm and it adds new discovery
code at the same time we start moving stuff _out_ of the kernel
to initramfs.

If you can SuSE has merged it any IBM patch posted here should get
in, coming from big blue seems to be a basic merge criteria in
Nuernberg :)

Re: What's left over.

[Stephen Lord]

On Wed, 2002-10-30 at 20:31, Linus Torvalds wrote:
> 
> On Thu, 31 Oct 2002, Rusty Russell wrote:
> > 
> > 	Here is the list of features which have are being actively
> > pushed, not NAK'ed, and are not in 2.5.45.  There are 13 of them, as
> > appropriate for Halloween.
> 
> I'm unlikely to be able to merge everything by tomorrow, so I will 
> consider tomorrow a submission deadline to me, rather than a merge 
> deadline. That said, I merged everything I'm sure I want to merge today, 
> and the rest I simply haven't had time to look at very much.
> 

> 
> > ext2/ext3 ACLs and Extended Attributes
> 
> I don't know why people still want ACL's. There were noises about them for 
> samba, but I'v enot heard anything since. Are vendors using this?
> 

There are a fair number of NAS vendors who do linux boxes with Samba
and XFS because of the ACL support, Quantum being the one Tridge now
works for by the way. The reason they want it is so they can support
the features NT folks are used to having in their file servers.
Now, we could just let the NT folks use NT servers instead....

Even getting XFS ACLs running in 2.5 requires part of this patch set.

Steve

Re: What's left over.

[tridge]

> > > ext2/ext3 ACLs and Extended Attributes
> > 
> > I don't know why people still want ACL's. There were noises about them for 
> > samba, but I'v enot heard anything since. Are vendors using this?
> 
> SAMBA needs them, which is why serious Samba boxes use XFS.  Tridge,
> Ted?

oh yes, all the Linux based storage appliances use ACLs. Posix ACLs
aren't ideal for Samba, but they are *much* better than having no ACLs
at all. The Posix ACL code has been in Samba for a long time (getting
close to 3 years now?). 

Eventually I'd like to see a combination of LSM with a new ACL system
give the ability to support full NT ACLs on Linux (which is also
needed for full nfsv4 support), but that is way too much to do for
the 2.6 kernel.

For the majority of windows users the mapping Samba does internally
between Posix ACLs and NT ACLs is sufficient for now. 

I think that it would be a very good thing for Posix ACLs to be
included in the 2.6 kernel, especially in ext3.

Extended attributes are also important as they give a place to store
all the extra DOS info that has no other logical place in a posix
filesystem. For example, we can put the 'read only', 'archive', 'hidden'
and 'system' attributes there. If we don't have extended attributes
then we need to use a nasty kludge where these map to various unix
permission bits, but the mapping is terrible and doesn't give the
correct semantics (especially for things like read only on
directories). 

My main concern with using extended attributes in this way is
performance. My experience with XFS is that as soon as you start
adding extended attributes then the performance drops a lot, but I
haven't tested performance with the ext3 extended attributes so maybe
they don't have the same problem.

Cheers, Tridge

--
http://samba.org/~tridge/

Re: What's left over.

[Stephen Frost]

* Rik van Riel (riel@conectiva.com.br) wrote:
> On Wed, 30 Oct 2002, Linus Torvalds wrote:
> > On Thu, 31 Oct 2002, Rusty Russell wrote:
> 
> > > ext2/ext3 ACLs and Extended Attributes
> >
> > I don't know why people still want ACL's. There were noises about them for
> > samba, but I'v enot heard anything since. Are vendors using this?
> 
> Yes, people use it.  Not quite sure why though, I guess ACLs
> buy some flexibility over the user/group/other model but if
> the "unlimited groups" patch goes in (is in?) I'm happy ;)
> 
> Personally I do think either the unlimited groups patch or
> ACLs are needed in order to sanely run a large anoncvs setup.

The feeling I got on this was the ability to let users define their own
groups.  Perhaps I'm not following it closely enough but that was the
impression I got in terms of "what this does for us"; I'm probably
missing other things.  Just that ability would be nice in my view
though.  Isn't it something that's been in AFS for a long time too?
I've got a few friends who've played with AFS before (at CMU and the
like) and really enjoyed the ACLs there.

	Just my thoughts,

		Stephen

Re: What's left over.

[Karim Yaghmour]

Linus Torvalds wrote:
> > Linux Trace Toolkit (LTT)
> 
> I don't know what this buys us.

How about being able to:
- Debug synchronization problems among processes (there is no other
tool to do this, not gdb, not strace, not printf, ...)
- Measure exact time spent wainting for kernel and which other
processes a process had to wait for.
- Measure exact time it takes for an interrupt's effects to propagate
throughout the entire system.
- Understand the exact behavior the system has to input. (what is
the exact sequence of processes that run when I press a key).
- Identify sporadic problems in very saturated systems. (thousands
of servers and one of them is doing weird stuff).
- etc.

Providing system tracing is a necessity for any sort of complex
application development and system monitoring. Some people simply
can't use Linux without this sort of tool and I am at pains to
explain to them why they actually have to patch their kernel to
be able to debug their inter-process synchronization problems.

Users don't have to patch their kernel to use gdb and I don't
see why they should need to patch their kernel to understand how
their various processes interact with the kernel and vice-versa.

Karim

===================================================
                 Karim Yaghmour
               karim@opersys.com
      Embedded and Real-Time Linux Expert
===================================================

Re: What's left over.

[Rik van Riel]

On Wed, 30 Oct 2002, Linus Torvalds wrote:
> On Thu, 31 Oct 2002, Rusty Russell wrote:

> > ext2/ext3 ACLs and Extended Attributes
>
> I don't know why people still want ACL's. There were noises about them for
> samba, but I'v enot heard anything since. Are vendors using this?

Yes, people use it.  Not quite sure why though, I guess ACLs
buy some flexibility over the user/group/other model but if
the "unlimited groups" patch goes in (is in?) I'm happy ;)

Personally I do think either the unlimited groups patch or
ACLs are needed in order to sanely run a large anoncvs setup.

regards,

Rik
-- 
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/		http://distro.conectiva.com/
Current spamtrap:  <a href=mailto:"october@surriel.com">october@surriel.com</a>

Re: What's left over.

[Rusty Russell]

In message <Pine.LNX.4.44.0210301823120.1396-100000@home.transmeta.com> you wri
te:
> 
> On Thu, 31 Oct 2002, Rusty Russell wrote:
> > 
> > 	Here is the list of features which have are being actively
> > pushed, not NAK'ed, and are not in 2.5.45.  There are 13 of them, as
> > appropriate for Halloween.
> 
> I'm unlikely to be able to merge everything by tomorrow, so I will 
> consider tomorrow a submission deadline to me, rather than a merge 
> deadline. That said, I merged everything I'm sure I want to merge today, 
> and the rest I simply haven't had time to look at very much.
> 
> > In-kernel Module Loader and Unified parameter support
> 
> This apparently breaks things like DRI, which I'm fairly unhappy about,
> since I think 3D is important.

Yes, the patch stubs out inter_module_*, in favor of get_symbol() &
put_symbol().

This breaks the three users: one in drivers/mtd/ and two in
drivers/char/drm/.  I have a patch which fixes them (untested), or I
can simply put the inter_module_* code back in.

> > Fbdev Rewrite
> 
> This one is just huge, and I have little personal judgement on it.

It's been around for a while.  Geert, Russell?

> > Linux Trace Toolkit (LTT)
> 
> I don't know what this buys us.

Haven't looked at it.

> > statfs64
> 
> I haven't even seen it.

It's fairly old, but Peter Chubb said there was some vendor interest
for v. large devices.  Peter?

> > ext2/ext3 ACLs and Extended Attributes
> 
> I don't know why people still want ACL's. There were noises about them for 
> samba, but I'v enot heard anything since. Are vendors using this?

SAMBA needs them, which is why serious Samba boxes use XFS.  Tridge,
Ted?

> > Hotplug CPU Removal Support
> 
> No objections, but very little visibility into it either.

The controls are in driverfs etc, and that's always been in flux. 8(

The rest is v. small, basically extending ksoftirqd, workqueues and
migration threads to disable them.  Then it's all arch-specific.

> > Hires Timers
> 
> This one is likely another "vendor push" thing.
> 
> > EVMS
> 
> Not for the feature freeze, there are some noises that imply that SuSE may 
> push it in their kernels. 

They have, IIRC.  Interestingly, it was less invasive (existing source
touched) than the LVM2/DM patch you merged.

> > initramfs
> 
> I want this.

Good.  The big payoff is moving stuff out of the kernel, which can't
really be done in a stable series.

> > Kernel Probes
> 
> Probably.

Sent.

Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: What's left over.

[Alexander Viro]

On Wed, 30 Oct 2002, Linus Torvalds wrote:

> > ext2/ext3 ACLs and Extended Attributes
> 
> I don't know why people still want ACL's. There were noises about them for 
> samba, but I'v enot heard anything since. Are vendors using this?

Because People Are Stupid(tm).  Because it's cheaper to put "ACL support: yes"
in the feature list under "Security" than to make sure than userland can cope
with anything more complex than  "Me Og.  Og see directory.  Directory Og's.
Nobody change it".  C.f. snake oil, P.T.Barnum and esp. LSM users

Re: What's left over.

[Linus Torvalds]

On Thu, 31 Oct 2002, Rusty Russell wrote:
> 
> 	Here is the list of features which have are being actively
> pushed, not NAK'ed, and are not in 2.5.45.  There are 13 of them, as
> appropriate for Halloween.

I'm unlikely to be able to merge everything by tomorrow, so I will 
consider tomorrow a submission deadline to me, rather than a merge 
deadline. That said, I merged everything I'm sure I want to merge today, 
and the rest I simply haven't had time to look at very much.

> In-kernel Module Loader and Unified parameter support

This apparently breaks things like DRI, which I'm fairly unhappy about,
since I think 3D is important.

> Fbdev Rewrite

This one is just huge, and I have little personal judgement on it.

> Linux Trace Toolkit (LTT)

I don't know what this buys us.

> statfs64

I haven't even seen it.

> ext2/ext3 ACLs and Extended Attributes

I don't know why people still want ACL's. There were noises about them for 
samba, but I'v enot heard anything since. Are vendors using this?

> ucLinux Patch (MMU-less support)

I've seen this, it looks pretty ok.

> Crash Dumping (LKCD)

This is definitely a vendor-driven thing. I don't believe it has any 
relevance unless vendors actively support it.

> POSIX Timer API

I think I'll do at least the API, but there were some questions about the 
config options here, I think.

> Hotplug CPU Removal Support

No objections, but very little visibility into it either.

> Hires Timers

This one is likely another "vendor push" thing.

> EVMS

Not for the feature freeze, there are some noises that imply that SuSE may 
push it in their kernels. 

> initramfs

I want this.

> Kernel Probes

Probably.

		Linus

What's left over.

[Rusty Russell]

Hi Linus,

	Here is the list of features which have are being actively
pushed, not NAK'ed, and are not in 2.5.45.  There are 13 of them, as
appropriate for Halloween.

	Most were submitted repeatedly *well* before the freeze.  It'd
be nice for you to give feedback, and decide which ones (if any) are
still up for review.

Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

From: http://www.kernel.org/pub/linux/kernel/people/rusty/2.6-not-in-yet/

Rusty's Remarkably Unreliable List of Pending 2.6 Features
[aka. Rusty's Snowball List]

A: Author
M: lkml posting describing patch
D: Download URL
S: Size of patch, number of files altered (source/config), number of new files.
X: Impact summary (only parts of patch which alter existing source files, not config/make files)
T: Diffstat of whole patch
N: Random notes

In rough order of invasiveness (number of altered source files):

In-kernel Module Loader and Unified parameter support
A: Rusty Russell
D: http://www.kernel.org/pub/linux/kernel/people/rusty/patches/Module/
S: 841 kbytes, 302/36 files altered, 22 new
T: Diffstat
X: Summary patch (598k)
N: Requires new modutils

Fbdev Rewrite
A: James Simmons
M: http://www.uwsg.iu.edu/hypermail/linux/kernel/0111.3/1267.html
D: http://phoenix.infradead.org/~jsimmons/fbdev.diff.gz
S: 4852 kbytes, 168/29 files altered, 124 new
T: Diffstat
X: Summary patch (182k)

Linux Trace Toolkit (LTT)
A: Karim Yaghmour
M: http://www.uwsg.iu.edu/hypermail/linux/kernel/0204.1/0832.html
M: http://marc.theaimsgroup.com/?l=linux-kernel&m=103491640202541&w=2
M: http://marc.theaimsgroup.com/?l=linux-kernel&m=103423004321305&w=2
M: http://marc.theaimsgroup.com/?l=linux-kernel&m=103247532007850&w=2
D: http://opersys.com/ftp/pub/LTT/ExtraPatches/patch-ltt-linux-2.5.44-vanilla-021026-2.2.bz2
S: 257 kbytes, 67/4 files altered, 9 new
T: Diffstat
X: Summary patch (90k)

statfs64
A: Peter Chubb
M: http://marc.theaimsgroup.com/?l=linux-kernel&m=103490436228016&w=2
D: http://marc.theaimsgroup.com/?l=linux-kernel&m=103490436228016&w=2
S: 42 kbytes, 53/0 files altered, 1 new
T: Diffstat
X: Summary patch (32k)

ext2/ext3 ACLs and Extended Attributes
A: Ted Ts'o
M: http://lists.insecure.org/lists/linux-kernel/2002/Oct/6787.html
B: bk://extfs.bkbits.net/extfs-2.5-update
D: http://thunk.org/tytso/linux/extfs-2.5/
S: 497 kbytes, 96/34 files altered, 34 new
T: Diffstat
X: Summary patch (167k)

ucLinux Patch (MMU-less support)
A: Greg Ungerer
M: http://lwn.net/Articles/11016/
D: http://www.uclinux.org/pub/uClinux/uClinux-2.5.x/linux-2.5.44uc3.patch.gz
S: 2218 kbytes, 25/34 files altered, 429 new
T: Diffstat
X: Summary patch (40k)

Crash Dumping (LKCD)
A: Matt Robinson, LKCD team
M: http://lists.insecure.org/lists/linux-kernel/2002/Oct/8552.html
D: http://lkcd.sourceforge.net/download/latest/
S: 18479 kbytes, 18/10 files altered, 10 new
T: Diffstat
X: Summary patch (18k)

POSIX Timer API
A: George Anzinger
M: http://marc.theaimsgroup.com/?l=linux-kernel&m=103553654329827&w=2
D: http://unc.dl.sourceforge.net/sourceforge/high-res-timers/hrtimers-posix-2.5.44-1.0.patch
S: 66 kbytes, 18/2 files altered, 4 new
T: Diffstat
X: Summary patch (21k)

Hotplug CPU Removal Support
A: Rusty Russell
D: http://www.kernel.org/pub/linux/kernel/people/rusty/patches/Hotcpu/hotcpu-cpudown.patch.gz
S: 32 kbytes, 16/0 files altered, 0 new
T: Diffstat
X: Summary patch (29k)

Hires Timers
A: George Anzinger
M: http://marc.theaimsgroup.com/?l=linux-kernel&m=103557676007653&w=2
M: http://marc.theaimsgroup.com/?l=linux-kernel&m=103557677207693&w=2
M: http://marc.theaimsgroup.com/?l=linux-kernel&m=103558349714128&w=2
D: http://unc.dl.sourceforge.net/sourceforge/high-res-timers/hrtimers-core-2.5.44-1.0.patch http://unc.dl.sourceforge.net/sourceforge/high-res-timers/hrtimers-i386-2.5.44-1.0.patch http://unc.dl.sourceforge.net/sourceforge/high-res-timers/hrtimers-hrposix-2.5.44-1.1.patch
S: 132 kbytes, 15/4 files altered, 10 new
T: Diffstat
X: Summary patch (44k)
N: Requires POSIX Timer API patch

EVMS
A: EVMS Team
M: http://www.uwsg.iu.edu/hypermail/linux/kernel/0208.0/0109.html
D: http://evms.sourceforge.net/patches/2.5.44/
S: 1101 kbytes, 7/10 files altered, 44 new
T: Diffstat
X: Summary patch (4k)

initramfs
A: Al Viro
M: http://www.cs.helsinki.fi/linux/linux-kernel/2001-30/0110.html
D: ftp://ftp.math.psu.edu/pub/viro/N0-initramfs-C21
S: 16 kbytes, 5/1 files altered, 2 new
T: Diffstat
X: Summary patch (5k)

Kernel Probes
A: Vamsi Krishna S
M: lists.insecure.org/linux-kernel/2002/Aug/1299.html
D: http://www.kernel.org/pub/linux/kernel/people/rusty/patches/Misc/kprobes.patch.gz
S: 18 kbytes, 4/2 files altered, 4 new
T: Diffstat
X: Summary patch (5k)