Unable to continue testing of 2.6.25

Unable to continue testing of 2.6.25

[Frans Pop]

Yesterday, after spending quite a few hours over the last days on bisecting 
some serious regressions and finding workarounds for them, I thought I 
could start using 2.6.25-rc2 as the new kernel for my desktop. 
Unfortunately I found that I cannot because it would make my other main 
activity - working on the Debian installation system - impossible.

For my work on the Debian Installer I heavily rely on emulators to run test 
installs and ATM my emulator of choice is VirtualBox (the fully open "ose" 
version). This requires the vboxdrv kernel module, but unfortunately:
   vboxdrv: Unknown symbol change_page_attr

At first I traced this to:
e1271f68
 x86: deprecate change_page_attr() for drivers
 With the introduction of the new API, no driver or non-archcore code needs
 to use c-p-a anymore, so this patch also deprecates the EXPORT_SYMBOL of
 CPA (it's a horrible API after all).
which had:
-EXPORT_SYMBOL(change_page_attr);
+EXPORT_UNUSED_SYMBOL(change_page_attr); /* to be removed in 2.6.27 */

Which seemed entirely reasonable but left me wondering about the error I 
got.

But then I found:
d1028a15
 x86: make various pageattr.c functions static
 change_page_attr_add is only used in pageattr.c now, so we can
 make this function static.
 change_page_attr() isn't used anywere at all anymore; this function
 is a really bad API anyway so just remove the bloat entirely.

Which removed the entire function (without even properly mentioning it in 
the shortlog).


OF COURSE it is up to the VirtualBox developers to adjust to the new 
interface (and based on past experience I expect they will with their next 
version). And it may very well be that they were totally braindead to use 
the function in the first place. I don't know and I really don't care.
The important fact for me is that I can no longer use a piece of software 
that is essential to me and thereby lose the motivation to do any work on 
the kernel.

Lesson of the day: thinking only about in-kernel users of published API when 
doing restructuring is going to lose you testers who do MORE with their 
systems than just kernel development.

Please allow external users some decent period for transitioning. The 
initial plan to "remove the old function in 2.6.27" was entirely sensible. 
It's a pity it was not followed through.

Thanks,
FJP

P.S. Of course I _will_ follow up on issues that I've already reported. I 
will just not be doing any new testing.

Re: Unable to continue testing of 2.6.25

[Adrian Bunk]

On Sun, Feb 17, 2008 at 10:25:30AM +0100, Frans Pop wrote:
>...
> For my work on the Debian Installer I heavily rely on emulators to run test 
> installs and ATM my emulator of choice is VirtualBox (the fully open "ose" 
> version). This requires the vboxdrv kernel module, but unfortunately:
>...
> OF COURSE it is up to the VirtualBox developers to adjust to the new 
> interface (and based on past experience I expect they will with their next 
> version). And it may very well be that they were totally braindead to use 
> the function in the first place. I don't know and I really don't care.
> The important fact for me is that I can no longer use a piece of software 
> that is essential to me and thereby lose the motivation to do any work on 
> the kernel.
> 
> Lesson of the day: thinking only about in-kernel users of published API when 
> doing restructuring is going to lose you testers who do MORE with their 
> systems than just kernel development.
>...

I get your problem, but you are looking in the wrong direction for
a solution.

The real problem is that the kernel seems to lack functionality you 
require for doing some work.

Why does your work on the Debian Installer depend on VirtualBox and
can't be done with what the kernel already ships?

> Thanks,
> FJP
>...

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Unable to continue testing of 2.6.25

[Tilman Schmidt]

[-- Attachment #1: Type: text/plain, Size: 911 bytes --]

Am 17.02.2008 14:16 schrieb Adrian Bunk:
> The real problem is that the kernel seems to lack functionality you 
> require for doing some work.
> 
> Why does your work on the Debian Installer depend on VirtualBox and
> can't be done with what the kernel already ships?

No, that's not the real problem. Even if the kernel didn't lack
any required functionality and it could all be done today without
VirtualBox, pulling the rug from underneath it like that leaves
all those who are currently relying on it without the ability to
continue testing newer kernels until they find the time to redesign
their working environment. Sticking to the removal schedule could
have avoided that.

HTH
T.

-- 
Tilman Schmidt                          E-Mail: tilman@imap.cc
Bonn, Germany
Diese Nachricht besteht zu 100% aus wiederverwerteten Bits.
Ungeöffnet mindestens haltbar bis: (siehe Rückseite)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 253 bytes --]

Re: Unable to continue testing of 2.6.25

[Adrian Bunk]

On Sun, Feb 17, 2008 at 08:24:56PM +0100, Tilman Schmidt wrote:
> Am 17.02.2008 14:16 schrieb Adrian Bunk:
>> The real problem is that the kernel seems to lack functionality you  
>> require for doing some work.
>>
>> Why does your work on the Debian Installer depend on VirtualBox and
>> can't be done with what the kernel already ships?
>
> No, that's not the real problem. Even if the kernel didn't lack
> any required functionality and it could all be done today without
> VirtualBox, pulling the rug from underneath it like that leaves
> all those who are currently relying on it without the ability to
> continue testing newer kernels until they find the time to redesign
> their working environment. Sticking to the removal schedule could
> have avoided that.

Frans said that he requires VirtualBox for his work on the Debian 
Installer.

If the kernel would offer everything Frans needs for his work on the 
Debian Installer he wouldn't have sent his email.

So let's fix the problem (kernel lacks functionality) and not the 
symptom (external module requires some export).

> HTH
> T.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Unable to continue testing of 2.6.25

[Paul Jackson]

Adrian wrote:
> So let's fix the problem (kernel lacks functionality)

That's the problem as understood by Adrian.

I hear another problem as well ... 

Frans wrote:
> Please allow external users some decent period for transitioning. The 
> initial plan to "remove the old function in 2.6.27" was entirely sensible. 
> It's a pity it was not followed through.

That seems plain enough to me.  It's not just the lack of functionality,
but that such lack apparently happened with too little warning.

If this is a fair representation of what happened, then seems to me that
we could have left that EXPORT_UNUSED_SYMBOL(change_page_attr) in place
a bit longer.

Or at least, if we really did have to make Frans life difficult like
this, we could offer more appreciation for how things might look from
his perspective.

-- 
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj@sgi.com> 1.940.382.4214

Re: Unable to continue testing of 2.6.25

[Arjan van de Ven]

On Sun, 17 Feb 2008 10:25:30 +0100
Frans Pop <elendil@planet.nl> wrote:

> Yesterday, after spending quite a few hours over the last days on
> bisecting some serious regressions and finding workarounds for them,
> I thought I could start using 2.6.25-rc2 as the new kernel for my
> desktop. Unfortunately I found that I cannot because it would make my
> other main activity - working on the Debian installation system -
> impossible.
> 
> For my work on the Debian Installer I heavily rely on emulators to
> run test installs and ATM my emulator of choice is VirtualBox (the
> fully open "ose" version). This requires the vboxdrv kernel module,
> but unfortunately: vboxdrv: Unknown symbol change_page_attr
> 
> At first I traced this to:
> e1271f68
>  x86: deprecate change_page_attr() for drivers
>  With the introduction of the new API, no driver or non-archcore code
> needs to use c-p-a anymore, so this patch also deprecates the
> EXPORT_SYMBOL of CPA (it's a horrible API after all).
> which had:
> -EXPORT_SYMBOL(change_page_attr);
> +EXPORT_UNUSED_SYMBOL(change_page_attr); /* to be removed in 2.6.27 */
> 
> Which seemed entirely reasonable but left me wondering about the
> error I got.
> 
> But then I found:
> d1028a15
>  x86: make various pageattr.c functions static
>  change_page_attr_add is only used in pageattr.c now, so we can
>  make this function static.
>  change_page_attr() isn't used anywere at all anymore; this function
>  is a really bad API anyway so just remove the bloat entirely.
> 
> Which removed the entire function (without even properly mentioning
> it in the shortlog).
> 

the initial plan was for a depreciation period. Sadly it was untenable since the API
was changing entirely to fix bugs and add a really important feature
(the ability to clflush the exact range rather than wbinvd'ing the caches of all cpus in the system), 
at which point we had to pull the function right away.

As for virtualbox; a fix for that already exists, and it consists of removing 25 lines of workarounds
for c-p-a bugs, and just using set_memory_x / set_memory_nx.



-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: Unable to continue testing of 2.6.25

[Arjan van de Ven]

On Sun, 17 Feb 2008 14:38:51 -0600
Paul Jackson <pj@sgi.com> wrote:

> Adrian wrote:
> > So let's fix the problem (kernel lacks functionality)
> 
> That's the problem as understood by Adrian.
> 
> I hear another problem as well ... 
> 
> Frans wrote:
> > Please allow external users some decent period for transitioning.
> > The initial plan to "remove the old function in 2.6.27" was
> > entirely sensible. It's a pity it was not followed through.
> 
> That seems plain enough to me.  It's not just the lack of
> functionality, but that such lack apparently happened with too little
> warning.
> 
> If this is a fair representation of what happened, then seems to me
> that we could have left that EXPORT_UNUSED_SYMBOL(change_page_attr)
> in place a bit longer.
> 

it's not a fair repersentation. Again.. this export was unkeepable due to the
API being nasty and having to be fixed anyway ;(.

One of the problems was that the c-p-a api has to be followed by a cache flush function call.
Sadly that does a TOTAL flush of the caches of all cpus in the system. As part of the -rc1
changes, it is now done only on the exact pages that need to be flushed (so you no longer
flush 12Mb of caches when you only needed to flush 4Kb), but to achieve that, it was no longer
an option to keep this as 2 separate function calls.
Add to this that some very fundemanteal bugs couldn't be fixed without the function underlying
cpa changing prototype and behavior, so the function had to go.

> Or at least, if we really did have to make Frans life difficult like
> this, we could offer more appreciation for how things might look from
> his perspective.

I understand where he's coming from; at the same time it's a very small change to virtualbox to fix this
and has been done already... in minutes. Frans should take that up with the virtual box support forum, I'm sure
they have the patch available there. (it's mostly removing workarounds for cpa bugs and then just calling set_memory_x / set_memory_nx).

-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: Unable to continue testing of 2.6.25

[David Miller]

From: Tilman Schmidt <tilman@imap.cc>
Date: Sun, 17 Feb 2008 20:24:56 +0100

> No, that's not the real problem. Even if the kernel didn't lack
> any required functionality and it could all be done today without
> VirtualBox, pulling the rug from underneath it like that leaves
> all those who are currently relying on it without the ability to
> continue testing newer kernels until they find the time to redesign
> their working environment.

No, it is VirtualBox's problem.  Nobody outside of the kernel
should be using that symbol, it can only be used for totally
unsupportable things as far as upstream is concerned.

Re: Unable to continue testing of 2.6.25

[Tilman Schmidt]

[-- Attachment #1: Type: text/plain, Size: 1833 bytes --]

David Miller schrieb:
> From: Tilman Schmidt <tilman@imap.cc>
> Date: Sun, 17 Feb 2008 20:24:56 +0100
> 
>> No, that's not the real problem. Even if the kernel didn't lack
>> any required functionality and it could all be done today without
>> VirtualBox, pulling the rug from underneath it like that leaves
>> all those who are currently relying on it without the ability to
>> continue testing newer kernels until they find the time to redesign
>> their working environment.
> 
> No, it is VirtualBox's problem.

Why's that beeping noise coming from my SEP field detector? :-)

>  Nobody outside of the kernel
> should be using that symbol, it can only be used for totally
> unsupportable things as far as upstream is concerned.

Then why was it exported in the first place?

Still, we are talking about two completely different types of
problem here: (a) the technical problem of why something stopped
working and how it should be fixed and (b) the practical problem
of users left standing in the rain while such fixing is taking
place. While (a) might look like the only real problem to kernel
developers, (b) often feels much more real to users.
Unfortunately, discussing alternative solutions to (a) does
little to solve (b).

The conventional way of addressing (b) is to avoid it altogether
by giving sufficient advance warning before removing a feature,
and sticking to the feature removal schedule. If (as apparently
in this case) that isn't possible, then other ways to mitigate
the problem need to be found. That may require switching off
some SEP field generators, though. :-)

HTH
T.

-- 
Tilman Schmidt                    E-Mail: tilman@imap.cc
Bonn, Germany
Diese Nachricht besteht zu 100% aus wiederverwerteten Bits.
Ungeöffnet mindestens haltbar bis: (siehe Rückseite)


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 250 bytes --]

Re: Unable to continue testing of 2.6.25

[Andi Kleen]

David Miller <davem@davemloft.net> writes:

> From: Tilman Schmidt <tilman@imap.cc>
> Date: Sun, 17 Feb 2008 20:24:56 +0100
>
>> No, that's not the real problem. Even if the kernel didn't lack
>> any required functionality and it could all be done today without
>> VirtualBox, pulling the rug from underneath it like that leaves
>> all those who are currently relying on it without the ability to
>> continue testing newer kernels until they find the time to redesign
>> their working environment.
>
> No, it is VirtualBox's problem.  Nobody outside of the kernel
> should be using that symbol, 

> it can only be used for totally
> unsupportable things as far as upstream is concerned.

Creating any uncacheable or write protected mappings in unsupportable? 
I suspect you would have a hard time actually justifying this. That's
a relatively common and useful operation in device drivers.

The only special case I would agree with you is doing write-combined
mappings using this which is indeed generally unsupported yet on x86
before the full PAT infrastructure goes in (but 
I cannot imagine VirtualBox would use WC for anything) 

Most likely if they just want to write protect something
they should either use the new interface or just open code
it using lookup_address() / change ptes / flush tlbs.

-Andi

Re: Unable to continue testing of 2.6.25

[Andi Kleen]

Arjan van de Ven <arjan@infradead.org> writes:
>
> the initial plan was for a depreciation period. Sadly it was untenable since the API
> was changing entirely to fix bugs and add a really important feature
> (the ability to clflush the exact range rather than wbinvd'ing the caches of all cpus in the system), 

Just for the record: I posted full patches to implement clflush
support some time ago without changing any exported API. So your
claims that changing the API was needed to implement CLFLUSH are not
correct.

Also I believe some assumptions behind the new API are faulty (in
particular that the caller doesn't fully own the to be changed pages)
and make it actually impossible to implement the cache attribute PTE
changing operation fully correct according to the Intel x86 manual
(which requires temporary unmap)

-Andi

Re: Unable to continue testing of 2.6.25

[Arjan van de Ven]

On Mon, 18 Feb 2008 13:31:48 +0100
Andi Kleen <andi@firstfloor.org> wrote:

> Arjan van de Ven <arjan@infradead.org> writes:
> >
> > the initial plan was for a depreciation period. Sadly it was
> > untenable since the API was changing entirely to fix bugs and add a
> > really important feature (the ability to clflush the exact range
> > rather than wbinvd'ing the caches of all cpus in the system), 
> 
> Just for the record: I posted full patches to implement clflush
> support some time ago without changing any exported API. So your
> claims that changing the API was needed to implement CLFLUSH are not
> correct.

yeah of course it is possible to make things "smart" by having hidden state.
doesn't make it right.

> 
> Also I believe some assumptions behind the new API are faulty (in
> particular that the caller doesn't fully own the to be changed pages)
> and make it actually impossible to implement the cache attribute PTE
> changing operation fully correct according to the Intel x86 manual
> (which requires temporary unmap)

the Intel x86 manual explicitly only has a temporary unmap when going from a 
cached state to a write-combining state. Any other transition does not require
an unmap. Which makes this not impossible, all a cached->WC transition needs
to do is go via an intermediate UC state and the really expensive process from
the manual is not needed.


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: Unable to continue testing of 2.6.25

[Andi Kleen]

On Mon, Feb 18, 2008 at 08:50:18AM -0800, Arjan van de Ven wrote:
> On Mon, 18 Feb 2008 13:31:48 +0100
> Andi Kleen <andi@firstfloor.org> wrote:
> 
> > Arjan van de Ven <arjan@infradead.org> writes:
> > >
> > > the initial plan was for a depreciation period. Sadly it was
> > > untenable since the API was changing entirely to fix bugs and add a
> > > really important feature (the ability to clflush the exact range
> > > rather than wbinvd'ing the caches of all cpus in the system), 
> > 
> > Just for the record: I posted full patches to implement clflush
> > support some time ago without changing any exported API. So your
> > claims that changing the API was needed to implement CLFLUSH are not
> > correct.
> 
> yeah of course it is possible to make things "smart" by having hidden state.
> doesn't make it right.

Not sure how you can call global_flush_tlb() "hidden". It was a quite
exposed interface.

Anyways there was also no principle reason in the old interface why
the flush couldn't have been done immediately. The only reason
it wasn't done was that Linus long ago asked for separate
flushing to improve efficiency of large remaps.

> > Also I believe some assumptions behind the new API are faulty (in
> > particular that the caller doesn't fully own the to be changed pages)
> > and make it actually impossible to implement the cache attribute PTE
> > changing operation fully correct according to the Intel x86 manual
> > (which requires temporary unmap)
> 
> the Intel x86 manual explicitly only has a temporary unmap when going from a 
> cached state to a write-combining state. Any other transition does not require
> an unmap. Which makes this not impossible, all a cached->WC transition needs

True, that's a possible workaround for this deficiency of the new API.

> to do is go via an intermediate UC state and the really expensive process from
> the manual is not needed.

Ok then you're proposing to use a even more expensive operation just
to patch this over. I guess that will work as long as we assume
none of the callers cares too much about performance, but trying to describe
it as an improvement is quite a stretch.

-Andi

Re: Unable to continue testing of 2.6.25

[Arjan van de Ven]

On Mon, 18 Feb 2008 18:11:59 +0100
Andi Kleen <andi@firstfloor.org> wrote:
> > to do is go via an intermediate UC state and the really expensive
> > process from the manual is not needed.
> 
> Ok then you're proposing to use a even more expensive operation just
> to patch this over. I guess that will work as long as we assume
> none of the callers cares too much about performance, 

I've yet to see a user who wants WC. Lets face it, WC *sucks*. This is why
the folks who care about performance (the graphics guys) stopped using it.
WC is slow, and on modern cpus leads to really bad performance. I'm really
half tempted to just ignore WC entirely and suggest that we don't even implement
it in the kernel. Yes it's really that bad.


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: Unable to continue testing of 2.6.25

[Arjan van de Ven]

On Mon, 18 Feb 2008 19:52:28 +0100
Andi Kleen <andi@firstfloor.org> wrote:

> > I've yet to see a user who wants WC. Lets face it, WC *sucks*. This
> > is why
> 
> Interesting.
>  
> > the folks who care about performance (the graphics guys) stopped
> > using it.
> 
> I didn't know this. What do they do instead?

they use a cached mapping and use clflush for the cases they want to be sure
the GPU sees the data. This turns out to be faster than WC.
(think "double to tripple digit percentage")

 
> I understand that WC was added originally because AGP was really slow
> at IO towards the CPU. You mean on PCI-E it is fast enough now
> that standard cached WB works well enough?

since then the graphics programming paradigm has changed as well;
WC is really bad for reading data no matter what; it focuses on group writes 
so that you don't get one transaction per write, but reads are extremely slow.
And apparently in the current graphics systems (with "composite" and the like)
all of this stuff doesn't get used anymore. Instead they use cached mappings,
acceleration engines etc.

> 
> > WC is slow, and on modern cpus leads to really bad performance. I'm
> > really half tempted to just ignore WC entirely and suggest that we
> > don't even implement it in the kernel. Yes it's really that bad.
> 
> At least the X server still uses it.

depends on which driver. The new generation Intel graphics ones don't.
I doubt the proprietary ones do either, those are even more performance tuned.
I understand that the radeon driver is doing or going to do something similar to the
Intel driver as well.

> 
> What would you recommend should the X server use instead? Always
> map standard WB?  How about on older AGP systems?
>

the X drivers need to do what is best; the ones with acceleration use WB nowadays.
(For the Intel driver that is somewhat recent, so it could be that your distro
doesn't do that yet).



-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: Unable to continue testing of 2.6.25

[Alan Cox]

> I've yet to see a user who wants WC. Lets face it, WC *sucks*. This is why
> the folks who care about performance (the graphics guys) stopped using it.

WC for main memory or WC for mmio spaces ?

Re: Unable to continue testing of 2.6.25

[Andi Kleen]

> I've yet to see a user who wants WC. Lets face it, WC *sucks*. This is why

Interesting.
 
> the folks who care about performance (the graphics guys) stopped using it.

I didn't know this. What do they do instead?

I understand that WC was added originally because AGP was really slow
at IO towards the CPU. You mean on PCI-E it is fast enough now
that standard cached WB works well enough?

> WC is slow, and on modern cpus leads to really bad performance. I'm really
> half tempted to just ignore WC entirely and suggest that we don't even implement
> it in the kernel. Yes it's really that bad.

At least the X server still uses it. In fact there are already some
performance regression regarding this from differing kernel behavioun
in the sysfs interfaces vs /dev/mem.

What would you recommend should the X server use instead? Always
map standard WB?  How about on older AGP systems?

-Andi

Re: Unable to continue testing of 2.6.25

[Arjan van de Ven]

On Mon, 18 Feb 2008 18:40:50 +0000
Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> > I've yet to see a user who wants WC. Lets face it, WC *sucks*. This
> > is why the folks who care about performance (the graphics guys)
> > stopped using it.
> 
> WC for main memory or WC for mmio spaces ?

if you mean 'framebuffer' with mmio space, they stopped using it there
as well afaik.


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: Unable to continue testing of 2.6.25

[Roland Dreier]

 > I've yet to see a user who wants WC. Lets face it, WC *sucks*. This is why
 > the folks who care about performance (the graphics guys) stopped using it.
 > WC is slow, and on modern cpus leads to really bad performance. I'm really
 > half tempted to just ignore WC entirely and suggest that we don't even implement
 > it in the kernel. Yes it's really that bad.

I know of one case at least where WC is very useful.  Some InfiniBand
adapters allow small messages to be written directly into the
adapter's PCI space BAR to lower latency (having the CPU write the
message avoids doing something like build descriptor, ring doorbell
register on adapter, adapter DMA message out of CPU memory).  And
mapping the PCI space with WC is a pretty big win -- for example for
mlx4 hardware it gets MPI latency from ~1.8 usec to ~1.3 usec which is
a big deal.  I think most real users of mlx4 hardware are using a
hacky out-of-tree patch to allow using PAT to set WC.

AFAIK mapping PCI memory WB is not allowed, so WC is really our only choice.

 - R.

Re: Unable to continue testing of 2.6.25

[Arjan van de Ven]

On Mon, 18 Feb 2008 10:53:42 -0800
Roland Dreier <rdreier@cisco.com> wrote:

> AFAIK mapping PCI memory WB is not allowed, so WC is really our only
> choice.

afaik that depends on the BAR being prefetchable or not.

(and by your argument, ioremap_cached() would not be useful, and since that was, until
2.6.25-rc1, the default behavior for ioremap(), would have caused massive problems)


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: Unable to continue testing of 2.6.25

[Ingo Molnar]

* Andi Kleen <andi@firstfloor.org> wrote:

> > I've yet to see a user who wants WC. Lets face it, WC *sucks*. This 
> > is why
> 
> Interesting.

does this refresh your memory:

  http://lkml.org/lkml/2008/1/10/99

	Ingo

Re: Unable to continue testing of 2.6.25

[Roland Dreier]

 > > AFAIK mapping PCI memory WB is not allowed, so WC is really our only
 > > choice.

 > afaik that depends on the BAR being prefetchable or not.

In my case the BAR is prefetchable.

 > (and by your argument, ioremap_cached() would not be useful, and since that was, until
 > 2.6.25-rc1, the default behavior for ioremap(), would have caused massive problems)

I'm not sure what ioremap_cached() would really do in my case, since
the MTRRs for PCI memory are set to UC, so without monkeying with MTRR
contents (which can't really be done safely) the only choices we have
are leaving the mapping as UC or using PAT to get WC.

Also in my case I'm more concerned about latency of finishing a small
write rather than througput.  So I'm not sure that I would really want
to do a write to a WB mapping followed by CLFLUSH anyway.

 - R.

Re: Unable to continue testing of 2.6.25

[Alan Cox]

> if you mean 'framebuffer' with mmio space, they stopped using it there
> as well afaik.

And also bursting commands to mmio fifos - eg the 3Dfx where it makes a
*huge* difference.

Alan

Re: Unable to continue testing of 2.6.25

[Andi Kleen]

On Mon, Feb 18, 2008 at 08:18:41PM +0100, Ingo Molnar wrote:
> 
> * Andi Kleen <andi@firstfloor.org> wrote:
> 
> > > I've yet to see a user who wants WC. Lets face it, WC *sucks*. This 
> > > is why
> > 
> > Interesting.
> 
> does this refresh your memory:
> 
>   http://lkml.org/lkml/2008/1/10/99

I'm still far from convinced it covers all cases (see also Roland's example)
Arjan might be right for some modern graphics hardware, but there is
a lot more hardware out there than only this, old and new and non graphics
and graphics that Arjan didn't cover.

If Arjan was 100% right then PAT would not be needed at all. I wonder
why Venki/Suresh went through all the pain of resurrecting the old
PAT patchkit then!? Surely there is more about this.

-Andi

Re: Unable to continue testing of 2.6.25

[Siddha, Suresh B]

On Mon, Feb 18, 2008 at 11:18:48AM -0800, Roland Dreier wrote:
>  > > AFAIK mapping PCI memory WB is not allowed, so WC is really our only
>  > > choice.
> 
>  > afaik that depends on the BAR being prefetchable or not.
> 
> In my case the BAR is prefetchable.

Even if the BAR is prefetchable, on some platforms mapping MMIO space
as WB can cause bad results like machine check etc.

>  > (and by your argument, ioremap_cached() would not be useful, and since that was, until
>  > 2.6.25-rc1, the default behavior for ioremap(), would have caused massive problems)
> 
> I'm not sure what ioremap_cached() would really do in my case, since
> the MTRRs for PCI memory are set to UC, so without monkeying with MTRR
> contents (which can't really be done safely) the only choices we have
> are leaving the mapping as UC or using PAT to get WC.

thanks,
suresh

Re: Unable to continue testing of 2.6.25

[Frans Pop]

On Sunday 17 February 2008, Adrian Bunk wrote:
> The real problem is that the kernel seems to lack functionality you
> require for doing some work.

Not sure how you reached that conclusion.

> Why does your work on the Debian Installer depend on VirtualBox and
> can't be done with what the kernel already ships?

Work on the installer does not so much hard depend on VirtualBox (or any 
other emulator), but can be done much more effectively and efficiently 
using one.

It allows me to run the installer inside the emulator without the need for a 
second computer (and using the same keyboard). It allows me to take 
snapshots just before stages I'm interested in and then add debugging or 
try changes. If what I tried does not work, I can just revert to the 
snapshot and try something else without having to run the full installation 
from scratch. It allows me to easily test RAID setups without having 
multiple physical disks. Etc.

The fact that VirtualBox does not (yet) work for me with 2.6.25 means that 
I'm unable to run 2.6.25 on my main desktop and do any real work. Rebooting 
my desktop whenever I want to use VirtualBox is just not a realistic 
option.
That in turn means that I cannot do any more testing of 2.6.25, because most 
of the testing I do consists of just using a new kernel and critically 
observing the behavior of my system. As I've caught a fair number of bugs 
that way for the past few releases, I'd say that's a useful contribution.

Re: Unable to continue testing of 2.6.25

[Arjan van de Ven]

On Tue, 19 Feb 2008 22:55:01 +0100
Frans Pop <elendil@planet.nl> wrote:

> On Sunday 17 February 2008, Adrian Bunk wrote:
> > The real problem is that the kernel seems to lack functionality you
> > require for doing some work.
> 
> Not sure how you reached that conclusion.
> 
> > Why does your work on the Debian Installer depend on VirtualBox and
> > can't be done with what the kernel already ships?
> 
> Work on the installer does not so much hard depend on VirtualBox (or
> any other emulator), but can be done much more effectively and
> efficiently using one.
> 
> It allows me to run the installer inside the emulator without the
> need for a second computer (and using the same keyboard). It allows
> me to take snapshots just before stages I'm interested in and then
> add debugging or try changes. If what I tried does not work, I can
> just revert to the snapshot and try something else without having to
> run the full installation from scratch. It allows me to easily test
> RAID setups without having multiple physical disks. Etc.
> 
> The fact that VirtualBox does not (yet) work for me with 2.6.25 means
> that I'm unable to run 2.6.25 on my main desktop and do any real
> work. Rebooting my desktop whenever I want to use VirtualBox is just
> not a realistic option.
> That in turn means that I cannot do any more testing of 2.6.25,

I assume you've read the other mails right and went to the virtualbox form
to get the small patch to make it work on .25 right?


-- 
If you want to reach me at my work email, use arjan@linux.intel.com
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org

Re: Unable to continue testing of 2.6.25

[Harvey Harrison]

On Tue, 2008-02-19 at 22:55 +0100, Frans Pop wrote:
> On Sunday 17 February 2008, Adrian Bunk wrote:
> > The real problem is that the kernel seems to lack functionality you
> > require for doing some work.
> 
> Not sure how you reached that conclusion.
> 
> > Why does your work on the Debian Installer depend on VirtualBox and
> > can't be done with what the kernel already ships?
> 
> The fact that VirtualBox does not (yet) work for me with 2.6.25 means that 
> I'm unable to run 2.6.25 on my main desktop and do any real work. Rebooting 
> my desktop whenever I want to use VirtualBox is just not a realistic 
> option.
> That in turn means that I cannot do any more testing of 2.6.25, because most 
> of the testing I do consists of just using a new kernel and critically 
> observing the behavior of my system. As I've caught a fair number of bugs 
> that way for the past few releases, I'd say that's a useful contribution.

And it will be just as valuable when you stay with 2.6.24 finding bugs
there.

And once VirtualBox has been updated to work with 2.6.25, you can test
2.6.25.

Harvey

Re: Unable to continue testing of 2.6.25

[Adrian Bunk]

On Tue, Feb 19, 2008 at 10:55:01PM +0100, Frans Pop wrote:
> On Sunday 17 February 2008, Adrian Bunk wrote:
> > The real problem is that the kernel seems to lack functionality you
> > require for doing some work.
> 
> Not sure how you reached that conclusion.
> 
> > Why does your work on the Debian Installer depend on VirtualBox and
> > can't be done with what the kernel already ships?
> 
> Work on the installer does not so much hard depend on VirtualBox (or any 
> other emulator), but can be done much more effectively and efficiently 
> using one.
>...

The "or any other emulator" is exactly where my question is directed at.

Xen, KVM or even qemu come into my mind, but considering how loudly you 
complained about a temporary breakage for VirtualBox there must be a 
reason why your work on the Debian Installer can only be done 
effectively and efficiently with an emulator module that has AFAIK not 
been submitted for inclusion in the kernel.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Unable to continue testing of 2.6.25

[Frans Pop]

On Sunday 17 February 2008, Arjan van de Ven wrote:
> On Sun, 17 Feb 2008 14:38:51 -0600 Paul Jackson <pj@sgi.com> wrote:
> > Adrian wrote:
> > > So let's fix the problem (kernel lacks functionality)
> >
> > That's the problem as understood by Adrian.
> >
> > I hear another problem as well ...
> >
> > Frans wrote:
> > > Please allow external users some decent period for transitioning.
> > > The initial plan to "remove the old function in 2.6.27" was
> > > entirely sensible. It's a pity it was not followed through.
> >
> > That seems plain enough to me.  It's not just the lack of
> > functionality, but that such lack apparently happened with too little
> > warning.
> >
> > If this is a fair representation of what happened, then seems to me
> > that we could have left that EXPORT_UNUSED_SYMBOL(change_page_attr)
> > in place a bit longer.
>
> it's not a fair repersentation. Again.. this export was unkeepable due to
> the API being nasty and having to be fixed anyway ;(.
>
> One of the problems was that the c-p-a api has to be followed by a cache
> flush function call. Sadly that does a TOTAL flush of the caches of all
> cpus in the system. As part of the -rc1 changes, it is now done only on
> the exact pages that need to be flushed (so you no longer flush 12Mb of
> caches when you only needed to flush 4Kb), but to achieve that, it was no
> longer an option to keep this as 2 separate function calls.
> Add to this that some very fundemanteal bugs couldn't be fixed without
> the function underlying cpa changing prototype and behavior, so the
> function had to go.

That's a much better explanation than can be found in the changelog.

The changelog effectively says: this commits makes a few (new) functions 
static; oh, and by the way, let's delete the old function _because it is 
unused and ugly_.

Well, I've shown that the first is not true and the second by itself is 
absolutely not sufficient reason to remove an exported function that has 
long been part of the kernel.
I would probably have started this thread differently if the removal had 
been done in a separate commit and had included the explanation you give 
above.

> I understand where he's coming from; at the same time it's a very small
> change to virtualbox to fix this and has been done already... in minutes.

The problem here is that it may be a simple change for you and other 
similarly gifted people, but it's something that's above _my_ skills.

> Frans should take that up with the virtual box support forum, I'm sure

I did actually manage to create such a patch for the breakage in the 
VirtualBox source because of 2.6.24, but those really were very minor 
issues (basically overlapping definitions). I posted that on their user 
list (to which I am subscribed), but I never saw any response to it.

I also don't think it is really realistic to expect Innotek to provide 
patches for unreleased kernel versions. Maybe some other user could provide 
me with the patch, but I'm not as confident as you are.

My point still is that I feel they should not have to. That it's also the 
kernel developers responsibility to ensure backwards compatibility or at 
least a grace period for conversion whenever possible.
And IMO that not only goes for user-space API, but also for interfaces used 
by out-of-tree kernel modules.

Is it really impossible in this case for example to rewrite the old function 
so that it becomes a wrapper around the new interface? If it is impossible, 
then so be it.

> they have the patch available there.

I haven't seen anything on the user mailing list yet...

On Tuesday 19 February 2008, Arjan van de Ven wrote:
> I assume you've read the other mails right and went to the virtualbox
> form to get the small patch to make it work on .25 right?

If you can give me a link to that patch, I'd be grateful. As I said, I'm 
subscribed to their user list but have not seen any patch there.
I've also googled for it, without result.
I've also just quickly checked their "VirtualBox on Linux" forum, but did 
not see any obvious existing post about it.

Cheers,
FJP

Re: Unable to continue testing of 2.6.25

[Frans Pop]

On Tuesday 19 February 2008, Adrian Bunk wrote:
> The "or any other emulator" is exactly where my question is directed at.
>
> Xen, KVM or even qemu come into my mind, but considering how loudly you
> complained about a temporary breakage for VirtualBox there must be a
> reason why your work on the Debian Installer can only be done
> effectively and efficiently with an emulator module that has AFAIK not
> been submitted for inclusion in the kernel.

- Xen is currently not supported by the kernel Debian Installer uses (though 
work is being done to change that.
- KVM AFAIK requires hardware support that I don't have.
- QEMU is completely useless because of its slow speed without the (also out 
of tree) kqemu module, which does not work when the host system is x86_64 
[1]. Also, I very much prefer the VirtualBox user interface over what qemu 
has to offer.
- I've actually used VMWare for a long time (licenced), but stopped after 
the 5 series stopped working with current kernels and around that time 
VirtualBox became available as an alternative.

Hope that explains.

Cheers,
FJP

[1] http://bugs.debian.org/444160