2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1255 bytes --]

(Yes, I know NVidia is evil and all that.. If you're not Ingo or NVidia,
consider this "documenting the workaround" ;)

For reasons I can't explain, the NVidia module won't initialize
correctly with V0-0.7.33-03 if built with CONFIG_SPINLOCK_BKL.  It however
works fine with CONFIG_PREEMPT_BKL, changing nothing else in the config.
It also works fine with 2.6.10-rc3-mm1 without Ingo's patch.

Relevant .config snippet from the /proc/config.gz I'm running with right now:

# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT_DESKTOP=y
# CONFIG_PREEMPT_RT is not set
CONFIG_PREEMPT=y
CONFIG_PREEMPT_SOFTIRQS=y
CONFIG_PREEMPT_HARDIRQS=y
# CONFIG_SPINLOCK_BKL is not set
CONFIG_PREEMPT_BKL=y

If built with SPINLOCK_BKL, we get this in the kernel messages:

Dec 16 01:12:41 turing-police kernel: NVRM: rm_init_adapter failed

rm_init_adapter is in NVidia's binary code, so I can't shoot it.

The *odd* part is that it's failing with the spinlock but *not* the preempt version
or the unpatched version. (I was *expecting* that code that was happy with
the old-style BKL in -mm1 would be happy with the spinlock version and complain
if somebody hit the preempt flavor, not the other way around...)

Ingo? This ring any bells?

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Steven Rostedt]

On Thu, 2004-12-16 at 11:26 -0500, Valdis.Kletnieks@vt.edu wrote:
> (Yes, I know NVidia is evil and all that.. If you're not Ingo or NVidia,
> consider this "documenting the workaround" ;)
> 
> For reasons I can't explain, the NVidia module won't initialize
> correctly with V0-0.7.33-03 if built with CONFIG_SPINLOCK_BKL.  It however
> works fine with CONFIG_PREEMPT_BKL, changing nothing else in the config.
> It also works fine with 2.6.10-rc3-mm1 without Ingo's patch.

You were able to get the NVidia driver to work? Most of my machines have
the NVidia card (yes evil, but I like them) and I haven't been able to
get them to work on the rc3-mm1 (and a few earlier). Grant you, I didn't
try hard, but I did try a little on V0.7.33-0, and gave up later. 

How did you get by the...

1) pgd_offset_k_is_obsolete (not too hard, just a few patches for me)
2) class_simple_create and friends going to GPL (I just removed the GPL
from my code)
3) for Ingo's patches only, the might_sleep in the os_interface section.
having interrupts turned off. (here's where mine failed, I tried saving
and restoring them, turning them on that is, backwards from the normal
local_irq_save, and it would just be unstable here).

Do you have it working for the 2.6.10-rc3-mm1 without Ingo's patches?

Thanks,

-- Steve

Re: 2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Steven Rostedt]

On Fri, 2004-12-17 at 11:19 -0500, Steven Rostedt wrote:
> On Thu, 2004-12-16 at 11:26 -0500, Valdis.Kletnieks@vt.edu wrote:
> > (Yes, I know NVidia is evil and all that.. If you're not Ingo or NVidia,
> > consider this "documenting the workaround" ;)
> > 
> > For reasons I can't explain, the NVidia module won't initialize
> > correctly with V0-0.7.33-03 if built with CONFIG_SPINLOCK_BKL.  It however
> > works fine with CONFIG_PREEMPT_BKL, changing nothing else in the config.
> > It also works fine with 2.6.10-rc3-mm1 without Ingo's patch.
> 
> You were able to get the NVidia driver to work? Most of my machines have
> the NVidia card (yes evil, but I like them) and I haven't been able to
> get them to work on the rc3-mm1 (and a few earlier). Grant you, I didn't
> try hard, but I did try a little on V0.7.33-0, and gave up later. 
> 

Update: I just tried some of my fixes to the rc3-mm1 kernel, and that
worked without a problem.  But I still didn't get by the sleep problem
in Ingo's RT patch.  Did you get further, and did you make fixes to both
the nvidia module as well as the kernel?


> How did you get by the...
> 
> 1) pgd_offset_k_is_obsolete (not too hard, just a few patches for me)
> 2) class_simple_create and friends going to GPL (I just removed the GPL
> from my code)
> 3) for Ingo's patches only, the might_sleep in the os_interface section.
> having interrupts turned off. (here's where mine failed, I tried saving
> and restoring them, turning them on that is, backwards from the normal
> local_irq_save, and it would just be unstable here).
> 
> Do you have it working for the 2.6.10-rc3-mm1 without Ingo's patches?
> 
> Thanks,
> 
> -- Steve
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: 2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Valdis.Kletnieks]

[-- Attachment #1.1: Type: text/plain, Size: 1852 bytes --]

On Fri, 17 Dec 2004 12:03:31 EST, Steven Rostedt said:

> Update: I just tried some of my fixes to the rc3-mm1 kernel, and that
> worked without a problem.  But I still didn't get by the sleep problem
> in Ingo's RT patch.  Did you get further, and did you make fixes to both
> the nvidia module as well as the kernel?

I have to admit I haven't *hit* a sleep problem specific to Ingo's code, unless
you have a different config/hardware and my BKL wierdness and your sleep are
2 different manifestations of the same problem.  Or maybe they're 2 different
bugs... ;)

I'm running Ingo's patch and the nvidia 6629 drivers as I'm typing this. Given
you had to fool with pgd_offset_k and friends, you're probably trying an older
driver (6111?) and should upgrade - the 6629 picked up a *bunch* of 2.6-related
fixes.  Maybe 6629 fixed your sleep issue?

I needed to make 1 small patch to the nvidia code to support the 4-level
page table code in -mm, and one small kernel patch from Zander at NVidia to work
around another issue.  Both attached.  You apparently already have the GPL-export
fix.  If you're not running Ingo's patch, you'll need the non-NVidia-related
patch to fix the ioctl-cleanup.patch breakage (Ingo rolled that one into -RT),
but if you got it to come up at all in -rc3-mm1, you have that one too (but those
of you playing along at home will need it)...

Modulo those patches, the only thing I've hit since 2.6.10-rc1-mm1 or so is
this BKL strangeness, and that only bites me in -RT with one configuration...

YMMV, binary modules are evil, Works For Me with 6629 on my Dell laptop, and
all that. If this isn't enough to get it working for you, your best bet is
probably to wander over to

http://www.nvnews.net/vbulletin/forumdisplay.php?s=&forumid=14

and see if Zander has new words of wisdom (that's where I found the VM-IO patch).


[-- Attachment #1.2: VM_IO.diff --]
[-- Type: text/plain , Size: 479 bytes --]

--- linux-2.6.10-rc2-mm2/mm/mmap.c.nvidia	2004-11-18 22:10:39.259326934 -0500
+++ linux-2.6.10-rc2-mm2/mm/mmap.c	2004-11-18 22:22:46.877972513 -0500
@@ -1026,7 +1026,8 @@ out:	
 	__vm_stat_account(mm, vm_flags, file, len >> PAGE_SHIFT);
 	if (vm_flags & VM_LOCKED) {
 		mm->locked_vm += len >> PAGE_SHIFT;
-		make_pages_present(addr, addr + len);
+		if (!(vm_flags & VM_IO))
+			make_pages_present(addr, addr + len);
 	}
 	if (flags & MAP_POPULATE) {
 		up_write(&mm->mmap_sem);

[-- Attachment #1.3: nv.diff --]
[-- Type: text/plain , Size: 1033 bytes --]

diff -upr NVIDIA-Linux-x86-1.0-6629-pkg1/usr/src/nv/nv.c ../src/NVIDIA-Linux-x86-1.0-6629-pkg1/usr/src/nv/nv.c
--- NVIDIA-Linux-x86-1.0-6629-pkg1/usr/src/nv/nv.c	2004-11-03 16:53:00.000000000 -0500
+++ ../src/NVIDIA-Linux-x86-1.0-6629-pkg1/usr/src/nv/nv.c	2004-11-10 16:18:29.000000000 -0500
@@ -2492,6 +2492,7 @@ _get_phys_address(
 )
 {
     struct mm_struct *mm;
+    pml4_t *pg_l4_dir;
     pgd_t *pg_dir;
     pmd_t *pg_mid_dir;
     pte_t *pte;
@@ -2500,10 +2501,15 @@ _get_phys_address(
     mm = (kern) ? &init_mm : current->mm;
     spin_lock(&mm->page_table_lock);
 
-    if (kern) pg_dir = pgd_offset_k(address);
-    else pg_dir = pgd_offset(mm, address);
+    if (kern) pg_l4_dir = pml4_offset_k(address);
+    else pg_l4_dir = pml4_offset(mm, address);
 
-    if (!pg_dir || pgd_none(*pg_dir))
+    if (!pg_l4_dir || pml4_none(*pg_l4_dir))
+        goto failed;
+
+    pg_dir = pml4_pgd_offset_k(pg_l4_dir,address);
+
+    if (pgd_none(*pg_dir))
         goto failed;
 
     NV_PMD_OFFSET(address, pg_dir, pg_mid_dir);

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Steven Rostedt]

On Fri, 2004-12-17 at 13:10 -0500, Valdis.Kletnieks@vt.edu wrote:
> On Fri, 17 Dec 2004 12:03:31 EST, Steven Rostedt said:
> 
> > Update: I just tried some of my fixes to the rc3-mm1 kernel, and that
> > worked without a problem.  But I still didn't get by the sleep problem
> > in Ingo's RT patch.  Did you get further, and did you make fixes to both
> > the nvidia module as well as the kernel?
> 
> I have to admit I haven't *hit* a sleep problem specific to Ingo's code, unless
> you have a different config/hardware and my BKL wierdness and your sleep are
> 2 different manifestations of the same problem.  Or maybe they're 2 different
> bugs... ;)
> 
> I'm running Ingo's patch and the nvidia 6629 drivers as I'm typing this. Given
> you had to fool with pgd_offset_k and friends, you're probably trying an older
> driver (6111?) and should upgrade - the 6629 picked up a *bunch* of 2.6-related
> fixes.  Maybe 6629 fixed your sleep issue?
> 

Nope! I have the 6629. Actually, the patch you have for NV solved the
pgd_offset problem. But I'm amazed that you didn't get into the
may_sleep calls.  In the nvidia code os-interface.c, would call hooks
into the kernel with interrupts turned off and hit the may_sleep.  But
looking into it further now, one of the crashes came from
ioremap_nocache and the sleep happened in kmem_cache_alloc. So maybe
Ingo fixed this.  I lied earlier (not intentionally), the last kernel I
tried with the NVidia was V0.7.32-18, so let me try again.

Thanks, for the references though.

-- Steve

Re: 2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 968 bytes --]

On Fri, 17 Dec 2004 15:04:21 EST, Steven Rostedt said:

> Nope! I have the 6629. Actually, the patch you have for NV solved the
> pgd_offset problem.

Glad to hear it, I'm just the messenger on that one (I forget which lkml
denizen actually wrote/posted that one).

>                       But I'm amazed that you didn't get into the
> may_sleep calls.

I'm actually not *that* surprised - my test base consists entirely of one Dell
Latitude C840 laptop with a GeForce4 440 Go card, so there's no SMP issues or
similar, and I don't do heavy 3D or anything unless xscreensaver decides to use
a random OpenGL display hack. Most likely, your hardware and/or CONFIG_* setup
is getting it into code paths that never get hit on my system. And the
might_sleep() is almost certainly on one of those paths.

(For the record, I *did* see a few might_sleep hits on 2.6.10-rc2-mm4-V0.7.31-15,
but they only hit sendmail, gpg, bash, and stuff like that, never an X-based program.)

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Steven Rostedt]

On Fri, 2004-12-17 at 15:04 -0500, Steven Rostedt wrote:
> On Fri, 2004-12-17 at 13:10 -0500, Valdis.Kletnieks@vt.edu wrote:
> > On Fri, 17 Dec 2004 12:03:31 EST, Steven Rostedt said:
> > 
> > > Update: I just tried some of my fixes to the rc3-mm1 kernel, and that
> > > worked without a problem.  But I still didn't get by the sleep problem
> > > in Ingo's RT patch.  Did you get further, and did you make fixes to both
> > > the nvidia module as well as the kernel?
> > 
> > I have to admit I haven't *hit* a sleep problem specific to Ingo's code, unless
> > you have a different config/hardware and my BKL wierdness and your sleep are
> > 2 different manifestations of the same problem.  Or maybe they're 2 different
> > bugs... ;)
> > 
> > I'm running Ingo's patch and the nvidia 6629 drivers as I'm typing this. Given
> > you had to fool with pgd_offset_k and friends, you're probably trying an older
> > driver (6111?) and should upgrade - the 6629 picked up a *bunch* of 2.6-related
> > fixes.  Maybe 6629 fixed your sleep issue?
> > 
> 
> Nope! I have the 6629. Actually, the patch you have for NV solved the
> pgd_offset problem. But I'm amazed that you didn't get into the
> may_sleep calls.  In the nvidia code os-interface.c, would call hooks
> into the kernel with interrupts turned off and hit the may_sleep.  But
> looking into it further now, one of the crashes came from
> ioremap_nocache and the sleep happened in kmem_cache_alloc. So maybe
> Ingo fixed this.  I lied earlier (not intentionally), the last kernel I
> tried with the NVidia was V0.7.32-18, so let me try again.
> 

I just tried it again, but this time with V0.7.33-04 and it still has
the same problems.  I also forced the nv.c not use the class_simple
code. But it still breaks on start-up.  Do you have either
CONFIG_DEBUG_SPINLOCK_SLEEP or CONFIG_DEBUG_PREEMPT defined?  If not,
you won't see the dump I see with the might_sleep. This is only tested
if you have either of the two configure.

But just to let you know, here's the dump when it happens.

BUG: sleeping function called from invalid context XFree86(5029) at
kernel/rt.c:1443
in_atomic():0 [00000000], irqs_disabled():1
 [<c01041ae>] dump_stack+0x1e/0x20 (20)
 [<c011af04>] __might_sleep+0xd4/0xf0 (36)
 [<c0137893>] __spin_lock+0x33/0x50 (24)
 [<c0137928>] _spin_lock_irqsave+0x18/0x20 (16)
 [<c01dfc8e>] pci_bus_write_config_word+0x2e/0x80 (40)
 [<f93c4ca3>] os_pci_write_word+0x3b/0x3d [nvidia] (24)
 [<f91d910c>] _nv001740rm+0x1c/0x20 [nvidia] (32)
 [<f91d481d>] _nv002409rm+0x99/0x110 [nvidia] (80)
 [<f91d53e7>] _nv002398rm+0xc7/0x328 [nvidia] (80)
 [<f91df93c>] rm_init_agp+0x48/0x50 [nvidia] (48)
 [<f93c2edb>] nv_agp_init+0x13d/0x192 [nvidia] (36)
 [<f91db522>] _nv001779rm+0xda/0x110 [nvidia] (64)
 [<f91dbefd>] _nv004480rm+0x131/0x158 [nvidia] (64)
 [<f91dbda9>] _nv004434rm+0xf9/0x11c [nvidia] (64)
 [<f934e1fb>] _nv002078rm+0x73/0x94 [nvidia] (64)
 [<f92e5e5a>] _nv002236rm+0x1e6/0x1f4 [nvidia] (64)
 [<f92e5a9e>] _nv002163rm+0x9e/0xb8 [nvidia] (64)
 [<f91dbbde>] _nv001315rm+0x5e/0xc0 [nvidia] (48)  [<f91dc392>]
_nv001320rm+0x1de/0x2bc [nvidia] (80)
 [<f91df08b>] rm_init_adapter+0x5f/0x8c [nvidia] (48)
              ^^^^^^^^^^^^^^^^
Here's your rm_init_adapter call. 

 [<f93c09dd>] nv_kern_open+0x288/0x31d [nvidia] (56)
 [<c0169b09>] chrdev_open+0xf9/0x1e0 (40)
 [<c015f614>] dentry_open+0x164/0x290 (36)
 [<c015f4ab>] filp_open+0x5b/0x60 (92)
 [<c015f926>] sys_open+0x46/0x90 (32)
 [<c0103297>] syscall_call+0x7/0xb (-8124)
---------------------------
| preempt count: 00000001 ]
| 1-level deep critical section nesting:
----------------------------------------
.. [<c01382d8>] .... print_traces+0x18/0x50
.....[<c01041ae>] ..   ( <= dump_stack+0x1e/0x20)


Anyway, I don't have CONFIG_SPINLOCK_BLK set, it's not even in
my .config file, and I can't get past the NVidia logo. But I haven't
tried the kernel patch you had, I'll try that now.

-- Steve

Re: 2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Steven Rostedt]

On Fri, 2004-12-17 at 16:49 -0500, Valdis.Kletnieks@vt.edu wrote:
> On Fri, 17 Dec 2004 15:04:21 EST, Steven Rostedt said:
> 
> > Nope! I have the 6629. Actually, the patch you have for NV solved the
> > pgd_offset problem.
> 
> Glad to hear it, I'm just the messenger on that one (I forget which lkml
> denizen actually wrote/posted that one).
> 
> >                       But I'm amazed that you didn't get into the
> > may_sleep calls.

I keep typing may_sleep for some reason, it really gets annoying when
searching for that function ;-)

> 
> I'm actually not *that* surprised - my test base consists entirely of one Dell
> Latitude C840 laptop with a GeForce4 440 Go card, so there's no SMP issues or
> similar, and I don't do heavy 3D or anything unless xscreensaver decides to use
> a random OpenGL display hack. Most likely, your hardware and/or CONFIG_* setup
> is getting it into code paths that never get hit on my system. And the
> might_sleep() is almost certainly on one of those paths.
> 
> (For the record, I *did* see a few might_sleep hits on 2.6.10-rc2-mm4-V0.7.31-15,
> but they only hit sendmail, gpg, bash, and stuff like that, never an X-based program.)

I just sent my last email out just before I received this one.

Just for the record, I'm testing this on a SMP Athlon system (my main
desktop) and a SMP HT Thinkpad G41 (just hyper threaded, not really SMP,
but it does act the same).

-- Steve

Re: 2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1436 bytes --]

On Fri, 17 Dec 2004 16:52:34 EST, Steven Rostedt said:

> I just tried it again, but this time with V0.7.33-04 and it still has
> the same problems.  I also forced the nv.c not use the class_simple
> code. But it still breaks on start-up.  Do you have either
> CONFIG_DEBUG_SPINLOCK_SLEEP or CONFIG_DEBUG_PREEMPT defined?  If not,
> you won't see the dump I see with the might_sleep. This is only tested
> if you have either of the two configure.

I had SPINLOCK_SLEEP defined on -rc3-mm1, and have DEBUG_PREEMPT defined both
now and on the base -rc3-mm1. So at least on my box with a UP kernel, neither
of those is a problem... (SPINLOCK_SLEEP doesn't show up on -RT because of an
added Kconfig 'depends on !DEBUG_PREEMPT' the -RT patch adds).
 
> But just to let you know, here's the dump when it happens.

> BUG: sleeping function called from invalid context XFree86(5029) at kernel/rt.c:1443

Hmm.. best I can do is:

Dec 13 11:59:06 turing-police kernel:  <3>BUG: sleeping function called from invalid context sendmail(10011) at include/linux/rwsem.h:47

Multiple hits on rwsem.h:47, sendmail/gpg/bash, only on V0.7.31-15, nothing
X-related, and quite possibly due to my ham-handed attempt to make Ingo's
-rc2-mm3 patch apply to -rc2-mm4 myself. ;)

Most likely, the fact you have SMP/HT and I'm just on a PREEMPT-UP kernel is
what's making the difference.  There's almost certainly a '#ifdef CONFIG_SMP'
involved here somehow....

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: 2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Steven Rostedt]

On Fri, 2004-12-17 at 17:42 -0500, Valdis.Kletnieks@vt.edu wrote:

> Most likely, the fact you have SMP/HT and I'm just on a PREEMPT-UP kernel is
> what's making the difference.  There's almost certainly a '#ifdef CONFIG_SMP'
> involved here somehow....

Yep! I just compiled my system without SMP and I was able to start up X
with NVidia on my HT laptop (with V0.7.33-04).  

Ingo, do you think this is a bug with NVidia (bad proprietary module) or
might be with something in the RT SMP side? I'll look a little more on
Monday, but if you know of something, let me know.  I'm curious to why
it works fine without RT but will not work with RT (and SMP). I
shouldn't say NVidia bug, since it only is a problem with the RT SMP, so
I should say incompatibility w.r.t. RT SMP and NVidia.

If you are one of those that don't want anything to do with the NVidia
driver, and don't care less if it works or not, let me know that too.
That way I won't bother you with this anymore and will only communicate
with Valdis :-)


Thanks,

-- Steve

Re: 2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Joe]

On Sun, 19 Dec 2004 11:20:03 -0500, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Fri, 2004-12-17 at 17:42 -0500, Valdis.Kletnieks@vt.edu wrote:
> 
> > Most likely, the fact you have SMP/HT and I'm just on a PREEMPT-UP kernel is
> > what's making the difference.  There's almost certainly a '#ifdef CONFIG_SMP'
> > involved here somehow....
> 
> Yep! I just compiled my system without SMP and I was able to start up X
> with NVidia on my HT laptop (with V0.7.33-04).
> 
> Ingo, do you think this is a bug with NVidia (bad proprietary module) or
> might be with something in the RT SMP side? I'll look a little more on
> Monday, but if you know of something, let me know.  I'm curious to why
> it works fine without RT but will not work with RT (and SMP). I
> shouldn't say NVidia bug, since it only is a problem with the RT SMP, so
> I should say incompatibility w.r.t. RT SMP and NVidia.
> 
> If you are one of those that don't want anything to do with the NVidia
> driver, and don't care less if it works or not, let me know that too.
> That way I won't bother you with this anymore and will only communicate
> with Valdis :-)
> 
> Thanks,
> 
> -- Steve

Nope, I've experianced the same problem without SMP.  It also appears
to be a bug where if make menuconfig is not run after using an old
kernel, for some odd reason CONFIG_SPINLOCK_BKL is set to be on. 
Anyways, I just wanted to reassure you, this is NOT an SMP bug.

Re: 2.6.10-rc3-mm1-V0.7.33-03 and NVidia wierdness, with workaround...

[Steven Rostedt]

On Sun, 2004-12-19 at 21:46 -0500, Joe wrote:
> Nope, I've experianced the same problem without SMP.  It also appears
> to be a bug where if make menuconfig is not run after using an old
> kernel, for some odd reason CONFIG_SPINLOCK_BKL is set to be on. 
> Anyways, I just wanted to reassure you, this is NOT an SMP bug.

I'm not sure Valdis' problem is the same as mine, so I can't say that
mine is not an SMP problem. There may be two problems here, and mine was
indeed SMP (since turning it off allowed me to get X up and running with
the NVidia module). The problem Valdis has mentioned, may be a separate
issue that has nothing to do with SMP. Now, w.r.t. the problem you saw,
was that only when CONFIG_SPINLOCK_BKL was set?

Now, I'm trying the NVidia again (well tomorrow, since I need to
recompile my entire kernel again to get back to SMP), and I've changed
the nvidia driver to us raw_spinlocks instead of normal ones to see if
this fixes things.  Ah, I've only changed the GPL part of the nvidia
driver (obviously).

This isn't really necessary for my real work, since I'm not required to
get this working with NVidia, but since three of my machines have an
NVidia card, this is more a personally fix I would like to have.
Otherwise, I just keep the old nv driver.

-- Steve