linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Unable to handle kernel NULL pointer dereference at virtual address 00000000
@ 2004-12-15  1:32 Ray Van Dolson
  2004-12-15 16:29 ` Matt Domsch
  0 siblings, 1 reply; 5+ messages in thread
From: Ray Van Dolson @ 2004-12-15  1:32 UTC (permalink / raw)
  To: linux-kernel

Hoping someone can shed some light on this, or help me out in narrowing the
problem down.

Our servers exhibit these error messages (at bottom) off and on and
eventually appears to lead to system instability.  I guess the three
possibilities are that this is either a) a hardware problem, b) a problem
with poptop, pppd or the mppe modules, or c) a problem with the kernel

This happens on every single one of our DL140's, so if it's a hardware
problem, then it is widespread on this platform.

If it's a problem with poptop, pppd or mppe modules I'd at least like to be
able to give some evidence to that effect...

If it's c, I'm hoping someone here can give some suggestions on how to
resolve this. :-)  Please let me know if I can provide further information.
I have a box on which this happens fairly regularly, so I can reproduce it
pretty easily.

Thanks in advance!

System Specs:
HP DL140
Dual 2.4GHz P4 Xeon w/ Hyperthreading
Fedora Core 2
2.6.9 kernel (static compile, downloaded from ftp.kernel.org with the
              following patches):
 - http://marc.theaimsgroup.com/?l=linux-kernel&m=109926628920398
   This fixes an oops I was getting in mm/prio_tree.c
 - http://mdomsch.bkbits.net:8080/linux-2.6-mppe
   This provides MPPE support for clients connecting via PoPToP 1.2.1.

Intel E100 Pro Dual Port NIC (Using built-in e100 driver).
Onboard Broadcom BCM5700 Gigabit NIC's are *not* in use (no driver support
 compiled in)

Software:
Poptop 1.2.1
PPP 2.4.3

Kernel config:

CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y
CONFIG_LOCALVERSION=""
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_SYSCTL=y
CONFIG_AUDIT=y
CONFIG_AUDITSYSCALL=y
CONFIG_LOG_BUF_SHIFT=17
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_EXTRA_PASS=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
CONFIG_SHMEM=y
CONFIG_X86_PC=y
CONFIG_MPENTIUM4=y
CONFIG_X86_GENERIC=y
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=7
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_SMP=y
CONFIG_NR_CPUS=8
CONFIG_SCHED_SMT=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_TSC=y
CONFIG_X86_MCE=y
CONFIG_X86_MCE_NONFATAL=y
CONFIG_X86_MCE_P4THERMAL=y
CONFIG_MICROCODE=y
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_HIGHMEM4G=y
CONFIG_HIGHMEM=y
CONFIG_HIGHPTE=y
CONFIG_MTRR=y
CONFIG_IRQBALANCE=y
CONFIG_HAVE_DEC_LOCK=y
CONFIG_ACPI=y
CONFIG_ACPI_BOOT=y
CONFIG_ACPI_INTERPRETER=y
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_FAN=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_THERMAL=y
CONFIG_ACPI_ASUS=y
CONFIG_ACPI_TOSHIBA=y
CONFIG_ACPI_BLACKLIST_YEAR=2001
CONFIG_ACPI_BUS=y
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_PCI=y
CONFIG_ACPI_SYSTEM=y
CONFIG_X86_PM_TIMER=y
CONFIG_PCI=y
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_LEGACY_PROC=y
CONFIG_PCI_NAMES=y
CONFIG_ISA=y
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_MISC=y
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_PNP=y
CONFIG_ISAPNP=y
CONFIG_BLK_DEV_FD=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_CRYPTOLOOP=y
CONFIG_BLK_DEV_NBD=y
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_SIZE=16384
CONFIG_BLK_DEV_INITRD=y
CONFIG_LBD=y
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_IDEDISK_MULTI_MODE=y
CONFIG_BLK_DEV_IDECD=y
CONFIG_BLK_DEV_IDESCSI=y
CONFIG_IDE_GENERIC=y
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
CONFIG_BLK_DEV_GENERIC=y
CONFIG_BLK_DEV_IDEDMA_PCI=y
CONFIG_IDEDMA_PCI_AUTO=y
CONFIG_BLK_DEV_SVWKS=y
CONFIG_BLK_DEV_IDEDMA=y
CONFIG_IDEDMA_AUTO=y
CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
CONFIG_BLK_DEV_SR=y
CONFIG_BLK_DEV_SR_VENDOR=y
CONFIG_CHR_DEV_SG=y
CONFIG_SCSI_SPI_ATTRS=y
CONFIG_SCSI_FC_ATTRS=y
CONFIG_SCSI_QLA2XXX=y
CONFIG_I2O=y
CONFIG_I2O_CONFIG=y
CONFIG_I2O_BLOCK=y
CONFIG_I2O_SCSI=y
CONFIG_I2O_PROC=y
CONFIG_NET=y
CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
CONFIG_NETLINK_DEV=y
CONFIG_UNIX=y
CONFIG_NET_KEY=y
CONFIG_INET=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_FWMARK=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_VERBOSE=y
CONFIG_NET_IPIP=y
CONFIG_NET_IPGRE=y
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=y
CONFIG_INET_ESP=y
CONFIG_INET_IPCOMP=y
CONFIG_INET_TUNNEL=y
CONFIG_NETFILTER=y
CONFIG_IP_NF_CONNTRACK=y
CONFIG_IP_NF_CT_ACCT=y
CONFIG_IP_NF_CT_PROTO_SCTP=y
CONFIG_IP_NF_FTP=y
CONFIG_IP_NF_IRC=y
CONFIG_IP_NF_TFTP=y
CONFIG_IP_NF_AMANDA=y
CONFIG_IP_NF_QUEUE=y
CONFIG_IP_NF_IPTABLES=y
CONFIG_IP_NF_MATCH_LIMIT=y
CONFIG_IP_NF_MATCH_IPRANGE=y
CONFIG_IP_NF_MATCH_MAC=y
CONFIG_IP_NF_MATCH_PKTTYPE=y
CONFIG_IP_NF_MATCH_MARK=y
CONFIG_IP_NF_MATCH_MULTIPORT=y
CONFIG_IP_NF_MATCH_TOS=y
CONFIG_IP_NF_MATCH_RECENT=y
CONFIG_IP_NF_MATCH_ECN=y
CONFIG_IP_NF_MATCH_DSCP=y
CONFIG_IP_NF_MATCH_AH_ESP=y
CONFIG_IP_NF_MATCH_LENGTH=y
CONFIG_IP_NF_MATCH_TTL=y
CONFIG_IP_NF_MATCH_TCPMSS=y
CONFIG_IP_NF_MATCH_HELPER=y
CONFIG_IP_NF_MATCH_STATE=y
CONFIG_IP_NF_MATCH_CONNTRACK=y
CONFIG_IP_NF_MATCH_OWNER=y
CONFIG_IP_NF_MATCH_ADDRTYPE=y
CONFIG_IP_NF_MATCH_REALM=y
CONFIG_IP_NF_MATCH_SCTP=y
CONFIG_IP_NF_MATCH_COMMENT=y
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_REJECT=y
CONFIG_IP_NF_TARGET_LOG=y
CONFIG_IP_NF_TARGET_ULOG=y
CONFIG_IP_NF_TARGET_TCPMSS=y
CONFIG_IP_NF_NAT=y
CONFIG_IP_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_IP_NF_TARGET_REDIRECT=y
CONFIG_IP_NF_TARGET_NETMAP=y
CONFIG_IP_NF_TARGET_SAME=y
CONFIG_IP_NF_NAT_LOCAL=y
CONFIG_IP_NF_NAT_SNMP_BASIC=y
CONFIG_IP_NF_NAT_IRC=y
CONFIG_IP_NF_NAT_FTP=y
CONFIG_IP_NF_NAT_TFTP=y
CONFIG_IP_NF_NAT_AMANDA=y
CONFIG_IP_NF_MANGLE=y
CONFIG_IP_NF_TARGET_TOS=y
CONFIG_IP_NF_TARGET_ECN=y
CONFIG_IP_NF_TARGET_DSCP=y
CONFIG_IP_NF_TARGET_MARK=y
CONFIG_IP_NF_TARGET_CLASSIFY=y
CONFIG_IP_NF_RAW=y
CONFIG_IP_NF_TARGET_NOTRACK=y
CONFIG_IP_NF_ARPTABLES=y
CONFIG_IP_NF_ARPFILTER=y
CONFIG_IP_NF_ARP_MANGLE=y
CONFIG_XFRM=y
CONFIG_XFRM_USER=y
CONFIG_NET_DIVERT=y
CONFIG_NET_SCHED=y
CONFIG_NET_SCH_CLK_JIFFIES=y
CONFIG_NET_SCH_CBQ=y
CONFIG_NET_SCH_HTB=y
CONFIG_NET_SCH_HFSC=y
CONFIG_NET_SCH_PRIO=y
CONFIG_NET_SCH_RED=y
CONFIG_NET_SCH_SFQ=y
CONFIG_NET_SCH_TEQL=y
CONFIG_NET_SCH_TBF=y
CONFIG_NET_SCH_GRED=y
CONFIG_NET_SCH_DSMARK=y
CONFIG_NET_SCH_NETEM=y
CONFIG_NET_SCH_INGRESS=y
CONFIG_NET_QOS=y
CONFIG_NET_ESTIMATOR=y
CONFIG_NET_CLS=y
CONFIG_NET_CLS_TCINDEX=y
CONFIG_NET_CLS_ROUTE4=y
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=y
CONFIG_NET_CLS_U32=y
CONFIG_CLS_U32_PERF=y
CONFIG_NET_CLS_IND=y
CONFIG_NET_CLS_RSVP=y
CONFIG_NET_CLS_RSVP6=y
CONFIG_NET_CLS_POLICE=y
CONFIG_NETPOLL=y
CONFIG_NET_POLL_CONTROLLER=y
CONFIG_NETDEVICES=y
CONFIG_DUMMY=y
CONFIG_NET_ETHERNET=y
CONFIG_MII=y
CONFIG_NET_PCI=y
CONFIG_E100=y
CONFIG_PPP=y
CONFIG_PPP_MULTILINK=y
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=y
CONFIG_PPP_SYNC_TTY=y
CONFIG_PPP_DEFLATE=y
CONFIG_PPP_BSDCOMP=y
CONFIG_PPP_MPPE=y
CONFIG_NETCONSOLE=y
CONFIG_INPUT=y
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_SOUND_GAMEPORT=y
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
CONFIG_MOUSE_SERIAL=y
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PCSPKR=y
CONFIG_INPUT_UINPUT=y
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_NR_UARTS=1024
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_SHARE_IRQ=y
CONFIG_SERIAL_8250_DETECT_IRQ=y
CONFIG_SERIAL_8250_MULTIPORT=y
CONFIG_SERIAL_8250_RSA=y
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_UNIX98_PTYS=y
CONFIG_IPMI_HANDLER=y
CONFIG_IPMI_DEVICE_INTERFACE=y
CONFIG_IPMI_SI=y
CONFIG_IPMI_WATCHDOG=y
CONFIG_IPMI_POWEROFF=y
CONFIG_HW_RANDOM=y
CONFIG_NVRAM=y
CONFIG_RTC=y
CONFIG_AGP=y
CONFIG_AGP_ATI=y
CONFIG_AGP_INTEL=y
CONFIG_HANGCHECK_TIMER=y
CONFIG_I2C=y
CONFIG_I2C_CHARDEV=y
CONFIG_I2C_ALGOBIT=y
CONFIG_I2C_ALGOPCF=y
CONFIG_I2C_ALGOPCA=y
CONFIG_I2C_I801=y
CONFIG_I2C_I810=y
CONFIG_I2C_ISA=y
CONFIG_I2C_PIIX4=y
CONFIG_I2C_SENSOR=y
CONFIG_SENSORS_ADM1021=y
CONFIG_SENSORS_ADM1025=y
CONFIG_SENSORS_ADM1031=y
CONFIG_SENSORS_ASB100=y
CONFIG_SENSORS_DS1621=y
CONFIG_SENSORS_FSCHER=y
CONFIG_SENSORS_GL518SM=y
CONFIG_SENSORS_IT87=y
CONFIG_SENSORS_LM75=y
CONFIG_SENSORS_LM77=y
CONFIG_SENSORS_LM78=y
CONFIG_SENSORS_LM80=y
CONFIG_SENSORS_LM83=y
CONFIG_SENSORS_LM85=y
CONFIG_SENSORS_LM90=y
CONFIG_SENSORS_MAX1619=y
CONFIG_SENSORS_SMSC47M1=y
CONFIG_SENSORS_VIA686A=y
CONFIG_SENSORS_W83781D=y
CONFIG_SENSORS_W83L785TS=y
CONFIG_SENSORS_W83627HF=y
CONFIG_SENSORS_EEPROM=y
CONFIG_SENSORS_PCF8574=y
CONFIG_SENSORS_PCF8591=y
CONFIG_SENSORS_RTC8564=y
CONFIG_VIDEO_DEV=y
CONFIG_VIDEO_SELECT=y
CONFIG_VGA_CONSOLE=y
CONFIG_DUMMY_CONSOLE=y
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
CONFIG_JBD=y
CONFIG_FS_MBCACHE=y
CONFIG_FS_POSIX_ACL=y
CONFIG_ROMFS_FS=y
CONFIG_AUTOFS_FS=y
CONFIG_AUTOFS4_FS=y
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_ZISOFS_FS=y
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
CONFIG_RAMFS=y
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
CONFIG_NFS_V4=y
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=y
CONFIG_SUNRPC=y
CONFIG_SUNRPC_GSS=y
CONFIG_RPCSEC_GSS_KRB5=y
CONFIG_SMB_FS=y
CONFIG_CIFS=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
CONFIG_PARTITION_ADVANCED=y
CONFIG_OSF_PARTITION=y
CONFIG_MAC_PARTITION=y
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
CONFIG_MINIX_SUBPARTITION=y
CONFIG_SOLARIS_X86_PARTITION=y
CONFIG_UNIXWARE_DISKLABEL=y
CONFIG_SGI_PARTITION=y
CONFIG_SUN_PARTITION=y
CONFIG_EFI_PARTITION=y
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="utf8"
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_CODEPAGE_737=y
CONFIG_NLS_CODEPAGE_775=y
CONFIG_NLS_CODEPAGE_850=y
CONFIG_NLS_CODEPAGE_852=y
CONFIG_NLS_CODEPAGE_855=y
CONFIG_NLS_CODEPAGE_857=y
CONFIG_NLS_CODEPAGE_860=y
CONFIG_NLS_CODEPAGE_861=y
CONFIG_NLS_CODEPAGE_862=y
CONFIG_NLS_CODEPAGE_863=y
CONFIG_NLS_CODEPAGE_864=y
CONFIG_NLS_CODEPAGE_865=y
CONFIG_NLS_CODEPAGE_866=y
CONFIG_NLS_CODEPAGE_869=y
CONFIG_NLS_CODEPAGE_936=y
CONFIG_NLS_CODEPAGE_950=y
CONFIG_NLS_CODEPAGE_932=y
CONFIG_NLS_CODEPAGE_949=y
CONFIG_NLS_CODEPAGE_874=y
CONFIG_NLS_ISO8859_8=y
CONFIG_NLS_CODEPAGE_1250=y
CONFIG_NLS_CODEPAGE_1251=y
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=y
CONFIG_NLS_ISO8859_2=y
CONFIG_NLS_ISO8859_3=y
CONFIG_NLS_ISO8859_4=y
CONFIG_NLS_ISO8859_5=y
CONFIG_NLS_ISO8859_6=y
CONFIG_NLS_ISO8859_7=y
CONFIG_NLS_ISO8859_9=y
CONFIG_NLS_ISO8859_13=y
CONFIG_NLS_ISO8859_14=y
CONFIG_NLS_ISO8859_15=y
CONFIG_NLS_KOI8_R=y
CONFIG_NLS_KOI8_U=y
CONFIG_NLS_UTF8=y
CONFIG_DEBUG_KERNEL=y
CONFIG_MAGIC_SYSRQ=y
CONFIG_EARLY_PRINTK=y
CONFIG_DEBUG_STACKOVERFLOW=y
CONFIG_DEBUG_STACK_USAGE=y
CONFIG_X86_FIND_SMP_CONFIG=y
CONFIG_X86_MPPARSE=y
CONFIG_CRYPTO=y
CONFIG_CRYPTO_HMAC=y
CONFIG_CRYPTO_NULL=y
CONFIG_CRYPTO_MD4=y
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_SHA256=y
CONFIG_CRYPTO_SHA512=y
CONFIG_CRYPTO_WP512=y
CONFIG_CRYPTO_DES=y
CONFIG_CRYPTO_BLOWFISH=y
CONFIG_CRYPTO_TWOFISH=y
CONFIG_CRYPTO_SERPENT=y
CONFIG_CRYPTO_AES_586=y
CONFIG_CRYPTO_CAST5=y
CONFIG_CRYPTO_CAST6=y
CONFIG_CRYPTO_TEA=y
CONFIG_CRYPTO_ARC4=y
CONFIG_CRYPTO_KHAZAD=y
CONFIG_CRYPTO_DEFLATE=y
CONFIG_CRYPTO_MICHAEL_MIC=y
CONFIG_CRYPTO_CRC32C=y
CONFIG_CRC_CCITT=y
CONFIG_CRC32=y
CONFIG_LIBCRC32C=y
CONFIG_ZLIB_INFLATE=y
CONFIG_ZLIB_DEFLATE=y
CONFIG_X86_SMP=y
CONFIG_X86_HT=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_X86_TRAMPOLINE=y
CONFIG_PC=y

lspci output:

00:00.0 Host bridge: ServerWorks GCNB-LE Host Bridge (rev 32)
00:00.1 Host bridge: ServerWorks GCNB-LE Host Bridge
00:03.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
00:0f.0 ISA bridge: ServerWorks CSB6 South Bridge (rev a0)
00:0f.1 IDE interface: ServerWorks CSB6 RAID/IDE Controller (rev a0)
00:0f.2 USB Controller: ServerWorks CSB6 OHCI USB Controller (rev 05)
00:0f.3 Host bridge: ServerWorks GCLE-2 Host Bridge
00:10.0 Host bridge: ServerWorks CIOB-E I/O Bridge with Gigabit Ethernet (rev 12)
00:10.2 Host bridge: ServerWorks CIOB-E I/O Bridge with Gigabit Ethernet (rev 12)
01:06.0 PCI bridge: Digital Equipment Corporation DECchip 21152 (rev 03)
02:04.0 Ethernet controller: Intel Corp. 82557/8/9 [Ethernet Pro 100] (rev 05)
02:05.0 Ethernet controller: Intel Corp. 82557/8/9 [Ethernet Pro 100] (rev 05)
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 02)
03:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 02)

ksymoops output of problem:

ksymoops 2.4.9 on i686 2.6.9.  Options used
     -V (default)
     -K (specified)
     -l /proc/modules (default)
     -O (specified)
     -m /boot/System.map (specified)

No ksyms, skipping lsmod
Unable to handle kernel NULL pointer dereference<7>divert: not allocating divert_blk for non-ethernet device ppp440
00000000
*pde = 00000000
Oops: 0000 [#1]
CPU:    2
EIP:    0060:[<00000000>]    Not tainted VLI
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010286   (2.6.9) 
eax: ed13b000   ebx: d1d0a000   ecx: c029e9de   edx: f795ef40
esi: d1d0a000   edi: 00000000   ebp: e2f30080   esp: d2b0dea0
ds: 007b   es: 007b   ss: 0068
Stack: c02a205a ed13b000 00000000 c02a122c d1d0a000 13208a2e c040956f d1d0a000 
       d1d0a00c e2f30080 00000000 c029cda9 d1d0a000 e2f30080 00000000 c01552cd 
       e2f30080 00000010 00000004 00000004 c0166aa0 e2f30080 00000000 00000000 
Call Trace:
 [<c02a205a>] pty_chars_in_buffer+0x2c/0x49
 [<c02a122c>] normal_poll+0xed/0x150
 [<c040956f>] schedule_timeout+0x75/0xbf
 [<c029cda9>] tty_poll+0xa0/0xb0
 [<c01552cd>] fget+0x49/0x5e
 [<c0166aa0>] do_select+0x269/0x2c6
 [<c0166691>] __pollwait+0x0/0xc7
 [<c0166dd5>] sys_select+0x2b3/0x4c6
 [<c0105971>] sysenter_past_esp+0x52/0x71
Code:  Bad EIP value.


>>EIP; 00000000 Before first symbol

>>eax; ed13b000 <pg0+2ca80000/3f943400>
>>ebx; d1d0a000 <pg0+1164f000/3f943400>
>>ecx; c029e9de <n_tty_chars_in_buffer+0/78>
>>edx; f795ef40 <pg0+372a3f40/3f943400>
>>esi; d1d0a000 <pg0+1164f000/3f943400>
>>ebp; e2f30080 <pg0+22875080/3f943400>
>>esp; d2b0dea0 <pg0+12452ea0/3f943400>

Trace; c02a205a <pty_chars_in_buffer+2c/49>
Trace; c02a122c <normal_poll+ed/150>
Trace; c040956f <schedule_timeout+75/bf>
Trace; c029cda9 <tty_poll+a0/b0>
Trace; c01552cd <fget+49/5e>
Trace; c0166aa0 <do_select+269/2c6>
Trace; c0166691 <__pollwait+0/c7>
Trace; c0166dd5 <sys_select+2b3/4c6>
Trace; c0105971 <sysenter_past_esp+52/71>

Yet another ksymoops output:


ksymoops 2.4.9 on i686 2.6.9.  Options used
     -V (default)
     -K (specified)
     -l /proc/modules (default)
     -O (specified)
     -m /boot/System.map (specified)

No ksyms, skipping lsmod
Unable to handle kernel NULL pointer dereference<7>divert: not allocating divert_blk for non-ethernet device ppp504
00000000
*pde = 00000000
Oops: 0000 [#2]
CPU:    2
EIP:    0060:[<00000000>]    Not tainted VLI
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010286   (2.6.9) 
eax: cc8ae000   ebx: d46ca000   ecx: c029e9de   edx: f7801880
esi: d46ca000   edi: 00000000   ebp: e16a1d80   esp: d2907ea0
ds: 007b   es: 007b   ss: 0068
Stack: c02a205a cc8ae000 00000000 c02a122c d46ca000 1da86df5 c040956f d46ca000 
       d46ca00c e16a1d80 00000000 c029cda9 d46ca000 e16a1d80 00000000 c01552cd 
       e16a1d80 00000010 00000004 00000004 c0166aa0 e16a1d80 00000000 00000000 
Call Trace:
 [<c02a205a>] pty_chars_in_buffer+0x2c/0x49
 [<c02a122c>] normal_poll+0xed/0x150
 [<c040956f>] schedule_timeout+0x75/0xbf
 [<c029cda9>] tty_poll+0xa0/0xb0
 [<c01552cd>] fget+0x49/0x5e
 [<c0166aa0>] do_select+0x269/0x2c6
 [<c0166691>] __pollwait+0x0/0xc7
 [<c0166dd5>] sys_select+0x2b3/0x4c6
 [<c0105971>] sysenter_past_esp+0x52/0x71
Code:  Bad EIP value.


>>EIP; 00000000 Before first symbol

>>eax; cc8ae000 <pg0+c1f3000/3f943400>
>>ebx; d46ca000 <pg0+1400f000/3f943400>
>>ecx; c029e9de <n_tty_chars_in_buffer+0/78>
>>edx; f7801880 <pg0+37146880/3f943400>
>>esi; d46ca000 <pg0+1400f000/3f943400>
>>ebp; e16a1d80 <pg0+20fe6d80/3f943400>
>>esp; d2907ea0 <pg0+1224cea0/3f943400>

Trace; c02a205a <pty_chars_in_buffer+2c/49>
Trace; c02a122c <normal_poll+ed/150>
Trace; c040956f <schedule_timeout+75/bf>
Trace; c029cda9 <tty_poll+a0/b0>
Trace; c01552cd <fget+49/5e>
Trace; c0166aa0 <do_select+269/2c6>
Trace; c0166691 <__pollwait+0/c7>
Trace; c0166dd5 <sys_select+2b3/4c6>
Trace; c0105971 <sysenter_past_esp+52/71>

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Unable to handle kernel NULL pointer dereference at virtual address 00000000
  2004-12-15  1:32 Unable to handle kernel NULL pointer dereference at virtual address 00000000 Ray Van Dolson
@ 2004-12-15 16:29 ` Matt Domsch
  2004-12-15 16:36   ` Ray Van Dolson
  2005-01-03 17:22   ` Ray Van Dolson
  0 siblings, 2 replies; 5+ messages in thread
From: Matt Domsch @ 2004-12-15 16:29 UTC (permalink / raw)
  To: Ray Van Dolson, linux-kernel

On Tue, Dec 14, 2004 at 05:32:28PM -0800, Ray Van Dolson wrote:
> Hoping someone can shed some light on this, or help me out in narrowing the
> problem down.
> 
> Our servers exhibit these error messages (at bottom) off and on and
> eventually appears to lead to system instability.  I guess the three
> possibilities are that this is either a) a hardware problem, b) a problem
> with poptop, pppd or the mppe modules, or c) a problem with the kernel
> 
> This happens on every single one of our DL140's, so if it's a hardware
> problem, then it is widespread on this platform.
> 
> If it's a problem with poptop, pppd or mppe modules I'd at least like to be
> able to give some evidence to that effect...
> 
> If it's c, I'm hoping someone here can give some suggestions on how to
> resolve this. :-)  Please let me know if I can provide further information.
> I have a box on which this happens fairly regularly, so I can reproduce it
> pretty easily.
> 
> Thanks in advance!
> 
> System Specs:
> HP DL140
> Dual 2.4GHz P4 Xeon w/ Hyperthreading
> Fedora Core 2
> 2.6.9 kernel (static compile, downloaded from ftp.kernel.org with the
>               following patches):
>  - http://marc.theaimsgroup.com/?l=linux-kernel&m=109926628920398
>    This fixes an oops I was getting in mm/prio_tree.c
>  - http://mdomsch.bkbits.net:8080/linux-2.6-mppe
>    This provides MPPE support for clients connecting via PoPToP 1.2.1.
> 
> Intel E100 Pro Dual Port NIC (Using built-in e100 driver).
> Onboard Broadcom BCM5700 Gigabit NIC's are *not* in use (no driver support
>  compiled in)
> 
> Software:
> Poptop 1.2.1
> PPP 2.4.3
> 
> Kernel config:
[snip]
> lspci output:
[snip] 

> <7>divert: not allocating divert_blk for non-ethernet device ppp440

These messages I have gotten for years.  They're debugging messages,
as a result of:
register_netdevice()
  alloc_divert_blk()
     (sees it's not an ethernet device, printk's the message)

So I believe they're harmless.


> ksymoops output of problem:
> Unable to handle kernel NULL pointer dereference
> 00000000
> *pde = 00000000
> Oops: 0000 [#1]
> CPU:    2
> EIP:    0060:[<00000000>]    Not tainted VLI
> Using defaults from ksymoops -t elf32-i386 -a i386
> EFLAGS: 00010286   (2.6.9) 
> eax: ed13b000   ebx: d1d0a000   ecx: c029e9de   edx: f795ef40
> esi: d1d0a000   edi: 00000000   ebp: e2f30080   esp: d2b0dea0
> ds: 007b   es: 007b   ss: 0068
> Stack: c02a205a ed13b000 00000000 c02a122c d1d0a000 13208a2e c040956f d1d0a000 
>        d1d0a00c e2f30080 00000000 c029cda9 d1d0a000 e2f30080 00000000 c01552cd 
>        e2f30080 00000010 00000004 00000004 c0166aa0 e2f30080 00000000 00000000 
> Call Trace:
>  [<c02a205a>] pty_chars_in_buffer+0x2c/0x49
>  [<c02a122c>] normal_poll+0xed/0x150
>  [<c040956f>] schedule_timeout+0x75/0xbf
>  [<c029cda9>] tty_poll+0xa0/0xb0
>  [<c01552cd>] fget+0x49/0x5e
>  [<c0166aa0>] do_select+0x269/0x2c6
>  [<c0166691>] __pollwait+0x0/0xc7
>  [<c0166dd5>] sys_select+0x2b3/0x4c6
>  [<c0105971>] sysenter_past_esp+0x52/0x71
> Code:  Bad EIP value.

It looks like pty_chars_in_buffer() dereferenced a NULL function
pointer, but I don't see how that can be, the one deference is tested
for NULL before doing so.

I can't rule out the ppp_mppe code, but I haven't seen this crash
before myself.  Does this happen on simlar systems that aren't running poptop?

Thanks,
Matt

-- 
Matt Domsch
Sr. Software Engineer, Lead Engineer
Dell Linux Solutions linux.dell.com & www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Unable to handle kernel NULL pointer dereference at virtual address 00000000
  2004-12-15 16:29 ` Matt Domsch
@ 2004-12-15 16:36   ` Ray Van Dolson
  2004-12-17 16:11     ` Ray Van Dolson
  2005-01-03 17:22   ` Ray Van Dolson
  1 sibling, 1 reply; 5+ messages in thread
From: Ray Van Dolson @ 2004-12-15 16:36 UTC (permalink / raw)
  To: Matt Domsch; +Cc: linux-kernel

On Wed, Dec 15, 2004 at 10:29:43AM -0600, Matt Domsch wrote:
> These messages I have gotten for years. They're debugging messages,
> as a result of:
> register_netdevice( alloc_divert_blk( (sees it's not an ethernet device,     )
> printk's the message                                                         )
>
> So I believe they're harmless.

This is what I was thinking too.  I see plenty of these messages that do
not result in crashes.

> > ksymoops output of problem:
> > Unable to handle kernel NULL pointer dereference
> > 00000000
> > *pde = 00000000
> > Oops: 0000 [#1]
> > CPU: 2
> > EIP: 0060:[<00000000>] Not tainted VLI
> > Using defaults from ksymoops -t elf32-i386 -a i386
> > EFLAGS: 00010286 (2.6.9)
> > eax: ed13b000 ebx: d1d0a000 ecx: c029e9de edx: f795ef40
> > esi: d1d0a000 edi: 00000000 ebp: e2f30080 esp: d2b0dea0
> > ds: 007b es: 007b ss: 0068
> > Stack: c02a205a ed13b000 00000000 c02a122c d1d0a000 13208a2e c040956f       
> > d1d0a000 d1d0a00c e2f30080 00000000 c029cda9 d1d0a000 e2f30080 00000000     
> > c01552cd e2f30080 00000010 00000004 00000004 c0166aa0 e2f30080 00000000     
> > 00000000                                                                    
> > Call Trace: [<c02a205a>] pty_chars_in_buffer+0x2c/0x49 [<c02a122c>]
> > normal_poll+0xed/0x150 [<c040956f>] schedule_timeout+0x75/0xbf
> > [<c029cda9>] tty_poll+0xa0/0xb0 [<c01552cd>] fget+0x49/0x5e [<c0166aa0>]
> > do_select+0x269/0x2c6 [<c0166691>] __pollwait+0x0/0xc7 [<c0166dd5>]
> > sys_select+0x2b3/0x4c6 [<c0105971>] sysenter_past_esp+0x52/0x71
> > Code: Bad EIP value.

> It looks like pty_chars_in_buffer() dereferenced a NULL function
> pointer, but I don't see how that can be, the one deference is tested
> for NULL before doing so.
>
> I can't rule out the ppp_mppe code, but I haven't seen this crash
> before myself. Does this happen on simlar systems that aren't running poptop?
>
> Thanks,
> Matt

Fortunately we use the DL140 server pretty widely here.  None of the
non-poptop servers exhibit this problem.  Every single one of them that has
been used as a Poptop server however does exhibit this issue.  It seems to
happen more frequenly on the machines with a higher number of users
connected.

I have switched one of the servers (the one with the highest load) to use
Jan Dubiec's MPPC/MPPE patches for ppp 2.4.3 and Kernel 2.6.9.  We'll see
how that goes.

Any information I can provide or something I can do on my end to better
trace this down?

Ray

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Unable to handle kernel NULL pointer dereference at virtual address 00000000
  2004-12-15 16:36   ` Ray Van Dolson
@ 2004-12-17 16:11     ` Ray Van Dolson
  0 siblings, 0 replies; 5+ messages in thread
From: Ray Van Dolson @ 2004-12-17 16:11 UTC (permalink / raw)
  To: Matt Domsch, linux-kernel

Just a follow-up on this.  Things seem to run better (no crashes yet after
two days of runtime) after booting with nosmp noapic.

Any issues with the MPPE or CryptoAPI code that might only be triggered
when running in SMP mode?

Ray

On Wed, Dec 15, 2004 at 08:36:51AM -0800, Ray Van Dolson wrote:
> On Wed, Dec 15, 2004 at 10:29:43AM -0600, Matt Domsch wrote:
> > These messages I have gotten for years. They're debugging messages,
> > as a result of:
> > register_netdevice( alloc_divert_blk( (sees it's not an ethernet device, )
> > printk's the message )
> >
> > So I believe they're harmless.
> This is what I was thinking too. I see plenty of these messages that do
> not result in crashes.
>
> > > ksymoops output of problem:
> > > Unable to handle kernel NULL pointer dereference
> > > 00000000
> > > *pde = 00000000
> > > Oops: 0000 [#1]
> > > CPU: 2
> > > EIP: 0060:[<00000000>] Not tainted VLI
> > > Using defaults from ksymoops -t elf32-i386 -a i386
> > > EFLAGS: 00010286 (2.6.9)
> > > eax: ed13b000 ebx: d1d0a000 ecx: c029e9de edx: f795ef40
> > > esi: d1d0a000 edi: 00000000 ebp: e2f30080 esp: d2b0dea0
> > > ds: 007b es: 007b ss: 0068
> > > Stack: c02a205a ed13b000 00000000 c02a122c d1d0a000 13208a2e c040956f
> > > d1d0a000 d1d0a00c e2f30080 00000000 c029cda9 d1d0a000 e2f30080 00000000
> > > c01552cd e2f30080 00000010 00000004 00000004 c0166aa0 e2f30080 00000000
> > > 00000000
> > > Call Trace: [<c02a205a>] pty_chars_in_buffer+0x2c/0x49 [<c02a122c>]
> > > normal_poll+0xed/0x150 [<c040956f>] schedule_timeout+0x75/0xbf
> > > [<c029cda9>] tty_poll+0xa0/0xb0 [<c01552cd>] fget+0x49/0x5e [<c0166aa0>]
> > > do_select+0x269/0x2c6 [<c0166691>] __pollwait+0x0/0xc7 [<c0166dd5>]
> > > sys_select+0x2b3/0x4c6 [<c0105971>] sysenter_past_esp+0x52/0x71
> > > Code: Bad EIP value.
> > It looks like pty_chars_in_buffer() dereferenced a NULL function
> > pointer, but I don't see how that can be, the one deference is tested
> > for NULL before doing so.
> >
> > I can't rule out the ppp_mppe code, but I haven't seen this crash
> > before myself. Does this happen on simlar systems that aren't running
> > poptop?
> >
> > Thanks,
> > Matt
> Fortunately we use the DL140 server pretty widely here. None of the
> non-poptop servers exhibit this problem. Every single one of them that has
> been used as a Poptop server however does exhibit this issue. It seems to
> happen more frequenly on the machines with a higher number of users
> connected.
>
> I have switched one of the servers (the one with the highest load) to use
> Jan Dubiec's MPPC/MPPE patches for ppp 2.4.3 and Kernel 2.6.9. We'll see
> how that goes.
>
> Any information I can provide or something I can do on my end to better
> trace this down?
>
> Ray

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: Unable to handle kernel NULL pointer dereference at virtual address 00000000
  2004-12-15 16:29 ` Matt Domsch
  2004-12-15 16:36   ` Ray Van Dolson
@ 2005-01-03 17:22   ` Ray Van Dolson
  1 sibling, 0 replies; 5+ messages in thread
From: Ray Van Dolson @ 2005-01-03 17:22 UTC (permalink / raw)
  To: Matt Domsch; +Cc: linux-kernel

On Wed, Dec 15, 2004 at 10:29:43AM -0600, Matt Domsch wrote:
> > ksymoops output of problem:
> > Unable to handle kernel NULL pointer dereference
> > 00000000
> > *pde = 00000000
> > Oops: 0000 [#1]
> > CPU: 2
> > EIP: 0060:[<00000000>] Not tainted VLI
> > Using defaults from ksymoops -t elf32-i386 -a i386
> > EFLAGS: 00010286 (2.6.9)
> > eax: ed13b000 ebx: d1d0a000 ecx: c029e9de edx: f795ef40
> > esi: d1d0a000 edi: 00000000 ebp: e2f30080 esp: d2b0dea0
> > ds: 007b es: 007b ss: 0068
> > Stack: c02a205a ed13b000 00000000 c02a122c d1d0a000 13208a2e c040956f       
> > d1d0a000 d1d0a00c e2f30080 00000000 c029cda9 d1d0a000 e2f30080 00000000     
> > c01552cd e2f30080 00000010 00000004 00000004 c0166aa0 e2f30080 00000000     
> > 00000000                                                                    
> > Call Trace: [<c02a205a>] pty_chars_in_buffer+0x2c/0x49 [<c02a122c>]
> > normal_poll+0xed/0x150 [<c040956f>] schedule_timeout+0x75/0xbf
> > [<c029cda9>] tty_poll+0xa0/0xb0 [<c01552cd>] fget+0x49/0x5e [<c0166aa0>]
> > do_select+0x269/0x2c6 [<c0166691>] __pollwait+0x0/0xc7 [<c0166dd5>]
> > sys_select+0x2b3/0x4c6 [<c0105971>] sysenter_past_esp+0x52/0x71
> > Code: Bad EIP value.
> It looks like pty_chars_in_buffer() dereferenced a NULL function
> pointer, but I don't see how that can be, the one deference is tested
> for NULL before doing so.
>
> I can't rule out the ppp_mppe code, but I haven't seen this crash
> before myself. Does this happen on simlar systems that aren't running poptop?
>
> Thanks,
> Matt

Yet another follow-up on this.  After completely disabling SMP/Hyperthreading
on these systems, we have had zero NULL pointer dereferences.  Obviously can't
say for sure that this points to problems with the MPPE module--the only true
test would be to allow our customers to connect without MPPE and reenable
SMP/HT.  Obviously not an option in a live environment. :)

Guess we just won't be using SMP/HT on our busier systems!

Ray

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2005-01-03 17:25 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2004-12-15  1:32 Unable to handle kernel NULL pointer dereference at virtual address 00000000 Ray Van Dolson
2004-12-15 16:29 ` Matt Domsch
2004-12-15 16:36   ` Ray Van Dolson
2004-12-17 16:11     ` Ray Van Dolson
2005-01-03 17:22   ` Ray Van Dolson

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).