Kernel Panic

Kernel Panic

[Christophe DUMONT]

[-- Attachment #1.1.1: Type: text/plain, Size: 253 bytes --]

Hello Kernel World,

We're facing Kernel Panic on CentOS 7 since upgrading from 3.10.0-957 to 3.10.0-1062. I'm thinking about a java memory leak, but not sure.
Do you know what's going on here ?

Christophe

[cid:image001.png@01D58339.CD24E760]

[-- Attachment #1.1.2: Type: text/html, Size: 3284 bytes --]

[-- Attachment #1.2: image001.png --]
[-- Type: image/png, Size: 52202 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Kernel Panic

[Valentin Vidić]

On Tue, Oct 15, 2019 at 07:21:18AM +0000, Christophe DUMONT wrote:
> We're facing Kernel Panic on CentOS 7 since upgrading from 3.10.0-957 to 3.10.0-1062. I'm thinking about a java memory leak, but not sure.
> Do you know what's going on here ?

Looks similar to this: https://access.redhat.com/solutions/4164501

-- 
Valentin

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Kernel Panic

[Valdis Klētnieks]

[-- Attachment #1.1: Type: text/plain, Size: 728 bytes --]

On Tue, 15 Oct 2019 07:21:18 -0000, Christophe DUMONT said:

> We're facing Kernel Panic on CentOS 7 since upgrading from 3.10.0-957 to 3.10.0-1062. I'm thinking about a java memory leak, but not sure.
> Do you know what's going on here ?

Well, what made you think "Java memory leak"?

Java is userspace.  If it's leaking memory so far that the kernel has problems, it would probably:

a) Have been leaking memory and causing problems in -957 as well
b) Died in the OOM (Out Of Memory) code, rather than in the futex() system call.

Yes, poorly written Java code will leak memory like a sieve, but this doesn't smell
anything remotely like a memory leak.

I agree with Valentin that it's probably the bug report he references.

[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

RE: Kernel Panic

[Christophe DUMONT]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1263 bytes --]

Hello, 

 

What made me think about a memory leak is the message : Java Not Tainted 3.10.0-1062.1.1.el7.x86_64. 

I can’t access the bug report referenced : an active redhat subscription is needed !

 

Christophe Dumont

Ligne directe : 0476842574



 

De : Valdis Kletnieks <valdis@vt.edu> De la part de Valdis Kletnieks
Envoyé : mardi 15 octobre 2019 22:20
À : Christophe DUMONT <Christophe.DUMONT@yoni.fr>
Cc : kernelnewbies@kernelnewbies.org
Objet : Re: Kernel Panic

 

On Tue, 15 Oct 2019 07:21:18 -0000, Christophe DUMONT said: 

> We're facing Kernel Panic on CentOS 7 since upgrading from 3.10.0-957 to 3.10.0-1062. I'm thinking about a java memory leak, but not sure.

> Do you know what's going on here ? 

Well, what made you think "Java memory leak"? 

Java is userspace.  If it's leaking memory so far that the kernel has problems, it would probably: 

a) Have been leaking memory and causing problems in -957 as well 
b) Died in the OOM (Out Of Memory) code, rather than in the futex() system call. 

Yes, poorly written Java code will leak memory like a sieve, but this doesn't smell 
anything remotely like a memory leak. 

I agree with Valentin that it's probably the bug report he references. 

[-- Attachment #1.1.1.2: Type: text/html, Size: 6329 bytes --]

[-- Attachment #1.1.2: image001.png --]
[-- Type: image/png, Size: 8367 bytes --]

[-- Attachment #1.2: Type: application/pgp-signature, Size: 489 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Kernel Panic

[Valdis Klētnieks]

[-- Attachment #1.1: Type: text/plain, Size: 876 bytes --]

On Wed, 16 Oct 2019 07:34:01 -0000, Christophe DUMONT said:

> What made me think about a memory leak is the message : Java Not Tainted 3.10.0-1062.1.1.el7.x86_64.=20

That just tells you that the currently executing process was java.

It says nothing at all about a memory leak, and as I already mentioned, if Java
was leaking memory, it would almost certainly have been leaking memory on a
previous kernel.

The important part almost always isn't the running process, it's the kernel
stack traceback, which in this case has 'futex' scribbled *all* over it.

General rule of thumb:

If you get more than one crash that has a similar traceback that points at a
specific syscall, or file system driver, etc, the bug is almost guaranteed to
be in that code.

If you get a rash of crashes with *different* tracebacks, you probably have
some other code that's overlaying memory.


[-- Attachment #1.2: Type: application/pgp-signature, Size: 832 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

RE: Kernel Panic

[Christophe DUMONT]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1436 bytes --]

We can put aside Java Memory Leak.

I downgraded to kernel 3.10.0-957.27.2.el7.x86_64, there are no more crashes. 

 

The bug comes from ‘futex‘ syscall. What can causes that bug and how can i investigate ?

 

Christophe Dumont

Ligne directe : 0476842574



 

De : Valdis Kletnieks <valdis@vt.edu> De la part de Valdis Kletnieks
Envoyé : mercredi 16 octobre 2019 16:17
À : Christophe DUMONT <Christophe.DUMONT@yoni.fr>
Cc : kernelnewbies@kernelnewbies.org
Objet : Re: Kernel Panic

 

On Wed, 16 Oct 2019 07:34:01 -0000, Christophe DUMONT said: 

> What made me think about a memory leak is the message : Java Not Tainted 3.10.0-1062.1.1.el7.x86_64.=20 

That just tells you that the currently executing process was java. 

It says nothing at all about a memory leak, and as I already mentioned, if Java 
was leaking memory, it would almost certainly have been leaking memory on a 
previous kernel. 

The important part almost always isn't the running process, it's the kernel 
stack traceback, which in this case has 'futex' scribbled *all* over it. 

General rule of thumb: 

If you get more than one crash that has a similar traceback that points at a 
specific syscall, or file system driver, etc, the bug is almost guaranteed to 
be in that code. 

If you get a rash of crashes with *different* tracebacks, you probably have 
some other code that's overlaying memory. 

 

[-- Attachment #1.1.1.2: Type: text/html, Size: 6353 bytes --]

[-- Attachment #1.1.2: image001.png --]
[-- Type: image/png, Size: 8367 bytes --]

[-- Attachment #1.2: Type: application/pgp-signature, Size: 489 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Kernel Panic

[Valentin Vidić]

On Thu, Oct 17, 2019 at 07:48:49AM +0000, Christophe DUMONT wrote:
> We can put aside Java Memory Leak.
> 
> I downgraded to kernel 3.10.0-957.27.2.el7.x86_64, there are no more crashes. 
> 
> The bug comes from ‘futex‘ syscall. What can causes that bug and how
> can i investigate ?

You can try to get the source for these two versions and compare. If the
diff is not too big it might give you some idea what changed...

-- 
Valentin

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

RE: Kernel Panic

[Christophe DUMONT]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1332 bytes --]

Hello, 

 

We downgraded to older kernel. We still facing the same Kernel Panic with different Kernel. 

 

 

Christophe Dumont

Ligne directe : 0476842574



 

De : Valdis Kletnieks <valdis@vt.edu> De la part de Valdis Kletnieks
Envoyé : mercredi 16 octobre 2019 16:17
À : Christophe DUMONT <Christophe.DUMONT@yoni.fr>
Cc : kernelnewbies@kernelnewbies.org
Objet : Re: Kernel Panic

 

On Wed, 16 Oct 2019 07:34:01 -0000, Christophe DUMONT said: 

> What made me think about a memory leak is the message : Java Not Tainted 3.10.0-1062.1.1.el7.x86_64.=20 

That just tells you that the currently executing process was java. 

It says nothing at all about a memory leak, and as I already mentioned, if Java 
was leaking memory, it would almost certainly have been leaking memory on a 
previous kernel. 

The important part almost always isn't the running process, it's the kernel 
stack traceback, which in this case has 'futex' scribbled *all* over it. 

General rule of thumb: 

If you get more than one crash that has a similar traceback that points at a 
specific syscall, or file system driver, etc, the bug is almost guaranteed to 
be in that code. 

If you get a rash of crashes with *different* tracebacks, you probably have 
some other code that's overlaying memory. 

 

[-- Attachment #1.1.1.2: Type: text/html, Size: 6241 bytes --]

[-- Attachment #1.1.2: image001.png --]
[-- Type: image/png, Size: 8367 bytes --]

[-- Attachment #1.2: Type: application/pgp-signature, Size: 489 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Kernel Panic

[Greg KH]

On Tue, Oct 22, 2019 at 09:38:59AM +0000, Christophe DUMONT wrote:
> Hello, 
> 
>  
> 
> We downgraded to older kernel. We still facing the same Kernel Panic with different Kernel. 

Please contact your vendor for support, they are the only ones that can
support you as it's a very odd and old kernel you are using, and you are
paying for support for it already.  Might as well take advantage of it!

good luck!

greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

kernel panic

[Mulyadi Santosa]

On Thu, Oct 15, 2015 at 6:23 PM, jinzhao at wingtech.com <jinzhao@wingtech.com>
wrote:

> Dear all:
>      I met a problem as below, what the function of walk_page_range??
> Exception Class: Kernel (KE)
> PC is at [<ffffffc00017fc90>] walk_page_range+0x3c/0x314
>
> Current Executing Process:
> [Binder_C, 3611][main, 354]
>
> Backtrace:
> [<ffffffc0009f21dc>] __do_kernel_fault.part.5+0x70/0x84
> [<ffffffc0000942f4>] do_bad_area+0x90/0x94
> [<ffffffc000094340>] do_translation_fault+0x30/0x4c
> [<ffffffc0000813fc>] do_mem_abort+0x38/0x98
> [<ffffffc000083c58>] el1_da+0x1c/0x88
> [<ffffffc0001eb3e0>] show_smap+0x88/0x284
> [<ffffffc0001eb5e8>] show_pid_smap+0xc/0x18
> [<ffffffc0001aefb8>] seq_read+0x1a4/0x40c
> [<ffffffc00018e75c>] vfs_read+0x88/0x170
> [<ffffffc00018ebf0>] SyS_read+0x40/0x8c
> [<ffffffc000084308>] cpu_switch_to+0x48/0x4c
> [<ffffffffffffffff>] 0xffffffffffffffff
>
>
> Thanks!
> ------------------------------
> jinzhao at wingtech.cn
>
>
AFAIK, that's the function to get PTE of the related pages, then get the
virtual address of them.

Maybe the caller hit invalid PTE ?

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20151016/0ef8f374/attachment.html 

kernel panic

[jinzhao at wingtech.com]

Dear all:
     I met a problem as below, what the function of walk_page_range??
Exception Class: Kernel (KE) 
PC is at [<ffffffc00017fc90>] walk_page_range+0x3c/0x314 

Current Executing Process: 
[Binder_C, 3611][main, 354] 

Backtrace: 
[<ffffffc0009f21dc>] __do_kernel_fault.part.5+0x70/0x84 
[<ffffffc0000942f4>] do_bad_area+0x90/0x94 
[<ffffffc000094340>] do_translation_fault+0x30/0x4c 
[<ffffffc0000813fc>] do_mem_abort+0x38/0x98 
[<ffffffc000083c58>] el1_da+0x1c/0x88 
[<ffffffc0001eb3e0>] show_smap+0x88/0x284 
[<ffffffc0001eb5e8>] show_pid_smap+0xc/0x18 
[<ffffffc0001aefb8>] seq_read+0x1a4/0x40c 
[<ffffffc00018e75c>] vfs_read+0x88/0x170 
[<ffffffc00018ebf0>] SyS_read+0x40/0x8c 
[<ffffffc000084308>] cpu_switch_to+0x48/0x4c 
[<ffffffffffffffff>] 0xffffffffffffffff


Thanks!


jinzhao at wingtech.cn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20151015/440e76ee/attachment.html