[Cluster-devel] message in syslog: shrink_slab: gfs2_glock_shrink_scan+0x0/0x240 [gfs2] negative objects to delete nr=xxxxxxxxxxxx

[Cluster-devel] message in syslog: shrink_slab: gfs2_glock_shrink_scan+0x0/0x240 [gfs2] negative objects to delete nr=xxxxxxxxxxxx

[Lentes, Bernd]

Hi,

i hope this is the right place for asking about GFS2.
Yesterday one of my two nodes HA-cluster got slower and slower, until it was fenced.
In /var/log/messages i found this message repeated often before the system got slower:
shrink_slab: gfs2_glock_shrink_scan+0x0/0x240 [gfs2] negative objects to delete nr=xxxxxxxxxxxx
What does it mean ? Is it a problem ?

My Setup:
SuSE SLES 12 SP5
Kernel 4.12.14-122.46-default
Pacemaker 1.1.23
corosync 2.3.6-9.13.1
gfs2-utils-3.1.6-1.101.x86_64

Thanks.

Bernd

-- 

Bernd Lentes 
System Administrator 
Institute for Metabolism and Cell Death (MCD) 
Building 25 - office 122 
HelmholtzZentrum M?nchen 
bernd.lentes at helmholtz-muenchen.de 
phone: +49 89 3187 1241 
phone: +49 89 3187 3827 
fax: +49 89 3187 2294 
http://www.helmholtz-muenchen.de/mcd 


Public key: 

30 82 01 0a 02 82 01 01 00 b3 72 3e ce 2c 0a 6f 58 49 2c 92 23 c7 b9 c1 ff 6c 3a 53 be f7 9e e9 24 b7 49 fa 3c e8 de 28 85 2c d3 ed f7 70 03 3f 4d 82 fc cc 96 4f 18 27 1f df 25 b3 13 00 db 4b 1d ec 7f 1b cf f9 cd e8 5b 1f 11 b3 a7 48 f8 c8 37 ed 41 ff 18 9f d7 83 51 a9 bd 86 c2 32 b3 d6 2d 77 ff 32 83 92 67 9e ae ae 9c 99 ce 42 27 6f bf d8 c2 a1 54 fd 2b 6b 12 65 0e 8a 79 56 be 53 89 70 51 02 6a eb 76 b8 92 25 2d 88 aa 57 08 42 ef 57 fb fe 00 71 8e 90 ef b2 e3 22 f3 34 4f 7b f1 c4 b1 7c 2f 1d 6f bd c8 a6 a1 1f 25 f3 e4 4b 6a 23 d3 d2 fa 27 ae 97 80 a3 f0 5a c4 50 4a 45 e3 45 4d 82 9f 8b 87 90 d0 f9 92 2d a7 d2 67 53 e6 ae 1e 72 3e e9 e0 c9 d3 1c 23 e0 75 78 4a 45 60 94 f8 e3 03 0b 09 85 08 d0 6c f3 ff ce fa 50 25 d9 da 81 7b 2a dc 9e 28 8b 83 04 b4 0a 9f 37 b8 ac 58 f1 38 43 0e 72 af 02 03 01 00 01
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2217 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/cluster-devel/attachments/20211208/0a1b4bd5/attachment.p7s>

[Cluster-devel] message in syslog: shrink_slab: gfs2_glock_shrink_scan+0x0/0x240 [gfs2] negative objects to delete nr=xxxxxxxxxxxx

[Steven Whitehouse]

Hi,

On Wed, 2021-12-08 at 17:50 +0100, Lentes, Bernd wrote:
> Hi,
> 
> i hope this is the right place for asking about GFS2.
> Yesterday one of my two nodes HA-cluster got slower and slower, until
> it was fenced.
> In /var/log/messages i found this message repeated often before the
> system got slower:
> shrink_slab: gfs2_glock_shrink_scan+0x0/0x240 [gfs2] negative objects
> to delete nr=xxxxxxxxxxxx
> What does it mean ? Is it a problem ?
> 
> My Setup:
> SuSE SLES 12 SP5
> Kernel 4.12.14-122.46-default
> Pacemaker 1.1.23
> corosync 2.3.6-9.13.1
> gfs2-utils-3.1.6-1.101.x86_64
> 
> Thanks.
> 
> Bernd
> 

That message in itself is a consequence of (I believe) a race relating
to shrinking of that slab cache, but it is harmless. However the fact
that you've seen it suggests that the system might be short on free
memory. So I would check to see if a process is hogging memory as that
would explain the slowness too,

Steve.

[Cluster-devel] message in syslog: shrink_slab: gfs2_glock_shrink_scan+0x0/0x240 [gfs2] negative objects to delete nr=xxxxxxxxxxxx

[Andrew Price]

Hi Bernd,

On 08/12/2021 16:50, Lentes, Bernd wrote:
> Hi,
> 
> i hope this is the right place for asking about GFS2.
> Yesterday one of my two nodes HA-cluster got slower and slower, until it was fenced.
> In /var/log/messages i found this message repeated often before the system got slower:
> shrink_slab: gfs2_glock_shrink_scan+0x0/0x240 [gfs2] negative objects to delete nr=xxxxxxxxxxxx
> What does it mean ? Is it a problem ?

This could be the same issue as 
https://bugzilla.redhat.com/show_bug.cgi?id=1599324

The last comment in that bug mentions a kernel patch that is probably 
missing from your distro kernel version.

Cheers,
Andy

> 
> My Setup:
> SuSE SLES 12 SP5
> Kernel 4.12.14-122.46-default
> Pacemaker 1.1.23
> corosync 2.3.6-9.13.1
> gfs2-utils-3.1.6-1.101.x86_64
> 
> Thanks.
> 
> Bernd
>