All of lore.kernel.org
 help / color / mirror / Atom feed
From: Eric Sandeen <sandeen@sandeen.net>
To: Blake Golliher <bgolliher@box.com>
Cc: Chris Holcombe <cholcombe@box.com>,
	"Darrick J. Wong" <darrick.wong@oracle.com>,
	linux-xfs@vger.kernel.org
Subject: Re: XFS: possible memory allocation deadlock in kmem_alloc
Date: Tue, 5 Nov 2019 14:47:50 -0600	[thread overview]
Message-ID: <33b9bde4-fd14-8754-f98a-ae0f363e76be@sandeen.net> (raw)
In-Reply-To: <CAC752AnZ4biDGk6V17URQm5YVp=MwZBhiMH8=t733zaypxUsmA@mail.gmail.com>

On 11/5/19 2:36 PM, Blake Golliher wrote:
> We don't get anything more then this messages.
> 
> [Tue Nov  5 11:18:34 2019] XFS: nginx(2540) possible memory allocation deadlock size 63960 in kmem_alloc (mode:0x250)
> 
> [Tue Nov  5 11:18:34 2019] XFS: nginx(2517) possible memory allocation deadlock size 56880 in kmem_alloc (mode:0x250)
> 
> [Tue Nov  5 11:18:34 2019] XFS: nginx(2540) possible memory allocation deadlock size 63960 in kmem_alloc (mode:0x250)
> 
> [Tue Nov  5 11:18:35 2019] XFS: nginx(2517) possible memory allocation deadlock size 56880 in kmem_alloc (mode:0x250)
> 
> [Tue Nov  5 11:18:36 2019] XFS: nginx(2514) possible memory allocation deadlock size 63960 in kmem_alloc (mode:0x250)
> 
> [Tue Nov  5 11:18:36 2019] XFS: nginx(2540) possible memory allocation deadlock size 63960 in kmem_alloc (mode:0x250)
> 
> [Tue Nov  5 11:18:37 2019] XFS: nginx(2517) possible memory allocation deadlock size 56880 in kmem_alloc (mode:0x250)

no sure what to say.  In that kernel, when we print the message:

                        xfs_err(NULL,
        "%s(%u) possible memory allocation deadlock size %u in %s (mode:0x%x)",
                                current->comm, current->pid,
                                (unsigned int)size, __func__, lflags);

xfs_err() is:

define_xfs_printk_level(xfs_err, KERN_ERR);

which is the macro:

#define define_xfs_printk_level(func, kern_level)               \
void func(const struct xfs_mount *mp, const char *fmt, ...)     \
{                                                               \
        struct va_format        vaf;                            \
        va_list                 args;                           \
        int                     level;                          \
                                                                \
        va_start(args, fmt);                                    \
                                                                \
        vaf.fmt = fmt;                                          \
        vaf.va = &args;                                         \
                                                                \
        __xfs_printk(kern_level, mp, &vaf);                     \
        va_end(args);                                           \
                                                                \
        if (!kstrtoint(kern_level, 0, &level) &&                \ 
            level <= 3 /* LOGLEVEL_ERR */ &&                    \
            xfs_error_level >= XFS_ERRLEVEL_HIGH)               \
                xfs_stack_trace();                              \
}                                                               \

which should dump a stack if called w/ ERR priority and the xfs_error_level is at 11.

I suppose your general kernel ring buffer needs to be turned up high enough
as well (i.e. dmesg -n 8 to be sure?)

The resulting stack trace would tell us exactly how you got to the 
allocation message.

-Eric

  parent reply	other threads:[~2019-11-05 20:47 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-11-04 23:38 XFS: possible memory allocation deadlock in kmem_alloc Chris Holcombe
2019-11-05  0:01 ` Darrick J. Wong
2019-11-05  0:31   ` Eric Sandeen
     [not found]     ` <CAC752AmahECFry9x=pvqDkwQUj1PEJjoWGa2KFG1uaTzT1Bbnw@mail.gmail.com>
2019-11-05  4:21       ` Eric Sandeen
2019-11-05 16:25         ` Chris Holcombe
2019-11-05 17:11           ` Eric Sandeen
2019-11-05 19:53             ` Chris Holcombe
2019-11-05 20:08               ` Eric Sandeen
     [not found]                 ` <CAC752AnZ4biDGk6V17URQm5YVp=MwZBhiMH8=t733zaypxUsmA@mail.gmail.com>
2019-11-05 20:47                   ` Eric Sandeen [this message]
     [not found]                     ` <CAC752A=y9PMEQ1e4mXskha1GFeKXWi8PsdBW-nX40pgFCYp1Uw@mail.gmail.com>
2019-11-05 21:23                       ` Eric Sandeen
  -- strict thread matches above, loose matches on Subject: below --
2016-05-30  4:45 baotiao
2016-05-30  5:04 ` Dave Chinner
2016-05-30  8:48   ` baotiao
2016-05-30  9:20     ` Carlos Maiolino
2016-05-31  2:43     ` 陈宗志
2016-05-31  3:10       ` Dave Chinner
2016-05-31 11:00         ` 陈宗志
2016-05-31 12:14           ` Carlos Maiolino

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=33b9bde4-fd14-8754-f98a-ae0f363e76be@sandeen.net \
    --to=sandeen@sandeen.net \
    --cc=bgolliher@box.com \
    --cc=cholcombe@box.com \
    --cc=darrick.wong@oracle.com \
    --cc=linux-xfs@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.