* Dave Chinner:

>>   OBJS ACTIVE  USE OBJ SIZE  SLABS OBJ/SLAB CACHE SIZE NAME 
>> 4121208 4121177  99%    0.88K 1030302        4   4121208K xfs_inode
>> 986286 985229  99%    0.19K  46966       21    187864K dentry
>> 723255 723076  99%    0.10K  18545       39     74180K buffer_head
>> 270263 269251  99%    0.56K  38609        7    154436K radix_tree_node
>> 140310  67409  48%    0.38K  14031       10     56124K mnt_cache
>
> That's not odd at all. It means your workload is visiting millions
> on inodes in your filesystem between serious memory pressure events.

Okay.

>> (I have attached the /proc/meminfo contents in case it offers further
>> clues.)
>> 
>> Confronted with large memory allocations (from “make -j12” and
>> compiling GCC, so perhaps ~8 GiB of memory), the OOM killer kicks in
>> and kills some random process.  I would have expected that some
>> xfs_inodes are freed instead.
>
> The oom killer is unreliable and often behaves very badly, and
> that's typicaly not an XFS problem.
>
> What is the full output off the oom killer invocations from dmesg?

I've attached the dmesg output (two events).

>> The last time I saw
>> something like the slabtop output above, I could do “sysctl
>> vm.drop_caches = 3”, and the amount of memory allocated reported by
>> slabtop was reduced considerably.  (I have not checked if the memory
>> was actually returned to the system.)  I have not done this now so
>> that I can gather further data for debugging.i
>
> How long did the sysctl take to run to free those inodes? A few
> seconds, or minutes?

Seconds.  Very few of them.  Definitely not minutes.

>> I am not sure what
>> triggers this huge allocation.  It could be related to my Gnus mail
>> spool (which contains lots and lots of small files).
>
> OK - does that regularly dirty lots of those small files?

I use relatime for the file system, and the atime of old mail does not
show any recent access (certainly not since the last boot).

During normal operation, Gnus doesn't even read individual files or
list mail spool directories (which contain one message per file,
similar to Maildir).  I don't do much else on the box, and the GCC
compilation I started only involes 300,000 files or so.

> What sort of storage are you using, and what fs config?

It's a Samsung SATA SSD (MZ7WD960HAGP), supposedly enterprise-grade.

Mount flags:

/dev/sda1 on / type xfs (rw,relatime,attr2,inode64,noquota)

xfs_info:

meta-data=/dev/sda1              isize=256    agcount=4, agsize=57690368 blks
         =                       sectsz=512   attr=2, projid32bit=0
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=230761472, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal               bsize=4096   blocks=112676, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0


> http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F

Core i7 X980 CPU, 6 cores, 12 threads, hyperthreading enabled.

What is BBWC?

>> Dirty:                28 kB
>> Writeback:             0 kB
>
> There's no dirty data, and dropping caches makes progress, so this
> doesn't /sound/ like reclaim is getting stalled by dirty object
> writeback. More info needed.

Sorry, this is all new to me, I don't really know where to look.