archive mirror
 help / color / mirror / Atom feed
* Processes being killed without obvious reason or log entries
@ 2020-03-07  1:57 Whoza Ran
  0 siblings, 0 replies; only message in thread
From: Whoza Ran @ 2020-03-07  1:57 UTC (permalink / raw)
  To: linux-mm

[-- Attachment #1: Type: text/plain, Size: 2501 bytes --]

hi folks

I have some issue its reasons I am currently unaware of on multiple hosts.

Currently I noticed this when trying to access a postgresql database while
doing `su postgres`. please note that I do not think that this is related
to postgresql, but the kernel.

as soon as I do `su postgres` sometimes, but not always, the su-process
responds immediately with `Killed` and does not su to postgres. this does
not happen every time, sometimes I can turn into the postgres user.
executing anything else in that shell, e.g. psql then results in `Killed`
all the time. I may be hitting some system limits or something I am not
aware of.

but if there were limits hit, like things the oomkiller would handle, this
would be logged to syslog in any case afaik, yet there are absolutely no
logs in syslog, dmesg or the journal. since psql is a perl wrapper script,
it forks lots of processes and does not make it to a forked postgresql
client. so I tried this script:

as you can observe, even `date` gets killed sometimes. In this short
example at line #22

My best guess now is that there is something wrong with the logging there,
the kernel is vanilla 5.5.2, the OS is debian stretch. the ulimits are not
hit afaik, at least the usual suspects were set to unlimited, yet any log
entry should clarify what's going wrong.

also terminating all processes running as user postgres did not help. afaik
ulimits can not be the cause then, as there are no processes running as
that user. one weird thing: the database server itself, postgresql, can
start perfectly fine.

# ps -elf | grep postgres
0 S postgres   680     1  0  80   0 - 42460 -      Feb07 ?        00:00:41
/usr/lib/postgresql/9.6/bin/postgres [... some args]
1 S postgres   681   680  0  80   0 - 34755 -      Feb07 ?        00:00:02
postgres: logger process
1 S postgres   684   680  0  80   0 - 42518 -      Feb07 ?        00:00:17
postgres: checkpointer process
1 S postgres   685   680  0  80   0 - 42460 -      Feb07 ?        00:00:14
postgres: writer process
1 S postgres   686   680  0  80   0 - 42460 -      Feb07 ?        00:00:15
postgres: wal writer process
1 S postgres   687   680  0  80   0 - 38389 -      Feb07 ?        00:00:12
postgres: stats collector process

what am I missing? could this be some issue with the memory management
killing processes but not logging things?
Also, should that help: strace psql - which
is being killed as +++ killed by SIGKILL +++

[-- Attachment #2: Type: text/html, Size: 3252 bytes --]

^ permalink raw reply	[flat|nested] only message in thread

only message in thread, other threads:[~2020-03-07  1:57 UTC | newest]

Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-07  1:57 Processes being killed without obvious reason or log entries Whoza Ran

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).