archive mirror
 help / color / mirror / Atom feed
From: Jonathan Rajotte-Julien via lttng-dev <>
To: Rocky Dunlap <>
Subject: Re: Large number of stream files in CTF trace -- too many file handles
Date: Mon, 16 Mar 2020 10:51:25 -0400	[thread overview]
Message-ID: <20200316145125.GC14185@joraj-alpa> (raw)
In-Reply-To: <>


> If this is not the right approach, how should I proceed?  E.g., should the
> source-ctf-fs manage a limited pool of file handles?  I would think this
> would be pretty inefficient as you would need to constantly open/close
> files--expensive.

I would probably start looking at the soft and hardlimit for the babeltrace2
process in terms of open file:

On my machine:

joraj@~[]$ ulimit -Sn

joraj@~[]$ ulimit -Hn

That is a lot of headspace.

I might have a setting somewhere increasing the base hardlimit but
in any case you will see how much room you have.

Based on the number of streams you have, I would say that you will
need more than 2000 as a base soft limit for this trace.

We do have a FD pooling system in place in another project we maintain
(lttng-tools[1] GPL-2.0) that might be pertinent for babeltrace2 at some point
in time. As for the overhead that would occur in a scenario with not enough FD
available, I think it is a good compromise between either reading a trace or
not reading it at all. A warning informing the user that we reached
the limit of the pool might be a good start in such case.



Jonathan Rajotte-Julien

  reply	other threads:[~2020-03-16 14:51 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-13 21:55 Rocky Dunlap via lttng-dev
2020-03-16 14:51 ` Jonathan Rajotte-Julien via lttng-dev [this message]
2020-03-17  3:38   ` Rocky Dunlap via lttng-dev

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200316145125.GC14185@joraj-alpa \ \ \ \
    --subject='Re: Large number of stream files in CTF trace -- too many file handles' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).