archive mirror
 help / color / mirror / Atom feed
From: Rocky Dunlap via lttng-dev <>
To: Jonathan Rajotte-Julien <>
Subject: Re: Large number of stream files in CTF trace -- too many file handles
Date: Mon, 16 Mar 2020 21:38:11 -0600	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <20200316145125.GC14185@joraj-alpa>

[-- Attachment #1.1: Type: text/plain, Size: 1576 bytes --]


Increasing the soft FD limit worked great, both for the command line and
the Python script.  Thanks for the help!


On Mon, Mar 16, 2020 at 8:51 AM Jonathan Rajotte-Julien <> wrote:

> Hi,
> > If this is not the right approach, how should I proceed?  E.g., should
> the
> > source-ctf-fs manage a limited pool of file handles?  I would think this
> > would be pretty inefficient as you would need to constantly open/close
> > files--expensive.
> I would probably start looking at the soft and hardlimit for the
> babeltrace2
> process in terms of open file:
> On my machine:
> joraj@~[]$ ulimit -Sn
> 1024
> joraj@~[]$ ulimit -Hn
> 1048576
> That is a lot of headspace.
> I might have a setting somewhere increasing the base hardlimit but
> in any case you will see how much room you have.
> Based on the number of streams you have, I would say that you will
> need more than 2000 as a base soft limit for this trace.
> We do have a FD pooling system in place in another project we maintain
> (lttng-tools[1] GPL-2.0) that might be pertinent for babeltrace2 at some
> point
> in time. As for the overhead that would occur in a scenario with not
> enough FD
> available, I think it is a good compromise between either reading a trace
> or
> not reading it at all. A warning informing the user that we reached
> the limit of the pool might be a good start in such case.
> [1]
> Cheers
> --
> Jonathan Rajotte-Julien
> EfficiOS

[-- Attachment #1.2: Type: text/html, Size: 2174 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

lttng-dev mailing list

      reply	other threads:[~2020-03-17  3:38 UTC|newest]

Thread overview: 3+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-03-13 21:55 Rocky Dunlap via lttng-dev
2020-03-16 14:51 ` Jonathan Rajotte-Julien via lttng-dev
2020-03-17  3:38   ` Rocky Dunlap via lttng-dev [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='' \ \ \ \
    --subject='Re: Large number of stream files in CTF trace -- too many file handles' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).