[lttng-dev] [babeltrace2]about python self-defined plugin loading

[lttng-dev] [babeltrace2]about python self-defined plugin loading

[Wu, Yannan via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 2023 bytes --]

Hi, There,

I am trying to construct a customized filter and sink based on babeltrace2 python binding. However, nether mine plugin nor the plugins sample I could find on the internet all dont work.

For example, https://github.com/simark/babeltrace-fun-plugins/tree/master/my-first-components

I just downloaded the py file and run the exact command, it failed. The log is as following:

babeltrace2 --plugin-path . -c source.demo.MyFirstSource -c sink.demo.MyFirstSink
04-24 16:52:04.349 919805 919805 E CLI add_descriptor_to_component_descriptor_set@babeltrace2.c:1720 Cannot find component class: plugin-name="demo", comp-cls-name="MyFirstSource", comp-cls-type=1
04-24 16:52:04.349 919805 919805 E CLI cmd_run_ctx_init@babeltrace2.c:1882 Cannot find an operative message interchange protocol version to use to create the `run` command's graph: status=ERROR
04-24 16:52:04.349 919805 919805 E CLI cmd_run@babeltrace2.c:2465 Cannot initialize the command's context.

ERROR:    [Babeltrace CLI] (babeltrace2.c:2465)
  Cannot initialize the command's context.
CAUSED BY [Babeltrace CLI] (babeltrace2.c:1882)
  Cannot find an operative message interchange protocol version to use to create the `run` command's graph: status=ERROR
CAUSED BY [Babeltrace CLI] (babeltrace2.c:1720)
  Cannot find component class: plugin-name="demo", comp-cls-name="MyFirstSource", comp-cls-type=1

babeltrace2 --version
Babeltrace 2.0.7 "Amqui" [v2.0.6-1-g825a0ed6d]

Amqui (/ɒmkwiː/) is a town in eastern Québec, Canada, at the base of the Gaspé peninsula in Bas-Saint-Laurent. Located at the
confluence of the Humqui and Matapédia Rivers, its proximity to woodlands makes it a great destination for outdoor activities such as
camping, hiking, and mountain biking.
yannanwu@ue91e96f2951b5c:~/trees/lttng_test_run$

Is the cli changed or something? How can I make it right?

Besides, is it possible we create a pipeline in python and make use the the python drafted plugin? Can you advise me how?

Amanda


[-- Attachment #1.2: Type: text/html, Size: 3229 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [babeltrace2]about python self-defined plugin loading

[Kienan Stewart via lttng-dev]

Hi Amanda,

could you double-check to ensure that babeltrace2 was built with 
`--enable-python-plugins`, and that `import bt2` works?

There should be a babeltrace2-python-plugin-provider.so in 
`<libdir>/babeltrace2/plugin-provides`

thanks,
kienan


On 4/24/24 11:28 PM, Wu, Yannan via lttng-dev wrote:
> Hi, There,
> 
> I am trying to construct a customized filter and sink based on 
> babeltrace2 python binding. However, nether mine plugin nor the plugins 
> sample I could find on the internet all dont work.
> 
> For example, 
> https://github.com/simark/babeltrace-fun-plugins/tree/master/my-first-components <https://github.com/simark/babeltrace-fun-plugins/tree/master/my-first-components>
> 
> I just downloaded the py file and run the exact command, it failed. The 
> log is as following:
> 
> 
> babeltrace2 --plugin-path . -c source.demo.MyFirstSource -c 
> sink.demo.MyFirstSink
> 04-24 16:52:04.349 919805 919805 E CLI 
> add_descriptor_to_component_descriptor_set@babeltrace2.c:1720 Cannot 
> find component class: plugin-name="demo", comp-cls-name="MyFirstSource", 
> comp-cls-type=1
> 04-24 16:52:04.349 919805 919805 E CLI 
> cmd_run_ctx_init@babeltrace2.c:1882 Cannot find an operative message 
> interchange protocol version to use to create the `run` command's graph: 
> status=ERROR
> 04-24 16:52:04.349 919805 919805 E CLI cmd_run@babeltrace2.c:2465 Cannot 
> initialize the command's context.
> 
> ERROR:    [Babeltrace CLI] (babeltrace2.c:2465)
>    Cannot initialize the command's context.
> CAUSED BY [Babeltrace CLI] (babeltrace2.c:1882)
>    Cannot find an operative message interchange protocol version to use 
> to create the `run` command's graph: status=ERROR
> CAUSED BY [Babeltrace CLI] (babeltrace2.c:1720)
>    Cannot find component class: plugin-name="demo", 
> comp-cls-name="MyFirstSource", comp-cls-type=1
> 
> babeltrace2 --version
> Babeltrace 2.0.7 "Amqui" [v2.0.6-1-g825a0ed6d]
> 
> Amqui (/ɒmkwiː/) is a town in eastern Québec, Canada, at the base of the 
> Gaspé peninsula in Bas-Saint-Laurent. Located at the
> confluence of the Humqui and Matapédia Rivers, its proximity to 
> woodlands makes it a great destination for outdoor activities such as
> camping, hiking, and mountain biking.
> yannanwu@ue91e96f2951b5c:~/trees/lttng_test_run$
> 
> Is the cli changed or something? How can I make it right?
> 
> Besides, is it possible we create a pipeline in python and make use the 
> the python drafted plugin? Can you advise me how?
> 
> Amanda
> 
> 
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [babeltrace2]about python self-defined plugin loading

[Kienan Stewart via lttng-dev]

My apologies,

there is a typo in my previous e-mail, the library directory should be:

`<libdir>/babeltrace2/plugin-providers/'

thanks,
kienan

On 4/25/24 11:41 AM, Kienan Stewart via lttng-dev wrote:
> Hi Amanda,
> 
> could you double-check to ensure that babeltrace2 was built with 
> `--enable-python-plugins`, and that `import bt2` works?
> 
> There should be a babeltrace2-python-plugin-provider.so in 
> `<libdir>/babeltrace2/plugin-provides`
> 
> thanks,
> kienan
> 
> 
> On 4/24/24 11:28 PM, Wu, Yannan via lttng-dev wrote:
>> Hi, There,
>>
>> I am trying to construct a customized filter and sink based on 
>> babeltrace2 python binding. However, nether mine plugin nor the 
>> plugins sample I could find on the internet all dont work.
>>
>> For example, 
>> https://github.com/simark/babeltrace-fun-plugins/tree/master/my-first-components <https://github.com/simark/babeltrace-fun-plugins/tree/master/my-first-components>
>>
>> I just downloaded the py file and run the exact command, it failed. 
>> The log is as following:
>>
>>
>> babeltrace2 --plugin-path . -c source.demo.MyFirstSource -c 
>> sink.demo.MyFirstSink
>> 04-24 16:52:04.349 919805 919805 E CLI 
>> add_descriptor_to_component_descriptor_set@babeltrace2.c:1720 Cannot 
>> find component class: plugin-name="demo", 
>> comp-cls-name="MyFirstSource", comp-cls-type=1
>> 04-24 16:52:04.349 919805 919805 E CLI 
>> cmd_run_ctx_init@babeltrace2.c:1882 Cannot find an operative message 
>> interchange protocol version to use to create the `run` command's 
>> graph: status=ERROR
>> 04-24 16:52:04.349 919805 919805 E CLI cmd_run@babeltrace2.c:2465 
>> Cannot initialize the command's context.
>>
>> ERROR:    [Babeltrace CLI] (babeltrace2.c:2465)
>>    Cannot initialize the command's context.
>> CAUSED BY [Babeltrace CLI] (babeltrace2.c:1882)
>>    Cannot find an operative message interchange protocol version to 
>> use to create the `run` command's graph: status=ERROR
>> CAUSED BY [Babeltrace CLI] (babeltrace2.c:1720)
>>    Cannot find component class: plugin-name="demo", 
>> comp-cls-name="MyFirstSource", comp-cls-type=1
>>
>> babeltrace2 --version
>> Babeltrace 2.0.7 "Amqui" [v2.0.6-1-g825a0ed6d]
>>
>> Amqui (/ɒmkwiː/) is a town in eastern Québec, Canada, at the base of 
>> the Gaspé peninsula in Bas-Saint-Laurent. Located at the
>> confluence of the Humqui and Matapédia Rivers, its proximity to 
>> woodlands makes it a great destination for outdoor activities such as
>> camping, hiking, and mountain biking.
>> yannanwu@ue91e96f2951b5c:~/trees/lttng_test_run$
>>
>> Is the cli changed or something? How can I make it right?
>>
>> Besides, is it possible we create a pipeline in python and make use 
>> the the python drafted plugin? Can you advise me how?
>>
>> Amanda
>>
>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [babeltrace2]about python self-defined plugin loading

[Wu, Yannan via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 4169 bytes --]

Hihi, Kienan,

I finally find a workaround. I could success with the following code:
file_name = "bt_plugin_latency_filter.py"
current_directory=os.getcwd()

file_path = os.path.join(current_directory, file_name)
pset=bt2.find_plugins_in_path(file_path)

forplugininpset:

if plugin.name=="latency_filter"and hasattr(plugin,"filter_component_classes"):

    latency_filter_comp_cls=plugin.filter_component_classes['LatencyFilter']



But loading the whole directory, it just doesnt work from command line.

Amanda















________________________________
From: Kienan Stewart <kstewart@efficios.com>
Sent: Thursday, April 25, 2024 11:43:05 PM
To: Wu, Yannan; lttng-dev@lists.lttng.org
Subject: RE: [EXTERNAL] [lttng-dev] [babeltrace2]about python self-defined plugin loading

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



My apologies,

there is a typo in my previous e-mail, the library directory should be:

`<libdir>/babeltrace2/plugin-providers/'

thanks,
kienan

On 4/25/24 11:41 AM, Kienan Stewart via lttng-dev wrote:
> Hi Amanda,
>
> could you double-check to ensure that babeltrace2 was built with
> `--enable-python-plugins`, and that `import bt2` works?
>
> There should be a babeltrace2-python-plugin-provider.so in
> `<libdir>/babeltrace2/plugin-provides`
>
> thanks,
> kienan
>
>
> On 4/24/24 11:28 PM, Wu, Yannan via lttng-dev wrote:
>> Hi, There,
>>
>> I am trying to construct a customized filter and sink based on
>> babeltrace2 python binding. However, nether mine plugin nor the
>> plugins sample I could find on the internet all dont work.
>>
>> For example,
>> https://github.com/simark/babeltrace-fun-plugins/tree/master/my-first-components <https://github.com/simark/babeltrace-fun-plugins/tree/master/my-first-components>
>>
>> I just downloaded the py file and run the exact command, it failed.
>> The log is as following:
>>
>>
>> babeltrace2 --plugin-path . -c source.demo.MyFirstSource -c
>> sink.demo.MyFirstSink
>> 04-24 16:52:04.349 919805 919805 E CLI
>> add_descriptor_to_component_descriptor_set@babeltrace2.c:1720 Cannot
>> find component class: plugin-name="demo",
>> comp-cls-name="MyFirstSource", comp-cls-type=1
>> 04-24 16:52:04.349 919805 919805 E CLI
>> cmd_run_ctx_init@babeltrace2.c:1882 Cannot find an operative message
>> interchange protocol version to use to create the `run` command's
>> graph: status=ERROR
>> 04-24 16:52:04.349 919805 919805 E CLI cmd_run@babeltrace2.c:2465
>> Cannot initialize the command's context.
>>
>> ERROR:    [Babeltrace CLI] (babeltrace2.c:2465)
>>    Cannot initialize the command's context.
>> CAUSED BY [Babeltrace CLI] (babeltrace2.c:1882)
>>    Cannot find an operative message interchange protocol version to
>> use to create the `run` command's graph: status=ERROR
>> CAUSED BY [Babeltrace CLI] (babeltrace2.c:1720)
>>    Cannot find component class: plugin-name="demo",
>> comp-cls-name="MyFirstSource", comp-cls-type=1
>>
>> babeltrace2 --version
>> Babeltrace 2.0.7 "Amqui" [v2.0.6-1-g825a0ed6d]
>>
>> Amqui (/ɒmkwiː/) is a town in eastern Québec, Canada, at the base of
>> the Gaspé peninsula in Bas-Saint-Laurent. Located at the
>> confluence of the Humqui and Matapédia Rivers, its proximity to
>> woodlands makes it a great destination for outdoor activities such as
>> camping, hiking, and mountain biking.
>> yannanwu@ue91e96f2951b5c:~/trees/lttng_test_run$
>>
>> Is the cli changed or something? How can I make it right?
>>
>> Besides, is it possible we create a pipeline in python and make use
>> the the python drafted plugin? Can you advise me how?
>>
>> Amanda
>>
>>
>>
>> _______________________________________________
>> lttng-dev mailing list
>> lttng-dev@lists.lttng.org
>> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

[-- Attachment #1.2: Type: text/html, Size: 10024 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [babeltrace2]about python self-defined plugin loading

[Simon Marchi via lttng-dev]

On 2024-04-26 09:56, Wu, Yannan via lttng-dev wrote:
> Hihi, Kienan,
> 
> I finally find a workaround. I could success with the following code:
> file_name = "bt_plugin_latency_filter.py"
> current_directory=os.getcwd()
> 
> file_path = os.path.join(current_directory, file_name)
> pset=bt2.find_plugins_in_path(file_path)
> 
> forplugininpset:
> if plugin.name=="latency_filter"andhasattr(plugin,"filter_component_classes"):
> 
>     latency_filter_comp_cls=plugin.filter_component_classes['LatencyFilter']
> 
> But loading the whole directory, it just doesnt work from command line.
> Amanda

So, since this works, it confirms that your babeltrace build has support
for Python plugins.  Then there's no reason for your original example
using babeltrace-fun-plugins/my-first-components not to work.

When running the command:

  babeltrace2 --plugin-path . -c source.demo.MyFirstSource -c sink.demo.MyFirstSink

is your working directory babeltrace-fun-plugins/my-first-component?
`--plugin-path .` refers to the current working directory, so it must be
the directory that directly contains the `bt_plugin_*.py` file.

I made an example with docker to show it working, hopefully this can
help you figure out what's wrong on your side.  Put this inside a
`Dockerfile` file:

```
FROM debian:12

RUN apt-get -y update
RUN apt-get -y full-upgrade
RUN apt-get -y install babeltrace2 git
RUN git clone https://github.com/simark/babeltrace-fun-plugins.git
WORKDIR /babeltrace-fun-plugins/my-first-components
ENTRYPOINT babeltrace2 --plugin-path . -c source.demo.MyFirstSource -c sink.demo.MyFirstSink
```

Then, do either (whatever works for you):

  $ docker buildx build -t bt2-example .
  $ docker build -t bt2-example .

And finally:

  $ docker run -it bt2-example
  Stream beginning
  event my-event, timestamp 123
  Stream end

This last command runs babeltrace2, loading the Python plugin from
babeltrace-fun-plugins/my-first-components.

Finally, to investigate more in depth, you can increase the log level of
babeltrace2, like so:

  $ babeltrace2 --log-level DEBUG --plugin-path . -c source.demo.MyFirstSource -c sink.demo.MyFirstSink

This will output all sorts of things about babeltrace looking for
plugins and trying to load them.

Simon
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] [babeltrace2]about python self-defined plugin loading

[Wu, Yannan via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 2812 bytes --]

ok thanks for the help. I will try it later

Amanda

________________________________
From: Simon Marchi <simark@simark.ca>
Sent: Friday, April 26, 2024 10:57:59 PM
To: Wu, Yannan; Kienan Stewart; lttng-dev@lists.lttng.org
Subject: RE: [EXTERNAL] [lttng-dev] [babeltrace2]about python self-defined plugin loading

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you can confirm the sender and know the content is safe.



On 2024-04-26 09:56, Wu, Yannan via lttng-dev wrote:
> Hihi, Kienan,
>
> I finally find a workaround. I could success with the following code:
> file_name = "bt_plugin_latency_filter.py"
> current_directory=os.getcwd()
>
> file_path = os.path.join(current_directory, file_name)
> pset=bt2.find_plugins_in_path(file_path)
>
> forplugininpset:
> if plugin.name=="latency_filter"andhasattr(plugin,"filter_component_classes"):
>
>     latency_filter_comp_cls=plugin.filter_component_classes['LatencyFilter']
>
> But loading the whole directory, it just doesnt work from command line.
> Amanda

So, since this works, it confirms that your babeltrace build has support
for Python plugins.  Then there's no reason for your original example
using babeltrace-fun-plugins/my-first-components not to work.

When running the command:

  babeltrace2 --plugin-path . -c source.demo.MyFirstSource -c sink.demo.MyFirstSink

is your working directory babeltrace-fun-plugins/my-first-component?
`--plugin-path .` refers to the current working directory, so it must be
the directory that directly contains the `bt_plugin_*.py` file.

I made an example with docker to show it working, hopefully this can
help you figure out what's wrong on your side.  Put this inside a
`Dockerfile` file:

```
FROM debian:12

RUN apt-get -y update
RUN apt-get -y full-upgrade
RUN apt-get -y install babeltrace2 git
RUN git clone https://github.com/simark/babeltrace-fun-plugins.git
WORKDIR /babeltrace-fun-plugins/my-first-components
ENTRYPOINT babeltrace2 --plugin-path . -c source.demo.MyFirstSource -c sink.demo.MyFirstSink
```

Then, do either (whatever works for you):

  $ docker buildx build -t bt2-example .
  $ docker build -t bt2-example .

And finally:

  $ docker run -it bt2-example
  Stream beginning
  event my-event, timestamp 123
  Stream end

This last command runs babeltrace2, loading the Python plugin from
babeltrace-fun-plugins/my-first-components.

Finally, to investigate more in depth, you can increase the log level of
babeltrace2, like so:

  $ babeltrace2 --log-level DEBUG --plugin-path . -c source.demo.MyFirstSource -c sink.demo.MyFirstSink

This will output all sorts of things about babeltrace looking for
plugins and trying to load them.

Simon

[-- Attachment #1.2: Type: text/html, Size: 4265 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev