[lttng-dev] LTTng Support for Mount Namespace

[lttng-dev] LTTng Support for Mount Namespace

[Mohammad Kavousi via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 901 bytes --]

Dear LTTng developers,

Our group at Northwestern has been using your amazing tool for the purpose
of provenance tracking on Linux-based systems and we are very fond of the
performance and accuracy it provides.

Our analysis shows that mnt_ns context is supported in the 2.12 version of
LTTng. However, though, adding the mnt_ns context using the add-context command
produces this error:

Error: mnt_ns: Context unavailable on this kernel

We have tried adding the context to the more recent version of the kernel
(5.8) on Ubuntu 20.04, as well as older kernel versions such as the 4.4
version on Ubuntu 16.04. However, we always receive the above error trying
to add the mnt_ns context.

We could not find which kernel versions are supported for adding this
context, or whether they need to be built with special flags. I would
appreciate your guidance on resolving this issue.


Thank you,
Mohammad

[-- Attachment #1.2: Type: text/html, Size: 1663 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] LTTng Support for Mount Namespace

[Michael Jeanson via lttng-dev]

On 2021-01-06 10 h 50, Mohammad Kavousi via lttng-dev wrote:
> Dear LTTng developers,
> 
> Our group at Northwestern has been using your amazing tool for the 
> purpose of provenance tracking on Linux-based systems and we are very 
> fond of the performance and accuracy it provides.
> 
> Our analysis shows that mnt_ns context is supported in the 2.12 version 
> of LTTng. However, though, adding the mnt_ns context using 
> the add-context command produces this error:
> 
> Error: mnt_ns: Context unavailable on this kernel
> 
> We have tried adding the context to the more recent version of the 
> kernel (5.8) on Ubuntu 20.04, as well as older kernel versions such as 
> the 4.4 version on Ubuntu 16.04. However, we always receive the above 
> error trying to add the mnt_ns context.
> 
> We could not find which kernel versions are supported for adding this 
> context, or whether they need to be built with special flags. I would 
> appreciate your guidance on resolving this issue.
> 
> 
> Thank you,
> Mohammad

Hi,

Unfortunately the definition of 'struct mnt_namespace' is in a private 
kernel header (fs/mount.h) unlike other namespaces.  Private headers are 
not included in the kernel headers package of distributions like Ubuntu, 
to build support for this namespace context in lttng-modules you need to 
use the full kernel source tree.

Or as a quick hack, you could copy 'fs/mount.h' from the original source 
tree to your kernel headers package build directory, which on Ubuntu is 
usually '/lib/modules/$(uname -r)/build'.

Hoe this helps,

Michael
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] LTTng Support for Mount Namespace

[Mohammad Kavousi via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 2209 bytes --]

Michael,

Thank you for your swift response. However, for the first part, I'm not
sure what it means by "using" the full kernel source. How would I use it
with lttng-modules?

As for the quick hack part, I grabbed the kernel source and copied the
mount.h onto the folder you mentioned, and it did not help. I don't expect
lttng to automatically find that file and use it; also there are no other
header files in the package build directory. Could you kindly give us more
instructions?

Thanks,
Mohammad

On Wed, Jan 6, 2021 at 11:02 AM Michael Jeanson <mjeanson@efficios.com>
wrote:

> On 2021-01-06 10 h 50, Mohammad Kavousi via lttng-dev wrote:
> > Dear LTTng developers,
> >
> > Our group at Northwestern has been using your amazing tool for the
> > purpose of provenance tracking on Linux-based systems and we are very
> > fond of the performance and accuracy it provides.
> >
> > Our analysis shows that mnt_ns context is supported in the 2.12 version
> > of LTTng. However, though, adding the mnt_ns context using
> > the add-context command produces this error:
> >
> > Error: mnt_ns: Context unavailable on this kernel
> >
> > We have tried adding the context to the more recent version of the
> > kernel (5.8) on Ubuntu 20.04, as well as older kernel versions such as
> > the 4.4 version on Ubuntu 16.04. However, we always receive the above
> > error trying to add the mnt_ns context.
> >
> > We could not find which kernel versions are supported for adding this
> > context, or whether they need to be built with special flags. I would
> > appreciate your guidance on resolving this issue.
> >
> >
> > Thank you,
> > Mohammad
>
> Hi,
>
> Unfortunately the definition of 'struct mnt_namespace' is in a private
> kernel header (fs/mount.h) unlike other namespaces.  Private headers are
> not included in the kernel headers package of distributions like Ubuntu,
> to build support for this namespace context in lttng-modules you need to
> use the full kernel source tree.
>
> Or as a quick hack, you could copy 'fs/mount.h' from the original source
> tree to your kernel headers package build directory, which on Ubuntu is
> usually '/lib/modules/$(uname -r)/build'.
>
> Hoe this helps,
>
> Michael
>

[-- Attachment #1.2: Type: text/html, Size: 2836 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] LTTng Support for Mount Namespace

[Michael Jeanson via lttng-dev]

On 2021-01-09 09 h 03, Mohammad Kavousi wrote:
> Michael,
> 
> Thank you for your swift response. However, for the first part, I'm not 
> sure what it means by "using" the full kernel source. How would I use it 
> with lttng-modules?
> 
> As for the quick hack part, I grabbed the kernel source and copied the 
> mount.h onto the folder you mentioned, and it did not help. I don't 
> expect lttng to automatically find that file and use it; also there are 
> no other header files in the package build directory. Could you kindly 
> give us more instructions?
> 
> Thanks,
> Mohammad

The lttng modules are shipped in Ubuntu as a dkms package which means 
they are built directly on your machine for each kernel version you 
install. So after adding the required file in your headers you'll need 
to rebuild the modules with dkms and unload / reload them in your 
running kernel or just reboot the machine.

To rebuild all installed dkms modules for the current kernel:

   sudo dkms autoinstall
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] LTTng Support for Mount Namespace

[Mohammad Kavousi via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 1476 bytes --]

Thanks again for your reply,

To be specific, I did exactly this:

- Install linux source code, then copy the mentioned file to the folder
using: cp /usr/src/linux-headers-5.4.0-59/fs/mount.h /lib/modules/$(uname
-r)/build

- Run sudo dkms autoinstall (I also did reboot)

- Create the session and attempt to add the context

However, I'm getting the same error.

On Tue, Jan 12, 2021 at 9:05 AM Michael Jeanson <mjeanson@efficios.com>
wrote:

> On 2021-01-09 09 h 03, Mohammad Kavousi wrote:
> > Michael,
> >
> > Thank you for your swift response. However, for the first part, I'm not
> > sure what it means by "using" the full kernel source. How would I use it
> > with lttng-modules?
> >
> > As for the quick hack part, I grabbed the kernel source and copied the
> > mount.h onto the folder you mentioned, and it did not help. I don't
> > expect lttng to automatically find that file and use it; also there are
> > no other header files in the package build directory. Could you kindly
> > give us more instructions?
> >
> > Thanks,
> > Mohammad
>
> The lttng modules are shipped in Ubuntu as a dkms package which means
> they are built directly on your machine for each kernel version you
> install. So after adding the required file in your headers you'll need
> to rebuild the modules with dkms and unload / reload them in your
> running kernel or just reboot the machine.
>
> To rebuild all installed dkms modules for the current kernel:
>
>    sudo dkms autoinstall
>

[-- Attachment #1.2: Type: text/html, Size: 2110 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] LTTng Support for Mount Namespace

[Michael Jeanson via lttng-dev]

On 2021-01-12 12 h 29, Mohammad Kavousi wrote:
> Thanks again for your reply,
> 
> To be specific, I did exactly this:
> 
> - Install linux source code, then copy the mentioned file to the folder 
> using: cp 
> /usr/src/linux-headers-5.4.0-59/fs/mount.h /lib/modules/$(uname -r)/build

The path to the header has to stay the same, so you would need to copy 
it to '/lib/modules/$(uname -r)/build/fs'.

The rest of the steps seem fine.

One thing you can do if it still doesn't work, is modify 'Makefile' and 
'Kbuild.common' in the lttng-modules source directory and remove the 
conditions on 'mnt_ns_dep'. That would make the build fail when 
'fs/mount.h' isn't found and may help you find the cause.

> 
> - Run sudo dkms autoinstall (I also did reboot)
> 
> - Create the session and attempt to add the context
> 
> However, I'm getting the same error.
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] LTTng Support for Mount Namespace

[Mohammad Kavousi via lttng-dev]

[-- Attachment #1.1: Type: text/plain, Size: 1601 bytes --]

Before, I was installing using the ppa repository. Based on your
instructions, I made lttng using the source code in a totally new
environment, having the mount.h under '/lib/modules/$(uname -r)/build/fs'.

I removed the "ifneq" and "endif" statements for 'Makefile' and
'Kbuild.common' which were surrounding "mnt_ns_dep". lttng was installed
successfully and the kernel trace session started, but I'm still getting
the same error.

As a hint, in the build process for lttng-modules, I found messages such as
"./fs/ext4/*.h not found". Is that part of the problem? Meaning, is
enabling the mnt context filesystem-specific?



On Tue, Jan 12, 2021 at 12:51 PM Michael Jeanson <mjeanson@efficios.com>
wrote:

> On 2021-01-12 12 h 29, Mohammad Kavousi wrote:
> > Thanks again for your reply,
> >
> > To be specific, I did exactly this:
> >
> > - Install linux source code, then copy the mentioned file to the folder
> > using: cp
> > /usr/src/linux-headers-5.4.0-59/fs/mount.h /lib/modules/$(uname -r)/build
>
> The path to the header has to stay the same, so you would need to copy
> it to '/lib/modules/$(uname -r)/build/fs'.
>
> The rest of the steps seem fine.
>
> One thing you can do if it still doesn't work, is modify 'Makefile' and
> 'Kbuild.common' in the lttng-modules source directory and remove the
> conditions on 'mnt_ns_dep'. That would make the build fail when
> 'fs/mount.h' isn't found and may help you find the cause.
>
> >
> > - Run sudo dkms autoinstall (I also did reboot)
> >
> > - Create the session and attempt to add the context
> >
> > However, I'm getting the same error.
>

[-- Attachment #1.2: Type: text/html, Size: 2207 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

Re: [lttng-dev] LTTng Support for Mount Namespace

[Michael Jeanson via lttng-dev]

On 2021-01-12 15 h 30, Mohammad Kavousi wrote:
> Before, I was installing using the ppa repository. Based on your 
> instructions, I made lttng using the source code in a totally new 
> environment, having the mount.h under '/lib/modules/$(uname -r)/build/fs'.
> 
> I removed the "ifneq" and "endif" statements for 'Makefile' and 
> 'Kbuild.common' which were surrounding "mnt_ns_dep". lttng was installed 
> successfully and the kernel trace session started, but I'm still getting 
> the same error.

Make sure you delete 'ccflags-y += -DLTTNG_MNT_NS_MISSING_HEADER' from 
Kbuild.common, this is what disables the mount namespace support when 
the modules are built.

> 
> As a hint, in the build process for lttng-modules, I found messages such 
> as "./fs/ext4/*.h not found". Is that part of the problem? Meaning, is 
> enabling the mnt context filesystem-specific?

No this is related to the probe that enables ext4 specific tracepoints.
_______________________________________________
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev