Re: Is lttng namespace-ready?

Re: Is lttng namespace-ready?

[Thibault, Daniel]

Date: Mon, 8 Jul 2013 12:33:41 -0400
From: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>

> * Thibault, Daniel (Daniel.Thibault@drdc-rddc.gc.ca) wrote:
> >    But what of per-UID traces?  What happens to a trace in progress if
> >    one or more processes switch user namespaces?
>
> nothing.
>
> >  Presumably new
> >    sub-directories would be created under "session-name/ust/uid/"?
>
> no.

   Back on 26 Jun 2013, you said:

> Subject: Re: [lttng-dev] [RELEASE] LTTng Tools 2.2.0 - Cuda (STABLE)
> Message-ID: <20130626211926.GC4725@Krystal>
...
> > I also presume the uid used [in the path (e.g. $HOME/lttng-traces/session-20130611-1223344/ust/uid/1000/64-bit)] is the real uid (ruid)?
>
> yes, this is correct. The one returned by getuid().

   This getuid() call occurs from the tracepoint provider attached to the instrumented application, right?  If user namespaces are in use, this will therefore be the virtual uid, and if a process is cloned() into a new user namespace it will thus tell the daemons that its events are issuing from that virtual uid.  If it is a heretofore unknown uid, a new subdirectory should thus appear under /ust/uid.  If it is an already known uid (more precisely, an already-encountered numerical value), its events should just flow into the existing channel files.  There is (I suspect) no risk of trace files being overwritten accidentally.

   The difficulty lies at the analysis end, if the user wants to sort out event streams that issue from the same numerical virtual uids but that are actually different (issuing from different real uids or from different user namespaces).  One could figure it out manually by capturing namespace API events (clone(), unshared() and setns()), but it would be nicer if lttng's metadata held this information or if lttv did it for us, wouldn't it?  Maybe the namespace identifiers could be added as context using enable-event?

   Another question that occurs to me as I write this is: will a given real uid's lttng-consumerd daemon service the processes belonging to this uid and its subordinate user namespaces, or will each new user namespace spawn a new lttng-consumerd daemon?  Is the answer the same in the per-pid and per-uid cases?

> >    What about the case where two processes run by two different users (and thus being recorded under different uid subdirectories) switch
> >    to private user namespaces which happen to both map to the same numerical virtual uid?  We need to make sure the second process
> >    switch does not overwrite the trace files of the first (recall that per-uid trace paths do not include timestamps like per-pid trace
> >    paths do).
>
> Disambiguation between different PIDs requires the vpid context if you use per-UID buffers.

   As a user, I'd probably be better off using the pid (real pid) context.  It would disambiguate between unrelated processes bearing the same vpids.

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada
<http://www.valcartier.drdc-rddc.gc.ca/>

Re: Is lttng namespace-ready?

[Mathieu Desnoyers]

* Thibault, Daniel (Daniel.Thibault@drdc-rddc.gc.ca) wrote:
>    I was reading Michael Kerrisk's "Namespaces in operation"
>    (http://lwn.net/Articles/531114/), which explains most of the
>    namespace functionalities, and this brings to mind a number of
>    questions.
> 
>    Are trace paths namespace-ready?  Per-PID traces are apparently not
>    much at risk since a process can join a PID namespace only upon its
>    creation (that is to say, PIDs are immutable).  However, collisions
>    are still possible, such as if two user-space processes start
>    within the same one-second window in two sibling PID namespaces
>    such that they have the same virtual PID.

yes, this is a possible race indeed for per-PID tracing in UST, since I
think we're using namespaced PIDs to identify programs.

> 
>    But what of per-UID traces?  What happens to a trace in progress if
>    one or more processes switch user namespaces?

nothing.

>  Presumably new
>    sub-directories would be created under "session-name/ust/uid/"?

no.

>    What about the case where two processes run by two different users
>    (and thus being recorded under different uid subdirectories) switch
>    to private user namespaces which happen to both map to the same
>    numerical virtual uid?  We need to make sure the second process
>    switch does not overwrite the trace files of the first (recall that
>    per-uid trace paths do not include timestamps like per-pid trace
>    paths do).

Disambiguation between different PIDs requires the vpid context if you
use per-UID buffers.

> 
>    Finally, in the case of a remote trace being received by the relay
>    daemon, could a network or UTS namespace switch of the traced
>    processes cause trouble?  I suspect not, since the daemons at
>    either end of the connection are lttng-owned and would presumably
>    not undergo namespace switches themselves.

I don't see it causing an issue. It might be good to try it out though.

Thanks,

Mathieu

> 
> Daniel U. Thibault
> Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
> Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
> R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
> 2459 route de la Bravoure
> Québec QC  G3J 1X5
> CANADA
> Vox : (418) 844-4000 x4245
> Fax : (418) 844-4538
> NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
> Gouvernement du Canada | Government of Canada
> <http://www.valcartier.drdc-rddc.gc.ca/>
> 
> _______________________________________________
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Is lttng namespace-ready?

[Thibault, Daniel]

   I was reading Michael Kerrisk's "Namespaces in operation" (http://lwn.net/Articles/531114/), which explains most of the namespace functionalities, and this brings to mind a number of questions.

   Are trace paths namespace-ready?  Per-PID traces are apparently not much at risk since a process can join a PID namespace only upon its creation (that is to say, PIDs are immutable).  However, collisions are still possible, such as if two user-space processes start within the same one-second window in two sibling PID namespaces such that they have the same virtual PID.

   But what of per-UID traces?  What happens to a trace in progress if one or more processes switch user namespaces?  Presumably new sub-directories would be created under "session-name/ust/uid/"?  What about the case where two processes run by two different users (and thus being recorded under different uid subdirectories) switch to private user namespaces which happen to both map to the same numerical virtual uid?  We need to make sure the second process switch does not overwrite the trace files of the first (recall that per-uid trace paths do not include timestamps like per-pid trace paths do).

   Finally, in the case of a remote trace being received by the relay daemon, could a network or UTS namespace switch of the traced processes cause trouble?  I suspect not, since the daemons at either end of the connection are lttng-owned and would presumably not undergo namespace switches themselves.

Daniel U. Thibault
Protection des systèmes et contremesures (PSC) | Systems Protection & Countermeasures (SPC)
Cyber sécurité pour les missions essentielles (CME) | Mission Critical Cyber Security (MCCS)
R & D pour la défense Canada - Valcartier (RDDC Valcartier) | Defence R&D Canada - Valcartier (DRDC Valcartier)
2459 route de la Bravoure
Québec QC  G3J 1X5
CANADA
Vox : (418) 844-4000 x4245
Fax : (418) 844-4538
NAC : 918V QSDJ <http://www.travelgis.com/map.asp?addr=918V%20QSDJ>
Gouvernement du Canada | Government of Canada
<http://www.valcartier.drdc-rddc.gc.ca/>