how to do large-ish IPC transfers

how to do large-ish IPC transfers

[Brad Bishop]

Hi everyone

I have an application or two that need to collect and save ~ 16KiB of  
debugging data.  The applications have various triggers but they are very  
infrequent when deployed (on the order of months/years).  In the lab it  
would happen much more frequently.  It isn’t really debugging information  
for the developer of the application but rather for customer service  
personnel debugging broken components on a deployed server, after the event  
(and state) that caused the data to be collected.

The applications generating this data will not be the same applications  
that publish it (the webserver), so there is a need to transfer it using  
IPC.

16KiB seemed a little big for a DBus interface, and we’d have to ascii  
encode all that into a string.  Some other ideas were:

1 - passing a file descriptor over dbus
2 - passing a file name over dbus
3 - shared memory

Are there reasons to steer clear of any of these?  Security concerns?   
Performance concerns?  Maintainability concerns?

Are there other choices I’m not thinking of?

thanks in advance!

-brad

Re: how to do large-ish IPC transfers

[Vernon Mauery]

On 11-Sep-2019 01:49 PM, Brad Bishop wrote:
> Hi everyone
> 
> I have an application or two that need to collect and save ~ 16KiB of
> debugging data.  The applications have various triggers but they are very
> infrequent when deployed (on the order of months/years).  In the lab it
> would happen much more frequently.  It isn’t really debugging information
> for the developer of the application but rather for customer service
> personnel debugging broken components on a deployed server, after the event
> (and state) that caused the data to be collected.
> 
> The applications generating this data will not be the same applications that
> publish it (the webserver), so there is a need to transfer it using IPC.
> 
> 16KiB seemed a little big for a DBus interface, and we’d have to ascii
> encode all that into a string.  Some other ideas were:
> 
> 1 - passing a file descriptor over dbus

This would probably be the best, given our architecture. It would allow, 
at some point in the future, to have the sender have different 
permissions on the file and allow access to a less-privileged process.

> 2 - passing a file name over dbus
> 3 - shared memory

Two and three could be the same (if that file is in a tmpfs). System V 
shared memory is a pain to use, so simply mmap'ing a file in tmpfs is 
the easiest way to go. But both of these require permissions to align 
between sender and receiver.

--Vernon

> 
> Are there reasons to steer clear of any of these?  Security concerns?
> Performance concerns?  Maintainability concerns?
> 
> Are there other choices I’m not thinking of?
> 
> thanks in advance!
> 
> -brad

Re: how to do large-ish IPC transfers

[Andrew Jeffery]

On Thu, 12 Sep 2019, at 04:01, Vernon Mauery wrote:
> On 11-Sep-2019 01:49 PM, Brad Bishop wrote:
> > Hi everyone
> > 
> > I have an application or two that need to collect and save ~ 16KiB of
> > debugging data.  The applications have various triggers but they are very
> > infrequent when deployed (on the order of months/years).  In the lab it
> > would happen much more frequently.  It isn’t really debugging information
> > for the developer of the application but rather for customer service
> > personnel debugging broken components on a deployed server, after the event
> > (and state) that caused the data to be collected.
> > 
> > The applications generating this data will not be the same applications that
> > publish it (the webserver), so there is a need to transfer it using IPC.
> > 
> > 16KiB seemed a little big for a DBus interface, and we’d have to ascii
> > encode all that into a string.  Some other ideas were:
> > 
> > 1 - passing a file descriptor over dbus
> 
> This would probably be the best, given our architecture. It would allow, 
> at some point in the future, to have the sender have different 
> permissions on the file and allow access to a less-privileged process.
> 
> > 2 - passing a file name over dbus
> > 3 - shared memory
> 
> Two and three could be the same (if that file is in a tmpfs). System V 
> shared memory is a pain to use, so simply mmap'ing a file in tmpfs is 
> the easiest way to go. But both of these require permissions to align 
> between sender and receiver.
> 

I agree with Vernon here on all fronts.

Andrew

Re: how to do large-ish IPC transfers

[Alexander Amelkin]

[-- Attachment #1.1: Type: text/plain, Size: 2828 bytes --]

11.09.2019 21:31, Vernon Mauery wrote:
> On 11-Sep-2019 01:49 PM, Brad Bishop wrote:
>> Hi everyone
>>
>> I have an application or two that need to collect and save ~ 16KiB of
>> debugging data.  The applications have various triggers but they are very
>> infrequent when deployed (on the order of months/years).  In the lab it
>> would happen much more frequently.  It isn’t really debugging information
>> for the developer of the application but rather for customer service
>> personnel debugging broken components on a deployed server, after the event
>> (and state) that caused the data to be collected.
>>
>> The applications generating this data will not be the same applications that
>> publish it (the webserver), so there is a need to transfer it using IPC.
>>
>> 16KiB seemed a little big for a DBus interface, and we’d have to ascii
>> encode all that into a string.  Some other ideas were:
>>
>> 1 - passing a file descriptor over dbus
> This would probably be the best, given our architecture. It would allow, 
> at some point in the future, to have the sender have different 
> permissions on the file and allow access to a less-privileged process.

What is a 'file descriptor' you're referring to here? The Linux file descriptor
(the integer you get from open()) is process-specific and doesn't keep its
meaning in another process' context.

>> 2 - passing a file name over dbus
>> 3 - shared memory

Classic IPC in POSIX are named UNIX sockets, named pipes, message queues, or
indeed shared memory (access requires a shared key for SysV or a name for POSIX,
both could be sent over d-bus).

> Two and three could be the same (if that file is in a tmpfs). System V 
> shared memory is a pain to use, so simply mmap'ing a file in tmpfs is 
> the easiest way to go. But both of these require permissions to align 
> between sender and receiver.

Well, System V shm isn't that much of a pain actually (just some), but there is
also POSIX shared memory (man shm_overview), mmap() being a part of that spec.
You'll need synch primitives anyway, even for a file. For sendinding
message-like data I'd probably choose message queues (man mq_overview). They are
well capable of such small amounts as 16KiB.

> --Vernon
>
>> Are there reasons to steer clear of any of these?  Security concerns?
>> Performance concerns?  Maintainability concerns?
>>
>> Are there other choices I’m not thinking of?
I personally consider classic POSIX IPC mechanisms much better performing, much
less complex, much more robust, more thought-through and overall more proven
than d-bus and systemd. I wouldn't even think of d-bus when developing an
embedded system, but `that's just me...

With best regards,
Alexander Amelkin,
BIOS/BMC Team Lead, YADRO
https://yadro.com



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: how to do large-ish IPC transfers

[Brad Bishop]

at 6:30 AM, Alexander Amelkin <a.amelkin@yadro.com> wrote:

> 11.09.2019 21:31, Vernon Mauery wrote:
>> On 11-Sep-2019 01:49 PM, Brad Bishop wrote:
>>> Hi everyone
>>>
>>> I have an application or two that need to collect and save ~ 16KiB of
>>> debugging data.  The applications have various triggers but they are very
>>> infrequent when deployed (on the order of months/years).  In the lab it
>>> would happen much more frequently.  It isn’t really debugging information
>>> for the developer of the application but rather for customer service
>>> personnel debugging broken components on a deployed server, after the  
>>> event
>>> (and state) that caused the data to be collected.
>>>
>>> The applications generating this data will not be the same applications  
>>> that
>>> publish it (the webserver), so there is a need to transfer it using IPC.
>>>
>>> 16KiB seemed a little big for a DBus interface, and we’d have to ascii
>>> encode all that into a string.  Some other ideas were:
>>>
>>> 1 - passing a file descriptor over dbus
>> This would probably be the best, given our architecture. It would allow,
>> at some point in the future, to have the sender have different
>> permissions on the file and allow access to a less-privileged process.
>
> What is a 'file descriptor' you're referring to here? The Linux file  
> descriptor
> (the integer you get from open()) is process-specific and doesn't keep its
> meaning in another process' context.

On Linux you can pass file descriptors (the ones returned by open) over  
unix sockets.  The implementation of the ‘h’ DBus type relies on that.

Re: how to do large-ish IPC transfers

[Alexander Amelkin]

[-- Attachment #1.1: Type: text/plain, Size: 690 bytes --]

12.09.2019 16:24, Brad Bishop wrote:
>
>> What is a 'file descriptor' you're referring to here? The Linux file descriptor
>> (the integer you get from open()) is process-specific and doesn't keep its
>> meaning in another process' context.
>
> On Linux you can pass file descriptors (the ones returned by open) over unix
> sockets.  The implementation of the ‘h’ DBus type relies on that.

Oh, great! Didn't know about that. Thanks.

Anyway, I think I'd just use message queues or sockets (directly). What's the
profit of using a file whose descriptor is sent over a socket instead of just
using the socket for communication?

With best regards,
Alexander Amelkin



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: how to do large-ish IPC transfers

[Brad Bishop]

at 3:42 AM, Alexander Amelkin <a.amelkin@yadro.com> wrote:

> 12.09.2019 16:24, Brad Bishop wrote:
>>> What is a 'file descriptor' you're referring to here? The Linux file  
>>> descriptor
>>> (the integer you get from open()) is process-specific and doesn't keep  
>>> its
>>> meaning in another process' context.
>>
>> On Linux you can pass file descriptors (the ones returned by open) over  
>> unix
>> sockets.  The implementation of the ‘h’ DBus type relies on that.
>
> Oh, great! Didn't know about that. Thanks.
>
> Anyway, I think I'd just use message queues or sockets (directly). What's  
> the
> profit of using a file whose descriptor is sent over a socket instead of  
> just
> using the socket for communication?

I feel like you might be trolling me a tiny bit on our usage of DBus but  
I’ll answer just in case you aren’t…

In this particular case we already have a socket established and are  
passing other data over it, using an existing protocol (DBus).  Obviously  
we can’t stuff this file data onto that socket without breaking the  
protocol and all the code that implements it.  We could open yet another  
socket for this but it seems simpler to me to just pass the fd over the  
existing one…especially when the DBus protocol has allowances for that.

-brad