Shared memory usage

Shared memory usage

[linux-os (Dick Johnson)]

Hello,

I have an embedded system in which there are about 18.4
megabytes of shared libraries using shared memory, plus
12 pages of shared memory, memory-mapped into user-space.
This all works fine.

But the customer complained during certification testing
that shared memory in use is not measured and therefore
cannot be verified. This means that there may be rogue
communications channels, using shared memory, in the
system. I need to prove that there are no such channels
by metering the shared memory and then accounting for
every bit shown.

/proc/meminfo does not show any shared memory in use!

         total:    used:    free:  shared: buffers:  cached:
Mem:  255332352  7806976 247525376        0    94208  2764800
Swap:        0        0        0
MemTotal:       249348 kB
MemFree:        241724 kB
MemShared:           0 kB         <----------
Buffers:            92 kB
Cached:           2700 kB
SwapCached:          0 kB
Active:            608 kB
Inactive:         4348 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:       249348 kB
LowFree:        241724 kB
SwapTotal:           0 kB
SwapFree:            0 kB


This used Linux version 2.4.26. In attempting to troubleshoot
this, I found that current kernel versions don't even have a
shared memory entry in /proc/meminfo.

So, how do I find the total amount of shared memory in use
on a system? This does not need to be in /proc. I could
write a driver that finds all the shared memory and reports
it, but I need to know how. Also, anybody certifying systems
for secure use needs to know the amount of shared memory in
use so it would be a real good idea if that information was
available in /proc as it was in the past before it was
removed.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.54 BogoMips).
Warning : 98.36% of all statistics are fiction.
.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Shared memory usage

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 1244 bytes --]

On Mon, 16 Jan 2006 09:15:16 EST, "linux-os (Dick Johnson)" said:
> But the customer complained during certification testing
> that shared memory in use is not measured and therefore
> cannot be verified. This means that there may be rogue
> communications channels, using shared memory, in the
> system. I need to prove that there are no such channels
> by metering the shared memory and then accounting for
> every bit shown.

The customer is confused, and your test is broken as designed.

The fact that you look in /proc/meminfo and account for every shared
memory page *at this instant* doesn't mean there isn't a communication
channel *at some other time*. Even if you run a daemon that does nothing
but monitor this usage 10 times a second, and complain if a discrepancy
is found, it *still* won't work:

1) It's racy - 2 processes can mmap() some space during that 0.1 seconds,
transfer the info, and detach the memory without your knowledge.

2) It's racy - if you inquire *while* some other process is in some intermediate
state, causing false positives that will drive the SSO nuts.

The *proper* solution is to use something like SELinux that will flat-out
*prohibit* the attachment of a shared memory segment that isn't permitted.

[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Shared memory usage

[linux-os (Dick Johnson)]

On Mon, 16 Jan 2006 Valdis.Kletnieks@vt.edu wrote:

> On Mon, 16 Jan 2006 09:15:16 EST, "linux-os (Dick Johnson)" said:
>> But the customer complained during certification testing
>> that shared memory in use is not measured and therefore
>> cannot be verified. This means that there may be rogue
>> communications channels, using shared memory, in the
>> system. I need to prove that there are no such channels
>> by metering the shared memory and then accounting for
>> every bit shown.
>
> The customer is confused, and your test is broken as designed.
>
> The fact that you look in /proc/meminfo and account for every shared
> memory page *at this instant* doesn't mean there isn't a communication
> channel *at some other time*. Even if you run a daemon that does nothing
> but monitor this usage 10 times a second, and complain if a discrepancy
> is found, it *still* won't work:
>
> 1) It's racy - 2 processes can mmap() some space during that 0.1 seconds,
> transfer the info, and detach the memory without your knowledge.
>
> 2) It's racy - if you inquire *while* some other process is in some
> intermediate
> state, causing false positives that will drive the SSO nuts.
>
> The *proper* solution is to use something like SELinux that will flat-out
> *prohibit* the attachment of a shared memory segment that isn't permitted.
>

The customer is not confused and is, in fact, a known expert.
This is not a "general purpose" setup. There are no executables
except what is burned into R/O PROM. There is no code that can
upload new executables nor any way to execute them. There is no
shell and no init.

There is a startup program called 'init' for convenience, but
whatever it does is hard-coded. It starts a bunch of predefined
tasks and sleeps after performing some initialization.

So, if the executable files are exactly like those which
were reviewed during a security review, and if all
communications between processes can be accounted for,
including shared memory, then it can be guaranteed that
whatever is being executed is exactly what passed the
security review. However, if all communications channels
cannot be accounted for, then it is not possible for such
a guarantee.

It can be argued that, once the executable code is known,
there is no reason to review possible rogue communications
channels. However, the customer is an expert in this field
and requires an audit of all possible communications
channels, including shared memory.

Note that this is an audit. The code for every communications
channel is reviewed. To verify that all such channels are
audited, one needs to find them. For instance even 'vdso' is
audited. These are not continuous audits that operate at
run-time. The reason for this 'static' audit is so that
CPU time eating checks don't have to be done at runtime.


Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.54 BogoMips).
Warning : 98.36% of all statistics are fiction.
.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Shared memory usage

[Jan Knutar]

On Monday 16 January 2006 16:15, linux-os (Dick Johnson) wrote:

> /proc/meminfo does not show any shared memory in use!

echo m > /proc/sysrq-trigger ; dmesg | grep shared

> available in /proc as it was in the past before it was
> removed.

I think this was all over FAQs covering 2.4 -> 2.6 transition...

Re: Shared memory usage

[linux-os (Dick Johnson)]

On Mon, 16 Jan 2006, Jan Knutar wrote:

> On Monday 16 January 2006 16:15, linux-os (Dick Johnson) wrote:
>
>> /proc/meminfo does not show any shared memory in use!
>
> echo m > /proc/sysrq-trigger ; dmesg | grep shared
>
>> available in /proc as it was in the past before it was
>> removed.
>
> I think this was all over FAQs covering 2.4 -> 2.6 transition...
>

Thanks. That works.

Cheers,
Dick Johnson
Penguin : Linux version 2.6.13.4 on an i686 machine (5589.54 BogoMips).
Warning : 98.36% of all statistics are fiction.
.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Shared memory usage

[Ram Gupta]

Did you think about getting shared memory information using
shmctl(IPC_STAT). It provides useful information but I am not sure if
that will serve your purpose fully.

Regards
Ram Gupta

On 1/16/06, linux-os (Dick Johnson) <linux-os@analogic.com> wrote:
>
> On Mon, 16 Jan 2006 Valdis.Kletnieks@vt.edu wrote:
>
> > On Mon, 16 Jan 2006 09:15:16 EST, "linux-os (Dick Johnson)" said:
> >> But the customer complained during certification testing
> >> that shared memory in use is not measured and therefore
> >> cannot be verified. This means that there may be rogue
> >> communications channels, using shared memory, in the
> >> system. I need to prove that there are no such channels
> >> by metering the shared memory and then accounting for
> >> every bit shown.
> >
> > The customer is confused, and your test is broken as designed.
> >
> > The fact that you look in /proc/meminfo and account for every shared
> > memory page *at this instant* doesn't mean there isn't a communication
> > channel *at some other time*. Even if you run a daemon that does nothing
> > but monitor this usage 10 times a second, and complain if a discrepancy
> > is found, it *still* won't work:
> >
> > 1) It's racy - 2 processes can mmap() some space during that 0.1 seconds,
> > transfer the info, and detach the memory without your knowledge.
> >
> > 2) It's racy - if you inquire *while* some other process is in some
> > intermediate
> > state, causing false positives that will drive the SSO nuts.
> >
> > The *proper* solution is to use something like SELinux that will flat-out
> > *prohibit* the attachment of a shared memory segment that isn't permitted.
> >
>
> The customer is not confused and is, in fact, a known expert.
> This is not a "general purpose" setup. There are no executables
> except what is burned into R/O PROM. There is no code that can
> upload new executables nor any way to execute them. There is no
> shell and no init.
>
> There is a startup program called 'init' for convenience, but
> whatever it does is hard-coded. It starts a bunch of predefined
> tasks and sleeps after performing some initialization.
>
> So, if the executable files are exactly like those which
> were reviewed during a security review, and if all
> communications between processes can be accounted for,
> including shared memory, then it can be guaranteed that
> whatever is being executed is exactly what passed the
> security review. However, if all communications channels
> cannot be accounted for, then it is not possible for such
> a guarantee.
>
> It can be argued that, once the executable code is known,
> there is no reason to review possible rogue communications
> channels. However, the customer is an expert in this field
> and requires an audit of all possible communications
> channels, including shared memory.
>
> Note that this is an audit. The code for every communications
> channel is reviewed. To verify that all such channels are
> audited, one needs to find them. For instance even 'vdso' is
> audited. These are not continuous audits that operate at
> run-time. The reason for this 'static' audit is so that
> CPU time eating checks don't have to be done at runtime.
>
>
> Cheers,
> Dick Johnson
> Penguin : Linux version 2.6.13.4 on an i686 machine (5589.54 BogoMips).
> Warning : 98.36% of all statistics are fiction.
> .
>
> ****************************************************************
> The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.
>
> Thank you.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Shared memory usage

[Hugh Dickins]

On Mon, 16 Jan 2006, Ram Gupta wrote:
> Did you think about getting shared memory information using
> shmctl(IPC_STAT). It provides useful information but I am not sure if
> that will serve your purpose fully.

/proc/sysvipc/shm would be easier (showing for all the shmids).

But yes, these won't go very far towards "auditing all possible
communications channels".  Nor would a "MemShared" number, nor
a total of "pages shared", whatever they mean.

Hugh