Stupid user with user-space questions, matrix LED driving with user space code only.

Stupid user with user-space questions, matrix LED driving with user space code only.

[Jonathan Andrews]

I hope this is the correct place, I expect to get abused.

I'm trying to do a mostly soft real-time task with a very small hard
real time element.

I've written some code to drive matrix LED signs using a Raspberry Pi.

Source here:
http://www.jonshouse.co.uk/download/128x32_red_green_led_sign.tar.gz


Since I last used linux you kernel people have fiddled with it yet
again:

Linux raspberrypi 3.6.11+ #375 PREEMPT Tue Feb 12 01:41:07 GMT 2013 armv6l GNU/Linux

I'm scanning an LED display to produce a 2 bits per pixel image.  The
code simply alters the amount of time any one LED is on, for higher
intensity pixels the true amount of "on" time is non critical.

I've marked my process as realtime.

My problem is that for very dim pixels the amount of "on" time for the
LED  is very critical, this is only a fraction of a percent of the total
processes timeslice.
It scans 100 frames of brightest, 100 frames of brighter and 1 frame of
dim pixels for example, so 200:1 ratio of don't care much /vs care a lot
timing !

To this end I'm using a hard coded small delay instead of usleep for the
tight timing section:

// Delay for ARM without yeilding to schedular, roughly calibrated but better than usleep for short delays
inline usleep_arm_hard(int usecs)
{
        long int outer,inner;

        for (outer=0;outer<usecs;outer++)
        {
                for (inner=0;inner<300;inner++)
                        asm("andeq r0, r0, r0");		// NOP
        }
}

The dim pixel code is timing critical (but as I said only a tiny
fraction of the total CPU usage). What I need to do is grab the CPU and
prevent any context switch (IRQ or PREEMPT) for this period.

I cant find a user space mechanism other than changing the kernel to
disable preemtion ? No simple /proc switch to turn it on/off ? If not
why - I cant be the only one who wants to toggle preemption off without
swapping kernels ?

The other issue is that of IRQs, my dim pixels on the display seem to
flash brighter from time to time, this I assume is partly preemption
(maybe possibly) and partly IRQ handling (more likely) allowing context
switches or just taking a while on slow hardware.

I need only a tiny fraction of the runtime to be hard real time, on
intel in the past i've simply disabled IRQs briefly with some inline
assembler.  
The Raspberry Pi board would also probably survive this as the only
active peripheral is ethenet, I suspect couple of missed IRQs would not
matter as once IRQs are re-enabled the USB/ethernet hardware will likely
have the data or it can be re-tried.  Does anyone have an example of a
dirty hack along these lines they can share with me :-) 

Do I have any simple mechanism available to disable (or defer) kernel
IRQ handling briefly from user space code, I suspect not but no harm in
asking ?

Thanks,
Jon

PS I'm not a kernel hacker - yes I know I could write a proper driver
for the task but I lack any real skill and the required time !

Re: Stupid user with user-space questions, matrix LED driving with user space code only.

[anish kumar]

On Sat, 2013-02-16 at 14:37 +0000, Jonathan Andrews wrote:
> I hope this is the correct place, I expect to get abused.
> 
> I'm trying to do a mostly soft real-time task with a very small hard
> real time element.
> 
> I've written some code to drive matrix LED signs using a Raspberry Pi.
> 
> Source here:
> http://www.jonshouse.co.uk/download/128x32_red_green_led_sign.tar.gz
> 
> 
> Since I last used linux you kernel people have fiddled with it yet
> again:
> 
> Linux raspberrypi 3.6.11+ #375 PREEMPT Tue Feb 12 01:41:07 GMT 2013 armv6l GNU/Linux
> 
> I'm scanning an LED display to produce a 2 bits per pixel image.  The
> code simply alters the amount of time any one LED is on, for higher
> intensity pixels the true amount of "on" time is non critical.
> 
> I've marked my process as realtime.
> 
> My problem is that for very dim pixels the amount of "on" time for the
> LED  is very critical, this is only a fraction of a percent of the total
> processes timeslice.
> It scans 100 frames of brightest, 100 frames of brighter and 1 frame of
> dim pixels for example, so 200:1 ratio of don't care much /vs care a lot
> timing !
> 
> To this end I'm using a hard coded small delay instead of usleep for the
> tight timing section:
> 
> // Delay for ARM without yeilding to schedular, roughly calibrated but better than usleep for short delays
> inline usleep_arm_hard(int usecs)
> {
>         long int outer,inner;
> 
>         for (outer=0;outer<usecs;outer++)
>         {
>                 for (inner=0;inner<300;inner++)
>                         asm("andeq r0, r0, r0");		// NOP
>         }
> }
> 
> The dim pixel code is timing critical (but as I said only a tiny
> fraction of the total CPU usage). What I need to do is grab the CPU and
> prevent any context switch (IRQ or PREEMPT) for this period.
why you want to do this?
> 
> I cant find a user space mechanism other than changing the kernel to
> disable preemtion ? No simple /proc switch to turn it on/off ? If not
> why - I cant be the only one who wants to toggle preemption off without
> swapping kernels ?
you can't disable pre-emption from user space.
> 
> The other issue is that of IRQs, my dim pixels on the display seem to
> flash brighter from time to time, this I assume is partly preemption
> (maybe possibly) and partly IRQ handling (more likely) allowing context
> switches or just taking a while on slow hardware.
> 
> I need only a tiny fraction of the runtime to be hard real time, on
> intel in the past i've simply disabled IRQs briefly with some inline
> assembler. 
you shouldn't fiddle with irq's from user space but...
> The Raspberry Pi board would also probably survive this as the only
> active peripheral is ethenet, I suspect couple of missed IRQs would not
> matter as once IRQs are re-enabled the USB/ethernet hardware will likely
> have the data or it can be re-tried.  Does anyone have an example of a
> dirty hack along these lines they can share with me :-) 

> Do I have any simple mechanism available to disable (or defer) kernel
> IRQ handling briefly from user space code, I suspect not but no harm in
> asking ?
Use any sysfs to disable/enable the irq. This approach is very bad but
as you said you wanted a hack.
> Thanks,
> Jon
> 
> PS I'm not a kernel hacker - yes I know I could write a proper driver
> for the task but I lack any real skill and the required time !
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Stupid user with user-space questions, matrix LED driving with user space code only.

[Jonathan Andrews]

> > The dim pixel code is timing critical (but as I said only a tiny
> > fraction of the total CPU usage). What I need to do is grab the CPU and
> > prevent any context switch (IRQ or PREEMPT) for this period.
> why you want to do this?
Because I need an I/O pin to change state for a short accurately timed
period. If the scheduler is activated during the generation of this
pulse then the timing will be stretched, this stretching is seen as
bright flash by the viewer of the LED display. 

>  
> > I cant find a user space mechanism other than changing the kernel to
> > disable preemtion ? No simple /proc switch to turn it on/off ? If not
> > why - I cant be the only one who wants to toggle preemption off without
> > swapping kernels ?
> you can't disable pre-emption from user space.
Part of why I asked this here.

Why not !

I would expect a "/proc/sys/kernel/sched_preemption" that I could toggle
a 1 or 0 into to turn it on/off.  

>From a user perspective it seems a bit crap to have to change the kernel
if you have a workload that preemption is harmful to.  
In the case of something like the Raspberry Pi changing the kernel if
the distribution has not done the work for me sounds like real effort.
The kernel is tied to binary obscurity from broadcom... To build I need
a working cross compiler, toolchain, kernel sources, Pi specific patches
then to get everything in the correct place on an SD card containing two
filesystems.  Its possible but its not going to "just work" at my skill
level....
> > The other issue is that of IRQs, my dim pixels on the display seem to
> > flash brighter from time to time, this I assume is partly preemption
> > (maybe possibly) and partly IRQ handling (more likely) allowing context
> > switches or just taking a while on slow hardware.
> > 
> > I need only a tiny fraction of the runtime to be hard real time, on
> > intel in the past i've simply disabled IRQs briefly with some inline
> > assembler. 
> you shouldn't fiddle with irq's from user space but...
> > The Raspberry Pi board would also probably survive this as the only
> > active peripheral is ethenet, I suspect couple of missed IRQs would not
> > matter as once IRQs are re-enabled the USB/ethernet hardware will likely
> > have the data or it can be re-tried.  Does anyone have an example of a
> > dirty hack along these lines they can share with me :-) 
> 
> > Do I have any simple mechanism available to disable (or defer) kernel
> > IRQ handling briefly from user space code, I suspect not but no harm in
> > asking ?
> Use any sysfs to disable/enable the irq. This approach is very bad but
> as you said you wanted a hack.
Sounds interesting, can I have some more specific clues how. Device is
not intel or PCI so I need to toggle something relevant to ARMs native
interrupt system, do I still have anything I can toggle that is relevant
in "Linux raspberrypi 3.6.11+ #375 PREEMPT" /sys ?

Many thanks,
Jon

Re: Stupid user with user-space questions, matrix LED driving with user space code only.

[anish kumar]

On Sun, 2013-02-17 at 14:37 +0000, Jonathan Andrews wrote:
> > > The dim pixel code is timing critical (but as I said only a tiny
> > > fraction of the total CPU usage). What I need to do is grab the CPU and
> > > prevent any context switch (IRQ or PREEMPT) for this period.
> > why you want to do this?
> Because I need an I/O pin to change state for a short accurately timed
> period. If the scheduler is activated during the generation of this
> pulse then the timing will be stretched, this stretching is seen as
> bright flash by the viewer of the LED display. 
You can do that in kernel.
> 
> >  
> > > I cant find a user space mechanism other than changing the kernel to
> > > disable preemtion ? No simple /proc switch to turn it on/off ? If not
> > > why - I cant be the only one who wants to toggle preemption off without
> > > swapping kernels ?
> > you can't disable pre-emption from user space.
> Part of why I asked this here.
> 
> Why not !
> 
> I would expect a "/proc/sys/kernel/sched_preemption" that I could toggle
> a 1 or 0 into to turn it on/off.  
Do it in kernel and that is why drivers exist in the kernel.
> 
> >From a user perspective it seems a bit crap to have to change the kernel
> if you have a workload that preemption is harmful to.  
> In the case of something like the Raspberry Pi changing the kernel if
> the distribution has not done the work for me sounds like real effort.
> The kernel is tied to binary obscurity from broadcom... To build I need
> a working cross compiler, toolchain, kernel sources, Pi specific patches
> then to get everything in the correct place on an SD card containing two
> filesystems.  Its possible but its not going to "just work" at my skill
> level....
AFAIK raspberry pi kernel is very easy to build and there are already
source code out there which you just need to clone and compile.This
includes all the necessary tools so...just try and fail before
deciding!!
> > > The other issue is that of IRQs, my dim pixels on the display seem to
> > > flash brighter from time to time, this I assume is partly preemption
> > > (maybe possibly) and partly IRQ handling (more likely) allowing context
> > > switches or just taking a while on slow hardware.
> > > 
> > > I need only a tiny fraction of the runtime to be hard real time, on
> > > intel in the past i've simply disabled IRQs briefly with some inline
> > > assembler. 
> > you shouldn't fiddle with irq's from user space but...
> > > The Raspberry Pi board would also probably survive this as the only
> > > active peripheral is ethenet, I suspect couple of missed IRQs would not
> > > matter as once IRQs are re-enabled the USB/ethernet hardware will likely
> > > have the data or it can be re-tried.  Does anyone have an example of a
> > > dirty hack along these lines they can share with me :-) 
> > 
> > > Do I have any simple mechanism available to disable (or defer) kernel
> > > IRQ handling briefly from user space code, I suspect not but no harm in
> > > asking ?
> > Use any sysfs to disable/enable the irq. This approach is very bad but
> > as you said you wanted a hack.
> Sounds interesting, can I have some more specific clues how. Device is
> not intel or PCI so I need to toggle something relevant to ARMs native
> interrupt system, do I still have anything I can toggle that is relevant
> in "Linux raspberrypi 3.6.11+ #375 PREEMPT" /sys ?
So try building the raspberry kernel.
> 
> Many thanks,
> Jon
> 
> 

Re: Stupid user with user-space questions, matrix LED driving with user space code only.

[Greg KH]

On Sun, Feb 17, 2013 at 02:37:24PM +0000, Jonathan Andrews wrote:
> From a user perspective it seems a bit crap to have to change the kernel
> if you have a workload that preemption is harmful to.  
> In the case of something like the Raspberry Pi changing the kernel if
> the distribution has not done the work for me sounds like real effort.
> The kernel is tied to binary obscurity from broadcom... To build I need
> a working cross compiler, toolchain, kernel sources, Pi specific patches
> then to get everything in the correct place on an SD card containing two
> filesystems.  Its possible but its not going to "just work" at my skill
> level....

As you can not boot a kernel.org kernel on the RPI platform just yet,
there's very little that the kernel.org community can do here to help
you out.  I suggest you go take this up with the developers whom you got
this specific kernel build from, there's nothing we can do here about
it.

Best of luck,

greg k-h

Re: Stupid user with user-space questions, matrix LED driving with user space code only.

[Jonathan Andrews]

On Sun, 2013-02-17 at 18:05 -0800, Greg KH wrote: 
> On Sun, Feb 17, 2013 at 02:37:24PM +0000, Jonathan Andrews wrote:
> > From a user perspective it seems a bit crap to have to change the kernel
> > if you have a workload that preemption is harmful to.  
> > In the case of something like the Raspberry Pi changing the kernel if
> > the distribution has not done the work for me sounds like real effort.
> > The kernel is tied to binary obscurity from broadcom... To build I need
> > a working cross compiler, toolchain, kernel sources, Pi specific patches
> > then to get everything in the correct place on an SD card containing two
> > filesystems.  Its possible but its not going to "just work" at my skill
> > level....
> 
> As you can not boot a kernel.org kernel on the RPI platform just yet,
> there's very little that the kernel.org community can do here to help
> you out.
Somebody could stick in the code to enable/disable preemption at runtime
on PREEMPT compiled kernels :-) - Then all I have to do is wait for it
to filter down to the Raspian kernel maintainers, i'm patient ;-) ?   

In all seriousness I assume preemption has a timer and an interrupt
vector, cant the timer simply be enabled/disabled leaving just the
kernel call mechanism untouched. IE a "preemption" kernel that is now
not preempting ...... OK, OK - I have many other egg sucking suggestions
but am I the only one who wants the functionality ?

Seems a half step to add all this wizzy real-time code to the kernel
then stop users doing real-time in user space by allowing the schedular
to yield during what a user wishes to be a citical section (from a
timing point of view not the atomic point of view).

What about a yield alignment mechanism for user space. IE the process
calls the kernel with a request "schedule me first after a yeild" - then
the process at least has whatever the timer granularity is to do
something timing critical... add a flag to ignore or defer interrupts
and you have a semi 'hard-realtime' behaviour for user space, allowing
user space to grab small chunks of real time. Yes a nasty looking
facility for SMP intel servers but really useful for embedded.

>   I suggest you go take this up with the developers whom you got
> this specific kernel build from, there's nothing we can do here about
> it.
I suspected it was not going to be simple. As I suspect my feature
request is not that simple but if you don't ask........

Thanks,
Jon

Re: Stupid user with user-space questions, matrix LED driving with user space code only.

[el es]

Jonathan Andrews <jon <at> jonshouse.co.uk> writes:


> 
> What about a yield alignment mechanism for user space. IE the process
> calls the kernel with a request "schedule me first after a yeild" - then
> the process at least has whatever the timer granularity is to do
> something timing critical... add a flag to ignore or defer interrupts
> and you have a semi 'hard-realtime' behaviour for user space, allowing
> user space to grab small chunks of real time. Yes a nasty looking
> facility for SMP intel servers but really useful for embedded.
> 

Seems you have some (bad?) habits from embedded programming, 
you think Linux is FreeRTOS ;)

Linux as such, as far as I read, is not a real-time OS, it will 
NOT do what you want in userspace, (maybe unless you build it with
the RT patchset?)

Better take the advice and go build a kernel driver for this display.

Or use a small microcontroller that won't have the limitations. 

> Thanks,
> Jon
> 
> 
Lukasz