[PATCH]: uClinux (MMU-less) patches against 2.5.28

[PATCH]: uClinux (MMU-less) patches against 2.5.28

[Greg Ungerer]

Hi All,

A new set of uClinux (MMU-less) patches agains 2.5.28.
You can get it at:

http://www.uclinux.org/pub/uClinux/uClinux-2.5.x/linux-2.5.28uc0.patch.gz

Some amount of hacking was required to support the underlying
interrupt/cli/sti changes. But it seems to work pretty well
on my test boards.

Regards
Greg


------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Wizard        EMAIL:  gerg@snapgear.com
SnapGear Pty Ltd                               PHONE:    +61 7 3435 2888
825 Stanley St,                                  FAX:    +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia              WEB:   www.snapgear.com

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[David Woodhouse]

gerg@snapgear.com said:
>  A new set of uClinux (MMU-less) patches agains 2.5.28. You can get it
> at:

> http://www.uclinux.org/pub/uClinux/uClinux-2.5.x/linux-2.5.28uc0.patch.gz 

Perhaps drop drivers/block/blkmem.c or justify reinventing the wheel?

--
dwmw2

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[Greg Ungerer]

Hi David,

David Woodhouse wrote:
> gerg@snapgear.com said:
> 
>> A new set of uClinux (MMU-less) patches agains 2.5.28. You can get it
>>at:
> 
>>http://www.uclinux.org/pub/uClinux/uClinux-2.5.x/linux-2.5.28uc0.patch.gz 
> 
> 
> Perhaps drop drivers/block/blkmem.c or justify reinventing the wheel?

Indeed. That blkmem driver is a complete mess.
I can't think of any justification for it existing :-)
I'll work on cleaning that out.

Work is on going to remove the remaining bogosity we are
still carrying in this patch.

Regards
Greg



------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Wizard        EMAIL:  gerg@snapgear.com
SnapGear Pty Ltd                               PHONE:    +61 7 3435 2888
825 Stanley St,                                  FAX:    +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia              WEB:   www.snapgear.com

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[Greg Ungerer]

Hi David,

 >David Woodhouse wrote:
 > > gerg@snapgear.com said:
 > >
 > > A new set of uClinux (MMU-less) patches agains 2.5.28. You can get 
it at:
 > >
 > > 
http://www.uclinux.org/pub/uClinux/uClinux-2.5.x/linux-2.5.28uc0.patch.gz
 > >
 > >
 > > Perhaps drop drivers/block/blkmem.c or justify reinventing the wheel?
 >
 > Indeed. That blkmem driver is a complete mess.
 > I can't think of any justification for it existing :-)
 > I'll work on cleaning that out.

You may have noticed some MAGIC_ROM_PTR patches in the mtd
driver code in this patch. This is to allow XIP for applications.
We have been support XIP for quite some time on uClinux, it
works really well.

A problem we have experienced with MTD is that the nature of
asynchronous writing to FLASH does not play nice with apps
runing XIP.  Any thoughts on how to deal with this?

(blkmem does synchronous writes so it doesn't suffer any
problem. It just locks everyone out for an in-ordinate amount
of time while it does the FLASH write).

Regards
Greg



------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Wizard        EMAIL:  gerg@snapgear.com
SnapGear Pty Ltd                               PHONE:    +61 7 3435 2888
825 Stanley St,                                  FAX:    +61 7 3891 3630
Woolloongabba, QLD, 4102, Australia              WEB:   www.snapgear.com

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[David Woodhouse]

gerg@snapgear.com said:
>  You may have noticed some MAGIC_ROM_PTR patches in the mtd driver
> code in this patch. This is to allow XIP for applications. We have
> been support XIP for quite some time on uClinux, it works really well.

> A problem we have experienced with MTD is that the nature of
> asynchronous writing to FLASH does not play nice with apps runing XIP.
>  Any thoughts on how to deal with this?

Other than burying my head in the sand and hoping that someone starts 
making dual-port flash? Sort of...

XIP of the kernel makes it harder, of course -- writing to the chip you're
running the _kernel_ from turns your entire text segment into status words,
so really I can't see any alternative but to copy the flash manipulation
routines into RAM, disable all interrupts, and get on with it. 

If we're talking about XIP of just userspace pages, I have a vague plan for 
that which may be slightly more feasible. If it holds up to the cold light 
of day, perhaps you can help adjust it to work with uClinux.

The plan is to hand out pages to be mapped into userspace (or indeed kernel 
space -- JFFS2 can speed up mounts this way too), but which get marked 
absent when the flash driver talks to the chip. If your applications then 
take a page fault, you can either suspend the operation or just make them 
wait till it's complete.

The actual implementation of this should be relatively simple. We can 
provide a generic_mtd_vm_ops and the chip drivers just need to know about 
an extra state for 'mapped' and have a method to enter that state, and a 
callback for when they want to _leave_ that state. Oh, and a way to get the 
raw physical address for a given range of flash.

It occurs to me that this doesn't work too well without an MMU though. Got 
any better ideas? Could we disable entire processes when one of their pages 
is inaccessible?

--
dwmw2

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[David Woodhouse]

alan@lxorguk.ukuu.org.uk said:
>  It doesn't work with an MMU either. Consider O_DIRECT file writing
> from such a page

Hmmm. 

O_DIRECT writing of data from an XIP-mapped page to a file on the same chip 
ain't ever going to work. Writing to something elsewhere should be fine -- 
if a page is locked when the chip driver wants to go talk to the chip, the 
chip driver has to wait for that page to become unlocked. No?

--
dwmw2

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[David Woodhouse]

alan@lxorguk.ukuu.org.uk said:
>  Now consider two events each locking the page the other wishes to
> write to. At that point you may want to stop digging any deeper holes 

Maybe, but only if we can come up with a better alternative. 

Bear in mind that XIP is not for general-purpose use, and strict 
restrictions on what you can do with it are not unreasonable. Assume we 
remove O_DIRECT support entirely from any kernel which supports XIP. Then 
what breaks?

--
dwmw2

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[Alan Cox]

> It occurs to me that this doesn't work too well without an MMU though. Got 
> any better ideas? Could we disable entire processes when one of their pages 
> is inaccessible?

It doesn't work with an MMU either. Consider O_DIRECT file writing from
such a page

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[Alan Cox]

On Fri, 2002-07-26 at 10:39, David Woodhouse wrote:
> 
> alan@lxorguk.ukuu.org.uk said:
> >  It doesn't work with an MMU either. Consider O_DIRECT file writing
> > from such a page
> 
> Hmmm. 
> 
> O_DIRECT writing of data from an XIP-mapped page to a file on the same chip 
> ain't ever going to work. Writing to something elsewhere should be fine -- 
> if a page is locked when the chip driver wants to go talk to the chip, the 
> chip driver has to wait for that page to become unlocked. No?

Now consider two events each locking the page the other wishes to write
to. At that point you may want to stop digging any deeper holes

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[David Woodhouse]

alan@lxorguk.ukuu.org.uk said:
>  Anything where you rely on locking the pages and can get a loop of
> locked/absent pages and deadlock

Good point -- so if a writer encounters a locked page when it's trying to 
unmap them all, it needs to still allow other pages to be mapped in while 
it waits for the original page to become unlocked. That avoids the deadlock 
-- but leaves us with the potential for livelock, with the writer being 
starved by too many other things locking down the pages in question.

I suspect the number of pages getting locked will be sufficiently small 
that the deadlock does not occur. 

In what circumstances will such mmapped pages get locked?

The case of O_DIRECT or rawio writes to flash is easily avoided -- just 
don't allow raw or O_DIRECT writes to flash from _anywhere_ while XIP is in 
use. O_DIRECT and rawio to other destinations is fine. What else? Assume 
we don't permit mlock() :)

--
dwmw2

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[Alan Cox]

On Fri, 2002-07-26 at 10:58, David Woodhouse wrote:
> 
> alan@lxorguk.ukuu.org.uk said:
> >  Now consider two events each locking the page the other wishes to
> > write to. At that point you may want to stop digging any deeper holes 
> 
> Maybe, but only if we can come up with a better alternative. 
> 
> Bear in mind that XIP is not for general-purpose use, and strict 
> restrictions on what you can do with it are not unreasonable. Assume we 
> remove O_DIRECT support entirely from any kernel which supports XIP. Then 
> what breaks?

Anything where you rely on locking the pages and can get a loop of
locked/absent pages and deadlock

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[gerg]

David Woodhouse wrote:
> gerg@snapgear.com said:
>> You may have noticed some MAGIC_ROM_PTR patches in the mtd driver
>>code in this patch. This is to allow XIP for applications. We have
>>been support XIP for quite some time on uClinux, it works really well.
> 
>>A problem we have experienced with MTD is that the nature of
>>asynchronous writing to FLASH does not play nice with apps runing XIP.
>> Any thoughts on how to deal with this?
> 
> Other than burying my head in the sand and hoping that someone starts 
> making dual-port flash? Sort of...

:-)


> XIP of the kernel makes it harder, of course -- writing to the chip you're
> running the _kernel_ from turns your entire text segment into status words,
> so really I can't see any alternative but to copy the flash manipulation
> routines into RAM, disable all interrupts, and get on with it. 

Not at all unresonable. I don't think there is any other real
alternative if you want your kernel running from flash, and
you want to write to flash.


> If we're talking about XIP of just userspace pages, I have a vague plan for 
> that which may be slightly more feasible. If it holds up to the cold light 
> of day, perhaps you can help adjust it to work with uClinux.
> 
> The plan is to hand out pages to be mapped into userspace (or indeed kernel 
> space -- JFFS2 can speed up mounts this way too), but which get marked 
> absent when the flash driver talks to the chip. If your applications then 
> take a page fault, you can either suspend the operation or just make them 
> wait till it's complete.
> 
> The actual implementation of this should be relatively simple. We can 
> provide a generic_mtd_vm_ops and the chip drivers just need to know about 
> an extra state for 'mapped' and have a method to enter that state, and a 
> callback for when they want to _leave_ that state. Oh, and a way to get the 
> raw physical address for a given range of flash.

The MAGIC_ROM_PTR support in the uClinux patch adds a field to
the block_device_operations and file_operations structures that
allows getting at the physical address in flash.



> It occurs to me that this doesn't work too well without an MMU though. Got 
> any better ideas? Could we disable entire processes when one of their pages 
> is inaccessible?

Disabling processes that are known to be running direct from flash
sounds workable. (There is no real notion of separating pages under
uClinux - it is an all or nothing mapping. The text, data, bss, etc
are always a single contiguous region in the address space).

More generous lock that really required for general VM linux, but at
least the whole process model works for both VM and non-VM linux.
I would expect this avoids any potential loop/deadlock with pages
(going on the discussion in follow up emails anyway).

Regards
Greg


------------------------------------------------------------------------
Greg Ungerer  --  Chief Software Wizard        EMAIL:  gerg@snapgear.com
Snapgear Pty Ltd                               PHONE:    +61 7 3279 1822
825 Stanley St,                                  FAX:    +61 7 3279 1820
Woolloongabba, QLD, 4102, Australia              WEB:   www.snapgear.com

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[David Woodhouse]

gerg@snapgear.com said:
>  The MAGIC_ROM_PTR support in the uClinux patch adds a field to the
> block_device_operations and file_operations structures that allows
> getting at the physical address in flash.

Sick. Just provide your own mmap() instead. 

>  Disabling processes that are known to be running direct from flash
> sounds workable. (There is no real notion of separating pages under
> uClinux - it is an all or nothing mapping. The text, data, bss, etc
> are always a single contiguous region in the address space).

Yep. For uClinux we could probably get away with that. I don't want to
suggest it for normal Linux though.

> More generous lock that really required for general VM linux, but at
> least the whole process model works for both VM and non-VM linux. I
> would expect this avoids any potential loop/deadlock with pages (going
> on the discussion in follow up emails anyway).

I don't see that many cases where these pages would get locked; I'm not 
convinced it's a problem. But I'm aware of the percentages from the 
previous times I've argued with Alan :)

--
dwmw2

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[David Woodhouse]

alan@lxorguk.ukuu.org.uk said:
>  Well for a starter lets try user writing a block to flash which needs
> an erase from a block which happens to be not yet paged in and so
> still on the flash you are writing to.

What locks the page? Surely our write() implementation is just using 
copy_from_user(), to get the data from the user, and each erase and write 
to the flash chip can be considered atomic -- what's holding a lock, and 
what's causing the deadlock?

--
dwmw2

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[David Woodhouse]

>  Also a page can be locked because another I/O read is pending on it
> (ie its already in someone elses read/page in handling and currently
> has invalid data). 

If it has invalid data, then it's not an XIP page -- it's in RAM. XIP pages 
are either present or not present because the chip is otherwise occupied -- 
in neither case do they need to be locked for I/O. 

> You would need to be sure we never ended up with deadlocks for any case 
> where we have  
> 	process 1 copying page X from flash 2 to flash 1 page Y
> 	process 2 copying page Y from flash 1 to flash 2 page X
> With jffs2 and temporary buffering I guess it works out.  

Buffering is required for other reasons, and it's what saves us here. The 
thing that'll do the 'write lock' on the flash is the raw flash write 
operation, and that can't take a userspace pointer, it takes a kernel 
pointer. So we were buffering anyway (yes we could have locked it down and 
used it directly but we don't). 

So you get process 1 doing copy_from_user which reads from flash 2, then
passing the buffer into the write function for flash 1. And process 2 doing 
the converse. Each drops the read lock before getting a write lock -- in 
fact the page was never _locked_, it could be paged back out again to allow 
writes to happen between each byte that copy_from_user() fetched.

--
dwmw2

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[Alan Cox]

On Fri, 2002-07-26 at 15:03, David Woodhouse wrote:
> Good point -- so if a writer encounters a locked page when it's trying to 
> unmap them all, it needs to still allow other pages to be mapped in while 
> it waits for the original page to become unlocked. That avoids the deadlock 
> -- but leaves us with the potential for livelock, with the writer being 
> starved by too many other things locking down the pages in question.
> 
> I suspect the number of pages getting locked will be sufficiently small 
> that the deadlock does not occur. 

Well for a starter lets try user writing a block to flash which needs an
erase from a block which happens to be not yet paged in and so still on
the flash you are writing to.

Re: [PATCH]: uClinux (MMU-less) patches against 2.5.28

[Alan Cox]

On Fri, 2002-07-26 at 17:01, David Woodhouse wrote:
> What locks the page? Surely our write() implementation is just using 
> copy_from_user(), to get the data from the user, and each erase and write 
> to the flash chip can be considered atomic -- what's holding a lock, and 
> what's causing the deadlock?

That is going to depend on the exact ordering that is done. You would
need to copy the user data into a temporary memory buffer (possibly
paging) then lock the flash write and restore.

Also a page can be locked because another I/O read is pending on it (ie
its already in someone elses read/page in handling and currently has
invalid data). You would need to be sure we never ended up with
deadlocks for any case where we have 

	process 1 copying page X from flash 2 to flash 1 page Y
	process 2 copying page Y from flash 1 to flash 2 page X

With jffs2 and temporary buffering I guess it works out.