RE: firmware separation filesystem (fwfs)

RE: firmware separation filesystem (fwfs)

[Perez-Gonzalez, Inaky]

> From: David Gibson [mailto:david@gibson.dropbear.id.au]
>
> My personal feeling is that this would probably make more sense as a
> type of sysfs node, rather than a separate filesystem, but the basic
> concept seems sound.

Would not the new binary interface to sysfs help to sort 
this out?

Iñaky Pérez-González -- Not speaking for Intel -- all opinions are my own
(and my fault)

Re: firmware separation filesystem (fwfs)

[Manuel Estrada Sainz]

On Sat, Apr 19, 2003 at 10:52:44PM +0100, Alan Cox wrote:
> On Sad, 2003-04-19 at 21:41, Manuel Estrada Sainz wrote:
> > > fwfs is a broken idea because it leaves the data in kernel space. On
> > > a giant IBM monster maybe nobody cares about a few hundred K of cached
> > > firmware in the kernel, but the rest of us happen to run real world
> > > computers.
> > 
> >  Many drivers currently include this same data in kernel space, in in
> >  headers, what I am trying to do is make it easy for them to support
> >  fwfs (or whatever it becomes in the end). 
> 
> So what is the value in changing them to this, then changing them again
> to put the firmware in userspace ? Surely you'd be better off writing
> a generally usable request_firmware() hotplug interface ?

  Sorry, I modify plans in my head with the feedback, but am probably
  not telling my new plans very well as I go along.

  My current plan is actualy making a request_firmware() hotplug
  interface. What I am still not sure about is if it should be build
  around some evolution of fwfs, sysfs binary support or something even
  simpler. In any case, I'll try to make the same interface, so even if
  one is taken and later found to be wrong, drivers won't have to be
  rewritten.
 
  I plan to implement at least two alternatives, the one around fwfs and
  the one around sysfs, so we can compare the merits and limitations of
  each looking at working code. If there is interest in an even simpler
  alternative or the other two are found to be wrong I'll do it.

  Specially if you (Alan) don't like the sysfs alternative either, I'll
  try to make the third alternative based on your feedback.

  I am really more interested in finding a good way to handle firmware
  than in pushing any actual implementation in special.

  Have a nice day

  	Manuel

  PS: I'll be away most of the week, I'll respond to any new feedback
  and get back to coding when I come back.
-- 
--- Manuel Estrada Sainz <ranty@debian.org>
                         <ranty@bigfoot.com>
			 <ranty@users.sourceforge.net>
------------------------ <manuel.estrada@hispalinux.es> -------------------
Let us have the serenity to accept the things we cannot change, courage to
change the things we can, and wisdom to know the difference.

Re: firmware separation filesystem (fwfs)

[Alan Cox]

On Sad, 2003-04-19 at 21:41, Manuel Estrada Sainz wrote:
> > fwfs is a broken idea because it leaves the data in kernel space. On
> > a giant IBM monster maybe nobody cares about a few hundred K of cached
> > firmware in the kernel, but the rest of us happen to run real world
> > computers.
> 
>  Many drivers currently include this same data in kernel space, in in
>  headers, what I am trying to do is make it easy for them to support
>  fwfs (or whatever it becomes in the end). 

So what is the value in changing them to this, then changing them again
to put the firmware in userspace ? Surely you'd be better off writing
a generally usable request_firmware() hotplug interface ?

Re: firmware separation filesystem (fwfs)

[Manuel Estrada Sainz]

On Thu, Apr 17, 2003 at 02:12:03PM +0100, Alan Cox wrote:
> On Iau, 2003-04-17 at 02:23, David Gibson wrote:
> > > But so would loading it from hotplug via ioctl. It might be we want
> > > a clean hotplug way to ask for 'firmare for xyz'.
> > 
> > True, but ioctl()s are horrid.  And the driver needs to set up a
> > suitable device to which the ioctl() is applied, and deal with binding
> > the right image to the right instance, which can get messy in some
> 
> You are ignoring the main issue of discussion. I don't care if its
> ioctl, tcp/ip over carrier pigeon or a pipe.
> 
> fwfs is a broken idea because it leaves the data in kernel space. On
> a giant IBM monster maybe nobody cares about a few hundred K of cached
> firmware in the kernel, but the rest of us happen to run real world
> computers.

 Many drivers currently include this same data in kernel space, in in
 headers, what I am trying to do is make it easy for them to support
 fwfs (or whatever it becomes in the end). This way, they may be able to
 support it with little effort (which increases the chances of them
 actualy supporting it) and people worried about memory usage can easly
 free up that memory with a simple unlink.


> Catting the firmware to a device node also works fine for me as an 
> API, but keep the firmware in userspace.

 When sysfs binary support is integrated I'll try to do something on top
 of it, and in any case, I'll at the minimum make sure that in-kernel
 data is optional.

 Have a nice day

 	Manuel
 
-- 
--- Manuel Estrada Sainz <ranty@debian.org>
                         <ranty@bigfoot.com>
			 <ranty@users.sourceforge.net>
------------------------ <manuel.estrada@hispalinux.es> -------------------
Let us have the serenity to accept the things we cannot change, courage to
change the things we can, and wisdom to know the difference.

RE: firmware separation filesystem (fwfs)

[Alan Cox]

On Mer, 2003-04-16 at 17:57, Riley Williams wrote:
> I know that PCI uses a 32-bit number (or two 16-bit numbers if one
> prefers to think of it that way) to identify each piece of equipment.
> Is there a similar number for USB, Firewire, etc?

Yes but these often don't tell you the firmware you need. The firmware
querying is often rather more device specific .

RE: firmware separation filesystem (fwfs)

[Alan Cox]

On Mer, 2003-04-16 at 17:57, Riley Williams wrote:
> I know that PCI uses a 32-bit number (or two 16-bit numbers if one
> prefers to think of it that way) to identify each piece of equipment.
> Is there a similar number for USB, Firewire, etc?

Yes but these often don't tell you the firmware you need. The firmware
querying is often rather more device specific .

Re: firmware separation filesystem (fwfs)

[Alan Cox]

On Iau, 2003-04-17 at 02:23, David Gibson wrote:
> > But so would loading it from hotplug via ioctl. It might be we want
> > a clean hotplug way to ask for 'firmare for xyz'.
> 
> True, but ioctl()s are horrid.  And the driver needs to set up a
> suitable device to which the ioctl() is applied, and deal with binding
> the right image to the right instance, which can get messy in some

You are ignoring the main issue of discussion. I don't care if its
ioctl, tcp/ip over carrier pigeon or a pipe.

fwfs is a broken idea because it leaves the data in kernel space. On
a giant IBM monster maybe nobody cares about a few hundred K of cached
firmware in the kernel, but the rest of us happen to run real world
computers.

Catting the firmware to a device node also works fine for me as an 
API, but keep the firmware in userspace.



Alan

RE: firmware separation filesystem (fwfs)

[Perez-Gonzalez, Inaky]

> From: Manuel Estrada Sainz [mailto:ranty@debian.org]
>
> > With the risk of repeating myself (again) and being a PITA,
> > I really think it'd be easier to copy the firmware file to a
> > /sysfs binary file registered by the device driver during
> > initialization; then the driver can wait for the file to be
> > written with a valid firmware before finishing the init
> > sequence. The infrastructure is already there (or isn't ...
> > is it?).
> 
>  I don't know that much about sysfs, after a little investigation, it
>  seams like sysfs entries are restricted in size to PAGE_SIZE, which on
>  i386 is 4K, and ezusb firmware is already 6.9K in size.

You are right, at least that in 2.5.66 (the only tree I have handy now),
it is still limited to PAGE_SIZE; however, there were some
changes recently to the bin file interface, so it might have
been removed.

But this thing (firmware uploading) seems like a good reason
to remove that limit. 

If you [or somebody else :)] did the necessary modifications 
to fs/sysfs/bin.c and submit them to Patrick Mochel, with 
the reasoning on why and usage, I'd say he would mostly
accept them - it does not seem to be too hard.

Iñaky Pérez-González -- Not speaking for Intel -- all opinions are my own
(and my fault)

Re: firmware separation filesystem (fwfs)

[Greg KH]

On Thu, Apr 17, 2003 at 05:31:54AM +0200, Manuel Estrada Sainz wrote:
> 
>  I don't know that much about sysfs, after a little investigation, it
>  seams like sysfs entries are restricted in size to PAGE_SIZE, which on
>  i386 is 4K, and ezusb firmware is already 6.9K in size.
> 
>  I would really appreciate someone more knowledgeable than myself
>  commenting on the possibility of extending sysfs to fill this gap.

Yes, I don't think this would be that tough to add.  Matthew Wilcox
posted a patch to clean up a lot of problems with the current binary
file sysfs interface, and after that patch is in the main tree, it
shouldn't be that touch to change to support these kinds of files.

Although I do like the idea and implementation of fwfs, nice job :)

Hope this helps,

greg k-h

Re: firmware separation filesystem (fwfs)

['David Gibson']

On Wed, Apr 16, 2003 at 07:00:00PM -0700, Perez-Gonzalez, Inaky wrote:
> 
> > From: David Gibson [mailto:david@gibson.dropbear.id.au]
> > 
> > Incidentally another approach that also avoids nasty ioctl()s would be
> > to invoke the userland helper with specially set up FD 1, which lets
> > the kernel capture the program's stdout. 
> 
> I think this makes too many assumptions specially taking into
> account that most hotplug stuff are shell scripts - they are
> probably going to be writing all kinds of stuff to stdout.
> 
> With the risk of repeating myself (again) and being a PITA,
> I really think it'd be easier to copy the firmware file to a 
> /sysfs binary file registered by the device driver during 
> initialization; then the driver can wait for the file to be
> written with a valid firmware before finishing the init
> sequence. The infrastructure is already there (or isn't ... 
> is it?).

Well, I guess that would be basically what I mean by an equivalent
sysfs thing.  I haven't looked at the binary file support in sysfs, as
yet.

-- 
David Gibson			| For every complex problem there is a
david@gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.
http://www.ozlabs.org/people/dgibson

Re: firmware separation filesystem (fwfs)

[Manuel Estrada Sainz]

On Wed, Apr 16, 2003 at 07:00:00PM -0700, Perez-Gonzalez, Inaky wrote:
> 
> > From: David Gibson [mailto:david@gibson.dropbear.id.au]
> > 
> > Incidentally another approach that also avoids nasty ioctl()s would be
> > to invoke the userland helper with specially set up FD 1, which lets
> > the kernel capture the program's stdout. 
> 
> I think this makes too many assumptions specially taking into
> account that most hotplug stuff are shell scripts - they are
> probably going to be writing all kinds of stuff to stdout.

 Well, FD 3 could be used instead and "cat firmware_image >&3". shell
 scripts should not be writing to FD 3.
 
> With the risk of repeating myself (again) and being a PITA,
> I really think it'd be easier to copy the firmware file to a 
> /sysfs binary file registered by the device driver during 
> initialization; then the driver can wait for the file to be
> written with a valid firmware before finishing the init
> sequence. The infrastructure is already there (or isn't ... 
> is it?).

 I don't know that much about sysfs, after a little investigation, it
 seams like sysfs entries are restricted in size to PAGE_SIZE, which on
 i386 is 4K, and ezusb firmware is already 6.9K in size.

 I would really appreciate someone more knowledgeable than myself
 commenting on the possibility of extending sysfs to fill this gap.

 Have a nice day

 	ranty

-- 
--- Manuel Estrada Sainz <ranty@debian.org>
                         <ranty@bigfoot.com>
			 <ranty@users.sourceforge.net>
------------------------ <manuel.estrada@hispalinux.es> -------------------
Let us have the serenity to accept the things we cannot change, courage to
change the things we can, and wisdom to know the difference.

RE: firmware separation filesystem (fwfs)

[Perez-Gonzalez, Inaky]

> From: David Gibson [mailto:david@gibson.dropbear.id.au]
> 
> Incidentally another approach that also avoids nasty ioctl()s would be
> to invoke the userland helper with specially set up FD 1, which lets
> the kernel capture the program's stdout. 

I think this makes too many assumptions specially taking into
account that most hotplug stuff are shell scripts - they are
probably going to be writing all kinds of stuff to stdout.

With the risk of repeating myself (again) and being a PITA,
I really think it'd be easier to copy the firmware file to a 
/sysfs binary file registered by the device driver during 
initialization; then the driver can wait for the file to be
written with a valid firmware before finishing the init
sequence. The infrastructure is already there (or isn't ... 
is it?).

CU,

Iñaky Pérez-González -- Not speaking for Intel -- all opinions are my own
(and my fault)

Re: firmware separation filesystem (fwfs)

[David Gibson]

On Wed, Apr 16, 2003 at 04:47:09PM +0100, Alan Cox wrote:
> On Mer, 2003-04-16 at 17:36, Manuel Estrada Sainz wrote:
> >  On the other hand, there are already many drivers in the kernel that
> >  include firmware in headers, keyspan, io_edgeport, dabusb, ser_a2232,
> >  sym53c8xx_2, ...
> 
> But so would loading it from hotplug via ioctl. It might be we want
> a clean hotplug way to ask for 'firmare for xyz'.

True, but ioctl()s are horrid.  And the driver needs to set up a
suitable device to which the ioctl() is applied, and deal with binding
the right image to the right instance, which can get messy in some
cases.  As I see it the chief purpose of fwfs, or an equivalent sysfs
formulation would be to provide a clean mechanism for passing firmware
images to the kernel - figuring out which image to provide and
supplying it at the relevant moment can and should be the job of
hotplug or a similar userland helper.

Incidentally another approach that also avoids nasty ioctl()s would be
to invoke the userland helper with specially set up FD 1, which lets
the kernel capture the program's stdout.  When the driver needs a
firmware it invokes the helper, which figures out the relevant image
from its parameters and cats it.  The driver (presumably sleeping
waiting for this) grabs the image, stuffs it into the hardware, then
throws it away.

-- 
David Gibson			| For every complex problem there is a
david@gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.
http://www.ozlabs.org/people/dgibson

RE: firmware separation filesystem (fwfs)

[Riley Williams]

Hi Alan.

 >> On the other hand, there are already many drivers in the kernel
 >> that include firmware in headers, keyspan, io_edgeport, dabusb,
 >> ser_a2232, sym53c8xx_2, ...

 > But so would loading it from hotplug via ioctl. It might be we
 > want a clean hotplug way to ask for 'firmware for xyz'.

I know that PCI uses a 32-bit number (or two 16-bit numbers if one
prefers to think of it that way) to identify each piece of equipment.
Is there a similar number for USB, Firewire, etc?

If so, the hotplug firmware driver could use those numbers prefixed
by a code for the relevant bus to identify the relevant firmware.
For example...

	P:1234:5678		PCI Bus code 1234:5678
	U:1234:5678		USB Bus code 1234:5678

...and the hotplug driver would know whether those two were the same
device on different buses, or two different devices.

Best wishes from Riley.
---
 * Nothing as pretty as a smile, nothing as ugly as a frown.

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.471 / Virus Database: 269 - Release Date: 10-Apr-2003

Re: firmware separation filesystem (fwfs)

[Manuel Estrada Sainz]

On Thu, Apr 17, 2003 at 12:46:31AM +1000, David Gibson wrote:
> On Wed, Apr 16, 2003 at 12:31:21PM +0100, Alan Cox wrote:
> > How is this better than simply having your hotplug helper load
> > the firmware from disk ? Its nice code but you have to ask the
> > question "why". I don't want the firmware wasting ram, I don't
> > need the firmware fs wasting ram. I may need to load the right
> > firmware from a choice of 16 odd types (usb_serial has some
> > examples there).
> 
> The obvious case is where it's impossible, or at least difficult/messy
> for userspace to directly access the device to upload the firmware.
> The latter is the case for the orinoco devices: the same firmwares can
> be used by USB, PCMCIA and PCI devices - these all have somewhat
> different register configurationss and different upload methods.
> Having an upload program that handles all these cases (not to mention
> endian and IO vs. memory mapped registers) seems silly when the driver
> already knows how to talk to the devices properly (actually we don't
> do firmware upload yet, but one day).  

 In the case of Orinoco USB devices, we even have fxload to download the
 ez-usb firmware, but the device doesn't change identity on firmware
 load, so both the hotplug helper and the driver end up accessing the
 device at the same time and it blows up.
 
> The driver could provide a chardev interface to upload the firmware
> through, but that's a lot more work - and there are issues of mapping
> the right chardev to the right network device instance, coping with
> the device in half-alive mode (i.e. initialized enough that firmware
> can be uploaded, but not ready to use because there's no firmware yet)
> etc.
> 
> I believe the idea is that hotplug (or other scripts) would copy the
> appropriate firmware into the relevant path in fwfs, so you don't need
> RAM for every firmware variant - just the one you're using. 

 Yes, that is the idea, although "hotplug" support is not implemented
 jet. I wanted some more feedback before expending more time on this.

 Taking a closer look, it should be easy to hook into the regular
 hotplug mechanism. I could just run '/sbin/hotplug firmware' setting
 FW_NAME=image_name in the environment and wait for it to finish like
 kmod does with modprobe.

 And if I also run it with 'ACTION=unload' when the firmware is not
 needed, it can easily be removed to save memory.

> And if we're really worried about memory it shouldn't be hard for the
> driver to arrange to unlink the firmware image once it's done with it
> (at the cost of not being able to reload the firmware on a
> reset/reinitialize, which might be sensible or necessary for some
> devices).

 On the other hand, there are already many drivers in the kernel that
 include firmware in headers, keyspan, io_edgeport, dabusb, ser_a2232,
 sym53c8xx_2, ...
 
 Something like fwfs would allow the removal of such firmware and save
 memory.

 If the firmware is not needed to boot, it could simply be removed from
 the kernel and be loaded from userspace.
 
 If the firmware is needed to boot, the driver could declare the
 firmware as __init data and register it on boot via fwfs_write_default.
 And after boot it could just be unlinked from fwfs recovering the
 memory.

 Currently the firmware image is copied into kmalloced memory then a
 drivers asks for the image, but if memory is such an issue, I'll look
 for a way to use the page cache data directly all the time. I'll take a
 closer look to the vmalloc code.
 
 Have a nice day

 	Manuel
-- 
--- Manuel Estrada Sainz <ranty@debian.org>
                         <ranty@bigfoot.com>
			 <ranty@users.sourceforge.net>
------------------------ <manuel.estrada@hispalinux.es> -------------------
Let us have the serenity to accept the things we cannot change, courage to
change the things we can, and wisdom to know the difference.

Re: firmware separation filesystem (fwfs)

[Alan Cox]

On Mer, 2003-04-16 at 17:36, Manuel Estrada Sainz wrote:
>  On the other hand, there are already many drivers in the kernel that
>  include firmware in headers, keyspan, io_edgeport, dabusb, ser_a2232,
>  sym53c8xx_2, ...

But so would loading it from hotplug via ioctl. It might be we want
a clean hotplug way to ask for 'firmare for xyz'.

Re: firmware separation filesystem (fwfs)

[David Gibson]

On Wed, Apr 16, 2003 at 12:31:21PM +0100, Alan Cox wrote:
> How is this better than simply having your hotplug helper load
> the firmware from disk ? Its nice code but you have to ask the
> question "why". I don't want the firmware wasting ram, I don't
> need the firmware fs wasting ram. I may need to load the right
> firmware from a choice of 16 odd types (usb_serial has some
> examples there).

The obvious case is where it's impossible, or at least difficult/messy
for userspace to directly access the device to upload the firmware.
The latter is the case for the orinoco devices: the same firmwares can
be used by USB, PCMCIA and PCI devices - these all have somewhat
different register configurationss and different upload methods.
Having an upload program that handles all these cases (not to mention
endian and IO vs. memory mapped registers) seems silly when the driver
already knows how to talk to the devices properly (actually we don't
do firmware upload yet, but one day).  

The driver could provide a chardev interface to upload the firmware
through, but that's a lot more work - and there are issues of mapping
the right chardev to the right network device instance, coping with
the device in half-alive mode (i.e. initialized enough that firmware
can be uploaded, but not ready to use because there's no firmware yet)
etc.

I believe the idea is that hotplug (or other scripts) would copy the
appropriate firmware into the relevant path in fwfs, so you don't need
RAM for every firmware variant - just the one you're using.  And if
we're really worried about memory it shouldn't be hard for the driver
to arrange to unlink the firmware image once it's done with it (at the
cost of not being able to reload the firmware on a reset/reinitialize,
which might be sensible or necessary for some devices).

My personal feeling is that this would probably make more sense as a
type of sysfs node, rather than a separate filesystem, but the basic
concept seems sound.

-- 
David Gibson			| For every complex problem there is a
david@gibson.dropbear.id.au	| solution which is simple, neat and
				| wrong.
http://www.ozlabs.org/people/dgibson

Re: firmware separation filesystem (fwfs)

[Alan Cox]

How is this better than simply having your hotplug helper load
the firmware from disk ? Its nice code but you have to ask the
question "why". I don't want the firmware wasting ram, I don't
need the firmware fs wasting ram. I may need to load the right
firmware from a choice of 16 odd types (usb_serial has some
examples there).

Alan

firmware separation filesystem (fwfs)

[Manuel Estrada Sainz]

[-- Attachment #1: Type: text/plain, Size: 5691 bytes --]

 Hello all,

 A little context: I am writing support for Orinoco USB devices, and
 they require some firmware to work. But I got the firmware from
 communication dumps and a windows .sys file, which doesn't qualify for
 redistribution.

 After reading the linux-kernel thread started by Pavel Roskin I
 implemented a little filesystem for this purpose. It was based on
 ramfs.  And after some feedback from Pavel Roskin and David Gibson it
 became what it is now. I'll try to make a resume:

 I couldn't find the blobs support patch that someone talked about, so I
 did it myself. I called it 'fwfs' for obvious reasons, but I would
 gladly change the name for something more general if required.

 The idea is making the filesystem as simple as possible and making it
 as easy and codeless as possible to be used from drivers.

 From userspace it would be as simple as:

 # mount -t fwfs fwfs /firmware/
 # cp my_firmware_data /firmware/orinoco_ezusb_fw
 # modprobe orinoco_usb

 On the simplest case the usage could be:

        static const struct fwfs_entry *fwfs_entry;

        int init_module(void)
        {
                fwfs_entry = fwfs_get("orinoco_ezusb_fw")
                if(!fwfs_entry)
                        -EAGAIN;
                /* fwfs_entry->size holds the size of the firmware */
                /* fwfs_entry->data holds a pointer to the firmware data */
                /* Both will be there until we fwfs_put it so we don't
                 * even have to copy it */
        }

        void exit_module(void)
        {
                fwfs_put(fwfs_entry);
        }

 In its current implementation, to get dynamic firmware updates, what I
 do in orinoco_usb.c is:

        static const struct fwfs_entry *fwfs_entry;

        static void * skel_probe(struct usb_device *udev, unsigned int
                                 ifnum, const struct usb_device_id *id)
        {
                const struct fwfs_entry *fwfs_entry_new;


                if((fwfs_entry_new = fwfs_get("orinoco_ezusb_fw"))){
                        fwfs_put(fwfs_entry);
                        if (fwfs_entry != fwfs_entry_new){
                                fwfs_entry = fwfs_entry_new;
                                firmware.size = fwfs_entry->size;
                                firmware.code = fwfs_entry->data;
                                find_fw_variant_offset(&firmware);
                        }
                }
        }
        int init_module(void)
        {
        	if(firmware.size)
                	fwfs_write_default("orinoco_ezusb_fw", firmware.code,
                        	           firmware.size);
	}
        void exit_module(void)
        {
                fwfs_put(fwfs_entry);
        }

 With the second, you can copy a new firmware from userspace and it
 gets used for the next device you plug.

 Interesting ideas:
 	- Integrate something like this with sysfs.
		- It looks like sysfs is limited in size to PAGE_SIZE,
		  which wouldn't be appropriate here.
        - allow registering a callback with fwfs to get informed when
          the firmware gets loaded/updated from userspace.
        - Transparently calling a userspace helper if there is no entry
          with the specified name.
	- implement a modified version of insmod that would  load any
	  binary as module with any given name.
	- Make the drivers read directly from the page cache, although
	  that would complicate the interface. An alternative interface
	  could be added, so drivers with huge firmwares could expend a
	  little more code and save some memory.

>From the linux-kernel thread:
> 1) Register a file on procfs and use "cat" to load the firmware into
> the
>    kernel.
>
> 2) Register a device for the same purpose.
>
> 3) Register a device, but use ioctl().
>
> 4) Open a network socket and use ioctl() on it (like ifconfig does).
[snip]
> 7) Encode the firmware into a header file, add it to the driver and
>    pretend that the copyright issue doesn't exist (like it's done in
>    the
>    Keyspan USB driver).

 This implementation is equivalent to 1), 2), 3) and 4) but involves
 less code on the driver. And has very simple userspace like 1) and 2).

 7) A default firmware can be included in a header and registered via
 fwfs_write_default(), this:
	- gives userspace access to the inkernel image.
	- makes it easy to update it at runtime.
	- enables the Debian guys to remove the firmware without
	  removing the whole driver.

Pavel Roskin wrote:
> I believe the reply will be that it's 2.7 material.  You could consider
> variants when an existing filesystem is reused as the file repository
> for drivers - it could increase chances that something will be done
> before 2.6 kernel.

 I removed all the filesystem code, and used ramfs directly as storage,
 just giving it the "fwfs" name for mounting from userspace.

David Gibson wrote:
> I think it would be better to leave the image in the page cache, and
> just put it together when the driver actually uses it.  It might make
> the driver interface a bit more complicated, but I think it's worth
> it.  And actually it shouldn't be too bad, because you could use
> vmalloc()-like methods to make a virtually contiguous mapping of the
> pages from the page cache.

 All kinds of comments/suggestions/criticism are welcomed.

 Have a nice day

 	ranty
-- 
--- Manuel Estrada Sainz <ranty@debian.org>
                         <ranty@bigfoot.com>
			 <ranty@users.sourceforge.net>
------------------------ <manuel.estrada@hispalinux.es> -------------------
Let us have the serenity to accept the things we cannot change, courage to
change the things we can, and wisdom to know the difference.

[-- Attachment #2: fwfs.h --]
[-- Type: text/x-chdr, Size: 359 bytes --]

#ifndef _LINUX_FWFS_H
#define _LINUX_FWFS_H
struct fwfs_entry {
	size_t size;
	u8 *data;
	/* the rest is private */
	atomic_t use_count;
	struct dentry *dentry;
	time_t ctime;
};
const struct fwfs_entry *fwfs_get(const char *name);
void fwfs_put(const struct fwfs_entry *entry);
void fwfs_write_default(const char *name, const u8 *data, size_t size);

#endif

[-- Attachment #3: fwfs.c --]
[-- Type: text/x-csrc, Size: 7963 bytes --]

/* firmware filesystem 
 *
 * Copyright (C) 2003 Manuel Estrada Sainz <ranty@debian.org>.
 * 
 */

/* The main idea is making it easy both to users and driver developers to
 * provide binary firmware from userspace to the kernel drivers.
 * Although any other kind of binary data could be handled */

#include <linux/module.h>
#include <linux/fs.h>
#include <linux/pagemap.h>
#include <linux/init.h>
#include <linux/string.h>
#include <linux/locks.h>
#include <linux/types.h>
#include <linux/slab.h>
#include <linux/spinlock.h>
#include <asm/uaccess.h>

#include "fwfs.h"

static struct vfsmount *fwfs_mnt;
static spinlock_t fwfs_lock = SPIN_LOCK_UNLOCKED;

static struct fwfs_entry *fwfs_grab_image(struct inode *inode,
					  struct dentry *dentry);

/* Refcounting fwfs_entrys is probably not worth it.
 * In the general case, there will be just one client per entry, so allocating
 * on fwfs_get and freeing on fwfs_put should be good enough.
 * But since I wrote it, here it is for consideration. */

static void put_entry(struct fwfs_entry *entry)
{
	spin_lock(&fwfs_lock);
	if (entry && atomic_dec_and_test(&entry->use_count)){
		if(entry->dentry->d_inode->u.generic_ip == entry)
			entry->dentry->d_inode->u.generic_ip = NULL;
		kfree(entry->data);
		kfree(entry);
	}
	spin_unlock(&fwfs_lock);
}

static struct fwfs_entry *get_entry(struct inode *inode)
{
	struct fwfs_entry *entry;
	spin_lock(&fwfs_lock);
	entry = inode->u.generic_ip;
	if(entry){
		atomic_inc(&entry->use_count);
	}
	spin_unlock(&fwfs_lock);
	return entry;
}
static struct fwfs_entry *alloc_entry(size_t size)
{
	struct fwfs_entry *entry = kmalloc(sizeof(struct fwfs_entry),
					   GFP_KERNEL);
	if(entry){
		entry->dentry = NULL;
		atomic_set(&entry->use_count, 1);
		entry->size = size;
		if(size){
			entry->data = kmalloc(size, GFP_KERNEL);
			if(!entry->data){
				kfree(entry);
				entry = NULL;
			}
		} else {
			entry->data = NULL;
		}
	}
	return entry;
}

static struct fwfs_entry *entry_lookup(const char *str_name)
{
	struct dentry *dentry;
	struct dentry *parent = dget(fwfs_mnt->mnt_root);
	struct qstr name= {str_name, strlen(str_name), 0};
	struct fwfs_entry *entry;

	name.hash = full_name_hash(name.name, name.len);
	dentry = d_lookup(parent, &name);
	dput(parent);
	if(!dentry)
		return NULL;
	if (!dentry->d_inode){
		dput(dentry);
		return NULL;
	}
	entry = fwfs_grab_image(dentry->d_inode, dentry);
	if (!entry)
		dput(dentry);
	return entry;
}

const struct fwfs_entry *fwfs_get(const char *name)
{
	struct fwfs_entry *entry;

	entry = entry_lookup(name);
	if (entry && !entry->data){
		dput(entry->dentry);
		entry = NULL;
	}
	return entry;
}
void fwfs_put(const struct fwfs_entry *entry)
{
	if(entry){
		struct dentry *dentry = entry->dentry;
		put_entry((struct fwfs_entry *)entry);
		dput(dentry);
	}
}

static struct fwfs_entry *fwfs_grab_image(struct inode *inode,
					  struct dentry *dentry)
{
	struct fwfs_entry *fwfs_entry_old = get_entry(inode);
	struct fwfs_entry *fwfs_entry;
	struct address_space *mapping = inode->i_mapping;
	unsigned long index, end_index;

	down(&inode->i_sem);

	if(fwfs_entry_old && inode->i_ctime == fwfs_entry_old->ctime){
		printk("fwfs_entry is up to date\n");
		up(&inode->i_sem);
		return fwfs_entry_old;
	}

#if 0
	/* There is a race here, if userspace is writing the file
	   while we read it, we can get just a piece of it.

	   Locking i_sem is not enough.

	   The following looks neat to me, but I don't have a file
	   pointer to do it.

	   I would appreciate any sugestions.
	*/
	while (deny_write_access(struct file)){
		set_current_state(TASK_UNINTERRUPTIBLE);
		schedule_timeout(HZ);
	}
	/* get the data */
	allow_write_access(struct file);
#else
	/* This at least reduces the chances of the problem */
	while (atomic_read(&inode->i_writecount) > 0) {
		printk(KERN_WARNING "fwfs: inode busy\n");
		set_current_state(TASK_UNINTERRUPTIBLE);
		schedule_timeout(HZ);
	}
#endif

	fwfs_entry = alloc_entry(inode->i_size);
	if(!fwfs_entry){
		up(&inode->i_sem);
		return fwfs_entry_old;
	}
	fwfs_entry->dentry = dentry;
	fwfs_entry->ctime = inode->i_ctime;
	end_index = inode->i_size >> PAGE_CACHE_SHIFT;
	for (index=0; index <= end_index; index++) {
		struct page *page, **hash;
		unsigned long bytes;
		u8 *buf = fwfs_entry->data + index * PAGE_CACHE_SIZE;


		bytes = PAGE_CACHE_SIZE;
		if (index == end_index) {
			bytes = inode->i_size & ~PAGE_CACHE_MASK;
			if(!bytes)
				break;
		}

		/* in ramfs, the data is always in the page cache , right? */
		hash = page_hash(mapping, index);
		page = __find_get_page(mapping, index, hash);
		BUG_ON(!Page_Uptodate(page));

		memcpy(buf, kmap(page), bytes);
		kunmap(page);

		page_cache_release(page);
	}
	UPDATE_ATIME(inode);

	put_entry(fwfs_entry_old);
	spin_lock(&fwfs_lock);
	inode->u.generic_ip=fwfs_entry;
	spin_unlock(&fwfs_lock);
	up(&inode->i_sem);
	return fwfs_entry;
}

int fwfs_write_image(struct inode *inode, const u8 *buf, size_t size)
{
	struct address_space *mapping = inode->i_mapping;
	loff_t          pos = 0;
	struct page     *page, *cached_page=NULL;
	long            retval = 0;
	unsigned        bytes;

	down(&inode->i_sem);

	inode->i_ctime = inode->i_mtime = CURRENT_TIME;
	mark_inode_dirty_sync(inode);
	do {
		unsigned long index;
		char *kaddr;
		index = pos >> PAGE_CACHE_SHIFT;
		bytes = PAGE_CACHE_SIZE;
		if (bytes > size)
			bytes = size;
		page = grab_cache_page(mapping, index);
		if (!page){
			retval = -ENOMEM;
			break;
		}
		BUG_ON (!PageLocked(page));
		kaddr = kmap(page);
		/* prepare_write and commit_write never fail in ramfs, and
		 * they don't use the 'file' argument */
		mapping->a_ops->prepare_write(NULL, page, 0, bytes);
		memcpy(kaddr, buf, bytes);
		flush_dcache_page(page);
		mapping->a_ops->commit_write(NULL, page, 0, bytes);
		size -= bytes;
		pos += bytes;
		buf += bytes;
		
		kunmap(page);
		SetPageReferenced(page);
		UnlockPage(page);
		page_cache_release(page);
	} while (size);

	if (cached_page)
		page_cache_release(cached_page);

	up(&inode->i_sem);
	return retval;
	
}
void fwfs_write_default(const char *name, const u8 *data, size_t size)
{
	struct dentry * dentry;
	struct dentry * parent = fwfs_mnt->mnt_root;
	int error;
	struct qstr d_name = { name, strlen(name) };
	d_name.hash = full_name_hash(d_name.name, d_name.len);

	down(&parent->d_inode->i_sem);
	dentry = lookup_hash(&d_name, parent);
	if(dentry->d_inode){
		/* We already have an image */
		printk("we already have %s\n", name);
		goto up_parent;
	}
	error = vfs_create(parent->d_inode, dentry, 0644);
	if(error){
		printk("%d\n", error);
		goto up_parent;
	}
	if((error = fwfs_write_image(dentry->d_inode, data, size)))
		printk("fwfs_write_image: status %d\n", error);
	
up_parent:
	up(&parent->d_inode->i_sem);
	dput(dentry);
	return;
}
EXPORT_SYMBOL(fwfs_get);
EXPORT_SYMBOL(fwfs_put);
EXPORT_SYMBOL(fwfs_write_default);

static DECLARE_FSTYPE(fwfs_fs_type, "fwfs", NULL,
		      FS_LITTER|FS_SINGLE);

static int can_unload (void)
{
	/* kern_mount makes the module busy for ever, so I have to keep track
	 * of busyness myself */
	int usecount = atomic_read(&THIS_MODULE->uc.usecount);
	int s_active = atomic_read(&fwfs_mnt->mnt_sb->s_active);

	if (usecount == 1 && s_active == 1)
		/* both will go on kern_umount */
		return 0;
	
	return usecount;
}

static int __init init_fwfs_fs(void)
{
	int err;
	struct file_system_type *ramfs_fs_type = get_fs_type("ramfs");

	if (!ramfs_fs_type)
		return -ENOENT;

	fwfs_fs_type.read_super = ramfs_fs_type->read_super;

	err = register_filesystem(&fwfs_fs_type);
	if (err)
		return err;
	fwfs_mnt = kern_mount(&fwfs_fs_type);
	if (IS_ERR(fwfs_mnt)){
		unregister_filesystem(&fwfs_fs_type);
		return PTR_ERR(fwfs_mnt);
	}
	THIS_MODULE->can_unload = can_unload;
	return 0;
}

static void __exit exit_fwfs_fs(void)
{
	kern_umount(fwfs_mnt);
	BUG_ON(MOD_IN_USE);
	unregister_filesystem(&fwfs_fs_type);
}

module_init(init_fwfs_fs)
module_exit(exit_fwfs_fs)


MODULE_LICENSE("GPL");