FPGA programming driver architecture

FPGA programming driver architecture

[Hugo Villeneuve]

Hi,
I have written some code to program a FPGA in Linux, for two different types of boards: one uses a serial interface (SPI) and the second a parallel interface. I have been able to sucessfully program both boards. I'm now trying to clean my code and make it more generic, as well as better in line with the Linux driver model. I would also like to include it in the mainline kernel if there is interest.

Here is a description of the current architecture (refer to diagrams below): The fpgaload module controls one output GPIOs (PROG), and two input GPIOs (INIT and DONE). These GPIOs are specified in board setup code. Both fpgaload_ser and fpgaload_par modules export a single function to write a byte. The fpgaload driver is a char device to which we can write (/dev/fpgaload) to program a bitstream (FPGA firmware) inside the FPGA. The fpgaload driver will toggle the GPIOs to initiate programming and the then call the corresponding write_byte function based on the interface type specified in board setup code (serial or parallel, or any future interface desired).


FPGA serial bitstream loading architecture

+----------------------+
| FPGA load driver     |
| (fpgaload)           |
+----------------------+
  |             |
  |    +----------------+
  |    | FPGA serial    |
  |    | load driver    |
  |    | (fpgaload_ser) |
  |    +----------------+
  |             |
  |    +-------------------+
  |    | SPI master        |
  |    | controller driver |
  |    +-------------------+
  |             |
  |             |           Linux
------------------------------------
  |             |           HARDWARE
  |             |spi
  |             |
  |      +--------------+
  |      | SPI          |
  +----->| Programming  |
   GPIOs | interface    |
         +--------------+
                    FPGA


FPGA parallel bitstream loading architecture

+----------------------+
| FPGA load driver     |
| (fpgaload)           |
+----------------------+
  |             |
  |    +----------------+
  |    | FPGA parallel  |
  |    | load driver    |
  |    | (fpgaload_par) |
  |    +----------------+
  |             |
  |             |           Linux
------------------------------------
  |             |           HARDWARE
  |             |parallel
  |             |
  |      +--------------+
  |      | Parallel     |
  +----->| Programming  |
   GPIOs | interface    |
         +--------------+
                    FPGA


The problem with that approach is that when loading the fpgaload module with modprobe, it will automatically try to load the fpgaload_ser and fpgaload_par modules, even if only serial interface was specified in board setup code for example. This is not good when building a kernel for similar but different boards.

Probably a better approach would be for the fpgaload_Ser and fpgaload_par modules to register themselves with the fpgaload module. Then, should the fpgaload module be built using a BUS driver structure? Or anyone having any suggestions on how it should be implemented?

Hugo Villeneuve

Re: FPGA programming driver architecture

[Florian Fainelli]

(CC'ing linux-embedded)

Salut Hugo,

Le Friday 12 December 2008 21:03:14 Hugo Villeneuve, vous avez écrit :
> Hi,
> I have written some code to program a FPGA in Linux, for two different
> types of boards: one uses a serial interface (SPI) and the second a
> parallel interface. I have been able to sucessfully program both boards.
> I'm now trying to clean my code and make it more generic, as well as better
> in line with the Linux driver model. I would also like to include it in the
> mainline kernel if there is interest.

Is it a platform-driver ? What do you provide in platform_data ?

>
> Here is a description of the current architecture (refer to diagrams
> below): The fpgaload module controls one output GPIOs (PROG), and two input
> GPIOs (INIT and DONE). These GPIOs are specified in board setup code. Both
> fpgaload_ser and fpgaload_par modules export a single function to write a
> byte. The fpgaload driver is a char device to which we can write
> (/dev/fpgaload) to program a bitstream (FPGA firmware) inside the FPGA. 

You should probably consider using request_firmware to load the bitstream from 
the userspace and possibly add a /sys interface to export some attributes 
like :

- GPIOs being used between the host and the FPGA
- status (i.e : programmed, not programmed ...)
- FPGA vendor, type ...

> The 
> fpgaload driver will toggle the GPIOs to initiate programming and the then
> call the corresponding write_byte function based on the interface type
> specified in board setup code (serial or parallel, or any future interface
> desired).

> The problem with that approach is that when loading the fpgaload module
> with modprobe, it will automatically try to load the fpgaload_ser and
> fpgaload_par modules, even if only serial interface was specified in board
> setup code for example. This is not good when building a kernel for similar
> but different boards.

What about something like that :

- fpgaload-core which contains all the code that can be shared between the 
drivers like requesting firmware, providing sysfs attributes, 
- fpgaload-spi would handle the low-level SPI connection
- fpgaload-par would handle the low-level parallel connection

fpgaload-ser and par would register with fpgaload-core and they could register 
a fpga loading callback which is low-level specific for instance. Platform 
code would instantiate the core driver. That way, adding support for other 
loading protocols like slave serial or master serial can be done easily.

>
> Probably a better approach would be for the fpgaload_Ser and fpgaload_par
> modules to register themselves with the fpgaload module. Then, should the
> fpgaload module be built using a BUS driver structure? Or anyone having any
> suggestions on how it should be implemented?
-- 
Best regards, Florian Fainelli
Email : florian@openwrt.org
http://openwrt.org
-------------------------------

Re: FPGA programming driver architecture

[Hugo Villeneuve]

On Sat, 13 Dec 2008 13:58:01 +0100
Florian Fainelli <florian@openwrt.org> wrote:

> (CC'ing linux-embedded)
> 
> Salut Hugo,
> 
> Le Friday 12 December 2008 21:03:14 Hugo Villeneuve, vous avez écrit :
> > Hi,
> > I have written some code to program a FPGA in Linux, for two
> > different types of boards: one uses a serial interface (SPI) and
> > the second a parallel interface. I have been able to sucessfully
> > program both boards. I'm now trying to clean my code and make it
> > more generic, as well as better in line with the Linux driver
> > model. I would also like to include it in the mainline kernel if
> > there is interest.
> 
> Is it a platform-driver ? What do you provide in platform_data ?

Hi Florian,
Yes, currently it is a platform driver. Here is my platform data:

static struct fpgaload_pdata_t fpgaload_pdata = {
        .fpga_family            = FPGA_FAMILY_XILINX_XC3S,
        .payload_full_size      = XC3S1400A_PAYLOAD_SIZE,
        .program_b              = GPIO(21),
        .done                   = GPIO(19),
        .init_b                 = GPIO(20),
        .swapbits               = 0,
        .interface              = FPGALOAD_INTERFACE_SERIAL,
        .bitstream_name         = "fpga.bit",
};

> > Here is a description of the current architecture (refer to diagrams
> > below): The fpgaload module controls one output GPIOs (PROG), and
> > two input GPIOs (INIT and DONE). These GPIOs are specified in board
> > setup code. Both fpgaload_ser and fpgaload_par modules export a
> > single function to write a byte. The fpgaload driver is a char
> > device to which we can write (/dev/fpgaload) to program a bitstream
> > (FPGA firmware) inside the FPGA. 
> 
> You should probably consider using request_firmware to load the
> bitstream from the userspace and possibly add a /sys interface to
> export some attributes like :

This is already the case with request_firmware :) The bitstream_name field in platform data is used to specify the name of the file.
 
> - GPIOs being used between the host and the FPGA
> - status (i.e : programmed, not programmed ...)
> - FPGA vendor, type ...
> 
> > The 
> > fpgaload driver will toggle the GPIOs to initiate programming and
> > the then call the corresponding write_byte function based on the
> > interface type specified in board setup code (serial or parallel,
> > or any future interface desired).
> 
> > The problem with that approach is that when loading the fpgaload
> > module with modprobe, it will automatically try to load the
> > fpgaload_ser and fpgaload_par modules, even if only serial
> > interface was specified in board setup code for example. This is
> > not good when building a kernel for similar but different boards.
> 
> What about something like that :
> 
> - fpgaload-core which contains all the code that can be shared
> between the drivers like requesting firmware, providing sysfs
> attributes, 
> - fpgaload-spi would handle the low-level SPI connection
> - fpgaload-par would handle the low-level parallel connection

This is pretty much like that right now, exceptfor sysfs attributes.

> fpgaload-ser and par would register with fpgaload-core and they could
> register a fpga loading callback which is low-level specific for
> instance. Platform code would instantiate the core driver. That way,
> adding support for other loading protocols like slave serial or
> master serial can be done easily.

Yes, I think this could work, but I would like to know what kind of driver fpgaload-core would be. Currently it is a platform driver and a chardev (to support open and write methods for programming). How shall I implement it? Bus driver, platform driver?

Ciao, Hugo V.

Re: FPGA programming driver architecture

[Hugo Villeneuve]

On Sat, 13 Dec 2008 13:58:01 +0100
Florian Fainelli <florian@openwrt.org> wrote:

> (CC'ing linux-embedded)
> 
> Salut Hugo,
> 
> Le Friday 12 December 2008 21:03:14 Hugo Villeneuve, vous avez écrit :
> > Hi,
> > I have written some code to program a FPGA in Linux, for two
> > different types of boards: one uses a serial interface (SPI) and
> > the second a parallel interface. I have been able to sucessfully
> > program both boards. I'm now trying to clean my code and make it
> > more generic, as well as better in line with the Linux driver
> > model. I would also like to include it in the mainline kernel if
> > there is interest.
> ...
> What about something like that :
> 
> - fpgaload-core which contains all the code that can be shared
> between the drivers like requesting firmware, providing sysfs
> attributes, 
> - fpgaload-spi would handle the low-level SPI connection
> - fpgaload-par would handle the low-level parallel connection
> 
> fpgaload-ser and par would register with fpgaload-core and they could
> register a fpga loading callback which is low-level specific for
> instance. Platform code would instantiate the core driver. That way,
> adding support for other loading protocols like slave serial or
> master serial can be done easily.

Yes, but how I actually implement fpgaload_core, fpgaload_ser and
fpgaload_par?

Bus driver?
class driver?
platform driver?

I am not sure how I should implement things and how each low-level
specific driver should register with the fpgaload_core driver...

Hugo V.

---------------
Hugo Villeneuve
www.hugovil.com
---------------

Re: FPGA programming driver architecture

[Thiago Galesi]

> > > Hi,
> > > I have written some code to program a FPGA in Linux, for two
> > > different types of boards: one uses a serial interface (SPI) and
> > > the second a parallel interface. I have been able to sucessfully
> > > program both boards. I'm now trying to clean my code and make it
> > > more generic, as well as better in line with the Linux driver
> > > model.

Considering the several FPGA models available and ways to program it,
I guess the important thing to consider is what can be made generic.
(that is what will become fpgaload)

Also, there may be cases where after FPGA is programmed it "becomes a
device" (PCI or whatever)

> >
> > - fpgaload-core which contains all the code that can be shared
> > between the drivers like requesting firmware, providing sysfs
> > attributes,
> > - fpgaload-spi would handle the low-level SPI connection
> > - fpgaload-par would handle the low-level parallel connection

I think this maybe split into three layers (maybe)

1 - low level connection: paralell / spi / i2c / whatever
2 - vendor: xylinx, altera, etc
3 - generic stuff

Take a look at the MTD and SPI drivers and how they split things

--
-
Thiago Galesi

Re: FPGA programming driver architecture

[Leon Woestenberg]

Hello,

On Thu, Jan 8, 2009 at 11:14 PM, Thiago Galesi <thiagogalesi@gmail.com> wrote:
>> > > Hi,
>> > > I have written some code to program a FPGA in Linux, for two
>> > > different types of boards: one uses a serial interface (SPI) and
>> > > the second a parallel interface. I have been able to sucessfully
>> > > program both boards. I'm now trying to clean my code and make it
>> > > more generic, as well as better in line with the Linux driver
>> > > model.
>
> Considering the several FPGA models available and ways to program it,
> I guess the important thing to consider is what can be made generic.

Indeed.

The programming back end should be generic enough so that it can use
other subsystems.

The FPGA configuration interface could be on the memory bus (and in
many cases the DMA helpers can be used) or behind a PCI bus, or can
even have SPI etc front-ends.

In general the programming API should accept a blob of data,
preferably in one chunk (pointer plus length) or at least 4kB chunks
or so, so that configuration is not severy overheaded by a callback
per byte (or even bit).

Regards,
-- 
Leon

Re: FPGA programming driver architecture

[Florian Fainelli]

[-- Attachment #1: Type: text/plain, Size: 854 bytes --]

Hello,

Le Thursday 08 January 2009 23:47:58 Leon Woestenberg, vous avez écrit :
> Indeed.
>
> The programming back end should be generic enough so that it can use
> other subsystems.
>
> The FPGA configuration interface could be on the memory bus (and in
> many cases the DMA helpers can be used) or behind a PCI bus, or can
> even have SPI etc front-ends.
>
> In general the programming API should accept a blob of data,
> preferably in one chunk (pointer plus length) or at least 4kB chunks
> or so, so that configuration is not severy overheaded by a callback
> per byte (or even bit).

That's the idea behind using request_firmware, which provides you with both a 
pointer to the data and the size of the blob.
-- 
Best regards, Florian Fainelli
Email : florian@openwrt.org
http://openwrt.org
-------------------------------

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: FPGA programming driver architecture

[Leon Woestenberg]

Hello,

On Thu, Jan 8, 2009 at 11:57 PM, Florian Fainelli <florian@openwrt.org> wrote:
> Le Thursday 08 January 2009 23:47:58 Leon Woestenberg, vous avez écrit :

>> The programming back end should be generic enough so that it can use
>> other subsystems.
>>
> That's the idea behind using request_firmware, which provides you with both a
> pointer to the data and the size of the blob.

I know, that's what I would call the front-end (near userspace).

I meant the low-level back-end (towards hardware):

> - fpgaload-core which contains all the code that can be shared
> between the drivers like requesting firmware, providing sysfs
> attributes,
> - fpgaload-spi would handle the low-level SPI connection
> - fpgaload-par would handle the low-level parallel connection
>
> fpgaload-ser and par would register with fpgaload-core and they could
> register a fpga loading callback which is low-level specific for
> instance.

My $0.02 was:

Those callbacks should also be able to accept chunks of data
(preferably in the same way using ptr/len).


Regards,
-- 
Leon

Re: FPGA programming driver architecture

[Duane Ellis]

 >> [various FPGA config methods]

Don't forget JTAG, and playing back an SVF files, or BIT files.

-Duane.

Re: FPGA programming driver architecture

[Alexander Clouter]

Duane Ellis <duane@duaneellis.com> wrote:
> >> [various FPGA config methods]
> 
> Don't forget JTAG, and playing back an SVF files, or BIT files.
> 
On the board we have, the FPGA is programmable over JTAG which is 
conveniently strapped to the GPIO pins of the CPU.

http://www.embeddedarm.com/products/board-detail.php?product=TS-7800

Cheers

Alex

-- 
Alexander Clouter
.sigmonster says: This screen intentionally left blank.

Re: FPGA programming driver architecture

[Alexander Clouter]

Duane Ellis <duane@duaneellis.com> wrote:
> >> [various FPGA config methods]
> 
> Don't forget JTAG, and playing back an SVF files, or BIT files.
> 
On the board we have, the FPGA is programmable over JTAG which is 
conveniently strapped to the GPIO pins of the CPU.

http://www.embeddedarm.com/products/board-detail.php?product=TS-7800

Cheers

Alex

-- 
Alexander Clouter
.sigmonster says: This screen intentionally left blank.

Re: FPGA programming driver architecture

[Hugo Villeneuve]

On Thu, 8 Jan 2009 20:14:36 -0200
"Thiago Galesi" <thiagogalesi@gmail.com> wrote:

> > > > Hi,
> > > > I have written some code to program a FPGA in Linux, for two
> > > > different types of boards: one uses a serial interface (SPI) and
> > > > the second a parallel interface. I have been able to sucessfully
> > > > program both boards. I'm now trying to clean my code and make it
> > > > more generic, as well as better in line with the Linux driver
> > > > model.
> 
> Considering the several FPGA models available and ways to program it,
> I guess the important thing to consider is what can be made generic.
> (that is what will become fpgaload)

I have a pretty good idea of what goes into the core and
what goes into the specific interface modules (spi, parallel, etc).

> Also, there may be cases where after FPGA is programmed it "becomes a
> device" (PCI or whatever)

This is exactly the case for my board, but it doesn´t change anything
to the programming side of things, apart for the fact that the FPGA
must be configured before using any in-FPGA PCI bus for example.

> > > - fpgaload-core which contains all the code that can be shared
> > > between the drivers like requesting firmware, providing sysfs
> > > attributes,
> > > - fpgaload-spi would handle the low-level SPI connection
> > > - fpgaload-par would handle the low-level parallel connection
> 
> I think this maybe split into three layers (maybe)
> 
> 1 - low level connection: paralell / spi / i2c / whatever
> 2 - vendor: xylinx, altera, etc
> 3 - generic stuff
> 
> Take a look at the MTD and SPI drivers and how they split things

I think that for the moment, I will focus on implementing the core
module and the parallel/SPI interfaces only. Later we may want to add
more fonctionality (vendors, JTAG, etc).

But for now, my main problem is that I need to have some insight on how
I actually implement the different modules:

Bus driver?
class driver?
platform driver?

Can anybody give some advice on that?

Thank-you, Hugo.

---------------
Hugo Villeneuve
www.hugovil.com
---------------

Re: FPGA programming driver architecture

[Hans J. Koch]

On Fri, Jan 09, 2009 at 10:07:25AM -0500, Hugo Villeneuve wrote:
> 
> I think that for the moment, I will focus on implementing the core
> module and the parallel/SPI interfaces only. Later we may want to add
> more fonctionality (vendors, JTAG, etc).

That's a good plan, but you'll have to make sure different FPGA programming
drivers can be added easily to some fpga-core no matter which interface they
use.

> 
> But for now, my main problem is that I need to have some insight on how
> I actually implement the different modules:
> 
> Bus driver?
> class driver?
> platform driver?

Those are not mutually exclusive. If you register an spi driver, it'll appear
on that bus. It would be a good idea to also create a class "fpga". And an
FPGA will likely be a platform device unless you create a complete subsystem
that offers something like register_fpga_device(). The latter would be a nice
thing to have, I had to write FPGA programming drivers and had to implement
them as some char device in drivers/misc because there was no place where
they could go...

IMHO, the ideal solution would be to have an FPGA programming subsystem.
Somewhere in my board support I'd setup some struct fpga_device that
contains an element "bus_type" which can be set to FPGA_BUS_PARALLEL or
FPGA_BUS_SPI or whatever. A "name" element declares which driver I'd like
to use for my chip. A call to some fpga_register_device() with a
pointer to that struct would make the fpga subsystem remember that device and
look for an apropriate driver. Drivers register themselves by a call to some
register_fpga_driver() function. The fpga core would then make sure the
drivers probe function is called with pointers to the struct fpga_device of
the registered devices. That's how other subsystems work.

Such a subsystem would solve any FPGA programming problems (and your "main
problem" mentioned above) once and for all.

Thanks,
Hans