Proposal: r8152 firmware patching framework

Proposal: r8152 firmware patching framework

[Prashant Malani]

Hi,

The r8152 driver source code distributed by Realtek (on
www.realtek.com) contains firmware patches. This involves binary
byte-arrays being written byte/word-wise to the hardware memory
Example: grundler@chromium.org (cc-ed) has an experimental patch which
includes the firmware patching code which was distributed with the
Realtek source :
https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1417953

It would be nice to have a way to incorporate these firmware fixes
into the upstream code. Since having indecipherable byte-arrays is not
possible upstream, I propose the following:
- We use the assistance of Realtek to come up with a format which the
firmware patch files can follow (this can be documented in the
comments).
       - A real simple format could look like this:
               +
<section1><size_in_bytes><address1><data1><address2><data2>...<addressN><dataN><section2>...
                + The driver would be able to understand how to parse
each section (e.g is each data entry a byte or a word?)

- We use request_firmware() to load the firmware, parse it and write
the data to the relevant registers.

I'm unfamiliar with what the preferred method of firmware patching is,
so I hope the maintainers can help suggest the best path forward.

As an aside: It would be great if Realtek could publish a list of
fixes that the firmware patches implement (I think a list on the
driver download page on the Realtek website would be an excellent
starting point).

Thanks and Best regards,

-Prashant

Re: Proposal: r8152 firmware patching framework

[Prashant Malani]

(Adding a few more Realtek folks)

Friendly ping. Any thoughts / feedback, Realtek folks (and others) ?

On Thu, Aug 29, 2019 at 11:40 AM Prashant Malani <pmalani@chromium.org> wrote:
>
> Hi,
>
> The r8152 driver source code distributed by Realtek (on
> www.realtek.com) contains firmware patches. This involves binary
> byte-arrays being written byte/word-wise to the hardware memory
> Example: grundler@chromium.org (cc-ed) has an experimental patch which
> includes the firmware patching code which was distributed with the
> Realtek source :
> https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1417953
>
> It would be nice to have a way to incorporate these firmware fixes
> into the upstream code. Since having indecipherable byte-arrays is not
> possible upstream, I propose the following:
> - We use the assistance of Realtek to come up with a format which the
> firmware patch files can follow (this can be documented in the
> comments).
>        - A real simple format could look like this:
>                +
> <section1><size_in_bytes><address1><data1><address2><data2>...<addressN><dataN><section2>...
>                 + The driver would be able to understand how to parse
> each section (e.g is each data entry a byte or a word?)
>
> - We use request_firmware() to load the firmware, parse it and write
> the data to the relevant registers.
>
> I'm unfamiliar with what the preferred method of firmware patching is,
> so I hope the maintainers can help suggest the best path forward.
>
> As an aside: It would be great if Realtek could publish a list of
> fixes that the firmware patches implement (I think a list on the
> driver download page on the Realtek website would be an excellent
> starting point).
>
> Thanks and Best regards,
>
> -Prashant

Re: Proposal: r8152 firmware patching framework

[Amber Chen]

+ acct mgr, Stephen



> Prashant Malani <pmalani@chromium.org> 於 2019年8月31日 上午6:24 寫道：
> 
> (Adding a few more Realtek folks)
> 
> Friendly ping. Any thoughts / feedback, Realtek folks (and others) ?
> 
>> On Thu, Aug 29, 2019 at 11:40 AM Prashant Malani <pmalani@chromium.org> wrote:
>> 
>> Hi,
>> 
>> The r8152 driver source code distributed by Realtek (on
>> www.realtek.com) contains firmware patches. This involves binary
>> byte-arrays being written byte/word-wise to the hardware memory
>> Example: grundler@chromium.org (cc-ed) has an experimental patch which
>> includes the firmware patching code which was distributed with the
>> Realtek source :
>> https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1417953
>> 
>> It would be nice to have a way to incorporate these firmware fixes
>> into the upstream code. Since having indecipherable byte-arrays is not
>> possible upstream, I propose the following:
>> - We use the assistance of Realtek to come up with a format which the
>> firmware patch files can follow (this can be documented in the
>> comments).
>>       - A real simple format could look like this:
>>               +
>> <section1><size_in_bytes><address1><data1><address2><data2>...<addressN><dataN><section2>...
>>                + The driver would be able to understand how to parse
>> each section (e.g is each data entry a byte or a word?)
>> 
>> - We use request_firmware() to load the firmware, parse it and write
>> the data to the relevant registers.
>> 
>> I'm unfamiliar with what the preferred method of firmware patching is,
>> so I hope the maintainers can help suggest the best path forward.
>> 
>> As an aside: It would be great if Realtek could publish a list of
>> fixes that the firmware patches implement (I think a list on the
>> driver download page on the Realtek website would be an excellent
>> starting point).
>> 
>> Thanks and Best regards,
>> 
>> -Prashant
> 
> ------Please consider the environment before printing this e-mail.

RE: Proposal: r8152 firmware patching framework

[Hayes Wang]

Prashant Malani <pmalani@chromium.org> 
> >
> > (Adding a few more Realtek folks)
> >
> > Friendly ping. Any thoughts / feedback, Realtek folks (and others) ?
> >
> >> On Thu, Aug 29, 2019 at 11:40 AM Prashant Malani
> <pmalani@chromium.org> wrote:
> >>
> >> Hi,
> >>
> >> The r8152 driver source code distributed by Realtek (on
> >> www.realtek.com) contains firmware patches. This involves binary
> >> byte-arrays being written byte/word-wise to the hardware memory
> >> Example: grundler@chromium.org (cc-ed) has an experimental patch
> which
> >> includes the firmware patching code which was distributed with the
> >> Realtek source :
> >>
> https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel
> /+/1417953
> >>
> >> It would be nice to have a way to incorporate these firmware fixes
> >> into the upstream code. Since having indecipherable byte-arrays is not
> >> possible upstream, I propose the following:
> >> - We use the assistance of Realtek to come up with a format which the
> >> firmware patch files can follow (this can be documented in the
> >> comments).
> >>       - A real simple format could look like this:
> >>               +
> >>
> <section1><size_in_bytes><address1><data1><address2><data2>...<addressN
> ><dataN><section2>...
> >>                + The driver would be able to understand how to parse
> >> each section (e.g is each data entry a byte or a word?)
> >>
> >> - We use request_firmware() to load the firmware, parse it and write
> >> the data to the relevant registers.

I plan to finish the patches which I am going to submit, first. Then,
I could focus on this. However, I don't think I would start this
quickly. There are many preparations and they would take me a lot of
time.

Best Regards,
Hayes

RE: Proposal: r8152 firmware patching framework

[Bambi Yeh]

Hi Prashant:

We will try to implement your requests.
Based on our experience, upstream reviewer often reject our modification if they have any concern.
Do you think you can talk to them about this idea and see if they will accept it or not?
Or if you can help on this after we submit it?

Also, Hayes is now updating our current upstream driver and it goes back and forth for a while.
So we will need some time to finish it and the target schedule to have your request done is in the end of this month.

Thank you very much.

Best Regards,
Bambi Yeh

-----Original Message-----
From: Hayes Wang <hayeswang@realtek.com> 
Sent: Monday, September 2, 2019 2:31 PM
To: Amber Chen <amber.chen@realtek.com>; Prashant Malani <pmalani@chromium.org>
Cc: David Miller <davem@davemloft.net>; netdev@vger.kernel.org; Bambi Yeh <bambi.yeh@realtek.com>; Ryankao <ryankao@realtek.com>; Jackc <jackc@realtek.com>; Albertk <albertk@realtek.com>; marcochen@google.com; nic_swsd <nic_swsd@realtek.com>; Grant Grundler <grundler@chromium.org>
Subject: RE: Proposal: r8152 firmware patching framework

Prashant Malani <pmalani@chromium.org> 
> >
> > (Adding a few more Realtek folks)
> >
> > Friendly ping. Any thoughts / feedback, Realtek folks (and others) ?
> >
> >> On Thu, Aug 29, 2019 at 11:40 AM Prashant Malani
> <pmalani@chromium.org> wrote:
> >>
> >> Hi,
> >>
> >> The r8152 driver source code distributed by Realtek (on
> >> www.realtek.com) contains firmware patches. This involves binary 
> >> byte-arrays being written byte/word-wise to the hardware memory
> >> Example: grundler@chromium.org (cc-ed) has an experimental patch
> which
> >> includes the firmware patching code which was distributed with the 
> >> Realtek source :
> >>
> https://chromium-review.googlesource.com/c/chromiumos/third_party/kern
> el
> /+/1417953
> >>
> >> It would be nice to have a way to incorporate these firmware fixes 
> >> into the upstream code. Since having indecipherable byte-arrays is 
> >> not possible upstream, I propose the following:
> >> - We use the assistance of Realtek to come up with a format which 
> >> the firmware patch files can follow (this can be documented in the 
> >> comments).
> >>       - A real simple format could look like this:
> >>               +
> >>
> <section1><size_in_bytes><address1><data1><address2><data2>...<address
> N
> ><dataN><section2>...
> >>                + The driver would be able to understand how to 
> >> parse each section (e.g is each data entry a byte or a word?)
> >>
> >> - We use request_firmware() to load the firmware, parse it and 
> >> write the data to the relevant registers.

I plan to finish the patches which I am going to submit, first. Then, I could focus on this. However, I don't think I would start this quickly. There are many preparations and they would take me a lot of time.

Best Regards,
Hayes

Re: Proposal: r8152 firmware patching framework

[Prashant Malani]

Hi Bambi,

Thank you for your response. We'd be more than happy to assist in
working out a solution that would be acceptable by the upstream
maintainers.
I think having a maintainable and safe way to deploy firmware fixes
would be much appreciated by hardware users as well as upstream devs,
and certainly more manageable than big static byte-arrays in the
source code!

I've moved David to the TO list to hopefully get his suggestions and
guidance about how to design this in a upstream-compatible way.

I'd be happy to implement it too (I feel this can occur concurrent to
Hayes' upstreaming efforts).

David, could you kindly advise the best way to incorporate deploying
these firmware patches? This change link gives an idea of what we're
dealing with: https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/1417953

My original strawman is to just have a simple firmware format like so:
<section1><size_in_bytes><address1><data1><address2><data2>...<addressN><dataN><section2>

The driver code can have parts to deal with each section in an
appropriate fashion (e.g is each data entry a word or a byte? does
this section have a key which needs to be written to a certain
register etc.)

We'd be grateful if you can offer your advice about best practices (or
suggestions about who might be a good reviewer), so that we can have a
design in place before sending out any patches.

Thanks and best regards,

-Prashant

On Tue, Sep 3, 2019 at 2:01 AM Bambi Yeh <bambi.yeh@realtek.com> wrote:
>
> Hi Prashant:
>
> We will try to implement your requests.
> Based on our experience, upstream reviewer often reject our modification if they have any concern.
> Do you think you can talk to them about this idea and see if they will accept it or not?
> Or if you can help on this after we submit it?
>
> Also, Hayes is now updating our current upstream driver and it goes back and forth for a while.
> So we will need some time to finish it and the target schedule to have your request done is in the end of this month.
>
> Thank you very much.
>
> Best Regards,
> Bambi Yeh
>
> -----Original Message-----
> From: Hayes Wang <hayeswang@realtek.com>
> Sent: Monday, September 2, 2019 2:31 PM
> To: Amber Chen <amber.chen@realtek.com>; Prashant Malani <pmalani@chromium.org>
> Cc: David Miller <davem@davemloft.net>; netdev@vger.kernel.org; Bambi Yeh <bambi.yeh@realtek.com>; Ryankao <ryankao@realtek.com>; Jackc <jackc@realtek.com>; Albertk <albertk@realtek.com>; marcochen@google.com; nic_swsd <nic_swsd@realtek.com>; Grant Grundler <grundler@chromium.org>
> Subject: RE: Proposal: r8152 firmware patching framework
>
> Prashant Malani <pmalani@chromium.org>
> > >
> > > (Adding a few more Realtek folks)
> > >
> > > Friendly ping. Any thoughts / feedback, Realtek folks (and others) ?
> > >
> > >> On Thu, Aug 29, 2019 at 11:40 AM Prashant Malani
> > <pmalani@chromium.org> wrote:
> > >>
> > >> Hi,
> > >>
> > >> The r8152 driver source code distributed by Realtek (on
> > >> www.realtek.com) contains firmware patches. This involves binary
> > >> byte-arrays being written byte/word-wise to the hardware memory
> > >> Example: grundler@chromium.org (cc-ed) has an experimental patch
> > which
> > >> includes the firmware patching code which was distributed with the
> > >> Realtek source :
> > >>
> > https://chromium-review.googlesource.com/c/chromiumos/third_party/kern
> > el
> > /+/1417953
> > >>
> > >> It would be nice to have a way to incorporate these firmware fixes
> > >> into the upstream code. Since having indecipherable byte-arrays is
> > >> not possible upstream, I propose the following:
> > >> - We use the assistance of Realtek to come up with a format which
> > >> the firmware patch files can follow (this can be documented in the
> > >> comments).
> > >>       - A real simple format could look like this:
> > >>               +
> > >>
> > <section1><size_in_bytes><address1><data1><address2><data2>...<address
> > N
> > ><dataN><section2>...
> > >>                + The driver would be able to understand how to
> > >> parse each section (e.g is each data entry a byte or a word?)
> > >>
> > >> - We use request_firmware() to load the firmware, parse it and
> > >> write the data to the relevant registers.
>
> I plan to finish the patches which I am going to submit, first. Then, I could focus on this. However, I don't think I would start this quickly. There are many preparations and they would take me a lot of time.
>
> Best Regards,
> Hayes
>
>

Re: Proposal: r8152 firmware patching framework

[David Miller]

From: Prashant Malani <pmalani@chromium.org>
Date: Tue, 3 Sep 2019 14:32:01 -0700

> I've moved David to the TO list to hopefully get his suggestions and
> guidance about how to design this in a upstream-compatible way.

I am not an expert in this area so please do not solicit my opinion.

Thank you.