Addressing the problem of noisy GPUs under Nouveau

Addressing the problem of noisy GPUs under Nouveau

[Martin Peres]

Hello,

Some users have been complaining for years about their GPU sounding like
a jet engine at take off. Last year, I finally laid my hand on one of
these GPUs and have been trying to fix this issue on and off since then.

After failing to find anything in the HW, I figured out that the duty
cycle set by nvidia's proprietary driver would be way under the expected
value. By randomly changing values in the unknown tables of the vbios, I
found out that there is a fan calibration table at the offset 0x18 in
the BIT P table (version 2).

In this table, I identified 2 major 16 bits parameters at offset 0xa and
0xc[2]. The first one, I named pwm_max, while naming the latter
pwm_offset. As expected, these parameters look like a mapping function
of the form aX + b. However, after gathering more samples, I found out
that the output was not continuous when linearly increasing pwm_offset
[1]. Even more funnily, the period of this square function is linear
with the frequency used for the fan's PWN.

I tried reverse engineering the formula to describe this function, but
failed to find a version that would work perfectly for all PWM
frequency. This is the closest I have got to[3], and I basically stopped
there about a year ago because I could not figure it out and got
frustrated :s.

I started again on this project 2 weeks ago, with the intent of finding
a good-enough solution for nouveau, and modelling the rest of the
equation that that would allow me to compute what duty I should set for
every wanted fan speed (%). I again mostly succeeded... but it would
seem that the interpretation of the table depends on the generation of
chipset (Tesla behaves one way, Fermi+ behaves another way). Also, the
proprietary is not consistent for rules such as what to do when the
computed duty value is going to be lower than 0 or not (sometimes we
clamp it to 0, some times we set it to the same value as the divider,
some times we set it to a slightly lower value than the divider).

I have been trying to cover all edge cases by generating a randomized
set of values for the PWM frequency, pwm_max, and pwm_offset values,
flashed the vbios, and iterate from 0% to 100% fan speed while dumping
the values set by your driver. Using half a million sample points (which
took a week to acquire), my model computes 97% of the values correctly
(ignoring off by ones), while the remaining 3% are worryingly off (by up
to 100%)... It is clear that the code is not trivial and is full of
branching, which makes clean-room reverse engineering a chore.

As a final attempt to make a somewhat complete solution, I tried this
weekend to make a "safe" model that would still make the GPUs quiet. I
managed to improve the pass rate from 97 to 99.6%, but the remaining
failures conflict with my previous findings, which are also way more
prevalent. In the end, the only completely-safe way of driving the fan
is the current behaviour of nouveau...

At this point, I am ready to throw in the towel and hardcode parameters
in nouveau to address the problem of the loudest GPUs, but this is of
course suboptimal. This is why I am asking for your help. Would you have
some documentation about this fan calibration table that could help me
here? Code would be even more appreciated.

Thanks a lot in advance,
Martin

PS: here is most of the code you may want to see:
http://fs.mupuf.org/nvidia/fan_calib/

[1] http://fs.mupuf.org/nvidia/fan_calib/pwm_offset.png
[2] https://github.com/envytools/envytools/blob/master/nvbios/power.c#L333
[3] https://github.com/envytools/envytools/blob/master/nvbios/power.c#L298

_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[John Hubbard]

Hi Martin,

This is just a quick ACK. I've started an internal email thread and
we'll see if we can get back to you soon.

Yes, our thermal and fan control definitely changes a lot which
the various chip architectures. I'm continually impressed by how much
the SW+HW has been able to improve performance per watt, year after
year, but of course the side effect is a very complex system, as 
you are seeing. But even so, let's see if there is any sort of
simpler approximation that would work for you here...no promises,
because I'm about to be humbled when the thermal experts respond. :)


thanks,
John Hubbard
NVIDIA

On 11/12/2017 06:29 PM, Martin Peres wrote:
> Hello,
> 
> Some users have been complaining for years about their GPU sounding like
> a jet engine at take off. Last year, I finally laid my hand on one of
> these GPUs and have been trying to fix this issue on and off since then.
> 
> After failing to find anything in the HW, I figured out that the duty
> cycle set by nvidia's proprietary driver would be way under the expected
> value. By randomly changing values in the unknown tables of the vbios, I
> found out that there is a fan calibration table at the offset 0x18 in
> the BIT P table (version 2).
> 
> In this table, I identified 2 major 16 bits parameters at offset 0xa and
> 0xc[2]. The first one, I named pwm_max, while naming the latter
> pwm_offset. As expected, these parameters look like a mapping function
> of the form aX + b. However, after gathering more samples, I found out
> that the output was not continuous when linearly increasing pwm_offset
> [1]. Even more funnily, the period of this square function is linear
> with the frequency used for the fan's PWN.
> 
> I tried reverse engineering the formula to describe this function, but
> failed to find a version that would work perfectly for all PWM
> frequency. This is the closest I have got to[3], and I basically stopped
> there about a year ago because I could not figure it out and got
> frustrated :s.
> 
> I started again on this project 2 weeks ago, with the intent of finding
> a good-enough solution for nouveau, and modelling the rest of the
> equation that that would allow me to compute what duty I should set for
> every wanted fan speed (%). I again mostly succeeded... but it would
> seem that the interpretation of the table depends on the generation of
> chipset (Tesla behaves one way, Fermi+ behaves another way). Also, the
> proprietary is not consistent for rules such as what to do when the
> computed duty value is going to be lower than 0 or not (sometimes we
> clamp it to 0, some times we set it to the same value as the divider,
> some times we set it to a slightly lower value than the divider).
> 
> I have been trying to cover all edge cases by generating a randomized
> set of values for the PWM frequency, pwm_max, and pwm_offset values,
> flashed the vbios, and iterate from 0% to 100% fan speed while dumping
> the values set by your driver. Using half a million sample points (which
> took a week to acquire), my model computes 97% of the values correctly
> (ignoring off by ones), while the remaining 3% are worryingly off (by up
> to 100%)... It is clear that the code is not trivial and is full of
> branching, which makes clean-room reverse engineering a chore.
> 
> As a final attempt to make a somewhat complete solution, I tried this
> weekend to make a "safe" model that would still make the GPUs quiet. I
> managed to improve the pass rate from 97 to 99.6%, but the remaining
> failures conflict with my previous findings, which are also way more
> prevalent. In the end, the only completely-safe way of driving the fan
> is the current behaviour of nouveau...
> 
> At this point, I am ready to throw in the towel and hardcode parameters
> in nouveau to address the problem of the loudest GPUs, but this is of
> course suboptimal. This is why I am asking for your help. Would you have
> some documentation about this fan calibration table that could help me
> here? Code would be even more appreciated.
> 
> Thanks a lot in advance,
> Martin
> 
> PS: here is most of the code you may want to see:
> http://fs.mupuf.org/nvidia/fan_calib/
> 
> [1] http://fs.mupuf.org/nvidia/fan_calib/pwm_offset.png
> [2] https://github.com/envytools/envytools/blob/master/nvbios/power.c#L333
> [3] https://github.com/envytools/envytools/blob/master/nvbios/power.c#L298
> 
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[John Hubbard]

On 11/12/2017 06:29 PM, Martin Peres wrote:
> Hello,
> 
> Some users have been complaining for years about their GPU sounding like
> a jet engine at take off. Last year, I finally laid my hand on one of
> these GPUs and have been trying to fix this issue on and off since then.

Some early feedback: can you tell us the exact SKUs you have? And are these
production boards with production VBIOSes?  

Normally, it's just our bringup boards that we'd expect to be noisy like 
this, so we're looking for a few more details.

thanks,
John Hubbard
NVIDIA

> 
> After failing to find anything in the HW, I figured out that the duty
> cycle set by nvidia's proprietary driver would be way under the expected
> value. By randomly changing values in the unknown tables of the vbios, I
> found out that there is a fan calibration table at the offset 0x18 in
> the BIT P table (version 2).
> 
> In this table, I identified 2 major 16 bits parameters at offset 0xa and
> 0xc[2]. The first one, I named pwm_max, while naming the latter
> pwm_offset. As expected, these parameters look like a mapping function
> of the form aX + b. However, after gathering more samples, I found out
> that the output was not continuous when linearly increasing pwm_offset
> [1]. Even more funnily, the period of this square function is linear
> with the frequency used for the fan's PWN.
> 
> I tried reverse engineering the formula to describe this function, but
> failed to find a version that would work perfectly for all PWM
> frequency. This is the closest I have got to[3], and I basically stopped
> there about a year ago because I could not figure it out and got
> frustrated :s.
> 
> I started again on this project 2 weeks ago, with the intent of finding
> a good-enough solution for nouveau, and modelling the rest of the
> equation that that would allow me to compute what duty I should set for
> every wanted fan speed (%). I again mostly succeeded... but it would
> seem that the interpretation of the table depends on the generation of
> chipset (Tesla behaves one way, Fermi+ behaves another way). Also, the
> proprietary is not consistent for rules such as what to do when the
> computed duty value is going to be lower than 0 or not (sometimes we
> clamp it to 0, some times we set it to the same value as the divider,
> some times we set it to a slightly lower value than the divider).
> 
> I have been trying to cover all edge cases by generating a randomized
> set of values for the PWM frequency, pwm_max, and pwm_offset values,
> flashed the vbios, and iterate from 0% to 100% fan speed while dumping
> the values set by your driver. Using half a million sample points (which
> took a week to acquire), my model computes 97% of the values correctly
> (ignoring off by ones), while the remaining 3% are worryingly off (by up
> to 100%)... It is clear that the code is not trivial and is full of
> branching, which makes clean-room reverse engineering a chore.
> 
> As a final attempt to make a somewhat complete solution, I tried this
> weekend to make a "safe" model that would still make the GPUs quiet. I
> managed to improve the pass rate from 97 to 99.6%, but the remaining
> failures conflict with my previous findings, which are also way more
> prevalent. In the end, the only completely-safe way of driving the fan
> is the current behaviour of nouveau...
> 
> At this point, I am ready to throw in the towel and hardcode parameters
> in nouveau to address the problem of the loudest GPUs, but this is of
> course suboptimal. This is why I am asking for your help. Would you have
> some documentation about this fan calibration table that could help me
> here? Code would be even more appreciated.
> 
> Thanks a lot in advance,
> Martin
> 
> PS: here is most of the code you may want to see:
> http://fs.mupuf.org/nvidia/fan_calib/
> 
> [1] http://fs.mupuf.org/nvidia/fan_calib/pwm_offset.png
> [2] https://github.com/envytools/envytools/blob/master/nvbios/power.c#L333
> [3] https://github.com/envytools/envytools/blob/master/nvbios/power.c#L298
> 
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[Martin Peres]

Hello,

On 13/11/17 09:15, John Hubbard wrote:
> On 11/12/2017 06:29 PM, Martin Peres wrote:
>> Hello,
>>
>> Some users have been complaining for years about their GPU sounding like
>> a jet engine at take off. Last year, I finally laid my hand on one of
>> these GPUs and have been trying to fix this issue on and off since then.
> 
> Some early feedback: can you tell us the exact SKUs you have? And are these
> production boards with production VBIOSes?  
> 
> Normally, it's just our bringup boards that we'd expect to be noisy like 
> this, so we're looking for a few more details.

Thanks for the quick feedback.

We only have access to production hardware with production vbioses, as
far as I know. In any case, I made all my experiments on the following
GPU (with a stock vbios, albeit modified to perform the experiment):

NVIDIA Corporation GF108 [GeForce GT 620] (rev a1) (prog-if 00 [VGA
controller])
        Subsystem: eVga.com. Corp. Device 2625

I pushed my vbios to http://fs.mupuf.org/nvidia/fan_calib/ if this is
interesting to you (I doubt it, but if that can save us a round trip,
then let's do this :)).

Thanks,
Martin

> 
> thanks,
> John Hubbard
> NVIDIA
> 
>>
>> After failing to find anything in the HW, I figured out that the duty
>> cycle set by nvidia's proprietary driver would be way under the expected
>> value. By randomly changing values in the unknown tables of the vbios, I
>> found out that there is a fan calibration table at the offset 0x18 in
>> the BIT P table (version 2).
>>
>> In this table, I identified 2 major 16 bits parameters at offset 0xa and
>> 0xc[2]. The first one, I named pwm_max, while naming the latter
>> pwm_offset. As expected, these parameters look like a mapping function
>> of the form aX + b. However, after gathering more samples, I found out
>> that the output was not continuous when linearly increasing pwm_offset
>> [1]. Even more funnily, the period of this square function is linear
>> with the frequency used for the fan's PWN.
>>
>> I tried reverse engineering the formula to describe this function, but
>> failed to find a version that would work perfectly for all PWM
>> frequency. This is the closest I have got to[3], and I basically stopped
>> there about a year ago because I could not figure it out and got
>> frustrated :s.
>>
>> I started again on this project 2 weeks ago, with the intent of finding
>> a good-enough solution for nouveau, and modelling the rest of the
>> equation that that would allow me to compute what duty I should set for
>> every wanted fan speed (%). I again mostly succeeded... but it would
>> seem that the interpretation of the table depends on the generation of
>> chipset (Tesla behaves one way, Fermi+ behaves another way). Also, the
>> proprietary is not consistent for rules such as what to do when the
>> computed duty value is going to be lower than 0 or not (sometimes we
>> clamp it to 0, some times we set it to the same value as the divider,
>> some times we set it to a slightly lower value than the divider).
>>
>> I have been trying to cover all edge cases by generating a randomized
>> set of values for the PWM frequency, pwm_max, and pwm_offset values,
>> flashed the vbios, and iterate from 0% to 100% fan speed while dumping
>> the values set by your driver. Using half a million sample points (which
>> took a week to acquire), my model computes 97% of the values correctly
>> (ignoring off by ones), while the remaining 3% are worryingly off (by up
>> to 100%)... It is clear that the code is not trivial and is full of
>> branching, which makes clean-room reverse engineering a chore.
>>
>> As a final attempt to make a somewhat complete solution, I tried this
>> weekend to make a "safe" model that would still make the GPUs quiet. I
>> managed to improve the pass rate from 97 to 99.6%, but the remaining
>> failures conflict with my previous findings, which are also way more
>> prevalent. In the end, the only completely-safe way of driving the fan
>> is the current behaviour of nouveau...
>>
>> At this point, I am ready to throw in the towel and hardcode parameters
>> in nouveau to address the problem of the loudest GPUs, but this is of
>> course suboptimal. This is why I am asking for your help. Would you have
>> some documentation about this fan calibration table that could help me
>> here? Code would be even more appreciated.
>>
>> Thanks a lot in advance,
>> Martin
>>
>> PS: here is most of the code you may want to see:
>> http://fs.mupuf.org/nvidia/fan_calib/
>>
>> [1] http://fs.mupuf.org/nvidia/fan_calib/pwm_offset.png
>> [2] https://github.com/envytools/envytools/blob/master/nvbios/power.c#L333
>> [3] https://github.com/envytools/envytools/blob/master/nvbios/power.c#L298
>>

_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[Andy Ritger]

Hi Martin,

I was asked to clarify a few things:

(1) Are all the user reports of loud fans on Fermi-era GPUs?

(2) When the VBIOS POSTs the card, it loads initial ucode onto the Falcon
processor (PMU), which will do basic fan management on its own.  We call this
init ucode "IFR" (Init From ROM).  nvidia.ko will restore the IFR ucode when
unloaded.  I assume the loud fan symptom occurs after Nouveau is loaded and
running, correct?  I.e., this is a problem in Nouveau's fan control
programming, rather than a problem in IFR.

(3) IFR will run until something else is loaded on the Falcon processor (PMU).
On Fermi, I assume the Nouveau kernel driver is uploading the Nouveau-written
ucode from here:

    drivers/gpu/drm/nouveau/nvkm/subdev/pmu/fuc

correct?  I only ask to rule out the possibility that IFR and Nouveau are both
attempting to program fans simultaneously.  The symptoms you describe don't
sound like that, but just double checking...

(4) Given the PMU ucode debacle, I'm embarrassed to ask, but at least on Fermi,
how much does Nouveau strictly depend on Nouveau's PMU ucode?  Would it be an
option to just let IFR continue to manage fans?

(5) Lastly, I was asked how Nouveau determines what fan speed to (attempt
to) program.

Thanks,
- Andy


On Sun, Nov 12, 2017 at 11:15:45PM -0800, John Hubbard wrote:
> On 11/12/2017 06:29 PM, Martin Peres wrote:
> > Hello,
> > 
> > Some users have been complaining for years about their GPU sounding like
> > a jet engine at take off. Last year, I finally laid my hand on one of
> > these GPUs and have been trying to fix this issue on and off since then.
> 
> Some early feedback: can you tell us the exact SKUs you have? And are these
> production boards with production VBIOSes?  
> 
> Normally, it's just our bringup boards that we'd expect to be noisy like 
> this, so we're looking for a few more details.
> 
> thanks,
> John Hubbard
> NVIDIA
> 
> > 
> > After failing to find anything in the HW, I figured out that the duty
> > cycle set by nvidia's proprietary driver would be way under the expected
> > value. By randomly changing values in the unknown tables of the vbios, I
> > found out that there is a fan calibration table at the offset 0x18 in
> > the BIT P table (version 2).
> > 
> > In this table, I identified 2 major 16 bits parameters at offset 0xa and
> > 0xc[2]. The first one, I named pwm_max, while naming the latter
> > pwm_offset. As expected, these parameters look like a mapping function
> > of the form aX + b. However, after gathering more samples, I found out
> > that the output was not continuous when linearly increasing pwm_offset
> > [1]. Even more funnily, the period of this square function is linear
> > with the frequency used for the fan's PWN.
> > 
> > I tried reverse engineering the formula to describe this function, but
> > failed to find a version that would work perfectly for all PWM
> > frequency. This is the closest I have got to[3], and I basically stopped
> > there about a year ago because I could not figure it out and got
> > frustrated :s.
> > 
> > I started again on this project 2 weeks ago, with the intent of finding
> > a good-enough solution for nouveau, and modelling the rest of the
> > equation that that would allow me to compute what duty I should set for
> > every wanted fan speed (%). I again mostly succeeded... but it would
> > seem that the interpretation of the table depends on the generation of
> > chipset (Tesla behaves one way, Fermi+ behaves another way). Also, the
> > proprietary is not consistent for rules such as what to do when the
> > computed duty value is going to be lower than 0 or not (sometimes we
> > clamp it to 0, some times we set it to the same value as the divider,
> > some times we set it to a slightly lower value than the divider).
> > 
> > I have been trying to cover all edge cases by generating a randomized
> > set of values for the PWM frequency, pwm_max, and pwm_offset values,
> > flashed the vbios, and iterate from 0% to 100% fan speed while dumping
> > the values set by your driver. Using half a million sample points (which
> > took a week to acquire), my model computes 97% of the values correctly
> > (ignoring off by ones), while the remaining 3% are worryingly off (by up
> > to 100%)... It is clear that the code is not trivial and is full of
> > branching, which makes clean-room reverse engineering a chore.
> > 
> > As a final attempt to make a somewhat complete solution, I tried this
> > weekend to make a "safe" model that would still make the GPUs quiet. I
> > managed to improve the pass rate from 97 to 99.6%, but the remaining
> > failures conflict with my previous findings, which are also way more
> > prevalent. In the end, the only completely-safe way of driving the fan
> > is the current behaviour of nouveau...
> > 
> > At this point, I am ready to throw in the towel and hardcode parameters
> > in nouveau to address the problem of the loudest GPUs, but this is of
> > course suboptimal. This is why I am asking for your help. Would you have
> > some documentation about this fan calibration table that could help me
> > here? Code would be even more appreciated.
> > 
> > Thanks a lot in advance,
> > Martin
> > 
> > PS: here is most of the code you may want to see:
> > http://fs.mupuf.org/nvidia/fan_calib/
> > 
> > [1] http://fs.mupuf.org/nvidia/fan_calib/pwm_offset.png
> > [2] https://github.com/envytools/envytools/blob/master/nvbios/power.c#L333
> > [3] https://github.com/envytools/envytools/blob/master/nvbios/power.c#L298
> > 
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[Ilia Mirkin]

On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger@nvidia.com> wrote:
> Hi Martin,

Martin should have complete answers,

>
> I was asked to clarify a few things:
>
> (1) Are all the user reports of loud fans on Fermi-era GPUs?

Yes. Although I believe some GK208 users are also having trouble,
including yours truly. (It's been quite a while since I've checked
though... my memory is weak in that regard.)

>
> (2) When the VBIOS POSTs the card, it loads initial ucode onto the Falcon
> processor (PMU), which will do basic fan management on its own.  We call this
> init ucode "IFR" (Init From ROM).  nvidia.ko will restore the IFR ucode when
> unloaded.  I assume the loud fan symptom occurs after Nouveau is loaded and
> running, correct?  I.e., this is a problem in Nouveau's fan control
> programming, rather than a problem in IFR.

Correct.

>
> (3) IFR will run until something else is loaded on the Falcon processor (PMU).
> On Fermi, I assume the Nouveau kernel driver is uploading the Nouveau-written
> ucode from here:
>
>     drivers/gpu/drm/nouveau/nvkm/subdev/pmu/fuc
>
> correct?  I only ask to rule out the possibility that IFR and Nouveau are both
> attempting to program fans simultaneously.  The symptoms you describe don't
> sound like that, but just double checking...

Correct.

>
> (4) Given the PMU ucode debacle, I'm embarrassed to ask, but at least on Fermi,
> how much does Nouveau strictly depend on Nouveau's PMU ucode?  Would it be an
> option to just let IFR continue to manage fans?

Reclocking is still on our horizon, which clearly won't happen without
nouveau PMU code loaded. Not sure what it's used for until reclocking
becomes a thing on Fermi.

>
> (5) Lastly, I was asked how Nouveau determines what fan speed to (attempt
> to) program.

I'll let Martin answer this, but as you're probably aware, there's 2
different ways this can be done - there might be a PWM, we might have
to toggle it manually. Maybe something else still.

Have a look at drm/nouveau/nvkm/subdev/therm/fan.c and the various
bits it ends up calling (pre-GF119 fermi's end up with the nv50
fan_set, I believe).

The bios stuff is parsed in nvkm/subdev/bios/fan.c and therm.c,
although I believe Martin's latest analysis is more advanced than
what's in that code.

Martin's question was very long, but it boils down to this:

How do we compute the correct values to write into the e114/e118 pwm
registers based on the VBIOS contents and current state of the board
(like temperature).

We generally do this right, but appear to get it extra-wrong for certain GPUs.

Cheers,

  -ilia
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[Karol Herbst]

On Wed, Nov 22, 2017 at 3:06 AM, Ilia Mirkin <imirkin@alum.mit.edu> wrote:
> On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger@nvidia.com> wrote:
>> Hi Martin,
>
> Martin should have complete answers,
>
>>
>> I was asked to clarify a few things:
>>
>> (1) Are all the user reports of loud fans on Fermi-era GPUs?
>
> Yes. Although I believe some GK208 users are also having trouble,
> including yours truly. (It's been quite a while since I've checked
> though... my memory is weak in that regard.)
>

I think there are some Keplers where we drive the fans too loud? Maybe
it got fixed, but I am sure some users complaint about this on Kepler
GPUs.

>>
>> (2) When the VBIOS POSTs the card, it loads initial ucode onto the Falcon
>> processor (PMU), which will do basic fan management on its own.  We call this
>> init ucode "IFR" (Init From ROM).  nvidia.ko will restore the IFR ucode when
>> unloaded.  I assume the loud fan symptom occurs after Nouveau is loaded and
>> running, correct?  I.e., this is a problem in Nouveau's fan control
>> programming, rather than a problem in IFR.
>
> Correct.
>
>>
>> (3) IFR will run until something else is loaded on the Falcon processor (PMU).
>> On Fermi, I assume the Nouveau kernel driver is uploading the Nouveau-written
>> ucode from here:
>>
>>     drivers/gpu/drm/nouveau/nvkm/subdev/pmu/fuc
>>
>> correct?  I only ask to rule out the possibility that IFR and Nouveau are both
>> attempting to program fans simultaneously.  The symptoms you describe don't
>> sound like that, but just double checking...
>
> Correct.
>
>>
>> (4) Given the PMU ucode debacle, I'm embarrassed to ask, but at least on Fermi,
>> how much does Nouveau strictly depend on Nouveau's PMU ucode?  Would it be an
>> option to just let IFR continue to manage fans?
>
> Reclocking is still on our horizon, which clearly won't happen without
> nouveau PMU code loaded. Not sure what it's used for until reclocking
> becomes a thing on Fermi.
>

well I plan to use the PMU for the PMU counters readout code. Not that
it matters much on Fermi...

>>
>> (5) Lastly, I was asked how Nouveau determines what fan speed to (attempt
>> to) program.
>
> I'll let Martin answer this, but as you're probably aware, there's 2
> different ways this can be done - there might be a PWM, we might have
> to toggle it manually. Maybe something else still.
>
> Have a look at drm/nouveau/nvkm/subdev/therm/fan.c and the various
> bits it ends up calling (pre-GF119 fermi's end up with the nv50
> fan_set, I believe).
>
> The bios stuff is parsed in nvkm/subdev/bios/fan.c and therm.c,
> although I believe Martin's latest analysis is more advanced than
> what's in that code.
>
> Martin's question was very long, but it boils down to this:
>
> How do we compute the correct values to write into the e114/e118 pwm
> registers based on the VBIOS contents and current state of the board
> (like temperature).
>
> We generally do this right, but appear to get it extra-wrong for certain GPUs.
>

well short answer is: Nouveau parses the vbios and see what it has to
do. Apparently it is wrong in some cases. I don't think there is
anything else Nouveau tries to do like having its own curves for
calculating fan speeds or so.

> Cheers,
>
>   -ilia
> _______________________________________________
> Nouveau mailing list
> Nouveau@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/nouveau
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[Martin Peres]

Hey,

Thanks for your answer, Andy!

On 22/11/17 04:06, Ilia Mirkin wrote:
> On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger@nvidia.com> wrote:
>> Hi Martin,
> 
> Martin should have complete answers,
> 
>>
>> I was asked to clarify a few things:
>>
>> (1) Are all the user reports of loud fans on Fermi-era GPUs?
> 
> Yes. Although I believe some GK208 users are also having trouble,
> including yours truly. (It's been quite a while since I've checked
> though... my memory is weak in that regard.)

We did not hear back from a lot of users about these issues, but I can
see that most GF108 vbios in our vbios repo are problematic, and some
GK106/GT215/GT216/GT218 might be too.

> 
>>
>> (2) When the VBIOS POSTs the card, it loads initial ucode onto the Falcon
>> processor (PMU), which will do basic fan management on its own.  We call this
>> init ucode "IFR" (Init From ROM).  nvidia.ko will restore the IFR ucode when
>> unloaded.  I assume the loud fan symptom occurs after Nouveau is loaded and
>> running, correct?  I.e., this is a problem in Nouveau's fan control
>> programming, rather than a problem in IFR.
> 
> Correct.

Indeed.

> 
>>
>> (3) IFR will run until something else is loaded on the Falcon processor (PMU).
>> On Fermi, I assume the Nouveau kernel driver is uploading the Nouveau-written
>> ucode from here:
>>
>>     drivers/gpu/drm/nouveau/nvkm/subdev/pmu/fuc
>>
>> correct?  I only ask to rule out the possibility that IFR and Nouveau are both
>> attempting to program fans simultaneously.  The symptoms you describe don't
>> sound like that, but just double checking...
> 
> Correct.

Indeed.

> 
>>
>> (4) Given the PMU ucode debacle, I'm embarrassed to ask, but at least on Fermi,
>> how much does Nouveau strictly depend on Nouveau's PMU ucode?  Would it be an
>> option to just let IFR continue to manage fans?
> 
> Reclocking is still on our horizon, which clearly won't happen without
> nouveau PMU code loaded. Not sure what it's used for until reclocking
> becomes a thing on Fermi.

Yeah, this would hinder our reclocking efforts :s

The best idea I can come up with is to fake the temperature (register
0x20408) to 1°C (minimum the hardware allows us) and read the PWM duty,
then we can get the maximum duty by setting the temperature to the
fan_boost threshold.

Not sure we have a sure-way of computing the fan_boost threshold though,
maybe we can just use of the thermal throttling threshold for this (more
on this later in the email).

In any case, all of these solutions are workarounds. Given that the code
to compute these values is already found in vbioses, why is it a problem
to share the meaning of all the values in the fan calibration table,
and/or the algorithm?

> 
>>
>> (5) Lastly, I was asked how Nouveau determines what fan speed to (attempt
>> to) program.

Oh, thanks for giving me an idea about what the other values in this
table may be about :D

Anyways, the current code uses the entry id 0x46 of the thermal table
(bit P, offset 0x10) to find out what are the thermal points for
$fan_min and $fan_max. The $fan_min and $fan_max values are found in the
entry id 0x22 of the same table.

If the 0x46 entry is not present in the thermal table (which seems to be
the norm for Fermi), we revert to default values: 40 -> 85°C.

With these 4 values, we get 2 trip points (temp_min, fan_min)
and(temp_max, fan_max), and we merely do linear interpolation between them.

> 
> I'll let Martin answer this, but as you're probably aware, there's 2
> different ways this can be done - there might be a PWM, we might have
> to toggle it manually. Maybe something else still.

The manual toggle fans are only present on pre-tesla GPUs, let's ignore
them here, because we know what to do there.

All recent (2006+) GPUs use PWM, and anything after the GT215 use this
fan calibration table which took me a while to find, and that is still
mostly a mystery to me :s

> 
> Have a look at drm/nouveau/nvkm/subdev/therm/fan.c and the various
> bits it ends up calling (pre-GF119 fermi's end up with the nv50
> fan_set, I believe).
> 
> The bios stuff is parsed in nvkm/subdev/bios/fan.c and therm.c,
> although I believe Martin's latest analysis is more advanced than
> what's in that code.

Absolutely :) I have not updated Nouveau yet, in fear of setting a value
lower than what the proprietary driver does...

> 
> Martin's question was very long, but it boils down to this:
> 
> How do we compute the correct values to write into the e114/e118 pwm
> registers based on the VBIOS contents and current state of the board
> (like temperature).

Unfortunately, it can also be the e11c/e120 couple, or 0x200d8/dc on
GF119+, or 0x200cd/d0 on Kepler+.

At least, it looks like we know which PWM controler we need to drive, so
I did not want to muddy the water even more by giving register
addresses, rather concentrating on the problem at hand: How to compute
the duty value for the PWM controler.

> 
> We generally do this right, but appear to get it extra-wrong for certain GPUs.

Yes... So far, we are always safe, but users tend to mind when their
computer sound like a jumbo jet at take off... Who would have thought? :D

Anyway, looking forward to your answer!

Cheers,
Martin
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[John Hubbard]

On 11/22/2017 05:07 PM, Martin Peres wrote:
> Hey,
> 
> Thanks for your answer, Andy!
> 
> On 22/11/17 04:06, Ilia Mirkin wrote:
>> On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger@nvidia.com> wrote:
>> Martin's question was very long, but it boils down to this:
>>
>> How do we compute the correct values to write into the e114/e118 pwm
>> registers based on the VBIOS contents and current state of the board
>> (like temperature).
> 
> Unfortunately, it can also be the e11c/e120 couple, or 0x200d8/dc on
> GF119+, or 0x200cd/d0 on Kepler+.
> 
> At least, it looks like we know which PWM controler we need to drive, so
> I did not want to muddy the water even more by giving register
> addresses, rather concentrating on the problem at hand: How to compute
> the duty value for the PWM controler.
> 
>>
>> We generally do this right, but appear to get it extra-wrong for certain GPUs.
> 
> Yes... So far, we are always safe, but users tend to mind when their
> computer sound like a jumbo jet at take off... Who would have thought? :D
> 
> Anyway, looking forward to your answer!
> 
> Cheers,
> Martin


Hi Martin,

One of our firmware engineers thinks that this looks a lot like PWM inversion.
For some SKUs, the interpretation of the PWM duty cycle is inverted. That 
would probably make it *very* difficult to find a sensible algorithm that 
covered all the SKUs, given that some are inverted and others are not.

For the noisy GPUs, a very useful experiment would be to try inverting it, 
like this:

	pwmDutyCycle = pwmPeriod - pwmDutyCycle;

...and then see if fan control starts behaving closer to how you've actually 
programmed it.

Would that be easy enough to try out? It should help narrow down the
problem at least.

thanks,
John Hubbard
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[Martin Peres]

On 23/11/17 10:06, John Hubbard wrote:
> On 11/22/2017 05:07 PM, Martin Peres wrote:
>> Hey,
>>
>> Thanks for your answer, Andy!
>>
>> On 22/11/17 04:06, Ilia Mirkin wrote:
>>> On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger@nvidia.com> wrote:
>>> Martin's question was very long, but it boils down to this:
>>>
>>> How do we compute the correct values to write into the e114/e118 pwm
>>> registers based on the VBIOS contents and current state of the board
>>> (like temperature).
>>
>> Unfortunately, it can also be the e11c/e120 couple, or 0x200d8/dc on
>> GF119+, or 0x200cd/d0 on Kepler+.
>>
>> At least, it looks like we know which PWM controler we need to drive, so
>> I did not want to muddy the water even more by giving register
>> addresses, rather concentrating on the problem at hand: How to compute
>> the duty value for the PWM controler.
>>
>>>
>>> We generally do this right, but appear to get it extra-wrong for certain GPUs.
>>
>> Yes... So far, we are always safe, but users tend to mind when their
>> computer sound like a jumbo jet at take off... Who would have thought? :D
>>
>> Anyway, looking forward to your answer!
>>
>> Cheers,
>> Martin
> 
> 
> Hi Martin,
> 
> One of our firmware engineers thinks that this looks a lot like PWM inversion.
> For some SKUs, the interpretation of the PWM duty cycle is inverted. That 
> would probably make it *very* difficult to find a sensible algorithm that 
> covered all the SKUs, given that some are inverted and others are not.
> 
> For the noisy GPUs, a very useful experiment would be to try inverting it, 
> like this:
> 
> 	pwmDutyCycle = pwmPeriod - pwmDutyCycle;
> 
> ...and then see if fan control starts behaving closer to how you've actually 
> programmed it.
> 
> Would that be easy enough to try out? It should help narrow down the
> problem at least.
>

Hey John,

Unfortunately, we know about PWM inversion, and one can know which mode
to use based on the GPIO entry associated to the fan (inverted). We have
had support for this in Nouveau for a long time. At the very least, this
is not the problem on my GF108.

I am certain that the problem I am seeing is related to this vbios table
I wrote about (BIT P, offset 0x18). It is used to compute what PWM duty
I should use for both 0 and 100% of the fan speed.

Computing the value for 0% fan speed is difficult because of
non-continuous nature of some of the functions[1], but I can always
over-approximate. However, I failed to accurately compute the duty I
need to write to get the 100% fan speed (I have cases where I greatly
over-estimate it...).

Could you please check out the vbios table I am pointing at? I am quite
sure that your documentation will be clearer than my babbling :D

Thanks,
Martin

[1] http://fs.mupuf.org/nvidia/fan_calib/pwm_offset.png
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[John Hubbard]

On 11/23/2017 02:48 PM, Martin Peres wrote:
> On 23/11/17 10:06, John Hubbard wrote:
>> On 11/22/2017 05:07 PM, Martin Peres wrote:
>>> Hey,
>>>
>>> Thanks for your answer, Andy!
>>>
>>> On 22/11/17 04:06, Ilia Mirkin wrote:
>>>> On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger@nvidia.com> wrote:
>>>> Martin's question was very long, but it boils down to this:
>>>>
>>>> How do we compute the correct values to write into the e114/e118 pwm
>>>> registers based on the VBIOS contents and current state of the board
>>>> (like temperature).
>>>
>>> Unfortunately, it can also be the e11c/e120 couple, or 0x200d8/dc on
>>> GF119+, or 0x200cd/d0 on Kepler+.
>>>
>>> At least, it looks like we know which PWM controler we need to drive, so
>>> I did not want to muddy the water even more by giving register
>>> addresses, rather concentrating on the problem at hand: How to compute
>>> the duty value for the PWM controler.
>>>
>>>>
>>>> We generally do this right, but appear to get it extra-wrong for certain GPUs.
>>>
>>> Yes... So far, we are always safe, but users tend to mind when their
>>> computer sound like a jumbo jet at take off... Who would have thought? :D
>>>
>>> Anyway, looking forward to your answer!
>>>
>>> Cheers,
>>> Martin
>>
>>
>> Hi Martin,
>>
>> One of our firmware engineers thinks that this looks a lot like PWM inversion.
>> For some SKUs, the interpretation of the PWM duty cycle is inverted. That 
>> would probably make it *very* difficult to find a sensible algorithm that 
>> covered all the SKUs, given that some are inverted and others are not.
>>
>> For the noisy GPUs, a very useful experiment would be to try inverting it, 
>> like this:
>>
>> 	pwmDutyCycle = pwmPeriod - pwmDutyCycle;
>>
>> ...and then see if fan control starts behaving closer to how you've actually 
>> programmed it.
>>
>> Would that be easy enough to try out? It should help narrow down the
>> problem at least.
>>
> 
> Hey John,
> 
> Unfortunately, we know about PWM inversion, and one can know which mode
> to use based on the GPIO entry associated to the fan (inverted). We have
> had support for this in Nouveau for a long time. At the very least, this
> is not the problem on my GF108.
> 
> I am certain that the problem I am seeing is related to this vbios table
> I wrote about (BIT P, offset 0x18). It is used to compute what PWM duty
> I should use for both 0 and 100% of the fan speed.
> 
> Computing the value for 0% fan speed is difficult because of
> non-continuous nature of some of the functions[1], but I can always
> over-approximate. However, I failed to accurately compute the duty I
> need to write to get the 100% fan speed (I have cases where I greatly
> over-estimate it...).
> 
> Could you please check out the vbios table I am pointing at? I am quite
> sure that your documentation will be clearer than my babbling :D

Yes. We will check on this. There has been some productive discussion 
internally, but it will take some more investigation.

thanks,
John Hubbard


> 
> Thanks,
> Martin
> 
> [1] http://fs.mupuf.org/nvidia/fan_calib/pwm_offset.png
> 
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[Martin Peres]

On 28/11/17 07:32, John Hubbard wrote:
> On 11/23/2017 02:48 PM, Martin Peres wrote:
>> On 23/11/17 10:06, John Hubbard wrote:
>>> On 11/22/2017 05:07 PM, Martin Peres wrote:
>>>> Hey,
>>>>
>>>> Thanks for your answer, Andy!
>>>>
>>>> On 22/11/17 04:06, Ilia Mirkin wrote:
>>>>> On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger@nvidia.com> wrote:
>>>>> Martin's question was very long, but it boils down to this:
>>>>>
>>>>> How do we compute the correct values to write into the e114/e118 pwm
>>>>> registers based on the VBIOS contents and current state of the board
>>>>> (like temperature).
>>>>
>>>> Unfortunately, it can also be the e11c/e120 couple, or 0x200d8/dc on
>>>> GF119+, or 0x200cd/d0 on Kepler+.
>>>>
>>>> At least, it looks like we know which PWM controler we need to drive, so
>>>> I did not want to muddy the water even more by giving register
>>>> addresses, rather concentrating on the problem at hand: How to compute
>>>> the duty value for the PWM controler.
>>>>
>>>>>
>>>>> We generally do this right, but appear to get it extra-wrong for certain GPUs.
>>>>
>>>> Yes... So far, we are always safe, but users tend to mind when their
>>>> computer sound like a jumbo jet at take off... Who would have thought? :D
>>>>
>>>> Anyway, looking forward to your answer!
>>>>
>>>> Cheers,
>>>> Martin
>>>
>>>
>>> Hi Martin,
>>>
>>> One of our firmware engineers thinks that this looks a lot like PWM inversion.
>>> For some SKUs, the interpretation of the PWM duty cycle is inverted. That 
>>> would probably make it *very* difficult to find a sensible algorithm that 
>>> covered all the SKUs, given that some are inverted and others are not.
>>>
>>> For the noisy GPUs, a very useful experiment would be to try inverting it, 
>>> like this:
>>>
>>> 	pwmDutyCycle = pwmPeriod - pwmDutyCycle;
>>>
>>> ...and then see if fan control starts behaving closer to how you've actually 
>>> programmed it.
>>>
>>> Would that be easy enough to try out? It should help narrow down the
>>> problem at least.
>>>
>>
>> Hey John,
>>
>> Unfortunately, we know about PWM inversion, and one can know which mode
>> to use based on the GPIO entry associated to the fan (inverted). We have
>> had support for this in Nouveau for a long time. At the very least, this
>> is not the problem on my GF108.
>>
>> I am certain that the problem I am seeing is related to this vbios table
>> I wrote about (BIT P, offset 0x18). It is used to compute what PWM duty
>> I should use for both 0 and 100% of the fan speed.
>>
>> Computing the value for 0% fan speed is difficult because of
>> non-continuous nature of some of the functions[1], but I can always
>> over-approximate. However, I failed to accurately compute the duty I
>> need to write to get the 100% fan speed (I have cases where I greatly
>> over-estimate it...).
>>
>> Could you please check out the vbios table I am pointing at? I am quite
>> sure that your documentation will be clearer than my babbling :D
> 
> Yes. We will check on this. There has been some productive discussion 
> internally, but it will take some more investigation.
> 
> thanks,
> John Hubbard

Have the productive discussions panned out?

Thanks in advance,
Martin
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[Martin Peres]

On 29/01/18 01:24, Martin Peres wrote:
> On 28/11/17 07:32, John Hubbard wrote:
>> On 11/23/2017 02:48 PM, Martin Peres wrote:
>>> On 23/11/17 10:06, John Hubbard wrote:
>>>> On 11/22/2017 05:07 PM, Martin Peres wrote:
>>>>> Hey,
>>>>>
>>>>> Thanks for your answer, Andy!
>>>>>
>>>>> On 22/11/17 04:06, Ilia Mirkin wrote:
>>>>>> On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger@nvidia.com> wrote:
>>>>>> Martin's question was very long, but it boils down to this:
>>>>>>
>>>>>> How do we compute the correct values to write into the e114/e118 pwm
>>>>>> registers based on the VBIOS contents and current state of the board
>>>>>> (like temperature).
>>>>>
>>>>> Unfortunately, it can also be the e11c/e120 couple, or 0x200d8/dc on
>>>>> GF119+, or 0x200cd/d0 on Kepler+.
>>>>>
>>>>> At least, it looks like we know which PWM controler we need to drive, so
>>>>> I did not want to muddy the water even more by giving register
>>>>> addresses, rather concentrating on the problem at hand: How to compute
>>>>> the duty value for the PWM controler.
>>>>>
>>>>>>
>>>>>> We generally do this right, but appear to get it extra-wrong for certain GPUs.
>>>>>
>>>>> Yes... So far, we are always safe, but users tend to mind when their
>>>>> computer sound like a jumbo jet at take off... Who would have thought? :D
>>>>>
>>>>> Anyway, looking forward to your answer!
>>>>>
>>>>> Cheers,
>>>>> Martin
>>>>
>>>>
>>>> Hi Martin,
>>>>
>>>> One of our firmware engineers thinks that this looks a lot like PWM inversion.
>>>> For some SKUs, the interpretation of the PWM duty cycle is inverted. That 
>>>> would probably make it *very* difficult to find a sensible algorithm that 
>>>> covered all the SKUs, given that some are inverted and others are not.
>>>>
>>>> For the noisy GPUs, a very useful experiment would be to try inverting it, 
>>>> like this:
>>>>
>>>> 	pwmDutyCycle = pwmPeriod - pwmDutyCycle;
>>>>
>>>> ...and then see if fan control starts behaving closer to how you've actually 
>>>> programmed it.
>>>>
>>>> Would that be easy enough to try out? It should help narrow down the
>>>> problem at least.
>>>>
>>>
>>> Hey John,
>>>
>>> Unfortunately, we know about PWM inversion, and one can know which mode
>>> to use based on the GPIO entry associated to the fan (inverted). We have
>>> had support for this in Nouveau for a long time. At the very least, this
>>> is not the problem on my GF108.
>>>
>>> I am certain that the problem I am seeing is related to this vbios table
>>> I wrote about (BIT P, offset 0x18). It is used to compute what PWM duty
>>> I should use for both 0 and 100% of the fan speed.
>>>
>>> Computing the value for 0% fan speed is difficult because of
>>> non-continuous nature of some of the functions[1], but I can always
>>> over-approximate. However, I failed to accurately compute the duty I
>>> need to write to get the 100% fan speed (I have cases where I greatly
>>> over-estimate it...).
>>>
>>> Could you please check out the vbios table I am pointing at? I am quite
>>> sure that your documentation will be clearer than my babbling :D
>>
>> Yes. We will check on this. There has been some productive discussion 
>> internally, but it will take some more investigation.
>>
>> thanks,
>> John Hubbard
> 
> Have the productive discussions panned out?

Oh, I see you pushed new vbios documentation:
 1)
http://download.nvidia.com/open-gpu-doc/BIOS-Information-Table/1/BIOS-Information-Table.html
 2)
http://download.nvidia.com/open-gpu-doc/MemoryClockTable/1/MemoryClockTable.html
 3)
http://download.nvidia.com/open-gpu-doc/MemoryTweakTable/1/MemoryTweakTable.html

Is there any chance to get the documentation of the "Thermal Coolers
Table", and the "Thermal Device Table" (the latter does not seem super
important though).

Anyway, thanks for the new documentation, reverse engineering of power
management will be greatly simplified as we have a better idea which
bits will control what. Too bad it won't help for the current issue
though...

Thanks,
Martin
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[John Hubbard]

On 01/28/2018 04:05 PM, Martin Peres wrote:
> On 29/01/18 01:24, Martin Peres wrote:
>> On 28/11/17 07:32, John Hubbard wrote:
>>> On 11/23/2017 02:48 PM, Martin Peres wrote:
>>>> On 23/11/17 10:06, John Hubbard wrote:
>>>>> On 11/22/2017 05:07 PM, Martin Peres wrote:
>>>>>> Hey,
>>>>>>
>>>>>> Thanks for your answer, Andy!
>>>>>>
>>>>>> On 22/11/17 04:06, Ilia Mirkin wrote:
>>>>>>> On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger@nvidia.com> wrote:
>>>>>>> Martin's question was very long, but it boils down to this:
>>>>>>>
>>>>>>> How do we compute the correct values to write into the e114/e118 pwm
>>>>>>> registers based on the VBIOS contents and current state of the board
>>>>>>> (like temperature).
>>>>>>
>>>>>> Unfortunately, it can also be the e11c/e120 couple, or 0x200d8/dc on
>>>>>> GF119+, or 0x200cd/d0 on Kepler+.
>>>>>>
>>>>>> At least, it looks like we know which PWM controler we need to drive, so
>>>>>> I did not want to muddy the water even more by giving register
>>>>>> addresses, rather concentrating on the problem at hand: How to compute
>>>>>> the duty value for the PWM controler.
>>>>>>
>>>>>>>
>>>>>>> We generally do this right, but appear to get it extra-wrong for certain GPUs.
>>>>>>
>>>>>> Yes... So far, we are always safe, but users tend to mind when their
>>>>>> computer sound like a jumbo jet at take off... Who would have thought? :D
>>>>>>
>>>>>> Anyway, looking forward to your answer!
>>>>>>
>>>>>> Cheers,
>>>>>> Martin
>>>>>
>>>>>
>>>>> Hi Martin,
>>>>>
>>>>> One of our firmware engineers thinks that this looks a lot like PWM inversion.
>>>>> For some SKUs, the interpretation of the PWM duty cycle is inverted. That 
>>>>> would probably make it *very* difficult to find a sensible algorithm that 
>>>>> covered all the SKUs, given that some are inverted and others are not.
>>>>>
>>>>> For the noisy GPUs, a very useful experiment would be to try inverting it, 
>>>>> like this:
>>>>>
>>>>> 	pwmDutyCycle = pwmPeriod - pwmDutyCycle;
>>>>>
>>>>> ...and then see if fan control starts behaving closer to how you've actually 
>>>>> programmed it.
>>>>>
>>>>> Would that be easy enough to try out? It should help narrow down the
>>>>> problem at least.
>>>>>
>>>>
>>>> Hey John,
>>>>
>>>> Unfortunately, we know about PWM inversion, and one can know which mode
>>>> to use based on the GPIO entry associated to the fan (inverted). We have
>>>> had support for this in Nouveau for a long time. At the very least, this
>>>> is not the problem on my GF108.
>>>>
>>>> I am certain that the problem I am seeing is related to this vbios table
>>>> I wrote about (BIT P, offset 0x18). It is used to compute what PWM duty
>>>> I should use for both 0 and 100% of the fan speed.
>>>>
>>>> Computing the value for 0% fan speed is difficult because of
>>>> non-continuous nature of some of the functions[1], but I can always
>>>> over-approximate. However, I failed to accurately compute the duty I
>>>> need to write to get the 100% fan speed (I have cases where I greatly
>>>> over-estimate it...).
>>>>
>>>> Could you please check out the vbios table I am pointing at? I am quite
>>>> sure that your documentation will be clearer than my babbling :D
>>>
>>> Yes. We will check on this. There has been some productive discussion 
>>> internally, but it will take some more investigation.
>>>
>>> thanks,
>>> John Hubbard
>>
>> Have the productive discussions panned out?

Yes, we concluded our discussions, and decided that I should study the situation 
and write some documentation.  I just finished my research and writeup late last Friday, 
though, so my colleagues haven't had a chance to review it. Not to put undue
pressure on them, but I'm hoping that will go quickly now. The long pole is
done. :)

I was going to wait until the review was done, to respond, but I wanted to ACK 
this and to let you know that I do realize that the tables below are not directly 
answering your question.

(What happened here is: the new tables below are not actually what I've 
personally been working on; they just happen to be a very good set of supporting 
documentation in the exact same area. One of our teammates was already working 
on these independently, and managed to get them released.)

thanks,
-- 
John Hubbard

> 
> Oh, I see you pushed new vbios documentation:
>  1)
> http://download.nvidia.com/open-gpu-doc/BIOS-Information-Table/1/BIOS-Information-Table.html
>  2)
> http://download.nvidia.com/open-gpu-doc/MemoryClockTable/1/MemoryClockTable.html
>  3)
> http://download.nvidia.com/open-gpu-doc/MemoryTweakTable/1/MemoryTweakTable.html
> 
> Is there any chance to get the documentation of the "Thermal Coolers
> Table", and the "Thermal Device Table" (the latter does not seem super
> important though).
> 
> Anyway, thanks for the new documentation, reverse engineering of power
> management will be greatly simplified as we have a better idea which
> bits will control what. Too bad it won't help for the current issue
> though...
> 
> Thanks,
> Martin
> 
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[Martin Peres]

On 29/01/18 09:51, John Hubbard wrote:
> On 01/28/2018 04:05 PM, Martin Peres wrote:
>> On 29/01/18 01:24, Martin Peres wrote:
>>> On 28/11/17 07:32, John Hubbard wrote:
>>>> On 11/23/2017 02:48 PM, Martin Peres wrote:
>>>>> On 23/11/17 10:06, John Hubbard wrote:
>>>>>> On 11/22/2017 05:07 PM, Martin Peres wrote:
>>>>>>> Hey,
>>>>>>>
>>>>>>> Thanks for your answer, Andy!
>>>>>>>
>>>>>>> On 22/11/17 04:06, Ilia Mirkin wrote:
>>>>>>>> On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger@nvidia.com> wrote:
>>>>>>>> Martin's question was very long, but it boils down to this:
>>>>>>>>
>>>>>>>> How do we compute the correct values to write into the e114/e118 pwm
>>>>>>>> registers based on the VBIOS contents and current state of the board
>>>>>>>> (like temperature).
>>>>>>>
>>>>>>> Unfortunately, it can also be the e11c/e120 couple, or 0x200d8/dc on
>>>>>>> GF119+, or 0x200cd/d0 on Kepler+.
>>>>>>>
>>>>>>> At least, it looks like we know which PWM controler we need to drive, so
>>>>>>> I did not want to muddy the water even more by giving register
>>>>>>> addresses, rather concentrating on the problem at hand: How to compute
>>>>>>> the duty value for the PWM controler.
>>>>>>>
>>>>>>>>
>>>>>>>> We generally do this right, but appear to get it extra-wrong for certain GPUs.
>>>>>>>
>>>>>>> Yes... So far, we are always safe, but users tend to mind when their
>>>>>>> computer sound like a jumbo jet at take off... Who would have thought? :D
>>>>>>>
>>>>>>> Anyway, looking forward to your answer!
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Martin
>>>>>>
>>>>>>
>>>>>> Hi Martin,
>>>>>>
>>>>>> One of our firmware engineers thinks that this looks a lot like PWM inversion.
>>>>>> For some SKUs, the interpretation of the PWM duty cycle is inverted. That 
>>>>>> would probably make it *very* difficult to find a sensible algorithm that 
>>>>>> covered all the SKUs, given that some are inverted and others are not.
>>>>>>
>>>>>> For the noisy GPUs, a very useful experiment would be to try inverting it, 
>>>>>> like this:
>>>>>>
>>>>>> 	pwmDutyCycle = pwmPeriod - pwmDutyCycle;
>>>>>>
>>>>>> ...and then see if fan control starts behaving closer to how you've actually 
>>>>>> programmed it.
>>>>>>
>>>>>> Would that be easy enough to try out? It should help narrow down the
>>>>>> problem at least.
>>>>>>
>>>>>
>>>>> Hey John,
>>>>>
>>>>> Unfortunately, we know about PWM inversion, and one can know which mode
>>>>> to use based on the GPIO entry associated to the fan (inverted). We have
>>>>> had support for this in Nouveau for a long time. At the very least, this
>>>>> is not the problem on my GF108.
>>>>>
>>>>> I am certain that the problem I am seeing is related to this vbios table
>>>>> I wrote about (BIT P, offset 0x18). It is used to compute what PWM duty
>>>>> I should use for both 0 and 100% of the fan speed.
>>>>>
>>>>> Computing the value for 0% fan speed is difficult because of
>>>>> non-continuous nature of some of the functions[1], but I can always
>>>>> over-approximate. However, I failed to accurately compute the duty I
>>>>> need to write to get the 100% fan speed (I have cases where I greatly
>>>>> over-estimate it...).
>>>>>
>>>>> Could you please check out the vbios table I am pointing at? I am quite
>>>>> sure that your documentation will be clearer than my babbling :D
>>>>
>>>> Yes. We will check on this. There has been some productive discussion 
>>>> internally, but it will take some more investigation.
>>>>
>>>> thanks,
>>>> John Hubbard
>>>
>>> Have the productive discussions panned out?
> 
> Yes, we concluded our discussions, and decided that I should study the situation 
> and write some documentation.  I just finished my research and writeup late last Friday, 
> though, so my colleagues haven't had a chance to review it. Not to put undue
> pressure on them, but I'm hoping that will go quickly now. The long pole is
> done. :)
> 
> I was going to wait until the review was done, to respond, but I wanted to ACK 
> this and to let you know that I do realize that the tables below are not directly 
> answering your question.
> 
> (What happened here is: the new tables below are not actually what I've 
> personally been working on; they just happen to be a very good set of supporting 
> documentation in the exact same area. One of our teammates was already working 
> on these independently, and managed to get them released.)

Thanks for the information and your work, it is greatly appreciated.

No need to hurry, I will be away from home for 2 weeks.

Thanks,
Martin
> 
> thanks,
> 

_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[John Hubbard]

[-- Attachment #1: Type: text/plain, Size: 18501 bytes --]

On 01/28/2018 04:05 PM, Martin Peres wrote:
> On 29/01/18 01:24, Martin Peres wrote:
>> On 28/11/17 07:32, John Hubbard wrote:
>>> On 11/23/2017 02:48 PM, Martin Peres wrote:
>>>> On 23/11/17 10:06, John Hubbard wrote:
>>>>> On 11/22/2017 05:07 PM, Martin Peres wrote:
>>>>>> Hey,
>>>>>>
>>>>>> Thanks for your answer, Andy!
>>>>>>
>>>>>> On 22/11/17 04:06, Ilia Mirkin wrote:
>>>>>>> On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger-DDmLM1+adcrQT0dZR+AlfA@public.gmane.org> wrote:
>>>>>>> Martin's question was very long, but it boils down to this:
>>>>>>>
>>>>>>> How do we compute the correct values to write into the e114/e118 pwm
>>>>>>> registers based on the VBIOS contents and current state of the board
>>>>>>> (like temperature).
>>>>>>
>>>>>> Unfortunately, it can also be the e11c/e120 couple, or 0x200d8/dc on
>>>>>> GF119+, or 0x200cd/d0 on Kepler+.
>>>>>>
>>>>>> At least, it looks like we know which PWM controler we need to drive, so
>>>>>> I did not want to muddy the water even more by giving register
>>>>>> addresses, rather concentrating on the problem at hand: How to compute
>>>>>> the duty value for the PWM controler.
>>>>>>
>>>>>>>
>>>>>>> We generally do this right, but appear to get it extra-wrong for certain GPUs.
>>>>>>
>>>>>> Yes... So far, we are always safe, but users tend to mind when their
>>>>>> computer sound like a jumbo jet at take off... Who would have thought? :D
>>>>>>
>>>>>> Anyway, looking forward to your answer!
>>>>>>
>>>>>> Cheers,
>>>>>> Martin
>>>>>
[...]

Hi Martin,

I strongly suspect you are seeing a special behavior, which is: on
some GF108 boards we use only a very limited range of PWM,
0.4 to 2.5%, due to the particular type of DC power conversion
circuit on those boards. However, it could also just be difficulties
in interpreting the fixed-point variables in the tables. In either
case, the answer is to explain those formats, so I'll do that now.

I am attaching the fan cooler table, in HTML format. We have also
published the BIT (BIOS Information Table) format, separately:

    http://download.nvidia.com/open-gpu-doc/BIOS-Information-Table/1/BIOS-Information-Table.html

, but I don't think it has any surprises for you, in this regard. You
can check it, to be sure you're looking at the right subtable, though,
just in case.

The interesting parts of that table are:

PWM Scale Slope (16 bits):

  Slope to scale effective PWM to actual PWM (1/4096, F4.12, signed).
  For backwards compatibility, a value of 0.0 (0x0000) is interpreted as 1.0 (0x1000).
  This value is used to scale the effective PWM duty cycle, a conceptual fraction
  of full speed (0% to 100%), to the actual electrical PWM duty cycle.
  PWM(actual) = Slope × PWM(effective) + Offset

PWM Scale Offset (16 bits):

  Offset to scale effective PWM to actual PWM (1/4096, F4.12, signed).
  This value is used to scale the effective PWM duty cycle, a conceptual fraction
  of full speed (0% to 100%), to the actual electrical PWM duty cycle.
  PWM(actual) = Slope × PWM(effective) + Offset


However, the calculations are hard to get right, and the table stores
values in fixed-point format, so I'm showing a few simplified code excerpts
that use these. The various fixed point macro definitions are found as part of
our normal driver package, in nvmisc.h and nvtypes.h. Any other definitions
that you need are included right here (I ran a quick compiler check to be sure.)

#define VBIOS_THERM_COOLER_TABLE_10_ENTRY_SIZE_10                    0x00000010

// Fan Cooler Table Version
#define NV_FAN_COOLER_TABLE_V2_VERSION_10                             0x10

// We limit the size of V2 tables
#define NV_FAN_COOLER_TABLE_V2_MAX_ENTRIES                            10

// Entry skip.
#define NV_FAN_COOLER_TABLE_V2_SKIP_ENTRY                             0x000000FF

#define NV_FAN_COOLER_TABLE_TYPE                                      3:0       // field1- dword
#define NV_FAN_COOLER_TABLE_TYPE_PASSIVE_HEAT_SINK                    0x00000000
#define NV_FAN_COOLER_TABLE_TYPE_ACTIVE_FAN_SINK                      0x00000001
#define NV_FAN_COOLER_TABLE_TYPE_ACTIVE_SKIP                          0x0000000F
#define NV_FAN_COOLER_TABLE_TYPE_DEFAULT                              NV_FAN_COOLER_TABLE_TYPE_PASSIVE_HEAT_SINK

#define NV_FAN_COOLER_TABLE_TARGET_AFFINITY                           6:4
#define NV_FAN_COOLER_TABLE_TARGET_AFFINITY_GPU                       0x00000000
#define NV_FAN_COOLER_TABLE_TARGET_AFFINITY_ALL                       0x00000001
#define NV_FAN_COOLER_TABLE_TARGET_AFFINITY_DEFAULT                   NV_FAN_COOLER_TABLE_TARGET_AFFINITY_GPU

#define NV_FAN_COOLER_TABLE_CONTROL_DEVICE                            10:8
#define NV_FAN_COOLER_TABLE_CONTROL_DEVICE_NONE                       0x00000000
#define NV_FAN_COOLER_TABLE_CONTROL_DEVICE_GPU                        0x00000001
#define NV_FAN_COOLER_TABLE_CONTROL_DEVICE_EXTERNAL                   0x00000002
#define NV_FAN_COOLER_TABLE_CONTROL_DEVICE_DEFAULT                    NV_FAN_COOLER_TABLE_CONTROL_DEVICE_NONE

#define NV_FAN_COOLER_TABLE_TACHOMETER_DEVICE                         14:12
#define NV_FAN_COOLER_TABLE_TACHOMETER_DEVICE_NONE                    0x00000000
#define NV_FAN_COOLER_TABLE_TACHOMETER_DEVICE_GPU                     0x00000001
#define NV_FAN_COOLER_TABLE_TACHOMETER_DEVICE_EXTERNAL_INSTANCE0      0x00000002
#define NV_FAN_COOLER_TABLE_TACHOMETER_DEVICE_DEFAULT                 NV_FAN_COOLER_TABLE_TACHOMETER_DEVICE_NONE

#define NV_FAN_COOLER_TABLE_CONTROL_SPEED_MAXIMUM                     25:16

#define NV_FAN_COOLER_TABLE_CONTROL_SIGNAL                            29:26
#define NV_FAN_COOLER_TABLE_CONTROL_SIGNAL_NONE                       0x00000000
#define NV_FAN_COOLER_TABLE_CONTROL_SIGNAL_UNKNOWN                    0x00000001
#define NV_FAN_COOLER_TABLE_CONTROL_SIGNAL_FAN_SPECIFIC_INSTANCE0     0x00000002
#define NV_FAN_COOLER_TABLE_CONTROL_SIGNAL_GPIO_FAN_FUNC_INSTANCE0    0x00000003
#define NV_FAN_COOLER_TABLE_CONTROL_SIGNAL_DEFAULT                    NV_FAN_COOLER_TABLE_CONTROL_SIGNAL_NONE

#define NV_FAN_COOLER_TABLE_CONTROL_POLARITY                          31:30
#define NV_FAN_COOLER_TABLE_CONTROL_POLARITY_GPIO                     0x00000000
#define NV_FAN_COOLER_TABLE_CONTROL_POLARITY_LOW                      0x00000001
#define NV_FAN_COOLER_TABLE_CONTROL_POLARITY_HIGH                     0x00000002
#define NV_FAN_COOLER_TABLE_CONTROL_POLARITY_DEFAULT                  NV_FAN_COOLER_TABLE_CONTROL_POLARITY_GPIO

#define NV_FAN_COOLER_TABLE_CONTROL_SPEED_MINIMUM                     9:0      // field2- dword

#define NV_FAN_COOLER_TABLE_TACHOMETER_SIGNAL                         13:10
#define NV_FAN_COOLER_TABLE_TACHOMETER_SIGNAL_NONE                    0x00000000
#define NV_FAN_COOLER_TABLE_TACHOMETER_SIGNAL_UNKNOWN                 0x00000001
#define NV_FAN_COOLER_TABLE_TACHOMETER_SIGNAL_TACH_SPECIFIC_INSTANCE0 0x00000002
#define NV_FAN_COOLER_TABLE_TACHOMETER_SIGNAL_GPIO_TACH_FUNC_INSTANCE0 0x00000003
#define NV_FAN_COOLER_TABLE_TACHOMETER_SIGNAL_DEFAULT                 NV_FAN_COOLER_TABLE_TACHOMETER_SIGNAL_NONE

#define NV_FAN_COOLER_TABLE_TACHOMETER_RATE                           15:14
#define NV_FAN_COOLER_TABLE_TACHOMETER_RATE_1                         0x00000000
#define NV_FAN_COOLER_TABLE_TACHOMETER_RATE_2                         0x00000001
#define NV_FAN_COOLER_TABLE_TACHOMETER_RATE_3                         0x00000002
#define NV_FAN_COOLER_TABLE_TACHOMETER_RATE_4                         0x00000003
#define NV_FAN_COOLER_TABLE_TACHOMETER_RATE_DEFAULT                   NV_FAN_COOLER_TABLE_TACHOMETER_RATE_1

#define NV_FAN_COOLER_TABLE_PWM_MINIMUM                               22:16

#define NV_FAN_COOLER_TABLE_CONTROL_STOP                              23:23
#define NV_FAN_COOLER_TABLE_CONTROL_STOP_PWM                          0x00000000
#define NV_FAN_COOLER_TABLE_CONTROL_STOP_POWER                        0x00000001
#define NV_FAN_COOLER_TABLE_CONTROL_STOP_DEFAULT                      NV_FAN_COOLER_TABLE_CONTROL_STOP_PWM

#define NV_FAN_COOLER_TABLE_PWM_START                                 30:24

#define NV_FAN_COOLER_TABLE_PWM_FREQUENCY                             11:0     // field3 - dword
#define NV_FAN_COOLER_TABLE_PWM_FREQUENCY_UNDEFINED                   0x00000000

#define NV_FAN_COOLER_TABLE_FIELD3_RSVD                               15:12

#define NV_FAN_COOLER_TABLE_FIELD3_PWM_SCALE_SLOPE                    31:16

#define NV_FAN_COOLER_TABLE_FIELD4_PWM_SCALE_OFFSET                   15:0     // field4 - dword

#define NV_FAN_COOLER_TABLE_FIELD4_LOW_ENDPOINT_EXPECTED_ERROR        23:16

#define NV_FAN_COOLER_TABLE_FIELD4_INTERPOLATION_EXPECTED_ERROR       31:24

#define NV_FAN_COOLER_TABLE_FIELD5_HIGH_ENDPOINT_EXPECTED_ERROR       7:0

#define NV_FAN_COOLER_TABLE_FIELD5_RSVD                               31:8

// Fan Cooler Table entry

typedef struct
{
    NvU32 field1;
    NvU32 field2;
    NvU32 field3;
    NvU32 field4;
    NvU32 field5;
} FAN_COOLER_TABLEENTRY;

// Fan Cooler Table
typedef struct
{
    NvU32                   version;
    NvU32                   entrySize;
    NvU32                   entryCount;
    FAN_COOLER_TABLEENTRY   entries[NV_FAN_COOLER_TABLE_V2_MAX_ENTRIES];
} FAN_COOLER_TABLE, *PFAN_COOLER_TABLE;

// Default minimum fan level - *cannot* be overriden by VBIOS:
#define FAN_SPEEDCONTROL_MINIMUM_LEVEL_DEFAULT          30

// Default maximum fan level - can be overriden by VBIOS:
#define FAN_SPEEDCONTROL_MAXIMUM_LEVEL_DEFAULT          100

/*!
 * Scales a PWM ratio (raw duty cycle / period) to a functional percentage,
 * which more adequately represents the percent of full phsyical fan speed on
 * the fan.
 *
 * @param[in] fanPwmScaleOffset   Fan PWM scale offset, in F4.12
 * @param[in] fanPwmScaleSlope    Fan PWM scale slope, in F4.12
 * @param[in] pwmRatio    PWM ratio in F16.16.  However, expected values are in
 *            the range [0,1] so it's actually F1.16.
 *
 * @return Scaled percent in F16.16 - Note, this is an actual percentage stored
 *         in the fractional part.  The value is not scaled up by 100, as
 *         elsewhere in the RM.
 */
static NvUFXP16_16
fanPwmScalePwmRatioToPct
(
    NvSFXP4_12 fanPwmScaleOffset,
    NvSFXP4_12 fanPwmScaleSlope,
    NvUFXP16_16 pwmRatio
)
{
    NvUFXP16_16 pwmPct;

    if (fanPwmScaleSlope == 0)
    {
        // Various logging/tracing actions here...
        return 0;
    }

    //
    //   (F1.16 << 12) - (F4.12 << 16) => F4.28
    // / F4.12                         => F4.12
    // ----------------------------------------
    //                                    F4.16
    NvSFXP16_16 signedPwmPct = ((pwmRatio << 12) -
                                (((NvS32) fanPwmScaleOffset) << 16) +
                                 (fanPwmScaleSlope / 2)) /
                               fanPwmScaleSlope;

    if (signedPwmPct > NV_TYPES_S32_TO_SFXP_X_Y(16, 16, 1))
    {
        pwmPct = NV_TYPES_U32_TO_UFXP_X_Y(16, 16, 1);
    }
    else if (signedPwmPct < NV_TYPES_S32_TO_SFXP_X_Y(16, 16, 0))
    {
        pwmPct = NV_TYPES_U32_TO_UFXP_X_Y(16, 16, 0);
    }
    else
    {
        pwmPct = (NvUFXP16_16) signedPwmPct;
    }

    return pwmPct;
}

/*!
 * Scales a functional percentage (represing the percent of full physical fan
 * speed) to a PWM ratio (raw duty cycle / period).
 *
 * @param[in] fanPwmScaleOffset   Fan PWM scale offset, in F4.12
 * @param[in] fanPwmScaleSlope    Fan PWM scale slope, in F4.12
 * @param[in] pwmPct      Percent to scale in F16.16.  Note, this is an actual
 *            percentage stored in the fractional part.  The value is not scaled
 *            up by 100, as elsewhere in the RM.  So expected values are in the
 *            range [0,1] and actual value is really F.16.
 *
 * @return Scaled pwm ratio in F16.16
 */
static NvUFXP16_16
fanPwmScalePctToPwmRatio
(
    NvSFXP4_12 fanPwmScaleOffset,
    NvSFXP4_12 fanPwmScaleSlope,
    NvUFXP16_16 pwmPct
)
{
    NvUFXP16_16 pwmRatio;

    //
    //   (F1.16 * F4.12) >> 12  => F4.28 => F4.16
    // + F4.12 << 4                      => F4.16
    // ------------------------------------------
    //                                      F4.16
    //
    NvSFXP16_16 signedPwmRatio = pwmPct * fanPwmScaleSlope;
    signedPwmRatio = (signedPwmRatio >> 12) +
                     (DRF_VAL(_TYPES, _FXP, _FRACTIONAL_MSB(4, 12),
                            signedPwmRatio)) +
                     (((NvSFXP16_16) fanPwmScaleOffset) << 4);

    if (signedPwmRatio > NV_TYPES_S32_TO_SFXP_X_Y(16, 16, 1))
    {
        pwmRatio = NV_TYPES_U32_TO_UFXP_X_Y(16, 16, 1);
    }
    else if (signedPwmRatio < NV_TYPES_S32_TO_SFXP_X_Y(16, 16, 0))
    {
        pwmRatio = NV_TYPES_U32_TO_UFXP_X_Y(16, 16, 0);
    }
    else
    {
        pwmRatio = (NvUFXP16_16) signedPwmRatio;
    }

    return pwmRatio;
}

/*!
 * Parses the PWM Scale slope and offset from the Fan Coolers Table.
 *
 * @param[in]   pTable          FAN_COOLER_TABLE pointer from which to pull the
 *                              information.
 * @param[out]  pPwmScaleSlope  Pointer in which to return the slope
 * @param[out]  pPwmScaleOffset Pointer in which to return the offset
 *
 * @return NV_OK if values successfully returned in the pointers
 * @return NV_ERR_INVALID_ARGMENT if NULL pointers are provided.
 * @return NV_ERR_NOT_SUPPORTED if this Fan Coolers Table does not support
 *         these fields.
 */

NV_STATUS
fanCoolerTableGetPwmScale
(
    PFAN_COOLER_TABLE   pTable,
    NvSFXP4_12         *pPwmScaleSlope,
    NvSFXP4_12         *pPwmScaleOffset
)
{
    NV_STATUS status = NV_ERR_NOT_SUPPORTED;
    NvU32     i;

    if ((pPwmScaleSlope == NULL) ||
        (pPwmScaleOffset == NULL))
    {
       status = NV_ERR_INVALID_ARGUMENT;
       goto fanCoolerTableGetPwmScale_exit;
    }

    //
    // Make sure that we know the table supports this field.
    //
    // Ideally, this should have been abstracted out in the devinit
    // function to parse this table, so that our storage isn't
    // version-dependent.
    //
    if ((pTable->version >= NV_FAN_COOLER_TABLE_V2_VERSION_10) &&
        (pTable->entrySize >= VBIOS_THERM_COOLER_TABLE_10_ENTRY_SIZE_10))
    {
        for (i = 0; i < pTable->entryCount; i++)
        {
            // Slope of zero is unsupported, and the error case for this data.
            if ((FLD_TEST_DRF(_FAN, _COOLER_TABLE, _TYPE, _ACTIVE_FAN_SINK,
                              pTable->entries[i].field1)) &&
                (FLD_TEST_DRF(_FAN, _COOLER_TABLE, _CONTROL_DEVICE, _GPU,
                              pTable->entries[i].field1)))
            {
                //
                // Scale of zero is an invalid/unexpected case.  This is a major
                // failure.
                //
                if (FLD_TEST_DRF_NUM(_FAN_COOLER_TABLE, _FIELD3,
                      _PWM_SCALE_SLOPE, NV_TYPES_S32_TO_SFXP_X_Y(4, 12, 0),
                      pTable->entries[i].field3))
                {
                    // Warning/assertion here: Found a PWM Scaling Slope of
                    // zero!  This is invalid and would effectively lock the
                    // fan speed.
                }
                else
                {
                    *pPwmScaleSlope = (NvSFXP4_12) DRF_VAL(_FAN_COOLER_TABLE,
                        _FIELD3, _PWM_SCALE_SLOPE, pTable->entries[0].field3);

                    *pPwmScaleOffset = (NvSFXP4_12) DRF_VAL(_FAN_COOLER_TABLE,
                        _FIELD4, _PWM_SCALE_OFFSET, pTable->entries[0].field4);

                    status = NV_OK;
                }
                break;
            }
        }
    }

fanCoolerTableGetPwmScale_exit:
    return status;
}

/**
 * Converts a PWM duty cycle to a fan level (fan speed percentage) based on the
 * fan period and duty cycle.
 *
 * @param[in] fanPeriod          The fan period to convert
 * @param[in] dutyCycle          The duty cycle to convert
 * @param[in] fanPwmScaleOffset  PWM scale offset, from the VBIOS table
 * @param[in] fanPwmScaleSlope   PWM scale slope, from the VBIOS table
 *
 * @return The converted level
 */
NvU32
fanConvertDutyCycleToLevel
(
    NvU32 fanPeriod,
    NvU32 dutyCycle,
    NvSFXP4_12 fanPwmScaleOffset,
    NvSFXP4_12 fanPwmScaleSlope
)
{
    NvU32       level;
    NvUFXP16_16 pwmRatio;
    NvU64       pwmRatio64;

    // If the fan period is 0, we dont have a cooler, so default to OFF.
    if (fanPeriod == 0)
    {
        // OFF
        level = 0;
    }
    // If the fanPeriod is 1, we have an ON/OFF cooler
    else if (fanPeriod == 1)
    {
        // If the dutyCycle matches the fanPeriod, the cooler is ON
        if (dutyCycle == fanPeriod)
        {
            // ON.
            level = 100;
        }
        else
        {
            // OFF.
            level = 0;
        }
    }
    else
    {
        //
        // Variable speed, so calc level from dutyCycle and fanPeriod
        //

        //
        // On our legacy boards, we can have periods > 16-bits, meaning the
        // dutyCycle can be also be > 16-bits.  This means that if we shift left
        // by 16 we can overflow 32-bits.
        //
        // However, once we divide by the period, the result will always be in
        // the range [0,1] because dutyCycle <= period.  Thus, if we shift and
        // do the division/scaling in 64-bits the result can be truncated back
        // to 32-bits.
        //
        //   32.0 << 16   => 32.16
        // / 32.0
        // -----------------------
        //                  1.16
        //
        pwmRatio64 = NV_UNSIGNED_ROUNDED_DIV(((NvU64)dutyCycle) <<
                      DRF_SHIFT(NV_TYPES_FXP_INTEGER(32, 16)), fanPeriod);
        pwmRatio = (NvUFXP16_16) pwmRatio64;

        // F16.16 >> 16 => F16.0
        level = NV_TYPES_UFXP_X_Y_TO_U32_ROUNDED(16, 16,
                    fanPwmScalePwmRatioToPct(fanPwmScaleOffset,
                                             fanPwmScaleSlope,
                                             pwmRatio) * 100);

        // Make sure we're within the max and min bounds - this can be due to
        // rounding/truncation issues due to integer math.
        if (level > FAN_SPEEDCONTROL_MAXIMUM_LEVEL_DEFAULT)
        {
            level = FAN_SPEEDCONTROL_MAXIMUM_LEVEL_DEFAULT;
        }
        else if (level < FAN_SPEEDCONTROL_MINIMUM_LEVEL_DEFAULT)
        {
            level = FAN_SPEEDCONTROL_MINIMUM_LEVEL_DEFAULT;
        }
    }
    return level;
}

thanks,
-- 
John Hubbard
NVIDIA


[-- Attachment #2: thermal_coolers_table_1.0.html --]
[-- Type: text/html, Size: 13383 bytes --]

[-- Attachment #3: Type: text/plain, Size: 154 bytes --]

_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Addressing the problem of noisy GPUs under Nouveau

[Martin Peres]

On 07/02/18 05:31, John Hubbard wrote:
> On 01/28/2018 04:05 PM, Martin Peres wrote:
>> On 29/01/18 01:24, Martin Peres wrote:
>>> On 28/11/17 07:32, John Hubbard wrote:
>>>> On 11/23/2017 02:48 PM, Martin Peres wrote:
>>>>> On 23/11/17 10:06, John Hubbard wrote:
>>>>>> On 11/22/2017 05:07 PM, Martin Peres wrote:
>>>>>>> Hey,
>>>>>>>
>>>>>>> Thanks for your answer, Andy!
>>>>>>>
>>>>>>> On 22/11/17 04:06, Ilia Mirkin wrote:
>>>>>>>> On Tue, Nov 21, 2017 at 8:29 PM, Andy Ritger <aritger@nvidia.com> wrote:
>>>>>>>> Martin's question was very long, but it boils down to this:
>>>>>>>>
>>>>>>>> How do we compute the correct values to write into the e114/e118 pwm
>>>>>>>> registers based on the VBIOS contents and current state of the board
>>>>>>>> (like temperature).
>>>>>>>
>>>>>>> Unfortunately, it can also be the e11c/e120 couple, or 0x200d8/dc on
>>>>>>> GF119+, or 0x200cd/d0 on Kepler+.
>>>>>>>
>>>>>>> At least, it looks like we know which PWM controler we need to drive, so
>>>>>>> I did not want to muddy the water even more by giving register
>>>>>>> addresses, rather concentrating on the problem at hand: How to compute
>>>>>>> the duty value for the PWM controler.
>>>>>>>
>>>>>>>>
>>>>>>>> We generally do this right, but appear to get it extra-wrong for certain GPUs.
>>>>>>>
>>>>>>> Yes... So far, we are always safe, but users tend to mind when their
>>>>>>> computer sound like a jumbo jet at take off... Who would have thought? :D
>>>>>>>
>>>>>>> Anyway, looking forward to your answer!
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Martin
>>>>>>
> [...]
> 
> Hi Martin,
> 
> I strongly suspect you are seeing a special behavior, which is: on
> some GF108 boards we use only a very limited range of PWM,
> 0.4 to 2.5%, due to the particular type of DC power conversion
> circuit on those boards. However, it could also just be difficulties
> in interpreting the fixed-point variables in the tables. In either
> case, the answer is to explain those formats, so I'll do that now.
> 
> I am attaching the fan cooler table, in HTML format. We have also
> published the BIT (BIOS Information Table) format, separately:
> 
>     http://download.nvidia.com/open-gpu-doc/BIOS-Information-Table/1/BIOS-Information-Table.html
> 
> , but I don't think it has any surprises for you, in this regard. You
> can check it, to be sure you're looking at the right subtable, though,
> just in case.
> 
> The interesting parts of that table are:
> 
> PWM Scale Slope (16 bits):
> 
>   Slope to scale effective PWM to actual PWM (1/4096, F4.12, signed).
>   For backwards compatibility, a value of 0.0 (0x0000) is interpreted as 1.0 (0x1000).
>   This value is used to scale the effective PWM duty cycle, a conceptual fraction
>   of full speed (0% to 100%), to the actual electrical PWM duty cycle.
>   PWM(actual) = Slope × PWM(effective) + Offset
> 
> PWM Scale Offset (16 bits):
> 
>   Offset to scale effective PWM to actual PWM (1/4096, F4.12, signed).
>   This value is used to scale the effective PWM duty cycle, a conceptual fraction
>   of full speed (0% to 100%), to the actual electrical PWM duty cycle.
>   PWM(actual) = Slope × PWM(effective) + Offset
> 
> 
> However, the calculations are hard to get right, and the table stores
> values in fixed-point format, so I'm showing a few simplified code excerpts
> that use these. The various fixed point macro definitions are found as part of
> our normal driver package, in nvmisc.h and nvtypes.h. Any other definitions
> that you need are included right here (I ran a quick compiler check to be sure.)

Wow John, thanks a lot! Sorry for the delay, I was on vacation when you
posted this, but this definitely is what I was looking for!

Thanks a lot for the code example, I will try to make use of it soon and
come back to you if I still have issues!

Martin
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau