[PATCH v2] r8169: set coherent DMA mask as well as streaming DMA mask

[PATCH v2] r8169: set coherent DMA mask as well as streaming DMA mask

[Ard Biesheuvel]

PCI devices that are 64-bit DMA capable should set the coherent
DMA mask as well as the streaming DMA mask. On some architectures,
these are managed separately, and so the coherent DMA mask will be
left at its default value of 32 if it is not set explicitly. This
results in errors such as

     r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
     hwdev DMA mask = 0x00000000ffffffff, dev_addr = 0x00000080fbfff000
     swiotlb: coherent allocation failed for device 0000:02:00.0 size=4096
     CPU: 0 PID: 1062 Comm: systemd-udevd Not tainted 4.8.0+ #35
     Hardware name: AMD Seattle/Seattle, BIOS 10:53:24 Oct 13 2016

on systems without memory that is 32-bit addressable by PCI devices.

Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
v2: dropped the hunk that sets the coherent DMA mask to DMA_BIT_MASK(32),
    which is unnecessary given that it is the default

 drivers/net/ethernet/realtek/r8169.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c
index e55638c7505a..bf000d819a21 100644
--- a/drivers/net/ethernet/realtek/r8169.c
+++ b/drivers/net/ethernet/realtek/r8169.c
@@ -8273,7 +8273,8 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
 	if ((sizeof(dma_addr_t) > 4) &&
 	    (use_dac == 1 || (use_dac == -1 && pci_is_pcie(pdev) &&
 			      tp->mac_version >= RTL_GIGA_MAC_VER_18)) &&
-	    !pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
+	    !pci_set_dma_mask(pdev, DMA_BIT_MASK(64)) &&
+	    !pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64))) {
 
 		/* CPlusCmd Dual Access Cycle is only needed for non-PCIe */
 		if (!pci_is_pcie(pdev))
-- 
2.7.4

RE: [PATCH v2] r8169: set coherent DMA mask as well as streaming DMA mask

[David Laight]

From: Of Ard Biesheuvel
> Sent: 14 October 2016 14:41
> PCI devices that are 64-bit DMA capable should set the coherent
> DMA mask as well as the streaming DMA mask. On some architectures,
> these are managed separately, and so the coherent DMA mask will be
> left at its default value of 32 if it is not set explicitly. This
> results in errors such as
> 
>      r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
>      hwdev DMA mask = 0x00000000ffffffff, dev_addr = 0x00000080fbfff000
>      swiotlb: coherent allocation failed for device 0000:02:00.0 size=4096
>      CPU: 0 PID: 1062 Comm: systemd-udevd Not tainted 4.8.0+ #35
>      Hardware name: AMD Seattle/Seattle, BIOS 10:53:24 Oct 13 2016
> 
> on systems without memory that is 32-bit addressable by PCI devices.
> 
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
> v2: dropped the hunk that sets the coherent DMA mask to DMA_BIT_MASK(32),
>     which is unnecessary given that it is the default
> 
>  drivers/net/ethernet/realtek/r8169.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c
> index e55638c7505a..bf000d819a21 100644
> --- a/drivers/net/ethernet/realtek/r8169.c
> +++ b/drivers/net/ethernet/realtek/r8169.c
> @@ -8273,7 +8273,8 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
>  	if ((sizeof(dma_addr_t) > 4) &&
>  	    (use_dac == 1 || (use_dac == -1 && pci_is_pcie(pdev) &&
>  			      tp->mac_version >= RTL_GIGA_MAC_VER_18)) &&
> -	    !pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
> +	    !pci_set_dma_mask(pdev, DMA_BIT_MASK(64)) &&
> +	    !pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64))) {

Isn't there a dma_set_mask_and_coherent() function ?

	David

Re: [PATCH v2] r8169: set coherent DMA mask as well as streaming DMA mask

[Ard Biesheuvel]

> On 14 Oct 2016, at 14:42, David Laight<David.Laight@ACULAB.COM> wrote:
> 
> From: Of Ard Biesheuvel
>> Sent: 14 October 2016 14:41
>> PCI devices that are 64-bit DMA capable should set the coherent
>> DMA mask as well as the streaming DMA mask. On some architectures,
>> these are managed separately, and so the coherent DMA mask will be
>> left at its default value of 32 if it is not set explicitly. This
>> results in errors such as
>> 
>>     r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
>>     hwdev DMA mask = 0x00000000ffffffff, dev_addr = 0x00000080fbfff000
>>     swiotlb: coherent allocation failed for device 0000:02:00.0 size=4096
>>     CPU: 0 PID: 1062 Comm: systemd-udevd Not tainted 4.8.0+ #35
>>     Hardware name: AMD Seattle/Seattle, BIOS 10:53:24 Oct 13 2016
>> 
>> on systems without memory that is 32-bit addressable by PCI devices.
>> 
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>> v2: dropped the hunk that sets the coherent DMA mask to DMA_BIT_MASK(32),
>>    which is unnecessary given that it is the default
>> 
>> drivers/net/ethernet/realtek/r8169.c | 3 ++-
>> 1 file changed, 2 insertions(+), 1 deletion(-)
>> 
>> diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c
>> index e55638c7505a..bf000d819a21 100644
>> --- a/drivers/net/ethernet/realtek/r8169.c
>> +++ b/drivers/net/ethernet/realtek/r8169.c
>> @@ -8273,7 +8273,8 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
>>    if ((sizeof(dma_addr_t) > 4) &&
>>        (use_dac == 1 || (use_dac == -1 && pci_is_pcie(pdev) &&
>>                  tp->mac_version >= RTL_GIGA_MAC_VER_18)) &&
>> -        !pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
>> +        !pci_set_dma_mask(pdev, DMA_BIT_MASK(64)) &&
>> +        !pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64))) {
> 
> Isn't there a dma_set_mask_and_coherent() function ?
> 
>    David
> 

Re: [PATCH v2] r8169: set coherent DMA mask as well as streaming DMA mask

[Ard Biesheuvel]

> On 14 Oct 2016, at 14:42, David Laight <David.Laight@ACULAB.COM> wrote:
> 
> From: Of Ard Biesheuvel
>> Sent: 14 October 2016 14:41
>> PCI devices that are 64-bit DMA capable should set the coherent
>> DMA mask as well as the streaming DMA mask. On some architectures,
>> these are managed separately, and so the coherent DMA mask will be
>> left at its default value of 32 if it is not set explicitly. This
>> results in errors such as
>> 
>>     r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
>>     hwdev DMA mask = 0x00000000ffffffff, dev_addr = 0x00000080fbfff000
>>     swiotlb: coherent allocation failed for device 0000:02:00.0 size=4096
>>     CPU: 0 PID: 1062 Comm: systemd-udevd Not tainted 4.8.0+ #35
>>     Hardware name: AMD Seattle/Seattle, BIOS 10:53:24 Oct 13 2016
>> 
>> on systems without memory that is 32-bit addressable by PCI devices.
>> 
>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>> ---
>> v2: dropped the hunk that sets the coherent DMA mask to DMA_BIT_MASK(32),
>>    which is unnecessary given that it is the default
>> 
>> drivers/net/ethernet/realtek/r8169.c | 3 ++-
>> 1 file changed, 2 insertions(+), 1 deletion(-)
>> 
>> diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c
>> index e55638c7505a..bf000d819a21 100644
>> --- a/drivers/net/ethernet/realtek/r8169.c
>> +++ b/drivers/net/ethernet/realtek/r8169.c
>> @@ -8273,7 +8273,8 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
>>    if ((sizeof(dma_addr_t) > 4) &&
>>        (use_dac == 1 || (use_dac == -1 && pci_is_pcie(pdev) &&
>>                  tp->mac_version >= RTL_GIGA_MAC_VER_18)) &&
>> -        !pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
>> +        !pci_set_dma_mask(pdev, DMA_BIT_MASK(64)) &&
>> +        !pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64))) {
> 
> Isn't there a dma_set_mask_and_coherent() function ?
> 

Not of the pci_xxx variety afaik

Re: [PATCH v2] r8169: set coherent DMA mask as well as streaming DMA mask

[Francois Romieu]

Ard Biesheuvel <ard.biesheuvel@linaro.org> :
> PCI devices that are 64-bit DMA capable should set the coherent
> DMA mask as well as the streaming DMA mask. On some architectures,
> these are managed separately, and so the coherent DMA mask will be
> left at its default value of 32 if it is not set explicitly. This
> results in errors such as
> 
>      r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
>      hwdev DMA mask = 0x00000000ffffffff, dev_addr = 0x00000080fbfff000
>      swiotlb: coherent allocation failed for device 0000:02:00.0 size=4096
>      CPU: 0 PID: 1062 Comm: systemd-udevd Not tainted 4.8.0+ #35
>      Hardware name: AMD Seattle/Seattle, BIOS 10:53:24 Oct 13 2016
> 
> on systems without memory that is 32-bit addressable by PCI devices.
> 
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Acked-by: Francois Romieu <romieu@fr.zoreil.com>

Unless someone plans to plug an acenic, a 83820 (pci-e gem board, anyone ?)
on top of a pci <-> pci-e adapter on this kind of motherboard, no other
network driver that uses the pci_... dma api exhibits this mixed 32 / 64 bit
support bug. I haven't checked devices with 32 < mask < 64 nor plain DMA api
converted ones.

-- 
Ueimor

Re: [PATCH v2] r8169: set coherent DMA mask as well as streaming DMA mask

[David Miller]

From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Date: Fri, 14 Oct 2016 14:40:33 +0100

> PCI devices that are 64-bit DMA capable should set the coherent
> DMA mask as well as the streaming DMA mask. On some architectures,
> these are managed separately, and so the coherent DMA mask will be
> left at its default value of 32 if it is not set explicitly. This
> results in errors such as
> 
>      r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
>      hwdev DMA mask = 0x00000000ffffffff, dev_addr = 0x00000080fbfff000
>      swiotlb: coherent allocation failed for device 0000:02:00.0 size=4096
>      CPU: 0 PID: 1062 Comm: systemd-udevd Not tainted 4.8.0+ #35
>      Hardware name: AMD Seattle/Seattle, BIOS 10:53:24 Oct 13 2016
> 
> on systems without memory that is 32-bit addressable by PCI devices.
> 
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>

Applied.

Re: [PATCH v2] r8169: set coherent DMA mask as well as streaming DMA mask

[David Miller]

From: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Date: Fri, 14 Oct 2016 14:48:51 +0100

> 
>> On 14 Oct 2016, at 14:42, David Laight <David.Laight@ACULAB.COM> wrote:
>> 
>> From: Of Ard Biesheuvel
>>> Sent: 14 October 2016 14:41
>>> PCI devices that are 64-bit DMA capable should set the coherent
>>> DMA mask as well as the streaming DMA mask. On some architectures,
>>> these are managed separately, and so the coherent DMA mask will be
>>> left at its default value of 32 if it is not set explicitly. This
>>> results in errors such as
>>> 
>>>     r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded
>>>     hwdev DMA mask = 0x00000000ffffffff, dev_addr = 0x00000080fbfff000
>>>     swiotlb: coherent allocation failed for device 0000:02:00.0 size=4096
>>>     CPU: 0 PID: 1062 Comm: systemd-udevd Not tainted 4.8.0+ #35
>>>     Hardware name: AMD Seattle/Seattle, BIOS 10:53:24 Oct 13 2016
>>> 
>>> on systems without memory that is 32-bit addressable by PCI devices.
>>> 
>>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>> ---
>>> v2: dropped the hunk that sets the coherent DMA mask to DMA_BIT_MASK(32),
>>>    which is unnecessary given that it is the default
>>> 
>>> drivers/net/ethernet/realtek/r8169.c | 3 ++-
>>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>> 
>>> diff --git a/drivers/net/ethernet/realtek/r8169.c b/drivers/net/ethernet/realtek/r8169.c
>>> index e55638c7505a..bf000d819a21 100644
>>> --- a/drivers/net/ethernet/realtek/r8169.c
>>> +++ b/drivers/net/ethernet/realtek/r8169.c
>>> @@ -8273,7 +8273,8 @@ static int rtl_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
>>>    if ((sizeof(dma_addr_t) > 4) &&
>>>        (use_dac == 1 || (use_dac == -1 && pci_is_pcie(pdev) &&
>>>                  tp->mac_version >= RTL_GIGA_MAC_VER_18)) &&
>>> -        !pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
>>> +        !pci_set_dma_mask(pdev, DMA_BIT_MASK(64)) &&
>>> +        !pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64))) {
>> 
>> Isn't there a dma_set_mask_and_coherent() function ?
>> 
> 
> Not of the pci_xxx variety afaik

You can often use the "dev_*" variants intechangably with the pci_*()
ones.

In fact you'll find that for several architectures pci_*() is
implemented via calls to dev_*().