[PATCH v3] PCI: Add pci_iounmap

[PATCH v3] PCI: Add pci_iounmap

[George Cherian]

In case if any architecture selects CONFIG_GENERIC_PCI_IOMAP and not
CONFIG_GENERIC_IOMAP, then the pci_iounmap function is reduced to a NULL
function. Due to this the managed release variants or even the explicit
pci_iounmap calls doesn't really remove the mappings.

This issue is seen on an arm64 based system. arm64 by default selects
only CONFIG_GENERIC_PCI_IOMAP and not CONFIG_GENERIC_IOMAP from this
'commit cb61f6769b88 ("ARM64: use GENERIC_PCI_IOMAP")'

Also '66eab4df288a ("lib: add GENERIC_PCI_IOMAP")' moved only  the iomap
functions to lib/pci_iomap.c. The pci_iounmap() was left in lib/iomap.c
as different achitectures has its own pci_iounmap implementation.
For architectures, which doesn't have pci_iounmap implemented, this
would lead to a potential leak. So provide a generic iounmap function in
lib/pci_iomap.c.

Simple bind/unbind test of any pci driver using pcim_iomap/pci_iomap,
would lead to the following error message after long hour tests

"allocation failed: out of vmalloc space - use vmalloc=<size> to
increase size."

Signed-off-by: George Cherian <george.cherian@marvell.com>
---
* Changes from v2
	- Get rid of the #ifdefs around pci_iounmap()
* Changes from v1
	- Fix the 0-day compilation error.
	- Mark the lib/iomap pci_iounmap call as weak incase
	if any architecture have there own implementation.
 include/asm-generic/io.h        | 4 ++++
 include/asm-generic/iomap.h     | 1 -
 include/asm-generic/pci_iomap.h | 1 +
 lib/pci_iomap.c                 | 6 ++++++
 4 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/include/asm-generic/io.h b/include/asm-generic/io.h
index dabf8cb7203b..5986b37226b7 100644
--- a/include/asm-generic/io.h
+++ b/include/asm-generic/io.h
@@ -915,12 +915,16 @@ static inline void iowrite64_rep(volatile void __iomem *addr,
 struct pci_dev;
 extern void __iomem *pci_iomap(struct pci_dev *dev, int bar, unsigned long max);
 
+#ifdef CONFIG_GENERIC_PCI_IOMAP
+extern void pci_iounmap(struct pci_dev *dev, void __iomem *p);
+#else
 #ifndef pci_iounmap
 #define pci_iounmap pci_iounmap
 static inline void pci_iounmap(struct pci_dev *dev, void __iomem *p)
 {
 }
 #endif
+#endif /* CONFIG_GENERIC_PCI_IOMAP */
 #endif /* CONFIG_GENERIC_IOMAP */
 
 /*
diff --git a/include/asm-generic/iomap.h b/include/asm-generic/iomap.h
index 649224664969..68c75e26edbd 100644
--- a/include/asm-generic/iomap.h
+++ b/include/asm-generic/iomap.h
@@ -104,7 +104,6 @@ extern void ioport_unmap(void __iomem *);
 #ifdef CONFIG_PCI
 /* Destroy a virtual mapping cookie for a PCI BAR (memory or IO) */
 struct pci_dev;
-extern void pci_iounmap(struct pci_dev *dev, void __iomem *);
 #elif defined(CONFIG_GENERIC_IOMAP)
 struct pci_dev;
 static inline void pci_iounmap(struct pci_dev *dev, void __iomem *addr)
diff --git a/include/asm-generic/pci_iomap.h b/include/asm-generic/pci_iomap.h
index d4f16dcc2ed7..3684307a6b44 100644
--- a/include/asm-generic/pci_iomap.h
+++ b/include/asm-generic/pci_iomap.h
@@ -18,6 +18,7 @@ extern void __iomem *pci_iomap_range(struct pci_dev *dev, int bar,
 extern void __iomem *pci_iomap_wc_range(struct pci_dev *dev, int bar,
 					unsigned long offset,
 					unsigned long maxlen);
+extern void pci_iounmap(struct pci_dev *dev, void __iomem *p);
 /* Create a virtual mapping cookie for a port on a given PCI device.
  * Do not call this directly, it exists to make it easier for architectures
  * to override */
diff --git a/lib/pci_iomap.c b/lib/pci_iomap.c
index 2d3eb1cb73b8..e97b73995af7 100644
--- a/lib/pci_iomap.c
+++ b/lib/pci_iomap.c
@@ -134,4 +134,10 @@ void __iomem *pci_iomap_wc(struct pci_dev *dev, int bar, unsigned long maxlen)
 	return pci_iomap_wc_range(dev, bar, 0, maxlen);
 }
 EXPORT_SYMBOL_GPL(pci_iomap_wc);
+
+void __weak pci_iounmap(struct pci_dev *dev, void __iomem *addr)
+{
+	iounmap(addr);
+}
+EXPORT_SYMBOL(pci_iounmap);
 #endif /* CONFIG_PCI */
-- 
2.25.1

Re: [PATCH v3] PCI: Add pci_iounmap

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 1407 bytes --]

Hi George,

I love your patch! Yet something to improve:

[auto build test ERROR on pci/next]
[also build test ERROR on linux/master linus/master asm-generic/master v5.9-rc2 next-20200824]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/George-Cherian/PCI-Add-pci_iounmap/20200824-212149
base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
config: powerpc-allyesconfig (attached as .config)
compiler: powerpc64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=powerpc 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   powerpc64-linux-ld: lib/pci_iomap.o: in function `__crc_pci_iounmap':
>> (.rodata+0x10): multiple definition of `__crc_pci_iounmap'; lib/iomap.o:(.rodata+0x68): first defined here

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 70366 bytes --]

Re: [PATCH v3] PCI: Add pci_iounmap

[Yang Yingliang]

On 2020/8/25 9:25, kernel test robot wrote:
> Hi George,
>
> I love your patch! Yet something to improve:
>
> [auto build test ERROR on pci/next]
> [also build test ERROR on linux/master linus/master asm-generic/master v5.9-rc2 next-20200824]
> [If your patch is applied to the wrong git tree, kindly drop us a note.
> And when submitting patch, we suggest to use '--base' as documented in
> https://git-scm.com/docs/git-format-patch]
>
> url:    https://github.com/0day-ci/linux/commits/George-Cherian/PCI-Add-pci_iounmap/20200824-212149
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git next
> config: powerpc-allyesconfig (attached as .config)
> compiler: powerpc64-linux-gcc (GCC) 9.3.0
> reproduce (this is a W=1 build):
>          wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>          chmod +x ~/bin/make.cross
>          # save the attached .config to linux build tree
>          COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=powerpc
>
> If you fix the issue, kindly add following tag as appropriate
> Reported-by: kernel test robot <lkp@intel.com>
>
> All errors (new ones prefixed by >>):
>
>     powerpc64-linux-ld: lib/pci_iomap.o: in function `__crc_pci_iounmap':
>>> (.rodata+0x10): multiple definition of `__crc_pci_iounmap'; lib/iomap.o:(.rodata+0x68): first defined here
EXPORT_SYMBOL(pci_iounmap) in lib/iomap.c need be removed.
> ---
> 0-DAY CI Kernel Test Service, Intel Corporation
> https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

Re: [PATCH v3] PCI: Add pci_iounmap

[Yang Yingliang]

Hi,

On 2020/9/2 2:05, George Cherian wrote:
> Hi Yang,
>
>> -----Original Message-----
>> From: Yang Yingliang <yangyingliang@huawei.com>
>> Sent: Tuesday, September 1, 2020 6:59 PM
>> To: George Cherian <gcherian@marvell.com>; linux-kernel@vger.kernel.org;
>> linux-arch@vger.kernel.org; linux-pci@vger.kernel.org
>> Cc: kbuild-all@lists.01.org; bhelgaas@google.com; arnd@arndb.de;
>> mst@redhat.com
>> Subject: Re: [PATCH v3] PCI: Add pci_iounmap
>>
>>
>>
>>
>> On 2020/8/25 9:25, kernel test robot wrote:
>>> Hi George,
>>>
>>> I love your patch! Yet something to improve:
>>>
>>> [auto build test ERROR on pci/next]
>>> [also build test ERROR on linux/master linus/master asm-generic/master
>>> v5.9-rc2 next-20200824] [If your patch is applied to the wrong git tree,
>> kindly drop us a note.
>>> And when submitting patch, we suggest to use '--base' as documented in
>>> https://urldefense.proofpoint.com/v2/url?u=https-3A__git-
>> 2Dscm.com_doc
>>> s_git-2Dformat-2Dpatch&d=DwIC-
>> g&c=nKjWec2b6R0mOyPaz7xtfQ&r=TjMsEFPc7di
>> rkF6u2D3eSIS0cA8FeYpzRkkMzr4aCbk&m=dvtRkwC273FmalEZE_KonLRWrIV
>> WLSWfG61
>>> NWTWG5LI&s=ycW6SZOVRuKAm3YwdhyAuSh22oPuengSMVuv-
>> EwaUew&e= ]
>>> url:    https://urldefense.proofpoint.com/v2/url?u=https-
>> 3A__github.com_0day-2Dci_linux_commits_George-2DCherian_PCI-2DAdd-
>> 2Dpci-5Fiounmap_20200824-2D212149&d=DwIC-
>> g&c=nKjWec2b6R0mOyPaz7xtfQ&r=TjMsEFPc7dirkF6u2D3eSIS0cA8FeYpzRkk
>> Mzr4aCbk&m=dvtRkwC273FmalEZE_KonLRWrIVWLSWfG61NWTWG5LI&s=6c
>> UOYHeDOBZ0HaFc2z-vaDgDmbIK4LCBRt9kNkn1sto&e=
>>> base:   https://urldefense.proofpoint.com/v2/url?u=https-
>> 3A__git.kernel.org_pub_scm_linux_kernel_git_helgaas_pci.git&d=DwIC-
>> g&c=nKjWec2b6R0mOyPaz7xtfQ&r=TjMsEFPc7dirkF6u2D3eSIS0cA8FeYpzRkk
>> Mzr4aCbk&m=dvtRkwC273FmalEZE_KonLRWrIVWLSWfG61NWTWG5LI&s=h-
>> TMyLlEdAwew-u52q4dgWBUMgm0ys-xKzvOO86e1Lw&e=  next
>>> config: powerpc-allyesconfig (attached as .config)
>>> compiler: powerpc64-linux-gcc (GCC) 9.3.0 reproduce (this is a W=1
>>> build):
>>>           wget https://urldefense.proofpoint.com/v2/url?u=https-
>> 3A__raw.githubusercontent.com_intel_lkp-
>> 2Dtests_master_sbin_make.cross&d=DwIC-
>> g&c=nKjWec2b6R0mOyPaz7xtfQ&r=TjMsEFPc7dirkF6u2D3eSIS0cA8FeYpzRkk
>> Mzr4aCbk&m=dvtRkwC273FmalEZE_KonLRWrIVWLSWfG61NWTWG5LI&s=az
>> QcL0MQmPpr9UfvyBSSdQiu1UbjJgFrzNJOtcZ_--E&e=  -O ~/bin/make.cross
>>>           chmod +x ~/bin/make.cross
>>>           # save the attached .config to linux build tree
>>>           COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0
>>> make.cross ARCH=powerpc
>>>
>>> If you fix the issue, kindly add following tag as appropriate
>>> Reported-by: kernel test robot <lkp@intel.com>
>>>
>>> All errors (new ones prefixed by >>):
>>>
>>>      powerpc64-linux-ld: lib/pci_iomap.o: in function `__crc_pci_iounmap':
>>>>> (.rodata+0x10): multiple definition of `__crc_pci_iounmap';
>>>>> lib/iomap.o:(.rodata+0x68): first defined here
>> EXPORT_SYMBOL(pci_iounmap) in lib/iomap.c need be removed.
> I really don't think that is the way to fix this. I have also seen your other patch
> in which iomap being moved out of lib/iomap.c to header file.
>
> There was a reason for moving iomap and its variants to a lib since most of
> the arch's implementation of map was similar. Whereas the unmap had multiple
> implementation per arch's. So, the lib/iomap never implemented the generic unmap.
>
> I see either of the following solution.
> a. Have an arm64 specific implementation for the unmap function.
> Or
> b. something on the lines of v2[1], which accommodates all the arch's but has the #ifdef
> for which Bjorn raised his concerns.
I think 'a' may be better, and I have already make a patch, I can send 
it after testing.
>
> Bjorn, any comments?
>
> Regards
> -George
>
> [1] - https://lkml.org/lkml/2020/8/20/28